Out of memory crash of 3000 routes in BGP

[Justin Cattle]

Are there any thoughts as to why I still see quite large memory usage, and
only on some [ seemingly random ] hosts ?



Cheers,
Just

On 11 October 2016 at 19:05, Madhu <informmadhu at gmail.com> wrote:

> Hi everyone,
>
>   Now i tested with 16 path and 32 path, I don't see the crash. Below are
> the memory usage for each.
>
> *16 path*
>
> bird> show memory
> BIRD memory usage
> Routing tables:   5109 kB
> Route attributes:   13 kB
> ROA tables:        192  B
> Protocols:         163 kB
> Total:            5384 kB
>
>
> *32 path*
>
> bird> show memory
> BIRD memory usage
> Routing tables:     10 MB
> Route attributes:   16 kB
> ROA tables:        192  B
> Protocols:         239 kB
> Total:              10 MB
>
>
> Now I increased to* 62 path*, I see that* CPU is going high to 100%*, and
> the memory usage is 0.1%. birdc socket got struck. I was not able to access
> the socket.
>
> bird> show memory
> BIRD memory usage
> Routing tables:   5127 kB
> Route attributes:   20 kB
> ROA tables:        192  B
> Protocols:         382 kB
> Total:            5641 kB
> bird> show memory >>. After this it got struck
>
> # top
> top - 17:47:50 up  1:02,  2 users,  load average: 1.35, 1.42, 1.10
> Tasks: 163 total,   2 running, 161 sleeping,   0 stopped,   0 zombie
> %Cpu(s): 29.7 us,  0.5 sy,  0.2 ni, 69.4 id,  0.0 wa,  0.0 hi,  0.2 si,
> 0.0 st
> KiB Mem:   8127384 total,  1279304 used,  6848080 free,    36040 buffers
> KiB Swap:   812736 total,        0 used,   812736 free.   456220 cached Mem
>
>   PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
>  8768 bird      20   0   21728   8808   2352 R* 100.0*  0.1   8:12.97
> bird
>
> Attached the bird config file.
>
> Madhu
>
>
>
>
>
>
> On Tue, Oct 11, 2016 at 5:56 AM, Justin Cattle <j at ocado.com> wrote:
>
>> It looks like oom-kill from the call trace I think.
>>
>>
>>
>> Cheers,
>> Just
>>
>> On 11 October 2016 at 13:07, Ondrej Zajicek <santiago at crfreenet.org>
>> wrote:
>>
>>> On Mon, Oct 10, 2016 at 06:21:44PM -0700, Madhu wrote:
>>> > I have 3000 routes with 64 path ecmp. It is crashing . I don't see the
>>> coredump . Is there anyway to fix using trace ?
>>>
>>> Hi
>>>
>>> I would like to confirm, it is BIRD that was killed by out of memory
>>> killer and could be just restarted? Or Linux kernel crashed? (You wrote
>>> 'the kernel crashed with out of memory issue' in one prior mail.)
>>>
>>> In the first case, how much memory BIRD takes after start and before it
>>> crashes?
>>> How long it takes to crash to crash due to OOM?
>>>
>>>
>>> > On Oct 10, 2016, at 6:08 PM, Jonathan Stewart <
>>> jonathan.stewart at gmail.com> wrote:
>>> > It must be something specific to your environment.
>>>
>>> It is likely related to 64 path BGP multipath exported to kernel.
>>>
>>> --
>>> Elen sila lumenn' omentielvo
>>>
>>> Ondrej 'Santiago' Zajicek (email: santiago at crfreenet.org)
>>> OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net)
>>> "To err is human -- to blame it on a computer is even more so."
>>>
>>>
>>
>> Notice:  This email is confidential and may contain copyright material of
>> members of the Ocado Group. Opinions and views expressed in this message
>> may not necessarily reflect the opinions and views of the members of the
>> Ocado Group.
>>
>>
>>
>> If you are not the intended recipient, please notify us immediately and
>> delete all copies of this message. Please note that it is your
>> responsibility to scan this message for viruses.
>>
>>
>>
>> Fetch and Sizzle are trading names of Speciality Stores Limited and
>> Fabled is a trading name of Marie Claire Beauty Limited, both members of
>> the Ocado Group.
>>
>>
>>
>> References to the “Ocado Group” are to Ocado Group plc (registered in
>> England and Wales with number 7098618) and its subsidiary undertakings (as
>> that expression is defined in the Companies Act 2006) from time to time.
>> The registered office of Ocado Group plc is Titan Court, 3 Bishops Square,
>> Hatfield Business Park, Hatfield, Herts. AL10 9NE.
>>
>
>

-- 


Notice:  This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group. 

 

If you are not the intended recipient, please notify us immediately and 
delete all copies of this message. Please note that it is your 
responsibility to scan this message for viruses. 

 

Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled 
is a trading name of Marie Claire Beauty Limited, both members of the Ocado 
Group.

 

References to the “Ocado Group” are to Ocado Group plc (registered in 
England and Wales with number 7098618) and its subsidiary undertakings (as 
that expression is defined in the Companies Act 2006) from time to time. 
 The registered office of Ocado Group plc is Titan Court, 3 Bishops Square, 
Hatfield Business Park, Hatfield, Herts. AL10 9NE.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://trubka.network.cz/pipermail/bird-users/attachments/20161012/699827d4/attachment.html>