AW: BIRD crashes with 1000 peers

[Daniel Seidenstücker]

tried this today:

-          took fresh 1.5.0

-          applied the 2 patches manually

-          changed the following defines to 65535:

o   /usr/include/linux/posix_types.h:#define __FD_SETSIZE 1024

o   /usr/include/x86_64-linux-gnu/bits/typesizes.h:#define __FD_SETSIZE  1024

-          configure …, make, sudo make install

-          verified that the binaries I use were the ones that I built today

 

But the result is  the same behavior as before: 

-          without raise of user limits (/etc/security/limits.conf or ulimit in init-script); user limit is 1024 and lower than FD_SETSIZE

o   1000 works, but too many open files in syslog; 1250 doesn’t work

-          with raise of user limits (/etc/security/limits.conf or ulimit in init-script); user limit is 10,000 and lower than FD_SETSIZE

o   limits were accepted by BIRD (cat /proc/<pid>/limits | grep files)

o   750 works, 1000 crashes without anything in syslog

 

Due to my understanding and research some call of select() assumes FD_SETSIZE to be 1024 and doesn’t get my FD_SETSIZE changes. If too many peers connect the amount of open files grow larger than 1024 and that crushes BIRD. Without me raising the user limits, the user limit of 1024 prevents BIRD from taking more than 1024 and therefore select() from crushing BIRD.

 

I will try the master branch tomorrow but nevertheless any further help will be appreciated.

 

Thanks,

Daniel Seidenstuecker

 

Von: Alexander V. Chernikov [mailto:melifaro at ipfw.ru] 
Gesendet: Freitag, 12. Februar 2016 17:34
An: Daniel Seidenstьcker; Bird-users at network.cz
Betreff: Re: BIRD crashes with 1000 peers

 

https://gitlab.labs.nic.cz/labs/bird/commit/3aed0a6ff7b2b811a535202fd787281d2ac3340912.02.2016, 19:15, "Daniel Seidenstücker" <d.seidenstuecker at googlemail.com>:

Dear BIRD community,

 

I want to connect BIRD 1.5.0 with more than 1000 ExaBGP peers on Ubuntu 14.04 LTS 64 Bit. Connecting 1000 peers works but if I increase the peer count to more than 1024 it fails due to the soft file limit of 1024.

To raise that I tried the usual (ulimit, /etc/security/limits.conf) but BIRD doesn’t take these new limits. Later I found the advice to add  “ulimit -n 10000” to the init.d script before each “start-stop-daemon” command. After restarting BIRD it gets the correct soft and hard limits (cat /proc/<pid>/limits | grep files). But now only 750 peers work and connecting 1000 peers crashs bird without any error message in syslog (“log syslog all” configured). (I did not test 751-999 peers.)

You need to recompile bird with different FD_SETSIZE limit. For example, https://cs.uwaterloo.ca/~brecht/servers/openfiles.html can be used as a guide.

As for the crashes, they were fixed by https://gitlab.labs.nic.cz/labs/bird/commit/338f85ca7721fac16394ccabd561ddb5ccaacb36 && https://gitlab.labs.nic.cz/labs/bird/commit/3aed0a6ff7b2b811a535202fd787281d2ac33409

 

It would be nice if somebody could help meJ.

 

Thanks,

Daniel Seidenstuecker

 

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://trubka.network.cz/pipermail/bird-users/attachments/20160215/bec7b18d/attachment.html>