Bird6 freeze under high load

[Baptiste Jonglez]

Hi,

We are experiencing regular "freezes" of bird6 on a BGP router.  When this
happens, bird6 maxes out a CPU for several minutes.  If a command is run
in birdc6 during such a freeze, the command hangs, and the result is only
returned when birdc6 has stopped using the CPU.  Note that this also
applies to "cheap" commands like "show protocols", which usually complete
instantly (both with bird, and with bird6 in non-freeze conditions).

Sometimes (but not always), the non-responsiveness of bird6 causes all BGP
sessions to drop, which is really annoying on a full-view BGP router.

The freezes happen at random, but seem to happen more frequently when the
router is under load (typically, at peak time, each CPU spends ~20%
forwarding packets, on a 4-core box).

The BGP setup is made of multiple transit and peerings, on multiple VLANs
(some BGP neighbours share the same VLAN).  The setup is pretty similar on
bird and bird6, but only bird6 exhibits these freezes, bird works just fine.

The box is running Debian wheezy on amd64, with bird from backports: 1.4.5-1~bpo70+1

Attached is the configuration, and two extracts of the logs when all BGP
sessions dropped (with debug { states, interfaces, events }).  All files
are anonymised, but should be consistent.

What do you think?  It looks like bird6 gets stuck on some very expensive
operation, which prevents it from doing anything else (include maintaining
BGP sessions alive).

Thanks,
Baptiste
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bird6-log.tar.gz
Type: application/octet-stream
Size: 1865 bytes
Desc: not available
URL: <http://trubka.network.cz/pipermail/bird-users/attachments/20150129/1582b480/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bird6-conf.tar.gz
Type: application/octet-stream
Size: 1217 bytes
Desc: not available
URL: <http://trubka.network.cz/pipermail/bird-users/attachments/20150129/1582b480/attachment-0001.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://trubka.network.cz/pipermail/bird-users/attachments/20150129/1582b480/attachment.asc>