1 out of 2 BGP sessions down, all routes become unreachable

[Dez C]

Hi,

all external routes were unreachable (apart from the local loopback 
which was imported via direct protocol). master6 consisted solely of 
entries like:

2a02:xxxx:d00d::/45 unreachable [bgp_v6_a 19:38:23.222 from 2a02:xxxx:yyyy:53::a] * (100) [ASxxxxx1i]

No entries whatsoever from the other bgp session (bgp_v6_b), like in 
normal operation:

2a02:xxxx:d00d::/45   unicast [bgp_v6_a 2021-04-07] * (100) [ASxxxxx1i]
         via 2a02:xxxx:yyyy:53::a on eth2
                      unicast [bgp_v6_b 2021-04-07] (100) [ASxxxxx1i]
         via 2a02:xxxx:yyyy:53::b on eth2

  

regards,
Yannis


On 4/9/21 11:22 PM, Alexander Zubkov wrote:
> Hi,
>
> And what do you see in your bird's console? "show route" etc.
>
> On Fri, Apr 9, 2021 at 7:10 PM Dez C <dez at otenet.gr> wrote:
>> Hello,
>>
>> First message here, although a long-time bird user (not an expert in any
>> way). We're using bird as a bgp speaker for all our VMs that either need
>> to announce routes or import routes from VRFs etc.
>>
>> The setup is pretty straight forward I'd guess. The VM has 2 IPv6 BGP
>> sessions with 2 L3 switches (one per switch). The VM announces one /128
>> route and imports all routes learned from the BGP sessions. The same
>> routes are learned from both sessions, so whenever I do a 'ip -6 r' I
>> see 2 entries for each route (with different next hops) like:
>> 2a02:xxxx:d00d::/45 via 2a02:xxxx:yyyy:zz::a dev eth2 proto bird metric
>> 32 pref medium
>> 2a02:xxxx:d00d::/45 via 2a02:xxxx:yyyy:zz::b dev eth2 proto bird metric
>> 32 pref medium
>>
>> At some point, both BGP sessions went down and one of them did not
>> recover for an hour. Normally, this should not be a problem but in this
>> specific case, I could no longer see 2 routes in the kernel table, one
>> reachable and another unreachable, just an unreachable one, so even
>> though the anycast route was being announced and traffic was reaching
>> the VM, it couldn't send the packets to their destination as all routes
>> were unreachable in the kernel table:
>> unreachable 2a02:xxxx:d00d::/45 dev lo proto bird metric 32 error -113
>> pref medium .
>>
>> I'm running 2.0.7 (upgrading soon) on centos7 and here's the relevant
>> part of my config:
>>
>> protocol device {
>> }
>> protocol direct direct_v6_lo {
>>           interface "lo*";  # announce loopback
>>           ipv6;
>> }
>> protocol direct direct_v6_eth {
>>           interface "eth*";
>>           ipv6;
>> }
>> protocol bgp bgp_v6_a {
>>           local 2a02:xxxx:yyyy:53::37 as 66666
>>           neighbor 2a02:xxxx:yyyy:53::a as 66666;  # switch A
>>           ipv6 {
>>                  export where proto = "direct_v6_lo";
>>           };
>> }
>> protocol bgp bgp_v6_b {
>>           local 2a02:1388:4000:53::37 as 66666;
>>           neighbor 2a02:1388:4000:53::b as 66666;  # switch B
>>           ipv6 {
>>                   export where proto = "direct_v6_lo";
>>           };
>> }
>> protocol kernel {
>>       ipv6 {
>>           export filter {
>>               if proto = "direct_v6_eth" then reject;
>>               accept;
>>           };
>>       };
>>      merge paths limit 2;
>> }
>>
>> any ideas?
>> thanks in advance and regards,
>> Yannis