Invalid NEXT_HOP attribute for OSPF known route

[Ondrej Zajicek]

On Tue, Dec 14, 2021 at 08:35:48PM +0100, Nico Schottelius wrote:
> 
> Hello,
> 
> every year or then bird is putting me into the Invalid NEXT_HOP
> message.

Hello

Yes, this is kind of confusing error message (as i noted in response
to Simon Ruderich).


> TL;DR:
>     Why does bird on router1+router2 refuse the route
>     2a0a:e5c0:0:12:b01a:5ae3:1bd4:1e00/122 via
>     2a0a:e5c0::225:90ff:fe1e:3e62
> 
>     even though router1+router2 know how to reach 2a0a:e5c0::/64 via
>     fe80::20d:b9ff:fe57:2f91 by means of ospf?

Note that in gateway-recursive mode, the fact of whether reachability is
known is not relevant for 'Invalid NEXT_HOP' message. If it is
unreachable, then the route would be accepted as 'unreachable' and
changes to reachable if reachability is changed. It is usually some
other issue like zero or local IP address.


> In detail:
> 
> router1, router2 are peered to apu-router1,apu-router2 via OSPF + BGP.
> 
> apu-router1,apu-router2 are peered to a set of kubernetes hosts.
> 
> The goal is to have router1 + router2 import the routes sent by the
> kubernetes hosts:
> 
>            router1      router2---------|
>               |  \          |           |
>               |   \         |           |
>               |    \        |           |
>      apu-router1    apu-router2         |
>         .     |           .             |
>               |--------------------------
>         .                 .
>      [ kubernetes cluster via apu-routers ]
> 
> 
> The problem: router1+router reject the routes with:
> 
>     Dec 14 20:33:51 router1 daemon.err bird: apu_router1_place5_ungleich_ch_v6: Invalid NEXT_HOP attribute
> 
> The setup:
> 
>     router1, router2, apu-router1, apu-router2 = ASN209898
>     kubernetes hosts = ASN65533
>     kubernetes peers with apu-routers only.

So the BGP link between kubernetes and APU-ROUTER is EBGP, while between
APU-ROUTER and ROUTER is IBGP? I expect it is in multihop /
gateway-recursive mode, as it is default for IBGP?


> The routes:
>     Kubernetes announces parts of 2a0a:e5c0:0:12::/64 and
>     2a0a:e5c0:0:13::/64, for instance the route
>     2a0a:e5c0:0:12:b01a:5ae3:1bd4:1e00/122.
> 
>     Kubernetes nodes live in 2a0a:e5c0::/64.
> 
>     apu-routers have a leg in 2a0a:e5c0::/64, via eth1.2. They reach the
>     cluster directly. They have the routes.
> 
>     routers1+2 receive the route for 2a0a:e5c0::/64 via ospf:
> 
>     bird> show route 2a0a:e5c0::/64
>     Table master6:
>     2a0a:e5c0::/64       unicast [ospf6 17:08:18.515] * I (150/20) [0.0.0.47]
>                          via fe80::20d:b9ff:fe57:2f91 on bond0.8
> The apu-routers:
>     - They import the route [0]

Could you show 'show route all' on apu-router for failed routes to see
their BGP_NEXT_HOP attribute? And also ideally tcpdump output to see
BGP_NEXT_HOP as sent from apu-router to router?


>     - They export the route to the routers [1]

Could you also show 'show protocol all' for the session from apu-router to the kubernetes hosts?

> The routers:
>     - print 4x the Invalid NEXT_HOP attribute, once per exported
>     kubernetes network
>     - They ignore the 4 routes [2]
> 
> Question: why does bird on the routers not accept the routes? Or is
> there a different problem I am not seeing? Aside from that, shouldn't
> bird on the apu-routers set itself as nexthop for the kubernetes routes?

Not, because sending it to routerx over IBGP link, where BGP_NEXT_HOP is
kept unmodified by default (unless 'next hop self' option is used).

It might be an issue related to IPv6 dual next-hops (global and link-local),
where global is empty.


-- 
Elen sila lumenn' omentielvo

Ondrej 'Santiago' Zajicek (email: santiago at crfreenet.org)
OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net)
"To err is human -- to blame it on a computer is even more so."