Invalid NEXT_HOP attribute for OSPF known route

[Ondrej Zajicek]

On Thu, Dec 16, 2021 at 04:41:09PM +0100, Nico Schottelius wrote:
> 
> 
> Ondrej Zajicek <santiago at crfreenet.org> writes:
> >> > I expect it is in multihop /
> >> > gateway-recursive mode, as it is default for IBGP?
> >>
> >> It's in direct mode for all links, as the APU-ROUTERS have physical links
> >> inside the kubernetes cluster as well as physical connection to the ROUTER.
> >
> > This is likely the issue. In BIRD, relations between default values
> > are:
> 
> Removing the direct statement from the iBGP sessions indeed fixes the
> problem. So technically speaking, what is the disadvantage/drawback
> of avoiding the "direct" statement everywhere? We have so far added it
> whenever the connection was in the same network segment (hence: direct),
> but if it is enabled for eBGP by default (makes sense) and it does not
> "help" in the iBGP case, should we ever use it?

It has some minor advantages (like BGP is enabled immediately after iface
with neighbor address appears, BGP state can depend on layer-2 link
state, you can use TTL security more reliably, you can run BGP over
link-local addresses).


> > EBGP/IBGP -> direct / multihop -> gateway direct/recursive.
> >
> > So, by changing link mode to direct, you implicitly changed gateway mode
> > from recursive to direct.
> >
> > In gateway direct mode, the next hop is not resolved through routing table,
> > just through interface ranges. And if not found, it is rejected with
> > 'Invalid next hop', so having OSPF route is irrelevant. But because link
> > is IBGP, the original (indirect) next hop is there.
> 
> I understand the code path logic, but I don't understand the
> "configuration logic". Or as said above: does it ever make sense to
> apply direct explicitly?

Yes, if you have flat topology (all EBGP routers connected to shared
layer-2 segment), do not run IGP/OSPF, and use 'next hop self' on all
IBGP links, then enabling 'direct / gateway direct' can make sense
compared to regular multihop IBGP with recursive next hop resolution.

For example, in your case, you can decide to drop OSPF, propagate
2a0a:e5c0::/64 also through BGP and enable 'next hop self' on IBGP
sessions. Then enabling 'direct' would make sense. But in your current
setup i would probably recommend to use regular multihop IBGP.

Also note the reverse case for this option - switching EBGP to multihop
(which is used mostly in diagnostic setups / route collectors).


One could argue that perhaps configuration logic:

 EBGP/IBGP -> direct / multihop
 EBGP/IBGP -> gateway direct/recursive


would make more sense (i.e. switching IBGP to direct would not implicitly
enable gatewa direct mode), but it makes less sense for the reverese case
 - if you switch EBGP to multihop then you do not want gateway direct.

-- 
Elen sila lumenn' omentielvo

Ondrej 'Santiago' Zajicek (email: santiago at crfreenet.org)
OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net)
"To err is human -- to blame it on a computer is even more so."