Possibility to treat /32 and /128 non-gateway routes as onlink on BSD?

[Ondrej Zajicek]

On Sun, Apr 18, 2021 at 08:55:12PM +0200, Stefan Haller wrote:
> Hi everyone,
> 
> In a recent change in the FreeBSD Wireguard kernel module the
> POINTTOPOINT interface flag is dropped for wg interfaces[1]. This
> behaviour will probably not change, because Wireguard is not
> peer-to-peer or a broadcast domain, but peer-to-multipoint. Hence my
> previous configuration of setting peer addresses on the interface is not
> working anymore:
> 
>   ifconfig wg0 inet 192.168.0.10/32 192.168.0.4

Hi

Just to be sure, Wireguard is really PtMP (some peers on the iface may
not be able to communicate between themselves directly) and not NBMA?
In that case this network setup makes sense.

> Note that this actually models the real world better, because Wireguard
> is not point-to-point but point-to-multipoint. So multiple peers can be
> done in the following way:
> 
>   ifconfig wg0 192.168.0.10/32
>   route add 192.168.0.4 -iface wg0
>   route add 192.168.0.8 -iface wg0
> 
> However, this does not work with bird 2.0.8: bird will not recognize the
> p2p peer addresses as gateways and it will log on kernel table rescan:

Yes, the main issue is that (sans onlink flag) BIRD validates next-hops
against interface ranges and not against direct (non-gateway) routes. In
most cases it does not matter but with PtMP it would require manually
configuring multiple PtP address pairs for an iface.


>   KRT: Received route 192.168.42.0/24 with strange next-hop 192.168.0.4
>   KRT: Error sending route 192.168.42.0/24 to kernel: File exists
> 
> I did have a look why this is happening: bird discards the route,
> because the gateway does not belong to any of its interface's addresses
> (the peer address is configured as a static host route and not on the
> interface). At the same time it forgets that it was bird itself who
> added the route (because the route is discarded on kernel table rescan,
> it thinks it is absent in the kernel table but present in bird's
> internal routing table -> "add" logic triggers) and tries to add the
> route again (instead of replacing/keeping the route). This explains the
> cascade of error messages above.

True


> While the Linux netlink code allows to handle onlink routes, this
> functionality is absent in the BSD code path. Unfortunately, on BSD
> there is no dedicated flag to mark routes as being directly connected
> onlink routes. On Linux one could add an onlink route like so:
> 
>   ip route add 192.168.0.4 via 192.168.0.4 dev wg0 onlink

This does not make sense to me. You can do this just by:

 ip route add 192.168.0.4 dev wg0

The onlink flag is useful if you have some network behind 192.168.0.4,
then you can do:

 ip route add 192.168.42.0/24 via 192.168.0.4 dev wg0 onlink


> The command looks quite odd, but it is working. The via clause is
> necessary in conjunction with onlink. Afterwards a `birdc dump
> neighbors` verifies that the peer is recognized:
> 
>   000055db793035c0 192.168.0.4 wg0 wg0 kernel1 000000000000000000000000 scope univ ONLINK

So the strange ip-route command (192.168.0.4 via 192.168.0.4) is here
just to inject such neighbor entry to neighbor cache? That seems more
like a bug in neighbor cache (where two requests with different flags
influence themselves so they produce different results than when run
independently), which unexpectedly helped in your case.

The proper solution (on Linux) is that the second route (for
192.168.42.0/24) also has onlink flag, so it does not depend on
existence of route for 192.168.0.4/32. Babel in BIRD generates
routes with onlink flag.


> I created a patch that emulates the behaviour on FreeBSD. In this patch
> the onlink setting is heuristically determined by inspecting the
> RTF_GATEWAY flag (which must be absent) and checking for a /32 or /128
> prefix length. I do not see how to handle cases where the destination is
> a subnet. The patch was tested on FreeBSD 13.0-RELEASE and 14.0-CURRENT.
> During rescan of the kernel table the above route is now not dropped
> because the gateway is recognized to be onlink.
> 
> While in the Linux route the gateway is still set, on FreeBSD it is
> unset (~RTF_GATEWAY). So I actually use the destination address of the
> route, because that's the address of the peer.
> 
> While I tested the functionality on my machines and also checked the
> output of `birdc dump neighbors`, the patch might be faulty. Please be
> kind with me and treat it more as a proof of concept. As I am not
> experienced with bird's code base I do not fully understand how the
> neighbor cache is working in detail, so the suggested patch might have
> unexpected side effects. I would appreciate feedback.
> 
> In case you think the whole idea is not sensible, I would also love to
> hear suggestions how to get the above setup working.

As i wrote above, i think this behavior of the neighbor case is a bug,
not feature :-), so we should not depend on it.

I see the problem as BIRD internally support onlink flag, but BSD kernel
does not support that flag, so onlink routes exported to BSD kernel are
not read back properly. Seems to me that there is a simple woraround:

When i read a route (from kernel on BSD) that has gateway on iface, which
has only /32 or /128 IP address(es), so no proper iface range, then i would
assume onlink flag for the route (its nexthops).

-- 
Elen sila lumenn' omentielvo

Ondrej 'Santiago' Zajicek (email: santiago at crfreenet.org)
OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net)
"To err is human -- to blame it on a computer is even more so."