Possibility to treat /32 and /128 non-gateway routes as onlink on BSD?

[Stefan Haller]

Hi everyone,

In a recent change in the FreeBSD Wireguard kernel module the
POINTTOPOINT interface flag is dropped for wg interfaces[1]. This
behaviour will probably not change, because Wireguard is not
peer-to-peer or a broadcast domain, but peer-to-multipoint. Hence my
previous configuration of setting peer addresses on the interface is not
working anymore:

  ifconfig wg0 inet 192.168.0.10/32 192.168.0.4

The above syntax is for POINTTOPOINT interfaces. In this case
192.168.0.10 is the local address of the interface and 192.168.0.4 is
the address of the remote peer.

So if one does not want to add a dedicated subnet to every tunnel
device, one has to add static host routes similar to these ones:

  ifconfig wg0 192.168.0.10/32
  route add 192.168.0.4 -iface wg0

Note that this actually models the real world better, because Wireguard
is not point-to-point but point-to-multipoint. So multiple peers can be
done in the following way:

  ifconfig wg0 192.168.0.10/32
  route add 192.168.0.4 -iface wg0
  route add 192.168.0.8 -iface wg0

However, this does not work with bird 2.0.8: bird will not recognize the
p2p peer addresses as gateways and it will log on kernel table rescan:

  KRT: Received route 192.168.42.0/24 with strange next-hop 192.168.0.4
  KRT: Error sending route 192.168.42.0/24 to kernel: File exists

I did have a look why this is happening: bird discards the route,
because the gateway does not belong to any of its interface's addresses
(the peer address is configured as a static host route and not on the
interface). At the same time it forgets that it was bird itself who
added the route (because the route is discarded on kernel table rescan,
it thinks it is absent in the kernel table but present in bird's
internal routing table -> "add" logic triggers) and tries to add the
route again (instead of replacing/keeping the route). This explains the
cascade of error messages above.

While the Linux netlink code allows to handle onlink routes, this
functionality is absent in the BSD code path. Unfortunately, on BSD
there is no dedicated flag to mark routes as being directly connected
onlink routes. On Linux one could add an onlink route like so:

  ip route add 192.168.0.4 via 192.168.0.4 dev wg0 onlink

The command looks quite odd, but it is working. The via clause is
necessary in conjunction with onlink. Afterwards a `birdc dump
neighbors` verifies that the peer is recognized:

  000055db793035c0 192.168.0.4 wg0 wg0 kernel1 000000000000000000000000 scope univ ONLINK

(Also the "strange next-hop" message will go away.)

I created a patch that emulates the behaviour on FreeBSD. In this patch
the onlink setting is heuristically determined by inspecting the
RTF_GATEWAY flag (which must be absent) and checking for a /32 or /128
prefix length. I do not see how to handle cases where the destination is
a subnet. The patch was tested on FreeBSD 13.0-RELEASE and 14.0-CURRENT.
During rescan of the kernel table the above route is now not dropped
because the gateway is recognized to be onlink.

While in the Linux route the gateway is still set, on FreeBSD it is
unset (~RTF_GATEWAY). So I actually use the destination address of the
route, because that's the address of the peer.

While I tested the functionality on my machines and also checked the
output of `birdc dump neighbors`, the patch might be faulty. Please be
kind with me and treat it more as a proof of concept. As I am not
experienced with bird's code base I do not fully understand how the
neighbor cache is working in detail, so the suggested patch might have
unexpected side effects. I would appreciate feedback.

In case you think the whole idea is not sensible, I would also love to
hear suggestions how to get the above setup working.


Kind regards,
Stefan


[1]: https://git.zx2c4.com/wireguard-freebsd/commit/?id=8801509656e955c27ebf4b9b3e39fbf717bb0367


 sysdep/bsd/krt-sock.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/sysdep/bsd/krt-sock.c b/sysdep/bsd/krt-sock.c
index ea0cc4d9..4a0f3a7a 100644
--- a/sysdep/bsd/krt-sock.c
+++ b/sysdep/bsd/krt-sock.c
@@ -547,6 +547,12 @@ krt_read_route(struct ks_msg *msg, struct krt_proto *p, int scan)
 	    net->n.addr, a.nh.gw);
 	return;
       }
+  } else if ((!ipv6 && pxlen == 32) || (ipv6 && pxlen == 128)) {
+	/* Treat non gateway /32 and /128 routes as onlink routes and inject this
+	   information into neighbor cache. */
+    if (!neigh_find(&p->p, idst, a.nh.iface, NEF_ONLINK))
+      log(L_ERR, "KRT: Failed to inject onlink neighbor %I for interface %s",
+          idst, a.nh.iface->name);
   }
 
  done:
-- 
2.31.1