[PATCH] Netlink on FreeBSD support

[Ondrej Zajicek]

On Wed, Dec 14, 2022 at 01:17:05PM +0000, Alexander Chernikov wrote:
> > On 12 Dec 2022, at 15:17, Ondrej Zajicek via Bird-users <bird-users at trubka.network.cz> wrote:
>
> > BTW, do you have some suggestions how to improve BIRD rtsock BSD
> > interface in BIRD? Especially one sore point that is collisions with
> > kernel provided direct routes. In recent FreeBSD, these routes seems
> > to be non-overwritable / non-overridable, which is safe, but causes
> > plenty of error messages in logs.
>
> It may be a big one :-)
>
> In the current [Free]BSD networking model, there is no specific
> treatment of interface routes - they reside in the same routing table and
> that route table is the master source of truth w.r.t routing decisions.
>
> Bird model is slightly different - krt ignores all directly-connected
> routes and re-generates them IFF the appropriate direct protocol is
> configured. Not all installations configure the direct protocol.
> Additionally, there are other corner cases like multiple interfaces with
> the same network prefix or just the static interface routes where direct
> protocol may behave differently from the routing table.
>
> Personally, I tend to think that addressing this “impedance mismatch”
> would help with the specified and related problems - for the rtsock
> world.
>
> What do you think?

Definitely.

In ideal world, BIRD routing tables should be complete and equal to
kernel tables, perhaps by BIRD having the full responsibility for
managing the tables (i.e. being the single source of truth about
routing). In practice, there are several complications:

1) In all supported systems, kernel crates its own direct routes anyways.

2) Sometimes, direct routes require some magic OS-specific handing, that
would be hard to get right (like RTF_CLONING in OpenBSD).

3) We should keep OS usable even after BIRD is stopped, so we should remove
BIRD-specific routes but keep direct routes, even if they were temporarily
overriden.

4) There are also third-party alien routes that we should take into account.

So in practice, we just trying to not destroy kernel-provided direct
routes in kernel table, but we would like to be able to safely override
them (e.g. if OSPF decides that some indirect path should be more
preferred than the direct one).

But we still keep the idea that BIRD is the single/master source of truth
about routing, so we just close eyes about the fact that direct routes in
the kernel routing table are provided by kernel and not by us. That is
why krt ignores kernel-provided routes (as it is supposed to provide only
third-party alien routes) and why direct routes are instead generated by
the direct protocol.

It also ensures that we have consistent internal behavior between
systems, and not import say cloned host routes on systems that use them.

(I would ignore the fact that some installations do not configure the
direct protocol, that is just a result of limited usefulness of direct
routes inside BIRD, due to some hacks related to next hop resolution, so
in some cases they are not useful, or they are provided by OSPF anyways.
But conceptually, i think there should be direct routes in BIRD tables
and we should resolve next hops through them.)

One thing that reasonably works is Linux-style routing table with
priorities, esp. when kernel-provided routes have lower priority (like
in IPv6 case, where default kernel priority is 256, but it can be set as
kernel option for IPv4 case). In such case we can just export our routes
as higher priority ones, but still keep kernel ones.


> Speaking of the immutability - it happened for a reason. For example, there is a situation when you have an active-passive router configuration, sharing the state via OSPF/bgp and you need to migrate certain network/interface from one router to another.
> As minimal downtime is desired, it can happen that bird override the newly-added interface route with the remote route. When withdraw message propagates, bird will remove this route, leaving the interface w/o the directly connected route.
> There is still a way to remove such routes by specifying RTF_PINNED flag in the delete message if so desired.

One backwards-compatible solution would be to consider RTF_PINNED as
'shadow priority level', one could just overwrite the route without using
RTF_PINNED, but the kernel route is just shadowed and reappear when the
overwriting route is removed. While add/remove/replace with RTF_PINNED
specified would modify directly the kernel route.

-- 
Elen sila lumenn' omentielvo

Ondrej 'Santiago' Zajicek (email: santiago at crfreenet.org)
OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net)
"To err is human -- to blame it on a computer is even more so."