[PATCH] Netlink on FreeBSD support

[Alexander Chernikov]

> On 12 Dec 2022, at 15:17, Ondrej Zajicek via Bird-users <bird-users at trubka.network.cz> wrote:
> 
> On Sat, Dec 10, 2022 at 04:54:52PM +0000, Alexander Chernikov wrote:
>>>> In FreeBSD, the default routing table is #0 and there are no special 25X tables.
>>>> Some attributes (like RTA_PREFSRC) are not supported. Interface address flags are not fully mapped yet & there might be some unsupported stuff w.r.t IPv6 prefixes lifetime.
>>>> “Prohibit” nexthops are not supported yet (as the kernel only provides blackhole / reject ATM).
>>>> Resilient nexthop groups are not supported yet as well.
>>>> 
>>>> I may be missing something, but I’m pretty sure that the implementation is close to supporting all of the current bird features (except mpls).
>>>> 
>>>>> Like Linux keep (and Netlink API assumes) multiple routes with different
>>>>> metric, and keeps source protocol.
>>>> Yes, metrics & source protocol works.
>>> 
>>> Does source protocol also works with the native API (krt socket)?
> 
>> No. The data is in the kernel but there are no properties in the
>> protocol to propagate it. There are some spare fields in the rt_metrics
>> structure that can be used for that purpose, but I’d prefer not to extend
>> rtsock anymore.
> 
>> Additionally, there is no such feature in OpenBSD / NetBSD.
> 
> What about multiple routes with different metrics / priorities? Is that
> supported on FreeBSD through rtsock API? That one seems to work in OpenBSD.
It is, but there is no way get all of the paths from a single message - one has to implement buffering similar to netlink IPv6 code bird [used to] have IIRC.
> 
> BTW, do you have some suggestions how to improve BIRD rtsock BSD
> interface in BIRD? Especially one sore point that is collisions with
> kernel provided direct routes. In recent FreeBSD, these routes seems
> to be non-overwritable / non-overridable, which is safe, but causes
> plenty of error messages in logs.
It may be a big one :-)
In the current [Free]BSD networking model, there is no specific treatment of interface routes - they reside in the same routing table and that route table is the master source of truth w.r.t routing decisions.
Bird model is slightly different - krt ignores all directly-connected routes and re-generates them IFF the appropriate direct protocol is configured. Not all installations configure the direct protocol. Additionally, there are other corner cases like multiple interfaces with the same network prefix or just the static interface routes where direct protocol may behave differently from the routing table.
Personally, I tend to think that addressing this “impedance mismatch” would help with the specified and related problems - for the rtsock world.
What do you think?

Speaking of the immutability - it happened for a reason. For example, there is a situation when you have an active-passive router configuration, sharing the state via OSPF/bgp and you need to migrate certain network/interface from one router to another.
As minimal downtime is desired, it can happen that bird override the newly-added interface route with the remote route. When withdraw message propagates, bird will remove this route, leaving the interface w/o the directly connected route.
There is still a way to remove such routes by specifying RTF_PINNED flag in the delete message if so desired.

> 
> In recent OpenBSD, our routes are added with a different priority (RTP_*),
> which is done by default, but we sometimes overwrite/remove the kernel
I’d love to introduce variation of route priorities in the kernel, so the interface routes and backup routes are treated differently. The problem is that it would be a disrupting rtsock change, requiring changes of all existing customers.
Problems like this and stuff like multipath inefficiencies turned me to netlink.
I’m now focused on improving general netlink kernel support and making bird working with netlink flawlessly.
I’d love be able to eventually switch bird to netlink as the default protocol on new FreeBSD versions provided that it works correctly and there is no objection from you & other persons on the team.


> provided direct routes (probably because we do not use explicit priority
> in add/remove operations, and remove operation defaults to removing the
> kernel one, but that looks like something that could be easily fixed).
> 
> I have no idea how this works in NetBSD.
> 
> -- 
> Elen sila lumenn' omentielvo
> 
> Ondrej 'Santiago' Zajicek (email: santiago at crfreenet.org)
> OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net)
> "To err is human -- to blame it on a computer is even more so."
>