IPv6 routes not imported into Kernel

[Robert Finze]

Hello all,

I've decided to learn more about BGP and Bird.
To do so I'm currently setting up a small BGP setup with 2 VMs and Bird 
as routing daemons.
While the first server is running smoothly, the second one is not 
importing IPv6 routes into the kernel. There are no issues with IPv4 
routes on either system.

The Bird config on both systems is nearly identical (only IPs differ) 
and also the systems are setup in a similar manner.

Both VMs run in ProxmoxCE 8.0.4
The system without issues is running:

BIRD 2.14
on
Linux version 5.4.0-165-generic (buildd at lcy02-amd64-078) (gcc version 
9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.2)) #182-Ubuntu SMP Mon Oct 2 
19:43:28 UTC 2023

The system with issues is running:

BIRD 2.14
Linux version 5.15.0-88-generic (buildd at lcy02-amd64-058) (gcc (Ubuntu 
11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU Binutils for Ubuntu) 2.38) 
#98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023


The routes are correctly learned from upstream and exported to the 
kernel, but the kernel is not "learning" them.
The journal shows the following logs:

--------------
Nov 04 21:31:51 moon bird[850]: Netlink: Invalid argument
Nov 04 21:31:51 moon bird[850]: Netlink: Invalid argument
Nov 04 21:31:51 moon bird[850]: Netlink: Invalid argument
Nov 04 21:31:51 moon bird[850]: ...
Nov 04 21:31:51 moon bird[850]: Netlink: Invalid argument
Nov 04 21:31:51 moon bird[850]: ...
Nov 04 21:31:52 moon bird[850]: Netlink: Invalid argument
Nov 04 21:31:52 moon bird[850]: ...
Nov 04 21:31:53 moon bird[850]: Netlink: Invalid argument
Nov 04 21:31:54 moon bird[850]: ...
--------------

One example route from Birds point of view shows:
--------------
bird> show route for 2620:12f:f000::/44
Table master6:
2620:12f:f000::/44   unicast [upstream_1v6 2023-11-04 16:47:37] ! (100) 
[AS43i]
         via 2a0d:xxxxx:1 on ens20
                      unicast [upstream_2v6 2023-11-04 16:47:36] (100) 
[AS43i]
         via 2a0d:xxxxx:2 on ens20
--------------

Linux shows:
# ip -6 r get 2620:12f:f000::
RTNETLINK answers: Network is unreachable

Manually adding the routes to the kernel works:
# ip -6 r a 2620:12f:f000::/44 nexthop via 2a0d:xxxx:1 nexthop via 
2a0d:xxxx:2 dev ens20


After creating a netlink monitor interface I was able to capture some 
netlink messages which show that the routes are trying to be created but 
an error is returned.
I don't get from the error message what the issue is.


ip link add nl0 type nlmon
ip link set nl0 up

Netlink route
0000   00 04 03 38 00 00 00 00 00 00 00 00 00 00 00 00   ...8............
0010   68 00 00 00 18 00 05 05 13 0d 66 02 00 00 00 00   h.........f.....
0020   0a 28 00 00 fe 0c 00 01 00 00 00 00 14 00 01 00   .(..............
0030   26 07 ff 00 0b 00 00 00 00 00 00 00 00 00 00 00   &...............
0040   08 00 06 00 20 00 00 00 14 00 07 00 2a 0e 39 40   .... .......*.9@
0050   10 00 00 00 00 00 00 00 00 00 00 02 08 00 04 00   ................
0060   02 00 00 00 14 00 05 00 2a 0e 39 40 de ad 00 00   ........*.9 at ....
0070   00 00 00 00 00 00 00 01                           ........

Netlink
0000   00 04 03 38 00 00 00 00 00 00 00 00 00 00 00 00   ...8............
0010   7c 00 00 00 02 00 00 00 13 0d 66 02 7a 31 09 81   |.........f.z1..
0020   ea ff ff ff 68 00 00 00 18 00 05 05 13 0d 66 02   ....h.........f.
0030   00 00 00 00 0a 28 00 00 fe 0c 00 01 00 00 00 00   .....(..........
0040   14 00 01 00 26 07 ff 00 0b 00 00 00 00 00 00 00   ....&...........
0050   00 00 00 00 08 00 06 00 20 00 00 00 14 00 07 00   ........ .......
0060   2a 0e 39 40 10 00 00 00 00 00 00 00 00 00 00 02   *.9 at ............
0070   08 00 04 00 02 00 00 00 14 00 05 00 2a 0e 39 40   ............*.9@
0080   de ad 00 00 00 00 00 00 00 00 00 01               ............

Happy to forward a pcap or any other information as well.


After upgrading to newer Kernel it still shows the same behaviour, that 
is, "Netlink: Invalid argument" for IPv6 routes..

/proc/version
Linux version 6.2.0-36-generic (buildd at lcy02-amd64-050) 
(x86_64-linux-gnu-gcc-11 (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld 
(GNU Binutils for Ubuntu) 2.38) #37~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC 
Mon Oct  9 15:34:04 UTC 2


It would be great if anyone had any pointer to what I am missing or what 
is going wrong here.


Best,
Robert