Update on BIRD 3 progress

[Maria Matejka]

Hello!

I'd like to share some information on what has been done until now. My 
update is based on the "roadmap" I've shared with you on May 26 this year:

http://trubka.network.cz/pipermail/bird-users/2022-May/016139.html

On 5/26/22 10:52 PM, Maria Matejka wrote:
> Anyway, here is the current state dump, written from my perspective.
> 
> Once upon a time, there was a version 3.0-alpha0 which has been released 
> and found broken soon afterwards. It has three major flaws which we know 
> about right now:
> 
> 1. it can't be easily merged with version 2.0.9
> (I tried to do it, believe me, it is impossible.)
> 
> 1A. so let's create the next version 3 by merging its commits into 2.0.9 
> by short intervals and fix what is broken by these merges [IN PROGRESS]
> 1B. something in the original branch looks weird, let's do it better as 
> we now know the goal better [IN PROGRESS]
> 1C. some of these changes deserve to be merged back into the 2.0.x 
> branch to avoid future merge conflicts [IN PROGRESS]

1C. is prepared now in the "backport" branch. Needs some code review and 
comments but yes, it is there.

> 1D. oops, now there is the commit with original import/export table 
> rework which I don't want to merge at all [jump to 3].
> 1E. dozens of following commits to be merged and fixed [PENDING]
> 
> 2. MRT dumps are insecure and will crash your BIRD [PENDING]
> (Found out during some internal team call after the release.)
> 
> 2A. needs a special service layer
> 2B. this has to wait until tables are properly threaded
> 2C. it is better to wait for BMP to be merged to do multithreading of 
> both of these monitoring services at once

Not yet done but soon on the table.

> 3. calling ROA check from a filter in 'show route' crashes BIRD.
> (Reported by DE-CIX soon after release. Thanks for testing!)
> 
> 3A. this is due to lock ordering violation
> 3B. we wanted to rework the show route anyway to not lock the table 
> while formatting the route, so let's do it [PENDING]

Done.

> 3C. but there is also a significant performance problem in channel 
> auxiliary tables (import / export) – the original rework made them more 
> of a full-size table, which adds lots of unwanted overhead [PENDING]

Partially fixed by making the import table just a lower layer of route 
attributes. The attribute implementation deserves some optimizations yet 
to be done.

> 3D. in fact, the import table may be done by storing the pre-filter 
> route attributes "under" the actual attributes; filter modifications 
> would create an overlay over the original route attributes [PENDING]

Done and seems working.

> 3E. the export table also deserves some rework and it would help 
> performance a lot [PENDING]

Reworked to store the routes right before sending in BGP, therefore 
_after_ applying after-export modifications of attributes, e.g. 
attribute stripping, nexthop replacement or ASN appending.

This will be quite a major behavior change between version 2 and 3, yet 
we think storing the exported routes in this place is much more useful 
than where it was before.

> 3F. well, it would be less work in total to first do the import / export 
> table rework and then the show-route rework [PENDING]

Well, done interleaved somehow.

> 3G. the current route attribute implementation is a two-layer mess 
> originating in version 1 where everything was an IP route, let's squash 
> these into one layer [IN PROGRESS]

Done.

> 3H. also the nexthop resolving procedures and flowspec validation 
> procedures kinda suck, let's make them more obvious, transparent and 
> thread-friendly [IN PROGRESS]
> 3I. well, also the current "extended" attribute layer doesn't scale 
> well, let's fix it [IN PROGRESS]
> 3J. and finally also the type-value structures are different in filters 
> and attributes, let's fix it [IN PROGRESS]

These three are still pending.

> The appropriate branches are (named in a weird way, as common):
> * haugesund --> this should become the -alpha1, shouldn't rebase

Renamed to thread-next as we are now quite sure that it is the way to go.

> * haugesund-to-2.0 --> this should merge into 2.0.x soon, code review 
> and other feedback pending, rebases may occur when some bug is found to 
> fix the original commit rather than adding a fix commit

Renamed to backport.

> It's quite a lot of work but still just a little compared to the amount 
> of work needed before to get -alpha0 to release. Most of the tasks 
> depend on the previous ones deeply so it will still take several weeks 
> until something can be offloaded. For now, these branches are quite a 
> one-woman-show (I don't like that).

I'd like to say that it still seems slow from my POV, yet we're moving 
forwards. Let's see how much of coding opportunities is there during 
this summer as work performance always drops during the holiday season. 
Naively, I thought we might have got 3.0-alpha1 like now or so, yet it's 
now more like mid-September.

I'll be anyway very upset if we miss the opportunity to announce 
3.0-alpha1 during the next RIPE session (autumn 2022, should be in Beograd).

Maria

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2839 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://trubka.network.cz/pipermail/bird-users/attachments/20220711/37c48393/attachment.p7s>