Improving Anycast routing with Bird

[Keenan Tims]

These decisions are entirely made by Comcast's equipment - you have  
very little influence over their routing choices, especially because  
you do not peer directly with them. They do seem a bit nonsensical -  
Comcast's internal metrics should know that SEA is closer to PDX than  
LAX is, and should route accordingly, but this may be done for  
capacity reasons facing Choopa in Seattle or on the PDX-SEA links, an  
observation that the majority of Choopa traffic is destined for LA,  
and Seattle would be a detour, or who knows why else. You could ask  
Vultr to raise this with Comcast and see if you can get an adjustment  
or answer, some providers are responsive to this kind of request, but  
only from a direct customer/peer.

There is one mechanism built into BGP for influencing route metrics -  
MED, but this is only transitive to the neighbour AS, so Comcast  
wouldn't see your MEDs, and it's very common to ignore them anyway.

The other mechanism for influencing this kind of thing is through BGP  
community signalling, though this method is usually pretty blunt.  
Typically you can suppress announcements toward a specific AS  
entirely, or prepend toward them (lengthening the AS path and  
affecting BGP route selection); usually you can also modify the  
localpref of your announcement within your peer's network. The goal  
being, more or less, to encourage remote networks to avoid the problem  
AS/link entirely. If you're trying to avoid Comcast's messed up  
internal routing, your only real option is not to announce toward  
Comcast at all, and hope that Comcast will hand it off quickly to  
someone that routes it better. I wouldn't really count on it.  
Prepending might work, if Choopa and Comcast are peers, but if Choopa  
is a customer, Comcast will likely local-pref their routes above their  
peering/transit, and AS path length can't defeat this.

Vultr indicates they support some BGP communities to do this kind of  
manipulation:

https://www.vultr.com/docs/as20473-bgp-customer-guide

Comcast might also support something similar, though it is common to  
strip communities before passing the routes upstream. You might be  
able to work with Vultr to have them signal to Comcast that your  
announcements are backup only or the like, or do so yourself and the  
Vultr/Choopa routers pass this along. But again, this has the same  
effect as not announcing to them at all - they will choose a  
*different* route that doesn't connect directly to Choopa, but it may  
not be better.

More or less you're at their mercy.

K

Quoting Kyle Drake <kyle at kyledrake.net>:

>>
>>
>> Normally you would use anycast to get you to a DNS server (which doesn't
>> have to be that near), then a geographic DNS server to get you to the right
>> CDN element.
>>
>>
> That's what I was doing previously, but I need to control the IPs for the
> CDN, and I only have the budget for one /24, so I'm trying to make the best
> of it. Aside from some occasional weird routing, the network has worked
> really well. State has not been an issue for what we're doing (short-lived
> HTTP connections).
>
> I'm just trying to see what the extent of my powers to control weird
> routing are. It's odd to see Comcast cold-potatoing connections to the
> wrong routes, sometimes on the other side of the continent (
> https://gist.github.com/kyledrake/7a4cd36ea276ec3134b4a51a42a37f48). I'm
> wondering if there is a way to configure Bird to help steer these sorts of
> things a little better, even if it's on a case-by-case or region-by-region
> level.
>
> My apologies if these are all dumb questions. Again, not much anycast
> documentation out there (I'm planning to improve this later by putting
> together a web resource for people doing this).
>
> -Kyle


-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: PGP Digital Signature
URL: <http://trubka.network.cz/pipermail/bird-users/attachments/20160921/207e674a/attachment.asc>