BGP + routing w/ multiple providers/uplinks

[Tore Anderson]

* Gregg Berkholtz

> My goals & confusion:
> - The intent is to announce two /24 netblocks across both of two
> separate uplinks, w/ my ASN via BGP.
> - It's not clear if I need to maintain multiple routing tables, with
> a single "internal" autonomous system, and if so, how to facilitate
> that with bird's pipe function.
> - I believe I should be focusing on ensuring replies return via the
> originating interface - however at this point, another set of eyes
> would really help in hashing this out.

Hi Gregg,

BGP can't really help you here.  A BGP router only considers the
destination field of the IP packet when forwarding it.  It doesn't know
if it is a "reply packet" - from the router's point of view, it's just a
packet like any other.

Asymmetric routing like this is pretty normal, and there's seldom any
reason to try to avoid it.  If you have to anyway, you will have to look
into policy-based routing to replace or supplement BGP.  But I'd
recommend against it if you have a choice, because of the added complexity.

> From the perspective of my router:
> - BGP sessions are active w/ uplinks, and providers are not filtering
> announcements.
> - ingress TCP connections to router's "source address" IP are
> established with no problem, if my BGP/routing tables instruct
> replies via the same interface packets happen to originate via.
> - ingress TCP connections cannot be established, if my BGP/routing
> tables instruct replies via a different interface than what packets
> happen to originate via.
> - tcpdump shows packets ingress from 1st provider, and egress towards
> 2nd provider, only if routing table instructs to 2nd
> provider...although replies never arrive at destination.
> - outbound TCP connections originating from the router itself, can be
> established with no issue, to either provider. Source IP matches
> local address of outbound route.
> - All addresses are public; this router is not doing any NAT.
> 
> Below IPs were changed to protect the innocent, guilty, and everyone
> in-between - all replacements were via sed -i s///g...

In my opinion, that's a bit silly.  Leaving out information just makes
it harder to help you figure out what's going on, and in your case I
think specific information is essential.  BGP announcements and such are
inherently public information anyway.  So I just looked it up...

Your two /24s are 65.49.94.0/24 and 199.223.127.0/24, and that one of
the providers are AS6939 (Hurricane Electric), correct?  You might have
problems using 65.49.94.0/24 with another provider, as these addresses
belong to HE and provider #2 might drop them as an anti-spoof measure.
You might have to provide them with a LOA from HE before they'll allow
the traffic.  Similarly, 199.223.127.0/24 appears to belong to Stephouse
Networks if I'm reading ARIN's whois database correctly.  Is this your
provider #2?

You said your providers aren't filtering announcements, but I can not
see any routes originating from AS14613 that are not via AS6939.  Are
you 100% certain that provider #2 doesn't filter your announcements?
If it is, and furthermore is running uRPF in strict mode on your transit
port, any packets sent to that interface will be dropped.

In any case, if you want to multihome, I'd recommend against borrowing
address space from your upstreams.  You won't be truly
provider-independent if you do, and you'll be running into these
filtering issues from time to time.  If I were you I'd instead go and
get your very own PA allocation directly from ARIN, or if you're not an
ARIN member, get one of your upstreams to request a PI assignment on
your behalf.

Best regards,
-- 
Tore Anderson
Redpill Linpro AS - http://www.redpill-linpro.com/