Designing a BGP Large Community Scheme for a Small Network

Once you have more than a couple of upstreams and some IX peers, the question “where did this route come from?” gets harder to answer from the AS path alone. BGP communities solve that: tag a route when it enters your network, and every router after that knows where it was learned and how it should be treated.

In the Peering Manager post I already used large communities to set the local preference per location. This post takes a step back and describes how to design the scheme itself.

All examples use AS64500, a documentation ASN. Replace it with your own.

Why large communities

Classic communities (RFC 1997) are two 16-bit values: ASN:value. That works fine until your ASN is a 32-bit one, which is the case for most ASNs issued in recent years: it simply does not fit in 16 bits.

Large communities (RFC 8092) are three 32-bit values: ASN:function:parameter. That solves the 32-bit ASN problem, and the extra field gives you a natural structure: the middle value says what the community means, the last one carries the argument.

The scheme

I split the function field into two ranges: informational communities that we set ourselves, and action communities that peers and customers may send to us.

Informational (set on import, by us)

Community	Meaning
64500:1:<location>	Learned at location (1 = Amsterdam, 2 = Frankfurt, …)
64500:2:1	Learned from a transit provider
64500:2:2	Learned from an IX route server
64500:2:3	Learned from a bilateral IX peer
64500:2:4	Learned from a private interconnect (PNI)
64500:2:5	Learned from a customer
64500:3:<ix-id>	Learned at Internet Exchange <ix-id>

Actions (may be set by customers)

Community	Action
64500:100:<asn>	Do not announce to <asn>
64500:101:<asn>	Prepend once towards <asn>
64500:102:<asn>	Prepend twice towards <asn>
64500:103:<asn>	Prepend three times towards <asn>
64500:110:<location>	Do not announce at location <location>

Plus the well-known ones that every network should honour:

• 65535:666, BLACKHOLE (RFC 7999), for customers under DDoS
• 65535:0, GRACEFUL_SHUTDOWN (RFC 8326): lower the local preference so traffic moves away before maintenance

Rules that keep it sane

Strip before you set. On every eBGP import, first delete all communities in your own ASN range that the neighbor sent, then add your informational communities. Otherwise a peer can send you 64500:2:5 and their routes are suddenly treated as customer routes. Only accept action communities from customers, and only within the action range.

Export based on type, not on lists. The most important rule in any network is that routes from transit and peers are never announced to other transit and peers. With communities that becomes one simple rule: towards transit and peers, only announce your own prefixes and routes tagged 64500:2:5 (customer). No prefix lists to forget to update.

Document it publicly. A scheme is only useful to customers if they can find it. Put it in your PeeringDB record, on your website, or in your IRR object.

On VyOS

Tagging on import, for a transit session in Amsterdam:

set policy route-map TRANSIT-IN rule 10 action 'permit'
set policy route-map TRANSIT-IN rule 10 set large-community delete 'OWN-COMMUNITIES'
set policy route-map TRANSIT-IN rule 10 set large-community add '64500:1:1 64500:2:1'

The list of our own communities, used to strip them:

set policy large-community-list OWN-COMMUNITIES rule 10 action 'permit'
set policy large-community-list OWN-COMMUNITIES rule 10 regex '64500:.*:.*'

A list that matches customer routes, for the export:

set policy large-community-list FROM-CUSTOMER rule 10 action 'permit'
set policy large-community-list FROM-CUSTOMER rule 10 regex '64500:2:5'

And the export policy towards transit and peers. Own prefixes are matched with a prefix list, customer routes with the community, everything else is dropped:

set policy route-map TRANSIT-OUT rule 10 action 'permit'
set policy route-map TRANSIT-OUT rule 10 match ip address prefix-list 'OWN-PREFIXES-V4'
set policy route-map TRANSIT-OUT rule 20 action 'permit'
set policy route-map TRANSIT-OUT rule 20 match large-community large-community-list 'FROM-CUSTOMER'
set policy route-map TRANSIT-OUT rule 100 action 'deny'

The action communities (“don’t announce to AS x”) follow the same pattern: per neighbor, a rule in the export route-map that matches 64500:100:<neighbor-asn> and denies. With a generator like Peering Manager, that rule comes for free with every session.

Start small

You don’t need the full scheme on day one. Start with the informational communities on import: they don’t change routing at all, but show bgp ipv4 unicast large-community 64500:2:1 immediately gives you an overview of everything learned via transit. Once that works, convert the export filters to the community-based rules, and add action communities when the first customer asks for them.