L3VPN

Introduction

Virtual Private Network (VPN) is exactly that - a virtual connection over a shared medium that allows customer traffic to be separate from traffic of other customers. It's not a new concept - ATMs, Frame Relay and leased line work on the same basic principle. Nowadays however, VPNs refer to the general idea, not a specific technology of transferring packets. Most common use today is interconnecting multiple locations over an IP based packet switching/routing technology. There are few ways of going about it - ones that do not interact with customer's routing domain and ones that do:

• L3VPN - MPLS based method that interacts with customer over layer 3 (shared medium becomes a logical router that interconnects multiple locations).

• L2VPN - MPLS based method that interacts with customer over layer 2 (shared medium becomes a logical switch - customer takes care of routing on their own).

In this article, we will discuss the first option - L3VPNs.

Basic definitions:

• Customer Edge (CE) - router physically located in customer's location. Forms routing protocol adjacency with provider's side for exchange of customer's routes between interconnected locations or uses static routes.

• Provider Edge (PE) - router belonging to a provider which encapsulates customer's data and sends it over the provider's network along with redistributing customer's routes to other PEs connected to customer's locations.

• Provider Router (P) - routers in provider's core that don't peer with customers and just forward MPLS labeled customer data.

• VPN Routing and Forwarding table (VRF) - every customer gets its own routing table, that stores received routes. Contents of VRF table are then advertised to other PE routers that are connected to locations belonging to the same customer.

 
To rephrase: L3VPN customers send routes to the provider which redistributes them among customer's locations and allows for data transfer between those locations in a matter that separates it from other traffic.

Control Plane

BGP takes care of routes redistribution between customer locations. However, the usual BGP rules will prevent distribution of overlapping prefixes - for example - internal networks of different clients both running 192.168/16 network. Only one of those routes will be shared with other PEs, since only the best route is advertised.
To circumvent this limitation, special kind of NLRI is used for sharing L3VPN routes, which is configured with "inet-vpn unicast" family under appropriate BGP neighbors group.
With that MP-BGP extension, customer routes are prefixed with a value uniquely identifying customer VPNs - Route Distinguisher.

Route Distinguishers

RDs are a 32bit values that are prefixed to customer's internal networks to make them unique and not allow for mentioned earlier prefix overlapping. Prefixes with RD attached are called VPN-IPv4 and are transmitted in NLRI field of BGP.

RDs are split into two parts:

• Administrator Field

• Assigned Number

Administrator can assign any kind of value to those fields (as long as it's unique) but common practice is putting AS number or originating PE address into the Administrator Field and a number into the Assigned Number field to differentiate between VPNs on the same PE. Full VPN-IPv4 addresses look like this:

Administrator_Field:Assigned_Number:Prefix

For example:

• 65000:1234:192.168.0.0/16 (AS based)

• 112.45.32.53:1:192.168.0.0/16 (IP based)

In Junos, PEs can generate RDs automatically after configuring "route-distinguisher-id" command. Specified IP address is used in the Admin Field and an integer is generated by the router to fill the Assigned Number field.

Route Targets

There are two ways that route redistribution can be done:

• Manual configuration of BGP neighborships between PEs connected to the customer's location.

• Full mesh of BGP connections and route redistribution controlled by import/export rules.

Most popular and least demanding in terms of configuration is the second one. Commonly import/export rules are handled with one value called Route Target.
RTs identify a customer - i.e. all VPN connections that should share routes. RTs have form of extended BGP communities, for example:

target:65000:1111

All RTs have to start with the "target" value and are followed by an AS number and an Assigned Number, similar to AS based Route Distinguishers.
RT is attached to routes advertised to other PEs. When a route is received, RT value is checked against local VRFs and imported only to ones that have the same value. Routes not matching any VRFs are discarded.
If more control is required, separate import and export rules can be specified. They coexist with RTs, so common practice is specifying RTs and additional import/export rules in case a subset of routes from another customer also need to be shared.
All incoming routes that meet RTs or import rules are put into the bgp.l3vpn.0 routing table. Each customer gets also its own routing table in the form of customer.inet.0 which contains only routes belonging to that specific customer.

PE-CE Peering
Now that we have a way to redistribute routes between VPN locations, we need... routes to redistribute.
The simplest solutions are static routes pointing toward the Provider Edge routers. However if less configuration intensive solution is needed an exterior or interior routing protocol can be used to peer between CEs and PEs.

BGP

Peering with BGP is easy - from customer point of view, only external group configuration is required for basic functionality.
From provider's point of view however, one thing needs to be kept in mind - BGP relies on AS Path for loop detection. Since different customer's locations often have the same AS number, by default routes won't be accepted on remote end since the same AS number will appear twice in route's AS Path (or to be more precise - CE will see it's own AS Number in the incoming route and immediately discard it since it detects a routing loop).
To circumvent that behavior, AS Path needs to be modified. Most commonly used solution is "as-override" command that needs to be configured on PEs.
Now, when a PE receives a route which contains AS Number in the AS Path of the CE it's peering with, it replaces it with it's own AS Number. Since the CE receiving the route won't see it's own AS Number in the AS Path, route will be placed in its routing table.

OSPF

Peering using OSPF is equally simple. From customer's point of view PE router is just another neighbor. From provider's point of view, once again, one thing has to be kept in mind.
When route from CE is received by a PE, it's placed into the a VRF table and advertised using BGP to a remote PE (BGP by default advertises all active routes in the VRF table, no matter their source).
However, when the route is received on the remote PE and placed into the VRF table, it's marked as a BGP route since it was received from BGP. By default this route won't be advertised to remote CE since OSPF advertises routes learned only from OSPF. Routes from other protocols need to be specifically injected. This can be taken care of by setting up a BGP to OSPF import policy.

Data Forwarding Plane

Once the routes are distributed among PEs, data forwarding can start between customer's locations. Quite obviously, MPLS is used for that.
Once a PE receives a packet destined to a remote location, it checks VRF table associated with the interface for appropriate route. This route has to contain interface through which to forward the packet and labels:

• Top Label(s)- label(s) used for MPLS switching in the provider's core.

• Bottom Label - label that identifies the VPN.

Usually, just two labels are needed (double push), but in cases that the LDP LSP used for forwarding traffic to PE is tunneled inside RSVP LSP three may be required (triple push).
Once the packet crosses the core where P routes do the usual label swap operations on top labels, it reaches the remote PE, which examines the bottom label to know with which VRF table the packet is associated. Then, after consulting the VRF table, packet is forwarded to the CE router.

Internet Access

Often, customers require Internet access in addition to L3VPN services over the same physical link. This can be achieved in multiple ways. Internet access with separate physical link won't be discussed, since it has nothing to do with the VPN itself.
Separate Logical Connection
First and the most simple solution is setting up two logical connections on 2 subinterfaces - one of them will take care of VPN traffic and be assigned to the VRF table, the second one will be a "normal" interface used for Internet access.
Only configuration steps outside of setting up two subinterfaces are default route on the CE towards the PE on the "normal" subinterface and a static route to customer's prefix on the PE also using the "normal" subinterface.
Separate Logical Connection and a Single Uplink

Second solution is also using two logical connections, but the second one is used only for downstream Internet traffic.
On the CE a default route is configured toward the PE on VPN's logical connection. PE has two solutions:

• copy Internet routes into customer's VRF table

• default route in the VRF table with next hop set to inet.0 that contains Internet routes

Obviously, second solution is more popular and a lot more efficient when there are several customers connected to the PE.
Downstream traffic goes back to the customer exactly the same was as in first Internet access option - static route is configured for customer's prefix using the second, "normal" subinterface.

Single Logical Connection

The last way to provide Internet access uses just one logical connection and requires the least amount of additional configuration on customer's side - just a default route.
Provider to allow for customer upstream traffic to the Internet does the same thing as in the second access solution - default route with next-hop to the inet.0 table.
For downstream traffic however, whole VRF table is copied into the inet.0 table to allow for reaching the customer from the Internet.