A TLOC is a Transport Locator that represents an attachment point where a Cisco WAN Edge device connects to a WAN transport. A TLOC is uniquely identified by a tuple of three values - (System-IP address, Color, Encapsulation).
- System-IP: The System-IP is the unique identifier of the WAN edge device across the SD-WAN fabric. It is similar to the Router-ID in traditional routing protocols such as BGP. It does not need to be routable or reachable across the fabric.
- Transport Color: The color is an abstraction used to identify different WAN transports such as MPLS, Internet, LTE, 5G, etc. In scenarios where transport types are duplicated (for example two different Internet providers) and should be treated differently from each other, the colors could be arbitrary, such as Green, Blue, Silver, Gold, etc.
- Encapsulation Type: This value specifies the type of encapsulation this TLOC uses - IPsec or GRE. To successfully form a data plane tunnel to another TLOC, both sides must use the same encryption type.
Let's stop here and look at the example shown in figure 1. WAN edge router 1 has an interface Ge0/0 with IP address 10.1.2.3 that connects to the Internet. As we all know, this IP is private and not routable over the Internet, so on the provider side, this private address is translated to the public address 31.1.2.3 through one-to-one NAT. On the other side, vEdge-2 has interface Ge0/1 with a publicly routable IP address 150.2.2.2.
Now let's say that we are tasked to manually configure an overlay tunnel between WAN edge 1 and WAN edge 2. If the devices are regular Cisco IOS routers and we are configuring a GRE tunnel with IPsec on top, we are going to specify a tunnel interface with tunnel-source IP and tunnel-destination IP on both ends taking the NAT into account. Also, we are going to specify the IPsec authentication and encryption parameters, the IPsec session keys, tunnel keys, and many additional settings depending on the situation. Now imagine that vEdge-1 and vEdge-2 must establish the same tunnel but in an automatic fashion and on-demand. Well, they will need to exchange all these parameters with each other. That is what the concept of Transport Locators (TLOCs) is created for.
A TLOC route consists of all required information needed by a remote peer in order to establish an overlay tunnel with that TLOC. This includes private and public IP addresses and ports, site-id, preference, weight, status, encapsulation info such as encryption and authentication parameters, and much more. Note that the three-tuple uniquely identifies a particular TLOC but the TLOC route itself consists of much more info. For example, the vEdge-1 connection to the Internet is uniquely identified by the TLOC (1.1.1.1, green, ipsec), but the corresponding TLOC route that vEdge-1 advertises to the SD-WAN controller consists of the IP address of interface Ge0/0, the public IP address after the NAT, all IPsec parameters and many more values required by vEdge-2 in order to establish an IPsec tunnel to vEdge-1.
Having all we have said about TLOCs in mind, let's now follow the process of forming an overlay tunnel between WAN edge-1 and WAN edge-2. When vEdge-1 is first connected to the Internet via interface Ge0/0, it gets an IP address/Mask/Default Gateway via DHCP and it gets the transport color and encapsulation type from the controller or from a manual configuration. It then tries to establish all control plane connections to vBond/vManage/vSmart and in the process, it detects its publicly routable NATed address. Once it is able to communicate to the control plane via this interface, it creates a unique TLOC identified by the system-IP 1.1.1.1 (not the interface IP), the WAN transport color - green, and the encapsulation type - IPSec. This TLOC is then advertised to vSmart along with many other attributes that you can see in the output below.
vEdge-1# show omp tlocs detail
---------------------------------------------------
tloc entries for 1.1.1.1
green
ipsec
---------------------------------------------------
ADVERTISED TO:
peer 9.9.9.9
Attributes:
encap-key not set
encap-proto 0
encap-spi 256
encap-auth sha1-hmac,ah-sha1-hmac
encap-encrypt aes256
public-ip 31.1.2.3
public-port 12346
private-ip 10.1.2.3
private-port 12346
public-ip ::
public-port 0
private-ip ::
private-port 0
domain-id not set
site-id 10
overlay-id not set
preference 0
tag not set
stale not set
weight 1
version 2
gen-id 0x80000011
carrier default
restrict 0
groups ( 0 )
border not set
unknown-attr-len not set
preference 0
tag not set
stale not set
weight 1
version 2
gen-id 0x80000015
carrier default
restrict 0
groups ( 0 )
border not set
unknown-attr-len not set
Once vSmart receives this TLOC route via OMP, it redistributes it to WAN edge router 2. The same process happens on the other side where vEdge-2 advertises its WAN connection to the Internet as TLOC (2.2.2.2, green, ipsec). In the end, both WAN edge routers have all the required information in order to form an overlay tunnel between each other.
Now with the TLOC routes redistributed to both WAN Edge routers, they can attempt to form an overlay tunnel using IPsec and to form a BFD session with each other.
Multiple TLOCs
At this point, it is very likely that you might have thought - "Ah I see, a TLOC is just a tunnel endpoint and one TLOC is equal to one tunnel". If we have a WAN edge router that has two connections to two different WAN providers, for example, two Internet circuits, it would have two TLOCs. One that represents Internet-1 and one that represents Internet-2 as it is visualized on the example shown in figure 5. Well, this is the point where it gets a little bit more complicated.
By default, WAN Edge routers try to form an overlay tunnel to every TLOC over each available WAN transport, including TLOCs that belong to other colors if there is IP reachability between the two transport networks. In the example shown in figure 5, there is any-to-any reachability between the two clouds because they are basically two Internet providers. In this scenario, four IPsec tunnels will form T1-T3, T1-T4, T2-T3, T2-T4. There is one important exception, WAN edge devices do not form overlay tunnels to other devices in the same site ie having the same site-id.
This is where the Cisco SD-WAN overlay magic happens. If you advertise all TLOCs to all WAN edge routers, a full-mesh overlay fabric will be formed (assuming there is IP reachability between the underlay transports). However, if you want to have a custom overlay topology, you just control which particular TLOCs are advertised to which particular WAN edge routers. The logic is as follows - if vEdge-X receives TLOCs Y and Z, it will attempt to form IPsec tunnels from each own TLOC. If vEdge-X does not receive TLOCs Y and Z, it won't ever attempt to establish overlay tunnels to these TLOCs.
Overlay routing and TLOCs
Ok, but what does that mean for the overlay routing? Transport Locators are also a key part of the overlay OMP routing. Each omp route points to a TLOC next-hop and not to the particular overlay tunnel used to get there. By doing that, Cisco SD-WAN abstracts the overlay routing away from underlay routing, and WAN edge devices do not have to keep any underlay information in the overlay routing tables. For an OMP route to be considered as valid, the vEdge device must have at least one IPsec tunnel with a BFD session in UP state to the next-hop TLOC.
vEdge-1# show omp routes
---------------------------------------------------
omp route entries for vpn 1 route 172.16.50.0/24
---------------------------------------------------
RECEIVED FROM:
peer 9.9.9.9
path-id 10
label 1
status C,I,R
loss-reason not set
lost-to-peer not set
lost-to-path-id not set
Attributes:
originator 2.2.2.2
type installed
tloc 2.2.2.2, green, ipsec
ultimate-tloc not set
domain-id not set
overlay-id 1
site-id 20
preference not set
tag not set
origin-proto connected
origin-metric 0
as-path not set
unknown-attr-len not set