In real-world deployments, a typical use case is a dual-homed branch with each vEdge router directly connected to a single WAN transport only. We can see such an example shown in figure 1 below, where vEdge-1 is connected only to the Internet while vEdge-2 is connected only to the MPLS. Having only one WAN transport per-router seriously reduces the network resiliency and creates traffic inefficiencies in the overlay fabric. If vEdge-1's ge0/0 interface or link goes down, the router will lose connectivity to the SD-WAN overlay fabric.

One solution is to connect both routers to both WAN clouds. However, this might not be the most cost-efficient way because of the contracts with service providers. For example, in many parts of the world, service providers provision and charge per access port, which means that connecting vEdge-1 to the MPLS cloud would require the organization to pay extra fees for a second port/circuit.

What is a TLOC Extension?

TLOC extension is a Cisco SD-WAN feature that allows a vEdge router to communicate over the adjacent vEdge's WAN transports through a TLOC extension interface. Figure 2 shows an example where vEdge-1 is directly connected to the Internet and uses the feature to connect to the MPLS transport via vEdge-2. In the end, vEdge-1 has overlay tunnels built over both WAN clouds even though it is directly attached only to one of the WAN transports. On the opposite side, vEdge-2 is attached to the MPLS cloud and uses the TLOC extension feature to connect to the Internet via vEdge-1.
 

The TLOC extension feature is set up per interface and provides transparent connectivity from one interface (called a TLOC extension interface) to a particular WAN transport. Let's look at vEdge-1, for example. It is unaware that the green tunnel to the MPLS cloud passes through another vEdge router because vEdge-2 extends their directly connected interface transparently to the MPLS transport.

TLOC Extension Types

We can extend a vEdge router to the opposite transport in multiple ways. The simplest, most instructive use-case we will see in this example is directly connecting the vEdge routers using two separate interfaces, as illustrated in figure 2 above. However, if we want to be a little bit more efficient port-wise, we can use a physical connection between the routers configured as a dot1Q trunk and then use two separate directly connected VLANs. We can also configure TLOC extension through the site-local LAN, typically over the distribution switches. In this case, the routers must be L2-adjacent and in the same subnet.

We can go even further and configure TLOC extension over the site-local L3 routing in some exceptional circumstances. In this case, the vEdge routers are separated by an L3 device and their extension links, which could be dedicated interfaces or dot1q subinterfaces, are in different subnets. The L3 TLOC-extension is implemented using GRE tunnels and is typically not recommended unless necessary. Figure 3 illustrates the various deployment types.

Configuring TLOC Extension

We always configure a TLOC extension interface in the transport VPN 0. The configuration is pretty simple - we assign an IP address and specify the WAN interface to which the TLOC Extension is bound. Let's look at figure 4, for example. To extend vEdge-2 to the Internet, we connect it to vEdge-1's extension interface ge0/7, which is bound to the WAN interface connected to the Internet - ge0/0. On the opposite side, to extend vEdge-1 to the MPLS cloud, we link it to vEdge-2's extension interface ge0/7, which is bound to the MPLS cloud through ge0/1. Then we configure static default routes in VPN0 on each WAN Edge router, pointing to the adjacent vEdge's extension interface IP as a next-hop, and that's it.

Routing Considerations 

You can see that the TLOC extension config is pretty straightforward. However, we must always consider the reverse IP reachability from the transport networks back to the TLOC extension interfaces. Otherwise, the overlay tunnels and BFD sessions won't come up. 

For example, to ensure that packets can be routed from the MPLS cloud back to the TLOC extension interface, we must configure vEdge-2 to advertise subnet-B to the MPLS provider over the PE-CE routing protocol. On the other hand, in a typical production deployment, subnet-A would always be from the private IP address space. Therefore, we must enable NAT on vEdge-1's interface to the Internet so that subnet-A can be translated to a publicly routable address before being sent out. Otherwise, the packets would be dropped at the Internet edge.

Let's now go through a configuration example and see how we implement the TLOC-Extention feature.

Initial State

Before we begin with the actual configuration, let's quickly check the initial state of the network. We have vEdge-1 connected only to the Internet and vEdge-2 connected to the MPLS, as shown in the following lab diagram. Let's verify that using CLI.

If we check the interfaces on vEdge-1 using the show interface command, we will see that vEdge-1 has only one TLOC configured on ge0/0. Notice the column PORT TYPE. If an interface is listed as "transport," that means the interface has a TLOC configuration. The interfaces listed as port type transport are the local TLOCs of a vEdge router. They are not regular L3 interfaces that we are used to in traditional networking, though. A transport interface acts as a tunnel endpoint and will only accept data traffic destined for itself coming from remote DTLS/TLS/IPsec peers. It won't work as a transit interface and won't forward traffic coming from the underlay destined for other destinations, even though the router may have routing entries for the destination subnet. For example, vEdge-1's ge0/0 interface has an IP address 39.3.0.1/24 and is configured as a local TLOC. It will only accept traffic from the underlay that is destined to IP address 39.3.0.1.  

vEdge-1# sh int | t
#some rows and columns are omitted for clarity
                                            IF      IF              
                      AF                    ADMIN   OPER            
VPN    INTERFACE      TYPE  IP ADDRESS      STATUS  STATUS PORT TYPE
--------------------------------------------------------------------
0      ge0/0          ipv4  39.3.0.1/24     Up      Up     transport
0      ge0/1          ipv4  -               Down    Down   service
0      ge0/6          ipv4  -               Down    Down   service  
0      ge0/7          ipv4  -               Down    Down   service  
0      system         ipv4  1.1.1.1/32      Up      Up     loopback 
2      ge0/2          ipv4  10.1.2.1/24     Up      Up     service  
3      ge0/3          ipv4  10.1.3.1/24     Up      Up     service  
4      ge0/4          ipv4  10.1.4.1/24     Up      Up     service  
5      ge0/5          ipv4  10.1.5.1/24     Up      Up     service  
512    eth0           ipv4  -               Down    Down   service  

If we check the running configuration of the transport interface ge0/0 that is connected to the Internet, we can see that it has an IP address and tunnel-interface configuration. Under the tunnel-interface configuration (highlighted in orange), we configure the TLOC parameters such as color and encapsulation type. This interface is marked with the "biz-internet" color and configured with encapsulation type "ipsec." Therefore, this interface is a local tunnel endpoint (TLOC) that can be uniquely identified with the tuple {1.1.1.1, biz-internet, ipsec}. Notice that the tuple that identifies this TLOC uses the system-ip address 1.1.1.1 and not the interface IP address 39.3.0.1. This is a common mistake that engineers make in the beginning.

vEdge-1# sh run vpn 0 int ge0/0
#some rows and columns are omitted for clarity
vpn 0
 interface ge0/0
  description WAN-INET
  ip address 39.3.0.1/24
  !
  tunnel-interface
   encapsulation ipsec
   color biz-internet
   allow-service all
  !
  no shutdown
!

Ok, so vEdge-1 has only one local TLOC on the interface connected to the Internet. The easiest way to check how many overlay tunnels are established to a local TLOC is by looking at the BFD sessions. Remember that once an overlay tunnel is established, BFD is automatically started on top and cannot be disabled. Therefore, each BFD session represents one overlay tunnel.

If we look at the output of show bfd sessions on vEdge-1, we can see that it has tunnels to all other vEdges with the exception of vEdge-2 because vEdge-2 is at the same location (has the same site-id 1). Notice that all tunnels are established over the local color "biz-internet". That is because vEdge-1 does not have a connection to other WAN networks and hence does not have other local TLOCs that can form tunnels. 

Also, notice that vEdge-1 has formed overlay tunnels to all colors. Recall that a vEdge attempts to form an overlay tunnel to all remote TLOCs it has received (from vSmart) over each of its own TLOCs (over each WAN transport).
For example, the highlighted BFD session is to TLOC {3.3.3.3, mpls, ipsec} over the local colors biz-internet. This is the BFD session that runs on the tunnel between vEdge-1's biz-internet interface and vEdge-3's mpls interface. The second BFD session is between vEdge-1's biz-internet interface and vEdge-3's biz-internet interface, and so on.

vEdge-1# show bfd sessions | t
#some columns are omitted for clarity

                             SYSTEM   SITE                                   
SRC IP     DST IP     PROTO  IP       ID    LOCAL COLOR   COLOR         STATE
-----------------------------------------------------------------------------
39.3.0.1   10.10.0.3  ipsec  3.3.3.3  3     biz-internet  mpls          up   
39.3.0.1   39.3.0.3   ipsec  3.3.3.3  3     biz-internet  biz-internet  up   
39.3.0.1   10.10.0.4  ipsec  4.4.4.4  4     biz-internet  mpls          up   
39.3.0.1   39.3.0.4   ipsec  4.4.4.4  4     biz-internet  biz-internet  up   
39.3.0.1   10.10.0.5  ipsec  5.5.5.5  5     biz-internet  mpls          up   
39.3.0.1   39.3.0.5   ipsec  5.5.5.5  5     biz-internet  biz-internet  up   
39.3.0.1   10.10.0.6  ipsec  6.6.6.6  6     biz-internet  mpls          up   
39.3.0.1   39.3.0.6   ipsec  6.6.6.6  6     biz-internet  biz-internet  up   

If we quickly check the initial state of vEdge-2, we will see that it has only one transport interface, ge0/1.

vEdge-2# sh int | t
#some columns are omitted for clarity
                                            IF      IF              
                      AF                    ADMIN   OPER            
VPN    INTERFACE      TYPE  IP ADDRESS      STATUS  STATUS PORT TYPE
--------------------------------------------------------------------
0      ge0/0          ipv4  -               Down    Down   service
0      ge0/1          ipv4  10.10.0.2/24    Up      Up     transport
0      ge0/6          ipv4  -               Down    Down   service  
0      ge0/7          ipv4  -               Down    Down   service  
0      system         ipv4  1.1.1.2/32      Up      Up     loopback 
2      ge0/2          ipv4  10.1.2.2/24     Up      Up     service  
3      ge0/3          ipv4  10.1.3.2/24     Up      Up     service  
4      ge0/4          ipv4  10.1.4.2/24     Up      Up     service  
5      ge0/5          ipv4  10.1.5.2/24     Up      Up     service  
512    eth0           ipv4  -               Down    Down   service 

We can see that the transport interface is marked with the "mpls" color, which means that we have a local TLOC {2.2.2.2, mpls, ipsec}.

vEdge-1# sh run vpn 0 int ge0/1
#some columns are omitted for clarity
vpn 0
 interface ge0/1
  description WAN-MPLS
  ip address 10.10.0.2/24
  tunnel-interface
   encapsulation ipsec
   color mpls
   allow-service all
  !
  no shutdown
!

If we check the BFD sessions, we will see that it has established an overlay tunnel to all vEdges with the exception of vEdge-1 because it has the same site-id. Notice that vEdge-2 has only one local TLOC marked with the "mpls" color and has formed a tunnel to all remote colors over the mpls interface. 

vEdge-2# show bfd sessions | t
#some columns are omitted for clarity

                               SYSTEM   SITE                                 
SRC IP       DST IP     PROTO  IP       ID   LOCAL COLOR   COLOR        STATE
-----------------------------------------------------------------------------
10.10.0.2    10.10.0.3  ipsec  3.3.3.3  3    mpls          mpls         up   
10.10.0.2    39.3.0.3   ipsec  3.3.3.3  3    mpls          biz-internet up   
10.10.0.2    10.10.0.4  ipsec  4.4.4.4  4    mpls          mpls         up   
10.10.0.2    39.3.0.4   ipsec  4.4.4.4  4    mpls          biz-internet up     
10.10.0.2    10.10.0.5  ipsec  5.5.5.5  5    mpls          mpls         up   
10.10.0.2    39.3.0.5   ipsec  5.5.5.5  5    mpls          biz-internet up     
10.10.0.2    10.10.0.6  ipsec  6.6.6.6  6    mpls          mpls         up   
10.10.0.2    39.3.0.6   ipsec  6.6.6.6  6    mpls          biz-internet up  

Extending vEdge-1 to the MPLS

Let's first extend vEdge-1 to the MPLS cloud. For this example, we will use the private subnet 10.10.1.0/30 between vEdge-1's Ge0/1 and vEdge-2's ge0/7 as illustrated in figure 6 below. 

To have IP reachability in the direction from the MPLS cloud back to the TLOC extension interface, we will just add static routing on the MPLS router. However, in real-world deployments, the subnet 10.10.1.0/30 would most likely be advertised to the provider's edge router via BGP.

It is important to understand that in order to extend vEdge-1 to the MPLS cloud, the actual TLOC extension happens on the adjacent router. From the perspective of vEdge-1, Ge0/1 is connected to the MPLS provider in the same way as it would have been with a direct attachment link to the provider itself. Therefore, there is nothing special in the configuration on the vEdge-1, as you can see below. The transport interface is configured with IP address, color, and encapsulation type, and a default route to the MPLS cloud is specified. That's about it.