In the last lesson, we saw the differences between traditional routing protocols and overlay routing with OMP. We have seen that OMP does not reroute around failures in the overlay by default. This lesson will demonstrate a Cisco SD-WAN feature that protects against direct and indirect overlay failures using a control-policy option called tloc-action.

Failures in the overlay

Before we begin, let's again see how OMP handles failures. If you haven't checked our previous lesson on OMP routing, it is good to do so before continuing with this one.

Direct Failures

Figure 1 shows a basic overlay topology with three vEdges connected to a single transport. Direct failure in the topology leads to a network outage, even though an alternative overlay path exists. For example, if tunnel T2-T3 goes down, branches lose connectivity to each other, even though an alternative overlay path via the data center is available.

This happens because overlay routing is different from traditional underlay routing:

• There is no auto-discovery of routing peers and next-hops like in traditional routing protocols. All nodes peer only with the vSmart controller and not between each other.
• Subsequently, the routing topology never changes (because it is an overlay).
• There is no topology recalculation when there is a failure in the overlay.
• What actually changes is only the reachability to the next-hop tlocs (tracked with BFD).

Indirect Failures

Figure 2 shows an example where traffic is directed (via control policy) to an intermediate hop, which then forwards the traffic to its final destination. In this case, an indirect failure leads to traffic blackholes. For example, if the tunnel T1-T3 goes down, there is no way for Branch-2 to know that the data center doesn't have an overlay path to Branch-3, so it still sends the traffic destined to Branch-3 to the DC, and the traffic gets dropped.

You can clearly see the difference between overlay routing and the well-known routing protocols such as EIGRP and OSPF. Although the topology is a triangle and an alternative path always exists in case of one failure, one tunnel down leads to a network outage.

What is TLOC-Action?

TLOC Action is a control-policy option that inserts an intermediate hop in an OMP route to a destination prefix. The Smart controller then tracks the overlay path between the intermediate hop and the ultimate tloc. If that path goes down, the controller informs the WAN edge routers that received this OMP route.

Figure 3 shows an example of how tloc-action works.

1. vEdge-3 advertises its connected network 10.1.3.0/24 with next-hop its local tloc T3. The route is "10.1.3.0/24 via T3".
2. This route matches a control-policy sequence on vSmart. In the action portion of the sequence, the policy sets a new tloc and sets one of the available tloc-actions. This new tloc that is inserted by the policy is the intermediate hop.
3. The controller advertises the OMP route to vEdge-2. The route has two next-hops now - intermediate tloc (inserted by the policy) and ultimate tloc (the original route's next-hop). This tells vEdge-2 that the destination prefix 10.1.3.0/24 is reachable via T1 and via T3.
4. Now it is up to the specified tloc action to define how the traffic to 10.1.3.0 will be forwarded. There are four tloc-actions:
   1. primary
   2. backup
   3. ecmp
   4. strict (default)  

Now vEdge-2 knows that 10.1.3.0/24 is reachable via T1 and via T3. The router tracks both overlay tunnels T2-T1 and T2-T3 with BFD, but the router doesn't have a way to track the tunnel T1 - T3. However, if that tunnel goes down, vSmart will inform vEdge-2 about the status change. From the perspective of vEdge-2, this creates a triangle in the overlay and gives a choice - in normal circumstances, vEdge-2 can send the traffic destined to 10.1.3.0/24 via T1, it can send it via T3, it can send it via both tlocs or it can send it via none. Well, this is basically what the tloc-action parameter defines - where to send the traffic in normal circumstances and where to send the traffic in case of a failure. Let's look closely at each tloc-action.

TLOC-Action Primary

The "tloc-action primary" specifies that in normal circumstances, the communication between vEdge-2 and vEdge-3 goes through the intermediate hop.

If the overlay tunnel between T1 and T3 goes down, vSmart informs vEdge-2 and the router forwards the traffic directly to T3.

This option is useful when we want to force specific traffic through a data center or a regional hub for security inspection and at the same time keep the direct overlay path to the destination as an alternative. 

TLOC-Action Backup

The "tloc-action backup" specifies that in normal circumstances, the communication between vEdge-2 and vEdge-3 goes directly to the ultimate-tloc (the original route's next-hop).

If the overlay tunnel between T2 and T3 goes down, vEdge-2 forwards the traffic through the intermediate hop T1.

This option is useful when we want to force specific traffic through the direct overlay path to a destination and at the same time have an alternative path via intermediate hop.

TLOC-Action ECMP

The "tloc-action ecmp" specifies that in normal circumstances, the communication between vEdge-2 and vEdge-3 is load-balanced through the intermediate hop T1 and the direct next-hop T3.

If the overlay tunnel between T1 and T3 goes down, vSmart informs vEdge-1 to stop sending traffic via the intermediate router.

If the direct tunnel T2-T3 goes down, vEdge-2 stops forwarding traffic over that overlay path.

TLOC-Action Strict (Default Option)

The "tloc-action strict" specifies that in normal circumstances, the communication between vEdge-2 and vEdge-3 goes through T1, the intermediate hop.

If the overlay tunnel between T1 and T3 goes down, vEdge-2 drops the traffic.

This option is useful in cases where we want to ensure that traffic either gets inspected at the intermediate hop or gets dropped but does not reach the final destination uninspected.

TLOC-Action Example

The output below shows vEdge-2's OMP route to 10.1.3.0/24 with tloc-action primary applied. The tloc highlighted with orange is the intermediate hop, and the tloc highlighted with green is the original route's next hop.

---------------------------------------------------
omp route entries for vpn 1 route 10.1.3.0/24
---------------------------------------------------
            RECEIVED FROM:                   
peer            1.1.1.30
path-id         17
label           1004
status          C,I,R
loss-reason     not set
lost-to-peer    not set
lost-to-path-id not set
    Attributes:
     originator       3.3.3.3
     type             installed
     tloc             1.1.1.1, blue, ipsec
     ultimate-tloc    3.3.3.3, blue, ipsec -- primary
     domain-id        not set
     overlay-id        1
     site-id          2
     preference       not set
     tag              not set
     origin-proto     connected
     origin-metric    0
     as-path          not set
     community        not set
     unknown-attr-len not set

Notice the following restrictions:

• The intermediate tloc and the ultimate tloc must be the same color for the tloc-action to work.
• The intermediate router must be enabled for service TE.
   

Key Takeaways

• TLOC Action is a control-policy option that inserts an intermediate hop in an OMP route to a destination prefix.
• The tloc-action parameter specifies how to forward traffic to the destination prefix via an intermediate hop. 
• There are four different tloc actions:
  • Primary
    • In normal operation, forwards the traffic via the intermediate tloc.
    • In case of failure, forwards the traffic via the ultimate tloc.
  • Backup
    • In normal operation, forwards the traffic via the ultimate tloc.
    • In case of failure, forwards the traffic via the intermediate tloc.
  • ECMP
    • In normal operation, load-balance the traffic via both overlay paths.
    • In case of failure on one path, forward the traffic via the other.
  • Strict (default option)
    • In normal operation, forwards the traffic via the intermediate tloc.
    • In case of failure, drop the traffic.

For further reading on this topic, check out our configuration examples on TLOC Action:

• End-to-End Path Tracking
• Preferred DC in a dual-DC design