In the previous lab lesson, we have enabled Direct Internet access (DIA) to employees in VPN 1. For the users at the remote branches, we have even provided a redundant Internet breakout path in case of local ISP failures. However, we have not examined the security side of things. Allowing employees to access the entirety of the Internet is a significant security concern for any organization. Bypassing the security stack exposes the remote sites to cybersecurity vulnerabilities such as

  • Users accessing unauthorized web and storage locations;
  • Infiltrating data;
  • Fishing, drive-by, and password attacks;
  • Ultimately - ransomware, spyware, trojans, viruses, and so on.

There are two solutions to this problem. The first one that we will implement in this lab is to provide local Internet breakout only to a list of trusted applications. The second solution that we will see in the next lab lesson is to implement cloud-delivered security at branches.

DIA to trusted application only
Figure 1. DIA to trusted applications only

We are going to edit the existing data policy applied to the spokes in such a way so that a few trusted applications are allowed a local Internet breakout at branches while at the same time the untrusted applications are routed through the data center (the hub). In real-world deployments, there would probably be a security stack hosted in the data center that would provide threat defense, URL filtering, and malware protection to the untrusted apps. However, in our lab, the traffic going through the hub will just be routed out the Internet because the focus is on how we provide application-specific policy rules and not how we provide external security. 

Before jumping into the configuration portion, let’s first discuss how Cisco SD-WAN devices classify and recognize applications.

The DPI Engine

Each WAN edge router comes with a Deep Packet Inspection (DPI) engine that allows one to look into packets past the basic information found in the headers. The DPI engine can determine the content of packets and then classify the traffic flow as part of a specific app. This is a key part of the transition from packet-oriented to application-oriented networking. The level of network visibility that the DPI provides allows administrators to encode routing and firewall decisions based on applications.

DPI Engine
Figure 2. DPI Engine

In Cisco SD-WAN, we use the DPI as a match criteria in a centralized data policy. We define the applications of interest in an app-list and then call this list in a match-action sequence. This allows us to set specific actions to the application traffic. For example, we can match all VoIP traffic and route it through the highly available MPLS circuits while at the same time we match all web traffic and route it through the best-effort Internet links.

From a configuration standpoint, we enable the application visibility directly on vEdge routers using the following command:

vEdge-6(config)# policy app-visibility

Then we can monitor the applications using the “show app dpi flows” and “show app dpi flows” commands.

vEdge-6# show app dpi flows | t

                                 Source Dest                                         
VPN  Source IP  Destination IP   Port   Port   PROTOCOL  APPLICATION  FAMILY         
------------------------------------------------------------------------------- 
1    10.6.1.4   8.8.8.8          64279  53     udp       dns          Network Srvc
1    10.6.1.4   34.240.223.28    61481  443    tcp       adobe        Web            
1    10.6.1.4   104.244.42.131   60663  443    tcp       twitter      Web                    
1    10.6.1.4   142.250.187.162  56218  443    tcp       google_ads   Web            
1    10.6.1.4   157.240.9.35     56200  443    tcp       facebook     Web            
1    10.6.1.4   172.217.17.214   59290  443    tcp       youtube      Web            
1    10.6.1.4   172.217.17.227   53390  80     tcp       gstatic      Web            
1    10.6.1.4   172.217.17.234   60956  443    tcp       google_gen   Web                  
1    10.6.1.4   212.39.68.50     57756  443    tcp       linkedin     Forum

Cisco vEdge devices use Network-Based Application Recognition (NBAR2) as a deep packet inspection (DPI) engine. NBAR is a classification engine that recognizes a wide variety of applications and network protocols. It inspects packets content and identifies the applications according to predefined traffic characteristics called application signatures. Cisco packages a large database of app signatures directly into the routers’ software image (Vitela OS or IOS-XE). Cisco updates the signature database in each new software revision according to the latest application characteristics. Over the years, Cisco NBAR evolved to a DPI that can successfully differentiate 2000+ network apps, as illustrated in figure 3.

NBAR Evolution
Figure 3. NBAR Evolution

The first packet in flow (FIF) problem

Deep packet inspection engines such as Cisco NBAR are not perfect. There are specific network scenarios where the NBAR cannot classify a particular traffic flow from the very beginning, as illustrated in figure 4. Typically, most DPI engines score poorly or even erroneously when classifying the very first packets in a traffic flow (FIF). 

FIF / Traffic Symmetry problem.
Figure 4. FIF / Traffic Symmetry problem.

We will not cover in detail why this happens but instead will focus on what this means for us network engineers. If a router’s DPI engine cannot successfully recognize a network application from the very first packet, the execution of any application-specific rules specified in a data policy would not trigger instantaneously. Additionally, suppose the unclassified packets are not explicitly accounted for. In that case, they could be dropped by the default action at the end of the policy, and consequently, the app-specific rules may never trigger. 

When designing data policies with application-specific rules, the general rule of thumb is to always change the default action at the end of the policy to “accept” and have appropriate routing in the VPN route table. Or alternatively, to explicitly account for any unclassified packets in a match-action rule that is placed after the app-specific rules. 

The Traffic Symmetry Problem

Figure 4 highlights another fundamental property of the application visibility engine - the DPI/NBAR2 engine is a localized network function. It only sees what goes through the WAN edge router and has no way of knowing what happens across the entire network. Therefore, for a router to correctly identify most applications, it has to be able to see the application traffic in both directions. In dual-router branches without any data policy applied, the traffic is unlikely to constantly pass through the same WAN edge router because each network flow is ECMP hashed based on the IP header and forwarded over all equal-cost paths.

The solutions to this problem are a few:

  • We can design our data policies in such a way so that they account for the fact that the DPI engine might never identify some applications;
  • In a dual-router branch office, we can set up the routing in such a way so that one WAN Edge router is always preferred over the other, basically resulting in an active-standby design. This will guarantee that both directions of every application traffic flow will pass through the same router (the active one);
  • We can use a more sophisticated application visibility service called Cisco Software-Defined Application Visibility and Control (SD-AVC) that is designed to solve both the FIF and Traffic Symmetry problems. 

SD-AVC Overview

Cisco SD-AVC is a centralized network service that aggregates app classification information and synchronizes application states between network devices. It works by defining two roles in the network - Sensors and Consumers. Sensors are routers with NBAR2 that export classification information to the SD-AVC service. Consumers are routers that receive this classification information from the SD-AVC service. A WAN edge router can be both a sensor and a consumer at the same time.

The SD-AVC service runs in a container on vManage and as a linux container (LXC) on WAN edge routers (currently supported only on IOS-XE). The SD-AVC service does not replace the DPI/NBAR2 but complements it and provides the following improvements: 

  • Better app classification in asymmetric routing environments;
  • Better app classification for encrypted applications;
  • Improved FIF recognition (first packet in flow);
  • Improved DPI/NBAR2 performance;
  • Automatic Protocol-Pack updates;
  • Cloud connector that provides the latest available app information;
  • Better visibility of unclassified traffic.

Figure 5 below shows how Cisco SD-AVC would improve the application recognition in the asymmetric routing scenario illustrated in figure 4.

How SD-AVC solves the FIF problem
Figure 5. How SD-AVC solves the FIF problem

Configuring Application-specific policy rules

In our lab example, we will edit the existing data policy to spokes and add an application-specific rule at sequence 21 that will match the Internet-bound traffic to linkedin.com and provide local Internet breakout at branches. However, to account for the FIF packets, we will configure one additional match-all rule at sequence 31, that sends all unclassified packets to the hub so they can be successfully routed through the hub’s DIA. Figure 6 illustrates what will happen once the data policy is applied and in effect. vEdge6’s DPI engine will not recognize the traffic flow to LinkedIn from the very first packet. Therefore, the flow will not be matched at sequence 21. However, the flow will be matched at sequence 31 and sent to the hub. The hub will route the traffic out to the Internet and back to vEdge-6. Once the DPI engine recognizes that this traffic belongs to the application LinkedIn, the traffic will be matched at sequence 21 of the data policy and forwarded locally by the DIA at vEdge-6.

Configuring Application-specific policy rules
Figure 6. Configuring Application-specific policy rules

Let’s now see all this in action. First, we will need to turn on the application visibility on all vEdge routers in the topology.

vEdge-6# conf t 
#this should be configured on all vEdges (1 through 6)
Entering configuration mode terminal
vEdge-6(config)# policy app-visibility
vEdge-6(config-policy)# commit and-quit 
Commit complete.

Then we are going to specify a new app list that matches the LinkedIn predefined app. Of course, you can play around and specify a longer list of applications.

policy
 lists
  app-list TRUSTED-APPS
   app linkedin

Then, we will edit the current data policy DATA-POLICY-TO-SPOKES that is applied to the branches. At sequence 21, we are going to match the app-list TRUSTED-APPS. This will match all packets that the DPI classifies as part of the application LinkedIn and forward them through the local Internet breakout. However, as we said earlier, the very first packets would not be recognized and matched at sequence 21. That is why we will configure one additional sequence 31 that matches all packets that had not been matched earlier in the policy and send them to the hub. For that, we will use a tloc-list that we have already configured in lab#1 in the previous chapter for control policies. 

policy 
 data-policy DATA-POLICY-TO-SPOKES
  vpn-list VPN1
   sequence 11
    match
     destination-data-prefix-list ENTERPRISE-IPS
    !
    action accept
   !
   sequence 21
    match
     app-list TRUSTED-APPS
    !
    action accept
     nat use-vpn 0
     nat fallback
  !
   sequence 31
    action accept
     set
      tloc-list HUB-TLOCS vpn 1
     !
   !
   default-action drop
  !

Once the policy is edited and the changes are committed, we can check the forwarding decision of vEdge-6 using the “show policy service-path” command. Notice that the destination IP does not matter because the forwarding decision is based on the application and not on the destination IP address.

vEdge-6# show policy service-path vpn 1 interface ge0/2 source-ip 10.6.1.4 dest-ip 0.0.0.0 protocol 6 app linkedin all
Number of possible next hops: 1
Next Hop: Remote
  Remote IP: 39.3.1.1, Interface ge0/0 Index: 4

Any other traffic flow part of another application (for example youtube) will be sent to the hub using any of the hub’s tlocs as next hop.

vEdge-6# show policy service-path vpn 1 interface ge0/2 source-ip 10.6.1.4 dest-ip 0.0.0.0 protocol 6 app youtube all 
Number of possible next hops: 4
Next Hop: IPsec
  Source: 10.10.0.3 12346 Destination: 10.10.0.1 12346 Color: mpls
Next Hop: IPsec
  Source: 39.3.0.3 12346 Destination: 39.3.0.1 12346 Color: biz-internet
Next Hop: IPsec
  Source: 10.10.0.3 12346 Destination: 10.10.0.2 12346 Color: mpls
Next Hop: IPsec
  Source: 39.3.0.3 12346 Destination: 39.3.0.2 12346 Color: biz-internet

Key takeaways

  • Notice that prior to SD-WAN version 20.6.1, vEdge and IOS XE routers were using different classification engines. vEdge routers were using the Qosmos classification engine while IOS XE devices were using Cisco NBAR2;
  • As of version 20.6.1 All Cisco SD-WAN routers use NBAR2 as a Deep Packet Inspection (DPI) engine;
  • Each Cisco device comes with a software image that has an embedded application signature database called Application Pack.
  • Data policies that have application-specific rules must account for the first packets in flow (FIF) and the Traffic Symmetry problems.
  • For more sophisticated application-specific routing designs, it is highly recommended that the network utilizes the centralized Cisco SD-AVC service, which is not covered in this course.