In the last lesson, we looked at the main architectural components of the Cisco SD-WAN solution. Now in this lesson, we are going to try to explain how everything fits together to create an operational overlay fabric that can move user flows more securely and intelligently than Traditional WAN.

SD-WAN Deployment

When a company decides to migrate its traditional WAN architecture to Software-Defined WAN, the thing that always comes first is to deploy the controllers. The next step is to migrate the main data centers and hub sites and lastly the remote sites such as campuses and branches. 

The main idea for doing it in this sequence is to have the hub sites route the traffic between the SD-WAN and non-SD-WAN sites for the period of the migration. Of course, if it is a brand new ground-up deployment, the sequence does not matter that much.

Controllers Deployment Options

One of the main advantages of the Software-Defined WAN is that the controllers can be deployed in the public cloud. This can significantly reduce the CAPEX/OPEX costs and improve the overall availability and redundancy of the management plane/control plane. Compare this model to the scenario in which you have all controllers deployed on-premises. You need to accommodate rack space, power, cooling, physical servers, hypervisor, and virtual machines or containers. You have to manage redundancy and backups on your own. Using the cloud options, you can consume the management/control plane as IaaS (Infrastructure-as-a-Service) or even SaaS (Software-as-a-Service).

Cisco offers the following options to customers to choose from:

• Cisco-hosted cloud - The information that I have found regarding the existing deployments shows that most customers (above 90%) opt for this approach. This is also the vendor's recommended model because Cisco takes care of provisioning all controllers, they handle the backup and disaster recovery. The customer is basically consuming the SD-WAN control plane as a Software-as-a-Service (SaaS) by using the vManage to create custom configuration templates for their device and administer the overlay fabric.
• Public cloud - The customer could decide to host the controllers in the public clouds such as Azure and AWS. In this scenario, the controllers could be managed by a service provider or by the customer. 
• On-prem - Of course, the controllers can be deployed in the company's data centers or private clouds. In this scenario, the customer is responsible for backups and disaster recoveries. This is usually the case with financial and government institutions that must be compliant with regional regulators.

Once the controllers are up and running, they must establish secure connections between them. As of now, there are two options to choose from when it comes to the underlying secure protocol - TLS which uses TCP transport, and DTLS which uses UDP transport. By default, all controllers use the DTLS option.

If the SD-WAN is deployed in a zero-trust environment, figure 3 shows the Layer 4 information for all permanent connections between the controllers. Note that each core on vManage and vSmart makes a permanent DTLS connection to the vBond resulting in four connections between vManage and vBond and two connections between vSmart and vBond.

WAN Edge Routers Onboarding

Secure onboarding of the WAN edge devices is a very important part of the SD-WAN solution.

Identity and Whitelisting

The Cisco SD-WAN solution uses a whitelisting model for authenticating and trusting the vEdge devices. This means that before a WAN edge router is allowed to join the control plane, it has to be known by all SD-WAN controllers beforehand. Each device is uniquely identified by its Chassis ID and certificate serial number. 

Controllers reachability 

Once the SD-WAN controllers are deployed and have valid certificates, WAN edge routers can start the onboarding process. At this point, the most important thing is to make sure that the vEdge appliances have reachability to all controllers via all available transports. It sounds like an easy and straightforward step at first, but when you look more deeply into it, you can see that there are some decisions to be made. 

The Edge device tries to establish a control connection over each provisioned transport, first initiating one to the vBond orchestrator before attempting to connect to the other controllers. All available transports are tried one at a time, starting with the WAN connection to the lowest interface number. Let's look at the typical scenario where a remote site has one private MPLS circuit and one broadband Internet connection. The WAN edge device would try to connect to controllers over the Internet and the MPLS line. But if the controllers are deployed in a public cloud or any other 3rd part cloud, would the public IP addresses of the controllers be reachable over the MPLS circuit by default? I guess not, there is no MPLS service having all public prefixes redistributed into it. 

There are three common implementations that solve this issue:

1. The MPLS has reachability to the public cloud by being routed through a data center or regional hub that has both transport. This method is shown on the left side of figure 4. 
2. The public routable IP addresses of the controllers are redistributed into the MPLS cloud and the provider edge router advertises them to the vEdge. This method is shown on the right side of figure 4. 
3. It is possible to establish a control plane connection through the Internet connection only. The Edge would be able to join the SD-WAN fabric but would not have a control plane redundancy, so this approach is not recommended at all.

Onboarding process

Joining the overlay fabric

Let's now put everything together and follow each step of the process of joining a WAN edge device to the overlay fabric. Let's say that for this example we are going to use a Viptela vEdge 1000. 

• Step 0. IP Reachability - Upon bootup, the vEdge device obtains an IP address, default gateway, and DNS information via DHCP. If no DHCP service is available onsite, this information could be configured manually using CLI or using a configuration template.
• Step 1. Zero-Touch Provisioning - The WAN Edge router tries to reach the ZTP server by resolving the URL ztp.viptela.com and uses HTTPS to get information about the SD-WAN vBond orchestrator along with the organization name.
• Step 2. Authentication -  The WAN edge device authenticates to the orchestrator with its root-certificate and serial number. If the authentication is successful, the vBond sends back the vManage and vSmart
  controller information.
• Step 3. Connection to the Management Plane - The Edge then establishes a secure connection to the vManage and downloads the configuration using NETCONF. 
• Step 4. Connection to the Control Plane - If all previous steps are successful, the router establishes a secure connection to the vSmart controllers and joins the SD-WAN overlay fabric

The process of jointing the SD-WAN overlay is visualized in Figure 6. Note that some secure connections are transient and some are permanent.

SD-WAN Operation, Administration, and Management

The beauty of the Software-Defined WAN is that it is operated as-a-System rather than the traditional box-to-box approach. This opens up a whole new world of possibilities when it comes to the Operation, Administration, and Management (OAM) of the solution. Some of the main benefits are:

• Centralized management  - and operational simplicity, resulting in reduced change and deployment times.
• Transport-independent overlay - Because the underlay transport is abstracted away from the overlay fabric, any combination of transports can be used in an active/active fashion. This significantly reduces the bandwidth costs of the company.
• Sophisticated security - If you compare the traditional control plane security of OSPF and BGP to the control plane encryption of the SD-WAN, the latter is obviously more comprehensive using certificate identity with a zero-trust security model.  
• Application visibility - Real-time analysis and application visibility are a core part of the solution. This enables the enforcement of service-level agreements (SLA) and tracking of specific performance metrics

  

We obviously won't be able to go deeply into any step of the process of establishing the overlay fabric. However, this is a short summary of how Cisco SD-WAN works. We are going to deep-dive into each step of the process in the next sections of this course.