Cisco SD-WAN is a major topic in the CCE Enterprise blueprint now. Network engineers that want to pass the lab exam should have extensive hands-on experience with the solutions. However, for people who do not have the chance to touch it at work, it is actually not easy to get access to a practice lab. In this lesson, I will show you one way to set up a fully functional Cisco SD-WAN home lab on EVE-NG that can be used to practice every topic in the exam's blueprint. 

Setup EVE-NG

One thing I'd like to mention is that Cisco SD-WAN requires a lot of processing power. For a small practicing topology consisting of 1 controller of each type and 3-4 vEdges, you should give the EVE-NG VM at least 8 vCPUs and 16GB of RAM. However, if you want to make a large topology with redundant controllers and many vEdge devices, you must have a lot of computing resources at your disposal.

Cisco SD-WAN Images

You will need to have the following Cisco SD-WAN images to set up this practice lab environment on EVE-NG:

Software Images Required
EVE-NG Image Filename Version
vmanage-16.3.2 viptela-vmanage-genericx86-64.ova 16.3.2
vsmart-16.3.2 viptela-smart-genericx86-64.ova 16.3.2
vbond-16.3.2 viptela-edge-genericx86-64.ova 16.3.2
vedge-17.1 viptela-edge-genericx86-64.ova 17.1

Once you have the images, you need to create a folder for each one and then transfer the image to EVE-NG using a FileZilla or WinSCP tool. After you upload the images to their respective folders in EVE-NG, you need to convert the ova files to qcow2. Note that we need to create an additional virtual disk for vManage. This is done with the command highlighted in green: 

#  Creating Folders 
mkdir /opt/unetlab/addons/qemu/vtbond-16.3.2
mkdir /opt/unetlab/addons/qemu/vtsmart-16.3.2
mkdir /opt/unetlab/addons/qemu/vtmgmt-16.3.2
mkdir /opt/unetlab/addons/qemu/vtedge-17.1

##vManage
cd /opt/unetlab/addons/qemu/vtmgmt-16.3.2
tar -xvf viptela-vmanage-genericx86-64.ova
/opt/qemu/bin/qemu-img convert -f vmdk -O qcow2 viptela-vmanage-genericx86-64-disk1.vmdk virtioa.qcow2
/opt/qemu/bin/qemu-img create -f qcow2 virtiob.qcow2 150G 

##vSmart
cd /opt/unetlab/addons/qemu/vtsmart-16.3.2
tar -xvf viptela-smart-genericx86-64.ova
/opt/qemu/bin/qemu-img convert -f vmdk -O qcow2 viptela-smart-genericx86-64-disk1.vmdk virtioa.qcow2

##vBond
cd /opt/unetlab/addons/qemu/vtbond-16.3.2
tar -xvf viptela-edge-genericx86-64.ova
/opt/qemu/bin/qemu-img convert -f vmdk -O qcow2 viptela-edge-genericx86-64-disk1.vmdk virtioa.qcow2

##vEdge
cd /opt/unetlab/addons/qemu/vtedge-17.1
tar -xvf viptela-edge-genericx86-64.ova
/opt/qemu/bin/qemu-img convert -f vmdk -O qcow2 viptela-edge-genericx86-64-disk1.vmdk virtioa.qcow2

##Fix permissions
/opt/unetlab/wrappers/unl_wrapper -a fixpermissions

Physical Topology

The physical topology that we are going to use is as follows. You should re-create it on EVE-NG. If you want to copy/paste some of the configs, make sure that you use the same interfaces when creating the topology on EVE-NG.

Physical Topology
Figure 1. Physical Topology

You have to decide how many vEdge devices you are going to need and add them in the same manner as vEdge-1. The process of building the lab starts with Cisco SD-WAN version 16.3.2. Once you upgrade to the higher versions you won't be able to add more vEdges.

Default credentials for all devices are admin/admin. When vManage boots for the first time, it will ask on which storage device to install the software. Please make sure to use the virtual disk you have created in the previous step.

viptela 16.3.2 

vmanage login: admin
Password: 
Welcome to Viptela CLI
admin connected from 127.0.0.1 using console on vmanage
Available storage devices:
vda     18GB
vda1    2GB
vda2    16GB
vdb     150GB
hdc     3GB
1) vda
2) vda1
3) vda2
4) vdb
5) hdc
Select storage device to use: 4
Would you like to format vdb? (y/n): y

Bootstrap Configuration

Once all devices boot up it is time to enable basic connectivity between the controllers and all WAN edge devices. The following bootstrap snippets are the minimum required configuration in order to achieve basic connectivity.

vBond

!
system
 host-name vBond
 system-ip 1.1.1.10
 site-id 1
 organization-name networkacademy-io
 vbond 10.1.1.10 local vbond-only
!
vpn 0
 interface ge0/0
  ip address 10.1.1.10/24
  no tunnel-interface
  no shutdown
 !
!

vManage

!
system
 host-name vManage
 system-ip 1.1.1.20
 site-id 1
 organization-name networkacademy-io
 vbond 10.1.1.10
!
vpn 0
 interface eth0
  ip address 10.1.1.20/24
  no shutdown
 !
!
vpn 512
 interface eth1
  ip address 192.168.115.20/24
  no shutdown
 !
!

vSmart

system
 host-name vSmart
 system-ip 1.1.1.30
 site-id 1
 organization-name networkacademy-io
 vbond 10.1.1.10
!
vpn 0
 interface eth0
  ip address 10.1.1.30/24
  no tunnel-interface
  no shutdown
 !
!

vEdges

system
 host-name vEdge-1
 system-ip 1.1.1.40
 site-id 100
 organization-name networkacademy-io
 vbond 10.1.1.10
!
vpn 0
 interface ge0/0
  ip address 10.1.1.40/24
  no tunnel-interface
  no shutdown
 !
!

At this point, each device should successfully ping any other in VPN0. If for whatever reason there is no reachability to one of the devices, you should not continue ahead but troubleshoot and resolve the issue.

Certificates

Cisco SD-WAN Controllers can not be brought into operation unless their identity is validated by an established chain of trust. This identity validation process is intended to ensure that only trusted devices can join the SD-WAN solution while still retaining flexibility. Each controller must have a root certificate installed and a controller certificate installed and signed by a trusted CA (Certification Authority).

For creating this lab environment we are going to use the vBond controller as a Root CA.

vBond as Root CA

Configure vBond to act as a root of trust. The first step is to generate an RSA private key. Then we generate a ROOTCA.pem certificate and sign it with the ROOTCA.key private key that we have just created.

##Use vshell / viptela_cli to switch between shell and cli modes
vBond# vshell 
vBond:~$ openssl genrsa -out ROOTCA.key 2048
Generating RSA private key, 2048 bit long modulus
........+++..............................................+++
e is 65537 (0x10001)

vBond:~$ openssl req -x509 -new -nodes -key ROOTCA.key -sha256 -days 1024 \
>     -subj "/C=AU/ST=NSW/L=NSW/O=networkacademy-io/CN=sdwan.lab" \
>     -out ROOTCA.pem

vBond:~$ ls -l
total 12
-rw-r--r-- 1 admin admin 1679 Mar 23 09:41 ROOTCA.key
-rw-r--r-- 1 admin admin 1285 Mar 23 09:41 ROOTCA.pem
-rw-r--r-- 1 admin admin  392 Mar 23 09:37 archive_id_rsa.pub

Once that is completed, the root certificate should be installed on all other devices. It will act as a root-of-trust for all controller certificates.

# Excecute the following command in CLI mode on all devices
request root-cert-chain install scp://admin@10.1.1.10:/home/admin/ROOTCA.pem vpn 0

Uploading root-ca-cert-chain via VPN 0
Copying ... admin@10.1.1.10:/home/admin/ROOTCA.pem via VPN 0
Warning: Permanently added '10.1.1.10' (ECDSA) to the list of known hosts.
viptela 16.3.2 
admin@10.1.1.10's password: 
ROOTCA.pem                                    100% 1285     1.1MB/s   00:00    
Successfully installed the root certificate chain

If that step is successful, that means that all devices will have their chain-of-trust pointing to the vBond controllers ROOTCA.pem.

Certificates' Relations
Certificates' Relations

At this point, each controller should have a root certificate installed. This can be checked with the following command:

vSmart# show certificate root-ca-cert 
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            a4:f7:2a:2f:d8:64:74:98
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C=AU, ST=NSW, L=NSW, O=networkacademy-io, CN=sdwan.lab
        Validity
            Not Before: Apr 10 09:25:18 2021 GMT
            Not After : Jan 29 09:25:18 2024 GMT
        Subject: C=AU, ST=NSW, L=NSW, O=networkacademy-io, CN=sdwan.lab
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
## lines omitted

Now you need to log in to the vManage GUI interface. This is done using a web browser and entering the URL https://[vManage-VPN512-IP-address]:8443. Default credentials are admin/admin.

Once logged in, you need to go to Administration > Settings and set the Organization Name to be networkacademy-io, then edit the vBond address, and set it to 10.1.1.10. Make sure that Certificate Authorization is set to Manual as shown on the screenshot below.

vManage GUI Initial Config
vManage GUI Initial Config

Once that is done, go to Configuration > Devices > Controllers > Add Controller and add both vBond (10.1.1.10) and vSmart (10.1.1.30) using the GUI.

Adding the controllers
Adding the controllers

When both controllers are added you should see something similar to the screenshot below. Note that the Certificate Status of all controllers is Not-installed.

Adding the controllers via vManage GUI
Adding the controllers via vManage GUI

Now you need to go to Configuration > Certificates > Controllers and Generate CSR for all controllers. When you are done, all should be in Status "CSR Generated".

Generating CSRs
Generating CSRs

At this point, each controller must have a certificate signing request (CSR) generated. Now it is time to transfer all CSRs to vBond and sign them with the Root CA Private Key.

vBond# vshell
vBond:~$ scp admin@10.1.1.30:/home/admin/vsmart_csr vSmart.csr
vBond:~$ scp admin@10.1.1.20:/home/admin/vmanage_csr vManage.csr
vBond:~$ scp admin@10.1.1.10:/home/admin/vbond_csr vBond.csr

vBond:~$ ls -l
total 28
-rw-r--r-- 1 admin admin 1679 Mar 23 09:41 ROOTCA.key
-rw-r--r-- 1 admin admin 1285 Mar 23 09:41 ROOTCA.pem
-rw-r--r-- 1 admin admin  392 Mar 23 09:37 archive_id_rsa.pub
-rw-r--r-- 1 admin admin 1224 Mar 23 12:12 vBond.csr
-rw-r--r-- 1 admin admin 1228 Mar 23 12:12 vManage.csr
-rw-r--r-- 1 admin admin 1224 Mar 23 12:12 vSmart.csr
-rw-r--r-- 1 root  root  1224 Mar 23 12:05 vbond_csr

When you list the directory and see that all CSRs are present, you can go ahead and sign them.

vBond:~$ openssl x509 -req -in vBond.csr \
>     -CA ROOTCA.pem -CAkey ROOTCA.key -CAcreateserial \
>     -out vBond.crt -days 500 -sha256
Signature ok

vBond:~$ openssl x509 -req -in vSmart.csr \
>     -CA ROOTCA.pem -CAkey ROOTCA.key -CAcreateserial \
>     -out vSmart.crt -days 500 -sha256
Signature ok
		
vBond:~$ openssl x509 -req -in vManage.csr \
>     -CA ROOTCA.pem -CAkey ROOTCA.key -CAcreateserial \
>     -out vManage.crt -days 500 -sha256
Signature ok

Now you should see all controller certificates.

vBond:~$ ls -alh | grep crt
-rw-r--r-- 1 admin admin 1.3K Mar 23 12:16 vBond.crt
-rw-r--r-- 1 admin admin 1.3K Mar 23 12:16 vManage.crt
-rw-r--r-- 1 admin admin 1.3K Mar 23 12:18 vSmart.crt

It is time to install all controller certificates via the vManage GUI. Go to Configuration > Certificates > Controllers > Install Certificate

Installing the certificates using vManage GUI
Installing the certificates using vManage GUI

Now that you have all certificates (.crt) in vBond's directory, you just cat each of them and paste the output in the Install Certificate window as shown below:

vBond:~$ cat vManage.crt
-----BEGIN CERTIFICATE-----
MIIDpjCCAo4CCQCpxofKmIPl+TANBgkqhkiG9w0BAQsFADBZMQswCQYDVQQGEwJB
VTEMMAoGA1UECAwDTlNXMQwwCgYDVQQHDANOU1cxGjAYBgNVBAoMEW5ldHdvcmth
Y2FkZW15LWlvMRIwEAYDVQQDDAlzZHdhbi5sYWIwHhcNMjEwNDEwMDk0MDQwWhcN
MjIwODIzMDk0MDQwWjCB0DELMAkGA1UEBhMCVVMxEzARBgNVBAgTCkNhbGlmb3Ju
aWExETAPBgNVBAcTCFNhbiBKb3NlMRowGAYDVQQLExFuZXR3b3JrYWNhZGVteS1p
bzEUMBIGA1UEChMLdklQdGVsYSBJbmMxQzBBBgNVBAMUOnZtYW5hZ2VfOWVmYTcz
Y2MtMmIxNi00NzNhLWFjMGUtNTQyZjhjNGRhMzg5XzAudmlwdGVsYS5jb20xIjAg
BgkqhkiG9w0BCQEWE3N1cHBvcnRAdmlwdGVsYS5jb20wggEiMA0GCSqGSIb3DQEB
AQUAA4IBDwAwggEKAoIBAQDO4mHeqgQ/i59TRuKjmB+i70wbzvTVf606nBzf44MD
qgsFY3hwBgWcu5dcOEkkI8uCkTFoQqinQtgtw5Ucd3aE0e8skLwy6W1xzcl1YtvV
88C23BSVc66A7p/dT5g7Z4UWCm4O4BER46uNMalTZKYBGGRONMv0e1VfJgAUEN9/
bLgQMwyFdawhIYq/32rEUhms3mSunO56pAjmUBWcO19FVAFBJQXdUlpqFvniMfWW
K6/YXFZQ5Y9kj7TH3tLLfkZQJ62/vohBVkdwLQvm9N2XJqL/ZEa8XPRSLs56Wy0W
/WbqX2gs0emSXTQN99Npq7ILX8fF+w7W+Rqcb9Pqe3cRAgMBAAEwDQYJKoZIhvcN
AQELBQADggEBABssqE1Ds8bEVp5+NXDzw1gNZmU1S+j/1Affk+FVHivIXXp8tJq5
iJU8yUYFQQeXSXQHtJWeDEIGKZ5q3Q7MB/NnxqQtyDzF3ejQ69DDWwoWI2YcCz0e
g5H1Lk+2wKLRXYCvFkarMDIlbMNwoehUSiJDQUbDBJsNo5oqFa2bARB5NHoLYHrS
O7yU24F0qSRBR4HmZuwAbvBgTrVMUyjj0NIgprb3IqpqdSpwIxtL1o8GS10Q82sR
ICO1TBbYlqPvozd7HYRRgFGmG+9VIaORKLhPoJcM7tvIdQxmdrCItYATUy2djp6E
GyOO7U0MLBDbTzCkfl4huCOcbu/dWNk6zJk=
-----END CERTIFICATE-----
Paste the certificate into vManage
Paste the certificate into vManage

If everything is good up to this point, the certificate should install successfully. 

Certificate installed successfully
Certificate installed successfully

You repeat this for all controllers. In the end, when you go to Configuration > Certificates > Controllers, you should see that all controllers have Certificate Serial numbers. If that is the case, you click the Send-to-vBond function to propagate this information to vBond.

Updating vBond
Updating vBond

At this point, all controllers should be operational with valid certificates. Now it is time to validate the vEdges.

vEdges Validation

We need to generate a certificate signing request (CSR) for every vEdge. This is done using the following command in CLI mode. Note that the name of the CSR changes for each vEdge.

vEdge-1# request csr upload scp://admin@10.1.1.10:/home/admin/vedge-01.csr vpn 0
vEdge-2# request csr upload scp://admin@10.1.1.10:/home/admin/vedge-02.csr vpn 0
...
vEdge-N# request csr upload scp://admin@10.1.1.10:/home/admin/vedge-nn.csr vpn 0

Uploading CSR via VPN 0
Enter organization name            : networkacademy-io
Re-enter organization name          : networkacademy-io
Generating private/public pair and CSR for this vedge device          
Generating CSR for this vedge device   ........(DONE) 
Copying ... admin@10.1.1.10:/home/admin/vedge-01.csr via VPN 0
Warning: Permanently added '10.1.1.10' (ECDSA) to the list of known hosts.
viptela 16.3.2 
admin@10.1.1.10's password: 
server.csr                                    100% 1224    56.4KB/s   00:00    
CSR upload successful

When you finish with the above, on vBond you should see all WAN Edges' CSRs.

vBond:~$ ls -alh | grep vedge
-rw-r--r-- 1 admin admin 1.2K Mar 23 13:34 vedge-01.csr
-rw-r--r-- 1 admin admin 1.2K Mar 23 13:34 vedge-02.csr
-rw-r--r-- 1 admin admin 1.2K Mar 23 13:34 vedge-nn.csr

Now it is time to sign each one of them. Note that the certification name changes for each vEdge - vEdge-01.crt, vEdge-02.crt and so on.

vBond:~$ openssl x509 -req -in vedge-01.csr \
    -CA ROOTCA.pem -CAkey ROOTCA.key -CAcreateserial \
    -out vEdge-01.crt -days 500 -sha256
	
vBond:~$ openssl x509 -req -in vedge-02.csr \
    -CA ROOTCA.pem -CAkey ROOTCA.key -CAcreateserial \
    -out vEdge-02.crt -days 500 -sha256

vBond:~$ openssl x509 -req -in vedge-nn.csr \
    -CA ROOTCA.pem -CAkey ROOTCA.key -CAcreateserial \
    -out vEdge-nn.crt -days 500 -sha256

Once all CSRs are signed and a valid certificate is generated for each WAN edge router, it is time to install the certificates. This is done by executing the following command on each vEdge in CLI mode:

vEdge-1# request certificate install scp://admin@10.1.1.10:/home/admin/vEdge-01.crt vpn 0
vEdge-2# request certificate install scp://admin@10.1.1.10:/home/admin/vEdge-02.crt vpn 0
...
vEdge-nn# request certificate install scp://admin@10.1.1.10:/home/admin/vEdge-nn.crt vpn 0

Installing certificate via VPN 0
Copying ... admin@10.1.1.10:/home/admin/vEdge-01.crt via VPN 0
Warning: Permanently added '10.1.1.10' (ECDSA) to the list of known hosts.
viptela 16.3.2 

admin@10.1.1.10's password: 
vEdge-01.crt                                  100% 1326    31.3KB/s   00:00    
Successfully installed the certificate

In the end, each vEdge must have a certificate serial number.

vEdge-1# show certificate serial
Chassis number: 37a459dc-9447-4f44-b6cb-07e3c3190c8a serial number: E3C382FF265B63F1
...
vEdge-N# show certificate serial
Chassis number: bfba45b7-038b-476c-83b7-a19742ff6091 serial number: E3C382FF265B63F2

Now you need to copy the output of the show certificate serial of all vEdges in a Notepad file in the format shown below, and then save the file as vedges.csv.

Collect all vEdges serial numbers
Collect all vEdges serial numbers

Then you go to Configuration > Devices > vEdge List > Upload vEdge List and select the vedges.csv file. Make sure to check the "Validate the uploaded vEdge list and send to controllers" option.

Upload vEdge-list
Upload vEdge-list

If the upload is successful you should see all vEdge devices having a Chassis number and Certificate Serial as shown below. 

All vEdges
All vEdges

At this point, the validation of all devices is done. We need to bring up the control plane and upgrade to the CCIE Enterprise Infrastructure lab exam's version 18.4.4. 

Tunnel-Interfaces bring up

To bring up the control plane, we must enable the overlay tunnels on all devices. This is done using the following configuration:

# on vManage and vSmart
!
vpn 0
 interface eth0
  tunnel-interface
!

# on vBond and vEdges
!
vpn 0
 interface ge0/0
  tunnel-interface
   encapsulation ipsec
   allow-service all
!

Once this is applied, you should be able to see that the control plane is up on all controllers. However, the vEdges would not be able to join the overlay before we upgrade to at least version17.2.8.

vBond# show orchestrator connections 
                                                                            PEER                      PEER                                                    
PEER     PEER     PEER             SITE        DOMAIN      PEER             PRIVATE  PEER             PUBLIC                                                  
TYPE     PROTOCOL SYSTEM IP        ID          ID          PRIVATE IP       PORT     PUBLIC IP        PORT    REMOTE COLOR     STATE           UPTIME         
--------------------------------------------------------------------------------------------------------------------------------------------------------------
vsmart   dtls     1.1.1.30         1           1           10.1.1.30        12346    10.1.1.30        12346   default          up              0:00:00:38     
vsmart   dtls     1.1.1.30         1           1           10.1.1.30        12446    10.1.1.30        12446   default          up              0:00:00:37     
vmanage  dtls     1.1.1.20         1           0           10.1.1.20        12346    10.1.1.20        12346   default          up              0:00:00:49     
vmanage  dtls     1.1.1.20         1           0           10.1.1.20        12446    10.1.1.20        12446   default          up              0:00:00:48     
vmanage  dtls     1.1.1.20         1           0           10.1.1.20        12546    10.1.1.20        12546   default          up              0:00:00:48     
vmanage  dtls     1.1.1.20         1           0           10.1.1.20        12646    10.1.1.20        12646   default          up              0:00:00:48    


vManage# show control connections 
                                                                                             PEER                                          PEER                                              
      PEER    PEER PEER            SITE       DOMAIN PEER                                    PRIV  PEER                                    PUB                                               
INDEX TYPE    PROT SYSTEM IP       ID         ID     PRIVATE IP                              PORT  PUBLIC IP                               PORT  REMOTE COLOR    STATE           UPTIME     
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
0     vsmart  dtls 1.1.1.30        1          1      10.1.1.30                               12346 10.1.1.30                               12346 default         up              0:00:01:03 
0     vbond   dtls 1.1.1.10        0          0      10.1.1.10                               12346 10.1.1.10                               12346 default         up              0:00:01:19 
1     vbond   dtls 1.1.1.10        0          0      10.1.1.10                               12346 10.1.1.10                               12346 default         up              0:00:01:18 
2     vbond   dtls 1.1.1.10        0          0      10.1.1.10                               12346 10.1.1.10                               12346 default         up              0:00:01:18 
3     vbond   dtls 1.1.1.10        0          0      10.1.1.10                               12346 10.1.1.10                               12346 default         up              0:00:01:19 

Software Upgrade

The images required for upgrading to 17.2.8 are as follow:

Images for upgrading to 17.2.8
Device Image Name
vSmart/vBond viptela-18.4.4-x86_64.tar
vManage vmanage-18.4.4-x86_64.tar

The software upgrade is pretty simple and straightforward. You upload the necessary files in the Software Repository by going to Maintenance > Software Upgrade > Controller > Repository

vManage Repository
vManage Repository

In there, you select Add new software and upload the files for version 17.2.8 to vManage.

Uploading Software images to the vManage repository
Uploading Software images to the vManage repository

Then you go to Maintenance > Software Upgrade > vManage > Upgrade, select version 17.2.8 and select Upgrade. The upgrading process is quick, the new software will be installed on vManage, but will not be activated.

Upgrading vManage
Upgrading vManage to 17.2.8

The last thing that you need to do is to Activate the new version software. This is done at Maintenance > Software Upgrade > vManage > Activate. At this point, the controller will reload. The process usually takes some time. In some cases, you may need to delete the browser's cookies to log in to the new version of the GUI once it boots with the new image.

Activating the new software image on vManage
Activating the new software image on vManage

Once you log in again, you will note that the GUI's layout is more modern. At this point, vManage is upgraded. However, the other controllers are not upgraded yet. You must go to Maintenance > Software Upgrade > Controller and select all controllers. Then select the new version 17.2.8 and check the Activate and Reboot option. Then click Upgrade. The controllers will reload.

Upgrading vBond and vManage
Upgrading vBond and vManage

Once all controllers are upgrade and fully loaded, you must reload all vEdges in order to make them join the control plane.

Full Content Access is for Registered Users Only (it's FREE)...

  • Learn any CCNA, DevNet or Network Automation topic with animated explanation.
  • We focus on simplicity. Networking tutorials and examples written in simple, understandable language for beginners.