VxLAN EVPN Fabrics External Connectivity – VRF Lite

Leave a Comment

Overview

VXLAN EVPN fabrics address the need for workload mobility, flexible resource allocation and multi-tenancy by decoupling workloads from the underlying physical infrastructure. This enables to treat workloads as portable units that can be easily moved across different compute resources within the data center.

Workloads in the data center need the ability to connect to external networks. This external connectivity is essential for connecting systems to networks outside the VXLAN EVPN fabric, such as the internet, a WAN, or another Layer 3 network segment.

The external network can be configured with:

  • Shared external connectivity – In a multi-tenant VXLAN EVPN fabric, a shared external connection can be established to allow workloads within different tenant VRFs to access resources on external networks.
  • VRF Lite handoff – extend end to end segmentation using external connectivity / routing per tenant/VRF
  • SR / MPLS handoff – extend tenant VRFs using a single route peering leveraging SR / MPLS

The choice of whether to implement shared, VRF Lite or SR MPLS external connectivity depends on specific requirements, such as the number of tenants, security needs, hardware support, supportability, and traffic volume. By carefully considering these factors, a scalable solution can be established for external connectivity in a multi-tenant VXLAN EVPN fabric

On this blog VRF Lite handoff using NDFC is demonstrated based on the topology below.

Topology

Assumptions

  • Internal Fabric is already established
  • The topology is built on CML
  • CML version used for the lab – Version: 2.6.0+build.5
  • NDFC version used for the lab – 12.1.3b
  • Nexus 9K is used as edge router

Configuration

Configuration steps:

  1. Create External Fabric
  2. Configure the VRF Lite under the Data Center VXLAN EVPN fabric template
  3. Attach the VRF Lite extensions to VRF on the border nodes
  4. Recalculate and Deploy configurations on Data Center VXLAN EVPN Fabric
  5. Recalculate and Deploy configurations on External Fabric
  6. Verification

Create External Fabric

NDFC (Nexus Dashboard Fabric Controller) allows you to manage the creation of external fabrics that connect your internal network to external resources. Here’re the steps involved to create external fabric:

1. Log to NDFC –

  • navigate to LAN -> Fabrics
  • Actions -> Create Fabric
  • Fabric name – Provide a descriptive name to your external fabric for easy identification within NDFC
  • Click ‘Choose Fabric’ and choose ‘External Connectivity Network’ as a fabric type from the list

2. Define general parameters

NDFC offer options to define general parameters like the AS # for BGP routing and Fabric Monitor Mode

Disable ‘Fabric Monitor Mode’. If enabled fabric is only monitored, no configuration will be deployed

Leave the other configuration parameters unchanged and Click ‘Save’ to create the external fabric

3. Add external devices / switches to the external fabric – Specify the edge routers within external fabric that will act as the peering device with the VxLAN EVPN fabric border switches for the external connection.

Open ‘External Fabric’ -> Switches -> Actions -> Add switches then

Input the ‘Seed IP’, Device Type, and credential (Username & Password) then click ‘Discover Switches’

Select the edge routers / switches from the list and click ‘Add Switches’

Edge routers / switches under the External Fabric

Set role of the switches / routers added to the external fabric

Select the switches (edge routers) and Actions -> Set Role -> Edge Router -> Select

Configure the VRF Lite under the Data Center VXLAN EVPN fabric template

NDFC offers a Data Center VXLAN EVPN fabric template to leverage NDFC’s functionalities to configure VRF Lite on the Border Leaf Switches.

LAN -> Fabrics -> Select a VxLAN EVPN Fabric -> Actions -> Edit Fabric

Verify and update the information under ‘Resources’ tab of the VxLAN EVPN fabric template

  • Change the VRF Lite Deployment from ‘Manual’ to ‘Back2Back&ToExternal’
  • Add VRF Lite Subnet IP Range and VRF Lite Subnet Mask
  • Set the setting and add Subnet IP Range for per VRF per VTEP loopbacks

Attach the VRF Lite extensions to VRF on the border nodes

Based on the topology above, a VXLAN EVPN Fabric is connected to External Fabric. The Data Center VXLAN EVPN fabric has border leaf role, a role for leaf switches within the Data Center VXLAN EVPN fabric designated to handle external connectivity. They act as VTEPs (VXLAN Tunnel Endpoint) and perform routing functions between the VXLAN EVPN fabric and the external network. The External Fabric has edge router role, a role for external devices within the external fabric responsible for routing traffic between the external network and the Data Center VXLAN EVPN fabric. They peer with the Border Leaf Switches .

NDFC shows physical and logical representation of the topology with CDP/LLDP Link discovery.

NDFC topology showing VxLAN EVPN fabric and External Fabrics

Verify the links between VxLAN EVPN Fabric and External Fabric

Navigate LAN > Fabrics, double-click on VxLAN EVPN Fabric (VxLAN-EVPN-Brownfield) ->  Fabric Overview window, click on Links tab. You can view the links detected by NDFC with appropriate policy assigned automatically.

Attach the links to VRF

On the border switches, click VRFs -> VRF Attachments -> Select the VRF from the list -> Actions -> Edit

On the ‘Edit VRF Attachment’ window, make sure Attach is selected and VRF_LITE extension is selected.

Click ‘Edit’ for each link and verify all the information are right, then select ‘Attach’ and click ‘Save’ or click ‘Attach All’ and click save

Repeat this for all VRFs and all border switches

Recalculate and Deploy configurations on Data Center VXLAN EVPN Fabric

From Fabric Overview go to Actions -> Recalculate and Deploy -> Deploy All or Under the border switch overview -> VRFs -> VRF Attachments -> Select VRF -> Actions -> Deploy

This will trigger the VRF and VRF Lite configurations on the Border devices.

Recalculate and Deploy configurations on External Fabric

Configuration is auto generated for the cisco nexus switch edges. Auto configuration is only provided for the Border or Border Spine role in the VXLAN fabric and Edge Router role in the connected external fabric device.

‘Fabric Overview’ of the external fabric -> Actions -> Recalculate and Deploy

Verify the generated config and click ‘Deploy All’

Verification

BGP show commands 

### Border Leaf Switches

### BL-1
BL-1# sh run bgp  #(showing the configuration only relevant to the external connectivity) 

router bgp 65125
  vrf blue
    neighbor 192.168.11.22
      remote-as 65325
      address-family ipv4 unicast
        send-community
        send-community extended
        route-map extcon-rmap-filter out
    neighbor 192.168.11.26
      remote-as 65325
      address-family ipv4 unicast
        send-community
        send-community extended
        route-map extcon-rmap-filter out

  vrf orange
    neighbor 192.168.11.30
      remote-as 65325
      address-family ipv4 unicast
        send-community
        send-community extended
        route-map extcon-rmap-filter out
    neighbor 192.168.11.34
      remote-as 65325
      address-family ipv4 unicast
        send-community
        send-community extended
        route-map extcon-rmap-filter out

BL-1# sh bgp ipv4 unicast summary vrf Blue 
BGP summary information for VRF blue, address family IPv4 Unicast
BGP router identifier 10.10.111.6, local AS number 65125
BGP table version is 38, IPv4 Unicast config peers 4, capable peers 4
11 network entries and 21 paths using 1724 bytes of memory
BGP attribute entries [12/2064], BGP AS path entries [1/6]
BGP community entries [0/0], BGP clusterlist entries [10/40]

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
192.168.11.22   4 65325    1244    1243       38    0    0 20:38:04 1         
192.168.11.26   4 65325    1244    1243       38    0    0 20:38:04 1         
       
BL-1# sh bgp ipv4 unicast summary vrf Orange
BGP summary information for VRF orange, address family IPv4 Unicast
BGP router identifier 10.10.111.6, local AS number 65125
BGP table version is 43, IPv4 Unicast config peers 4, capable peers 4
11 network entries and 19 paths using 1724 bytes of memory
BGP attribute entries [12/2064], BGP AS path entries [1/6]
BGP community entries [0/0], BGP clusterlist entries [10/40]

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
192.168.11.30   4 65325    1245    1244       43    0    0 20:38:15 1         
192.168.11.34   4 65325    1245    1244       43    0    0 20:38:15 1   

### BL-2
BL-2# sh run bgp  #(showing the configuration only relevant to the external connectivity)      
router bgp 65125
  vrf blue
    neighbor 192.168.11.6
      remote-as 65325
      address-family ipv4 unicast
        send-community
        send-community extended
        route-map extcon-rmap-filter out
    neighbor 192.168.11.10
      remote-as 65325
      address-family ipv4 unicast
        send-community
        send-community extended
        route-map extcon-rmap-filter out
  vrf orange
    neighbor 192.168.11.14
      remote-as 65325
      address-family ipv4 unicast
        send-community
        send-community extended
        route-map extcon-rmap-filter out
    neighbor 192.168.11.18
      remote-as 65325
      address-family ipv4 unicast
        send-community
        send-community extended
        route-map extcon-rmap-filter out
  
BL-2# sh bgp ipv4 unicast summary vrf Blue 
BGP summary information for VRF blue, address family IPv4 Unicast
BGP router identifier 10.10.111.3, local AS number 65125
BGP table version is 36, IPv4 Unicast config peers 4, capable peers 4
11 network entries and 21 paths using 1724 bytes of memory
BGP attribute entries [12/2064], BGP AS path entries [1/6]
BGP community entries [0/0], BGP clusterlist entries [10/40]

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
192.168.11.6    4 65325    1253    1252       36    0    0 20:46:50 1         
192.168.11.10   4 65325    1253    1252       36    0    0 20:46:48 1         
       
BL-2# sh bgp ipv4 unicast summary vrf Orange 
BGP summary information for VRF orange, address family IPv4 Unicast
BGP router identifier 10.10.111.3, local AS number 65125
BGP table version is 37, IPv4 Unicast config peers 4, capable peers 4
11 network entries and 19 paths using 1724 bytes of memory
BGP attribute entries [12/2064], BGP AS path entries [1/6]
BGP community entries [0/0], BGP clusterlist entries [10/40]

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
192.168.11.14   4 65325    1253    1252       37    0    0 20:46:55 1         
192.168.11.18   4 65325    1253    1252       37    0    0 20:46:54 1         
        

### Edge Routers
### Ext-1 
Ext-1# show run bgp  #(showing the configuration only relevant to the external connectivity)
router bgp 65325
  vrf Blue
    address-family ipv4 unicast
    neighbor 192.168.11.9
      remote-as 65125
      address-family ipv4 unicast
        send-community
        send-community extended
    neighbor 192.168.11.21
      remote-as 65125
      address-family ipv4 unicast
        send-community
        send-community extended
  vrf Orange
    address-family ipv4 unicast
    neighbor 192.168.11.17
      remote-as 65125
      address-family ipv4 unicast
        send-community
        send-community extended
    neighbor 192.168.11.29
      remote-as 65125
      address-family ipv4 unicast
        send-community
        send-community extended
Ext-1# show bgp ipv4 unicast summary vrf Blue 
BGP summary information for VRF Blue, address family IPv4 Unicast
BGP router identifier 192.168.12.9, local AS number 65325
BGP table version is 11, IPv4 Unicast config peers 3, capable peers 3
3 network entries and 5 paths using 972 bytes of memory
BGP attribute entries [2/344], BGP AS path entries [1/6]
BGP community entries [0/0], BGP clusterlist entries [0/0]

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
192.168.11.9    4 65125    1257    1256       11    0    0 20:50:43 2         
192.168.11.21   4 65125    1257    1256       11    0    0 20:50:44 2                 
         
Ext-1# show bgp ipv4 unicast summary vrf Orange 
BGP summary information for VRF Orange, address family IPv4 Unicast
BGP router identifier 192.168.12.1, local AS number 65325
BGP table version is 11, IPv4 Unicast config peers 3, capable peers 3
3 network entries and 5 paths using 972 bytes of memory
BGP attribute entries [2/344], BGP AS path entries [1/6]
BGP community entries [0/0], BGP clusterlist entries [0/0]

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
192.168.11.17   4 65125    1258    1256       11    0    0 20:50:59 2         
192.168.11.29   4 65125    1258    1256       11    0    0 20:50:59 2         


### Ext-2 
Ext-2# show run bgp  #(showing the configuration only relevant to the external connectivity)
router bgp 65325
  vrf Blue
    address-family ipv4 unicast
    neighbor 192.168.11.5
      remote-as 65125
      address-family ipv4 unicast
        send-community
        send-community extended
    neighbor 192.168.11.25
      remote-as 65125
      address-family ipv4 unicast
        send-community
        send-community extended
  vrf Orange
    address-family ipv4 unicast
    neighbor 192.168.11.13
      remote-as 65125
      address-family ipv4 unicast
        send-community
        send-community extended
    neighbor 192.168.11.33
      remote-as 65125
      address-family ipv4 unicast
        send-community
        send-community extended
Ext-2# sh bgp ipv4 unicast summary vrf Blue 
BGP summary information for VRF Blue, address family IPv4 Unicast
BGP router identifier 192.168.12.13, local AS number 65325
BGP table version is 11, IPv4 Unicast config peers 3, capable peers 3
3 network entries and 5 paths using 972 bytes of memory
BGP attribute entries [2/344], BGP AS path entries [1/6]
BGP community entries [0/0], BGP clusterlist entries [0/0]

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
192.168.11.5    4 65125    1260    1259       11    0    0 20:53:28 2         
192.168.11.25   4 65125    1260    1259       11    0    0 20:53:27 2         
         
Ext-2# sh bgp ipv4 unicast summary vrf Orange 
BGP summary information for VRF Orange, address family IPv4 Unicast
BGP router identifier 192.168.12.5, local AS number 65325
BGP table version is 11, IPv4 Unicast config peers 3, capable peers 3
3 network entries and 5 paths using 972 bytes of memory
BGP attribute entries [2/344], BGP AS path entries [1/6]
BGP community entries [0/0], BGP clusterlist entries [0/0]

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
192.168.11.13   4 65125    1261    1259       11    0    0 20:53:32 2         
192.168.11.33   4 65125    1261    1259       11    0    0 20:53:32 2

Ping test

### Ping test from Host in the Fabric to External endpoints

### from a host (10.10.20.10) in vrf Orange to an external endpoint (172.16.20.1) reachable through vrf Orange
cisco@inserthostname-here:/home$ ping 172.16.20.1
PING 172.16.20.1 (172.16.20.1) 56(84) bytes of data.
64 bytes from 172.16.20.1: icmp_seq=1 ttl=252 time=48.0 ms
64 bytes from 172.16.20.1: icmp_seq=2 ttl=252 time=39.9 ms
64 bytes from 172.16.20.1: icmp_seq=3 ttl=252 time=39.6 ms
64 bytes from 172.16.20.1: icmp_seq=4 ttl=252 time=81.2 ms
64 bytes from 172.16.20.1: icmp_seq=5 ttl=252 time=161 ms
64 bytes from 172.16.20.1: icmp_seq=6 ttl=252 time=83.8 ms
^C
--- 172.16.20.1 ping statistics ---
6 packets transmitted, 6 received, 0% packet loss, time 5017ms
rtt min/avg/max/mdev = 39.627/75.628/161.254/42.357 ms

### from a host (10.10.30.10) in vrf Orange to an external endpoint (172.16.20.1) reachable through vrf Orange
cisco@S-2:~$ ping 172.16.20.1
PING 172.16.20.1 (172.16.20.1): 56 data bytes
64 bytes from 172.16.20.1: seq=0 ttl=252 time=34.622 ms
64 bytes from 172.16.20.1: seq=1 ttl=252 time=49.974 ms
64 bytes from 172.16.20.1: seq=2 ttl=252 time=64.229 ms
64 bytes from 172.16.20.1: seq=3 ttl=252 time=67.741 ms
64 bytes from 172.16.20.1: seq=4 ttl=252 time=36.913 ms
64 bytes from 172.16.20.1: seq=5 ttl=252 time=76.701 ms
^C
--- 172.16.20.1 ping statistics ---
6 packets transmitted, 6 packets received, 0% packet loss
round-trip min/avg/max = 34.622/55.030/76.701 ms

### from a host (10.10.30.10) in vrf Orange to an external endpoint (172.16.30.1) reachable through vrf Blue as expected ping test is not successful
cisco@S-2:~$ ping 172.16.30.1
PING 172.16.30.1 (172.16.30.1): 56 data bytes
^C
--- 172.16.30.1 ping statistics ---
11 packets transmitted, 0 packets received, 100% packet loss

### from a host (10.10.40.10) in vrf Blue to an external endpoint (172.16.30.1) reachable through vrf Blue
cisco@S-3:~$ ping 172.16.30.1
PING 172.16.30.1 (172.16.30.1): 56 data bytes
64 bytes from 172.16.30.1: seq=0 ttl=252 time=41.151 ms
64 bytes from 172.16.30.1: seq=1 ttl=252 time=36.849 ms
64 bytes from 172.16.30.1: seq=2 ttl=252 time=59.567 ms
64 bytes from 172.16.30.1: seq=3 ttl=252 time=40.677 ms
64 bytes from 172.16.30.1: seq=4 ttl=252 time=94.315 ms
64 bytes from 172.16.30.1: seq=5 ttl=252 time=64.158 ms
^C
--- 172.16.30.1 ping statistics ---
6 packets transmitted, 6 packets received, 0% packet loss
round-trip min/avg/max = 36.849/56.119/94.315 ms

### from a host (10.10.50.10) in vrf Blue to an external endpoint (172.16.30.1) reachable through vrf Blue
cisco@S-4:~$ ping 172.16.30.1
PING 172.16.30.1 (172.16.30.1): 56 data bytes
64 bytes from 172.16.30.1: seq=0 ttl=252 time=34.380 ms
64 bytes from 172.16.30.1: seq=1 ttl=252 time=47.235 ms
64 bytes from 172.16.30.1: seq=2 ttl=252 time=54.635 ms
64 bytes from 172.16.30.1: seq=3 ttl=252 time=39.834 ms
64 bytes from 172.16.30.1: seq=4 ttl=252 time=81.218 ms
64 bytes from 172.16.30.1: seq=5 ttl=252 time=64.845 ms
^C
--- 172.16.30.1 ping statistics ---
6 packets transmitted, 6 packets received, 0% packet loss
round-trip min/avg/max = 34.380/53.691/81.218 ms
### from a host (10.10.50.10) in vrf Blue to an external endpoint (172.16.20.1) reachable through vrf Orange as expected ping test is not successful
cisco@S-4:~$ ping 172.16.20.1
PING 172.16.20.1 (172.16.20.1): 56 data bytes
^C
--- 172.16.20.1 ping statistics ---
11 packets transmitted, 0 packets received, 100% packet loss

Related Links

https://deliabtech.com/data-center/vxlan-evpn-fabric/

https://www.cisco.com/c/en/us/td/docs/dcn/ndfc/121x/configuration/fabric-controller/cisco-ndfc-fabric-controller-configuration-guide-121x/m-vrf-lite.html#concept_ihm_rb1_5tb

About

Leave a Comment

Your email address will not be published. Required fields are marked *