Cisco VxLAN EVPN Fabric Configuration

Leave a Comment

1. Overview

VxLAN EVPN fabric / architecture is one of the widely deployed DC network solutions that can be scaled to thousands of networks across a wide range of geographical regions.

VLANs have been used to provide network segmentation in data center networks. But it’s limitation on addressing the growing need of scale, multi-tenancy and resiliency have made it unfit to the ever-changing demand of the new digital transformation paradigm. VLAN uses 12-bit identifier which limits the segmentation to about 4000 distinct logical networks. The spanning tree loop prevention mechanism also results in an inefficient use of available network links as a way of ensuring a loop free network topology. As modern apps are now a mesh of micro-services with truly distributed codes and data, VLAN based infrastructure limits the ability to build a large, secure, and multi-tenant DC infrastructure.

VxLAN is an overlay technology designed to provide Layer 2 and Layer 3 connectivity extension over a generic IP network. VxLAN, with its 24-bit identifier has the capability to scale the layer 2 segment isolation to about 16 million distinct logical segments. Since the underlay is IP based no spanning tree required and use the links available efficiently. It addresses the shortcoming of VLAN based DC fabric seen today.

VxLAN EVPN uses Multi-Protocol BGP (MP-BGP) with l2vpn evpn address family as a control plane to exchange layer 2 and Layer 3 information.

VxLAN and MP-BGP creates a powerful technology used to build a large, secure, and resilient multi-tenant web scale fabric that can scale to host hundreds of thousands of systems.

This blog focuses on VxLAN EVPN fabric using Nexus 9K

2. Topology, Connectivity and IP addressing

The following topology is used to demonstrate step by step configuration of a fully functioning Cisco VxLAN EVPN fabric.

Figure 1 – VxLAN Topology

Figure 2 – Connectivity and IP addressing

3. Configuration Steps

3.1 Underlay Network

In a VXLAN EVPN fabric, the underlay network forms the physical foundation that carries the VXLAN tunnels. Here’s a breakdown of key considerations for configuring the underlay network for VXLAN EVPN:

  1. IP addressing and routing:
    • The underlay network provides IP connectivity between all Leaf and Spine switches in the fabric using any unicast routing protocol (e.g., OSPF, IS-IS, or eBGP). This allows reachability between VTEPs for VXLAN tunnel establishment and communication.
  2. MTU:
    • VXLAN adds 50 bytes (or 54 bytes with 802.1Q header) to the original packet size. Consider increasing the MTU on the underlay network devices if necessary.
  3. Underlay BUM traffic:
    • Cisco VxLAN utilizes BGP EVPN for the control plane but it still needs mechanism to handle Broadcast, Unknown Unicast, and Multicast (BUM) traffic within the VxLAN fabric to be forwarded.
  4. vPC Configuration:
    • vPC configuration is required on leaf switches supporting M-LAG (Multi-chassis Link Aggregation Group) connectivity from systems.

3.1.1 Spines configuration

### Spine-1 
feature nxapi
feature tacacs+
nv overlay evpn
feature ospf
feature bgp
feature pim
feature lldp
feature nv overlay
feature ngoam


ip pim rp-address 10.10.100.100 group-list 239.1.25.0/24
ip pim anycast-rp 10.10.100.100 10.10.100.1
ip pim anycast-rp 10.10.100.100 10.10.100.2

interface Ethernet1/1
  description connected-to-BL-1-Ethernet1/1
  mtu 9216
  ip address 10.10.25.1/30
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2
  description connected-to-BL-2-Ethernet1/1
  mtu 9216
  ip address 10.10.25.9/30
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/3
  description connected-to-L-1-Ethernet1/1
  mtu 9216
  ip address 10.10.25.17/30
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/4
  description connected-to-L-2-Ethernet1/1
  mtu 9216
  ip address 10.10.25.25/30
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/5
  description connected-to-L-3-Ethernet1/1
  mtu 9216
  ip address 10.10.25.33/30
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/6
  description connected-to-L-4-Ethernet1/1
  mtu 9216
  ip address 10.10.25.41/30
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface mgmt0
  vrf member management
  ip address 10.14.14.21/24

interface loopback0
  description Routing loopback interface
  ip address 10.10.100.1/32
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode

interface loopback254
  description RP loopback interface
  ip address 10.10.100.100/32
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode


router ospf UNDERLAY
  router-id 10.10.100.1

### Spine-2
feature nxapi
feature tacacs+
nv overlay evpn
feature ospf
feature bgp
feature pim
feature lldp
feature nv overlay
feature ngoam


ip pim rp-address 10.10.100.100 group-list 239.1.25.0/24
ip pim anycast-rp 10.10.100.100 10.10.100.1
ip pim anycast-rp 10.10.100.100 10.10.100.2

interface Ethernet1/1
  description connected-to-BL-1-Ethernet1/2
  mtu 9216
  ip address 10.10.25.5/30
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2
  description connected-to-BL-2-Ethernet1/2
  mtu 9216
  ip address 10.10.25.13/30
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/3
  description connected-to-L-1-Ethernet1/2
  mtu 9216
  ip address 10.10.25.21/30
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/4
  description connected-to-L-2-Ethernet1/2
  mtu 9216
  ip address 10.10.25.29/30
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/5
  description connected-to-L-3-Ethernet1/2
  mtu 9216
  ip address 10.10.25.37/30
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/6
  description connected-to-L-4-Ethernet1/2
  mtu 9216
  ip address 10.10.25.45/30
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface mgmt0
  vrf member management
  ip address 10.14.14.22/24

interface loopback0
  description Routing loopback interface
  ip address 10.10.100.2/32
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode

interface loopback254
  description RP loopback interface
  ip address 10.10.100.100/32
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode


router ospf UNDERLAY
  router-id 10.10.100.2

3.1.2 BLeaf nodes configuration

### BL-1
feature nxapi
cfs eth distribute
nv overlay evpn
feature ospf
feature bgp
feature pim
feature interface-vlan
feature vn-segment-vlan-based
feature lacp
feature dhcp
feature lldp
feature nv overlay
feature ngoam

ip pim rp-address 10.10.100.100 group-list 239.1.25.0/24


interface Ethernet1/1
  description connected-to-spine-1-Ethernet1/1
  no switchport
  mtu 9216
  ip address 10.10.25.2/30
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2
  description connected-to-spine-2-Ethernet1/1
  no switchport
  mtu 9216
  ip address 10.10.25.6/30
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface mgmt0
  vrf member management
  ip address 10.14.14.11/24

interface loopback0
  description Routing loopback interface
  ip address 10.10.100.11/32
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode

interface loopback1
  description VTEP loopback interface
  ip address 10.10.100.111/32
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode

router ospf UNDERLAY
  router-id 10.10.100.11
### BL-2
feature nxapi
cfs eth distribute
nv overlay evpn
feature ospf
feature bgp
feature pim
feature interface-vlan
feature vn-segment-vlan-based
feature lacp
feature dhcp
feature lldp
feature nv overlay
feature ngoam

ip pim rp-address 10.10.100.100 group-list 239.1.25.0/24


interface Ethernet1/1
  description connected-to-spine-1-Ethernet1/2
  no switchport
  mtu 9216
  ip address 10.10.25.10/30
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2
  description connected-to-spine-2-Ethernet1/2
  no switchport
  mtu 9216
  ip address 10.10.25.14/30
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface mgmt0
  vrf member management
  ip address 10.14.14.12/24

interface loopback0
  description Routing loopback interface
  ip address 10.10.100.12/32
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode

interface loopback1
  description VTEP loopback interface
  ip address 10.10.100.112/32
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode

router ospf UNDERLAY
  router-id 10.10.100.12
BL-1# sh ip route 
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

10.10.25.0/30, ubest/mbest: 1/0, attached
    *via 10.10.25.2, Eth1/1, [0/0], 00:15:28, direct
10.10.25.2/32, ubest/mbest: 1/0, attached
    *via 10.10.25.2, Eth1/1, [0/0], 00:15:28, local
10.10.25.4/30, ubest/mbest: 1/0, attached
    *via 10.10.25.6, Eth1/2, [0/0], 00:15:27, direct
10.10.25.6/32, ubest/mbest: 1/0, attached
    *via 10.10.25.6, Eth1/2, [0/0], 00:15:27, local
10.10.25.8/30, ubest/mbest: 1/0
    *via 10.10.25.1, Eth1/1, [110/80], 00:15:04, ospf-UNDERLAY, intra
10.10.25.12/30, ubest/mbest: 1/0
    *via 10.10.25.5, Eth1/2, [110/80], 00:15:11, ospf-UNDERLAY, intra
10.10.25.16/30, ubest/mbest: 1/0
    *via 10.10.25.1, Eth1/1, [110/80], 00:15:04, ospf-UNDERLAY, intra
10.10.25.20/30, ubest/mbest: 1/0
    *via 10.10.25.5, Eth1/2, [110/80], 00:15:11, ospf-UNDERLAY, intra
10.10.25.24/30, ubest/mbest: 1/0
    *via 10.10.25.1, Eth1/1, [110/80], 00:15:04, ospf-UNDERLAY, intra
10.10.25.28/30, ubest/mbest: 1/0
    *via 10.10.25.5, Eth1/2, [110/80], 00:15:11, ospf-UNDERLAY, intra
10.10.25.32/30, ubest/mbest: 1/0
    *via 10.10.25.1, Eth1/1, [110/80], 00:15:04, ospf-UNDERLAY, intra
10.10.25.36/30, ubest/mbest: 1/0
    *via 10.10.25.5, Eth1/2, [110/80], 00:15:11, ospf-UNDERLAY, intra
10.10.25.40/30, ubest/mbest: 1/0
    *via 10.10.25.1, Eth1/1, [110/80], 00:15:04, ospf-UNDERLAY, intra
10.10.25.44/30, ubest/mbest: 1/0
    *via 10.10.25.5, Eth1/2, [110/80], 00:15:11, ospf-UNDERLAY, intra
10.10.100.1/32, ubest/mbest: 1/0
    *via 10.10.25.1, Eth1/1, [110/41], 00:15:04, ospf-UNDERLAY, intra
10.10.100.2/32, ubest/mbest: 1/0
    *via 10.10.25.5, Eth1/2, [110/41], 00:15:11, ospf-UNDERLAY, intra
10.10.100.11/32, ubest/mbest: 2/0, attached
    *via 10.10.100.11, Lo0, [0/0], 00:15:27, local
    *via 10.10.100.11, Lo0, [0/0], 00:15:27, direct
10.10.100.12/32, ubest/mbest: 2/0
    *via 10.10.25.1, Eth1/1, [110/81], 00:13:43, ospf-UNDERLAY, intra
    *via 10.10.25.5, Eth1/2, [110/81], 00:13:41, ospf-UNDERLAY, intra
10.10.100.13/32, ubest/mbest: 2/0
    *via 10.10.25.1, Eth1/1, [110/81], 00:12:09, ospf-UNDERLAY, intra
    *via 10.10.25.5, Eth1/2, [110/81], 00:12:09, ospf-UNDERLAY, intra
10.10.100.14/32, ubest/mbest: 2/0
    *via 10.10.25.1, Eth1/1, [110/81], 00:11:37, ospf-UNDERLAY, intra
    *via 10.10.25.5, Eth1/2, [110/81], 00:11:33, ospf-UNDERLAY, intra
10.10.100.15/32, ubest/mbest: 2/0
    *via 10.10.25.1, Eth1/1, [110/81], 00:04:46, ospf-UNDERLAY, intra
    *via 10.10.25.5, Eth1/2, [110/81], 00:04:46, ospf-UNDERLAY, intra
10.10.100.16/32, ubest/mbest: 2/0
    *via 10.10.25.1, Eth1/1, [110/81], 00:04:01, ospf-UNDERLAY, intra
    *via 10.10.25.5, Eth1/2, [110/81], 00:04:01, ospf-UNDERLAY, intra
10.10.100.100/32, ubest/mbest: 2/0
    *via 10.10.25.1, Eth1/1, [110/41], 00:15:04, ospf-UNDERLAY, intra
    *via 10.10.25.5, Eth1/2, [110/41], 00:15:11, ospf-UNDERLAY, intra
10.10.100.111/32, ubest/mbest: 2/0, attached
    *via 10.10.100.111, Lo1, [0/0], 00:15:27, local
    *via 10.10.100.111, Lo1, [0/0], 00:15:27, direct
10.10.100.112/32, ubest/mbest: 2/0
    *via 10.10.25.1, Eth1/1, [110/81], 00:13:43, ospf-UNDERLAY, intra
    *via 10.10.25.5, Eth1/2, [110/81], 00:13:41, ospf-UNDERLAY, intra
10.10.100.113/32, ubest/mbest: 2/0
    *via 10.10.25.1, Eth1/1, [110/81], 00:12:09, ospf-UNDERLAY, intra
    *via 10.10.25.5, Eth1/2, [110/81], 00:12:09, ospf-UNDERLAY, intra
10.10.100.114/32, ubest/mbest: 2/0
    *via 10.10.25.1, Eth1/1, [110/81], 00:11:37, ospf-UNDERLAY, intra
    *via 10.10.25.5, Eth1/2, [110/81], 00:11:33, ospf-UNDERLAY, intra
10.10.100.115/32, ubest/mbest: 2/0
    *via 10.10.25.1, Eth1/1, [110/81], 00:04:46, ospf-UNDERLAY, intra
    *via 10.10.25.5, Eth1/2, [110/81], 00:04:46, ospf-UNDERLAY, intra
10.10.100.116/32, ubest/mbest: 2/0
    *via 10.10.25.1, Eth1/1, [110/81], 00:04:01, ospf-UNDERLAY, intra
    *via 10.10.25.5, Eth1/2, [110/81], 00:04:01, ospf-UNDERLAY, intra
10.10.100.121/32, ubest/mbest: 2/0
    *via 10.10.25.1, Eth1/1, [110/81], 00:12:09, ospf-UNDERLAY, intra
    *via 10.10.25.5, Eth1/2, [110/81], 00:12:09, ospf-UNDERLAY, intra
10.10.100.143/32, ubest/mbest: 2/0
    *via 10.10.25.1, Eth1/1, [110/81], 00:04:46, ospf-UNDERLAY, intra
    *via 10.10.25.5, Eth1/2, [110/81], 00:04:46, ospf-UNDERLAY, intra
BL-1# 
BL-1# sh ip ospf neighbors 
 OSPF Process ID UNDERLAY VRF default
 Total number of neighbors: 2
 Neighbor ID     Pri State            Up Time  Address         Interface
 10.10.100.1       1 FULL/ -          00:15:23 10.10.25.1      Eth1/1 
 10.10.100.2       1 FULL/ -          00:15:30 10.10.25.5      Eth1/2 
BL-1# sh ip int br

IP Interface Status for VRF "default"(1)
Interface            IP Address      Interface Status
Lo0                  10.10.100.11    protocol-up/link-up/admin-up       
Lo1                  10.10.100.111   protocol-up/link-up/admin-up       
Eth1/1               10.10.25.2      protocol-up/link-up/admin-up       
Eth1/2               10.10.25.6      protocol-up/link-up/admin-up

3.1.3 Leaf nodes configuration

### L-1
feature nxapi
cfs eth distribute
nv overlay evpn
feature ospf
feature bgp
feature pim
feature interface-vlan
feature vn-segment-vlan-based
feature lacp
feature dhcp
feature vpc
feature lldp
feature nv overlay
feature ngoam

ip pim rp-address 10.10.100.100 group-list 239.1.25.0/24


interface Ethernet1/1
  description connected-to-spine-1-Ethernet1/3
  no switchport
  mtu 9216
  ip address 10.10.25.18/30
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2
  description connected-to-spine-2-Ethernet1/3
  no switchport
  mtu 9216
  ip address 10.10.25.22/30
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface mgmt0
  vrf member management
  ip address 10.14.14.13/24

interface loopback0
  description Routing loopback interface
  ip address 10.10.100.13/32
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode

interface loopback1
  description VTEP loopback interface
  ip address 10.10.100.113/32
  ip address 10.10.100.121/32 secondary
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode

router ospf UNDERLAY
  router-id 10.10.100.13

vpc domain 2
  peer-switch
  peer-keepalive destination 10.14.14.14 source 10.14.14.13
  delay restore 150
  peer-gateway
  auto-recovery reload-delay 360
  ipv6 nd synchronize
  ip arp synchronize

interface Ethernet1/5
  description "PO 10 (vpc-peer-link) member L-1-Ethernet1/5 to L-2-Ethernet1/5"
  switchport
  switchport mode trunk
  channel-group 10 mode active
  no shutdown

interface port-channel10
  description "vpc-peer-link L-1-L-2
  switchport mode trunk
  vpc peer-link

Vlan 3600

interface Vlan3600
  description VPC-Peer-Link SVI
  no shutdown
  mtu 9216
  no ip redirects
  ip address 10.10.10.45/30
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode

### L-3
feature nxapi
cfs eth distribute
nv overlay evpn
feature ospf
feature bgp
feature pim
feature interface-vlan
feature vn-segment-vlan-based
feature lacp
feature dhcp
feature vpc
feature lldp
feature nv overlay
feature ngoam

ip pim rp-address 10.10.100.100 group-list 239.1.25.0/24

interface Ethernet1/1
  description connected-to-spine-1-Ethernet1/5
  no switchport
  mtu 9216
  ip address 10.10.25.34/30
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2
  description connected-to-spine-2-Ethernet1/5
  no switchport
  mtu 9216
  ip address 10.10.25.38/30
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface mgmt0
  vrf member management
  ip address 10.14.14.15/24

interface loopback0
  description Routing loopback interface
  ip address 10.10.100.15/32
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode

interface loopback1
  description VTEP loopback interface
  ip address 10.10.100.115/32
  ip address 10.10.100.143/32 secondary
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode

router ospf UNDERLAY
  router-id 10.10.100.15

vpc domain 2
  peer-switch
  peer-keepalive destination 10.14.14.16 source 10.14.14.15
  delay restore 150
  peer-gateway
  auto-recovery reload-delay 360
  ipv6 nd synchronize
  ip arp synchronize

interface Ethernet1/5
  description "PO 10 (vpc-peer-link) member L-3-Ethernet1/5 to L-4-Ethernet1/5"
  switchport
  switchport mode trunk
  channel-group 10 mode active
  no shutdown

interface port-channel10
  description "vpc-peer-link L-3-L-4
  switchport mode trunk
  spanning-tree port type network
  vpc peer-link

vlan 3600

interface Vlan3600
  description VPC-Peer-Link SVI
  no shutdown
  mtu 9216
  no ip redirects
  ip address 10.10.10.49/30
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
### L-2
feature nxapi
cfs eth distribute
nv overlay evpn
feature ospf
feature bgp
feature pim
feature interface-vlan
feature vn-segment-vlan-based
feature lacp
feature dhcp
feature vpc
feature lldp
feature nv overlay
feature ngoam

ip pim rp-address 10.10.100.100 group-list 239.1.25.0/24


interface Ethernet1/1
  description connected-to-spine-1-Ethernet1/4
  no switchport
  mtu 9216
  ip address 10.10.25.26/30
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2
  description connected-to-spine-2-Ethernet1/4
  no switchport
  mtu 9216
  ip address 10.10.25.30/30
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface mgmt0
  vrf member management
  ip address 10.14.14.14/24

interface loopback0
  description Routing loopback interface
  ip address 10.10.100.14/32
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode

interface loopback1
  description VTEP loopback interface
  ip address 10.10.100.114/32
  ip address 10.10.100.121/32 secondary
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode

router ospf UNDERLAY
  router-id 10.10.100.14

vpc domain 2
  peer-switch
  peer-keepalive destination 10.14.14.13 source 10.14.14.14
  delay restore 150
  peer-gateway
  auto-recovery reload-delay 360
  ipv6 nd synchronize
  ip arp synchronize

interface Ethernet1/5
  description "PO 10 (vpc-peer-link) member L-2-Ethernet1/5 to L-1-Ethernet1/5"
  switchport
  switchport mode trunk
  channel-group 10 mode active
  no shutdown

interface port-channel10
  description "vpc-peer-link L-1-L-2
  switchport mode trunk
  vpc peer-link

Vlan 3600

interface Vlan3600
  description VPC-Peer-Link SVI
  no shutdown
  mtu 9216
  no ip redirects
  ip address 10.10.10.46/30
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode

### L-4
feature nxapi
cfs eth distribute
nv overlay evpn
feature ospf
feature bgp
feature pim
feature interface-vlan
feature vn-segment-vlan-based
feature lacp
feature dhcp
feature vpc
feature lldp
feature nv overlay
feature ngoam

ip pim rp-address 10.10.100.100 group-list 239.1.25.0/24

interface Ethernet1/1
  description connected-to-spine-1-Ethernet1/6
  no switchport
  mtu 9216
  ip address 10.10.25.42/30
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2
  description connected-to-spine-2-Ethernet1/6
  no switchport
  mtu 9216
  ip address 10.10.25.46/30
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface mgmt0
  vrf member management
  ip address 10.14.14.16/24

interface loopback0
  description Routing loopback interface
  ip address 10.10.100.16/32
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode

interface loopback1
  description VTEP loopback interface
  ip address 10.10.100.116/32
  ip address 10.10.100.143/32 secondary
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode

router ospf UNDERLAY
  router-id 10.10.100.16

vpc domain 2
  peer-switch
  peer-keepalive destination 10.14.14.15 source 10.14.14.16
  delay restore 150
  peer-gateway
  auto-recovery reload-delay 360
  ipv6 nd synchronize
  ip arp synchronize

interface Ethernet1/5
  description "PO 10 (vpc-peer-link) member L-4-Ethernet1/5 to L-3-Ethernet1/5"
  switchport
  switchport mode trunk
  channel-group 10 mode active
  no shutdown

interface port-channel10
  description "vpc-peer-link L-4-L-3
  switchport mode trunk
  spanning-tree port type network
  vpc peer-link

vlan 3600

interface Vlan3600
  description VPC-Peer-Link SVI
  no shutdown
  mtu 9216
  no ip redirects
  ip address 10.10.10.50/30
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
L-1# sh ip ospf neighbors 
 OSPF Process ID UNDERLAY VRF default
 Total number of neighbors: 2
 Neighbor ID     Pri State            Up Time  Address         Interface
 10.10.100.1       1 FULL/ -          00:15:14 10.10.25.17     Eth1/1 
 10.10.100.2       1 FULL/ -          00:15:15 10.10.25.21     Eth1/2 
 
L-1# sh ip int br

IP Interface Status for VRF "default"(1)
Interface            IP Address      Interface Status
Lo0                  10.10.100.13    protocol-up/link-up/admin-up       
Lo1                  10.10.100.113   protocol-up/link-up/admin-up       
Eth1/1               10.10.25.18     protocol-up/link-up/admin-up       
Eth1/2               10.10.25.22     protocol-up/link-up/admin-up 
      
L-1# sh ip route 
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

10.10.25.0/30, ubest/mbest: 1/0
    *via 10.10.25.17, Eth1/1, [110/80], 00:15:20, ospf-UNDERLAY, intra
10.10.25.4/30, ubest/mbest: 1/0
    *via 10.10.25.21, Eth1/2, [110/80], 00:15:25, ospf-UNDERLAY, intra
10.10.25.8/30, ubest/mbest: 1/0
    *via 10.10.25.17, Eth1/1, [110/80], 00:15:20, ospf-UNDERLAY, intra
10.10.25.12/30, ubest/mbest: 1/0
    *via 10.10.25.21, Eth1/2, [110/80], 00:15:25, ospf-UNDERLAY, intra
10.10.25.16/30, ubest/mbest: 1/0, attached
    *via 10.10.25.18, Eth1/1, [0/0], 00:15:39, direct
10.10.25.18/32, ubest/mbest: 1/0, attached
    *via 10.10.25.18, Eth1/1, [0/0], 00:15:39, local
10.10.25.20/30, ubest/mbest: 1/0, attached
    *via 10.10.25.22, Eth1/2, [0/0], 00:15:37, direct
10.10.25.22/32, ubest/mbest: 1/0, attached
    *via 10.10.25.22, Eth1/2, [0/0], 00:15:37, local
10.10.25.24/30, ubest/mbest: 1/0
    *via 10.10.25.17, Eth1/1, [110/80], 00:15:20, ospf-UNDERLAY, intra
10.10.25.28/30, ubest/mbest: 1/0
    *via 10.10.25.21, Eth1/2, [110/80], 00:15:25, ospf-UNDERLAY, intra
10.10.25.32/30, ubest/mbest: 1/0
    *via 10.10.25.17, Eth1/1, [110/80], 00:15:20, ospf-UNDERLAY, intra
10.10.25.36/30, ubest/mbest: 1/0
    *via 10.10.25.21, Eth1/2, [110/80], 00:15:25, ospf-UNDERLAY, intra
10.10.25.40/30, ubest/mbest: 1/0
    *via 10.10.25.17, Eth1/1, [110/80], 00:15:20, ospf-UNDERLAY, intra
10.10.25.44/30, ubest/mbest: 1/0
    *via 10.10.25.21, Eth1/2, [110/80], 00:15:25, ospf-UNDERLAY, intra
10.10.100.1/32, ubest/mbest: 1/0
    *via 10.10.25.17, Eth1/1, [110/41], 00:15:20, ospf-UNDERLAY, intra
10.10.100.2/32, ubest/mbest: 1/0
    *via 10.10.25.21, Eth1/2, [110/41], 00:15:25, ospf-UNDERLAY, intra
10.10.100.11/32, ubest/mbest: 2/0
    *via 10.10.25.17, Eth1/1, [110/81], 00:15:20, ospf-UNDERLAY, intra
    *via 10.10.25.21, Eth1/2, [110/81], 00:15:25, ospf-UNDERLAY, intra
10.10.100.12/32, ubest/mbest: 2/0
    *via 10.10.25.17, Eth1/1, [110/81], 00:15:20, ospf-UNDERLAY, intra
    *via 10.10.25.21, Eth1/2, [110/81], 00:15:25, ospf-UNDERLAY, intra
10.10.100.13/32, ubest/mbest: 2/0, attached
    *via 10.10.100.13, Lo0, [0/0], 00:15:37, local
    *via 10.10.100.13, Lo0, [0/0], 00:15:37, direct
10.10.100.14/32, ubest/mbest: 2/0
    *via 10.10.25.17, Eth1/1, [110/81], 00:14:48, ospf-UNDERLAY, intra
    *via 10.10.25.21, Eth1/2, [110/81], 00:14:44, ospf-UNDERLAY, intra
10.10.100.15/32, ubest/mbest: 2/0
    *via 10.10.25.17, Eth1/1, [110/81], 00:07:57, ospf-UNDERLAY, intra
    *via 10.10.25.21, Eth1/2, [110/81], 00:07:57, ospf-UNDERLAY, intra
10.10.100.16/32, ubest/mbest: 2/0
    *via 10.10.25.17, Eth1/1, [110/81], 00:07:12, ospf-UNDERLAY, intra
    *via 10.10.25.21, Eth1/2, [110/81], 00:07:12, ospf-UNDERLAY, intra
10.10.100.100/32, ubest/mbest: 2/0
    *via 10.10.25.17, Eth1/1, [110/41], 00:15:20, ospf-UNDERLAY, intra
    *via 10.10.25.21, Eth1/2, [110/41], 00:15:25, ospf-UNDERLAY, intra
10.10.100.111/32, ubest/mbest: 2/0
    *via 10.10.25.17, Eth1/1, [110/81], 00:15:20, ospf-UNDERLAY, intra
    *via 10.10.25.21, Eth1/2, [110/81], 00:15:25, ospf-UNDERLAY, intra
10.10.100.112/32, ubest/mbest: 2/0
    *via 10.10.25.17, Eth1/1, [110/81], 00:15:20, ospf-UNDERLAY, intra
    *via 10.10.25.21, Eth1/2, [110/81], 00:15:25, ospf-UNDERLAY, intra
10.10.100.113/32, ubest/mbest: 2/0, attached
    *via 10.10.100.113, Lo1, [0/0], 00:15:36, local
    *via 10.10.100.113, Lo1, [0/0], 00:15:36, direct
10.10.100.114/32, ubest/mbest: 2/0
    *via 10.10.25.17, Eth1/1, [110/81], 00:14:48, ospf-UNDERLAY, intra
    *via 10.10.25.21, Eth1/2, [110/81], 00:14:44, ospf-UNDERLAY, intra
10.10.100.115/32, ubest/mbest: 2/0
    *via 10.10.25.17, Eth1/1, [110/81], 00:07:57, ospf-UNDERLAY, intra
    *via 10.10.25.21, Eth1/2, [110/81], 00:07:57, ospf-UNDERLAY, intra
10.10.100.116/32, ubest/mbest: 2/0
    *via 10.10.25.17, Eth1/1, [110/81], 00:07:12, ospf-UNDERLAY, intra
    *via 10.10.25.21, Eth1/2, [110/81], 00:07:12, ospf-UNDERLAY, intra
10.10.100.121/32, ubest/mbest: 2/0, attached
    *via 10.10.100.121, Lo1, [0/0], 00:15:36, local
    *via 10.10.100.121, Lo1, [0/0], 00:15:36, direct
10.10.100.143/32, ubest/mbest: 2/0
    *via 10.10.25.17, Eth1/1, [110/81], 00:07:57, ospf-UNDERLAY, intra
    *via 10.10.25.21, Eth1/2, [110/81], 00:07:57, ospf-UNDERLAY, intra


L-1# sh vpc
Legend:
                (*) - local vPC is down, forwarding via vPC peer-link

vPC domain id                     : 2   
Peer status                       : peer adjacency formed ok      
vPC keep-alive status             : peer is alive                 
Configuration consistency status  : success 
Per-vlan consistency status       : success                       
Type-2 consistency status         : success 
vPC role                          : primary                       
Number of vPCs configured         : 0   
Peer Gateway                      : Enabled
Dual-active excluded VLANs        : -
Graceful Consistency Check        : Enabled
Auto-recovery status              : Enabled, timer is off.(timeout = 360s)
Delay-restore status              : Timer is off.(timeout = 150s)
Delay-restore SVI status          : Timer is off.(timeout = 10s)
Operational Layer3 Peer-router    : Disabled
Virtual-peerlink mode             : Disabled

vPC Peer-link status
---------------------------------------------------------------------
id    Port   Status Active vlans    
--    ----   ------ -------------------------------------------------
1     Po10   up     1

3.2 iBGP and NVE configuration

Full reachability between loop back addresses of every spine and leaf is established with the configuration at step 1 (section 3.1)

Now, iBGP and NVE interface are configured between the VTEPs (BLeaf & Leaf nodes)

iBGP (Interior Border Gateway Protocol) plays a crucial role in establishing communication channels and exchanging routing information between VXLAN Tunnel Endpoints (VTEPs) residing on the Leaf and Spine switches.

NVE is a logical interface that identifies the switch as a VXLAN endpoint.

### Spine-1
router bgp 65125
  router-id 10.10.100.1
  template peer iBGP-VxLAN-EVPN
    remote-as 65125
    update-source loopback0
    address-family l2vpn evpn
      send-community both
      route-reflector-client
  neighbor 10.10.100.11
    inherit peer iBGP-VxLAN-EVPN
  neighbor 10.10.100.12
    inherit peer iBGP-VxLAN-EVPN
  neighbor 10.10.100.13
    inherit peer iBGP-VxLAN-EVPN
  neighbor 10.10.100.14
    inherit peer iBGP-VxLAN-EVPN
  neighbor 10.10.100.15
    inherit peer iBGP-VxLAN-EVPN
  neighbor 10.10.100.16
    inherit peer iBGP-VxLAN-EVPN
### BL-1
router bgp 65125
  router-id 10.10.100.11
  address-family l2vpn evpn
  template peer iBGP-VxLAN-EVPN
    remote-as 65125
    update-source loopback0
    address-family l2vpn evpn
      send-community both
  neighbor 10.10.100.1
    inherit peer iBGP-VxLAN-EVPN
  neighbor 10.10.100.2
    inherit peer iBGP-VxLAN-EVPN

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback1
  

### L-1
router bgp 65125
  router-id 10.10.100.13
  address-family l2vpn evpn
    advertise-pip
  template peer iBGP-VxLAN-EVPN
    remote-as 65125
    update-source loopback0
    address-family l2vpn evpn
      send-community both
  neighbor 10.10.100.1
    inherit peer iBGP-VxLAN-EVPN
  neighbor 10.10.100.2
    inherit peer iBGP-VxLAN-EVPN

interface nve1
  no shutdown
  host-reachability protocol bgp
  advertise virtual-rmac
  source-interface loopback1

### L-3
router bgp 65125
  router-id 10.10.100.15
  address-family l2vpn evpn
    advertise-pip
  template peer iBGP-VxLAN-EVPN
    remote-as 65125
    update-source loopback0
    address-family l2vpn evpn
      send-community both
  neighbor 10.10.100.1
    inherit peer iBGP-VxLAN-EVPN
  neighbor 10.10.100.2
    inherit peer iBGP-VxLAN-EVPN

interface nve1
  no shutdown
  host-reachability protocol bgp
  advertise virtual-rmac
  source-interface loopback1
### Spine-2
router bgp 65125
  router-id 10.10.100.2
  template peer iBGP-VxLAN-EVPN
    remote-as 65125
    update-source loopback0
    address-family l2vpn evpn
      send-community both
      route-reflector-client
  neighbor 10.10.100.11
    inherit peer iBGP-VxLAN-EVPN
  neighbor 10.10.100.12
    inherit peer iBGP-VxLAN-EVPN
  neighbor 10.10.100.13
    inherit peer iBGP-VxLAN-EVPN
  neighbor 10.10.100.14
    inherit peer iBGP-VxLAN-EVPN
  neighbor 10.10.100.15
    inherit peer iBGP-VxLAN-EVPN
  neighbor 10.10.100.16
    inherit peer iBGP-VxLAN-EVPN
### BL-2
router bgp 65125
  router-id 10.10.100.12
  address-family l2vpn evpn
  template peer iBGP-VxLAN-EVPN
    remote-as 65125
    update-source loopback0
    address-family l2vpn evpn
      send-community both
  neighbor 10.10.100.1
    inherit peer iBGP-VxLAN-EVPN
  neighbor 10.10.100.2
    inherit peer iBGP-VxLAN-EVPN

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback1


### L-2
router bgp 65125
  router-id 10.10.100.14
  address-family l2vpn evpn
    advertise-pip
  template peer iBGP-VxLAN-EVPN
    remote-as 65125
    update-source loopback0
    address-family l2vpn evpn
      send-community both
  neighbor 10.10.100.1
    inherit peer iBGP-VxLAN-EVPN
  neighbor 10.10.100.2
    inherit peer iBGP-VxLAN-EVPN

interface nve1
  no shutdown
  host-reachability protocol bgp
  advertise virtual-rmac
  source-interface loopback1

### L-4
router bgp 65125
  router-id 10.10.100.16
  address-family l2vpn evpn
    advertise-pip
  template peer iBGP-VxLAN-EVPN
    remote-as 65125
    update-source loopback0
    address-family l2vpn evpn
      send-community both
  neighbor 10.10.100.1
    inherit peer iBGP-VxLAN-EVPN
  neighbor 10.10.100.2
    inherit peer iBGP-VxLAN-EVPN

interface nve1
  no shutdown
  host-reachability protocol bgp
  advertise virtual-rmac
  source-interface loopback1
Spine-1# sh bgp l2vpn evpn  summary 
BGP summary information for VRF default, address family L2VPN EVPN
BGP router identifier 10.10.100.1, local AS number 65125
BGP table version is 8, L2VPN EVPN config peers 6, capable peers 6
0 network entries and 0 paths using 0 bytes of memory
BGP attribute entries [0/0], BGP AS path entries [0/0]
BGP community entries [0/0], BGP clusterlist entries [0/0]

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.10.100.11    4 65125       9       9        8    0    0 00:03:08 0         
10.10.100.12    4 65125       8       8        8    0    0 00:02:37 0         
10.10.100.13    4 65125       8       8        8    0    0 00:02:12 0         
10.10.100.14    4 65125       7       7        8    0    0 00:01:30 0         
10.10.100.15    4 65125       6       6        8    0    0 00:00:52 0         
10.10.100.16    4 65125       6       6        8    0    0 00:00:28 0  

Spine-2# sh bgp l2vpn evpn summary 
BGP summary information for VRF default, address family L2VPN EVPN
BGP router identifier 10.10.100.2, local AS number 65125
BGP table version is 8, L2VPN EVPN config peers 6, capable peers 6
0 network entries and 0 paths using 0 bytes of memory
BGP attribute entries [0/0], BGP AS path entries [0/0]
BGP community entries [0/0], BGP clusterlist entries [0/0]

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.10.100.11    4 65125      16      21        8    0    0 00:10:20 0         
10.10.100.12    4 65125      21      21        8    0    0 00:15:39 0         
10.10.100.13    4 65125      15      19        8    0    0 00:09:45 0         
10.10.100.14    4 65125      15      19        8    0    0 00:09:40 0         
10.10.100.15    4 65125      15      19        8    0    0 00:09:36 0         
10.10.100.16    4 65125      15      18        8    0    0 00:09:32 0  
L-1# sh bgp l2vpn evpn summary 
BGP summary information for VRF default, address family L2VPN EVPN
BGP router identifier 10.10.100.11, local AS number 65125
BGP table version is 4, L2VPN EVPN config peers 2, capable peers 2
0 network entries and 0 paths using 0 bytes of memory
BGP attribute entries [0/0], BGP AS path entries [0/0]
BGP community entries [0/0], BGP clusterlist entries [0/0]

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.10.100.1     4 65125      23      23        4    0    0 00:17:14 0         
10.10.100.2     4 65125      17      17        4    0    0 00:11:25 0         

BL-1# sh int nve 1
nve1 is up
admin state is up,  Hardware: NVE
  MTU 9216 bytes
  Encapsulation VXLAN
  Auto-mdix is turned off
  RX
    ucast: 0 pkts, 0 bytes - mcast: 0 pkts, 0 bytes
  TX
    ucast: 0 pkts, 0 bytes - mcast: 0 pkts, 0 bytes

BL-2# sh bgp l2vpn evpn summary 
BGP summary information for VRF default, address family L2VPN EVPN
BGP router identifier 10.10.100.12, local AS number 65125
BGP table version is 4, L2VPN EVPN config peers 2, capable peers 2
0 network entries and 0 paths using 0 bytes of memory
BGP attribute entries [0/0], BGP AS path entries [0/0]
BGP community entries [0/0], BGP clusterlist entries [0/0]

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.10.100.1     4 65125      23      23        4    0    0 00:17:38 0         
10.10.100.2     4 65125      23      23        4    0    0 00:17:39 0         
BL-2# sh int nve 1
nve1 is up
admin state is up,  Hardware: NVE
  MTU 9216 bytes
  Encapsulation VXLAN
  Auto-mdix is turned off
  RX
    ucast: 0 pkts, 0 bytes - mcast: 0 pkts, 0 bytes
  TX
    ucast: 0 pkts, 0 bytes - mcast: 0 pkts, 0 bytes
L-1# sh bgp l2vpn evpn summary 
BGP summary information for VRF default, address family L2VPN EVPN
BGP router identifier 10.10.100.13, local AS number 65125
BGP table version is 4, L2VPN EVPN config peers 2, capable peers 2
0 network entries and 0 paths using 0 bytes of memory
BGP attribute entries [0/0], BGP AS path entries [0/0]
BGP community entries [0/0], BGP clusterlist entries [0/0]

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.10.100.1     4 65125      24      24        4    0    0 00:18:07 0         
10.10.100.2     4 65125      18      18        4    0    0 00:12:40 0         
L-1# sh int nve 1
nve1 is up
admin state is up,  Hardware: NVE
  MTU 9216 bytes
  Encapsulation VXLAN
  Auto-mdix is turned off
  RX
    ucast: 0 pkts, 0 bytes - mcast: 0 pkts, 0 bytes
  TX
    ucast: 0 pkts, 0 bytes - mcast: 0 pkts, 0 bytes

L-2# sh bgp l2vpn evpn summary 
BGP summary information for VRF default, address family L2VPN EVPN
BGP router identifier 10.10.100.14, local AS number 65125
BGP table version is 4, L2VPN EVPN config peers 2, capable peers 2
0 network entries and 0 paths using 0 bytes of memory
BGP attribute entries [0/0], BGP AS path entries [0/0]
BGP community entries [0/0], BGP clusterlist entries [0/0]

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.10.100.1     4 65125      24      24        4    0    0 00:18:13 0         
10.10.100.2     4 65125      19      19        4    0    0 00:13:23 0         

L-2# sh int nve 1
nve1 is up
admin state is up,  Hardware: NVE
  MTU 9216 bytes
  Encapsulation VXLAN
  Auto-mdix is turned off
  RX
    ucast: 0 pkts, 0 bytes - mcast: 0 pkts, 0 bytes
  TX
    ucast: 0 pkts, 0 bytes - mcast: 0 pkts, 0 bytes
L-3# sh bgp l2vpn evpn summary 
BGP summary information for VRF default, address family L2VPN EVPN
BGP router identifier 10.10.100.15, local AS number 65125
BGP table version is 4, L2VPN EVPN config peers 2, capable peers 2
0 network entries and 0 paths using 0 bytes of memory
BGP attribute entries [0/0], BGP AS path entries [0/0]
BGP community entries [0/0], BGP clusterlist entries [0/0]

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.10.100.1     4 65125      24      24        4    0    0 00:18:40 0         
10.10.100.2     4 65125      20      20        4    0    0 00:14:24 0         

L-3# sh int nve 1
nve1 is up
admin state is up,  Hardware: NVE
  MTU 9216 bytes
  Encapsulation VXLAN
  Auto-mdix is turned off
  RX
    ucast: 0 pkts, 0 bytes - mcast: 0 pkts, 0 bytes
  TX
    ucast: 0 pkts, 0 bytes - mcast: 0 pkts, 0 bytes

L-4# sh bgp l2vpn evpn summary 
BGP summary information for VRF default, address family L2VPN EVPN
BGP router identifier 10.10.100.16, local AS number 65125
BGP table version is 4, L2VPN EVPN config peers 2, capable peers 2
0 network entries and 0 paths using 0 bytes of memory
BGP attribute entries [0/0], BGP AS path entries [0/0]
BGP community entries [0/0], BGP clusterlist entries [0/0]

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.10.100.1     4 65125      25      25        4    0    0 00:19:12 0         
10.10.100.2     4 65125      21      21        4    0    0 00:15:15 0         

L-4# sh int nve 1
nve1 is up
admin state is up,  Hardware: NVE
  MTU 9216 bytes
  Encapsulation VXLAN
  Auto-mdix is turned off
  RX
    ucast: 0 pkts, 0 bytes - mcast: 0 pkts, 0 bytes
  TX
    ucast: 0 pkts, 0 bytes - mcast: 0 pkts, 0 bytes

3.3 Overlay configuration


A VXLAN EVPN overlay network combines two protocols, VXLAN (Virtual Extensible LAN) and EVPN (Ethernet VPN), to create a scalable and secure Layer 2 overlay network on top of a Layer 3 physical network.

Configuring overlay includes:

  1. VRF
  2. VLAN and VLAN to VNI mapping
  3. NVE and VNI mapping
  4. iBGP related to the VRF
  5. L2 EVPN
  6. Interface 
### BL-1 & BL-2 (dedicated border leaf, no server attachment)

fabric forwarding anycast-gateway-mac 2020.0000.00aa
vlan 2000
  vn-segment 32000

vrf context myvrf_1
  description VRF for prod systems 1
  vni 32000
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn

interface Vlan2000
  description VRF for prod systems 1
  no shutdown
  mtu 9216
  vrf member myvrf_1
  no ip redirects
  ip forward

interface nve1
  member vni 32000 associate-vrf

router bgp 65125
  vrf myvrf_1
    address-family ipv4 unicast
      advertise l2vpn evpn
      maximum-paths ibgp 2
#### L-1
interface Ethernet1/10
  switchport 
  switchport mode access
  switchport access vlan 30 
  no shutdown

#### L-2
interface Ethernet1/10
  switchport 
  switchport mode access
  switchport access vlan 50
  no shutdown

#### L-3
interface Ethernet1/11
  switchport 
  switchport mode trunk
  switchport trunk allowed vlan 20
  channel-group 20 mode active
  no shutdown

interface port-channel20
  switchport mode trunk
  switchport trunk allowed vlan 20
  vpc 20

#### L-4
interface Ethernet1/10
  switchport 
  switchport mode access
  switchport access vlan 40
  no shutdown

interface Ethernet1/11
  switchport 
  switchport mode trunk
  switchport trunk allowed vlan 20
  channel-group 20 mode active
  no shutdown

interface port-channel20
  switchport mode trunk
  switchport trunk allowed vlan 20
  vpc 20
### L-1, L-2, L-3 & L-4 

fabric forwarding anycast-gateway-mac 2020.0000.00aa

vlan 20,30,40,50,2000
vlan 20
  vn-segment 20020
vlan 30
  vn-segment 20030
vlan 40
  vn-segment 20040
vlan 50
  vn-segment 20050
vlan 2000
  vn-segment 32000

vrf context myvrf_1
  description VRF for prod systems 1
  vni 32000
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn

interface Vlan20
  no shutdown
  mtu 9216
  vrf member myvrf_1
  no ip redirects
  ip address 10.10.20.1/24 tag 12345
  fabric forwarding mode anycast-gateway

interface Vlan30
  no shutdown
  mtu 9216
  vrf member myvrf_1
  no ip redirects
  ip address 10.10.30.1/24 tag 12345
  fabric forwarding mode anycast-gateway

interface Vlan40
  no shutdown
  mtu 9216
  vrf member myvrf_1
  no ip redirects
  ip address 10.10.40.1/24 tag 12345
  fabric forwarding mode anycast-gateway

interface Vlan50
  no shutdown
  mtu 9216
  vrf member myvrf_1
  no ip redirects
  ip address 10.10.50.1/24 tag 12345
  fabric forwarding mode anycast-gateway

interface Vlan2000
  description VRF for prod systems 1
  no shutdown
  mtu 9216
  vrf member myvrf_1
  no ip redirects
  ip forward

interface nve1
  member vni 20020
    mcast-group 239.1.25.20
  member vni 20030
    mcast-group 239.1.25.30
  member vni 20040
    mcast-group 239.1.25.40
  member vni 20050
    mcast-group 239.1.25.50
  member vni 32000 associate-vrf

route-map fabric-rmap-redist-subnet permit 10
  match tag 12345 

router bgp 65125
  vrf myvrf_1
    address-family ipv4 unicast
      advertise l2vpn evpn
      redistribute direct route-map fabric-rmap-redist-subnet
      maximum-paths ibgp 2

evpn
  vni 20020 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 20030 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 20040 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 20050 l2
    rd auto
    route-target import auto
    route-target export auto

3.4 Internal Fabric Verification

Testing a VXLAN EVPN fabric involves verifying various aspects to ensure proper functioning and communication between workloads. Some of the test performed to verify the proper functioning of the internal fabric are:

  1. Ping test – The most basic test to verify connectivity between workloads residing within the fabric.
  2. EVPN Route Verification – Confirm VXLAN tunnels are established and routes are exchanged between VTEPs that need to communicate.
  3. Mac address – Verify mac address info are exchanged using the control plane.
L-1# SH IP route vrf myvrf_1 
IP Route Table for VRF "myvrf_1"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

10.10.20.0/24, ubest/mbest: 1/0, attached
    *via 10.10.20.1, Vlan20, [0/0], 03:35:18, direct, tag 12345
10.10.20.1/32, ubest/mbest: 1/0, attached
    *via 10.10.20.1, Vlan20, [0/0], 03:35:18, local, tag 12345
10.10.20.10/32, ubest/mbest: 1/0
    *via 10.10.100.143%default, [200/0], 00:12:14, bgp-65125, internal, tag 6512
5, segid: 32000 tunnelid: 0xa0a648f encap: VXLAN
 
10.10.30.0/24, ubest/mbest: 1/0, attached
    *via 10.10.30.1, Vlan30, [0/0], 03:35:18, direct, tag 12345
10.10.30.1/32, ubest/mbest: 1/0, attached
    *via 10.10.30.1, Vlan30, [0/0], 03:35:18, local, tag 12345
10.10.30.10/32, ubest/mbest: 1/0, attached
    *via 10.10.30.10, Vlan30, [190/0], 03:26:42, hmm
10.10.40.0/24, ubest/mbest: 1/0, attached
    *via 10.10.40.1, Vlan40, [0/0], 03:35:18, direct, tag 12345
10.10.40.1/32, ubest/mbest: 1/0, attached
    *via 10.10.40.1, Vlan40, [0/0], 03:35:18, local, tag 12345
10.10.40.10/32, ubest/mbest: 1/0
    *via 10.10.100.143%default, [200/0], 02:55:54, bgp-65125, internal, tag 6512
5, segid: 32000 tunnelid: 0xa0a648f encap: VXLAN
 
10.10.50.0/24, ubest/mbest: 1/0, attached
    *via 10.10.50.1, Vlan50, [0/0], 03:35:18, direct, tag 12345
10.10.50.1/32, ubest/mbest: 1/0, attached
    *via 10.10.50.1, Vlan50, [0/0], 03:35:18, local, tag 12345
10.10.50.10/32, ubest/mbest: 1/0, attached
    *via 10.10.50.10, Vlan50, [190/0], 03:27:40, hmm

L-1# sh mac address-table 
Legend: 
        * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
        age - seconds since last seen,+ - primary entry using vPC Peer-Link,
        (T) - True, (F) - False, C - ControlPlane MAC, ~ - vsan
   VLAN     MAC Address      Type      age     Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
C   20     6efe.94ff.d9f7   dynamic  0         F      F    nve1(10.10.100.143)
*   30     5254.0005.5fb3   dynamic  0         F      F    Eth1/10
C   40     5254.000a.9126   dynamic  0         F      F    nve1(10.10.100.143)
* 2000     0200.0a0a.6479   static   -         F      F    Vlan2000
* 2000     0200.0a0a.648f   static   -         F      F    nve1(10.10.100.143)
* 2000     5204.ee2d.1b08   static   -         F      F    nve1(10.10.100.115)
* 2000     5206.d4eb.1b08   static   -         F      F    Vlan2000
* 2000     5214.adc0.1b08   static   -         F      F    nve1(10.10.100.116)
+   50     5254.000b.9a01   dynamic  0         F      F    vPC Peer-Link
G    -     0200.0a0a.6479   static   -         F      F    sup-eth1(R)
G    -     2020.0000.00aa   static   -         F      F    sup-eth1(R)
G    -     5206.d4eb.1b08   static   -         F      F    sup-eth1(R)
G 3600     5206.d4eb.1b08   static   -         F      F    sup-eth1(R)
G   20     5206.d4eb.1b08   static   -         F      F    sup-eth1(R)
G   30     5206.d4eb.1b08   static   -         F      F    sup-eth1(R)
G   40     5206.d4eb.1b08   static   -         F      F    sup-eth1(R)
G   50     5206.d4eb.1b08   static   -         F      F    sup-eth1(R)
G 2000     5206.d4eb.1b08   static   -         F      F    sup-eth1(R)
G 3600     520d.c05b.1b08   static   -         F      F    vPC Peer-Link(R)
G   20     520d.c05b.1b08   static   -         F      F    vPC Peer-Link(R)
G   30     520d.c05b.1b08   static   -         F      F    vPC Peer-Link(R)
G   40     520d.c05b.1b08   static   -         F      F    vPC Peer-Link(R)
G   50     520d.c05b.1b08   static   -         F      F    vPC Peer-Link(R)
G 2000     520d.c05b.1b08   static   -         F      F    vPC Peer-Link(R)
L-3(config-if)# sh ip route vrf myvrf_1 
IP Route Table for VRF "myvrf_1"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

10.10.20.0/24, ubest/mbest: 1/0, attached
    *via 10.10.20.1, Vlan20, [0/0], 03:24:49, direct, tag 12345
10.10.20.1/32, ubest/mbest: 1/0, attached
    *via 10.10.20.1, Vlan20, [0/0], 03:24:49, local, tag 12345
10.10.20.10/32, ubest/mbest: 1/0, attached
    *via 10.10.20.10, Vlan20, [190/0], 00:14:32, hmm
10.10.30.0/24, ubest/mbest: 1/0, attached
    *via 10.10.30.1, Vlan30, [0/0], 03:36:57, direct, tag 12345
10.10.30.1/32, ubest/mbest: 1/0, attached
    *via 10.10.30.1, Vlan30, [0/0], 03:36:57, local, tag 12345
10.10.30.10/32, ubest/mbest: 1/0
    *via 10.10.100.121%default, [200/0], 03:23:09, bgp-65125, internal, tag 6512
5, segid: 32000 tunnelid: 0xa0a6479 encap: VXLAN
 
10.10.40.0/24, ubest/mbest: 1/0, attached
    *via 10.10.40.1, Vlan40, [0/0], 03:36:57, direct, tag 12345
10.10.40.1/32, ubest/mbest: 1/0, attached
    *via 10.10.40.1, Vlan40, [0/0], 03:36:57, local, tag 12345
10.10.40.10/32, ubest/mbest: 1/0, attached
    *via 10.10.40.10, Vlan40, [190/0], 02:58:13, hmm
10.10.50.0/24, ubest/mbest: 1/0, attached
    *via 10.10.50.1, Vlan50, [0/0], 03:36:57, direct, tag 12345
10.10.50.1/32, ubest/mbest: 1/0, attached
    *via 10.10.50.1, Vlan50, [0/0], 03:36:57, local, tag 12345
10.10.50.10/32, ubest/mbest: 1/0
    *via 10.10.100.121%default, [200/0], 03:23:09, bgp-65125, internal, tag 6512
5, segid: 32000 tunnelid: 0xa0a6479 encap: VXLAN
 
L-3(config-if)# sh mac address-table 
Legend: 
        * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
        age - seconds since last seen,+ - primary entry using vPC Peer-Link,
        (T) - True, (F) - False, C - ControlPlane MAC, ~ - vsan
   VLAN     MAC Address      Type      age     Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
*   20     6efe.94ff.d9f7   dynamic  0         F      F    Po20
C   30     5254.0005.5fb3   dynamic  0         F      F    nve1(10.10.100.121)
C   50     5254.000b.9a01   dynamic  0         F      F    nve1(10.10.100.121)
* 2000     0200.0a0a.6479   static   -         F      F    nve1(10.10.100.121)
* 2000     0200.0a0a.648f   static   -         F      F    Vlan2000
* 2000     5204.ee2d.1b08   static   -         F      F    Vlan2000
* 2000     5206.d4eb.1b08   static   -         F      F    nve1
* 2000     520d.c05b.1b08   static   -         F      F    nve1(10.10.100.114)
+   40     5254.000a.9126   dynamic  0         F      F    vPC Peer-Link
G    -     0200.0a0a.648f   static   -         F      F    sup-eth1(R)
G    -     2020.0000.00aa   static   -         F      F    sup-eth1(R)
G    -     5204.ee2d.1b08   static   -         F      F    sup-eth1(R)
G 3600     5204.ee2d.1b08   static   -         F      F    sup-eth1(R)
G   20     5204.ee2d.1b08   static   -         F      F    sup-eth1(R)
G   30     5204.ee2d.1b08   static   -         F      F    sup-eth1(R)
G   40     5204.ee2d.1b08   static   -         F      F    sup-eth1(R)
G   50     5204.ee2d.1b08   static   -         F      F    sup-eth1(R)
G 2000     5204.ee2d.1b08   static   -         F      F    sup-eth1(R)
G 3600     5214.adc0.1b08   static   -         F      F    vPC Peer-Link(R)
G   20     5214.adc0.1b08   static   -         F      F    vPC Peer-Link(R)
G   30     5214.adc0.1b08   static   -         F      F    vPC Peer-Link(R)
G   40     5214.adc0.1b08   static   -         F      F    vPC Peer-Link(R)
G   50     5214.adc0.1b08   static   -         F      F    vPC Peer-Link(R)
G 2000     5214.adc0.1b08   static   -         F      F    vPC Peer-Link(R)
cisco@S-1$ ping 10.10.20.10
PING 10.10.20.10 (10.10.20.10) 56(84) bytes of data.
64 bytes from 10.10.20.10:icmp_seq=1 ttl=64 time=0.031 ms
64 bytes from 10.10.20.10:icmp_seq=2 ttl=64 time=0.089 ms
64 bytes from 10.10.20.10:icmp_seq=3 ttl=64 time=0.055 ms
64 bytes from 10.10.20.10:icmp_seq=4 ttl=64 time=0.056 ms

cisco@S-1$ ping 10.10.30.10
PING 10.10.30.10 (10.10.30.10) 56(84) bytes of data.
64 bytes from 10.10.30.10: icmp_seq=1 ttl=62 time=57.2 ms
64 bytes from 10.10.30.10: icmp_seq=2 ttl=62 time=34.1 ms
64 bytes from 10.10.30.10: icmp_seq=3 ttl=62 time=35.0 ms
64 bytes from 10.10.30.10: icmp_seq=7 ttl=62 time=29.4 ms

cisco@S-1$ ping 10.10.40.10
PING 10.10.40.10 (10.10.40.10) 56(84) bytes of data.
64 bytes from 10.10.40.10: icmp_seq=1 ttl=62 time=57.2 ms
64 bytes from 10.10.40.10: icmp_seq=2 ttl=62 time=34.1 ms
64 bytes from 10.10.40.10: icmp_seq=3 ttl=62 time=35.0 ms
64 bytes from 10.10.40.10: icmp_seq=7 ttl=62 time=29.4 ms

cisco@S-1$ ping 10.10.50.10
PING 10.10.50.10 (10.10.50.10) 56(84) bytes of data.
64 bytes from 10.10.50.10: icmp_seq=1 ttl=62 time=37.3 ms
64 bytes from 10.10.50.10: icmp_seq=2 ttl=62 time=23.2 ms
64 bytes from 10.10.50.10: icmp_seq=4 ttl=62 time=46.9 ms
64 bytes from 10.10.50.10: icmp_seq=11 ttl=62 time=25.0 
cisco@S-3:~$ ping 10.10.20.10
PING 10.10.20.10 (10.10.20.10): 56 data bytes
64 bytes from 10.10.20.10: seq=0 ttl=62 time=15.859 ms
64 bytes from 10.10.20.10: seq=1 ttl=62 time=17.699 ms
64 bytes from 10.10.20.10: seq=2 ttl=62 time=21.679 ms
64 bytes from 10.10.20.10: seq=3 ttl=62 time=23.526 ms

cisco@S-3:~$ ping 10.10.30.10
PING 10.10.30.10 (10.10.30.10): 56 data bytes
64 bytes from 10.10.30.10: seq=0 ttl=62 time=15.859 ms
64 bytes from 10.10.30.10: seq=1 ttl=62 time=17.699 ms
64 bytes from 10.10.30.10: seq=2 ttl=62 time=21.679 ms
64 bytes from 10.10.30.10: seq=3 ttl=62 time=23.526 ms

cisco@S-3:~$ ping 10.10.40.10
PING 10.10.40.10 (10.10.40.10): 56 data bytes
64 bytes from 10.10.40.10: seq=0 ttl=64 time=0.065 ms
64 bytes from 10.10.40.10: seq=1 ttl=64 time=0.075 ms
64 bytes from 10.10.40.10: seq=2 ttl=64 time=0.075 ms
64 bytes from 10.10.40.10: seq=3 ttl=64 time=0.083 ms

cisco@S-3:~$ ping 10.10.50.10
PING 10.10.50.10 (10.10.50.10): 56 data bytes
64 bytes from 10.10.50.10: seq=0 ttl=62 time=17.809 ms
64 bytes from 10.10.50.10: seq=1 ttl=62 time=34.829 ms
64 bytes from 10.10.50.10: seq=2 ttl=62 time=49.042 ms
64 bytes from 10.10.50.10: seq=3 ttl=62 time=38.418 ms
cisco@S-2:~$ ping 10.10.20.10
PING 10.10.20.10 (10.10.20.10): 56 data bytes
64 bytes from 10.10.20.10: seq=0 ttl=62 time=28.948 ms
64 bytes from 10.10.20.10: seq=1 ttl=62 time=27.921 ms
64 bytes from 10.10.20.10: seq=5 ttl=62 time=34.761 ms
64 bytes from 10.10.20.10: seq=7 ttl=62 time=16.770 ms

cisco@S-2:~$ ping 10.10.30.10
PING 10.10.30.10 (10.10.30.10): 56 data bytes
64 bytes from 10.10.30.10: seq=0 ttl=64 time=0.066 ms
64 bytes from 10.10.30.10: seq=1 ttl=64 time=0.081 ms
64 bytes from 10.10.30.10: seq=2 ttl=64 time=0.153 ms
64 bytes from 10.10.30.10: seq=3 ttl=64 time=0.067 ms

cisco@S-2:~$ ping 10.10.40.10
PING 10.10.40.10 (10.10.40.10): 56 data bytes
64 bytes from 10.10.40.10: seq=0 ttl=62 time=20.557 ms
64 bytes from 10.10.40.10: seq=1 ttl=62 time=15.453 ms
64 bytes from 10.10.40.10: seq=2 ttl=62 time=64.280 ms
64 bytes from 10.10.40.10: seq=3 ttl=62 time=23.320 ms

cisco@S-2:~$ ping 10.10.50.10
PING 10.10.50.10 (10.10.50.10): 56 data bytes
64 bytes from 10.10.50.10: seq=0 ttl=63 time=8.152 ms
64 bytes from 10.10.50.10: seq=1 ttl=63 time=7.002 ms
64 bytes from 10.10.50.10: seq=2 ttl=63 time=9.542 ms
64 bytes from 10.10.50.10: seq=3 ttl=63 time=11.052 ms
cisco@S-4:~$ ping 10.10.20.10
PING 10.10.20.10 (10.10.20.10): 56 data bytes
64 bytes from 10.10.20.10: seq=6 ttl=62 time=23.577 ms
64 bytes from 10.10.20.10: seq=7 ttl=62 time=18.244 ms
64 bytes from 10.10.20.10: seq=9 ttl=62 time=30.697 ms
64 bytes from 10.10.20.10: seq=11 ttl=62 time=42.495 ms

cisco@S-4:~$ ping 10.10.30.10
PING 10.10.30.10 (10.10.30.10): 56 data bytes
64 bytes from 10.10.30.10: seq=0 ttl=63 time=10.126 ms
64 bytes from 10.10.30.10: seq=1 ttl=63 time=15.323 ms
64 bytes from 10.10.30.10: seq=2 ttl=63 time=14.607 ms
64 bytes from 10.10.30.10: seq=3 ttl=63 time=6.446 ms

cisco@S-4:~$ ping 10.10.40.10
PING 10.10.40.10 (10.10.40.10): 56 data bytes
64 bytes from 10.10.40.10: seq=0 ttl=62 time=17.461 ms
64 bytes from 10.10.40.10: seq=1 ttl=62 time=30.959 ms
64 bytes from 10.10.40.10: seq=2 ttl=62 time=22.874 ms
64 bytes from 10.10.40.10: seq=3 ttl=62 time=42.906 ms

cisco@S-4:~$ ping 10.10.50.10
PING 10.10.50.10 (10.10.50.10): 56 data bytes
64 bytes from 10.10.50.10: seq=0 ttl=64 time=0.056 ms
64 bytes from 10.10.50.10: seq=1 ttl=64 time=1.270 ms
64 bytes from 10.10.50.10: seq=2 ttl=64 time=0.495 ms
64 bytes from 10.10.50.10: seq=3 ttl=64 time=0.097 ms

3.5 Layer 3 External Connectivity – Border Leaf Nodes

In a VXLAN EVPN fabric, a border leaf acts as a router, forwarding traffic between the VXLAN EVPN fabric and the external Layer 3 network. Border leaves play a critical role in enabling external communication for workloads residing within the VXLAN environment. They perform essential function of external network reachability to ensure that workloads within the fabric can communicate with resources or other workloads located outside the fabric. It uses routing protocols like BGP (Border Gateway Protocol), OSPF (Open Shortest Path First), and EIGRP to exchange routing information with external routers.

3.5.1 Configuration Based on the above Topology 

### BL-1
ip prefix-list host-route seq 5 permit 0.0.0.0/0 eq 32 ### filer host routes
route-map rmap-filter-out deny 10
  match ip address prefix-list host-route  
route-map rmap-filter-out permit 1000

interface e1/6
  description connected-to-R-1-G1 
  vrf member myvrf_1
  ip address 192.168.1.1/30
  no shutdown

interface e1/7
  description connected-to-R-2-G1 
  vrf member myvrf_1
  ip address 192.168.1.9/30
  no shutdown

router bgp 65125
  template peer eBGP-external
    remote-as 65225
    address-family ipv4 unicast
      send-community both
      route-map rmap-filter-out out
  vrf myvrf_1
   address-family ipv4 unicast
      advertise l2vpn evpn
      maximum-paths ibgp 2
      maximum-paths 2
    neighbor 192.168.1.2
      inherit peer eBGP-external
    neighbor 192.168.1.10
      inherit peer eBGP-external
### R-1
interface g1
  description connected-to-L-1-E1/6
  ip address 192.168.1.2 255.255.255.252
  no shutdown

interface g2
  description connected-to-L-2-E1/6
  ip address 192.168.1.6 255.255.255.252
  no shutdown

interface g3
  description connected-to-R-3-G1
  ip address 192.168.2.1 255.255.255.252
  no shutdown

interface l0
  description router-id
  ip address 192.168.100.1 255.255.255.255

router bgp 65225
 bgp router-id 192.168.100.1
 neighbor 192.168.1.1 remote-as 65125
 neighbor 192.168.1.5 remote-as 65125
 neighbor 192.168.2.2 remote-as 65225
 !
 address-family ipv4
  neighbor 192.168.1.1 activate
  neighbor 192.168.1.1 send-community both
  neighbor 192.168.1.5 activate
  neighbor 192.168.1.5 send-community both
  neighbor 192.168.2.2 activate
  neighbor 192.168.2.2 send-community both
 exit-address-family
### R-3
interface g1
  description connected-to-R-1-G3
  ip address 192.168.2.2 255.255.255.252
  no shutdown

interface g2
  description connected-to-R-2-G3
  ip address 192.168.2.6 255.255.255.252
  no shutdown


interface l0
  description router-id
  ip address 192.168.100.3 255.255.255.255

interface l0
  description test network
  ip address 172.16.1.1 255.255.255.0

router bgp 65225
 bgp router-id 192.168.100.3
 neighbor 192.168.2.1 remote-as 65225
 neighbor 192.168.2.5 remote-as 65225
 network 172.16.1.0 mask 255.255.255.0
 !
 address-family ipv4
  neighbor 192.168.2.1 activate
  neighbor 192.168.2.1 send-community both
  neighbor 192.168.2.5 activate
  neighbor 192.168.2.5 send-community both
 exit-address-family
### BL-2
ip prefix-list host-route seq 5 permit 0.0.0.0/0 eq 32 ### filer host routes
route-map rmap-filter-out deny 10
  match ip address prefix-list host-route  
route-map rmap-filter-out deny 1000

interface e1/6
  description connected-to-R-1-G2 
  vrf member myvrf_1
  ip address 192.168.1.5/30
  no shutdown

interface e1/7
  description connected-to-R-2-G2 
  vrf member myvrf_1
  ip address 192.168.1.13/30
  no shutdown

router bgp 65125
  template peer eBGP-external
    remote-as 65225
    address-family ipv4 unicast
      send-community both
      route-map rmap-filter-out out
  vrf myvrf_1
   address-family ipv4 unicast
      advertise l2vpn evpn
      maximum-paths ibgp 2
      maximum-paths 2
    neighbor 192.168.1.6
      inherit peer eBGP-external
    neighbor 192.168.1.14
      inherit peer eBGP-external
### R-2
interface g1
  description connected-to-L-1-E1/6
  ip address 192.168.1.10 255.255.255.252
  no shutdown

interface g2
  description connected-to-L-2-E1/6
  ip address 192.168.1.14 255.255.255.252
  no shutdown

interface g3
  description connected-to-R-3-G1
  ip address 192.168.2.5 255.255.255.252
  no shutdown

interface l0
  description router-id
  ip address 192.168.100.2 255.255.255.255

router bgp 65225
 bgp router-id 192.168.100.2
 neighbor 192.168.1.9 remote-as 65125
 neighbor 192.168.1.13 remote-as 65125
 neighbor 192.168.2.6 remote-as 65225
 !
 address-family ipv4
  neighbor 192.168.1.9 activate
  neighbor 192.168.1.9 send-community both
  neighbor 192.168.1.13 activate
  neighbor 192.168.1.13 send-community both
  neighbor 192.168.2.6 activate
  neighbor 192.168.2.6 send-community both
 exit-address-family

3.5.1 Test between Internal Hosts and External IP

### Show ip route on R3 external router with subnets from internal fabric
R-3#sh ip route 
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2, m - OMP
       n - NAT, Ni - NAT inside, No - NAT outside, Nd - NAT DIA
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       H - NHRP, G - NHRP registered, g - NHRP registration summary
       o - ODR, P - periodic downloaded static route, l - LISP
       a - application route
       + - replicated route, % - next hop override, p - overrides from PfR
       & - replicated local route overrides by connected

Gateway of last resort is not set

      10.0.0.0/24 is subnetted, 4 subnets
B        10.10.20.0 [200/0] via 192.168.2.1, 21:49:41
B        10.10.30.0 [200/0] via 192.168.2.1, 21:49:41
B        10.10.40.0 [200/0] via 192.168.2.1, 21:49:41
B        10.10.50.0 [200/0] via 192.168.2.1, 21:49:41
      172.16.0.0/16 is variably subnetted, 2 subnets, 2 masks
C        172.16.1.0/24 is directly connected, Loopback0
L        172.16.1.1/32 is directly connected, Loopback0
      192.168.2.0/24 is variably subnetted, 4 subnets, 2 masks
C        192.168.2.0/30 is directly connected, GigabitEthernet1
L        192.168.2.2/32 is directly connected, GigabitEthernet1
C        192.168.2.4/30 is directly connected, GigabitEthernet2
L        192.168.2.6/32 is directly connected, GigabitEthernet2
### Show ip route on BL-1 & 2 with the externaltest network and internal networks on the routing table
BL-1# sh ip route vrf myvrf_1 
IP Route Table for VRF "myvrf_1"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

10.10.20.0/24, ubest/mbest: 2/0
    *via 10.10.100.113%default, [200/0], 22:13:10, bgp-65125, internal, tag 6512
5, segid: 32000 tunnelid: 0xa0a6471 encap: VXLAN
    *via 10.10.100.115%default, [200/0], 22:13:10, bgp-65125, internal, tag 6512
5, segid: 32000 tunnelid: 0xa0a6473 encap: VXLAN 
10.10.20.10/32, ubest/mbest: 1/0
    *via 10.10.100.143%default, [200/0], 22:13:10, bgp-65125, internal, tag 6512
5, segid: 32000 tunnelid: 0xa0a648f encap: VXLAN 
10.10.30.0/24, ubest/mbest: 2/0
    *via 10.10.100.113%default, [200/0], 22:13:10, bgp-65125, internal, tag 6512
5, segid: 32000 tunnelid: 0xa0a6471 encap: VXLAN
    *via 10.10.100.115%default, [200/0], 22:13:10, bgp-65125, internal, tag 6512
5, segid: 32000 tunnelid: 0xa0a6473 encap: VXLAN
10.10.30.10/32, ubest/mbest: 1/0
    *via 10.10.100.121%default, [200/0], 22:13:10, bgp-65125, internal, tag 6512
5, segid: 32000 tunnelid: 0xa0a6479 encap: VXLAN 
10.10.40.0/24, ubest/mbest: 2/0
    *via 10.10.100.113%default, [200/0], 22:13:10, bgp-65125, internal, tag 6512
5, segid: 32000 tunnelid: 0xa0a6471 encap: VXLAN
    *via 10.10.100.115%default, [200/0], 22:13:10, bgp-65125, internal, tag 6512
5, segid: 32000 tunnelid: 0xa0a6473 encap: VXLAN
10.10.40.10/32, ubest/mbest: 1/0
    *via 10.10.100.143%default, [200/0], 22:13:10, bgp-65125, internal, tag 6512
5, segid: 32000 tunnelid: 0xa0a648f encap: VXLAN 
10.10.50.0/24, ubest/mbest: 2/0
    *via 10.10.100.113%default, [200/0], 22:13:10, bgp-65125, internal, tag 6512
5, segid: 32000 tunnelid: 0xa0a6471 encap: VXLAN
    *via 10.10.100.115%default, [200/0], 22:13:10, bgp-65125, internal, tag 6512
5, segid: 32000 tunnelid: 0xa0a6473 encap: VXLAN
10.10.50.10/32, ubest/mbest: 1/0
    *via 10.10.100.121%default, [200/0], 22:13:10, bgp-65125, internal, tag 6512
5, segid: 32000 tunnelid: 0xa0a6479 encap: VXLAN
172.16.1.0/24, ubest/mbest: 1/0
    *via 192.168.1.10, [20/0], 22:12:47, bgp-65125, external, tag 65225
192.168.1.0/30, ubest/mbest: 1/0, attached
    *via 192.168.1.1, Eth1/6, [0/0], 22:25:39, direct
192.168.1.1/32, ubest/mbest: 1/0, attached
    *via 192.168.1.1, Eth1/6, [0/0], 22:25:39, local
192.168.1.8/30, ubest/mbest: 1/0, attached
    *via 192.168.1.9, Eth1/7, [0/0], 22:25:35, direct
192.168.1.9/32, ubest/mbest: 1/0, attached
    *via 192.168.1.9, Eth1/7, [0/0], 22:25:35, local
BL-2# show ip route vrf myvrf_1 
IP Route Table for VRF "myvrf_1"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

10.10.20.0/24, ubest/mbest: 2/0
    *via 10.10.100.113%default, [200/0], 1d03h, bgp-65125, internal, tag 65125, 
segid: 32000 tunnelid: 0xa0a6471 encap: VXLAN 
    *via 10.10.100.115%default, [200/0], 1d03h, bgp-65125, internal, tag 65125, 
segid: 32000 tunnelid: 0xa0a6473 encap: VXLAN 
10.10.20.10/32, ubest/mbest: 1/0
    *via 10.10.100.143%default, [200/0], 1d00h, bgp-65125, internal, tag 65125, 
segid: 32000 tunnelid: 0xa0a648f encap: VXLAN 
10.10.30.0/24, ubest/mbest: 2/0
    *via 10.10.100.113%default, [200/0], 1d03h, bgp-65125, internal, tag 65125, 
segid: 32000 tunnelid: 0xa0a6471 encap: VXLAN
    *via 10.10.100.115%default, [200/0], 1d03h, bgp-65125, internal, tag 65125, 
segid: 32000 tunnelid: 0xa0a6473 encap: VXLAN 
10.10.30.10/32, ubest/mbest: 1/0
    *via 10.10.100.121%default, [200/0], 1d03h, bgp-65125, internal, tag 65125, 
segid: 32000 tunnelid: 0xa0a6479 encap: VXLAN
10.10.40.0/24, ubest/mbest: 2/0
    *via 10.10.100.113%default, [200/0], 1d03h, bgp-65125, internal, tag 65125, 
segid: 32000 tunnelid: 0xa0a6471 encap: VXLAN
    *via 10.10.100.115%default, [200/0], 1d03h, bgp-65125, internal, tag 65125, 
segid: 32000 tunnelid: 0xa0a6473 encap: VXLAN 
10.10.40.10/32, ubest/mbest: 1/0
    *via 10.10.100.143%default, [200/0], 1d03h, bgp-65125, internal, tag 65125, 
segid: 32000 tunnelid: 0xa0a648f encap: VXLAN
10.10.50.0/24, ubest/mbest: 2/0
    *via 10.10.100.113%default, [200/0], 1d03h, bgp-65125, internal, tag 65125, 
segid: 32000 tunnelid: 0xa0a6471 encap: VXLAN
    *via 10.10.100.115%default, [200/0], 1d03h, bgp-65125, internal, tag 65125, 
segid: 32000 tunnelid: 0xa0a6473 encap: VXLAN 
10.10.50.10/32, ubest/mbest: 1/0
    *via 10.10.100.121%default, [200/0], 1d03h, bgp-65125, internal, tag 65125, 
segid: 32000 tunnelid: 0xa0a6479 encap: VXLAN
172.16.1.0/24, ubest/mbest: 1/0
    *via 192.168.1.6, [20/0], 22:22:58, bgp-65125, external, tag 65225
192.168.1.4/30, ubest/mbest: 1/0, attached
    *via 192.168.1.5, Eth1/6, [0/0], 22:26:33, direct
192.168.1.5/32, ubest/mbest: 1/0, attached
    *via 192.168.1.5, Eth1/6, [0/0], 22:26:33, local
192.168.1.12/30, ubest/mbest: 1/0, attached
    *via 192.168.1.13, Eth1/7, [0/0], 22:26:26, direct
192.168.1.13/32, ubest/mbest: 1/0, attached
    *via 192.168.1.13, Eth1/7, [0/0], 22:26:26, local
### Ping tests from the external and internal hosts
### from external system to internal hosts
R-3#ping 10.10.20.10 source 172.16.1.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.10.20.10, timeout is 2 seconds:
Packet sent with a source address of 172.16.1.1 
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 43/44/45 ms
R-3#ping 10.10.30.10 source 172.16.1.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.10.30.10, timeout is 2 seconds:
Packet sent with a source address of 172.16.1.1 
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 20/28/50 ms
R-3#ping 10.10.40.10 source 172.16.1.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.10.40.10, timeout is 2 seconds:
Packet sent with a source address of 172.16.1.1 
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 16/27/35 ms
R-3#ping 10.10.50.10 source 172.16.1.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.10.50.10, timeout is 2 seconds:
Packet sent with a source address of 172.16.1.1 
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 19/20/24 ms

### from Internal hosts to external system 

cisco@S-2:~$ ping 172.16.1.1
PING 172.16.1.1 (172.16.1.1): 56 data bytes
64 bytes from 172.16.1.1: seq=0 ttl=252 time=55.105 ms
64 bytes from 172.16.1.1: seq=1 ttl=252 time=44.408 ms
64 bytes from 172.16.1.1: seq=2 ttl=252 time=26.869 ms
64 bytes from 172.16.1.1: seq=3 ttl=252 time=50.347 ms
64 bytes from 172.16.1.1: seq=4 ttl=252 time=68.280 ms
^C
--- 172.16.1.1 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 26.869/49.001/68.280 ms

cisco@S-3:~$ ping 172.16.1.1
PING 172.16.1.1 (172.16.1.1): 56 data bytes
64 bytes from 172.16.1.1: seq=0 ttl=252 time=33.930 ms
64 bytes from 172.16.1.1: seq=1 ttl=252 time=23.624 ms
64 bytes from 172.16.1.1: seq=2 ttl=252 time=22.862 ms
64 bytes from 172.16.1.1: seq=3 ttl=252 time=18.312 ms
64 bytes from 172.16.1.1: seq=4 ttl=252 time=20.479 ms
^C
--- 172.16.1.1 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 18.312/23.841/33.930 ms

cisco@S-4:~$ ping 172.16.1.1
PING 172.16.1.1 (172.16.1.1): 56 data bytes
64 bytes from 172.16.1.1: seq=0 ttl=252 time=24.677 ms
64 bytes from 172.16.1.1: seq=1 ttl=252 time=20.368 ms
64 bytes from 172.16.1.1: seq=2 ttl=252 time=27.852 ms
64 bytes from 172.16.1.1: seq=3 ttl=252 time=28.689 ms
64 bytes from 172.16.1.1: seq=4 ttl=252 time=37.514 ms
^C
--- 172.16.1.1 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 20.368/27.820/37.514 ms

Related Links

https://deliabtech.com/blogs/underlay-multicast-routing-for-vxlan-bum-traffic/

https://deliabtech.com/data-center/vxlan-evpn-multi-site/

About

Leave a Comment

Your email address will not be published. Required fields are marked *