EVPN/VXLAN anycast gateway troubleshooting is easiest when you separate the underlay, BGP EVPN control plane, VNI mapping and host evidence instead of staring at one broken ping. The common failure pattern is simple: hosts in the same rack work, but inter-rack or inter-subnet traffic fails after a new VLAN, VRF or leaf pair is added.
This checklist is vendor-neutral, but the commands use familiar NX-OS style wording because many data-center fabrics expose EVPN state in that form. Adapt the command names to your platform and keep the verification order. If you are new to the fabric vocabulary, start with the Data Center hub and the Start Here page.
The problem: local VLAN works, routed overlay fails
A typical ticket says: “VMs on VLAN 10 can talk on Leaf-1, but cannot reach VLAN 20 behind Leaf-4.” That sentence hides four possible faults:
- the underlay cannot carry VXLAN packets between VTEP loopbacks;
- BGP EVPN is not advertising or importing the right MAC/IP and IP-prefix routes;
- the VLAN-to-L2VNI or VRF-to-L3VNI binding differs across leaves;
- the anycast gateway SVI is inconsistent, so hosts ARP for a gateway that behaves differently depending on attachment point.
The key discipline is to prove each layer before moving up. This avoids random fixes such as bouncing NVE interfaces, clearing BGP sessions or moving workloads without knowing the failing boundary.
Design baseline: what must be identical and what must be unique
Keep the anycast gateway identical
For a stretched VLAN, every leaf that hosts that VLAN should use the same default gateway IP and the same anycast gateway MAC. The host should see one stable gateway, regardless of the physical leaf where it lands.
fabric anycast-gateway-mac 0200.0aaa.bbbb
vlan 10
vn-segment 10100
interface Vlan10
vrf member APP
ip address 10.10.10.1/24
fabric forwarding mode anycast-gatewayKeep VTEP loopbacks unique and reachable
VTEP loopback addresses are unique. They must be reachable through the underlay with the right MTU. If the VTEP loopback route is missing or ECMP sends jumbo VXLAN packets into a small-MTU path, EVPN may look healthy while data traffic fails.
interface loopback1
description VTEP source
ip address 192.0.2.11/32
interface nve1
source-interface loopback1
host-reachability protocol bgpRoute targets must match the intent
L2VNIs carry bridge-domain reachability. L3VNIs carry VRF routing. If import/export route targets are different between leaves, the fabric may learn local hosts but never install remote prefixes into the tenant VRF.
Implementation checklist before blaming EVPN
1. Prove underlay reachability and MTU
Start with the transport. From each leaf, test every remote VTEP loopback. Use a payload size that represents the post-encapsulation path. For a 1500-byte tenant packet, the underlay normally needs room for outer Ethernet, IP, UDP and VXLAN headers.
show ip route 192.0.2.14/32
ping 192.0.2.14 source 192.0.2.11 df-bit packet-size 8972
show interface counters errors
show ip ospf neighbor ! or IS-IS/BGP underlay equivalentIf this step fails, stay in the underlay. Check ECMP asymmetry, unnumbered link state, MTU, ACLs and CoPP before touching EVPN.
2. Check BGP EVPN neighbors and route types
EVPN needs control-plane reachability. Route type 2 advertises MAC/IP host reachability; route type 5 advertises IP prefixes. In symmetric IRB designs, both often matter.
show bgp l2vpn evpn summary
show bgp l2vpn evpn route-type 2 mac-ip
show bgp l2vpn evpn route-type 5 ip-prefix
show bgp l2vpn evpn vni-id 10100Useful questions: does the remote host MAC/IP appear? Is the next hop the remote VTEP? Are the expected route targets attached? Is the route accepted but not installed because the local VNI or VRF is missing?
3. Validate VLAN, L2VNI, VRF and L3VNI mapping
The fastest way to find config drift is to compare a working leaf with a broken leaf. Build a tiny table: VLAN, L2VNI, SVI state, VRF, L3VNI, NVE membership and route targets.
show vlan id 10
show nve vni 10100
show vrf APP
show nve vni | include 50000
show running-config interface vlan 10
show running-config vrf context APPA single wrong VNI number can create a silent black hole: the VLAN exists, the SVI is up, BGP is up, but the local leaf is advertising into a different overlay segment.
Verification and troubleshooting workflow
Host-side verification
After the fabric checks, look from the host edge inward. Confirm that the host has the expected default gateway, ARP entry and VLAN attachment. Then check the fabric tables on the ingress leaf.
show mac address-table dynamic vlan 10
show ip arp vrf APP 10.10.10.50
show forwarding route vrf APP 10.20.20.0/24
show system internal l2fwder mac ! platform-specific; use carefullyDo not ignore duplicate IP or MAC movement signals. In a virtualized environment, a vMotion event, stale port profile or duplicate test VM can look like an EVPN control-plane issue.
Packet path test
- Ping from Host A to its default gateway. If this fails, fix access VLAN/SVI/host configuration.
- Ping Host A to Host B in the same L2VNI if applicable. If this fails only across leaves, inspect route type 2 and NVE peers.
- Ping Host A to a remote subnet. If L2 works but L3 fails, inspect L3VNI, VRF route targets and route type 5.
- Run a DF-bit VTEP-to-VTEP ping. If control plane is fine but larger packets fail, suspect MTU.
Practical takeaways
- Always start below VXLAN. A broken underlay or MTU mismatch can masquerade as an EVPN problem.
- Compare a working leaf to a broken leaf. Most new-VLAN incidents are VNI, SVI, VRF or route-target drift.
- Use route type 2 and type 5 deliberately. Know which one should prove host reachability and which one should prove prefix reachability in your design.
- Standardize anycast gateway values. The gateway IP and MAC are boring until one leaf is different.
- Capture the evidence before clearing sessions. A clear command may hide the real root cause until the next change window.
For deeper reading paths, see the AI Infrastructure and Automation page for fabric automation ideas, the BGP Table Watch page for control-plane thinking, and the Resources page for external references.
Post a Comment