EVPN/VXLAN troubleshooting becomes much easier when you stop treating it as a magic overlay and split the fault into three layers: local switching, BGP EVPN control plane and VXLAN data plane. This guide gives a practical checklist for the common case: two endpoints should communicate through a leaf-spine fabric, but ping, ARP, TCP or a VM migration test fails.
The examples use documentation-style names such as leaf-1, vni-10100 and RFC 5737/RFC 3849 addressing. Replace them with your platform commands and real values in the lab, but keep the same order of operations.
The problem: one broken flow can have many EVPN causes
A simple symptom like “Host A cannot reach Host B” may come from any of these places:
- the endpoint VLAN is not allowed or is mapped to the wrong VNI;
- the local leaf never learned the source MAC or ARP/ND entry;
- BGP EVPN route-target import/export is wrong, so the remote leaf never receives route-type 2 or route-type 5;
- the underlay can route to the remote VTEP loopback, but MTU or UDP/4789 filtering breaks VXLAN packets;
- security policy, contracts, ACLs or distributed firewall rules block the flow after control-plane learning looks healthy.
For wider design context, keep the Data Center networking hub and AI Infrastructure and automation hub open while building your own checklist.
Design model: separate underlay, overlay and tenant policy
Before running commands, write down the expected facts for the failing flow. This prevents random changes and makes the escalation useful.
Minimum evidence table
| Item | Example | Why it matters |
|---|---|---|
| Source / destination | 10.10.10.11 → 10.10.10.22 | Defines the exact endpoints; avoid testing a different subnet by accident. |
| Tenant VRF | vrf-app | Route-targets and route leaking are VRF-specific. |
| VLAN / VNI | VLAN 100 / VNI 10100 | Wrong mapping is a classic silent failure. |
| Local and remote VTEP | 192.0.2.11 / 192.0.2.12 | Needed for underlay reachability and VXLAN counters. |
| Expected route types | RT-2 for MAC/IP, RT-3 for flood list, RT-5 for routed prefix | Tells you whether L2 extension or L3 routing should carry the flow. |
Implementation checklist: commands to adapt to your NOS
The exact syntax differs between NX-OS, EOS, Junos, SONiC and other platforms. The logic is portable: prove one layer, then move to the next.
1. Prove local endpoint learning
show vlan id 100
show interface trunk | include 100
show mac address-table vlan 100 | include 0011.2233.4455
show ip arp vrf vrf-app 10.10.10.11
show nve vni 10100 detailIf the source MAC is not local on the access leaf, EVPN is not the first problem. Check the access port, MLAG/vPC state, VLAN pruning, NIC bonding mode and whether the endpoint moved to another rack.
2. Prove EVPN control-plane propagation
show bgp l2vpn evpn route-type 2 mac-ip 0011.2233.4455
show bgp l2vpn evpn route-type 2 ip 10.10.10.22
show bgp l2vpn evpn route-type 3
show bgp l2vpn evpn route-type 5 10.10.10.0/24
show bgp l2vpn evpn summaryCommon findings:
- Route exists locally but not remotely: check BGP neighbors, address-family activation, route-reflector policy and route-target filters.
- Route imported into the wrong VRF: compare RD/RT values and tenant templates.
- Stale MAC after VM move: inspect duplicate MAC detection, mobility sequence numbers and hold timers before clearing large tables.
If BGP behavior is suspicious, the BGP Table Watch page is also a useful hub for thinking in terms of prefixes, next-hops and route visibility.
3. Prove VXLAN data-plane reachability
ping 192.0.2.12 source 192.0.2.11
ping 192.0.2.12 source 192.0.2.11 size 8972 df-bit
show nve peers
show interface nve1 counters
show hardware forwarding drops | include vxlan|mtu|aclDo not skip the large-packet test. A fabric may pass small ICMP between VTEP loopbacks while dropping encapsulated tenant traffic because the underlay MTU is too small. For AI or storage networks, this is even more important because the applications will expose packet loss quickly.
4. Check tenant security after routing works
When MAC, ARP/ND, BGP EVPN and VXLAN counters all look healthy, move to policy:
- distributed firewall or contract denies;
- VRF route leaking missing between frontend, application and backup segments;
- asymmetric return path through a service firewall;
- host firewall or hypervisor security group blocking the application port.
This order matters. If you start by changing ACLs while the VNI is wrong, you create two faults instead of one.
Verification and troubleshooting workflow
- Refresh endpoint state: clear only the specific ARP/ND or MAC entry if needed, not the whole fabric.
- Retest L2 and L3 separately: ARP/ND, same-subnet ping, routed ping and the real TCP/UDP application flow.
- Watch counters while testing: interface, NVE, ACL and drop counters should move in the expected direction.
- Capture sparingly: capture on the endpoint or leaf only for the failing VNI and host pair.
- Record the fix: note whether the root cause was access, BGP EVPN, underlay MTU, VXLAN data plane or policy.
Practical takeaways
- A working EVPN/VXLAN flow needs local endpoint learning, correct EVPN routes, reachable VTEP loopbacks and policy allowing the tenant traffic.
- Route-type 2 tells you MAC/IP reachability; route-type 3 tells you flood-list membership; route-type 5 tells you routed prefix reachability.
- MTU and UDP/4789 filtering are frequent causes when BGP looks perfect but applications fail.
- Use one evidence table per incident. It is faster than jumping between random show commands.
- For more foundations and related checklists, start from Start Here: Networking Topics and the Resources page.
Material usage note: This post uses generic diagrams redrawn for publication. The local material index was used for topic inspiration and safe lab-style visual direction only; no customer names, private hostnames, project names, private addressing, ticket data or vendor/customer-specific diagrams were published.
Post a Comment