EVPN/VXLAN BUM traffic troubleshooting becomes painful when everyone starts at the wrong layer. A missing ARP reply, a silent PXE boot, or a VM that only fails before it has learned a unicast MAC address is often blamed on the application, but the real issue is usually in broadcast, unknown-unicast and multicast replication.
This article gives a practical checklist for finding BUM failures in an EVPN/VXLAN fabric, with special attention to head-end replication versus multicast underlay designs. It is written for operators who need a repeatable workflow, not a protocol theory lecture.
The problem: unicast looks healthy, but first contact fails
BUM problems usually show up as symptoms that are easy to misread:
- ARP or IPv6 neighbor discovery is intermittent between hosts in the same stretched VLAN.
- PXE, DHCP, clustering heartbeats or service discovery works on one rack but not another.
- Known unicast traffic works after a manual ping, but new conversations fail until entries are learned somewhere else.
- Only one VNI or VLAN is affected while the rest of the fabric looks clean.
If these symptoms appear in an EVPN/VXLAN fabric, do not jump straight to BGP route targets or server NIC drivers. First separate the forwarding path into four checks: local VLAN flooding, VNI mapping, replication delivery, and remote decapsulation. For broader fabric design context, see the Data Center networking hub and the Start Here networking topics page.
Design diagnosis: head-end replication versus multicast underlay
Head-end replication
With head-end replication, the ingress VTEP creates separate VXLAN packets for each remote VTEP that participates in the VNI. This is operationally simple because the underlay only needs IP unicast reachability between VTEPs. The trade-off is replication load on the ingress leaf. A wrong flood list, missing EVPN Inclusive Multicast Ethernet Tag route, or stale VTEP membership can break BUM delivery even when normal unicast VXLAN traffic works.
Multicast underlay replication
With multicast underlay replication, a VNI maps to an underlay multicast group. The ingress VTEP sends one packet to the group, and the underlay replicates it. This scales replication load differently, but adds PIM, IGMP, rendezvous point or multicast routing state to the troubleshooting path. If multicast state is incomplete, BUM traffic can disappear between spines and leaves while VTEP loopbacks still ping perfectly.
EVPN control-plane clues
For either model, check the control plane before packet captures become a fishing trip. Type-3 IMET routes tell you which VTEPs should receive BUM traffic for a given VNI. Type-2 MAC/IP routes tell you whether hosts are eventually learned correctly. Type-5 routes are usually not the first place to look for same-subnet BUM failures, but wrong route leaking can create confusing side effects. The BGP Table Watch page is a useful anchor for related routing checks.
Implementation workflow: isolate the failing layer
1. Confirm the local VLAN and VNI mapping
show vlan id 100
show interface trunk | include 100
show nve vni 10100
show mac address-table dynamic vlan 100
Look for simple mismatches: VLAN not allowed on the access-facing port, wrong VLAN-to-VNI mapping, VNI administratively down, or a bridge-domain configuration that exists on one leaf but not the other. In multi-vendor environments, also verify whether the platform requires an explicit flood list, EVPN route import, or per-VNI multicast group.
2. Check EVPN IMET membership
show bgp l2vpn evpn route-type 3
show bgp l2vpn evpn vni 10100
show nve peers
Every VTEP that should receive BUM for the VNI must appear in the membership view. If Leaf A does not know Leaf B participates in VNI 10100, head-end replication will never send the copy. If the IMET route exists but is not imported, focus on route targets, EVPN address-family activation, and policy filters.
3. Validate underlay reachability and MTU
ping <remote-vtep-loopback> source <local-vtep-loopback> df-bit size 8972
traceroute <remote-vtep-loopback> source <local-vtep-loopback>
show interface counters errors
BUM frames are still VXLAN packets in the underlay. MTU loss can make small pings pass while real frames fail. This is especially visible when DHCP options, storage discovery, or larger broadcast packets are involved. If the underlay has ECMP, test more than one flow when the platform allows it.
4. Add multicast-specific checks when the design uses multicast
show ip mroute <group>
show ip pim neighbor
show ip pim rp mapping
show ip igmp groups
Confirm that leaves join the correct group and that spines replicate toward the expected receivers. A single wrong group value on one VNI can look exactly like an EVPN problem from the host side.
5. Capture at ingress and egress VTEPs
When counters and control-plane state disagree, use packet capture in two places: before encapsulation on the ingress leaf and after decapsulation on the remote leaf. The goal is to answer one question at a time: did the frame enter the fabric, was it encapsulated, did the remote VTEP receive it, and was it emitted into the local VLAN?
Verification and troubleshooting checklist
- Local only test: two hosts on the same leaf and VLAN should ARP without using VXLAN.
- Remote same-subnet test: hosts on different VTEPs in the same VNI should learn each other through BUM and then switch to known unicast.
- Control-plane test: the VNI should have correct Type-3 membership and expected Type-2 host routes after traffic starts.
- Data-plane test: VXLAN encapsulated packets should increment on ingress and decapsulated frames should increment on egress.
- Negative test: remove a lab VTEP from the VNI and confirm BUM membership changes as expected before using the same logic in production.
If automation is available, make this a pre-change and post-change validation. An AI-assisted workflow can summarize diffs and highlight missing VNI membership, but keep the actual checks deterministic: parsed command output, expected VTEP set, expected route targets, and a pass/fail result. For more automation ideas, see the AI Infrastructure and automation hub and the networking resources page.
Practical takeaways
- Do not troubleshoot BUM as one giant EVPN problem. Split it into local VLAN, VNI, replication and remote VLAN checks.
- For head-end replication, verify IMET membership and the actual remote VTEP list for the VNI.
- For multicast replication, verify multicast state with the same discipline as BGP state.
- Always test MTU from VTEP loopback to VTEP loopback with the encapsulation overhead in mind.
- Capture only after you know what packet should exist and where it should appear.
A good EVPN/VXLAN BUM troubleshooting runbook should fit on one page. Start at the host-facing VLAN, prove the replication target set, prove the underlay can deliver the encapsulated packet, and only then chase advanced edge cases.
Post a Comment