EVPN/VXLAN silent host troubleshooting is a common data-center problem: the fabric looks healthy, BGP EVPN sessions are up, VNIs are present, but one server, VM, storage node or appliance cannot be reached until it sends traffic first.
The usual cause is not “VXLAN is broken”. It is missing or stale endpoint state: the remote leaf does not have a usable MAC/IP binding, ARP/ND suppression answers from the wrong cache entry, or BUM flooding is constrained so discovery never reaches the quiet host.
For related reading, start with the Start Here networking topics page, the Data Center Networking hub, AI Infrastructure & Automation, BGP Table Watch and the Resources page.
The problem: the host works only after it talks first
Silent hosts are endpoints that do not frequently generate ARP, Neighbor Discovery, gratuitous ARP, unsolicited NA or application traffic. Examples include standby appliances, backup targets, storage controllers, bare-metal servers after a NIC reset, BMC-like management interfaces, and VMs that moved but kept quiet after migration.
In a classic VLAN, flooding often hides the problem. In an EVPN/VXLAN fabric, we intentionally reduce flooding with control-plane learning, ARP suppression and tighter VNI membership. That is good design, but it means stale or absent endpoint state becomes visible.
Design background: what must be true for reachability?
For a remote endpoint to reach a silent host behind another VTEP, the fabric needs a consistent chain of facts:
- The local leaf learned the host MAC on the correct access interface, VLAN and bridge domain.
- The leaf has a correct ARP/ND binding if the fabric advertises MAC/IP routes.
- BGP EVPN advertises a valid Type-2 route with the right route-target, VNI and VTEP next-hop.
- The remote leaf imports that route and programs hardware forwarding.
- If the route is missing, BUM replication must still be able to discover the host or intentionally fail closed.
Diagnosis workflow
1. Start at the leaf where the silent host is attached
Do not begin by clearing BGP sessions. First prove whether the local leaf learned the endpoint at all.
# Generic local checks
show mac address-table vlan <tenant-vlan> | include <host-mac>
show arp vrf <tenant-vrf> | include <host-ip>
show ipv6 neighbors vrf <tenant-vrf> | include <host-ipv6>
show interface <access-port> counters errors
show interface <access-port> switchport / vlan membershipIf the MAC is absent locally, EVPN cannot advertise it. Look for link state, wrong VLAN, port-security, storm-control, server NIC teaming, virtualization port groups or a firewall/appliance that suppresses gratuitous ARP.
2. Check the EVPN Type-2 route, not only the VNI
A healthy NVE interface and an up BGP session are not enough. You need the exact MAC/IP route for the endpoint.
# Generic EVPN checks
show bgp l2vpn evpn route-type 2 mac <host-mac>
show bgp l2vpn evpn route-type 2 ip <host-ip>
show nve vni <vni-id> detail
show bgp l2vpn evpn route-target <tenant-rt>
show forwarding vxlan route vni <vni-id> mac <host-mac>If the local leaf has the MAC but does not advertise it, check bridge-domain to VNI mapping, route-target export, duplicate MAC detection, EVPN multi-homing state and any policy that filters Type-2 routes.
3. Compare ARP/ND suppression cache with reality
ARP suppression is valuable because it reduces broadcast noise, especially in large tenant segments. It is also a common place for stale state. If the fabric answers ARP for an address that moved, the remote host may keep sending to the wrong VTEP.
# Questions to answer
Is the ARP/ND cache bound to the current MAC?
Is the MAC advertised from the current leaf or an old one?
Did a VM move without a gratuitous ARP / unsolicited NA?
Are suppression timers longer than endpoint mobility timers?4. Validate BUM replication before blaming the endpoint
When control-plane learning is absent, unknown unicast, broadcast and multicast behavior determines whether the endpoint can be discovered. Ingress replication lists, multicast groups, flood lists and VNI membership must include the relevant VTEPs. Also check whether storm-control, ACLs or firewall insertion drops ARP/ND.
Implementation and remediation options
Use a small fix first
The safest first action is to refresh endpoint state without changing the whole fabric. Trigger a gratuitous ARP or neighbor advertisement from the host, bounce only the affected access port if the owner approves, or clear the single stale ARP/MAC entry if your platform supports precise clearing.
Tune the fabric if the issue repeats
If the same pattern returns, tune the design rather than relying on manual clears:
- Align MAC ageing, ARP/ND cache and EVPN mobility timers.
- Confirm duplicate MAC detection thresholds are not suppressing legitimate moves.
- Keep ARP/ND suppression enabled only where the vendor behavior and operational visibility are understood.
- Ensure EVPN route-target import/export policy matches the tenant design.
- For appliances and clusters, document whether they send gratuitous ARP on failover.
Verification checklist
After remediation, verify all layers agree. Do not stop after one successful ping.
# Minimum verification set
local leaf: MAC learned on the expected access port
local leaf: ARP/ND entry maps expected IP to expected MAC
BGP EVPN: Type-2 MAC/IP route exported with correct VNI and RT
remote leaf: Type-2 route imported and programmed in hardware
traffic: bidirectional test passes from more than one remote rack
telemetry: no new MAC moves, drops, BUM spikes or BGP flapsFor storage, backup or AI cluster networks, add application-level probes: mount checks, small read/write tests, TCP retransmits, RoCE counters if applicable, and latency before/after comparison.
Practical takeaways
- Silent host issues are usually endpoint-state problems, not generic VXLAN failures.
- Always check the exact EVPN Type-2 MAC/IP route and the hardware table on the remote leaf.
- ARP/ND suppression should reduce noise, not hide stale bindings; monitor and tune it.
- Do not globally enable flooding or disable suppression as a first reaction.
- Build a small runbook for single-entry clears, endpoint GARP/NA triggers and post-fix verification.
Post a Comment