A pool member is red, but the application opens in your browser. Or every member looks healthy while users report errors. Start by asking what the health check actually requests, from which source, and what response it accepts—not by removing the monitor.
This F5 BIG-IP LTM troubleshooting guide provides a reusable symptom matrix, a monitor request worksheet and a controlled acceptance test plan. It targets conventional BIG-IP TMOS LTM, with the official BIG-IP 17.0 monitoring guide as a conceptual reference; confirm settings against your installed release.[3] All examples are hypothetical and unexecuted, not customer configurations or measured incident results.
Why a browser test and an F5 monitor can disagree
F5 monitors check a particular protocol, service or application at a configured interval; a successful check therefore needs to be interpreted in the context of that monitor type.[3] An HTTP monitor request with a malformed Send string, an unsuitable Receive expression, or an unexpected redirect can mark a member down even though another request succeeds.[4]
For incident triage, treat a browser request and a health-monitor request as separate experiments. Record their destination, transport, application name, path and authentication rather than assuming they are equivalent. A browser reaching the public virtual server is not a direct test of the same backend endpoint the monitor targets.
Recommended first decision: if the member is down, investigate the exact failing monitor and its path. If the member is up but the application fails, compare the monitor's success condition with the user's failed transaction before changing load balancing or persistence.
Durable asset: pool-member troubleshooting matrix
The table is an original diagnostic workflow, not a claim that one symptom proves a particular root cause. Send/Receive checks, logs and monitor captures follow F5's troubleshooting guidance.[4]
| Symptom | First evidence to collect | Discriminating test | Avoid this shortcut |
|---|---|---|---|
| Member is down; browser works | Assigned monitor, target port, Send and Receive settings | Reproduce the monitor's exact path and application Host value against that backend | Replacing the application check with ping |
| Member flaps during busy periods | Status-transition timestamps and backend response timing | Correlate one failure window with server load and captured monitor responses | Raising every timeout before locating the delay |
| All members fail together after a change | Monitor revision and shared network/application changes | Compare an approved pre-change request with the current request | Editing every backend independently |
| Only one member fails | Per-member response, inherited monitor settings and node status | Send the same controlled request to each member | Assuming identical deployment means identical content |
| TCP connection opens; HTTP check fails | Actual HTTP response and Receive expression | Check Host, URI, status, redirect, authentication and response body | Calling TCP reachability an application-health result |
| HTTP succeeds; HTTPS check fails | TLS handshake details and monitor TLS settings | Compare protocol, cipher and server-name requirements with the actual monitor behavior | Disabling certificate checks as a permanent fix |
| Pool is green; users see errors | User transaction plus exact monitor success condition | Exercise the failing dependency or application action separately | Assuming a health endpoint proves every business transaction |
| Check works from an admin laptop only | Monitor source address, route and backend access policy | Observe the request leaving BIG-IP and the corresponding reply | Treating the management network as the data path |
Step 1: establish state before changing anything
Capture the affected virtual server, pool, member and node; include the partition and route domain where applicable. Record the software release, current active unit and change window. Keep real identifiers in your private incident record, not a public ticket attachment.
F5 documents tmsh show /ltm pool and tmsh show /ltm node for status collection and identifies /var/log/ltm as the health-monitor event log.[4] These are illustrative read-only commands; replace the uppercase placeholders with approved object names:
tmsh show /ltm pool /Common/POOL_NAME
tmsh show /ltm node /Common/NODE_NAME
Use the Configuration utility to inspect the monitor assignment and any member-level overrides. Save the administrative state separately from the health-check result. Do not turn an intentionally isolated member back on because its application responds.
Build a small timeline: last known good transaction, configuration change, first monitor failure, first user error and recovery. The useful evidence is the order of events, not merely a screenshot of a red icon.
Step 2: complete the monitor request worksheet
F5's ECV monitor guidance identifies Send and Receive settings and authentication fields as key checks; the Receive value is a regular expression, and redirects or a match appearing too late in the response can cause failure.[4]
Copy this worksheet into your change record before proposing a fix:
| Field | What to write down |
|---|---|
| Scope | Pool, member and node monitor assignments; any override or alias target |
| Destination | Backend endpoint and port actually checked |
| Transport | HTTP or HTTPS; required TLS behavior if applicable |
| Application selection | Expected Host header; required TLS server name assessed separately |
| Request | Method, URI, HTTP version and line termination |
| Authentication | Whether required; credential reference only, never the secret |
| Success condition | Expected status/body and exact Receive expression |
| Timing | Interval, timeout and observed response timing |
| Source and path | Observed monitor source, VLAN, route domain and return path |
| Ownership | Application owner who approved the endpoint's health meaning |
A hypothetical HTTP health contract
For a lab application named app.example, define a dedicated /health/ready endpoint returning a small body containing a distinctive token such as APP_READY_OK. Agree that this token appears only when the instance should accept new work. This is a proposed application contract, not an F5 default.
An illustrative request representation is:
GET /health/ready HTTP/1.1\r\nHost: app.example\r\nConnection: close\r\n\r\n
Here \r\n denotes wire-format line endings. This is not a paste-ready tmsh command: confirm how your GUI or configuration interface encodes the Send field, then verify the actual request on the wire. F5 explicitly calls out HTTP request construction and CR/LF handling as monitor troubleshooting topics.[4]
Test both success and failure content. A generic word such as OK is a poor proposed contract if it can appear on an error page. Do not define health as a redirect to a login page unless that is deliberately the service you intend to test.
For HTTPS, record the HTTP Host value and the TLS server-name requirement separately. F5 notes that HTTPS monitors use OpenSSL for cipher negotiation; investigate the monitor's TLS behavior rather than assuming a browser's successful negotiation proves compatibility.[4]
Step 3: prove the monitor path with a bounded capture
F5's documented monitor-capture workflow uses ip route get to help identify the source self IP and then captures on the relevant internal VLAN.[4] Confirm the applicable route-domain context; do not treat a default-context route lookup as universal evidence for every deployment.
The following is an unexecuted template, not a runnable command until placeholders are replaced:
ip route get BACKEND_IP
tcpdump -nnvi INTERNAL_VLAN:nnn -s0 -c 200 -w /var/tmp/monitor-check.pcap 'host BACKEND_IP and port BACKEND_PORT'
The packet-count cap limits packets, not elapsed time. Stop the capture manually at the approved time even if the count has not been reached. Check storage first, restrict access to the capture and remove it according to your retention policy. Full-payload captures can contain credentials or application data; use a dedicated non-sensitive health endpoint where possible.
Read the capture as a sequence of questions:
- Does a request leave the expected VLAN for the intended destination?
- Is the source address one the backend and intermediate policy permit?
- Does the backend reply, and does the reply return to the originating BIG-IP?
- For HTTPS, does the exchange progress beyond the TLS handshake?
- For visible HTTP, does the response match the configured success condition?
These observations choose the next owner: missing egress evidence calls for appliance/path investigation; unanswered requests call for network and server investigation; a valid response rejected by the success condition calls for monitor/application-contract review. Encrypted packet payloads alone will not reveal an HTTPS response body—correlate with backend and monitor logs.
Step 4: fix the smallest demonstrated mismatch
Prefer a targeted, reversible change. Examples include correcting the requested URI, matching the application's required Host header, repairing backend reachability or adjusting an incorrectly defined Receive expression. Clone a shared monitor for a scoped trial rather than silently changing the health policy of unrelated pools.
F5 recommends checking the interval/timeout relationship and backend connectivity when diagnosing monitor failures.[4] Use captured timing to justify any timer change; a longer timeout should not substitute for understanding an application stall. Agree the acceptable removal and recovery behavior with the application owner.
If more detail is needed, F5 provides per-member or per-node Monitor Logging and stores those logs under /var/log/monitors/; its guidance warns that leaving logging enabled can fill /var/log.[4] Enable it only for a bounded diagnostic window and disable it afterward. Avoid restarting health-monitor processes as an exploratory first step.
Durable asset: acceptance and rollback checklist
This proposed test plan belongs in a lab or an approved maintenance window. It has not been executed for this article.
| Test | Evidence required before sign-off |
|---|---|
| Healthy member | Intended request succeeds and normal client transaction succeeds |
| Application readiness failure | Approved failure injection removes the member from new-work selection as intended |
| Listener/network failure | Failure is detected; another member handles the agreed test transaction |
| Recovery | Restored service returns according to configured recovery/manual-resume policy |
| Redirect or wrong content | Negative response does not accidentally satisfy the success condition |
| Other pool using old monitor | Its behavior remains unchanged by the scoped repair |
| Cleanup | Temporary logging stopped, captures secured, temporary changes removed |
| Rollback | Previous monitor object, assignment and administrative states recorded and recoverable |
Before testing, identify a stop condition: unacceptable user errors, unexpected impact outside the target pool or failure of the healthy alternative member. Restore the approved prior configuration if the change itself causes harm. Do not blindly roll back to a known-broken monitor and declare the incident resolved; escalate with the captured evidence and application owner.
Practical takeaways
A useful monitor is a small, explicit application-health contract. Troubleshoot the exact request and response, retain the distinction between administrative state and health, and change only what the evidence supports. Keep the symptom matrix and worksheet with your runbook so the next engineer does not have to reconstruct the same questions under pressure.
If the capture points to an intermediate firewall rather than the monitor itself, continue with Palo Alto NAT troubleshooting. For adjacent topology and operations guides, use the Data Center hub and Start Here index.
Post a Comment