An allowed Palo Alto session ending with aged-out is not, by itself, proof that the firewall blocked the connection. Palo Alto defines allow as a session allowed by policy, while aged-out describes why a session ended.[3] The useful question is: did the application complete its transaction, and if not, where did the expected packet disappear?
This guide provides a reusable troubleshooting matrix, a NAT-aware capture worksheet and an acceptance checklist. It focuses on traffic passing through a PAN-OS firewall managed directly or through Panorama, not Prisma SD-WAN ION diagnostics. The workflow and hypothetical examples are recommendations, not results from a production incident or an executed lab.
Quick answer: is aged-out normal or a fault?
Palo Alto's knowledge base describes aging out as a normal way for UDP traffic such as DNS to end; for TCP, it can also occur when the handshake never completes and the session times out.[1] Therefore, distinguish a successful transaction followed by inactivity from a connection that never became usable.
| What you observed | First interpretation to test | Next evidence to collect | What not to conclude |
|---|---|---|---|
| DNS query succeeds, response is received, UDP session later ages out | Normal session expiration is plausible | Matching query/response and endpoint result | Every aged-out entry requires remediation |
| TCP connect fails; only client-to-server packets appear | No server response observed in this firewall session | SYN path, server listener, return route and reverse capture filter | The firewall must be dropping the SYN |
| Server capture shows a reply, but firewall capture does not | Return path or capture visibility may differ | NAT tuple, actual next hop, capture point and offload conditions | Missing capture data proves an on-wire drop |
| TCP transaction succeeds; idle reuse later fails | Idle-state lifetime mismatch is worth investigating | Idle interval, applicable timeout, endpoint keepalive and intermediate devices | Increase all firewall timeouts immediately |
| Packets are present in both directions, but application fails | Transport visibility alone does not establish application success | Full handshake, request/response and application logs | Nonzero receive counters mean the service is healthy |
Matrix: proposed diagnostic decisions. The UDP/TCP expiration distinction comes from the vendor knowledge base; the investigation branches are an original operational checklist.[1]
Read the traffic log before changing policy
The PAN-OS traffic log exposes original and translated addresses and ports, rule name, application, interfaces, session ID, action and session-end reason.[3] Its sent/received packet fields are directional: sent means client-to-server and received means server-to-client.[3] Do not interpret “received” as “received on the physical interface I am looking at.”
Build one evidence record for one failed attempt:
- Record the test time with timezone, firewall identity and virtual system privately in the change ticket.
- Save the original source/destination, protocol and ports, plus translated addresses and ports where applicable.
- Record the matched rule, ingress/egress interfaces, application label, action and end reason.
- Record directional packet/byte counts and the session start and elapsed times.
- Record the endpoint result separately: DNS answer, TCP connection, TLS handshake or actual application operation.
These proposed worksheet fields are drawn from the documented traffic-log schema, with the endpoint outcome added as an independent acceptance measure.[3]
Correlate precisely. Avoid combining the log from one client retry with a packet capture of another. In your private worksheet, keep the timestamp, protocol and full tuple together; retain session ID as another correlation field, not as a globally unique incident identifier.
The vendor documentation also notes that, if session termination has multiple causes, the end-reason field displays only the highest-priority reason.[3] Treat a single field as a summary, not a complete packet timeline. If the question is “was it denied?”, inspect the action, matching rule and associated evidence rather than translating aged-out into deny.
Troubleshooting workflow: find the first missing packet
1. Define an application-level pass condition
Before capturing anything, write down what success means. “The DNS client receives the expected answer from the intended resolver” is better than “UDP/53 is allowed.” “The application completes a read-only health request” is better than “TCP/443 is open.”
Use one authorized client and one intended service. Start with a fresh application connection so the capture window contains the setup. Do not clear every firewall session merely to simplify a test; coordinate any narrowly scoped session intervention under your normal change procedure.
2. Separate forward-path failure from missing return traffic
For a failed TCP connection, compare the client attempt with the firewall and server observations. The vendor's packet-capture example demonstrates the distinction between a request that gets no server response and a successful exchange that includes the three-way handshake and data.[2]
Use this proposed sequence:
- Does the client actually attempt the expected destination and port?
- Is that attempt present at the firewall's relevant receive capture point?
- Does the corresponding packet appear in the transmit capture with the expected addressing?
- Does the intended server receive the attempt?
- Does the server emit a response, and to which destination tuple?
- Does the response return through the expected firewall path and reach the client?
Stop at the first boundary where the evidence changes. A server that never received the request needs a different investigation from a server that sent a response through another gateway. This workflow localizes a hypothesis; it does not make any single capture point infallible.
3. Capture the translated return path, not just the client address
Palo Alto's custom-capture procedure explicitly calls for two filters in its source-NAT example: the original source toward the destination, and the server toward the translated source address for the reply.[2] A client-only filter can therefore leave your reverse-path investigation incomplete.
Hypothetical worksheet: the symbolic values below are placeholders, not customer addresses or copy-and-paste firewall syntax. This example is source NAT only; if destination NAT also applies, document that mapping separately before building filters.
| Worksheet item | Forward direction | Return direction |
|---|---|---|
| Endpoint identity | CLIENT to SERVER | SERVER to CLIENT after reverse translation |
| Tuple to identify at firewall ingress | CLIENT:CLIENT_PORT to SERVER:SERVICE_PORT | SERVER:SERVICE_PORT to SNAT_ADDRESS:TRANSLATED_PORT |
| Protocol | Chosen test protocol | Same test protocol |
| Supporting evidence | Original tuple plus translated tuple from session/log | Server capture or next-hop evidence for the reply |
| Capture objective | Find request entering and leaving | Find reply returning and reaching client |
Do not guess that source ports remain unchanged. Copy the actual translated port from the relevant session/log evidence, or initially use appropriately narrow address/protocol filters without over-restricting the source port. Keep the scope small enough to remain safe.
4. Run a bounded custom capture
The documented PAN-OS workflow uses Monitor → Packet Capture, capture filters with Filtering On, and the receive, firewall, transmit and drop stages as needed.[2] The vendor warns that capture can degrade system performance and instructs operators to turn it off after collecting the required data.[2]
Recommended operational sequence:
- Coordinate ownership of existing capture settings before clearing or replacing them.
- Define the forward and reverse filters from the worksheet; verify that Filtering is on.
- Select only the needed stages, use distinct filenames and apply available packet/byte limits.
- Start a short approved capture window and reproduce one controlled transaction.
- Turn capture off immediately, refresh the file list and download the evidence securely.
- Confirm capture is off, remove temporary settings as agreed and record any cleanup still required.
Hardware offload can affect capture completeness; Palo Alto notes that disabling it may be needed to capture all traffic.[2] Do not disable hardware offload globally as an automatic first step. Check the exact platform/release procedure, assess performance risk and use your change process or vendor support. An empty capture is inconclusive until filters, timing, path and capture visibility have been checked.
Packet captures may contain credentials, payloads and personal data. Use authorized test traffic, restricted storage and your retention policy. Do not upload production captures to public troubleshooting tools or publish them with a blog comment.
When should you investigate a timeout change?
Only after proving that the connection worked before becoming idle. The vendor's definition establishes that TCP sessions can time out, but the log label alone does not identify which timer or device caused the user-visible failure.[1]
For an idle-connection hypothesis, record these as measurements to collect, not assumed defaults:
| Measurement | Why it belongs in the ticket |
|---|---|
| Last successful application exchange | Establishes that setup and data transfer worked |
| First failed reuse after inactivity | Establishes the user-visible failure point |
| Applicable firewall session timeout | Lets you compare the configured behavior, not a remembered default |
| Application keepalive/reconnect behavior | Helps distinguish intended reconnect from unwanted failure |
| Load balancer, proxy or server idle settings | Prevents attributing another device's timeout to the firewall |
| Result after a scoped change or application fix | Tests the specific hypothesis |
Prefer a controlled comparison with one changed variable. Consider whether the application should reconnect cleanly or send an appropriate keepalive before proposing longer network state retention. If a timeout adjustment is justified, document its scope, owner, rollback and expected resource impact. Do not treat a broad timeout increase as a substitute for fixing a listener, NAT mapping or return route.
Acceptance checklist: copy into the incident or change ticket
This is a proposed acceptance asset, not a claim that these tests have been run.
| Test | Pass condition | Evidence to attach |
|---|---|---|
| Fresh intended connection | Expected application transaction completes | Client result and matched session/log |
| Bidirectional packet path | Expected request and response observed at the necessary boundaries | Timestamped capture notes and tuples |
| Source-NAT return path, if used | Response targets the actual translated tuple and reaches the client | Session mapping plus server/firewall observations |
| Idle reuse, if relevant | Application resumes or reconnects as designed after the agreed idle interval | Recorded interval and application outcome |
| Negative control | One approved disallowed test remains blocked | Matching policy/log evidence; no broad scanning |
| Regression check | A known-good related service still works | Before/after application result |
| Cleanup | Capture stopped; temporary changes removed or formally retained | Operator confirmation and final change diff |
Do not make “zero aged-out logs” the acceptance criterion. Successful UDP exchanges can still age out normally.[1] Choose criteria that demonstrate the intended service works while security boundaries remain intact.
Common questions
Does allow plus aged-out mean Palo Alto dropped the connection?
No. allow says the session was allowed by policy; aged-out says the session aged out.[3] Establish the cause of a failed transaction from the packet path and endpoint evidence, not from that combination alone.
Does a missing drop-stage file prove the firewall is innocent?
No. Palo Alto's worked capture example produces no drop file because no dropped packets were captured in that example.[2] For your investigation, first validate the capture's scope and visibility, then correlate it with other observations. Absence of a file is not an end-to-end delivery guarantee.
Should I increase the TCP timeout when the handshake never completes?
Not as the initial remedy. The vendor explicitly identifies an unestablished handshake as one situation that can time out.[1] Investigate the request, listener and reply path first; a longer wait does not demonstrate that those paths work.
Practical takeaways and related guides
Start with the application outcome, preserve the original and translated tuples, and find the first missing packet. Use the matrix to choose the next observation, the worksheet to scope the capture, and the acceptance table to prove the fix. Change a timeout only when evidence supports that specific hypothesis.
For a tunnel-specific failure, continue with Palo Alto IPsec tunnel up but no traffic. If the failure is confined to inspected TLS applications, use the SSL decryption troubleshooting matrix. Browse Start Here and the networking resources hub for the broader troubleshooting library.
Sources
- [1] https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000PN02CAG&lang=en_US
- [2] https://docs.paloaltonetworks.com/ngfw/administration/monitoring/take-packet-captures/take-a-custom-packet-capture
- [3] https://docs.paloaltonetworks.com/pan-os/10-1/pan-os-admin/monitoring/use-syslog-for-monitoring/syslog-field-descriptions/traffic-log-fields
Post a Comment