Build a Read-Only Network Change Reviewer with Claude Fable 5.2
This is a design article: a reference architecture you can build, not a writeup of a shipped product. Model behaviour and prices are from Anthropic's documentation, checked 12 November 2026. Disclosure: drafted with AI assistance and reviewed against the linked sources.
Network changes fail in reviewable ways. The permit that lands behind an earlier deny, the prefix-list edit that silently stops matching a customer block, the interface that ends up in the default VRF - most of it is visible in the diff, the ticket and the topology, if someone has the time to read all three carefully. That is exactly the kind of reading a large language model is good at, and exactly the kind of system you should not give write access to.
This article designs a change reviewer around that tension: a worker that can read everything and touch nothing. Claude Fable 5.2 is the model we size it for, because long multi-file correlation is its stated specialty, but the architecture is model-agnostic and the cheaper models are the right starting point.[1]
- The reviewer is read-only by construction: no device credentials, no deploy tokens, egress limited to the model API. Prompts are instructions, not security controls.
- Deterministic tools check what is mechanical (syntax, shadowing, IPAM); the model correlates diff, intent and topology and explains risk.
- The report is a JSON contract: every finding carries evidence pointing into the bundle, and the schema has no "approved" value.
- Trust is earned with a seeded-fault evaluation set, scoring recall, precision and evidence accuracy separately.
- Token cost per review is cents; the real cost driver is engineer time spent on wrong or noisy reports.
Define the contract before connecting anything · Trust boundaries and least privilege · Deterministic checks beside the model, not inside it · The report contract · What a Fable 5.2 wrapper must handle · Effort: the knob that replaces model-switching · The evaluation set: seeded faults, scored three ways · What a review costs · The whole workflow, end to end
1. Define the contract before connecting anything
Write the reviewer's job description first, as three lists:
- Input: a frozen change bundle - config diff, the ticket's intent text, a topology snapshot, the relevant policy/standards documents, and the rollback plan. The bundle is hashed; the review refers to that hash.
- Output: one structured report. Findings with severity and evidence, open questions, and an explicit list of what was not reviewed.
- Non-goals: the reviewer does not approve, does not deploy, does not log in to devices, and does not fetch anything beyond the bundle.
The decisive property is that the reviewer sits in a different trust zone than deployment. If the same pipeline identity can both review and push, a model failure can become an outage. If the reviewer's zone has no route and no credential to any device, the worst a bad review can do is waste reading time.
2. Trust boundaries and least privilege
Concretely:
- The review worker runs in an ephemeral container with no SSH client and no routes to the management network.
- Git, IPAM and the ticket system are reachable with read-only tokens; the only write scope is "comment on the ticket".
- Network egress is an allow-list with one entry: the model API endpoint.
- The audit log records the bundle hash, model ID, effort level, prompt version and token usage for every run - enough to reproduce or dispute any review later.
Note what is absent from this list: any instruction to the model about what it is "allowed" to do. Those lines can exist in the prompt for clarity, but the guarantee comes from the container, the tokens and the firewall. A reviewer that is read-only only because the prompt says so is not read-only.
3. Deterministic checks beside the model, not inside it
A language model should never be asked a question that a parser can answer. The pipeline runs the mechanical checks first and hands the model their results as part of the bundle:
This split has two benefits. The obvious one is reliability: ACL shadowing analysis from a rule engine is a fact, not an opinion. The subtler one is that it changes what you are grading the model on. With syntax and arithmetic off its plate, the model's job narrows to the things only it can do: does the diff match the stated intent, what is missing from the rollback plan, which neighbouring services could be affected. Those are the questions where its findings are worth an engineer's attention.
4. The report contract
Fix the output schema before you write a single prompt. It is what makes the reviewer auditable:
Five rules do most of the work:
- Evidence or silence. Every finding must point at a file and line range in the bundle, or at a named deterministic check result. The pipeline rejects a report citing a line that does not exist - a cheap, fully mechanical guard against invented findings.
- No approval verb. The
verdictenum containsneeds_changes,findings_reported,nothing_found- neverapproved. Evennothing_foundis phrased as a review result, not a go-ahead. - Say what was skipped. A reviewer that silently ignored the QoS policy because the baseline was missing is worse than one that lists it under
not_reviewed. - Stable finding IDs across re-runs, so a reviewer can see that F3 from yesterday is gone after the fix.
- Machine-validated. The pipeline validates the JSON against the schema before anything reaches the ticket. A malformed report is a failed run, not a published comment.
5. What a Fable 5.2 wrapper must handle
If your harness was written for an earlier Claude model, four documented behaviours of Claude Fable 5.2 can break it, and two more need handling:[2]
- Append-only history. A request that edits the system prompt, tools or earlier messages after a thinking block fails the prefix check with a 400 error. Build the conversation once, append only.
- Adaptive thinking is always on.
thinking: enabledwith a token budget, ordisabled, both return 400. Control depth with theeffortparameter instead.[2][3] - No forced tool use.
tool_choiceof typeanyortoolreturns 400. For a reviewer this is no loss - you validate the output schema anyway. - Omit sampling parameters. Leave out
temperature,top_p,top_k.[4] - Refusals are in-band. A declined request returns HTTP 200 with
stop_reason: "refusal"and a category. A reviewer harness should map that to "review not performed", never to "no findings".[2] - Data retention. Fable 5.2 requires 30-day retention and is not available under zero data retention without express authorisation. Decide before you send production configs whether that is acceptable, and start with synthetic bundles regardless.[4]
6. Effort: the knob that replaces model-switching
On current Claude models the effort parameter sets how much the model thinks and how many tokens it burns per turn; the API default on Fable 5.2 is high.[3][4] Anthropic's own guidance is that most of the capability gain is already present at high, with xhigh reserved for the hardest long-running work and max for measuring ceilings.[2]
For a reviewer: run the evaluation set at medium, high and xhigh, and keep the cheapest level whose recall matches the best one. Anthropic also documents that early stops - a turn ending with a to-do list instead of the finished job - are more common at low effort levels, which for a reviewer shows up as half-reviewed bundles.[5]
7. The evaluation set: seeded faults, scored three ways
Do not trust a reviewer because its reports read well. Build an evaluation set of synthetic change bundles where you know the ground truth:
Score three numbers separately, because they fail independently:
- Recall - of the seeded faults, how many did it find?
- Precision - of its findings, how many are real? The clean bundle exists to measure this; a reviewer that invents a finding on a correct change trains engineers to ignore it.
- Evidence accuracy - do the cited lines exist, and do they support the claim? This is the LLM-specific failure mode, and the one that silently destroys trust.
Rerun the set on every prompt change, model change and effort change. It is the regression suite of the reviewer. We proposed the same evidence-first scoring for public demo claims in our review of Fable 5.2 project evidence - the principle is identical.
8. What a review costs
Assume a generous bundle: 60,000 input tokens (diff, ticket, topology, policies, check results) and 6,000 output tokens of report. At list prices that is about $0.09 on Sonnet 5.5, $0.34 on Opus 5.5 and $0.72 on Fable 5.2 - and less when the stable part of the prompt (standards, policy, instructions) is served from the prompt cache, which costs 2.5% of the input price on Fable 5.2 and 5% on Opus and Sonnet.[6]
The numbers make the economics blunt: tokens are not the cost. A single engineer-hour spent chasing a hallucinated finding costs more than a thousand reviews' worth of Sonnet tokens. Optimise accuracy first, price last - and per Anthropic's model guidance, start the evaluation on Opus 5.5 and escalate to Fable 5.2 only if the evals demand it.[1]
9. The whole workflow, end to end
The sequence is deliberately boring: engineer opens the change, pipeline freezes and hashes the bundle, deterministic checks run, the model reviews, the pipeline validates the report schema and evidence references, the approver reads the report and decides, the engineer fixes or deploys. The model is one box in the middle with no arrows going anywhere dangerous.
Measure the whole loop, not the model: time from change opened to decision, how often the report changed the decision, how many findings were accepted versus dismissed. Those are the numbers that tell you whether the reviewer deserves to stay - and they are also the honest answer to "did AI help here?", which no demo video can give you. For the broader model-selection question, see our November 2026 ChatGPT vs Claude comparison; for more project patterns like this one, six useful Fable 5.2 projects.
Frequently asked questions
Can an AI safely review network configuration changes?
Yes, if the reviewer is architecturally read-only: it receives a frozen change bundle, holds no device credentials, and its output is a report that a human reads before approving. The safety comes from the environment, not from prompt wording.
Should the AI reviewer be allowed to approve changes?
No. The report schema should not even contain a value that means "approved". The model reports findings with evidence; a human makes the deployment decision in the change system.
Which Claude model is best for network change review?
Start with Claude Opus 5.5, which Anthropic recommends as the starting point for most workloads, and move to Claude Fable 5.2 only if your seeded-fault evaluation set shows it misses findings at higher effort. Token cost per review is under a dollar on either.
How much does one AI change review cost?
With a 60,000-token bundle and 6,000 output tokens, list prices give roughly 9 cents on Claude Sonnet 5.5, 34 cents on Opus 5.5 and 72 cents on Fable 5.2, less with prompt caching. Engineer time reading a noisy report costs far more than the tokens.
How do you test an LLM change reviewer before trusting it?
Build synthetic change bundles with seeded faults - ACL shadowing, a route leak, a wrong VRF, an MTU mismatch - plus clean bundles, and score three numbers separately: seeded faults found, findings that are real, and whether cited evidence exists.
Sources
Primary sources, checked 12 November 2026.
Related reading
- Claude Fable 5.2: Six Useful Projects Beyond the Viral Demos
- Can an LLM Pick the Right EIGRP Feasible Successor? DUAL's Strict-Inequality Trap
- Claude Fable 5.2 in the Wild: What the Public Project Evidence Shows
- GPT-6 Cyber and Prompt Injection: How Untrusted Text Reaches Agent Tools
- Hugging Face AI-Agent Intrusion: What the Public Logs Show—and Why It Wasn’t GPT-6 Cyber
- Prisma SD-WAN Branch Migration Checklist: Circuits, Cutover and Rollback
Comments
Post a Comment