Build a Read-Only Network Change Reviewer with Claude Fable 5.2

Cover image: building a read-only network change reviewer with Claude Fable 5.2

This is a design article: a reference architecture you can build, not a writeup of a shipped product. Model behaviour and prices are from Anthropic's documentation, checked 12 November 2026. Disclosure: drafted with AI assistance and reviewed against the linked sources.

Network changes fail in reviewable ways. The permit that lands behind an earlier deny, the prefix-list edit that silently stops matching a customer block, the interface that ends up in the default VRF - most of it is visible in the diff, the ticket and the topology, if someone has the time to read all three carefully. That is exactly the kind of reading a large language model is good at, and exactly the kind of system you should not give write access to.

This article designs a change reviewer around that tension: a worker that can read everything and touch nothing. Claude Fable 5.2 is the model we size it for, because long multi-file correlation is its stated specialty, but the architecture is model-agnostic and the cheaper models are the right starting point.[1]

Key takeaways
  • The reviewer is read-only by construction: no device credentials, no deploy tokens, egress limited to the model API. Prompts are instructions, not security controls.
  • Deterministic tools check what is mechanical (syntax, shadowing, IPAM); the model correlates diff, intent and topology and explains risk.
  • The report is a JSON contract: every finding carries evidence pointing into the bundle, and the schema has no "approved" value.
  • Trust is earned with a seeded-fault evaluation set, scoring recall, precision and evidence accuracy separately.
  • Token cost per review is cents; the real cost driver is engineer time spent on wrong or noisy reports.

1. Define the contract before connecting anything

Write the reviewer's job description first, as three lists:

  • Input: a frozen change bundle - config diff, the ticket's intent text, a topology snapshot, the relevant policy/standards documents, and the rollback plan. The bundle is hashed; the review refers to that hash.
  • Output: one structured report. Findings with severity and evidence, open questions, and an explicit list of what was not reviewed.
  • Non-goals: the reviewer does not approve, does not deploy, does not log in to devices, and does not fetch anything beyond the bundle.
Architecture diagram of a read-only AI network change reviewer: change bundle, deterministic checks, Claude model review, structured report, human approver, separate deployment zone
The reference architecture. Review and deployment are separate zones with separate identities.

The decisive property is that the reviewer sits in a different trust zone than deployment. If the same pipeline identity can both review and push, a model failure can become an outage. If the reviewer's zone has no route and no credential to any device, the worst a bad review can do is waste reading time.

2. Trust boundaries and least privilege

Trust boundary diagram for an AI change reviewer showing read-only sources of truth, an isolated reviewer sandbox without device credentials, and report outputs
What the reviewer can reach (read-only sources), what lives in its sandbox, and what must not be present there.

Concretely:

  • The review worker runs in an ephemeral container with no SSH client and no routes to the management network.
  • Git, IPAM and the ticket system are reachable with read-only tokens; the only write scope is "comment on the ticket".
  • Network egress is an allow-list with one entry: the model API endpoint.
  • The audit log records the bundle hash, model ID, effort level, prompt version and token usage for every run - enough to reproduce or dispute any review later.

Note what is absent from this list: any instruction to the model about what it is "allowed" to do. Those lines can exist in the prompt for clarity, but the guarantee comes from the container, the tokens and the firewall. A reviewer that is read-only only because the prompt says so is not read-only.

3. Deterministic checks beside the model, not inside it

A language model should never be asked a question that a parser can answer. The pipeline runs the mechanical checks first and hands the model their results as part of the bundle:

Table dividing network change review tasks between deterministic tools and a language model
Who checks what. If it can be evaluated mechanically, a tool does it; the model correlates and explains.

This split has two benefits. The obvious one is reliability: ACL shadowing analysis from a rule engine is a fact, not an opinion. The subtler one is that it changes what you are grading the model on. With syntax and arithmetic off its plate, the model's job narrows to the things only it can do: does the diff match the stated intent, what is missing from the rollback plan, which neighbouring services could be affected. Those are the questions where its findings are worth an engineer's attention.

4. The report contract

Fix the output schema before you write a single prompt. It is what makes the reviewer auditable:

Example JSON report schema for an AI network change reviewer with findings, evidence references, open questions and a not-reviewed list
An example report schema. Evidence or silence; no approval verb; "not_reviewed" is mandatory.

Five rules do most of the work:

  1. Evidence or silence. Every finding must point at a file and line range in the bundle, or at a named deterministic check result. The pipeline rejects a report citing a line that does not exist - a cheap, fully mechanical guard against invented findings.
  2. No approval verb. The verdict enum contains needs_changes, findings_reported, nothing_found - never approved. Even nothing_found is phrased as a review result, not a go-ahead.
  3. Say what was skipped. A reviewer that silently ignored the QoS policy because the baseline was missing is worse than one that lists it under not_reviewed.
  4. Stable finding IDs across re-runs, so a reviewer can see that F3 from yesterday is gone after the fix.
  5. Machine-validated. The pipeline validates the JSON against the schema before anything reaches the ticket. A malformed report is a failed run, not a published comment.

5. What a Fable 5.2 wrapper must handle

If your harness was written for an earlier Claude model, four documented behaviours of Claude Fable 5.2 can break it, and two more need handling:[2]

Checklist of Claude Fable 5.2 API breaking changes: prefix check, always-on adaptive thinking, no forced tool choice, sampling parameters, refusals, data retention
Six documented API behaviours to handle before any comparison or production run.
  • Append-only history. A request that edits the system prompt, tools or earlier messages after a thinking block fails the prefix check with a 400 error. Build the conversation once, append only.
  • Adaptive thinking is always on. thinking: enabled with a token budget, or disabled, both return 400. Control depth with the effort parameter instead.[2][3]
  • No forced tool use. tool_choice of type any or tool returns 400. For a reviewer this is no loss - you validate the output schema anyway.
  • Omit sampling parameters. Leave out temperature, top_p, top_k.[4]
  • Refusals are in-band. A declined request returns HTTP 200 with stop_reason: "refusal" and a category. A reviewer harness should map that to "review not performed", never to "no findings".[2]
  • Data retention. Fable 5.2 requires 30-day retention and is not available under zero data retention without express authorisation. Decide before you send production configs whether that is acceptable, and start with synthetic bundles regardless.[4]

6. Effort: the knob that replaces model-switching

On current Claude models the effort parameter sets how much the model thinks and how many tokens it burns per turn; the API default on Fable 5.2 is high.[3][4] Anthropic's own guidance is that most of the capability gain is already present at high, with xhigh reserved for the hardest long-running work and max for measuring ceilings.[2]

Diagram of Claude Fable 5.2 effort levels low, medium, high, xhigh and max with guidance on when to use each
The five effort levels, with Anthropic's own guidance on when each is appropriate.

For a reviewer: run the evaluation set at medium, high and xhigh, and keep the cheapest level whose recall matches the best one. Anthropic also documents that early stops - a turn ending with a to-do list instead of the finished job - are more common at low effort levels, which for a reviewer shows up as half-reviewed bundles.[5]

7. The evaluation set: seeded faults, scored three ways

Do not trust a reviewer because its reports read well. Build an evaluation set of synthetic change bundles where you know the ground truth:

Table of an evaluation set for an AI network change reviewer with seeded faults such as ACL shadowing, route leak, wrong VRF and MTU mismatch
Eight starter bundles. Clean bundles matter as much as faulty ones.

Score three numbers separately, because they fail independently:

  • Recall - of the seeded faults, how many did it find?
  • Precision - of its findings, how many are real? The clean bundle exists to measure this; a reviewer that invents a finding on a correct change trains engineers to ignore it.
  • Evidence accuracy - do the cited lines exist, and do they support the claim? This is the LLM-specific failure mode, and the one that silently destroys trust.

Rerun the set on every prompt change, model change and effort change. It is the regression suite of the reviewer. We proposed the same evidence-first scoring for public demo claims in our review of Fable 5.2 project evidence - the principle is identical.

8. What a review costs

Assume a generous bundle: 60,000 input tokens (diff, ticket, topology, policies, check results) and 6,000 output tokens of report. At list prices that is about $0.09 on Sonnet 5.5, $0.34 on Opus 5.5 and $0.72 on Fable 5.2 - and less when the stable part of the prompt (standards, policy, instructions) is served from the prompt cache, which costs 2.5% of the input price on Fable 5.2 and 5% on Opus and Sonnet.[6]

Bar chart of the estimated token cost of one AI network change review on Claude Haiku 5.5, Sonnet 5.5, Opus 5.5 and Fable 5.2 with and without prompt caching
Estimated token cost of one review, by model, with and without caching. Arithmetic from list prices, not measured usage.

The numbers make the economics blunt: tokens are not the cost. A single engineer-hour spent chasing a hallucinated finding costs more than a thousand reviews' worth of Sonnet tokens. Optimise accuracy first, price last - and per Anthropic's model guidance, start the evaluation on Opus 5.5 and escalate to Fable 5.2 only if the evals demand it.[1]

9. The whole workflow, end to end

Swimlane sequence diagram of an AI-assisted network change review from engineer to pipeline, deterministic checks, model review, report validation and human approver
One change from engineer to decision. The model appears once; everything else is ordinary pipeline code and a human.

The sequence is deliberately boring: engineer opens the change, pipeline freezes and hashes the bundle, deterministic checks run, the model reviews, the pipeline validates the report schema and evidence references, the approver reads the report and decides, the engineer fixes or deploys. The model is one box in the middle with no arrows going anywhere dangerous.

Measure the whole loop, not the model: time from change opened to decision, how often the report changed the decision, how many findings were accepted versus dismissed. Those are the numbers that tell you whether the reviewer deserves to stay - and they are also the honest answer to "did AI help here?", which no demo video can give you. For the broader model-selection question, see our November 2026 ChatGPT vs Claude comparison; for more project patterns like this one, six useful Fable 5.2 projects.

Frequently asked questions

Can an AI safely review network configuration changes?

Yes, if the reviewer is architecturally read-only: it receives a frozen change bundle, holds no device credentials, and its output is a report that a human reads before approving. The safety comes from the environment, not from prompt wording.

Should the AI reviewer be allowed to approve changes?

No. The report schema should not even contain a value that means "approved". The model reports findings with evidence; a human makes the deployment decision in the change system.

Which Claude model is best for network change review?

Start with Claude Opus 5.5, which Anthropic recommends as the starting point for most workloads, and move to Claude Fable 5.2 only if your seeded-fault evaluation set shows it misses findings at higher effort. Token cost per review is under a dollar on either.

How much does one AI change review cost?

With a 60,000-token bundle and 6,000 output tokens, list prices give roughly 9 cents on Claude Sonnet 5.5, 34 cents on Opus 5.5 and 72 cents on Fable 5.2, less with prompt caching. Engineer time reading a noisy report costs far more than the tokens.

How do you test an LLM change reviewer before trusting it?

Build synthetic change bundles with seeded faults - ACL shadowing, a route leak, a wrong VRF, an MTU mismatch - plus clean bundles, and score three numbers separately: seeded faults found, findings that are real, and whether cited evidence exists.

Sources

Primary sources, checked 12 November 2026.

  1. Claude models overview (Anthropic docs)
  2. What's new in Claude Fable 5.2 (Anthropic docs)
  3. Effort parameter (Anthropic docs)
  4. Claude Fable 5.2 model overview (Anthropic docs)
  5. Prompting Claude Fable 5.2 (Anthropic docs)
  6. Claude API pricing (Anthropic docs)

Comments

Popular posts from this blog

BGP Neighbor States

Reason legend in show ip ospf statistics