A plausible answer can become an operational error
I have spent years building automation for ISP operators. In network operations, a plausible diagnosis can lead to a command on live equipment. An incorrect action can interrupt service even when the model’s answer looks confident.
Model error rates depend on the task, the evidence supplied and the evaluation method. A summarization result cannot be treated as the error rate for generating commands on a particular access platform.
The operational question is specific: does the proposed action match the current device state, the documented procedure and the authority granted to the agent?
Use multiple opinions to find disagreements
XSI LodeStone’s quorum approach compares sub-agent assessments before selected operational actions proceed. A second assessment can challenge assumptions, identify missing evidence or recommend escalation.
Models can share training material, prompts and failure modes. They may agree on the same wrong answer. Agreement is evidence to examine; it does not establish truth or grant permission to execute.
A review should expose the reason for a disagreement and the evidence behind each proposal. The next step may be another diagnostic query, a documented command check or an operator decision.
Ground the proposal in the equipment
For an OLT fault, the relevant sources include the vendor’s procedure, hardware model, firmware version and current telemetry. A curated skill can bind that evidence to an allowed set of operations.
A command still needs its prerequisites checked. The right syntax for the wrong firmware or the wrong device can be unsafe. Tool validation and scoped permissions provide checks outside the model’s answer.
The system should record the proposed action, the evidence used, the review outcome and the execution decision. Sensitive material should remain under the access controls that govern it.
Resolve the evidence gap
Select diagram to enlargeBudget for review work
Additional model calls consume input and output tokens, accelerator time and memory. Parallel calls can reduce waiting compared with sequential calls, but the slowest reviewer, shared hardware and comparison step still affect response time.
Local hosting can avoid per-token API charges. It does not make additional inference free: review calls consume finite capacity and power. The review policy can reserve additional inference for actions where it is useful, then measure the resulting cost and latency.
Compare observed failure rates and review outcomes on the intended workflow. A useful policy catches consequential errors without turning every routine request into an unnecessary multi-model exercise.
Keep execution authority separate
Passing a review does not replace the operator’s authorization. The execution layer must enforce the allowed action, target, scope and current policy. Service-affecting work can require a named human decision before it runs.
This separation lets the agent reason about a problem while the application controls what happens next. Disagreement, missing evidence and denied authority all have explicit paths to further investigation or operator review.
