Declare the requirements of a call

Telecom least-cost routing selects among routes that meet the requirements of a call. XSI Compass applies that idea to model inference: establish eligibility first, then compare cost among the available candidates.

A request can specify tool support, modality, context size, latency and cost limits, vendor restrictions or a pinned model. The registry supplies the model profiles and deployment information used to evaluate those requirements.

A new model becomes useful to the router after it is registered and evaluated for the relevant work. A lower price alone does not establish that it can replace the selected model.

A practical cost example

When eligible calls cost less

Select diagram to enlarge
Using the article’s stated rates and traffic, moving 70% of calls to an eligible lower-cost route reduces the illustrative monthly inference total from $225,000 to $99,000.

Assume a fleet makes ten million calls per month, each with 2,000 input tokens and 500 output tokens. For an illustrative premium route priced at $5 per million input tokens and $25 per million output tokens, each call costs $0.0225. Using that route for every call costs $225,000 per month.

Now assume 70% of those calls qualify for another route priced at $1 input and $5 output per million tokens. Those seven million calls cost $31,500. The remaining three million premium calls cost $67,500. Total inference becomes $99,000, a $126,000 or 56% reduction under these assumptions.

These illustrative savings depend on 70% of calls meeting the workload’s evaluation criteria on the lower-cost route. Tool charges, retries and additional review calls are excluded.

Requirements before cost
A cheaper model is useful only when it meets the call’s requirements and workload evaluation criteria.

Eligibility and outcome are different checks

A model profile can establish supported capabilities and measured behavior under particular conditions. It cannot guarantee the quality or latency of every future response.

When no eligible route is available, the request needs a defined failure or escalation path. A cost limit should not silently remove a required capability, and a policy exclusion should not be bypassed to obtain an answer.

For selected consequential calls, Compass’s adversarial-review approach can compare a primary answer with a challenge. Disagreement becomes evidence for review. Agreement still does not authorize an external action or prove that an answer is correct.

Record the routing decision

The operational record can identify the selected model, applicable requirements, relevant profile versions and excluded alternatives. Access to that record remains governed; prompts and completions do not have to be published to explain the route.

Observed outcomes help update the profiles used for future selection. The same record can connect a changing invoice to traffic volume, token sizes, retries or a shift toward more expensive capabilities.

Compass’s purpose is to make that selection explicit and inspectable. Its economic benefit has to be measured on the fleet that uses it.