Count the workload first
An agent can make several model calls while completing one customer request. Each call can include the prompt, retrieved documents, prior tool results and conversation history. Retries, review calls and generated output add to that workload.
Cloud inference bills commonly separate input and output tokens. There is no universal output-to-input ratio for an agent: a document review can consume a large input and produce a short answer, while drafting can produce much more output. Count both from the workflow you intend to run.
For a planning example, assume 30,000 model calls per month, each with 3,000 input tokens and 1,000 output tokens. That is 90 million input tokens and 30 million output tokens per month.
A worked inference bill
For instance, use illustrative rates of $3 per million input tokens and $15 per million output tokens. The monthly input charge is 90 × $3 = $270. The output charge is 30 × $15 = $450. Total model inference is $720 per month.
At three times that workload, with the same token sizes and rates, the inference charge becomes $2,160. A second full review call for every request would double the modeled inference volume. Selected review calls add only their own usage.
These are assumptions for comparison, not current vendor quotes or an XSI benchmark. Tool fees, storage, network charges and application operations are additional. Caching and batch discounts apply only when the workflow qualifies for the provider’s terms.
Local inference has a different cost structure
A local model can avoid a provider’s per-token inference charge. The work still consumes electricity, accelerator time, memory and storage. Hardware capacity also limits concurrent requests and response time.
Assume a local system costs $6,000 to acquire and commission, with a three-year planning life. Straight-line hardware allocation is $2,000 per year. If power costs another $600 per year, those two items total $2,600 annually, or about $217 per month. Software subscriptions, support, staff time, repairs and additional capacity remain outside that subtotal.
Compare systems that can complete the same tasks at acceptable quality, latency and concurrency. A small local model and a frontier cloud model do not become equivalent because their invoices can be compared.
Calculate payback separately
Hardware allocation and cash payback answer different questions. Under the example above, replacing a $720 monthly inference bill with $50 monthly power cost leaves $670 per month before other operating costs. A $6,000 initial purchase would take about nine months to recover on that simplified basis.
Add $300 per month of incremental local software, support or staffing and the difference becomes $370 per month. The same purchase then takes about sixteen months to recover. If local capacity cannot meet the workload, additional hardware changes the calculation again.
Allocation and payback differ
Select diagram to enlargeChoose the deployment around the work
Cloud services can provide capacity for bursts and access to models that exceed local hardware limits. Local deployment gives the operator control over model versions, scheduling and the inference environment. Both require queues, monitoring and recovery paths.
XSI LodeStone’s local deployment approach addresses the workloads operators want to keep under their own control. The economic case comes from measured utilization and the operating model: which work stays local, which work needs a larger model, and what the complete deployment costs to run.
