The cost of carrying a conversation
An agent can spend much of its input budget on work it has already done. Requirements, earlier answers, corrections and decisions accumulate in the conversation. With full-transcript replay, that growing history goes back to the model on every turn. The next answer starts with another charge for processing the past.
XSI Reckoning carries the current state of the work into the next reasoning step. Within the Memory Continuum, the application retains history and evidence while the active context holds the commitments and information needed now. An old requirement can remain in the record without continuing to govern the task after a correction.
In XSI's controlled research comparison, the difference reached approximately forty to one at turn 50. Full-transcript replay used 11,450 prompt tokens; bounded memory used 286. At an equal input-token price, traditional transcript replay cost approximately 40× as much for that turn's input. The reduction was 97.5%.
The comparison used one 50-turn conversation, repeated three times with prefix caching disabled. All three repetitions produced those turn-50 counts. The measurement covers the input side of inference at that point in the conversation; generated output and tool use sit outside that comparison.
XSI also ran separate long-horizon tests extending beyond 5,000 turns. In those runs, XSI observed no catastrophic loss of performance or memory and no increase in reasoning-token cost.
| Full-transcript replay | 11,450 prompt tokens |
|---|---|
| Bounded memory | 286 prompt tokens |
From tokens to dollars
For instance, assume a frontier inference price of $5 per million uncached input tokens. At that rate, 11,450 input tokens cost $0.05725. The 286-token input costs $0.00143. The difference is a little over five and a half cents for one reasoning step.
Across 100,000 steps with those respective input sizes, the arithmetic becomes $5,725 versus $143: $5,582 less spent on input. This is an illustrative volume calculation using the observed turn-50 sizes, rather than a measured total for 100,000 tasks. Output charges, cache discounts and tool fees would be accounted for separately.
The rate can change without changing the relationship. At the same price per input token, forty times the input carries forty times the input cost. The absolute saving depends on the workload, the amount of history otherwise replayed and the price paid for processing it.
Tools add another layer of token cost
A reasoning step may lead to a database query, a document search or code execution. The tool can return far more material than the agent needs for its next decision: thousands of rows, a complete document or a lengthy execution log. Placing that material in the conversation adds to the model's input.
When subsequent turns replay the result, the payload is charged again. Additional calls add their own definitions, arguments, responses and control messages. Reducing the conversational history addresses one part of inference cost. Tool interactions create another source of growth, especially when large results remain in context through many later steps.
XSI is building new tools and a tool-acceleration framework to sharply reduce that overhead. The framework routes tool results into working state and returns a compact governed reference to the agent. The reference identifies the result and carries the metadata needed to request further operations under the relevant permissions.
A result can stay in working state
Consider an agent reviewing purchase orders across several operating locations. A database query returns a large set of orders. The agent needs to identify late deliveries, compare affected suppliers and prepare an exception report. Repeatedly sending every returned row through the model makes each stage carry the entire query result.
In XSI's developing framework, the query result stays in working state. The agent receives a reference, a description of the result and the permitted ways to work with it. It can ask a tool to filter for overdue orders, calculate totals or retrieve the specific records needed for the report.
The model still needs the information on which a judgment depends. A reference provides access; it does not convey the contents of an unseen table. Selected rows, aggregates or passages enter context when required. The larger result remains available for further inspection and operations without being copied into every prompt.
Authority and provenance accompany this flow. The tooling layer checks the requested operation, the result receives an identity, and subsequent access is checked against the applicable scope. The proposed framework also records the relationship between the operation and its result so a later review can follow what happened.
The same pattern applies to a document search. The search result can stay available as a collection while the agent requests particular passages and their citations. A later step can inspect another passage without first carrying every document through the intervening conversation.
A practical tool-cost example
Suppose that purchase-order result occupies 100,000 tokens and is replayed in ten subsequent model inputs. The repeated payload alone contributes one million input tokens. At the illustrative $5 rate, that is $5 of input cost for carrying one result through those ten steps.
Now suppose each step carries a 200-token reference and its metadata, while two steps each retrieve 1,000 tokens of selected information. That contributes 2,000 reference tokens plus 2,000 retrieved tokens: 4,000 input tokens, costing $0.02. Across 1,000 such workflows, the corresponding payload-input costs would be $5,000 and $20.
Those assumed sizes illustrate the economics of the proposed design; they are not a measured tooling result. The calculation covers the carried payload, references and selected retrievals. Query execution, storage, tool definitions, other messages and generated output add their own costs. If a task needs more of the result in context, its retrieval cost rises accordingly.
Memory and tool acceleration
Reckoning maintains current commitments, including corrections and revocations. Atlas provides durable custody of evidence. Plotter maintains execution continuity across interruptions, retries and handoffs. Together, these responsibilities let an application retain the record, keep track of where the work stands and supply context for the next operation.
XSI's tool framework extends that approach to the material produced during execution. Working state holds a result while it is being used. Governed references connect later operations to it. Selected information reaches the agent as the task requires, and durable evidence can remain available through Atlas after the immediate operation ends.
XSI adopted published methods, science and mathematics, then adapted and tuned them in a house blend of Go tools with CUDA acceleration. That implementation work follows the earlier prompt-token research. Reckoning reduces the history carried into reasoning; the developing tool framework targets the volume of tool material carried forward as the work continues.
