Prompt caching cut a tool-use agent’s input cost by about 80%

A tool-use agent resends its whole context on every turn. What prompt caching changes about that, and how I read a cut of about 80% in input cost.

CRT line drawing: a seven-turn helix strung with beads

A tool-use agent pays for its context on every turn. Each request in the loop carries the system prompt, the tool definitions and the whole conversation so far. Input is billed per token, on every request. So the agent keeps paying for text it has already sent, and that part grows with each tool call and each result. Prompt caching exists for that repeated part.

YieldMap has a card in the R&D Labs section of my site, with the badge AI_Research. The card is three sentences long:

Real-world-asset yields compared for retail investors. A hand-written Claude tool-use agent researches each product with five tested tools, and its output waits in a queue for human approval. Prompt caching cut its input cost by about 80%.

YieldMap’s spec describes a platform that aggregates yield, TVL, risk and on-chain data from free-tier APIs and processes it with AI. The problem it names is fragmented data: users visit each protocol separately to compare APY, risk and liquidity.

My stack list makes the same pairing as the card. Under AI & Agents, the Anthropic API entry carries two notes in brackets: tool use and caching. This note is about the card’s last sentence and the two ideas under it: what a tool-use loop sends on every turn, and what prompt caching changes about that.

What a tool-use loop sends

In a tool-use loop, the request lists the tools the model may call. Each tool has a name, a description and a schema for its input. The model replies with an answer or with a call to one of those tools. The program runs the tool, adds the result to the conversation and sends the next request. The loop ends when the model stops calling tools.

In the usual setup the model keeps nothing between requests, so every request carries everything again. Here is a sketch of three turns:

request 1:  system | tools | task
request 2:  system | tools | task | call 1 | result 1
request 3:  system | tools | task | call 1 | result 1 | call 2 | result 2
A sketch, not real code: what three consecutive requests in a generic tool-use loop carry.

Two things stand out. The front of every request is the same: the same system prompt, the same tool definitions, the same task. And the conversation grows only at the end, so each request repeats all of the one before it and adds one call and one result. A tool result can be much longer than the call that asked for it. The input bill pays for all of that repetition, turn after turn.

What prompt caching changes

Prompt caching keeps the processed form of a prompt’s opening part, its prefix, for a short time. When the next request starts with exactly the same tokens, that prefix is read from the cache instead of being processed again. Cached input is billed at a lower rate than fresh input. Caching doesn’t change how the model answers. What changes is the price of the repeated input and the time spent processing it.

The match is exact, and it runs from the start. If one token changes early in the prompt, everything after it is processed fresh. A timestamp in the system prompt does that, and so does a tool list in a different order. So a cache-friendly request runs from the most stable part to the least: system prompt and tool definitions first, then the conversation, newest last.

A tool-use loop already has that shape. Its stable parts sit at the front, and its history grows only at the end. The part that repeats is the part a cache can serve.

That is the background to the card’s last sentence: “Prompt caching cut its input cost by about 80%.” A few things help me read a figure like that:

  • It is a cut in input cost. Output tokens are billed separately, and caching doesn’t change them.
  • Caching doesn’t shrink the prompt. The same tokens are still sent, and the ones read from the cache are billed at the lower rate.
  • A cache has to be written before it can be read, and entries expire when nothing reads them for a while. The first request with a new prefix gets no discount.
  • The size of the cut depends on how much of each request repeats, and how many times.

Not paying twice, elsewhere in the plans

Prompt caching avoids paying twice for the same input.

  • A project rule says the Anthropic API is never called at request time. Under that rule, risk layers, FAQs and news summaries are computed ahead of time by workers and stored in the database. The server loaders read them as regular structured data.
  • In the plan, a product’s documents are parsed once, when the product is new, and parsed again when they change.
  • The planned news worker keeps only items that contain one of four terms, such as RWA or tokenization. It skips URLs already stored and starts one summary job per new article. Articles already summarised must not be summarised again.
  • The spec says every data source must be free tier at MVP. The plan’s scraping budget is a free tier of 500 credits a month, and credit use is to be logged. One treasury product with no API is to be scraped daily instead of hourly, to save those credits.
  • A project rule gives each route its own Cache-Control header, such as public, max-age=1800 on a product page and private, no-store on auth and user pages. The spec has the CDN follow those headers.

In the plans, then, the model runs per product and per new article. A page view never calls it.

Cheaper input still needs a reader

The card’s middle sentence ends on a different point: “its output waits in a queue for human approval.” A cache hit changes what the input costs. It doesn’t change what the model writes, and it doesn’t make that text correct.

In YieldMap’s plans, AI-written risk text and FAQs go through an approval step in the admin, in a flow from pending to approved. The planned review has the owner read the content, edit text that is factually wrong or misleading, and set approved_at for each product. That field already exists in the Products collection:

    {
      name: 'approved_at',
      type: 'date',
      admin: { description: 'Set by admin after reviewing AI output' },
    },
From apps/admin/src/collections/Products.ts, lines 112–116, copied verbatim.

The rest of that flow is planned. The plan has the model’s reply validated as JSON, with every required field, before anything is saved. If a product comes back as malformed JSON, the prompt gets debugged and the job runs again. The spec also makes one line a non-optional legal requirement on every product page and in the site footer: “This is not financial advice. Do your own research.”

What I’d keep

  • Order a tool-use request from stable to changing, and keep the stable front identical from turn to turn, so a cache can serve it.
  • Read a caching figure for what it measures. Input cost is not the whole bill.
  • Keep the model out of the request path. Compute AI text ahead of time and read it as data.
  • Don’t pay twice anywhere: parse each product’s documents once until they change, skip what is already stored, and never summarise the same article twice.
  • Put a person between the model and the page. A cheaper prompt doesn’t make a correct answer.