Prompt Caching: Cut Your API Costs by Up to 90%

The Problem

Every time you send a large system prompt - a 50-page document, a long tool definition block, a huge few-shot example set - the model has to process it from scratch. This prefill step is expensive and slow.

How Caching Works

Prompt caching stores the KV cache from the prefill computation. On subsequent requests that share the same prefix, the model skips the prefill for that prefix entirely.

The result: up to 90% cost reduction and 3–5x lower latency on cached prefixes.

Using It Through Infyrence

Set cache_control on the messages or system block you want to cache:

{
  "model": "anthropic/claude-3-5-sonnet",
  "system": [
    {
      "type": "text",
      "text": "You are a legal document analyst...",
      "cache_control": { "type": "ephemeral" }
    }
  ],
  "messages": [{ "role": "user", "content": "Summarize section 4." }]
}

Infyrence normalizes cache control across providers - the same syntax works whether the model is Claude, GPT-4o, or Gemini.

Cache Lifetime

Caches are ephemeral (5 minutes) by default. Persistent cache tiers are coming in Q3.