Prompt caching

What it is

Prompt caching stores the model’s processing of a prompt prefix, so later requests with the same prefix skip that work. All three major APIs offer it, with different controls. OpenAI and Anthropic let you rely on automatic caching or set cache breakpoints yourself. Google turns implicit caching on by default for Gemini 2.5 and newer models.

How it works

1. Put the unchanging part of the prompt first: instructions, examples, documents.
2. The first request processes and caches that prefix.
3. Later requests that share the exact prefix within the cache lifetime are billed at the cached rate.

Current terms (checked 2026-10-17):

• Anthropic. Cache reads cost 0.1× the base input price on most models. Writes cost 1.25× for a 5-minute cache or 2× for a 1-hour cache.
• OpenAI. On GPT-5.6 and later, cache writes cost 1.25× the uncached input rate, the minimum cacheable prefix is 1,024 tokens, and the default lifetime is 30 minutes.
• Google. Each Gemini model has a minimum prompt size before caching applies.

Benefits

• Lower cost for chatbots, agents and document Q&A that resend the same context.
• Faster responses on long prompts.
• Makes long, detailed system prompts more affordable.

Limits and risks

• Only exact prefix matches count. A small change early in the prompt breaks the cache.
• Caches expire, and writes can cost more than normal input. Short or rarely reused prompts may not save money.
• Rates, lifetimes and minimums differ by vendor and model, so re-check before you budget.