Prompt Caching
Reusing the processed form of a repeated prompt prefix so later requests are cheaper and faster.
/ quick answer
Providers can cache the computed state of a long, unchanged prefix — system prompt, tool definitions, reference documents — and bill cached tokens at a steep discount. Caching only works when the prefix is byte-identical, so stable content must come first and variable content (user message, timestamps) last.
/ frequently asked
What is Prompt Caching?
Providers can cache the computed state of a long, unchanged prefix — system prompt, tool definitions, reference documents — and bill cached tokens at a steep discount. Caching only works when the prefix is byte-identical, so stable content must come first and variable content (user message, timestamps) last.
What is an example of Prompt Caching?
Moving a 6,000-token policy document to the start of every support prompt and the customer question to the end lets most requests hit the cache.
Why does Prompt Caching matter?
Reusing the processed form of a repeated prompt prefix so later requests are cheaper and faster.
What should I learn after Prompt Caching?
Open the related workflows and prompts further down this page: they show how this idea is applied in a real task, step by step.
/ continue exploring
Related concepts
The vocabulary this page depends on.
- →Small Language Model (SLM)
A language model with roughly 1–15B parameters, small enough to run on a laptop, phone or single GPU while handling narrow tasks well.
- →Semantic Cache
A cache that returns a stored LLM answer when a new question means the same thing as an earlier one, even if worded differently.
- →Knowledge Distillation
Training a small model to imitate the outputs of a large one, keeping most of the quality on a target task at a fraction of the cost.
Related workflows
Turn this into a repeatable process.
- →Cut LLM Inference Costs
A step-by-step method to reduce model spend without degrading output quality.
- →Route Requests Between Models
Send each request to the cheapest model that can handle it, escalating only when needed.
Related tool stacks
The tools that run it in production.
- →LLM Cost Optimization Stack
The layers that keep AI spend predictable as usage grows.
Related prompts
Reusable prompts for this job.
- →Model Routing Classifier Prompt
A cheap-model prompt that labels request difficulty so it can be routed to the right model.
- →Prompt Improver Prompt
Iterate a weak prompt into a strong one using best practices.
- →Strategic Analyst Prompt
Turn a business situation into a McKinsey-style strategic memo.
- →Buyer Persona Builder Prompt
Generate a detailed persona from a product description and target market.
Related use cases
How people apply it, and what came out.
- →SaaS Cuts AI Bill With Routing
Illustrative example: routing easy requests to a small model reduces spend while holding quality steady.