Verdict
Run the ratio before you build anything: at current prices a cached read costs 0.1x base input on Claude (Anthropic prompt caching: 1.25x base input to write with the 5-minute TTL, 2x to write with the 1-hour TTL, 0.1x to read - platform.claude.com/docs pricing) and 0.1x on OpenAI's current GPT-5.4 family (gpt-5.4 input $2.50/M, cached input $0.25/M - a 90% discount - per developers.openai.com/api/docs/pricing). RAG sends roughly 2,000 retrieved tokens at full price. So caching beats retrieval on per-call cost only while corpus_tokens x 0.1 < retrieved_tokens - that is, while your corpus is under roughly 10x your retrieved context, around 20,000-30,000 tokens for typical top-k sizes. Below that line, cache everything: it is cheaper AND the model sees the whole corpus. Above it, caching costs more per call than RAG but buys whole-corpus visibility and zero retrieval-quality risk - a quality-for-money trade, not a cost win - until the corpus outgrows the context window entirely and RAG becomes mandatory. Agent loops tilt the math toward caching harder than single-shot apps because every loop iteration re-reads the same prefix, amortizing the write premium across many reads within the TTL. And the strongest production answer is often the hybrid neither camp markets: cache the stable system prompt, tool schemas, and core docs, and RAG the long tail behind a retrieval tool.
Every number in the worked examples below comes from a current vendor price sheet. Anthropic: cache writes cost 1.25x base input (5-minute TTL) or 2x (1-hour TTL), cache reads 0.1x, per the pricing page at platform.claude.com/docs; with Claude Sonnet-class base input at $3/M, that is $3.75/M to write and $0.30/M to read. OpenAI: gpt-5.4 input $2.50/M, cached input $0.25/M (90% off), applied automatically once the prefix reaches 1,024 tokens with no write premium - and the same 0.1x ratio holds down the family, e.g. gpt-5.4-mini at $0.75/M input and $0.075/M cached (developers.openai.com/api/docs/pricing). RAG's cost side: OpenAI text-embedding-3-small at $0.02/M tokens and text-embedding-3-large at $0.13/M (openai.com/api/pricing) make embedding nearly free - a 2M-token corpus costs $0.04 to embed with the small model - so RAG's real per-query cost is the retrieved tokens themselves at full input price, plus vector-store hosting (pgvector in your existing Postgres makes that line item approximately zero; our pgvector vs Qdrant comparison covers when it does not). Worked example, Sonnet-class prices, 2,000-token retrieval, 20-iteration agent run inside one cache TTL: (a) 30k-token corpus - cached: one write $0.1125 + 20 reads at $0.009 = $0.29 total; RAG: 20 calls x 2k tokens at $3/M = $0.12 plus retrieval infrastructure. Close to parity, and caching sends the model everything. (b) 200k-token corpus (at the context ceiling) - cached: write $0.75 + 20 reads at $0.06 = $1.95; RAG: still $0.12. Caching now costs ~16x more per run - you are buying whole-corpus attention, and you should ask whether the task needs it. (c) 2M-token corpus - no mainstream window fits it; RAG (or hierarchical summarization) is forced, and the embedding bill is four cents. No published comparison we found states this crossover ratio explicitly; SERP checks show the two techniques covered separately or framed as 'caching kills RAG', which the arithmetic does not support.
Decision Table
| Criterion | Edge | Explanation |
|---|---|---|
| Per-call cost, small corpus (under ~30k tokens) | Prompt caching (put the whole corpus in a cached prefix and stuff the window) Best edge | With cached reads at 0.1x base input (Anthropic at $0.30/M for Sonnet-class and OpenAI gpt-5.4 at $0.25/M per their pricing pages), a 30k-token cached prefix reads at about $0.009 per call on Sonnet - comparable to sending 2,000 retrieved tokens at full price ($0.006), while giving the model the entire corpus instead of top-k chunks. Below roughly 10x your retrieved-context size, caching is both cheaper-or-equal and strictly better for quality, and you delete the whole retrieval subsystem. |
| Per-call cost, large corpus (100k+ tokens) | RAG (embed the corpus, retrieve top-k chunks per query or loop iteration) Best edge | The 0.1x multiplier scales with corpus size; retrieval does not. A 200k-token cached prefix on Sonnet-class pricing reads at $0.06 per call - sixteen times the $0.006 of a 2k-token retrieval - and every loop iteration pays it (worked math above, from platform.claude.com/docs and openai.com/api/pricing figures). At scale, RAG's per-query cost is flat in corpus size while caching's grows linearly. Teams that cache 200k-token prefixes into high-iteration agent loops discover this line item quickly. |
| Answer quality and retrieval risk | Prompt caching (put the whole corpus in a cached prefix and stuff the window) Best edge | Caching has no retrieval failure mode: the model attends over the whole corpus, so there is no chunking strategy to tune, no embedding model mismatch, no 'the answer was in a chunk we didn't retrieve.' RAG's quality ceiling is set by its weakest retrieval, and cross-document reasoning (schema joins, config interactions, code call-graphs) is exactly where top-k retrieval silently drops needed context. If wrong-but-confident answers are expensive for you, that risk difference is worth real money. |
| Corpus size ceiling | RAG (embed the corpus, retrieve top-k chunks per query or loop iteration) Best edge | Caching dies at the context window; RAG scales to arbitrary corpus size. A 2M-token corpus cannot be a prefix on mainstream models, but embeds for $0.04 with text-embedding-3-small at $0.02/M tokens (openai.com/api/pricing) and serves from pgvector at negligible marginal cost. Past the window, this is not a trade-off - RAG (or hierarchical summarization feeding a cached digest) is simply mandatory. |
| Latency in the agent loop | Prompt caching (put the whole corpus in a cached prefix and stuff the window) Best edge | A cache hit removes prefill work on the provider side and removes an entire retrieval round-trip (embed query, vector search, assemble context) from your side of every loop iteration. RAG inserts network hops and a search in the hot path of each step. For interactive agents where per-iteration latency compounds across a multi-step loop, cached-prefix calls are the faster architecture; RAG's latency cost is amortizable only when iterations are few or parallel. |
| Freshness and update cost | RAG (embed the corpus, retrieve top-k chunks per query or loop iteration) Best edge | Caching's dirty secret is invalidation: change one token of the prefix and the next call pays a full re-write - 1.25x base input on the whole corpus for Anthropic's 5-minute TTL, 2x for the 1-hour TTL (platform.claude.com/docs). A frequently-changing corpus turns the write premium into a recurring tax and shreds hit rates. RAG updates incrementally: re-embed the changed document for fractions of a cent and the next query sees it. Fast-moving corpora belong in a vector store. |
| Operational simplicity | Prompt caching (put the whole corpus in a cached prefix and stuff the window) Best edge | Prompt caching is a header and prompt-ordering discipline (stable prefix first; OpenAI applies it automatically past 1,024 prefix tokens, Anthropic via cache_control breakpoints). RAG is a subsystem: chunking pipeline, embedding jobs, a vector store, retrieval evaluation, and re-ranking tuning - components that need monitoring and break independently. Where the corpus fits and the cost math is close, the architecture with fewer moving parts should win the tie. |
| Zero-engineering default | Tie | A third option wins this row: implicit caching you get without designing for either strategy. OpenAI applies the 90% cached-input discount automatically to any stable prefix past 1,024 tokens (openai.com/api/pricing), and Gemini's implicit caching discounts reused prefixes without explicit cache management (Google's docs describe implicit caching as automatic, with explicit caching adding a per-token-hour storage charge - verify current rates on ai.google.dev before modeling costs on them). If your prompts already lead with a stable system block, you are collecting most of the caching win with no architecture decision at all - measure your existing cache-hit discount before building anything. |
Choose Prompt caching (put the whole corpus in a cached prefix and stuff the window) if...
- Agents consulting a stable corpus under ~30k tokens: cheaper than RAG per call, better answers, no retrieval subsystem.
- High-iteration agent loops re-reading the same prefix many times per TTL - the regime the write/read pricing was designed for.
- Tasks needing cross-document reasoning where top-k retrieval silently drops context: whole-corpus attention is the product.
- Teams that want the fewest moving parts: prompt ordering plus a header versus a chunking-embedding-retrieval pipeline.
Choose RAG (embed the corpus, retrieve top-k chunks per query or loop iteration) if...
- Corpora past the context window - there is no choice to make, and embedding costs cents.
- Large corpora (100k+ tokens) queried with low iteration counts, where 0.1x-of-everything costs a multiple of retrieving 2k tokens.
- Fast-changing corpora where cache invalidation would trigger repeated full-prefix re-writes at 1.25-2x base input.
- Multi-tenant products where per-tenant prefixes would thrash provider caches, but a shared vector store partitions cleanly by tenant filter.
Decision rules
- 01Compute the ratio first: if corpus_tokens x 0.1 < your typical retrieved tokens (roughly: corpus under 10x retrieval size, ~20-30k tokens for 2-3k-token retrievals), cache everything and skip RAG entirely.
- 02If the corpus exceeds the model's context window, the decision is made: RAG (or hierarchical summarization feeding a cached digest). Embedding is nearly free - $0.02/M tokens with text-embedding-3-small.
- 03If your agent loop runs many iterations against the same context within a cache TTL, weight caching up: the write premium amortizes across reads, and Anthropic's 1-hour TTL at a 2x write premium pays for itself when reads-per-write within the hour exceed roughly 2 (each read saving 0.9x base on the prefix).
- 04If the corpus changes faster than your cache TTL, weight RAG up: every prefix edit re-triggers a full cache write at 1.25-2x base input on the entire corpus.
- 05If wrong answers from missed retrieval are expensive (compliance text, safety-critical config), pay the caching premium for whole-corpus attention even above the cost crossover - it is a quality purchase, priced by the worked math above.
- 06Before building either: check your current cache-hit rates. OpenAI and Gemini apply prefix caching automatically; if your prompts already lead with stable content you may be most of the way there.
- 07Do not pay a vector-store vendor to test the RAG side - pgvector in the Postgres you already run is the zero-commitment way to measure retrieval quality on your corpus.
Migration Notes
- RAG to caching: collapse retrieval by moving the corpus into the prompt behind a cache breakpoint, and enforce prefix stability - any dynamic content (timestamps, user context) must move below the cached block or every call becomes a cache miss at write prices.
- Caching to RAG: your prompts shrink but you inherit a retrieval-quality problem that did not exist before; build a small eval set of real agent queries first and measure recall before cutting over.
- The hybrid is usually the destination, not a compromise: cache system prompt + tool schemas + the hot core of the corpus; expose the long tail through a retrieval tool the agent calls when needed. Both pricing models reward exactly this split.