Prompt caching or batch - which saves more?

They save on different things, so the honest answer is "it depends, and often both". Caching discounts the part of your prompt you send again and again - a long system prompt or shared context - and it keeps the response instant. Batch discounts 50% off every token, cached or not, in exchange for a queue wait - a measured median of 78s on the fastest model (gpt-5.6-sol at OpenAI) right now. If your prompts repeat a big prefix and you need the answer now, caching wins. If the work can wait, batch wins. If both are true, you stack them.

When is prompt caching the better choice?

When you need the answer immediately and your requests share a large, stable prefix - a long system prompt, a document you ask many questions about, a few-shot example block. Caching keeps full synchronous latency and discounts only the repeated tokens, so it is the right tool for interactive and low-latency work. Batch cannot help there: its saving is paid for with a wait.

When is the batch API the better choice?

When the work can wait - nightly jobs, backfills, evaluations, bulk generation. Batch takes 50% off every token, not just a repeated prefix, so it beats caching on cost for one-off inputs with no shared context. The only question is whether the queue wait fits your deadline, and that is the number we measure - a measured median of 78s on the fastest model (gpt-5.6-sol at OpenAI) right now.

Can I use prompt caching and batch together?

Usually yes, and it is often the cheapest option: cache the repeated prefix and submit the job to the batch queue, and you get both discounts. One important exception is Google Gemini, whose own docs say implicit caching takes precedence over the batch discount - they do not stack there. Check your provider's page; each states its own rule, and we link all three above.

Are these caching numbers measured by batchwatch?

No, and we say so plainly. The caching discounts are each provider's OWN published figure, linked to the source so you can check it - we do not measure caching. The one number on this page that is ours is the batch queue time, an observed completion of a real batch job, rendered live and published only once a model has at least 8 completions.

Prompt caching vs batch: which saves more?

Two different discounts

Prompt caching instant

Discounts the tokens you send again and again - a long system prompt, shared context, a few-shot block. The response comes back at full synchronous speed; only the repeated prefix is cheaper.

Best for interactive and low-latency work.
Saves nothing on tokens that are not repeated.
Each provider's own published figure, linked below.

Batch API queued

Cuts 50% off every token - cached or not, input and output - in exchange for a queue wait. The saving is flat and provider-published; the wait is the part everyone else guesses at, and the part we measure.

Best for work that can wait: nightly jobs, backfills, evals.
Discounts one-off inputs caching cannot touch.
The wait is measured, a measured median of 78s on the fastest model (gpt-5.6-sol at OpenAI) right now.

The batch discount, and its measured cost in time

50% of synchronous the providers' own published batch rate, on every input and output token

OpenAI, Anthropic and Google all price their batch API at 50% of the synchronous rate. What they publish is a "within 24 hours" ceiling and nothing more precise; the real median is far below it and different for every model. We measure it - a measured median of 78s on the fastest model (gpt-5.6-sol at OpenAI) right now - so you can weigh the flat 50% saving against a wait you can actually see rather than folklore. Compare every model's measured wait.

Batch queue time rendered live from the rollup; newest measurement 2026-08-30. Caching discounts are the providers' own published figures, linked above.

Prompt caching, per provider (their published numbers)

Provider	Published caching discount	How it works
OpenAI	up to 75% off cached input tokens	Automatic once a prompt prefix exceeds 1024 tokens; the cached portion of the input is discounted, output is not.
Anthropic	up to 90% off cached input tokens	Explicit cache_control breakpoints; you pay a one-time write premium, then reads of the cached prefix are heavily discounted.
Google (Gemini)	up to 90% off cached input tokens (implicit caching)	Implicit caching applies automatically on repeated prefixes; the cache discount takes precedence over the batch discount - they do NOT stack.

These are the providers' OWN published caching discounts, each linked to its source. batchwatch does not measure caching - we quote it as theirs, exactly as we quote the batch discount as theirs.

So which should you use?

Need it now, and your prompts share a big prefix? Cache. Batch cannot help when latency matters.

Can it wait? Batch - it discounts every token, not just a repeated one, and the only cost is a queue wait you can now see measured.

Both true? Stack them where the provider allows it (all but Gemini's implicit cache, which takes precedence over batch). That is usually the cheapest path of all. The point of this page is that you should not have to pick blind - and on the one number nobody else publishes, the batch wait, you no longer have to.

Questions this answers

Prompt caching or batch - which saves more?: They save on different things, so the honest answer is "it depends, and often both". Caching discounts the part of your prompt you send again and again - a long system prompt or shared context - and it keeps the response instant. Batch discounts 50% off every token, cached or not, in exchange for a queue wait - a measured median of 78s on the fastest model (gpt-5.6-sol at OpenAI) right now. If your prompts repeat a big prefix and you need the answer now, caching wins. If the work can wait, batch wins. If both are true, you stack them.
When is prompt caching the better choice?: When you need the answer immediately and your requests share a large, stable prefix - a long system prompt, a document you ask many questions about, a few-shot example block. Caching keeps full synchronous latency and discounts only the repeated tokens, so it is the right tool for interactive and low-latency work. Batch cannot help there: its saving is paid for with a wait.
When is the batch API the better choice?: When the work can wait - nightly jobs, backfills, evaluations, bulk generation. Batch takes 50% off every token, not just a repeated prefix, so it beats caching on cost for one-off inputs with no shared context. The only question is whether the queue wait fits your deadline, and that is the number we measure - a measured median of 78s on the fastest model (gpt-5.6-sol at OpenAI) right now.
Can I use prompt caching and batch together?: Usually yes, and it is often the cheapest option: cache the repeated prefix and submit the job to the batch queue, and you get both discounts. One important exception is Google Gemini, whose own docs say implicit caching takes precedence over the batch discount - they do not stack there. Check your provider's page; each states its own rule, and we link all three above.
Are these caching numbers measured by batchwatch?: No, and we say so plainly. The caching discounts are each provider's OWN published figure, linked to the source so you can check it - we do not measure caching. The one number on this page that is ours is the batch queue time, an observed completion of a real batch job, rendered live and published only once a model has at least 8 completions.

See the measured side for yourself

The batch wait is measured, and it is all public.

The live dashboard and /compare show every model's real queue time with its sample size and confidence. /v1/coverage is the same data as JSON, no key required. Send your own deadline to /v1/should-i-batch and we tell you whether the batch saving is worth the wait for your job.

Prompt caching or batch: which actually saves more?