batchwatch
Live dashboard Use the API

batchwatch › Questions › LLM batch API vs synchronous: which should I use?

LLM batch API vs synchronous: which should I use?

Batch when you can wait: half the price, a wait usually measured in minutes. Synchronous when you cannot.

The answer

The trade is simple and it is yours to make. Batch costs 50% of the synchronous rate at OpenAI, Anthropic, Google - their published price - in exchange for a queue wait. Synchronous costs double and answers immediately. The only variable is your deadline.

What tips it toward batch for most workloads is that the wait is short: we measured a median of 77s for gpt-5.6-sol, with nine in ten jobs finishing within 22 min. Against half the price, minutes of wait is an easy call unless you are on a hard, immediate deadline.

Questions this answers

Should I use the batch API or synchronous?
Use batch whenever the work can wait: it is 50% cheaper at every major provider. Use synchronous only when you need the answer immediately. batchwatch measures the real batch wait so you can size that "can wait" against actual numbers rather than the 24-hour ceiling.
How much does the batch API save over synchronous?
50% - batch is half the synchronous price on both input and output tokens at OpenAI, Anthropic and Google. That is their published price, not a batchwatch estimate.

See it live

Every number here is measured, and it is all public.

The live dashboard shows every model with its distribution, n and confidence grade. /v1/coverage is the same data as JSON, no key required. Contributing five measurements in seven days cuts the public delay to five minutes and unlocks the decision endpoints (/v1/should-i-batch, /v1/estimate-batchtime).

More questions we have measured the answer to

How long does the OpenAI batch API take? How long does the Anthropic Message Batches API take? How long does the Google (Gemini) batch API take? Is the OpenAI Batch API worth it? Is the Anthropic Message Batches API worth it? How much cheaper is the OpenAI Batch API? How much cheaper is the LLM batch API?