Batch processing (batch APIs)

What it is

A batch API accepts a file or list of requests, processes them asynchronously within a stated window, and returns the results together. Vendors price batch work below real-time calls.

How it works

1. Prepare a file of requests, one per item.
2. Submit it to the batch endpoint and note the job ID.
3. Collect the results when the job finishes, then load them into your system.

Vendor terms (checked 2026-10-17):

• OpenAI’s Batch API offers 50% lower costs, a separate pool of higher rate limits and a 24-hour turnaround.
• Anthropic’s Message Batches API charges 50% of standard prices. Most batches finish within an hour, and batches that don’t complete within 24 hours expire. A batch can hold up to 100,000 requests or 256 MB, and results stay available for 29 days.

Benefits

• Lower cost for bulk work.
• Higher volume without hitting real-time rate limits.
• A simple pattern for nightly reporting and enrichment.

Limits and risks

• No instant answers. Results arrive later.
• One bad input format can fail many items, so validate a small sample first.
• Anthropic notes that batches may go slightly over a workspace’s spend limit because of concurrent processing.
• Discounts and windows change, so re-check the vendor page before you plan around them.