Batch API
Process large volumes of chat requests asynchronously at roughly half the cost.
The Batch API is OpenAI Batch-compatible. You upload a JSONL file of requests, create a batch, and collect the results once it finishes — at roughly 50% of the synchronous price. It's ideal for bulk classification, evaluation, and offline content generation.
The supported batch endpoint is /v1/chat/completions. Input is a JSONL file uploaded via the
File API with purpose set to batch.
Workflow
Build a JSONL input file
Each line is one request. custom_id lets you match results back to inputs; url is the batch endpoint;
body is a normal Chat Completions request.
{"custom_id": "req-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "model-id", "messages": [{"role": "user", "content": "Summarize: AI is transforming healthcare..."}]}}
{"custom_id": "req-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "model-id", "messages": [{"role": "user", "content": "Translate to French: Hello World"}]}}Upload it as a batch file
batch_file = client.files.create(
file=open("batch_input.jsonl", "rb"),
purpose="batch",
)Create the batch
batch = client.batches.create(
input_file_id=batch_file.id,
endpoint="/v1/chat/completions",
completion_window="24h",
metadata={"job": "nightly-summaries"},
)
print(batch.id, batch.status)Poll until it completes
batch = client.batches.retrieve(batch.id)
print(batch.status)
print(f"{batch.request_counts.completed}/{batch.request_counts.total} done")Download the results
import json
if batch.status == "completed":
content = client.files.content(batch.output_file_id)
for line in content.text.strip().split("\n"):
result = json.loads(line)
body = result["response"]["body"]
print(result["custom_id"], body["choices"][0]["message"]["content"])Any failed rows are written to error_file_id.
Create batch
POST /v1/batches| Parameter | Type | Required | Description |
|---|---|---|---|
input_file_id | string | Yes | Id of an uploaded JSONL file with purpose=batch. |
endpoint | string | Yes | Batch endpoint. Supported: /v1/chat/completions. |
completion_window | string | No | Target window for completion. Default 24h. |
metadata | object | No | Up to a small set of string key/value pairs. |
List, retrieve, and cancel
GET /v1/batches
GET /v1/batches/{id}
POST /v1/batches/{id}/cancelclient.batches.list()
client.batches.retrieve(batch.id)
client.batches.cancel(batch.id)Batch status
| Status | Description |
|---|---|
validating | The input file is being validated. |
in_progress | Requests are being processed. |
finalizing | Processing is done; the output file is being assembled. |
completed | All requests processed — results are in output_file_id. |
failed | The batch failed (e.g. invalid input file). |
expired | Not finished within the completion window. |
cancelling | A cancel request is being applied. |
cancelled | The batch was cancelled. |
The normal lifecycle is validating → in_progress → finalizing → completed. Inspect
batch.request_counts (total, completed, failed) to track progress.
Batch and collection-management paths are excluded from per-minute rate limiting — submit large jobs freely. See Pricing for the batch discount and Rate Limits.
How is this guide?