Batch Processing (Batch API)
The Batch API is under development and will be available in upcoming platform updates.
The Batch API lets you submit a large number of requests at once and retrieve results asynchronously. This is ideal for bulk data processing when immediate responses are not required.
When to Use Batch API
- Bulk processing — Processing thousands of texts, images, or classification requests.
- Cost savings — Batch requests cost 50% less than standard API requests.
- Higher rate limits — The Batch API has a separate, significantly higher request limit.
- Async tasks — When results can wait — report generation, ETL pipelines, data processing.
Batch jobs typically complete within 24 hours. Most finish significantly faster.
How It Works
The process consists of four steps:
- 1. Prepare the file — Create a JSONL file where each line is a separate request.
- 2. Upload the file — Upload the file via the Files API and get a file_id.
- 3. Create the batch — Submit a batch creation request with the file_id.
- 4. Retrieve results — Poll the batch status and download results when ready.
Input File Format (JSONL)
Each line in the file is a JSON object with custom_id, method, url, and body fields:
requests.jsonl
{"custom_id": "req-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "mira", "messages": [{"role": "user", "content": "Translate to French: Hello"}], "max_tokens": 100}}
{"custom_id": "req-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "mira", "messages": [{"role": "user", "content": "Translate to French: Thank you"}], "max_tokens": 100}}
{"custom_id": "req-3", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "mira", "messages": [{"role": "user", "content": "Translate to French: Goodbye"}], "max_tokens": 100}}custom_id must be unique for each request. It lets you match results back to the original request.
Step 1: Upload File
Python
import requests
# Upload the JSONL file
with open("requests.jsonl", "rb") as f:
upload = requests.post(
"https://api.vmira.ai/v1/files",
headers={"Authorization": "Bearer sk-mira-YOUR_KEY"},
files={"file": ("requests.jsonl", f, "application/jsonl")},
data={"purpose": "batch"}
)
file_id = upload.json()["id"]
print(f"File uploaded: {file_id}")Step 2: Create Batch
Python
batch = requests.post(
"https://api.vmira.ai/v1/batches",
headers={
"Authorization": "Bearer sk-mira-YOUR_KEY",
"Content-Type": "application/json",
},
json={
"input_file_id": file_id,
"endpoint": "/v1/chat/completions",
"completion_window": "24h"
}
)
batch_id = batch.json()["id"]
print(f"Batch created: {batch_id}")
# batch_abc123Step 3: Check Status
Python
import time
while True:
status = requests.get(
f"https://api.vmira.ai/v1/batches/{batch_id}",
headers={"Authorization": "Bearer sk-mira-YOUR_KEY"}
).json()
print(f"Status: {status['status']} | "
f"Completed: {status['request_counts']['completed']}/{status['request_counts']['total']}")
if status["status"] in ("completed", "failed", "expired"):
break
time.sleep(30) # Poll every 30 secondsBatch Statuses
StatusDescription
validatingFile is being validated
in_progressRequests are being processed
completedAll requests have been processed
failedBatch failed with an error
expiredBatch did not complete in time
cancellingCancellation in progress
cancelledBatch was cancelled
Step 4: Retrieve Results
Python
import json
# Get the output file ID from the completed batch
output_file_id = status["output_file_id"]
# Download the results
results = requests.get(
f"https://api.vmira.ai/v1/files/{output_file_id}/content",
headers={"Authorization": "Bearer sk-mira-YOUR_KEY"}
)
# Parse JSONL results
for line in results.text.strip().split("\n"):
result = json.loads(line)
custom_id = result["custom_id"]
content = result["response"]["body"]["choices"][0]["message"]["content"]
print(f"{custom_id}: {content}")JavaScript Example
JavaScript
const API_KEY = "sk-mira-YOUR_KEY";
const BASE = "https://api.vmira.ai/v1";
// Step 1: Upload
const formData = new FormData();
formData.append("file", new Blob([jsonlContent], { type: "application/jsonl" }), "requests.jsonl");
formData.append("purpose", "batch");
const upload = await fetch(`${BASE}/files`, {
method: "POST",
headers: { "Authorization": `Bearer ${API_KEY}` },
body: formData,
});
const { id: fileId } = await upload.json();
// Step 2: Create batch
const batch = await fetch(`${BASE}/batches`, {
method: "POST",
headers: {
"Authorization": `Bearer ${API_KEY}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
input_file_id: fileId,
endpoint: "/v1/chat/completions",
completion_window: "24h",
}),
});
const { id: batchId } = await batch.json();
// Step 3: Poll status
let status;
do {
await new Promise(r => setTimeout(r, 30000));
const res = await fetch(`${BASE}/batches/${batchId}`, {
headers: { "Authorization": `Bearer ${API_KEY}` },
});
status = await res.json();
console.log(`Status: ${status.status}`);
} while (!["completed", "failed", "expired"].includes(status.status));
// Step 4: Download results
const output = await fetch(`${BASE}/files/${status.output_file_id}/content`, {
headers: { "Authorization": `Bearer ${API_KEY}` },
});
const text = await output.text();
const results = text.trim().split("\n").map(line => JSON.parse(line));
console.log(results);Error Handling
If some requests in the batch failed, they will be recorded in error_file_id. Download this file for diagnostics:
Python
if status.get("error_file_id"):
errors = requests.get(
f"https://api.vmira.ai/v1/files/{status['error_file_id']}/content",
headers={"Authorization": "Bearer sk-mira-YOUR_KEY"}
)
for line in errors.text.strip().split("\n"):
err = json.loads(line)
print(f"{err['custom_id']}: {err['error']['message']}")Maximum input file size is 100 MB. Maximum number of requests per batch is 50,000.
Best Practices
- Small batches for testing — Start with 5-10 requests to verify the format is correct before submitting thousands.
- Unique custom_ids — Use meaningful identifiers (e.g., row-1234) to easily match results.
- Reasonable polling — Do not poll status more than once every 30 seconds. For large batches, poll once per minute.
- Handle partial failures — Always check error_file_id after batch completion and handle failed requests.