Cut your AI bill without changing your prompts

There's a tempting category of cost-cutting advice that involves rewriting your prompts, distilling to a smaller model, or trimming output tokens. It works, but it costs you quality and engineering time, and the savings are usually in the 10–20% range. This post is about the other category: five routing-layer moves that cut spend 30–50% without touching the prompt, the model, or the output. The output bytes you'd ship today are the same output bytes you'd ship after applying these. The bill is just lower.

1. Move sync jobs to batch wherever latency allows

OpenAI's Batch API is 50% off sync prices for the same models. Anthropic's Message Batches is also 50% off, and on some models the effective delta is larger once volume discounts compound. The 24-hour SLA sounds long but is irrelevant for most of what teams actually run on LLMs: nightly enrichment, classification queues, eval runs, document pipelines, agent backfills, dataset labeling, RAG re-embedding sweeps.

If a job runs on a cron, or kicks off when a user uploads a file they don't expect back instantly, or processes a queue that drains overnight — it's batch-eligible. Audit one week of your sync usage and tag anything that doesn't need to finish in under 60 seconds. That's your first 50%.

2. Route across providers, not just within one

If a job is OpenAI Batch eligible and Anthropic Message Batch eligible, the cheaper provider wins on that particular run. Prices move. Native batch lanes on Batchrouter carry an automatic 10% discount on top of the provider rate, which often flips the ranking from one week to the next. Without a router, you've committed to whichever provider you wired up first. With one, your custom_id semantics stay identical and the lane selector picks the floor.

Typical real-world deltas we see between equivalent-tier models on a given week:

• Small-tier (Haiku-class vs GPT-4o-mini-class): 8–18% spread, alternates direction.
• Mid-tier (Sonnet-class vs GPT-4o-class): 10–25% spread.
• Routing to the floor across a quarter saves roughly 12% blended on top of whatever batch savings you already booked.

3. Use workflow products for repeatable shapes

A lot of batch work isn't general inference — it's the same task over and over: classify a document, extract a structured object, score a reranking, summarise, translate, moderate. We ship these as workflow products at /workflows: outcome contracts where you describe the input schema, the output schema, and the quality target, and we pick the lane.

Counter-intuitively, this is often cheaper net than DIY routing even though the workflow fee is 11.8% versus the 5.9% direct-route fee. Two reasons. First, schema-side caching: when the system prompt and output schema are stable across runs, we can share cache state across customers in a privacy-safe way that's hard to replicate when each team rolls their own prompt. Second, curated lane selection: a workflow product knows, for example, that a Haiku-class model hits the structured-extraction quality bar at ~30% of the cost of a Sonnet-class one for typical inputs — so we route there by default and only escalate on disagreement.

If you're running classification, extraction, rerank, summarise, translate, or moderation at any meaningful volume, check whether the workflow contract beats your current setup before optimising your own prompt further.

4. Let the router split mixed-model JSONL

Production JSONL rarely uses one model. You might want Haiku for the cheap rows, Sonnet for the hard ones, GPT-4o-mini for a specific structured-output subset. The naive path is to run three separate batch jobs, get three results files back, and stitch them in application code. That's three poll loops, three failure modes, and three reconciliation paths.

Batchrouter accepts mixed-model JSONL natively. We split internally into one lane per distinct model, run each on the cheapest eligible provider, and merge results back into a single output file ordered by your original custom_id. You get one batch_id to track. The control-plane fee for the split is $0.02 per internal lane — real, but utterly tiny against the engineering cost of running this yourself.

5. Pay only for actual usage

Most prepaid-credit setups round to the cent or to the credit pack. Batchrouter holds your reservations in micro-dollars and settles to actual usage. When the job lands, anything you didn't use comes back to your credit balance. The estimate is conservative on purpose — we'd rather reserve a little extra and release than under-reserve and fail the precheck — and the gap between reserved and settled is yours.

Over a year of nightly batches, the released-back delta compounds. See /pricing for the full breakdown of credit packs and the settlement model.

Quick check: what to do today

1. If you have any sync workload that doesn't need a sub-minute response — move it to batch. That alone is ~50%.
2. If you're locked to one provider for batch — route across both. Add another ~10–12% blended.
3. If you're running classification, extraction, rerank, summarise, translate, or moderation at scale — check the workflow products before re-tuning your prompt.
4. If your JSONL uses more than one model — let the router split, don't run parallel jobs.
5. If you're on flat credit packs that round up — switch to a settlement model that pays back what you don't use.

If you're submitting via OpenAI's Batch shape today, switching to Batchrouter is a base URL change. The endpoint, the file format, the polling loop, and the custom_id semantics are identical.

# Same OpenAI Batch shape, different host.
curl https://api.batchrouter.com/v1/batches \
  -H "Authorization: Bearer $BATCHROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input_file_id": "file_abc123",
    "endpoint": "/v1/chat/completions",
    "completion_window": "24h"
  }'

Five moves, applied in order, typically land between 30% and 50% lower spend on the same workload with the same outputs. None of them touches a prompt. None of them swaps a model. None of them changes a single byte of what your users see.

Want a real number on your real workload? Drop one of your monthly batches into the calculator and see what comes back. If the saving looks worth a base URL change, /docs/quickstart gets you running in under ten minutes. If you want to understand why we built it this way, the manifesto post covers it.