Notes from the routing desk
Engineering, routing economics, and product updates from the team building Batchrouter.
Hello world: introducing the Batchrouter blog
Engineering notes from the routing engine, plain-language posts on AI batch economics, and product updates — all in one place, with RSS.
Batchrouter SLA tiers and what to use each one for
Not every AI request has to come back right now. The more time you can give the work, the cheaper it can run. A tour of Batchrouter's six SLA tiers, the workloads each one fits, and three questions that point to the right one.
Keep your GPUs busy around the clock with Batchrouter
You size a cluster for your busiest hour, and the rest of the day part of it waits. Here's how Batchrouter providers fill the troughs with batch work: open your lane when you have room, close it when you don't, and get paid monthly for every item you complete.
AI spending guardrails should be a first-class API feature
Teams running batch AI workloads at scale need spending guardrails they can trust — not approximate spend limits bolted on after the fact, but controls built into the infrastructure itself.
See where your results landed — and find the docs at their new home
A delivery status badge and one-click retry on the batch detail page — so you always know whether your results reached their destination. Plus: the docs have a proper home at docs.batchrouter.com.
Spending controls, one-click feedback, and a steadier sign-in
Cap your spend with limits and alerts, report a bug or share an idea (screenshot included) without leaving the page, and a momentary network blip no longer signs you out. Here's everything in the June 11 release.
Sign in before you verify — and a verify link that always works
You can now sign in the moment you sign up, the email-verification link works on the first click even after a mail scanner prefetches it, and buying credits is gated on a verified address. Here's what changed and why.
Routing you can read back: tool-aware lanes, rejection receipts, and privacy proofs
When Batchrouter quotes a batch it weighs every eligible provider lane. Three new upgrades make that decision legible: routing that understands required hosted tools, a receipt for every rejected lane, and a privacy proof on each one.
Lessons from routing 100M tokens
100M tokens through the router. Five lessons that changed the engine: mixed-model JSONL, latency-vs-price flips, webhook redelivery costs, idempotency, and model deprecations.
Cut your AI bill without changing your prompts
Five concrete moves that cut LLM cost 30–50% with no prompt changes, no model swaps, no quality regressions. Just smarter routing.
Routing a JSONL batch in under 50ms
A JSONL file hits POST /v1/quotes/model. ~50ms later, the client has an estimate with a routing explanation. Here's exactly what happens in between, and where the time goes.
OpenAI Batch vs Anthropic Message Batches: a routing-first comparison
Same discount. Same clock. Different shapes, different limits, different status names. Here's an honest, routing-first comparison of OpenAI Batch and Anthropic Message Batches.
Usage-based receipts: the missing billing primitive for agentic commerce
Charge-on-checkout assumes the buyer knows the price. Monthly invoices assume the buyer trusts the seller. Neither works when the buyer is an agent. Usage-based receipts do.
Why we built Batchrouter
Async AI inference is roughly half the price of sync. Yet most teams still pay sync rates. The reason isn't ignorance — it's operational tax. Here's how we attack it.