Notes from the routing desk
Engineering, routing economics, and product updates from the team building Batchrouter.
Hello world: introducing the Batchrouter blog
Engineering notes from the routing engine, plain-language posts on AI batch economics, and product updates — all in one place, with RSS.
AI spending guardrails should be a first-class API feature
Teams running batch AI workloads at scale need spending guardrails they can trust — not approximate spend limits bolted on after the fact, but controls built into the infrastructure itself.
See where your results landed — and find the docs at their new home
A delivery status badge and one-click retry on the batch detail page — so you always know whether your results reached their destination. Plus: the docs have a proper home at docs.batchrouter.com.
Spending controls, one-click feedback, and a steadier sign-in
Cap your spend with limits and alerts, report a bug or share an idea (screenshot included) without leaving the page, and a momentary network blip no longer signs you out. Here's everything in the June 11 release.
Sign in before you verify — and a verify link that always works
You can now sign in the moment you sign up, the email-verification link works on the first click even after a mail scanner prefetches it, and buying credits is gated on a verified address. Here's what changed and why.
Routing you can read back: tool-aware lanes, rejection receipts, and privacy proofs
When BatchRouter quotes a batch it weighs every eligible provider lane. Three new upgrades make that decision legible: routing that understands required hosted tools, a receipt for every rejected lane, and a privacy proof on each one.
Lessons from routing 100M tokens
100M tokens through the router. Five lessons that changed the engine: mixed-model JSONL, latency-vs-price flips, webhook redelivery costs, idempotency, and model deprecations.
Cut your AI bill without changing your prompts
Five concrete moves that cut LLM cost 30–50% with no prompt changes, no model swaps, no quality regressions. Just smarter routing.
Routing a JSONL batch in under 50ms
A JSONL file hits POST /v1/quotes/model. ~50ms later, the client has an estimate with a routing explanation. Here's exactly what happens in between, and where the time goes.
OpenAI Batch vs Anthropic Message Batches: a routing-first comparison
Same discount. Same clock. Different shapes, different limits, different status names. Here's an honest, routing-first comparison of OpenAI Batch and Anthropic Message Batches.
Usage-based receipts: the missing billing primitive for agentic commerce
Charge-on-checkout assumes the buyer knows the price. Monthly invoices assume the buyer trusts the seller. Neither works when the buyer is an agent. Usage-based receipts do.
Why we built Batchrouter
Async AI inference is roughly half the price of sync. Yet most teams still pay sync rates. The reason isn't ignorance — it's operational tax. Here's how we attack it.