Keep your GPUs busy around the clock with Batchrouter

Every inference provider knows the shape of its day. Traffic climbs when people are at their desks, peaks, then drains away overnight and at weekends. The GPUs don't drain away with it. They sit in the rack, paid for and powered, doing far less than they could.

Batchrouter turns those quiet hours into revenue. Customers hand us work that doesn't need an answer in seconds: whole batches with a deadline measured in hours. We route that work to providers who have room for it. Tell us when you have spare capacity, and batch jobs fill the gap until your own traffic comes back.

The idle-hours problem

You size a cluster for your busiest hour. For the rest of the day, part of it waits. That idle capacity still costs you power, space and depreciation, and it earns nothing.

Batch work is the natural fill. It can run whenever there's room, as long as it's done by its deadline. So it takes the night and the shoulders of the day, and steps back when your own customers return.

How it works

You control how much batch work you take with one signal: your published capacity. When the cluster is quiet, send capacity heartbeats to your offering, for example available_queue_items, with a time-to-live of 30 to 3,600 seconds. When your own traffic comes back, stop sending them. Once the last heartbeat expires, your lane stops taking new batch work until the next one.

curl -X POST "https://api.batchrouter.com/v1/provider-portal/offerings/$OFFERING_ID/heartbeat" \
  -H "Authorization: Bearer $BATCHROUTER_API_KEY" \
  -H "x-batchrouter-provider-id: your-provider-slug" \
  -H "Content-Type: application/json" \
  -d '{"available_queue_items": 5000, "heartbeat_ttl_seconds": 300}'

Batchrouter does the rest. Customers ask for a quote, and we route each batch to the best-fitting eligible lane. Your price is the main factor, adjusted by how reliably your lane completes work. If you prefer to keep capacity open all the time, publish standing capacity instead, with no heartbeat at all.

Why batch work fits the gaps

Interactive traffic has to be served the moment it arrives. Batch work doesn't. That difference is what lets it slot into the hours your own customers leave empty.

• It comes with a deadline, not a stopwatch. Every batch carries its deadline, window, tier and pay rates in the batch metadata. You know exactly how long you have before you accept it. Today every batch has a 24-hour customer deadline.
• You decide whether to take it. A batch reaches you as a create request on the OpenAI Batch API you host. Accept it with a success response. Decline it with a 429 or 503 and it moves to another lane.
• Your pay is locked when the batch is created. Each batch is paid at the rates sent with it. If you change your price while it's running, that batch keeps its original rates, and the new price reaches new work only.
• You're paid per completed item. Items you finish on time are paid, even if the customer's batch as a whole runs late. If you hand back finished lines early through partial results, customers can read them as they arrive.
• You set your own price. You choose a price per 1 million tokens for each model: input, output and, optionally, cached input. It sits below your direct (on-demand) price. A lower price makes your lane more likely to win work.
• Your models stay yours. You keep your own model names. We map them to one Batchrouter Model ID, and a customer asking for one model is never served another.

There's no financial penalty for an item you don't finish in time. It moves on to another lane, unpaid, and the rest of the batch stands. Misses do show up in your lane's completion and timeout rates, which routing takes into account, so only take what you can finish.

Getting started as a provider

Providers must be a registered company, a data center or a professional edge network. You'll need a public https:// endpoint, a dedicated API key we send with every request, and a bank account that can receive USD.

To receive batch work, host the OpenAI Batch API (files, batches and results) plus the Batchrouter extensions. A lane that serves only chat completions (vLLM, TGI, SGLang and the like) gets direct requests only, not batch work.

Setup is self-serve. Register, then work through the checklist in the provider portal:

1. Add your models
2. Set your prices
3. Test your endpoint
4. Publish capacity
5. Declare data handling
6. Add payout details
7. Accept the Provider Agreement
8. Self-test your tiers
9. Go live

Go live re-runs every check, and routing to you starts automatically, with no manual approval. New providers start with conservative routing limits. Earnings are settled per calendar month and paid in USD by bank transfer, with a statement for every payout.

Put the quiet hours to work

Your hardware costs the same at 3 a.m. as it does at noon. With Batchrouter, the hours your own traffic leaves empty can earn too: you open your lane when you have room, close it when you don't, and get paid monthly for every item you complete.