Skip to content

Pools

The model name your code sends, and the routing, budget and lifecycle policy behind it.

Updated View as Markdown
For humans

A Pool is a stable model name with a complete policy attached. Your code sends the slug; the platform decides which upstream answers and what it may spend.

What a Pool fixes

Setting What it decides
Capability Which interfaces the Pool may serve, such as text, embedding or ocr
Provider connection The reviewed credential and endpoint used for the call
Model The upstream model behind the slug
Fallback An optional second provider/model used when the primary cannot answer safely
Safe retries How many confirmed-non-billable attempts may be repeated
Circuit policy Consecutive failure threshold and cooldown before the Pool stops being called
Hard budget The monthly ceiling the Pool may settle against
Rate limit Requests per minute for the Pool
Status active or paused

Every one of these travels together. Updating a Pool writes the whole policy atomically and uses optimistic conflict protection, so two operators cannot silently overwrite each other.

The slug is the model

GET /v1/models returns the Pools a key can call, and the Pool slug is the value your client uses as the model name. Vendor model ids never appear in your configuration or your requests.

Lifecycle

  • Pause stops traffic immediately. Callers receive an error rather than a silent reroute.
  • Resume is allowed only after entitlement and the provider connection pass revalidation.
  • Delete ends the slug. Keys granted to it lose that grant with it.

Budgets and limits

Each accepted request reserves against the Pool’s hard budget before the upstream is called, and settles the confirmed cost after it returns. A request that would exceed the remaining budget is refused with 429 instead of being served and billed later. Rate limits are enforced per Pool.

See Usage and billing for the money view and Requests for the exact error envelope.

Type to search…

↑↓ navigate↵ selectEsc close