Riven Spot Broker

Buys interruptible GPU capacity when the fleet needs to grow, and drains it gracefully when the market takes it back.

Owned hardware is sized for normal load, not for peaks. Spot Broker handles the difference by renting interruptible GPU capacity — cheap, plentiful, and liable to disappear with little warning.

What it does#

• Watches demand — sustained queue depth from the Fleet Scheduler is what triggers acquisition, not a momentary spike.
• Shops across suppliers — compares price and availability per GPU class and region before committing.
• Brings capacity up — provisions a worker, loads weights, waits for health, and only then registers it as schedulable.
• Handles preemption — on a reclaim notice, stops taking new work, lets in-flight requests finish where possible, and deregisters before the instance dies.
• Releases capacity — returns instances once demand subsides, so burst cost stays proportional to burst load.

Preemption is expected, not exceptional#

Spot capacity is treated as unreliable by construction. A spot worker is never the only home for a model, and it is drained rather than killed whenever the provider gives notice. Requests already in flight either complete or are retried on another pool by Ultra Router — preemption should not be visible in a response.

Economics#

The broker's decisions are priced against the alternative: serving the same request through a cloud provider at list rates. Renting a GPU is worth it when sustained overflow is cheaper than per-token fallback, and Pricing Brain is what supplies that comparison.