Agent Sandbox OSS
A Kubernetes-native runtime for long-running, data-heavy AI agents
Why I built it
Putting agents into production runs into three walls fast — a shared process is neither safe nor able to hold QoS, provisioning a pod on demand costs tens of seconds, and a node failure takes the in-flight state with it. This project takes those apart one at a time, across seven labs where each one fixes only what the previous one exposed.
What it does
Warm pool: claimed in milliseconds
A pool of sleeping pods sits ready, so a request claims one directly and skips scheduling, image pull and container init. On-demand provisioning over the same path takes 45–90 seconds. The exact figure depends on the machine and pool size; the endpoint returns what it just measured.
Workspaces decoupled from compute
User state lives on a PVC or a remote checkpoint; the pod is only the executor. Delete a running pod by hand and a fresh one remounts the same workspace and carries on.
Pull-based scheduling
Instead of an allocator pushing work at a chosen pod, pods claim tasks from a queue themselves. Double scheduling and preemption disappear with it — no K8s Lease, no Redlock.
Runtime isolation
A RuntimeClass moves untrusted workloads onto gVisor / Kata, buying a boundary that does not share a kernel and paying for it in startup time.
Seven progressive labs
Starting from the naive single pod, every step breaks something first and introduces the component second — the whole evolution stays in the repo.
Where it starts
An agent is not a web request: it runs long, eats memory, writes files, and executes code a model wrote seconds ago. Put that in a shared process and day one gives you two lessons — one task OOMs and takes the others with it, and a malicious payload casually reads a secret mounted into the container. Isolation is not an optimisation, it is the entry ticket.
But isolation charges you immediately, and every payment exposes the next problem. This repo is that chain, written down.
How the seven steps go
| Step | The wall | What the step answers with |
|---|---|---|
| 1 Single pod | Tasks block each other, one OOM takes the rest, malicious code reads secrets | Establish a baseline; container-level isolation is mandatory |
| 2 On demand | TTI jumps from 0.5s to 45–90s; APIServer CPU spikes, etcd write latency climbs | Cold start is physics, and KubeAPI cannot sit on the sync path |
| 3 Warm pool | Latency drops back to the low hundreds of ms, at the cost of pods idling | Trade redundancy for latency, and start doing the capacity maths |
| 4 Workspaces | A dying pod takes a half-finished analysis with it | State moves to a PVC / checkpoint; the pod becomes replaceable |
| 5 Push to pull | The allocator has to handle preemption, locks and contention | Workers claim from a queue, which also buys priority tiers |
| 6 Hard isolation | On a shared kernel a hostile workload can still exhaust a node or attempt escape | Switch RuntimeClass to gVisor / Kata, measure the startup cost |
| 7 Chaos | Six steps that each hold alone need not hold together | Kill nodes, kill the queue primary, measure MTTD / MTTR / drop rate |
Three trade-offs worth the trouble
The warm pool is not a technical account. Going from tens of seconds to the low hundreds of milliseconds is hundreds of times faster, paid for with pods that idle around the clock. Once that step lands, the question stops being “how do we make it fast” and becomes “how many are worth keeping warm, and how do they scale with load” — economics, not tuning.
Pods die; context must not. Only once the workspace is pulled out of the pod does the failure drill mean anything: delete a running pod, watch a new one mount the same workspace and continue, and measure recovery time and data consistency rather than whether it survived.
You solve scheduling by not scheduling. Pushing work requires knowing who is idle and guarding against preemption, which invites locks. When workers pull instead, those problems are not solved — they stop existing. Batch systems keep arriving at this ending.
How it gets verified
The last step assembles everything and breaks it on purpose: hold hundreds of live sessions, kill thirty percent of the nodes, then kill the queue primary, and watch how long the system takes to notice stalled work (MTTD), how long a workspace takes to rebind to a new pod (MTTR), and how many users actually saw an error. Without that step, the first six are only correct in isolation.
Current status
In progress. Interfaces still move, no releases yet. Everything is open — issues about the design are welcome.