A container per task fixes isolation, and leaves the user waiting 45 seconds

The previous post ended here: with every task crammed into one program, one failure takes down all of them, and anybody’s code can read everybody’s data and keys.

The fix is natural — give every task a container of its own. Throw it away afterwards; nobody touches anybody.

On Kubernetes this is almost embarrassingly easy to write. Easy enough to make you wonder whether the last post was overreacting.

The code: a task arrives, a container gets built

# request comes in → ask K8s to build a new container
v1.create_namespaced_pod(namespace="default", body=pod_manifest)

# then keep asking "ready yet? ready yet?" — this is the wait the user sits through
while True:
    pod = v1.read_namespaced_pod(name=task_id, namespace="default")
    if pod.status.phase == "Running":
        break
    time.sleep(0.5)          # asking too often overwhelms the management system

return {"time_to_interactive_seconds": round(time.time() - start_time, 2)}

The isolation is genuine: one container per task, a memory overrun kills only its own task, keys are no longer shared. Both walls from the last post really are gone.

Then you look at that last line — it records how long the user waited between pressing the button and any work starting.

From half a second to 45–90 seconds

That is not slow code. There is simply a queue of errands to run first:

Where the seconds go	Who is busy	Can it be optimised
Registering the new container	the cluster’s ledger writes it down and confirms it stuck	Barely
Picking a machine for it	the scheduler filters, scores, selects one	Somewhat
Downloading the program image	that machine pulls the package from a registry	Pre-pulling helps, for images you know about
Creating the container, wiring the network	the runtime starts it, the network plugin assigns an IP	Barely
The program starting itself	Python boots, dependencies load	Somewhat

The usual first instinct is to blame image size. But an image is cached on the machine after the first pull, and it still never gets fast enough to be acceptable — the bulk is register, place, initialise, which is the inherent rhythm of the system rather than your code being slow. That is why “optimise the image” runs out of road: it optimises the one slice that happens to be optimisable.

The “keep asking” loop is the worse half

Look at the comment: asking too often overwhelms the management system. The moment that line gets written, the problem has changed shape.

An analogy. K8s has two halves: an administration office that registers things, approves them, and hands out resources, and a factory floor where work actually happens. Creating a container and checking its status are both errands at the administration office. And an office like that is designed for internal paperwork — unhurried, retryable, fine if it takes a moment.

What this code does is walk every single customer up to the office window, and nag every half second.

One customer is fine. A crowd is not:

• every waiting request checks twice a second, so 100 people at once is 200 queries a second;
• every new task must be registered, and registration means writing to the central ledger and confirming it stuck;
• the office falls behind first, ledger writes slow down next — and once those two slow down, the whole cluster does, including workloads that have nothing to do with your agents.

The line worth keeping: the interface for managing the system is for management, not for user clicks. Cold start is only the first invoice; dragging down the entire cluster is the real risk.

What this step settles

Two things hold at once:

• isolation was right, no need to walk it back;
• building on demand is the wrong timing — you cannot start construction once the guest has arrived.

Which forces the next move: shift the building to before the user shows up. Keep a batch of containers standing by, and let a request claim one.

The wait drops from 45 seconds to a fraction of a second. And that batch of idling containers starts burning money by the minute. The next post does that arithmetic.

In one line: isolation was not the mistake — waiting for the user before preparing was.

Code: agent-sandbox-oss/lab2 — a single file, and running it once beats reading the numbers.