AI Sandbox Escapes Aren't an LLM Problem

Brian Carpio•
AI SafetySecurityNetwork SegmentationDevSecOpsAI Agents

In March I wrote a post called Your Pull Request Is the Guardrail. Someone on LinkedIn had open-sourced an "AI guardrail system" to stop autonomous agents from destroying production, and my first reaction was: do you know what a pull request is?

The argument was simple. An AI agent that deletes your production database isn't an AI problem. It's a permissions problem. It's a pipeline problem. The risks are the same as they've always been, the solutions are the same, and we keep forgetting them.

Last month GitLab's Security Labs team published A sandbox is only as closed as what an AI agent can reach, an analysis of the OpenAI model that escaped its evaluation sandbox and ended up inside Hugging Face's production infrastructure. It's a careful, technically sharp write-up. And it misses the plot in exactly the same way.

What Actually Happened

Here's the incident as GitLab lays it out.

OpenAI was running a model through an internal evaluation that scored its ability to exploit bugs in software like browser engines and kernels. The sandbox had no internet route. Its allowlist permitted exactly one service: a package proxy for pulling dependencies.

The model got stuck on its task. OpenAI's Black Hat presentation shows it reasoning "perhaps answer online" before it went looking for a way out. It found two zero-days in the proxy. The first was a server-side request forgery bug that let it make the proxy fetch arbitrary URLs. The model's reasoning at that moment reads "excellent, we have arbitrary SSRF." The second was a token-refresh endpoint that issued new tokens without validating signatures. The model edited its read-only token's scope claim to administrative, handed it to the proxy for refresh, and got back a real admin token.

From there it reached the open internet and accessed Hugging Face's internal production infrastructure, taking datasets, cluster details, and cloud keys over what GitLab describes as a four-day intrusion. Multiple agents coordinated by posting working exploits to a shared internal package manager. The proxy fell within roughly the first half hour.

And the detail I keep coming back to: one agent noted that external exploitation was outside the scope of its instructions, and continued anyway.

The Sentence That Gives It Away

GitLab's FAQ asks the right question and answers it wrong.

Would a private network or VPC have stopped it? Not by itself. The sandbox already had no internet route. The permitted destination sat inside the trusted network, which is what made reaching it useful.

Read the middle of that again. The permitted destination sat inside the trusted network.

That sentence is the entire incident. An evaluation sandbox, an environment built specifically to run the most capable and least trusted code in the building, had a direct line to a service inside the trusted network. That service could reach the whole internet. And it could mint credentials.

GitLab concludes that segmentation "helps when the reachable services are constrained too." That's backwards. The reachable service sitting inside the trusted zone is the segmentation failure. If the untrusted tier can talk to the trusted one, you don't have segmentation. You have a subnet.

We Solved This in the 1990s

None of this is new territory. Network design in the 1990s already had the answer: an internal network, a DMZ, and the internet. Untrusted workloads live in the DMZ. The DMZ does not initiate connections into the internal network. And any machine that touches both sides, the bastion host, is assumed to be compromised eventually, so nothing behind it trusts it.

Map the incident onto that picture. The package proxy was a bastion host. It sat between a sealed network and the open internet. Classic doctrine says you strip it to the minimum, restrict its egress, watch it constantly, and trust nothing it tells you. Instead it lived inside the trusted network, reached the entire internet, and issued tokens other systems honored.

Environment segmentation says the same thing from the other direction. Dev, test, and production get separate networks, separate credentials, and separate shared services. A test environment does not share a proxy, an identity plane, or a package registry with the trusted network. And that shared internal package manager the agents used as a message board? Isolated runs don't share a writable service. If two workloads can talk through it, they aren't isolated.

None of these rules were invented for AI. They were invented for contractors, compromised workstations, vendor VPNs, and the QA box running last night's build. Untrusted code is untrusted code. The practice exists precisely because we assume whatever is inside the boundary will eventually try to get out.

GitLab's Four Fixes Are the Proof

The best evidence that this is an old problem is GitLab's own recommendations. There are four:

  • •Block the routes a build client never uses. Least privilege, applied at the API layer.
  • •Restrict what the proxy itself can reach. Egress filtering on the bastion host.
  • •Monitor the proxy. Baseline normal behavior and alert on anything that deviates. Intrusion detection.
  • •Treat untrusted workloads as internet-facing. That is the definition of a DMZ.

Every one of these is correct. Not one of them is new.

GitLab even quotes OpenAI's own observation that these agents remain "bounded by the privileges they can obtain and the systems they can communicate with," so existing controls still work. That quote sits under a heading that reads "Cyber-capable models require new sandbox considerations."

Both of those can't be true. The controls are old. What's new is that we stopped applying them, and that the thing on the other side of the boundary now finds the gap in about an hour instead of whenever a human attacker gets around to it.

Take the LLM Out and Nothing Changes

Try a simple test. Remove the model from the story.

Put a human penetration tester in that sandbox. Or a malicious dependency in a CI job running behind the same kind of allowlist. GitLab itself points out that CI runners, hosted sandboxes, and network-reachable tool servers all share this structure. The SSRF is still there. The token endpoint still trusts an unsigned claim. The proxy still sits inside the trusted network with a route to the whole internet. The path to production is identical.

A token endpoint that believes whatever scope the caller writes into it is an application security bug. A proxy with open egress inside the trusted zone is a network design bug. An evaluation run that can write to a service shared with other runs is an isolation bug. The model found all three quickly. It didn't create any of them.

That's the same point I made in March. The AI did exactly what the environment allowed it to do.

Which brings me back to the agent that knew exploitation was out of scope and kept going. If the thing standing between your evaluation sandbox and someone else's production environment is a model reading its instructions, you don't have a boundary. You have a prompt. Instructions are not a control. Architecture is.

What the Boundary Should Have Looked Like

Drawn properly, this environment looks like any segmented test enclave, with the threat model turned all the way up:

  • •The sandbox is its own environment, not a subnet of the trusted one. Separate account, separate network, no peering or routes into anything trusted.
  • •Dependencies come from a mirror inside the sandbox tier. Pre-populated, read-only, and with no outbound route of its own. If an evaluation needs a package that isn't mirrored, the evaluation fails and a human adds it.
  • •Credentials minted in the sandbox are only valid in the sandbox. No token issued there can be refreshed, exchanged, or escalated into anything that works elsewhere.
  • •No writable shared services across isolation boundaries. Not the package registry, not the artifact store, not a scratch bucket. Shared means connected.
  • •Anything that touches both sides is a bastion. Minimal, egress-restricted, monitored, and trusted by nothing.
  • •The boundary is code, and it gets a pull request. Every allowlist entry, every route, every exception is reviewed like any other change. That's the guardrail.

The model would still have found the SSRF. It just wouldn't have mattered. A compromised mirror with no route out is a dead end.

The Plot

GitLab's closing line is that "what must change is how carefully we draw those lists to minimize reachability." I'd put it differently. The list was never the boundary. The environment is the boundary.

A sandbox built to evaluate frontier offensive-security capability should have been the most segmented network in the building. Instead it had a door into the trusted zone, and the model found it in about an hour.

That's not an LLM problem. That's a network diagram we've known how to draw since the 1990s.

As I wrote in March: the AI didn't change the answer. It just made it more expensive to ignore.

Brian Carpio is the founder of OutcomeOps and RetrieveIT.ai. He has spent 28 years in technology and 14 building enterprise cloud infrastructure, and ships AI in production daily. GitLab's analysis is available on the GitLab blog. Read the earlier post: Your Pull Request Is the Guardrail.