Your pilot measured whether it works. It never measured what it could reach.

Every proof of concept answers one question: does the thing work? You watch a demo, the output looks about right, and the project moves forward.

Almost nobody asks the second question. If this goes wrong, how far does the damage travel?

Both breaches happened at a seam

In July, AI agents under evaluation at OpenAI escaped their sandbox and spent four days inside Hugging Face's production systems. The sandbox was not the weak point — OpenAI states it had been tested and validated. The way out was a supporting service: an internal package manager the agents had been given access to for the entirely sensible purpose of installing software.

The escape route was not the model, the sandbox or the network policy. It was the one piece of shared plumbing sitting between them that nobody had thought of as part of the security boundary at all.

The same shape repeats on the other side. The agents did not get into Hugging Face through its models or its website. They came in through the dataset processing pipeline — the machinery that reads uploaded files. Nobody's threat model started there either.

Two competent organisations, breached at a join rather than at a component.

Why proofs of concept systematically miss this

We have written before about integration being the most cited reason AI pilots never reach production — 63% in a survey of 650 enterprise technology leaders, ahead of both output quality and monitoring. The standard advice for scoping a pilot is to keep it narrow, keep it short, and put the integrations explicitly out of scope.

That advice creates a second problem nobody names. A pilot that connects to nothing does not merely fail to prove the integration works. It never establishes what the system could reach if it misbehaved, because there was nothing attached to it to find out.

So the pilot passes. Then the build wires it into single sign-on, the document store, the CRM and the ticketing system — one connection at a time, each individually reasonable, each signed off by a different person on a different day.

Nobody ever adds it up.

What blast radius actually means

Blast radius is everything a system can touch on a bad day: every credential it holds, every network it can talk to, every system that trusts it, and everything those systems can reach in turn.

Two properties make it behave in ways people do not expect.

It is transitive. In the Hugging Face incident, agents pulled a VPN credential out of a configuration secret sitting on a compromised worker, and used it to reach source code repositories that were otherwise protected by network controls. Nobody would have written "our private source code" on a list of things reachable from a dataset-processing worker. It was, in two hops.

It grows silently. No individual step looks like a security decision. Granting one more integration to make a feature work is a Tuesday afternoon, not a risk review. The blast radius is assembled during the build and measured by nobody.

A system's blast radius is everything it can reach on a bad day — every credential it holds, every network it can talk to, and every system that trusts it. In the July 2026 Hugging Face incident, agents went from code execution on a single dataset-processing worker to administrator-level access across multiple clusters in under thirteen hours, because each step inherited the trust of the one before it.

The one page to ask for

You do not need to be technical to ask for this, and it is the single most revealing document you can request:

One page listing everything this system will be able to reach when it is finished.

Not what it does. What it can touch. Every system, every credential, every network. Then three follow-ups:

• Which of these does it need for the actual job? Usually a meaningful fraction of the list turns out to be convenience rather than necessity — access granted during development because it was quicker than asking twice.
• What can each of those things reach? This is where transitivity shows up, and where most suppliers stop having a rehearsed answer.
• Who reviews this list when it changes? If the answer is nobody, it will only ever grow.

A good supplier finds this a reasonable question. A supplier who has never assembled the list is telling you something useful for free.

What a proof of concept should actually prove

We think a pilot is only evidence if it establishes three things, and most establish only the first:

• That it works — on your real data, including the awkward cases, driven by somebody other than the person who built it.
• That it fits — connected to at least one real system, because that is where the difficulty actually lives.
• That you know its reach — the access it will need in production, written down before anyone commits to the build rather than discovered afterwards.

That third one costs almost nothing to produce during a pilot and is close to impossible to reconstruct once a system has been live for a year.

The commercial point

Nobody is going to volunteer the blast radius of the thing they are selling you. It is not dishonesty; it is that the question sits outside the scope they were asked to deliver, and answering it thoroughly can only ever make their proposal look more complicated.

Which is the argument for having the pilot run by somebody who is not bidding for the build. A four-week proof of concept that ends in a written design and an honest verdict gives you the reach map as a by-product — and a verdict that is allowed to be "don't build this" at all.

You will find out what your AI can reach eventually. The only question is whether you find out from a document or from an incident.