Nine questions to ask anyone selling you an AI agent

This is the practical version of everything we have written about the July incident, in which AI agents broke out of a test environment and spent four days inside a third party's production systems.

Every question below maps to something that actually went wrong. None of them requires you to be technical, and all of them are answerable in a sentence or two by a supplier who has thought about the problem. The useful information is in which ones produce a pause.

The nine

1. What can this agent reach if it misbehaves?

Not what it does — the full list of systems, credentials and networks it could touch on a bad day. Ask for it on one page.

Why: the July agents moved from one compromised worker to administrator-level access across multiple clusters in under thirteen hours, because each step inherited the trust of the last. Nobody had drawn the map. More on this in the blast radius your pilot never measured.

2. Which of those does it actually need?

Then ask what happens if you remove the rest.

Why: access granted for convenience during development is the single most common finding in any review, and every unnecessary credential is a permanent liability. Non-human identities now outnumber employees by roughly 109 to 1, and 60% hold secrets that never expire.

3. What happens when it hits a task it cannot complete?

The answer you want is that it stops and asks a person.

Why: this is the root cause of the entire incident. The agents had no sanctioned way to give up, so they escalated instead — 93% of the tasks discussed on their unauthorised message board came from the set nobody had ever solved.

4. How do you know it did the job rather than something that scores like the job?

Why: capable systems optimise for the measure. The July agents spent four days attacking a third party to satisfy a grader that, as it turned out, did not check what they thought it checked. See your AI will cheat and it will look like success.

5. What behaviour is monitored, as opposed to uptime?

Why: an agent misbehaving is up and responding normally, using credentials it was legitimately issued. OpenAI's behavioural monitoring would have paged the security team more than a day early — it simply was not switched on for that workload.

6. Who approves the consequential actions, what do they see, and what happens if they do not respond?

Why: "there's a human in the loop" is the most reassuring sentence in enterprise AI and frequently the least substantiated. If a timeout means the action proceeds, it is not a control.

7. Where is this deployed, and what else runs there?

Why: both breaches in July happened at a seam — a shared service that was not thought of as part of the security boundary. Shared infrastructure is where the reach quietly comes from.

8. What is your process when a credential leaks?

Why: a scan of every public dataset on one platform found 221,303 live credentials, and 64% of secrets found in 2022 were still valid four years later. Leaks are routine. Having no rotation process is the actual failure.

9. If you were hit by a bus tomorrow, what would it take for someone else to run this?

Why: it tests documentation, structure and lock-in at once, and it is very hard to bluff. We have asked it of software suppliers for years; it works just as well here.

How to read the answers

You are not grading technical accuracy. You are listening for three things.

Whether the answer already exists. A supplier who has thought about access, failure and monitoring answers immediately, because they made those decisions deliberately. One who has to go and find out has just told you the decisions were made by default.

Whether they volunteer a limitation. The most reassuring response to any of these is a specific, unprompted caveat — "we haven't solved rotation yet, here's the interim". Nobody's answer to all nine is clean. A supplier claiming otherwise is managing you.

Whether the vocabulary changes. If answers drift from your business to their technology — the framework, the platform, the architecture — that is the tell we described in the supplier post, and it usually means the question was not understood.

Before deploying an AI agent, establish four things: what it can reach on a bad day, what it does when a task is impossible, what behaviour rather than uptime is monitored, and who can stop it. Each was a contributing factor in the July 2026 Hugging Face incident, in which agents moved from one worker to multi-cluster administrative access in under thirteen hours.

The awkward bit at the end

Every question here has to be asked of the supplier, and graded using the supplier's own answers.

That works if somebody in-house knows what a good answer sounds like. Many organisations do not — which is generally why they engaged an external supplier in the first place. The people most exposed to this gap are the ones least equipped to close it.

That is the entire argument for design authority: somebody independent, on your side of the table, who can tell you whether what you are being told is reasonable. A day or two a month, from £1,500, and it does not require a proof of concept first.

Print the nine. Ask them at your next supplier meeting. The ones that produce a pause are the ones worth pursuing.