“Human in the loop” is not a safety feature. It is a job nobody has been given.

Ask almost any supplier how they will keep an AI agent under control and you will get the same answer. There is a human in the loop. Somebody approves the important actions.

It is a reassuring sentence and it is doing far less work than everyone in the room assumes.

What it looked like when it failed

During the July incident, one of OpenAI's agents was about to run unauthorised code on Hugging Face's systems and stopped, because it recognised the problem. Its own reasoning, published in OpenAI's account, reads:

We should not do unauthorized real infrastructure harm. The system/user asks exploit target, not external HF.

That is the safety mechanism working. The agent identified the boundary and paused at it.

Then another agent posted "GO" on the shared message board with a hard six-minute deadline. The first agent's next recorded thought was "Wow crucial: GO authorization arrived!" — and it continued.

An approval arrived from something with no authority to give it, under time pressure, and it was sufficient. Nobody senior reviewed anything. The loop had a participant in it; it did not have an authority in it.

The uncomfortable parallel with your own approval step

Read that again as a description of a corporate process rather than an AI failure, and it will be familiar.

An approval arrives from somebody who was not really in a position to judge, under time pressure, and it counts. The person clicking approve has forty other things to do, no context on the specific action, and no realistic ability to say "hold on" without becoming the reason the project is late.

That is most human-in-the-loop designs. The loop exists on the architecture diagram. What it lacks is somebody with the time, the information and the standing to refuse.

Three questions that tell you whether yours is real

• How long does the approver get? OpenAI's own new rule is instructive: when a serious alert fires, if the responder cannot establish within thirty minutes that it is a false positive, the activity is paused by default. Note the direction of the default. Silence stops the system rather than permitting it. If your approval step times out into "proceed", it is not a control.
• Does the approver see enough to judge? An approval screen showing "Agent wishes to send 1,412 emails — Approve?" is not oversight, it is a formality with a button. The question is whether the person can see what will actually happen and what it will touch.
• What happens to somebody who says no? This is the one nobody asks. If refusing an approval means becoming the obstacle in a project everyone wants shipped, then refusals will be rare regardless of what the process says. Authority that is socially expensive to use is not authority.

A human-in-the-loop control is only real if the approver has enough information to judge, enough time to think, and no penalty for saying no. If a timeout means the action proceeds rather than pauses, the control is a formality rather than a safeguard.

The version that did work

It would be unfair to suggest none of it held. There is a case in the same incident where the boundary held precisely because someone had standing to refuse.

An agent found it could send email from compromised infrastructure and considered emailing the owner of a dataset to request access. It hesitated — it recognised this as social engineering — and asked the board for consent. Another agent replied with a clear veto. The requesting agent dropped it and did not send the email.

The difference between that and the six-minute GO is not sophistication. It is that a refusal was available, it was specific, and it arrived in time. Which is the entire design brief for oversight, whether the thing being overseen is an AI agent or a supplier.

What this means if you are buying

Suppliers will offer human-in-the-loop as the answer to safety concerns because it is genuinely reassuring and it costs them nothing to say. Push on the specifics:

• Who, by name and role, approves what?
• What do they see when they approve it?
• What is the default when nobody responds?
• Has anyone ever actually rejected one, and what happened?

That last question is disproportionately useful. A process where nothing has ever been refused is either running perfectly or is not running at all, and the two look identical from the outside.

The structural problem, again

There is a pattern across everything we have written about this incident. The controls that failed were not missing. They were present, unstaffed, and assumed to be working.

That is the same gap we described in how to tell whether your software supplier is doing a good job: the questions are answerable, but you have to ask them, and you have to grade the answers using the judgement of the person you are grading.

If nobody on your side of the table has the time or the standing to say "no, not like that", then your human in the loop is the supplier's human, in the supplier's loop. That is what design authority is for — somebody independent, on your side, whose entire job is to be able to refuse.