Human-in-the-loop

What it is

IBM defines human-in-the-loop (HITL) as a system or process in which a human actively takes part in the operation, supervision or decision-making of an automated system. In practice that means approval steps, review queues or correction feedback.

How it works

1. Map the workflow and mark the steps with real consequences, such as sending, paying, deleting or publishing.
2. Require human approval at those steps and let low-risk steps run automatically.
3. Feed corrections back into prompts, docs or evals.

In the loop vs. on the loop (IonBin’s framing). In the loop means a person approves each action before it happens. On the loop means the agent acts and a person monitors and can stop it. On the loop suits low-risk, reversible work such as tagging or drafting. Anything that sends, pays, publishes or deletes should stay in the loop: reviewing a message after it has gone out is not a control. This matters most for background agents that run while nobody is watching.

Simon Willison frames much of agent design as this kind of permissions work. For example, give a coding agent a key that works in staging but not in production.

Benefits

• Catches errors before customers see them.
• Keeps accountability with a named person.
• Builds the track record needed to automate more later.

Limits and risks

• Rubber-stamping. Pascal Bornet argues that when AI is right 99 times in a row, reviewers can lose the judgment the check depends on. “Human-in-the-loop should not become human-clicks-approve.” His suggestions include forming your own view before seeing the AI’s recommendation and keeping people close to genuinely ambiguous cases. This is a practitioner view, not a study.
• Too many approvals erase the time savings.
• Ownership. Approval steps need clear owners and deadlines.