How do we keep people in control when AI makes recommendations?

What it is

As AI recommendations get better, the main risk shifts to people no longer checking them.

• Research. A 2010 review in Human Factors by Parasuraman and Manzey found that automation bias leads to errors of both omission and commission when decision aids are imperfect. It affects experts as well as novices, teams as well as individuals, and can’t be prevented by training or instructions alone.
• A practitioner view. Pascal Bornet adds that relying on AI that’s usually right can erode the judgment people build by deciding for themselves.

How it works

1. Classify decisions by impact and reversibility.
2. Set the AI’s role for each: decide, recommend or assist.
3. For important decisions, have people form their own view before seeing the AI’s recommendation, as Bornet suggests. Then require a short written reason for accepting or rejecting it.
4. Audit a sample regularly, rotate who checks and track override rates.

Benefits

• Keeps accountability clear.
• Catches drift in AI quality.
• Keeps staff skills sharp.

Limits and risks

• Extra steps slow work, so reserve them for important decisions.
• Zero overrides can mean great AI or disengaged reviewers. Investigate which.
• Because training alone doesn’t fix automation bias, the review design matters more than reminders to “be careful”.