Terry Mills Ph.D.Start a conversation

Chapter 23

Human-in-the-Loop Is Not Enough

A person clicking approve is not governance.

The phrase “human in the loop” sounds reassuring. It suggests that a person remains in control. But simply placing a person near an automated decision does not create meaningful oversight.

A reviewer who lacks expertise cannot challenge a weak recommendation. A reviewer given thirty seconds to approve a complex decision is not exercising judgment. And a worker whose performance targets reward agreement with the automated system may technically be in the loop while having little practical freedom to disagree.

Meaningful oversight requires five things: competence, authority, information, time, and responsibility.

Responsibility matters because it changes behavior. A 2026 Decision Support Systems study found that people who felt more responsible for AI-assisted work made fewer errors. The study also found a tension worth remembering: as AI appeared more capable, people could feel less personally responsible for the result.

That is exactly when governance should become clearer, not weaker.

For consequential work, name the person who owns the decision. Give that person the evidence needed to challenge the system. Define what must be escalated. Record important assumptions. Preserve the ability to stop or reverse the process when possible.

This leads to a simple rule: the easier a decision is to reverse, the more freedom AI can have. The harder a decision is to reverse, and the greater the harm if it is wrong, the stronger the human authority and evidence should be.

This is the Accountability Premium. When polished recommendations become cheap, recommendations that a credible person or institution is willing to stand behind become more valuable.

Source checks, professional sign-off, clear methods, audit trails, and named decision rights may look like friction. In an AI-rich environment, they are signals of quality.

Responsibility also needs support. Accountability without enough authority, time, information, or a realistic workload can turn into overload. Asking a human to absorb every risk after a process has been automated is not sound governance.

Accountability without agency is not governance. It is liability transfer.

Trust, then, is not a personality trait of the model. It is an outcome of the system around it.

Higher automation can also change how people attribute responsibility. Recent experimental work finds that as automation increases, users may process less information and shift responsibility toward the system. That makes passive oversight especially dangerous when the AI appears highly reliable.

Meaningful governance therefore has to preserve engagement, not merely human presence. The reviewer should know what evidence to inspect, what uncertainty matters, what conditions require escalation, and what authority the reviewer actually possesses.

Reversibility becomes actionable when it is translated into decision classes. A low-cost draft can be regenerated, compared, or discarded, so autonomy can be broad. A hiring decision, grant award, clinical recommendation, disciplinary action, or public commitment carries consequences that may persist after an error is discovered. Those decisions need stronger evidence, explicit authority, and a named human who can stop the process. The principle is not 'more human review everywhere.' It is governance proportional to consequence and recoverability.

The ceremonial human

Many governance diagrams contain a human box near the end of the process. That box can create false comfort. If the reviewer lacks expertise, time, evidence, authority, or permission to disagree, the human is present but functionally decorative. Clicking approve is not the same as governing.

Meaningful oversight begins by matching authority to accountability. A person expected to own a decision needs access to the relevant evidence and the practical ability to stop, revise, or escalate the process. Otherwise the organization has assigned blame without assigning control.

Reversibility as a design rule

A useful governance principle is reversibility. Low-consequence, easily reversible decisions can tolerate greater machine autonomy. As consequences increase and reversal becomes harder, the burden of evidence and human authority should rise. This avoids the impossible goal of treating every AI interaction as high risk while still protecting the decisions that deserve serious oversight.

Authority before approval

Human oversight fails when the designated reviewer lacks the authority to change the outcome. This can happen subtly. An employee may technically be allowed to reject an AI recommendation but know that doing so will delay the process, upset a senior leader, or require documentation no one has time to produce. The human is present but structurally discouraged from exercising judgment.

Meaningful governance therefore begins before the approval screen. Define which decisions AI may make, which it may recommend, which evidence the reviewer receives, what uncertainty must be disclosed, who can escalate, and what happens when the human and system disagree.

Reversibility helps calibrate the design. A draft social post can be revised before publication. A denied benefit, terminated employee, medical intervention, or irreversible financial commitment may impose costs that cannot be repaired by simply generating another answer. As reversibility falls and consequence rises, the required human authority and evidence should increase.

This principle also prevents governance theater. The goal is not to maximize the number of approvals. Excessive approvals create fatigue and encourage rubber-stamping. The goal is to place competent human judgment at the points where it can actually alter consequential outcomes.

A human can be present and absent at the same time A reviewer who has three seconds to approve a recommendation is technically in the loop. A junior employee who lacks authority to challenge a system is technically in the loop. A physician who receives an algorithmic recommendation without access to the relevant evidence may be in the loop. None of these arrangements necessarily constitutes meaningful oversight.

This is why governance should be described in verbs rather than labels. Can the human inspect? Can the human question? Can the human override? Can the human escalate? Can the human delay? Can the human explain? Can the human be held responsible only for decisions over which the human had genuine control?

Governance is a distribution problem

Recent human-AI governance research increasingly treats governance as a distribution of decision authority, process autonomy, and accountability rather than a binary choice between human and machine control. That is a better model for real organizations. Different stages of one workflow can legitimately assign different degrees of autonomy.

A system may autonomously classify low-risk documents, recommend action on medium-risk cases, and be prohibited from acting on high-risk cases without named human authorization. The important design question is not whether humans are somewhere in the process. It is whether authority matches consequence.

Counterargument: humans are fallible too

Calls for human oversight can become sentimental. Humans are biased, tired, inconsistent, political, and sometimes less accurate than automated systems. Requiring a human decision can reduce quality if the human lacks expertise or overrides a stronger system for bad reasons.

The answer is not human supremacy. It is comparative governance. Assign each part of the decision to the actor, human or machine, best suited to perform it, then design checks for the predictable weaknesses of both. Human judgment is not valuable because humans are perfect. It is valuable when humans contribute context, legitimacy, responsibility, and forms of reasoning the system does not reliably possess.