A human in the loop needs a way to check.

The difficult part may be verification itself.

Lyell and Coiera reviewed automation-bias studies and found that problems were not confined to multitasking: some single-task settings involved substantial verification complexity. Their analysis points to cognitive load as a research direction, with important limitations in the underlying evidence. Read the review.

Our application to patient access is a testable question: can a staff member establish whether a claimed action happened without reconstructing the workflow across inaccessible systems? The review did not test that particular question.

Inspect what the reviewer can actually see.

A reassuring summary and a completed-status label may come from the same unverified assertion. Giving a human both does not necessarily add independent evidence. Ask which authoritative record, event or receiving-team acknowledgment is available to that person.

Record the permissions, systems, steps, time and assistance needed. A test with a specialist engineer beside the reviewer may not represent the ordinary operating arrangement.

Detection needs a recovery path.

Review four things together: the signal that something needs attention, the evidence required to verify it, the authority to correct or escalate it, and the point at which someone accepts responsibility. A person may have only some of these.

For an appointment issue, correction could require a scheduling owner. For a clinical decision, an appropriately authorized professional may be required. Agree the role boundaries with the institution; do not give a reviewer broader authority just to make a test pass.

Questions for operations, quality and security.

1. What triggers a review, including requests that never produce a success message?
2. Can the reviewer access the authoritative evidence during the relevant shift?
3. What happens when the evidence is incomplete or conflicting?
4. Can the reviewer recover the request or obtain acceptance from the correct team?
5. Does the correction preserve a usable route for the legitimate patient or proxy?
6. Have these steps been observed with the current configuration and staffing assumptions?

Interpret automation research carefully.

In Gaube and colleagues’ experimental study, inaccurate advice reduced diagnostic accuracy regardless of whether it was labeled as AI or human advice. The advice itself was generated by human experts. That makes it useful evidence about advice and reliance, not a measured error rate for a deployed AI product. Read the study.

The assessment opportunity is to inspect the actual oversight arrangement and its evidence. It is not to presume staff will fail, or to promise that additional training alone resolves the problem.