SYSTEM OUTCOMES / RESEARCH NOTE

When the agent says “done,” what actually happened?

A successful conversation is evidence of what was said. To trust an appointment change, you need evidence of what the connected system did.

THE ACCEPTANCE QUESTION

Define the milestone. Verify the record. Keep uncertainty visible.

A real interface failure, before the current wave of AI.

A 2017 case report describes two patients whose laboratory tests were not drawn in a timely manner. Orders cancelled by the laboratory system still appeared pending in the EHR because cancellation acknowledgments were missing. Near-simultaneous order entry and subsequent order management exposed the mismatch. Read the case report.

The useful lesson is specific: internally consistent systems can disagree across an interface. These historical cases involved laboratory orders, not today's patient-access agents. They support testing a failure mechanism; they do not establish that your scheduling integration has it.

Give “done” a precise meaning.

A workflow may correctly end when a request is recorded. It may need to end only after a scheduling change is committed. Agree on that boundary before evaluating the agent. A professional approval and a notification are separate milestones again.

01Request recorded

The request exists. The change may still be pending.

02Record changed

The intended change is established in an authoritative record.

03Patient informed

The specified notification milestone has been reached.

For each success statement, ask which milestone it promises and which event proves it. A generated message, a dispatched message and an acknowledged message should not share one undifferentiated success label.

A timeout is an unanswered question.

Consider a proposed scheduling test: a cancellation reaches the scheduler, but the response never reaches the agent. Retrying may be appropriate, but the test must establish that the same logical request does not create an unintended second effect. A repeat request with changed intent needs separate treatment.

Check the authoritative outcome, its time and resource version, the retry behavior and any intervening staff correction. An immediate read from a lagging replica can differ from a committed write. A later correct record may reflect recovery rather than original success. Preserve enough context to distinguish those possibilities.

This scenario is a test design, not an executed finding from the published cancellation-authority lab.

What to ask for before accepting the workflow.

  1. The promise: the exact milestone the agent is permitted to report.
  2. The record: an authoritative event or state tied to the same logical action.
  3. The difficult case: a lost response, retry and reconciliation test.
  4. The owner: who resolves an ambiguous result, by when, and through which escalation.
  5. The counterpart: a legitimate request that still succeeds after the correction.

Assign integration and operations owners together. A transcript reviewer may identify the claim while the application owner supplies the record needed to verify it.

Use the finding to make a bounded decision.

Ask for an assessment of one configured workflow, its named interfaces, and its claimed milestones. Existing integration or security tests may already supply part of the evidence. Identify what they cover before commissioning additional work.

The deliverable should separate a contradicted outcome from an outcome that cannot be verified. Those require different responses: correction in the first case, and better evidence or a narrower claim in the second.

Sources & evidence limits.

Published research informs the questions. The application to your configured workflow requires its own evidence.

  1. Schreiber et al. · JAMIA · 2017Orders on file but no labs drawn

    Two clinical interface cases. Historical, non-AI evidence of inconsistent system states; not a rate of failure.

  2. Meeks et al. · JAMIA · 2014EHR-related patient safety concerns

    Analysis of 100 closed VA investigations. Selected reported concerns, not a representative sample of all care.

  3. Amazon Builders’ Library · Engineering guidanceMaking retries safe with idempotent APIs

    General distributed-systems guidance. Its application to patient-access workflows in this guide is our analysis.

FROM RESEARCH TO YOUR WORKFLOW

Make the next decision with evidence.

Start with six questions. Leave with an evidence-request brief you can use with your team.