Medical AI’s Next Test Is the Appointment

The appointment is the product

A useful medical AI could change an appointment before the doctor enters the room. A patient would have time to explain what brought them in, answer relevant follow-up questions and organize concerns that might otherwise surface at the door. The clinician would begin with a clearer account and more room for discussion. The appeal is easy to understand. The harder question is whether the technology actually improves that shared encounter, rather than adding another conversation that someone must review.

Google brought that question back into focus in an October 20 interview about AMIE, its research system for clinical reasoning and dialogue. The discussion follows an October 8 publication in The Lancet; initial findings had already been shared in March. This is a developing research story, not a newly launched medical service. Its significance lies in the move toward testing AI inside an actual care process. A sensible assessment therefore follows the information from the patient’s first message through the clinician’s decisions, including the work and uncertainty introduced along the way.

What the clinical study actually establishes

The published study was a prospective, single-center, single-arm feasibility study at Beth Israel Deaconess Medical Center. English-speaking adults used AMIE before a single-complaint urgent primary care appointment. Of 114 enrolled patients, 98 completed both the AI interaction and the visit. Physicians monitored the conversations in real time, with predefined criteria for stopping them. No safety stops were required. These are encouraging observations within a particular supervised setting, rather than a comparison demonstrating superior care.

The usefulness result needs an equally precise denominator. Clinicians completed post-visit surveys for 60 of the 98 cases; in 44, they had reviewed the AMIE transcript beforehand. They found it helpful for preparation in 33 of those 44 cases and said it might have changed their behavior in 25. The often-repeated 75% figure therefore describes the reviewed subset, not every enrolled patient or every completed visit.

That distinction changes the product question. Producing information and getting it used are separate steps. A future service needs to fit the moment when a clinician prepares, present the important material clearly and make review practical. A useful summary arriving after the appointment has lost much of its intended value. An excellent conversation that nobody has time to inspect may become another record in a crowded system.

The appropriate conclusion is that this approach merits further testing. The study does not establish that consultations became shorter, that health outcomes improved or that the same process would work without live supervision. Those are different claims requiring different evidence. Keeping them separate makes the result more useful: it tells developers and care organizations where progress has been demonstrated and which questions still deserve a direct test.

The handoff needs to preserve uncertainty

The most consequential output may be a small piece of text. A pre-visit summary becomes the point where an open-ended conversation is compressed into something a busy professional can use. Compression requires selection. Which details survive, which are treated as uncertain, and which become the opening frame for the appointment? A fluent summary can feel authoritative even when the underlying conversation was tentative or incomplete.

Consider a hypothetical patient who cannot remember whether a symptom began before or after a medication change. A careful handoff would preserve that uncertainty. A careless one might turn the sequence into a definite timeline. The wording difference is small, but the clinician would receive a different starting point. This example is an illustration of the design problem, not a reported AMIE error or a claim about a particular medical decision.

A useful interface should therefore distinguish what the patient said, what was verified elsewhere and what the model inferred. It should make unresolved questions easy to find and let the clinician inspect the relevant exchange without rereading an entire transcript. The patient should also have a practical way to correct important facts. These capabilities would make the summary an inspectable aid rather than a polished replacement for its evidence.

Evaluation should examine omissions as closely as invented statements. A summary may contain nothing obviously false while leaving out the concern that matters most to the patient. Reviewers could compare the original conversation, the condensed account and what the clinician ultimately needed to ask again. The goal would be to discover whether the AI preserves meaning under compression, including ambiguity and priorities, instead of rewarding brevity or an impressive clinical tone alone.

Count the work on both sides of the chat

Google’s October interview describes the ambition of giving doctors more time for collaborative care. That is an attractive outcome, but it needs a measurement plan that includes the entire workflow. Time spent typing, correcting a summary, monitoring a conversation, opening a separate interface and repeating information in person all belongs in the accounting. Otherwise a task can appear automated simply because its remaining work has moved to someone else.

Patient effort deserves its own measure. Completing a thoughtful conversation at home could feel easier than recalling everything under time pressure in a clinic. It could also become an additional obligation for someone who is tired, worried or uncomfortable with a digital interface. Those experiences should not disappear into a single completion rate. Researchers should ask who finishes, who abandons the process and what alternatives remain available to each group.

Clinician time also has different forms. A few minutes of focused preparation may be valuable if they prevent disruptive searching during the encounter. A shorter visit is not automatically a better one if questions go unanswered. An evaluation could therefore track total staff effort alongside interruptions, correction work and the quality of the discussion. The intended benefit should be defined before deployment, rather than selected afterward from whichever metric looks favorable.

Supervision must remain visible in any economic claim. A system tested with a physician following every exchange cannot inherit the cost assumptions of a service operating without that support. Researchers may eventually identify a different oversight arrangement, but it would need evaluation on its own terms. Until then, the honest unit of comparison is the complete care workflow, including the people whose attention makes the AI interaction possible.

Read the safety endpoint carefully

A safety endpoint describes a defined observation, not every possible failure. Harvard Medical School’s account reports that supervisors identified one hallucination, involving a surgery date, and added clinical clarification in five interactions. It also describes one provider’s concern that a possible diagnosis presented by AMIE may have made a patient anxious. These details can coexist with a finding of no safety stops because the measures capture different things.

The lesson for evaluating future systems is to keep several records rather than compressing everything into a safe-or-unsafe label. Conversation interruptions, factual corrections, patient distress, inappropriate reassurance and downstream consequences answer different questions. Some issues may become apparent only after a clinician reviews the record or after the patient leaves. A favorable result on one measure should direct attention to the next question, not close the investigation.

Oversight itself also needs to be understood. Who is watching, what information can they see, and how quickly can they act? A monitor who reads a transcript later has a different role from someone present during the conversation. Neither arrangement should be described vaguely as having a human involved. The responsibilities, timing and escalation process are part of the intervention being evaluated.

For a care organization, this suggests a practical purchasing discipline: ask vendors to describe the tested workflow before discussing the model’s headline score. A change in patient population, interface or staffing may alter the meaning of the original evidence. Procurement should compare the proposed service with the conditions under which it was studied and identify the differences that require additional validation.

Trust requires a usable way to say no

The same Harvard account notes that patients retained concerns about confidentiality and the chatbot’s honesty and trustworthiness. Those concerns belong near the center of product design. A helpful conversation does not answer who can access its contents, how long they remain available or whether later uses differ from the reason the patient supplied them. The explanation needs to be understandable before participation, not merely discoverable in a long policy afterward.

A practical service should make the AI’s role unmistakable and explain how its output reaches the care team. Patients should know whether a person is monitoring the interaction, when someone will review it and how to correct information. Declining the AI should lead to a usable alternative. Consent has little practical value if the interface makes a person believe that answering a chatbot is the only way to secure ordinary care.

The study’s registered eligibility criteria also limit what can be generalized. They focused on adults with English listed as a primary language and excluded pregnancy-related and psychiatric visit reasons, among other categories. Evidence from this population does not settle performance for children, other languages or those excluded concerns. A deployment plan should name the people it can currently serve and investigate the needs of those outside that group.

Access can be tested through ordinary details: whether a person can pause and resume, use assistive technology, get help with the interface or switch to another channel without starting over. These are proposed evaluation questions, not capabilities established by the AMIE study. They matter because a system that works beautifully for willing, confident users may still create friction for people who need care through a different route.

Video adds evidence, and new ways to lose it

AMIE’s research program is also moving beyond text. An August Google Research report described real-time video consultations using separate agents for conversation, planning and audiovisual perception. Its evaluation used simulated consultations with patient actors. That is a different evidence base from the real-patient text-chat feasibility study. Strong results in a structured simulation cannot simply be attached to a clinical deployment with another interface.

Video could reduce the burden of translating every observation into words. It also creates new questions about what the system actually perceived. A poor camera angle, an interrupted connection or uncertain speech recognition can affect the information reaching the reasoning process. A useful system should make such gaps visible rather than quietly converting an incomplete signal into a confident account.

The multi-agent design adds another handoff to inspect. If one component perceives something and another decides what to ask, evaluators need to understand whether uncertainty survives that transfer. The spoken response must also remain aligned with the system’s current understanding. These are engineering questions suggested by the architecture, not evidence that the published system failed in those ways.

Google’s August report described its nationwide randomized study with Included Health as ongoing, while the October interview again emphasized broader controlled comparisons. The research direction is important, but an ongoing study is not a completed result. Readers should look for the eventual design, participant mix and measured outcomes, rather than treating a larger planned scale as proof of a larger benefit.

What a convincing next result would look like

The next compelling result would connect technical performance to a clearly defined care benefit. That might be a more complete patient history with less repeated questioning, less total preparation work without poorer communication, or better handling of uncertainty before a visit. Researchers should choose the intended benefit explicitly and compare against a relevant existing workflow. A favorable satisfaction survey would complement that evidence, rather than substitute for it.

The report should also show where the process breaks down. How often was the summary unavailable, ignored or corrected? Which patients needed another route? What work did staff perform outside the main interface? Publishing these details would help other organizations judge whether they can reproduce the benefit with their own staffing and patients. Reporting the average alone would leave too much of the implementation unexplained.

Funding and independence deserve attention as well. The feasibility study was funded by Alphabet. That does not invalidate its observations, but it reinforces the value of transparent methods and evaluation by other teams. Buyers should be able to distinguish a developer’s account of what its system might enable from evidence that a particular service reliably delivers that benefit.

AMIE’s early clinical work is valuable because it gives these questions a concrete setting. The promising future is an appointment in which patients feel better understood and clinicians have more useful information with less avoidable work. Reaching that point requires careful handling of summaries, honest accounting of effort and evidence that follows the entire encounter. Medical AI will earn a place through the quality of care it helps people deliver together.

• AI Agents Need Boundaries They Cannot Rewrite
• AI Is Becoming a Scientist