The Value of Physicians in the AI Lifecycle

And physicians' under-recognized contributions to healthcare

Geoffrey Hinton famously declared in 2016 that “We should stop training radiologists now” and that in five years the field would be obsolete. That clearly didn’t happen, but pessimistic assessments of the future of physicians abound. For example, surgeons might be surprised to learn that the Bureau of Labor Statistics considers their field to be at high risk of automation in the next ten years, listed next to ‘fast food and counter workers’. 

These dire predictions about automation of physician roles underscores how poorly understood the holistic, multifaceted role a physician plays in healthcare.

Growth trends for selected occupations considered at risk from automation : Monthly Labor Review

AI now performs extremely well on the USMLE, can identify lesions in images in radiology, pathology, and ophthalmology with startling accuracy, and can answer many medical queries accurately. However, those are small pieces of a physician’s role in a clinical setting, and an even smaller part of a physician’s role in the overall health system. 

As a profession, we’ve done a terrible job communicating the contributions of physicians outside our role of seeing patients. 

Non-clinical physician contributions to health systems

Hospitals generally rely formally and informally on physicians for leadership related to:

• Cultural change 

• Suggesting system and practice changes based on medical literature 

• Updating and raising standards of care

• Implementing new clinical programs or approaches

• Technology implementation

• Addressing concerns of nurses, pharmacists, and other clinicians

The association between hospital performance and physician engagement is underlined by data showing that hospitals run by physicians have a higher ranking, and studies showing that “separation of clinical and managerial knowledge inside hospitals was associated with worse management”. Interestingly, this correlation of “domain-expert leader” and better business performance has been demonstrated in many other industries including universities and the NBA.

This leadership has been mostly taken for granted in many hospital systems over the years. Often, though not always, these duties are done on a volunteer basis. In other industries, people who perform similar tasks have…actual jobs. And formal titles and roles reflecting their contributions.

Physician contributions to the AI lifecycle

Well-Architected machine learning lifecycle

Just as physicians contribute to a multitude of processes in the hospital, they can and should contribute to the development, implementation, and use of AI tools. A well-defined lifecycle exists for AI/ML platforms, and each stage benefits from physician contributions.

Business goal 

Physicians possess a deep understanding of the current state for a particular problem, in terms of severity and impact on patients, providers, and the medical system. Most physicians are full of ideas about how to improve patient care and experience but lack the time and expertise to implement them. They’ve often practiced in multiple health systems across their careers and can identify inefficiencies and opportunities in the workflow. 

A little-appreciated aspect of medicine is constant interaction with people of all ages, life situations, concerns, and priorities. Physicians do constant market research: they actually know people who fit “user profiles'' and can speak to real-life use cases. Physicians also speak to many other physicians both personally and professionally and can often identify additional use cases in other specialties or parts of the care journey. 

ML Problem Framing

Physicians understand whether the problem actually exists clinically and what problem the tool is actually solving. They can also provide context about the size of the problem warrants the processing power and people power required for an AI solution. Often the current non-tech solution actually works quite well. Physicians can identify the true end users of the technology (ie, office staff rather than physician).

Data processing

Physicians’ nuanced understanding of the clinical situation allows for training or outcome data to be aligned more closely with the business case. In the case of training data, physicians can identify advantages and disadvantages of ML training sets. They can help determine missing pockets of data that would be unlikely to be well-represented in a specific data set, or point out a change in clinical practice patterns that may bias an algorithm to an approach that is no longer relevant. 

Model development

Determining the outcome of interest or a clinically relevant proxy has long been a role of physicians in medical studies. Knowing what outcomes are actually meaningful to clinicians, patients, and in the medical literature is crucial to understanding the utility of AI models. 

Additionally, physicians know how to present the data to clinicians to make it useful. Almost all physicians will expect to receive information in similar formats with similar mental models. Physicians can identify the level of complexity and that’s actually helpful clinically. 

Physicians can also identify workflow impact of implementation and failure of AI, which can have huge patient care implications. Modern medical care is so complex that it relies on many pieces of the healthcare machine to function correctly. Any doctor can give you examples of a mistake that was made when one part of the system workflow was disrupted. Identifying and making contingency plans for these circumstances can reduce the risk to patients in these situations. 

Model deployment

Routing data to clinicians in a way that augments rather than burdens their workflow is a major issue. Unlike many professionals, physicians spend much of their time away from computers (though they still spend about half their time in front of a screen). Though a computer may be physically present nearby in the operating room or in a patient’s room while making rounds, the machine isn’t interacting with the physician during these times. Finding a way to provide AI support in real time without disrupting these crucial activities is a major barrier. Physicians are well-positioned to identify points in the workflow in which AI support would be welcome and helpful.

Monitoring

Physician oversight of clinical products should be part of an ethical obligation of healthcare companies. Physicians can identify technology that has the potential for harm due to bias, technical failure, or hallucinations. Advice from AI has been well-documented to produce incorrect and/or dangerous results. Physicians can identify the size, scope, impact, and likelihood of AI errors. They can also help develop plans and systems to mitigate these risks, which likely include some degree of physicians or clinician oversight. 

The importance of embedding physician input into the AI/ML lifecycle

The predictions about the end of physicians are far-fetched, but they’re true in the sense that if we don’t speak up about our contributions to AI, health tech, and healthcare overall, our role will morph into being more of a widget than we’d like to imagine. 

Unless we are vocal about how physicians contribute to the development of AI platforms, we are at risk of our input becoming invisible, just as it is for the myriad contributions we make in the clinical setting. Physicians must advocate to be integrally involved in the process and advocate for acknowledgement of their insights. Physician leaders can assist with this by hard-wiring a physician role into the committees. AI/ML companies also must acknowledge the value of physicians in their process, not just because it’s safer clinically but also because they can provide information that will make products more impactful and more useable. 

Are you a physician interested in health tech careers? Join ML for MDs for a special event on January 24, 2024. Sign up here!