The Healthcare AI Triad
Understanding the main drivers of success and failure in healthcare AI
“A single sentence can summarize the complexities of modern artificial intelligence: Machine learning systems use computing power to execute algorithms that learn from data. Everything [you] need to know is captured in those 13 words.”
Center for Security and Emerging Technology Report on AI Triad
This month we’ll discuss the three main factors that affect AI and how they relate to healthcare:
Algorithms/People
Compute (computational processing power)
Data
Sometimes the “algorithms” in the triad is replaced with “people”, since (at least for now) people have to create, implement, and manage the algorithms.
Algorithms and People
The algorithms are what turns a dataset into a model, and in machine learning they are designed to learn from data and improve their performance through training.
Below are some examples of algorithms and their uses
Neural networks/deep learning: Neural networks are used for pattern recognition for tasks like medical imaging and chatbots.
Support Vector Machines (SVMs): SVMs find the best boundary between two classes and is used for classification, like “cancer” vs “not cancer”.
Decision Trees: Decision trees are a type of machine learning algorithm that is used for classification and regression analysis. Random forests work by creating multiple decision trees and combining their results to make a final prediction.
Most of the advances in AI in the last 10 years have come from advances in neural networks, and more recently large language models (LLMs) specifically. This is where we’ll mostly focus this month.
People
To develop, run, monitor, and troubleshoot these algorithms, you need knowledgeable people. A McKinsey report in 2022 noted that most companies have hit a plateau in their adoption of AI mostly because they just don’t have people who know how to do it. Companies have been trying desperately to hire the roles they need on an AI team; not just engineers; not just engineers, but product managers, data scientists, data engineers, and designers.
To deal with the labor shortage, employers have started hiring more people right out of school, and many are focusing on reskilling employees, spending thousands per engineer to train their engineers to keep up with the new technology. Despite more focus on diversity, the percentages of women and minorities is still pretty bad.
Clearly there’s an opportunity to expand the workforce by drawing in people who haven’t traditionally been in these roles. These percentages are very similar to applicants to Clinical Informatics fellowships in 2020: 34% were women and 12% were underrepresented minorities.
A paper from 2019 described the characteristics of physicians board-certified in clinical informatics. 65% are from primary care specialties, and very few are from procedural specialties.
Graphic based on data from Desai et al
And there are just not very many physician informaticists total: about 1700 in the country. There are some states with zero physicians boarded in Clinical Informatics, and many states with fewer than 2 physician informaticists/1000 physicians.
Let me preface this section by saying how surprised I am at the dearth of research about the physician workforce and its readiness to contribute to AI. I couldn’t find any articles that quantified how many physicians might have the background to contribute to healthcare AI. I’m hopeful there will be an effort to map transferable skills and invest in training and education for the physician workforce.
Until then, this is my summary of the levels of knowledge and skills needed for physicians at different levels of interaction with AI:
We don’t have any good data on how many of each type is in each hospital, but I am going to assume that it will be similar to EHR implementation, possibly with increasing numbers in categories 3 and 4 due to the increasing numbers of computer science majors in universities who are now in medical school, training, and practice.
Let’s make a few back-of-the-envelope assumptions:
In a health system with 1,000 physicians, let’s assume there are:
20 clinician AI champions, about one per department
4 of those are CMIOs or similar, at level 3, 2 based on national CMIO data plus a few additional
one person able to create their own AI products.
I actually think these are fairly generous numbers, and in many parts of the country the numbers would be lower. I’ll add that most physician AI champions are thrust into that role with relatively little training and most of them learn on the job.
A perspective piece in the NEJM suggested that physicians don’t have the skills to use clinical decision support algorithms correctly. But let’s think for a moment about skills physicians already have, and how they can be reframed to be helpful in an AI context:
My hope is that healthcare will follow the example of the tech industry and upskill their workforce, including physicians and other healthcare professionals, to optimize the possibilities of AI.
In the coming weeks we’ll discuss the rest of the AI triad including:
An overview of AI compute
Compute in the clinical setting
Data in the healthcare AI triad
The future of the AI triad
Machine Learning for MDs Member Spotlights
Each week I’ll feature some of the amazing physicians in our Slack Community. They’re an incredibly talented group!
Kevin Maloy is a DC-based ER physician who not only runs codes, but writes code. You can check out his musings on FHIR (the standard defines how healthcare information can be exchanged between different computer systems), EHRs, and building AI in healthcare at www.patient.dev
Matt Sakumoto is a virtualist primary care physician in San Francisco. He is also fellowship-trained in clinical informatics at UCSF, with a focus on virtual care and clinician efficiency tools for the EHR. In his role as regional CMIO and as a clinician-advisor to many early-stage health tech companies, he is passionate about exploring and implementing effective and evidence-based AI into clinical workflows.
Waqas Haque is a third-year Internal Medicine resident at New York University in a Clinical Investigator Track with consulting experiences at Regeneron, OneOncology, and Sarepta Therapeutics in the areas of clinical trial operations and digital health. During medical school, he took a leave to pursue an MPH at Johns Hopkins as a Sommer Scholar with a focus on clinical trials, and before medical school earned a master’s degree at the business school at Cambridge University. He looks forward to conversations and opportunities related to innovation or drug development in the oncology or internal medicine space as he continues his training next year as an oncology fellow.