The Spectrum of AI in Healthcare: Understanding the Levels of Intelligence in AI

Key Takeaways:

There is enormous potential for leveraging AI in healthcare, including removing toil from staff workloads, increasing efficiency and productivity, improving consistency in results, and
It can be difficult to truly evaluate AI powered solutions due to buzzword inflation, and the fact that AI is an umbrella term covering a wide range of technology
Asking some foundational questions can be key to truly understanding if an AI powered solution aligns with your business need.

Iodine Intelligence tackles a new challenge in healthcare’s mid-revenue cycle every month, shedding light on problems and solutions and sharing valuable insights from industry experts. Listen to Episode 12 The Spectrum of AI in Healthcare: Understanding the Levels of Intelligence in AI to learn more.

Artificial Intelligence (AI) has been an area of significant interest for the healthcare industry for years, and that interest is only growing in the face of prevailing financial headwinds and staffing shortages. However, in a crowded field of AI powered solutions promising to solve healthcare’s more pressing issues, it can be difficult to truly evaluate their capabilities and potential. In this month’s episode, Priti Shah, Iodine’s Chief Product and Technology officer, provides a framework for making sense of AI and its potential, and some of its applications in the mid-revenue cycle space.

The Promise of AI

The real world applications for leveraging AI to solve some of the pain points in healthcare can be sorted into a few broad buckets:

Automation: One of the most basic applications is simply automating tasks that you don’t actually need a human to perform.
Efficiency: If you have a task that can’t be automated entirely, it still requires human judgement, you can still supplement and enhance your staff with AI-powered tools.
Timeliness: You can ask an AI model to actively evaluate your entire patient census 24/7, which, realistically, you will never be able to staff humans to that degree.
Consistency: An AI model can help establish a baseline level of competence, rather than relying on strengths and weaknesses of individual staff with different experiences and clinical background.

AI is Confusing

Although recognizing the promise of AI is easy, evaluating an AI powered solution can become confusing quickly, because the truth is not all AI is the same, and not all AI can solve all of healthcare’s unique problems.

Today, there are two main barriers to understanding and evaluating AI. The first is that AI is just a hot space right now and the term gets bandied around a lot. Everyone is trying to claim that they use some form of AI, and they all mean something slightly different when they make that claim. The second problem is even within the field of computer science AI is not well defined, it’s an umbrella term that covers a lot of different tools and technologies for solving a lot of different types of problems. The only real common theme is: applying computer systems to perform tasks that normally require human intelligence because they’re too hard or complicated for computers.

AI models are constantly walking that tightrope, balancing precision and sensitivity, and you can’t over-pivot on either axis because it essentially renders that model impractical to use. But that also means we have to understand no model is perfect, and you have to chose what balance of false negatives and false positives you can live with.
” AI’s accuracy, or success, is always on a spectrum, and there’s always tradeoffs that I think we should be aware of.
PRITI SHAH, CHIEF PRODUCT AND TECHNOLOGY OFFICER

De-Mystifying AI

Knowing the benefits that AI can bring you, but acknowledging the challenges of truly understanding and evaluating AI, Priti Shah offers the following framework for thinking about AI, with some basic, fundamental questions to have in the back of your mind when thinking about making an investment in an AI powered tool.

Use Case

The first question to ask yourself is: what problem are you trying to solve?

This is important for two reasons, one, there’s a wide range of AI tools and they’re not all equally good at all tasks, and two, almost every AI model you’re going to encounter right now is trained to do one very specific task.

There are different tools available in the AI space that are better suited for solving some types of problems than others.

Documentation Interpretation: NLP or large language models
Classification Problems: Gradient boosted machines or neural networks
Image Recognition: Deep learning models

Different tools have different applicability to different problems, so you shouldn’t just focus on the latest and greatest. ChatGPT is currently the talk of the own, and while there are some things it does very well, its limitations have been well demonstrated. While powerful in its domain, no one’s trusting it to make a clinical diagnosis.

There is no silver bullet, you should not expect any one technology to solve all your problems, so when selecting an AI for investment, focus on: what is this AI actually trying to do for me, and does that match up with the business problem that I am trying to solve.

Performance

The second question to ask yourself is: how well does the model actually perform?

You can’t assume that just because a solution is powered by AI, the AI is highly successful. While some may think, “If the AI is trained to do this, it’s doing it perfectly,” that’s rarely the case. The reality is AI’s success is always on a spectrum, and generally that comes with trade-offs that you need to be aware of.

Artificial intelligence models are always balancing sensitivity and precision. You can create a model that’s so precise it has no false positives, but your criteria will be so narrow you’ll miss most of what you’re searching for, and have a ton of false negatives. Conversely, you can create a model that is incredibly sensitive, but by casting a wider net you’ll also catch a bunch of false positives.

Dialing up either precision or sensitivity too high can result in a model that is essentially useless for practical purposes, so you have to choose what balance of false positives and false negatives you can live with.

Timing

The third question to ask yourself when evaluating AI is: when is it capable of making its predictions.

An example of this coming into play would be giving your sepsis coordinators an AI model that can predict sepsis. If the model is only capable of making its predictions post-discharge, it doesn’t actually fit the use case of your sepsis coordinators, who want to identify sepsis within 24 hours of a patient’s stay. Timeframe is critical when making a decision about deploying AI.

Explainability

And the final thing you should consider is: how much insight can the model actually give you into why it’s making its predictions.

There are use cases where all you care about is the answer and how confident the model is in the answer. But when it comes to healthcare, where you’re interacting with other people, and you’re dealing with someone’s health information, it cannot be a black box. You have to be able to discuss and explain why is the AI making this determination.

Especially because, consider what we discussed earlier: that no AI is perfect, that it’s always a balance of sensitivity and precision, and therefore also a balance of false positives and false negatives. You need clear explanations of the predictions so you can know when to trust the system’s predictions, and when to apply your own judgment.

Iodine and AI

To further help conceptualize this, below are example of an AI company answering these four fundamental questions.

What problem are we trying to solve
- At Iodine, we leverage machine learning models to take in raw clinical data (lab results, ordered medications, performed treatments, clinician observations, etc.) and look at all those disparate pieces of data in conjunction with one another to make predictions about various disease conditions. We leverage those clinical predictions in a variety of ways. In the CDI space, we compare the clinical reality of the patient against what’s documented, and then look for gaps in between that can be clarified. So when it’s time to bill and code, the documentation is complete and accurate, and health systems will get paid. In the utilization management space, we compare the clinical reality of the patient to the level of care we would expect a patient like that to receive, to aid UM nurses with determining the appropriate level of care and admission status for patients
How well do our models perform
- Iodine is fortunate enough to have access to a vast clinical datasdet, and this enormous set of clinical data is fueling our models. Having more data enables the ability to target more rare diseases. We’ve also been iterating, and experimenting, and improving on our models for seven years. Data science is a process of discovery; for some of the more complicated disease states, we have gone through seven or eight different generations, each new version building upon previous advancements to increase performance. Across our client cohort, we’ve found that we’ve been able to substantially improve client’s metrics: 92% of facilities saw an increase in productivity (query volume) with the average facility generating more than twice as many queries per CDS as they did before Iodine. With our models surfacing those cases with the greatest likelihood of opportunity, CDI specialists were two-thirds more likely to query a reviewed patient. And this has real, measurable impact, including increased MCC capture rates, increased CMI, and, on average, an additional $3.5M in annual appropriate reimbursements per 10k admission.
When are our models making their predictions
- Our models are working concurrent with the patient stay, and our models are constantly reevaluating in real time as new information becomes available
How much insight do we give into why the model made its predictions.
- We bubble up the most relevant clinical evidence to our users so they can see why we think this way about a patient. We’re not speaking about a patient in the abstract, or this general type of patient, it’s specifically: why does the model think this patient, Jane Smith, has sepsis based on the way she’s presented so far.

Hopefully this framework is helpful for evaluating AI solutions, and having these four fundamental questions in the back of your mind will help to demystify the hype around AI, explain the different types of AI, and why it should be something you’re considering, but considering for the right reasons.

Interested in Being on the Show?

Iodine Software’s mission has been to change healthcare by applying our deep experience in healthcare along with the latest technologies like machine learning to improve patient care. The Iodine Intelligence podcast is always looking for leaders in the healthcare technology space to further the conversation in how technology and clinicians can work together to empower intelligent care. if that sounds like you, we want to hear from you!

Key Takeaways: There is enormous potential for leveraging AI in healthcare, including removing toil from staff workloads, increasing efficiency and productivity, improving consistency in results, and It can be difficult to truly evaluate AI powered solutions due to buzzword inflation, and the fact that AI is an umbrella term covering a wide range of technology Asking some foundational questions can be key to truly understanding if an AI powered solution aligns with your business need. Iodine Intelligence tackles a new challenge in healthcare’s mid-revenue cycle every month, shedding light on problems and solutions and sharing valuable insights from industry experts. Listen to Episode 12 The Spectrum of AI in Healthcare: Understanding the Levels of Intelligence in AI to learn more. Artificial Intelligence (AI) has been an area of significant interest for the healthcare industry for years, and that interest is only growing in the face of prevailing financial headwinds and staffing shortages. However, in a crowded field of AI powered solutions promising to solve healthcare’s more pressing issues, it can be difficult to truly evaluate their capabilities and potential. In this month's episode, Priti Shah, Iodine's Chief Product and Technology officer, provides a framework for making sense of AI and its potential, and some of its applications in the mid-revenue cycle space. The Promise of AI The real world applications for leveraging AI to solve some of the pain points in healthcare can be sorted into a few broad buckets: Automation: One of the most basic applications is simply automating tasks that you don't actually need a human to perform. Efficiency: If you have a task that can't be automated entirely, it still requires human judgement, you can still supplement and enhance your staff with AI-powered tools. Timeliness: You can ask an AI model to actively evaluate your entire patient census 24/7, which, realistically, you will never be able to staff humans to that degree. Consistency: An AI model can help establish a baseline level of competence, rather than relying on strengths and weaknesses of individual staff with different experiences and clinical background. AI is Confusing Although recognizing the promise of AI is easy, evaluating an AI powered solution can become confusing quickly, because the truth is not all AI is the same, and not all AI can solve all of healthcare’s unique problems. Today, there are two main barriers to understanding and evaluating AI. The first is that AI is just a hot space right now and the term gets bandied around a lot. Everyone is trying to claim that they use some form of AI, and they all mean something slightly different when they make that claim. The second problem is even within the field of computer science AI is not well defined, it’s an umbrella term that covers a lot of different tools and technologies for solving a lot of different types of problems. The only real common theme is: applying computer systems to perform tasks that normally require human intelligence because they’re too hard or complicated for computers. AI models are constantly walking that tightrope, balancing precision and sensitivity, and you can't over-pivot on either axis because it essentially renders that model impractical to use. But that also means we have to understand no model is perfect, and you have to chose what balance of false negatives and false positives you can live with." AI's accuracy, or success, is always on a spectrum, and there's always tradeoffs that I think we should be aware of. PRITI SHAH, CHIEF PRODUCT AND TECHNOLOGY OFFICER De-Mystifying AI Knowing the benefits that AI can bring you, but acknowledging the challenges of truly understanding and evaluating AI, Priti Shah offers the following framework for thinking about AI, with some basic, fundamental questions to have in the back of your mind when thinking about making an investment in an AI powered tool. Use Case The first question to ask yourself is: what problem are you trying to solve? This is important for two reasons, one, there’s a wide range of AI tools and they’re not all equally good at all tasks, and two, almost every AI model you’re going to encounter right now is trained to do one very specific task. There are different tools available in the AI space that are better suited for solving some types of problems than others. Documentation Interpretation: NLP or large language models Classification Problems: Gradient boosted machines or neural networks Image Recognition: Deep learning models Different tools have different applicability to different problems, so you shouldn't just focus on the latest and greatest. ChatGPT is currently the talk of the own, and while there are some things it does very well, its limitations have been well demonstrated. While powerful in its domain, no one's trusting it to make a clinical diagnosis. There is no silver bullet, you should not expect any one technology to solve all your problems, so when selecting an AI for investment, focus on: what is this AI actually trying to do for me, and does that match up with the business problem that I am trying to solve. Performance The second question to ask yourself is: how well does the model actually perform? You can’t assume that just because a solution is powered by AI, the AI is highly successful. While some may think, "If the AI is trained to do this, it’s doing it perfectly," that’s rarely the case. The reality is AI’s success is always on a spectrum, and generally that comes with trade-offs that you need to be aware of. Artificial intelligence models are always balancing sensitivity and precision. You can create a model that's so precise it has no false positives, but your criteria will be so narrow you'll miss most of what you're searching for, and have a ton of false negatives. Conversely, you can create a model that is incredibly sensitive, but by casting a wider net you'll also catch a bunch of false positives. Dialing up either precision or sensitivity too high can result in a model that is essentially useless for practical purposes, so you have to choose what balance of false positives and false negatives you can live with. Timing The third question to ask yourself when evaluating AI is: when is it capable of making its predictions. An example of this coming into play would be giving your sepsis coordinators an AI model that can predict sepsis. If the model is only capable of making its predictions post-discharge, it doesn't actually fit the use case of your sepsis coordinators, who want to identify sepsis within 24 hours of a patient's stay. Timeframe is critical when making a decision about deploying AI. Explainability And the final thing you should consider is: how much insight can the model actually give you into why it’s making its predictions. There are use cases where all you care about is the answer and how confident the model is in the answer. But when it comes to healthcare, where you’re interacting with other people, and you’re dealing with someone’s health information, it cannot be a black box. You have to be able to discuss and explain why is the AI making this determination. Especially because, consider what we discussed earlier: that no AI is perfect, that it’s always a balance of sensitivity and precision, and therefore also a balance of false positives and false negatives. You need clear explanations of the predictions so you can know when to trust the system’s predictions, and when to apply your own judgment. Iodine and AI To further help conceptualize this, below are example of an AI company answering these four fundamental questions. What problem are we trying to solve At Iodine, we leverage machine learning models to take in raw clinical data (lab results, ordered medications, performed treatments, clinician observations, etc.) and look at all those disparate pieces of data in conjunction with one another to make predictions about various disease conditions. We leverage those clinical predictions in a variety of ways. In the CDI space, we compare the clinical reality of the patient against what’s documented, and then look for gaps in between that can be clarified. So when it’s time to bill and code, the documentation is complete and accurate, and health systems will get paid. In the utilization management space, we compare the clinical reality of the patient to the level of care we would expect a patient like that to receive, to aid UM nurses with determining the appropriate level of care and admission status for patients How well do our models perform Iodine is fortunate enough to have access to a vast clinical datasdet, and this enormous set of clinical data is fueling our models. Having more data enables the ability to target more rare diseases. We’ve also been iterating, and experimenting, and improving on our models for seven years. Data science is a process of discovery; for some of the more complicated disease states, we have gone through seven or eight different generations, each new version building upon previous advancements to increase performance. Across our client cohort, we’ve found that we’ve been able to substantially improve client’s metrics: 92% of facilities saw an increase in productivity (query volume) with the average facility generating more than twice as many queries per CDS as they did before Iodine. With our models surfacing those cases with the greatest likelihood of opportunity, CDI specialists were two-thirds more likely to query a reviewed patient. And this has real, measurable impact, including increased MCC capture rates, increased CMI, and, on average, an additional $3.5M in annual appropriate reimbursements per 10k admission. When are our models making their predictions Our models are working concurrent with the patient stay, and our models are constantly reevaluating in real time as new information becomes available How much insight do we give into why the model made its predictions. We bubble up the most relevant clinical evidence to our users so they can see why we think this way about a patient. We’re not speaking about a patient in the abstract, or this general type of patient, it’s specifically: why does the model think this patient, Jane Smith, has sepsis based on the way she’s presented so far. Hopefully this framework is helpful for evaluating AI solutions, and having these four fundamental questions in the back of your mind will help to demystify the hype around AI, explain the different types of AI, and why it should be something you’re considering, but considering for the right reasons.

Iodine Software Iodine Software Iodine Software

AwareCDI

AwareUM

Iodine News

Events

About

Partnerships

Careers

Support

Contact