Key Takeaways:
- Iodine Software has been iterating on its models for the last seven years, with each new generation unveiling advancements and improvements in accuracy
- Artificial Intelligence (AI) is an umbrella term, natural language processing (NLP) that merely scans the written record is not the same as machine learning (ML) which surfaces clinical predictions based on the clinical evidence
Iodine Intelligence tackles a new challenge in healthcare’s mid-revenue cycle every month, shedding light on problems and solutions and sharing valuable insights from industry experts. Listen to Episode 5: Progression of Accuracy with Lance Eason to learn more.
Data analytics is nothing new, and big data is leveraged by everyone from Google in targeting ads to Starbucks in picking the best location for a new store, however, surfacing clinical insights out of healthcare data is an incredible challenge. Iodine’s CognitiveML seeks to conquer this final frontier: processing raw clinical data to make precise predictions about specific disease conditions.
While there is an incredible range of tools at one’s disposal (with dozens of different model types that can be applied to a problem) each disease has its own idiosyncrasies, and the same model often doesn’t work equally well for every type of disease state. While some conditions, such as an electrolyte balance, may be relatively straightforward to predict, other conditions are incredibly complex.
The genesis of Iodine’s machine-learning models lies, in part, in the inadequacies of traditional methods in the face of complicated conditions, namely: sepsis. F1 scores are frequently used when measuring a model’s accuracy, and examine both the number of predictions made and the number of predictions which are correct; the profoundly limiting nature of a rules-based approach resulted in an F1 of only 0.21 in the original sepsis model. This spurred Iodine into investigating different technologies; the first machine-learning model applied to sepsis more than doubled the model’s accuracy from 0.21 to 0.53. Since its original launch, Iodine has gone through numerous iterations of the sepsis model, with each subsequent generation building upon previous advancements and introducing new improvements. Iodine’s F1 for sepsis is now in the 0.80 range, with plans to continue experimenting, iterating, and improving.
““If we had gone and said, ‘We are using AI to predict sepsis’ seven years ago, and we say it today, we’re saying two different things, because our predictions at that time were 0.5 versus 0.8 nowadays. So there’s been a significant increase in the actual quality of the predictions we’re making over time”.”
– LANCE EASON, Chief DATA SCIENTIST
These improvements stem from a variety of advancements and experiments,
including:
- a growing pool of data
- mapping of input values (i.e. labeling lab values)
- applying successful models to similar disease conditions (ex. UTIs and pneumonia are both infectious diseases, a technique that is successful for one might be similarly successful with the other)
These combine to create a ML-AI engine seven years in the making that is now
uncatchable.
Competitors wishing to switch to a similar format, would be starting almost a decade behind, but the reality is, many of the other technologies in the clinical documentation integrity (CDI) space take a radically different approach to solving the problem of documentation leakage: focusing on interpreting the clinical documentation to determine what’s missing. “The problem with that approach,” says Lane Eason, Chief Data Scientist, “is the documentation itself is not where most of the CDI opportunity is. What CDI is about is making sure what is documented for the patient matches what’s actually going on clinically with the patient, and if you’re not looking at that other side of the equation…then just reading the documentation is not going to tell you all the things that aren’t documented.”
AI is a wide umbrella that encompasses everything from NLP to image recognition, and the degree of complexity and ability varies across technologies. Claiming to leverage AI because you use NLP to scan documentation is not equivalent to using machine-learning to make clinical predictions about specific disease states based on the clinical evidence. And as Iodine itself learned in the beginning, the method of AI applied can have dramatic affects on the efficacy and accuracy of the results.
Interested in Being on the Show?
Iodine Software’s mission has been to change healthcare by applying our deep experience in healthcare along with the latest technologies like machine learning to improve patient care. The Iodine Intelligence podcast is always looking for leaders in the healthcare technology space to further the conversation in how technology and clinicians can work together to empower intelligent care. if that sounds like you, we want to hear from you!