The Pleasant Grove, Utah-based KLAS Research on March 10 published its 2022 update of its 2019 research report on artificial intelligence (AI) data science solutions. As KLAS explained on its website on Thursday morning, “An update to that research, this report examines how outcomes and customer satisfaction have evolved in the years since. Though progress has been somewhat hamstrung by the financial and operational constraints of COVID-19, many of the organizations interviewed for this research have found results by focusing on the right problems.” The new report is entitled “Healthcare AI 2022: Proven Outcomes with Data Science Solutions,” and was authored by Ryan Pretnik (lead author), Jennifer Hickenlooper (co-author), and Elizabeth Pew (writer). The report notes that “The data in this report comes from two sources: (1) case study interviews with two of each vendor’s deepest AI adopters and (2) KLAS performance data.”
“The potential for artificial intelligence (AI) to transform healthcare has been both championed and challenged. In 2019, KLAS published our first research on healthcare AI data science solutions, exploring healthcare organizations’ clinical, financial, and operational use cases and the early outcomes they were achieving. An update to that research, this report examines how outcomes and customer satisfaction have evolved in the years since. Though progress has been somewhat hamstrung by the financial and operational constraints of COVID-19, many of the organizations interviewed for this research have found results by focusing on the right problems.”
After sharing their analyses of health IT leaders’ perceptions of the performance of several vendors—Cerner, CLosedLoop.ai, Epic, Health Catalyst, and Jvion—the report shares brief case studies from several healthcare organizations—ChristianaCare, INTEGRIS Health, SCAN Health Plan, St. Luke’s University Health Network, Geisinger Health System, and Kaiser Health Plan Foundation of the Northwest.
The report’s authors then provide a set of recommendations for health IT leaders, “from successful organizations.” They are:
Preimplementation:
• Analytics solutions are foundational to AI and often need to be implemented before predictive/prescriptive models.
• Identify the problem you are trying to solve. Then determine whether AI is the right solution for the problem.
• Set meaningful, clearly articulated goals for your use cases.
• An out-of-the-box solution is a great place to start, but the models will require examination before the predictions or prescriptive output can be implemented.
• For each use case, collaborate and get buy-in across teams before, during, and after the implementation; create an inclusive governance structure that represents all stakeholder groups and plan carefully for change management.
Implementation:
• Don’t get hung up on perfecting the incoming data. The machine learning needs to happen with your data as it really is.
• Plan for the model testing to take more time and resources than expected.
• Don’t spend too much time trying to perfect a model. Models that are “good enough” can still drive significant outcomes.
• Figuring out how to operationalize the model (i.e., defining the intervention) often requires more work than the actual build.
• Foster buy-in and adoption by helping end users understand the models and how the data is generated.
The authors also quote a vice president, who told them that “The biggest challenge with AI is operationalizing the models. We could create models left and right all day long, but they don’t mean anything until they are put into operation and can make a difference in decision-making.”
