The analytics pathway is described as moving upward in complexity from descriptive to diagnostic to predictive to prescriptive, with each step adding more complexity and business value. Jaclyn Bernard, manager of the innovations team at Texas Children's Hospital (TCH), recently described some lessons learned on her organization’s journey to predictive analytics.
With three hospitals in the Houston area, TCH is the largest pediatric hospital in the country, with more than 14,000 faculty and staff, and a health plan that covers 460,000 members in the Texas CHIP and Medicaid programs. Bernard’s team develops and deploys predictive analytics solutions and does custom application development.
Speaking a few weeks ago at the Healthcare Analytics Summit put on by vendor Health Catalyst, Bernard said they started doing machine learning proofs of concept (POCs) in 2017, beginning with central line associated bloodstream infections (CLABSI). In 2018, they targeted other topics such as well checks for Texas Children’s Health Plan, asthma costs, emergency center utilization and throughput, and readmissions.
Initially TCH used outside consultants for machine learning expertise but eventually built a team internally with the right mix of expertise, including data architects, operational champions, and subject matter experts.
“We have had an enterprise data warehouse for 10 years being fed by 60 source systems. It has more than 200 subject area marts and reporting tools,” Bernard said. “We also needed data pipelines to machine learning platforms."
Obviously, predictive analytics requires investment. “To get buy-in from senior leaders, make sure you have alignment with their strategic initiatives,” Bernard recommended. For example, quality leaders are interested in CLABSI; care coordination execs care about readmissions; Finance execs are interested in no-shows.
The TCH team had to show predictive analytics was a worthwhile investment. “You have to show executives the value of investing in predictive capability and demonstrate to them that it is not only valuable but broadly applicable and doable,” she said. “We decided to pursue multiple proofs of concept and strategically chose our use cases.”
She said pursuing multiple proofs of concept allowed for broad experimentation. Multiple quick-and dirty proofs of concept show the potential value quickly compared with taking a long time to prove real value through one full implementation from start to finish. It could show the organization the potential to be valuable across a wide variety of contexts. “When an organization is starting a predictive analytics journey, you don’t want it to look like a mammoth task,” Bernard said. “This approach allows for rapid learning about what makes a good use case.” Some POCs were more successful than others, she added. By the end of the second year of their journey, they had built 14 proof of concepts.
In the first year they were able to successfully demonstrate that predictive analytics is valuable, broadly applicable and doable. In the second year, TCH began work to operationalize models, build competency in-house and formalize and expand the program.
In the early projects, they learned that different stakeholders may have different goals and that some use cases require deep expertise. They learned to watch out for incomplete, low-quality data. The accessibility and quality of data matters. Also, data literacy varies across the organization and operational partnership is key. They had to learn to manage expectations and remind people that predictive analytics is not a silver bullet.
TCH created a use case evaluation framework that focused on:
• Understanding the problem —what domain expertise is needed and how straightforward is the question?
• Defining business or clinical value (Is this impacting a hospital metric?)
• Data quality
• Resource availability
Each project moves through steps from project intake and prioritization to project kickoff to model development to operationalization.
One example is HPV immunizations. HEDIS immunization measure targets the completion of adolescent immunization by age 13. HPV immunization compliance is 27 to 37 percent. “If we knew who was likely to be non-compliant, outreach could begin earlier,” she said. The problem is straightforward, and no clinical expertise is needed. The data is readily available, including past vaccine adherence, flu shot adherence, demographics, social determinants of health, and history of refusing vaccines. The project has a high clinical quality impact and is a 2020 organizational quality goal. HEDIS Immunization is a pay-for-quality metric for some business lines. So the decision was made to begin the project.
Bernard said that since launching the use case evaluation framework, Texas Children’s has demonstrated significant improvement in project selection, as evidenced by higher efficiency in progressing past the proof of concept stage. A greater number of projects are reaching the full development, operational and live stages The perception of the use of predictive analytics over the past three years at Texas Children’s have evolved favorably, she added.
“This has been a huge learning process for us as an organization. When we started very few people knew what predictive analytics was,” she said. “Now people are excited about it.”
