Providence St. Joseph Health’s Forward Push on Analytics During the Crisis

Patient care organizations’ data analytics journeys took a sharp turn earlier this year in the face of the pandemic as hospitals and health systems had to quickly mobilize data to prepare and synthesize information about COVID-19 patients throughout their system.

Just like it was all over the U.S., in the Pacific Northwest, the situation that presented itself was unprecedented at the Renton, Wash.-based Providence St. Joseph Health (PSJH), a 51-hospital, 9,000-physician healthcare system. The earliest thing that data analytics leaders were focused on when the crisis first struck was trying to characterize the epidemiology of COVID-19 in a region, recalls Ari Robiscek, M.D., the chief medical analytics officer at PSJH, who also specializes in infectious diseases.

“We weren’t doing a lot of COVID testing, since its availability was very limited early on and we had no idea how much [of the virus] was in our region, so the very first thing we tried to do was syndromic surveillance—looking for patients who had clinical features that we thought were suggestive of COVID, and basically screening all clinical notes that were written on all of our patients for clusters of symptoms that we thought would tell us if those patients might have COVID. Then we would look to see where those patients were geographically situated. That was one of the first things we focused on,” Robiscek explains.

Another early realization from Robiscek and his team was trying to understand the prevalence of COVID that PSJH would be seeing in its hospitals, so they can then do surge modeling. At that point, he notes, situational awareness is what became most important, such as how many patients were in the hospital at that moment, what the ventilator capacity was, and if they were running out of ICU beds. “Those were our early pre-occupations,” he says.

Early on, one of the major challenges for health system analytics departments in the face of the pandemic was developing data models that would forecast when COVID-19 would hit a specific region the hardest. Robiscek recounts that the initial models his team built that projected PSJH’s hospital surges “were terrible” since they didn’t know what the starting number of COVID was in its populations, which he says is the most important entry point when building out exponential models.

“We didn’t have a good sense of how COVID behaved in populations, which [meant] we didn’t have a good sense of what its effective reproduction number would be,” Robiscek admits. “That was especially true when we initially built these models, since we hadn’t even started aggressive social distancing measures that we subsequently implemented such as closing schools, restricting public gatherings, and wearing face masks. When we started doing [the modeling], we had no clue what the impact of those measures would be.”

Robiscek further acknowledges that there still isn’t a great sense of what percentage of the population is highly susceptible to being infected. “We didn’t have and still don’t have good data related to how many people are infected at any given moment, and even now with all the testing, we’re probably substantially underestimating the percentage of the population that has COVID-19.”

For all these reasons, the initial modeling was quite difficult, Robiscek admits. He adds that the organization’s judgement in trying to build out models was essentially based on one single surge several months into the future. “It seems obvious in retrospect, but we now realize that this isn’t the kind of disease that will observe the classic epidemiological dynamic of one or two surges. This is going to be a disease of ups and downs that vary from one region to another, depending on lots of things—including the hard-to-predict super-spreader events,” he says.

A new path forward

Now, however, thanks to new learnings over the course of the last several months, the analytics group is able to be “more modest” in its ambitions, making predictions two weeks in advance on what its COVID census is going to be in each of the health system’s regions, says Robiscek. “The data we use to feed into [those models] includes information on how many patients we have been censusing over time and data on the percentage of outpatients in a particular region who are displaying a particular constellation of symptoms, which we find is a good leading indicator of what will eventually happen in our hospitals.” He adds that the percentage of patients who test positive, as well as data that is openly available on Google Health Trends, are also used for modeling.

Another core challenge for the PSJH analytics team, Robiscek offers, has been dealing with the constantly changing business requirements over the course of the pandemic. “It’s hard for an analytics team to pivot every few days to doing a whole different set of analyses, as well as chasing after different key goals. Having a structure where you are open to and can accommodate flexibility is very important,” he contends. Robiscek also points to the value of having an established underlying infrastructure around people, process and technology. On the people side, having individuals with a variety of different skillsets on the team has been incredibly useful, he notes. For example, his team has some people with geographical information system skills, some with natural language processing skills, others who are proficient in web development, and also some who have connections to researchers.

An additional win for the analytics team, Robiscek says, was having everything built on a cloud platform. “We had strong underlying connections to our EHR system in the form of existing warehouse data, so it wasn’t that hard for us to adapt data in existing tables into marts that ultimately were tailored around COVID. In the first two weeks or so, we were in fact able to stand up the marts that underlay almost all of the many analytics we have subsequently done. So having that underlying data infrastructure has been really helpful. And in general, being on the cloud has enabled us to do rapid processing, spin up data science activities, and just generally respond quickly to things,” he says.

Robiscek believes that one of the keys to making progress with data analytics when crisis strikes is to be on the same page as the clinical team. At PSJH, he explains, the data group is closely connected with the clinical enterprise and doesn’t sit alone in an IT department. In fact, Robiscek’s team is housed within the main clinical division of the organization, called Clinical Care. “My boss is the system chief clinical officer,” he says. “Right from the get-go we were called to action by the clinical leaders, so my peers are folks such as the system’s chief quality officer, the person who heads up the telehealth division, and various other individuals who lead major clinical groups.”

To that end, more, every Monday morning Robiscek attends an emergency operations command group meeting, consisting of 250 leaders from around the health system including regional chief executives, chief nursing officers, and the people who are most involved at coordinating emergency response. Robiscek takes feedback on the needs of that group, and also does a data roundup during that meeting in which he presents new information, including projections about what the next few weeks will look like in specific regions related to COVID, to disparities in care, or to testing efforts. “That’s been a really good way to stay closely in touch with what’s going on the clinical world. We need to deliver the things that the clinical leaders on the ground need,” he says.

Speaking to the importance of analytics and data scientists when there’s a pandemic, Robiscek says that all of a sudden, the people doing this type of work get massive amounts of attention, which can actually be a scary thing. “But it’s also understandable, given that we have a brand new disease affecting many millions of people all around the globe, which also has huge effects on the economy and way of life, in addition to peoples’ health. So there’s a huge thirst for information.” He adds, “People rely on our ability to mobilize the right data and interpret that data well. Anyone who does science of any kind related to COVID is under way more security than usual. We’re doing the work we have always done, but just a lot faster and under a much larger microscope.”