“It's what I call the diabetic foot exam example,” she says. In this scenario, a health system wants to analyze the performance of physicians who specialize in caring for diabetic patients, with regard to the routine ordering of a foot exam when a patient comes in for a primary care doctor visit. Logically, the health system would want to track which physicians order foot exams and perhaps set up differential reward programs around such tasks.
But what if Mrs. Jones has already had both of her feet amputated? Her physician's “failure” to order the test should not count negatively, no matter what the data says. Yet few systems that gather process-oriented clinical data are sophisticated enough to take such contingencies into consideration, Drazen points out. And CIOs remain less than confident in the data they're analyzing that comes from multiple sources.
Still, many agree that as pay-for-performance and other quality-related initiatives gain traction, CIOs will need to cement their confidence in the validity and integrity of all types of data in order to survive in the emerging operating environment. Nor are concerns over data integrity abstract and futuristic.
“There was a big brouhaha in Massachusetts recently,” Drazen says, “Because they were going to use claims data for qualifying the physicians; for allowing them to participate in the health plans in that state. Physicians actually filed a lawsuit last fall, claiming the data was so inaccurate” that it should not be used for such purposes, she notes.
The solutions
Fortunately, more CIOs are becoming aware of the data integrity issue, and those whose organizations are participating in such programs as the CMS/Premier Hospital Quality Incentive Demonstration (HQID) P4P project, are doing something about it. “I think the foot-exam situation is a great example,” says Mike McCurry, vice president and CIO of the 20-hospital Sisters of Mercy Health System, based in St. Louis, “And it exists in many dimensions. As processes change, you get different relevancy of data before and after the process change. And part of what you're trying to do, of course, is to look at data in order to promote wide-scale process change. If the data has changed, it's difficult to appropriately use it.”
McCurry and his colleagues are working on the data integrity problem, not only because of their participation in the HQID project, but as part of their strategic vision of quality.
“The larger scope of what Mercy is doing around quality is really superseding what we're doing with the CMS/Premier program,” he says. “We're on a six-year journey to replace all of our major systems,” and are in the midst of a system-wide implementation of a core clinical IS right now.
With regard to the kinds of issues around production and analysis of meaningful clinical data, “On a technical level, conceptually, it's a fairly straightforward concept, that A must equal A,” says Paul Helmering, executive director of business intelligence at Sisters of Mercy. “But the intuitiveness often is not there. Leaders often dismiss it and move past the problem, only to have to go back to it again later when the data doesn't conform. So it's the single biggest challenge to have to approach with regard to performance improvement work.” Helmering's business intelligence team is currently at work on a major initiative to improve the meaningfulness and usability of data.
But the need to move forward is increasingly important, she says, not only from the pay-for-performance standpoint, “but also because our data is becoming more visible to patients and customers.”
“Quite honestly, there is a lot of bad data” in patient care organizations' databases, says Mark Budd, a partner at CSC. “People pull data into warehouses, where the algorithms aren't correctly written, so it's not clear what level of quality of data you're talking about,” notes the Washington, D.C.-based Budd. “And a lot of data comes in from transactions, where it's fuzzy. So a lot of the focus becomes, who's accountable for making the data accurate?”
Budd, who is managing a project at the federal National Institutes of Health that involves development of a broad-based clinical data repository for analytics, says the path ahead will be long and complicated.
At the prestigious Memorial Sloan-Kettering Cancer Center in Manhattan, vice president and CIO Patricia Skarulis and medical director of information systems and CMIO David Artz, M.D., are working on keeping their data pure.
One example that Skarulis and Artz are trying to work out is the routine alert they've created that requires physicians to order a pregnancy test for any female patient between the ages of 11 and 55, when preparing for chemotherapy and radiation treatment. Within the gynecological oncology department at Sloan-Kettering, however, the vast majority of patients have already had hysterectomies or other operations that obviate the need to order a pregnancy test. As a result, Skarulis and Artz say the key is to analyze data and situations appropriately, and create systems, such as alerts, that optimize care while not turning physicians and clinicians off.
Even the largest and most sophisticated integrated health systems find the problem vexing. “As people begin to implement and wind down their EHR systems and say, ‘Now we can get to that granular level,’ they're discovering in fact that, for example, they don't enter all the patient problems for the problem list, or that an antibiotic given five years ago is out of date,” says John Glaser, Ph.D., senior vice president and CIO of Partners HealthCare in Boston.
What's more, says Glaser, “The question that arises with regard to all this is effectiveness of use of data. For example,” he says, “we found that in one of our institutions, one of four prescriptions was still being hand-written. That means that if you're analyzing your prescription drug expenditures, you're not able to analyze one-quarter of your data.” And that question begs yet another, which is why clinician adoption of the EMR and its tools might be lagging—which leads to workflow, screen design, and other issues to sort out. In other words, says Glaser, the critical need to examine the integrity of an organization's data inevitably provokes deeper questions about many interrelated issues.
The ROI
Many agree there is a return on investment in data integrity work, though it's accepted that provable and concrete ROI won't be realized for years to come. The future towards which organizations are working is one of data transparency and usability, not only for P4P participation, but for internal performance improvement and care quality.
At the seven-hospital Ochsner Health System in New Orleans, system vice president and CIO Lynn Witherspoon, M.D., sees strong potential in data analytics going forward; and with that in mind, Witherspoon has created a data analytics group. The work of that group will be deeply involved in optimizing the use of the organization's data warehouse. At the same time, Witherspoon and his colleagues are pushing ahead on multiple fronts, including sorting out patient identifier issues and managing the streaming of data from diverse sources. “We have 200-odd clinical information systems across our organization,” he notes. That fact, and the reality that Ochsner has evolved through multiple hospital acquisitions, necessitates continuous data integrity and validity work ahead in the coming years, he says.
The challenges are manifold, agrees Beth Just, president and CEO of Just Associates, a Centennial, Colo.-based consulting firm specializing in data integration and integrity issues. “Fortunately,” says Just, “CIOs get it now. Until recently, they weren't aware of the problem. But by the same token, it wasn't common back then to have so many different systems all talking to each other,” she says. “Now, there's just so much more integration going on, that this problem becomes more visible.”
