AHRQ Tool to Help Health Systems Learn From Diagnostic Safety Events

The federal Agency for Healthcare Research and Quality has introduced a new clinical tool, Measure Dx, that health systems can use to help prevent diagnostic errors. In a recent blog post, Robert Otto Valdez, Ph.D., M.H.S.A.,  director of AHRQ, said the new tool “offers practical guidance and modular, scalable strategies to detect and learn from diagnostic safety events.”

The Measure Dx program was led by the Center for Innovations in Quality, Effectiveness, and Safety at the Michael E. DeBakey Veterans Affairs Medical Center and Baylor College of Medicine in Houston, under the direction of Andrea Bradford, Ph.D., and Hardeep Singh, M.D., M.P.H., with support from the MedStar Institute for Quality and Safety.

Valdez noted that the strategies associated with Measure Dx are laid out in a four-part guide that provides:

• Strategies to engage people in your organization so that adequate resources are devoted to implementing measurement and learning activities.
• A self-assessment checklist to gauge readiness for implementation, as well as guidance for choosing a measurement strategy that fits within organizational resources.
• Measurement strategies for different types of data sources.
• Recommendations for systematically reviewing and analyzing case data and translating findings into valuable insights for improvement.

During a recent webinar hosted by AHRQ, Baylor’s Bradford began with a stark reminder of why they are doing this work. An estimated one in 20 outpatients each year will experience a diagnostic error. “That's a staggering number of people when you extrapolate to the volume of patients that are seen in outpatient settings every year,” she said. “In the inpatient setting each year, it's estimated that a quarter of a million people will not only experience a diagnostic error in that setting, they will also incur some level of harm as a result. This is a huge problem. But unless a misdiagnosis leads to a major event or a malpractice claim or a news story, it's likely to go undetected — and what that means is that the system really doesn't get an opportunity to learn from it or to act on it. Very few systematic strategies exist to detect and measure diagnostic safety. Up until now, most of those strategies that do exist have usually not gone very far beyond the academic medical centers where they were developed.”

The National Academy of Medicine recognizes this problem, she said, adding that a 2015 report recommends that payers and accrediting organizations start paying attention to this problem, too. “There's every reason to believe that we're on the road toward payers and accrediting organizations looking at what healthcare organizations are doing in the service of improving diagnostic safety,” Bradford added. “How are they monitoring and learning from the events that happened within their own walls around vulnerabilities in the diagnostic process?”

In 2020, the Measure Dx team surveyed the state of measurement of diagnostic safety.  What they found was that some of these measurement strategies are still very much in the research and development stage. They haven't really been tested in operational settings. But there are some strategies that have real proofs of concept and have been applied in operational healthcare settings, Bradford said. “They've yielded actual diagnostic safety intelligence on real patients in a real healthcare system. We thought that a subset of these strategies was ready for prime time. Our next initiative through AHRQ was to then translate this further and get these strategies into the hands of clinicians, patient safety professionals, and healthcare leaders.”

The team created Measure Dx in close collaboration with the experts who had developed and initially tested some of these measurement approaches. “In the initial phase of development, we partnered with 12 experts who worked with us in small groups to co-create different sections of this resource depending on their expertise and the types of measurement strategies that they themselves had developed and tried,” Bradford said. As this resource came together, they brought in some additional patient safety experts, including a patient representative to review and revise the tool.  “Most recently, we've concluded pilot testing with 10 healthcare organizations,” she said. “We asked them to take this resource, give it a try and give us some feedback about how we can make this even better and more accessible. All of that work has gone into forming the finished version that has just been released and that you can download on AHRQ’s website.”

Bradford stressed that within a health system this is a team effort requiring several different kinds of expertise. “At a basic minimum, we recommend that the team consist of at least one clinician whose scope of practice includes diagnosis, need someone who understands diagnosis, and can be can lend their effort to looking more closely at the cases that you can apply,” she said. “We also recommend that your team include at least one quality and safety professional, someone who is well versed in quality improvement methodologies, and who can apply a quality and safety lens to this work in order to push any lessons learned forward and result in some kind of meaningful improvement in your system.”

How a health system decides to proceed with event detection is going to depend on the capabilities of the organization and the data that's available, Bradford noted. Most organizations can start by looking at data that they already routinely collect for other purposes — things like mortality reviews or general quality and safety event reviews. An organization can re-examine these data, maybe by looking a little bit further upstream in the diagnostic process to look at these events, applying a lens of diagnostic safety and learning more from them from that perspective.

She said there are other data sources that the research suggests that organizations can consider for detecting these events in the first place. “For instance, some organizations have a mature safety event reporting system, and that can be leveraged as a way of collecting safety data from frontline clinicians,” Bradford added. “Some organizations have taken an existing safety event reporting system and then modified it so that diagnostic safety is its own category of event. Other organizations can even build a separate stream of collecting information specific to diagnostic safety events from clinicians.”

The Measure DX resource provides some guidance for leveraging the capabilities of an electronic health record, combined with a data warehouse that's able to be queried in order to use an algorithm to detect potential safety events from patterns of care that suggests that there may have been some breakdown in the diagnostic process. “For example, there might be unexpected escalations of care, or it may be return visits to an emergency department,” Bradford said.

Later this year, Bradford said, there will be an opportunity for some organizations to work with the Measure Dx team to do some further and more rigorous evaluation of the tool.