OIG Finds VA Metrics Gaps One Year After EHR Go-Live
A U.S. Department of Veterans Affairs Office of Inspector General (OIG) report found that more than a year after go-live with the new EHR at Mann-Grandstaff VA Medical Center, gaps in available metrics due to the EHR transition impaired the facility’s ability to measure and act on issues of organizational performance and patient safety.
Spokane, Washington-based Mann-Grandstaff was the first Veterans Health Administration (VHA) medical center to implement the new EHR system.
The OIG determined that, one year after go-live, gaps existed between required metrics and those that were available using new EHR data. These gaps impeded assessment and action to address organizational performance, quality and patient safety, and access to care at the facility.
According to the June 1 report, the OIG found that following go-live, facility staff used workarounds to mitigate the post go-live metrics gap. Facility staff shared with the OIG that the workarounds created a “tremendous” increase in additional workload, at times requiring numerous hours or days to prepare just one metrics report. Despite time-intensive workarounds and concerns with metrics accuracy, a facility leader shared that facility service chiefs had been forced at times to “provide their best estimates” to inform decisions because of the gaps in metrics.
VA uses a collection of organizational performance metrics, the Strategic Analytics for Improvement and Learning (SAIL) model, to facilitate internal and external benchmarking, identify strengths and areas of improvement, and facilitate the sharing of strong practices across VA healthcare systems.
However, the OIG learned from a leader in VHA’s Office of Performance Measurement that of 103 metrics necessary to populate the facility’s SAIL metrics, only 13 were available to the facility, and 90 were partially or not available. The OIG accessed VHA’s public website and found that in fiscal year 2021, no SAIL metrics were provided for the facility. The OIG said it is concerned that the lack of organizational performance metrics precludes an understanding of actual performance and data-driven decision-making at the facility.
EHR metrics of quality and patient safety enable the assessment of timely, effective, safe and veteran-centered care at VA facilities, which allow for comparison to private-sector care. However, the OIG determined that many quality and patient safety metrics were unavailable for the facility. One year after go-live, the VA’s Mission Act Quality Community Comparison website listed only 4 of 12 effective care measures for the facility. Further, VHA did not publish data that compares the facility’s quality outcomes to established quality benchmarks.
The OIG was told, one year after go-live, that VHA’s Office of Analytics and Performance Integration was working with Cerner to provide certified National Committee for Quality Assurance’s Healthcare Effectiveness Data and Information Set (HEDIS) measures for the facility. A leader in VHA’s Office of Performance Measurement informed the OIG that 17 metrics needed for hospital accreditation by The Joint Commission were unavailable at the facility.
Another VHA leader told the OIG, “Absolutely not,” when asked about the facility’s readiness for an upcoming accreditation survey by The Joint Commission. The OIG is concerned that missing quality and patient safety metrics thwart accurate and timely patient safety monitoring and could impede identification of opportunities for quality improvement.
The OIG remains concerned that, despite the concerted efforts of facility staff to use workarounds to manage gaps in the new EHR’s metrics, deficits in new EHR metrics may negatively affect organizational performance, quality and patient safety, and access to care.
The OIG identified multiple factors contributing to the significant gap in metrics available to the facility following go-live. Challenges with the new EHR’s metrics included the following factors:
• Cerner’s failure to deliver metrics reports,
• New EHR’s metrics could not be assessed prior to go-live,
• New EHR’s metrics utility was impaired, and
• Training deficits with new EHR metrics.
VHA-generated metrics using new EHR data also created challenges:
• VHA resources were insufficient for generating new EHR metrics,
• VHA metrics using new EHR data were not validated and unavailable, and
• VHA changed metrics required from the facility.
The OIG determined that deficiencies related to the new EHR’s metrics and challenges with VHA-generated metrics using new EHR data impaired the facility’s access to and utilization of metrics.
The OIG is concerned that further deployment of the new EHR in VHA without addressing the gap in metrics available to the facility will affect the facility and future sites’ ability to utilize metrics effectively.
Accordingly, to address the gaps in metrics available to the facility and future sites, the OIG said that the VA must resolve the factors that affect the availability of metrics.
The OIG made two recommendations to the Deputy Secretary related to evaluating gaps in new EHR metrics and the factors affecting the availability of metrics and taking action as warranted. The Office of the Deputy Secretary concurred with the recommendations and provided acceptable action plans, the OIG said.
