Evaluating the Reliability of EHR-Generated Clinical Outcomes Reports: A Case
Study
Authors: Kanger Chatrian , Brown Lisanne , Mukherjee Snigdha , Xin Haichang , Diana Mark , Khurshid Anjum 
Published: October 2014

Introduction: Quality incentive programs, such as Meaningful Use, operate under
the assumption that clinical quality measures can be reliably extracted from EHRs.
Safety Net providers, particularly Federally Qualified Health Centers and Look-Alikes,
tend to be high adopters of EHRs; however, recent reports have shown that only about 9%
of FQHCs and Look-Alikes were demonstrating meaningful use as of 2013. Our experience
working with the Crescent City Beacon Community (CCBC) found that many health centers
relied on chart audits to report quality measures as opposed to electronically
generating reports directly from their EHRs due to distrust in the data. This paper
describes a step-by-step process for improving the reliability of data extracted from
EHRs to increase reliability of quality measure reports, to support quality improvement,
and to achieve alignment with national clinical quality reporting
requirements.Background: Lack of standardization in data capture and reporting within
EHRs drives distrust in EHR-reported data. Practices or communities attempting to
achieve standardization may look to CCBC’s experience for guidance on where to start and
the level of resources required in order to execute a data standardization project.
During the time of this data standardization project, CCBC was launching an HIE. Lack of
trust in EHR data was a driver for distrust in the HIE data.Methods: We present a case
study where a five-step process was used to harmonize measures, reduce data errors, and
increase trust in EHR clinical outcomes reports among a community of Safety Net
providers using a common EHR. Primary outcomes were the incidence of reporting errors
and the potential effect of error types on quality measure percentages. The activities
and level of resources required to achieve these results were also documented by the
CCBC program.Findings: Implementation of a community-wide data reporting project
resulted in measure harmonization, reduced reporting burden, and error reduction in
EHR-generated clinical outcomes reporting across participating clinics over a nine-month
period. Increased accuracy of clinical outcomes reports provided physicians and clinical
care teams with better information to guide their decision-making around quality
improvement planning.Discussion: A number of challenges exist to achieving reliable
population level quality reporting from EHRs at the practice, vendor, and community
levels. Our experience demonstrates that quality measure reporting from EHRs is not a
straightforward process, and it requires time and close collaboration between clinics
and vendors to improve reliability of reports. Our experience found that practices
valued the opportunity and step-wise process to validate their data locally (out of
their EHRs) prior to reporting out of the HIE.Conclusion and Next Steps: Communities can
achieve higher levels of confidence in quality measure reporting at the population level
by establishing collaborative user groups that work with EHR vendors as partners and use
technical assistance to build relationships and trust in EHR-generated reports. While
this paper describes the first phase of our work around improving standardization and
reliability of EHR reports, vendors should continue to explore modifications for
improving data capture (at the front-end) via standardized data entry
templates.