A Harmonized Data Quality Assessment Terminology and Framework for the Secondary
Use of Electronic Health Record Data
Authors: Kahn Michael G. , Callahan Tiffany J. , Barnard Juliana , Bauck Alan E. , Brown Jeff , Davidson Bruce N. , Estiri Hossein , Goerg Carsten , Holve Erin , Johnson Steven G. , Liaw Siaw-Teng , Hamilton-Lopez Marianne , Meeker Daniella , Ong Toan C. , Ryan Patrick , Shang Ning , Weiskopf Nicole G. , Weng Chunhua , Zozus Meredith N. , Schilling Lisa 
Published: September 2016

Objective: Harmonized data quality (DQ) assessment terms, methods, and reporting
practices can establish a common understanding of the strengths and limitations of
electronic health record (EHR) data for operational analytics, quality improvement, and
research. Existing published DQ terms were harmonized to a comprehensive unified
terminology with definitions and examples and organized into a conceptual framework to
support a common approach to defining whether EHR data is ‘fit’ for specific
uses.Materials and Methods: DQ publications, informatics and analytics experts, managers
of established DQ programs, and operational manuals from several mature EHR-based
research networks were reviewed to identify potential DQ terms and categories. Two
face-to-face stakeholder meetings were used to vet an initial set of DQ terms and
definitions that were grouped into an overall conceptual framework. Feedback received
from data producers and users was used to construct a draft set of harmonized DQ terms
and categories. Multiple rounds of iterative refinement resulted in a set of terms and
organizing framework consisting of DQ categories, subcategories, terms, definitions, and
examples. The harmonized terminology and logical framework’s inclusiveness was evaluated
against ten published DQ terminologies.Results: Existing DQ terms were harmonized and
organized into a framework by defining three DQ categories: (1) Conformance (2)
Completeness and (3) Plausibility and two DQ assessment contexts: (1) Verification and
(2) Validation. Conformance and Plausibility categories were further divided into
subcategories. Each category and subcategory was defined with respect to whether the
data may be verified with organizational data, or validated against an accepted gold
standard, depending on proposed context and uses. The coverage of the harmonized DQ
terminology was validated by successfully aligning to multiple published DQ
terminologies.Discussion: Existing DQ concepts, community input, and expert review
informed the development of a distinct set of terms, organized into categories and
subcategories. The resulting DQ terms successfully encompassed a wide range of disparate
DQ terminologies. Operational definitions were developed to provide guidance for
implementing DQ assessment procedures. The resulting structure is an inclusive DQ
framework for standardizing DQ assessment and reporting. While our analysis focused on
the DQ issues often found in EHR data, the new terminology may be applicable to a wide
range of electronic health data such as administrative, research, and patient-reported
data.Conclusion: A consistent, common DQ terminology, organized into a logical
framework, is an initial step in enabling data owners and users, patients, and policy
makers to evaluate and communicate data quality findings in a well-defined manner with a
shared vocabulary. Future work will leverage the framework and terminology to develop
reusable data quality assessment and reporting methods.