databases, or patient surveys, as opposed to being calculated from EHR derived data. This was driven in large part by the historically low rates of EHR use nationwide and the ensuing lack of data availability, quality, and comparability. Studies comparing claims data with clinical data have noted signiﬁcant disparities between the two sources,4 5 yet claims-based quality measurement has continued to be the dominant form of large- scale quality analysis because no other data sources have been readily available either as a complement or replacement. In some measures, such as breast cancer screening, the use of adminis- trative or claims data may still be more reliable until health information exchanges are established, more broadly adopted, and can integrate multiple data sources to establish more comprehensive measures on recommended care delivery and health outcomes. EHR offer new potential for performance measurement given that most commercially available EHR use standard dictionaries to capture information in coded forms, such as ICD for problem list and SNOMED for medications. Using these codiﬁed data, EHR can help identify patient populations and calculate a signiﬁcant number of quality measures that leverage data available in the EHR.6 These measures can range from adherence to clinical guidelines to assessments of rates of clinical preven- tive services to rates of screening. However, EHR-derived quality measurement has limitations due to several factors, most notably variations in EHR content, structure and data format, as well as local data capture and extraction procedures.7 8 Several steps can be taken to mitigate the variability of EHR documentation. As part of PCIP’s programme, providers are trained on proper documentation techniques during their initial EHR training, and then quality improvement specialists rein- force their use, but ultimately there are no mechanisms to force providers to document in a particular location in the chart. Providers need regular prompts, training and feedback to alter their documentation habits. Studies have shown that clinical decision support can help improve the quality and accuracy of documentation.9 Another way to mitigate the variability of documentation would be to include claims data to populate the EHR, thereby providing a more robust and complete proﬁle of the patient. In addition, standards need to be developed for what needs to be documented in the various medical record compo- nents, such as a clinical encounter note or a care plan document. Much work is being done to standardize the output of EHR for use in health information exchange (eg, the continuity of care document), but few efforts are aimed at standardizing what data inputs should go into the EHR. Research and applications More studies are needed to assess the validity of EHR-derived quality measures and to ascertain which measures are best calculated using claims or administrative data or a combination of data sources. If provider-speciﬁc quality measurements are to be reported and made public, as is the plan for the meaningful use quality measures, further analysis is needed to understand the limitations of these data, particularly if they are prone to underestimation of true provider performance. Acknowledgments The authors would like to acknowledge Dr Farzad Mostashari for his initial concept and design of PCIP as well as his vision for the prevention- oriented EHR tools deployed in this project. They would also like to thank the participating practices and the Island Peer Review Organization staff for their dedication and many hours spent conducting chart reviews. They also wish to thank the PCIP staff for their tireless dedication to improving health. Funding This study was supported by the Agency for Healthcare Research and Quality (grant nos R18HS17059 and 17294). The funder played no role in the study design, in the collection, analysis and interpretation of data, in the writing of the report or in the decision to submit the paper for publication. Competing interests None. Ethics approval This study was approved by the Department of Health and Mental Hygiene institutional review board no 09067. Provenance and peer review Not commissioned; externally peer reviewed. REFERENCES 1. Blumenthal D. Promoting use of health IT: why be a meaningful user? Md Med 2010;11:18e19. 2. CMS Ofﬁce of Public Affairs. Fact Sheet. Details for Medicare and Medicaid Health Information Technology: Title IV of the American Recovery and Reinvestment Act. 2009. http://www.cms.hhs.gov/apps/media/press/factsheet.asp?Counter¼3466& intNumPerPage¼10&checkDate¼&checkKey¼&srchType¼1&numDays¼3500& srchOpt¼0&srchData¼&keywordType¼All&chkNewsType¼6&intPage¼& showAll¼&pYear¼&year¼&desc¼&cboOrder¼date (accessed 18 Aug 2011). 3. Frieden TR, Mostashari F. Health care as if health mattered. JAMA 2008;299:950e2. 4. Jollis JG, Ancukiewicz M, DeLong ER, et al. Discordance of databases designed for claims payment versus clinical information systems. Implications for outcomes research. Ann Intern Med 1993;119:844e50. 5. Tang PC, Ralston M, Arrigotti MF, et al. Comparison of methodologies for calculating quality measures based on administrative data versus clinical data from an electronic health record system: implications for performance measures. J Am Med Inform Assoc 2007;14:10e15. 6. Roski J, McClellan M. Measuring health care performance now, not tomorrow: essential steps to support effective health reform. Health Aff (Millwood) 2011;30:682e9. 7. Kahn M, Ranade D. The impact of electronic medical records data sources on an adverse drug event quality measure. J Am Med Inform Assoc 2010;17:185e91. 8. Chan K, Fowles J, Weiner J. Electronic health records and the reliability and validity of quality measures: a review of the literature. Med Care Res Rev 2010;67:503e27. 9. Galanter WL, Hier DB, Jao C, et al. Computerized physician order entry of medications and clinical decision support can improve problem list documentation compliance. Int J Med Inform 2010;79:332e8. PAGE fraction trail=5.75 J Am Med Inform Assoc 2012;19:604e609. doi:10.1136/amiajnl-2011-000557 609