Can't you just pull the data? The limitations of using of the electronic medical record for research

With the proliferation of electronic medical records (EMR), there is an increasing interest in using the information for the purposes of research. Although the EMR is an important source of data and information, care must be taken by researchers and readers to understand the limitations surrounding these types of data. It is vital to understand the design of the EMR and its benefits and limitations for research. An electronic medical record is ‘an electronic record of health-related information on an individual that can be created, gathered, managed, and consulted by authorized clinicians and staff within one health care organization’ (1). The main purpose of an EMR is to allow for the documentation of observations by a clinician and to allow for informed treatment plans. Large volumes of data are created that may have multiple secondary purposes including administrative data for billing and reporting; public health for the detection of patterns of disease outbreaks or complications; clinical support through decision-making software such as medication interactions; and rich information for research studies. However, an EMR contains large amounts of data with little, or no ability, to know or report the quality of the data contained. The medical data in an EMR can be divided into categories of data including narrative, structured, and imaging (2). Narrative data include any description of the patient by a health care provider. These data may be handwritten and scanned, typed, or dictated, and subsequently transcribed. Structured data include any measurement performed on a patient’s physical exam, laboratory tests, and measurements from imaging studies. Structured data also include standardized data such as the patient’s name and primary diagnosis. Imaging data include both radiological images and related letters or other documents that have been scanned into the EMR. A single medical datum or one observation can be divided into five elements: the patient identity, the parameter observed, the value of the parameter, the time of the observation, and the circumstances of the observation (2). However, data elements often do not incorporate any degree, measure, or range of uncertainty in the values of the data. For example, a physician listens to the heart in a patient who has chest pain and thinks that he or she hears a murmur. Or, a nurse thinks that the oximeter was not reading properly and does not record the possible hypoxemia. Thus, whenever data are used from an EMR for research, consideration must be made not only to what is recorded but also what is or is not reliably entered. There are no specific measures of quality that are readily available for what is entered into the EMR. Data quality in research can be divided into five primary areas: relevance, timeliness, usability, comparability, and accuracy (3). Relevance refers to the ability of a dataset to answer relevant health questions and to adapt over time. Timeliness is the ability of a dataset to be current and meet established timelines. Usability is the availability of documentation about the data, both a description of the data elements contained as well as availability of standardized analyses as appropriate. Comparability should be maintained using established definitions and always maintaining original data elements as well as any calculated elements. Accuracy is perhaps the most important component that needs to be ensured to attain the highest level of data quality. Without the assurance of data accuracy, the other components of quality become moot, as the interpretation of any derived analysis, subsequent information, and future knowledge is, at best, compromised or, at worst, false. To achieve accuracy in data, the coverage of the data should be well defined (3). Coverage refers to the difference between the data that should be included in the dataset and the data that are actually included in the dataset. Three features of data should be emphasized as indicators of the accuracy of the data reported: measurement error, bias, and consistency (3). Measurement error is what degree at which the data reported matches what should have been reported, or some measure of the ‘truth’. As an example, mismeasurement occurs when a blood pressure cuff consistently overreports the true or actual blood pressure. Bias is the systematic degree of difference between reported and true data elements. Possible bias can be difficult to prove, and may be correlated with other variables in a dataset. Consistency is the reliability or repeatability of measures. Subjective variables can be very difficult to ensure consistency. Obviously, having this level of detail of data quality for the entire EMR is impossible. There are thousands of fields entered including discrete data points, continuous physiologic variables, and free text. It is not possible or even practical to consider reporting the quality of an entire EMR. However, when research is considered, a plan should be developed to assess the quality of the data. Simply ‘pulling data’ from an electronic source