A b s t r a c t Current efforts to define and implement health data standards are driven by issues related to the quality, cost and continuity of care, patient safety concerns, and desires to speed clinical research findings to the bedside. The President's goal for national adoption of electronic medical records in the next decade, coupled with the current emphasis on translational research, underscore the urgent need for data standards in clinical research. This paper reviews the motivations and requirements for standardized clinical research data, and the current state of standards development and adoption-including gaps and overlaps-in relevant areas. Unresolved issues and informatics challenges related to the adoption of clinical research data and terminology standards are mentioned, as are the collaborations and activities the authors perceive as most likely to address them. Ⅲ J Am Med Inform Assoc. 2007;14:687-696. DOI 10.1197/jamia.M2470. Efforts to build a national health information infrastructure (NHII) and supporting data standards must address the needs of clinical research. 1 Clinical research, as defined by the National Institutes of Health (NIH) is patient-oriented research conducted with human subjects (or on material of human origin that can be linked to an individual). 2 Clinical research includes investigation of the mechanisms of human disease, therapeutic interventions, clinical trials, development of new technologies, epidemiology, behavioral studies, and outcomes and health services research. The broad scope of clinical research, coupled with the infusion of technology, has generated increasing amounts of data, and the scientific community needs to identify strategies to share it in meaningful ways. The NIH policy on the sharing of research data 3 is bringing forth questions about how data should be represented for data sharing, and making the need for clinical research data standards critical and immediate. Data standards are defined here as consensual specifications for the representation of data from different sources or settings. Standards are necessary for the sharing, portability, and reusability of data. 4 -7 The notion of standardized data includes specifications for both data fields (ϳvariables) and value sets (ϳcodes) that encode the data within these fields. Although the current data standards focus is on regulated research (often the narrower context of clinical trials) and their business activities (e.g., safety reporting, study reporting to regulatory bodies), it is important to mention that clinical research includes many other types of research, including observational, epidemiological, and outcomes research, as well as molecular and biology research (e.g., genetics and biomarkers for disease). Although important, this discussion does not address the "-omics" standards, 8 but rather clinical, laboratory, procedure and observation data collected in the context of clinical research subject visits. The permeation of clinical research data standards that are harmonious with clinical care standards is required for the sharing of patient data between healthcare and researchone ambition of the NHII. 9 The goals for the NHII include the seamless integration of clinical research data to/from patient care data to/from population data and existing medical knowledge bases, making standardized data in clinical research a high priority. 6,9,10 Interoperability between healthcare and clinical research data can create opportunities for increased subject enrollment, evidence-based medicine, and population monitoring. This paper describes data standards requirements for subject data in the clinical research domain, the nature of overlaps and gaps in current standards coverage, and highlights key informatics challenges that remain. Affiliations of the authors: Pediatrics Epidemiology Center (RLR, JK), University of South Florida, Tampa, FL The project described was supported by Grant Number RR019259 from the National Center for Research Resources (NCRR), a component of the National Institutes of Health (NIH). The authors thank the members of HL7 and CDISC vocabulary and terminology teams whose hard work and dedication provided inspiration for this paper, as well as the members of the RDCRN Standards Committee. The authors also thank the Office of Rare Diseases for their support. Contents of the project are solely the responsibility of the authors and do not necessarily represent the official views of NCRR or NIH. The authors are grateful for the thorough reviews and insightful comments of the two anonymous reviewers, whose contributions have strengthened the value and accuracy of this manuscript.  There are several stakeholders (Appendix 1 available as an online JAMIA supplement at www.jamia.org) for the uniform adoption of data standards in clinical research including both pharmaceutical companies and regulatory agencies such as the U.S. Food and Drug Administration (FDA) and its international equivalent the International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH). Other stakeholders include academic and government-based clinical researchers, and government agencies that are large sponsors of clinical research in specified domains (e.g., National Cancer Institute, National Heart, Lung, and Blood Institute) and those NIH institutes that sponsor clinical research agendas (e.g., National Center for Research Resources, NCRR). Because of the importance of data standards in healthcare delivery and federally funded research, various NIH institutes have standards efforts and dedicated resources for examining data standards within their domains of interest. The NCRR is exploring the impact of current government standards initiatives upon broad clinical research interests expressly. In 2003, all federal agencies with health data interests committed to adopt a set of voluntary data standards recommended by the Consolidated Health Informatics (CHI) initiative, a U.S. based multi-agency movement for healthcare data standards. 12 Their recommended standards for over 20 areas of healthcare data have been endorsed by the Department of Health and Human Services' Office of the National Coordinator for Health Information Technology (ONC). 13 While the CHI standards defined in 2004 have yet to be widely implemented in the commercial world, government agencies, such as Centers for Medicare and Medicaid Services (CMS) and the Veteran's Administration (VA), are striving to incorporate them. Some CHI standards, such as HL7, were well-used prior to being named CHI standards, while others, such as SNOMED CT, were not widely implemented and their adoption in federal healthcare activities is moving more slowly. The CHI standards identification teams of 2004 postponed recommendations on many key data areas, including physical exam, medical history, and adverse events, and it is not clear if the ONC's Health Information Technology Standards Panel will continue with that agenda. Clinical research interests were not specifically represented in the CHI activities of 2003-4, but advocates for clinical research needs in health IT data standards do have a formal presence in the current ONC activities. The National Library of Medicine (NLM) operates in a leadership and coordination role in the identification of an interlocking set of standards that would completely address all NHII data representation needs. The NLM has developed and maintains knowledge sources and tools to facilitate access to standard terminologies as well and to coordinate their use. 14 In 2003, the NLM procured a public U.S. license for the use of SNOMED CT via their Unified Medical Language System (UMLS). The UMLS is a multi-purpose resource that includes concepts and terms from over 100 different source vocabularies, and establishes linkages across these source vocabularies with its semantic network representation of important concepts and relationships in the biomedical domain. 15,16 The UMLS tools support the mapping of multiple data standards to a common set of concepts via the Metathesaurus, 17 and the NLM historically functions in a coordinator and funder role in creating needed mappings across heterogeneous data standards. Additionally, the NLM has been supporting the development of tools for the access and use of CHI standards, and for achieving interoperability between them. They have funded LOINC and special projects that facilitate government agency interests in standards, including HL7. Additionally, NLM has developed and delivers RxNorm, a database of drug concepts, including the clinical drug (drug ϩ route ϩ dose) which is being linked with the FDA's new structured product labels (i.e., package inserts) effort. The ICH is a collaboration of the regulatory authorities of Europe, Japan and the United States that formed in 1990 with the aim of harmonizing scientific and technical aspects of product registration in order to eliminate the need for duplicate testing in the development of new medicines. The ICH has developed multiple reference models for the clinical research domain and the E2B specification of data elements for transmission of Individual Case Safety Reports of all types of individual case safety reports, regardless of source and destination. 18 The E2B data model standard is the foundation that other clinical research data standards groups are drawing from to support development of standardized electronic regulatory data reporting applications in the short term. The dominant discussion forums for moving toward clinical research data standards that support applied uses are the Clinical Data Standards Interchange Consortium (CDISC) and the Regulated Clinical Research (RCRIM) Technical Committee of Health Level Seven (HL7). These groups are very different in terms of membership, organization, and purpose. The CDISC membership is dominantly from large pharmaceutical companies world-wide, but also includes the FDA as well as representation from governmental agencies such as the VA and NCI. The immediate goal of CDISC is to create standard data models for regulatory submissions. While CDISC has commendable industry participation and motivation, it is not a formal standards development organization, and there is a risk that the organization might not address the needs of all stakeholders. HL7 is a not-for-profit volunteer organization dedicated to produce standards for clinical and administrative data in all health settings, and is an American National Standards Institute (ANSI)-accredited Standards Developing Organization (SDO). 19 Like all ANSI-accredited SDOs, HL7 adheres to a strict and well-defined set of operating procedures that ensures consensus, openness and balance of interest. The Regulated Clinical Research Information Management (RCRIM) Technical Committee shares some of the same goals as CDISC, but also represents broader clinical research and patient safety interests. Because HL7 also addresses more stakeholders than just clinical researchers, the time for discussion and approval of the standards can be lengthy. This formal and required discussion and approval of developing standards throughout HL7 increases the likelihood that the standards that are defined in technical committees and special interest groups such as the RCRIM, will be 688 by guest on January 6, 2016 http://jamia.oxfordjournals.org/ Downloaded from interoperable, or harmonious with, the emerging messaging standards in other healthcare domains. Both CDISC and HL7 are developing models for messaging or transferring data (e.g., records, reports, or data sets) to drug research regulatory organizations. The CDISC effort focuses on building formal data models for regulatory reporting (to FDA). Although current (2.x) versions of HL7 use relatively simple models to support messaging in clinical settings, the HL7 version 3 (not widely implemented) relies upon a very abstract information model, the Reference Information Model (RIM) that is broad and flexible enough to address any messaging need in the healthcare domain. CDISC is developing more practical data models (designed for very narrow, explicit regulatory needs), with a finite set of variables needing controlled vocabulary. Both HL7 and CDISC have terminology teams or liaisons with terminology groups tasked to put the terminology (or controlled value sets) in the slots of the information models that they have developed, but the extensive differences in the HL7 and CDISC models raise the concern that "ideal" terminology standards for each application might differ between the groups. Heterogeneity across HL7 and CDISC models also invites the risk that the same terminology standards can be applied differently in different applications. In addition to terminology issues, the harmonization of HL7 and CDISC standards for message/information structure, encoding of data types, and communication conventions are imperative. CDISC and HL7's RCRIM groups began formally working together in 2001 and are committed to achieving syntactic and semantic interoperability between their standards. It is not clear whether the CDISC data models and the broader HL7 messages will ever be synchronized, but there is promising work in harmonizing CDISC lab data reporting standards with HL7 (version 2 and 3) messages, including the use of LOINC test codes within both models. The incentive to harmonize the CDISC and HL7 models is strong, although the task is daunting due to the differences in complexity, conceptualization, and in the levels of abstraction between the models. The Biomedical Research Integrated Domain Group (BRIDG) formed in 2005 solely to link the CDISC data reporting models with the HL7 RIM. The (BRIDG) model is a domain analysis model of protocol-driven biomedical and clinical research, developed to provide a comprehensive conceptual model of the clinical research domain as a basis for harmonization across information model standards. The domain model is the result of work from HL7 RCRIM, CDISC, NCI, and the FDA. The BRIDG Model has recently been formally adopted by both CDISC and the HL7 (RCRIM) Technical Committee as their domain analysis model and it is supporting National Cancer Institute's (NCI) cancer Bioinformatics Grid (caBIG). The BRIDG model is intended to be the conceptual backbone to which all CDISC and HL7 RCRIM implementation models link, thereby creating interoperable applications within and across both organizations. Proof of concept and pilot demonstrations of this harmonization are in early development. Although not a formal SDO, the NCI has been building a standards infrastructure for years, offers robust terminology resources, and is an important and active participant in most clinical research data standards venues. The NCI has developed the Common Terminology Criteria for Adverse Events (CTCAE), a standard for adverse events (AE), which is perhaps the most comprehensive AE classification for general clinical research, despite its origins in oncology. The NCI has created a strong infrastructure for terminology maintenance, mapping, and access activities. 20 The NCI serves as a host for CDISC data elements, value sets, and terminology as they are being defined. 21 In addition, the NCI is hosting controlled terminology for the FDA. The NCI has been successful at standardizing (to some extent) clinical research data within its many sponsored studies, although they too are struggling with the standards gaps and overlaps we describe in the next section. One of the greatest challenges in identifying data standards for the clinical research domain is to reconcile the requirements of varied investigators and data users with the need for common standards. While individual investigators might only want a subset of an information model or a standardized terminology, collectively clinical research needs terminologies with comprehensive coverage, including broad concepts (e.g., "abnormal nervous system finding") and detailed ones (e.g., "abnormal lateral conjugate gaze"). Further, clinical research workflows create requirements for representing important nuances of clinical and research data, particularly in regulatory contexts, which differ from other healthcare areas. For example, although AEs are clinical findings and observations, they are defined in the context of a research protocol, imply subject participation in a study procedure (usually intervention), and contain other semantic dimensions (e.g., attribution, expectedness). The clinical research domain, as a whole, includes data from the spectrum of broad constructs shown in online Table 1 . If there is a current U.S. standard, the standard and the organization naming the standard are listed. We characterized whether each construct had a gap or an overlap in named U.S. standards. Competing standards are listed for the constructs with overlaps, and potential standards (with at least some relevant content) are listed for the constructs with gaps in standards coverage. Because standardized data includes specifications for both data fields (ϳvariables) and value sets (ϳcoding systems), the state of standards adoption is described by both variables and value sets for most constructs examined. Conceptualizations of the clinical research domain (i.e., the definition and organization of constructs) can be based upon data content types (e.g., clinical observations, reported symptoms, diagnoses) or by workflow activities or artifacts (e.g., physical exam, medical history, adverse events). In Table 1 , we present constructs that are related to both data content types and workflow activities in order to mesh with how other groups such as CHI conceptualized health care data domains for endorsing standards. We present Table 1 as a starting point to visualize broad target areas for data standards in the clinical research domain, and to illustrate the number of data areas that are missing named standards ("gaps") or have one or more competing standards ("overlaps").  While some clinical research constructs are covered by existing data standards, there are important areas that have gaps in standards coverage. These include study descriptive information (including study design and randomization/ blinding features), subject and provider identifiers, study eligibility criteria, subject disposition descriptors (e.g., eligible, enrolled, loss to follow-up), study events (e.g., baseline visit, follow-up visit, physical exam), protocol deviations, medical device names, and vital signs. Although data standards for medical device names and vital signs are relevant to broader healthcare interests, the majority of gaps in clinical research data standards are those that are very unique to clinical research. This is understandable considering that the CHI initiative primarily was focused on naming data standards for health care delivery. For constructs with standards gaps, some (subject identifiers, protocol deviations) have just one or a few candidate standards and no apparent de facto standards with a broad user base in clinical research. Other constructs with formal standards gaps (study descriptors, demographics, subject disposition, medical devices, and vital signs) have multiple candidate standards and overlap of standards is likely in the future. Another gap, listed as a separate construct in Table 1 but relevant to all construct areas, is needed standards for encoding missing data-e.g., unknown, not reported, not assessed, refused, etc. While this has been addressed by the "flavors of null" work within HL7 version 3, it needs to be simplified and adopted by data collection and management in the clinical research domain. There is a conspicuous lack of named standards for the structuring of questions and case report forms, particularly in the areas of physical exam, medical history, family history, and eligibility criteria. It is important to note that while named standards (i.e., SNOMED CT and MedDRA) exist for the content of these activity areas, 22,23 standards for how they are used (metadata and question modeling) are both important and lacking. Such standards would support a consistent structure and use of standardized terminologies in a variety of applications, including interfaces to electronic health records, public health questionnaires, message models such as HL7, or clinical research case report forms. The importance of question-level metadata is evidenced by applications such as the NCI's Cancer Data Standards Repository 20 and CDISC's Clinical Data Acquisition Standards Harmonization (CDASH) project-a new project focused on the development of data standards in case report form design. 21, 17 The importance of the exchange and reuse of federally-required patient/client assessment and other functioning and disability content to Centers for Medicare and Medicaid Services (CMS), and other U.S. Department of Health and Human Services (DHHS) agencies, resulted in placing this area on the CHI agenda. The CHI standard (Fall 2006) for the area of Functioning and Disability 24 includes a combination of data standards, including LOINC, to represent the items and batteries of items on standardized federally-required patient/clinical assessments and other disability content across the federal healthcare enterprise. The recommendations were preceded by a DHHS-sponsored work group that looked at many of the questions and determined that the majority of clinical (ϳhealth related) content (which the team called "usefully related" content) was covered to some degree by relevant vocabularies such as International Classification for Functioning, Disability, and Health (ICH) and SNOMED CT, but that structural features of questions, including answer groups, were not represented well by recommended vocabularies.
In particular, the lack of a standards-based information model has been recognized as a major barrier for achieving computable diagnostic criteria REF .
Data Standards in Clinical Research: Gaps, Overlaps, Challenges and Future Directions
Journal of the American Medical Informatics Association : JAMIA
Journal of the American Medical Informatics Association : JAMIA
Computer Science Medicine