Department of Health Sciences Research, Mayo Clinic College of Medicine, Rochester, Minnesota, USA Correspondence to Dr Guoqian Jiang, Department of Health Sciences Research, Mayo Clinic College of Medicine, 200 First Street, SW, Rochester, MN 55905, USA; jiang.guoqian@mayo.edu Received 30 November 2011 Accepted 18 March 2012 Published Online First 17 April 2012 This paper is freely available online under the BMJ Journals unlocked scheme, see http:// jamia.bmj.com/site/about/ unlocked.xhtml Research and applications Quality evaluation of value sets from cancer study common data elements using the UMLS semantic groups Guoqian Jiang, Harold R Solbrig, Christopher G Chute ABSTRACT Objective The objective of this study is to develop an approach to evaluate the quality of terminological annotations on the value set (ie, enumerated value domain) components of the common data elements (CDEs) in the context of clinical research using both uniﬁed medical language system (UMLS) semantic types and groups. Materials and methods The CDEs of the National Cancer Institute (NCI) Cancer Data Standards Repository, the NCI Thesaurus (NCIt) concepts and the UMLS semantic network were integrated using a semantic web-based framework for a SPARQL-enabled evaluation. First, the set of CDE-permissible values with corresponding meanings in external controlled terminologies were isolated. The corresponding value meanings were then evaluated against their NCI- or UMLS-generated semantic network mapping to determine whether all of the meanings fell within the same semantic group. Results Of the enumerated CDEs in the Cancer Data Standards Repository, 3093 (26.2%) had elements drawn from more than one UMLS semantic group. A random sample (n¼100) of this set of elements indicated that 17% of them were likely to have been misclassiﬁed. Discussion The use of existing semantic web tools can support a high-throughput mechanism for evaluating the quality of large CDE collections. This study demonstrates that the involvement of multiple semantic groups in an enumerated value domain of a CDE is an effective anchor to trigger an auditing point for quality evaluation activities. Conclusion This approach produces a useful quality assurance mechanism for a clinical study CDE repository. INTRODUCTION With the importance of value sets gradually being recognized by the clinical research community,1 standardization of value sets is becoming impera- tive, as it can enable comparison across disparate datasets and facilitate reuse of well-deﬁned value sets to advance clinical research studies. This standardization involves two componentsdaccess and content. Access to standardized value sets is a component of standardized terminological services. Notably, Health Level Seven International (HL7) has been developing effective and deployable solutions for value sets for many years, and is presently working on extending its standard speciﬁcation Common Terminology Services 2 (CTS2) to support formal subsets and value set deﬁnition functionality.2 The Centers for Disease Control and Prevention (CDC) has been publishing value sets through its Public Health Information Network Vocabulary Access and Distribution System (PHIN VADS).3 In the context of terminology/ontology services, a value set is a uniquely identiﬁable set of discrete mean- ings or ‘concepts’ that can be resolved as a unique collection of identiﬁers.4 Value sets can be derived from existing coding schemes such as SNOMED CT, either by enumerated lists or by constraints based on logical expressions (eg, all sub-codes of the code ‘breast cancer ’). Standardized value set content can be realized through the perspective of meta-data management and registry. One such effort is the ISO/IEC 11179 standard, which speciﬁes a meta-data model for representing the common data elements (CDEs).5 6 In the ISO/IEC 11179 standard, a ‘data element’ is deﬁned as ‘a unit of data for which the deﬁnition, identiﬁcation, representation and permissible values are speciﬁed by means of a set of attributes’.6 ISO/IEC 11179 refers to the set of permissible values for a CDE as a ‘value domain’. These value domains come in two ‘ﬂavors’d‘described value domains’ and ‘enumerated value domains’. Described value domains represent free text and numeric ﬁelds, while enumerated value domains represent coded informationda ﬁnite set of discrete values, each of which represents a predeﬁned meaning. Typical examples of a described value domain might include a ﬁeld that represents a person’s name or age, while the person’s gender would usually be represented as an enumerated domain. Enumerated value domains are composed of one or more ‘permissible values’, each of which represents a valid value for the ﬁeld. Each of these values, in turn, is tied to a corresponding ‘value meaning’, which represents the intended meaning of the permissible value in the context of the value domain. As an example, a patient gender value domain might have the permissible values ‘0’, ‘1’ and ‘9’, where ‘0’ represents the meaning ‘male’, ‘1’ ‘female’ and ‘9’ ‘indeterminate’. Sets of value meanings are grouped into ‘enumerated conceptual domains’. The solid lines in ﬁgure 1 illustrate link- ages among the components of the standard. Part of value of the ISO/IEC 11179 standard comes from the ability to align ﬁelds from different databases by connecting local value domains with common conceptual domains. A ﬁeld that repre- sents a person’s age in weeks can be aligned with a ﬁeld in another database that represents it in years. Similarly, a ﬁeld that represents patient J Am Med Inform Assoc 2012;19:e129ee136. doi:10.1136/amiajnl-2011-000739 e129