Research and applications Valid value Palpatation CT Chest x-ray Spiral CT Plain x-ray MRI Value meaning Palpation Computed Tomography Chest Radiography Spiral CT X-Ray Magnetic Resonance Imaging NCIt code (meaning concept) C16950 C17204 C38103 C20645 C17262 C16809 Table 3 Permissible values of the ﬁrst example from the data elements identiﬁed with inconsistent codes (highlighted in bold and italic) NCIt concept label Semantic type Type code Semantic group Group code Palpation Computed Tomography Chest Radiography Spiral CT X-Ray Magnetic Resonance Imaging Diagnostic Procedure Diagnostic Procedure Diagnostic Procedure Diagnostic Procedure Natural Phenomenon or Process Diagnostic Procedure T060 T060 T060 T060 T070 T060 Procedures Procedures Procedures Procedures Phenomena Procedures PROC PROC PROC PROC PHEN PROC Data element: Malignant Neoplasm Measurable Disease Evaluation Method Clinical Trial Eligibility Criteria Type (3179024). Value domain: Malignant Neoplasm Measurable Disease Evaluation Method Type (3179022). NCIt, National Cancer Institute Thesaurus. Table 3 shows the ﬁrst example CDE and permissible values. Using the semantic groups linked to each permissible value, we could see that the dominant semantic group of the permissible values is PROC (Procedures), whereas there is an outlier, the semantic group of which is PHEN (Phenomena). Although the label of the asserted NCIt concept code C17262|X-Ray of the outlier matched exactly the strings of the valid value and value meaning, this is not a correct code for the case. The value domain of the CDE actually references the values related to Procedure. The valid value Plain X-Ray here corresponds to the concept C38101|Radiography, the deﬁnition of which is ‘A radiographic procedure using the emission of X-Rays to form an image of the structure penetrated by the radiation’. The term ‘X- Ray’ has been listed as a synonym of the concept in the NCIt. Table 4 shows the second example CDE and permissible values. Using the semantic groups linked to each permissible value, it is clear that the dominant semantic group of the permissible values is CONC (Concepts & Ideas), whereas there is an outlier, the semantic group of which is DISO (Disorders). The asserted NCIt concept code of the outlier is C14714| Metastatic, the semantic type of which in the NCIt is Finding. We argue that the valid value Metastatic in the context of this CDE is used to indicate the tumor status and should be a code with the semantic type Qualitative Concept or Functional Concept as those other asserted codes. More interestingly, we found that the corresponding NCI Meta-thesaurus code of the NCIt code C14714|Metastatic is C0036525, the semantic type of which is classiﬁed as Functional Concept. This code may be more appropriate for the annotation. DISCUSSION Our results demonstrate that the semantic group is a potential target for detecting inconsistent permissible values for a CDE. We hypothesized that all the permissible values of the enumer- ated value domain of a CDE should come from a single semantic group. A total of 8705 (73.8%) enumerated CDEs met the criteria, while the remaining 3093 (26.2%) fell into multiple groups. A manual evaluation on 100 of the 3093 CDEs identiﬁed 17 that had inconsistency issues. The majority of the remaining 83 exhibited a dominant + residual/contextual pattern, where the permissible values of a CDE had a dominant semantic group (ie, the semantic group for the majority of permissible values), in addition to one or more residual/contextual semantic group(s). A typical example is illustrated in table 1. For the permissible values of the value domain Lesion Anatomical Site, the domi- nant semantic group is ANAT (Anatomy), and we can also see the other semantic group CONC (Concepts & Ideas), which is linked to a residual value, C17649|Other. From other examples, in addition to a dominant semantic group, we could see the contextual values such as C17998|Unknown, C41132|None, C48660|Not Applicable, etc, which mostly belong to the semantic group, CONC (Concepts & Ideas). The residual cate- gory issue is a variant of the categorization NEC (not elsewhere classiﬁed) issue,23 where the code is intended to represent a member of the base class not included in the set itself (eg, other ‘anatomical site’, other ‘procedure’, etc). The contextual values represent a mixture of negation (‘NO’ anatomical site) and meta-data about the data element itself (‘it doesn’t apply’, ‘we weren’t able to ﬁll it out because .’). Rules could be put into place to detect these patterns that could improve the sensitivity of the inconsistency auditing, but it should be noted that they represent another potential barrier to interoperability and will need to be addressed. Within the 17 CDEs with identiﬁed inconsistency issues, we ﬁrst identiﬁed situations where permissible values were used to create a single, composite meaning. As an example, the value Hormone Therapeutic Procedure For Breast Carcinoma was associated with four NCIt concept codes: C2315|Hormone, C49236|Therapeutic Procedure, C64956|For and C4872|Breast Carcinoma. This posed a challenge to our approach, as we had to determine a primary code to represent the semantic group of the permissible value. In our manual review, we heuristically Table 4 Permissible values of the second example from the data elements identiﬁed with inconsistent code (highlighted in bold and italic) Value meaning Residual RECURRENT Invasive PRIMARY METASTATIC MALIGNANT Not Applicable BENIGN NCIt code (meaning concept) C37895 C14173 C14159 C25251 C14174 C14143 C48660 C14172 NCIt concept label Semantic type Type code Residual Recurrent Invasive Primary Metastatic Malignant Not Applicable Benign Qualitative Concept Qualitative Concept Qualitative Concept Qualitative Concept Finding Qualitative Concept Qualitative Concept Qualitative Concept T080 T080 T080 T080 T033 T080 T080 T080 Semantic group Concepts & Ideas Concepts & Ideas Concepts & Ideas Concepts & Ideas Disorders Concepts & Ideas Concepts & Ideas Concepts & Ideas Group code CONC CONC CONC CONC DISO CONC CONC CONC Data element: Glioblastoma Pathology Or Primary Neoplasm Metastatic Neoplasm Status Tumor Status Text Name (2673966). Value domain: Tumor Status Text Name (2231026). NCIt, National Cancer Institute Thesaurus. Valid value Residual Recurrent Invasive Primary Metastatic Malignant Benign e134 J Am Med Inform Assoc 2012;19:e129ee136. doi:10.1136/amiajnl-2011-000739