JMIR MEDICAL INFORMATICS

Lin et al

Original Paper

Dermoscopy Differential Diagnosis Explorer (D3X) Ontology toAggregate and Link Dermoscopic Patterns to Differential
Diagnoses: Development and Usability Study
Rebecca Z Lin1, MD; Muhammad Tuan Amith2,3,4, PhD; Cynthia X Wang5, MPHS, MD; John Strickley6, MD; Cui
Tao7, PhD
1Division of Dermatology, Washington University School of Medicine, St. Louis, MO, United States

2

Department of Information Science, University of North Texas, Denton, TX, United States

3

Department of Biostatistics and Data Science, The University of Texas Medical Branch, Galveston, TX, United States

4

Department of Internal Medicine, The University of Texas Medical Branch, Galveston, TX, United States

5

Department of Dermatology, Kaiser Permanente Redwood City Medical Center, Redwood City, CA, United States

6

Division of Dermatology, University of Louisville, Louisville, KY, United States

7

Department of Artificial Intelligence and Informatics, Mayo Clinic, Jacksonville, FL, United States

Corresponding Author:
Cui Tao, PhD
Department of Artificial Intelligence and Informatics
Mayo Clinic
4500 San Pablo Road
Jacksonville, FL, 32224
United States
Phone: 1 9049530255
Email: Tao.Cui@mayo.edu

Abstract
Background: Dermoscopy is a growing field that uses microscopy to allow dermatologists and primary care physicians to
identify skin lesions. For a given skin lesion, a wide variety of differential diagnoses exist, which may be challenging forinexperienced users to name and understand.
Objective: In this study, we describe the creation of the dermoscopy differential diagnosis explorer (D3X), an ontology linking
dermoscopic patterns to differential diagnoses.
Methods: Existing ontologies that were incorporated into D3X include the elements of visuals ontology and dermoscopy
elements of visuals ontology, which connect visual features to dermoscopic patterns. A list of differential diagnoses for eachpattern was generated from the literature and in consultation with domain experts. Open-source images were incorporated from
DermNet, Dermoscopedia, and open-access research papers.
Results: D3X was encoded in the OWL 2 web ontology language and includes 3041 logical axioms, 1519 classes, 103 object
properties, and 20 data properties. We compared D3X with publicly available ontologies in the dermatology domain using asemiotic theory–driven metric to measure the innate qualities of D3X with others. The results indicate that D3X is adequately
comparable with other ontologies of the dermatology domain.
Conclusions: The D3X ontology is a resource that can link and integrate dermoscopic differential diagnoses and supplementary
information with existing ontology-based resources. Future directions include developing a web application based on D3X for
dermoscopy education and clinical practice.
(JMIR Med Inform 2024;12:e49613) doi: 10.2196/49613
KEYWORDS
medical informatics; biomedical ontology; ontology; ontologies; vocabulary; OWL; web ontology language; skin; semiotic; webapp; web application; visual; visualization; dermoscopic; diagnosis; diagnoses; diagnostic; information storage; information
retrieval; skin lesion; skin diseases; dermoscopy differential diagnosis explorer; dermatology; dermoscopy; differential diagnosis;
information storage and retrieval
https://medinform.jmir.org/2024/1/e49613

XSL• FO
RenderX

JMIR Med Inform 2024 | vol. 12 | e49613 | p. 1
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Introduction
Dermoscopy is a noninvasive, in vivo microscopic techniqueused to examine skin lesions by detecting morphological featuresthat may not be seen by the naked eye [1-4]. Studies havedemonstrated that dermoscopy improves the diagnosis of bothpigmented skin lesions [3,5-7] and nonpigmented skin lesions[8], including neoplasms [9] and infectious and inflammatory
skin diseases [4,10]. Notably, the diagnostic accuracy ofdermoscopy is dependent on the examiner’s experience, as
dermoscopy by untrained or less experienced examiners wasfound to be no better than clinical inspection without
dermoscopy [6]. Learning dermoscopy is not just relevant todermatologists, but also for physicians in other medical
specialties. Patients with new or changing skin lesions often
first consult their primary care physician (PCP) rather than adermatologist. Dermoscopy has shown to be an effective tool
for the assessment and triage of pigmented skin lesions inprimary care, with improved diagnostic accuracy and referral
accuracy to dermatologists [11-13]. However, dermoscopy
training for PCPs is currently highly variable, with many PCPs
citing a lack of training as a key barrier to the use of dermoscopy[14-16]. Furthermore, short dermoscopy training programs [14]may be insufficient to establish long-term competency indermoscopy, with poor continuing use of dermoscopy and theneed for refresher sessions [17]. The need for dermoscopy
training among plastic surgeons has recently been documented
as well [18]. Thus, the development of machine-based tools for
dermoscopy may enhance clinical practice for dermatology
providers and other medical professionals.
The use of standard terminologies organized through taxonomies
has a long history with the life sciences, starting with Carl
Linnaeus’ taxonomy [19]: a classification system to name andgroup species according to their shared characteristics. Centuries
later and with advances in computing infrastructure, these typesof classification systems have continued to be of interest to thescience community. An ontology is “a representational artifact
comprising a taxonomy as proper part, whose representational
units are intended to designate some combination of universals,
defined classes, and certain relations between them” [20].
Essentially, an ontology is a graphical representation of linked
concepts to formalize a schema (Tbox) for data (Abox). The
formalization leverages semantic links (Rbox) between theconcepts to give data more meaning and to aggregate related
data of any heterogeneous format. This ensures thenormalization of heterogeneous data. Furthermore, withsemantics, ontologies could support machine reasoning to
generate references via deductive reasoning. As related to the
medical field, ontologies can extend the computability ofstandard controlled terminologies to provide descriptive andcomposite representations of medical information (such asfeatures related to various diagnoses). Ontologies represent thedata in a machine-readable format to give computing tools more
context, making them highly valuable for artificial intelligence.
Within the dermatology domain, some existing ontologies aim
to describe cutaneous disorders. For example, the dermatology
lexicon (DERMLEX) was created with the American Academyof Dermatology with a nosology, anatomical distributions,
https://medinform.jmir.org/2024/1/e49613

XSL• FO
RenderX

Lin et alclassical signs, and therapeutic procedures; however,maintenance was discontinued in 2009 [21,22]. More recently,
the human dermatological disease ontology (DERMO) was
developed to classify cutaneous diseases by etiology, anatomical
location or cell type, and phenotype consistent with current
clinical practice [23,24]. Some other dermatology-specificontologies exist, including the skin physiology ontology (SPO;
last updated in 2008) [25], but notably, none of these ontologies
connect cutaneous disorders to metaphoric terms like“strawberry pattern” which may be difficult for a machine tounderstand. Similarly, none of the aforementioned ontologies
specifically address dermoscopy, which is a specialized
technique that may have special considerations when used indiagnosis. For instance, the colors of certain lesions are best
seen under polarized light [26]. As such, there is a need todevelop an ontology that adequately addresses the field of
dermoscopy, with the capability of processing both descriptive
and metaphoric terminology.
In our previous work, we developed the elements of visuals
ontology (EVO) to decompose the fundamental features ofvisualizations, such as shapes, colors, and textures. The
dermoscopy elements of visuals ontology (DEVO) then applied
the visual features described in EVO to dermoscopic
terminology [27]. For instance, DEVO characterizes
dermoscopic metaphoric terms such as “shiny white streaks”
and “leaflike areas” by shapes, colors, and textures, along withother features involved. Discussion with domain experts
revealed that while DEVO is capable of responding to queries
to find visual features associated with metaphoric terms andvice versa, linking the dermoscopic terms to differential
diagnoses would significantly enhance its clinical utility. A listof differential diagnoses indicates many possible diagnoses thatshare similar features to the patient’s symptoms and signs. These
differential diagnoses can then be narrowed down to aid theclinician in identifying the final diagnosis. As dermatology isa technical field, the landscape of differential diagnoses is wideand difficult to parse [28]. In this study, we describe theextension of EVO and DEVO to create the dermoscopy
differential diagnosis explorer (D3X), an ontology linking
metaphoric terms to differential diagnoses. We further propose
a use case integrating D3X into a web application in dermoscopy
education and clinical practice.

Methods
Ethical Considerations
This article adheres to the Committee on Publication Ethics
guidelines. This research did not involve human subjects.

Integration of Existing Ontologies
Overview
A common practice in the development of ontologies [29] is toreuse existing ontologies’ components to ensure semanticinteroperability. We used the following ontologies to build theD3X ontology.

JMIR Med Inform 2024 | vol. 12 | e49613 | p. 2
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

About EVO
EVO is a foundational ontology model that describes the basic
constituents of visualizations: shapes, colors, strokes (lines),size, perceived texture, etc. It also represents the dimensional
extended 9-intersection model, a mathematical model for spatial
relationships between elements [30]. Further, EVO imports andreuses controlled terminologies and standards from the W3C
scalable vector graphics, Wikidata, phenotype and trait ontology,
and the simple knowledge organization system to supplement
our core representational model of visualizations. EVO is hosted
on GitHub for public release and is coded in the OWL 2 web
ontology language.

About DEVO
DEVO is an extension of EVO that reuses the foundational
understanding of visualizations for the dermoscopy domain.
DEVO
incorporates
some
of
the
controlled
terminologies—“metaphoric” and “descriptive”—that are usedin practice by dermatologists, with a focus on the metaphoric
terminologies. With DEVO, we developed a core model that
encodes and describes the “visual language” of the dermoscopic
terms’ definitions. Further, one important outcome of this work
was a computable representational model of an agreed
understanding of visual elements of dermoscopic patterns, which
we used as a framework to generate differential diagnoses.
Similarly, DEVO was coded in OWL 2 and is hosted on GitHub
for public consumption.

Miscellaneous Ontologies and Vocabularies
We also aligned D3X with commonly used top-level ontologies.
The information artifact ontology (IAO) [31] is part of the open
biological and biomedical ontology (OBO) foundry. IAO
represents a general abstraction of informational objects (likedocuments and components within those documents—eg,
figures, images). Like many OBO foundry ontologies, IAO uses
the basic formal ontology and relation ontology as part of its
architecture model. We minimally reused some of the term
entities and properties like IAO:image and “denoted by.” Wealso reused the software ontology (SWO) [32] for its licensing
entity terms—SWO:license and “has license”—to describe thelicensing information for any imaging resource of skin lesions.
Lastly, we used Schema.org’s [33] schema::image to link image
resources.

Development of D3X
To generate a list of differential diagnoses, we started with themetaphoric terms defined in DEVO from the third consensus
conference conducted by the International Society of
Dermoscopy [34]. We then searched the literature [34-36] forcorresponding differential diagnoses for each term and consulted
2 domain experts to independently edit the list of diagnoses for
accuracy. These differential diagnoses were later encoded using
Protégé [37] in our ontology. Following this, we reviewed
open-source resources (DermNet, Dermoscopedia, andopen-access research papers) for a collection of hosted images
that matched individual differential diagnoses. We tracked theprovenance information and associated data (caption,
description, etc) in a spreadsheet as a central organized resourcethat mapped the images for each diagnosis to the concept
https://medinform.jmir.org/2024/1/e49613

XSL• FO
RenderX

Lin et al
diagnosis used in D3X. To streamline the data transfer process,we developed a management code to transfer data from the
spreadsheet to the ontology. The source code is available on
our GitHub repository, using the OWL API to facilitate efficientcustom import. This approach allowed centralized datacollection and also enabled an ad-hoc import and data creation
pipeline.

Semiotic Evaluation
Semiotic theory is the study of signs and symbols, andconsidering ontologies are symbolic representations of a specificdomain, we used a metric suite grounded in that theoreticalframework [38]. Semiotic theory is composed of 3 basic
qualities: syntactic, semantic, and pragmatic. Essentially, in thecontext of ontologies, the metric suite components refer toaspects of the ontology artifact—syntactic concerning encoding
adherence and standards; semantic concerning the effective useof human-friendly labels for entities and concepts; andpragmatic concerning function. Each of these qualities isquantified based on a computation of representative quantifiablefeatures of an ontology file (eg, the number of classes, theaverage number of word senses for labels, etc). This is describedin detail in previously published works [38]. This suite helps
to measure some of the intrinsic qualities of our ontology
concerning other ontologies in the same domain. We usedpublicly available ontologies from the skin and dermatology
domain—DERMLEX, DERMO, and SPO—that are found in
the National Center for Biomedical Ontology (NCBO)
BioPortal. We used a command line version of our tool
OntoKeeper [39] to quickly generate scores from the metricsuite and then calculated z scores to determine how D3X fares
in terms of intrinsic quality with other ontologies of thedermatology domain.

Results
Development of D3X
The D3X ontology was encoded in the OWL 2 web ontology
language. In terms of the size of the ontology, there are 3041
logical axioms, 1519 classes, 103 object properties, and 20 data
properties. Imported image data are encoded as 387 instances.
Figure 1 displays a sample series of screenshots showing Kaposi
sarcoma, as an example entity, linked to DEVO’s rainbow
pattern, standard medical terminologies (eg, Systematized
Nomenclature of Medicine–Clinical Terms [SNOMED CT],
National Cancer Institute [NCI] Thesaurus), and theopen-sourced image example. For ongoing data management,
we host the spreadsheet with image data (n=364 images) andthe OWL API software code to allow for an automated processof adding new image data. The software will pull the data fromthe spreadsheet and will add and export a version of ourontology that has the image instance data. Both the spreadsheet
and the software are available on our GitHub repository [32].
As more dermoscopy images become available for the publicdomain, we will include them in our spreadsheet and generate
an encoded export with the new instance data. D3X uses our
pre-existing work of DEVO and also leverages terminology
from the IAO, SWO, and Schema.org. Figure 2 shows a global

JMIR Med Inform 2024 | vol. 12 | e49613 | p. 3
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
overview of the D3X ontology and the various linked

Lin et alterminologies that compose the entire model.

Figure 1. Sample screenshot of the D3X ontology through Protégé showing related metadata and information about Kaposi sarcoma. D3X: dermoscopy
differential diagnosis explorer; DDX: differential diagnosis.

Figure 2. Global overview of the D3X ontology and linked ontologies, including the DEVO, BFO, IAO, and SWO. BFO: basic formal ontology; D3X:
dermoscopy differential diagnosis explorer; DEVO: dermoscopy elements of visuals ontology; IAO: information artifact ontology; rdfs: resource
description framework schema; RO: relation ontology; SWO: software ontology.

Each dermoscopy sample image is represented as a single
instance data value with a unique ID (DDX_IMG_[DIGITS]).
As an instance data value representing the digital image, it links
to the exact file on the web using schema:image fromhttps://medinform.jmir.org/2024/1/e49613

XSL• FO
RenderX

Schema.org. Caption information is used as an annotation for
RDF:comment (RDF: resource description framework) andrdf:label (rdf: resource description framework). The instancedata value is an instantiation of a specific image class from the
JMIR Med Inform 2024 | vol. 12 | e49613 | p. 4
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Lin et alfollowing sources: IAO—open access journal images, DermNet
images, National Center for Biotechnology Information–hosted
images, and Dermoscopedia images. For some of them, the “haslicenses” predicate links to a license, signifying that any instanceof this image class has some license agreement (eg, Creative
Commons). The licensing terminology is derived from the SWO
and is hosted on our GitHub repository as an external import.
With D3X, we declared a new dermoscopy differential diagnosis
class. This class provides a list of associated diagnoses for skinlesions. Each of the dermoscopy differential diagnosis classes
is linked to a pattern from DEVO. The pattern in DEVO
ontologically describes each dermoscopic pattern (metaphoricterm) using visual elements, such as lines, shapes, colors, andspatial relationships. Table S1 in Multimedia Appendix 1
provides a comprehensive list of the metaphoric patterns listed
in DEVO and their corresponding differential diagnoses in D3X.
Each pattern in DEVO is linked to its differential diagnoses
using OBO’s “is marker for” (eg, angular lines > is a marker
for > Lentigo_maligna). Additionally, the instance images
described above are linked to the differential diagnoses using“denoted by,” such that each diagnosis is provided with at least
one visual example. Lastly, for each of the dermoscopy
differential diagnosis classes, there are associated annotations
that link the class to the other standardized ontologies like theMedical Dictionary for Regulatory Activities (MedDRA),SNOMED CT, NCI Thesaurus, and LOINC (logical observation
identifier names and codes). MedRA covered 63% (n=25) of
the classes, while SNOMED CT and NCI Thesaurus covered

53% (n=21) and 55% (n=22) of the classes, respectively. The
remaining, like DERMO and LOINC, covered 15% (n=6) and
3% (n=1) of the classes.

Semiotic Evaluation
Semiotic theory is composed of 3 basic qualities: syntactic,
semantic, and pragmatic. Table 1 displays the z scores for eachof the qualities and subqualities of D3X compared to otherpublicly available ontologies in the dermatology domain.
Examining the syntactic quality of D3X (z=0.17), while it lacks
diverse syntactic richness (z=–0.74) in comparison with its otherdomain counterparts, D3X does adhere to syntactic lawfulness(z=0.49). D3X compares satisfactorily with other ontologies inthe semantic quality (z=0.77). Although the semantic clarity
subquality was below average than its peers (z=–0.91; theambiguity of labels), D3X does better with semantic consistency(z=0.56; the number of essentially unique labels) and semanticinterpretability (z=0.65; whether the label has meaning). The
pragmatic quality is composed of 1 score: comprehensiveness,a measure of the coverage of the domain scope of the ontology
based on the number of entities encoded, which was nearly
below average for D3X (z=–0.66). Lastly, the overall score ofD3X (z=0.58) points to a somewhat better overall quality scorethan DERMLEX and SPO (z=–1.41 and 0.00, respectively).
Although DERMO had a slightly higher overall quality than
D3X (z=0.83), its score is still within 1 SD of the D3X ontology
score, so the quality of D3X appears at least comparable to thatof the other ontologies within its own domain.

Table 1. Semiotic comparison of D3Xa to other dermatology ontologies: the DERMLEXb, DERMOc, and SPOd using z scores.
Quality and subquality

Mean (SD)D3X-z

DERMLEX-z

DERMO-z

SPO-z

Syntactic

0.57 (0.11)

0.17e

–1.33

0.08

1.09e

Richness

0.26 (0.11)

–0.74

0.62e

–0.96

1.08e

Lawfulness

0.87 (0.25)

0.49

–1.50

0.51e

0.51 eSemantic

0.85 (0.13)

0.77e

–1.38

0.70e

–0.08

Clarity

0.99 (0.01)

–0.91

0.91e

0.82e

–0.82

Consistency

0.73 (0.49)

0.56e

–1.50

0.43

0.51e

Interpretability

0.87 (0.21)

0.65e

0.65e

0.16

–1.46

0.11 (0.15)

–0.66

1.44e

–0.09e–0.69

0.11 (0.15)

–0.66

1.44e

–0.09e–0.69

0.52 (0.03)

0.58 e

–1.41

0.83e

0.00

Pragmatic
ComprehensivenessOverall scorea

D3X: dermoscopy differential diagnosis explorer.

b

DERMLEX: dermatology lexicon.

c

DERMO: human dermatological disease ontology.

d

SPO: skin physiology ontology.

eThese values indicate the 2 highest values for each quality and subquality.

https://medinform.jmir.org/2024/1/e49613

XSL• FO
RenderX

JMIR Med Inform 2024 | vol. 12 | e49613 | p. 5
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Discussion
Principal Results and Limitations
D3X is an ontology that connects dermoscopic patterns(metaphoric terms) with differential diagnoses. It is an extensionof the DEVO to describe patterns based on their visual elements,
which is in turn an extension of the EVO. D3X also leverages
terminology from IAO, SWO, Relation Ontology, andSchema.org, and its differential diagnoses are linked to
MedDRA, SNOMED CT, and NCI Thesaurus. Using thesemiotic theory framework proposed by Burton-Jones et al [38],
we measured D3X in comparison with similar publicly available
ontologies to assess its intrinsic quality. Our assessment
indicates that while comparably better to the other ontologiesof the same domain in its overall score, D3X does lack diverse
syntactic richness and could improve its semantic clarity(despite a better overall semantic quality score than its ilk) andpragmatic comprehensiveness. Leveraging additional OWL 2
syntactic features could improve the syntactic richness.
However, since the purpose of our ontology is to retrieve andaggregate information and metadata about dermoscopic features,some of the more sophisticated OWL 2 features like symmetry,
inverse, etc, may not be necessary for our use case. As for the
pragmatic score, it might improve over time as we collect moreinstance data of images to link to our ontology. Further, ourassessment was limited to 3 ontologies as there are no otherpublicly available ontologies that deal solely with a dermatology
subject. Additionally, OntoKeeper uses a subset of scores as
the social quality (composed of authority and history), and thepragmatic subscores of accuracy and relevancy are difficult tocompute, so they are not listed in our semiotic analysis [39].
Despite this, the quality scores are sufficient for an application
ontology, since the role of this artifact is to aggregate andconsolidate skin diagnostic information—an area where it islikely to shine.
The aforementioned evaluation included DERMLEX, DERMO,
and the SPO. DERMLEX was originally created by the

Lin et al
American Academy of Dermatology to describe dermatologicaldiagnosis and related domain vocabularies, aligned toInternational Classification of Diseases, Ninth Revision (ICD-9).
However, the upkeep ended in 2009 [22]. DERMO is anotherontology that also aims to describe dermatological diseases, but
unlike DERMLEX, it is aligned to International Classificationof Diseases, Tenth Revision (ICD-10). The latest version waslast released in 2015, according to the NCBO BioPortal record[23]. Not much is known about SPO, other than a presence on
NCBO BioPortal and the latest release dating back to 2008 [25].
Compared to these existing works, D3X yields richer semantics
and applicability by the OWL2 encoding in EVO and DEVO
that describes lesions using primitive visualization elements.
Another advantage of this work is the use of semantic web
properties of our work, namely the linking of heterogeneous
resources (external entities, images, metadata, etc). This allows
D3X to be an application-driven artifact that can be integrated
into software tools, and other analytical and educational tools.
According to researchers, terminologies enriched with semantics
will yield opportunities to develop innovative tools andapplications [40]. We further discuss our vision in thesubsequent sections (see Proposed Web Application and Use
Case). Overall, we presume this work provides a richer
ontological artifact compared to similar ontologies of the same
domain.
Aside from our aforementioned application use case, this work
can advance machine learning models for dermoscopy diagnosis
support. There has been some preliminary evidence that machinelearning models can be supported or improved by ontologies[41-43]. Potentially, the combined stack of EVO, DEVO, andD3X could augment tools that analyze real-world entities (eg,lesions). In Figure 3, we illustrate a hypothetical example wherea software application segments signals from an entity using
machine learning in a sensor array to detect shape, color, andpattern data. The structured information from the sensor array
could then be linked to an ontological knowledge base systemthat expresses meaning and context.

Figure 3. Diagram of a software system using segmented machine learning with a sensor array linked to an ontological knowledge base system. Pixel
art icon by DesignContest is licensed under CC BY 4.0. Earth icon by Treetog ArtWork. Globe icon and user female icon by paomedia.

https://medinform.jmir.org/2024/1/e49613

XSL• FO
RenderX

JMIR Med Inform 2024 | vol. 12 | e49613 | p. 6
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Data Upkeep and Management Plan
Noted earlier, we produced a basic management system to allow
for continued integration of data and information from externalsources to be added to D3X. Continued data management is an
issue with some ontology and controlled terminology resources.
By having this basic management system, we can ensure that
D3X will be up to date with little resources and time needed tointegrate new diagnostic information and metadata. Figure 4
shows the basic management pipeline, with the tools needed,hostedon
our
GitHubrepository
under
the

Lin et al
ddx_data_management folder. In the aforementioned figure,any new or updated digital resources (images, web page text,
knowledge graph, and ontology resources) will be added to acentralized spreadsheet for the human-friendly organization ofdata for diagnosis information. The management software will
import the spreadsheet and parse the data for the D3X ontology.
The final output of the software is the D3X ontology with theupdated linked information. Future plans could include using
shapes constraint language (SHACL) to ensure the quality ofthe data is validated, and further development of data
management software to facilitate ease of use.

Figure 4. Outline of the D3X ontology data upkeep and management plan. D3X: dermoscopy differential diagnosis explorer. OWL Lite icon and OWL
Lite document icon by Picol Team are licensed under CC BY 4.0. File excel icon, database icon, window system icon, user female alt icon by paomedia.

Proposed Web Application and Use CaseFor use in clinical practice, an ontology-based software
application could be designed using D3X to guide theidentification of differential diagnoses. After the user performs
dermoscopy on a skin lesion, they can open the web applicationand select various features about the lesion, which will thensuggest differential diagnoses. A mock-up for the use case of
Kaposi sarcoma is shown in Figure 5. The user can selectoptions for the first 3 boxes (“dermoscopy,” “feature,” and thechosen feature [eg, “color”], with multi-select functionality
available for the latter). The web application would thengenerate the relevant patterns and a list of differential diagnoses.
Thus, if the user indicates that under polarized light, red, pink,
blue, violet, and white colors were visualized, this corresponds
to a “rainbow pattern,” which is associated with Kaposi sarcoma,among other differential diagnoses. Clicking on each differential
diagnosis will then display any relevant dermoscopic images
in the “viewer,” as well as a description under “details” with alink to learn more about the condition. By reviewing images
and descriptions of these differential diagnoses, this would
ideally help the user narrow down the list and identify the most
likely diagnosis. This web application was independently
reviewed by 2 domain experts who agreed that the format washttps://medinform.jmir.org/2024/1/e49613

XSL• FO
RenderX

understandable to the user; they also stated that the information
provided would be useful in clinical practice as a quick searchfor differential diagnoses. Of note, the aforementioned example
illustrates a query based on features or patterns, but the web
application would also be capable of querying based on
diagnoses (eg, “what dermoscopy patterns are associated with
Kaposi sarcoma?”). This would be more useful in an educationalsetting for those who want to gain an understanding of
dermoscopic patterns and the features comprising each pattern.
Furthermore, we proposed the development of a web application
harnessing D3X capable of carrying out the following queries:(1) given dermoscopic features or patterns, output a list ofdifferential diagnoses; and (2) given a differential diagnosis,
output associated features and patterns. Along with a descriptionof each differential diagnosis, the application would also display
images from DermNet, Dermoscopedia, National Center for
Biotechnology Information Hosted, or open-access journals forease of understanding the relationship between each differential
diagnosis and its visual elements. There is a growing body ofliterature on machine learning models for automated diagnosis
of dermoscopic images, such as convolutional neural networks(CNNs) [9]. Both ontologies and CNNs fall under the artificial
intelligence umbrella, but ontologies relate to knowledge
representation, while CNN is statistical machine learning. These
JMIR Med Inform 2024 | vol. 12 | e49613 | p. 7
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
are fundamentally different approaches to power artificial
intelligence that are difficult to compare directly. Whileautomated diagnosis via CNNs is a very promising area of study,
research has largely focused on the diagnosis of melanoma[44-46], with few studies including pigmented nonmelanocytic
lesions [47,48] and largely ignoring nonpigmented lesions. D3X
labels dermoscopic patterns of pigmented and nonpigmented
lesions, so it may apply to a broader range of patient visits.

https://medinform.jmir.org/2024/1/e49613

XSL• FO
RenderX

Lin et al
Additionally, the likelihood of provider acceptance of automateddiagnosis systems is unclear. With our proposed web
application, providers would be able to input search criteria
themselves and see a list of differential diagnoses, rather thana binary output for 1 diagnosis (eg, melanoma) suggested by
the machine, which may not be as likely to be accepted byphysicians.

JMIR Med Inform 2024 | vol. 12 | e49613 | p. 8
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Lin et al

Figure 5. Mock-up of a web application harnessing the D3X ontology to perform queries for differential diagnoses associated with dermoscopic featuresand patterns, with Kaposi sarcoma as a use case. Doctor Icon by MedicalWP is licensed under CC BY 4.0. Computer icon and text x java icon by Papirus
Dev Team are licensed under GNU GPL (version 3.0). Database icon, file text icon, file picture icon, device mobile phone icon by paomedia. Abox:
assertion component of a knowledge base; D3X: dermoscopy differential diagnosis explorer; DEVO: dermoscopy elements of visuals ontology; DL:
description logic; EVO: elements of visuals ontology; OBO: open biological and biomedical ontology; SPARQL: SPARQL protocol and RDF querylanguage (recursive acronym); Tbox: terminology component of a knowledge base.

In discussion with domain experts, we chose not to integratediagnostic rules into D3X, as experienced dermoscopy users
could assess the list of differential diagnoses fairly quickly anddecide on the most likely diagnosis using their clinical expertise.
If the user is relatively inexperienced, they may gain morehttps://medinform.jmir.org/2024/1/e49613

XSL• FO
RenderX

understanding by reading the description of each differential
diagnosis, viewing the images, and accessing additional
information by clicking “learn more” in the web application.
Nevertheless, in the future, it may be useful to integratediagnostic rules into D3X for more sophisticated suggestions
JMIR Med Inform 2024 | vol. 12 | e49613 | p. 9
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
of differential diagnoses (eg, ranking the most to least likely
differential diagnoses in a prioritized list). Another limitationis that there are dermoscopic terms and differential diagnoses
not included in D3X, given that we built D3X from the terms
mentioned in the International Society of Dermoscopy’s third
consensus conference [34]. Similarly, while we aimed to include
images from a variety of external sources, we acknowledge thatthey may not be fully representative of all patient skin tones.
Our work is only a starting point, as we plan to continue
updating D3X and anticipate that its library will become morecomprehensive with time.
To conclude, the web application based on D3X has great
potential for use in several areas. First, it could be included
alongside formal dermoscopy training as a supplementary
educational tool for dermatology trainees and providers in otherspecialties (PCPs, plastic surgeons). Given that providers may
require ongoing dermoscopy refresher sessions to feel fully
comfortable even after completing an initial training program[17], this web application could be a helpful reference to deepen
understanding of dermoscopic patterns associated with different
skin conditions. Furthermore, in a clinical setting, providers
could quickly query the web application for a list of differential
diagnoses after dermoscopic examination of a lesion, which
would aid in the identification of their patient’s diagnosis. Thismay be useful for inexperienced and experienced dermoscopy
users alike, as it is intended to augment, not replace, theprovider’s clinical reasoning. The next steps include using our

Lin et al
proposed interface to build a functional web application.
Creating the web application could reveal additional flaws in
the design that require clarification, and we would continue toimprove aspects of D3X and the web application in an iterative
process. After a beta version of the web application is finalized,
we would aim to conduct user testing to evaluate the user
experience as well as clinical or educational utility among
physicians.

Conclusions
We introduce and discuss the design and development of theD3X ontology as a resource that can link and integratedermoscopic differential diagnoses and supplementary
information with existing ontology-based resources (MedDRA,
SNOMED CT, and NCI). We repurposed a previous work of
the EVO and DEVO to construct and support D3X, along withother supplementary standardized ontologies, like IAO andSWO. Using the semiotic theoretical framework to compare
D3X with other dermatology-related ontologies, its overall
quality score was similar to existing ontologies’ scores. One ofthe outcomes of this work is providing a means to aggregate
and link dermoscopic patterns to differential diagnoses, thereby
enhancing understanding of dermoscopy for educational andclinical use. This outcome has fueled our next objective indeveloping a web-based application that can query D3X andfetch the linked information for the user. Currently, D3X and
its resources are available on GitHub for public release and use.

Acknowledgments
This work was partially supported by the National Institutes of Health (R01AI130460 and U24CA194215) and the Cancer
Prevention Research Institute of Texas (RP220244).

Conflicts of Interest
None declared.

Multimedia Appendix 1
Patterns in the dermoscopy elements of visuals ontology and their associated differential diagnoses in the dermoscopy differential
diagnosis explorer.
[DOCX File , 19 KB-Multimedia Appendix 1]

References
1.
2.

3.
4.

5.

6.

Argenziano G, Soyer HP. Dermoscopy of pigmented skin lesions—a valuable tool for early diagnosis of melanoma. Lancet
Oncol. 2001;2(7):443-449. [doi: 10.1016/s1470-2045(00)00422-8] [Medline: 11905739]
Vázquez-López F, Manjón-Haces JA, Maldonado-Seral C, Raya-Aguado C, Pérez-Oliva N, Marghoob AA. Dermoscopic
features of plaque psoriasis and lichen planus: new observations. Dermatology. 2003;207(2):151-156. [doi:10.1159/000071785] [Medline: 12920364]
Rosendahl C, Tschandl P, Cameron A, Kittler H. Diagnostic accuracy of dermatoscopy for melanocytic and nonmelanocytic
pigmented lesions. J Am Acad Dermatol. 2011;64(6):1068-1073. [doi: 10.1016/j.jaad.2010.03.039] [Medline: 21440329]
Lallas A, Argenziano G, Apalla Z, Gourhant JY, Zaballos P, Di Lernia V, et al. Dermoscopic patterns of common facial
inflammatory skin diseases. J Eur Acad Dermatol Venereol. 2014;28(5):609-614. [doi: 10.1111/jdv.12146] [Medline:
23489377]
Bafounta ML, Beauchet A, Aegerter P, Saiag P. Is dermoscopy (epiluminescence microscopy) useful for the diagnosis ofmelanoma? Results of a meta-analysis using techniques adapted to the evaluation of diagnostic tests. Arch Dermatol.
2001;137(10):1343-1350. [doi: 10.1001/archderm.137.10.1343] [Medline: 11594860]
Kittler H, Pehamberger H, Wolff K, Binder M. Diagnostic accuracy of dermoscopy. Lancet Oncol. 2002;3(3):159-165.
[doi: 10.1016/s1470-2045(02)00679-4] [Medline: 11902502]

https://medinform.jmir.org/2024/1/e49613

XSL• FO
RenderX

JMIR Med Inform 2024 | vol. 12 | e49613 | p. 10(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
7.

8.

9.
10.

11.

12.

13.
14.

15.

16.

17.

18.

19.
20.

21.

22.
23.

24.
25.
26.

27.

28.

Vestergaard ME, Macaskill P, Holt PE, Menzies SW. Dermoscopy compared with naked eye examination for the diagnosis
of primary melanoma: a meta-analysis of studies performed in a clinical setting. Br J Dermatol. 2008;159(3):669-676. [doi:10.1111/j.1365-2133.2008.08713.x] [Medline: 18616769]
Sinz C, Tschandl P, Rosendahl C, Akay BN, Argenziano G, Blum A, et al. Accuracy of dermatoscopy for the diagnosis of
nonpigmented cancers of the skin. J Am Acad Dermatol. 2017;77(6):1100-1109. [doi: 10.1016/j.jaad.2017.07.022] [Medline:
28941871]
Weber P, Tschandl P, Sinz C, Kittler H. Dermatoscopy of neoplastic skin lesions: recent advances, updates, and revisions.
Curr Treat Options Oncol. 2018;19(11):56. [FREE Full text] [doi: 10.1007/s11864-018-0573-6] [Medline: 30238167]
Haliasos EC, Kerner M, Jaimes-Lopez N, Rudnicka L, Zalaudek I, Malvehy J, et al. Dermoscopy for the pediatricdermatologist part I: dermoscopy of pediatric infectious and inflammatory skin lesions and hair disorders. Pediatr Dermatol.
2013;30(2):163-171. [doi: 10.1111/pde.12097] [Medline: 23405886]
Westerhoff K, McCarthy WH, Menzies SW. Increase in the sensitivity for melanoma diagnosis by primary care physicians
using skin surface microscopy. Br J Dermatol. 2000;143(5):1016-1020. [doi: 10.1046/j.1365-2133.2000.03836.x] [Medline:
11069512]
Argenziano G, Puig S, Zalaudek I, Sera F, Corona R, Alsina M, et al. Dermoscopy improves accuracy of primary care
physicians to triage lesions suggestive of skin cancer. J Clin Oncol. 2006;24(12):1877-1882. [doi: 10.1200/JCO.2005.05.0864][Medline: 16622262]
Herschorn A. Dermoscopy for melanoma detection in family practice. Can Fam Physician. 2012;58(7):740-745. [FREEFull text] [Medline: 22859635]
Chappuis P, Duru G, Marchal O, Girier P, Dalle S, Thomas L. Dermoscopy, a useful tool for general practitioners inmelanoma screening: a nationwide survey. Br J Dermatol. 2016;175(4):744-750. [doi: 10.1111/bjd.14495] [Medline:
26914613]
Morris JB, Alfonso SV, Hernandez N, Fernández MI. Examining the factors associated with past and present dermoscopy
use among family physicians. Dermatol Pract Concept. 2017;7(4):63-70. [FREE Full text] [doi: 10.5826/dpc.0704a13][Medline: 29214111]
Fee JA, McGrady FP, Rosendahl C, Hart ND. Training primary care physicians in dermoscopy for skin cancer detection:a scoping review. J Cancer Educ. 2020;35(4):643-650. [FREE Full text] [doi: 10.1007/s13187-019-01647-7] [Medline:
31792723]
Robinson JK, MacLean M, Reavy R, Turrisi R, Mallett K, Martin GJ. Dermoscopy of concerning pigmented lesions andprimary care providers' referrals at intervals after randomized trial of mastery learning. J Gen Intern Med. 2018;33(6):799-800.
[FREE Full text] [doi: 10.1007/s11606-018-4419-5] [Medline: 29637481]
Brennan MC, Kabuli MN, Dargan MD, Pinder MR. A short correspondence piece to the editor in chief: the need forincreased training in the technique of dermoscopy amongst plastic surgeons and the under recognised value of dermoscopy
in the assessment of non-pigmented cutaneous lesions. J Plast Reconstr Aesthet Surg. 2022;75(1):496-498. [doi:10.1016/j.bjps.2021.11.014] [Medline: 34852970]
Paterlini M. There shall be order. The legacy of Linnaeus in the age of molecular biology. EMBO Rep. 2007;8(9):814-816.
[FREE Full text] [doi: 10.1038/sj.embor.7401061] [Medline: 17767191]
Smith B, Kusnierczyk W, Schober D, Ceusters W. Towards a reference terminology for ontology research and developmentin the biomedical domain. 2006. Presented at: Proceedings of the Second International Workshop on Formal Biomedical
Knowledge Representation (KR-MED 2006) ;222?65; November 8, 2006:57; Baltimore, MA.
Papier A, Chalmers RJG, Byrnes JA, Goldsmith LA, Dermatology Lexicon Project. Framework for improved communication:
the Dermatology Lexicon Project. J Am Acad Dermatol. 2004;50(4):630-634. [doi: 10.1016/s0190-9622(03)01571-8][Medline: 15034516]
Dermatology lexicon. NCBO BioPortal. 2009. URL: https://bioportal.bioontology.org/ontologies/DERMLEX/?p=summary[accessed 2024-01-05]
Fisher HM, Hoehndorf R, Bazelato BS, Dadras SS, King LE, Gkoutos GV, et al. DermO; an ontology for the descriptionof dermatologic disease. J Biomed Semantics. 2016;7:38. [FREE Full text] [doi: 10.1186/s13326-016-0085-x] [Medline:
27296450]
Human dermatological disease ontology. NCBO BioPortal. URL: https://bioportal.bioontology.org/ontologies/DERMO[accessed 2024-01-05]
Skin physiology ontology. NCBO BioPortal. URL: https://bioportal.bioontology.org/ontologies/SPO [accessed 2024-01-05]
Benvenuto-Andrade C, Dusza SW, Agero ALC, Scope A, Rajadhyaksha M, Halpern AC, et al. Differences between polarized
light dermoscopy and immersion contact dermoscopy for the evaluation of skin lesions. Arch Dermatol. 2007;143(3):329-338.
[doi: 10.1001/archderm.143.3.329] [Medline: 17372097]
Lin R, Amith M, Zhang X, Wang C, Light J, Strickley J, et al. Developing ontologies to standardize descriptions of visualand dermoscopic elements. 2021. Presented at: 2021 IEEE International Conference on Bioinformatics and Biomedicine(BIBM); December 09-12, 2021:1813; Houston, TX. [doi: 10.1109/bibm52615.2021.9669477]
Ashton R, Leppard B. Differential Diagnosis in Dermatology. In: CRC Press. London. CRC Press; 2014:978-1001.

https://medinform.jmir.org/2024/1/e49613

XSL• FO
RenderX

Lin et al

JMIR Med Inform 2024 | vol. 12 | e49613 | p. 11(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
29.
30.
31.
32.

33.
34.

35.

36.
37.
38.
39.

40.

41.

42.

43.

44.

45.

46.

47.

48.

Lin et al

Noy NF, McGuinness DL. Ontology development 101: A guide to creating your first ontology. Corais. 2021. URL: https://corais.org/sites/default/files/ontology_development_101_aguide_to_creating_your_first_ontology.pdf [accessed 2024-05-23]
Egenhofer M, Herring J. A mathematical framework for the definition of topological relations. 1990. Presented at: Proceedingsof the Fourth International Symposium on Spatial Data Handing; September 1990:803; California.
Ceusters W. An information artifact ontology perspective on data collections and associated representational artifacts. In:
Studies in Health Technology and Informatics. Amsterdam, the Netherlands. IOS Press; 2012:68-72.
Malone J, Brown A, Lister AL, Ison J, Hull D, Parkinson H, et al. The Software Ontology (SWO): a resource forreproducibility in biomedical data analysis, curation and digital preservation. J Biomed Semantics. 2014;5:25. [FREE Fulltext] [doi: 10.1186/2041-1480-5-25] [Medline: 25068035]
Guha RV, Brickley D, Macbeth S. Schema.org: evolution of structured data on the web. Commun ACM. 2016;59(2):44-51.
[doi: 10.1145/2844544]
Kittler H, Marghoob AA, Argenziano G, Carrera C, Curiel-Lewandrowski C, Hofmann-Wellenhof R, et al. Standardizationof terminology in dermoscopy/dermatoscopy: results of the third consensus conference of the International Society of
Dermoscopy. J Am Acad Dermatol. 2016;74(6):1093-1106. [FREE Full text] [doi: 10.1016/j.jaad.2015.12.038] [Medline:
26896294]
Yélamos O, Braun RP, Liopyris K, Wolner ZJ, Kerl K, Gerami P, et al. Dermoscopy and dermatopathology correlates of
cutaneous neoplasms. J Am Acad Dermatol. 2019;80(2):341-363. [FREE Full text] [doi: 10.1016/j.jaad.2018.07.073][Medline: 30321581]
Draghici C, Vajaitu C, Solomon I, Voiculescu VM, Popa MI, Lupu M. The dermoscopic rainbow pattern—a review of theliterature. Acta Dermatovenerol Croat. 2019;27(2):111-115. [Medline: 31351506]
Musen MA, Protégé Team. The Protégé project: a look back and a look forward. AI Matters. 2015;1(4):4-12. [FREE Fulltext] [doi: 10.1145/2757001.2757003] [Medline: 27239556]
Burton-Jones A, Storey VC, Sugumaran V, Ahluwalia P. A semiotic metrics suite for assessing the quality of ontologies.
Data Knowl Eng. 2005;55(1):84-102. [doi: 10.1016/j.datak.2004.11.010]
Amith M, Manion F, Liang C, Harris M, Wang D, He Y, et al. Architecture and usability of OntoKeeper, an ontology
evaluation tool. BMC Med Inform Decis Mak. 2019;19(Suppl 4):152. [FREE Full text] [doi: 10.1186/s12911-019-0859-z][Medline: 31391056]
Obrst L. Ontologies for semantically interoperable systems. 2003. Presented at: Proceedings of the Twelfth International
Conference on Information and Knowledge Management; November 3, 2003:366-369; New York, NY. [doi:10.1145/956863.956932]
Mullin S, Zola J, Lee R, Hu J, MacKenzie B, Brickman A, et al. Longitudinal K-means approaches to clustering andanalyzing EHR opioid use trajectories for clinical subtypes. J Biomed Inform. 2021;122:103889. [doi:10.1016/j.jbi.2021.103889] [Medline: 34411708]
Sahoo SS, Kobow K, Zhang J, Buchhalter J, Dayyani M, Upadhyaya DP, et al. Ontology-based feature engineering inmachine learning workflows for heterogeneous epilepsy patient records. Sci Rep. 2022;12(1):19430. [FREE Full text] [doi:10.1038/s41598-022-23101-3] [Medline: 36371527]
Zemmouchi-Ghomari L. Ontology and machine learning: a two-way street to improved knowledge representation andalgorithm accuracy. Springer Nature; 2023. Presented at: Proceedings of International Conference on Paradigms ofCommunication, Computing and Data Analytics; October 11, 2023:181-189; Singapore. [doi: 10.1007/978-981-99-4626-6_15]
Rajpara SM, Botello AP, Townend J, Ormerod AD. Systematic review of dermoscopy and digital dermoscopy/ artificial
intelligence for the diagnosis of melanoma. Br J Dermatol. 2009;161(3):591-604. [doi: 10.1111/j.1365-2133.2009.09093.x][Medline: 19302072]
Marchetti MA, Codella NCF, Dusza SW, Gutman DA, Helba B, Kalloo A, et al. International Skin Imaging Collaboration.
Results of the 2016 International Skin Imaging Collaboration international symposium on biomedical imaging challenge:
comparison of the accuracy of computer algorithms to dermatologists for the diagnosis of melanoma from dermoscopic
images. J Am Acad Dermatol. 2018;78(2):270-277. [doi: 10.1016/j.jaad.2017.08.016] [Medline: 28969863]
Pham TC, Luong CM, Hoang VD, Doucet A. AI outperformed every dermatologist in dermoscopic melanoma diagnosis,
using an optimized deep-CNN architecture with custom mini-batch logic and loss function. Sci Rep. 2021;11(1):17485.
[FREE Full text] [doi: 10.1038/s41598-021-96707-8] [Medline: 34471174]
Tschandl P, Kittler H, Argenziano G. A pretrained neural network shows similar diagnostic accuracy to medical students
in categorizing dermatoscopic images after comparable training conditions. Br J Dermatol. 2017;177(3):867-869. [doi:10.1111/bjd.15695] [Medline: 28569993]
Shetty B, Fernandes R, Rodrigues AP, Chengoden R, Bhattacharya S, Lakshmanna K. Skin lesion classification of
dermoscopic images using machine learning and convolutional neural network. Sci Rep. 2022;12(1):18134. [doi:10.1038/s41598-022-22644-9] [Medline: 36307467]

Abbreviations
CNN: convolutional neural networkhttps://medinform.jmir.org/2024/1/e49613

XSL• FO
RenderX

JMIR Med Inform 2024 | vol. 12 | e49613 | p. 12(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Lin et al

D3X: dermoscopy differential diagnosis explorer
DERMLEX: dermatology lexicon
DERMO: human dermatological disease ontology
DEVO: dermoscopy elements of visuals ontology
EVO: elements of visuals ontology
IAO: information artifact ontology
ICD-9: International Classification of Diseases, Ninth Revision
ICD-10: International Classification of Diseases, Tenth Revision
LOINC: logical observation identifier names and codes
MedDRA: Medical Dictionary for Regulatory Activities
NCBO: National Center for Biomedical Ontology
NCI: National Cancer Institute
OBO: Open Biological and Biomedical Ontology
OWL: web ontology language
PCP: primary care physician
rdf: resource description framework
RDF: resource description framework
SHACL: shapes constraint language
SNOMED CT: Systematized Nomenclature of Medicine—Clinical Terms
SPO: skin physiology ontology
SWO: software ontology

Edited by C Lovis, G Eysenbach; submitted 12.07.23; peer-reviewed by C Gaudet-Blavignac; comments to author 09.12.23; revised
version received 18.04.24; accepted 04.05.24; published 21.06.24
Please cite as:
Lin RZ, Amith MT, Wang CX, Strickley J, Tao C
Dermoscopy Differential Diagnosis Explorer (D3X) Ontology to Aggregate and Link Dermoscopic Patterns to Differential Diagnoses:
Development and Usability StudyJMIR Med Inform 2024;12:e49613
URL: https://medinform.jmir.org/2024/1/e49613
doi: 10.2196/49613
PMID: 38904996©Rebecca Z Lin, Muhammad Tuan Amith, Cynthia X Wang, John Strickley, Cui Tao. Originally published in JMIR Medical
Informatics (https://medinform.jmir.org), 21.06.2024. This is an open-access article distributed under the terms of the Creative
Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, andreproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The
complete bibliographic information, a link to the original publication on https://medinform.jmir.org/, as well as this copyrightand license information must be included.

https://medinform.jmir.org/2024/1/e49613

XSL• FO
RenderX

JMIR Med Inform 2024 | vol. 12 | e49613 | p. 13(page number not for citation purposes)