Knowledge-based semantic enrichment of medical
imaging data for automatic phenotyping and pattern
discovery in metastatic lung cancer

Francesco Cremonesi, Lucie Chambon, Nelson Mokkadem, Huyen Thi Trang

Nguyen, Oliver Humbert, Marco Lorenzi

► To cite this version:

Francesco Cremonesi, Lucie Chambon, Nelson Mokkadem, Huyen Thi Trang Nguyen, Oliver Humbert, et al.. Knowledge-based semantic enrichment of medical imaging data for automatic phenotyping and pattern discovery in metastatic lung cancer. 2025 SNMMI Annual Meeting - Society for Nuclear Medicine and Molecular Imaging, Jun 2025, New orleans, LA, United States. hal-04879690

HAL Id: hal-04879690

https://hal.science/hal-04879690v1

Submitted on 7 Feb 2025

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Knowledge-based semantic enrichment of
medical imaging data for automatic
phenotyping and pattern discovery in
metastatic lung cancer

Francesco Cremonesi (1), Lucie Chambon (1), Nelson Mokkadem (2), Huyen Thi Trang Nguyen (1), Oliver Humbert (2), Marco Lorenzi (3, 4, 1)

1 EPIONE - E-Patient : Images, données & mOdèles pour la médecine numérique

2 UNICANCER/CAL - Centre de Lutte contre le Cancer Antoine Lacassagne [Nice]

3 Inria - Institut National de Recherche en Informatique et en Automatique

4 ASCLEPIOS - Analysis and Simulation of Biomedical Images

Abstract

Introduction

The emergence of AI models for the analysis of 18FDG PET/CT images has opened the way for automatic quantification of clinically-relevant parameters in metastatic cancer patients such as e.g. the number, metabolic volume, and dispersion of lesions. However, current approaches lack specificity as they do not provide information about the localisation of lesions, which is a major clinical parameter for patient management. These methods also lack generality, as they fail to integrate the medical community's body of anatomical knowledge, which can nowadays be found in machine-readable ontology formats. Such integration may improve analysis depth as well as reusability for secondary purposes of the data, by enabling reasoning at different anatomical scales and linking with other sources of information, such as clinical records. To fill this gap, we present in this work a novel framework for the enrichment of 18FDG PET-based lesion annotations through the automatic segmentation of anatomical structures and inclusion of ontological knowledge.

Methods

Starting from aligned whole body 18FDG PET/CT images from a cohort of metastatic non-small
cell lung cancer (NSCLC) patients undergoing immunotherapy treatment (n=101), we obtain
automatic organ segmentations for over 100 anatomical structures using the publicly available
TotalSegmentator (TS) tool applied on CT scans, and combine them with manual lesion
annotation (including both segmentation and gold-standard anatomic labels of lesions) by
nuclear physicians on the PET images. We map the TotalSegmentator structures to the
Foundational Model of Anatomy (FMA) ontology, which in turn allows us to associate each
structure with the gold standard labels by querying the ontology for relationships such as
subClassOf, part_of, and others.
We then compute the intersection between the lesion volumes contoured by human experts on
PET and the anatomical structure segmentations on CT, and combine intersections belonging to
the same class as defined by the gold standard label. We localise lesions by finding the class
with the maximal intersection, allowing us to directly compare to the expert annotations.

We insert the lesion location information into a Knowledge Graph (KG) integrating the FMA ontology, restricting our analysis to locations present in the set of structures segmented by TotalSegmentator. From our KG we extract the co-occurrence of lesions for each patient at two semantic scales: the raw TotalSegmentator structures and the gold standard anatomic labels. In both cases, we perform KMeans clustering to identify patterns of lesion co-occurrence.

## Results

We found a good overlap between gold-standard anatomic localisations of lesions and automatically segmented anatomical structures by TS: compared to gold-standard labels, our method successfully localized 1568 out of 1926 lesions (79.9 %).

The co-occurrence of lesions across selected anatomical structures is shown in Figure 1. Unsupervised clustering of the co-occurrence patterns shows distinct groups of lesion distributions associated with different survival likelihood. The most pathological clusters mapped to liver and bones (OS 24M rate resp. 0.12 and 0.29), in accordance with published literature (PMID: 25130083). On the contrary, clusters mapping to lesions in muscles and adrenal glands were associated with better prognosis (OS 24M rate resp. 1.00 and 0.73). When applying the clustering on the segmented structures directly, without ontologies, no clinically-relevant subgroups were found (OS 24M rates from 0.33 to 0.46).

## Conclusion

This study introduces the methodology of augmenting annotations of tumor lesions on 18FDG PET images with automatic organ segmentation from CT and anatomical knowledge from ontologies. Our approach improved disease phenotyping by automatically characterizing the distribution of lesions across clinically relevant anatomical groupings and identifying patterns of lesion dispersion correlating with survival parameters.
# Methods

Starting from aligned whole body 18FDG PET/CT images from a cohort of metastatic non-small cell lung cancer (NSCLC) patients undergoing immunotherapy treatment (n=101), we obtain automatic organ segmentations for over 100 anatomical structures using the publicly available TotalSegmentator (TS) tool applied on CT scans, and combine them with manual lesion annotation (including both segmentation and gold-standard anatomic labels of lesions) by nuclear physicians on the PET images (see Table 1).

<table><caption>Table 1 Gold-standard labels for anatomical locations of lesions as identified by nuclear physicians.</caption><thead><tr><td>Gold-standard anatomical location label</td><td>Description</td><td>FMA label</td></tr></thead><tbody><tr><td>Mediastinum</td><td></td><td></td></tr><tr><td>Cervical</td><td></td><td></td></tr><tr><td>Lung</td><td></td><td></td></tr><tr><td>Bone</td><td></td><td></td></tr><tr><td>Adrenal gland</td><td></td><td></td></tr><tr><td>SD</td><td>Sub-Diaphragmatic</td><td></td></tr><tr><td>Liver</td><td></td><td></td></tr><tr><td>Brain</td><td></td><td></td></tr><tr><td>Pancreas</td><td></td><td></td></tr><tr><td>CP</td><td>Peritoneal carcinomatosis</td><td></td></tr><tr><td>Thyroid</td><td></td><td></td></tr><tr><td>Muscle</td><td></td><td></td></tr><tr><td>Axilla</td><td></td><td></td></tr><tr><td>Spleen</td><td></td><td></td></tr><tr><td>C1</td><td>Obsolete - Bone</td><td></td></tr><tr><td>C2</td><td>Obsolete - Mediastinum</td><td></td></tr></tbody></table>

## Mapping automatic organ segmentation structures to gold-standard labels for lesion localization

This section describes the process of mapping the Totalsegmentator structure labels to the gold-standard lesion localization labels. The approach uses the Foundational Model of Anatomy (FMA) ontology as a bridge between the two concept sets.

### Baseline mapping

We mapped the Totalsegmentator structures to the FMA ontology using a semi-manual approach: a first heuristic based on matching the Totalsegmentator name to the FMA label allowed to identify roughly 70% of mappings, while the rest was mapped manually. A manual revision based on human oversight was also required, as several subtleties needed to be fixed in order to improve the quality of the mapping, as shown in Table 2. The final results are shown in Table 3.

<table><caption>Table 2 Challenges encountered while mapping Totalsegmentator structures to FMA concepts.</caption><thead><tr><td>Structure</td><td>Mapped FMA label</td><td>Comments</td></tr></thead></table>
<table><caption>Table 3 Mapping of Totalsegmentator structures to FMA ontology</caption><tr><td>Autochthon</td><td>Set of erector spinae</td><td>The mapped structure is not a subClassOf "Muscle organ", because it is a set of muscles.</td></tr><tr><td>Sacrum</td><td>Sacral Vertebra</td><td>The alternative mapping (i.e. "Sacrum") was discarded because it is not a SubClassOf "Bone Organ".</td></tr><tr><td>Hip</td><td>Hip Bone</td><td>The alternative mapping (i.e. "Hip") was discarded because Totalsegmentator only segments the hip bone, not the whole hip region.</td></tr></table>

<table><thead><tr><td>Structure</td><td>fma_id</td><td>fma_label</td></tr></thead><tbody><tr><td>adrenal_gland_left</td><td>http://purl.org/sig/ont/fma/fma15630</td><td>Left adrenal gland</td></tr><tr><td>adrenal_gland_right</td><td>http://purl.org/sig/ont/fma/fma15629</td><td>Right adrenal gland</td></tr><tr><td>aorta</td><td>http://purl.org/sig/ont/fma/fma3734</td><td>Aorta</td></tr><tr><td>autochthon_left</td><td>http://purl.org/sig/ont/fma/fma22732</td><td>Set of left erector spinae</td></tr><tr><td>autochthon_right</td><td>http://purl.org/sig/ont/fma/fma22731</td><td>Set of right erector spinae</td></tr><tr><td>brain</td><td>http://purl.org/sig/ont/fma/fma50801</td><td>Brain</td></tr><tr><td>clavicula_left</td><td>http://purl.org/sig/ont/fma/fma13323</td><td>Left clavicle</td></tr><tr><td>clavicula_right</td><td>http://purl.org/sig/ont/fma/fma13322</td><td>Right clavicle</td></tr><tr><td>colon</td><td>http://purl.org/sig/ont/fma/fma14543</td><td>Colon</td></tr><tr><td>duodenum</td><td>http://purl.org/sig/ont/fma/fma7206</td><td>Duodenum</td></tr><tr><td>esophagus</td><td>http://purl.org/sig/ont/fma/fma7131</td><td>Esophagus</td></tr><tr><td>face</td><td>http://purl.org/sig/ont/fma/fma24728</td><td>Face</td></tr><tr><td>femur_left</td><td>http://purl.org/sig/ont/fma/fma24475</td><td>Left femur</td></tr><tr><td>femur_right</td><td>http://purl.org/sig/ont/fma/fma24474</td><td>Right femur</td></tr><tr><td>gallbladder</td><td>http://purl.org/sig/ont/fma/fma7202</td><td>Gallbladder</td></tr><tr><td>gluteus_maximus_left</td><td>http://purl.org/sig/ont/fma/fma22329</td><td>Left gluteus maximus</td></tr><tr><td>gluteus_maximus_right</td><td>http://purl.org/sig/ont/fma/fma22328</td><td>Right gluteus maximus</td></tr><tr><td>gluteus_medius_left</td><td>http://purl.org/sig/ont/fma/fma22331</td><td>Left gluteus medius</td></tr><tr><td>gluteus_medius_right</td><td>http://purl.org/sig/ont/fma/fma22330</td><td>Right gluteus medius</td></tr><tr><td>gluteus_minimus_left</td><td>http://purl.org/sig/ont/fma/fma22333</td><td>Left gluteus minimus</td></tr><tr><td>gluteus_minimus_right</td><td>http://purl.org/sig/ont/fma/fma22332</td><td>Right gluteus minimus</td></tr><tr><td>heart_atrium_left</td><td>http://purl.org/sig/ont/fma/fma7097</td><td>Left atrium</td></tr><tr><td>heart_atrium_right</td><td>http://purl.org/sig/ont/fma/fma7096</td><td>Right atrium</td></tr><tr><td>heart_myocardium</td><td>http://purl.org/sig/ont/fma/fma9462</td><td>Myocardium</td></tr><tr><td>heart_ventricle_left</td><td>http://purl.org/sig/ont/fma/fma7101</td><td>Left ventricle</td></tr><tr><td>heart_ventricle_right</td><td>http://purl.org/sig/ont/fma/fma7098</td><td>Right ventricle</td></tr><tr><td>hip_left</td><td>http://purl.org/sig/ont/fma/fma16587</td><td>Left hip bone</td></tr><tr><td>hip_right</td><td>http://purl.org/sig/ont/fma/fma16586</td><td>Right hip bone</td></tr><tr><td>humerus_left</td><td>http://purl.org/sig/ont/fma/fma23131</td><td>Left humerus</td></tr><tr><td>humerus_right</td><td>http://purl.org/sig/ont/fma/fma23130</td><td>Right humerus</td></tr><tr><td>iliac_artery_left</td><td>http://purl.org/sig/ont/fma/fma14766</td><td>Left common iliac artery</td></tr><tr><td>iliac_artery_right</td><td>http://purl.org/sig/ont/fma/fma14765</td><td>Right common iliac artery</td></tr><tr><td>iliac_vena_left</td><td>http://purl.org/sig/ont/fma/fma21388</td><td>Left common iliac vein</td></tr></tbody></table>
<table>
  <tr>
    <td>iliac_vena_right</td>
    <td>http://purl.org/sig/ont/fma/fma21387</td>
    <td>Right common iliac vein</td>
  </tr>
  <tr>
    <td>iliopsoas_left</td>
    <td>http://purl.org/sig/ont/fma/fma64920</td>
    <td>Left iliopsoas</td>
  </tr>
  <tr>
    <td>iliopsoas_right</td>
    <td>http://purl.org/sig/ont/fma/fma64919</td>
    <td>Right iliopsoas</td>
  </tr>
  <tr>
    <td>inferior_vena_cava</td>
    <td>http://purl.org/sig/ont/fma/fma10951</td>
    <td>Inferior vena cava</td>
  </tr>
  <tr>
    <td>kidney_left</td>
    <td>http://purl.org/sig/ont/fma/fma7205</td>
    <td>Left kidney</td>
  </tr>
  <tr>
    <td>kidney_right</td>
    <td>http://purl.org/sig/ont/fma/fma7204</td>
    <td>Right kidney</td>
  </tr>
  <tr>
    <td>liver</td>
    <td>http://purl.org/sig/ont/fma/fma7197</td>
    <td>Liver</td>
  </tr>
  <tr>
    <td>lung_lower_lobe_left</td>
    <td>http://purl.org/sig/ont/fma/fma7371</td>
    <td>Lower lobe of left lung</td>
  </tr>
  <tr>
    <td>lung_lower_lobe_right</td>
    <td>http://purl.org/sig/ont/fma/fma7337</td>
    <td>Lower lobe of right lung</td>
  </tr>
  <tr>
    <td>lungcoxmiddle_lobe_right</td>
    <td>http://purl.org/sig/ont/fma/fma7383</td>
    <td>Middle lobe of right lung</td>
  </tr>
  <tr>
    <td>lung_lower_lobe_left</td>
    <td>http://purl.org/sig/ont/fma/fma7370</td>
    <td>Upper lobe of left lung</td>
  </tr>
  <tr>
    <td>lung_lower_lobe_right</td>
    <td>http://purl.org/sig/ont/fma/fma7333</td>
    <td>Upper lobe of right lung</td>
  </tr>
  <tr>
    <td>pancreas</td>
    <td>http://purl.org/sig/ont/fma/fma7198</td>
    <td>Pancreas</td>
  </tr>
  <tr>
    <td>pulmonary_artery</td>
    <td>http://purl.org/sig/ont/fma/fma66326</td>
    <td>Pulmonary artery</td>
  </tr>
  <tr>
    <td>rib_left_1</td>
    <td>http://purl.org/sig/ont/fma/fma7987</td>
    <td>Left first rib</td>
  </tr>
  <tr>
    <td>rib_left_2</td>
    <td>http://purl.org/sig/ont/fma/fma8012</td>
    <td>Left second rib</td>
  </tr>
  <tr>
    <td>rib_left_3</td>
    <td>http://purl.org/sig/ont/fma/fma8039</td>
    <td>Left third rib</td>
  </tr>
  <tr>
    <td>rib_left_4</td>
    <td>http://purl.org/sig/ont/fma/fma8148</td>
    <td>Left fourth rib</td>
  </tr>
  <tr>
    <td>rib_left_5</td>
    <td>http://purl.org/sig/ont/fma/fma8093</td>
    <td>Left fifth rib</td>
  </tr>
  <tr>
    <td>rib_left_6</td>
    <td>http://purl.org/sig/ont/fma/fma8202</td>
    <td>Left sixth rib</td>
  </tr>
  <tr>
    <td>rib_left_7</td>
    <td>http://purl.org/sig/ont/fma/fma8256</td>
    <td>Left seventh rib</td>
  </tr>
  <tr>
    <td>rib_left_8</td>
    <td>http://purl.org/sig/ont/fma/fma8310</td>
    <td>Left eighth rib</td>
  </tr>
  <tr>
    <td>rib_left_9</td>
    <td>http://purl.org/sig/ont/fma/fma8391</td>
    <td>Left ninth rib</td>
  </tr>
  <tr>
    <td>rib_left_10</td>
    <td>http://purl.org/sig/ont/fma/fma8472</td>
    <td>Left tenth rib</td>
  </tr>
  <tr>
    <td>rib_left_11</td>
    <td>http://purl.org/sig/ont/fma/fma8532</td>
    <td>Left eleventh rib</td>
  </tr>
  <tr>
    <td>rib_left_12</td>
    <td>http://purl.org/sig/ont/fma/fma8534</td>
    <td>Left twelfth rib</td>
  </tr>
  <tr>
    <td>rib_right_1</td>
    <td>http://purl.org/sig/ont/fma/fma7857</td>
    <td>Right first rib</td>
  </tr>
  <tr>
    <td>rib_right_2</td>
    <td>http://purl.org/sig/ont/fma/fma7882</td>
    <td>Right second rib</td>
  </tr>
  <tr>
    <td>rib_right_3</td>
    <td>http://purl.org/sig/ont/fma/fma7909</td>
    <td>Right third rib</td>
  </tr>
  <tr>
    <td>rib_right_4</td>
    <td>http://purl.org/sig/ont/fma/fma7957</td>
    <td>Right fourth rib</td>
  </tr>
  <tr>
    <td>rib_right_5</td>
    <td>http://purl.org/sig/ont/fma/fma8066</td>
    <td>Right fifth rib</td>
  </tr>
  <tr>
    <td>rib_right_6</td>
    <td>http://purl.org/sig/ont/fma/fma8175</td>
    <td>Right sixth rib</td>
  </tr>
  <tr>
    <td>rib_right_7</td>
    <td>http://purl.org/sig/ont/fma/fma8229</td>
    <td>Right seventh rib</td>
  </tr>
  <tr>
    <td>rib_right_8</td>
    <td>http://purl.org/sig/ont/fma/fma8283</td>
    <td>Right eighth rib</td>
  </tr>
  <tr>
    <td>rib_right_9</td>
    <td>http://purl.org/sig/ont/fma/fma8364</td>
    <td>Right ninth rib</td>
  </tr>
  <tr>
    <td>rib_right_10</td>
    <td>http://purl.org/sig/ont/fma/fma8445</td>
    <td>Right tenth rib</td>
  </tr>
  <tr>
    <td>rib_right_11</td>
    <td>http://purl.org/sig/ont/fma/fma8531</td>
    <td>Right eleventh rib</td>
  </tr>
  <tr>
    <td>rib_right_12</td>
    <td>http://purl.org/sig/ont/fma/fma8533</td>
    <td>Right twelfth rib</td>
  </tr>
  <tr>
    <td>sacrum</td>
    <td>http://purl.org/sig/ont/fma/fma12526</td>
    <td>Sacral vertebra</td>
  </tr>
  <tr>
    <td>scapula_left</td>
    <td>http://purl.org/sig/ont/fma/fma13396</td>
    <td>Left scapula</td>
  </tr>
  <tr>
    <td>scapula_right</td>
    <td>http://purl.org/sig/ont/fma/fma13395</td>
    <td>Right scapula</td>
  </tr>
  <tr>
    <td>small_bowel</td>
    <td>http://purl.org/sig/ont/fma/fma7200</td>
    <td>Small intestine</td>
  </tr>
  <tr>
    <td>spleen</td>
    <td>http://purl.org/sig/ont/fma/fma7196</td>
    <td>Spleen</td>
  </tr>
</table>
<table>
  <tr>
    <td>stomach</td>
    <td>http://purl.org/sig/ont/fma/fma7148</td>
    <td>Stomach</td>
  </tr>
  <tr>
    <td>trachea</td>
    <td>http://purl.org/sig/ont/fma/fma7394</td>
    <td>Trachea</td>
  </tr>
  <tr>
    <td>urinary-bladder</td>
    <td>http://purl.org/sig/ont/fma/fma15900</td>
    <td>Urinary bladder</td>
  </tr>
  <tr>
    <td>vertebrae-S1</td>
    <td>http://purl.org/sig/ont/fma/fma13077</td>
    <td>First sacral vertebra</td>
  </tr>
  <tr>
    <td>vertebrae-C1</td>
    <td>http://purl.org/sig/ont/fma/fma12519</td>
    <td>Atlas</td>
  </tr>
  <tr>
    <td>vertebrae-C2</td>
    <td>http://purl.org/sig/ont/fma/fma12520</td>
    <td>Axis</td>
  </tr>
  <tr>
    <td>vertebrae-C3</td>
    <td>http://purl.org/sig/ont/fma/fma12521</td>
    <td>Third cervical vertebra</td>
  </tr>
  <tr>
    <td>vertebrae-C4</td>
    <td>http://purl.org/sig/ont/fma/fma12522</td>
    <td>Fourth cervical vertebra</td>
  </tr>
  <tr>
    <td>vertebrae-C5</td>
    <td>http://purl.org/sig/ont/fma/fma12523</td>
    <td>Fifth cervical vertebra</td>
  </tr>
  <tr>
    <td>vertebrae-C6</td>
    <td>http://purl.org/sig/ont/fma/fma12524</td>
    <td>Sixth cervical vertebra</td>
  </tr>
  <tr>
    <td>vertebrae-C7</td>
    <td>http://purl.org/sig/ont/fma/fma12525</td>
    <td>Seventh cervical vertebra</td>
  </tr>
  <tr>
    <td>vertebrae-L1</td>
    <td>http://purl.org/sig/ont/fma/fma13072</td>
    <td>First lumbar vertebra</td>
  </tr>
  <tr>
    <td>vertebrae-L2</td>
    <td>http://purl.org/sig/ont/fma/fma13073</td>
    <td>Second lumbar vertebra</td>
  </tr>
  <tr>
    <td>vertebrae-L3</td>
    <td>http://purl.org/sig/ont/fma/fma13074</td>
    <td>Third lumbar vertebra</td>
  </tr>
  <tr>
    <td>vertebrae-L4</td>
    <td>http://purl.org/sig/ont/fma/fma13075</td>
    <td>Fourth lumbar vertebra</td>
  </tr>
  <tr>
    <td>vertebrae-L5</td>
    <td>http://purl.org/sig/ont/fma/fma13076</td>
    <td>Fifth lumbar vertebra</td>
  </tr>
  <tr>
    <td>vertebrae-T1</td>
    <td>http://purl.org/sig/ont/fma/fma9165</td>
    <td>First thoracic vertebra</td>
  </tr>
  <tr>
    <td>vertebrae-T2</td>
    <td>http://purl.org/sig/ont/fma/fma9187</td>
    <td>Second thoracic vertebra</td>
  </tr>
  <tr>
    <td>vertebrae-T3</td>
    <td>http://purl.org/sig/ont/fma/fma9209</td>
    <td>Third thoracic vertebra</td>
  </tr>
  <tr>
    <td>vertebrae-T4</td>
    <td>http://purl.org/sig/ont/fma/fma9248</td>
    <td>Fourth thoracic vertebra</td>
  </tr>
  <tr>
    <td>vertebrae-T5</td>
    <td>http://purl.org/sig/ont/fma/fma9922</td>
    <td>Fifth thoracic vertebra</td>
  </tr>
  <tr>
    <td>vertebrae-T6</td>
    <td>http://purl.org/sig/ont/fma/fma9945</td>
    <td>Sixth thoracic vertebra</td>
  </tr>
  <tr>
    <td>vertebrae-T7</td>
    <td>http://purl.org/sig/ont/fma/fma9968</td>
    <td>Seventh thoracic vertebra</td>
  </tr>
  <tr>
    <td>vertebrae-T8</td>
    <td>http://purl.org/sig/ont/fma/fma9991</td>
    <td>Eighth thoracic vertebra</td>
  </tr>
  <tr>
    <td>vertebrae-T9</td>
    <td>http://purl.org/sig/ont/fma/fma10014</td>
    <td>Ninth thoracic vertebra</td>
  </tr>
  <tr>
    <td>vertebrae-T10</td>
    <td>http://purl.org/sig/ont/fma/fma10037</td>
    <td>Tenth thoracic vertebra</td>
  </tr>
  <tr>
    <td>vertebrae-T11</td>
    <td>http://purl.org/sig/ont/fma/fma10059</td>
    <td>Eleventh thoracic vertebra</td>
  </tr>
  <tr>
    <td>vertebrae-T12</td>
    <td>http://purl.org/sig/ont/fma/fma10081</td>
    <td>Twelfth thoracic vertebra</td>
  </tr>
  <tr>
    <td>heart</td>
    <td>http://purl.org/sig/ont/fma/fma7088</td>
    <td>Heart</td>
  </tr>
  <tr>
    <td>pulmonary-vein</td>
    <td>http://purl.org/sig/ont/fma/fma66643</td>
    <td>Pulmonary vein</td>
  </tr>
  <tr>
    <td>brachiocephalic-trunk</td>
    <td>http://purl.org/sig/ont/fma/fma3932</td>
    <td>Brachiocephalic artery</td>
  </tr>
  <tr>
    <td>subclavian-artery-right</td>
    <td>http://purl.org/sig/ont/fma/fma3953</td>
    <td>Right subclavian artery</td>
  </tr>
  <tr>
    <td>subclavian-artery-left</td>
    <td>http://purl.org/sig/ont/fma/fma4694</td>
    <td>Left subclavian artery</td>
  </tr>
  <tr>
    <td>common_carotid_artery-right</td>
    <td>http://purl.org/sig/ont/fma/fma3941</td>
    <td>Right common carotid artery</td>
  </tr>
  <tr>
    <td>common_carotid_artery-left</td>
    <td>http://purl.org/sig/ont/fma/fma4058</td>
    <td>Left common carotid artery</td>
  </tr>
  <tr>
    <td>brachiocephalic-vein-left</td>
    <td>http://purl.org/sig/ont/fma/fma4761</td>
    <td>Left brachiocephalic vein</td>
  </tr>
  <tr>
    <td>brachiocephalic-vein-right</td>
    <td>http://purl.org/sig/ont/fma/fma4751</td>
    <td>Right brachiocephalic vein</td>
  </tr>
  <tr>
    <td>superior-vena-cava</td>
    <td>http://purl.org/sig/ont/fma/fma4720</td>
    <td>Superior vena cava</td>
  </tr>
  <tr>
    <td>prostate</td>
    <td>http://purl.org/sig/ont/fma/fma9600</td>
    <td>Prostate</td>
  </tr>
  <tr>
    <td>femur</td>
    <td>http://purl.org/sig/ont/fma/fma9611</td>
    <td>Femur</td>
  </tr>
  <tr>
    <td>patella</td>
    <td>http://purl.org/sig/ont/fma/fma24485</td>
    <td>Patella</td>
  </tr>
  <tr>
    <td>tibia</td>
    <td>http://purl.org/sig/ont/fma/fma24476</td>
    <td>Tibia</td>
  </tr>
  <tr>
    <td>fibula</td>
    <td>http://purl.org/sig/ont/fma/fma24479</td>
    <td>Fibula</td>
  </tr>
</table>
<table><tr><td>humerus</td><td>http://purl.org/sig/ont/fma/fma13303</td><td>Humerus</td></tr><tr><td>ulna</td><td>http://purl.org/sig/ont/fma/fma23466</td><td>Ulna</td></tr><tr><td>radius</td><td>http://purl.org/sig/ont/fma/fma23463</td><td>Radius</td></tr><tr><td>sternum</td><td>http://purl.org/sig/ont/fma/fma7485</td><td>Sternum</td></tr><tr><td>skull</td><td>http://purl.org/sig/ont/fma/fma0326476</td><td>Cranial bone</td></tr><tr><td>subcutaneous_fat</td><td>http://purl.org/sig/ont/fma/fma74315</td><td>Subcutaneous adipose tissue</td></tr><tr><td>skeletal_muscle</td><td>http://purl.org/sig/ont/fma/fma0329248</td><td>Skeletal muscle</td></tr><tr><td>spinal_cord</td><td>http://purl.org/sig/ont/fma/fma7647</td><td>Spinal cord</td></tr><tr><td>kidney</td><td>http://purl.org/sig/ont/fma/fma7203</td><td>Kidney</td></tr><tr><td>adrenal_gland</td><td>http://purl.org/sig/ont/fma/fma9604</td><td>Adrenal gland</td></tr><tr><td>thyroid_gland</td><td>http://purl.org/sig/ont/fma/fma9603</td><td>Thyroid gland</td></tr><tr><td>vertebrae_lumbar</td><td>http://purl.org/sig/ont/fma/fma9921</td><td>Lumbar vertebra</td></tr><tr><td>vertebrae_thoracic</td><td>http://purl.org/sig/ont/fma/fma9139</td><td>Thoracic vertebra</td></tr><tr><td>vertebrae_cervical</td><td>http://purl.org/sig/ont/fma/fma9915</td><td>Cervical vertebra</td></tr><tr><td>ribs</td><td>http://purl.org/sig/ont/fma/fma7574</td><td>Rib</td></tr><tr><td>scapula</td><td>http://purl.org/sig/ont/fma/fma13394</td><td>Scapula</td></tr><tr><td>hip</td><td>http://purl.org/sig/ont/fma/fma16585</td><td>Hip bone</td></tr><tr><td>gluteus_maximus</td><td>http://purl.org/sig/ont/fma/fma22314</td><td>Gluteus maximus</td></tr><tr><td>gluteus_medius</td><td>http://purl.org/sig/ont/fma/fma22315</td><td>Gluteus medius</td></tr><tr><td>gluteus_minimus</td><td>http://purl.org/sig/ont/fma/fma22317</td><td>Gluteus minimus</td></tr><tr><td>iliopsoas</td><td>http://purl.org/sig/ont/fma/fma64918</td><td>Iliopsoas</td></tr></table>

We also manually mapped the gold-standard anatomical location labels to the FMA label, as shown in Table 1. Note that the mapped concepts for the gold standard labels are not necessarily subclasses of the “Organ” concept. For example, the Mediastinum is not an organ, but it is part of the “Anatomical cluster” hierarchy (see Figure 1).
<figure><img src="image_2.png" /><figcaption>Figure 1 Class hierarchy for Mediastinum in the FMA ontology</figcaption></figure>

Finally, we are left with the challenge of describing the mapping between the Totalsegmentator structures and the gold-standard lesion localization labels. The level of granularity of the gold-standard labels varies widely, ranging from single organs (e.g. Liver) to aggregate entities such as organ sets (e.g. Bones is the set of all bones) and regions in the anatomical space (e.g. the Mediastinum).

Our approach consists of defining a set of SparQL queries searching for paths from a gold-standard label to a Totalsegmentator structure. The queries must be adapted to the level of granularity and type of gold-standard label. In cases where the gold-standard label is the organ itself, or a superclass of a Totalsegmentator structure, a simple query structured around the code shown in Figure 2 is sufficient.

```sql
{ #catch the same individuals
    ?lesion_organ a ?lesion_organ_class .
    ?totalseg_organ a ?totalsegclass .
    ?totalsegclass rdfs:subClassOf+ ?lesion_organ_class
}
UNION
{ # catch the same classes (that are not punned)
    ?lesion_organ rdfs:subClassOf? ?totalseg_organ .
    ?totalseg_organ rdfs:subClassOf? ?lesion_organ .
}
UNION
{ # unpunned classes that are subclasses of lesions
    ?totalseg_organ rdfs:subClassOf+ ?lesion_organ .
}
```

<caption>Figure 2 Sample query to identify cases where the Totalsegmentator structure and the gold-standard lesions are the same concepts, or have a subClassOf relationship.</caption>
In cases where the gold-standard label represents an aggregate of organs, we exploit the FMA relationships constitutional_part_of, regional_part_of, member_of, contained_in to identify a path from a Totalsegmentator structure to a gold-standard label. Additional care must be taken to account for subclasses: we link a Totalsegmentator structure to a gold-standard label also in cases where a parent class of the structure is contained in the gold-standard label, as well as cases where the structure is contained in a child class of the gold-standard label. For example, the FMA concept fma:7371 "Lower lobe of left lung" is asserted as regional_part_of fma:7310 "Left lung", and fma:7310 "Left lung" is a subClassOf the gold-standard label concept fma:7195 "Lung". The sample query shown in Figure 3 covers these cases. The full query is shown in Figure 4.

<figure><img src="image_3.png" /><figcaption>Figure 3 Sample query to identify cases where the gold-standard label is an aggregate of structures.</figcaption></figure>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>

PREFIX fma: <http://purl.org/sig/ont/fma/fma>

PREFIX fmapred: <http://purl.org/sig/ont/fma/>

select distinct ?lesion_organ ?totalseg_organ ?organlabel ?lesionorganlabel
WHERE
  {

values ?lesion_organ {fma:14543 fma:18252 fma:24864 fma:5018 fma:5022
fma:50801 fma:7163 fma:7195 fma:7196 fma:7197 fma:7198 fma:7203 fma:79646
fma:9578 fma:9583 fma:9584 fma:9601 fma:9603 fma:9604 fma:9608 fma:9826
fma:12266 fma:9738 } .

values ?totalseg_organ {fma:329248 fma:10014 fma:10037 fma:10059 fma:10081
fma:10951 fma:12519 fma:12520 fma:12521 fma:12522 fma:12523 fma:12524
fma:12525 fma:13072 fma:13073 fma:13074 fma:13075 fma:13076 fma:13077
fma:13303 fma:13322 fma:13323 fma:13394 fma:13395 fma:13396 fma:14543
fma:14765 fma:14766 fma:15629 fma:15630 fma:15900 fma:12526 fma:21387
fma:21388 fma:22314 fma:22315 fma:22317 fma:22328 fma:22329 fma:22330
fma:22331 fma:22332 fma:22333 fma:22731 fma:22732 fma:23130 fma:23131
fma:23463 fma:23466 fma:24474 fma:24475 fma:24476 fma:24479 fma:24485
fma:24728 fma:3734 fma:3932 fma:3941 fma:3953 fma:4058 fma:4694 fma:4720
fma:4751 fma:4761 fma:50801 fma:64918 fma:64919 fma:64920 fma:66326 fma:66643
fma:7088 fma:7096 fma:7097 fma:7098 fma:7101 fma:7131 fma:7148 fma:7196
fma:7197 fma:7198 fma:7200 fma:7202 fma:7203 fma:7204 fma:7205 fma:7206
fma:7333 fma:7337 fma:7370 fma:7371 fma:7383 fma:7394 fma:74315 fma:7485
fma:7574 fma:7647 fma:7857 fma:7882 fma:7909 fma:7957 fma:7987 fma:8012
fma:8039 fma:8066 fma:8093 fma:8148 fma:8175 fma:8202 fma:8229 fma:8256
fma:8283 fma:8310 fma:8364 fma:8391 fma:8445 fma:8472 fma:8531 fma:8532
fma:8533 fma:8534 fma:9139 fma:9165 fma:9187 fma:9209 fma:9248 fma:9462
fma:9600 fma:9603 fma:9604 fma:9611 fma:9915 fma:9921 fma:9922 fma:9945
fma:9968 fma:9991 fma:16586 fma:16587 fma:0326476} .

{ ?totalseg_organ rdfs:subClassOf* ?y .
?y (fmapred:constitutional_part_of | fmapred:regional_part_of |
fmapred:member_of | fmapred:contained_in)+ ?x .
?x rdfs:subClassOf* ?lesion_organ
}
UNION

UNION
{ ?lesion_organ (fmapred:constitutional_part | fmapred:regional_part |
fmapred:member)* ?intermediate .
?intermediate (fmapred:constitutional_part_of |
fmapred:regional_part_of | fmapred:member_of)* ?totalseg_organ . }
UNION
{ #catch the same individuals
?lesion_organ a ?lesion_organ_class .
?totalseg_organ a ?totalsegclass .
?totalsegclass rdfs:subClassOf+ ?lesion_organ_class
}
UNION
{ # catch the same classes (that are not punned)
?lesion_organ rdfs:subClassOf? ?totalseg_organ .
?totalseg_organ rdfs:subClassOf? ?lesion_organ .
}
UNION

UNION
{ # unpunned classes that are subclasses of lesions
?totalseg_organ rdfs:subClassOf+ ?lesion_organ .
}
?totalseg_organ rdfs:label ?organlabel .
Figure 4 Full SparQL query to link Totalsegmentator structures to gold-standard labels. This query is executed on the rdf representation of the FMA ontology.

## Mapping improvement through heuristics

The process described thus far is helpful to establish a good baseline, but some imperfections remain, as shown in Table 4.

<table><caption>Table 4 Remaining issues after establishing a baseline link between Totalsegmentator structures and gold-standard labels.</caption><thead><tr><td>No.</td><td>Description</td><td>Example</td></tr></thead><tbody><tr><td>1</td><td>The same Totalsegmentator structure is linked with multiple gold-standard labels</td><td>Adrenal glands are matched with both the "Adrenal gland" gold-standard label and the "Sub-diaphragmatic" label. The latter match happens because of the following chain of relationships "Adrenal gland" member_of "Set of viscera of abdomen" contained_in "Abdominal cavity".</td></tr><tr><td>2</td><td>No valid path can be found between the Totalsegmentator structure and the correct gold-standard label</td><td>The "Set of left erector spinae" is not linked to "Muscle organ" by any valid relationship.</td></tr></tbody></table>

Issue number 1 is solved by establishing a priority among the different ways in which a Totalsegmentator structure may be linked to the gold-standard label:

1. The Structure is the same concept as the gold-standard label;

2. The Structure is a sub-class of the gold-standard label (in case of multiple matches, choose the one that minimizes the number of intermediate subclasses);

3. The Structure is linked to the gold-standard label through one or more of the constitutional_part_of, regional_part_of, member_of, contained_in relationships.

Issue number 2 was solved manually, due to the small number of structures falling in this category: the erector spinae and iliopsoas muscles.

The final result is shown in Table 5.

<table><caption>Table 5 Mapping of Totalsegmentator structures to gold-standard labels, both represented through their corresponding FMA concepts.</caption><thead><tr><td>totalsegmentator</td><td>gold-standard label</td></tr></thead><tbody><tr><td>Stomach</td><td>Abdominal cavity</td></tr><tr><td>Small intestine</td><td>Abdominal cavity</td></tr><tr><td>Adrenal gland</td><td>Adrenal gland</td></tr><tr><td>Left adrenal gland</td><td>Adrenal gland</td></tr><tr><td>Right adrenal gland</td><td>Adrenal gland</td></tr><tr><td>Gallbladder</td><td>Biliary system</td></tr><tr><td>Sternum</td><td>Bone organ</td></tr><tr><td>Cranial bone</td><td>Bone organ</td></tr><tr><td>Right scapula</td><td>Bone organ</td></tr><tr><td>Left scapula</td><td>Bone organ</td></tr><tr><td>Scapula</td><td>Bone organ</td></tr></tbody></table>
<table>
  <tr>
    <td>Sacral vertebra</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Rib</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Right ninth rib</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Right eighth rib</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Right sixth rib</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Tibia</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Right fifth rib</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Right fourth rib</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Right third rib</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Right second rib</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Right twelfth rib</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Right eleventh rib</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Right tenth rib</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Right first rib</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Left ninth rib</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Left eighth rib</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Left seventh rib</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Right seventh rib</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Ulna</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Axis</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Left sixth rib</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Lumbar vertebra</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Cervical vertebra</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Ninth thoracic vertebra</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Eighth thoracic vertebra</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Seventh thoracic vertebra</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Sixth thoracic vertebra</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Fifth thoracic vertebra</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Fourth thoracic vertebra</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Third thoracic vertebra</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Second thoracic vertebra</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Twelfth thoracic vertebra</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Eleventh thoracic vertebra</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Tenth thoracic vertebra</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>First thoracic vertebra</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>First sacral vertebra</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Fifth lumbar vertebra</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Fourth lumbar vertebra</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Third lumbar vertebra</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Second lumbar vertebra</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>First lumbar vertebra</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Seventh cervical vertebra</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Sixth cervical vertebra</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Fifth cervical vertebra</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Fourth cervical vertebra</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Third cervical vertebra</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Atlas</td>
    <td>Bone organ</td>
  </tr>
  <tr>
    <td>Left fifth rib</td>
    <td>Bone organ</td>
  </tr>
</table>
<table><tr><td>Left fourth rib</td><td>Bone organ</td></tr><tr><td>Left third rib</td><td>Bone organ</td></tr><tr><td>Left femur</td><td>Bone organ</td></tr><tr><td>Femur</td><td>Bone organ</td></tr><tr><td>Thoracic vertebra</td><td>Bone organ</td></tr><tr><td>Right clavicle</td><td>Bone organ</td></tr><tr><td>Left hip bone</td><td>Bone organ</td></tr><tr><td>Right hip bone</td><td>Bone organ</td></tr><tr><td>Left clavicle</td><td>Bone organ</td></tr><tr><td>Humerus</td><td>Bone organ</td></tr><tr><td>Left humerus</td><td>Bone organ</td></tr><tr><td>Right humerus</td><td>Bone organ</td></tr><tr><td>Right femur</td><td>Bone organ</td></tr><tr><td>Fibula</td><td>Bone organ</td></tr><tr><td>Patella</td><td>Bone organ</td></tr><tr><td>Left second rib</td><td>Bone organ</td></tr><tr><td>Left twelfth rib</td><td>Bone organ</td></tr><tr><td>Left eleventh rib</td><td>Bone organ</td></tr><tr><td>Left tenth rib</td><td>Bone organ</td></tr><tr><td>Left first rib</td><td>Bone organ</td></tr><tr><td>Radius</td><td>Bone organ</td></tr><tr><td>Brain</td><td>Brain</td></tr><tr><td>Colon</td><td>Colon</td></tr><tr><td>Left kidney</td><td>Kidney</td></tr><tr><td>Kidney</td><td>Kidney</td></tr><tr><td>Right kidney</td><td>Kidney</td></tr><tr><td>Liver</td><td>Liver</td></tr><tr><td>Middle lobe of right lung</td><td>Lung</td></tr><tr><td>Upper lobe of right lung</td><td>Lung</td></tr><tr><td>Upper lobe of left lung</td><td>Lung</td></tr><tr><td>Lower lobe of left lung</td><td>Lung</td></tr><tr><td>Lower lobe of right lung</td><td>Lung</td></tr><tr><td>Aorta</td><td>Mediastinum</td></tr><tr><td>Right brachiocephalic vein</td><td>Mediastinum</td></tr><tr><td>Esophagus</td><td>Mediastinum</td></tr><tr><td>Brachiocephalic artery</td><td>Mediastinum</td></tr><tr><td>Trachea</td><td>Mediastinum</td></tr><tr><td>Myocardium</td><td>Mediastinum</td></tr><tr><td>Right ventricle</td><td>Mediastinum</td></tr><tr><td>Left ventricle</td><td>Mediastinum</td></tr><tr><td>Heart</td><td>Mediastinum</td></tr><tr><td>Left atrium</td><td>Mediastinum</td></tr><tr><td>Right atrium</td><td>Mediastinum</td></tr><tr><td>Set of left erector spinae</td><td>Muscle organ</td></tr><tr><td>Set of right erector spinae</td><td>Muscle organ</td></tr><tr><td>Right iliopsoas</td><td>Muscle organ</td></tr><tr><td>Left iliopsoas</td><td>Muscle organ</td></tr><tr><td>Right gluteus minimus</td><td>Muscle organ</td></tr></table>
<table><tr><td>Left gluteus minimus</td><td>Muscle organ</td></tr><tr><td>Left gluteus maximus</td><td>Muscle organ</td></tr><tr><td>Right gluteus maximus</td><td>Muscle organ</td></tr><tr><td>Gluteus medius</td><td>Muscle organ</td></tr><tr><td>Left gluteus medius</td><td>Muscle organ</td></tr><tr><td>Right gluteus medius</td><td>Muscle organ</td></tr><tr><td>Gluteus minimus</td><td>Muscle organ</td></tr><tr><td>Gluteus maximus</td><td>Muscle organ</td></tr><tr><td>Pancreas</td><td>Pancreas</td></tr><tr><td>Prostate</td><td>Prostate</td></tr><tr><td>Spleen</td><td>Spleen</td></tr><tr><td>Thyroid gland</td><td>Thyroid gland</td></tr></table>

## Lesion localization

We preprocessed the lesion annotation masks provided by nuclear physicians by computing the connected components and considering each connected component as a separate lesion. We then computed the intersection between each Totalsegmentator structure and each lesion's connected component separately. For each component, we aggregated the intersections for structures that are mapped to the same gold-standard label, and finally assigned as lesion localization the label with the largest intersection.

For example, the red lesion in Figure 5 intersects the aorta on 50 voxels and the esophagus on 70 voxels. Since both aorta and esophagus are mapped by the ontology to the mediastinum, we sum these contributions for a total intersection of 120 voxels, and compare it with the total intersection for other lesion locations. Since the mediastinum is the label with the maximal intersection, we predict the lesion location as mediastinum.

<figure><img src="image_4.png" /><figcaption>Figure 5 Intersection between lesion masks annotated by nuclear physicians and Totalsegmentator segmentations.</figcaption></figure>
Knowledge graph

Patient stratification

<figure><img src="image_5.png" /><figcaption>**Figure 1:** *Top:* Co-occurrences of lesion locations. On the left, raw lesion locations as identified by the automatic segmentation tool. In the center, an example of the mapping to gold standard labels grounded on the ontological relationships between anatomical structures. On the right, clinically-interpretable representation of the co-occurrence information, leading to improved patient phenotyping.</figcaption></figure>

<caption>**Bottom:** Phenotyping through patient clustering. On the left, groups arising from clustering on the raw lesion locations as identified by the automatic segmentation tool. On the right, groups arising from clustering on the lesion locations after the ontology-driven mapping to gold standard labels, leading to more interpretable clusters with a stronger correlation to overall survival.</caption>