# Automatic Segmentation of Primary Central Nervous System Lymphoma at Clinical Routine Postcontrast T1-weighted MRI

Guanghui Fu, Lucia Nichelli, Darío Herrán de la Gala, Sophie Loizillon, Camile Bousfiha, Romain Valabregue, Agusti Alentorn, Khê Hoang-Xuan, Bertrand Mathon, Carole Soussain, et al.

► To cite this version:

Guanghui Fu, Lucia Nichelli, Darío Herrán de la Gala, Sophie Loizillon, Camile Bousfiha, et al. Automatic Segmentation of Primary Central Nervous System Lymphoma at Clinical Routine Postcontrast T1-weighted MRI. Radiology: Imaging Cancer, 2025, 7 (5), 10.1148/rycan.240446. hal-05283497

HAL Id: hal-05283497

https://u-picardie.hal.science/hal-05283497v1

Submitted on 28 Sep 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Automatic Segmentation of Primary Central Nervous System
Lymphoma on Clinical Routine Post-contrast T1-weighted
MRI

Guanghui Fu¹, Lucia Nichelli¹,², Dario Herran², Sophie Loizillon¹, Camile Bousfiha¹, Romain Valabregue³,⁴, Agusti Alentorn³,⁵, Khê Hoang-Xuan³,⁵, Bertrand Mathon³,⁶, Carole, Soussain⁷, Jean Pierre Marolleau⁸, Jérôme Paillassa⁹, Luc Taillandier¹⁰, Philippe, Agapé¹¹, Anna, Schmitt¹², Olivier, Chinot¹³, Guido, Ahle¹⁴, Didier Dormont¹,², Caroline Houillier³,⁵, Stéphane Lehéricy²,³,⁴, Olivier Colliot*¹, and for the French network for Oculocerebral Lymphoma (LOC Network)¹⁵⁻⁴²

¹Sorbonne Université, Institut du Cerveau - Paris Brain Institute - ICM, CNRS, Inria, Inserm, AP-HP, Hôpital de la Pitié Salpêtrière, Paris, France

²AP-HP, Pitié Salpêtrière, DMU DIAMENT, Dep. of Neuroradiology, Paris, France.

³ICM, Centre de NeuroImagerie de Recherche-CENIR, Paris, France.

⁴Sorbonne Université, Institut du Cerveau - Paris Brain Institute - ICM, CNRS, Inserm, AP-HP, Hôpital de la Pitié Salpêtrière, F-75013, Paris, France.

⁵AP-HP, Pitié Salpêtrière, DMU Neurosciences, Dep. of Neurology 2, Paris, France.

⁶AP-HP Pitié-Salpêtrière Hospital, Department of Neurosurgery, Sorbonne University, Paris, France

⁷Department of Clinical Hematology, Institut Curie, Saint Cloud, Paris, France.

⁸Department of Hematology, CHU Amiens-Picardie, Amiens, France

⁹Department of Hematology, CHU d'Angers, Angers, France

¹⁰Department of Neuro-Oncology, Université de Lorraine, CHRU Nancy, Nancy, France

¹¹Department of Hematology, Centre Hospitalier de Laval, Laval, France

¹²Department of Hematology, Institut Bergonié, Bordeaux, France

¹³Department of Neuro-Oncology, AP-HM, University Hospital Timone, Marseille, France

¹⁴Department of Neurology, Hôpital Pasteur - Hôpitaux Civils de Colmar, Colmar, France

¹⁵⁻⁴²For affiliations 15 to 42, see the Acknowledgements section.

**Article type** Original research

**Key Points**

* This retrospective study, based on a cohort of 135 subjects and utilizing post-contrast T1-weighted MRI acquired in a clinical context, led to the development of the first publicly available automatic segmentation model for primary central nervous system lymphoma (PCNSL). The model achieved high mean Dice scores of 0.84 on internal test data (n=44) and 0.88 on external test data (n=48).

* The model demonstrated strong volumetric correlation with manual annotations (r=0.99, P<.001 internal; r=0.98, P<.001 external) and achieved F1-scores of 76% and 67% on internal and external test sets, respectively.

* Performance varied by lesion type, with the best results in homogeneous, discrete lesions and decreased performance when multiple ill-defined infracentimetric lesions occur.

Further author information: (Send correspondence to olivier.colliot@cnrs.fr)
**Abbreviations** FLAIR = fluid-attenuated inversion recovery, IPCG= International Primary CNS Lymphoma Collaborative Group, MASD = mean average surface distance, PCNSL = primary central nervous system lymphoma, 3D = three-dimensional

**Summary** The trained deep learning model accurately segmented primary central nervous system lymphoma with high generalizability across eight centers, offering an open-source tool that may enhance quantitative imaging evaluation in brain lymphoma.

**Keywords:** Brain Lymphoma, Brain Tumor, Automatic Segmentation, Artificial Intelligence, Deep Learning

# ABSTRACT

**Purpose** To develop and validate a deep learning model for automatic segmentation of primary central nervous system lymphoma (PCNSL) at post-contrast T1-weighted MRI.

**Materials and Methods** Data were retrospectively collected from patients with pathologically proven immunocompetent PCNSL between September 2010 and February 2022. Postcontrast T1-weighted MRI scans were used to train and validate a deep learning model based on the nnU-net framework. Manual segmentation by neuroradiologists served as the reference standard. The model was trained using an internal dataset from a single center and tested on both internal and external test sets from seven additional centers. Performance was assessed using Dice score, mean average surface distance, and F1 score. Statistical comparisons were performed using Mann-Whitney U test and bootstrap resampling for CIs.

**Results** The study included 135 patients (68 female, 66 male, and one of unspecified sex; internal dataset: mean age [±SD], 67.0 years ± 12.0; external dataset: mean age, 75.5 years ± 13.6). The model achieved a mean Dice score of 0.84 (95% CI: 0.79, 0.88) on the internal test set (n = 44) and 0.88 (95% CI: 0.84, 0.91) on the external test set (n = 48), with no evidence of a difference between test sets (P = .59). Performance varied by lesion type; accuracy was highest in homogeneous discrete lesions, and performance was slightly decreased when numerous poorly defined infracentimetric lesions occurred. Strong volumetric correlation was observed between automatic and manual segmentations (internal: r = 0.99, P < .001; external: r = 0.98, P < .001).

**Conclusion** A deep learning model achieved accurate and robust automatic segmentation of PCNSL across multiple clinical centers with different MRI acquisition parameters.

# 1. INTRODUCTION

Primary central nervous system lymphoma (PCNSL) is a rare, highly malignant brain tumor entity within diffuse large B-cell lymphoma whose incidence is continuously rising especially in the elderly.<sup>1,2</sup> Several mutations, molecular alterations, and tumor micro-environment signatures have been linked to PCNSL in the past few decades, increasing the stratified complexity behind an apparently homogeneous abnormality.<sup>3</sup> Recently, multiomics analysis has identified four different molecular PCNSL subtypes with distinct clinical behaviors and survival characteristics, laying the foundation for a deeper and more integrated clinicopathologic understanding.<sup>4,5</sup> The oncologic management of PCNSL has also progressed substantially and is still rapidly evolving with the advent of targeted therapies and CD19-directed chimeric antigen receptor T cells.<sup>6-8</sup>

In this flourishing, personalized, therapeutic scenario, accurate objective evaluation of tumor burden is pivotal. To date, according to the International Primary CNS Lymphoma Collaborative Group (IPCG),<sup>9</sup> lymphoma tumor burden is defined by blood-brain barrier leakiness visible as abnormal contrast enhancement on gadolinium-enhanced T1-weighted MRI sequences. Brain lymphoma pathologic abnormality is known to be infiltrative and is not limited to the rupture in blood-brain barrier visible on a gadolinium-enhanced T1-weighted sequence; it is also visible as hyperintense on T2-weighted fluid-attenuated inversion recovery (FLAIR) images. However, T2-weighted FLAIR hyperintensities have lower specificity than the blood-brain barrier disruption visible at
gadolinium-enhanced T1-weighted MRI, and evidence on the role of T2-weighted FLAIR sequences in brain lymphoma is scarce. Thus, nonenhancing tumor burden evaluation has not been included in IPCG criteria.

Although deep learning segmentation models have already been proposed for diffuse gliomas segmenta-
tion,¹⁰,¹¹ no publicly-available segmentation tool exist for brain lymphoma, probably given the rarity of the
disease. The only automated volumetric models have no independent clinical validation.¹²,¹³ Pennig et al.¹³
trained a deep learning segmentation model (DeepMedic¹⁴) using MRI data from 220 patients with gliomas, and
applied the model to 69 scans from 43 patients with lymphoma, achieving a median Dice score of 0.76. However,
since the model was trained with gliomas, it is unclear if this approach is optimal for lymphomas. Lymphomas
often include a varying number of lesions, which is rare for gliomas.¹⁵,¹⁶ Multiple periventricular lesions are
also common in lymphomas and less common in gliomas. In addition, in immunocompetent individuals, lym-
phomas usually display intense homogeneous contrast enhancement; in contrast, gliomas are more commonly
heterogeneous, with several necrotic and hemorrhagic components. Finally, there are no reports on external
generalization of models using the current segmentation algorithms, which is crucial for clinical applications.

Zhang et al.¹² built deep learning methods for automatic classification of brain tumors, including 92 patients with lymphoma. Their approach involved a segmentation network, the results of which were used for classification, but did not evaluate the segmentation accuracy. Furthermore, none of these models can be downloaded and tested. Hence, a publicly available, automatic, validated segmentation model for brain lymphomas is highly desirable. Brain tumor segmentation is a complex task because of the high variability of lesion location, morphologic features, and dimension. The task is even more complex for real-life data because in daily clinical settings, MRI examinations are heterogeneous in terms of scanners, magnetic fields, acquisition parameters, two-dimensional or three-dimensional (3D) sequences, image resolutions, and sequence types (spin echo vs gradient echo).

In this study, we trained and evaluated, with both internal and external testing, a nnU-net model¹⁷ for au-
tomatic segmentation of histologically proven primary brain lymphomas at clinical routine gadolinium-enhanced
T1-weighted MRI.

## 2. MATERIALS & METHODS

2.1 Study Sample

A multicenter retrospective study was conducted on 135 immunocompetent patients with pathologically confirmed Epstein-Barr virus negative PCNSL from the French Network for Oculocerebral Lymphoma network database. The study was approved by the Institutional Ethical Committee (Pitié-Salpêtrière Hospital, Île-de-France VI, no. DC-2009-95 ) and by the French Data Protection Authority (CNIL, Commission Nationale de l'Informatique et des Libertés, DR-2013-279). According to French regulation, consent was waived as these images were acquired as part of the routine clinical care of the patients.

Each patient underwent T1-weighted MRI after injection of gadolinium-based contrast agent. Patient selection process is described in Figure 1, with additional details provided in Figure S1. First, we included patients from a single tertiary center (Pitié Salpêtrière University Hospital) based on the following inclusion criteria: (a) pathological diagnosis of diffuse large B-cell lymphoma of the CNS, (b) MRI availability, (c) absence of systemic involvement assessed either with full-body CT scan or PET scan, (d) immunocompetent status as well as negative HIV serology, and (e) age over 18 years.

All images were converted from DICOM to NIfTI using dicom2niix¹⁸ and organized according to the Brain
Imaging Data Structure standard. Images were then selected by a certified neuro-radiologist (L.N) according
to the following exclusion criteria (a) absence of a T1 weighted sequence after injection of gadolinium contrast
agent, (b) severe image artifacts on gadolinium-enhanced T1-weighed MRI that affected the diagnostic interpre-
tation of the images, (c) errors in DICOM- NIFTI conversion that affected the diagnostic interpretation of the
images. Previous treatment status (corticotherapy as well as different anti-tumor treatment strategies) was not
an exclusion criterion, but in cases with multiple MRIs for a single individual, baseline MRI (eg, MRI before
therapy) was chosen. Artifacts of various origins (eg, motion, blooming, aliasing) that did not interfere sub-
stantially with image interpretation were not excluded. In case of multiple gadolinium-enhanced T1-weighted
MRI sequences, high-resolution thin slices volumetric sequences were preferred. If more than one high-resolution
<figure><img src="image_2.png" /><figcaption>Figure 1. Data distribution of training and test sets. “External” refers to the sets with patients scanned from September 2010 to February 2022 at institutions different from the one where the training patients were scanned. N denotes the number of patients.</figcaption></figure>

sequence was present, a 3D spin echo sequence was preferred to a 3D gradient echo sequence, as it provides an improved signal-to-noise-ratio for the detection of T1 abnormal contrast enhancement.⁹

Eighty-seven patients came from a single center (Pitié Salpêtrière University Hospital) meet the inclusion criteria. This cohort was used for training/validation and for internal testing. Subsequently, we included patients from other hospitals using the same selection process of the internal cohort. Forty-eight patients met the criteria and are referred to as the external dataset. They came from seven centers (Center I: Clermont-Ferrand University Hospital, seven patients; center II: Amiens University Hospital, seven patients; center III: Nancy University Hospital, seven patients; center IV: Timone University Hospital, seven patients; center V: Pasteur University Hospital, seven patients; center VI: Bergonié Institute, six patients; center VII: Oncologic Institute of the West, seven patients.). Forty-eight patients met the inclusion and exclusion criteria. The study flowchart is shown in Figure S1.

## 2.2 MRI's Characteristics

The images were acquired as part of clinical routine and were thus not harmonized (different MRI scanners, sequence types, resolutions, contrast to noise, etc.).

Internal data included 68 isotropic (or quasi-isotropic) 3D acquisitions and 19 non-isotropic 2D acquisitions across different field strengths (3T: 46, 1.5T: 35, 1.0T: six) and manufacturers (GE Healthcare: 78, Philips: six, Siemens: three). The dataset contained 53 gradient echo and 34 spin echo images. In-plane resolution ranged from 0.47 mm to 2 mm and slice thickness ranged from 0.47 mm to 6.5 mm. The mean (± SD) number of lesions per scan was 3.68±3.26. Seventy-six MRI scans were acquired at baseline, 10 were acquired after induction therapy and one patient had an unspecified treatment status.

External data included 35 isotropic (or quasi-isotropic) 3D acquisitions and 13 non-isotropic 2D acquisitions. Field strengths and manufacturers also varied (field strengths: 3T: 10 patients, 1.5T: 38 patients; manufacturers: General Electric Healthcare: 23 patients, Philips: 10 patients, Siemens: 14 patients, Toshiba: one patients), as well as sequence types (37 gradient echo, 11 spin echo). In-plane resolution ranged from 0.38 mm to 1.4 mm and slice thickness ranged from 0.47 mm to 6.3 mm. The average number of lesions was 5.42±5.54 per scan. The dataset included 34 pre-treatment patients, 13 post-treatment patients, and one patient with an unspecified treatment status.

A representation of the cohorts' division used for the development of model is shown in Figure 1. Figure 2 illustrates the variability in lesion appearance, location, and image quality.
<figure><img src="image_3.png" /><figcaption>Figure 2. Gadolinium T1-weighted contrast-enhanced MRI scans acquired in clinical practice for patients with primary central nervous system lymphoma. For each of four patients (A–D), an image on an axial plane is shown, followed by the same image overlaid with the annotated segmentation, and a zoom-in of the area(s) marked with a box.</figcaption></figure>

## 2.3 Manual Annotation of Tumors

The full dataset was annotated by a certified neuroradiologist (L.N) with five years of experience in neuro-oncology imaging. According to IPCG consensus recommendations, lymphoma tumor burden is defined by the measure of blood-brain barrier leakiness visible as an abnormal contrast-enhancement on gadolinium T1-weighted MRI. Therefore, in this study we decided to focus on enhancing tumor burden and we did not evaluate the non-enhancing tumor burden visible on T2-weighted FLAIR sequences.

All contrast-enhancing lesions were segmented using ITK-SNAP software (version 3.6) with a semi-automatic active contour segmentation method and subsequent manual correction. To evaluate the consistency of manual segmentations, which we considered the reference standard, we assessed both intra-reader and inter-reader variability using key segmentation metrics. Intra-reader variability was quantified after the primary reader (L.N) reannotated a random subset of 50 cases with a one-month interval. Inter-reader variability was assessed after a second board certified neuroradiologist (D.H), with one year of experience in neuro-oncology imaging, independently annotated the same 50 cases.

The primary reader (L.N) also assessed the type of MRI sequence (spin echo or gradient echo) and classified the PCNSL imaging patterns. Because there is no reference standard for MRI imaging characteristics of PCNSL, based on Figure 2 of the IPCG consensus recommendations,⁹ the radiological presentation of PCNSL was divided as follows: unique, homogeneous lesion (group A); multiple discrete homogeneous lesions (group B); single heterogeneous lesions (group C); multiple heterogeneous lesions, including linear ependymal enhancement (group D); and linear perivascular enhancement (group E).

## 2.4 Development of Automatic Segmentation Model

Each MRI volume in this dataset was resized and resampled to isotropic 1×1×1mm³ voxels and dimensions of 240×240×160 using resample and resize operations from TorchIO¹⁹ for model training propose. We also rescaled the intensity to a range of 0 and 1. At the evaluation step, we resample the images back to the original resolution.

The segmentation approach was based on nnU-Net V2, a supervised deep-learning segmentation method¹⁷ which has been shown to be one of the most consistently effective algorithms across many medical applications.²⁰ Initially, the pipeline autonomously extracted features from the dataset, including intensity distribution, image size, modalities, to create data fingerprints. Subsequently, rule-based parameters utilize these dataset fingerprints to modify specific properties of the segmentation pipeline, following established heuristic rules. This included the automatic selection of appropriate image resampling methods, batch sizes, etc. Certain robust configurations were predetermined, including the automatic determination of model architecture, training schedules, and implementation of data augmentation strategies. During the training time, 5-fold cross-validation was executed on the training/validation data for 1000 epochs of training, to select the optimal configuration. No modifications were
made to the nnU-Net algorithm, ensuring full compatibility, reproducibility, and easy deployment for researchers and clinicians. The model was trained based on a single Tesla V100SXM2-32GB GPU. This may vary depending on computational hardware in other environments.

We first tested the model trained on contrast-enhancing glioma data using the same nnU-Net framework to assess cross-disease transferability (Details in Section Appendix H). As shown in Table S3, this model performed poorly on PCNSL, with Dice scores of 0.14 (internal) and 0.07 (external), and MASD values of 44 mm (internal) and 69 mm (external). Given these low performances, we therefore utilized the same nnU-Net framework, trained on our Study Sample, and tested it.

### 2.4.1 Evaluation Metrics

To evaluate the performance of the nnU-net model we used the following metrics, based the consensus recommendations of Metrics Reloaded.²¹ Segmentation accuracy was assessed using the 3D Dice score and the mean average surface distance (MASD) which are computed at the voxel level and results in one measure per patient. The ability to detect the multiple lesion instances was assessed using F1-score which is computed at the instance-level and results in one measure per patient. The F1-score evaluates the detection performance by balancing precision and recall for the different lesion instances of a given patient.

The segmentation map of a given patient was broken down into instances using connected components analysis. The localization criterion was the mask Intersection over Union (IoU). We chose a small overlap threshold (IoU>0.1 in our experiment) as there was a high variability of structure sizes. The localization criterion allows computing the precision (proportion of predicted instances which are present in the manual segmentation) and the recall (proportion of manual segmentation instances which are retrieved by the automatic segmentation). The F1-score is then the harmonic mean of precision and recall. It is computed for each patient and quantifies how well the different lesions of a given patient are detected. In addition, even though this metric is not present in Metrics Reloaded recommendations, we assessed the concordance of manual and automatic volumetric measurements using Pearson's correlation because it is clinically useful information.

### 2.4.2 Statistical Analysis

For each metric reported, the mean value across patients as well as the corresponding 95% confidence interval (CI) computed using bootstrap resampling (5,000 samples) over the independent test set. P<.05 was considered statistically significant. To compare model performance across subgroups, we assessed the statistical significance of Dice score differences between the internal and external test sets using the Mann-Whitney U test. This subgroup analysis was pre-specified as part of our evaluation protocol. Statistical analyses were performed using Python (version 3.11.4).

### 2.4.3 Data Availability

The repository was made publicly available, containing the trained model to automatically segment brain lymphoma. We could not make the images of individual patients available due to regulatory constraints. The trained model is available at: https://github.com/GuanghuiFU/nnunet_lymphoma_segment/tree/main.

## 3. RESULTS

### 3.1 Patient Characteristics

The internal dataset was randomly split into a training/validation set with 43 patients (25 females, 18 males) aged 32-86 years (mean 67.07 years, ±11.79) and an internal test set with 44 patients (23 females, 20 males, one unspecified) aged 32-86 years (mean 67.03 years, ±12.23). Forty-eight patients came from seven other centers and formed the external test set (20 females, 28 males) aged 33-95 years (mean 75.52 years, ± 13.60). The test sets were only used to evaluate the performance of the models. MRI acquisition date ranged between 2010 and 2021. Data distribution can be seen in Table 1. The distribution of Karnofsky scores for the internal and external test sets is shown in Figure S2.
<table><caption>Table 1. Distribution of Datasets for Model Development. [Note] Unless otherwise noted, values are expressed as numbers of patients (corresponding to the number of scans). Values are the means ± SDs, with ranges in parentheses. The model was trained using an internal dataset and validated both internally (internal) and across seven external centers (external I-VII). In the location column, “S” denotes supratentorial and “I” denotes infratentorial. In the categories column, the classifications are as follows: unique, homogeneous lesion (group A); multiple discrete homogeneous lesions (group B); single discrete heterogeneous lesions (group C); multiple heterogeneous lesions, including linear ependymal enhancement (group D); and linear perivascular enhancement (group E). A tumor may be located in both supratentorial and infratentorial regions simultaneously and may belong to multiple categories.</caption><thead><tr><th>Medical center</th><th>Data split</th><th>N</th><th>N<sub>lesion</sub></th><th>Age (mean±SD)</th><th>Sex (N)</th><th>Location (N)</th><th>Categories (N)</th></tr></thead><tbody><tr><td rowspan="2">Internal</td><td>Train/Val</td><td>43</td><td>2.76±2.64</td><td>32-86, 67.07±11.79</td><td>M: 18; F: 25</td><td>S: 41; I: 5</td><td>1: 9; 2: 11; 3: 8; 4: 12; 5: 3</td></tr><tr><td>Test</td><td>44</td><td>3.68±3.22</td><td>32-86, 67.03±12.23</td><td>M: 20; F: 23, U: 1</td><td>S: 42; I: 13</td><td>1: 14; 2: 12; 3: 9; 4: 5; 5: 4</td></tr><tr><td>External-I</td><td rowspan="6">Test</td><td>7</td><td>7.29±8.92</td><td>33-95, 72.43±7.78</td><td>M: 2; F: 5</td><td>S: 5; I: 3</td><td>1: 2; 2: 1; 3: 2; 4: 2; 5: 0</td></tr><tr><td>External-II</td><td>7</td><td>9±6.93</td><td>72-89, 80.29±2.83</td><td>M: 6; F: 1</td><td>S: 6; I: 2</td><td>1: 2; 2: 1; 3: 1; 4: 2; 5: 1</td></tr><tr><td>External-III</td><td>7</td><td>4.57±3.29</td><td>59-91, 76.14±16.97</td><td>M: 4; F: 3</td><td>S: 5; I: 3</td><td>1: 0; 2: 4; 3: 1; 4: 2; 5: 0</td></tr><tr><td>External-IV</td><td>7</td><td>4.43±4.07</td><td>48-88, 71.71±12.02</td><td>M: 3; F: 4</td><td>S: 7; I: 0</td><td>1: 2; 2: 1; 3: 3; 4: 1; 5: 0</td></tr><tr><td>External-V</td><td>7</td><td>4.29±3.49</td><td>60-91, 72.29±15.56</td><td>M: 4; F: 3</td><td>S: 7; I: 1</td><td>1: 2; 2: 3; 3: 2; 4: 0; 5: 0</td></tr><tr><td>External-VI</td><td>6</td><td>3.83±2.61</td><td>75-89, 83.5±1.41</td><td>M: 5; F: 1</td><td>S: 5; I: 1</td><td>1: 3; 2: 1; 3: 1; 4: 1; 5: 0</td></tr><tr><td>External-VII</td><td rowspan="2">Test</td><td>7</td><td>3.28±2.60</td><td>45-90, 72.43±7.78</td><td>M: 4; F: 3</td><td>S: 6; I: 3</td><td>1: 1; 2: 3; 3: 2; 4: 1; 5: 0</td></tr><tr><td>External (total)</td><td>48</td><td>5.27±5.48</td><td>33-95, 75.52±13.60</td><td>M: 28; F: 20</td><td>S: 41; I: 13</td><td>1: 12; 2: 14; 3: 12; 4: 9; 5: 1</td></tr></tbody></table>

## 3.2 Manual Segmentation Reliability

For intra-reader reliability, where the primary annotator re-segmented cases after a one-month interval, the variability metrics (in %, mean [95% bootstrap CI]) were: Dice score: 0.85 [0.82, 0.88], F1-score: 75% [68%, 82%], and Mean average surface distance (MASD): 1.05 [0.77, 1.39]. For inter-reader reliability, where segmentations were independently performed by a second certified radiologist, the variability metrics were: Dice score 0.81 [0.78, 0.85]; MASD: 1.64 [1.20, 2.17], F1-score: 74% [68%, 80%].

## 3.3 Overall Segmentation Performance

The model achieved an average Dice score of 0.84 [0.79, 0.88] on the internal test set and 0.88 [0.84, 0.91] on the external test set (mean [95% confidence interval]). It also achieved F1-scores of 76% on the internal test set and 67% on the external test set. Table 2 details the performance of PCNSL segmentation in internal and external dataset using the three chosen evaluation metrics (Dice score, MASD, F1-score). Figure 3 displays the Dice score distribution of our model across the internal and external test sets, while the distributions of the MASD and F1-score are shown in Figure S4 and Figure S5. The Mann-Whitney U test showed the performance between the internal and external datasets were not statistically significant (P-value=0.59).

<table><caption>Table 2. Performance of Primary Central Nervous System Lymphoma Model Segmentation in Internal and External Test Set and Centers I-VII. [Note] The results are presented as means with 95% bootstrap CIs computed on the independent test set. MASD = mean average surface distance.</caption><thead><tr><th>Hospitals (Subjects)</th><th>Dice Score</th><th>MASD (mm)</th><th>F1-score (%)</th></tr></thead><tbody><tr><td>Internal (N=44)</td><td>0.84 (0.79, 0.88)</td><td>1.69 (0.94, 2.67)</td><td>76 (69, 83)</td></tr><tr><td>External (N=48)</td><td>0.87 (0.84, 0.90)</td><td>1.44 (0.90, 2.21)</td><td>67 (59, 74)</td></tr><tr><td>External-I (N=7)</td><td>0.85 (0.76, 0.92)</td><td>1.37 (0.76, 2.07)</td><td>62 (40, 80)</td></tr><tr><td>External-II (N=7)</td><td>0.88 (0.82, 0.92)</td><td>1.42 (0.88, 1.94)</td><td>53 (37, 72)</td></tr><tr><td>External-III (N=7)</td><td>0.75 (0.55, 0.89)</td><td>3.13 (0.40, 7.31)</td><td>71 (50, 90)</td></tr><tr><td>External-IV (N=7)</td><td>0.91 (0.89, 0.93)</td><td>1.33 (0.70, 2.17)</td><td>69 (54, 84)</td></tr><tr><td>External-V (N=7)</td><td>0.91 (0.86, 0.95)</td><td>0.48 (0.24, 0.72)</td><td>73 (57, 90)</td></tr><tr><td>External-VI (N=6)</td><td>0.91 (0.88, 0.93)</td><td>0.53 (0.42, 0.68)</td><td>66 (42, 90)</td></tr><tr><td>External-VII (N=7)</td><td>0.92 (0.89, 0.94)</td><td>1.69 (0.52, 3.44)</td><td>71 (56, 87)</td></tr></tbody></table>

## 3.4 Volumetric Assessment

Volumetric correlation between the automatic segmentation and the manual annotation had a correlation coefficient of r = 0.99 (P < 0.001) for the internal test set and r = 0.98 (P < 0.0001) for the external test set
<figure><img src="image_4.png" /><figcaption>Figure 3. Distribution of Dice score for the internal test set, the external test set, and the seven medical centers of the external test set. Each box is bounded by the lower quartile) and the upper quartile, with the center line representing the median. Whiskers extend from the box to the farthest data point within 1.5 times the IQR of the box.</figcaption></figure>

(Figure 4). After excluding patients with very large reference standard label tumor volumes (≥ 30 cm³), the correlation coefficients were r=0.98 (P < 0.0001) for the internal test set and r=0.94 (P < 0.0001) for the external test set (Figure S6). Bland-Altman analysis (Figure S3) further confirmed good agreement, with the model showing a slight systematic underestimation compared to manual segmentation.

## 3.5 Subgroup Analysis

Figure 5 illustrates the mean performance metrics of the model across different PCNSL tumor subtypes, as previously described. Table 3 details this performance in each center. The model performed best on homogeneous and discrete lesions (e.g., groups A and B), while performance slightly decreased for single heterogeneous lesions and periventricular pattern (group D), and markedly decreased in case of perivascular enhancement (group E).

We also re-evaluated model performance after excluding post-treatment or unknown treatment status cases from both the internal (n = 5) and external (n = 14) test sets (Details in Section Appendix. F). As shown in Table S2, Dice scores remained consistent (0.84 internal, 0.87 external), with no statistically significant difference compared to the full dataset (p = .81 and .82, respectively.

To further understand failure modes, we analyzed test cases with Dice scores below 0.60, as summarized in Table S4 (Details in Section Appendix. I). Poor image quality (e.g., strong motion artifacts), low lesion contrast, and under-segmentation of small (< 1cm) lesions, especially in the periventricular location were common error sources.

Figure 6 shows an example of expert annotation and automatic segmentation in a subject with movement artifacts.
<figure><img src="image_5.png" /><figcaption>Figure 4. Volumetric correlation between the manually annotated reference standard label and the automatic segmentation for total tumor volume per MRI scan was assessed using Pearson correlation coefficient (r) for both the internal and external test sets. The shaded area represents the 95% bootstrap CI of the fitted line.</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Figure 5. Automatic segmentation results in different primary central nervous system lymphoma (PCNSL) cases. (A) Single, homogeneous lesion in a 72-year-old male patient. (B) Multiple discrete homogeneous PCNSL lesions in a 67-year-old female patient. (C) Single heterogeneous lesion in a 73-year-old female patient. (D) Multiple ependymal enhancements in a 75-year-old male patient. (E) Multiple perivascular enhancements in a 64-year-old male patient. MASD = mean average surface distance, T1-GD = T1-weighted gadolinium-enhanced. The red areas represent the annotated lesions.</figcaption></figure>
<table><caption>Table 3. Performance of Primary Central Nervous System Lymphoma Model Segmentation in Internal and External Test Set Group by Tumor Type. [Note] Tumor types were as follows: unique, homogeneous lesion (group A); multiple discrete homogeneous lesions (group B); single heterogeneous lesions (group C); multiple heterogeneous lesions, including linear ependymal enhancement (group D); and linear perivascular enhancement (group E). The results are presented as means with 95% bootstrap CI computed on the independent test set.</caption><thead><tr><th>Hospitals</th><th>tumor (Subjects)</th><th>Dice Score</th><th>MASD (mm)</th><th>F1-score (%)</th></tr></thead><tbody><tr><td rowspan="6">Internal</td><td>(A) (N=11)</td><td>0.84 (0.75, 0.91)</td><td>0.93 (0.63, 1.36)</td><td>93 (86, 100)</td></tr><tr><td>(B) (N=11)</td><td>0.83 (0.71, 0.92)</td><td>2.95 (0.67, 6.63)</td><td>77 (64, 88)</td></tr><tr><td>(C) (N=14)</td><td>0.87 (0.83, 0.91)</td><td>0.90 (0.66, 1.14)</td><td>68 (56, 81)</td></tr><tr><td>(D) (N=4)</td><td>0.78 (0.63, 0.88)</td><td>3.01 (1.00, 6.35)</td><td>63 (48, 76)</td></tr><tr><td>(A+D) (N=3)</td><td>0.74 (0.36, 0.93)</td><td>2.02 (0.39, 5.08)</td><td>79 (36, 100)</td></tr><tr><td>(B+E) (N=1)</td><td>0.93 (0.93, 0.93)</td><td>0.89 (0.89, 0.89)</td><td>40 (40, 40)</td></tr><tr><td rowspan="4">External</td><td>(A) (N=12)</td><td>0.93 (0.91, 0.94)</td><td>0.92 (0.44, 1.59)</td><td>71 (54, 86)</td></tr><tr><td>(B) (N=12)</td><td>0.87 (0.81, 0.92)</td><td>1.39 (0.48, 2.62)</td><td>78 (69, 86)</td></tr><tr><td>(C) (N=21)</td><td>0.85 (0.78, 0.91)</td><td>1.78 (0.89, 3.26)</td><td>60 (49, 71)</td></tr><tr><td>(B+D) (N=3)</td><td>0.82 (0.75, 0.87)</td><td>1.36 (0.92, 2.16)</td><td>53 (44, 67)</td></tr></tbody></table>

<figure><img src="image_7.png" /><figcaption>Figure 6. Reference standard label versus automatic segmentation results for a primary central nervous system lymphoma visible on a gadolinium T1-weighted contrast-enhanced MRI sequence with movement artifacts in a 67-year-old female patient. The main prediction error regions are marked with red circles. MASD = mean average surface distance.</figcaption></figure>

## 4. DISCUSSION

We trained and extensively evaluated an automated segmentation model for PCNSL at clinical routine gadolinium-enhanced T1-weighted MRI, demonstrating its accuracy and robustness. Our model, developed at a single medical center and internally and externally tested, was highly accurate (Dice scores of 0.84 and 0.88 on internal and external test sets, respectively).

Of note, our model is publicly available to encourage its widespread use. To the best of our knowledge no other open-source model with external testing exists for this primary brain tumor. There is only one published paper on automated segmentation in PCNSL.¹³ In contrast to our model, which was trained on pathologically proven PCNSL, the deep-learning model was trained on MRI of subjects with glioblastoma. The experiments conducted on our population do not support the translation of nnU-Net models trained on glioma data to PCNSL (Dice scores of 0.14 and 0.07, in the internal and external population, respectively, and MASD values exceeding 40 mm). Instead, they advocate for disease-specific models to achieve accurate tumor segmentation. High-grade gliomas and PCNSL can have similar MRI findings, but they usually differ. Grade 4 gliomas are typically very necrotic and highly heterogeneous, while lymphomas display more uniform, homogeneous contrast enhancement. In addition, brain lymphomas often have multiple distant locations, unlike gliomas, and frequently
display periventricular and ependymal enhancement, which is rarer at the time of glioma diagnosis. Furthermore,
the performance of the proposed deep learning model for PCNSL segmentation is lower than that of our model,
with an average Dice score of 0.76, and it did not include external validation.¹³ In contrast, one of the main
strengths of our model is its ability to generalize well in real-world clinical settings, achieving similar Dice scores
between the internal and external test sets. Among the seven external centers, the performance was generally
consistent, except for Center III, that showed a lower but acceptable Dice score (0.75).

Another strength of our study is that we detailed the F1-score, according to Metrics Reloaded guidelines.²¹
The model achieved F1-scores of 76% on the internal test set and 67% on the external test set, indicating good
lesion detection performance. Since previous studies did not document these detection measures, a direct com-
parison with our results is not possible. F1-score is an infrequently reported accuracy metric in machine learning
models. However, this score is important, especially in real-world dataset, as it comprehensively computes the
number of times a model makes a correct prediction on the dataset by balancing precision and recall.

In terms of reliability of manual annotations, both intra-reader and inter-reader Dice scores exceeded 0.80,
indicating a high level of segmentation consistency. The lower MASD in intra-reader comparisons (1.05 mm)
suggests that the same annotator achieved greater spatial precision, while the slight decrease in inter-reader
agreement (1.64 mm) reflects expected variability between different radiologists.

In terms of the research dataset, although we did not conduct a formal power analysis, our cohort of 135
patients represents the largest PCNSL segmentation dataset to date, surpassing prior studies such as Pennig
et al.¹³ (43 patients) and Zhang et al.²² (92 patients). Given the rarity of PCNSL, this dataset constitutes a
substantial, clinically representative multicentric collection. The strong and consistent performance of our model
on both internal and external test sets supports the adequacy of this sample size for training and evaluation.

A strong volumetric correlation between automatic segmentation and manual annotation was also observed (Figure 4). These findings indicate that our model offers a consistent approach to volumetric assessment, potentially reducing current interobserver variability in clinical settings. Although PCNSL response assessment is often straightforward, interobserver variability in central review remains a challenge.²³ A recent study conducted on PCNSL MRI also suggest that standardized volumetric measurements could improve tumor monitoring and enable earlier identification of responders versus nonresponders.²⁴ The integration of automated segmentation tools with longitudinal imaging aims to refine response assessment protocols beyond conventional sum-product methods, especially for cases with multiple, irregularly shaped lesions. The robustness and reliability of our model in clinical practice provide a strong foundation for fully automated PCNSL MRI evaluation.

To better understand our model's performance, we visually categorized our dataset into five representative
PCNSL patterns at gadolinium-enhanced T1-weighted MRI (Figure 5), and we analyzed test cases with low Dice
scores (Section. Appendix. I). The model performed best on homogeneous and discrete lesions (e.g., subtypes
A and B). Its performance decreased for periventricular and perivascular patterns, which often involve multiple,
poorly defined, and highly heterogeneous infracentimetric lesions. These findings indicate that while the model
is reliable, its performance may be limited under these specific conditions. Additionally, since our dataset lacked
cases with purely meningeal spread, our model was neither trained nor validated for this pattern, limiting its
applicability in such cases. Future work should address this limitation by incorporating a broader dataset with
more representation of these rare patterns.

It is worth nothing that, in daily practice, leptomeningeal and perivascular enhancement are also challenging
for clinicians, as defining their boundaries is difficult and dependent on acquisition parameters. Interleukin 10
quantification in cerebrospinal fluid is very useful in these complex imaging scenarios.²²,²⁵

Our study had several limitations. First, despite being the largest PCNSL segmentation dataset to date,
our sample size was limited due to the rarity of the disease, potentially limiting the model's ability to learn
uncommon imaging patterns, underscoring the need for larger studies to validate our model. The hope is
that our open-source link will encourage further research and refinement of the model's performance. Second,
the absence of T2-weighted FLAIR sequences in the segmentation framework limits our model. T2-weighted
FLAIR hyperintense lesions are not specific, especially in older patients with concomitant vascular leukopathy
and/or methotrexate-related leukoencephalopathy, and evidence on its evaluation in brain lymphoma are scarce.
However, we recognize that an accurate quantification of non-enhancing tumor burden is advisable and we plan to
expand our dataset with this sequence. Third, our model is intended for PCNSL segmentation at different stages of diagnosis and treatment, it was primarily trained and evaluated on pre-treatment MRI scans. The relatively small number of post-treatment cases limits our ability to evaluate its performance for follow-up evaluations. Expanding the dataset with more post-treatment MRIs will help assess its robustness in long-term monitoring.

In summary, we constructed an automatic PCNSL segmentation model using post-contrast T1-weighted MRI data from clinical routine and tested it on data from seven other medical centers. Our results concurrently agreed that the proposed deep-learning method was accurate and generalizable. We achieved good performance comparable to manual annotation in both the internal and external test sets. This publicly available segmentation model can serve as a valuable analytical tool for clinical research on primary central nervous system lymphomas. Beyond tumor burden quantification, this tool can also support clinical decision-making by integrating radiomic features with survival outcomes, thereby aiding in the prediction of therapeutic resistance and disease progression. The model's ability to evaluate volumetric changes over time could enhance response assessment in clinical trials and improve patient stratification for treatment. Further research will explore how integrating molecular and genetic tumor profiles with automated MRI analysis can refine prognostic and predictive models.

## 5. ACKNOWLEDGMENTS

We acknowledge funding from the French government under management of Agence Nationale de la Recherche as part of the "Investissements d'avenir" program, reference ANR-19-P3IA-0001 (PRAIRIE 3IA Institute) and reference ANR-10-IAIHU-06 (Agence Nationale de la Recherche-10-IA Institut Hospitalo-Universitaire-6), from the European Union's Horizon Europe Framework Programme (grant number 101136607, project CLARA) and from the Fonds Recherche Neurosciences under the project "NEIMO - Monitoring Neuroinflammation: a Paris Brain Institute-Yale Neurology Collaborative program". G.F. is supported by the Chinese Government Scholarship provided by China Scholarship Council (CSC). L.N. is supported by an Inria/AP-HP "Poste d'accueil.

**On Behalf of the French Network for Oculocerebral Lymphoma (LOC Network)** Caroline Houillier<fnref n="15" />, Sylvain Choquet<fnref n="15" />, Khê Hoang-Xuan<fnref n="15" />, Valérie Touitou<fnref n="15" />, Carole Soussain<fnref n="16" />, Nathalie Cassoux<fnref n="16" />, Denis Malaise<fnref n="16" />, Renata Ursu<fnref n="17" />, Lise Willems<fnref n="18" />, Hervé Ghesquières<fnref n="19" />, Anna Schmitt<fnref n="20" />, Olivier Chinot<fnref n="21" />, Emeline Tabouret<fnref n="21" />, Luc Taillandier<fnref n="22" />, Marie Blonski<fnref n="22" />, Roch Houot<fnref n="23" />, Guido Ahle<fnref n="24" />, Gandhi Damaj<fnref n="25" />, Cécile Moluçon-Chabrot<fnref n="26" />, Vincent Delwail<fnref n="27" />, Michel Fabbro<fnref n="28" />, Fabrice Jardin<fnref n="29" />, Adrien Chauchet<fnref n="30" />, Franck Morschhauser<fnref n="31" />, Olivier Casasnovas<fnref n="32" />, Rémy Gressin³³, Luc-Matthieu Fornecker³⁴, Julie Abraham³⁵, Jean-Pierre Marolleau³⁶, Adrian Tempescul³⁷, Philippe Agapé³⁸, Lucie Oberic³⁹, Emmanuel Gyan⁴⁰, Frédéric Peyrade⁴¹, Thomas Gastinne⁴².

<footnote n="15">¹⁵CHU Pitié Salpêtrière, Paris, France</footnote>

<footnote n="16">¹⁶Institut Curie, Saint Cloud, France</footnote>

<footnote n="17">¹⁷Hôpital Saint-Louis, Paris, France</footnote>

<footnote n="18">¹⁸CHU Paris Centre Site Cochin, Paris, France</footnote>

<footnote n="19">¹⁹CHU Lyon Sud, Lyon, France</footnote>

<footnote n="20">²⁰Institut Bergonié, Bordeaux, France</footnote>

<footnote n="21">²¹CHU de la Timone, Marseille, France</footnote>

<footnote n="22">²²CHU de Nancy, Nancy, France</footnote>

<footnote n="23">²³CHU de Rennes, Rennes, France</footnote>

<footnote n="24">²⁴Hôpitaux Civils, Colmar, France</footnote>

<footnote n="25">²⁵CHU de Caen, Caen, France</footnote>

<footnote n="26">²⁶CHU de Clermont Ferrand, Clermont-Ferrand, France</footnote>

<footnote n="27">²⁷CHU de Poitiers, Poitiers, France</footnote>

<footnote n="28">²⁸Institut du Cancer de Montpellier Val d'Aurelle, Montpellier, France</footnote>

<footnote n="29">²⁹Centre Henri Becquerel, Rouen, France</footnote>

<footnote n="30">³⁰CHU de Besançon, Besançon, France</footnote>

<footnote n="31">³¹CHRU de Lille, Lille, France</footnote>

<footnote n="32">³²CHU de Dijon, Dijon, France</footnote>
<sup>33</sup>CHU de Grenoble, Grenoble, France

<sup>34</sup>CHU de Strasbourg, Strasbourg, France

<sup>35</sup>CHU de Limoges, Limoges, France

<sup>36</sup>CHU d'Amiens, Amiens, France

<sup>37</sup>CHU de Brest, Brest, France

<sup>38</sup>Institut de Cancérologie de l'Ouest, Saint Herblain, France

<sup>39</sup>Institut Universitaire du Cancer de Toulouse, Toulouse, France

<sup>40</sup>CHU Tours, Tours, France

<sup>41</sup>Centre Antoine Lacassagne, Nice, France

<sup>42</sup>CHU de Nantes, Nantes, France

# 6. DISCLOSURE STATEMENT

Competing financial interests related to the present article: none to disclose for all authors. Competing financial interests unrelated to the present article: OC reports having received consulting fees from Therapanacea. OC reports that other principal investigators affiliated to the team which he co-leads have received grants (paid to the institution) from Sanofi and Biogen. OC reports that his spouse was an employee of myBrainTechnologies and is an employee of DiamPark. OC holds a patent registered at the International Bureau of the World Intellectual Property Organization (PCT/IB2016/0526993, Allassonniere S, Colliot O, Durrleman S, Schiratti J-B, A method for determining the temporal progression of a biological phenomenon and associated methods and devices) (2017). OC has submitted a declaration of invention to the European Patent Office (File Number: EP23306632, Ayache N, Colliot O, Hamzaoui M, Soulier T, Stankoff B, Devices for generation of synthetic 3D representation representative of myelin content, Submitted to the European Patent Office on September 28th, 2023).

# REFERENCES

[1] Shiels, M. S., Pfeiffer, R. M., Besson, C., Clarke, C. A., Morton, L. M., Nogueira, L., Pawlish, K., Yanik, E. L., Suneja, G., and Engels, E. A., "Trends in primary central nervous system lymphoma incidence and survival in the US," *British journal of haematology* **174**(3), 417-424 (2016).

[2] Houillier, C., Soussain, C., Ghesquières, H., Soubeyran, P., Chinot, O., Taillandier, L., Lamy, T., Choquet, S., Ahle, G., Damaj, G., et al., "Management and outcome of primary CNS lymphoma in the modern era: an LOC network study," *Neurology* **94**(10), e1027-e1039 (2020).

[3] Ferreri, A. J., Calimeri, T., Cwynarski, K., Dietrich, J., Grommes, C., Hoang-Xuan, K., Hu, L. S., Illerhaus, G., Nayak, L., Ponzoni, M., et al., "Primary central nervous system lymphoma," *Nature Reviews Disease Primers* **9**(1), 29 (2023).

[4] Hernández-Verdin, I., Kirasic, E., Wienand, K., Mokhtari, K., Eimer, S., Loiseau, H., Rousseau, A., Paillassa, J., Ahle, G., Lerintiu, F., et al., "Molecular and clinical diversity in primary central nervous system lymphoma," *Annals of Oncology* **34**(2), 186-199 (2023).

[5] Hernández-Verdin, I., Morales-Martínez, A., Hoang-Xuan, K., and Alentorn, A., "Primary central nervous system lymphoma: advances in its pathogenesis, molecular markers and targeted therapies," *Current Opinion in Neurology* **35**(6), 779-786 (2022).

[6] Choquet, S., Soussain, C., Azar, N., Morel, V., Metz, C., Ursu, R., Waultier-Rascalou, A., Di Blasi, R., Houot, R., Souchet, L., et al., "CAR T-cell therapy induces a high rate of prolonged remission in relapsed primary CNS lymphoma: Real-life results of the LOC network," *American Journal of Hematology* (2024).

[7] von Roemeling, C., Ferreri, A. J., Soussain, C., Tun, H. W., and Grommes, C., "Targets and treatments in primary CNS lymphoma," *Leukemia & Lymphoma*, 1-13 (2024).

[8] Hoang-Xuan, K., Deckert, M., Ferreri, A. J., Furtner, J., Gallego Perez-Larraya, J., Henriksson, R., Hottinger, A. F., Kasenda, B., Lefranc, F., Lossos, A., et al., "European association of neuro-oncology (EANO) guidelines for treatment of primary central nervous system lymphoma (PCNSL)," *Neuro-oncology* **25**(1), 37-53 (2023).
[9] Barajas Jr, R. F., Politi, L. S., Anzalone, N., Schöder, H., Fox, C. P., Boxerman, J. L., Kaufmann, T. J., Quarles, C. C., Ellingson, B. M., Auer, D., et al., “Consensus recommendations for MRI and PET imaging of primary central nervous system lymphoma: guideline statement from the international primary CNS lymphoma collaborative group (IPCG),” *Neuro-oncology* **23**(7), 1056–1071 (2021).

[10] Ghaffari, M., Sowmya, A., and Oliver, R., “Automated brain tumor segmentation using multimodal brain scans: a survey based on models submitted to the BraTS 2012–2018 challenges,” *IEEE reviews in biomedical engineering* **13**, 156–168 (2019).

[11] van Kempen, E. J., Post, M., Mannil, M., Witkam, R. L., Ter Laan, M., Patel, A., Meijer, F. J., and Henssen, D., “Performance of machine learning algorithms for glioma segmentation of brain MRI: a systematic literature review and meta-analysis,” *European Radiology* **31**(12), 9638–9653 (2021).

[12] Zhang, Y., Liang, K., He, J., Ma, H., Chen, H., Zheng, F., Zhang, L., Wang, X., Ma, X., and Chen, X., “Deep learning with data enhancement for the differentiation of solitary and multiple cerebral glioblastoma, lymphoma, and tumefactive demyelinating lesion,” *Frontiers in Oncology* **11**, 665891 (2021).

[13] Pennig, L., Hoyer, U. C. I., Goertz, L., Shahzad, R., Persigehl, T., Thiele, F., Perkuhn, M., Ruge, M. I., Kabbasch, C., Borggrefe, J., et al., “Primary central nervous system lymphoma: clinical evaluation of automated segmentation on multiparametric MRI using deep learning,” *Journal of Magnetic Resonance Imaging* **53**(1), 259–268 (2021).

[14] Kamnitsas, K., Ferrante, E., Parisot, S., Ledig, C., Nori, A. V., Criminisi, A., Rueckert, D., and Glocker, B., “Deepmedic for brain tumor segmentation,” in [*Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: Second International Workshop, BrainLes 2016, with the Challenges on BRATS, ISLES and mTOP 2016, Held in Conjunction with MICCAI 2016, Athens, Greece, October 17, 2016, Revised Selected Papers* 2], 138–149, Springer (2016).

[15] Haldorsen, I., Kråkenes, J., Krossnes, B., Mella, O., and Espeland, A., “CT and MR imaging features of primary central nervous system lymphoma in norway, 1989–2003,” *American journal of neuroradiology* **30**(4), 744–751 (2009).

[16] Xing, Z., You, R., Li, J., Liu, Y., and Cao, D., “Differentiation of primary central nervous system lymphomas from high-grade gliomas by rCBV and percentage of signal intensity recovery derived from dynamic susceptibility-weighted contrast-enhanced perfusion MR imaging,” *Clinical neuroradiology* **24**, 329–336 (2014).

[17] Isensee, F., Jaeger, P. F., Kohl, S. A., Petersen, J., and Maier-Hein, K. H., “nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation,” *Nature methods* **18**(2), 203–211 (2021).

[18] Li, X., Morgan, P. S., Ashburner, J., Smith, J., and Rorden, C., “The first step for neuroimaging data analysis: Dicom to nifti conversion,” *Journal of neuroscience methods* **264**, 47–56 (2016).

[19] Pérez-García, F., Sparks, R., and Ourselin, S., “TorchIO: a python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning,” *Computer Methods and Programs in Biomedicine* **208**, 106236 (2021).

[20] Antonelli, M., Reinke, A., Bakas, S., Farahani, K., Kopp-Schneider, A., Landman, B. A., Litjens, G., Menze, B., Ronneberger, O., Summers, R. M., et al., “The medical segmentation decathlon,” *Nature communications* **13**(1), 4128 (2022).

[21] Maier-Hein, L., Reinke, A., Godau, P., Tizabi, M. D., Buettner, F., Christodoulou, E., Glocker, B., Isensee, F., Kleesiek, J., Kozubek, M., et al., “Metrics reloaded: recommendations for image analysis validation,” *Nature methods*, 1–18 (2024).

[22] Zhang, Y., Zou, D., Yin, J., Zhang, L., Zhang, X., Wang, W., Zhang, M., Zhou, D., and Zhang, W., “Changes in cerebrospinal fluid interleukin-10 levels display better performance in predicting disease relapse than conventional magnetic resonance imaging in primary central nervous system lymphoma,” *BMC cancer* **21**, 1–7 (2021).

[23] Van Der Meulen, M., Postma, A. A., Smits, M., Bakunina, K., Minnema, M. C., Seute, T., Cull, G., Enting, R. H., Van Der Poel, M., Stevens, W. B., et al., “Extent of radiological response does not reflect survival in primary central nervous system lymphoma,” *Neuro-oncology advances* **3**(1), vdab007 (2021).

[24] Lauer, E. M., Riegler, E., Mutter, J. A., Alig, S. K., Bleul, S., Kuehn, J., Ranganathan, L., Klingler, C., Demerath, T., Würtemberger, U., et al., “Improved early outcome prediction by MRI-based 3d tumor volume assessment in patients with cns lymphomas,” *Neuro-oncology* **26**(2), 374–386 (2024).
[25] Herzi, D., Le Garff-Tavernier, M., Sourdeau, E., Choquet, S., Soussain, C., Nichelli, L., Mathon, B., Mokhtari, K., Laurenge, A., Alentorn, A., et al., “Prognostic value of CSF IL-10 at early assessment of induction chemotherapy in primary CNS lymphomas: A LOC network study,” *Neurology* **102**(12), e209527 (2024).

[26] Simpson, A. L., Antonelli, M., Bakas, S., Bilello, M., Farahani, K., Van Ginneken, B., Kopp-Schneider, A., Landman, B. A., Litjens, G., Menze, B., et al., “A large annotated medical image dataset for the development and evaluation of segmentation algorithms,” *arXiv preprint arXiv:1902.09063* (2019).
# Appendix

In the appendix, we provide detailed information about our dataset, including the flowchart of data selection (Section Appendix. A), the Karnofsky scores of patients (Section Appendix. B) and the parameters of the MRI scanner across different medical centers (Section Appendix. C). Additionally, we conducted a Bland-Altman analysis to compare manual annotations with automatic segmentation, with details provided in Section Appendix. D. To better understand model performance, we included the distributions of the MASD and F1-score in Section Appendix. E. Section Appendix. F reports a sensitivity analysis that excludes post-treatment and unknown-treatment-status cases. Section Appendix. G shows volumetric correlation results after removing outliers with extremely large tumor volumes, indicating the model's reliability across typical cases. Section Appendix. H compares the lymphoma-trained model with a glioma-trained model using the same nnU-Net framework. Finally we did error analysis to the low performing cases in Section Appendix. I.

## Appendix. A Data Selection Flowchart

To clarify the patient selection process, we have added a flowchart in Fig S1 below, which outlines the inclusion and exclusion criteria.

<figure><img src="image_8.png" /><figcaption>Figure S1. Flowchart of data selection from the database, illustrating the inclusion and exclusion criteria.</figcaption></figure>

## Appendix. B Karnofsky Score of the Research Dataset

The Karnofsky Performance Status (KPS) is a standard tool used to assess a patient's overall health and functional status, particularly in the context of cancer and chronic diseases such as brain tumors. The KPS score ranges from 0 to 100, with higher scores indicating better functional status. We show the distribution of the internal and external data in Figure S2.
<figure><img src="image_9.png" /><figcaption>Figure S2. The Karnofsky score of the internal and external dataset, NaN represents no record.</figcaption></figure>

## Appendix. C Details of MRI Acquisition

To enhance transparency and allow readers to understand the distribution of MRI acquisition parameters across centers, we provide specific details about these parameters for each center as summarized in Table S1.

<table><caption>Table S1. MRI acquisition parameters</caption><thead><tr><th>Dataset</th><th>Pulse Sequences</th><th>Spatial Resolution</th><th>Magnetic Field Strength</th><th>Manufacturer</th></tr></thead><tbody><tr><td>Internal Train</td><td>Gradient echo: 27; Spin echo: 16</td><td>Isotropic: 31; Non-isotropic: 12</td><td>1T: 3; 1.5T: 17; 3T: 23</td><td>GE: 40; Philips: 3</td></tr><tr><td>Internal Test</td><td>Gradient echo: 25; Spin echo: 18</td><td>Isotropic: 37; Non-isotropic: 7</td><td>1T: 3; 1.5T: 18; 3T: 23</td><td>GE: 38; Siemens: 3; Philips: 3</td></tr><tr><td>External (I)</td><td>Gradient echo: 7; Spin echo: 0</td><td>Isotropic: 5; Non-isotropic: 2</td><td>1.5T: 4; 3T: 3</td><td>GE: 7</td></tr><tr><td>External (II)</td><td>Gradient echo: 5; Spin echo: 2</td><td>Isotropic: 5; Non-isotropic: 2</td><td>1.5T: 4; 3T: 3</td><td>GE: 7</td></tr><tr><td>External (III)</td><td>Gradient echo: 4; Spin echo: 3</td><td>Isotropic: 1; Non-isotropic: 6</td><td>1.5T: 7</td><td>GE: 6; Philips: 1</td></tr><tr><td>External (IV)</td><td>Gradient echo: 7; Spin echo: 0</td><td>Isotropic: 7</td><td>1.5T: 4; 3T: 3</td><td>GE: 2; Siemens: 3; Philips: 2</td></tr><tr><td>External (V)</td><td>Gradient echo: 4; Spin echo: 3</td><td>Isotropic: 5; Non-isotropic: 2</td><td>1.5T: 7</td><td>GE: 1; Siemens: 1; Philips: 5</td></tr><tr><td>External (VI)</td><td>Gradient echo: 6</td><td>Isotropic: 6</td><td>1.5T: 6</td><td>Siemens: 4; Philips: 1; Toshiba: 1</td></tr><tr><td>External (VII)</td><td>Gradient echo: 4; Spin echo: 3</td><td>Isotropic: 6; Non-isotropic: 1</td><td>1.5T: 7</td><td>Siemens: 6; Philips: 1</td></tr></tbody></table>

## Appendix. D Bland-Altman Analysis of Manual Annotations and Automatic Segmentation

We have now performed this analysis and included the Bland-Altman plots in Figure S3. For the internal dataset, the results show a mean bias of 1.64 *cm*³, with 95% limits of agreement ranging from -4.44 *cm*³ to 7.72 *cm*³. This bias represents approximately 9.5% of the mean ground truth tumor volume (17.18 *cm*³). For the external dataset, the mean bias is 1.40 *cm*³, with 95% limits of agreement from -5.11 *cm*³ to 7.90 *cm*³, which corresponds to around 10.7% of the mean ground truth tumor volume (13.07 *cm*³). These results indicate that our model systematically produces slightly lower volumetric estimates compared to manual segmentation. This highlights the general consistency of our model's segmentation performance across datasets, with minimal systematic bias.

## Appendix. E Performance Distribution on Other Evaluation Metrics

Figure S4 and S5 display the MASD and F1-score distribution for the internal and external test set.
<figure><img src="image_10.png" /><figcaption>Figure S3. The Bland-Altman analysis comparing the manually annotated ground truth and the automatic segmentation for total tumor volume per scan in both the internal and external test sets. The red dashed line represents the mean bias, while the blue dashed lines indicate the 95% limits of agreement (LoA). Each point corresponds to an individual scan, with the x-axis representing the mean volume of the two methods and the y-axis showing the volume difference (manual - automatic).</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>Figure S4. Distribution of MASD for the internal test set, the external test set and the seven medical centers of the external test set. Every box is bounded by the lower quartile (Q1) and the upper quartile (Q3), with the center line representing the median. Whiskers extend from the box to the furthest data point within 1.5x the interquartile range of the box.</figcaption></figure>
<figure><img src="image_12.png" /><figcaption>Figure S5. Distribution of F1-score for the internal test set, the external test set and the seven medical centers of the external test set. Every box is bounded by the lower quartile (Q1) and the upper quartile (Q3), with the center line representing the median. Whiskers extend from the box to the furthest data point within 1.5x the interquartile range of the box.</figcaption></figure>

## Appendix. F Impact of Excluding Post-treatment Cases on Model Performance

We have reanalyzed our model performance after excluding post-treatment and unknown treatment status cases from both the internal (n=5) and external (n=14) test sets. The updated results are presented in Table S2. The model achieved a Dice score of 83.88 [78.77, 88.20] on the internal test set and 87.47 [82.48, 91.03] on the external test set (in %, mean [95% bootstrap CI]). A Mann-Whitney U test comparing these results with the original full dataset (Table 2) yielded p = 0.8053 for the internal test set and p = 0.8249 for the external test set, indicating no statistically significant difference after excluding post-treatment and unknown treatment cases. These findings confirm that the inclusion of post-treatment or unknown treatment cases did not significantly impact segmentation performance.

<table><caption>Table S2. Performance of the PCNSL model segmentation on the internal and external test sets, both with and without excluding cases of post-treatment or unknown treatment status. The results are presented as mean with 95% bootstrap confidence interval computed on the independent test set. N denotes the number of patients.</caption><thead><tr><td>Hospitals (Subjects)</td><td>Dice Score</td><td>MASD (mm)</td><td>F1-score (%)</td></tr></thead><tbody><tr><td>Internal (All, N=44)</td><td>0.84 (0.79, 0.88)</td><td>1.69 (0.94, 2.67)</td><td>76 (69, 83)</td></tr><tr><td>External (All, N=48)</td><td>0.87 (0.84, 0.90)</td><td>1.44 (0.90, 2.21)</td><td>67 (59, 74)</td></tr><tr><td>Internal (Exclude, N=39)</td><td>0.84 (0.79, 0.88)</td><td>1.63 (0.90, 2.70)</td><td>77 (70, 83)</td></tr><tr><td>External (Exclude, N=34)</td><td>0.87 (0.82, 0.91)</td><td>1.62 (0.92, 2.63)</td><td>61 (53, 70)</td></tr></tbody></table>
# Appendix. G Volumetric Correlation after Removing Outlier

Figure S6 displays the volumetric correlation between the automatic segmentation and the manual annotation after removing outlier (volume > 30cm³). The correlation coefficients were r = 0.98 (P < 0.0001) for the internal test set and r = 0.94 (P < 0.0001) for the external test set.

<figure><img src="image_13.png" /><figcaption>Figure S6. The volumetric correlation between the manually annotated ground truth and the automatic segmentation for total tumor volume per scan was assessed using Pearson correlation (r) for both the internal and external test sets after removing outlier (volume ≥ 30cm³). The shaded area represents the 95% bootstrap confidence interval of the fitted line.</figcaption></figure>

# Appendix. H Inference on the Lymphoma Dataset Using the Trained Glioma Model

We aimed to assess whether a model trained on gliomas could be generalized to PCNSL. To that purpose, we trained a model from the MSD-BraTS public dataset from the Medical Segmentation Decathlon challenge.²⁰ BraTS consists of MRI scans from patients with glioblastoma or lower-grade glioma, with manual segmentations provided by expert raters.²⁶ The dataset has undergone uniform pre-processing, including co-registration to a common template, standardization to 1 mm³ resolution, and skull stripping. To make the task as comparable as possible to lymphoma segmentation, we selected T1-weighted MRI after gadolinium injection and focused on the enhancing tumor as the target region. To maintain a similar sample size across the two tasks, we randomly selected 113 patients for our experiments*. These were then randomly split into a training/validation set with 58 patients and a test set comprising 55 patients. The test set was exclusively used for model evaluation. The glioma-trained model achieved good performances on the 55 patients with glioma with a mean Dice of 79.98% (confidence interval [74.42, 85.12]), indicating that the model was adequately trained.

To assess the generalization ability of the trained glioma model, we applied it directly to the lymphoma dataset without fine-tuning. The segmentation performance is reported in Table S3. The result shows that the glioma model fails to generalize to lymphoma segmentation. The lymphoma model achieves high performance, with Dice scores of 83.51% (internal) and 87.40% (external), along with low MASD values (shown in Table 2), indicating accurate boundary delineation. In contrast, the glioma model performs poorly, with Dice scores dropping to 14.25% (internal) and 7.03% (external), and a substantial increase in MASD, suggesting poor spatial alignment. These results highlight the differences in tumor morphology and imaging characteristics between gliomas and lymphomas, making direct model transfer ineffective. Thus, training a dedicated lymphoma model is necessary for accurate segmentation, as evidenced by its superior performance.

<footnote>*The lymphoma dataset includes 112 patients due to a preprocessing defect identified later.</footnote>
<table><caption>Table S3. The performance of a lymphoma-trained model and a glioma-trained model when applied to a lymphoma dataset, evaluated on both the internal and external test sets. The results are presented as mean with a 95% bootstrap confidence interval computed on the independent test set. N denotes the number of patients.</caption><thead><tr><td>Model-Hospitals (Subjects)</td><td>Dice Score</td><td>MASD (mm)</td><td>F1-score (%)</td></tr></thead><tbody><tr><td>Lymphoma-Internal (N=44)</td><td>0.84 (0.79, 0.88)</td><td>1.69 (0.94, 2.67)</td><td>76 (69, 83)</td></tr><tr><td>Lymphoma-External (N=48)</td><td>0.87 (0.84, 0.90)</td><td>1.44 (0.90, 2.21)</td><td>67 (59, 74)</td></tr><tr><td>Glioma-Internal (N=44)</td><td>0.14 (0.09, 0.21)</td><td>44.21 (38.86, 49.61)</td><td>13 (10, 17)</td></tr><tr><td>Glioma-External (N=48)</td><td>0.07 (0.03, 0.11)</td><td>69.08 (60.24, 78.47)</td><td>10 (7, 14)</td></tr></tbody></table>

## Appendix. I Error Analysis of Low-performing Cases

We analyzed cases with Dice scores below 60% to identify key segmentation failure patterns, and the error analysis can be seen in Table S4. The primary issues include poor lesion contrast and image quality, where low

<table><caption>Table S4. Analysis of cases with Dice scores below 60%, highlighting key sources of segmentation errors.</caption><thead><tr><th>Hospitals</th><th>Dice Score</th><th>MASD (mm)</th><th>F1-score (%)</th><th>Error: Reason</th></tr></thead><tbody><tr><td rowspan="4">Internal</td><td>0.36</td><td>5.08</td><td>36</td><td>Incorrect lesion delineation: bad image quality (low contrast-to-noise ratio).</td></tr><tr><td>0.40</td><td>17.89</td><td>33</td><td>1) Unrelated lymphoma lesions: vestibular schwannoma.<br>2) Missed lesions: periventricular pattern.</td></tr><tr><td>0.44</td><td>2.89</td><td>100</td><td>Incorrect lesion delineation: bad image quality (severe movement artifacts).</td></tr><tr><td>0.56</td><td>7.92</td><td>40</td><td>Missed lesions: periventricular pattern.</td></tr><tr><td rowspan="3">External</td><td>0.00</td><td>1.33</td><td>67</td><td>Incorrect alignment between ground truth mask and prediction mask: technical issue.</td></tr><tr><td>0.23</td><td>14.97</td><td>17</td><td>Incorrect lesion delineation: multiple infracentimetric lesions.</td></tr><tr><td>0.59</td><td>4.65</td><td>80</td><td>Missed lesions: periventricular pattern.</td></tr></tbody></table>

contrast-to-noise ratio and severe motion artifacts impaired delineation. Missed or misclassified lesions were also observed, particularly periventricular lesions, which were frequently under-segmented, and a case of vestibular schwannoma misidentified as lymphoma. Technical errors such as ground truth misalignment and challenges in detecting multiple infracentimetric lesions further contributed to low performance. Addressing these issues may involve improving preprocessing techniques, lesion detection strategies, and model robustness to image artifacts.

While increasing dataset size generally improves model generalizability, obtaining periventricular and multiple infracentimetric lesions remains challenging, particularly for PCNSL, a rare disease with limited cases. Additionally, the poorly defined lesion boundaries in some cases make manual annotation difficult and subject to interobserver variability. Future efforts will focus on improving lesion detection in complex regions and enhancing delineation of ambiguous margins. Integrating T2-weighted FLAIR sequences may better visualize tumor infiltration, while standardized annotation protocols could reduce variability, improving segmentation reliability in PCNSL management.