# Evaluation of a deep learning segmentation tool to help detect spinal cord lesions from combined T2 and STIR acquisitions in people with multiple sclerosis

Baptiste Lode, Burhan Rashid Hussein, Cédric Meurée, Ricky Walsh, Malo Gaubert, Nicolas Lassalle, Guilhem Courbon, Agathe Martin, Jeanne Le Bars, Françoise Durand-Dubief, et al.

► To cite this version:

Baptiste Lode, Burhan Rashid Hussein, Cédric Meurée, Ricky Walsh, Malo Gaubert, et al. Evaluation of a deep learning segmentation tool to help detect spinal cord lesions from combined T2 and STIR acquisitions in people with multiple sclerosis. European Radiology, 2025, 35 (10), pp.5954-5964. 10.1007/s00330-025-11541-0. hal-05079968

HAL Id: hal-05079968

https://hal.science/hal-05079968v1

Submitted on 22 May 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

<figure><img src="image_3.png" /></figure>

Check for
updates

# Evaluation of a deep learning segmentation tool to help detect spinal cord lesions from combined T2 and STIR acquisitions in people with multiple sclerosis

Baptiste Lodé¹,², Burhan Rashid Hussein², Cédric Meurée², Ricky Walsh², Malo Gaubert², Nicolas Lassalle³, Guilhem Courbon⁴, Agathe Martin⁴, Jeanne Le Bars⁴, Françoise Durand-Dubief⁵,⁶, Bertrand Bourre⁷, Adil Maarouf⁸, Olivier Outteryck⁹, Clément Mehier³, Alexandre Poulin³, Camille Cathelineau³, Jeremy Hong³, Guillaume Criton¹⁰,¹¹, Sophie Motillon-Alonso³, Augustin Lecler¹²,¹³, Frédérique Charbonneau¹², Loïc Duron¹², Alexandre Bani-Sadr¹¹,¹⁴, Céline Delpierre¹⁵, Jean-Christophe Ferré³, Gilles Edan⁴,¹⁶, François Cotton¹¹,¹⁷, Romain Casey¹⁸, Francesca Galassi², Benoit Combès² and Anne Kerbrat²,⁴* ID

**Abstract**

**Objective** To develop a deep learning (DL) model for the detection of spinal cord (SC) multiple sclerosis (MS) lesions from both sagittal T2 and short tau inversion recovery (STIR) sequences and to investigate whether such a model could improve the performance of clinicians in detecting SC lesions.

**Materials and methods** A DL tool was developed based on SC sagittal T2 and STIR acquisitions from the imaging database of the French MS registry (OFSEP), including retrospective data from 40 different scanners. A multi-reader study based on retrospective data was performed between December 2023 and June 2024 to compare the performance of 20 clinicians in interpreting upper and lower SC acquisitions with and without the use of the tool. A ground truth was established by three experts. Sensitivity, precision, and inter-reader variability were evaluated.

**Results** We included 50 patients (39 females, median age: 41 years [range: 15–67]) with SC MRI acquired between February 2017 and December 2022. When reading with the tool, the clinicians’ mean sensitivity to detect SC lesions improved (from 74.3% [95% CI = 67.8–80.6%] to 79.2% [95% CI: 73.5–85.0%]; *p* < 0.0001), with no evidence of difference in the mean precision: (69.0% [95% CI: 62.8–75.2%] vs 70.1% [95% CI: 64.3–75.9%]; *p* = 0.08). Inter-reader variability in lesion detection was slightly improved with the tool (Light’s kappa = 0.55 vs 0.60), but without statistical difference (*p* = 0.056).

**Conclusion** The use of an automatic tool can help clinicians detect SC lesions in pwMS by increasing their sensitivity.

**Key Points**

**Question** No tool to help detect MS SC lesions is used in clinical practice despite their frequency and prognostic value.

**Findings** This DL-based tool led to improvement in clinicians’ sensitivity in detecting SC lesions from both sagittal T2 and STIR sequences, without decreasing precision.

<figure><img src="image_4.png" /></figure>

<footnote>Benoit Combès and Anne Kerbrat contributed equally to this work.</footnote>

<footnote>*Correspondence:</footnote>

<footnote>Anne Kerbrat
anne.kerbrat@chu-rennes.fr</footnote>

<footnote>Full list of author information is available at the end of the article</footnote>
**Clinical relevance** Our study indicated the potential of a DL-based tool to assist clinicians in the challenging task of detecting SC lesions in people with MS on a combination of sequences commonly acquired in clinical practice.

**Keywords** Spinal cord, MRI, Multiple sclerosis, Deep learning-based tool, STIR sequence

**Graphical Abstract**

## Evaluation of a deep learning segmentation tool to help detect spinal cord lesions from combined T2 and STIR acquisitions in people with multiple sclerosis

Can an automated tool help clinicians detect spinal cord lesions from T2 and STIR sequences in people with multiple sclerosis (MS)?

<figure><img src="image_5.png" /></figure>

50 patients
20 readers

Spinal
cord MRI

Multi-center

<figure><img src="image_6.png" /></figure>

The tool led to a significant improvement in clinicians' sensitivity in detecting spinal cord lesions. It has the potential to assist clinicians in the challenging task of detecting spinal cord lesions in people with MS.

Eur Radiol (2025) Lodé B, Hussein BR, Meurée C,;
DOI: 10.1007/s00330-025-11541-0

<figure><img src="image_7.png" /></figure>

EUROPEAN SOCIETY OF RADIOLOGY

European Radiology

**Introduction**

Spinal cord (SC) lesions are seen in up to 80% of people with multiple sclerosis (pwMS) [1], are included in diagnosis criteria [2], and have a strong prognosis value [3]. SC MRI is therefore performed very systematically during the initial assessment of the disease, although its interpretation is more challenging than that of brain MRI due to difficulties in acquiring images (e.g., small tissue size, pulsation artifacts). In view of the artifact risks, the international recommendations suggest acquiring at least two SC MRI sequences in pwMS from T2-weighted sequences, proton density-weighted sequences, or short tau inversion recovery (STIR) [4].

Deep-learning models have been widely applied to brain MRI analyses in pwMS [5, 6]. Some enable MS lesions to be segmented on a given brain MRI [7], others are dedicated to differentiating between MS lesions and non-specific white matter lesions [8], while others enable the detection of new MS lesions between two longitudinal brain MRI scans [9, 10]. The potential added value of similar tools for the analysis of SC MRI remains largely unexplored. Deep

learning (DL)-based tools that segment SC lesions have been developed in a research context and have been included in the open-source SC toolbox [11, 12], but they have not been evaluated as an aid to SC lesion detection for radiologists in a clinical context. A diagnostic tool to differentiate between tumoral and inflammatory SC lesions has recently been proposed [13]. At present, no tool to help detect SC lesions in pwMS is used in clinical practice by radiologists. Moreover, no tool combining the analysis of two SC sequences, as recommended in clinical routines, has been proposed and evaluated.

In this study, our main objective is to assess the impact of a segmentation tool as an aid to detect SC MS lesions from both T2 and STIR, on clinician's sensitivity and precision. Our secondary objectives were to evaluate its impact on interpretation time and inter-reader variability.

**Materials and methods**

We developed a DL tool to detect SC MS lesions, and we performed a multi-reader, multi-case retrospective study with a fully crossed design. The study was approved by the
<figure><img src="image_8.png" /><figcaption>**Fig. 1** Summary of the multi-reader study design</figcaption></figure>

local ethics advisory committee (no. 24.100). All data were extracted (de-identified) from the French MS registry in October 2023 (www.ofsep.org) and included clinical and imaging data from 38 expert MS centers collected during patients' routine follow-up visits [14, 15] (NCT02889965).

## Characteristics of the segmentation model

Details about the segmentation model are provided in the Supplemental Material. Briefly, it is based on a five-level U-net architecture [16] with one input channel for each of the two sequences of interest and one single output channel. The model was trained with a combined dice and cross-entropy loss function (deep supervision was applied) on an annotated dataset of 140 cases from 40 scanners and with a diverse range of lesion

presentations extracted from the OFSEP database. The model output post-processing was optimized on a dataset of 21 cases for the task assessed in our work (favoring a good sensitivity with the risk of deteriorated precision). The resulting model achieved a lesion-wise sensitivity of 0.89 and a precision of 0.64 on a test set of 40 cases.

## The multi-reader study design

The multi-reader study was conducted between December 2023 and June 2024 (Fig. 1). Twenty clinicians annotated cervical and thoracic SC sagittal T2 acquisitions from 50 pwMS, with the systematic help of sagittal STIR. A web-based annotation tool resembling a standard picture archiving and communication system was used to annotate lesions by clicking once on each lesion (Supplementary Fig. 1). Each reader underwent a standardized training protocol, including a 20-min presentation and a written tutorial. A timer recorded the time spent on each MRI volume. Each patient was analyzed twice by each reader, with and without the tool, in a randomized order across two sessions at least 15 days apart. At the end of the experiment, we asked each reader to complete a questionnaire concerning their potential expectations of a tool to assist in the detection of SC MS lesions in clinical practice. Each item was scored from 1 to 5, ranging from “completely disagree” to “completely agree”.

## Dataset

SC MRI acquisitions were extracted from the OFSEP database. Inclusion criteria were images acquired from 2017, with both sagittal T2 and STIR acquisitions available at both upper and lower SC levels, with at least another SC MRI timepoint available. Non-inclusion criteria were patients who were already included in the training dataset. Exclusion criteria were a pathologic condition of the SC other than MS and poor imaging quality. Initially, 55 patients were selected, five of whom were excluded due to poor imaging quality, resulting in a final cohort of 50 patients.

## Readers

Radiologists and neurologists with varying levels of expertise and who were not involved in the tool development process were recruited for the experiment. Senior neuroradiologists and neurologists were recruited at the meeting of the OFSEP imaging group and came from French MS expert centers. Junior radiologists and neurologists were residents from the same centers. The general radiologists had no particular specialization in the follow-up of MS patients.

## Ground truth

Ground truth segmentation was established independently by a neurologist with 10 years of experience in MS SC MRI
reading (A.K.) and a radiology resident (B.L.) using ITK-SNAP software (version 4.0.1). They had more data at their disposal than the readers (SC MRI data on at least one other time-point and axial acquisitions). Discrepancies between them were resolved through adjudication with a third expert, a neuroradiologist with six years of experience (N.L.).

Secondarily, the 20 readers' annotations were grouped together on each MRI volume and were independently reviewed by B.L. and A.K. to construct a revised ground truth to potentially reintegrate lesions that would have been missed in the initial ground truth. We considered that a reader had correctly identified a lesion if he had clicked on any voxel within the ground truth lesion mask.

## Statistical analysis

Statistical analyses were performed using R (version 4.2.2).

## Primary outcome

to assess a difference in mean readers' sensitivity and/or precision to detect SC lesions with or without the tool. Prior to the study, we established through simulation that, under simplifying assumptions and for a Type I error of 0.05, a Type II error of 0.20, and an improvement in sensitivity of 10% with the tool (based on an average performance of experts alone of 0.60), a set of 50 patients with at least seven experts was sufficient. The individual mean sensitivity together with its 95% confidence interval (CI) was computed for each condition, with or without the tool. The overall mean sensitivity was tested for equality between the two conditions using a regression including a condition, acquisition, and a reader-fixed effect. The same analysis was conducted for the mean precision.

## Secondary outcomes

We studied supplemental metrics as an exploratory analysis. Corresponding *p*-values associated with absences of effect are provided without correction for multiple comparisons.

## Supplemental assessment of difference in mean sensitivity and precision

The main analysis was repeated according to readers' experiences (< 10 and ≥ 10 years), readers' specialty (neurologists with expertise in MS, radiologists, and neuroradiologists), acquisitions coverage (upper and lower SC), MRI magnetic field, slice thickness, and image in-plane resolution. Then, sensitivity and precision were computed for the model alone and were compared to the performances of the reader alone using a linear model with an acquisition and a condition (reader without the tool vs model alone) fixed effect.

## Lesion count and lesion-wise sensitivity and precision

The average number of true positive (TP) lesions detected per reader was computed for each condition. This number was tested for equality between the two conditions using a paired *t*-test. Lesion-wise sensitivity and precision were analyzed aggregating all lesions from all scans. The individual lesion-wise sensitivity, together with its 95% CI, was computed for each condition and was then tested for equality between conditions using a logistic regression including a condition and an acquisition fixed effect. The same analysis was reported for the lesion-wise precision. These analyses were repeated according to lesion volume.

## Acquisition-wise categorization

Each acquisition was categorized in three classes: no lesion, 1 or 2 lesions, and ≥ 3 lesions. The overall accuracy associated with this categorization was computed for the two conditions and tested for equality using a logistic regression including a condition, a patient, and a reader effect.

## Task duration

The mean time elapsed for each acquisition, each reader, and each condition was estimated after having filtered out the 5% upper values. This was done to avoid outliers due to delay during the task as a result of external solicitations and tested for equality between conditions using a Wilcoxon rank sum test.

## Inter-reader variability

The inter-reader differences in detected lesions within each condition were reported as multi-rater Light's kappa. The pooled inter-reader standard deviation associated with the number of lesions detected in each condition was computed and compared using a linear model with a condition and an acquisition fixed effect.

## Revised ground truth

The main analyses were repeated with the revised ground truth.

## Results

### Patients, MRI acquisitions, and readers' characteristics

We included 50 patients (39 females, 11 males; 44 relapsing, 6 progressive MS) with SC MRI acquired between February 2017 and December 2022. The median age was 41 years (range: 15–67 years) and the median Expanded Disability Status Scale score was 2 (range: 0–6.5). The data come from Siemens (46), Philips (2), and GE (2) scanners (details in Supplementary Table 1). Thirty-eight datasets came from 1.5-T scanners and 12 from 3-T scanners. The median in-plane resolution was 0.42 × 0.42 mm² (range: 0.34–0.86) for the sagittal T2 and
<table><caption>**Table 1** Readers' characteristics</caption><thead><tr><th rowspan="2"></th><th rowspan="2">All readers<br/>n = 20</th><th colspan="3">Physicians category</th></tr><tr><th>Neuro-radiologists<br/>n = 8</th><th>Radiologists<br/>n = 5</th><th>Neurologists experienced in MS<br/>n = 7</th></tr></thead><tbody><tr><td>Number of years of experience (median, range)</td><td>8 (0–43)</td><td>12 (2–20)</td><td>2 (2–3)</td><td>12 (0–43)</td></tr><tr><td>Number of SC MRI of pwMS read per month (median, range)</td><td>9 (0–100)</td><td>35 (5–50)</td><td>1 (0–2)</td><td>10 (2–100)</td></tr></tbody></table>

pwMS people with MS

0.47 × 0.47 mm² (range: 0.42–1.00) for the sagittal STIR. The median slice thickness was 3 mm (range: 2.00–3.85) for the sagittal T2 and 3 mm (range: 2.50–3.85) for the sagittal STIR. The readers' characteristics are summarized in Table 1.

## Ground truth characteristics

A total of 170 lesions were detected by both experts 1 and 2. They disagreed on 46 lesions. Of these, 36 were validated during the consensual reading with the third expert, resulting in a total number of 206 lesions included in the ground truth (148 in upper SC and 58 in lower SC).

## Acquisition-wise sensitivity and precision

The sensitivity of the readers was significantly improved with the tool (79.2% [95% CI: 73.5–85.0%] vs 74.3% [95% CI: 67.8–80.6%]; *p* < 0.0001), with no evidence of a difference in precision (70.1% [95% CI: 64.3–75.9%] vs 69.0% [95% CI: 62.8–75.2; *p* = 0.08] (Table 2 and Figs. 2 and 3). For sensitivity, we observed similar results when grouping readers according to their experience or specialty. Regarding precision, we only found a significant improvement with the tool in readers with less than 10 years of experience (*p* = 0.032).

For the model alone, the mean acquisition-wise sensitivity and precision were 88.6% (83.8–93.1%) and 54.4% (46.0–62.8%), respectively. We observed a significant difference vs the reader alone in mean sensitivity (*p* < 0.0001) but not in mean precision (*p* = 0.14).

The impact of MRI magnetic field, slice thickness, and image in-plane resolution on tool and reader performance is reported in Table 3.

## Lesion count and lesion-wise sensitivity and precision

The mean number of TP lesions detected per reader was 145 without the tool vs 152 with the tool (*p* = 0.0004), and the mean number of FP was 75 with the tool vs 74 without (*p* = 0.67). The lesion-wise sensitivity was improved with the tool (73.9% [95% CI: 68.0–79.8%] vs 70.7% [95% CI: 64.3–77.0%]; *p* = 0.0003) with no evidence of lesion-wise precision improvement (69.8% [95% CI: 64.3–75.3%] vs

69.1% [95% CI: 63.1–75.0%], *p* = 0.47). The model alone has a TP of 175 and a number of FP of 103, providing a lesion-wise sensitivity and precision of 85.4% and 62.9%.

The lesion-wise sensitivity and precision of the model alone were 79.5% and 50.8% for small lesions (< 100 mm³) and 89.3% and 94.9% for large lesions (≥ 100 mm³). The lesion-wise sensitivity of the readers was increased with the tool for both small (*p* = 0.00009) and large lesions (*p* = 0.039) (Supplementary Table 2).

## Acquisition-wise categorization

Based on the ground truth, each of the 100 MRI volumes was categorized as no lesion (*n* = 33), one or two lesions (*n* = 36), or ≥ 3 lesions (*n* = 31). Results of the categorization from the readers are summarized in Supplementary Table 3. We observed no evidence of a difference in categorization with or without the tool (*p* = 0.22).

## Task duration

The median time taken by readers per SC MRI volume was increased with the tool (64.9 s [IQR = 36.0–108.2] vs 58.1 s [IQR = 32.3–103.2] without; *p* = 0.001).

## Inter-reader variability

The Light's kappa was 0.55 without the tool, indicating a moderate inter-expert agreement (Figs. 4 and 5), and was slightly increased at 0.60 with the tool. Similarly, the pooled standard deviation associated with the number of reported lesions was 1.54 without the tool and 1.39 with the tool (*p* = 0.056).

## Revised ground truth

Using revised ground truth, we observed a significant improvement in mean sensitivity with the tool (69.9% [95% CI = 63.8–76.0%] to 74.5% [95% CI = 68.9–80.1%]; *p* value < 0.0001) but no statistical difference in precision.

## Readers' survey

The main results from the survey are reported in Table 4, indicating rather positive feedback on the use of AI for SC MRI reading.
<table><caption>**Table 2** Acquisition-wise sensitivity and precision for all readers and for different subgroups, without the tool or with the tool (expressed in %, with 95% CIs)</caption><thead><tr><th rowspan="2"></th><th colspan="3">Sensitivity</th><th colspan="3">Precision</th></tr><tr><th>Without the tool</th><th>With the tool</th><th>p value</th><th>Without the tool</th><th>With the tool</th><th>p value</th></tr></thead><tbody><tr><td>All readers<br/>n = 20</td><td>74.3 (67.8-80.6)</td><td>79.2 (73.5-85.0)</td><td>&lt; 0.0001</td><td>69.0 (62.8-75.2)</td><td>70.1 (64.3-75.9)</td><td>0.082</td></tr><tr><td>According to the reader's experience</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>&lt; 10 years n = 11</td><td>74.1 (64.7-83.6)</td><td>78.2 (69.8-86.6)</td><td>0.006</td><td>69.3 (58.9-79.8)</td><td>72.0 (62.5-81.4)</td><td>0.032</td></tr><tr><td>≥ 10 years n = 9</td><td>74.4 (63.6-85.2)</td><td>80.5 (70.8-90.1)</td><td>0.0001</td><td>68.7 (60.3-77.0)</td><td>67.8 (59.9-75.6)</td><td>0.687</td></tr><tr><td>According to the reader's expertise</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Neurologist n = 7</td><td>69.0 (45.9-91.0)</td><td>75.6 (63.8-85.7)</td><td>0.0005</td><td>69.7 (61.1-78.3)</td><td>69.2 (60.4-78.0)</td><td>0.492</td></tr><tr><td>Radiologist n = 5</td><td>68.5 (44.9-89.7)</td><td>73.1 (50.9-92.4)</td><td>0.053</td><td>79.1 (58.3-99.8)</td><td>81.3 (63.2-99.4)</td><td>0.512</td></tr><tr><td>Neuro-radiologist n = 8</td><td>82.5 (77.3-84.8)</td><td>86.2 (80.3-89.1)</td><td>0.004</td><td>62.2 (53.0-71.3)</td><td>63.8 (56.2-71.4)</td><td>0.094</td></tr><tr><td>According to the level of the SC</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Upper part n = 20</td><td>73.4 (67.3-79.5)</td><td>78.0 (72.0-84.0)</td><td>0.0007</td><td>70.1 (64.2-75.9)</td><td>70.8 (65.4-76.2)</td><td>0.2787</td></tr><tr><td>Lower part n = 20</td><td>75.6 (65.2-79.5)</td><td>81.3 (72.1-83.6)</td><td>0.002</td><td>67.9 (60.4-75.4)</td><td>69.3 (62.6-76.1)</td><td>0.1076</td></tr></tbody></table>

<figure><img src="image_9.png" /><figcaption>**Fig. 2** Lesion detection performance for each reader and each session: A for sensitivity and B for precision. The individual performance of each reader is represented by a histogram, where light blue bars on top indicate performance augmented with the tool and red bars on top indicate performance lowered with the tool. The black line on each graph indicates the performance of the tool alone</figcaption></figure>

## Discussion

In this study, we developed and validated a DL tool to detect SC MS lesions on combined sagittal T2 and STIR acquisitions commonly acquired in clinical practice. We have demonstrated that the concurrent use of the tool led to significant improvement in readers' sensitivity without lowering precision and tended to reduce inter-reader variability. The majority of clinicians also reported that using the tool improved their comfort when reading SC images.

To date, no study has evaluated the use by clinicians of an AI-based tool to help detect MS SC lesions. As illustrated by our survey, there is a need for this difficult task,

which can have an impact on the diagnosis, the evaluation of the prognosis, and the treatment proposed to pwMS [1, 3, 17]. A precise count of the number and volume of SC lesions is not usually conducted in clinical practice, even though both the number and volume of MS SC lesions in a recent study were associated with future accumulation of disability progression independent of clinical relapses in pwMS [18]. The use of AI tools in clinical practice makes this precise monitoring realistic.

A previous study evaluated an AI model, segmenting MS SC lesions from sagittal T2w or axial T2\*w, compared to manual segmentation [12]. They reported lesion-wise
<figure><img src="image_10.png" /><figcaption>**Fig. 3** Examples of lesions that were detected more frequently by readers with the tool than without the tool. The first column, is sagittal T2 acquisition with the lesion of interest mask in red; the second column, is sagittal T2 acquisition; the third column, is sagittal STIR acquisition. The lesions of interest are localized by the red arrow. Lesion A was detected by eight more readers with the tool than without the tool, lesion B by seven, and lesion C by six</figcaption></figure>

sensitivity at 83.3% and lesion-wise precision at 76.9% (vs 85.4% and 62.9% in our study, based on data acquired only in clinical practice). Although direct comparison with this method is beyond the scope of this study, it should be emphasized that our tool was deliberately optimized for high sensitivity—and, thus, lower precision—to improve the number of MS lesions detected by the readers, based on the assumption that they would remove FP. Our results confirm this hypothesis, with a significant improvement in clinicians’ sensitivity in SC lesion detection with the tool, without lowering precision. It should also be noted that this increase in clinician sensitivity with the tool is observed for both small and large lesions, and whatever the magnetic field of the acquisitions, the thickness of the slices, and the in-plane resolution. However, clinicians must be aware of the potential FP and exercise their critical judgment when interpreting the tool’s suggestions.

Another original aspect of our study is the development of a tool that takes into account two different major sequences to detect SC lesions. Indeed, as recommended

by the latest recommendations [4], a single acquisition of a T2-weighted sequence is not sufficient, and a second sequence is required to confirm the presence of lesions and exclude artifacts [19–21]. While several combinations of sequences are proposed in the recommendations, we have chosen in this study to develop and test a model taking into account the sequences most commonly acquired in clinical practice in France (T2 and STIR). However, unlike brain imaging, in which a clear consensus exists (3D FLAIR), the acquisition combinations for the SC can be multiple, depending on the MRI scanner model and the habits. Recently, sequences such as phase-sensitive inversion recovery [22–24] or MP2RAGE [25] have demonstrated better detection, with better confidence, of cervical SC MS lesions compared to sagittal T2 and STIR, due to optimized contrast and high spatial resolution. The tool, therefore, could be adapted in the future to consider these different combinations of sequences.

The tool did not allow the readers to reduce their interpretation time. This result contrasts with the
<table><caption>**Table 3** Acquisition-wise sensitivity and precision according to MRI magnetic field, slice thickness, and in-plane resolution (expressed in %, with 95% CIs)</caption><thead><tr><th rowspan="2"></th><th colspan="3">Sensitivity</th><th colspan="4">Precision</th></tr><tr><th>Tool alone</th><th>Readers without the tool</th><th>Readers with the tool</th><th>p value (readers with vs without the tool)</th><th>Tool alone</th><th>Readers without the tool</th><th>Readers with the tool</th><th>p value (readers with vs without the tool)</th></tr></thead><tbody><tr><td>According to MRI magnetic field</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>1.5 T n = 38</td><td>86.9 (79.5-94.2)</td><td>74.2 (66.7-81.7)</td><td>79.2 (73.1-85.4)</td><td>0.0006</td><td>51.8 (40.8-62.8)</td><td>68.1 (62.2-74.1)</td><td>70.1 (64.0-76.2)</td><td>0.11</td></tr><tr><td>3 T n = 12</td><td>90.9 (85.8-96.0)</td><td>74.3 (68.8-79.8)</td><td>79.2 (73.6-84.8)</td><td>0.002</td><td>58.2 (44.6-71.8)</td><td>74.3 (63.5-77.0)</td><td>70.3 (64.3-76.3)</td><td>0.35</td></tr><tr><td>According to slice thickness for sagittal T2</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>≤ 3 mm n = 31</td><td>89.2 (84.2-94.2)</td><td>72.3 (65.7-78.8)</td><td>77.4 (61.7-83.1)</td><td>0.00008</td><td>56.2 (45.5-66.8)</td><td>70.6 (56.2-68.2)</td><td>71.3 (65.5-77.0)</td><td>0.65</td></tr><tr><td>> 3 mm n = 19</td><td>87.5 (77.6-97.4)</td><td>77.6 (70.8-84.3)</td><td>82.3 (76.1-88.4)</td><td>0.01</td><td>51.3 (36.9-65.7)</td><td>66.6 (59.5-73.6)</td><td>68.1 (64.6-76.6)</td><td>0.02</td></tr><tr><td>According to in plane resolution for sagittal T2</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>≤ 0.5 × 0.5 mm² n = 29</td><td>89.5 (84.4-94.5)</td><td>71.3 (64.6-77.9)</td><td>76.9 (70.8-82.8)</td><td>0.00008</td><td>53.6 (43.3-63.8)</td><td>68.2 (61.9-74.5)</td><td>68.7 (62.7-74.6)</td><td>0.50</td></tr><tr><td>> 0.5 × 0.5 mm² n = 21</td><td>87.0 (77.2-97.0)</td><td>79.3 (62.9-85.6)</td><td>83.2 (77.8-88.6)</td><td>0.01</td><td>55.6 (40.7-70.5)</td><td>70.7 (65.6-77.8)</td><td>72.5 (66.6-78.4)</td><td>0.02</td></tr></tbody></table>

reduction in interpretation time of brain MRIs reported with tools to help detect new brain lesions [10]. However, the interpretation of SC MRIs is a particularly delicate task, and the addition or removal of the mask to validate or invalidate the lesion proposed by the tool probably explains this result. Furthermore, despite this extended interpretation time, it remains reasonably short (approximately 2 min per patient for an analysis of the full SC), and is accompanied by an increase in sensitivity and a better feeling of comfort when interpreting the images. Finally, this was the first time that readers had used the tool. Reading time could be reduced with practice.

## Limits and perspectives

Our study has several limitations. First, it is well documented that radiologists' performance in interpreting examinations is different in a clinical context than in a retrospective research context [26]. Therefore, further experiments are needed to validate the model in routine use. Second, the task of detecting SC lesions is inherently variable and difficult, as illustrated by the moderate interreader agreement for SC MS lesion detection in our study, which also potentially affects the constitution of the ground truth by experts. To minimize this difficulty, the three experts involved had access to more data than the readers did, such as axial acquisitions and follow-up MRI scans. Moreover, the consistency of our findings with the exploratory analysis using the revised ground truth (from all the lesions detected by the 20 readers) suggests the robustness of our results. Thirdly, we did not ask the readers to segment each lesion, but we considered that they had correctly identified a lesion if they clicked on any voxel within the ground truth lesion mask. This methodological choice likely introduces a bias in the estimation of the clinicians' lesion detection sensitivity and precision. Thus, if the reader clicks *n* times on a TP lesion, it is considered the same as if they click once, whereas if they click *n* times on a given false positive lesion, it is considered *n* false positives. Sensitivity may therefore be slightly overestimated in our study, while precision may be underestimated. However, the main objective of our study was to compare the mean readers' sensitivity and precision in detecting SC lesions with or without the tool. This same bias related to our methodological choice was thus applied in both cases and probably had little impact on the main result of our study. Finally, the developed tool is designed to detect SC lesions in pwMS in cross-sectional MRI. It does not differentiate between SC MS lesions and differential diagnoses, such as an intramedullary tumor or a Neuromyelitis Optica Spectrum Disorder. Furthermore, a longitudinal tool, to assist new SC lesions detection between two MRI scans, remains to be developed and tested.
<figure><img src="image_11.png" /><figcaption>**Fig. 4** Number of experts having detected each lesion present in the ground truth without and with the tool. We observe a wide dispersion in the number of experts who discovered each lesion, which is slightly improved by using the tool</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>**Fig. 5** Example of inter-reader variability for the same SC MRI acquisition. Column 1: sagittal T2 acquisition with the ground truth mask in red, all the experts' clicks on what they thought was a lesion in yellow, and the intersection of ground truth and expert clicks in orange. Column 2: sagittal T2 acquisition. Column 3: sagittal STIR acquisition</figcaption></figure>
<table><caption>**Table 4** Readers' survey, number of responses per item (out of 20 readers)</caption><thead><tr><th></th><th>Completely disagree</th><th>Rather disagree</th><th>Don't know</th><th>Rather agree</th><th>Completely agree</th></tr></thead><tbody><tr><td>"I am confident when I read an SC MRI"</td><td>0</td><td>6</td><td>1</td><td>13</td><td>0</td></tr><tr><td>"I think AI can be useful to help me read an SC MRI"</td><td>0</td><td>0</td><td>0</td><td>11</td><td>9</td></tr><tr><td>"I think the proposed system allows me more comfort in reading an SC MRI"</td><td>0</td><td>3</td><td>3</td><td>12</td><td>2</td></tr><tr><td>"I would like to use such a system in my clinical practice"</td><td>0</td><td>3</td><td>4</td><td>9</td><td>4</td></tr><tr><td>"I would advise such a system to a junior"</td><td>1</td><td>4</td><td>2</td><td>8</td><td>5</td></tr><tr><td>"I would advise such a system to a senior"</td><td>0</td><td>2</td><td>4</td><td>12</td><td>2</td></tr></tbody></table>

SC spinal cord

## Conclusion

Our multi-reader study demonstrated that the concurrent use of AI in SC MRI interpretation increases the sensitivity of radiologists and neurologists in detecting MS SC lesions, without reducing their precision, and improves their comfort in performing this difficult task. Further studies are needed to evaluate the potential of these tools with other sequence combinations, and as an aid to the detection of new SC lesions.

### Abbreviations

DL Deep learning

MS Multiple sclerosis

PwMS People with multiple sclerosis

SC Spinal cord

STIR Short tau inversion recovery

TP True positive

## Supplementary information

The online version contains supplementary material available at https://doi.org/10.1007/s00330-025-11541-0.

### Acknowledgements

We thank Nathalie Blanc for her support in project management and Arthur Masson for his support in designing the annotation interface for the readers.

### Funding

This study was supported by the French National Research Agency (ANR) within the France 2030 program (ANR-21-RHUS-0014). Data collection (OFSEP database) was supported by a grant provided by the French National Research Agency (ANR) within the France 2030 program (ANR-10-COHO-002 OFSEP). Open access funding provided by Centre Hospitalier Universitaire de Rennes.

### Data availability

The individual de-identified imaging data can be obtained upon request to the OFSEP Scientific Board (https://www.ofsep.org/en/data-access). The data generated and analyzed during the study are available from the corresponding author by request. The full study protocol is available at https://osf.io/2q6sb/.

## Compliance with ethical standards

### Guarantor

The scientific guarantor of this publication is Anne Kerbrat.

### Conflict of interest

The authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article.

## Statistics and biometry

One of the authors, Benoit Combès, has significant statistical expertise.

### Informed consent

Written informed consent was obtained from all subjects (patients) in this study. All clinical and imaging data used in the study were extracted (identified) from the French MS registry in October 2023 (NCT02889965). All patients included in the OFSEP database consented to the use of their medical data for research purposes, by signed consent after being fully informed. Data confidentiality and security were consistent with the recommendations of the French Data Protection Agency (CNIL).

### Ethical approval

Institutional Review Board approval was obtained (Rennes local ethics advisory committee no. 24.100).

### Study subjects or cohorts overlap

Not applicable.

### Methodology

* Retrospective

* Diagnostic or prognostic study

* Multicenter study

### Author details

Received: 25 October 2024 Revised: 31 January 2025 Accepted: 20 February 2025

Published online: 04 April 2025

<footnote n="1">¹Department of Neuroradiology, Rennes University Hospital, Rennes, France.</footnote>

<footnote n="2">²EMPENN Research Team, IRISA, CNRS-INSERM-INRIA, Rennes University, Rennes, France. ³Department of Radiology, Rennes University Hospital, Rennes, France. ⁴Neurology Department, Rennes University Hospital, Rennes, France.</footnote>

<footnote n="5">⁵Neurology A, Hôpital Neurologique Pierre Wertheimer, Hospices Civils de Lyon, University Lyon 1, Lyon, France. ⁶Creatis LRMN, CNRS UMR 5220, Inserm U630, Université Claude Bernard Lyon 1, Lyon, France. ⁷CHU Rouen, Department of Neurology, Rouen, France. ⁸CRMBM, CNRS, Aix-Marseille Université, Marseille, France/APHM Hôpital de la Timone, Marseille, France.</footnote>

<footnote n="9">⁹University of Lille, INSERM, Department of Neuroradiology, CHU Lille, Lille, France. ¹⁰Department of Radiology, Lyon University Hospital, Lyon, France.</footnote>

<footnote n="11">¹¹CREATIS - CNRS UMR 5220 & INSERM U1044, University Claude Bernard Lyon 1, Lyon, France. ¹²Department of Neuroradiology, Foundation Adolphe de Rothschild Hospital, Paris, France. ¹³Paris Cité University, Paris, France.</footnote>

<footnote n="14">¹⁴Department of Neuroradiology, East Group Hospital, Hospices Civils de Lyon, Bron, France. ¹⁵Department of Neuroradiology, Rouen University Hospital, Rouen, France. ¹⁶CIC-P 1414 INSERM, University Hospital of Rennes, Rennes, France. ¹⁷Department of Radiology, Lyon Sud Hospital, Hospices Civils de Lyon, Lyon, France. ¹⁸Univ Lyon, Université Claude Bernard Lyon 1, Hospices Civils de Lyon, Fondation EDMUS, OFSEP, Centre de Recherche en Neurosciences de Lyon, Lyon, France.</footnote>
## References

1. Lukas C, Sombekke MH, Bellenberg B et al (2013) Relevance of spinal cord abnormalities to clinical disability in multiple sclerosis: MR imaging findings in a large cohort of patients. *Radiology* 269:542-552

2. Thompson AJ, Banwell BL, Barkhof F et al (2018) Diagnosis of multiple sclerosis: 2017 revisions of the McDonald criteria. *Lancet Neurol* 17:162-173

3. Brownlee WJ, Altmann DR, Prados F et al (2019) Early imaging predictors of long-term outcomes in relapse-onset multiple sclerosis. *Brain* 142:2276-2287

4. Wattjes MP, Ciccarelli O, Reich DS et al (2021) 2021 MAGNIMS-CMSC-NAIMS consensus recommendations on the use of MRI in patients with multiple sclerosis. *Lancet Neurol* 20:653-670

5. Belwal P, Singh S (2025) Deep learning techniques to detect and analysis of multiple sclerosis through MRI: a systematic literature review. *Comput Biol Med* 185:109530

6. Amin M, Martínez-Heras E, Ontaneda D, Prados Carrasco F (2024) Artificial intelligence and multiple sclerosis. *Curr Neurol Neurosci Rep* 24:233-243

7. Valverde S, Cabezas M, Roura E et al (2017) Improving automated multiple sclerosis lesion segmentation with a cascaded 3D convolutional neural network approach. *Neuroimage* 155:159-168

8. Amin M, Nakamura K, Ontaneda D (2024) Differentiating multiple sclerosis from non-specific white matter changes using a convolutional neural network image classification model. *Mult Scler Relat Disord* 82:105420

9. Rovira A, Corral JF, Auger C et al (2021) Assessment of automatic decision-support systems for detecting active T2 lesions in multiple sclerosis patients. *Mult Scler* 28:1209-1218

10. Combès B, Kerbrat A, Pasquier G et al (2021) A clinically-compatible workflow for computer-aided assessment of brain disease activity in multiple sclerosis patients. *Front Med (Lausanne)* 8:740248

11. van Leeuwen KG, Schalekamp S, Rutten MJCM, van Ginneken B, de Rooij M (2021) Artificial intelligence in radiology: 100 commercially available products and their scientific evidence. *Eur Radiol* 31:3797-3804

12. Gros C, De Leener B, Badji A et al (2019) Automatic segmentation of the spinal cord and intramedullary multiple sclerosis lesions with convolutional neural networks. *Neuroimage* 184:901-915

13. Zhuo Z, Zhang J, Duan Y et al (2022) Automated classification of intramedullary spinal cord tumors and inflammatory demyelinating lesions using deep learning. *Radiol Artif Intell* 4:e210292

14. Vukusic S, Casey R, Rollot F et al (2020) Observatoire Français de la Sclérose en Plaques (OFSEP): a unique multimodal nationwide MS registry in France. *Mult Scler* 26:118-122

15. Confavreux C, Compston DA, Hommes OR, McDonald WI, Thompson AJ (1992) EDMUS, a European database for multiple sclerosis. *J Neurol Neurosurg Psychiatry* 55:671-676

16. Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention—MICCAI 2015. Springer International Publishing, 234-241

17. Kerbrat A, Gros C, Badji A et al (2020) Multiple sclerosis lesions in motor tracts from brain to cervical cord: spatial distribution and correlation with disability. *Brain* 143:2089-2105

18. Lauerer M, McGinnis J, Bussas M et al (2023) Prognostic value of spinal cord lesion measures in early relapsing-remitting multiple sclerosis. *J Neurol Neurosurg Psychiatry*. https://doi.org/10.1136/jnnp-2023-331799

19. Chang CA, Chong AL, Chandra RV et al (2021) Detection of multiple sclerosis lesions in the cervical cord: Which of the MAGNIMS "mandatory" non-gadolinium enhanced sagittal sequences is optimal at 3T? *Neuroradiol J* 34:600-606

20. Bot JC, Barkhof F, Lycklama à Nijeholt GJ et al (2000) Comparison of a conventional cardiac-triggered dual spin-echo and a fast STIR sequence in detection of spinal cord lesions in multiple sclerosis. *Eur Radiol* 10:753-758

21. Philpott C, Brotchie P (2011) Comparison of MRI sequences for evaluation of multiple sclerosis of the cervical spinal cord at 3T. *Eur J Radiol* 80:780-785

22. Fechner A, Savatovsky J, El Methni J et al (2019) A 3T phase-sensitive inversion recovery MRI sequence improves detection of cervical spinal cord lesions and shows active lesions in patients with multiple sclerosis. *AJNR Am J Neuroradiol* 40:370-375

23. Poonawalla AH, Hou P, Nelson FA, Wolinsky JS, Narayana PA (2008) Cervical spinal cord lesions in multiple sclerosis: T1-weighted inversion-recovery MR imaging with phase-sensitive reconstruction. *Radiology* 246:258-264

24. Mirafzal S, Goujon A, Deschamps R et al (2020) 3D PSIR MRI at 3 Tesla improves detection of spinal cord lesions in multiple sclerosis. *J Neurol* 267:406-414

25. Demortière S, Lehmann P, Pelletier J, Audoin B, Callot V (2020) Improved cervical cord lesion detection with 3D-MP2RAGE sequence in patients with multiple sclerosis. *AJNR Am J Neuroradiol* 41:1131-1134

26. Gur D, Bandos AI, Cohen CS et al (2008) The "laboratory" effect: comparing radiologists' performance and variability during prospective clinical and laboratory mammography interpretations. *Radiology* 249:47-53

## Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.