<figure><img src="image_1.png" /></figure>

HAL
open science

# Evaluating Multi-Channel Input of Sagittal T2-weighted and STIR MRI for Automated Segmentation of Spinal Cord MS Lesions

Ricky Walsh, Anne Kerbrat, Benoît Combès, Francesca Galassi

► To cite this version:

Ricky Walsh, Anne Kerbrat, Benoît Combès, Francesca Galassi. Evaluating Multi-Channel Input of Sagittal T2-weighted and STIR MRI for Automated Segmentation of Spinal Cord MS Lesions. 2026. [hal-05487910](https://hal HAL Id: hal-05487910)

https://inria.hal.science/hal-05487910v1

Preprint submitted on 1 Feb 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# EVALUATING MULTI-CHANNEL INPUT OF SAGITTAL T2-WEIGHTED AND STIR MRI FOR AUTOMATED SEGMENTATION OF SPINAL CORD MS LESIONS

Ricky Walsh<fnref n="1" />

Anne Kerbrat<fnref n="1" />,<fnref n="2" />

Benoît Combès<fnref n="1" />,∗

Francesca Galassi<fnref n="1" />,∗

## ABSTRACT

Identifying multiple sclerosis lesions in spinal cord MRI is important but challenging, and automated methods have been proposed to reduce inter-rater variability. Although clinicians typically use multiple MR sequences to identify lesions for a given patient, most automated methods use only a single sequence as input. We investigated combining sagittal T2-weighted and STIR sequences using an early fusion U-Net for lesion segmentation, assessing the impact of re-sampling techniques and U-Net architectures (3D vs. hybrid 2D/3D). Results showed no overall improvement from adding STIR (p=0.17), though this varied among subjects. Furthermore, T2-only models trained on larger datasets outperformed T2+STIR models trained on subsets, indicating that a larger single-sequence dataset is preferable to a smaller multi-sequence one.

**Index Terms—** Deep Learning, Multiple Sclerosis, Spinal Cord, Lesion Segmentation, MRI, Early Fusion

## 1. INTRODUCTION

Multiple sclerosis (MS) is a chronic demyelinating disease of the central nervous system affecting over 2.8 million people globally [1]. Magnetic resonance imaging (MRI) of the spinal cord (SC) to identify MS lesions is an important tool for diagnosis and prognosis of the disease [2]. However, identifying these lesions is a complex task and gives rise to significant inter-rater variability [3], [4]. Automated tools have demonstrated potential to improve clinician sensitivity in detecting lesions [4], but there is still much room for improvement in the performance of these tools.

Clinicians often use more than one MR sequence for SC acquisitions to rule out artefacts and improve confidence in identified lesions [2]. In the same way, using information from multiple MR sequences is a promising direction to improve the performance of automated methods, yet this problem has not been well studied for SC MRI [5].

Sagittal T2-w and STIR are two of the most commonly used sequences in clinical practice for MS [2]. STIR may offer a better contrast between lesions and normal-appearing

tissue and higher sensitivity than sagittal T2-w [6], but is also prone to more artefacts [7] and so could increase the risk of false positives. These trade-offs motivate an investigation into whether combining the two sequences improves overall segmentation performance for an automated method.

In this work, we explored the effectiveness of combining sagittal T2-w and STIR using early fusion (i.e., multi-channel input) in a deep learning model to segment MS SC lesions. Early fusion is appealing for its simplicity and compatibility with existing architectures, and is a popular approach when using multiple MR sequences [8], also being the approach used by the state-of-the-art nnU-Net framework [9].

The main contribution of this work is a systematic study of the effect of early fusion of sagittal T2-w and STIR for lesion detection and segmentation compared to T2-w alone. To the best of our knowledge, this is the first work to study early fusion of T2-w and STIR volumes. In contrast to [4], which assessed the clinical impact of a single tool using T2-w and STIR in a multi-reader setting, our work provides a systematic methodological evaluation of early fusion versus single-sequence training. We also examined the effect of re-sampling techniques (isotropic and anisotropic) along with different types of U-Net (3D vs. hybrid 2D/3D). Finally, early fusion approaches combining two imaging modalities typically use subsets with both modalities for training. A direct disadvantage of this is that training a single-modality model on a larger dataset could outperform a two-modality model trained on a subset, and so we examined the performance of T2-only and T2+STIR models at several dataset sizes to investigate this question.

## 2. METHOD

### 2.1. Dataset

The dataset came from the OFSEP clinical database [10] and research studies.<fnref n="1" /> 236 subjects had at least one sagittal T2-w acquisition, with 379 acquisitions at a single timepoint, with the upper and lower spine imaged in two separate acquisitions. 19 of these were excluded due to poor image quality. Of the remaining 360 T2-w acquisitions, 201 had a corresponding STIR acquisition with the same coverage in the same session.

<footnote n="1">¹ Univ Rennes, Inria, CNRS, Inserm, IRISA UMR 6074, Empenn, Rennes, France</footnote>

<footnote n="2">² Department of Neurology, Rennes University Hospital, Rennes, France</footnote>

<footnote>*Equal contribution.</footnote>

<footnote n="1">¹Study IDs: NCT02117375, NCT04220814, NCT04918225</footnote>
16 scanner models were represented in the dataset across three vendors, Siemens (84%), GE (10%), and Philips (6%). The 10th, 50th, and 90th percentiles of the T2-w slice spacing were 2.75mm, 3.00mm, and 3.60mm, and of the in-plane spacing were 0.42mm, 0.59mm, and 0.80mm, respectively.

The 201 cases with both T2-w and STIR acquisitions were split into five folds for cross-validation, stratifying by scanner vendor and lesion load, and ensuring that each subject appeared in only one fold. For each run, three folds were used for training, one fold for validation (e.g., selecting the best epoch during training, or to decide on hyperparameters), and one fold was used for testing. For experiments with different training set sizes (30, 60, 120), subsets were sampled from each fold. T2-only models were also trained on 240 volumes.

One of six neuroradiologists and neurologists reviewed each sagittal T2-w volume, along with other timepoints or acquisitions from other sequences where available. The segmentation masks created by these experts were validated and adapted, if necessary, by a senior neurologist with 10 years' experience in reading SC MRI. In a second round, an automated tool developed in previous work [3] was applied to the sagittal T2-w volumes and the predicted segmentations were reviewed by the senior expert, who determined whether the proposed lesions should be accepted or rejected, and manually adapted the segmentations if necessary.

## 2.2. Preprocessing

### 2.2.1. Processing T2-w Volumes

The volume was first re-oriented to RPI orientation. We used the spinal cord toolbox (SCT v6.0) (sct_deepseg_sc) to segment the SC in the T2-w volume. SCT_get_centerline was then used to extract the centreline of the cord, or to manually annotate the centreline for the small number of cases with poor segmentations. We used two different strategies for spacing and interpolation, isotropic and anisotropic, and the differences in preprocessing are outlined in the first two rows of Table 1.

Finally, the volume was cropped and shifted around the SC centreline, as in [11]. In the isotropic case, the volume was cropped to a square of 48×48 voxels (24×24mm) around the centreline in each axial slice, whereas in the anisotropic case the area was 10×40 voxels (27.5×23.6mm). Then, each cropped axial slice was stacked vertically along the z-axis, creating a straightened version of the SC without introducing further interpolation. Unlike [11], we retained the full volumes and used large patches during training. This cropping and shifting was used primarily as a way to reduce training times. Exploratory experiments did not show significant differences in segmentation performance between this approach and simply cropping to the bounding box of the SC.

<table><caption>**Table 1.** Differences between the isotropic and anisotropic experimental settings.</caption><thead><tr><th></th><th>Iso</th><th>Aniso</th></tr></thead><tbody><tr><td>Voxel Spacing</td><td>0.5 × 0.5 × 0.5 (mm)</td><td>2.75 × 0.59 × 0.59 (mm)</td></tr><tr><td>Interpolation</td><td>Trilinear</td><td>B-spline in-plane, nearest neighbour through-plane</td></tr><tr><td>Patch size</td><td>48 × 48 × 384 (voxels)</td><td>10 × 40 × 384 (voxels)</td></tr><tr><td>Convolutions</td><td>3D</td><td>2D in first two stages of U-Net, then 3D</td></tr><tr><td>Downsampling (Strided Convolutions)</td><td>Applied along all axes</td><td>Applied in sagittal plane in first two stages, then all axes</td></tr></tbody></table>

### 2.2.2. Processing STIR Volumes

First, the T2-w and STIR volumes were cropped around the T2 SC segmentation, to a cylinder with radius at least 35mm around the SC centreline. The STIR volume was then registered to the T2-w volume, as the manual segmentations were created in the T2-w image space, using the SCT deep learning registration model (sct_register_multimodal). The same resampling and interpolation were used as for the T2-w volume. Finally, both volumes were cropped to the intersection of the coverage of both acquisitions.

## 2.3. Architecture

The architecture and training strategy were adapted from the state-of-the-art nnU-Net framework [9], with modifications to suit the characteristics of the preprocessed data.

The patch size was set to 48 × 48 × 384 for the isotropic setting and 10 × 40 × 384 for the anisotropic setting. nnU-Net suggested a patch height of 520, but limiting this to 384 reduced training time and gave slightly higher Dice in preliminary experiments. For the same reason, used four stages in the U-Net, rather than the six proposed by nnU-Net.

The last rows of Table 1 summarise differences in architecture between the isotropic and anisotropic settings. Specifically, the U-Net for the isotropic setting was fully 3D, whereas the anisotropic setting used a hybrid 2D/3D U-Net. For this latter case, the initial stages used 2D convolutions and downsampling applied only in the sagittal plane. 2D operations were used until the effective in-plane spacing was within a factor of two times the between-slices spacing – after that point, 3D convolutions were used and downsampling was applied along all axes. This was after two stages in our case.<fnref n="2" /> The aniso setting is the default nnU-Net setting for highly anisotropic data, but a recent update to nnU-Net found that the iso setting can sometimes improve performance [12].

To process sagittal T2-w and STIR (T2+STIR), an early fusion approach was used, i.e., the two volumes were provided as two input channels to the model, whereas for T2-only models, only the T2-w volume was provided as a single input channel, but the architectures were otherwise identical.

<footnote n="2">²The anisotropic spacing was (2.75, 0.59, 0.59)mm. After two downsampling steps, the effective in-plane spacing was 2.36mm.</footnote>
## 2.4. Training

Training was carried out using PyTorch v1.13.1 with the PyTorch Lightning (v2.0.0) framework. The batch size was increased to 8, to utilise the available 24GB GPU RAM. Experiments were run for 50000 iterations, and two patches were randomly sampled from every volume in every epoch. The SGD optimiser was used during training, with polynomial decay applied to the learning rate. The initial learning rate was set to 0.1 as this outperformed the default nnU-Net learning rate of 0.01 in preliminary experiments. The default loss from nnU-Net was used, i.e., Dice + cross-entropy, and deep supervision was applied during training.

Data augmentation was applied using TorchIO (v0.18.87) [13] and included gamma intensity transformations ($p = 0.3$), flipping each axis with $p = 0.5$, blurring with $\sigma \in (0.5, 1.5)$ and $p = 0.3$, applying a random bias field ($p = 0.2$), random downsampling ($p = 0.25$) with a factor drawn from (1.0, 2.0), simulating small motion artefacts ($p = 0.2$), an affine transform ($p = 0.4$) with scaling factor between (-0.2, 0.2) and rotation around the z-axis within (-30, 30) degrees, or alternatively an elastic deformation ($p = 0.1$). Finally, the patch was normalised to mean 0 and standard deviation 1 and noise sampled from $N(0, 0.1)$ was injected with $p = 0.15$.

## 2.5. Evaluation

We used the evaluation protocol of the MS-Multi-Spine Challenge on MS SC lesion detection.<fnref n="3" /> Given a prediction mask where each lesion has a single predicted score, the sensitivity is calculated at five levels of False Positive lesions Per Image (FPPI), namely 0.25, 0.5, 1, 2, and 3. A predicted lesion is considered a true positive if its intersection over union (IoU) with a ground-truth (GT) lesion is above 0.2. The FROC score is then the average of the five sensitivity values.

To convert from a segmentation output, where each voxel has a predicted score, to a single score per lesion, we binarised the output at a certain threshold, computed the 3D connected components, and assigned the lesion score to be the maximum score within the lesion. For the threshold, we selected that which gave the best FROC score on the validation set, among the thresholds $10^{-6}, 10^{-5}, 10^{-4}, 10^{-3}, 10^{-2}, 10^{-1}$, and 0.5.

# 3. RESULTS

## 3.1. T2 vs. T2+STIR with 120 training cases

Fig. 1 presents the results for experiments with 120 training cases (i.e., either 120 T2-w volumes for T2 models or 120 T2-w & 120 STIR volumes for T2+STIR models), where each experiment was run five times with 5-fold cross validation. In the aniso setting (Fig. 1.A), the mean FROC was slightly lower for T2+STIR compared to T2, whereas in the iso setting (Fig. 1.B), the mean FROC score was slightly higher for T2+STIR. However, in both of these cases, the differences

<figure><img src="image_3.png" /><figcaption>**Fig. 1. Performance with 120 training cases.** Each point is a single cross-validation run, the dashed lines connect corresponding runs, and the star shows the mean. No comparison was statistically significant – pairwise t-tests gave the p-values 0.31, 0.17, 0.27, 0.28 for A, B, C, D, respectively.</figcaption></figure>

between T2 and T2+STIR were not statistically significant ($p = 0.31$ for aniso and $p = 0.17$ for iso).

## 3.2. Effect of training set size

Fig. 2 shows results in the anisotropic setting for different sizes of the training dataset. In both the T2 and the T2+STIR setting, the performance increased as the number of training images increased, as one might expect, rising from a FROC of 0.38 for T2 models with 30 training volumes to 0.60 with 240 training volumes. We note that the confidence intervals at each scale overlap between the T2 and T2+STIR results, indicating again that there is no evidence to support our hypothesis that combining T2 and STIR with an early fusion approach improves the performance. Moreover, given the choice between training a model on *n* cases with both T2-w and STIR or training a model on *2n* cases with just T2-w, it is more likely that the latter would perform better.

## 3.3. Qualitative Examples

Fig. 3 presents examples of lesions that were detected consistently more or less frequently by T2+STIR models than T2 models. While the average FROC results may be similar between T2 and T2+STIR models, the outputs are not necessarily equivalent. Indeed, these examples show that there are some cases where a lesion that is less conspicuous in a T2-w volume can be more reliably detected if a STIR volume is also used. Conversely, there are also cases where a typical

<footnote n="3">³https://portal.fli-iam.irisa.fr/MS-Multi-Spine/</footnote>
<figure><img src="image_4.png" /><figcaption>**Fig. 2.** Effect of training set size. Mean FROC vs. the number of cases used for training. Error bars are ±1.96× standard error over five-fold cross validation. Points are offset for legibility.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>**Fig. 3.** Examples where STIR improves (top row) or impairs (bottom row) lesion detection. For each GT lesion, the rate of detection was calculated across all T2 and T2+STIR models in all experiments with either 120 or 240 training volumes, binarising at a threshold of 0.5. **Top:** detected by 25% of T2 models; 80% of T2+STIR models. **Bottom:** detected by 58% of T2 models; 0% of T2+STIR models.</figcaption></figure>

true positive for T2 models can become a false negative for T2+STIR models if the lesion is less clear in the STIR volume. The performance of T2+STIR models is thus partly dependent on the relative frequency of these types of lesions.

# 4. DISCUSSION

Although additional acquisitions with different image contrast help clinicians to better detect SC lesions and rule out artefacts, processing STIR alongside sagittal T2-w with early fusion did not outperform T2-w alone for automated lesion detection and segmentation in this work. However, while we did not observe an overall improvement with T2+STIR models, the information from the STIR volume did help for specific cases. The overall result thus likely depends on the relative frequency of lesions where the STIR acquisition provides useful or detrimental information. These findings suggest a hybrid approach may be beneficial: for each case, both T2-only and T2+STIR predictions could be generated, and a downstream classifier could decide which prediction is more reliable. This represents an interesting direction for future research.

There are several potential causes of the lack of overall improvement with an early fusion approach to combine the information from T2-w and STIR volumes. First, although STIR may provide higher lesion contrast, most lesions are nonetheless visible in the T2-w volumes and an automated method could thus potentially detect them without the aid of the STIR volume. Second, even if STIR does introduce beneficial information and allows certain lesions to be detected, artefacts in STIR acquisitions could introduce false positives or even false negatives if real lesions are not visible. Finally, it may be that the relationship between the information in T2-w and STIR acquisitions is too complex to be captured at the first layer of a CNN. Radiologists do not integrate information from multiple sequences at the pixel level, and automated methods may likewise benefit from first extracting higher-level features from each acquisition – an approach better suited to intermediate or late fusion strategies.

One limitation of the study is that most of the STIR (& corresponding T2-w) volumes were clinical acquisitions, and so the quality may be lower than in research settings where longer acquisition sessions can be more acceptable. However, as the developed automated tools are primarily aimed at clinical practice, it is important to evaluate them in this setting. Another limitation is that the manual segmentations were created on the T2-w volumes. STIR volumes were registered to the T2-w space, potentially affecting the signal, which may contribute to the lack of improvement with T2+STIR models.

# 5. COMPLIANCE WITH ETHICAL STANDARDS

This research study was conducted retrospectively using human subject data from a clinical database (OFSEP) as well as several research studies, as detailed in the text. Specific ethical approval was not required for the current study. All subjects gave written informed consent for their data to be used for research. Data storage and processing complied with GDPR and French regulations.
# 6. ACKNOWLEDGMENTS

R.W. was supported by the French doctoral programme in artificial intelligence, managed by the French National Research Agency (ANR) (ANR-20-THIA-0018). This work was supported in part by ANR through the France 2030 programme (ANR-21-RHUS-0014). Data collection was supported by ANR through the France 2030 programme (ANR-10-COH0-0002 OFSEP). Experiments presented in this paper were carried out using the Grid'5000 testbed, supported by a scientific interest group hosted by Inria and including CNRS, RE-NATER and several universities as well as other organisations (see https://www.grid5000.fr).

# References

[1] C. Walton, R. King, L. Rechtman, W. Kaye, E. Leray, R. A. Marrie, et al., "Rising prevalence of multiple sclerosis worldwide: Insights from the Atlas of MS, third edition," *Multiple Sclerosis*, vol. 26, no. 14, Dec. 2020.

[2] F. Barkhof, D. S. Reich, J. Oh, M. A. Rocca, D. K. B. Li, P. Sati, et al., "2024 MAGNIMS-CMSC-NAIMS consensus recommendations on the use of MRI for the diagnosis of multiple sclerosis," *The Lancet Neurology*, vol. 24, no. 10, Oct. 1, 2025.

[3] R. Walsh, C. Meurée, A. Kerbrat, A. Masson, B. R. Hussein, M. Gaubert, et al., "Expert Variability and Deep Learning Performance in Spinal Cord Lesion Segmentation for Multiple Sclerosis Patients," in *2023 IEEE 36th International Symposium on Computer-Based Medical Systems (CBMS)*, Jun. 2023.

[4] B. Lodé, B. R. Hussein, C. Meurée, R. Walsh, M. Gaubert, N. Lassalle, et al., "Evaluation of a deep learning segmentation tool to help detect spinal cord lesions from combined T2 and STIR acquisitions in people with multiple sclerosis," *European Radiology*, Apr. 4, 2025.

[5] R. Walsh, M. Gaubert, C. Meurée, B. R. Hussein, A. Kerbrat, R. Casey, et al., "Multi-Sequence Learning for Multiple Sclerosis Lesion Segmentation in Spinal Cord MRI," in *MICCAI*, M. G. Linguraru, Q. Dou, A. Feragen, S. Giannarou, B. Glocker, K. Lekadir, et al., Eds., Cham: Springer Nature Switzerland, 2024.

[6] P. Alcaide-Leon, A. Pauranik, L. Alshafai, S. Rawal, J. Oh, W. Montanera, et al., "Comparison of Sagittal FSE T2, STIR, and T1-Weighted Phase-Sensitive Inversion Recovery in the Detection of Spinal Cord Lesions in MS at 3T," *AJNR: American Journal of Neuroradiology*, vol. 37, no. 5, May 2016.

[7] J.-C. Brisset, S. Kremer, S. Hannoun, F. Bonneville, F. Durand-Dubief, T. Tourdias, et al., "New OFSEP recommendations for MRI assessment of multiple sclerosis patients: Special consideration for gadolinium deposition and frequent acquisitions," *Journal of Neuroradiology*, vol. 47, no. 4, Jun. 1, 2020.

[8] T. Zhou, S. Ruan, and S. Canu, "A review: Deep learning for medical image segmentation using multimodality fusion," *Array*, vol. 3–4, Sep. 1, 2019.

[9] F. Isensee, P. F. Jaeger, S. A. A. Kohl, J. Petersen, and K. H. Maier-Hein, "nnU-Net: A self-configuring method for deep learning-based biomedical image segmentation," *Nature Methods*, vol. 18, no. 2, 2 Feb. 2021.

[10] S. Vukusic, R. Casey, F. Rollot, B. Brochet, J. Pelletier, D.-A. Laplaud, et al., "Observatoire Français de la Sclérose en Plaques (OFSEP): A unique multimodal nationwide MS registry in France," *Multiple Sclerosis Journal*, vol. 26, no. 1, 2020.

[11] C. Gros, B. De Leener, A. Badji, J. Maranzano, D. Eden, S. M. Dupont, et al., "Automatic segmentation of the spinal cord and intramedullary multiple sclerosis lesions with convolutional neural networks," *NeuroImage*, vol. 184, Jan. 1, 2019.

[12] F. Isensee, T. Wald, C. Ulrich, M. Baumgartner, S. Roy, K. Maier-Hein, et al., "nnU-Net Revisited: A Call for Rigorous Validation in 3D Medical Image Segmentation," in *MICCAI*, M. G. Linguraru, Q. Dou, A. Feragen, S. Giannarou, B. Glocker, K. Lekadir, et al., Eds., vol. 15009, Cham: Springer Nature Switzerland, 2024.

[13] F. Pérez-García, R. Sparks, and S. Ourselin, "TorchIO: A Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning," *Computer Methods and Programs in Biomedicine*, vol. 208, Sep. 1, 2021.