# A recurrent network for segmenting the thrombus on brain MRI in patients with hyper-acute ischemic stroke

Sofia Vargas Ibarra, Vincent Vigneron, Hichem Maaref, Jonathan Kobold, Sonia Garcia-Salicetti, Nicolas Chausson, Didier Smadja, Yann L'Hermitte

<figure><img src="image_1.png" /></figure>

## To cite this version:

Sofia Vargas Ibarra, Vincent Vigneron, Hichem Maaref, Jonathan Kobold, Sonia Garcia-Salicetti, et al.. A recurrent network for segmenting the thrombus on brain MRI in patients with hyper-acute ischemic stroke. 7th International Conference on Medical Imaging with Deep Learning (MIDL 2024), Medical Imaging with Deep Learning Foundation, Jul 2024, Paris, France. pp.657-671. hal-04939296

HAL Id: hal-04939296

https://univ-evry.hal.science/hal-04939296v1

Submitted on 24 Jul 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# A recurrent network for segmenting the thrombus on brain MRI in patients with hyper-acute ischemic stroke

Sofia Vargas-Ibarra¹

Vincent Vigneron¹

Hichem Maaref¹

Jonathan Kobold¹

¹Univ Evry, Université Paris-Saclay, IBISC, Evry, France.

Sonia Garcia-Salicetti²

²SAMOVAR, Télécom SudParis, Institut Polytechnique de Paris, Palaiseau, France.

Nicolas Chausson³

Didier Smadja³

Yann Lhermitte³

³Neurology Department, Sud-Francilien Hospital, Corbeil-Essonnes, France.

SOFIA.VARGASIBARRA@UNIV-EVRY.FR

VINCENT.VIGNERON@UNIV-EVRY.FR

HICHEM.MAAREF@UNIV-EVRY.FR

JONATHAN.KOBOLD@GMX.DE

SONIA.GARCIA@TELECOM-SUDPARIS.EU

NICOLAS.CHAUSSON@CHSF.FR

DIDIER.SMADJA@CHSF.FR

YANN.LHERMITTE@CHSF.FR

## Abstract

In the stroke workflow, timely decision-making is crucial. Identifying, localizing, and measuring occlusive arterial thrombi during initial imaging is a critical step that triggers the choice of therapeutic treatment for optimizing vascular re-canalization. We present a recurrent model that segments the thrombus in patients suffering from a hyper-acute stroke. A cross-attention module is defined to merge the diffusion and susceptibility-weighted modalities available in magnetic resonance imaging (MRI), which are fed to a modified version of convolutional long-short-term memory (CLSTM). It detects almost all the thrombi with a Dice higher than 0.6. The lesion segmentation prediction reduces the false positives to almost zero and the performance is comparable between distal and proximal occlusions.

**Keywords:** hyperacute stroke, brain imagery, MRI modalities, lesion, thrombus, spatial recurrence, deep learning

## 1. Introduction

Stroke causes millions of deaths and cerebral palsy per day nowadays. acute ischemic stroke (AIS) occurs when a blood clot (thrombus) blocks an artery, restricting average blood circulation and initiating brain tissue damage. The lesion caused by the stroke can be separated into (1) the penumbra, which represents the brain tissue that is potentially recoverable but temporarily injured, and (2) the core, the irreversibly damaged tissue. The lesion's size and location are crucial pieces of information that help the doctors locate the thrombi and decide the treatment. In an emergency case, the pathology is in the hyper-acute phase (<6h from the onset of the symptoms) which is critical for saving lives and improving life expectancy of patients.

This paper proposes an experimentally validated and computationally affordable method capable of segmenting the thrombus of an AIS patient with few false positives using lesion segmentation as support. This technique is based on a light spatial recurrent neural network architecture.
## 1.1. Related works on classical image processing techniques

As the lesion is brighter (in the case of DWI) or darker (in ADC) depending on the MRI modalities, some approaches exploit these properties using non-learning algorithms. For instance, unsupervised clustering such as Fuzzy C-means produces considerably good automatic segmentation results (Tsai et al., 2014). It is applied on intensity values lower than the histogram peak (healthy brain tissue) using a fixed threshold, and it eliminates the artifacts thanks to several modalities. Also, anomaly detection can be done on DWI as in (Nazari-Farsani et al., 2020), where the Crawford-Howell t-test and the comparison of stroke images to healthy controls are used to develop a classifier to discriminate the images into stroke or non-stroke categories following the lesion segmentation. For the thrombi, (Santos et al., 2014) defined an automatic segmentation method in three steps: segmentation of the contralateral vasculature; creation of a mask in the occluded arterial segment by mapping the contralateral segmentation using mirror symmetry; and segmentation of the thrombus using intensity-based region growing. In (Löber et al., 2017) Random Forests are used to classify features extracted by thresholding and connected component clustering.

## 1.2. Deep learning works related to brain imagery

ISLES challenges (Petzsche et al., 2022) provide several public datasets that allow the training of lesion segmentation models. To manage 3D inputs, 2D CNN models have been used (Clerigues et al., 2019), (Shah et al., 2020), the mixture of 2D and 3D architectures have been proposed as D-Unet (Zhou et al., 2021), DFENet (Basak et al., 2021), and also using variations of Unet3D (Omarov et al., 2022), (Ashtari et al., 2023). Attention methods such as AABTS-Net (Tian et al., 2023) have been used, incorporating axial attention (Wang et al., 2020), skip-connections as PerfUnet (de Vries et al., 2023) or soft-attention mechanisms as DRANet (Liu et al., 2020). UNETR proposed by (Hatamizadeh et al., 2021) is an architecture adapted from Visual Transformers (Dosovitskiy et al., 2021) to 3D inputs for medical applications. But, all these methods rely on considerable big datasets and neither of them is only focused on the hyperacute phase, where the delineation of the affected area is more difficult to do (Jones and Cercignani, 2010). On the other hand, for the thrombi segmentation, (Lucas et al., 2019) learns to segment potential candidates using Unet3D and a Convolutional architecture classifies the candidates. Polar-UNet (Zoetmulder et al., 2022), uses a CNN to segment thrombi restricting the volume-of-interest to the brainstem, or in (Tolhuisen et al., 2020) patches from two hemispheres are used to segment the thrombi, as a healthy brain tissue should be symmetric. However, all these methods use CT images. For MRis in (Kobold et al., 2019) Multidirectional Unet was used and a recurrent method also denoted as Logic LLSTM (Kobold, 2019) but no distal thrombi were present.

# 2. Materials and methods

## 2.1. Dataset and preprocessing

Two datasets are used in this study: CHSF (proximal occlusions (P)) and MATAR (distal occlusions (D)). All MRIs come from patients treated for stroke at the Centre Hospitalier Sud-Francilien and were acquired from a 1.5 T and a 3 T GE Healthcare MRI machine in the hyper-acute phase, before treatment. The details of datasets are described in Table 1.
<table><caption>**Table 1:** Dataset description. The sizes are in pixels (pix.) and the average (avg) of the ground-truths sizes are provided</caption><thead><tr><th></th><th>CHSF</th><th>MATAR</th></tr></thead><tbody><tr><td>Age (years)</td><td>74.33 ± 0.74</td><td>72.95 ± 1.6</td></tr><tr><td>Male</td><td>43.3%</td><td>46.5%</td></tr><tr><td>Hypertension</td><td>62%</td><td>59%</td></tr><tr><td>Current smokers</td><td>11%</td><td>13%</td></tr><tr><td>DWI-ASPECTS<fnref n="1" /></td><td>3.69 (0-10)</td><td>9 (7.75-10)</td></tr><tr><td>SWAN slices</td><td>72 to 216</td><td>72 to 232</td></tr><tr><td>DWI slices</td><td>24 to 38</td><td>24 to 40</td></tr><tr><td>SWAN shape (pix. per slice)</td><td>512×512</td><td>512×512</td></tr><tr><td>DWI shape (pix. per slice)</td><td>256×256</td><td>256×256</td></tr><tr><td>Thrombi size (mm³)</td><td>231.97</td><td>77.04</td></tr><tr><td>Lesion size (mm³)</td><td>31770.21</td><td>5010.15</td></tr><tr><td>Number of patients</td><td>63</td><td>125</td></tr></tbody></table>

Even though they have been used for several clinical studies (Marnat et al., 2023), we use only a subset of it, segmented by neurologists. Notice the differences between MATAR and CHSF: the thrombi and lesion size and the ASPECT. Due to registration problems such as missing modalities, only 188 MRIs are used finally. The susceptibility-weighted angiography (SWAN) with its associated PHASE are used to segment the thrombi, and the apparent diffusion coefficient (ADC), and diffusion-weighted imaging

(DWI) modalities are used to segment the lesion. The susceptibility images (SWAN and PHASE) have the same geometries and alignment. This is because SWAN is calculated using the machine's PHASE and the magnitude image. It happens the same for the diffusion ones, as ADC is obtained from DWI and B0.

The dataset is normalized using Nyul's method (Nyul et al., 2000). Computing per modality an average histogram using the quantiles of all the data, the MRI's intensities are normalized using that histogram as a reference. The skull-stripped MRIs are used and DWI is coregistered to SWAN using ANTS software, producing images of size 512×512 in the (x,y)-plane and z varies between patients. As input for the model, we take 256×256×s-size crops, where s is the chosen number of slices. To reduce x and y the center of mass is calculated to add around 128 pixels in all dimensions (arriving at a size of 256×256 around that center). As DWI and SWAN have different resolutions and indeed the lesion is normally a bigger region than the thrombi, the crops in z are done in the original resolution, taking the corresponding slices. Having s slices from SWAN and s slices from DWI means that we are seeing a bigger brain region in the diffusion modality, which allows the model to have an increased attention impact.

## 2.2. Methodology: AttLLSTM

To detect the thrombi, the neurologists look for the lesion as the first step. From the location of the stroke consequence (damaged tissue) they know where to look for its cause (only one hemisphere is affected for example). Trying to mimic that reasoning, we propose AttLLSTM which merges DWI (where the lesion is visible) and SWAN and PHASE (where the thrombus is seen) using cross-attention. An improved version of the CLSTM (Shi et al., 2015) that we denote as Logic long-short time memory (LLSTM) is used to segment the thrombi. The longitudinal direction matches the time using s slices (the prediction is done by calculating the cell and hidden states with the previous slices). Finally, the segmentation is improved by merging the full lesion and thrombi predictions using a post-processing module. The details of AttLLSTM are shown in Fig. 1.

<footnote n="1">1. It measures the extent of early ischemic changes in anterior circulation hyperacute ischemic stroke. A point is subtracted if it touches one of the 10 brain divisions</footnote>
<figure><img src="image_3.png" /><figcaption>**Figure 1:** AttLLSTM. The thrombus is segmented using a cross-attention module between DWI, SWAN and PHASE followed by the LLSTM architecture.</figcaption></figure>

### 2.2.1. CROSS-ATTENTION

The cross-attention module merges the diffusion modality DWI (as it is just to guide where the affected area is, only one diffusion modality is used) and the susceptibility ones (where the thrombi information is). It is computed as in CrossVIT (Chen et al., 2021). First, we apply a patch embedding ($p_1$ (DWI), $p_2$ (SWAN, PHASE)) of the images. We compute $Q$ from the embedding of the diffusion modality and $K$ and $V$ from the embedding of the concatenation of susceptibility ones, normalizing and applying an MLP. The scaled dot-product attention (Vaswani et al., 2023) is then calculated from $Q$, $K$, and $V$. As $p_1$ and $p_2$ are different values in our case, the smaller embedding output (larger patch size) is resampled. The usual Transformer Encoder module is used where a residual connection is done with a normalization layer, followed by an MLP and another residual connection and normalization. After this module, we reshape the output (going back to a 3D image) and a convolution layer (CNN) of size $7 \times 7$ is included (having the same output channels as input ones) followed by Elu activation. All these operations are applied per slice (in 2D).

### 2.2.2. LOGIC LSTM

Denoting $V||^lS$ as the concatenation along the axis $l$ of the two tensors $V$ and $S$, as $\circ$ the Hadamard product and as $\mathcal{L}(A) = W*A + b$ the classic convolution with a kernel $W$ of size $w \times w$, CLSTM (Shi et al., 2015) is defined by the following equations:

$$A = c_t||^4h_t||^4x_t \quad (1) \quad o_t = \sigma(\mathcal{L}_o(A)) \quad (4) \quad c_{t+1} = f_t \circ c_t + i_t \circ d_t \quad (6)$$

$$f_t = \sigma(\mathcal{L}_f(A)) \quad (2) \quad d_t = \tanh(\mathcal{L}_d(A)) \quad (5) \quad h_{t+1} = o_t \circ \tanh(c_{t+1}) \quad (7)$$

$$i_t = \sigma(\mathcal{L}_i(A)) \quad (3)$$

where $x_t$ is the input image of size $n_1 \times n_2 \times 1 \times n_{4\text{Input}}$. The cell state $c_t$, which is the model's memory, and the hidden state $h_t$, which is the model's output, are of size $n_1 \times n_2 \times 1 \times n_{4\text{hidden}}$. The forget gate $f_t$ and input gate $i_t$ decide what is erased from and written to the memory $c_t$. The output gate $o_t$ determines which parts of the memory are retrieved to create the output. At time $t$, the model has access to its memory of all previous steps, in our case, the longitudinal direction matches time. To reduce the number
**Figure 2:** LSTM. The convolution operation is replaced by the Logic one. This operation reduces the trainable parameters as it is a concatenation of two parts ($a_1, a_2$) where smaller convolutions are applied and a pooling layer is included.

<figure><img src="image_4.png" /></figure>

of parameters and increase the receptive field we propose the operation $Logic(A)$, which replaces $\mathcal{L}(A)$, denoting the architecture as LSTM (Kobold, 2019) as shown in Figure 2.

The operation is reduced to the concatenation of a convolution part ($a_1$) and a logic part ($a_2$). The convolution operation is only applied on a part of $A$ ($A_c$), reducing considerably the number of learned parameters. To do so, we first slice $h_t$ and $c_t$ in $h_{1,t}, h_{2,t}$ and $c_{1,t}, c_{2,t}$.

$$c_t = c_{1,t}||^4 c_{2,t} \qquad h_t = h_{1,t}||^4 h_{2,t}, \qquad (8)$$

i.e. the hidden state $h_t$ is split into a convolution part $h_{1,t}$ with $n_c$ channels and a logic part with $n_l$ channels and likewise for the cell state $c_t$. Using them we define the splits of $A, A_c$ and $A_l$ as following

$$A_c = c_{1,t}||^4 h_{1,t}||^4 x_t \qquad A_l = c_{2,t}||^4 h_{2,t}. \qquad (9)$$

Considering that $\mathcal{L}_i$ are convolution layers with weights $W_i$ and $b_i$ and in particular $\mathcal{L}_2$ has $b_2 = 0$, we obtain the convolution ($a_1$) and the logic ($a_l$) part and $Logic(A)$ as following:

$$a_1 = \mathcal{L}_1(A_c) + \mathcal{L}_2(A_l) \qquad a_2 = \mathcal{L}_4(\mathcal{T}(\mathcal{L}_3(A_c)))||^4 A_l \qquad (10)$$

$$Logic(A) = a_1||^4 a_2 \qquad (11)$$

where $\mathcal{T}$ is put for the Transfer Layer, which produces the same output size as the input size. $\mathcal{L}_1, \mathcal{L}_3$ and $\mathcal{L}_2, \mathcal{L}_4$ are $w \times w$ and $1 \times 1$ convolutions respectively. Finally, denoting $(V_{i_1 i_2 i_3 i_4})_{i_k \in [r_1...r_2]}$ as the slicing $V$ by setting the dimension $i_k$ to a specific range $r_1, ..., r_2$ or a single value $r_1$, $\mathcal{T}$ is defined as follows:

$$(\mathcal{T}(I)_{i_1 i_2 i_3 i_4})_{i_4=u} = \maxpool((I_{i_1 i_2 i_3 i_4})_{i_4=u}, p), \text{where } u = im + j \qquad (12)$$

with $i \in \{1, ..., k\}, j \in \{1, ..., m\}$. $I$ is of dimensions $n_1, n_2, n_3 = 1, n_4 = n_{4l} = km$ where the multiplicity $m$ is a positive integer. We choose $p$ as powers of 2 using $p = \frac{2n_1}{2^i}$. For example, having $n_4 = 6$ and $m=3$, we have $k=2$ so we apply a pooling layer on the first three features, and another one in the last three ones, each one with different sizes. So, this layer performs a max pooling (Ranzato et al., 2007) operation on each channel separately, with varying window sizes ($p$).
The convolution result ($a_1$) can be seen as the original CLSTM, being applied to part of $A(\mathcal{L}_1(A_c))$ and receiving information from the logical part $(\mathcal{L}_2(A_l))$ as additional bias. The $T$ function moves features across the image in a series of different neighborhood sizes, from local to global. This allows the receptive field of the gates to be as large as the whole input image (which allows doing the concatenation with $A_l$) and captures the spatial distance as well. In addition, a double pass is done to not start with zero memory and to replicate a bidirectional recurrence method but without adding parameters. The first pass is done to save all the memory states, so the prediction is done using them in the second one (Kobold, 2019). Finally before the prediction, a convolutional layer is included of size 1×1 with output channels 2 (number of classes).

### 2.2.3. LESION SEGMENTATION AND POST-PROCESSING MODULE

The lesion and thrombi segmentation are merged to reduce the possible false positive thrombi detections. For this purpose, nnUnet (Isensee et al., 2021) is trained using ADC and DWI as inputs (the modalities in which the hyper-acute lesion is seen (Tsai et al., 2014)), reaching a Dice of 0.72 in average and roughly a 100% percent detection rate. More details about these results are in Appendix A.

Two steps are used as postprocessing. First, we choose the object that is closer to the lesion prediction (if there is one), reducing false positives. Using objects with a certain size (the ones smaller than 20 pixels are eliminated), the Euclidean distance between the center of mass of both segmentations is computed. In order to relax constraints, we select the closest object to the lesion with at least $N$ pixels of distance from the others. This produces as output in most cases just one object. Finally, the pixels that are around the predicted one are included reducing the pixel classification threshold ($T$), increasing the size of the predicted thrombus.

## 2.3. Training process and metrics

The training configuration is described in Table 2. We use flipping (in three dimensions) and Gaussian noise (each one is chosen in every iteration with 40% of probability) as data augmentation. As the thrombus is a small object, we use per iteration a crop including the thrombi and another without to manage the unbalanced dataset. We have a sequence of slices containing the full thrombus (successively as the thrombi is a dense object), some other slices just before, and after the thrombus (all of them without thrombi). So per iteration, one crop of $s$ slices is chosen from the first group of crops and another from the other two, both for the same patient.

<table><caption>**Table 2:** Training process. Adam optimizer is used with learning rate (lr) 0.01</caption><thead><tr><th>Batch size</th><th>Batch's crops</th><th>p<sub>1</sub></th><th>p<sub>2</sub></th><th>Att embed</th><th>n<sub>c</sub></th><th>n<sub>l</sub></th><th>m</th><th>n<sub>4</sub></th><th>Train, Val, Test (%)</th><th>Loss function</th><th>T</th><th>N</th></tr></thead><tbody><tr><td>2</td><td>4</td><td>4</td><td>32</td><td>32</td><td>4</td><td>8</td><td>3</td><td>32</td><td>70,20,10</td><td>Cross Entropy</td><td>0.3</td><td>20</td></tr></tbody></table>

The metrics used are the Dice score, which is calculated pixel-wise and measures the overlap between the prediction and the ground truth, the average count and size of false positives (FP) and false negatives (FN) (in pixel count), and the detection rate (det.) calculated by patients, being one if at least one pixel's prediction overlaps the ground truth.
## 3. Results

As the number of slices is a critical hyper-parameter on the recurrent methods presented (in nnUnet the software chooses the crops by itself), a specific study on that parameter is done. The results using several slices are in Table 3 for proximal (P) and distal (D) occlusions.

<table><caption>**Table 3:** Results using different numbers of slices for segmenting the thrombi using AttLLSTM with CHSF and MATAR dataset.</caption><thead><tr><th rowspan="3">Slices</th><th colspan="2">False Positives</th><th colspan="2">False Negatives</th><th colspan="2">Dice</th><th colspan="2">Det.</th></tr><tr><th colspan="2">Count</th><th colspan="2">Size</th><th colspan="2">Count</th><th colspan="2">Size</th></tr><tr><th>P</th><th>D</th><th>P</th><th>D</th><th>P</th><th>D</th><th>P</th><th>D</th></tr></thead><tbody><tr><td>10</td><td>1.3</td><td>0.9</td><td>74.1</td><td>102.7</td><td>0.3</td><td>0.3</td><td>292.7</td><td>79.4</td><td>0.48</td><td>0.51</td><td>0.9</td><td>0.91</td></tr><tr><td><strong>12</strong></td><td><strong>1.0</strong></td><td><strong>0.6</strong></td><td><strong>49.8</strong></td><td><strong>63.3</strong></td><td><strong>0.2</strong></td><td><strong>0.3</strong></td><td><strong>0.2</strong></td><td><strong>79.4</strong></td><td><strong>0.55</strong></td><td><strong>0.54</strong></td><td><strong>1.0</strong></td><td><strong>0.91</strong></td></tr><tr><td>14</td><td>3.3</td><td>2.3</td><td>79.4</td><td>82.8</td><td>0.2</td><td>0.4</td><td>0.2</td><td>108.5</td><td>0.59</td><td>0.42</td><td>1.0</td><td>0.85</td></tr></tbody></table>

Using *s* equal to 12 for all the recurrent methods, as it produces the best result using AttLLSTM, we compare the results with some of the SOTA methods for the thrombi segmentation in Table 4. nnUnet is trained using the general procedure and all our contributions are tested: first, just LLSTM and CLSTM (adding at the beginning a CNN with 5×5 filters instead of the attention), the attention module, and the two post-processing techniques (lesion and threshold (Thr)). As the other methods proposed for the thrombi use the hemisphere position (Tolhuisen et al., 2020) or the reference path through the brainstem as annotations (Zoetmulder et al., 2022), we do not include them.

<table><caption>**Table 4:** Results using different architectures and configurations for segmenting the thrombi using 12 slices for the recurrent methods. The best model is highlighted in bold and the best model without post-processing techniques is underlined.</caption><thead><tr><th rowspan="3">Model</th><th rowspan="3">Datasets</th><th rowspan="3">Lesion</th><th rowspan="3">Thr</th><th colspan="2">False Positives</th><th colspan="2">False Negatives</th><th colspan="2">Dice</th><th colspan="2">Det.</th></tr><tr><th colspan="2">Count</th><th colspan="2">Size</th><th colspan="2">Count</th><th colspan="2">Size</th></tr><tr><th>P</th><th>D</th><th>P</th><th>D</th><th>P</th><th>D</th><th>P</th><th>D</th><th>P</th><th>D</th></tr></thead><tbody><tr><td>nnUnet</td><td>CHSF+MATAR</td><td></td><td></td><td>0.6</td><td>1.1</td><td>37.1</td><td>222.4</td><td>0.4</td><td>0.5</td><td>373.6</td><td>125.3</td><td>0.46</td><td>0.45</td><td>0.7</td><td>0.7</td></tr><tr><td>CLSTM</td><td>CHSF+MATAR</td><td></td><td></td><td>3.8</td><td>2.6</td><td>178.1</td><td>153.5</td><td>0.2</td><td>0.4</td><td>0.2</td><td>28.1</td><td>0.39</td><td>0.33</td><td>1.0</td><td>0.81</td></tr><tr><td>LLSTM</td><td>CHSF+MATAR</td><td></td><td></td><td>0.8</td><td>1.7</td><td>111.9</td><td>34.9</td><td>0.2</td><td>0.6</td><td>0.2</td><td>136.4</td><td>0.48</td><td>0.36</td><td>1.0</td><td>0.64</td></tr><tr><td>AttCLSTM</td><td>CHSF+MATAR</td><td></td><td></td><td>1.2</td><td>1.6</td><td>43.81</td><td>99.3</td><td>0.3</td><td>0.4</td><td>141.1</td><td>100.7</td><td>0.41</td><td>0.38</td><td>0.9</td><td>0.72</td></tr><tr><td>AttLLSTM</td><td>CHSF</td><td></td><td></td><td>3.8</td><td>3.0</td><td>111.9</td><td>101.5</td><td>0.2</td><td>0.6</td><td>0.2</td><td>136.4</td><td>0.50</td><td>0.27</td><td>1.0</td><td>0.64</td></tr><tr><td><u>AttLLSTM</u></td><td>CHSF+MATAR</td><td></td><td></td><td><u>1.0</u></td><td><u>0.6</u></td><td><u>49.8</u></td><td><u>63.3</u></td><td><u>0.2</u></td><td><u>0.3</u></td><td><u>0.2</u></td><td><u>79.4</u></td><td><u>0.55</u></td><td><u>0.54</u></td><td><u>1.0</u></td><td><u>0.91</u></td></tr><tr><td>nnUnet</td><td>CHSF+MATAR</td><td>✓</td><td></td><td>0.0</td><td>0.7</td><td>0.0</td><td>161.3</td><td>0.4</td><td>0.5</td><td>373.6</td><td>130.9</td><td>0.48</td><td>0.51</td><td>0.7</td><td>0.7</td></tr><tr><td>AttCLSTM</td><td>CHSF+MATAR</td><td>✓</td><td></td><td>0.5</td><td>0.9</td><td>19.4</td><td>109.8</td><td>0.3</td><td>0.6</td><td>141.1</td><td>134.4</td><td>0.43</td><td>0.36</td><td>0.9</td><td>0.6</td></tr><tr><td>AttLLSTM</td><td>CHSF+MATAR</td><td>✓</td><td></td><td>0.0</td><td>0.1</td><td>0.0</td><td>8.5</td><td>0.2</td><td>0.3</td><td>0.2</td><td>79.4</td><td>0.52</td><td>0.58</td><td>1.0</td><td>0.91</td></tr><tr><td>nnUnet</td><td>CHSF+MATAR</td><td>✓</td><td>✓</td><td>0.0</td><td>0.8</td><td>0.0</td><td>177.9</td><td>0.5</td><td>0.5</td><td>373.6</td><td>130.9</td><td>0.49</td><td>0.51</td><td>0.7</td><td>0.7</td></tr><tr><td>AttCLSTM</td><td>CHSF+MATAR</td><td>✓</td><td>✓</td><td>0.5</td><td>0.9</td><td>59.9</td><td>245.8</td><td>0.3</td><td>0.6</td><td>141.1</td><td>134.4</td><td>0.54</td><td>0.32</td><td>0.9</td><td>0.6</td></tr><tr><td><u>AttLLSTM</u></td><td>CHSF+MATAR</td><td>✓</td><td>✓</td><td><u>0.0</u></td><td><u>0.1</u></td><td><u>0.0</u></td><td><u>8.5</u></td><td><u>0.2</u></td><td><u>0.3</u></td><td><u>0.2</u></td><td><u>79.4</u></td><td><u>0.63</u></td><td><u>0.59</u></td><td><u>1.0</u></td><td><u>0.91</u></td></tr></tbody></table>

nnUnet struggles to segment the thrombi: less than 80% of the patients are detected. LLSTM outperforms CLSTM in the Dice score, even though the detection rate is lower. Adding to both recurrent methods the attention module increases the performance. AttLLSTM has the highest detection rate (having 91% in the smaller cases) and Dice. Without using the distal occlusions, AttLLSTM misses 40% of the patients and including them in the training improves all metrics. For the post-processing techniques, adding the lesion (Lesion) reduces the number of false positives in more than half of them but also reduces the detection rate for distal cases (some wrong close objects are chosen) for AttCLSTM.
<figure><img src="image_5.png" /><figcaption>**Figure 3:** Visual prediction examples. Original slice and zoom versions are shown. The ground truth (Gt) and the predictions (pred) are shown in the zoomed version. The first row is concerned with proximal thrombus, and the second one with distal thrombus.</figcaption></figure>

Finally, the threshold technique (Thr) allows to have on average a Dice of 0.43 or higher for all. AttLLSTM gets the best performance (0.61 as Dice) detecting almost all the patients (missing less than 10% of distal thrombi). Indeed, the model is already better without the post-processing comparing it with nnUnet/AttCLSTM +lesion+thr in all metrics. To test the robustness of the model, the control MRI segmentation results are present in Appendix B. Figure 3 presents some predictions obtained, the original SWAN, zoomed, and predictions are shown, for a proximal thrombus (a,b,c,d) and distal one (e,f,g,h).

## 4. Conclusion

We have proposed a model relating the lesion and the thrombi segmentation on MRIs that outperforms the other methods tested. A cross-attention between the modalities as a recurrent model where the longitudinal direction matches time and a way of merging the lesion and the thrombus is defined. AttLLSTM detects almost all thrombi present in stroke patients in less than 3 minutes, producing promising results in the hyper-acute phase, for distal and proximal clots. This detection could save time, help to choose a treatment or estimate the survival of the patients.

But, due to the lack of public datasets, the multicentric robustness cannot be tested, even though augmentations and normalizations have been defined to overcome that problem. MRIs from different vendors will not only produce different images due to calibration but the susceptibility modalities can also be different: SWI instead of SWAN. The two postprocessing proposed techniques could be done differently: for example, merging the lesion and thrombus with a learning method or improving the Dice with another technique.
References

Pooya Ashtari, Diana M. Sima, Lieven De Lathauwer, Dominique Sappey-Marinier, Frederik Maes, and Sabine Van Huffel. Factorizer: A scalable interpretable approach to context modeling for medical image segmentation. *Medical Image Analysis*, 84:102706, feb 2023. doi: 10.1016/j.media.2022.102706. URL https://doi.org/10.1016%2Fj.media.2022.102706.

Hritam Basak, Rukhshanda Hussain, and Ajay Rana. DFENet: A novel dimension fusion edge guided network for brain MRI segmentation. *SN Computer Science*, 2 (6), aug 2021. doi: 10.1007/s42979-021-00835-x. URL https://doi.org/10.1007%2Fs42979-021-00835-x.

Chun-Fu Chen, Quanfu Fan, and Rameswar Panda. Crossvit: Cross-attention multi-scale vision transformer for image classification, 2021.

Albert Clerigues, Sergi Valverde, Jose Bernal, Jordi Freixenet, Arnau Oliver, and Xavier Lladó. Acute ischemic stroke lesion core segmentation in ct perfusion images using fully convolutional neural networks. *Computers in biology and medicine*, 115:103487, 2019.

Lucas de Vries, Bart J Emmer, Charles BLM Majoie, Henk A Marquering, and Efstratios Gavves. Perfu-net: Baseline infarct estimation from ct perfusion source data for acute ischemic stroke. *Medical Image Analysis*, 85:102749, 2023.

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale, 2021.

Ali Hatamizadeh, Yucheng Tang, Vishwesh Nath, Dong Yang, Andriy Myronenko, Bennett Landman, Holger Roth, and Daguang Xu. Unetr: Transformers for 3d medical image segmentation, 2021.

Fabian Isensee, Paul F Jaeger, Simon AA Kohl, Jens Petersen, and Klaus H Maier-Hein. nnu-net: a self-configuring method for deep learning-based biomedical image segmentation. *Nature methods*, 18(2):203–211, 2021.

Derek K. Jones and Mara Cercignani. Twenty-five pitfalls in the analysis of diffusion mri data. *NMR in Biomedicine*, 23(7):803–820, 2010. ISSN 1099-1492. doi: 10.1002/nbm.1543. URL http://dx.doi.org/10.1002/nbm.1543.

J. Kobold, V. Vigneron, H. Maaref, D. Fourer, M. Aghasaryan, C. Alecu, N. Chausson, Y. L’Hermitte, D. Smadja, E. Lang, and A. M. Tome. Stroke thrombus segmentation on swan with multi-directional u-nets. In *2019 Ninth International Conference on Image Processing Theory, Tools and Applications (IPTA)*, pages 1–6, 2019. doi: 10.1109/IPTA.2019.8936074.

Jonathan Kobold. *Deep Learning for lesion and thrombus segmentation from cerebral MRI*. Theses, Université Paris Saclay (COmUE), December 2019. URL https://theses.hal.science/tel-03592570.
Liangliang Liu, Lukasz Kurgan, Fang-Xiang Wu, and Jianxin Wang. Attention convolutional neural network for accurate segmentation and quantification of lesions in ischemic stroke disease. *Medical Image Analysis*, 65:101791, 2020.

Patrick Löber, Bernhard Stimpel, Christopher Syben, Andreas K Maier, Hendrik Ditt, Peter Schramm, Boy Raczkowski, and André Kemmling. Automatic thrombus detection in non-enhanced computed tomography images in patients with acute ischemic stroke. In *VCBM*, pages 125–129, 2017.

Christian Lucas, Jonas J Schöttler, André Kemmling, Linda F Aulmann, and Mattias P Heinrich. Automatic detection and segmentation of the acute vessel thrombus in cerebral ct. In *Bildverarbeitung für die Medizin 2019: Algorithmen-Systeme-Anwendungen. Proceedings des Workshops vom 17. bis 19. März 2019 in Lübeck*, pages 74–79. Springer, 2019.

Gaultier Marnat, Bertrand Lapergue, Benjamin Gory, Maeva Kyheng, Julien Labreuche, Guillaume Turc, Stephanze Olindo, Igor Sibon, Jildaz Caroff, Didier Smadja, et al. Intravenous thrombolysis with tenecteplase versus alteplase combined with endovascular treatment of anterior circulation tandem occlusions: A pooled analysis of etis and tetris. *European Stroke Journal*, page 23969873231206894, 2023.

Sanaz Nazari-Farsani, Mikko Nyman, Tomi Karjalainen, Marco Bucci, Janne Isojärvi, and Lauri Nummenmaa. Automated segmentation of acute stroke lesions using a data-driven anomaly detection on diffusion weighted mri. *Journal of neuroscience methods*, 333: 108575, 2020.

L.G. Nyul, J.K. Udupa, and Xuan Zhang. New variants of a method of mri scale standardization. *IEEE Transactions on Medical Imaging*, 19(2):143–150, 2000. doi: 10.1109/42.836373.

Batyrkhan Omarov, Azhar Tursynova, Octavian Postolache, Khaled Gamry, Aidar Batyrbekov, Sapargali Aldeshov, Zhanar Azhibekova, Marat Nurtas, Akbayan Aliyeva, and Kadrzhan Shiyapov. Modified unet model for brain stroke lesion segmentation on computed tomography images. *Computers, Materials & Continua*, 71(3), 2022.

Moritz Roman Hernandez Petzsche, Ezequiel de la Rosa, Uta Hanning, Roland Wiest, Waldo Enrique Valenzuela Pinilla, Mauricio Reyes, Maria Ines Meyer, Sook-Lei Liew, Florian Kofler, Ivan Ezhov, David Robben, Alexander Hutton, Tassilo Friedrich, Teresa Zarth, Johannes Bürkle, The Anh Baran, Bjoern Menze, Gabriel Broocks, Lukas Meyer, Claus Zimmer, Tobias Boeckh-Behrens, Maria Berndt, Benno Ikenberg, Benedikt Wiestler, and Jan S. Kirschke. Isles 2022: A multi-center magnetic resonance imaging stroke lesion segmentation dataset, 2022. URL https://arxiv.org/abs/2206.06694.

M. Ranzato, F. J. Huang, Y. Boureau, and Y. LeCun. Unsupervised learning of invariant feature hierarchies with applications to object recognition. In *2007 IEEE Conference on Computer Vision and Pattern Recognition*, pages 1–8, June 2007. doi: 10.1109/CVPR.2007.383157.
Emilie MM Santos, Henk A Marquering, Olvert A Berkhemer, Wim H van Zwam, Aad van der Lugt, Charles B Majoie, Wiro J Niessen, and MR CLEAN investigators. Development and validation of intracranial thrombus segmentation on ct angiography in patients with acute ischemic stroke. *PLoS One*, 9(7):e101985, 2014.

Pir Masoom Shah, Hikmat Khan, Uferah Shafi, Saif ul Islam, Mohsin Raza, Tran The Son, and Hoa Le-Minh. 2d-cnn based segmentation of ischemic stroke lesions in mri scans. In *Advances in Computational Collective Intelligence: 12th International Conference, ICCCI 2020, Da Nang, Vietnam, November 30–December 3, 2020, Proceedings 12*, pages 276–286. Springer, 2020.

Xingjian Shi, Zhourong Chen, Hao Wang, Dit-Yan Yeung, Wai-Kin Wong, and Wang-chun Woo. Convolutional LSTM network: A machine learning approach for precipitation nowcasting. *CoRR*, abs/1506.04214, 2015. URL http://arxiv.org/abs/1506.04214.

Weiwei Tian, Dengwang Li, Mengyu Lv, and Pu Huang. Axial attention convolutional neural network for brain tumor segmentation with multi-modality mri scans. *Brain Sciences*, 13 (1), 2023. ISSN 2076-3425. doi: 10.3390/brainsci13010012. URL https://www.mdpi.com/2076-3425/13/1/12.

Manon L. Tolhuisen, Elena Ponomareva, Anne M. M. Boers, Ivo G. H. Jansen, Miou S. Koopman, Renan Sales Barros, Olvert A. Berkhemer, Wim H. van Zwam, Aad van der Lugt, Charles B. L. M. Majoie, and Henk A. Marquering. A convolutional neural network for anterior intra-arterial thrombus detection and segmentation on non-contrast computed tomography of patients with acute ischemic stroke. *Applied Sciences*, 10(14), 2020. ISSN 2076-3417. doi: 10.3390/app10144861. URL https://www.mdpi.com/2076-3417/10/14/4861.

Jang-Zern Tsai, Syu-Jyun Peng, Yu-Wei Chen, Kuo-Wei Wang, Hsiao-Kuang Wu, Yun-Yu Lin, Ying-Ying Lee, Chi-Jen Chen, Huey-Juan Lin, Eric Edward Smith, et al. Automatic detection and quantification of acute cerebral infarct by fuzzy clustering and histographic characterization on diffusion weighted mr imaging and apparent diffusion coefficient map. *BioMed research international*, 2014, 2014.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2023.

Huiyu Wang, Yukun Zhu, Bradley Green, Hartwig Adam, Alan Yuille, and Liang-Chieh Chen. Axial-deeplab: Stand-alone axial-attention for panoptic segmentation, 2020.

Yongjin Zhou, Weijian Huang, Pei Dong, Yong Xia, and Shanshan Wang. D-unet: A dimension-fusion u shape network for chronic stroke lesion segmentation. *IEEE/ACM Transactions on Computational Biology and Bioinformatics*, 18(3):940–950, 2021. doi: 10.1109/TCBB.2019.2939522.

Riaan Zoetmulder, Agnetha A. E. Bruggeman, Ivana Išgum, Efstratios Gavves, Charles B. L. M. Majoie, Ludo F. M. Beenen, Diederik W. J. Dippel, Nikkie Boodt, Sanne J. den Hartog, Pieter J. van Doormaal, Sandra A. P. Cornelissen, Yvo B. W. E. M. Roos,
Josje Brouwer, Wouter J. Schonewille, Anne F. V. Pirson, Wim H. van Zwam, Christiaan van der Leij, Rutger J. B. Brans, Adriaan C. G. M. van Es, and Henk A. Marquering. Deep-learning-based thrombus localization and segmentation in patients with posterior circulation stroke. *Diagnostics*, 12(6), 2022. ISSN 2075-4418. doi: 10.3390/diagnostics12061400. URL https://www.mdpi.com/2075-4418/12/6/1400.
## Appendix A. Lesion segmentation

In Table 5 we can see the results for the lesion. nnUnet is used as a reference model, using exactly the training and validation procedure available in the library (just including our splits). It outperforms LLSTM using just the proximal dataset (CHSF). So nnUnet is trained on both datasets, arriving at a Dice of 0.72 on average and close to a 100% percent detection rate. Notice that there is a considerable difference between the performances on the proximal (P) and distal (D) datasets. That model is used for the post-processing module in AttLLSTM.

<table><caption>**Table 5:** Test set results using LLSTM and nnUnet for lesion detection on both datasets.</caption><thead><tr><th rowspan="2">Model</th><th rowspan="2">Datasets</th><th colspan="2">False Positives</th><th colspan="2">False Negatives</th><th colspan="2">Dice</th><th colspan="2">Det.</th></tr><tr><th>Count</th><th>Size</th><th>Count</th><th>Size</th><th>P | D</th><th>P | D</th><th>P | D</th><th>P | D</th></tr></thead><tbody><tr><td>LLSTM</td><td>CHSF</td><td>0.8</td><td>82.8</td><td>1.4</td><td>47.5</td><td>0.70</td><td>1.00</td><td></td><td></td></tr><tr><td>nnUnet</td><td>CHSF</td><td>3.7</td><td>33.2</td><td>1.9</td><td>5.64</td><td>0.75</td><td>1.00</td><td></td><td></td></tr><tr><td><strong>nnUnet</strong></td><td>Both</td><td><strong>1.1</strong></td><td><strong>0.2</strong></td><td><strong>19.3</strong></td><td><strong>9.7</strong></td><td><strong>1.3</strong></td><td><strong>2.4</strong></td><td><strong>23.9</strong></td><td><strong>14.6</strong></td></tr></tbody></table>

## Appendix B. AttLLSTM robustness using control MRIs

The annotations for the control MRIs are available for MATAR dataset. This MRI is taken just after the treatment is given, to evaluate the effect of it. In some cases, the thrombus completely disappears, and in other cases, it can remain almost in the same position so another treatment is used. To formalize the different cases we can define the distance between the initial thrombus (thrombus₁) and the one after the treatment (thrombus₂):

$$D = \text{dist}(\text{thrombus}_1, \text{thrombus}_2) \qquad (13)$$

Depending on its value, we have a different treatment outcome:

* If $D = 0$: No treatment effect

* If $D = \infty$: The treatment makes the thrombus to disappear.

We evaluate our model in the most similar cases to the first MRI (where the distance is smaller) and the cases where the treatment effect is higher (the distance is bigger). We use the distance quartiles $q_{25}$ and $q_{75}$ to obtain these patients, only for the patients where the distance can be calculated (the thrombus is present in both MRIs). In Table 6 we can see the results.

Our model maintains a comparable performance when the treatment has less effect (smaller distance), detecting more than 80% of the patients but it just detects the 25% when the treatment produced a bigger effect (having a Dice of 0.51 and 0.1 respectively). The thrombus movement changes the relationship between the modalities; for instance, the relationship between the lesion and the thrombus is not the same anymore: the treatment can dissolve the thrombus but the lesion remains the same, as it is the damaged tissue.
<table><caption>**Table 6:** Results using DWI, SWAN, and PHASE to detect thrombus using AttLLSTM. For the control MRIs a total of 80 patients are used (the thrombus is still present) and 30 patients satisfy the distance conditions. All the results are obtained with no post-processing.</caption><thead><tr><th rowspan="2">Model</th><th rowspan="2">Evaluation (MATAR dataset)</th><th colspan="2">False Positives</th><th colspan="2">False Negatives</th><th rowspan="2">Dice</th><th rowspan="2">Det.</th></tr><tr><th>Count</th><th>Size</th><th>Count</th><th>Size</th></tr></thead><tbody><tr><td>AttLLSTM</td><td>1st MRI</td><td>0.6</td><td>63.3</td><td>0.3</td><td>79.4</td><td>0.55</td><td>0.91</td></tr><tr><td>AttLLSTM</td><td>Control MRI (D &lt; q<sub>25</sub>)</td><td>0.6</td><td>19.3</td><td>0.4</td><td>78.8</td><td>0.51</td><td>0.81</td></tr><tr><td>AttLLSTM</td><td>Control MRI (D &gt; q<sub>75</sub>)</td><td>1.1</td><td>65.1</td><td>1.0</td><td>175.1</td><td>0.10</td><td>0.25</td></tr></tbody></table>

Because of that, the post-processing techniques are not included in this evaluation. We can conclude that our model maintains a similar performance segmenting the thrombus when the treatment effect is low.