# Multicenter Validation of Synthetic FLAIR as a substitute for FLAIR Sequence in Acute Ischemic Stroke

Guillaume Hamon, Laurence Legrand, Ghazi Hmeydia, Guillaume Turc, Wagih Ben Hassen, Sylvain Charron, Clement Debacker, Olivier Naggara, Bertrand Thirion, Bailiang Chen, et al.

► To cite this version:

Guillaume Hamon, Laurence Legrand, Ghazi Hmeydia, Guillaume Turc, Wagih Ben Hassen, et al. Multicenter Validation of Synthetic FLAIR as a substitute for FLAIR Sequence in Acute Ischemic Stroke. European Stroke Journal, 2024, pp.23969873241263418. 10.1177/23969873241263418. inserm-04667549

HAL Id: inserm-04667549

https://inserm.hal.science/inserm-04667549v1

Submitted on 5 Aug 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

Multicenter Validation of Synthetic FLAIR as a substitute for
FLAIR Sequence in Acute Ischemic Stroke

<table><tr><td>Journal:</td><td>European Stroke Journal</td></tr><tr><td>Manuscript ID</td><td>ESO-24-0232.R1</td></tr><tr><td>Manuscript Type:</td><td>Original Research Article</td></tr><tr><td>Date Submitted by the Author:</td><td>28-May-2024</td></tr><tr><td>Complete List of Authors:</td><td>Hamon, Guillaume; Université Paris Cité, Institute of Psychiatry and Neuroscience of Paris (IPNP), INSERM U1266<br>Legrand, Laurence; Université Paris Cité, Institute of Psychiatry and Neuroscience of Paris (IPNP), INSERM U1266; GHU Paris Psychiatrie et Neurosciences, Neuroradiology<br>Hmeydia, Ghazi; GHU Paris Psychiatrie et Neurosciences, Neuroradiology<br>Turc, Guillaume; Université Paris Cité, Institute of Psychiatry and Neuroscience of Paris (IPNP), INSERM U1266; GHU Paris Psychiatrie et Neurosciences, Neurology<br>Ben Hassen, Wagih; Université Paris Cité, Institute of Psychiatry and Neuroscience of Paris (IPNP), INSERM U1266; GHU Paris Psychiatrie et Neurosciences, Neuroradiology<br>Charron, Sylvain; Université Paris Cité, Institute of Psychiatry and Neuroscience of Paris (IPNP), INSERM U1266<br>Debacker, Clement; Université Paris Cité, Institute of Psychiatry and Neuroscience of Paris (IPNP), INSERM U1266<br>Naggara, Olivier; Université Paris Cité, Institute of Psychiatry and Neuroscience of Paris (IPNP), INSERM U1266; GHU Paris Psychiatrie et Neurosciences, Neuroradiology<br>Thirion, Bertrand; CEA, NEUROSPIN<br>Chen, Bailiang; CHRU de Nancy, CIC-IT 1433<br>Lapergue, Bertrand; Department of Neurology, Foch Hospital<br>Oppenheim, Catherine; Université Paris Cité, Institute of Psychiatry and Neuroscience of Paris (IPNP), INSERM U1266; GHU Paris Psychiatrie et Neurosciences, Neuroradiology<br>Benzakoun, Joseph; Université Paris Cité, Institute of Psychiatry and Neuroscience of Paris (IPNP), INSERM U1266; GHU Paris Psychiatrie et Neurosciences, Neuroradiology</td></tr><tr><td>Keywords:</td><td>Acute Stroke, Ischemic Stroke, DWI-FLAIR mismatch, Synthetic FLAIR</td></tr><tr><td>Abstract:</td><td>Purpose: To evaluate performance of synthetic and real FLAIR for identifying early stroke in a multicenter cohort.<br>Methods: This retrospective study was conducted using DWI and FLAIR extracted from the Endovascular Treatment in Ischemic Stroke image registry (2017-2021). The database was partitioned into subsets according to MRI field strength and manufacturer, and randomly divided into training set (70%) used for model fine-tuning, validation set (15%), and test set (15%). In test set, five readers, blinded to FLAIR sequence type, assessed DWI-FLAIR mismatch using real and synthetic FLAIR. Interobserver agreement for DWI-FLAIR rating and concordance between synthetic and real FLAIR were evaluated with kappa statistics. Sensitivity</td></tr></table>
<table><tr><td></td><td>and specificity for identification of ≤4.5 hours AIS were compared in patients with known onset-to-MRI delay using McNemar's test. Results: 1454 complete MRI sets (1172 patients, median [IQR] age: 73 years [62-82]; 762 women) acquired on 125 MRI units were analyzed. In test set (207 MRI), interobserver reproducibility for DWI-FLAIR mismatch labeling was substantial for real and synthetic FLAIR (Fleiss κ=0.79 [95%CI:0.73-0.84] and 0.77 [95%CI:0.71-0.82], respectively). After consensus, concordance between real and synthetic FLAIR was excellent (κ=0.85 [95%CI:0.78-0.92]). In 141 MRI sets with known onset-to-MRI delay, diagnostic performances for ≤4.5 hours AIS identification did not differ between real and synthetic FLAIR (sensitivity: 60/71 [85%] vs. 59/71 [83%], P=.56; specificity: 65/70 [93%] vs. 65/70 [93%], P>.99). Conclusion: A deep-learning-based FLAIR fine-tuned on multicenter data can provide comparable performances to real FLAIR for early AIS identification. This approach may help reducing MR protocol duration and motion artifacts.</td></tr></table>

Note: The following files were submitted by the author for peer review, but cannot be converted to PDF. You must view these files (e.g. movies) online.

Hamon 24-0232 VA 2.eps
DECLARATIONS

1. Conflicting interests: The Authors declare that there is no conflict of interest.

2. Funding: This work was supported by a government grant managed by the French National Research Agency (ANR) as part of the future investment program integrated into France 2030, under grant agreement No. ANR-18-RHUS-0001.

3. Informed consent: Written informed consent was obtained, and data collection and analysis were approved by ETIS review board.

4. Ethical approval: This study was performed in line with the principles of the Declaration of Helsinki. The study was conducted under the Reference Methodology MR-004 for data protection relating to the processing of retrospective and prospective personal data implemented in the framework of research not involving the human person and approved by our clinical research committee.

5. Guarantor: CO and JB.

6. Contributorship: All authors reviewed and edited the manuscript and approved the final version of the manuscript.

7. Acknowledgements: ETIS Registry Investigators (a list of the ETIS Investigators is given in the Appendix).
# Title page

## Multicenter Validation of Synthetic FLAIR as a substitute for FLAIR Sequence in Acute Ischemic Stroke

Guillaume Hamon, MD; Laurence Legrand, MD, PhD; Ghazi Hmeydia, MD; Guillaume Turc, MD, PhD; Wagih Ben Hassen, MD, PhD; Sylvain Charron, PhD; Clement Debacker, PhD; Olivier Naggara, MD, PhD; Bertrand Thirion, PhD; Bailiang Chen, PhD; Bertrand Lapergue, MD, PhD; Catherine Oppenheim, MD, PhD; Joseph Benzakoun, MD, PhD; and on behalf of the ETIS Registry Investigators*.

From Université Paris Cité, Institute of Psychiatry and Neuroscience of Paris (IPNP), INSERM U1266 (G.Ha., J.B., L.L., G.T., W.B.H., S.C., C.D., O.N., C.O.), Paris, France; Department of Neurology, Foch Hospital (B.L.), Versailles Saint-Quentin-en-Yvelines University, Suresnes, France; Department of Neuroradiology (L.L., G.Hm., W.B.H., C.D., O.N., C.O., J.B.) and Department of Neurology (G.T.), GHU Paris Psychiatrie et Neurosciences, Paris, France; Université Paris-Cité, FHU Neurovasc (L.L., G.T., W.B.H., S.C., C.D., O.N., C.O., J.B.), Paris, France; PARIETAL team (B.T.), INRIA, Saclay, France; and CIC, Innovation Technologique, Université de Lorraine, INSERM 1433 (B.C.), Nancy, France.

* A list of the ETIS Investigators is given in the Appendix.

## Corresponding author contact details:

Joseph Benzakoun, Department of Neuroradiology, GHU Paris Psychiatrie et Neurosciences, Site Sainte-Anne, 1, rue Cabanis, 75014 Paris, France.

Email: j.benzakoun@ghu-paris.fr; Phone: +33145657710

Manuscript type: Original research

Word count: 2821

Data sharing statement: Data analyzed during the study were provided by a third party. Requests for data should be directed to the provider indicated in the Acknowledgments.
DECLARATIONS

1. Conflicting interests: The Authors declare that there is no conflict of interest.

2. Funding: This work was supported by a government grant managed by the French National Research Agency (ANR) as part of the future investment program integrated into France 2030, under grant agreement No. ANR-18-RHUS-0001.

3. Informed consent: Written informed consent was obtained, and data collection and analysis were approved by ETIS review board.

4. Ethical approval: This study was performed in line with the principles of the Declaration of Helsinki. The study was conducted under the Reference Methodology MR-004 for data protection relating to the processing of retrospective and prospective personal data implemented in the framework of research not involving the human person and approved by our clinical research committee.

5. Guarantor: CO and JB.

6. Contributorship: All authors reviewed and edited the manuscript and approved the final version of the manuscript.

7. Acknowledgements: ETIS Registry Investigators (a list of the ETIS Investigators is given in the Appendix)
**Structured Abstract**

**Purpose:** To evaluate performance of synthetic and real FLAIR for identifying early stroke in a multicenter cohort.

**Methods:** This retrospective study was conducted using DWI and FLAIR extracted from the Endovascular Treatment in Ischemic Stroke image registry (2017-2021). The database was partitioned into subsets according to MRI field strength and manufacturer, and randomly divided into training set (70%) used for model fine-tuning, validation set (15%), and test set (15%). In test set, five readers, blinded to FLAIR sequence type, assessed DWI-FLAIR mismatch using real and synthetic FLAIR. Interobserver agreement for DWI-FLAIR rating and concordance between synthetic and real FLAIR were evaluated with kappa statistics.

Sensitivity and specificity for identification of ≤4.5 hours AIS were compared in patients with known onset-to-MRI delay using McNemar's test.

**Results:** 1454 complete MRI sets (1172 patients, median [IQR] age: 73 years [62-82]; 762 women) acquired on 125 MRI units were analyzed. In test set (207 MRI), interobserver reproducibility for DWI-FLAIR mismatch labeling was substantial for real and synthetic FLAIR (Fleiss κ=0.79 [95%CI:0.73-0.84] and 0.77 [95%CI:0.71-0.82], respectively). After consensus, concordance between real and synthetic FLAIR was excellent (κ=0.85 [95%CI:0.78-0.92]). In 141 MRI sets with known onset-to-MRI delay, diagnostic performances for ≤4.5 hours AIS identification did not differ between real and synthetic FLAIR (sensitivity: 60/71 [85%] vs. 59/71 [83%], P=.56; specificity: 65/70 [93%] vs. 65/70 [93%], P>.99).

**Conclusion:** A deep-learning-based FLAIR fine-tuned on multicenter data can provide comparable performances to real FLAIR for early AIS identification. This approach may help reducing MR protocol duration and motion artifacts.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
**Key words:** Acute Stroke – Ischemic Stroke – DWI-FLAIR mismatch – Synthetic FLAIR

**Abbreviations**

AIS = acute ischemic stroke

AUC = Area under the receiver operating characteristic curve

DWI = Diffusion-weighted imaging

FLAIR = Fluid-attenuated inversion recovery

realFLAIR = real FLAIR

synthFLAIR = synthetic FLAIR

rSI = ratio of signal intensity
# Manuscript

## Introduction

In acute ischemic stroke (AIS), recanalization treatment decision either by intravenous thrombolysis or endovascular therapy, is highly impacted by imaging data and time constraints¹⁻³. While treatment window has expanded, notably to late stroke patients⁴⁻⁶, imaging has emerged as a cornerstone to assess collaterals⁷,⁸ and identify potentially salvageable brain parenchyma beyond pre-defined timeframes⁹. In this setting and given the variability of stroke growth, optimizing the imaging workflow remains crucial for providing the best-informed clinical decision support in the shortest possible time¹⁰. Depending on available modality, CT or MRI scan are typically performed before treatment decision, leading to comparable treatment delays in known-onset stroke¹¹.

In up to 27% of AIS¹², imaging is a used for the stroke onset time estimation when unknown, such as in wake-up and unwitnessed strokes. In these clinical situations, MRI may estimate the symptom onset by assessing the DWI-FLAIR mismatch, i.e., the presence of a diffusion restriction on the DWI without any significant signal change on the FLAIR sequence¹³,¹⁴.

A previous, single-center, study showed that a synthetic FLAIR sequence (hereafter referred as synthFLAIR) could be computed based on DWI sequence and be as clinically relevant as real FLAIR (realFLAIR) sequence to assess this mismatch pattern. Indeed, the T2-weighted acquisition embedded in the DWI sequence before application of diffusion gradients (b=0 sec/mm² [b0]) has been shown to contain information about stroke FLAIR visibility, but its analysis is limited in cortical regions where cerebrospinal fluid intensity is high¹⁵. The synthFLAIR sequence, whose calculation relies on these signal changes, should in contrast keep a good diagnostic value near cortical areas. Furthermore, its use should allow to reduce
the MR protocol time by avoiding realFLAIR acquisition, and be used as an alternative to
realFLAIR in case of motion artifacts in restless patients¹⁶. However, the original synthFLAIR
model was developed on a homogeneous single-center dataset from a single MRI unit, and
its generalizability on images issued from different MRI vendors, magnetic fields and variable
sequence parameters is unknown¹⁷. For large-scale application, the synthFLAIR model must
be adapted and validated in a multicentric environment to overcome a potential domain
shift¹⁸. Multicenter-level studies are now part of AI recommendations driven by the FDA¹⁹
and guideline initiatives, such as the CLAIM (Checklist for Artificial Intelligence in Medical
Imaging)²⁰, because of potential training bias and confounding factors that may impede
future broader utilization. This is even more sensitive regarding some unsupervised
generative deep-learning models that can be inflected by the data distribution during the
training phase, as provocatively demonstrated by some produced hallucinated images²¹.

Here, we used a fine-tuning procedure to adapt the synthFLAIR model for DWI data acquired
with different manufacturers and field strengths, in order to make it compatible with DWI
data issued from any manufacturers at either 1.5T or 3T. The aim of the study was to
compare the diagnostic value of the new fine-tuned synthFLAIR to the realFLAIR sequence
for DWI-FLAIR mismatch assessment and identification of stroke patients within 4.5 hours
from symptom onset in a national multicenter cohort.

# Materials and Methods

## Data Source

This retrospective study included adult patients who underwent recanalization treatment for
AIS enrolled in the prospective multicenter observational Endovascular Treatment in
Ischemic Stroke (ETIS) registry (ClinicalTrials Identifier: NCT03776877, approved by ethics
committee ID-RCB number: 2017-A03457-46) between 2017 and December 2021. Written
informed consent was obtained, and data collection and analysis were approved by ETIS
review board. ETIS is a clinical multicenter registry including patients who present an acute
stroke due to a large vessel occlusion with indication for mechanical thrombectomy, aged 18
years or older, with or without intravenous thrombolysis treatment<sup>22</sup>. ETIS-image is an
associated daughter image database collecting MR and CT imaging performed in these
patients and in the subgroup of centers transmitting raw image data. Inclusion criteria in our
study were: 1) availability of baseline MRI acquired before treatment and/or at early follow-
up; 2) availability of paired FLAIR and DWI sequences with low and high b-values (hereafter
referred to as b<sub>low</sub> and b<sub>high</sub>). Clinical data including age, sex, National Institutes of Health
Stroke Scale (NIHSS) score at admission, recanalization treatment and stroke onset-to-MRI
delay were also collected.

Selection of Data Subsets and Stratified Data Partition

FLAIR and DWI sequences were evaluated by one reader (resident, G.Ha.) on an ordinal scale ranging from 1 (low quality) to 3 (high quality) and inadequate MRI quality sets due to major artifacts were excluded. MRI datasets were partitioned into subsets according to the MRI field strength (1.5 or 3 Tesla) and manufacturers (corresponding to General Electric Healthcare, Siemens Healthineers and Philips Healthcare, hereafter respectively referred as manufacturers 1, 2 and 3). A specific subset was used for DWI sequences from one MRI unit with DWI acquired with a b<sub>low</sub> value of 50 s/mm² (rather than 0 s/mm² in all other subsets) issued from manufacturer 2 (Supplemental Table S1). MRI sets were randomly divided into train (70%), validation (15%), and test (15%) sets with stratified randomization on each of 7 subsets, MRI time (baseline or follow-up) and FLAIR quality (1, 2 or 3). Each MRI was assigned to one of the 42 (7×2×3) stratification groups depending on these 3 variables, and
stratified splitting was performed using dedicated function from scikit-learn library²³. In order to fulfill independence assumption for statistical tests in the test set, follow-up MRIs were excluded from analysis if the same patient was included twice in the test set.

## Data Preprocessing

RealFLAIR sequences were co-registered onto the corresponding DWI data using a 6-parameter rigid registration using Advanced Normalization Tools version 2.3.5 (https://stnava.github.io/ANTs). All MRI sets were either up-scaled or down-scaled into a standard 256 x 256 squared matrix size after signal normalization. Additional preprocessing steps, including data augmentation, were performed during training only according to previously published pipeline¹⁶ (Supplemental Methods 1.).

## Deep-learning Model Update and Domain Adaptation

The original synthFLAIR model¹⁶ ("Vanilla model") was adapted to require only DWI source images as input data (i.e., without apparent diffusion coefficient [ADC] map) (Supplemental Methods 2. and Supplemental Figure S1). In addition, the updated model was fine-tuned for each subset of the database. We favored this supervised domain adaptation strategy after an exploratory analysis using supervised²⁴ and unsupervised²⁵ methods evaluated on the validation set (Supplemental Methods 3.). Source code and model weights are freely available on http://github.com/***ANONYMIZED FOR REVIEW***. After training, synthFLAIR were generated in the test set by applying each fine-tuned model on DWI source images.

## Image Analysis

For DWI-FLAIR mismatch assessment, DWI images and either realFLAIR or synthFLAIR were presented in a random order to four neuroradiologists (G.Hm., J.B., L.L., and C.O.), with respectively, 6, 4, 11 and 21 years of experience in stroke imaging, and one resident (G.Ha.). Readers were blinded to data subset, FLAIR sequence type and onset-to-MRI delay. FLAIR
lesion was categorized as not visible (i.e., presence of DWI-FLAIR mismatch), visible (i.e., absence of DWI-FLAIR mismatch), or not assessable (because of extensive white matter disease or artifacts), following the Wake-Up Stroke trial specifications²⁶. One reader (G.Ha.) repeated the procedure for intraobserver reproducibility assessment after a 2-month washout period. Discrepancies between readers were resolved by consensus, either automatically when a majority agreement was reached (i.e., > 3 readers among 5 assigned the same rating), or after agreement of two senior readers for dubious cases.

Besides the visual analysis, the ratio of signal intensity (rSI) corresponding to the relative signal intensity of the ischemic lesion to the contralateral signal intensity¹⁶ computed on both realFLAIR and synthFLAIR was used to assess FLAIR status and detect ≤4.5 hours AIS.

## Statistical Analysis

Statistical analyses were performed with open-source software (R, version 4.0.1; R Foundation). Inter-observer agreement for DWI-FLAIR mismatch rating between realFLAIR and synthFLAIR of the 5 readers was assessed using the Fleiss Kappa (κ) coefficient. Intra-observer reproducibility for DWI-FLAIR mismatch assessment and concordance between realFLAIR and synthFLAIR were evaluated with the Cohen Kappa coefficient. Sensitivity, specificity, positive and negative predictive values of DWI-FLAIR mismatch for the identification of ≤4.5 hours AIS were compared between realFLAIR and synthFLAIR using McNemar's test and the relative predictive value method²⁷. The rSIs were compared between realFLAIR and synthFLAIR using Pearson correlation coefficients. Areas under the receiver operating characteristic curve (AUCs) for identifying ≤4.5 hours AIS were computed using rSI in stroke patients with known onset-to-MRI delay, and AUC comparison was performed using DeLong's method. Subgroup analysis was additionally performed in the 2-9hrs target window, and subgroup analyses were also performed at the subset level. Given
that DWI obtained with $b_{low}=50$ s/mm² might not contain complete T2-weighted information, we performed a subgroup analysis in all subsets with $b_{low}=0$ s/mm². Additional post-hoc analysis for interobserver DWI-FLAIR mismatch assessment was performed in restless patients excluded from initial data partition for inadequate FLAIR quality due to major artifacts. Values are expressed with interquartile range (IQR) and/or 95% Cis. The statistical significance threshold was $P<.05$.

# Results

## Patients and MRI Set Characteristics

In total, 1490 complete MRI sets were screened. After exclusion of 27 low-quality datasets (including 16 with inadequate FLAIR quality) (Figure 1), 1463 MRI sets from 1172 patients (762 women; median age: 73 years [IQR, 62-82]) acquired from 125 different MRI units were included. After data splitting, 9 follow-up MRIs issued from the same subject included twice in the test set were excluded, leading to a final number of 1454 analyzed MRIs, of which 1023 (70%) were used for training, 224 (15%) for validation and 207 (15%) for testing.

Among these MRI sets, 1013 (70%) were acquired before treatment and 441 (30%) at early follow-up. Clinical data are summarized in Table 1. MRI units and DWI and FLAIR sequence parameters in the 7 subsets are reported in Supplemental Table S1.

## Reproducibility of DWI-FLAIR Mismatch Assessment

Intraobserver reproducibility was not statistically different between realFLAIR and synthFLAIR ($\kappa=0.82$ [95%CI: 0.74-0.90] and 0.75 [0.66-0.84], respectively, $P=.27$).

Interobserver reproducibility was substantial for realFLAIR and synthFLAIR sequence and not significantly different ($\kappa=0.79$ [95%CI: 0.73-0.84] and 0.77 [0.71-0.82], respectively, $P=.58$) (Table 2).
The lowest interobserver reproducibility for synthFLAIR was obtained in Subset C (MRI sets with $b_{low}=50$ s/mm²), ($\kappa$=0.80 [95%CI: 0.62-0.99] and 0.58 [95%CI: 0.31-0.85] respectively for realFLAIR and synthFLAIR, $P=.22$). In MRI sets with $b_{low}=0$ s/mm² (n=186), interobserver reproducibility was substantial for both realFLAIR and synthFLAIR sequence ($\kappa$=0.78 [95%CI: 0.72-0.84] and $\kappa$=0.77 [95%CI: 0.71-0.83], respectively, $P=.89$).

### Concordance between realFLAIR and synthFLAIR for Mismatch Assessment

Depending on the reader, rating ranged from substantial to excellent ($\kappa$=0.70 to 0.83) (Table 3). After consensus, 4 MRI sets were considered non-assessable and were thus excluded from analysis. Dubious cases were resolved by consensus review in 14/203 (7%) realFLAIR and 19/203 (9%) synthFLAIR ($P=.38$).

After consensus, concordance between realFLAIR and synthFLAIR on the 203 assessable MRI sets was excellent ($\kappa$=0.85 [0.78-0.92]). Concordance was also excellent in the subgroup of 182 assessable MRI sets with $b_{low}=0$ s/mm² ($\kappa$=0.87 [95%CI: 0.80-0.94]). Illustrative examples are presented in Figure 2. Consensus assessment will be used in what follows.

### Identification of ≤4.5 hours AIS with Qualitative and Quantitative Analysis

Stroke onset-to-MRI delay was known in 141 of 203 assessable MRI sets from the test set.

Early stroke (≤4.5 hours from stroke onset) was classified accurately by DWI-FLAIR mismatch with 125/141 (89%) realFLAIR and 124/141 (88%) synthFLAIR sequences ($P>$.99). The sensitivity and specificity of the DWI-FLAIR mismatch for the identification of ≤4.5 hours AIS were not significantly different between realFLAIR and synthFLAIR (sensitivity: 60/71 [85%] vs. 59/71 [83%], $P$=0.56; specificity: 65/70 [93%] vs. 65/70 [93%], $P>$.99) (Table 4).

rSIs measured on realFLAIR and synthFLAIR were highly correlated (Pearson $r$=0.83 [95%CI: 0.78-0.87]). Pearson coefficient ranged from 0.77 (subset C, MRI sets with $b_{low}=50$ s/mm²) to
0.92 (subset D) and was equal to 0.83 [95%CI: 0.78-0.87] in the subgroup of all MRI sets with $b_{low}=0$ s/mm².

AUCs using rSI for identification of ≤4.5 hours AIS were not significantly different between realFLAIR and synthFLAIR (respectively 0.90 [95%CI: 0.85-0.96] and 0.86 [95%CI: 0.84-0.95], $P=.85$), nor were they significantly different in the subgroup of MRI sets with $b_{low}=0$ s/mm² (respectively 0.89 [95%CI: 0.83-0.95] and 0.91 [95%CI: 0.86-0.97], $P=.60$).

## Subgroups of onset-to-MRI delays in the 2-9hrs target window

Among 141 MRI sets where onset-to-MRI delay was known, 40 (28%) were performed in the 2-9hrs target window. Interobserver reproducibility was moderate for both realFLAIR and synthFLAIR sequence (Fleiss $\kappa=0.69$ [95%CI: 0.53-0.84] and 0.65 [0.49-0.81], respectively, $p=0.73$). After consensus, concordance between realFLAIR and synthFLAIR was substantial ($\kappa=0.75$ [0.53-0.98]). DWI-FLAIR mismatch was present in 31/40 (78%) realFLAIR and 27/40 (68%) synthFLAIR sequences ($P=0.13$). Both sequences had identical accuracy for classifying stroke delay (≤4.5hrs or > 4.5hrs from stroke onset) (31/40, 77%). The sensitivity and specificity of the DWI-FLAIR mismatch for the identification of ≤4.5 hours AIS were not significantly different between realFLAIR and synthFLAIR (sensitivity: 27/32 [84%] vs. 25/32 [78%], $P=0.16$; specificity: 4/8 [50%] vs. 6/8 [75%], $P=.16$). The 4/40 (10%) discordant labelings corresponded to subjects labeled as DWI-FLAIR mismatch using realFLAIR and no mismatch using synthFLAIR; 2 subjects had early stroke (2-hrs and 3.25-hrs) and 2 subjects had late stroke (4.7-hrs and 6.5-hrs).
## Post-hoc Analysis in Restless Patients

In 16 MRI sets excluded from data partition and main analysis (inadequate FLAIR quality because of major artifacts), interobserver reproducibility for mismatch assessment on synthFLAIR sequence between the 5 readers was substantial (κ=0.79 [95%CI: 0.59-0.98]). Illustrative cases are shown in Figure 3.

## Discussion

We have demonstrated that the original synthFLAIR model trained on homogenous single-center data could be adapted to compute clinically relevant synthFLAIR sequence on a multicenter cohort using a fine-tuning procedure.

To our knowledge, our study is the first to evaluate the adaptation of synthFLAIR in a large multicentric cohort with various manufacturers, and to propose a technical approach for this adaptation. Supervised deep-learning models' generalizability is indeed a controversial topic, as evidenced by the recent literature, which has raised concerns about the reliability of models when faced with new heterogeneous target domains in medical imaging²⁸. Our work suggests the feasibility of adapting a pre-trained model using a specific supervised domain adaptation method to overcome field strength and manufacturer shift from multicenter MRI data. To achieve such a change of scale, we first adjusted its architecture and discarded the ADC map as input. Indeed, the ADC map computed from the native DWI source image introduced signal variability by adding noise that likely affected multi-site translation without providing information relevant to the model's purpose (see Supplemental Methods 2.). More than 100 comprehensive and primary stroke centers, using both 1.5 and 3 T MRI units from three different manufacturers participated in recruiting stroke patients and acquiring MRI data in this study. Such a broad aggregate leads to data heterogeneity beyond
manufacturer and field strength, including MRI model subtypes and variability in imaging
protocol and acquisition parameters either on DWI or FLAIR sequences. These variations
faithfully reflect the daily clinical workflow and ensure real-world training conditions, thus
allowing a widespread applicability without the need to re-train models against other MRI
units at a later stage, thanks to the variety of MRI units initially included within each subset.

Moreover, we purposely kept all diffusion data, including DWI with ≥3 gradient-encoding
directions and b-value variations, without considering those variations as specific domains
(except for the b<sub>low</sub> variations). This data heterogeneity and these model development
strategies facilitate clinical portability across any 1.5 or 3T MRI units from 3 main
manufacturers. Each fine-tuned model will be made available as open-source software on
http://github.com/***ANONYMIZED FOR REVIEW***, in order to facilitate external
validation of our technique by individual teams with different MRI units.

Within the subgroup of DWI data with $b_{low}=50$ s/mm², the fine-tuned model presented the lowest interobserver reproducibility for DWI-FLAIR mismatch assessment as well as the lowest Pearson coefficient of the rSI compared to other subsets. This finding reinforces the underlying hypothesis that the synthFLAIR is mainly driven by the T2 contrast yielded by $b_{low}=0$ s/mm² images¹⁵ and explains lower performances with increasing $b_{low}$ values.

Subgroup analysis in the 2-9hrs target window subgroup did not show any differences in
interobserver reproducibility and diagnostic accuracy between realFLAIR and synthFLAIR.

Despite the lack of statistical significance, synthFLAIR tended to be more “conservative” than
realFLAIR in this subgroup analysis, since the 4 labeling discrepancies in this subgroup could
have led to avoid thrombolysis using synthFLAIR mismatch definition. It should however be
noted that 2 of these 4 discrepancies were justified (no DWI-FLAIR mismatch unisng
synthFLAIR for subjects with >4.5-hrs onset-to-MRI delay). As a consequence, if this
"conservative" feature is confirmed, synthFLAIR may be "safe" to use (reducing the risk to perform thrombolysis on late stroke and thus reducing iatrogenic hemorrhagic risk) at the expense of the number of treated patients. Until further research confirming or infirming this result, it seems thus acceptable to perform synthFLAIR only in situations where realFLAIR is deemed uninformative (restless patients).

Our initial domain definitions may be questionable as the subsets we selected, based on the MR field strengths and manufacturers, gathered very heterogeneous data, which may have led to a greater variety in the distribution of data than one would expect from a single domain as defined by the framework of the computer vision model. Data partition based on sequence parameters would have been useful to increase data homogeneity in each subset but using smaller groups would have increased the risk of overfitting²⁹, especially using the fine-tuning strategy.

One of the potential strengths of synthFLAIR is its ability to overcome motion artifacts in restless patients, minimizing artifacts given the short acquisition time of DWI. This is supported by the substantial interobserver reproducibility for DWI-synthFLAIR mismatch evaluation in the post-hoc analysis of restless patients and could have a major impact on clinical practice. However, DWI, and by extension synthFLAIR, can be prone to other artifacts, including geometric distortions and susceptibility artifacts associated with EPI techniques particularly near skull base and temporal lobes.

Our study has several limitations. First, the number of MRI sets for each manufacturer and field strength was unbalanced in the 7 different subsets. Its impact on each model's performance after application of the domain adaptation technique may be difficult to apprehend. However, the smallest group (subset E), trained on only 62 subjects, reached relatively good performances as compared to other groups (realFLAIR - synthFLAIR
concordance $κ$ = 0.83). The number of MRI sets required for fine-tuning the synthFLAIR model may thus be limited. Preliminary ablation study seems to point that at least 50 MRIs may be required for this fine-tuning (see Supplemental Methods 3.d).

Second, we chose to include both early and follow-up imaging in our study, even if clinical challenges and time constraints are very different in these two situations. This was performed in line with the reference study¹⁶, in order to increase data diversity for model training and hence improve model generalizability³⁰. Moreover, if reducing acquisition time is not crucial for follow-up imaging, synthFLAIR could still be a supplementary tool in restless patients presenting with kinetic artifacts on realFLAIR sequences. This approach poses however the question of patients with two MRI in the dataset. From a learning standpoint, considering early and late imaging as independent seems acceptable. Indeed, given the important differences in MR signal, acquisition plane and image orientation between early and late acquisitions, it seems unlikely that the model could learn patient-specific brain morphology to generate the synthFLAIR signal, particularly since the model is trained on a slice-wise basis. From a statistical standpoint, we removed follow-up imaging from the test set when the patient had also an early imaging, in order to account for statistical independence assumptions. In this study, the analysis based on the rSI showed that the AUC for the identification of ≤4.5 hours AIS on synthFLAIR tended to be lower than with the realFLAIR, without reaching statistical significance. The AUC difference between realFLAIR and synthFLAIR was however smaller in the validation dataset (with AUCs respectively equal to 0.85 and 0.84, see Supplemental Table S2, as compared to 0.90 and 0.86 in the test set), suggesting either some overfitting on the validation dataset or a variation due to data sampling. Moreover, quantitative evaluation of the rSI on the FLAIR sequence may represent an additional tool for treatment decisions, but cutoff values vary among studies¹⁴,³¹⁻³⁴ and
this parameter may not yet replace visual rating for DWI-FLAIR mismatch status in clinical
practice<sup>35,36</sup>.

As the duration of realFLAIR acquisition was not available for all exams, the impact of
accelerating the diagnosis process with synthFLAIR cannot be as clearly assessable as in a
single-center study<sup>16</sup>. Due to its retrospective design, the impact of synthFLAIR on patient
outcomes and management strategies could not be fully assessed beyond the potential
expected benefits in time reduction acquisition in this study. Our results cannot be
extrapolated to stroke mimics<sup>37</sup>, as we only included AIS patients. Further study still needs to
be done to extend this synthFLAIR sequence to other pathologies in the setting of suspected
AIS, although time management may be less decisive in those clinical situations.

Research perspectives could also include the development of a multi-task model
that, beyond generating a synthFLAIR sequence from the DWI, would also predict DWI-FLAIR
mismatch status<sup>38</sup> to enhance decision-making.

In conclusion, a single-center generative pre-trained model, fine-tuned across DWI
data from different MRI manufacturers and field strengths can generate clinically relevant
synthFLAIR that can compete with realFLAIR to assess DWI-FLAIR mismatch and identify
early AIS at a multicenter scale. Beyond reduction time of the stroke MR protocol without
the prior need for a real FLAIR sequence acquisition, synthFLAIR may be a promising
alternative to overcome motion artifacts in restless patients at the acute phase of stroke.
# Appendix: Endovascular Treatment in Ischemic Stroke (ETIS)

## Investigators

**01- Foch, Suresnes:** Bertrand LAPERGUE, Adrien WANG, Arturo CONSOLI, Oguzhan COSKUN, Federico DI MARIA, Silvia PIZZUTO, Alessandro SGRECCIA, Charline BENOIT, Lucas GORZA, David WEISENBURGER-LILE, Waliyde JABEUR, TCHIKVILADZE Maia, Serge EVRARD, Georges RODESCH

**02- Fondation Ophtalmologique Rotschild, Paris:** Raphaël BLANC, Michael OBADIA, Jean-Philippe DESILLES, Michel PIOTIN, Pierre SENERS, Stanislas SMAJDA, Simon ESCALARD, Benjamin MAIER, Candice SABBEN, Hocine REDJEM, Mikael MAZHIGI, Grace ADWAN, François DELVOYE, Amira AL RAAISI, William BOISSEAU

**03- Lyon:** Omer EKER, Tae-Hee CHO, Laurent DEREX, Julia FONTAINE, Laura MECHTOUFF, Norbert NIGHOGHOSSIAN, Elodie ONG, Lucie RASCLE, Roberto RIVA, François TURJMAN, Morgane LAUBACHER, Mehdi BEYRAGUED, Yves BERTHEZENE, Marc HERMIER, Ameli ROXANNA, Alexandre BANI-SADR, Andrea FILIP, Matteo CAPPUCCI

**04- Nantes:** Romain BOURCIER, Benjamin DAUMAS DUPORT, Pierre Louis ALEXANDRE, Cédric LENOBLE, Hubert DESAL, Solene DE GAALON, Benoît GUILLON, Cécile PRETERRE, Guillaume TESSIER, Arthur LIONNET

**05- Nancy:** Benjamin GORY, Lisa HUMBERTJEAN-SELTON, René ANXIONNAT, Anne-Laure DERELLE, Liang LIAO, Emmanuelle SCHMITT, Sophie PLANEL, Sébastien RICHARD, Gioia MIONE, Jean-Christophe LACOUR, Marian DOUARINOU, Emilien MICARD, Bailiang CHEN, Gérard AUDIBERT, Agnès MASSON, Lionel ALB, Marine BEAUMONT, Adriana TABARNA, Marcela VOICU, Grégoire BARTHEL, Iona PODAR, Madalina BREZEANU, Marie REITTER, François ZHU
**06- Bordeaux:** Gaultier MARNAT, Jean-Sébastien LIEGEY, Pierre BRIAU, Lisa PAPILLON, Igor SIBON, Xavier BARREAU, Jean PAPAXANTHOS, Jérome BERGE, Sabrina DEBRUXELLES, Stephane OLINDO, Mathilde POLI, Pauline RENOU, Sharmila SAGNIER, Thomas TOURDIAS, Thomas COURRET, Ludovic LUCAS, Malgorzata Milnerowicz

**07- Montpellier:** Cyril DARGAZANLI, Vincent COSTALAT, Isabelle MOURAND, Caroline ARQUIZAN, Lucas CORTI, Adrien TER SCHIPHORST, CAGNAZZO Federico, Imad DERRAZ, Mehdi MAHMOUDI, Pierre-Henri LEFEVRE, Grégory GASCOU

**08- Kremlin Bicêtre, Paris:** Laurent SPELLE, Jildaz CAROFF, Christian DENIER, Vanessa CHALUMEAU, Cristian MIHALEA, Nicolas LEGRIS, Augustin OZANNE, Leon IKKA, Olivier CHASSIN, Sophie GALLAS, Laura VENDITTI, Mariana SAROV, Jonathan CORTESE

**09- Rennes:** Jean-Christophe FERRE, Stephane VANNIER, Thomas RONZIERE, Maria Veronica LASSALLE, Jean-Yves GAUVRIT, Clément TRACOL, Abdelghani Fakhreddine BOUSTIA, Cécile MALRAIN, Edouard BEAUFRETON, Thibault LAPOTRE, Quentin ALIAS, Julien HISSIER, Maud GUILLEN, François EUGENE

**10- Amiens:** Cyril CHIVOT, Audrey COURSELLE, Elisa OUIN, Chantal LAMY, Kevin DELAFORGE, Manuel FERNANDEZ, Jérémie VIAL, Quentin LAFERTE, Xavier DESDOIT

**11- Brest:** Serge TIMSIT, Aurore JOURDAIN, Jean-Christophe GENTRIC, Julien OGNARD, Irina VIAKHIREVA, Jordan CORIS, Sabine PRUD'HON, François-Mathias MERRIEN, Denis MARECHAL, Marie BRUGUET, Pierre Yves ROUSSEAU, Philippe GOAS

**12- Caen:** Marion BOULANGER, Emmanuel TOUZE, Denis VIVIEN, Charlotte BARBIER, Romain SCHNECKENBURGER, Fabrizio SALARIS, Julien COGEZ, Sophie GUETTIER, Estelle LA PORTE, Jean BOUCHART

**13- Limoges:** Charbel MOUNAYER, Aymeric ROUCHAUD, Suzana SALEME, Géraud FORESTIER
**14- Pitié-Salpêtrière, Paris:** Frédéric CLARENCON, Charlotte ROSSO, Sara LEDER, Flore BARONNET, Sophie CROZIER, Anne LEGER, Kevin PREMAT, Shotar EIMAD, Stéphanie LENCK, Nader SOUROUR, Laure BOTTIN, Sam GHAZANFARI, Marion YGER, Sonia ALAMOWITCH, Stephen DELORME, Aymeric WITTLER, Christine VASSILEV

**16- Ste Anne, Paris:** Olivier NAGGARA, Guillaume TURC, Wagih BEN HASSEN, Basile KERLEROUX, Denis TRYSTRAM, Christine RODRIGUEZ-REGENT

**17- Rouen:**
Ozlem OZKUL-WERMESTER, Chrysanthi PAPAGIANNAKI, Evelyne MASSARDIER, Aude TRIQUENOT, Margaux LEFEBVRE, Julien BUREL

**18- Toulouse:** Alain VIGUIER, Christophe COGNARD, Anne Christine JANUEL, Jean-François ALBUCHER, Lionel CALVIERE, Jean-Marc OLIVOT, Jean DARCOURT, Nicolas RAPOSO, Fabrice BONNEVILLE, Guillaume BELLANGER, Louis FONTAINE, Philippe TALL

**19- Bayonne:** Frédéric BOURDAIN, Patricia BERNADY, Guillaume BALLAN, Stephanie BANNIER, Emmanuel ELLIE, Olivier LABEAU, Julia POTENZA, Antoine SOULAGES, Laurent LAGOARDE-SEGOT, Hélène CAILLIEZ, Louis VEUNAC, David HIGUE

**20- Vannes:** Anthony LEBRAS, Sarah ADAM, Benoit PEGAT, Arnaud LE GUEN, François CHEDEVILLE, Jérémy JOUAN

**21- Pau:** Stéphanie DEMASLES, Johann Sebastian RICHTER, Bruno Thierry BARROSO, Camille DAHAN, Alexis GONNET, Régis HUBRECHT, Zoé LEPINE, Hélène CASTAGNET, Raluca MARASESCU

**22- Grenoble:** Olivier HECK, Pauline CUISENIER, Olivier DETANTE, Isabelle FAVRE WIKI, Clémentine BONAZ, GARAMBOIS Katia, Loic LEGRIS, Adrian KASTLER, Kamel BOUBAGRA, Corentin BERTHET, Stéphane Charara
**024-Strasbourg:** Valérie WOLFF, Raoul POP, Véronique QUENARDELLE, Valérie LAUER, Irène PIERRE-PAUL, Roxana GHEOCA, Malwina TRZECIAK

**026-Reims:** Solène MOULIN, HUA Vi Tuan, Paolo PAGANO, Alexandre DOUCET, Christophe GELMINI, Pierre- François MANCEAU, Laurentiu PAIUSAN, Isabelle SERRE, Sébastien SOIZE, Thi Ngoc Phuong NGUYEN, Maher SAHNOUN, Nathalie CAUCHETEUX

**28- Clermont-Ferrand:** Anna FERRIER, Abderrahim ZERROUG, Ricardo MORENO, Emmanuel CHABERT, Elie LTEIF, Pauline PARIS, Nathalie BOURGOIS, Marie RAQUIN

**29- Angers:** Anne PASCO-PAPON, Jean Baptiste GIROT, Alderic LECLUSE, Sophie GODARD, Vincent L'ALLINEC

**30- Tours:** Kevin JANOT, Richard BIBI, Marie GAUDRON, Arnaud BRETONNIERE, Mariam ANNAN, Héloïse IFERGAN, Grégoire BOULOUIS, Marco PASI, Séverine DEBIAIS, Elisabeth MOLINIER

**32- Bar le Duc:** Anthony WIETRICH, Valérie RUCHE, Karine LAVANDIER

**33- Dijon:** Yannick BEJOT, Brivale LEMOGNE, Fédéric RICOLFI, Laura BAPTISTE, Pierre THOUANT, Gaulthier DULOQUIN, Pierre olivier COMBY

**34- Besançon:** Guillaume CHARBONNIER, Louise BONNET, Nicolas RAYBAUD, Benjamin BOUAMRA, Thierry MOULIN, Alessandra BIONDI
# Supplemental Material

## Supplemental Methods

1. Data pre-processing

2. Vanilla synthFLAIR model input adjustment

3. SynthFLAIR domain adaptation technique choice

## Tables S1-S2

## Figures S1-S3

## References 16, 24, 25
References

1. Powers WJ, Rabinstein AA, Ackerson T, et al. Guidelines for the Early Management of Patients With Acute Ischemic Stroke: 2019 Update to the 2018 Guidelines for the Early Management of Acute Ischemic Stroke: A Guideline for Healthcare Professionals From the American Heart Association/American Stroke Association. *Stroke* 2019; 50: e344–e418.

2. Thomalla G, Simonsen CZ, Boutitie F, et al. MRI-Guided Thrombolysis for Stroke with Unknown Time of Onset. *New England Journal of Medicine* 2018; 379: 611–622.

3. Escalard S, Gory B, Kyheng M, et al. Unknown-onset strokes with anterior circulation occlusion treated by thrombectomy after DWI-FLAIR mismatch selection. *Eur J Neurol* 2018; 25: 732–738.

4. Nogueira RG, Jadhav AP, Haussen DC, et al. Thrombectomy 6 to 24 Hours after Stroke with a Mismatch between Deficit and Infarct. *N Engl J Med* 2018; 378: 11–21.

5. Albers GW, Marks MP, Kemp S, et al. Thrombectomy for Stroke at 6 to 16 Hours with Selection by Perfusion Imaging. *N Engl J Med* 2018; 378: 708–718.

6. Ma H, Campbell BCV, Parsons MW, et al. Thrombolysis Guided by Perfusion Imaging up to 9 Hours after Onset of Stroke. *N Engl J Med* 2019; 380: 1795–1803.

7. Rao VL, Mlynash M, Christensen S, et al. Collateral status contributes to differences between observed and predicted 24-h infarct volumes in DEFUSE 3. *J Cereb Blood Flow Metab* 2020; 40: 1966–1974.

8. Vagal A, Aviv R, Sucharew H, et al. Collateral Clock Is More Important Than Time Clock for Tissue Fate. *Stroke* 2018; 49: 2102–2107.

9. Puig J, Shankar J, Liebeskind D, et al. From “Time is Brain” to “Imaging is Brain”: A Paradigm Shift in the Management of Acute Ischemic Stroke. *Journal of Neuroimaging* 2020; 30: 562–571.

10. Saver JL. Time Is Brain—Quantified. *Stroke* 2006; 37: 263–266.

11. Provost C, Soudant M, Legrand L, et al. Magnetic Resonance Imaging or Computed Tomography Before Treatment in Acute Ischemic Stroke. *Stroke* 2019; 50: 659–664.

12. Fink JN, Kumar S, Horkan C, et al. The stroke patient who woke up: clinical and radiological features, including diffusion and perfusion MRI. *Stroke* 2002; 33: 988–993.

13. Thomalla G, Cheng B, Ebinger M, et al. DWI-FLAIR mismatch for the identification of
patients with acute ischaemic stroke within 4·5 h of symptom onset (PRE-FLAIR): a multicentre observational study. *The Lancet Neurology* 2011; 10: 978–986.

14. Petkova M, Rodrigo S, Lamy C, et al. MR imaging helps predict time from symptom onset in patients with acute stroke: implications for patients with unknown onset time. *Radiology* 2010; 257: 782–792.

15. Geraldo AF, Berner L-P, Haesebaert J, et al. Does b1000-b0 Mismatch Challenge Diffusion-Weighted Imaging-Fluid Attenuated Inversion Recovery Mismatch in Stroke? *Stroke* 2016; 47: 877–881.

16. Benzakoun J, Deslys M-A, Legrand L, et al. Synthetic FLAIR as a Substitute for FLAIR Sequence in Acute Ischemic Stroke. *Radiology* 2022; 303: 153–159.

17. Yu AC, Mohajer B, Eng J. External Validation of Deep Learning Algorithms for Radiologic Diagnosis: A Systematic Review. *Radiol Artif Intell* 2022; 4: e210064.

18. Yan W, Huang L, Xia L, et al. MRI Manufacturer Shift and Adaptation: Increasing the Generalizability of Deep Learning Segmentation for MR Images Acquired with Different Scanners. *Radiol Artif Intell* 2020; 2: e190195.

19. Health C for D and R. Artificial Intelligence and Machine Learning in Software as a Medical Device. FDA, https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-machine-learning-software-medical-device (2023, accessed 6 August 2023).

20. Mongan J, Moy L, Kahn CE. Checklist for Artificial Intelligence in Medical Imaging (CLAIM): A Guide for Authors and Reviewers. *Radiology: Artificial Intelligence* 2020; 2: e200029.

21. Cohen JP, Luck M, Honari S. Distribution Matching Losses Can Hallucinate Features in Medical Image Translation. *arXiv:180508841 [cs]*, http://arxiv.org/abs/1805.08841 (2018, accessed 8 February 2022).

22. Hopital Foch. *Evaluation of Clinical and Imaging Criteria, and Plasma Biomarkers of Patients Receiving an Endovascular Treatment for an Acute Ischemic Stroke*. Clinical Trial Registration NCT03776877, clinicaltrials.gov, https://clinicaltrials.gov/study/NCT03776877 (2 August 2022, accessed 1 January 2024).

23. Pedregosa F, Varoquaux G, Gramfort A, et al. Scikit-learn: Machine Learning in Python. *Journal of Machine Learning Research*; 12.

24. Mo S, Cho M, Shin J. Freeze the Discriminator: a Simple Baseline for Fine-Tuning
GANs. arXiv:200210964 [cs, stat], http://arxiv.org/abs/2002.10964 (2020, accessed 10 February 2022).

25. Zhu J-Y, Park T, Isola P, et al. Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. arXiv:170310593 [cs], http://arxiv.org/abs/1703.10593 (2020, accessed 2 February 2022).

26. Thomalla G, Fiebach JB, Østergaard L, et al. A multicenter, randomized, double-blind, placebo-controlled trial to test efficacy and safety of magnetic resonance imaging-based thrombolysis in wake-up stroke (WAKE-UP). Int J Stroke 2014; 9: 829-836.

27. Moskowitz CS, Pepe MS. Comparing the predictive values of diagnostic tests: sample size and analysis for paired study designs. Clin Trials 2006; 3: 272-279.

28. Zech JR, Badgeley MA, Liu M, et al. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study. PLoS Med 2018; 15: e1002683.

29. Eche T, Schwartz LH, Mokrane F-Z, et al. Toward Generalizability in the Deployment of Artificial Intelligence in Radiology: Role of Computation Stress Testing to Overcome Underspecification. Radiol Artif Intell 2021; 3: e210097.

30. Gong Z, Zhong P, Hu W. Diversity in Machine Learning. IEEE Access 2019; 7: 64323-64350.

31. Cheng B, Boutitie F, Nickel A, et al. Quantitative Signal Intensity in Fluid-Attenuated Inversion Recovery and Treatment Effect in the WAKE-UP Trial. Stroke 2020; 51: 209-215.

32. Legge J, Graham A, Male S, et al. Fluid-Attenuated Inversion Recovery (FLAIR) Signal Intensity Can Identify Stroke Within 6 and 8 Hours. J Stroke Cerebrovasc Dis 2017; 26: 1582-1587.

33. Galinovic I, Puig J, Neeb L, et al. Visual and Region of Interest-Based Inter-Rater Agreement in the Assessment of the Diffusion-Weighted Imaging- Fluid-Attenuated Inversion Recovery Mismatch. Stroke 2014; 45: 1170-1172.

34. Song SS, Latour LL, Ritter CH, et al. A pragmatic approach using magnetic resonance imaging to treat ischemic strokes of unknown onset time in a thrombolytic trial. Stroke 2012; 43: 2331-2335.

35. Scheldeman L, Wouters A, Dupont P, et al. Diffusion-Weighted Imaging and Fluid-Attenuated Inversion Recovery Quantification to Predict Diffusion-Weighted Imaging-Fluid-Attenuated Inversion Recovery Mismatch Status in Ischemic Stroke With Unknown Onset.
*Stroke* 2022; 53: 1665–1673.

36. Cheng B, Brinkmann M, Forkert ND, et al. Quantitative measurements of relative fluid-attenuated inversion recovery (FLAIR) signal intensities in acute stroke for the prediction of time from symptom onset. *J Cereb Blood Flow Metab* 2013; 33: 76–84.

37. Danière F, Edjlali-Goujon M, Mellerio C, et al. MR screening of candidates for thrombolysis: How to identify stroke mimics? *J Neuroradiol* 2014; 41: 283–295.

38. Polson JS, Zhang H, Nael K, et al. Identifying acute ischemic stroke patients within the thrombolytic treatment window using deep learning. *Journal of Neuroimaging* 2022; 32: 1153–1160.
# Figures Legends

**Figure 1: Flow chart for MRI set and patient inclusion.**

DWI = diffusion-weighted imaging, FLAIR = fluid-attenuated inversion recovery, ETIS = Endovascular Treatment in Ischemic Stroke, baseline MRI sets = MRI acquired before recanalization treatment, follow-up MRI sets = MRI acquired at early follow-up. *9 follow-up MRI were excluded from test set for statistical analysis if the same patient was randomized twice in the test set, to fulfill independence assumption for statistical tests.

**Figure 2: Diffusion-weighted imaging (DWI)–fluid-attenuated inversion recovery (FLAIR) mismatch assessment using acquired FLAIR sequence (realFLAIR) and synthetic FLAIR (synthFLAIR) in AIS.**

(A) DWI-FLAIR mismatch in a 69-year-old man (subset B). On 1.5T DWI ($b_{high}=1000$ s/mm²) obtained 2 hours and 15 minutes from symptom onset, diffusion restriction is seen in the left middle cerebral artery territory without signal change on the realFLAIR and synthFLAIR.

(B) DWI-FLAIR mismatch in a 59-year-old man (subset G). On a 3T DWI ($b_{high}=1000$ s/mm²) obtained 4 hours from symptom onset, large diffusion restriction is seen in the right middle cerebral artery territory without significant signal change on the 3D realFLAIR and synthFLAIR. Note that the DWI based on EPI technique is prone to artifacts on the periphery, which results in less accurate frontal cortex delineation on synthFLAIR compared to realFLAIR.

(C) Absence of DWI-FLAIR mismatch in a 54-year-old man (subset A). On 1.5T DWI ($b_{high}=1000$ s/mm²) obtained 6 hours and 10 minutes from symptom onset, diffusion restriction is seen in the left middle cerebral artery territory, also visible on realFLAIR and synthFLAIR.

**Figure 3: Diffusion-weighted imaging (DWI)–fluid-attenuated inversion recovery (FLAIR) mismatch assessment in restless patients.**

(A) DWI-FLAIR mismatch assessable on the synthFLAIR generated from 1.5T DWI data in a 73-year-old woman 2 hours after symptom onset (subset D). On DWI, a slight diffusion restriction is seen in the right middle artery territory without signal change on the

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
synthFLAIR. The realFLAIR sequence was excluded from main analysis due to artifacts
whereas the synthFLAIR was of diagnostic value.

(B) AIS with hemorrhagic transformation in a 59-year-old woman (subset A). The realFLAIR sequence acquired with a 1.5T MRI presented with severe motion artifacts and was excluded from main analysis. Note the absence of these artifacts on the synthFLAIR sequence.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60

For Peer Review
# Tables

<table><caption>Table 1: Data Sets and Patient Characteristics</caption><thead><tr><th>Variable</th><th>Training Set</th><th>Validation Set</th><th>Test Set</th></tr></thead><tbody><tr><td>No. of MRI sets</td><td>1023 (70)</td><td>224 (15)</td><td>207 (15)</td></tr><tr><td>No. of MRI sets with known onset-to-MRI delay</td><td>723 (71)</td><td>157 (70)</td><td>143 (70)</td></tr><tr><td>Onset-to-MRI delay (h), per MRI set*</td><td>3.75 (2 – 28)</td><td>3.6 (2 – 26)</td><td>4.2 (1.9 – 29.3)</td></tr><tr><td>No. of patients</td><td>885</td><td>218</td><td>207</td></tr><tr><td>Age (y)</td><td>71 (59 – 83)</td><td>69 (59 – 97)</td><td>70 (60 – 98)</td></tr><tr><td>No. of women</td><td>524 (51)</td><td>122 (54)</td><td>116 (54)</td></tr><tr><td>NIHSS score at admission*</td><td>13.5 (7 – 19)</td><td>14 (9 – 20)</td><td>14 (7 – 20)</td></tr><tr><td>Intravenous thrombolysis†</td><td>403 (52)</td><td>95 (54)</td><td>86 (50)</td></tr><tr><td>Mechanical thrombectomy‡</td><td>667 (89)</td><td>151 (88)</td><td>164 (96)</td></tr><tr><td>No. of MRI sets in each subset</td><td></td><td></td><td></td></tr><tr><td>- Subset A: Manufacturer 1, 1.5T</td><td>242 (24)</td><td>53 (24)</td><td>48 (23)</td></tr><tr><td>- Subset B: Manufacturer 2, 1.5T<br>b<sub>low</sub>=0 s/mm<sup>2</sup></td><td>375 (37)</td><td>81 (36)</td><td>77 (37)</td></tr><tr><td>- Subset C: Manufacturer 2, 1.5T<br>b<sub>low</sub>=50 s/mm<sup>2</sup></td><td>105 (10)</td><td>23 (10)</td><td>21 (10)</td></tr><tr><td>- Subset D: Manufacturer 3, 1.5T</td><td>67 (6)</td><td>15 (7)</td><td>13 (6)</td></tr><tr><td>- Subset E: Manufacturer 1, 3T</td><td>62 (6)</td><td>14 (6)</td><td>12 (9)</td></tr><tr><td>- Subset F: Manufacturer 2, 3T</td><td>92 (9)</td><td>20 (9)</td><td>19 (9)</td></tr><tr><td>- Subset G: Manufacturer 3, 3T</td><td>80 (8)</td><td>18 (8)</td><td>17 (8)</td></tr></tbody></table>

Note. – Values are expressed as numbers of patients with percentages in parentheses, unless otherwise specified. NIHSS = National Institutes of Health Stroke Scale. *Data are expressed as median with interquartile range in parentheses. NIHSS data were missing in respectively 46 (5%), 16 (7%) and 9 (4%) patients in training, validation and test sets. † Missing data in respectively 114, 43 and 34 patients. ‡ Missing data in respectively 135, 47 and 37 patients.
<table><caption>Table 2: Interobserver Reproducibility for DWI-FLAIR Mismatch Assessment between the 5 Readers</caption><thead><tr><td>Test set</td><td>realFLAIR</td><td>synthFLAIR</td><td>P value</td></tr></thead><tbody><tr><td>Subset A: Manufacturer 1, 1.5T (n=48)</td><td>0.71 (0.59, 0.83)</td><td>0.70 (0.57, 0.83)</td><td>.904</td></tr><tr><td>Subset B: Manufacturer 2, 1.5T b<sub>low</sub>=0 s/mm<sup>2</sup> (n=77)</td><td>0.83 (0.74, 0.92)</td><td>0.80 (0.71, 0.89)</td><td>.655</td></tr><tr><td>Subset C: Manufacturer 2, 1.5T b<sub>low</sub>=50 s/mm<sup>2</sup> (n=21)</td><td>0.80 (0.62, 0.99)</td><td>0.58 (0.31, 0.85)</td><td>.219</td></tr><tr><td>Subset D: Manufacturer 3, 1.5T (n=13)</td><td>0.59 (0.26, 0.92)</td><td>0.73 (0.42, 1.00)</td><td>.485</td></tr><tr><td>Subset E: Manufacturer 1, 3T (n=12)</td><td>0.65 (0.36, 0.93)</td><td>0.76 (0.50, 1.00)</td><td>.494</td></tr><tr><td>Subset F: Manufacturer 2, 3T (n=19)</td><td>0.81 (0.57, 1.00)</td><td>0.77 (0.54, 1.00)</td><td>.796</td></tr><tr><td>Subset G: Manufacturer 3, 3T (n=17)</td><td>0.94 (0.82, 1.00)</td><td>0.85 (0.67, 1.00)</td><td>.427</td></tr><tr><td>All subsets except Subset C (n=186)</td><td>0.78 (0.72, 0.84)</td><td>0.77 (0.71, 0.83)</td><td>.89</td></tr><tr><td>All subsets (n=207)</td><td>0.79 (0.73, 0.84)</td><td>0.77 (0.71, 0.82)</td><td>.58</td></tr></tbody></table>

Note. – FLAIR = fluid-attenuated inversion recovery. Values are expressed as κ Fleiss coefficient with 95%CI in parentheses.
<table><caption>Table 3: Concordance of DWI-FLAIR Mismatch Assessment between realFLAIR and synthFLAIR</caption><thead><tr><th>Test set</th><th>Reader 1</th><th>Reader 2</th><th>Reader 3</th><th>Reader 4</th><th>Reader 5</th><th>After consensus</th></tr></thead><tbody><tr><td>Subset A: Manufacturer 1, 1.5T (n=48)*</td><td>0.80</td><td>0.76</td><td>0.75</td><td>0.79</td><td>0.53</td><td>0.91 (0.79-1.00)</td></tr><tr><td>Subset B: Manufacturer 2, 1.5T<br>b<sub>low</sub>=0 s/mm<sup>2</sup> (n=77)*</td><td>0.90</td><td>0.72</td><td>0.77</td><td>0.84</td><td>0.71</td><td>0.89 (0.79-0.99)</td></tr><tr><td>Subset C: Manufacturer 2, 1.5T<br>b<sub>low</sub>=50 s/mm<sup>2</sup> (n=21)</td><td>0.53</td><td>0.67</td><td>0.59</td><td>0.39</td><td>0.49</td><td>0.63 (0.25-1.00)</td></tr><tr><td>Subset D: Manufacturer 3, 1.5T (n=13)*</td><td>0.71</td><td>0.37</td><td>0.52</td><td>0.37</td><td>0.59</td><td>0.47 (0.01-0.98)</td></tr><tr><td>Subset E: Manufacturer 1, 3T (n=12)</td><td>0.83</td><td>0.83</td><td>0.53</td><td>0.82</td><td>1.00</td><td>0.82 (0.5-1.00)</td></tr><tr><td>Subset F: Manufacturer 2, 3T (n=19)</td><td>0.87</td><td>0.87</td><td>0.88</td><td>1.00</td><td>0.89</td><td>0.87 (0.63-1.00)</td></tr><tr><td>Subset G: Manufacturer 3, 3T (n=17)</td><td>0.87</td><td>0.64</td><td>0.85</td><td>0.87</td><td>0.8</td><td>0.87 (0.61-1.00)</td></tr><tr><td>All subsets except Subset C (n=186)*</td><td>0.85</td><td>0.72</td><td>0.75</td><td>0.82</td><td>0.71</td><td>0.87 (0.80-0.94)</td></tr><tr><td>All subsets (n=207)*</td><td>0.83</td><td>0.73</td><td>0.75</td><td>0.79</td><td>0.70</td><td>0.85 (0.78-0.92)</td></tr></tbody></table>

Note. – Values are expressed as κ values with 95%CI in parentheses when applicable. * In subsets A, B and D, respectively 2, 1 and 1 MRI sets were considered non-assessable and were thus excluded from analysis after consensus.
<table><caption>Table 4: Comparison of the Diagnostic Value of DWI-FLAIR Mismatch after Consensus Review between realFLAIR and synthFLAIR to Estimate Stroke Onset Time within 4.5 Hours</caption><thead><tr><td>Statistic</td><td>realFLAIR</td><td>synthFLAIR</td><td>P value</td></tr></thead><tbody><tr><td>Sensitivity</td><td>60/71 (85)</td><td>59/71 (83)</td><td>.56</td></tr><tr><td>Specificity</td><td>65/70 (93)</td><td>65/70 (93)</td><td>&gt;.99</td></tr><tr><td>Positive predictive value</td><td>60/65 (92)</td><td>59/64 (92)</td><td>.97</td></tr><tr><td>Negative predictive value</td><td>65/76 (86)</td><td>65/77 (84)</td><td>.57</td></tr></tbody></table>

Note. — Diagnostic value was computed in the 141 MRI datasets when stroke onset-to-MRI delay was available. Data are expressed as number of MRI sets, with corresponding percentages in parentheses. Sensitivity and specificity were compared using the McNemar test. Predictive values were compared using the relative predictive value method.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60

<figure><img src="image_3.png" /><figcaption>Figure 1: Flow chart for MRI set and patient inclusion.</figcaption></figure>

DWI = diffusion-weighted imaging, FLAIR = fluid-attenuated inversion recovery, ETIS = Endovascular Treatment in Ischemic Stroke, baseline MRI sets = MRI acquired before recanalization treatment, follow-up MRI sets = MRI acquired at early follow-up. *9 follow-up MRI were excluded from test set for statistical analysis if the same patient was randomized twice in the test set, to fulfill independence assumption for statistical tests.

149x230mm (300 x 300 DPI)
<figure><img src="image_4.png" /><figcaption>Figure 2: Diffusion-weighted imaging (DWI)-fluid-attenuated inversion recovery (FLAIR) mismatch assessment using acquired FLAIR sequence (realFLAIR) and synthetic FLAIR (synthFLAIR) in AIS.</figcaption></figure>

(A) DWI-FLAIR mismatch in a 69-year-old man (subset B). On 1.5T DWI (bhigh=1000 s/mm2) obtained 2 hours and 15 minutes from symptom onset, diffusion restriction is seen in the left middle cerebral artery territory without signal change on the realFLAIR and synthFLAIR.

(B) DWI-FLAIR mismatch in a 59-year-old man (subset G). On a 3T DWI (bhigh=1000 s/mm2) obtained 4 hours from symptom onset, large diffusion restriction is seen in the right middle cerebral artery territory without significant signal change on the 3D realFLAIR and synthFLAIR. Note that the DWI based on EPI technique is prone to artifacts on the periphery, which results in less accurate frontal cortex delineation on synthFLAIR compared to realFLAIR.

(C) Absence of DWI-FLAIR mismatch in a 54-year-old man (subset A). On 1.5T DWI (bhigh=1000 s/mm2) obtained 6 hours and 10 minutes from symptom onset, diffusion restriction is seen in the left middle cerebral artery territory, also visible on realFLAIR and synthFLAIR.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

86x105mm (300 x 300 DPI)
<figure><img src="image_5.png" /><figcaption>Figure 3: Diffusion-weighted imaging (DWI)-fluid-attenuated inversion recovery (FLAIR) mismatch assessment in restless patients.</figcaption></figure>

(A) DWI-FLAIR mismatch assessable on the synthFLAIR generated from 1.5T DWI data in a 73-year-old woman 2 hours after symptom onset (subset D). On DWI, a slight diffusion restriction is seen in the right middle artery territory without signal change on the synthFLAIR. The realFLAIR sequence was excluded from main analysis due to artifacts whereas the synthFLAIR was of diagnostic value.

(B) AIS with hemorrhagic transformation in a 59-year-old woman (subset A). The realFLAIR sequence acquired with a 1.5T MRI presented with severe motion artifacts and was excluded from main analysis. Note the absence of these artifacts on the synthFLAIR sequence.

86x71mm (300 x 300 DPI)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
# Supplemental Material

## Appendix E1. Data pre-processing

MRI sets were processed following previously published pipeline¹.

### a. DWI (b1000) - FLAIR co-registration: subject space registration

Within each MRI set, a 6-parameter rigid co-registration of the FLAIR images to the corresponding DWI images (b1000) was performed with Advanced Normalization Tools version 2.3.5 (https://stnava.github.io/ANTs).

### b. DWI (b0) - MNI T2 registration: world space registration

All DWI sequences were also registered to the MNI152 brain atlas (Montreal Neurologic Institute, Montreal, Canada) using a 12-parameter affine registration based on mutual information (MI) metrics using Advanced Normalization Tools.

### c. Brain mask extraction

For each subject a binary brain mask was computed through an Otsu method from both $b_{low}$ and $b_{high}$ images using the DIPY library². Brain segmentation was manually adjusted or re-computed with the Brain Extraction Tool from FSL toolbox (version 6.0, Functional Magnetic Resonance Imaging of the Brain, Oxford, UK, noncommercial software) after visual inspection.

### d. Stroke segmentation and stroke ROI generation

All acute ischemic stroke lesions were roughly segmented with a large ROI manually drawn using Mango software (version 4.0.1, Research Imaging Institute, San Antonio, Texas, noncommercial software). These loose ROIs were intersected with the previously computed brain mask to remove non-brain structures and were thresholded with an ADC cut-off value
<620 × 10⁻⁶ mm²/s to focus on the ischemic core. This final ROI is hereafter referred as stroke ROI.

### e. Contralateral stroke ROI (or mirror ROI) on DWI

The stroke ROI obtained was secondarily projected onto the MNI152 brain atlas, flipped onto the contralateral hemisphere, and then back-projected onto the subject space, using a nearest-neighbor interpolation. The mirror ROI was then thresholded with an ADC cut-off value <1200 × 10⁻⁶ mm²/s to exclude cerebrospinal fluid (CSF) voxels. To prevent ROI overlap and cross-contamination between voxels in case of bilateral stroke lesions, the mirror ROI also excluded stroke ROI's voxels.

### f. Learning weighting map generation

To focus the learning process of the deep-learning model on stroke regions, the computed binary brain mask intersected with the first mentioned rough manual stroke segmentation was used to define a weighting map with three regions: stroke region with high weighting (×100), brain region with normal weighting (×1) and areas outside the brain with low weighting (×0.01). This weighting map-based approach was added as an additional filter on the input MRI sets to adjust the learning process of the model during the training phase, as previously developed in the previously published synthFLAIR model¹, hereafter referred as the “Vanilla model”.

### g. Data harmonization

The ETIS-image database included many different MRI image matrix sizes. Because the pre-trained synthFLAIR model was designed to process a fixed image matrix size of 256 × 256 as an input data shape and to ease data handling, all MRI sets were either up-scaled or down-scaled into this standard squared matrix size, both in the x and y axis.
A Z-score normalization was performed to standardize the signal intensities of each sequence ($b_{low}$, $b_{high}$ and FLAIR) of the database by mean centering and variance scaling.

## h. Data augmentation

Data augmentation was performed to artificially increase the database and prevent overfitting. It included: (1) flipping image along x-axis, (2) rotating image between -10 and +10 degrees, (3) shearing image between -5 and +5 degrees, (4) translating image between -10 and +10 pixels along x-axis and y-axis, (5) scaling the image of a factor between 0.9 and 1.1, and (6) adjusting the image contrast and brightness between -30% and +30%.

## i. Quality control

For each of the aforementioned pre-processing steps, we performed a systematic qualitative visual inspection to detect data processing failure. Consequently, all image pre-processing steps were achieved and checked before feeding them into the network.

# Appendix E2. Vanilla synthFLAIR model input adjustment

## 1. Principles

First attempts to adapt synthFLAIR to a multicenter dataset were impaired by the necessity to compute the ADC map. Indeed, the Vanilla synthFLAIR model, trained on single-center data from a unique MRI unit had 3 inputs: $b_{low}$ and $b_{high}$ images as well as ADC maps. However, the high level of noise in ADC maps and contours led to a lack of model convergence when trained on multicenter data.

The single-center Vanilla model was thus adapted to only require DWI source image as input data without the need for an apparent diffusion coefficient map. We hypothesized that this quantitative ADC map computed from DWI native $b_{low}$ and $b_{high}$ images was not mandatory for
model training since it is linearly correlated with the $b_{low}$ and $b_{high}$ signals. Furthermore, it
might increase signal variability by adding unnecessary noise that could affect multi-site
translation.

**1. Training**

Accordingly, we re-trained a new updated synthFLAIR model based on a similar architecture,
removing only the apparent diffusion coefficient map from the Vanilla model's inputs. We
used the training dataset of the previous single-center cohort to develop the model. The
detailed Generator and Discriminator's layers remained unchanged, as previously described.
An illustration of the updated network architecture is provided in Supplemental Figure 1.

**2. Evaluation**

After complete training, we performed a quantitative analysis on the MRI sets with known
stroke-onset time of the validation set to compare the diagnostic value of the rSI of the
realFLAIR and synthFLAIR for the stroke onset-time estimation between the Vanilla and the
updated model to confirm our hypothesis, using the DeLong method. The area under the ROC
curve (AUC) for identification of ≤4.5 hours AIS was not significantly different between
realFLAIR and synthFLAIR computed with (Vanilla model) or without (updated model) the ADC
map on the original single-center dataset (respectively AUC=0.86 and 0.88, P=.26) and the
pertinence of this approach was then confirmed on the multicenter ETIS dataset
(Supplemental Figure 2).

# Appendix E3. SynthFLAIR domain adaption technique choice

**1. Context**

Several approaches for domain adaptation onto ETIS registry data were initially evaluated:
no domain adaptation (i.e., testing from scratch, reference method), supervised domain
adaptation (using fine-tuning for each manufacturer) and unsupervised domain adaptation (using CycleGAN). Models were all developed on TensorFlow v2.8.0. All training sessions were performed using a JupyterLab notebook running on a python framework v3.8.10 in a Linux environment (Ubuntu 20.04 LTS). Training was done on a customized computer station with 2 GPUs (TITAN RTX with 24 GB VRAM), on Intel i9-9900 K CPU with 8 cores and 16 threads, 62 GB RAM and a 500-GB Solid State Drive.

## 2. Models evaluation

### a. SynthFLAIR model's testing from scratch

We performed a straightforward transfer learning strategy by testing the performance of the pre-trained synthFLAIR model (either vanilla or updated) on the validation set issued from the ETIS registry without any adjustment (i.e., without previous training on any data subsets).

### b. Supervised domain adaptation: fine-tuning the models for each data subset

We performed a supervised domain adaptation experiment relying on fine-tuning the pre-trained models for each subset of the database, which therefore included manufacturer and MRI field strength specific training. We opted for a simple approach specifically developed for GANs models³ by freezing the layers of the Discriminator (i.e., to set these specific layers to non-trainable) and exclusively fine-tuned the weights of the Generator for each data subset. The learning rate of the ADAM optimizer during training was slightly adjusted (set to 1.10⁻⁵ compared to the 1.10⁻⁶ original one). Therefore, for the 7 subsets of our database, we trained and selected 7 new fine-tuned models based on their performance on each corresponding validation set.

### c. Unsupervised domain adaptation: cycle-GAN

We explored another scenario based on an unsupervised domain adaptation method. The general idea behind this experiment is to pre-process the input DWI data before feeding them
into the pre-trained model with the adjunction of a module which aims to transform any given DWI data of the database to "look like" the original distribution of the DWI data source (i.e., DWI acquired with the 1.5T GE MRI unit of single-center data on which the model was initially developed). In this setting, we developed a cycle-GAN-based model⁴ to perform an unsupervised image to image translation from multi-domains into one reference domain.

The cycle-GAN model architecture is composed of 2 Generators, one to perform a translation from DWI data of the ETIS database (=E, target domain) into DWI data acquired in Sainte-Anne hospital (=S, source domain), hereafter referred as "Translator DWI E > S" and one other to perform the opposite tasks (referred as "Translator DWI S > E"). The model architecture also included 2 Discriminators for each domain to determine whether the output of either Translator was from the corresponding native DWI data domain. To conduct such an experiment, we used as domain source data the previous data used to train the initial synthFLAIR model.

Once the cycle GAN pre-processing module was trained, we applied the Translator-module directly into the validation set. We used a series configuration, allowing the DWI source image to pass through 2 networks: first, the pre-processing cycle-GAN module (i.e., cycle DWI Translator E > S), then the pre-trained synthFLAIR model; so that the output of the former becomes the input of the latter.

### d. Final model selection

In an exploratory analysis to select the best model, we performed a correlation study between real rSI and synthetic rSI for each of the domain adaptation experiments in the validation set and compared the receiver operating curve (ROC) for the identification of AIS within 4.5 hours using the rSI of the realFLAIR and synthFLAIR sequence of each experiment in stroke patients
with available stroke onset-to-MRI delay, using the DeLong method. Quantitative results are available in Supplemental Table 2.

We selected the final model based on the supervised domain adaptation strategy relying on the updated FLAIR model, fine-tuned for each subset of the database (highest correlation coefficient (0.86) and AUC (0.84) for identification of early AIS).

### e. Ablation study

In order to evaluate the quantity of data needed for proper training, we performed an ablation study in the smallest subset (subset E, n=92). The validation subset (n=14) was left unchanged, and the model was iteratively trained with subsections of the complete training set (n=62), in increments of 10 MRIs (i.e., 10, 20, 30, 40, 50, 60 and 62 MRIs). Visual analysis of the correlation coefficient evolution between real rSI and synthetic rSI during training (Supplemental Figure 3) displays a certain degree of overfitting during training procedures with 10 to 40 MRIs, and similar performances for models trained with 50, 60 or 62 MRIs.
Supplemental References

1. Benzakoun J, Deslys M-A, Legrand L, Hmeydia G, Turc G, Hassen WB, et al. Synthetic FLAIR as a Substitute for FLAIR Sequence in Acute Ischemic Stroke. *Radiology* 2022;303:153–159.

2. Garyfallidis E, Brett M, Amirbekian B, Rokem A, van der Walt S, Descoteaux M, et al. Dipy, a library for the analysis of diffusion MRI data. *Front Neuroinform* 2014;8:8.

3. Mo S, Cho M, Shin J. Freeze the Discriminator: a Simple Baseline for Fine-Tuning GANs. *arXiv:200210964 [cs, stat]* 2020 Feb 28.

4. Zhu J-Y, Park T, Isola P, Efros AA. Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. *arXiv:170310593 [cs]* 2020 Aug 24.
Supplemental Figures

<figure><img src="image_6.png" /><figcaption>**Supplemental Figure 1: Schematic overview of the updated synthFLAIR model.**</figcaption></figure>

The generator input (x) corresponds to the DWI data, (i.e., paired $b_{low}$-$b_{high}$ images) associated with two additional inputs during the training phase: (1) a weighting factor (that allows the learning rate between stroke and non-stroke regions to be adjusted through a weighting map) and (2) the FLAIR sequence quality scalar.

The generator output, G(x), corresponds to the synthetic FLAIR sequence produced by the Generator. This synthetic image is compared to the realFLAIR sequence (y) through the computation of the L1-loss. In addition, a complementary loss, computed through the edges map of the real and the synthetic image, stressed the learning process on the edges (so-called edge-aware GAN). It ensured the correspondence between the 2 images by increasing the definition of the brain structures.

The discriminator input corresponds to one set of 4 images:

- a synthetic quadruplet corresponding to: DWI source data (x), synthFLAIR ($\mathcal{L}_\text{Synth}$) and synthetic FLAIR edges map ($\mathcal{L}_\text{ edges}$)

- a real quadruplet corresponding to: DWI source data (x), realFLAIR (y) and realFLAIR edges map

The discriminator output corresponds to a probability ranging between 0 (if it identifies the quadruplet as synthetic) and 1 (if the quadruplet input is real). The two networks are thus trained in opposite task and jointly optimized at each step through these losses, so that the synthFLAIR finally closely match the realFLAIR sequence.
## Supplemental Figure 2: ROC curve analysis for identification of ≤4.5 hours AIS using rSI.

Results are given in the validation sets from the original single-center cohort and from the new ETIS multicenter cohort. Comparative results between the Vanilla model, the updated model (without fine-tuning) and realFLAIR are shown. The area under the ROC curve was not significantly different between updated synthFLAIR and Vanilla synthFLAIR (0.88 vs. 0.86, *P*=.26) in the original single-center dataset but the difference was significant in the ETIS dataset (0.79 [0.71-0.91] vs. 0.72 [0.64-0.80], *P*=.036).

<figure><img src="image_7.png" /></figure>

## Supplemental Figure 3: Ablation study - evolution of correlation coefficient during training.

<figure><img src="image_8.png" /></figure>
# Supplemental Tables

<table><caption>## Supplemental Table 1: MRI Characteristics and DWI-FLAIR Sequence Imaging Parameters for Each Subset of the Database</caption><thead><tr><th>Subsets</th><th>A</th><th>B</th><th>C</th><th>D</th><th>E</th><th>F</th><th>G</th></tr></thead><tbody><tr><td>Manufacturer</td><td>1</td><td>2</td><td>2</td><td>3</td><td>1</td><td>2</td><td>3</td></tr><tr><td>Field Strength (T)</td><td>1.5</td><td>1.5</td><td>1.5</td><td>1.5</td><td>3</td><td>3</td><td>3</td></tr><tr><td>No. of MRI units*</td><td>31</td><td>41</td><td>1</td><td>16</td><td>10</td><td>20</td><td>6</td></tr><tr><td>No. of MRI sets</td><td>346</td><td>536</td><td>150</td><td>96</td><td>89</td><td>131</td><td>115</td></tr><tr><td>Number of MRI unit model types</td><td>6</td><td>6</td><td>1</td><td>5</td><td>6</td><td>5</td><td>3</td></tr><tr><td colspan="8"><strong>DWI sequences parameters</strong></td></tr><tr><td>DWI b<sub>low</sub> value</td><td>0</td><td>0</td><td>50</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>DWI b<sub>high</sub> values</td><td>1000, 1300, 1500</td><td>1000, 2000</td><td>1000</td><td>1000</td><td>1000, 1500</td><td>1000</td><td>1000, 1400, 2000</td></tr><tr><td>Repetition Time (msec)</td><td>7000 ± 1400</td><td>4800 ± 1700</td><td>5772 ± 586</td><td>4000 ± 500</td><td>5800 ± 2500</td><td>5500 ± 2400</td><td>3700 ± 900</td></tr><tr><td>Echo Time (msec)</td><td>87 ± 9</td><td>81 ± 12</td><td>85 ± 0</td><td>95 ± 7</td><td>67 ± 8</td><td>67 ± 13</td><td>81 ± 11</td></tr><tr><td>Acquisition Matrix†</td><td>256x256</td><td>[154x154 - 332x332]</td><td>[130x130 - 384x384]</td><td>[176x176 - 512x512]</td><td>256x256</td><td>[132x132 - 384x384]</td><td>[160x160 - 288x288]</td></tr><tr><td>Diffusion tensor imaging (&gt; 3 gradient encoding direction)</td><td>99 (28.6%)</td><td>0</td><td>0</td><td>0</td><td>7 (7.9%)</td><td>0</td><td>69 (60%)</td></tr><tr><td colspan="8"><strong>FLAIR sequences parameters</strong></td></tr><tr><td>Repetition Time (msec)</td><td>9426 ± 1100</td><td>7830 ± 1591</td><td>8734 ± 931</td><td>9378 ± 2260</td><td>9621 ± 1565</td><td>8339 ± 887</td><td>9926 ± 1947</td></tr><tr><td>Echo Time (msec)</td><td>144 ± 21</td><td>112 ± 50</td><td>127 ± 48</td><td>181 ± 91</td><td>133 ± 14</td><td>102 ± 44</td><td>183 ± 93</td></tr><tr><td>Inversion Time (msec) ‡</td><td>2430 ± 204</td><td>2309 ± 270</td><td>2456 ± 157</td><td>2349 ± 474</td><td>2437 ± 227</td><td>2404 ± 140</td><td>2624 ± 305</td></tr><tr><td>Fat saturation</td><td>45 (13%)</td><td>179 (33.4%)</td><td>5 (3.3%)</td><td>27 (38%)</td><td>32 (36%)</td><td>68 (51.9%)</td><td>97 (84.3%)</td></tr><tr><td>3D FLAIR</td><td>3 (0.9%)</td><td>19 (3.5%)</td><td>3 (2%)</td><td>21 (22%)</td><td>13 (14.6%)</td><td>3 (2.3%)</td><td>37 (32%)</td></tr></tbody></table>

Notes –T = Tesla. Values are expressed as mean ± SD.

*Missing information for center affiliation in 12 MRI sets from subset A, 7 from subset B, 19 from subset D and 4 from subset F.

†Acquisition matrix size is expressed as range [min - max].

‡Inversion Time was missing for respectively 53, 14, 24, 1 and 1 MRI sets in subset A, B, D, E, F.
<table><caption>Supplemental Table 2: Performances of synthFLAIR Sequence in the Validation Set of the ETIS Cohort</caption><thead><tr><th>Domain Adaptation technique</th><th>Vanilla synthFLAIR model</th><th>Updated synthFLAIR model</th><th>realFLAIR</th></tr></thead><tbody><tr><td colspan="4">Pearson coefficient correlation between synthetic rSI and real rSI (n=224)*</td></tr><tr><td>No DA (test from scratch)</td><td>0.64 [0.55-0.71]</td><td>0.76 [0.70-0.81]</td><td>-</td></tr><tr><td>Supervised DA (fine-tuning)</td><td>0.84 [0.80-0.88]</td><td><strong>0.86 [0.82-0.89]</strong></td><td>-</td></tr><tr><td>Unsupervised DA (cycle-GAN)</td><td>0.74 [0.68-0.80]</td><td>0.72 [0.82-0.78]</td><td>-</td></tr><tr><td colspan="4">AUC for the identification of AIS within 4.5 hours from symptom onset based on rSI (n=157)§</td></tr><tr><td>No DA (test from scratch)</td><td>0.72 [0.64-0.80]</td><td>0.79 [0.71-0.91]</td><td>0.85 [0.78-0.91]</td></tr><tr><td>Supervised DA (fine-tuning)</td><td>0.79 [0.72-0.86]</td><td><strong>0.84 [0.77-0.91]</strong></td><td>0.85 [0.78-0.91]</td></tr><tr><td>Unsupervised DA (cycle-GAN)</td><td>0.79 [0.72-0.87]</td><td>0.77 [0.69-0.85]</td><td>0.85 [0.78-0.91]</td></tr></tbody></table>

Notes – DA = domain adaptation, AUC = area under the ROC curve.

* Values are expressed as Pearson correlation coefficients with 95% CIs in parentheses.

§ Values are expressed as AUC with 95% CIs in parentheses.