# SMART: A Flexible, Interpretable, and Scalable Spatio-temporal Brain Atlas from High-Resolution Imaging Data

John Kalkhof, Marco Lorenzi

► To cite this version:

John Kalkhof, Marco Lorenzi. SMART: A Flexible, Interpretable, and Scalable Spatio-temporal Brain Atlas from High-Resolution Imaging Data. 2026. hal-05489861

HAL Id: hal-05489861

https://hal.science/hal-05489861v1

Preprint submitted on 4 Feb 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# SMART: A Flexible, Interpretable, and Scalable Spatio-temporal Brain Atlas
from High-Resolution Imaging Data

John Kalkhof

Inria Center at University Côte d'Azur
Sophia Antipolis, France

john.kalkhof@inria.fr

Marco Lorenzi

Inria Center at University Côte d'Azur
Sophia Antipolis, France

marco.lorenzi@inria.fr

## Abstract

We introduce SMART, a framework for building a flexible, interpretable, and scalable spatio-temporal brain atlas from time-series of high-resolution 3D medical images. Existing approaches to spatio-temporal atlas construction rely on black-box generative models that lack flexibility, limit interpretability, and struggle to scale to high-dimensional data. SMART overcomes these challenges through a novel approach to obtain spatio-temporal disentanglement by decoupling global group-wise disease dynamics from patient-specific local anatomical changes. Guided by anatomically inspired priors, SMART models interpretable global trajectories across brain regions through the optimization of region-specific differential equations. Local anatomical changes are estimated via dense diffeomorphic displacements parameterized by a flexible and scalable multi-scale Neural Cellular Automata. Evaluated on five longitudinal MRI datasets in Alzheimer's disease (ADNI-1/GO/2, OASIS-3, AIBL; >1,400 subjects), SMART achieves state-of-the-art forecasting accuracy and improved temporal consistency over adversarial and diffusion baselines. Our approach offers a new paradigm to flexible, interpretable, and scalable modeling of spatio-temporal changes in high-dimensional image time-series.

## 1. Introduction

In the field of computational anatomy, spatio-temporal atlases serve as powerful tools that model anatomical changes over time based on the analysis of longitudinal imaging data, such as magnetic resonance images (MRI) [8, 24, 30]. Specifically, within the realm of Alzheimer's disease research, these spatio-temporal brain atlases are designed to capture the long-term evolution of the disease potentially spanning over decades. This is accomplished by analyzing collections of image time-series from patients participating in short-term studies, where each patient contributes

a “snapshot” of their pathological evolution over the study period, which typically lasts from several months to a few years [1, 7, 16]. By leveraging these atlases, researchers can better map the progression of Alzheimer’s changes, enhancing our ability to tailor interventions and monitor treatment efficacy. Despite their promise, current approaches often fall short on three properties critical for clinical impact: flexibility to accommodate heterogeneous patient trajectories and irregular image sampling over time, interpretability in representing fine-grained anatomical dynamics across the entire brain, and scalability to operate at the highest possible resolution of 3D imaging data.

These limitations are due to the challenging setup encountered in typical imaging studies from large patient cohorts. First, spatio-temporal methods often assume a reference timeline and well-defined timestamps, conditions rarely met in practice due to heterogeneity in disease onset across patients, irregular follow-ups, and diversity of data acquisition protocols. Second, the high dimensionality of dense 3D brain images raises scalability concerns, and existing pipelines often resort to downsampling or simplified parameterizations that undercut fine-grained anatomical fidelity. Finally, modern approaches to image time-series analysis, especially those based on large generative models, are based on black-box architectures which fail to provide intelligible parameterisation of the spatio-temporal dynamics, thus providing little understanding of which anatomical regions change, how much, and when.

We address these challenges by developing a novel spatio-temporal analysis framework based on an end-to-end two-level atlas-building strategy: (i) at the **global** level, we estimate long-term **group-wise trajectories** across the entire disease time-span associated with volume loss for different key brain regions; (ii) at the **local** level, the global trajectories are used to condition a dense diffeomorphic deformation model to predict fine-grained anatomical changes associated with the estimated individual's severity and adapted to the associated image data. This design preserves **flexibility** by accommodating heterogeneous
<figure><img src="image_2.png" /><figcaption>Figure 1. Overview of the proposed SMART framework for modeling Alzheimer's disease progression. (A) Given a baseline MRI, SMART forecasts anatomical changes over a future interval t, estimating how an individual brain is expected to deform as the disease advances. (B) A global module learns a population-level spatio-temporal atlas, capturing average region-wise progression patterns and modeling how each structure evolves over time over the entire span of the disease. (C) Local voxel interactions, conditioned on these global progression signals, generate the deformation field that produces anatomically coherent follow-up scans.</figcaption></figure>

anatomies and uncertain timing across individuals, en-
hances **interpretability** through the global model of re-
gional trajectories, and improves **scalability** by operating
directly on the voxel grid without latent compression.

Modern deep learning models can predict future scans
from past observations, but they typically frame the task
as image synthesis, mapping a scan at time *t* to *t* + *δ*.
Adversarial methods (e.g., CounterSynth [26]) enforce re-
alism via discriminators, while diffusion and sequence-
aware approaches (e.g., BrLP [28, 29], SADM [41]) im-
prove temporal smoothness via iterative denoising. While
visually compelling, these pipelines compress **nonlinear,**
**region-specific** disease dynamics into high-dimensional la-
tent spaces, limiting **interpretability** and **control,** and often
incurring substantial computational cost.

By **separating how much, where, and when** the disease evolves from **how** those changes appear, **SMART** combines for the first time the estimation of *interpretable long-term temporal trajectories with anatomically faithful deformations predicting short-term individual anatomical changes.* Trained end-to-end, it avoids the instability and overhead of large generative systems, supports continuous-time interpolation and extrapolation, and reframes forecasting as self-registration of a brain to its own future state.

We evaluate SMART on five longitudinal MRI datasets
(ADNI-1/GO/2 [25], OASIS-3 [18], AIBL [9]), providing

5647 MRI images for 1400 individuals followed over an
average of 3.5 years. Across all benchmarks, our approach
achieves **state-of-the-art disease progression forecasting**
and improves spatio-temporal consistency over diffusion
and GAN baselines, with a **fraction of required param-**
**eters.** Beyond metrics, the learned atlas recovers realistic
and clinically meaningful patterns of neurodegeneration of-
fering a flexible, interpretable, and scalable representation
of neurodegenerative change in large longitudinal imaging
cohorts.

The full codebase will be released upon acceptance at
github.com/anonymized.

Contributions.

* **Interpretable spatio-temporal atlas.** We introduce a new formulation of disease progression modeling as a decoupled spatio-temporal atlas, where global region-wise differential equations represent global continuous trajectories of change, while a Neural Cellular Automata renders them as anatomically coherent individualized deformations. This separation exposes *how much, where, and when* neurodegeneration occurs.

* **Flexible temporal modeling.** A subject-specific time shift estimator aligns heterogeneous patients' timelines without requiring synchronized follow-ups, while the continuous-time formulation supports interpolation and
extrapolation across arbitrary intervals.

* **Scalable full-resolution forecasting.** A multi-scale conditioned NCA predicts voxel-level deformations directly on the 3D grid, achieving high anatomical fidelity and diffeomorphic consistency with orders-of-magnitude fewer parameters than generative baselines.

## 2. Related Work

Forecasting Alzheimer's disease progression from longitudinal MRI remains a central task in brain image analysis [10, 36]. The challenge lies in providing a plausible and accurate representation of slow, spatially varying anatomical changes across the entire disease time-span, while accounting for the large heterogeneity of patients' trajectories.

**Foundations: Deformation-based spatio-temporal atlases.** The development of spatio-temporal atlases of morphological changes in times-series of medical images is a central topic in the established field of computational anatomy [22]. Typical approaches from this domain rely on geometric foundations and formulate the task of atlas computation as the problem of interpolating time-series of images by optimizing diffeomorphic flows in suitable deformation spaces [7, 19, 23, 30]. In particular, by combining the principle of mixed-effect models within a geodesic regression framework, [8, 16] developed a deformation-based approach to spatio-temporal atlases accounting for individual time-warp parameters. This approach is inspired by the theory of self-modeling regression, and allows to account for subject-specific evolution pace and timing with respect to a group-wise global disease trajectory [7, 20]. Nevertheless, the reliance of these approaches to spatio-temporal atlas estimation on a standard diffeomorphic registration setting requires expensive optimization routines to solve a problem associated to Lagrangian or Eulerian flows of diffeomorphism [22]. This limitation thus prevents the development of brain atlases from high-dimensional and large-scale collections of medical images. Moreover, these methods often rely on specific choices for the metric of the ambient deformation space, leading to expensive and often unfeasible hyperparameter tuning steps.

**Flexibility: Learning-based progression models.** To overcome the limitations of classical deformation-based methods, the problem of estimating spatio-temporal atlases from time-series of images has been recently reformulated by adopting more flexible learning-based approaches. For example, *Adversarial spatiotemporal frameworks* such as DaniNet [32] treat forecasting as a 4D generation problem, producing entire image sequences rather than isolated frames. They couple adversarial objectives with biologically informed constraints to stabilize training and maintain 3D consistency. Auxiliary modules for super-resolution and transfer learning allow fine-tuning to individual trajectories, anticipating patient-specific evolution in high-

resolution space. *Deformation-based synthesis methods*, exemplified by CounterSynth [26], view disease progression as a field of diffeomorphic transformations. By conditioning these deformations on diagnostic labels, the model produces anatomically plausible counterfactuals that can augment downstream discriminative models. This line of work introduces a crucial idea: representing disease evolution not as pixel-wise hallucination, but as a structured transformation within the geometry of the brain. More recently, *diffusion and latent sequence models* [28, 41] have reframed longitudinal prediction as a denoising process unfolding over time. Sequence-aware transformers or auxiliary regional priors guide the diffusion dynamics, allowing inference from incomplete follow-ups and improving spatiotemporal coherence. These models emphasize smooth temporal trajectories but operate in compressed latent spaces, trading interpretability for scalability.

**Interpretability: Latent Dynamics for Image Time-series Forecasting.** A common strategy for modeling multivariate time-series is to parameterize their evolution through latent temporal dynamics, using architectures such as recurrent networks [33] or neural ODEs [5]. These models have been applied to build temporal atlases of brain change in Alzheimer's disease [1, 6, 17, 21]. By introducing *suitable constraints for the latent dynamics*, for example under the form of logistic parameterization, these approaches allow the estimation of interpretable and actionable progression patterns allowing us to shed light on the pathophysiology of the disease. Nevertheless, these approaches have been so far demonstrated in the low-dimensional regime only, due to the challenge of integrating dynamics parameterizing high-dimensional and heterogeneous spatio-temporal data.

**Scalability: Neural Cellular Automata in Medical Imaging.** Neural Cellular Automata (NCA) [23] extend the classical concept of Cellular Automata introduced by von Neumann [39], by defining discrete systems in which each cell updates its state based on local neighborhood rules. In NCAs, these rules are no longer hand-crafted but learned and differentiable, allowing the update dynamics themselves to be optimized from data. Architecturally, NCAs function as recurrent, residual convolutional networks that operate directly in spatial image coordinates rather than a latent space, yielding spatially coherent models whose dynamics resemble physical or biological processes such as diffusion or morphogenesis. Recent variants such as MED-NCA [13, 15] and NCA-Morph [31] have demonstrated the potential of multi-scale NCAs for 3D segmentation and registration, achieving U-Net-level accuracy with 2 orders of magnitude fewer parameters. Nevertheless, the application of NCA architectures to time-series analysis is currently lacking, as it requires linking *voxel-level local rules* to *global temporal progression signals*.
<figure><img src="image_3.png" /><figcaption>Figure 2. Overview of the proposed SMART framework for modeling individualized Alzheimer's disease progression. (A) A scan-specific module estimates each subject's temporal time shift, aligning heterogeneous timelines to a shared disease trajectory. (B) A global spatio-temporal atlas models continuous region-wise progression functions, capturing population-level dynamics of atrophy and expansion. (C–D) These progression signals condition a multi-scale Neural Cellular Automata (NCA) backbone that operates directly on voxel grids to predict anatomically coherent deformation fields without latent compression. (E) The resulting subject-specific deformations are applied to the baseline MRI to generate future anatomies, producing interpretable and temporally consistent forecasts.</figcaption></figure>

### 3. Methodology

Forecasting neurodegeneration requires reasoning about both the timing of disease progression and its anatomical manifestation. SMART models these aspects jointly through two complementary processes: (i) a temporal module that estimates how much change is expected over time, and (ii) a spatial module that renders these changes as anatomically plausible diffeomorphic deformations. This decomposition aligns with our goal of separating *how much*, *where*, and *when* change occurs from *how* it appears in image space.

Given a set of baseline MRI $x_{t_1}^j \in \mathbb{R}^{D \times H \times W}$ acquired at time $t_1^j$ for $N$ individuals, $1 \le j \le N$, we define their associated anatomical segmentation mask across $K$ brain regions as $s_{t_1}^j \in [0, 1]^{K \times D \times H \times W}$. By defining an individual target interval $\delta t = t_2^j - t_1^j$, the task is to predict the future individual anatomies $\hat{y}_{t_2}^j \in \mathbb{R}^{D \times H \times W}$ while preserving anatomical plausibility and temporal consistency (Fig. 2). Without loss of generality, from now on we will consider $t_1^j = t_1$ and $t_2^j = t_2$ for every individual $j$.

SMART estimates a spatio-temporal atlas of anatomical changes across the disease time-span by relying on four interconnected components:

1. **Time Shift Estimator (TSE):** infers a subject-specific temporal shift $v^j$ to align heterogeneous patients' timelines to represent a global group-wise smooth trajectory of disease evolution across the long-term disease time axis.

2. **Spatio-Temporal Atlas (STA):** models differential equations representing global smooth group-wise trajectories across brain regions describing the disease evolution along the entire disease time-span.

3. **Conditioned Med-NCA Backbone (CMNB):** predicts voxel-level velocity fields conditioned on the individual anatomy, on the individual time-shift, and on the regional group-wise dynamics of change.

4. **Diffeomorphic Warper (DW):** integrates the velocity field into a displacement map that diffeomorphically warps $x_{t_1}^j$ to obtain $\hat{y}_{t_2}^j$.

Together, these components form a fully differentiable pipeline leading to interpretable temporal dynamics and anatomically faithful spatial deformations.

#### 3.1. Subject-Specific Temporal Alignment

Longitudinal neuroimaging cohorts exhibit substantial heterogeneity in disease onset and progression rate. Chronological age $t_1^j$ is therefore a poor proxy for disease stage. To construct a coherent group-wise spatio-temporal atlas, we
must first map each subject's baseline anatomy to a common disease time axis. We achieve this by estimating a subject-specific temporal shift $v^j$, which places each anatomy on a common disease time axis.

Each shift $v^j$ is inferred from the subject's baseline segmentation mask $s_{t_1}^j$. From this mask, we compute normalized regional volumes

$$f_k^j = \frac{1}{DHW} \sum_p 1\{s_{t_1}^j(k, p) = 1\}, \quad k = 1, \dots, K, \quad (1)$$

where $p$ indexes voxels and $K$ is the number of anatomical regions. A lightweight MLP regresses the temporal shift $v^j$ from these features yielding a mapping from anatomy to disease stage, aligning all subjects to a shared spatio-temporal frame before learning population-level dynamics.

## 3.2. Spatio-Temporal Atlas of Regional Progression (STA)

Our goal is to learn a global, monotonic progression function $g(t; \theta) \in \mathbb{R}_{\ge 0}^K$ where each component $g_k(t)$ describes the instantaneous rate of change for region $k$ at disease time $t$. This function forms the core of the spatio-temporal atlas: it captures how the *average Alzheimer's brain* evolves across structures and over time. We parameterize $g$ using a small MLP with Softplus activation to enforce non-negativity, reflecting the monotonic nature of neurodegenerative changes (from normal to pathological states with no possibility of recovery). For subject $j$ and a future interval $\delta t$, the respective time shift $v^j$ is optimized to compute the expected cumulative change by integration of $g(t)$ over the subject-specific window $[v^j, v^j + \delta t]$:

$$I^j(\delta t; v^j) = \int_{v^j}^{v^j + \delta t} g(t; \theta) dt. \quad (2)$$

The resulting vector $I^j \in \mathbb{R}^K$ quantifies how much change each region is expected to undergo for subject $j$ over the chosen time horizon. Optimizing Equation (2) thus allows to estimate individualized time reparameterization: the time shifts $v^j$ positions each subject along the time-axis underlying the global evolution $g(\cdot)$, and the integration yields coherent group-wise trajectories across the entire disease time-span.

## 3.3. Voxel-Level Growth Conditioning

To estimate personalized voxel-level anatomical changes, SMART projects the individual global region-wise progression $I(\delta t; v^j)$ back into image space. For each voxel $p$, we build a dense growth map

$$G(p) = \sum_{k=1}^{K} I(\delta t; v^j) 1\{s_{t_1}^j(k, p) = 1\}, \quad (3)$$

assigning to each location the expected magnitude of anatomical change implied by the underlying labelled region.

This growth signal is concatenated with the one-hot segmentation to form the conditioning tensor

$$\text{Cond} = [G, S_{\text{one-hot}}] \in \mathbb{R}^{(1+K) \times D \times H \times W}, \quad (4)$$

which encodes both spatial context and region-specific evolution. Through a set of $1 \times 1 \times 1$ convolutions, Cond is converted into gating fields ($G_1, G_2$) that modulate the perception and transition layers of the NCA (Fig. 2D, Section 3.4). This conditioning ensures that each local update is anatomically grounded and consistent with the predicted global disease dynamics.

## 3.4. Conditioned Multi-Scale Neural Cellular Automaton

The Conditioned Med-NCA Backbone (CMNB) models local deformation as a spatially distributed dynamical system. Each voxel (“cell”) iteratively updates its hidden state through localized convolutional interactions, enabling efficient, memory-efficient modeling of large 3D volumes, as illustrated in Figure 2D. To capture both coarse and fine-scale effects, we use a hierarchical design: lower levels predict large-scale trends, while higher levels refine them. During training, to improve efficiency, backpropagation is performed on randomly selected 3D patches; during inference, the model operates seamlessly on full-resolution volumes (introduced by MED-NCA [14]).

At each iteration:

$$X_{t+1} = X_t + f_{\theta}(X_t; G_1, G_2) \odot m, m \sim \text{Bernoulli}(\rho),$$

where $\rho = 0.5$ introduces asynchronous updates that stabilize dynamics. The update function $f_{\theta}$ consists of two $3^3$ convolutions with skip connections and a gated MLP, ensuring stable information flow and coherent spatial behavior across scales.

## 3.5. Diffeomorphic Warping

The CMNB outputs a stationary velocity field $v_{SVF}$, integrated through scaling-and-squaring to obtain a diffeomorphic displacement field $u = \exp(v_{SVF})$ [3, 38]. The predicted follow-up scan $\hat{y}$ is generated by warping the baseline (Figure 2E):

$$\hat{y} = \text{Warp}(x, u), \quad (5)$$

using trilinear interpolation for intensities and nearest-neighbor interpolation for labels. In addition to the scaling-and-squaring integration method, an inverse-consistency loss,

$$\mathcal{L}_{\text{cycle}} = \frac{1}{2} (\|u \circ u^{-1}\|_2^2 + \|u^{-1} \circ u\|_2^2), \quad (6)$$
<table><caption>Table 1. Mean absolute error (MAE) of regional volumes as percentage of total brain volume (mean ± standard deviation).</caption><thead><tr><th>Method</th><th>Hip ↓</th><th>Amyg ↓</th><th>L.-Vent ↓</th><th>C. WM ↓</th><th>C. Cortex ↓</th><th>Inf.-L. Vent ↓</th></tr></thead><tbody><tr><td>SMART (ours)</td><td>0.031 ± 0.091</td><td><strong>0.015 ± 0.037</strong></td><td><strong>0.217 ± 0.379</strong></td><td><strong>0.652 ± 1.948</strong></td><td>0.568 ± 0.815</td><td><strong>0.017 ± 0.027</strong></td></tr><tr><td>CounterSynth [26]</td><td><strong>0.030 ± 0.092</strong></td><td><strong>0.015 ± 0.037</strong></td><td>0.315 ± 0.425</td><td>0.747 ± 1.919</td><td><strong>0.534 ± 0.751</strong></td><td>0.018 ± 0.028</td></tr><tr><td>BrLP [28]</td><td>0.052 ± 0.078</td><td>0.029 ± 0.037</td><td>0.736 ± 0.773</td><td>1.393 ± 3.015</td><td>3.943 ± 1.131</td><td>0.055 ± 0.062</td></tr><tr><th>Method</th><th>Cereb. WM ↓</th><th>Cereb. Cortex ↓</th><th>Thalamus ↓</th><th>Caudate ↓</th><th>Putamen ↓</th><th>Pallidum ↓</th></tr><tr><td>SMART (ours)</td><td>0.101 ± 0.343</td><td><strong>0.315 ± 1.190</strong></td><td><strong>0.043 ± 0.153</strong></td><td>0.032 ± 0.092</td><td><strong>0.033 ± 0.115</strong></td><td><strong>0.013 ± 0.037</strong></td></tr><tr><td>CounterSynth [26]</td><td><strong>0.100 ± 0.343</strong></td><td>0.324 ± 1.190</td><td>0.061 ± 0.151</td><td><strong>0.027 ± 0.091</strong></td><td>0.035 ± 0.115</td><td>0.015 ± 0.037</td></tr><tr><td>BrLP [28]</td><td>0.293 ± 0.348</td><td>0.614 ± 1.137</td><td>0.073 ± 0.150</td><td>0.071 ± 0.090</td><td>0.063 ± 0.112</td><td>0.028 ± 0.037</td></tr><tr><th>Method</th><th>Accumbens ↓</th><th>VentralDC ↓</th><th>Brain Stem ↓</th><th>3rd Vent ↓</th><th>4th Vent ↓</th><th>CSF ↓</th></tr><tr><td>SMART (ours)</td><td><strong>0.006 ± 0.013</strong></td><td><strong>0.026 ± 0.091</strong></td><td><strong>0.063 ± 0.253</strong></td><td><strong>0.009 ± 0.013</strong></td><td><strong>0.010 ± 0.023</strong></td><td>1.318 ± 3.943</td></tr><tr><td>CounterSynth [26]</td><td><strong>0.006 ± 0.013</strong></td><td>0.027 ± 0.091</td><td>0.065 ± 0.253</td><td>0.010 ± 0.014</td><td>0.012 ± 0.023</td><td><strong>1.303 ± 3.976</strong></td></tr><tr><td>BrLP [28]</td><td>0.011 ± 0.014</td><td>0.082 ± 0.084</td><td>0.142 ± 0.247</td><td>0.024 ± 0.019</td><td>0.031 ± 0.029</td><td>4.731 ± 5.179</td></tr></tbody></table>

promotes smooth, invertible mappings between baseline and follow-up anatomies.

Additional detailed architectural configurations and training parameters are provided in the supplementary material. The full codebase will be released upon acceptance at github.com/anonymized.

# 4. Experimental Setup and Results

We assess **SMART** on large collections of longitudinal MRI data to validate its flexibility, interpretability, and scalability in modeling and forecasting the evolution of brain changes in Alzheimer's patients. Experiments span five cohorts (ADNI-1/GO/2, OASIS-3, AIBL) and compare against the state-of-the-art atlas building generative methods based on adversarial (CounterSynth [26]) and diffusion-based (BrLP [28]) models on next-timepoint forecasting and multi-step rollouts. We use the official implementations of CounterSynth and BrLP, following the authors' default hyperparameters and recommended training protocols. We retrain all baselines on our preprocessed data at the same resolution and with matched splits for a fair comparison. To assess clinical plausibility, we further analyze the learned temporal atlas and subject-specific time shifts.

## 4.1. Datasets

We evaluate our method across five longitudinal MRI datasets from three major sources: ADNI-1, ADNI-GO, and ADNI-2 (826 subjects in total), OASIS-3 (350 subjects), and AIBL (229 subjects). Only participants with at least two timepoints were included. For clinical relevance, the ADNI cohorts were further filtered to include only subjects with at least one scan where the Aβ (amyloid-beta) level was below 192 pg/mL, indicating confirmed Alzheimer's pathology [34]. The OASIS-3 and AIBL datasets were retained without additional filtering due to their smaller cohort sizes. To ensure comparability across baselines, all scans were skull-stripped, aligned, and resampled to a com-

mon voxel grid of 128 × 160 × 128. All datasets were divided on a patient level, using an 80 % / 5 % / 15 % split for training, validation, and testing, respectively.

## 4.2. Preprocessing - Turboprep

For the preprocessing of the dataset we use the turboprep pipeline [27] that consists of six steps to ensure consistent normalisation and preprocessing across samples. It starts with intensity homogeneity normalisation N4 [37], Skull Stripping SynthStrip [11], Affine registration SyN [2] to template MNI152 [40], Segmentation of Brain Tissue SynthSeg 2.0 [4], Brain Mask Extraction and lastly, Intensity normalisation with White Stripe [35].

## 4.3. Quantitative Results

Table 2 shows that SMART achieves the best image-level accuracy (MSE 0.009, SSIM 0.858) despite using only 334K parameters. Modeling change as a deformation rather than synthesizing intensities yields sharper anatomy and more stable temporal evolution. Region-wise MAE scores (Table 1) highlight the main advantage: SMART captures anatomically relevant changes in most of the considered regions. On the contrary, CounterSynth consistently underestimates progression, while BrLP exhibits substantially higher errors across almost all regions. Together, these results confirm that our individual diffeomorphic deformation model informed by global progression trajectories leads to quantitatively better forecasts, both in terms of quality of anatomical detail and accuracy of volume changes.

<table><caption>Table 2. Results with mean ± standard deviation.</caption><thead><tr><th>Method</th><th>MSE ↓</th><th>SSIM ↑</th><th>N PARAM ↓</th></tr></thead><tbody><tr><td>SMART (ours)</td><td><strong>0.009 ± 0.018</strong></td><td><strong>0.858 ± 0.127</strong></td><td><strong>334K</strong></td></tr><tr><td>CounterSynth [26]</td><td>0.010 ± 0.018</td><td>0.851 ± 0.127</td><td>908K</td></tr><tr><td>BrLP [28]</td><td>0.082 ± 0.035</td><td>0.368 ± 0.048</td><td>799M</td></tr></tbody></table>
<figure><img src="image_4.png" /><figcaption>Figure 3. Qualitative comparison of SMART compared to the baselines BrLP and CounterSynth</figcaption></figure>

## 4.4. Qualitative Comparison

Figure 3 illustrates qualitative results comparing SMART with BrLP and CounterSynth across multiple timepoints. SMART produces anatomically coherent deformations that accurately follow the expected temporal progression of Alzheimer's disease. Structural changes such as ventricular enlargement and hippocampal atrophy evolve smoothly and remain consistent with the predicted disease stage.

In contrast, CounterSynth tends to underestimate progression, generating follow-ups that remain overly close to the baseline appearance. BrLP, while producing visually plausible images, often fails to preserve subject-specific anatomy and instead converges toward an averaged, population-level representation.

<figure><img src="image_5.png" /><figcaption>Figure 4. Distribution of predicted time shifts grouped by diagnostic category.</figcaption></figure>

## 4.5. Analysis of the Individual Time Shift Estimates

To assess the relevance of the estimated time shifts across subjects, we examine their correlation with key demographic and clinical factors.

Because detailed clinical measures are available only in the ADNI cohorts, correlations are computed exclusively on this subset. The estimated time shifts show a positive correlation with age (Pearson $r = 0.50, p = 4.5 * 10^{-24}$), indicating that the inferred disease stage increases with chronological aging. A similar positive correlation is observed with the ADAS-13 cognitive score ($r = 0.35, p = 1.3 * 10^{-9}$), suggesting that higher time shifts correspond to greater cognitive impairment. Conversely, the estimator exhibits a negative correlation with the $A\beta$ (amyloid-beta) level ($r = -0.43, p = 1.5 * 10^{-4}$), consistent with expected patterns of neurodegeneration. In line with these results, a box-plot analysis (Figure 4) of the predicted time shifts across diagnostic groups (controls: CN, mild cognitive impairment: MCI, Alzheimer's disease: AD) shows a clear ordering of estimated disease stage, with progressively higher time shifts for more advanced diagnoses. These results suggest that the time shift estimator learns a meaningful representation of disease progression that aligns with both demographic and clinical markers across the entire course of Alzheimer's pathology.

## 4.6. Learned Functional Progression

We analyze the long-term regional group-wise disease progression curves learned by the spatio-temporal atlas, visualized in Figure 5. In spite of the short follow-up time across subjects (average 43 months), SMART identifies global group-wise progression patterns spanning more than 20 years. The dynamics of progression across regions reveal biologically consistent patterns of change: regions known to be strongly affected by Alzheimer's disease, such as the hippocampus, lateral ventricles, inferior lateral ventricles, and third ventricle, exhibit pronounced trajectories of atrophy or
<figure><img src="image_6.png" /><figcaption>Figure 5. Region-wise progression functions learned by SMART. Each curve represents the estimated temporal trajectory of structural change for a specific brain region, parameterized as a continuous function of disease time. The functions capture distinct rates of atrophy or expansion, reflecting the heterogeneous progression dynamics of Alzheimer's disease across anatomical regions.</figcaption></figure>

expansion over time. In contrast, regions with known limited atrophy patterns in disease progression, such as cerebellum or brain stem, remain largely stable with low growth signals.

## 5. Discussion

SMART demonstrates that disentangling temporal disease dynamics from spatial manifestation enables interpretable and efficient modeling of neurodegenerative progression. By formulating brain forecasting as a deformation process conditioned on region-wise ODE trajectories, the model combines statistical progression modeling and diffeomorphic registration, offering a unified view of how anatomical change unfolds over time. The resulting spatio-temporal atlas captures biologically plausible evolution patterns while remaining computationally lightweight and scalable to full-resolution volumes. A current limitation is that the learned progression functions represent *cohort-averaged trajectories*, which do not yet account for *disease subtypes*. Extending the framework to learn personalized or subgroup-specific dynamics, potentially through hierarchical modeling, represents a natural next step. Nevertheless, the present formulation provides a solid foundation for transparent,

controllable disease modeling and marks a shift toward mechanistic representations of brain change in Alzheimer's progression forecasting.

## 6. Conclusion

SMART reframes disease progression forecasting as a structured spatio-temporal atlas estimation problem rather than direct image synthesis. By coupling continuous-time ODE dynamics with voxel-level Neural Cellular Automata, it learns anatomically coherent deformation fields that capture interpretable, region-specific trajectories of neurodegeneration. Across ADNI, OASIS-3, and AIBL, SMART surpasses adversarial and diffusion-based baselines in forecasting accuracy and temporal consistency. The learned atlas recovers hallmark Alzheimer's patterns, hippocampal shrinkage, ventricular expansion, and cortical thinning, and supports interpretable simulations of individual disease trajectories. By disentangling what changes, where, and when from how those changes appear, SMART establishes a flexible, interpretable, and scalable paradigm for modeling neurodegenerative processes, moving beyond image-based prediction toward a more mechanistic understanding of brain evolution.
# 7. Acknowledgments

This work was supported by the grants ANR-22-FAI1-0003 (TRAIN) and ANR-24-IAS2-0001 (Fed-Ops), by the PEPR Santé Numérique program through the grant ANR-22-PESN-0016 (REWIND), and by the 3IA Côte d'Azur initiatives, including ANR-19-P3IA-0002 (3IA Côte d'Azur) and ANR-23-IACL-0001 (3IA Côte d'Azur 2030).

**ADNI** [12]: Data collection and sharing for the Alzheimer's Disease Neuroimaging Initiative (ADNI) is funded by the National Institute on Aging (National Institutes of Health Grant U19AG024904). The grantee organization is the Northern California Institute for Research and Education. In the past, ADNI has also received funding from the National Institute of Biomedical Imaging and Bioengineering, the Canadian Institutes of Health Research, and private sector contributions through the Foundation for the National Institutes of Health (FNIH) including generous contributions from the following: AbbVie, Alzheimer's Association; Alzheimer's Drug Discovery Foundation; Aragon Biotech; BioClinica, Inc.; Biogen; Bristol-Myers Squibb Company; CereSpir, Inc.; Cogstate; Eisai Inc.; Elan Pharmaceuticals, Inc.; Eli Lilly and Company; EuroImmun; F. Hoffmann-La Roche Ltd and its affiliated company Genentech, Inc.; Fujirebio; GE Healthcare; IXICO Ltd.; Janssen Alzheimer Immunotherapy Research & Development, LLC.; Johnson & Johnson Pharmaceutical Research & Development LLC.; Lumosity; Lundbeck; Merck & Co., Inc.; Meso Scale Diagnostics, LLC.; NeuroRx Research; Neurotrack Technologies; Novartis Pharmaceuticals Corporation; Pfizer Inc.; Piramal Imaging; Servier; Takeda Pharmaceutical Company; and Transition Therapeutics.

**OASIS** [18]: Data were provided in part by OASIS-3: Longitudinal Multimodal Neuroimaging: Principal Investigators: T. Benzinger, D. Marcus, J. Morris; NIH P30 AG066444, P50 AG00561, P30 NS09857781, P01 AG026276, P01 AG003991, R01 AG043434, UL1 TR000448, R01 EB009352. AV-45 doses were provided by Avid Radiopharmaceuticals, a wholly owned subsidiary of Eli Lilly

# References

[1] Clément Abi Nader, Nicholas Ayache, Giovanni B Frisoni, Philippe Robert, Marco Lorenzi, and Alzheimer's Disease Neuroimaging Initiative. Simulating the outcome of amyloid treatments in alzheimer's disease from imaging and clinical data. *Brain communications*, 3(2):fcab091, 2021. 1, 3

[2] Brian B Avants, Charles L Epstein, Murray Grossman, and James C Gee. Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. *Medical image analysis*, 12(1):26–41, 2008. 6

[3] Guha Balakrishnan, Amy Zhao, Mert R Sabuncu, John Guttag, and Adrian V Dalca. Voxelmorph: a learning framework for deformable medical image registration. *IEEE transactions on medical imaging*, 38(8):1788–1800, 2019. 5

[4] Benjamin Billot, Douglas N Greve, Oula Puonti, Axel Thielscher, Koen Van Leemput, Bruce Fischl, Adrian V Dalca, Juan Eugenio Iglesias, et al. Synthseg: Segmentation of brain mri scans of any contrast and resolution without retraining. *Medical image analysis*, 86:102789, 2023. 6

[5] Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. *Advances in neural information processing systems*, 31, 2018. 3

[6] Ruoxuan Cui, Manhua Liu, Alzheimer's Disease Neuroimaging Initiative, et al. Rnn-based longitudinal analysis for diagnosis of alzheimer's disease. *Computerized Medical Imaging and Graphics*, 73:1–10, 2019. 3

[7] Michael C Donohue, Hélène Jacqmin-Gadda, Mélanie Le Goff, Ronald G Thomas, Rema Raman, Anthony C Gamst, Laurel A Beckett, Clifford R Jack Jr, Michael W Weiner, Jean-François Dartigues, et al. Estimating long-term multivariate progression from short-term data. *Alzheimer's & Dementia*, 10:S400–S410, 2014. 1, 3

[8] Stanley Durrleman, Xavier Pennec, Alain Trouvé, Guido Gerig, and Nicholas Ayache. Spatiotemporal atlas estimation for developmental delay detection in longitudinal datasets. In *International Conference on Medical Image Computing and Computer-Assisted Intervention*, pages 297–304. Springer, 2009. 1, 3

[9] Kathryn A Ellis, Ashley I Bush, David Darby, Daniela De Fazio, Jonathan Foster, Peter Hudson, Nicola T Lautenschlager, Nat Lenzo, Ralph N Martins, Paul Maruff, et al. The australian imaging, biomarkers and lifestyle (aibl) study of aging: methodology and baseline characteristics of 1112 individuals recruited for a longitudinal study of alzheimer's disease. *International psychogeriatrics*, 21(4): 672–687, 2009. 2

[10] Nick C Fox and Jonathan M Schott. Imaging cerebral atrophy: normal ageing to alzheimer's disease. *The Lancet*, 363 (9406):392–394, 2004. 3

[11] Andrew Hoopes, Jocelyn S Mora, Adrian V Dalca, Bruce Fischl, and Malte Hoffmann. Synthstrip: skull-stripping for any brain image. *NeuroImage*, 260:119474, 2022. 6

[12] Clifford R Jack Jr, Matt A Bernstein, Nick C Fox, Paul Thompson, Gene Alexander, Danielle Harvey, Bret Borowski, Paula J Britson, Jennifer L. Whitwell, Chadwick Ward, et al. The alzheimer's disease neuroimaging initiative (adni): Mri methods. *Journal of Magnetic Resonance Imaging: An Official Journal of the International Society for Magnetic Resonance in Medicine*, 27(4):685–691, 2008. 9

[13] John Kalkhof and Anirban Mukhopadhyay. M3d-nca: Robust 3d segmentation with built-in quality control. In *International Conference on Medical Image Computing and Computer-Assisted Intervention*, pages 169–178. Springer, 2023. 3

[14] John Kalkhof, Camila González, and Anirban Mukhopadhyay. Med-nca: Robust and lightweight segmentation with neural cellular automata. In *International Conference on In-
*formation Processing in Medical Imaging*, pages 705–716.
Springer, 2023. 5

[15] John Kalkhof, Niklas Ihm, Tim Köhler, Bjarne Gregori, and Anirban Mukhopadhyay. Med-nca: Bio-inspired medical image segmentation. *Medical Image Analysis*, page 103601, 2025. 3

[16] Igor Koval, Alexandre Bône, Maxime Louis, Thomas Lar-
tigue, Simona Bottani, Arnaud Marcoux, Jorge Samper-
Gonzalez, Ninon Burgos, Benjamin Charlier, Anne Bertrand,
et al. Ad course map charts alzheimer's disease progression.
*Scientific Reports*, 11(1):8020, 2021. 1, 3

[17] Dmitrii Lachinov, Arunava Chakravarty, Christoph Grechenig, Ursula Schmidt-Erfurth, and Hrvoje Bogunović. Learning spatio-temporal model of disease progression with neuralodes from longitudinal volumetric data. *IEEE Transactions on Medical Imaging*, 43(3):1165–1179, 2023. 3

[18] Pamela J LaMontagne, Tammie LS Benzinger, John C Morris, Sarah Keefe, Russ Hornbeck, Chengjie Xiong, Elizabeth Grant, Jason Hassenstab, Krista Moulder, Andrei G Vlassenko, et al. Oasis-3: longitudinal neuroimaging, clinical, and cognitive dataset for normal aging and alzheimer disease. medrxiv, pages 2019-12. 2019. 2, 9

[19] Marco Lorenzi, Nicholas Ayache, Giovanni B Frisoni, Xavier Pennec, and ADNI. Mapping the effects of a β 1-42 levels on the longitudinal changes in healthy aging: Hierarchical modeling based on stationary velocity fields. In *International Conference on Medical Image Computing and Computer-Assisted Intervention*, pages 663–670. Springer, 2011. 3

[20] Marco Lorenzi, Maurizio Filippone, Giovanni B Frisoni, Daniel C Alexander, Sébastien Ourselin, Alzheimer's Disease Neuroimaging Initiative, et al. Probabilistic disease progression modeling to characterize diagnostic uncertainty: application to staging and prediction in alzheimer's disease. *NeuroImage*, 190:56–68, 2019. 3

[21] Gerard Marti-Juan, Marco Lorenzi, Gemma Piella, Alzheimer's Disease Neuroimaging Initiative, et al. Mc-rvae: multi-channel recurrent variational autoencoder for multimodal alzheimer's disease progression modelling. *NeuroImage*, 268:119892, 2023. 3

[22] Michael I Miller, Alain Trouvé, and Laurent Younes. On the metrics and euler-lagrange equations of computational anatomy. *Annual review of biomedical engineering*, 4(1): 375–405, 2002. 3

[23] Alexander Mordvintsev, Ettore Randazzo, Eyvind Niklason, and Michael Levin. Growing neural cellular automata. *Distill*, 5(2):e23, 2020. 3

[24] Marc Niethammer, Yang Huang, and François-Xavier Vialard. Geodesic regression for image time-series. In *International conference on medical image computing and computer-assisted intervention*, pages 655–662. Springer, 2011. 1, 3

[25] Ronald Carl Petersen, Paul S Aisen, Laurel A Beckett, Michael C Donohue, Anthony Collins Gamst, Danielle J Harvey, CR Jack Jr, William J Jagust, Leslie M Shaw, Arthur W Toga, et al. Alzheimer's disease neuroimaging

initiative (adni) clinical characterization. *Neurology*, 74(3): 201–209, 2010. 2

[26] Guilherme Pombo, Robert Gray, M Jorge Cardoso, Sebastien Ourselin, Geraint Rees, John Ashburner, and Parashkev Nachev. Equitable modelling of brain imaging by counterfactual augmentation with morphologically constrained 3d deep generative models. *Medical Image Analysis*, 84: 102723, 2023. 2, 3, 6

[27] Lemuel Puglisi. Turboprep. https://github.com/LemuelPuglisi/turboprep, 2023. Accessed: 2025-10-08. 6

[28] Lemuel Puglisi, Daniel C Alexander, and Daniele Ravì. Enhancing spatiotemporal disease progression models via latent diffusion and prior knowledge. In *International Conference on Medical Image Computing and Computer-Assisted Intervention*, pages 173–183. Springer, 2024. 2, 3, 6

[29] Lemuel Puglisi, Daniel C Alexander, and Daniele Ravì. Brain latent progression: Individual-based spatiotemporal disease progression on 3d brain mris via latent diffusion. *Medical Image Analysis*, 2025. 2

[30] Anqi Qiu, Marilyn Albert, Laurent Younes, and Michael I Miller. Time sequence diffeomorphic metric mapping and parallel transport track time-dependent shape changes. *NeuroImage*, 45(1):S51–S60, 2009. 1, 3

[31] Amin Ranem, John Kalkhof, and Anirban Mukhopadhyay. Nca-morph: Medical image registration with neural cellular automata. arXiv preprint arXiv:2410.22265, 2024. 3

[32] Daniele Ravì, Stefano B Blumberg, Silvia Ingala, Frederik Barkhof, Daniel C Alexander, Neil P Oxtoby, Alzheimer's Disease Neuroimaging Initiative, et al. Degenerative adversarial neuroimage nets for brain scan simulations: Application in ageing and dementia. *Medical Image Analysis*, 75: 102257, 2022. 3

[33] Mike Schuster and Kuldip K Paliwal. Bidirectional recurrent neural networks. *IEEE transactions on Signal Processing*, 45 (11):2673–2681, 1997. 3

[34] Leslie M Shaw, Hugo Vanderstichele, Malgorzata Knapik-Czajka, Christopher M Clark, Paul S Aisen, Ronald C Petersen, Kaj Blennow, Holly Soares, Adam Simon, Piotr Lewczuk, et al. Cerebrospinal fluid biomarker signature in alzheimer's disease neuroimaging initiative subjects. *Annals of neurology*, 65(4):403–413, 2009. 6

[35] Russell T Shinohara, Elizabeth M Sweeney, Jeff Goldsmith, Navid Shiee, Farrah J Mateen, Peter A Calabresi, Samson Jarso, Dzung L Pham, Daniel S Reich, Ciprian M Crainiceanu, et al. Statistical normalization techniques for magnetic resonance imaging. *NeuroImage: Clinical*, 6:9–19, 2014. 6

[36] Paul M Thompson, Kiralee M Hayashi, Rebecca A Dutton, MING-CHANG CHIANG, Alex D Leow, Elizabeth R Sowell, Greig De Zubicaray, James T Becker, Oscar L Lopez, Howard J Aizenstein, et al. Tracking alzheimer's disease. *Annals of the New York Academy of Sciences*, 1097(1):183–214, 2007. 3

[37] Nicholas J Tustison, Brian B Avants, Philip A Cook, Yuanjie Zheng, Alexander Egan, Paul A Yushkevich, and James C Gee. N4itk: improved n3 bias correction. *IEEE transactions on medical imaging*, 29(6):1310–1320, 2010. 6
[38] Tom Vercauteren, Xavier Pennec, Aymeric Merchant, and Nicholas Ayache. Non-parametric diffeomorphic image registration with the demons algorithm. In *International conference on medical image computing and computer-assisted intervention*, pages 319–326. Springer, 2007. 5

[39] John Von Neumann, Arthur Walter Burks, et al. Theory of self-reproducing automata. 1966. 3

[40] Washington University HCPpipelines. Mni152_t1_1mm_brain.nii.gz. GitHub repository, https://github.com/Washington-University/HCPpipelines, 2025. Template file in global/templates; Accessed: 2025-10-08. 6

[41] Jee Seok Yoon, Chenghao Zhang, Heung-Il Suk, Jia Guo, and Xiaoxiao Li. Sadm: Sequence-aware diffusion model for longitudinal medical image generation. In *International Conference on Information Processing in Medical Imaging*, pages 388–400. Springer, 2023. 2, 3