SlowGuess/NeurIPS_2023_OCR huggingface dataset:
For every individual in the combined dataset of HC and AD+ groups, we processed their cortical thickness data  $\mathbf{x}$  through the model  $\Phi (\mathbf{x};\mathbf{C}_{\mathsf{HA}},\mathcal{H})$  where parameters  $\mathcal{H}$  were learned in the regression task on the data from HC group as described previously. Hence, the vector of mean of all final layer outputs for cortical thickness input  $\mathbf{x}$  is given by  $\mathbf{p} = \frac{1}{F}\sum_{f = 1}^{F}[\Phi (\mathbf{x};\mathbf{C}_{\mathsf{HA}},\mathcal{H}]_f$  and the VNN output is  $\hat{y} = \frac{1}{148}\sum_{j = 1}^{148}[\mathbf{p}]_j$ . Furthermore, we define the residual for feature  $a$  (or brain region represented by feature  $a$  in this case) as
$$
[ \mathbf {r} ] _ {a} \triangleq [ \mathbf {p} ] _ {a} - \hat {y}. \tag {7}
$$
Thus, (7) allows us to characterize the residuals with respect to the VNN output  $\hat{y}$  at the regional level, where brain regions are indexed by  $a$ . Henceforth, we refer to the residuals (7) as "regional residuals". Recall that these are evaluated for an individual with cortical thickness data  $\mathbf{x}$ . We hypothesized accelerated aging for an individual to be an aggregated effect of contributions from certain biologically plausible brain regions. The brain regions contributing to the observed higher  $\Delta$ -Age (procedure described in Section 3.3) could be characterized at a regional level by the analysis of regional residuals as defined in (7). Thus, the elements of the residual vector  $\mathbf{r}$  can potentially act as a biomarker that can enable the isolation of brain regions impacted due to accelerated aging in AD.
In our experiments, for a given VNN model, the residual vector  $\mathbf{r}$  was evaluated for every individual in the OASIS-3 dataset. Also, the population of residual vectors for the HC group is denoted by  $\mathbf{r}_{\mathrm{HC}}$  and that for individuals in the AD+ group by  $\mathbf{r}_{\mathrm{AD + }}$ . The length of the residual vectors is the same as the number of cortical thickness features (i.e.,  $m = 148$ ). Further, each element of the residual vector was mapped to a distinct brain region and ANOVA was used to test for group differences between individuals in HC and AD+ groups. Also, since elevation in  $\Delta$ -Age is the biomarker of interest in this analysis, we hypothesized that the brain regions that exhibited higher means for regional residuals for AD+ group than HC group would be the most relevant to capturing accelerated aging. Hence, the results are reported only for brain regions that showed elevated regional residual distribution in AD+ group with respect to HC group. Further, the group difference between AD+ and HC groups in the residual vector element for a brain region was deemed significant if it met the following criteria: i) the corrected  $p$ -value (Bonferroni correction) for the clinical diagnosis label in the ANOVA model was smaller than 0.05; and ii) the uncorrected  $p$ -value for clinical diagnosis label in ANCOVA model with age and sex as covariates was smaller than 0.05. See Appendix H for an example of this analysis.
Recall that 100 distinct VNNs were trained as regression models on different permutations of the training set of cortical thickness features from HC group. We used these trained models to establish the robustness of observed group differences in the distributions of regional residuals.
Deriving anatomical interpretability or regional profile for  $\Delta$ -Age from the robustness of findings from regional analyses. We performed the regional analysis described above corresponding to each trained VNN model and tabulated the number of VNN models for which a brain region was deemed to be associated with a significantly elevated regional residual for the AD+ group. A larger number of VNN models isolating a brain region as significant suggested that this region was likely to be a highly robust contributor to accelerated aging in the AD+ group.
Explaining the anatomical interpretability. We further investigated the relationship between regional residuals derived from VNNs and the eigenvectors of  $\mathbf{C}_{\mathrm{HA}}$  to determine the specific eigenvectors (principal components) of  $\mathbf{C}_{\mathrm{HA}}$  that were instrumental to anatomical interpretability. For this
purpose, we evaluated the inner product of normalized residual vectors (norm = 1) obtained from VNNs and the eigenvectors of the covariance matrix  $\mathbf{C}_{\mathrm{HA}}$  for the individuals in AD+ group. The normalized residual vector is denoted by  $\bar{\mathbf{r}}_{\mathrm{AD + }}$ . For every individual, the mean of the absolute value of the inner product  $|\langle \bar{\mathbf{r}}_{\mathrm{AD + }},\mathbf{v}_i\rangle |$  (where  $\mathbf{v}_i$  is the  $i$ -th eigenvector of  $\mathbf{C}_{\mathrm{HA}}$ ) was evaluated for the 100 VNN models.
Note that the VNNs were trained as described in Section 3.1 and hence, their ability to exploit the eigenvectors of the covariance matrix in a specific manner was learned when trained to predict chronological age for a healthy population. Hence, to gauge whether the anatomical interpretability was contingent on the learned ability of VNNs to exploit the eigenvectors of the covariance matrix from the procedure in Section 3.1, we also derived the anatomical interpretability for randomly initialized VNNs.
3.3 Individual-level brain age prediction
Finally, a scalar estimate for the brain age was obtained from the VNN regression output through a procedure consistent with the existing studies in this domain. Note that 100 VNNs provide 100 estimates  $\hat{y}$  of the chronological age for each subject. For simplicity, we consider  $\hat{y}$  to be the mean of these estimates. A systemic bias in the gap between  $\hat{y}$  and  $y$  may potentially exist when the correlation between  $\hat{y}$  and  $y$  is smaller than 1. Such a bias can confound the interpretations of brain age due to underestimation for older individuals and overestimation for younger individuals [45]. Therefore, to correct for this age-induced bias in  $\hat{y} - y$ , we adopted a linear regression model-based approach [45, 46]. Specifically, the following bias correction steps were applied on the VNN estimated age  $\hat{y}$  to obtain the brain age  $\hat{y}_{\mathrm{B}}$  for an individual with chronological age  $y$ :
Step 1. Fit a regression model for the HC group to determine scalars  $\alpha$  and  $\beta$  in the following model:
$$
\hat {y} - y = \alpha y + \beta . \tag {8}
$$
Step 2. Evaluate brain age  $\hat{y}_{\mathrm{B}}$  as follows:
$$
\hat {y} _ {\mathrm {B}} = \hat {y} - (\alpha y + \beta). \tag {9}
$$
The gap between  $\hat{y}_{\mathrm{B}}$  and  $y$  is the  $\Delta$ -Age and is defined below. For an individual with cortical thickness  $\mathbf{x}$  and chronological age  $y$ , the brain age gap  $\Delta$ -Age is formally defined as
$$
\Delta - \mathrm {A g e} \triangleq \hat {y} _ {\mathrm {B}} - y, \tag {10}
$$
where  $\hat{y}_{\mathsf{B}}$  is determined from the VNN age estimate  $\hat{y}$  and  $y$  according to steps in (8) and (9). The age-bias correction in (8) and (9) was performed only for the HC group to account for bias in the VNN estimates due to healthy aging, and then applied to the AD+ group. Further, the distributions of  $\Delta$ -Age were obtained for all individuals in HC and AD+ groups.
$\Delta$ -Age for  $\mathrm{AD+}$  group was expected to be elevated as compared to HC group as a consequence of elevated regional residuals derived from the VNN model. To elucidate this, consider a toy example with two individuals of the same chronological age  $y$ , with one belonging to the  $\mathrm{AD+}$  group and another to the HC group. Equation (9) suggests that their corresponding VNN outputs (denoted by  $\hat{y}_{\mathrm{AD+}}$  for individual in the  $\mathrm{AD+}$  group and  $\hat{y}_{\mathrm{HC}}$  for individual in the HC group) are corrected for age-bias using the same term  $\alpha y + \beta$ . Hence,  $\Delta$ -Age for the individual in the  $\mathrm{AD+}$  group will be elevated with respect to that from the HC group only if the VNN prediction  $\hat{y}_{\mathrm{AD+}}$  is elevated with respect to  $\hat{y}_{\mathrm{HC}}$ . Since the VNN predictions  $\hat{y}_{\mathrm{AD+}}$  and  $\hat{y}_{\mathrm{HC}}$  are proportional to the unweighted aggregations of the estimates at the regional level [see (4) and (5)], larger  $\hat{y}_{\mathrm{AD+}}$  with respect to  $\hat{y}_{\mathrm{HC}}$  can be a direct consequence of a subset of regional residuals [see (7)] being robustly elevated in  $\mathrm{AD+}$  group with respect to HC group across the 100 VNN models. When the individuals in this example have different chronological age, the age-bias correction is expected to remove any variance due to chronological age in  $\Delta$ -Age. We also verified that the differences in  $\Delta$ -Age for  $\mathrm{AD+}$  and HC group were not driven by age or sex differences via ANCOVA with age and sex as covariates.
4.1 Chronological age prediction for the HC group
The performance achieved by the VNNs on the chronological age prediction task for the HC group has been summarized over the 100 nominal VNN models. VNNs achieved a mean absolute error (MAE)
of  $5.82 \pm 0.13$  years and Pearson's correlation of  $0.51 \pm 0.078$  between the chronological age estimates and the ground truth on the test set. Moreover, on the complete dataset, the MAE was  $5.44 \pm 0.18$  years and Pearson's correlation was  $0.47 \pm 0.074$ . Thus, the trained VNNs were not overfit on the training set.
Next, for every individual in the HC group, we evaluated the mean of the inner products (also equivalently referred to as dot product) between the vectors of contributions of every brain region [p in (5)] and the eigenvectors of the anatomical covariance matrix for all 100 VNN models. The strongest alignment was present between the first eigenvector of the anatomical covariance matrix (i.e., the eigenvector associated with the largest eigenvalue) and the vectors of regional contributions to the VNN output  $(0.987 \pm 0.0005$  across the HC group), with relatively smaller associations for second  $(0.051 \pm 0.003)$ , third  $(0.075 \pm 0.004)$ , and fourth  $(0.094 \pm 0.003)$  eigenvectors. Additional details are included in Appendix G. Thus, the VNNs exploited the eigenvectors of the anatomical covariance matrix to achieve the learning performance in this task. The first eigenvector of the anatomical covariance matrix predominantly included bilateral anatomic brain regions in the parahippocampal gyrus, precuneus, inferior medial temporal gyrus, and precentral gyrus.
We remark that several existing studies on brain age prediction have utilized deep learning and other approaches to report better MAE on their respective healthy populations [20-22, 47]. In contrast, our contribution here is conceptual, where we have explored the properties of VNNs when they are trained to predict the chronological age for the HC group. Subsequently, our primary focus in the context of brain age is on demonstrating the anatomical interpretability offered by VNNs and relevance of eigenvectors of the anatomical covariance matrix. Thus, we further provide the insights that have not been explored (or are infeasible to obtain) for most existing brain age evaluation frameworks based on less transparent deep learning models.
4.2 Analyses of regional residuals derived from VNNs revealed regions characteristic of AD
Figure 2a displays the robustness (determined via analyses of 100 VNN models) for various brain regions being associated with significantly larger residual elements for the AD+ group than the HC group. The most significant regions with elevated regional residuals in AD+ with respect to HC were concentrated in bilateral inferior parietal, temporal, entorhinal, parahippocampal, subcallosal, and precentral regions. All these brain regions, except for precentral and subcallosal, mirrored the cortical atrophy (Fig. 11 in supplementary material), and these regions are known to be highly interconnected with hippocampus [48]. Hence, brain regions characteristic of AD had significant differences in regional residual distributions for AD+ group as compared to HC group.
Although the results in Fig. 2a provided a meaningful regional profile for AD+ group, we further performed exploratory analysis to check whether the regional residuals had any clinical significance. To this end, we evaluated the Pearson's correlations between CDR sum of boxes and the regional residuals derived from final layer VNN outputs for the AD+ group for all 100 VNN models. Interestingly, the brain regions with the largest correlations with the CDR sum of boxes scores in Fig. 2c were concentrated in the parahippocampal, medial temporal lobe, and temporal pole regions (also isolated in Fig. 2a). This observation provides the evidence that the regional residuals for the AD+ group that led to the result in Fig. 2a could predict dementia severity.
4.3  $\Delta$ -Age was elevated in AD+ group and correlated with CDR
We evaluated the  $\Delta$ -Age for HC and AD+ groups according to the procedure specified in Section 3.3. We also investigated the Pearson's correlation between  $\Delta$ -Age and CDR sum of boxes scores in AD+ group. Figure 2b illustrates the distributions for  $\Delta$ -Age for HC and AD+ groups ( $\Delta$ -Age for HC:  $0 \pm 2.83$  years,  $\Delta$ -Age for AD+:  $3.54 \pm 4.49$  years). The difference in  $\Delta$ -Age for AD+ and HC groups was significant (Cohen's  $d = 0.942$ , ANCOVA with age and sex as covariates:  $p$ -value  $< 10^{-20}$ , partial  $\eta^2 = 0.158$ ). Also, age and sex were not significant in ANCOVA ( $p$ -value  $> 0.4$  for both). Hence, the group difference in  $\Delta$ -Age for the two groups was not driven by the difference in the distributions of their chronological age. Figure 2d plots CDR sum of boxes scores versus  $\Delta$ -Age for the AD+ group. Pearson's correlation between CDR sum of boxes score and  $\Delta$ -Age was  $0.474$  ( $p$ -value  $= 2.88 \times 10^{-12}$ ), thus, implying that the  $\Delta$ -Age evaluated for AD+ group captured information about dementia severity. Hence, as expected, the  $\Delta$ -Age for AD+ was likely to be larger
C
Figure 2: Anatomically interpretable  $\Delta$ -Age evaluation in OASIS-3. Panel a displays the robustness of the significantly elevated regional residuals for AD+ group with respect to HC group for different brain regions. For every VNN model in the set of 100 nominal VNN models that were trained on HC group, the analyses of regional residuals helped isolate brain regions that corresponded to significantly elevated regional residuals in AD+ group with respect to HC group. After performing this experiment for 100 VNN models, the robustness of the observed significant effects in a brain region was evaluated by calculating the number of times a brain region was identified to have significantly elevated regional residuals in AD+ group with respect to HC group. The number of times a brain region was linked with significantly elevated regional residuals in AD+ group with respect to HC group is projected on the brain template. Panel b displays the distribution of  $\Delta$ -Age for HC and AD+ groups. The elevated brain age effect here is characterized by regional profile in Panel a. Panel c projects the mean Pearson's correlation between regional residuals (derived for each VNN model in the set of 100 nominal VNN models) and CDR sum of boxes for AD+ group on the brain template. Panel d displays the scatter plot for CDR sum of boxes versus  $\Delta$ -Age in AD+ group. The correlation between  $\Delta$ -Age and CDR sum of boxes could be attributed to the observations in Panel c.
with an increase in CDR sum of boxes scores. For instance, the mean  $\Delta$ -Age for individuals with CDR sum of boxes greater than 4 was 6.04 years, and for CDR sum of boxes  $\leq 4$  was 2.42 years.
Given that the age-bias correction procedure is a linear transformation of VNN outputs, it can readily be concluded that the statistical patterns for regional residuals in Fig. 2a and Fig. 2c lead to elevated  $\Delta$ -Age and correlation between  $\Delta$ -Age and CDR sum of boxes scores. Therefore, our framework provides a feasible way to characterize accelerated biological aging in AD+ group with a regional profile. Additional figures and details pertaining to VNN outputs and brain age before and after the age-bias correction was applied are included in Appendix J.
4.4 Regional residuals derived from VNNs were correlated with eigenvectors of the anatomical covariance matrix
Figure 3a plots the mean inner product  $|\langle \bar{\mathbf{r}}_{\mathrm{AD +}},\mathbf{v}_i\rangle |$  for eigenvectors associated with 50 largest eigenvalues of  $\mathbf{C}_{\mathrm{HA}}$ . The three largest mean correlations with the regional residuals in  $\mathrm{AD}+$  group were observed for the third eigenvector of  $\mathbf{C}_{\mathrm{HA}}(|\langle \bar{\mathbf{r}}_{\mathrm{AD +}},\mathbf{v}_3\rangle | = 0.645\pm 0.016)$ , fourth eigenvector  $(|\langle \bar{\mathbf{r}}_{\mathrm{AD +}},\mathbf{v}_4\rangle | = 0.305\pm 0.02)$ , and the first eigenvector  $(|\langle \bar{\mathbf{r}}_{\mathrm{AD +}},\mathbf{v}_1\rangle | = 0.299\pm 0.001)$ . These eigenvectors are plotted on a brain template in the expanded Fig. 3b. Inspection of the first,
a
Inner product between normalized regional residuals and eigenvectors of  $\mathbf{C}_{\mathrm{HA}}$
b
Figure 3: a. Bar plots of the mean inner products between the normalized vector of regional residuals (norm = 1) of VNN outputs (VNNs trained on OASIS-3) obtained from AD+ group and the eigenvectors of  $\mathbf{C}_{\mathsf{HA}}$  (covariance matrix of combined HC and AD+ group) with respective standard deviations as error bars. Results with coefficient of variation  $>30\%$  across the AD+ group have been excluded. b. The eigenvectors associated with the top three largest values for  $|\langle \bar{\mathbf{r}}_{\mathrm{AD + }}\rangle ,\mathbf{v}_i\rangle |$  are plotted on the brain surface. Subcallosal region in the right hemisphere was associated with the element with the largest magnitude in  $\mathbf{v}_3$  and  $\mathbf{v}_4$  and is highlighted with a red circle in the corresponding plots.
Eigenvectors of  $\mathbf{C}_{\mathrm{HA}}$
third, and fourth eigenvectors of  $\mathbf{C}_{\mathrm{HA}}$  suggested that subcallosal, entorhinal, parahippocampal and temporal pole regions had the most dominant weights (in terms of magnitude) in these eigenvectors.
4.5 Anatomical interpretability was diminished for randomly initialized VNNs.
Finally, we leveraged 100 VNNs that were randomly initialized (i.e., not trained whatsoever) to evaluate the regional profiles for brain regions that exhibited elevated regional residuals for AD+ group with respect to HC group in OASIS-3. These VNNs had the same architecture as the VNNs that were trained on OASIS-3. Figure 13a in the supplementary material shows that the robustness of the regional residuals being elevated for AD+ group with respect to HC group was severely diminished as compared to the parallel results in Fig. 2a. Similarly, the correlation between the regional residual derived using randomly initialized VNNs and CDR scores was highly inconsistent (Fig. 13b in supplementary material) as compared to parallel results in Fig. 2c. These findings suggested that the ability of VNNs to exploit the covariance matrix (learned when trained to predict chronological age of the HC group) was instrumental to the results derived in Fig. 2.
4.6 Additional Experiments
Cross-validation: The results derived in Fig. 2a and Fig. 2b were cross-validated on the ADNI-1 dataset (see Appendix I for details).
Stability to perturbations in  $\mathbf{C}_{\mathrm{HA}}$ : As a consequence of the stability of VNNs, we observed that the regional profile for  $\Delta$ -Age in Fig. 2a was stable even when the covariance matrix  $\mathbf{C}_{\mathrm{HA}}$  was estimated by a variable composition of individuals from the HC and AD+ group (Appendix M).
Anatomical covariance matrix and brain age: Use of anatomical covariance matrix derived from only HC group provides results consistent with Fig. 2, albeit with a slightly smaller group difference between the  $\Delta$ -Age distributions for HC and AD+ groups. See Appendix N for details.
We refer the reader to Appendix C for discussions on various aspects of this work and its limitations and associated societal concerns.
OASIS-3 data were provided by Longitudinal Multimodal Neuroimaging: Principal Investigators: T. Benzinger, D. Marcus, J. Morris; NIH P30 AG066444, P50 AG00561, P30 NS09857781, P01 AG026276, P01 AG003991, R01 AG043434, UL1 TR000448, R01 EB009352. Data collection and sharing for ADNI-1 dataset was funded by the Alzheimer's Disease Neuroimaging Initiative (ADNI) (National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department of Defense award number W81XWH-12-2-0012). ADNI data are disseminated by the Laboratory for Neuro Imaging at the University of Southern California.
We are grateful to the anonymous reviewers, whose suggestions have greatly improved this paper.
[69] Achraf Esemlali, Etienne St-Onge, Maxime Descoteaux, and Pierre-Marc Jodoin. Understanding Alzheimer disease's structural connectivity through explainable AI. In Medical Imaging with Deep Learning, pages 217–229. PMLR, 2020.
[70] Helmet T Karim, Maria Ly, Gary Yu, Robert Krafty, Dana L Tudorascu, Howard J Aizenstein, and Carmen Andreescu. Aging faster: Worry and rumination in late life are associated with greater brain age. Neurobiology of Aging, 101:13-21, 2021.
[71] Steve Horvath. DNA methylation age of human tissues and cell types. Genome biology, 14(10): 1-20, 2013.
[72] Lisa Ronan, Aaron F Alexander-Bloch, Konrad Wagstyl, Sadaf Farooqi, Carol Brayne, Lorraine K Tyler, Paul C Fletcher, et al. Obesity associated with increased brain age from midlife. Neurobiology of aging, 47:63-70, 2016.
[73] Klara Mareckova, Radek Marecek, Lenka Andryskova, Milan Brazdil, and Yuliya S Nikolova. Maternal depressive symptoms during pregnancy and brain age in young adult offspring: findings from a prenatal birth cohort. Cerebral Cortex, 30(7):3991-3999, 2020.
[74] Lars T Westlye, Ivar Reinvang, Helge Rootwelt, and Thomas Espeseth. Effects of apoe on brain white matter microstructure in healthy adults. Neurology, 79(19):1961-1969, 2012.
[75] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems, 32, 2019.
[76] Christian Gaser, Robert Dahnke, Paul M Thompson, Florian Kurth, Eileen Luders, and Alzheimer's Disease Neuroimaging Initiative. Cat-a computational anatomy toolbox for the analysis of structural mri data. biorxiv, pages 2022-06, 2022.
[77] Gary W van Hoesen, Jean C Augustinack, Jason Dierking, Sarah J Redman, and Ramasamy Thangavel. The parahippocampal gyrus in alzheimer's disease: clinical and preclinical neuroanatomical correlates. Annals of the New York Academy of Sciences, 911(1):254-274, 2000.
OASIS-3 dataset is publicly available and hosted on central .xnat .org. Access to OASIS-3 dataset may be requested through https://www.oasis-brains.org/. Supplementary files include (i) the VNN models trained to predict chronological age for HC group, (ii) code for demonstrations of evaluating regional profiles corresponding to elevated regional residuals in AD+ group, and correlations between the eigenvectors of the anatomical covariance and VNN outputs using a small subset of  $z$ -score normalized cortical thickness data, (iii) the regional residuals derived from different VNN models for the OASIS-3 dataset that lead to results in Fig. 2, and (iv) the code for brain age evaluation on OASIS-3 dataset. All material is also available online athttps://github.com/sihags/VNN_Brain_Age.
B Relevant Literature