SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Cross-validation. The results in Fig. 9 were obtained from the models that were trained on OASIS-3 dataset. Here, the anatomical covariance matrix was estimated using cortical thickness measures from all individuals in ADNI1 dataset. The observations on ADNI1 were highly consistent with those made in the paper. Specifically, in ADNI1 dataset, brain age gap was significantly higher for individuals with dementia as compared to healthy controls, with MCI in between them. Also, the subcallosal, entorhinal, temporal pole, and superior temporal regions were cross-validated on this dataset as being contributors to elevated  $\Delta$ -Age in the combined cohort of dementia and MCI (equivalent to AD+ group from OASIS-3) with respect to the HC group.
(b)
Figure 9: (a) Distribution of  $\Delta$ -Age across HC, MCI, Dementia cohorts derived from VNN models trained on OASIS-3. Anatomical covariance matrix from ADNI-1 dataset was used in the VNNs.  $\Delta$ -Age for controls:  $0 \pm 3.92$  years,  $\Delta$ -Age for MCI  $3 \pm 5.74$  years,  $\Delta$ -Age for AD:  $4.49 \pm 5.34$  years. (b) Across 100 VNNs that had been trained on OASIS-3 dataset, we evaluated the number of times the regional residual mean was smaller for HC group than the AD+MCI group in ADNI-1 dataset.
J Additional details on brain age prediction in OASIS-3
In this section, we provide additional figures and discussions pertaining to the results for interpretable brain age prediction in Fig. 2. Figure 10 displays the distributions of chronological age for AD+ and HC groups in OASIS-3 dataset.
Figure 10: Distribution of chronological age in AD+ and HC groups.
Cortical Atrophy
$(\mathsf{AD} + \mathsf{vs}\mathsf{HC})$
OASIS-3
(DKT Atlas)
Figure 11: Results of group differences in cortical thickness between  $\mathrm{AD + }$  and HC groups. Regions with significant differences (two-sided t-test, Bonferroni corrected  $p$  -value  $< 0.05$  ) are identified and the corresponding  $t$  statistics are projected on a brain template. Negative  $t$  -statistic for a brain region suggests that the  $\mathrm{AD + }$  group had significant cortical atrophy in that region as compared to HC group.
t-statistic
Since VNNs were trained for the regression task, a VNN processed cortical thickness data and provided an estimate for the chronological age for each individual. Since we trained 100 VNN models on different permutations of the training set in OASIS-3, we use the mean of all VNN estimates as the VNN prediction for an individual. This VNN prediction is further leveraged to form brain age estimates and  $\Delta$ -Age metrics. Figure 12a displays the plot for VNN predictions versus chronological age (ground truth) for the complete HC group. The Pearson's correlation between VNN prediction and chronological age (ground truth) for HC group was 0.486, which was similar to that reported in Section 4.1. VNN outputs clearly under-estimated the chronological age for older individuals and over-estimated the chronological age for individuals on the younger end of the age distribution for HC group.
Figure 12b displays the plot for VNN predictions versus chronological age (ground truth) for the complete AD+ group. The Pearson's correlation between VNN prediction and chronological age (ground truth) for AD+ group was 0.28. We further note that the VNN architecture and our analysis of regional residuals helped quantify the contribution of each brain region to a data point in Fig. 12a and Fig. 12b. Hence, the scatter plot in Fig. 12b could be affected by larger contributions of certain brain regions for AD+ group relative to the HC group.
Figure 12c illustrates the box plots of residuals evaluated by the difference between VNN predictions and chronological age for HC and  $\mathrm{AD + }$  groups. Figure 12c suggests that the chronological age
for  $\mathrm{AD + }$  group was underestimated as compared to that for HC group. This observation was also expected since  $\mathrm{AD + }$  group is significantly older than the HC group. However, we expect that the robust elevated regional residuals from brain regions in Fig. 2b mitigated the under-estimation effect due to higher age of  $\mathrm{AD + }$  group to some extent.
Figures 12d-f display the results after age-bias correction is applied to the VNN outputs. As expected, the brain age for HC group in Fig. 12d is largely concentrated around the line of equality ( $x = y$  line). In contrast, the brain age for  $\mathrm{AD + }$  group in Fig. 12e is concentrated above the line of equality. These effects manifest into the box plots for  $\Delta$ -Age in Fig. 12f where we observe the  $\mathrm{AD + }$  group to have elevated  $\Delta$ -Age as compared to HC group.
VNN architecture facilitated isolation of the effects of accelerated aging before age-bias correction was applied. Hence, the transformation of VNN outputs to brain age from Fig. 12a-c to Fig. 12d-f was not surprising. However, such insights may be infeasible for machine learning approaches that lack transparency and hence, the impact of deviations due to neurodegeneration from the healthy control population cannot be interpreted or isolated. In this context, if the learning model was a black box, Fig. 12a-c may appear to be counter-intuitive to the goal of detecting accelerated aging in the AD+ group and the effect of age-bias correction can be unclear, thus, leading to several criticisms [23].
f
Figure 12: Supplementary figures to Fig. 2. Panel a displays the plot of VNN prediction versus chronological age for HC group. VNN predictions were obtained as the average of the outputs of 100 nominal VNNs that were trained on OASIS-3 and operated on the anatomical covariance matrix  $\mathbf{C}_{\mathsf{HA}}$ . Panel b displays the results similar to that in panel a for the AD+ group. The solid line in panels a and b is the least squares line. Panel c includes the boxplots for residuals derived from the difference between VNN predictions and chronological age for HC and AD+ groups. Panel d and e display the plots for brain age versus chronological age for HC and AD+ groups, respectively. The solid line in panels d and e is the identity line. Panel f displays the box plots for  $\Delta$ -Age in HC and AD+ groups.
K Adaptive readouts may penalize the interpretability of regional residuals and  $\Delta$ -Age
Thus far, we have focused on VNNs that operate with a non-adaptive readout (unweighted average) function. However, it is expected that the performance of the VNNs on their original task of chronological age prediction could be improved significantly with the help of an adaptive readout function. Our experiments showed that this was indeed the case. If a single-layer fully connected perceptron consisting of 10 neurons was added to the VNNs with the same architecture as the ones that were trained on OASIS-3 dataset, we could improve the performance on the chronological prediction task. For 100 VNNs with adaptive readout that were trained on random permutations of the training data, the median MAE for the HC group was 4.64 years, which was significantly smaller than the MAE achieved by VNNs with non-adaptive readouts (Section 4.1). Among the 100 VNN models with adaptive readouts, we analyzed the regional residuals for one VNN model with adaptive readout that had the best performance on chronological age prediction in HC group (test set:  $\mathrm{MAE} = 4.17$  years, Pearson's correlation between prediction and ground truth  $= 0.73$ ; complete HC group:  $\mathrm{MAE} = 4.26$  years, Pearson's correlation between prediction and ground truth  $= 0.725$ ). Our regional residuals revealed no significant difference between the regional residuals for  $\mathrm{AD+}$  group and HC group. This observation suggested that VNN lost its interpretability due to the addition of adaptive readout function. Moreover, we also observed a diminished gap between  $\Delta$ -Age for  $\mathrm{AD+}$  and HC groups determined using this VNN model ( $\Delta$ -Age for AD group:  $1.58 \pm 4.67$  years,  $\Delta$ -Age for HC group:  $0 \pm 3.45$  years, Cohen's  $d = 0.384$ ). The findings discussed here suggest that boosting the performance on chronological age prediction task by using an adaptive readout function may penalize the interpretability offered by VNNs with non-adaptive readouts and also diminish the  $\Delta$ -Age gap between  $\mathrm{AD+}$  and HC groups.
L Results for randomly initialized VNNs
b
Figure 13: The results here were derived by VNN models that were randomly initialized and had the same architecture as those in Section 3.1.
M Regional profiles corresponding to elevated regional residuals in AD+ group are stable to the composition of data used to estimate anatomical covariance matrix  $\mathrm{C}_{\mathrm{HA}}$
Recall that  $\Delta$ -Age and associated regional profiles were evaluated using VNNs that operated upon a composite anatomical covariance matrix  $\mathbf{C}_{\mathrm{HA}}$ . We next checked whether the results derived from VNNs relevant to  $\Delta$ -Age were stable to the changes in composition of the combined HC and AD+ groups used to estimate the anatomical covariance matrix. Note that the bilateral parahippocampal, entorhinal, subcallosal, and temporal pole regions are expected to be among the most relevant to  $\Delta$ -Age in AD based on the results in Fig. 3.
We performed two sets of experiments. In the first set, we included the whole HC group and gradually varied the number of individuals from the AD+ group to be included to form the estimate  $\mathbf{C}_{\mathrm{HA}}$ . Figure 14a includes the results obtained from a randomly selected VNN model corresponding to the anatomical covariance matrix formed by different combinations of the individuals from HC and AD+ groups. The results in Fig. 14a display the brain regions whose regional residuals from AD+ group were higher than that in the HC group (Bonferroni corrected  $p$ -value  $< 0.05$ ). The result obtained by the VNN when it used  $\mathbf{C}_{\mathrm{HA}}$  estimated from all 611 HC individuals and 194 AD+ individuals forms the baseline to evaluate stability in this context. When the covariance matrix  $\mathbf{C}_{\mathrm{HA}}$  was perturbed by using a smaller number of individuals from the AD+ group to estimate it, we observed that the significance of the relevant brain regions (parahippocampal, subcallosal and temporal pole) were preserved till exclusion of about 144 AD+ individuals from the estimate  $\mathbf{C}_{\mathrm{HA}}$ . Hence, the significant differences observed in the regional residuals for AD+ and HC groups in the aforementioned regions were robust to perturbations in  $\mathbf{C}_{\mathrm{HA}}$  due to variability in the number of individuals from the AD+ group.
Figure 14b illustrates the results obtained for a similar experiment as above, with the difference that the regional residuals were evaluated for the VNN when the anatomical covariance matrix  $\mathbf{C}_{\mathrm{HA}}$  was perturbed by reducing the number of individuals from the HC group used to estimate it. Using the result obtained for  $\mathbf{C}_{\mathrm{HA}}$  estimated from 611 HC individuals and 194 AD+ individuals in Fig. 14a as the baseline, the results pertaining to ANOVA between regional residuals for AD+ and HC groups (with AD+ elevated as compared to HC) remained consistent as long as 100 or more individuals from HC group were included in forming  $\mathbf{C}_{\mathrm{HA}}$ . With less than 100 number of HC individuals included in  $\mathbf{C}_{\mathrm{HA}}$ , the results became noticeably less significant in precuneus and supramarginal regions in the left hemisphere.
In summary, the group differences observed between the regional residuals for  $\mathrm{AD + }$  and HC groups in OASIS-3 dataset were robust to perturbations in the covariance matrix  $\mathbf{C}_{\mathrm{HA}}$  when it was perturbed from the baseline by using a different combination of HC and  $\mathrm{AD + }$  individuals to estimate it. However, we also remark that (nearly) complete exclusion of HC or  $\mathrm{AD + }$  groups from  $\mathbf{C}_{\mathrm{HA}}$  resulted in loss of significance of the elevation in regional residuals in  $\mathrm{AD + }$  for various regions, including bilateral parahippocampal and temporal pole regions, and precuneus and supramarginal regions in the left hemisphere. Thus, both HC and  $\mathrm{AD + }$  groups were relevant to the anatomical covariance matrix  $\mathbf{C}_{\mathrm{HA}}$  that resulted in regional profiles in Fig. 2a, and they were robust to the combination of individuals from HC and  $\mathrm{AD + }$  used to estimate  $\mathbf{C}_{\mathrm{HA}}$ .
611 (HC)
100 (HC)
Figure 14: Stability to perturbations in the anatomical covariance matrix for group differences between AD+ and HC groups observed in regional residuals. For a VNN model that was trained to predict chronological age for HC group in OASIS-3 dataset, the regional residuals were first determined using the anatomical covariance matrix  $\mathbf{C}_{\mathrm{HA}}$  formed by the cortical thickness data of complete OASIS-3 dataset (i.e., 611 HC individuals and 194 individuals in the AD+ group. The group differences in regional residuals between AD+ and HC group were investigated according to the procedure in subsection 3.2. In the procedure described therein, we evaluated the F-values for the ANOVA test between regional residuals for AD+ group and HC group. The brain regions associated with the regional residuals that were significantly elevated in AD+ group with respect to HC group are highlighted on the brain template. The stability of the group differences to perturbations in  $\mathbf{C}_{\mathrm{HA}}$  was further investigated by varying the composition of cortical thickness data from AD+ and HC groups used to estimate  $\mathbf{C}_{\mathrm{HA}}$ . Figures in the top row display the results obtained via analysis of regional residuals by VNNs that processes the cortical thickness data from the complete OASIS-3 dataset over the anatomical covariance matrix  $\mathbf{C}_{\mathrm{HA}}$  estimated from 611 HC individuals and a varying number of individuals from the AD+group. Figures in the bottom row illustrate the results of similar experiments, with the difference that the anatomical covariance matrix  $\mathbf{C}_{\mathrm{HA}}$  was estimated using all 194 individuals in the AD+ group and varying number of individuals from the HC group. The results corresponding to  $\mathbf{C}_{\mathrm{HA}}$  that was estimated using 194 AD+ individuals and 611 HC individuals formed the baseline for comparison for all scenarios.
300 (HC)
500 (HC)
N  $\triangle$ -Age evaluation with anatomical covariance matrix from HC group
Using the anatomical covariance matrix derived only from the HC group (denoted by  $\mathbf{C_H}$ ) resulted in observations that were consistent with Fig. 2 with a slightly diminished difference in  $\Delta$ -Age between HC and AD+ groups. Specifically,  $\Delta$ -Age for AD+ group in this setting was  $3.41 \pm 4.57$  years, which was significantly larger than that for the HC group (ANOVA: partial  $\eta^2 = 0.141$ , Cohen's  $d = 0.88$ ). The correlation between  $\Delta$ -Age and CDR sum of boxes scores in the AD+ group was 0.464. Thus, when the anatomical covariance matrix derived solely from the HC group was utilized in our brain age prediction framework, the magnitude of  $\Delta$ -Age and its utility as a marker of dementia severity was slightly diminished as compared to the results in Fig. 2.
Figure 15a displays the regional profile associated with  $\Delta$ -Age derived from VNNs with  $\mathbf{C_H}$  as the anatomical covariance matrix. Comparison with Fig. 2a reveals that the robustness of regional residuals being elevated in the AD+ group in subcallosal and temporal regions was preserved, but that in bilateral entorhinal and parahippocampal regions was diminished in this scenario. Figure 15b displays the distributions for  $\Delta$ -Age in AD+ and HC groups.
The variation in the regional residuals associated with entorhinal and parahippocampal regions in Fig. 15a and Fig. 2a could be explained by investigating the eigenvectors of  $\mathbf{C_H}$ . Specifically, Fig. 16a displays the associations between regional residuals for the AD+ group and the first 50 eigenvectors of  $\mathbf{C_H}$ , where we observed that the third, second, and first eigenvectors of  $\mathbf{C_H}$  had the top three largest associations. Figure 16b plots the projections of the first three eigenvectors of  $\mathbf{C_H}$  on a brain template. Note that the eigenvectors  $\mathbf{v}_3$  and  $\mathbf{v}_2$  have the largest weights associated with the subcallosal region in the right hemisphere, which is consistent with the relevant eigenvectors of  $\mathbf{C_HA}$  in Fig. 3. However, unlike the eigenvectors of  $\mathbf{C_H}$  in Fig. 16b, the eigenvectors of  $\mathbf{C_HA}$  in Fig. 3b are also characterized by comparatively larger weights in the entorhinal and parahippocampal regions. Since parapippocampal and entorhinal regions are well known to be associated with disease onset and cortical atrophy [77], the anatomical covariance matrix  $\mathbf{C_HA}$  may provide a more holistic perspective to  $\Delta$ -Age in AD.
Figure 15:  $\Delta$ -Age results for anatomical covariance matrix from only HC group ( $C_H$ ). Panel a projects the robustness of observing a significantly higher regional residual for AD+ group with respect to HC group for a brain region on the template. Panel b plots the  $\Delta$ -Age distributions for AD+ and HC groups derived from VNNs with  $C_H$  as the covariance matrix.
Figure 16: Panel a illustrates a bar plot for  $|\bar{\mathbf{r}}_{\mathrm{AD + }}|$ ,  $\mathbf{v}_i > |$  for  $i \in \{1, \dots, 30\}$ , where  $\mathbf{v}_i$  is the  $i$ -th eigenvector of covariance matrix  $\mathbf{C_H}$  associated with its  $i$ -largest eigenvalue. The bars are evaluated from the mean of  $|\bar{\mathbf{r}}_{\mathrm{AD + }}|$ ,  $\mathbf{v}_i > |$  obtained for individuals in the  $\mathrm{AD+}$  group (results for eigenvectors associated with coefficient of variation of  $|\bar{\mathbf{r}}_{\mathrm{AD+}}|$ ,  $\mathbf{v}_i > |$  larger than  $30\%$  excluded). For every individual in  $\mathrm{AD+}$  group, the association of its regional residuals with eigenvectors of  $\mathbf{C_H}$  were evaluated over 100 nominal VNN models (trained on the OASIS-3 dataset). The eigenvectors associated with top three largest values for  $|\bar{\mathbf{r}}_{\mathrm{AD+}}|$ ,  $\mathbf{v}_i > |$  are plotted on the brain template in Panel b. Subcallosal region in the right hemisphere was associated with the element with the largest magnitude in  $\mathbf{v}_2$  and  $\mathbf{v}_3$  and is highlighted with a red circle in the corresponding plots.