Understanding pre-training data effects in retinal foundation models using two large fundus cohorts

Domain: chemistry
Published: 2026-02-28
Editorial outcome: major_revision

Abstract
Medical foundation models, pre-trained on large-scale unlabelled data, show strong performance and
data efficiency when adapted to various clinically relevant applications. However, how pre-training data
shape the generalisability and fairness of these models remains unexplored. Here we address this using
two cohorts from Moorfields Eye Hospital (UK) and the Shanghai Diabetes Prevention Program (China),
each containing 904,170 fundus photographs for model pre-training. Using identical pipelines, we train
parallel foundation models using individual cohort and evaluate them on downstream tasks with publicly
available datasets and held-out data from each site. The parallel models show competitive performance
to data that differ substantially from their pre-training data. Nevertheless, we observe fairness gaps over
age subgroups, whereas sex and ethnicity show minimal impact. These results demonstrate the good
generalisability of retinal foundation models and indicate that pre-training demographic attributes shape
fairness differently, highlighting the importance of domain-specific, fine-grained data curation for efficient
foundation model development.

Introduction
Foundation models (FM) are large artificial intelligence (AI) models trained using data and computation
at scale [1,2]. Using self-supervised or unsupervised learning methods, FMs capture abundant data
patterns that can potentially be applied to diverse applications in real-world scenarios. This approach has
broad applications across fields of medical AI [3–5], such as ophthalmology [6–8], radiology [9–11],
pathology [12–16], and as a generalist [17,18] advancing clinically meaningful tasks like disease
diagnosis and prognosis. However, despite the increasing number of medical FMs being developed,
there remains very limited knowledge regarding how the composition of pre-training data, the “guts” of
these models, affects FM capabilities such as generalisability and fairness. The lack of this critical
knowledge makes pre-training data collection and medical FM development inefficient and highly
speculative.

Training data is the fundamental substrate for developing AI models, with its characteristics
encompassing attributes, properties, and features that influence data quality, usability, and its impact on
analytical processes. These characteristics define the scope of knowledge and largely determine model
capability including generalisability and fairness [19–25]. Previous studies have investigated the impact
of labelled data on traditional application-specific AI models in supervised learning [20,23,26–28],
providing effective guidance for labelled data selection during model training. For instance, some work
has demonstrated that model performance markedly drops when applied to external sites with distinct
data characteristics such as demographics, imaging devices, and disease phenotypes [20,23,26,28].
Additionally, imbalanced training data often leads to poor performance in underrepresented subgroups in
terms of age, sex, and ethnicity, raising concerns about model fairness and generalisability [27,29]. To
address these challenges, previous work has focused on building diverse and balanced training data for
traditional application-specific AI models, using techniques such as data augmentation and synthesis
[20,30–32].

Unlike application-specific AI models that rely solely on labelled data for training, FMs learn
generalisable features through extensive pre-training on substantial unlabelled data (e.g. via self-
supervised learning [33–35]), followed by fine-tuning on labelled data for specific target applications. The
performance of FMs is collectively shaped, where pre-training establishes the foundational capabilities
while fine-tuning refines performance in specific tasks. Prior studies have systematically shown that the
distribution of fine-tuning data, particularly demographic attributes such as age and sex, can be encoded
by AI models and lead to fairness gaps [23,25]. However, the impact of unlabelled pre-training data on
medical FMs remains underexplored, partly hampered by the substantial workload and resources
required to build and compare parallel FMs on matched large-scale datasets from different countries and
areas. This lack of knowledge leaves critical questions unanswered: 1) Do pre-training data
characteristics affect the generalisability of medical FMs, such as performing poorly on sites with distinct
demographics and imaging devices? 2) Do demographic attribute of pre-training data impact medical FM
fairness over age, sex, and ethnicity, similar to traditional application-specific AI models? Addressing
these questions is critical for understanding how pre-training impacts medical FMs, providing evidence
for efficient data sampling to optimise FM generalisability and fairness. Furthermore, revealing the
impact of pre-training data provides a strong basis for advocating data transparency, one of the least
transparent dimensions according to Foundation Model Transparency Index Scores [36]. Disclosing how
pre-training data impacts FMs, and providing details of pre-training data, are essential to understanding
FM strengths and limitations [37].

To address these gaps, we investigated the impact of pre-training data on retinal FMs, one of the
pioneering areas in medical FM, using two large data cohorts from Moorfields Eye Hospital (MEH), UK,
and Shanghai Diabetes Prevention Program (SDPP), China, each comprising 904,170 retinal colour
photographs for FM pre-training. We first characterised data cohorts using demographic and imaging
metadata, latent features (representative features encoded by models), and clinically meaningful
morphological indices, highlighting the differences between the datasets. We then developed retinal FMs
with MEH data (FM-MEH) and SDPP data (FM-SIDPP), using identical pre-training strategies and
implementation details. We evaluated the performance of parallel FMs across a wide range of
downstream tasks, including ocular disease diagnosis and systemic event prediction, using data from
each site (held out from pre-training data) and publicly available datasets. We finally assessed model
fairness across subgroups based on age, sex, and ethnicity. Our results showed the strong
generalisability of retinal FMs. For example, FM-MEH and FM-SDPP performed comparably across all
SDPP downstream tasks. However, the models exhibited performance gaps over age subgroups (not
seen in sex and ethnicity), likely caused by larger distributional shift across age subgroups. The age
fairness improved when we supplemented pre-training data with synthetic images that provided a
complementary age distribution. Through extensive experiments with real-world clinical data, this study
examined the impact of pre-training data on retinal FMs. While our study focuses on retinal images, the
observed combination of strong generalisability and uneven demographic effects may extend to broader
medical FMs. These findings suggest an evidence-based approach to data description and selection in
FM development to improve both efficiency and capabilities.

Results
Distinct data distribution in two cohorts
Figure 1 provides an overview of the development and application of retinal FMs. FM-MEH and FM-
SDPP were constructed using data from MEH and SDPP respectively. We randomly sampled 904,170
retinal fundus photographs from each database for FM pre-training, and analysed their characteristics
across demographic and imaging metadata, latent features, and clinically meaningful morphological
indices. As shown in Figure 2, significant differences were observed between the MEH and SDPP data
in terms of demographic and imaging metadata. The average age in the MEH cohort is 68.88 years
(95% Confidence Interval (CI) 68.85, 68.91), significantly older than the SDPP cohort, which had an
average age of 47.26 years (95% CI 47.21, 47.30) (P<0.001). MEH data had a more balanced sex
distribution, with 52.8% female participants compared to 36.8% in the SDPP data. MEH data was
ethnically diverse with individuals identifying as White (45.9%), Asian or Asian British (18.7%, of which
0.6% were Chinese), Black or Black British (9.2%), Mixed (0.9%), other ethnicity (12.7%) and not
reported (NR, 12.6%), based on the ethnicity grouping by the UK Office for National Statistics. In
contrast, the SDPP cohort comprised only Chinese participants. Imaging devices also varied between
the two datasets, with MEH primarily using the 3DOCT-2000SA (Topcon), FD-OCT (Topcon), and
CIRRUS (ZEISS), while the SDPP data was collected using TRC-NW300 and TRC-NW400 (Topcon).
MEH data, as a hospital-based cohort, exhibits higher disease prevalence than SDPP data, which is
derived from community screening. The prevalence of diabetic retinopathy, diabetic macular oedema,
and ischaemic stroke in the MREH cohort is 8.71%, 3.02%, and 1.57%, respectively, compared with 1.4%,
0.08%, and 0.66% in the SDPP cohort.

To analyse latent features, we extracted features from 5000 random samples respectively from each site
and visualised them using t-distributed stochastic neighbour embedding (t-SNE), a dimensionality
reduction algorithm widely used for visualising high-dimensional data [38]. As shown in Figure 2b and
Supplementary Figure 1, the t-SNE plots revealed clear clustering patterns between the MEH and SDPP
cohorts, underscoring the distinct data distributions of the two sites. Furthermore, we measured clinically
meaningful morphological indices, such as vascular fractal dimension, which have been proven to be
highly associated with systemic conditions like cardiovascular [39–41] and neurological health [42–45].
Figure 2c highlights significant differences in these indices between the two cohorts. For instance, the
artery fractal dimension in the MEH dataset is 1.25 (95% CI 1.24, 1.25), compared to 1.28 (95% CI 1.28,
1.29) in the SDPP dataset (p<0.001). More morphological indices are listed in Supplementary Figure 2.

For FM pre-training, we included two representative and widely used self-supervised learning strategies,
generative-based learning (Masked Autoencoder [46]) and self-distillation (DINOV2 [47]). We organised
downstream tasks using held-out data (i.e. isolated from FM pre-training data) from MEH and SDPP, as
well as publicly available datasets sourced from several countries. We adapted FMs to downstream
tasks via both fine-tuning (all model parameters tuned on downstream labelled data) and linear probes
(all model parameters frozen with one linear classifier tuned on downstream labelled data), as shown in
Supplementary Figure 3. All task performances were assessed using the area under the receiver
operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC). Details
of the downstream task datasets are listed in Supplementary Data 1 and 2. More details about data
curation, model adaptation, and model evaluation are introduced in the Methods section.

Generalisability on downstream tasks curated from each site
We compared FM-MEH and FM-SDPP on three clinically relevant applications (diabetic retinopathy
detection, diabetic macular oedema detection, and ischaemic stroke prediction) using held-out MEH and
SDPP data. As shown in Figure 3, the FMs demonstrated good generalisability, with FM-MEH and FM-
SDPP showing comparable performance on most evaluations. When adapted to SDPP downstream
tasks (Figure 3a and 3b), FM-MEH and FM-SDPP performed comparably on all evaluations. Similarly,
when adapted to MEH downstream tasks, FM-MEH significantly outperformed FM-SDPP on only three
evaluations (ischaemic stroke prediction tasks), as shown in Figures 3c and 3d. AUPRC performance
followed a similar trend, illustrated in Supplementary Figure 4. All quantitative results are listed in
Supplementary Data 3.

Generalisability on downstream tasks from publicly available datasets
We evaluated the generalisability of FM-MEH and FM-SDPP to diverse applications using six publicly
available datasets, comprising diabetic retinopathy detection (APTOS2019, IDRiD, and MESSIDOR2),
glaucoma detection (Glaucoma fundus), and multiple retinal disease detection (JSIEC and Retina). As
shown in Figure 4, FM-SDPP and FM-MEH showed comparable performance in a majority of
evaluations (8 out of 12 for Masked Autoencoder-based FMs; 10 out of 12 for DINOV2-based FMs).
When pre-trained with Masked Autoencoder and fine-tuned to the downstream tasks (Figure 4a), FM-
SDPP significantly outperformed FM-MEH on three out of six datasets (APTOS2019, MESSIDOR2, and
Retina datasets), while the average performance gap on APTOS2019 is minor. When adapted to
downstream tasks using linear probing (Figure 4c), FM-MEH performed significantly better on the
Glaucoma fundus dataset (p<0.01). When using DINOV2 for FM pre-training (Figure 4b and Figure 4d),
FM-MEH significantly outperformed FM-SDPP in two applications, i.e. fine-tuning to MESSIDOR2
(0.01<p<0.05) and linear probing to IDRiD (p<0.01). AUPRC results are illustrated in Supplementary
Figure 5. All quantitative results are listed in Supplementary Data 3.

FM fairness over demographic subgroups
We evaluated FM fairness across demographic attribute subgroups in downstream tasks. Using a
curated MEH dataset (details introduced in Supplementary Data 4), we examined subgroup performance
on diabetic retinopathy detection and diabetic macular oedema detection, considering FM-SDPP and
FM-MEH showed no significant differences in overall performance on these tasks (Figure 3c), and MEH
data includes multiple ethnical groups. The SDPP pre-training data is mainly distributed over the young
(<40 years) and middle-aged groups (40-70 years), with a subgroup ratio of (0.279, 0.685, 0.036), while
MEH data is skewed towards middle-age and older cohorts (>70 years), with a ratio of (0.015, 0.467,
0.518). As shown in Figure 5a, FM-SDPP performed consistently better than FM-MEH in the young
group (grey bars) but worse in the aged group (yellow bars). For instance, in diabetic retinopathy
detection using a linear probe, FM-SDPP outperformed FM-MEH by an average AUROC of 0.073 in the
young group (p<0.01) while underperforming by 0.047 in the aged group (p<0.01). FM-MEH and FM-
SDPP demonstrated similar performance in the middle-aged group. This consistent performance gap
across diverse applications demonstrated the FM bias introduced by differential age distribution in pre-
training data.

For ethnicity, despite being pre-trained exclusively on data from the Chinese cohort, FM-SDPP
sometimes outperformed FM-MEH in White and Black subgroups on downstream tasks. For instance,
FM-SDPP achieved an AUROC of 0.111 higher than FM-MEH in the White subgroup for diabetic
macular oedema detection (p<0.01, Figure 5b). When adapted to diabetic retinopathy detection with the
linear probe, FM-MEH pre-trained with DINOV2 significantly outperformed FM-SDPP in Asian cohorts
(p<0.01). The performance differences across ethnicity subgroups showed no consistent pattern and did
not correlate with the ethnicity distribution of the pre-training data.

For sex, FM-MEH and FM-SDPP showed varying performance across sex subgroups depending on the
task and adaptation method (Figure 5c). Although FM-SDPP pre-training data has a less balanced sex
distribution (36.8% female participants versus 52.8% in MEH data), it performed better in female cohorts
for certain tasks like diabetic macular oedema detection when pre-trained with DINOV2 and adapted with
linear probing. These results showed no correlation with the sex distribution in the pre-training data. All
quantitative results and p-values are listed in Supplementary Data 5.

Probing the reasons behind fairness and generalisability
We probed the reasons why demographic attribute unevenly shaped the model performance, particularly
fairness over subgroups. We mainly investigated this from two aspects, the morphological indices and
latent features across demographic subgroups in pre-training data. As shown in Figure 6, we observed
distinct distribution differences across age subgroups in both latent features (Figure 6a and 6c) and
morphological indices (Figure 6e and 6f). After controlling for confounders by looking at only Asian or
Asian British females (Figure 6b and 6d), there are still significant gaps in morphological indices (Figure
6f). This demonstrates that age subgroups exhibit clear morphological variations, and even with self-
supervised pre-training, retinal FMs learned distinct latent features across age subgroups, which likely
caused bias in FM fairness.

For ethnicity, we investigated three subgroups White, Asian or Asian British, and Black or Black British.
The t-SNE visualisations (Supplementary Figure 6a, FMs trained with Masked Autoencoder) showed
clear clustering partially for the White cohort. When FMs were pre-trained with DINOV2 (Supplementary
Figure 6c), latent features showed no distinct clustering across all ethnicities. Additionally, when
controlling for confounders, there were no significant differences across ethnic subgroups in
morphological indices (Supplementary Figure 6f).

For sex subgroups (i.e. female, male), we observed no distinct clustering in t-SNE visualisations, either
before or after removing confounding variables (Supplementary Figure 7). Only the vein fractal
dimension showed significant differences across the sex subgroups after removing confounding
variables (Supplementary Figure 7f). These findings suggest that, ethnicity and sex subgroups
contributed limited observable variations in morphological indices and less distinct latent compared to
age distribution, which are less likely to bias FM fairness and generalisability in downstream tasks.

Inspired by the observations above, we tested whether retinal FM generalisability and fairness improve
when pre-training on a dataset with a more diverse and balanced age distribution. We generated 300K
synthetic colour fundus photographs representative of SDPP young cohort (age<40) in age distribution,
using synthetic models (details introduced in the Method section) and merged them with randomly
sampled 604,170 MEH images, for a total of 904,170 images. Image synthesis was explicitly conditioned
on age to match the age profile in SDPP young cohort. The age distribution of this combined data is
shown in Supplementary Figure 8. Using this combined dataset, we trained FM-MEH-SDPP using
DINOV2 and compared it with FM-MEH in subgroup analysis. As shown in Supplementary Data 7, FM-
MEH-SDPP performed comparably with FM-MEH, with differences observed for diabetic oedema
detection on MEH data and diabetic retinopathy detection on MESSIDOR2 datasets after linear probing.
For model fairness across age subgroups (Supplementary Data 8), FM-MEH-SDPP showed significant
improvement in young groups for diabetic retinopathy detection after fine-tuning (p=0.02 after Holm-
Bonferroni correction) and for diabetic retinopathy after linear probing (p=0.033 after correction). These
results indicate that simply increasing age distribution diversity and balance in the pre-training data can
yield fairness gains, even with synthetic images.

Methods
Ethics statement
Ethical approval for this study was obtained from the London-Central Research Ethics Committee
(18/LO/1163, approved 1 August 2018), Advanced statistical modelling of multimodal data of genetic and
acquired retinal diseases (20/HRA/2158, approved 5 May 2020), Confidential Advisory Group for Section
251 support (18/CAG/0111, approved 13 September 2018), and the Ethics Committee of Shanghai Sixth
People’s Hospital (Approved No: 2019-087, approved 29 August 2019). The National Health Service
Health Research Authority gave final approval on 13 September 2018. Moorfields Eye Hospital NHS
Foundation Trust validated the de-identifications for MEH data. Only de-identified retrospective data
were used for research.

The requirement for informed consent using SDPP data was waived by the Shanghai Sixth People’s
Hospital ethics committee because the study used retrospectively collected, anonymised imaging data.
The requirement for informed consent using AlzEye data was waived following section 251 supported by
the UK CAG because the study used retrospectively collected, anonymised imaging data.

Source for pre-training data
This research complies with approved ethical regulations. The Moorfields Eye Hospital (MEH) cohort
was sourced from AlzEye [58], a retrospective cohort study linking ophthalmic data from 353,157
participants, who attended MEH between 2008 and 2018, with systemic health data from hospital
admissions across the whole of England. The ethnicity groups are reported based on the ethnicity
grouping by the UK Office for National Statistics. The Shanghai Diabetes Prevention Program (SDPP)
cohort was drawn from a community-based longitudinal study of 79,284 participants who underwent
physical examinations at Huadong Sanatorium and Shanghai Sixth People’s Hospital between
December 2015 and November 2022. We randomly sampled 904,170 retinal fundus photographs from
each database for FM pre-training. The corresponding data characteristics are listed in Figure 2. The
retinal fundus photographs have a normal field of view (<60°), i.e. no ultra-widefield fundus images were
used.

Data for downstream tasks
We evaluated the foundation model performance on clinically relevant applications using the data from
Moorfields Eye Hospital (MEH) UK, SDPP China, and publicly available datasets. First, we organised
ocular disease detection tasks, including diabetic retinopathy and diabetic macular oedema detection,
using MEH and SDPP data which were held out from FM pre-training data at the patient level. There was
no overlap of patients between pre-training and downstream data. We curated 2000 images with labels
of diabetic retinopathy and macular oedema from 2000 participants. The labels for diabetic retinopathy
are based on the International Clinical Diabetic Retinopathy Severity scale [59], indicating five stages
from no diabetic retinopathy to proliferative diabetic retinopathy. The 2000 images were evenly
distributed over the five categories. The labels for diabetic macular oedema included three categories:
no diabetic oedema, non-clinically significant diabetic macular oedema, and clinically significant diabetic
oedema [60,61]. For MEH data, the labels were obtained from clinical practice records. For SDPP data,
two independent ophthalmologists annotated the disease labels, with disagreements adjudicated by a
consultant-level ophthalmologist. Second, we curated the task of ischaemic stroke prediction using MEH
and SDPP data. The stroke labels include binary categories, i.e. no stroke event within three years from
imaging or stroke event within three years. For SDPP data, stroke labels were obtained from digital
hospital records and self-report records during longitudinal visits between December 2015 and
November 2022. For MEH data, systemic health data were derived from Hospital Episode Statistics
(HES) data relating to admitted patient care (inpatient records). Diagnostic codes in HES admitted
patient care were reported according to the tenth revision of the ICD (International Statistical
Classification of Diseases) [62]. ICD codes for stroke (I23-I24) were used in line with previous reports.
The stroke data from MEH included 2526 images with each category having 1263 images, while SDPP
data included 2000 images with each category including 1000 images. More details are listed in
Supplementary Data 2. For subgroup analysis to understand FM fairness, we increased the sample size
for diabetic retinopathy detection and diabetic oedema detection to 10,014. The image numbers for each
demographic subgroup are listed in Supplementary Data 4.

Similarly to the RETFound study [68], we organised six ocular disease detection tasks with publicly
available datasets. For diabetic retinopathy diagnosis, Kaggle APTOS2019 (India), IDRID (India) [63]
and MESSIDOR2 (France) [64] were used, with the labels defined by the International Clinical Diabetic
Retinopathy Severity scale. For glaucoma, Glaucoma Fundus (South Korea) [65] was included, with
three categorical labels, non-glaucoma, early glaucoma (suspected glaucoma) and advanced glaucoma.
For datasets with several diseases, JSIEC (China) [66] and Retina were included. JSIEC included 1,000
images with 39 categories of common referable fundus diseases and conditions. Retina had labels of
normal, glaucoma, cataract and retina disease. The grading protocols for the public datasets were
summarised as: IDRiD, two medical experts provided adjudicated consensus grades; MESSIDOR2,
adjudicated by a panel of three retina specialists in accordance with a published protocol; APTOS2019,
Kaggle dataset with limited information but possibly a single clinician grader; Glaucoma Fundus,
agreement of two specialists based on visual fields and extensive imaging, and JSIEC, labelled by
ophthalmologists and confirmed by senior retina specialists. Disagreements were resolved by a panel of
five senior retina specialists. Retina, details not available. The details of datasets, such as imaging
devices, country and label category, are listed in Supplementary Data 1.

Data processing for self-supervised learning
We used AutoMorph [67], an automated retinal image analysis tool, to exclude the background and keep
the retinal area. All images were resized to 256 × 256 with cubic interpolation. We followed the default
data augmentation settings as Masked Autoencoder and DINOV2. On pre-training with Masked
Autoencoder, we included random crop (lower bounds 20% of the whole image and upper bounds
100%) and resized the cropped patches to 224 × 224, random horizontal flipping and image
normalisation. For DINOV2, the global patch augmentation included random crop (lower bounds 32% of
the whole image and upper bounds 100%) and resizing the cropped patches to 224 × 224, random
horizontal flipping, colour jittering (brightness 0.4, contrast 0.4, saturation 0.2, and hue 0.1), followed by
either Gaussian blur or Gaussian blur and random image solarising (threshold 128, possibility 20%). The
local patch augmentation included random crop (lower bounds 5% of the whole image and upper bounds
32%) and resizing the cropped patches to 96 × 96, random horizontal flipping, colour jittering, and
random Gaussian blur (possibility 50%). All augmented patches were normalised. We also measured the
image quality and morphological indices with AutoMorph.

Data synthesis
We trained Stable Diffusion XL [68] to generate synthetic retinal images representative of the SDPP
cohort. To improve training efficiency on retinal data, we applied Low-Rank Adaptation [69]. For
conditional model training, we randomly sampled 200,000 images from the SDPP cohort aged below 40
years, along with their corresponding ages. After training, we generated 300,000 synthetic images that
followed the sampled SDPP age distribution. In parallel, we randomly selected 604,170 images from the
previously curated set of 904,170 real-world MEH images. The real and synthetic images were then
combined to pre-train retinal foundation models, using the same procedures described in sections below.
We note that image synthesis may not always strictly follow the age condition. However, as the SDPP
data used for Stable Diffusion XL training are drawn from a young cohort, the resulting synthetic images
should still be broadly representative of this group.

Foundation model implementations
For FM pre-training, we selected two representative self-supervised learning strategies, Masked
Autoencoder and DINOV2. Both have been widely used across various domains including medical
applications, and have demonstrated state-of-the-art performance in disease diagnosis [6,14,16,70]. We
used a specific configuration of Masked Autoencoder comprising an encoder and a decoder. The
encoder was a large vision Transformer (ViT-large) with 24 Transformer blocks and an embedding
vector size of 1024, while the decoder was a small vision Transformer (ViT-small) with eight Transformer
blocks and an embedding vector size of 512. The encoder took unmasked patches (with a patch size of
16 × 16) as input and projected them into feature vectors of size 1024. These feature vectors passed
through the 24 Transformer blocks, which consisted of multi-headed self-attention and multilayer
perceptrons to generate high-level features. The decoder reconstructed the image by inserting masked
placeholder patches into the extracted high-level features and then projecting them back to image
patches through a linear projection layer. During model pre-training, the objective was to reconstruct
retinal images from the highly masked version, with a mask ratio of 0.75. The pre-training batch size was
1792 (4 GPUs × 448 per GPU). The total pre-training epoch was 800 and the first 15 epochs were for
learning rate warming up (from 0 to a learning rate of 1 × 10−3). The model weights at the final epoch
were saved as the checkpoint for adapting to downstream tasks.

We specified DINOV2 with both teacher and student networks as ViT-large, with 24 Transformer blocks
and an embedding vector size of 1024. It included a projection head of three-layer perceptrons,
respectively with dimensions 2048, 384, and 131,072. The patch size was 14 × 14. The teacher network
processed the global patches while the student network processed both global and local patches. During
model pre-training, the objectives combined the original objectives of DINO [71] and iBOT [72]. The
DINO part calculated the cross-entropy loss between the categorical tokens from the teacher network
and the student network, while the iBOT part calculated the cross-entropy between masked patch tokens
between the two networks (the maximum number of masking patches was 128). The pre-training batch
size was 320 (4 GPUs × 80 per GPU). The total pre-training epoch was 100 and the first 10 epochs were
for the learning rate warming up (from 1 × 10−6 to a learning rate of 2 × 10−4) and the remaining 90
epochs for a cosine annealing schedule. The model weights at the final epoch were saved as the
checkpoint for adapting to downstream tasks.

Adaptation to downstream tasks
When adapting foundation models pre-trained with Masked Autoencoder to downstream tasks, we only
need the encoder (ViT-large) of the foundation model and discard the decoder. For foundation models
pre-trained with DINOV2, the teacher network was used and adapted to downstream tasks. Both the
encoder and teacher networks extracted high-level features from retinal images. A fully connected layer
took these features as input and output the probability distribution over the disease categories. The
category with the highest probability was selected as the final classification. The number of categories
determined the number of neurons in the fully connected layer. We used two adaptation strategies,
fine-tuning and the linear probe. Fine-tuning tuned the encoder and fully connected layer using the
downstream data while the linear probe tuned only the fully connected layer. The schematic diagram is
shown in Supplementary Figure 3. The training objective was to predict the same categorical output as
the label. The batch size was set to 16, and the model was trained for 50 epochs. The first 10 epochs
followed a learning rate warm-up schedule, increasing linearly from 0 to a learning rate of 5 × 10−4. This
was followed by a cosine annealing schedule, where the learning rate gradually decreased from 5 × 10−4
to 1 × 10−6 over the remaining 40 epochs. After each training epoch, the model performance was
evaluated on the validation set. The model checkpoint with the highest AUROC on the validation set was
saved for subsequent internal and external evaluations.

Computational resources
Four NVIDIA Tesla A100 (80 GB) were used for self-supervised pre-training in this project. It took about
16 days to finish pre-training with DINOV2 or Masked Autoencoder. We used an equal computational
cost from MEH and SDPP for foundation model development. For fine-tuning and linear probing
foundation models to downstream tasks, we use NVIDIA Tesla T4 (16 GB). Fine-tuning took about
70 mins for every 1,000 images, while linear probing took around 15 mins.

Evaluation and statistical analysis
All task performance was assessed using the classification metrics AUROC and AUPRC. For ischaemic
stroke prediction tasks, the AUROC and AUPRC were calculated in a binary setting. For multiclass
classification, such as five-stage diabetic retinopathy and multicategory disease diagnosis, AUROC and
AUPRC were calculated separately for each class and then averaged to obtain the overall AUROC and
AUPRC scores. For each task, we fine-tuned the model with five different random seeds, which
determined the implementations including shuffling of fine-tuning data and data augmentation. The mean
and standard deviation of the performance across the five runs were computed. The standard error is
estimated as (standard deviation / √5), and the 95% confidence interval (CI) is obtained by multiplying
the standard error by 1.96. The normality of the model performance was checked via Shapiro-Wilk test.
Statistical significance is calculated using two-sided Welch’s t-test, followed by Holm-Bonferroni
correction. When comparing FM-MEH and FM-SDPP, the p-value was adjusted to the size of 24
(3*SDPP tasks, 3*MEH tasks, 6*publicly available datasets, each including fine-tuning and linear
probing). In subgroup analysis, for each demographic attribute, the p-value was adjusted to the size of 8
(2*MEH tasks, 2*Self-supervised learning methods, each including fine-tuning and linear probing). All
raw p-values, adjusted p-values, and significances are listed in Supplementary Data 3-8.

Peer review rounds
[{"round": 1, "reviewer_comments": {"reviewer_1": "Paper Summary\nThis study investigates the influence of pre-training data characteristics (specifically, geographic origin and associated\ndemographic distributions) on the generalizability and fairness of medical foundation models (FMs) in ophthalmology. The\nauthors pre-train Vision Transformer-based FMs using two self-supervised learning methods (Masked Autoencoder (MAE),\nand DINOv2) on two large, distinct datasets of retinal fundus photographs: one from Moorfields Eye Hospital (MEH) in the\nUK and one from the Shanghai Diabetes Prevention Program (SDPP) in China. They evaluate these parallel-trained models\n(FM-MEH and FM-SDPP) on downstream tasks (diabetic retinopathy, diabetic macular edema, stroke prediction) using held-\nout data from both sites and several public datasets. Fairness is assessed by comparing performance across subgroups\ndefined by age, sex, and ethnicity using MEH test data.\nMain Claims and Observations\nFairness: The paper concludes that for these retinal FMs, the age distribution within the pre-training data introduces\nmeasurable performance bias across downstream task age subgroups, while pre-training data distributions for sex and\nethnicity did not result in similar consistent biases. [lines 96, 269 following]\nGeneralizability & Site Bias: The study reports that FMs demonstrate good generalizability, performing comparably across\nsites and datasets in approximately 70% of evaluations. However, in the remaining ~30% of cases, FMs perform significantly\nbetter when the downstream task data aligns with their specific pre-training data source (intra-site advantage). [lines 322\nfollowing]\nReview Summary\nThis paper explores how pre-training data influences the behaviour of medical foundation models (FMs), employing a robust\nparallel training strategy on large-scale, high-quality datasets. While it addresses important questions, its main conclusions\nadd little novelty or practical significance. The finding that models often perform better on data resembling their training\ndistribution (site bias) is already a well-documented principle in machine learning, and the approximately 30% figure\nreported here lacks deeper explanations of how or why this bias arises.\nFurthermore, the finding that age distribution affects fairness, while potentially valid for this specific retinal imaging context, is\npresented with claims that overreach its generalizability to \"medical FMs\" broadly. The finding regarding site-specific\nperformance benefits is somewhat inconclusive, lacking a clear framework for practical data selection beyond reaffirming\nthat data distribution matters. Statistical methods require further clarification (especially given multiple comparisons),\npotentially undermining the reliability of the reported quantitative findings. The authors would benefit from more thorough\ncontextualization of their results relative to previous works on fairness and bias in medical foundation models.\nCombined with numerous minor weaknesses, the paper overstates its contributions and requires substantial revision,\npotentially bordering on rejection due to the limited novelty and potential statistical issues.\nDetailed Review\nMajor Weaknesses\nLimited Novelty and Depth of “Site-Bias” Finding\nThe core finding that models can exhibit better performance on data distributions matching their training data is not novel; it's\na fundamental concept in domain adaptation and generalization research. While quantifying this effect for these specific FMs\n(~30% intra-site advantage) provides a data point, the paper falls short of providing significant new conceptual\nunderstanding or predictive insight into when or why this site bias occurs for FMs. This finding also stands in mild tension with\nliterature suggesting large-scale self-supervised training can mitigate certain biases.\nConcerns Regarding Statistical Rigor\nMultiple Comparisons: The paper reports a large number of statistical tests across many tasks, datasets, models, and\ndemographic subgroups (Figures 3, 4, 6, Supplementary Tables), yet there is no mention of adjusting for multiple\ncomparisons (e.g., via Bonferroni correction). Reporting multiple unadjusted p-values raises the risk of overstating the\nsignificance of any observed differences, casting doubt on the reliability of the ~30% site-bias estimate and subgroup\nfindings in Figure 6.\nSubgroup Analysis & Sample Size: The authors draw fairness-related conclusions from subgroup analyses (Figure 6).\nGiven the total dataset sizes and the age distribution data cited (Lines 281–282), sample counts within certain subgroups\n(especially ages <40 and >70) could be critically small, which may render results unstable or prone to outliers. The\ndistribution of test samples and disease prevalence within these subgroups should be explicitly reported.\nPrevalence: The potential influence of varying disease prevalence across demographic subgroups (e.g., DR/DME\nprevalence increasing with age) on subgroup performance metrics (like AUROC) needs clarification and discussion, even if\nthe overall test sets were balanced. Especially in combination with small subgroup sizes <30 (age below 40 in MHE) low\nprevalence in this group may even result in missing samples for a class.\nOvergeneralization of Findings to “Medical Foundation Models”\nScope of \"Medical FMs\": The paper repeatedly frames its findings in the context of \"medical FMs\" generally (Title, Abstract,\nDiscussion). However, the empirical results are derived solely from 2D colour fundus photography, specific ViT\narchitectures, and two SSL methods. The conclusion that age is the dominant fairness driver while sex/ethnicity are minimal\nis specific to this retinal context and cannot be readily generalized. The conclusion that age is the primary driver of fairness\nwhile sex and ethnicity show minimal effects applies to this retinal imaging context and cannot be generalized to all medical\nfoundation models. Sex and ethnicity are known to be significant contributors to bias in other medical AI domains (e.g.,\nradiology, dermatology), so any general statements require caution.\nLimited Technical Scope: The results may not directly extend to different imaging modalities (CT, MRI), task types\n(segmentation, image generation), model architectures (CNNs, multimodal models), or other SSL/training methods.\nClaims of Offering a “Data Selection Framework”:\nThe paper suggests its findings offer \"practical guidance for improving the construction and application of medical FMs\"\n(Lines 319-321) and proposes a \"pipeline for key metadata identification\" (Fig 5a, Lines 447-448). However, the practical\nadvice boils down to selecting FMs pre-trained on similar data (Lines 339-340), which lacks operational specificity. The\nflowchart in Figure 5a adds little value; it's a linear description, not a decision-making tool. This falls short of providing a\nrobust framework\nPotentially Confounded SSL Comparisons\nComparing MAE and DINOv2 performance is complicated by vastly different reported pre-training protocols (epochs: 800 vs\n100; batch size: 1792 vs 320) without discussion of hyperparameter tuning or ablation studies. Observed differences might\nstem from training budget or suboptimal parameters rather than solely the inherent properties of the SSL methods\nthemselves for this task\nMinor Weaknesses\nAmbiguity in Figure 6 - The figure does not specify if the test data is from MEH, SDPP, or both combined, nor how the\nsubgroups are distributed within those sets. Details on fine-tuning sets, class distributions, and data balancing are also\nunclear.\nLabel Sources and Potential Biases - MEH retinal labels come from clinical records; SDPP labels come from expert\nannotations. This difference in label acquisition may partly underlie the performance differences, not just the site distribution.\nThe paper briefly mentions it but does not delve into its implications.\nUnclear Link to Morphological Features - Although morphological feature differences between MEH and SDPP images are\nreported, the quantitative relationship between these differences and downstream task performance remains vague. The\nchoice of morphological indices (e.g., fractal dimension) and their direct relevance to diabetic retinopathy, DME, or stroke\nprediction should be explored.\nMissing Pre-training Disease Distribution Details - The paper reports disease prevalence in the downstream sets but not in\nthe pre-training sets. Figure 2 lacks information on the distribution of relevant diseases (DR, DME, stroke indicators if\npossible) within the MEH and SDPP pre-training datasets, which is highly relevant context. This information on the\ndistribution of relevant diseases (DR, DME, and possibly stroke indicators) in these datasets should be included in Figure 2.\nSmall Performance Gaps Treated as Significant - Some statistically significant differences reported involve very small\nabsolute AUC changes (e.g., <0.01), questioning their practical relevance and the conclusion of one model outperforming\nthe other. For APTOS2019 AUC difference is 0.003; for Glaucoma fundus 0.006; for Diabetic retinopathy\nshanghai fine tune 0.006\nContradictory Results Across Architectures - In some cases, FM-SDPP outperforms FM-MEH for one SSL method (MAE) but\nperforms worse under the other method (DINOv2) (e.g., MESSIDOR2, Retina datasets in Fig 4), making it hard to attribute\nperformance solely to dataset differences versus interactions with the pre-training approach. Summing p-values across\nmodels is not appropriate.\nFigures and Flowcharts -The t-SNE clusterings mentioned in the text (Fig. 5b) are not clearly visible or explained. The\nflowchart in Figure 5a adds no value; it's a linear description, not a decision-making tool. Lack of discussion on metadata\n(age, sex, device, etc.) for the public datasets hinders interpretation of performance differences on these sets in the context of\npre-training data similarity.\nMiscellaneous Issues\nPrecision Scores: AUPRC is calculated, but precision-recall trade-offs are not discussed in the text.\nFigure 2b Specificity: Mentions t-SNE for MAE features but not explicitly for DINOv2 features in the caption/text for\ncomparison.\nTypo: Extended Data Figure 4 caption (line 900) likely means AUPRC, not AUROC, based on the figure content.\nImage Quality: Plotted in Fig 2c but its impact or relevance is not discussed.\nStudy Origins: The different clinical origins of the datasets (MEH/AlzEye vs. SDPP/Diabetes Prevention) might imply\nsystematic differences beyond demographics (e.g., comorbidities, baseline health status) that are not explored.\nAge Binning: Justification for binning age rather than analyzing it as a continuous variable or using more granular bins is\nmissing.\nStrengths:\nParallel Training Design: The core strength is training identical FM architectures with identical methods (per SSL type) on\ntwo distinct, large-scale datasets, providing a clean setup to isolate the impact of pre-training data origin.\nValuable Datasets: Access to and use of large, real-world clinical datasets (MEH, SDPP) adds significant value.\nImportant Problem: The study addresses a timely and critical question about the foundations of generalizability and fairness\nin medical FMs.\nSuggestions\nDeeper Bias Analysis: Investigate potential mechanisms behind observed biases. How do models encode ethnicity/gender,\neven if downstream performance differences weren't consistently significant in this study?\nCombined Dataset Training: Explore if pre-training on a combined MEH+SDPP dataset mitigates the observed age bias and\nsite-specific performance gaps.\nSSL Comparison Rigor: Provide justification for the chosen training protocols or conduct ablation studies to better isolate the\neffect of the SSL strategy itself versus training budget/hyperparameters, especially for claims of DINOv2 superiority.\nExpand Scope (Future Work): Acknowledge limitations more strongly and suggest future work replicating this parallel design\nin other medical domains (e.g., radiology, pathology) to assess the generalizability of these specific findings.\nStatistical Correction: Re-evaluate statistical significance using appropriate corrections for multiple comparisons and report\ncorrected p-values or use confidence intervals more centrally. Clearly report subgroup sample sizes and prevalence.\nContextualize Novelty: Reframe the \"site bias\" finding by clearly acknowledging it confirms existing principles and focus\nnovelty on the quantification within the FM/retinal context, or on deeper analysis if performed.\nRelevant references:\nPractical selection of pre-training data and sub-group analysis to address bias\nYang et al. (Nature Medicine): The limits of fair medical imaging AI in real-world generalization.\nFoundation models do generalize\nCekmeceli et al.: Do Vision Foundation Models Enhance Domain Generalization in Medical Image Segmentation?\nhttps://arxiv.org/pdf/2409.07960\nMitigating intra- and intersite heterogeneity\nChaudhari et al.: Embracing the Disharmony in Heterogeneous Medical Data.\nBias in non-medical foundation models under long-tailed data\nChen et al.: Rethinking the Bias of Foundation Model under Long-tailed Distribution.\nhttps://arxiv.org/pdf/2501.15955\nFairness in DINOv2\nGustafson et al.: FACET: Fairness in Computer Vision Evaluation Benchmark.\nDataset bias in contrastive learning\nPurushwalkam et al. (NeurIPS 2020):\nhttps://proceedings.neurips.cc/paper_files/paper/2020/file/22f791da07b0d8a2504c2537c560001c-Paper.pdf", "reviewer_2": "Noteworthy results\nYukun Zhou and co-authors, in “Revealing the Impact of Pre-training Data on Medical Foundation Models,” explore fairness\nand generalizability of medical foundation models, using ophthalmic fundus photography as the prototypical case.\nThe most important finding is the impact of certain metadata on foundation model (FM) generalizability, in particular with\nregard to age. Since the authors created two different foundation models, one with MEH data and another with SDPP data.\nThe groups differed across a few domains, most notably age. The investigators demonstrated differences between\nfoundation model performance when comparing MEH and SDPP analysis of datasets and also showed the effect of age\nwithin models. This is a key finding, as it speaks to the importance of transparency and bias in model fairness and\ngeneralizability with regard to the datasets used to create the foundation models.\nEthnicity and sex did not appear to affect model performance. The investigators showed “that FMs demonstrate[d] good\ngeneralisability, achieving comparable performance in over 70% of downstream tasks. However, FMs sometimes\nperform[ed] better on application data that aligns with their pre-training data.”\nThe authors point out that “long-term and sustainable advancement of medical AI requires the development of domain-\nspecific techniques tailored to the unique characteristics of medical data and application scenarios… Pre-training clinical\ndata and self-supervised learning strategies have a synergistic effect on FM performance… need to simultaneously optimise\nboth model learning strategies and pre-training data characteristics to advance medical FM development.”\nThe authors make several excellent and important recommendations for future work in this area, including the following:\n• “Future studies should include extra factors particularly concerning disease phenotypes.”\n• “Further research involving a wider range of learning strategies and medical domains is needed.”\n• “There are clear performance differences across 435 various applications.”\nOf note is that the MEH and SDPP samples used to create and test the FMs were unbalanced and a subset of data were\nused to create balanced and curated samples. The authors should comment on how this might have affected the FMs and\ncomparisons.\nSignificance to the field\nThis work has significant potential to enhance the quality of FMs used for analysis of medical data. It is an interesting\nexamination of the role and effect of training data on FM performance and bias.\nConclusions and Claims\nThe authors’ conclusions and claims are supported by their data.\nFlaws in the data analysis, interpretation and conclusions\nNo significant flaws are noted.\nMethodology\nThe methodology is sound, well thought out and well executed. As noted above, the authors should comment on the\nimbalance of the MEH and SDPP datasets.\nReproduction\nThe authors have provided adequate information to enable reproduction of their results, although their data are not readily\navailable to other investigators."}, "author_response": "Dear editors and reviewers,\nThank you for your valuable and instructive comments on our paper studying pre-training\ndata effects on retinal foundation models. We have revised the manuscript in light of your\nsuggestions to bring more insights. The major revisions are summarised below. All revisions\nin the manuscript are highlighted in blue. Please find our point-by-point response below.\nSummary of major revisions to the manuscript\n● Scope aligned to retinal FMs and images.\nWe have tightened the framing to retinal FMs and retinal images, removing\nstatements that casually generalised to the broader medical domain. Revisions\ninvolve the Title, Abstract, Introduction, and Discussion sections.\n● Statistical analysis strengthened.\nWe now report Welch’s t-tests with Holm-Bonferroni correction. All raw p-values,\nadjusted p-values, and significance flags are provided in Supplementary Tables 3-8,\nand the relevant figures and main text have been updated accordingly.\n● Expanded subgroup analysis.\nThe dataset for subgroup analysis has been increased from 2,000 images to 10,014\nimages to mitigate small-sample effects and improve robustness. Subgroup details\nare reported in Supplementary Table 4. The results in Figure 5, Supplementary Table\n5, and the main text have been updated.\n● Clarified research focus and key findings.\nIn the revision, we clarified that our primary focus is to quantify how pre-training data\nshape the generalisability and fairness of retinal FMs. Head-to-head comparisons\nbetween parallel retinal FMs indicate strong cross-site generalisability. Subgroup\nanalyses reveal uneven impacts of pre-training demographic attributes on fairness.\nWe removed text implying a practical data-selection framework (including original\nFigure 5a) from the Abstract, Introduction, and Discussion sections.\n● Positioned within prior literature.\nWe added relevant references, particularly on AI model fairness and bias, to the\nIntroduction and Discussion sections.\n● Extra practical insight into model fairness.\nWe trained FM-MEH-SDPP using 300,000 synthetic images representative of the\nSDPP young cohort (age<40 years) and 604,170 real images from MEH. FM-MEH-\nSDPP achieved overall performance comparable to FM-MEH (Supplementary Table\n7), with improved fairness in certain applications (Supplementary Table 8).\nReviewer #1\nPaper Summary\nThis study investigates the influence of pre-training data characteristics (specifically,\ngeographic origin and associated demographic distributions) on the generalizability and\nfairness of medical foundation models (FMs) in ophthalmology. The authors pre-train Vision\nTransformer-based FMs using two self-supervised learning methods (Masked Autoencoder\n(MAE), and DINOv2) on two large, distinct datasets of retinal fundus photographs: one from\nMoorfields Eye Hospital (MEH) in the UK and one from the Shanghai Diabetes Prevention\nProgram (SDPP) in China. They evaluate these parallel-trained models (FM-MEH and FM-\nSDPP) on downstream tasks (diabetic retinopathy, diabetic macular edema, stroke\nprediction) using held-out data from both sites and several public datasets. Fairness is\nassessed by comparing performance across subgroups defined by age, sex, and ethnicity\nusing MEH test data.\nMain Claims and Observations\nFairness: The paper concludes that for these retinal FMs, the age distribution within the pre-\ntraining data introduces measurable performance bias across downstream task age\nsubgroups, while pre-training data distributions for sex and ethnicity did not result in similar\nconsistent biases. [lines 96, 269 following]\nGeneralizability & Site Bias: The study reports that FMs demonstrate good generalizability,\nperforming comparably across sites and datasets in approximately 70% of evaluations.\nHowever, in the remaining ~30% of cases, FMs perform significantly better when the\ndownstream task data aligns with their specific pre-training data source (intra-site\nadvantage). [lines 322 following]\nResponse: Thank you very much for the accurate summary of our study. We highly\nappreciate your thoughtful comments and suggestions. We hope that our point-to-point\nresponse has addressed the concerns, and we are happy to provide further clarification.\nReview Summary\n1. This paper explores how pre-training data influences the behaviour of medical foundation\nmodels (FMs), employing a robust parallel training strategy on large-scale, high-quality\ndatasets. While it addresses important questions, its main conclusions add little novelty or\npractical significance. The finding that models often perform better on data resembling their\nt resembling their\ntraining distribution (site bias) is already a well-documented principle in machine learning.\nResponse: Thank you for acknowledging the robustness of our parallel training strategy and\nthe significance of the research question. We agree that the impact of training data on\nmachine learning models is well established, including the tendency for models to perform\nbetter on data aligned with their training distribution. Below, we clarify our specific\ncontributions and their practical significance.\n● According to our knowledge, this is the first study investigating FM generalisability\nand fairness using parallel retinal FMs. We developed two parallel retinal FMs, FM-\nMEH and FM-SDPP, that differ only in their pre-training data, to directly test how pre-\ntraining shapes generalisability and fairness on downstream tasks.\n● Substantial and carefully controlled build. Constructing parallel FMs is non-trivial: we\ntrained on two matched, large-scale, real-world datasets (~904k images each) from\nMoorfields Eye Hospital (MEH, UK) and the Shanghai Diabetes Prevention Program\n(SDPP, China), controlling computational resources and implementation details so\nthat pre-training data were the only factor impacting FM performance.\n● Why a head-to-head comparison between parallel FMs matters for generalisability.\nPrior work often compares FMs with baseline models on external data, which shows\nrelative strength but does not investigate whether FMs are generalisable to address\nthe domain gap challenge. Our comparisons between FM-MEH and FM-SDPP on\nboth MEH and SDPP tasks provide direct evidence about this.\n● Isolating the role of pre-training data in fairness. Using parallel FMs allows us to\nattribute fairness differences to pre-training data, rather than to model or training\nconfounders, enabling clearer insight into demographic influences.\n● Main finding 1: Strong cross-site generalisability. The parallel FMs performed\ncomparably on most downstream tasks in each site, despite being pre-trained on\ndifferent populations and imaging distributions. This partially contrasts with the\nconventional expectation that models preferentially perform on data resembling their\ntraining set, posing FMs as generalisable solutions for medical AI.\n● Main finding 2: Demographic attributes have uneven fairness effects. We observed\nfairness gaps across age subgroups for retinal FMs, whereas sex and ethnicity did\nnot show such effects. Data profile indicates substantial distributional variation\n(clinically meaningful morphological indices and latent features) across age groups,\nlikely contributing to the observed age-related fairness gap.\n● Practical mitigation via synthetic data. The age fairness improved when we\nsupplemented pre-training data with synthetic images that provided a complementary\nage distribution.\nIn summary, our analyses provide robust evidence of how pre-training influences retinal\nFMs, highlighting the value of fine-grained, attribute-level data analysis and selection to\noptimise generalisability and fairness.\n2. The approximately 30% figure reported here lacks deeper explanations of how or why this\nbias arises.\nResponse: Thank you for raising this concern. Before addressing this question, we first\ncarefully revised the statistical analyses and added the corresponding description to the\nMethods and Results sections.\nLine 617-623, Page 19:\n“The normality of the model performance was checked via Shapiro-Wilk test.\nStatistical significance is calculated using Welch’s t-test, followed by Holm-Bonferroni\ncorrection. When comparing FM-MEH and FM-SDPP, the p-value was adjusted to\nthe size of 24 (3*SDPP tasks, 3*MEH tasks, 6*publicly available datasets, each\nincluding fine-tuning and linear probing). In subgroup analysis, for each demographic\nattribute, the p-value was adjusted to the size of 8 (2*MEH tasks, 2*Self-supervised\nlearning methods, each including fine-tuning and linear probing). All raw p-values,\nadjusted p-values, and significances are listed in Supplementary Table 3-8”\nLine 215-220, Page 7:\n“As shown in Figure 3, the FMs demonstrated good generalisability, with FM-MEH\nand FM-SDPP showing comparable performance on most evaluations. When\nadapted to SDPP downstream tasks (Figure 3a and 3b), FM-MEH and FM-SDPP\nperformed comparably on all evaluations. Similarly, when adapted to MEH\ndownstream tasks, FM-MEH significantly outperformed FM-SDPP on only three\nevaluations (ischaemic stroke prediction tasks), as shown in Figures 3c and 3d.\nAUPRC performance followed a similar trend, illustrated in Extended Data Figure 4.”\nAfter strengthening the statistical analyses, we found that only three significant differences\nremained in comparing intra-site and inter-site adaptation performance (Figure 3), all relating\nto ischaemic stroke prediction at the MEH site. Potential reasons for this were discussed in\nthe Discussion section.\nLine 342-355, Page 10:\n“Retinal FMs are more likely to demonstrate a generalisation gap in clinical tasks that\nhinge on subtle morphological manifestations, e.g. ischaemic stroke prediction in our\nstudy. FM-MEH significantly outperformed FM-Shanghai only for ischaemic stroke\nprediction, using both masked autoencoder and DINOv2 pre-training strategies\n(Figure 3). Compared to ocular disease detection (e.g. diabetic retinopathy and\ndiabetic oedema), which targets well-defined lesions with abnormal colour and\ntexture patterns, predicting systemic diseases from retinal images (termed as\nOculomics [49,50]) typically depends on subtle changes in retinal anatomical tissues,\nsuch as vasculature and optic disc, arising from metabolic and circulatory\ndysfunction. Prior work has shown that retinal vascular calibre, tortuosity, fractal\ndimension, and optic-disc features carry systemic risk signals [40–46]. For example,\nnarrower retinal arterioles and wider venules have been associated with long-term\nrisk of mortality and ischaemic stroke (both sexes) and coronary heart disease (in\nwomen) [42], and patients with Alzheimer’s disease exhibit narrower venules with\nsparser, more tortuous vasculature compared with controls. Our results point to\ngreater generalisation challenges for such Oculomics tasks and, for the time being,\nsupport using retinal FMs pre-trained on local data when available for these\napplications.”\nAdditionally, we investigated the generalisation gaps to publicly available datasets (Figure 4)\nand measured squared Maximum Mean Discrepancy to interpret the performance gaps.\nLine 355-364, Page 11:\n“Additionally, we measured the squared Maximum Mean Discrepancy (MMD²)\nbetween each public dataset and the MEH and SDPP cohorts (Supplementary Table\n9) and visualised feature distributions using t-SNE (Extended Data Fig. 9).\nMMD² offers partial explanatory power for the generalisation gap between FM-MEH\nand FM-SDPP. For example, with MAE-based features, MESSIDOR2 distributes\ncloser to SDPP than to MEH and FM-SDPP performs better; with DINOv2-based\nfeatures, MESSIDOR2 is closer to MEH and FM-MEH performs better. However, this\npattern does not hold for all datasets (e.g. Retina), indicating that distance metrics\nalone are insufficient to account for performance and generalisability differences.\nWhile FMs substantially mitigate cross-site generalisation gaps, residual disparities\nwarrant further investigation.”\n3. Furthermore, the finding that age distribution affects fairness, while potentially valid for this\nspecific retinal imaging context, is presented with claims that overreach its generalizability to\n\"medical FMs\" broadly.\nResponse: Thank you for raising this issue. We have tightened the framing to retinal FMs\nand retinal images, removing statements that improperly generalised to the broader medical\ndomain. These include:\nTitle: “Revealing the Impact of Pre-training Data on Retinal Foundation Models”\nAbstract: “Medical foundation models (FMs), pre-trained on large-scale unlabelled data,\nshow strong performance and data efficiency when adapted to various clinically relevant\napplications. However, how pre-training data shape the generalisability and fairness of these\nmodels, such as retinal FMs, remains unexplored. We address this using two large retinal\nimage cohorts from Moorfields Eye Hospital (UK) and the Shanghai Diabetes Prevention\nProgram (China), each containing 904,170 retinal images for FM pre-training. Using identical\npipelines, we trained parallel FMs using individual image cohort and evaluated them on\ndownstream tasks with publicly available datasets and held-out data from each site. The\nparallel FMs achieved comparable performance on most downstream tasks, indicating\nstrong generalisability to data that differ substantially from their pre-training data.\nNevertheless, we observed fairness gaps over age subgroups for retinal FMs, whereas\nsex and ethnicity showed no such impact. These results serve as quantitative evidence of\nthe good generalisability of retinal FMs and reveal that pre-training demographic attributes\nshape fairness differently. This inspires domain-specific, fine-grained investigation of pre-\ntraining data effect to support evidence-based data selection to improve both generalisability\nand fairness in FM development.”\nLine 147-148, Page 5:\n“To address these gaps, we investigated the impact of pre-training data on retinal\nFMs, one of the pioneering areas in medical FM”\nLine 157-166, Page 5:\n“Our results showed the strong generalisability of retinal FMs. For example, FM-MEH\nand FM-SDPP performed comparably across all SDPP downstream tasks. However,\nthe models exhibited performance gaps over age subgroups (not seen in sex and\nethnicity), likely caused by larger distributional shift across age subgroups. The age\nfairness improved when we supplemented pre-training data with synthetic images\nthat provided a complementary age distribution. Through extensive experiments with\nreal-world clinical data, this study examined the impact of pre-training data on retinal\nFMs. While our study focuses on retinal images, the observed combination of strong\ngeneralisability and uneven demographic effects may extend to broader medical\nFMs. These findings suggest an evidence-based approach to data description and\nselection in FM development to improve both efficiency and capabilities.”\nLine 315-321, Page 10:\n“We developed parallel retinal FMs with identical implementations that differed only in\ntheir pre-training data, so as to isolate the pre-training data effect and provide robust\nevidence. Our results show that retinal FMs have good generalisability, achieving\ncomparable performance in inter-site adaptation compared to intra-site one. Despite\nthis, fairness gaps were observed across age subgroups, but not across sex or\nethnicity, potentially driven by stronger age-related data variation captured by the\nFMs. These findings suggest fine-grained, attribute-level data analysis of pre-training\ndata, working as quantitative evidence for efficient data selection in FM\ndevelopment.”\nLine 365-366, Page 11:\n“Pre-training data attributes, such as demographics, shape retinal FM generalisability\nand fairness differently, highlighting the need for fine-grained, attribute-level data\nanalysis and selection.”\nLine 373-378, Page 11:\n“This suggests that, at least for retinal images, simply increasing range and balance\nof ethnicity and sex may not necessarily enhance FM fairness in downstream tasks,\nwhile ensuring a balanced and wide-ranging age distribution can improve both\nfairness and generalisability. These findings underscore the necessity of fine-grained\ndata analysis for efficient data sampling in medical FM development, before uniformly\nscaling data across all metadata dimensions.”\nLine 388-390, Page 12:\n“In our retinal FM setting, age distribution produced significant subgroup differences\nin both morphological indices and latent features, and it significantly impacted the FM\ngeneralisability and fairness.”\nLine 445-448, Page 13:\n“Whether good generalisability exhibited by retinal FMs and uneven impact of data\nattributes extend to other task types (e.g. segmentation, regression tasks) and\nmedical fields (e.g. pathology, radiology) is unknown. Further research involving a\nwider range of learning strategies, task types, and medical domains with a replicated\nparallel design is needed.”\nAdditionally, we replaced the term “medical FMs” with “retinal FMs” wherever it referred to\nour results and conclusions, to reflect the study’s scope accurately.\n4. The finding regarding site-specific performance benefits is somewhat inconclusive, lacking\na clear framework for practical data selection beyond reaffirming that data distribution\nmatters.\nResponse: data shape the generalisability and fairness of retinal FMs. From extensive\nexperiments, we draw two key findings: 1) retinal FMs generalise well across sites, and 2)\ndemographic attributes influence fairness unevenly. These results partly depart from the\nconventional expectation that models perform best on data resembling their training\ndistribution, motivating fine-grained, attribute-level data sampling in FM development to\noptimise both generalisability and fairness. While a practical data-selection framework based\non these insights would be valuable, it lies beyond the scope of this work and warrants a\ndedicated study with tailored designs and comprehensive evaluation. Here, we address the\ncore question, how pre-training data affect retinal FMs, by developing parallel retinal FMs,\nevaluating diverse downstream tasks, and conducting subgroup analyses across multiple\ndemographic attributes. To avoid any confusion, we have removed language implying a\npractical data-selection framework (including the original Figure 5a) from the Abstract,\nIntroduction, and Discussion sections.\n5. Statistical methods require further clarification (especially given multiple comparisons),\npotentially undermining the reliability of the reported quantitative findings.\nResponse: This suggestion is really important and helpful. We now report Welch’s t-tests\nwith Holm–Bonferroni correction. The raw p-values, adjusted p-values, and significance flags\nare provided in Supplementary Tables 3-8, and the relevant figures and main text have been\nupdated accordingly. This helps to provide a more reliable and robust statistical analysis.\nThe revision includes:\nLine 617-623, Page 19:\n“The normality of the model performance was checked via Shapiro-Wilk test.\nStatistical significance is calculated using Welch’s t-test, followed by Holm-Bonferroni\ncorrection. When comparing FM-MEH and FM-SDPP, the p-value was adjusted to\nthe size of 24 (3*SDPP tasks, 3*MEH tasks, 6*publicly available datasets, each\nincluding fine-tuning and linear probing). In subgroup analysis, for each demographic\nattribute, the p-value was adjusted to the size of 8 (2*MEH tasks, 2*Self-supervised\nlearning methods, each including fine-tuning and linear probing). All raw p-values,\nadjusted p-values, and significances are listed in Supplementary Table 3-8”\nLine 215-220, Page 7:\n“As shown in Figure 3, the FMs demonstrated good generalisability, with FM-MEH\nand FM-SDPP showing comparable performance on most evaluations. When\nadapted to SDPP downstream tasks (Figure 3a and 3b), FM-MEH and FM-SDPP\nperformed comparably on all evaluations. Similarly, when adapted to MEH\ndownstream tasks, FM-MEH significantly outperformed FM-SDPP on only three\nevaluations (ischaemic stroke prediction tasks), as shown in Figures 3c and 3d.\nAUPRC performance followed a similar trend, illustrated in Extended Data Figure 4.”\nLine 226-235, Page 7:\n“As shown in Figure 4, FM-SDPP and FM-MEH showed comparable performance in\na majority of evaluations (8 out of 12 for Masked Autoencoder-based FMs; 10 out of\n12 for DINOV2-based FMs). When pre-trained with Masked Autoencoder and fine-\ntuned to the downstream tasks (Figure 4a), FM-SDPP significantly outperformed FM-\nMEH on three out of six datasets (APTOS2019, MESSIDOR2, and Retina datasets),\nwhile the average performance gap on APTOS2019 is minor. When adapted to\ndownstream tasks using linear probing (Figure 4c), FM-MEH performed significantly\nbetter on the Glaucoma fundus dataset (p<0.01). When using DINOV2 for FM pre-\ntraining (Figure 4b and Figure 4d), FM-MEH significantly outperformed FM-SDPP in\ntwo applications, i.e. fine-tuning to MESSIDOR2 (0.01<p<0.05) and linear probing to\nIDRiD (p<0.01).”\nLine 240-253, Page 8:\n“We evaluated FM fairness across demographic attribute subgroups in downstream\ntasks. Using a curated MEH dataset (details introduced in Supplementary Table 4),\nwe examined subgroup performance on diabetic retinopathy detection and diabetic\nmacular oedema detection, considering FM-SDPP and FM-MEH showed no\nsignificant differences in overall performance on these tasks (Figure 3c), and MEH\ndata includes multiple ethnical groups. The SDPP pre-training data is mainly\ndistributed over the young (<40 years) and middle-aged groups (40-70 years), with a\nsubgroup ratio of (0.279, 0.685, 0.036), while MEH data is skewed towards middle-\nage and older cohorts (>70 years), with a ratio of (0.015, 0.467, 0.518). As shown in\nFigure 5a, FM-SDPP performed consistently better than FM-MEH in the young group\n(grey bars) but worse in the aged group (yellow bars). For instance, in diabetic\nretinopathy detection using a linear probe, FM-SDPP outperformed FM-MEH by an\naverage AUROC of 0.073 in the young group (p<0.01) while underperforming by\n0.047 in the aged group (p<0.01). FM-MEH and FM-SDPP demonstrated similar\nperformance in the middle-aged group. This consistent performance gap across\ndiverse applications demonstrated the FM bias introduced by differential age\ndistribution in pre-training data.”\nRevised Figures are listed below, while Supplementary Tables are not included here due to\nthe table size:\nFigure 3. AUROC performance of FM-MEH and FM-SDPP on downstream tasks using data from\neach site. Subgraphs a and b show the model performance on tasks at the SDPP site, with FMs\nrespectively pre-trained with Masked Autoencoder and DINOV2. Subgraphs c and d present the\nperformance of FMs on tasks at the MEH site. In SDPP downstream tasks, FM-SDPP and FM-MEH\nshowed comparable performance, while FM-MEH achieved superior performance on 3 out of 12\nevaluations when adapted to MEH downstream tasks. For each task, models were fine-tuned with five\ndifferent random seeds, controlling the shuffling of fine-tuning data, and evaluated on the test set to\ngenerate five replicates. The mean AUROC values are represented by bar centres, with error bars\nindicating 95% confidence intervals (CI). A Welch’s t-test followed by Holm-Bonferroni correction was\nused to assess whether the performance differences between FM-SDPP and FM-MEH were\nstatistically significant. * indicates 0.01<p<0.05 and ** indicates p<0.01. n indicates the number of\ncases showing significant differences.\nFigure 4. AUROC performance of FM-MEH and FM-SDPP on downstream tasks using publicly\navailable datasets sourced from multiple countries. Subgraphs a and b show the performance of FMs\npre-trained respectively with Masked Autoencoder and DINOV2 when fine-tuned to downstream\ntasks. Subgraphs c and d present the performance of FMs when adapted to downstream tasks with\nlinear probing. When pre-trained with Masked Autoencoder, FM-SDPP significantly outperformed FM-\nMEH on 3 downstream evaluations. When pre-trained with DINOV2, FM-MEH significantly\noutperformed FM-SDPP on 2 evaluations. For each task, models were fine-tuned with five different\nrandom seeds, controlling the shuffling of fine-tuning data, and evaluated on the test set to generate\nfive replicates. The mean AUROC values are represented by bar centres, with error bars indicating\n95% CI. A Welch’s t-test followed by Holm-Bonferroni correction was used to assess whether the\nperformance differences between FM-SDPP and FM-MEH were statistically significant. * indicates\n0.01<p<0.05 and ** indicates p<0.01. n indicates the number of cases showing significant differences.\nFigure 5. Performance differences between FM-MEH and FM-SDPP on diabetic retinopathy and\ndiabetic macular oedema detection across age (a), ethnicity (b), and sex (c) subgroups on MEH data.\nYellow bars indicate that FM-MEH outperforms FM-SDPP, while grey bars indicate the opposite. The\nleft two columns include the results for FMs pre-trained with Masked Autoencoder, while the right two\ncolumns show results for FMs pre-trained with DINOV2. Each section includes the results with fine-\ntuning and the linear probe. We observed consistent and significant performance gaps between FM-\nMEH and FM-SDPP in the age section, but not in the sex and ethnicity sections. The bar centres\nrepresent the mean AUROC differences. A Welch’s t-test followed by Holm-Bonferroni correction was\nused to assess whether the performance differences between FM-SDPP and FM-MEH were\nstatistically significant. * indicates 0.01<p<0.05 and ** indicates p<0.01. n indicates the number of\ncases showing significant differences.\nExtended Data Figure 4. AUPRC performance of FM-MEH and FM-SDPP on downstream tasks using\ndata from each site. Subgraphs a and b show the model performance on tasks at the SDPP site, with\nFMs respectively pre-trained with Masked Autoencoder and DINOV2. Subgraphs c and d present the\nperformance of FMs on tasks at the MEH site. In SDPP downstream tasks, FM-SDPP and FM-MEH\nachieved comparable performance, while FM-MEH achieved superior performance on 4 out of 12\nevaluations when adapted to MEH downstream tasks. For each task, models were fine-tuned with five\ndifferent random seeds, controlling the shuffling of fine-tuning data, and evaluated on the test set to\ngenerate five replicates. The mean AUPRC values are represented by bar centres, with error bars\nindicating 95% CI. A Welch’s t-test followed by Holm-Bonferroni correction was used to assess\nwhether the performance differences between FM-SDPP and FM-MEH were statistically significant. *\nindicates 0.01<p<0.05 and ** indicates p<0.01. n indicates the number of cases showing significant\ndifferences.\nExtended Data Figure 5. AUPRC performance of FM-MEH and FM-SDPP on downstream tasks using\npublicly available datasets sourced from multiple countries. Subgraphs a and b show the performance\nof FMs pre-trained respectively with Masked Autoencoder and DINOV2 when fine-tuned to\ndownstream tasks. Subgraphs c and d present the performance of FMs when adapted to downstream\ntasks with the linear probe. When pre-trained with Masked Autoencoder, FM-SDPP significantly\noutperformed FM-MEH on 4 downstream evaluations. When pre-trained with DINOV2, FM-MEH\nsignificantly outperformed FM-SDPP on 2 evaluations. For each task, models were fine-tuned with\nfive different random seeds, controlling the shuffling of fine-tuning data, and evaluated on the test set\nto generate five replicates. The mean AUROC values are represented by bar centres, with error bars\nindicating 95% CI. A Welch’s t-test followed by Holm-Bonferroni correction was used to assess\nwhether the performance differences between FM-SDPP and FM-MEH were statistically significant. *\nindicates 0.01<p<0.05 and ** indicates p<0.01. n indicates the number of cases showing significant\ndifferences.\n6. The authors would benefit from more thorough contextualization of their results relative to\nprevious works on fairness and bias in medical foundation models.\nResponse: Thank you for your suggestion. We have now added the description of previous\nliterature in the Introduction and Discussion sections:\nLine 131-133, Page 4:\n“Prior studies have systematically shown that the distribution of fine-tuning data,\nparticularly demographic attributes such as age and sex, can be encoded by AI\nmodels and lead to fairness gaps [23,25].”\nLine 407-423, Page 12:\n“This study provides quantitative evidence that the composition of pre-training data\nshapes the generalisability and fairness of retinal FMs. Some of our findings align\nwith prior work on generalisability and fairness in medical imaging. For example,\nYang et al. showed that models encoding more demographic attributes (e.g. age,\nsex) are prone to fairness gap under external evaluation across radiology,\ndermatology, and ophthalmology [60]. We extend this observation to FMs. As shown\nin Figure 5 and 6, differences in age distribution within the pre-training set are\nperceived by retinal FMs (age<40, 40<age<70, 70<age) and fairness gap across age\nsubgroups is observed. Kerem et al. verified the good generalisability of vision FMs\nin medical image segmentation [24]. We also confirmed strong generalisability of\nretinal FMs. Importantly, rather than comparing FMs to baselines in external\nevaluation, we trained parallel FMs and conducted head-to-head intra-site vs inter-\nsite comparison, showing that external performance matches internal performance, a\nstricter demonstration of generalisability. Wang et al. used auxiliary demographic\ntasks to improve external performance [25], while in our study, we verified that retinal\nFMs are highly generalisable. To our knowledge, this is the first study to use parallel\nretinal FMs to rigorously isolate the impact of pre-training data on both\ngeneralisability and fairness, an urgent and under-studied question. Together, these\nresults both converge with the existing literature and advance it in FM area by\nshowing how targeted pre-training curation can steer fairness improvement in\nmedical foundation models.”\nLine 425-434, Page 13:\n“Our experiments with FM-MEH-SDPP echo prior findings that synthetic data can\ncontribute effectively to FM pre-training [60,61]. Moreover, our fine-grained, age-\nconditioned image synthesis further verified the benefits of increasing the diversity\nand balance of age, a demographic attribute that strongly impacts both morphological\nindices and latent features. Although FM-MEH-SDPP performed comparably with\nFM-MEH on the general cohort, it significantly improved the performance on younger\ncohorts on certain applications. While synthetic images cannot perfectly recapitulate\nreal distributions, and we conditioned on age only, the observed subgroup\nimprovements indicate that age is indeed an important metadata that should be\nprioritised in data selection for retinal foundation models. Nevertheless, signals on\nFM generalisability and how pre-training data impacts FM fairness may differ across\nmedical domains and warrant domain-specific investigation, which we advocate\nthrough this study.”\n7. Combined with numerous minor weaknesses, the paper overstates its contributions and\nrequires substantial revision, potentially bordering on rejection due to the limited novelty and\npotential statistical issues.\nResponse: Thank you again for providing the detailed comments and suggestions. We hope\nour revision addressed your concerns.\nMajor Weaknesses\n8. Limited Novelty and Depth of “Site-Bias” Finding. The core finding that models can exhibit\nbetter performance on data distributions matching their training data is not novel; it's\na fundamental concept in domain adaptation and generalization research. While quantifying\nthis effect for these specific FMs (~30% intra-site advantage) provides a data point, the\npaper falls short of providing significant new conceptual understanding or predictive insight\ninto when or why this site bias occurs for FMs. This finding also stands in mild tension with\nliterature suggesting large-scale self-supervised training can mitigate certain biases.\nResponse: Thank you for raising this concern. As noted in our response to Question 1, after\nrevision, we report two main findings that we believe add value to the community:\n● Main finding 1: Strong cross-site generalisability. The parallel FMs, FM-MEH and FM-\nSDPP, performed comparably on most downstream tasks in each site, despite being\npre-trained on different populations and imaging distributions. This partially contrasts\nwith the conventional expectation that models preferentially perform on data resembling their\ntraining set, posing FMs as generalisable solutions for medical AI.\n● Main finding 2: Demographic attributes have uneven fairness effects. We observed\nfairness gaps across age subgroups for retinal FMs, whereas sex and ethnicity did\nnot show such effects. Data profile indicates substantial distributional variation\n(clinically meaningful morphological indices and latent features) across age groups,\nlikely contributing to the observed age-related fairness gap.\nWe also noted that the three significant differences between FM-MEH and FM-SDPP at the\nMEH site pertain to ischaemic stroke prediction. We discuss potential reasons in the\nDiscussion section\nLine 341-363, Page 10-11:\n“Retinal FMs are more likely to demonstrate a generalisation gap in clinical tasks that\nhinge on subtle morphological manifestations, e.g. ischaemic stroke prediction in our\nstudy. FM-MEH significantly outperformed FM-Shanghai only for ischaemic stroke\nprediction, using both masked autoencoder and DINOv2 pre-training strategies\n(Figure 3). Compared to ocular disease detection (e.g. diabetic retinopathy and\ndiabetic oedema), which targets well-defined lesions with abnormal colour and\ntexture patterns, predicting systemic diseases from retinal images (termed as\nOculomics [49,50]) typically depends on subtle changes in retinal anatomical tissues,\nsuch as vasculature and optic disc, arising from metabolic and circulatory\ndysfunction. Prior work has shown that retinal vascular calibre, tortuosity, fractal\ndimension, and optic-disc features carry systemic risk signals [40–46]. For example,\nnarrower retinal arterioles and wider venules have been associated with long-term\nrisk of mortality and ischaemic stroke (both sexes) and coronary heart disease (in\nwomen) [42], and patients with Alzheimer’s disease exhibit narrower venules with\nsparser, more tortuous vasculature compared with controls. Our results point to\ngreater generalisation challenges for such Oculomics tasks and, for the time being,\nsupport using retinal FMs pre-trained on local data when available for these\napplications. Additionally, we measured the squared Maximum Mean Discrepancy\n(MMD²) between each public dataset and the MEH and SDPP cohorts\n(Supplementary Table 9) and visualised feature distributions using t-SNE (Extended\nData Fig. 9). MMD² offers partial explanatory power for the generalisation gap\nbetween FM-MEH and FM-SDPP. For example, with MAE-based features,\nMESSIDOR2 distributes closer to SDPP than to MEH and FM-SDPP performs better;\nwith DINOv2-based features, MESSIDOR2 is closer to MEH and FM-MEH performs\nbetter. However, this pattern does not hold for all datasets (e.g. Retina), indicating\nthat distance metrics alone are insufficient to account for performance and\ngeneralisability differences. While FMs substantially mitigate cross-site generalisation\ngaps, residual disparities warrant further investigation.”\nUsing parallel retinal FMs, our study provided robust evidence for FM generalisability and\nfairness:\n● Why a head-to-head comparison between parallel FMs matters for generalisability.\nPrior work often compares FMs with baseline models on external data, which shows\nrelative strength but does not investigate whether FMs are generalisable to address\nthe domain gap challenge. Our comparisons between FM-MEH and FM-SDPP on\nboth MEH and SDPP tasks provide direct evidence about this.\n● Isolating the role of pre-training data in fairness. Using parallel FMs allows us to\nattribute fairness differences to pre-training data, rather than to model or training\nconfounders, enabling clearer insight into demographic influences.\n9. Concerns Regarding Statistical Rigor. Multiple Comparisons: The paper reports a large\nnumber of statistical tests across many tasks, datasets, models, and demographic\nsubgroups (Figures 3, 4, 6, Supplementary Tables), yet there is no mention of adjusting for\nmultiple comparisons (e.g., via Bonferroni correction). Reporting multiple unadjusted p-\nvalues raises the risk of overstating the significance of any observed differences, casting\ndoubt on the reliability of the ~30% site-bias estimate and subgroup findings in Figure 6.\nResponse: Thank you. Please see our response to Question 5.\n10. Subgroup Analysis & Sample Size: The authors draw fairness-related conclusions from\nsubgroup analyses (Figure 6). Given the total dataset sizes and the age distribution data\ncited (Lines 281–282), sample counts within certain subgroups (especially ages <40\nand >70) could be critically small, which may render results unstable or prone to outliers.\nThe distribution of test samples and disease prevalence within these subgroups should be\nexplicitly reported.\nResponse: We appreciate the reviewer highlighting this point. To minimise bias and\nsensitivity to outliers from small subgroup sizes, we expanded the subgroup analysis: the\ndataset increased from 2,000 to 10,014 images to improve analysis robustness. Detailed\ncounts are provided in Supplementary Table 4. Even in the least represented age group\n(<40 years) at MEH site, case numbers for each disease category are reasonably adequate.\nFigure 5, Supplementary Table 5, and the corresponding main text have been updated\naccordingly.\nLine 240-253, Page 8:\n“We evaluated FM fairness across demographic attribute subgroups in downstream\ntasks. Using a curated MEH dataset (details introduced in Supplementary Table 4),\nwe examined subgroup performance on diabetic retinopathy detection and diabetic\nmacular oedema detection, considering FM-SDPP and FM-MEH showed no\nsignificant differences in overall performance on these tasks (Figure 3c), and MEH\ndata includes multiple ethnical groups. The SDPP pre-training data is mainly\ndistributed over the young (<40 years) and middle-aged groups (40-70 years), with a\nsubgroup ratio of (0.279, 0.685, 0.036), while MEH data is skewed towards middle-\nage and older cohorts (>70 years), with a ratio of (0.015, 0.467, 0.518). As shown in\nFigure 5a, FM-SDPP performed consistently better than FM-MEH in the young group\n(grey bars) but worse in the aged group (yellow bars). For instance, in diabetic\nretinopathy detection using a linear probe, FM-SDPP outperformed FM-MEH by an\naverage AUROC of 0.073 in the young group (p<0.01) while underperforming by\n0.047 in the aged group (p<0.01). FM-MEH and FM-SDPP demonstrated similar\nperformance in the middle-aged group. This consistent performance gap across\ndiverse applications demonstrated the FM bias introduced by differential age\ndistribution in pre-training data.”\n11. Prevalence: The potential influence of varying disease prevalence across demographic\nsubgroups (e.g., DR/DME prevalence increasing with age) on subgroup performance metrics\n(like AUROC) needs clarification and discussion, even if the overall test sets were balanced.\nEspecially in combination with small subgroup sizes <30 (age below 40 in MHE) low\nprevalence in this group may even result in missing samples for a class.\nResponse: Thank you for suggesting clarification on disease prevalence within subgroups.\nWe have added fine-grained case counts for the subgroup analysis in Supplementary Table\n4. MEH data, as a hospital-based cohort, exhibits generally high disease. For example, in\nthe least represented age group (<40 years), the test set includes 21, 29, 31, 31, and 74\ncases across the five stages of diabetic retinopathy, and 134, 26, and 26 cases for diabetic\nmacular oedema. Accordingly, there is no long-tail issue or missing-class problem in the\nsubgroup analysis.\n12. Overgeneralization of Findings to “Medical Foundation Models”. Scope of \"Medical FMs\":\nThe paper repeatedly frames its findings in the context of \"medical FMs\" generally (Title,\nAbstract, Discussion). However, the empirical results are derived solely from 2D colour\nfundus photography, specific ViT architectures, and two SSL methods. The conclusion that\nage is the dominant fairness driver while sex/ethnicity are minimal is specific to this retinal\ncontext and cannot be readily generalized. The conclusion that age is the primary driver of\nfairness while sex and ethnicity show minimal effects applies to this retinal imaging context\nand cannot be generalized to all medical foundation models. Sex and ethnicity are known to\nbe significant contributors to bias in other medical AI domains (e.g., radiology, dermatology),\nso any general statements require caution.\nResponse: Thank you. We agree that precise scoping is important. Please see our\nresponse to Question 3.\n13. Limited Technical Scope: The results may not directly extend to different imaging\nmodalities (CT, MRI), task types (segmentation, image generation), model architectures\n(CNNs, multimodal models), or other SSL/training methods.\nResponse: Thank you for raising this concern. While our results may not directly generalise\nto other imaging modalities, task types, or model architectures, our head-to-head evaluation\nof parallel retinal FMs, trained on two large, real-world cohorts using two representative self-\nsupervised methods, offers valuable insights into how pre-training data influence retinal FM\ngeneralisability and fairness. These findings can inspire more efficient retinal FM\ndevelopment and, importantly, encourage domain-specific, fine-grained data analysis and\nselection across medical domains. We also acknowledge this as a limitation in the\nDiscussion section:\nLine 442-448, Page 13:\n“Second, due to the considerable workload involved in developing parallel FMs and\norganising diverse downstream tasks, this study primarily focused on representative\nself-supervised learning strategies, such as Masked Autoencoder and DINOV2, and\nused retinal images as an exemplar. Whether good generalisability exhibited by\nretinal FMs and uneven impact of data attributes extend to other task types (e.g.\nsegmentation, regression tasks) and medical fields (e.g. pathology, radiology) is\nunknown. Further research involving a wider range of learning strategies, task types,\nand medical domains with a replicated parallel design is needed.”\n14. Claims of Offering a “Data Selection Framework”: The paper suggests its findings offer\n\"practical guidance for improving the construction and application of medical FMs\" (Lines\n319-321) and proposes a \"pipeline for key metadata identification\" (Fig 5a, Lines 447-448).\nHowever, the practical advice boils down to selecting FMs pre-trained on similar data (Lines\n339-340), which lacks operational specificity. The flowchart in Figure 5a adds little value; it's a linear description, not a decision-making tool. This falls short of providing a robust\nframework\nResponse: We apologise for the wording and resulting confusion. In the revision, we clarify\nthat our primary aim is to quantify how pre-training data shape the generalisability and\nfairness of retinal FMs. Head-to-head comparisons between parallel retinal FMs show strong\ncross-site generalisability, and subgroup analyses reveal uneven impacts of pre-training\ndemographic attributes on fairness. We have removed text implying a practical data-\nselection framework (including the original Figure 5a) from the Abstract, Introduction, and\nDiscussion.\n15. Potentially Confounded SSL Comparisons. Comparing MAE and DINOv2 performance is\ncomplicated by vastly different reported pre-training protocols (epochs: 800 vs 100;\nbatch size: 1792 vs 320) without discussion of hyperparameter tuning or ablation studies.\nObserved differences might stem from training budget or suboptimal parameters rather than\nsolely the inherent properties of the SSL methods themselves for this task.\nResponse: We acknowledge that the previous descriptions of the MAE vs. DINOv2\ncomparison are arbitrary. Considering the complex confounders relevant to their\nperformance, we explicitly introduced our experimental conditions, which do not necessarily\nproduce the best-performing models for comparison.\nLine 392-405, Page 12:\n“Advancements in general-purpose AI techniques (e.g. self-supervised learning\nmethods) continue to push the performance boundaries in our applications. In this\nstudy, we included and compared two representative self-supervised learning\nstrategies, i.e. generative-based method (Masked Autoencoder) and self-distillation\nmethod (DINOV2). Under the same compute budget (identical GPU settings and\ntraining time), DINOV2 achieved superior performance in downstream tasks on both\nsites (Supplementary Table 5). This also extended to publicly available datasets\n(Supplementary Table 5). The advantage likely reflects a combination of pre-training\nstrategies (i.e. DINO [58] and iBoT [59] capturing image-level and patch-level\nfeatures respectively), practical refinements (e.g. Sinkhorn-Knopp centring [48]), and\ngeneralisable features learnt from initial large-scale pre-training on 142 million natural\nimages [23]. However, it is noted that retinal FMs developed in this study, with either\nMasked Autoencoder or DINOv2, were not optimised to their theoretical best. We\npre-trained these models out-of-the-box on large-scale retinal datasets, without\nextensive hyperparameter searches. Even so, these models achieved competitive\nperformance across diverse downstream applications, illustrating the value of\ntranslational approaches that initialise medical FMs from evolving general AI\nmodels.”\nMinor Weaknesses\n16. Ambiguity in Figure 6 - The figure does not specify if the test data is from MEH, SDPP, or\nboth combined, nor how the subgroups are distributed within those sets. Details on fine-\ntuning sets, class distributions, and data balancing are also unclear.\nResponse: Thank you for raising this issue. We have now explicitly mentioned that we used\nMEH data for subgroup analysis in the Figure caption and main text.\nFigure 5. Performance differences between FM-MEH and FM-SDPP on diabetic retinopathy and\ndiabetic macular oedema detection across age (a), ethnicity (b), and sex (c) subgroups on MEH data.\nYellow bars indicate that FM-MEH outperforms FM-SDPP, while grey bars indicate the opposite. The\nleft two columns include the results for FMs pre-trained with Masked Autoencoder, while the right two\ncolumns show results for FMs pre-trained with DINOV2. Each section includes the results with fine-\ntuning and the linear probe. We observed consistent and significant performance gaps between FM-\nMEH and FM-SDPP in the age section, but not in the sex and ethnicity sections. The bar centres\nrepresent the mean AUROC differences. A Welch’s t-test followed by Holm-Bonferroni correction was\nused to assess whether the performance differences between FM-SDPP and FM-MEH were\nstatistically significant. * indicates 0.01<p<0.05 and ** indicates p<0.01. n indicates the number of\ncases showing significant differences.\nLine 240-244, Page 8:\n“We evaluated FM fairness across demographic attribute subgroups in downstream\ntasks. Using a curated MEH dataset (details introduced in Supplementary Table 4),\nwe examined subgroup performance on diabetic retinopathy detection and diabetic\nmacular oedema detection, considering FM-SDPP and FM-MEH showed no\nsignificant differences in overall performance on these tasks (Figure 3c), and MEH\ndata includes multiple ethnical groups.”\n17. Label Sources and Potential Biases - MEH retinal labels come from clinical records;\nSDPP labels come from expert annotations. This difference in label acquisition may partly\nunderlie the performance differences, not just the site distribution. The paper briefly\nmentions it but does not delve into its implications.\nResponse: Thank you for the question. As we compared FM-MEH and FM-SDPP on MEH\ntasks and SDPP tasks separately, differences in label resources do not compromise fair\ncomparisons. As shown in the study pipeline (Figure 1), the fine-tuning and evaluation data\nfor each task were identical for both models, ensuring a fair comparison. We have also\nrevised the Limitations paragraph to clarify this point\nLine 448-453, Page 13:\n“Third, due to the differences in the sources of labels (e.g. MEH diabetic retinopathy\nlabels were extracted from clinical practice records; SDPP labels were annotated by\ntwo ophthalmologists with disagreements adjudicated by a consultant-level\nophthalmologist), there are clear performance differences across various\napplications, as shown in Figure 3. Although this does not bias the performance\ncomparison between FM-MEH and FM-SDPP, a well-aligned labelling system would\nenable extra cross-validation”\n18. Unclear Link to Morphological Features - Although morphological feature differences\nbetween MEH and SDPP images are reported, the quantitative relationship between these\ndifferences and downstream task performance remains vague. The choice of morphological\nindices (e.g., fractal dimension) and their direct relevance to diabetic retinopathy, DME, or\nstroke prediction should be explored.\nResponse: Thank you for this question. We introduced the morphological indices for two\npurposes: 1) to characterise substantial differences between MEH and SDPP data as an\nadditional modality of comparison (Figure 2 and the Results section), and 2) to quantify\nvariation across demographic subgroups (Figure 6 and Extended Data Figures 6-7). While\nlinking morphological indices (e.g. fractal dimension) to disease outcomes would give\ninsights (explored in prior literature [40-46]), they may not directly contribute to our main\nfindings.\n19. Missing Pre-training Disease Distribution Details - The paper reports disease prevalence\nin the downstream sets but not in the pre-training sets. Figure 2 lacks information on the\ndistribution of relevant diseases (DR, DME, stroke indicators if possible) within the MEH and\nSDPP pre-training datasets, which is highly relevant context. This information on the\ndistribution of relevant diseases (DR, DME, and possibly stroke indicators) in these datasets\nshould be included in Figure 2.\nResponse: Thank you for this helpful suggestion. We have added the disease prevalence in\nthe main text and caption of Figure 2.\nLine 184-187, Page 6:\n“MEH data, as a hospital-based cohort, exhibits higher disease prevalence than\nSDPP data, which is derived from community screening. The prevalence of diabetic\nretinopathy, diabetic macular oedema, and ischaemic stroke in the MEH cohort is\n8.71%, 3.02%, and 1.57%, respectively, compared with 1.4%, 0.08%, and 0.66% in\nthe SDPP cohort.”\nLine 874-876, Page 27:\n“Regarding disease prevalence, MEH (hospital-based) shows higher disease\nprevalence than SDPP (community-based): diabetic retinopathy 8.71%, diabetic\nmacular oedema 3.02%, and ischaemic stroke 1.57% in MEH vs 1.4%, 0.08%, and\n0.66% in SDPP.”\n20. Small Performance Gaps Treated as Significant - Some statistically significant\ndifferences reported involve very small absolute AUC changes (e.g., <0.01), questioning\ntheir practical relevance and the conclusion of one model outperforming the other. For\nAPTOS2019 AUC difference is 0.003; for Glaucoma fundus 0.006; for Diabetic retinopathy\nshanghai fine tune 0.006\nResponse: Thank you for raising this issue. After recalculating the p-values, most concerns\nhave been resolved. For the remaining case, a minor but statistically significant difference\nbetween FM-MEH and FM-SDPP when fine-tuned to APTOS2019, we have added an\nexplicit clarification in the main text.\nLine 229-231, Page 7:\n“When pre-trained with Masked Autoencoder and fine-tuned to the downstream tasks\n(Figure 4a), FM-SDPP significantly outperformed FM-MEH on three out of six\ndatasets (APTOS2019, MESSIDOR2, and Retina datasets), while the average\nperformance gap on APTOS2019 is minor”\n21. Contradictory Results Across Architectures - In some cases, FM-SDPP outperforms FM-\nMEH for one SSL method (MAE) but performs worse under the other method (DINOv2)\n(e.g., MESSIDOR2, Retina datasets in Fig 4), making it hard to attribute performance solely\nto dataset differences versus interactions with the pre-training approach. Summing p-values\nacross models is not appropriate.\nResponse: Thank you for this question. We have revised the main text to avoid summing p-\nvalues across models when evaluating and comparing FMs on publicly available datasets.\nThe main text is:\nLine 224-236, Page 7:\n“We evaluated the generalisability of FM-MEH and FM-SDPP to diverse applications\nusing six publicly available datasets, comprising diabetic retinopathy detection\n(APTOS2019, IDRiD, and MESSIDOR2), glaucoma detection (Glaucoma fundus),\nand multiple retinal disease detection (JSIEC and Retina). As shown in Figure 4, FM-\nSDPP and FM-MEH showed comparable performance in a majority of evaluations (8\nout of 12 for Masked Autoencoder-based FMs; 10 out of 12 for DINOV2-based FMs).\nWhen pre-trained with Masked Autoencoder and fine-tuned to the downstream tasks\n(Figure 4a), FM-SDPP significantly outperformed FM-MEH on three out of six\ndatasets (APTOS2019, MESSIDOR2, and Retina datasets), while the average\nperformance gap on APTOS2019 is minor. When adapted to downstream tasks\nusing linear probing (Figure 4c), FM-MEH performed significantly better on the\nGlaucoma fundus dataset (p<0.01). When using DINOV2 for FM pre-training (Figure\n4b and Figure 4d), FM-MEH significantly outperformed FM-SDPP in two applications,\ni.e. fine-tuning to MESSIDOR2 (0.01<p<0.05) and linear probing to IDRiD (p<0.01).\nAUPRC results are illustrated in Extended Data Figure 5. All quantitative results are\nlisted in Supplementary Table 3.”\n22. Figures and Flowcharts -The t-SNE clusterings mentioned in the text (Fig. 5b) are not\nclearly visible or explained. The flowchart in Figure 5a adds no value; it's a linear description,\nnot a decision-making tool. Lack of discussion on metadata (age, sex, device, etc.) for the\npublic datasets hinders interpretation of performance differences on these sets in the context\nof pre-training data similarity.\nResponse: Thank you for this feedback. In the revision, we clarify that our primary aim is to\nquantify how pre-training data shape the generalisability and fairness of retinal FMs. We\nremoved language implying a practical data-selection framework (including the former Figure\n5a) from the Abstract, Introduction, and Discussion. The updated Figure 6 (formerly Figure\n5) now serves to illustrate variations in latent features and morphological indices across age\nsubgroups.\nWe agree that discussing public dataset metadata (age, sex, device, etc.) would aid\ninterpretation in the context of similarity to pre-training data. However, most public datasets\nprovide limited demographic information, which constrains such analysis. To partly address\nthis, we added a t-SNE visualisation to show latent-feature distributions of the public\ndatasets with MEH and SDPP data in the Extended Data Figure 9. Additionally, we\nmeasured the squared Maximum Mean Discrepancy (MMD²) between each public dataset\nand the MEH and SDPP cohorts (Supplementary Table 9), and added relevant discussion.\nExtended Data Figure 9. t-SNE visualisation for MEH, SDPP data (5000 randomly sampled data\npoints), and publicly available datasets used in this study, respectively with features extracted by\nfoundation models developed in each site (FM-MEH and FM-SDPP) with Masked Autoencoder (MAE)\nand DINOv2.\nLine 354-363, Page 11:\n“Additionally, we measured the squared Maximum Mean Discrepancy (MMD²)\nbetween each public dataset and the MEH and SDPP cohorts (Supplementary Table\n9) and visualised feature distributions using t-SNE (Extended Data Fig. 9).\nMMD² offers partial explanatory power for the generalisation gap between FM-MEH\nand FM-SDPP. For example, with MAE-based features, MESSIDOR2 distributes\ncloser to SDPP than to MEH and FM-SDPP performs better; with DINOv2-based\nfeatures, MESSIDOR2 is closer to MEH and FM-MEH performs better. However, this\npattern does not hold for all datasets (e.g. Retina), indicating that distance metrics\nalone are insufficient to account for performance and generalisability differences.\nWhile FMs substantially mitigate cross-site generalisation gaps, residual disparities\nwarrant further investigation.”\nMiscellaneous Issues\n23. Precision Scores: AUPRC is calculated, but precision-recall trade-offs are not discussed\nin the text.\nResponse: Thank you for highlighting this. In this study, we constructed balanced\ndownstream evaluation sets (equal numbers of cases and controls) to support fair model\ncomparison and subgroup analyses. Given this design, we did not explore operating-point\nselection or precision–recall trade-offs, which are especially informative under imbalanced\nprevalence. We now acknowledge this as a limitation and, in future work, will 1) evaluate on\ndatasets reflecting hospital- and community-level prevalence and 2) report operating-point\nanalyses under realistic priors to better characterise precision–recall trade-offs.\nLine 453-457, Page 13:\n“Fourth, we balanced the downstream tasks by sampling equal numbers of disease\nand control images. This design facilitates fair model comparison and subgroup\nanalyses. However, future evaluations should also include datasets that reflect\ncommunity prevalence to assess real-world performance. Coupled with analysis at\nclinically relevant precision-recall operating points, this would provide deeper insight\ninto performance trade-offs.”\n24. Figure 2b Specificity: Mentions t-SNE for MAE features but not explicitly for DINOv2\nfeatures in the caption/text for comparison.\nResponse: Thanks, we have now included DINOv2 features in Extended Data Figure 1.\nExtended Data Figure 1. t-SNE visualisation for MEH and SDPP data (5000 randomly sampled data\npoints), respectively with features extracted by foundation models developed in each site (FM-MEH\nand FM-SDPP) with DINOV2.\n25. Typo: Extended Data Figure 4 caption (line 900) likely means AUPRC, not AUROC,\nbased on the figure content.\nResponse: Thank you for pointing out this mistake. We have corrected the typo.\n26. Image Quality: Plotted in Fig 2c but its impact or relevance is not discussed.\nResponse: Thank you for this question. We included image quality (Figure 2c) to\ncharacterise substantial differences between MEH and SDPP images. For all downstream\ntasks at both sites, images underwent quality control with AutoMorph (as introduced in the\nMethod section), minimising any bias in model performance caused by image quality.\n27. Study Origins: The different clinical origins of the datasets (MEH/AlzEye vs.\nSDPP/Diabetes Prevention) might imply systematic differences beyond demographics (e.g.,\ncomorbidities, baseline health status) that are not explored.\nResponse: Thank you for this suggestion. We have added description in the main text.\nAlthough with the substantial differences between MEH and SDPP in data origins,\ndemographics, morphological indices, and latent features distribution, retinal FMs achieved\nstrong generalisability.\nLine 184-187, Page 6:\n“MEH data, as a hospital-based cohort, exhibits higher disease prevalence than\nSDPP data, which is derived from community screening. The prevalence of diabetic\nretinopathy, diabetic macular oedema, and ischaemic stroke in the MEH cohort is\n8.71%, 3.02%, and 1.57%, respectively, compared with 1.4%, 0.08%, and 0.66% in\nthe SDPP cohort.”\n28. Age Binning: Justification for binning age rather than analyzing it as a continuous\nvariable or using more granular bins is missing.\nResponse: Thanks, we have revised Figure 2a to show granular age bins (one year per bar)\nfor the MEH and SDPP cohorts.\nStrengths:\n29. Parallel Training Design: The core strength is training identical FM architectures with\nidentical methods (per SSL type) on two distinct, large-scale datasets, providing a clean\nsetup to isolate the impact of pre-training data origin.\nValuable Datasets: Access to and use of large, real-world clinical datasets (MEH, SDPP)\nadds significant value.\nImportant Problem: The study addresses a timely and critical question about the foundations\nof generalizability and fairness in medical FMs.\nResponse: Thank you for recognising our parallel-training design. We appreciate your\nacknowledgement of the value of large-scale datasets and the importance of our\ninvestigation into FM generalisability and fairness.\nSuggestions\n30. Deeper Bias Analysis: Investigate potential mechanisms behind observed biases. How\ndo models encode ethnicity/gender, even if downstream performance differences weren't\nconsistently significant in this study?\nResponse: Thank you for this suggestion. We streamlined the paper’s logic as follows: we\nfirst developed parallel retinal FMs and evaluated them across multiple sites and\ndownstream tasks; we then assessed fairness across demographic subgroups and identified\nfairness gaps by age; next, we explored why an age-related gap emerged while gaps by sex\nand ethnicity did not. We examined data variation (latent features and morphological indices)\nacross demographic groups and found substantial differences by age, which likely contribute\nto the observed age-related fairness gap. Visualising latent features with t-SNE (Figure 6, Extended Data\nFigure 6-7) provides an alternative view of how models encode demographic attributes. Our\nfindings align with prior literature.\nLine 409-413, Page 12:\n“For example, Yang et al. showed that models encoding more demographic attributes\n(e.g. age, sex) are prone to fairness gap under external evaluation across radiology,\ndermatology, and ophthalmology [60]. We extend this observation to FMs. As shown\nin Figure 5 and 6, differences in age distribution within the pre-training set are\nperceived by retinal FMs (age<40, 40<age<70, 70<age) and fairness gap across age\nsubgroups is observed.”\nLine 271-293, Page 8-9:\n“We probed the reasons why demographic attribute unevenly shaped the model\nperformance, particularly fairness over subgroups. We mainly investigated this from\ntwo aspects, the morphological indices and latent features across demographic\nsubgroups in pre-training data. As shown in Figure 6, we observed distinct\ndistribution differences across age subgroups in both latent features (Figure 6a and\n6c) and morphological indices (Figure 6e and 6f). After controlling for confounders by\nlooking at only Asian or Asian British females (Figure 6b and 6d), there are still\nsignificant gaps in morphological indices (Figure 6f). This demonstrates that age\nsubgroups exhibit clear morphological variations, and even with self-supervised pre-\ntraining, retinal FMs learned distinct latent features across age subgroups, which\nlikely caused bias in FM fairness.\nFor ethnicity, we investigated three subgroups White, Asian or Asian British, and\nBlack or Black British. The t-SNE visualisations (Extended Data Figure 6a, FMs\ntrained with Masked Autoencoder) showed clear clustering partially for the White\ncohort. When FMs were pre-trained with DINOV2 (Extended Data Figure 6c), latent\nfeatures showed no distinct clustering across all ethnicities. Additionally, when\ncontrolling for confounders, there were no significant differences across ethnic\nsubgroups in morphological indices (Extended Data Figure 6f).\nFor sex subgroups (i.e. female, male), we observed no distinct clustering in t-SNE\nvisualisations, either before or after removing confounding variables (Extended Data\nFigure 7). Only the vein fractal dimension showed significant differences across the\nsex subgroups after removing confounding variables (Extended Data Figure 7f).\nThese findings suggest that, ethnicity and sex subgroups contributed limited\nobservable variations in morphological indices and less distinct latent compared to\nage distribution, which are less likely to bias FM fairness and generalisability in\ndownstream tasks.”\n31. Combined Dataset Training: Explore if pre-training on a combined MEH+SDPP dataset\nmitigates the observed age bias and site-specific performance gaps.\nResponse: Thank you for this suggestion. We agree that pre-training on a combined dataset\ncould yield additional insights, but merging real-world clinical data is challenging due to\nprivacy regulations. As a workaround, we synthesised images representative of the SDPP\ncohort by training a generative model on SDPP images and conditioning synthesis on age.\nUsing these synthetic images to complement the pre-training age distribution improved\nfairness across age subgroups in certain applications (Supplementary Table 8). We have\nadded details in the Results and Discussion sections.\nLine 295-309, Page 9:\n“Inspired by the observations above, we tested whether retinal FM generalisability\nand fairness improve when pre-training on a dataset with a more diverse and\nbalanced age distribution. We generated 300K synthetic colour fundus photographs\nrepresentative of SDPP young cohort (age<40) in age distribution, using synthetic\nmodels (details introduced in the Method section) and merged them with randomly\nsampled 604,170 MEH images, for a total of 904,170 images. Image synthesis was\nexplicitly conditioned on age to match the age profile in SDPP young cohort. The age\ndistribution of this combined data is shown in Extended Data Figure 8. Using this\ncombined dataset, we trained FM-MEH-SDPP using DINOV2 and compared it with\nFM-MEH in subgroup analysis. As shown in Supplementary Table 7, FM-MEH-SDPP\nperformed comparably with FM-MEH, with differences observed for diabetic oedema\ndetection on MEH data and diabetic retinopathy detection on MESSIDOR2 datasets\nafter linear probing. For model fairness across age subgroups (Supplementary Table\n8), FM-MEH-SDPP showed significant improvement in young groups for diabetic\nretinopathy detection after fine-tuning (p=0.02 after Holm-Bonferroni correction)\nand for diabetic retinopathy after linear probing (p=0.033 after correction). These\nresults indicate that simply increasing age distribution diversity and balance in the pre-\ntraining data can yield fairness gains, even with synthetic images.”\nLine 425-434, Page 13:\n“Our experiments with FM-MEH-SDPP echo prior findings that synthetic data can\ncontribute effectively to FM pre-training [60,61]. Moreover, our fine-grained, age-\nconditioned image synthesis further verified the benefits of increasing the diversity\nand balance of age, a demographic attribute that strongly impacts both morphological\nindices and latent features. Although FM-MEH-SDPP performed comparably with\nFM-MEH on the general cohort, it significantly improved the performance on younger\ncohorts on certain applications. While synthetic images cannot perfectly recapitulate\nreal distributions, and we conditioned on age only, the observed subgroup\nimprovements indicate that age is indeed an important metadata that should be\nprioritised in data selection for retinal foundation models. Nevertheless, signals on\nFM generalisability and how pre-training data impacts FM fairness may differ across\nmedical domains and warrant domain-specific investigation, which we advocate\nthrough this study.”\nExtended Data Figure 8. Age distribution of the combined dataset, comprising 604,170 real images\nfrom MEH and 300,000 synthetic images representative of the younger SDPP cohort (age < 40\nyears). Pre-training the retinal FM on this combined dataset allows us to examine whether age-related\nfairness improves, particularly within the younger subgroup in downstream tasks. We note that image\nsynthesis may not always strictly follow the age condition. However, as the SDPP data used for\nStable Diffusion XL training are drawn from a young cohort (age<40 years), the resulting synthetic\nimages should still be broadly representative of this group.\n32. SSL Comparison Rigor: Provide justification for the chosen training protocols or conduct\nablation studies to better isolate the effect of the SSL strategy itself versus training\nbudget/hyperparameters, especially for claims of DINOv2 superiority.\nResponse: Thank you for this suggestion. We have enhanced the description rigour. More\ndetails are included in the response to Question 15.\n33. Expand Scope (Future Work): Acknowledge limitations more strongly and suggest future\nwork replicating this parallel design in other medical domains (e.g., radiology, pathology)\nto assess the generalizability of these specific findings.\nResponse: Thank you for this suggestion. We have narrowed the description to retinal FMs\nand retinal images, as detailed in our response to Question 3. We also stated this explicitly\nin the limitations paragraph.\nLine 442-448, Page 13:\n“Second, due to the considerable workload involved in developing parallel FMs and\norganising diverse downstream tasks, this study primarily focused on representative\nself-supervised learning strategies, such as Masked Autoencoder and DINOV2, and\nused retinal images as an exemplar. Whether good generalisability exhibited by\nretinal FMs and uneven impact of data attributes extend to other task types (e.g.\nsegmentation, regression tasks) and medical fields (e.g. pathology, radiology) is\nunknown. Further research involving a wider range of learning strategies, task types,\nand medical domains with a replicated parallel design is needed”\n34. Statistical Correction: Re-evaluate statistical significance using appropriate corrections\nfor multiple comparisons and report corrected p-values or use confidence intervals more\ncentrally. Clearly report subgroup sample sizes and prevalence.\nResponse: We highly appreciate this valuable suggestion. We have updated all statistical\nanalyses. Please see our response to Question 5.\n35. Contextualize Novelty: Reframe the \"site bias\" finding by clearly acknowledging it\nconfirms existing principles and focus novelty on the quantification within the FM/retinal\ncontext, or on deeper analysis if performed.\nResponse: Thank you for this suggestion. We have included prior literature as context.\nPlease see our response to Question 6.\nReviewer #2\nNoteworthy results\nYukun Zhou and co-authors, in “Revealing the Impact of Pre-training Data on Medical\nFoundation Models,” explore fairness and generalizability of medical foundation models,\nusing ophthalmic fundus photography as the prototypical case.\nThe most important finding is the impact of certain metadata on foundation model (FM)\ngeneralizability, in particular with regard to age. Since the authors created two different\nfoundation models, one with MEH data and another with SDPP data. The groups differed\nacross a few domains, most notably age. The investigators demonstrated differences\nbetween foundation model performance when comparing MEH and SDPP analysis of\ndatasets and also showed the effect of age within models. This is a key finding, as it speaks\nto the importance of transparency and bias in model fairness and generalizability with regard\nto the datasets used to create the foundation models.\nEthnicity and sex did not appear to affect model performance. The investigators showed\n“that FMs demonstrate[d] good generalizability, achieving comparable performance in over\n70% of downstream tasks. However, FMs sometimes perform[ed] better on application data\nthat aligns with their pre-training data.”\nThe authors point out that “long-term and sustainable advancement of medical AI requires\nthe development of domain-specific techniques tailored to the unique characteristics of\nmedical data and application scenarios… Pre-training clinical data and self-supervised\nlearning strategies have a synergistic effect on FM performance… need to simultaneously\noptimise both model learning strategies and pre-training data characteristics to advance\nmedical FM development.”\nThe authors make several excellent and important recommendations for future work in this\narea, including the following:\n• “Future studies should include extra factors particularly concerning disease phenotypes.”\n• “Further research involving a wider range of learning strategies and medical domains is\nneeded.”\n• “There are clear performance differences across various applications.”\nSignificance to the field. This work has significant potential to enhance the quality of FMs\nused for analysis of medical data. It is an interesting examination of the role and effect of\ntraining data on FM performance and bias.\nConclusions and Claims\nThe authors’ conclusions and claims are supported by their data.\nFlaws in the data analysis, interpretation and conclusions\nNo significant flaws are noted.\nResponse: Thank you very much for the thoughtful and generous appraisal of our work. We\nappreciate your clear summary, particularly the emphasis on demographic effects, cross-site\ngeneralisability, the variable impacts of ethnicity and sex, and the recommendations for\nfuture work. We hope this study encourages fine-grained, attribute-level data analysis and\nselection across medical domains to support efficient FM development.\n1. Of note is that the MEH and SDPP samples used to create and test the FMs were\nunbalanced and a subset of data were used to create balanced and curated samples. The\nauthors should comment on how this might have affected the FMs and comparisons.\nThe methodology is sound, well thought out and well executed. As noted above, the authors\nshould comment on the imbalance of the MEH and SDPP datasets.\nResponse: Thank you for this helpful suggestion. The MEH (hospital-based) and SDPP\n(community-based) cohorts are indeed imbalanced in disease prevalence (e.g. diabetic\nretinopathy, diabetic macular oedema, and ischaemic stroke). To enable fair model\ncomparison and subgroup analyses, we constructed balanced downstream evaluation sets\nwith equal numbers of disease cases and controls. While this design reduces issue of data\nimbalance and supports stable estimation of group-wise effects, it also departs from real-\nworld prevalence, which can affect the evaluation on external applicability. We now\nacknowledge this as a limitation and, as part of future work, will 1) evaluate on datasets\nreflecting hospital- and community-level prevalence, and 2) report operating-point analyses\nunder realistic priors to provide deeper insight into performance trade-off.\nLine 184-187, Page 6:\n“MEH data, as a hospital-based cohort, exhibits higher disease prevalence than\nSDPP data, which is derived from community screening. The prevalence of diabetic\nretinopathy, diabetic macular oedema, and ischaemic stroke in the MEH cohort is\n8.71%, 3.02%, and 1.57%, respectively, compared with 1.4%, 0.08%, and 0.66% in\nthe SDPP cohort.”\nLine 453-457, Page 13:\n“Fourth, we balanced the downstream tasks by sampling equal numbers of disease\nand control images. This design facilitates fair model comparison and subgroup\nanalyses. However, future evaluations should also include datasets that reflect\ncommunity prevalence to assess real-world performance. Coupled with analysis at\nclinically relevant precision-recall operating points, this would provide deeper insight\ninto performance trade-offs.”\n3. The authors have provided adequate information to enable reproduction of their results,\nalthough their data are not readily available to other investigators.\nResponse: Thank you for your comments on reproducibility. We provide implementation\ndetails to replicate our study and have released all code and model weights (see the Code\navailability section). Public dataset links are listed, and access to private clinical data follows\nthe standardised application process outlined in the Data availability section.\nDear editors and reviewers,\nThank you for acknowledging our previous efforts in addressing the concerns and improving\nthe manuscript. We have now added further revision and clarification to address the\nremaining concerns. All revisions are marked in blue in the manuscript. Please find our\npoint-by-point response below.\nReviewer #1\n1. The authors have made substantial and constructive revisions. The statistical analysis\nwas re-run using Welch's t-tests with a Holm-Bonferroni correction for multiple\ncomparisons, directly addressing the main methodological concern from the first review\nround. The methods are now clearer, and the fundamental statistical flaws of the earlier\nversion appear resolved.\nThe conclusions have shifted in response to the re-analysis. Rather than attributing\nperformance differences to site bias, the revised results show few significant cross-site\ndifferences and emphasize cross-site robustness.\nResponse: Thank you for appreciating our substantial revisions. We are pleased that the\nmain concerns have been addressed. The updated analyses and results demonstrate strong\ncross-site generalisation of the trained foundation models, comparable to intra-site\nadaptation on most clinical applications.\n2. A remaining concern is that the comparison between MAE and DINOv2 is still\nconfounded by differences in training protocols and optimization, which limits the\nstrength of any conclusions about the relative performance of the two self-supervised\nlearning (SSL) approaches.\nResponse: Thank you for this insightful comment. We acknowledge that our previous\ndiscussion may cause some confusion. We intended to compare RETFound-MAE and\nRETFound-DINOv2, rather than to directly contrast the self-supervised learning (SSL)\nstrategies, MAE and DINOv2, in isolation. Even in the original DINOv2 paper [48], it is\nchallenging to conclude the intrinsic superiority of one SSL strategy over another, given that\nthe compared models were trained with distinct data and optimisation protocols. In our study,\nwe aimed to compare RETFound-MAE and RETFound-DINOv2 using identical pre-training\nretinal images and computational budgets, but differing in their initial weights and\noptimisation approaches, across diverse downstream applications. To clarify this point, we\nhave revised the relevant discussion as follows:\nLine 393-406, Page 12:\n“Advancements in general-purpose AI methods continue to push the performance\nboundaries in our applications. In this study, we incorporated two representative self-\nsupervised learning paradigms, a generative-based method (Masked Autoencoder\n[47]) and a self-distillation-based method (DINOv2 [48]), to develop RETFound-MAE\nand RETFound-DINOv2, using the same compute budget (identical GPU settings\nand training duration). Our experiments showed that RETFound-DINOv2 achieved\nsuperior performance in downstream tasks across both sites (Supplementary Table\n5), and this trend extended to publicly available datasets. This advantage likely\nreflects that strong initial weights from general-purpose AI models, such as DINOv2,\ncan facilitate subsequent domain-specific foundation model development and\napplications. However, it should be noted that the retinal foundation models\ndeveloped in this study, RETFound-MAE and RETFound-DINOv2, were not\noptimised to their theoretical best. Both were pre-trained out-of-the-box on large-\nscale retinal datasets without extensive hyperparameter tuning. Even so, they\nachieved competitive performance across diverse downstream applications,\nhighlighting the translational potential of medical foundation models initialised from\nevolving general AI models.”\n3. The paper's main contribution is framed around its experimental design and the\nresulting empirical evidence. However, the design itself is not a novel contribution. It is a\nstandard controlled experiment using off-the-shelf code on size-matched datasets. The\nsubsequent evaluation is comprehensive and well executed.\nResponse: Thank you for acknowledging that our evaluation is comprehensive and well\nexecuted. We would like to clarify that we have not claimed novelty in our experimental\ndesign, either in the manuscript or in the response. Our study is not a typical methodological\ndevelopment work that introduces new algorithms or methods. Instead, its main contribution\nlies in providing robust empirical evidence on how pre-training data impacts the\ngeneralisability and fairness of retinal foundation models through extensive experiments.\nThe insights are derived from a carefully controlled and standardised experimental setting.\nWe believe this evidence provides substantial value to the broader research community,\nincluding those involved in both the development and deployment of foundation models. For\ninstance, the observed strong generalisability of retinal foundation models may encourage\nresearchers to prioritise the collection of large-scale datasets from new imaging modalities\n(e.g. OCTA) to develop complementary foundation models, rather than replicating datasets\nfor existing modalities, such as retinal fundus photographs, for local pre-training. Likewise,\nthose applying retinal foundation models for clinical tasks may consider demographic\nalignment, such as age distribution, more critical than geographic origin. Given the rapid\nexpansion of retinal foundation model research and their wide use across diverse\napplications [1–5], we believe these findings are both highly relevant and impactful.\nIn addition, we have made our model weights fully open-source, together with detailed usage\ninstructions (https://github.com/rmaphoh/RETFound). This provides another strong retinal\nfoundation to advance research and applications in foundation models, ocular disease\ndetection, and oculomics tasks.\n4. The remaining weakness is the limited scope of the experiments. As the authors\nacknowledge (e.g., line 448), broader studies across learning strategies, task types, and\nmedical domains with a replicated parallel design are desireable. In this work, the\nevidence is restricted to two large but specialized fundus image datasets, i.e., a single\nimaging modality. The observed cross-site robustness may be specific to the\ncharacteristics of the MEH and SDPP cohorts and may not extend to other fundus\ndatasets, other retinal modalities (as hinted by the title) (for example OCT), or other\nareas of medical imaging.\nResponse: We agree that expanding the scope of experiments across broader learning\nstrategies, task types, and medical domains is desirable. This is exactly why we\nacknowledged it as one of the limitations and directions for future work. However, to the best\nof our knowledge, this study represents the first quantitative investigation into how pre-\ntraining data impacts the generalisability and fairness of retinal foundation models in clinical\napplications, with multiple parallel foundation models and extensive evaluation.\nThe primary bottleneck in starting this line of research lies in the substantial experimental\ndemands, developing multiple foundation models and conducting downstream evaluations\nand subgroup analysis, while carefully isolating the effects of pre-training data. Our current\nstudy already reflects a significant scale of investigation. Specifically, we included:\n● Two large-scale cohorts (~904K retinal fundus photographs each)\n● Two representative learning strategies (MAE and DINOv2);\n● Five foundation models (RETFound-MAE and RETFound-DINOv2 trained on each\nsite separately, plus one trained on merged data, including real MEH data and\nsynthetic data representative of the SDPP cohort);\n● Evaluation across six downstream datasets from two sites, covering both ocular\ndisease detection and oculomics tasks (diabetic retinopathy, diabetic oedema, and\nischaemic stroke prediction);\n● Six publicly available datasets representing different ocular diseases and varied\ndemographics;\n● A large-scale fairness analysis across demographic subgroups; and\n● Two adaptation strategies (fine-tuning and linear probing).\nThese experiments collectively underpin six main figures, nine Extended Data Figures, and\nnine Supplementary Tables (certain Figures and Tables include massive amounts of data).\nWhile extending this study to additional modalities (e.g., OCT or OCTA) would be valuable, it\nis beyond the feasible scope of a single research article, also requiring substantial\ncomputational and data availability (e.g. 904K OCT or OCTA scans from two large-scale,\ndistinct cohorts for pre-training; multiple curated datasets for evaluation).\nWe view this work as an initial but important step toward a comprehensive and evidence-\nbased understanding of foundation model generalisability and fairness in medical imaging.\nAlthough it cannot cover all possible learning strategies or imaging modalities, as no single\nstudy can, it provides solid empirical evidence and aims to encourage future collective efforts\nfrom the research community to advance this critical area.\nThat said, we appreciate the reviewer’s suggestion for clearer framing. While we had\nspecified the use of retinal fundus photographs in the introduction (e.g. line 150, page 5) and\nthe following sections (e.g. line 172, page 6), we have now revised the title and abstract to\nexplicitly highlight this focus, ensuring clarity and precision for the audience.\nTitle: Revealing the Impact of Pre-training Data on Retinal Foundation Models: Evidence\nfrom Two Large-Scale Fundus Photograph Cohorts\nAbstract: Medical foundation models (FMs), pre-trained on large-scale unlabelled data,\nshow strong performance and data efficiency when adapted to various clinically relevant\napplications. However, how pre-training data shape the generalisability and fairness of these\nmodels, such as retinal FMs, remains unexplored. We address this using two large retinal\nfundus photograph cohorts from Moorfields Eye Hospital (UK) and the Shanghai Diabetes\nPrevention Program (China), each containing 904,170 retinal fundus photographs for FM\npre-training. Using identical pipelines, we trained parallel FMs using individual image cohort\nand evaluated them on downstream tasks with publicly available datasets and held-out data\nfrom each site. The parallel FMs achieved comparable performance on most downstream\ntasks, indicating strong generalisability to data that differ substantially from their pre-training\ndata. Nevertheless, we observed fairness gaps over age subgroups for retinal FMs, whereas\nsex and ethnicity showed no such impact. These results serve as quantitative evidence of\nthe good generalisability of these FMs and reveal that pre-training demographic attributes\nshape fairness differently. This inspires domain-specific, fine-grained investigation of pre-\ntraining data effect to support evidence-based data selection to improve both generalisability\nand fairness in FM development.\n5. Consequently, the claims about robustness of retinal foundation models to pre-\ntraining data shifts are not sufficiently supported by the current evidence beyond the\ntested data. Moreover, the observation that supervised fine-tuning can overcome\ndomain gaps is well established; demonstrating it again in this narrow context does not\nconstitute a significant advance.\nResponse: Thank you for sharing your detailed and constructive feedback in this second-\nround review. We hope that the revisions and clarifications provided above have addressed\nyour concerns regarding the study scope.\nWe agree that fine-tuning is a well-established method for mitigating domain gaps. However,\nthis is not the central focus of our work. Instead, our findings go beyond the expected\nbenefits of fine-tuning: through extensive experiments, we observed that retinal foundation\nmodels demonstrate strong cross-site generalisability, achieving performance comparable to\nintra-site adaptation using a retinal foundation model pre-trained on the same site. This\nreflects not only a narrowing of performance gaps, but also an impressive level of\ngeneralisability rarely observed in broader machine learning models.\nFurthermore, our linear probing analyses independently confirmed this high level of\ngeneralisation. Beyond generalisability, our study also provides insights into fairness,\nidentifying disparities across demographic subgroups and demonstrating practical strategies\nfor partially improving fairness by involving synthetic data to pre-train retinal foundation\nmodels.\nOverall, we believe this work makes a meaningful contribution to the field by providing\nrigorous empirical evidence on the generalisability and fairness of retinal foundation models.\nOur previous RETFound model has made significant contributions to the ophthalmic AI\ncommunity, serving as one of the most widely used open-source models for diverse clinical\nand research applications [1-5]. In this study, we further introduce RETFound-DINOv2, a\nnew, generalisable, and openly available retinal foundation model that we expect will\ncontinue to benefit a broad range of ocular disease detection and oculomics studies.\nAdditionally, these research findings, particularly regarding generalisation and fairness, will\nhelp optimise future foundation model development and deployment efforts across the wider\ncommunity, as detailed in the response to Question 3.\nReferences:\n[1] Sun, Yuqi, et al. \"A data-efficient strategy for building high-performing medical foundation\nmodels.\" Nature Biomedical Engineering (2025): 1-13.\n[2] Wang, Meng, et al. \"Enhancing diagnostic accuracy in rare and common fundus diseases\nwith a knowledge-rich vision-language model.\" Nature Communications 16.1 (2025): 5528.\n[3] Jiang, Nan, et al. \"A deep learning system for detecting silent brain infarction and\npredicting stroke risk.\" Nature Biomedical Engineering (2025): 1-13.\n[4] Lee, Tae Kwan, et al. \"Vision transformer based interpretable metabolic syndrome\nclassification using retinal Images.\" npj Digital Medicine 8.1 (2025): 205.\n[5] Zhang, Juzhao, et al. \"RETFound-enhanced community-based fundus disease screening:\nreal-world evidence and decision curve analysis.\" NPJ digital medicine 7.1 (2024): 108.\nOpen-source Models:\nhttps://github.com/rmaphoh/RETFound\nReviewer #2\n1. All reviewer concerns have been satisfactorily addressed in the authors' extensive and\ncomplete revision and reply. This reviewer thanks the authors for the\ncomprehensiveness of their revision and reply and for their consideration of queries\nposed.\nResponse: Thank you for acknowledging that the concerns have been fully addressed. Your\ninsightful comments have greatly helped us improve the manuscript."}, {"round": 2, "reviewer_comments": {"reviewer_1": "The authors have made substantial and constructive revisions. The statistical analysis was re-run using Welch's t-tests with a Holm-Bonferroni correction for multiple comparisons, directly addressing the main methodological concern from the first review round. The methods are now clearer, and the fundamental statistical flaws of the earlier version appear resolved.\nThe conclusions have shifted in response to the re-analysis. Rather than attributing performance differences to site bias, the revised results show few significant cross-site differences and emphasize cross-site robustness.\nA remaining concern is that the comparison between MAE and DINOv2 is still confounded by differences in training protocols and optimization, which limits the strength of any conclusions about the relative performance of the two self-supervised learning (SSL) approaches.\nThe paper's main contribution is framed around its experimental design and the resulting empirical evidence. However, the design itself is not a novel contribution. It is a standard controlled experiment using off-the-shelf code on size-matched datasets. The subsequent evaluation is comprehensive and well executed.\nThe remaining weakness is the limited scope of the experiments. As the authors acknowledge (e.g., line 448), broader studies across learning strategies, task types, and medical domains with a replicated parallel design are desireable. In this work, the evidence is restricted to two large but specialized fundus image datasets, i.e., a single imaging modality. The observed cross-site robustness may be specific to the characteristics of the MEH and SDPP cohorts and may not extend to other fundus datasets, other retinal modalities (as hinted by the title) (for example OCT), or other areas of medical imaging. Consequently, the claims about robustness of retinal foundation models to pre-training data shifts are not sufficiently supported by the current evidence beyond the tested data. Moreover, the observation that supervised fine-tuning can overcome domain gaps is well established; demonstrating it again in this narrow context does not constitute a significant advance.", "reviewer_2": "All reviewer concerns have been satisfactorily addressed in the authors' extensive and complete revision and reply. This reviewer thanks the authors for the comprehensiveness of their revision and reply and for their consideration of queries posed."}, "author_response": "Dear editors and reviewers,\nThank you for acknowledging our previous efforts in addressing the concerns and improving\nthe manuscript. We have now added further revision and clarification to address the\nremaining concerns. All revisions are marked in blue in the manuscript. Please find our\npoint-by-point response below.\nReviewer #1\n1. The authors have made substantial and constructive revisions. The statistical analysis\nwas re-run using Welch's t-tests with a Holm-Bonferroni correction for multiple\ncomparisons, directly addressing the main methodological concern from the first review\nround. The methods are now clearer, and the fundamental statistical flaws of the earlier\nversion appear resolved.\nThe conclusions have shifted in response to the re-analysis. Rather than attributing\nperformance differences to site bias, the revised results show few significant cross-site\ndifferences and emphasize cross-site robustness.\nResponse: Thank you for appreciating our substantial revisions. We are pleased that the\nmain concerns have been addressed. The updated analyses and results demonstrate strong\ncross-site generalisation of the trained foundation models, comparable to intra-site\nadaptation on most clinical applications.\n2. A remaining concern is that the comparison between MAE and DINOv2 is still\nconfounded by differences in training protocols and optimization, which limits the\nstrength of any conclusions about the relative performance of the two self-supervised\nlearning (SSL) approaches.\nResponse: Thank you for this insightful comment. We acknowledge that our previous\ndiscussion may cause some confusion. We intended to compare RETFound-MAE and\nRETFound-DINOv2, rather than to directly contrast the self-supervised learning (SSL)\nstrategies, MAE and DINOv2, in isolation. Even in the original DINOv2 paper [48], it is\nchallenging to conclude the intrinsic superiority of one SSL strategy over another, given that\nthe compared models were trained with distinct data and optimisation protocols. In our study,\nwe aimed to compare RETFound-MAE and RETFound-DINOv2 using identical pre-training\nretinal images and computational budgets, but differing in their initial weights and\noptimisation approaches, across diverse downstream applications. To clarify this point, we\nhave revised the relevant discussion as follows:\nLine 393-406, Page 12:\n“Advancements in general-purpose AI methods continue to push the performance\nboundaries in our applications. In this study, we incorporated two representative self-\nsupervised learning paradigms, a generative-based method (Masked Autoencoder\n[47]) and a self-distillation-based method (DINOv2 [48]), to develop RETFound-MAE\nand RETFound-DINOv2, using the same compute budget (identical GPU settings\nand training duration). Our experiments showed that RETFound-DINOv2 achieved\nsuperior performance in downstream tasks across both sites (Supplementary Table\n5), and this trend extended to publicly available datasets. This advantage likely\nreflects that strong initial weights from general-purpose AI models, such as DINOv2,\ncan facilitate subsequent domain-specific foundation model development and\napplications. However, it should be noted that the retinal foundation models\ndeveloped in this study, RETFound-MAE and RETFound-DINOv2, were not\noptimised to their theoretical best. Both were pre-trained out-of-the-box on large-\nscale retinal datasets without extensive hyperparameter tuning. Even so, they\nachieved competitive performance across diverse downstream applications,\nhighlighting the translational potential of medical foundation models initialised from\nevolving general AI models.”\n3. The paper's main contribution is framed around its experimental design and the\nresulting empirical evidence. However, the design itself is not a novel contribution. It is a\nstandard controlled experiment using off-the-shelf code on size-matched datasets. The\nsubsequent evaluation is comprehensive and well executed.\nResponse: Thank you for acknowledging that our evaluation is comprehensive and well\nexecuted. We would like to clarify that we have not claimed novelty in our experimental\ndesign, either in the manuscript or in the response. Our study is not a typical methodological\ndevelopment work that introduces new algorithms or methods. Instead, its main contribution\nlies in providing robust empirical evidence on how pre-training data impacts the\ngeneralisability and fairness of retinal foundation models through extensive experiments.\nThe insights are derived from a carefully controlled and standardised experimental setting.\nWe believe this evidence provides substantial value to the broader research community,\nincluding those involved in both the development and deployment of foundation models. For\ninstance, the observed strong generalisability of retinal foundation models may encourage\nresearchers to prioritise the collection of large-scale datasets from new imaging modalities\n(e.g. OCTA) to develop complementary foundation models, rather than replicating datasets\nfor existing modalities, such as retinal fundus photographs, for local pre-training. Likewise,\nthose applying retinal foundation models for clinical tasks may consider demographic\nalignment, such as age distribution, more critical than geographic origin. Given the rapid\nexpansion of retinal foundation model research and their wide use across diverse\napplications [1–5], we believe these findings are both highly relevant and impactful.\nIn addition, we have made our model weights fully open-source, together with detailed usage\ninstructions (https://github.com/rmaphoh/RETFound). This provides another strong retinal\nfoundation to advance research and applications in foundation models, ocular disease\ndetection, and oculomics tasks.\n4. The remaining weakness is the limited scope of the experiments. As the authors\nacknowledge (e.g., line 448), broader studies across learning strategies, task types, and\nmedical domains with a replicated parallel design are desireable. In this work, the\nevidence is restricted to two large but specialized fundus image datasets, i.e., a single\nimaging modality. The observed cross-site robustness may be specific to the\ncharacteristics of the MEH and SDPP cohorts and may not extend to other fundus\ndatasets, other retinal modalities (as hinted by the title) (for example OCT), or other\nareas of medical imaging.\nResponse: We agree that expanding the scope of experiments across broader learning\nstrategies, task types, and medical domains is desirable. This is exactly why we\nacknowledged it as one of the limitations and directions for future work. However, to the best\nof our knowledge, this study represents the first quantitative investigation into how pre-\ntraining data impacts the generalisability and fairness of retinal foundation models in clinical\napplications, with multiple parallel foundation models and extensive evaluation.\nThe primary bottleneck in starting this line of research lies in the substantial experimental\ndemands, developing multiple foundation models and conducting downstream evaluations\nand subgroup analysis, while carefully isolating the effects of pre-training data. Our current\nstudy already reflects a significant scale of investigation. Specifically, we included:\n● Two large-scale cohorts (~904K retinal fundus photographs each)\n● Two representative learning strategies (MAE and DINOv2);\n● Five foundation models (RETFound-MAE and RETFound-DINOv2 trained on each\nsite separately, plus one trained on merged data, including real MEH data and\nsynthetic data representative of the SDPP cohort);\n● Evaluation across six downstream datasets from two sites, covering both ocular\ndisease detection and oculomics tasks (diabetic retinopathy, diabetic oedema, and\nischaemic stroke prediction);\n● Six publicly available datasets representing different ocular diseases and varied\ndemographics;\n● A large-scale fairness analysis across demographic subgroups; and\n● Two adaptation strategies (fine-tuning and linear probing).\nThese experiments collectively underpin six main figures, nine Extended Data Figures, and\nnine Supplementary Tables (certain Figures and Tables include massive amounts of data).\nWhile extending this study to additional modalities (e.g., OCT or OCTA) would be valuable, it\nis beyond the feasible scope of a single research article, also requiring substantial\ncomputational and data availability (e.g. 904K OCT or OCTA scans from two large-scale,\ndistinct cohorts for pre-training; multiple curated datasets for evaluation).\nWe view this work as an initial but important step toward a comprehensive and evidence-\nbased understanding of foundation model generalisability and fairness in medical imaging.\nAlthough it cannot cover all possible learning strategies or imaging modalities, as no single\nstudy can, it provides solid empirical evidence and aims to encourage future collective efforts\nfrom the research community to advance this critical area.\nThat said, we appreciate the reviewer’s suggestion for clearer framing. While we had\nspecified the use of retinal fundus photographs in the introduction (e.g. line 150, page 5) and\nthe following sections (e.g. line 172, page 6), we have now revised the title and abstract to\nexplicitly highlight this focus, ensuring clarity and precision for the audience.\nTitle: Revealing the Impact of Pre-training Data on Retinal Foundation Models: Evidence\nfrom Two Large-Scale Fundus Photograph Cohorts\nAbstract: Medical foundation models (FMs), pre-trained on large-scale unlabelled data,\nshow strong performance and data efficiency when adapted to various clinically relevant\napplications. However, how pre-training data shape the generalisability and fairness of these\nmodels, such as retinal FMs, remains unexplored. We address this using two large retinal\nfundus photograph cohorts from Moorfields Eye Hospital (UK) and the Shanghai Diabetes\nPrevention Program (China), each containing 904,170 retinal fundus photographs for FM\npre-training. Using identical pipelines, we trained parallel FMs using individual image cohort\nand evaluated them on downstream tasks with publicly available datasets and held-out data\nfrom each site. The parallel FMs achieved comparable performance on most downstream\ntasks, indicating strong generalisability to data that differ substantially from their pre-training\ndata. Nevertheless, we observed fairness gaps over age subgroups for retinal FMs, whereas\nsex and ethnicity showed no such impact. These results serve as quantitative evidence of\nthe good generalisability of these FMs and reveal that pre-training demographic attributes\nshape fairness differently. This inspires domain-specific, fine-grained investigation of pre-\ntraining data effect to support evidence-based data selection to improve both generalisability\nand fairness in FM development.\n5. Consequently, the claims about robustness of retinal foundation models to pre-\ntraining data shifts are not sufficiently supported by the current evidence beyond the\ntested data. Moreover, the observation that supervised fine-tuning can overcome\ndomain gaps is well established; demonstrating it again in this narrow context does not\nconstitute a significant advance.\nResponse: Thank you for sharing your detailed and constructive feedback in this second-\nround review. We hope that the revisions and clarifications provided above have addressed\nyour concerns regarding the study scope.\nWe agree that fine-tuning is a well-established method for mitigating domain gaps. However,\nthis is not the central focus of our work. Instead, our findings go beyond the expected\nbenefits of fine-tuning: through extensive experiments, we observed that retinal foundation\nmodels demonstrate strong cross-site generalisability, achieving performance comparable to\nintra-site adaptation using a retinal foundation model pre-trained on the same site. This\nreflects not only a narrowing of performance gaps, but also an impressive level of\ngeneralisability rarely observed in broader machine learning models.\nFurthermore, our linear probing analyses independently confirmed this high level of\ngeneralisation. Beyond generalisability, our study also provides insights into fairness,\nidentifying disparities across demographic subgroups and demonstrating practical strategies\nfor partially improving fairness by involving synthetic data to pre-train retinal foundation\nmodels.\nOverall, we believe this work makes a meaningful contribution to the field by providing\nrigorous empirical evidence on the generalisability and fairness of retinal foundation models.\nOur previous RETFound model has made significant contributions to the ophthalmic AI\ncommunity, serving as one of the most widely used open-source models for diverse clinical\nand research applications [1-5]. In this study, we further introduce RETFound-DINOv2, a\nnew, generalisable, and openly available retinal foundation model that we expect will\ncontinue to benefit a broad range of ocular disease detection and oculomics studies.\nAdditionally, these research findings, particularly regarding generalisation and fairness, will\nhelp optimise future foundation model development and deployment efforts across the wider\ncommunity, as detailed in the response to Question 3.\nReferences:\n[1] Sun, Yuqi, et al. \"A data-efficient strategy for building high-performing medical foundation\nmodels.\" Nature Biomedical Engineering (2025): 1-13.\n[2] Wang, Meng, et al. \"Enhancing diagnostic accuracy in rare and common fundus diseases\nwith a knowledge-rich vision-language model.\" Nature Communications 16.1 (2025): 5528.\n[3] Jiang, Nan, et al. \"A deep learning system for detecting silent brain infarction and\npredicting stroke risk.\" Nature Biomedical Engineering (2025): 1-13.\n[4] Lee, Tae Kwan, et al. \"Vision transformer based interpretable metabolic syndrome\nclassification using retinal Images.\" npj Digital Medicine 8.1 (2025): 205.\n[5] Zhang, Juzhao, et al. \"RETFound-enhanced community-based fundus disease screening:\nreal-world evidence and decision curve analysis.\" NPJ digital medicine 7.1 (2024): 108.\nOpen-source Models:\nhttps://github.com/rmaphoh/RETFound\nReviewer #2\n1. All reviewer concerns have been satisfactorily addressed in the authors' extensive and\ncomplete revision and reply. This reviewer thanks the authors for the\ncomprehensiveness of their revision and reply and for their consideration of queries\nposed.\nResponse: Thank you for acknowledging that the concerns have been fully addressed. Your\ninsightful comments have greatly helped us improve the manuscript."}, {"round": 3, "reviewer_comments": {"reviewer_3": "The authors' corrections substantially answer the concerns presented."}, "author_response": ""}]

Decision letter