Frequency-Balanced Retinal Representation Learning with Mutual Information Regularization
Authors: Seunghoon Lee
Seongjae Kang
Inhyuk Park
Gitaek Kwon
Jihyeon Baek
Doohyun Park
Venue: ICLR 2026 Poster (2026)
Keywords: Masked Image Modeling
Masked Autoencoders
Representation Learning
Mutual Information
Retinal Imaging
Medical Imaging
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: We propose a frequency-oriented perspective on retinal representation learning by analyzing masked autoencoders (MAE) through the lens of spatial frequency. Our analysis shows that MAE favors low-frequency content while under-encoding diagnostically critical high-frequency structures in retinal images. Because retinal pathology often manifests in high-frequency detail, this bias limits diagnostic performance and motivates frequency-balanced representations. Within a mutual-information (MI) formulation of MAE, we introduce the Frequency-Balanced Retinal Masked Autoencoder (RetMAE), which augments the reconstruction objective with a MI regularizer that suppresses low-frequency redundancy and accentuates clinically salient high-frequency information. Without altering the architecture, RetMAE learns frequency-balanced features that surpass those of MAE-based retinal encoders in both quantitative and qualitative evaluations. These results suggest that a frequency-oriented view provides a principled foundation for future advances in ophthalmic modeling, offering new insight into how MAE’s reconstruction objective amplifies ViT’s low-pass tendencies in spatially heterogeneous retinal images and enabling a simple MI-based correction that improves retinal encoders.

Published as a conference paper at ICLR 2026
FREQUENCY-BALANCED RETINAL REPRESENTATION LEARNING WITH MUTUAL INFORMATION REGULAR- IZATION
Seunghoon Lee',
Scongjae Kang'.
Inhyuk Park!,
Gitaek Kwon',
Jihyeon Back!,
Doohyun Park
VVNO Inc.
Gannaa771@@gail1 com, (seongjae kang, Enhyuk.park. par @vuno, co (gitaek. kwon, jihyeon.
@vuno.co
ABSTRACT
We propose frequency-orienteed perspective retinal representation learning by analyzing masked autoencoders (MAE) through the lens of spatial frequency. Our analysis shows that MAE favors low-frequency content while under-encoding diagnostically critical high-frequency structures in retinal images. Because reti- nal pathology often manifests in high-frequency detail, this bias limits diagnostic performance and motivates frequency- -balnnccd representations. Within a mutual- information (MI) formulation of MAE, we introduce the Prequency-Balancee Retinal Masked Autoencoder (ReMMAE), which augments the reconstruction ob- jective with a MI regularizer that suppresses low-frequency redundancy and centuates clinically salient high-frequency information. Without altering the archi- tecture, RetMAE learns eequuenc--balance features that surpass those of MAE- based retinal encoders in both quantitative and qualitative evaluations. These re- sults suggest that a frequency-oriented view provides principled foundation for future advances in ophthalmic modeling, offering new insight into how MAE's reconstruction objective amplifies ViT's low-pass tendencies spatially hetero- geneous retinal images and enabling a simple MI-based correction that improves retinal encoders.
INTRODUCTION
Image (tep orgenal
Vision foundation models learn generalizable representations from large-scale pre-training, transferable to diverse downstream tasks. This paradigm shows promise in medical imaging, particularly fundus photography, where spe- cialized domain knowledge crucial. In the
200-
Fundas ImageNet-1K
Pobbbbiity Donlty
12.5
fundus domain, recent foundation model ap-
Inc:
p-value 0.001
Funci Imase (ep: original bettone HPFI
proaches have explored two primary directions:
1) self-supervised learning (He et al. 2022 Oquab al., 2023) Fang et al., [2023] 2024b) and 2) viston-language pre-training (Radford
56
al., 2 Wang al. 2022). Self-supervised learning approaches design pretext tasks that generate supervisory signals directly from the unlabeled data, including masked autoen- coders (MAE) which reconstruct masked im- age patches without requiring manual annota- tions (Zhou et al. 2023). Vision-language pre- training approaches leverage contrastive learn- ing to learn joint representations by aligning vi-
Eligh Siequescy ratlo
Figure 1: High-frequency in natural vs. fun- dus images. (a) Histogram of high-freeuency ra- tios for fundus and ImageNet- IK (Appendix(A.1) (b) High-pass filtered examples: fundus high- frequency content concentrates near lesions, the optic disc, and vessels (green boxes).
Corresponning author

---

Published as a conference paper at ICLR 2026
sual features with clinical text descriptions (Du et al. 2024 Wu et al. 2024 Silva- Rodriguez al. 2023| Yu et al. 2024). However, high-quality paired image text data are scarce and costly in fundus imaging, which limits the practicality of lision-language models. As result, self-supervised meth- ods such as MAF, which leverage abundant unlabeled retinal images, are more suitable for most publicly available datasets.
However, MAE encounters fundamental limitations when applied to retinal fundus imaging due to its unique characteristics. Unlike natural images, fundus photographs exhibit a distinctive frequency distribution as described in Fig. D diagnostically critical structures are sparse and concentrated in high- free-frr bands, while the majority of image content comprises smooth,
www-freuuncy back-
grounds (Barriga et al.) 2209) [Agurto et al., [011] Jindra 1993; Zhang et al., [2022] Yu et al. [2025).
The standard MAE reconstruction objective with random masking and pixel-wise losses implicitly assumes uniform information density across regions, inducing a bias toward smooth, low-frequency backgrounds while underrepresenting the sparse but elinically crucial high-frequency details re- quired for reliable disease recognition. This goes beyond previously observed low-pass tendencies of ViTs by revealing a mismatch between MAE's uniform-informmtion assumption and the strongly heterogeneous spatial distribution of diagnostic signal in retinal images.
In our preliminary study in Section we systematically analyze two aspects using centered kernel alignment (CKA) (Korbllih et al. 2019) between MAE features and reqquency-separaae inputs: (1) how standard MAË representations exhibit a preference for low-frequency over high-frequency com- ponents, and (2) how this frequeney bias affects downstream diagnostic performance. We apply the discrete Fourier transform (DFT) with a Butterworth (Butterworth et al. 1930) filter to separate fun- dus images into high-frequency (vessel boundaries, lesion edges) and wwwffreyenccy (smooth back- grounds) components. Our analysis reveals a critical mismatch in standard MAE (Table D: while MAE representations strongly align with low-frequency components (CKA 0,990), they poorly capture high-free fighufrry structures (CKA = 0.164). Downstream linear probing performance fur- ther shows the opposite dependence- hiahiee--hhhhh-eeeee high-freyuency components outperform low-frequency ones across multiple datasets, achieving higher macro-average area under the receiver operating charac- teristic curve (AUROC) (0.641 vs. 0.727). In retinal fundus photography, this inverse relationship between representational alignment and diagnostic utility indicates that standard MAE preferen- tially encodes the least informative (low (low-frequency) band. These findings motivate us to propose a frequency- ballncce approach retinal representation learning.
To this end, we propose the Frequency-Balanne Balanced Retinal Masked Autoencoder (RetMAE). a apretrain ing framework that addresses frequency imbalance in fundus images under the mutual information perspective, MI provides a principled basis for learn-
(MI) principle. From an information- ing representations that are compact yet diagnostically sufficient. RetMAF instantiates this principle with a novel objective-the High-frequency MI regularizer (HighFreqMMI (Fig. 2 -hicc priori- tizes the efficient encoding sparse, clinically important high-frequency signals without requiring paired-tex, text supervision, while attenuating fow-frquuecyy redundancy. Importantly, no architectural changes are required performanefreeee gains arise from the MI objective alone. This objective encour- ages frequency-balaneee retinal representutions that suppress irrelevant content while retaining es- sential diagnostic cues.
Our main contributions are as follows:
Frequency bias of MAE: We show that standard MAE pretraining under-encodes clinically salient high-frequency information while over-representing low-frequency background. Building on these findings, we introduce RetMAF, which incorporates the High-frequency MI regularizer (HighFreqMI) to learn frequency-balanced retinal representations.
MI-regularized compactness and sufficiency: Through comprehensive representational anal- yses, we demonstrate that HighFreqMI yields embeddings that are both compact and suffi- cient-reduccng redundancy while preserving clinically meaningful features-thers providing a principled nutual-informaal based correction that directly targets MAE's s under-utilization of high-frequency retinal structure.
Data efficiency without paired text: RetMAE consistently outperforms retinal foundation mod- els, including MAE- based encoders, and also competes favorably with non-MAE paradigms (text- guided and lanionn models), while requiring substantially fewer pretraining samples. Us- ing only ~~25.6k unlabeled fundus images, it achieves a macro macro-average average AUROC of 0.940 across five benchmarks, highlighting strong data efficiency without paired text.
2

---

Published as a conference paper at ICLR 2026
Random Masking
Masked Autoencoder
HighFreqMi
Encoder
Decoder
Lree
Recerssiio
Maskebiig Shaai beceraae
HF Masking
Jumi
Input
Figure 2: Overview of RetMAE. RetMAE extends MAE with intuallinformation regularizer. High-frequency MI (HighFreqMI) aligns the encoder high-frequency context using frequency- selected (HF-maakedd) patches. Here, Z' are linearly projected latents and Zur are high-frequeng, latents; HE, high frequency; EMA, exponential moving average; LP, linear projection.
RELATED WORKS
Fundus foundation models. Large-scale pretraining on fundus images has enabled strong retinal encoders. RETFound (Zhou et al.
2023) shows that masked image modeling (MIM) unlabeled
fundus photographs can transfer across retinal diseases. Multimodal approaches RET-CLIP (Du
et al., 2024), KeepFIT (Wu et al.
2024 and FLAIR (Silva- Rara--odriuuzzzz et al. 2025 further in-
fuse expert knowledge by aligning images with diagnostic reports, while UrFound Yu et al.) 2024)
embeds anatomical priors via matomyy-uudee masking within MIM. Masking-basee fundus hier- archies (Lin et al. 2025) likewise employ aankinn-anddrrconsttrttiiio to learn disease-indicative features, but focus on stage-robust, spatially invariant cues rather than explicitly correcting the low- frequency bias of MAE. Our prequency-balancee MI regularizer instead steers the MAE bottleneck toward lesion-centric high-frequency tokens, and could in principle be combined with such hier- archical objectives. However, in clinical settings, high-quality paired image image-text text corpora are scarce and expensive curate, which constrains we keep the backbone architecture fixed and avoid paired text, using MIM modest unlabeled col- lections together with an optional, off-the-shelf, retina-informed context latent.
scaling. Our work targets this regime:
Frequency structure. Classical analyses show that low-frequue wwwffreueeccy bands capture coarse, global ap- pearance, whereas high-frequency bands encode fine structure (Oppenheim et al. 1979] Oppenheim Piotrowski & Campbell 1982; Hansen & Hess 2007). In retinal fundus images, clin- ically salient lesions (microaneurysms, exudates, and hemorrhages), drusen-related textures, and
& Lim 1981
spectral changes in retinal nerve fiber layer (RNFL) are predominantly contained in high-frequency
components (Zhang et al. 2022
Yu et al.
2025 Barriga et al. 2009 Agurto et al.) 2011 Jindra
1993
Recent MIM variants leverage Fourier or band-aware objectives on frequency-domain inputs or reconstruction targets (Xie et al. [202 Wang al.) 2024b). In contrast, regularize semantic latent representations derived from raw high-frequency regions, making our approach complementary.
MI-based representation learning. From
informmation-thorreti perspective, the information bottleneck favors encoders that preserve task-relevant content while discarding nuisances (Tishby) al., 2000 Tishby & Zaslavsky 2015). and MAE admits MI formulation linking inputs, masked regions, and latents (Huang et al. 2025). Within this view, RetMAE couples complementary MI regularizers to suppress low-frequency redundancy and amplify diagnostically informative high- frequency cues, yielding liqquency-balance retinal representations without architectural changes or paired text.
3 PRELIMINARIES
We briefly review MAE, an information-theoree interpretation of its objective, and the high- frequency token scoring procedure that underpins regularizer.

---

Published as a conference paper at ICLR 2026
Masked autoencoders. MAE (He et al. trains a ViT-based encoder- decoder to reconstruct the missing content from the remaining visible patches. We consider image that has been partitioned into N HW/P² P2 non-overlapping patches of size P x P. and denote by x; RRCC the vectorized pixel values of the i-th patch for i 1,..., N, where N denotes the number of patches per image. Let M C {l.....N] .N denote the masked indices and (l,..., N) \ the visible ones. The reconstruction loss computed only on the masked patches using mean squared error:
2022) randomly masks subset M of image patches and
Lrxe
|x; Σ iEM
8:112
where &; E P denotes the reconstructed pixel vector of the i-th patch. This objective encour- ages the encoder to infer masked content from visible context and yields transferable features for downstream tasks (Kong & Zhang 2023).
An information-theoreti view, MI offers a principled lens on representation learning (Tihbyy et al.) 2000 Tishby& & Zaslavsky 2015). A recent analysis (Huang et al. 2025) shows that the MAE ob- jective can be viewed as minimzzigg the Lagrangian
f = I(xv; 2) + 8 (IXv; XM 1
where X = [e],..... exl denotes the sequence of patch embeddings E RD obtained from an image (e.g., via linear projection), with D the embedding dimension. We denote the visible and masked subsets by Xv = (es)iev and XM [ei]ie.. respectively, and let Z be the encoder's s latent representation of Xy. The scalar 3 > is a weighting coefficient, and (::-) and [[: denote (conditional) Shannon mutual information (Shannon 1948).
In deep networks, successive layers compress inputs into internal representations; when this com- pression is too strong, task-relevant information can be lost, phenomenon known information distortion (Tishby et al., 2000: Tishby & Zaslavsky| 2015). From this viewpoint, Eq. can be in- terpreted as minimizing Lagrangian that explicitly trades off the complexity of the latent repre- sentation against such distortion: the first term I(Xv; Z) quantifies the complexity of the latent description Z, whereas the second term [(Xv; XM Z) plays the role of information-distortion term whose minimization drives Z to retain sufficient information to predict XM.
This view motivates MI-based regularization of the MAE bottleneck beyond the standard reconstruc- tion loss, and Huang et al. (2025) shows that constraining the complexity term / (Xv; 2) improves MAE performance on natural images. Building on the same we introduce domain -peecfc high-frequency regularizer: instead of enforcing mask invariance as in MI-MAE, we align Z with high-frequency retinal context so that the bottleneck prioritizes clinically salient high-frequene structure over low-frequency background in fundus images.
formulation,
High-frrquency extraction. For each fundus image, we first apply a Soft-FOV mask to the green channel. which provides the strongest vessel and lesion contrast (Biswas et al.) 2022) Ooi et al.) 2021 Kumar et al. 2020). We suppress low-frequency content via Gaussian blur, transform the re- sult to the Fourier domain (Brigham 1988), and apply Butterworth high-pass filter (Butterworth et al. 1930) tuned on a small held-out set with vessel/lesion annotations; the inverse transform yields high-pass response map. We then reapply binarized Foft-FOV mask to attenuate residual back- ground and boundary responses, and compute a scalar high-frequency score for each ViT (Dosovit- skiy et al. 2020 token by averaging the masked response over its corresponding non-overlapping P x P patch. Tokens in the top 25% of scores are treated as high- nigh-frequency tokens for our regular- izer; implementation details and hyperparameters are provided in Appendix A.2
UNCOVERING FREQUENCY BIASES IN MAE REPRESENTATIONS
We assess whether a standard MAE with a ViT backbone captures diagnostically salient high- frequency content by pretraining on fundus images (see Sec. A.5) and evaluating the encoder using CKA alongside linear probing AUROC (see Sec. D. Because ViT is a token-based architecture, changing the visible token set naturally induces different internal representations, Kong & Zhang (2023) uses CKA to compare such representations under different training schemes. Following this perspective, Table reports CKA-based comparison across token subsets. More specifically,
4

---

Published as a conference paper at ICLR 2026
treat the representation from the fill, unmasked imput the baseline representation learned by MAE and use CKA to quantify how closely each subset-induced representation aligns with it. In addition to random masking subsets, also consider frequency- -bsed subsets obtained by ranking tokens with our high-frequency scores and assigning the top 25% to high-freq. only and the remaining 75% to low-freq. only (see Appendix|A.2 for details). High CKA means that subset leaves this baseline largely unchanged, whereas low CKA indicates that information specific to that subset is not well reflected the baseline representation.
As summarized in Table [ three observations emerge. (1) 25% masked (which retains 75% of to- kens) maintains AUROC comparable to full and ex- hibits near-reference CKA, revealing substantial re- dundancy in the MAE representations. (2) low-freq. only shows very high CKA yet a clear AUROC drop, indicating that low-frequency background structure dominates the baseline representation while con- tributing limited diagnostic signal. (3) high-freq. only, which keeps only 25% of tokens, yields low CKA yet achieves the best AUROC, consistently outperforming 75% masked at the same token bud- get (25% siswble that small set of high- frequency tokens carries most of the diagnostic signal but is under-emphasized in the baseline rep- resentation. Taken together, standard MAE redundantly encodes www-frequency backgrounds and under-encodes frequecyyy diagnostic structure, motivating representation-leve correction.
Table 1: CKA and linear probing across to- ken subsets. AUROC is the macro-average across five benchmarks; per-dataset results are provided in Appendix A.9
Subset
CKA
AUROC
full
Baseline
0.685
25% masked
0.996
0.686
low-freq, only
0.990
0.641
75% masked
0.890
0.647
high-freq. only
0.164
0.727
5 BREQUENCY-BALANCEE RETINAL MASKED AUTOENCODERS
We introduce a frequency-balancedd retinal masked autoencoder (ReMMAE) that mitigates the fre- quency imbalance identified in Sec. throuugh an MI formulation. As illustrated in Fig. 2 RetMAE augments the standard MAE reconstruction loss with novel MI regularizer- -te high-frequency MI maximization objective to steer the encoder toward compact and task-sufficient representations. We ground the approach in the MI principle (Sec. 5.1) and then detail the objective and training procedure (Sec. 5.2
5.1 MUTUAL INFORMATION AS A PRINCIPLE
We optimize MAE under an MI perspective to embed diagnostically relevant retinal cues. In this framework (Eq. the conditional term [[Xv; XM Z) is instantiated by the standard MAE ob- jective. To regulate the marginal term I [(vv; Z), we align the trainable representation Z with www.--rruueecyfoccssee context latent used reference. Optimizing the conditional and marginal terms jointly yields a freluency-balanced encoder suitable for retinal diagnosis.
Reconstruction conditional mutual information minimization. In the decomposition of 2 the term T(Xp; XMs Z) corresponds to the MAE reconstruction objective. This relationship becomes clear when the decoder is modeled as an isotropic Gaussian with fixed variance, follow- ing probabilistic autoencoder formulations (Bishop & Nasrabadi] 2006 Kingma et al.) 2013 2014] Ciampiconi al. 2023). Under this assumption, the mean squared error (MSE) is proportional to the negative log likeelhood,, a standard and analytically convenient interpretation that links recon- struction to conditional mutual information, as shown the theorem below.
Theorem 1. Let = fo(Xy) be the encoder output and let the decoder fo map Z to the input space. Assume isotropic Gaussian reconstruction model with fixed variance,
(x Z) = N(X, 02),
= E NRNPX(O)),
where 2 is constant. Then minimizing the MAE reconstruction loss
min Lrees
5

---

Published as a conference paper at ICLR 2026
is equivalent, up a positive affine rescaling determined by o², minimizing the conditional mutual information between visible and masked patches,
min [(Xv; Z).
See Appendix A.3 for the proof. The standard reconstruction loss therefore serves principled surrogate for minimizing T(Xv; Xas Z).
Context alignment as marginal mutual information minimization. Within Eq. 2 the marginal term (Xv; 2) can be bounded by aligning the trainable representation to a compact and task- informative context. If the context encoder discards irrelevant variation while preserving diagnostic cues, then aligning the trainable encoder to this context drives Z toward a similarly compact encod- ing of the visible input.
Theorem 2. Let Ze g(X) be a context representation that is E-compact, meaning (X; Zo) S and let Z fXy be the trainable representation produced from visible patches. Suppose training aligument between Z and Ze up to small error daliga and capacity matching up a small mismatch deap- Define 6 == max (alignn 6eap] 2 0. Then
achieves qutuallinformann
I(Xv; Z) & I(Xv: Zz) + 5 I +
[..4 provides the proof. In practice, standard MAE training already produces a reasonably compact trainable representation, and the capacity gap between the trainable and context encoders typically modest. Maximizing I(: Ze) then aligns Z to the e-compact context and tightens control of the marginal term v(vv; Z) Eq. equation 2 When alignment error and capacity mismatch are negligible, the bound approaches I [(Xv; 2) S
Appendix A.4
Taken together, Theorems and 2 yield a clear protocol. The MAE loss reduces the conditional term T(Xv; XM Z), while alignment to s-compact context upper-bounds the marginal term (Xv; Z). Together, these mechanisms yield compact yet diagnostically sufficient representations and jointly optimize the MI Lagrangian in Eq. equation 2
5.2 TRAINING OBJECTIVE
Guided by Theorem 2 we control the marginal I(Xv; Z) by maximizing I(Ze; Z) between the trainable latent Z To (v) and compact context latent Ze- Since mutual information is gener- ally intractable to compute exactly, we instead maximize
barsed lower bound on ((Z.; 2) using the Mutual Information Neural Estimator (MINE) (Donsker & Varadhan 1983 BBelgazi et al. 2018). With critic f& R xR > IR scoring joint pairs (Z., Z') p(Zc, Z) and shuffled (product-marginall pairs (Z:, ; Z3) jipt 2')
p( 2( pp( 2), the objective is
LMINE Ze, 7) - Pzz. ( Ze, 7)
+ log (z) ( Z))}},
(3)
where Z' ~0pppp( 2) is independent. Our implementation of MINE is based on open-source reference implementation
High-frequency MI regularization. We construct feehefreyuency context latent !!! by feed- ing frequency-selecteed visible tokens (Appendix (A.2) into an exponential moving-average (EMA) teacher of the encoder. The HighFreq objective maximizes the mutual information between the trainable representation Z and this context, estimated with MINE:
Lumi LMINE Z2 ZHF).
Our base RetMAE augments MAE reconstruction with HighFreqMI,
Liotal Arec Lree + Aumi Lumi-
(5)
By Theorem 2 the context must be compact; accordingly, activate HighFreqMI only after short warm-up period so that the EMA teacher stabilizes. The loss Lhmi acts as a high-frequency MI reg- ularizer the shared latent Z, mitigating MAE's low-frequency bias and encouraging frequency- balanced representations. A detailed analysis of the computational overhead introduced by our high- frequency regularization is provided in Table 13 in Appendix A.9
https://gi http://gittubbcooo htmm:///eearr/uuuu

---

Published as a conference paper at ICLR 2026
Auxiliary-los-aaugmeine RetMAE. Recent work improves MAE encoders by adding auxiliary objectives beyond reconstruction (e.g., text supervision or aligmment to features from pretrained vision model) (Fang et al. 22023] 2024b Yu et al. 2024). Following this paradigm, consider an variant that adds a generic term Laux on top of MAE+HiggFreqMil:
Liotal = Aree free + Ahmi Limi + Xaux frax-
In our experiments, Laux is instantiated as MINE-based feature alignment between Z and frozen features zeux extracted from pretrained fundus encoder (e.g.. RET-CLIP):
Laux LMINE (Z, 7C2x))
Here, Arec: Ahmi: Aaux 2 0 are non-negative scalar weights; in all experiments, fix Arec and use Ahmi = 0.1 and Aaux = 0.01. A sensitivity analysis of these loss weights and the contribution of each component is provided in Appendix [.9 The results indicate that incorporating Lhmi provides the largest improvements, both alone and in combination with the auxiliary term.
EXPERIMENTS
We evaluate RetMAE on five retinal fundus benchmarks and probe the mechanisms behind gains, testing the hypothesis that it learns requency-balancee task-sufficient representations. Sec. 6.1 specifies baselines, datasets, and evaluation protocols. Sec. 6.22 reports downstream performance, loss ablations, and pretraining data efficiency. See. 6.3 examines frequency balance via (1) layer- wise CKA under frequency-masked inputs, (2) PCA visualizations of class-to-patch attention, and (3) the linear decodability of high-freyuency targets from frozen patch embeddings.
6.1 EXPERIMENTAL SETUP
Models. We evaluate a broad set of fundus pretraining approaches, including a vision-language baseline (RET-CLIP (Du et al. 2024)) and MAE-based methods (RETFound (Zhou et al.) (023))
UrFound Yu et al. 2024)). Complete model configurations and training protocols are provided in Appendix A.6
Datasets. We evaluate RetMAE on four public fundus benchmarks: IDRiD, RFMiD, CHAKSU, and APTOS, spanning three diagnostic categories diabetic retinopathy (DR), age-related macular degeneration (AMD), and glaucoma (GL). RFMiD is split into DR and AMD subsets, which are evaluated independently. APTOS is used solely as an out-of-distribution test sel to avoid data leak- age. A detailed description of tasks, labels, image counts, and splits is provided in Appendix A.7
Evaluation protocols. To evaluate the quality of the learned representations, we adopt a linear probing strategy in which the encoder is frozen and only linear head is trained. Performance is measured using the AUROC.
6.2 MAIN RESULTS
Comparison with signal-level baselines. We compare la- tent high-frequency (HF) regularization via Lami against two signal-level variants built on top of standard MAE coder and evaluated under the same linear probing protocol. HF masking augments MAE by ranking patches based on their high-frequency energy and allocating visible tokens ac- cordingly, echoing salience- and attention guidedd masking ap-
Table 2: Average (AVG) AUROC for frequency-aware baselines.
Method
AVG
MAE
0.685
+ Eymi
0.750
MAE w/ HF masking
0.679
proaches (Choi et al.] 2024 Sick et al.) 2025). HF input con- catenation modifies MAE by injecting an additional high-pass channel into the encoder, following prior frequency-guu -ogedddeggddddddeeoeo de-
Lami
0.737
MAE w/ HF input
0.746
Lymi
0.769
signs (Xie et al., 2022 Wang et al., 2024b). Both approaches expose HF cues directly at the input but do so through signal manipulation rather than latent-level guidance. As shown in Table adding Lumi consistently improves all MAE-based variants, indicat- ing that latent-level HF regularization captures information that input-level methods alone do not cover. Although MAE with HF input and Chmi attains the highest score, HF input requires architec- tural changes and additional preprocessing, whereas Lami architecture-agnotie and lightweight.
7

---

Published as a conference paper at ICLR 2026
Table 3: Linear probing performance (AUROC). Columns marked are out-of-distribution test sets. AVG is the macro-average across datasets. Values in light gray denote evaluation datasets during pretraining. Auxiliary loss: / indicates the use of auxiliary signals beyond images (e.g., text guidance retina-informed off-the-shelf encoder); indicates image-only self-supervised pretraining.
Method
Auxiliary loss
IDRiD
RFMID (DR)
RFMID (AMD)
CHAKSU
APTOS'
AVG
MAE
X
0.726
0.721
0.793
0.371
0.812
0.685
RETFound
X
0.736
0.760
0.784
0.464
0.706
0.690
RetMAE
X
0.816
0.848
0.852
0.516
0.862
0.779
UrFound
V
0.836
0.955
0.953
0.604
0.927
0.855
MAE
0.887
0.949
0.959
0,912
0.910
0,923
RET-CLIP
0.898
0.955
0,962
0.930
0,940
0.937
RetMAE
0.910
0.952
0.980
0.911
0.952
0.941
Table 4: Full fine-tuning performance (AUROC).
Method
IDRiD
RFMID (DR)
RFMiD (AMD)
CHAKSU
APTOS'
AVG
RETFound
0.856
0.926
0.942
0.755
0,902
RET-CLIP
0.879
0.947
0.916
0.836
0,973
0,910
ReMAA
0.856
0.956
0.963
0.903
0.961
0.928
For this reason, and to cleanly isolate the effect of latent-level HF guidance, the standard MAE as our primary baseline. Table Saleerree reports AUROC (linear probing) averaged over the datasets described in Sec. 6.1
Linear probing performance. Table reports linear probing results across five benchmarks. Ret- MAE attains the best macro-average AUROC (0.941). On APTOS, it achieves the top AUROC score (0.952), indicating strong out-of- out-of-distribution (OOD) generalization. Across datasets, RetMAE with auxiliary losses surpasses all image-only MAE variants (MAE, RETFound, UrFound), indicating that MI-based emphasis on high-frequency retinal structure improves MAE pretraining. Because knowledge transfer degrades under distribution shift (Zhang et al.) (025), the auxiliary-only base- line tends to underperform RET-CLIP. Nevertheless, adding HighFreqMi improves macro-average AUROC (AAUROC +0.018) and matches or surpasses RET-CLIP. These results indicate that ex- plicit high-frequency alignment, rather than language supervision, is the principal driver of the gains. Appendix A.9 reports additional results for the precision-recall curve (AUPRC) in Table II as well performance multi-disease diagnosis datasets (Table
Full fine-tuning performance. Tableerreport AUROC under full fine-tuning on five retinal bench- marks. RetMAE attains the best average performance (0.928). exceeding RET-CLIP (0.910) and RETFound (0.876), with particularly strong gains on REMiD (DR/AMD) and CHAKSU while re- maining competitive IDRiD and APTOS. These results show that high-frequency regulariza- tion improves not only linear-probe performance but also full fine-tuning, the regime most relevant for clinical deployment.
Pretraining data efficiency. We assess pretraining data efficiency by subsampling each training fold into nested random subsets at [75, 50, 25, 10, 13% of the full split (S75% > S50% 3
1.0
C S1%). Figure reports the
AUROC
0.8
macro-average AUROC across five datasets as a func- tion of pretraining set size; per-henchmark curves and ad- ditional details are provided in Appendix A.9 RetMAE achieves strong downstream performance with substan- tially fewer images: with only ~1% of the pretraining set (2.6k images), it surpasses RETFound trained on 904k images (AUROC 0.741 0.690); with 5% (12.8k im- ages). it also exceeds UrFound trained on 187k images (AUROC 0.925 vs. 0.855).
0.6
ReMAA
UrFoumd (187k)
0.4
FoTFound (904k)
128
256
Number Pretraining Data
Figure 3: Pretraining data efficiency.
8

---

Published as a conference paper at ICLR 2026
Till vs Thigh
That vs Time
Thigh vs Tuw
1.0
1.0
1.0-
0.3
0.8
0.8
CKA
0.4
0.4
0.4
MAE
0.2
RetMAE
0.2
0.2
5
10
11 12
Layer Index
Figure 4: Frequency-awarr CKA analysis across layers. The three panels compare (left) the full input with high-frequency tokens, (center) the full input with low-frequency tokens, and (right) the high- and low- wwwfrreuency subsets directly. RetMAE achieves more balanced alignment across frequencies, highlighting its frequency-balancee representation.
Although the curve plateaus beyond approximately 12.8k images, a full discussion of the under- lying reasons, including domain redundancy, effective data diversity, and scaling considerations, is provided in Appendix A.8
6.3 FREQUENCY-AWARE REPRESENTATION ANALYSIS
This section evaluates whether RetMAE learns frequency-balanced representations using rrequency-centric analysis of its embeddings. We employ three probes: (1) CKA to assess how high-frequency (HF) and low-frequency (LF) content is encoded and how these subsets align with the full representation; (2) principal component analysis (PCA) of class-to-patch attention to quali- tatively visualize the organization of frequency-specifg retinal structures; and (3) HF decodability to quantify the linear predictability of HF content from patch tokens. Together, these probes show that, in retinal imaging, MAE-style pretraining further biases ViT representations toward low-frequency backgrounds due to its mniform-informatio high-freen frehfrrguency sensitivity without sacrificing low-frequency structure, providing mechanistic in- sight into how MAE can be corrected in this domain.
assumption, while our MI-based regularizer restores
CKA similarity. Building on Sec. we compute CKA between representations obtained under dif- ferent frequency-visibilli conditions: the full-input tokens (Tun), the high-frequency-onl tokens (Trigh), and the low- www-frequency-on/) tokens (Tww). Concretely, we report layer-wise CKA for three pairs: Trall vs. Thigh- Fan vs. Too and Thigh vs. Tiow- Results for MAE and ReMMAE are shown in Fig. 4 (1) Tran vs. Thigh: MAE exhibits high similarity in early layers that declines with depth, indi- cating progressive attenuation of HF content the learned embedding; contrast, RetMAE sustains higher similarity through depth, consistent with preserving task-relevant HF cues under compres- sion. (2) Tunl vs. Tiow: Both models maintain consistently high similarity (++ 0.9-1.0) across layers. Taken together with (1), this shows that RetMAE preserves HF alignment without sacrificing LF structure, whereas MAE becomes increasingly LF-biased with depth. (3) Thigh vs. Tuw The models are similar in early layers, but with depth MAE similarity approaches zero (strong separation of frequency components), while RetMAE remains moderate-to-high indicating a more balanced co- embedding of HF/LF components rather than collapsing. Overall, MAE tends toward LF-dominated representations, whereas RetMAE maintains frequency diversity across layers and keeps HF infor- mation coupled the global embedding.
PCA visualization of class- ta-s-hh attention. We visualize how the global representation attends to local structure by applying PCA to the concatenated class-to-patch attention maps across heads and mapping the top three principal components to RGB (following Oquab et al. (2023)). This yields a compact chromatic embedding in which tokens with similar class semantics appear sim- ilar colors. Models that preserve richer HF detail exhibit sharper chromatic contrast aligned with retinal anatomy. As illustrated in Fig. 5. RetMAE shows clearer separation of clinically salient cleanly delineated from smooth back- ground, and the optic disc is consistently isolated). These visualizations indicate that RetMAE orga- nizes tokens into frequency- awaar, anatomically coherent clusters, preserving high-frequency detail
regions than baseline models (e.g., leston boundaries

---

Published as a conference paper at ICLR 2026
Input
High-Freq.
RetMAE (Ours)
RET-CLIP
RETFound
UrFound
Figure 5: PCA visualization of Blass-to-patch attention. RetMAE shows pronounced chromatic separation-optic disc (red), hemorraages/exuates (green), and background (blue)-with clear seg- regation aligned with clinical priors, a property important for retinal diagnosis. Additional examples, visualization details, and attention maps for large lesion
provided in Appendix A.9
while maintaining low-frequency structure. Across the three analyses, we find that RetMAE learns seqquency-balanee representations: aligns well with high-frequency information while preserving low-frequency structure. The learned representations maintain high- and low-frequency components distinct rather than collapsing and exhibit clearer anatomical organization. This frequency balance, in turn, contributes to improved diagnostic performance observed on downstream tasks (Sec. 6.2)
High-frequency decodability of patch tokens. To assess whether HF' information remains decod- able from learned features, we predict the patch- level HF targets defined in Eq. 22 from patch tokens using ridge regression, and compute the coefficient of determination R² layer-wise (aver- aged over images). Figure [6] summarizes results for early, middle, and late layers, while per-image R² distributions are provided in Appendix A.9 RetMAE yields near-ceiling R² across all depths (carly 0.975, middle 0.994, late 0.991; all < 10-15), demonstrating that HF content robustly linearly decodable from patch tokens. In contrast, other models show markedly lower R², often peaking middle layers but declining later ones (e.g., RET-CLIP 0.764 at middle; RETFound 0.724 at middle), reflecting reduced HF sensitivity the representation level. When averaged across layers, RetMAE achieves the highest mean HF decodability (R² 0.987), consistent with the frequency-aware structure revealed by analysis in Fig. 4
0.9
0.6
0.5
0.3
0.2
Early
Middle
Late
Layer
RETFound
RET-CLIP
UtFound
ReMMA
Figure 6: Layer-wise high-frequency decod-
ability (R²). Asterisks denote p 0.001.
DISCUSSION
This work provides, knowledge, the first latent-level mutual-informattin formulation that explicitly diagnoses and mitigates the low-frequency bias of MAE in medical imaging. We intro- duce RetMAE, which augments the reconstruction objective with high-frequency MI regularizer to reduce low-frequency redundancy and emphasize clinically salient structure, without modifying the architecture or requiring paired supervision. We further find that signal-level frequency- approaches, including high-frequency input augmentation and frequency-aware masking, provide complementary benefits when combined with our latent-level HighFreqMI regularizer, supporting the generality of the proposed framework. A key limitation of this study is that our evaluation is restricted color fundus images. While the same mechanism may extend to domains which task- relevant signals are partially concentrated in high-frequency structure, such industrial anomaly detection or other medical imaging modalities, this generalization remains to be validated. Future work includes extending RetMAE additional imaging modalities, exploring alternative backbone architectures, adopting adaptive or data-driven context targets, and conducting broader distribution- shift evaluations.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
All implementation details are in Appendix A.3 baselines, datasets, splits, and metrics are in Ap- pendix 6.1 with dataset notes in Appendix A.7 and frequency preprocessing in Appendix A.2 We will release the codebase with model implementation. All runs used fixed random seeds. Experi- ments were run on 8 NVIDIA RTX 3090 (24 GB) using PyTorch Lightning 2.4.0; training used torch. compile mode (no mixed precision). Training and evaluation were conducted on the same machine; multi-GPU runs used DDP (NCCL, fixed global batch size). The proposed method (Ret- MAB) and the MAE baseline instantiate backhones via timm (version 2 1.0.12); all other baselines were obtained from their official GitHub repositories.
ETHICS STATEMENT
This work uses only de-identified, publicly available retinal fundus datasets (see Appendix (.7]) no additional patient data were collected. Models and code released for research use only and are not intended for clinical decision-making without further validation and regulatory approval. We acknowledge potential dataset biases and report results across multiple benchmarks support transparent evaluation.
REFERENCES
Anum Abdul Salam, M Usman Akram, and Shoab Ahmed Khan. Retina identification database (ridb), 2020. lattps::///daa men.eely.eeee.....m.mmm. com/ cct.///aatattttteaaaaaaaaaa
Carla Agurto, E Simon Barriga, Victor Murray, Sheila Nemeth, Robert Crammer, Wendall Bauman, Gilberto Zamora, Marios S Pattichis, and Peter Soliz. Automatic detection of diabetic retinopathy and age-related macular degeneration in digital fundus images. Investigative ophthalmology & visual science, 528))5586,-5871, 2011.
Muhammad Usman Akram, Shahzad Akbar, Taimur Hassan, Sajid Gul Khawaja, Ubaidullah Yasin, and Imran Basit. Data fundus images for vessels segmentation, detection of hypertensive retinopathy, diabetic retinopathy and papilledema. Data in brief, 29:105282, 2020.
Ahmed Almazroa, Sami Alodhayb, Essameldin Osman, Eslam Ramadan, Mohammed Hum madi, Mohammed Dlaim, Muhannad Alkatee, Kaamran Raahemifar, and Vasudevan Lakshmi- narayanan. Retinal fundus images for glaucoma analysis: the riga dataset. In Medical Imaging 2018: Imaging Informatics for Healthcare, Rexearch, and Applications, volume 10579, pp. 55-62. SPIE, 2018
Muhammad Naseer Bajwa, Gur Amrit Pal Singh, Wolfgang Neumeier, Muhammad Imran Malik, Andreas Dengel, and Sheraz Ahmed. G1020: A benchmark retinal fundus image dataset for computer-aided glaucoma detection. In 2020 International Joint Conference Neural Networks (JJCN), pp. 1-7. IEEE, 2020.
Eduardo Barriga, Victor Murray, Carla Agurto, Marios S Pattichis, S Russell, MD Abramoff, Herbert Davis, and Peter Soliz. Multi-scale am-fm for lesion phenotyping on age-related macular degeneration. In 2009 22nd IEEE international symposium on computer-based medical systems, pp. 1-5. IEEE, 2009.
Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Devon Hjelm. Mutual information neural estimation. In International conference on machine learning, pp. 531-540. PMLR, 2018
Veronica Elisa Castillo Benítez, Ingrid Castro Matto, Julio César Mello Román, José Luis Vázquez Noguera, Miguel Gareía- -orres, Jordan Ayala, Diego Pinto-Roa, Pedro Gardel-Sotomayor Jacques Facon, and Sebastian Alberto Grillo. Dataset from fundus images for the study of diabetic retinopathy. Data brief, 36: 107068, 2021.
Christopher M Bishop and Nasser M Nasrabadi. Pattern recognition and machine learning, vol- ume 4. Springer, 2006.
11

---

Published as a conference paper at ICLR 2026
Sangeeta Biswas, Md Iqbal Aziz Khan, Md Tanvir Hossain, Angkan Biswas, Takayoshi Nakai, and Johan Rohdin. Which color channel is better for diagnosing retinal diseases automatically in color fundus photographs? Life, 12(7):973, 2022.
E Oran Brigham. The fast Fourier transform and its applications. Prentice-Hall Inc., 1988.
Attila Budai, Rüdiger Bock, Andreas Maier, Joachim Hornegger, and Georg Michelson. Robust vessel segmentation in fundus images. International journal of biomedical imaging, 2013(1): 154860, 2013.
Stephen Butterworth et al. On the theory of filter amplifiers. Wireless Engineer, 7(6):536-555 541,, 1930.
Enrique J Carmona, Mariano Rincón, Julián García-Feijoó, and José M Martínez-de-la Casa. Iden- tification of the optic nerve head with genetic algorithms. Artificial intelligence medicine, 43 (3):243-259, 2008
Ling-Ping Cen, Jie Ji, Jian-Wei Lin, Si-Tong Ju, Hong-Jie Lin, Tai-Ping Li, Yun Wang, Jian-Feng Yang, Yu-Fen Liu, Shaoying Tan, al. Automatic detection of 39 fundus diseases and conditions in retinal photographs using deep neural networks. Nature communications, 12(1))4828, 2021.
Subhadeep Chakraborty. Drimdb (diabetic retinopathy images database), 2024.
URL https:
77 www. www.lakglle..... com./ds.................ooo..... /
Hyesong Choi, Hyejin Park, Kwang Moo Yi, Sungmin Cha, and Dongbo Min. Salience-based adaptive masking: revisiting token dynamics for enhanced pre-training. In European Conference on Computer Vision, pp. 343-359. Springer, 2024.
Lorenzo Ciampiconi. Adam Elwood. Marco Leonardi, Ashraf Mohamed, and Alessandro Rozza. A survey and taxonomy of loss functions in machine learning. arXiv preprint arXiv:2301.0557, 2023.
Behdad Dashtbozorg, Jiong Zhang, Fan Huang, and Bart M. ter Haar Romeny. Retinal microa- neurysms detection using local convergence index features. IEEE Transactions on Image Pro- cessing, 27(7):3300-333 2018. doi: 10.1 (09//TI.2018.2815555
Mostafa Dehghani, Josip Djolonga, Basil Mustafa, Piotr Padlewski. Jonathan Heek, Justin Gilmer, Andreas Peter Steiner, Mathilde Caron, Robert Geirhos, Ibrahim Alabdulmohsin, al. Scaling vision transformers to 22 billion parameters. In International conference on machine learning, pp. 7480-7511 75100 PMLR, 2023.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 JEEE conference on computer vision and pattern recognition, pp. 248-255. Icee, 2009.
D Jeba Derwin, S Tamil Selvi, O Jeba Singh, and B Priestly Shan. A novel automated system of discriminating microaneurysms in fundus images. Biomedical Signal Processing and Control, 58: 101833 001839,00 2020.
M. D. Donsker and S. R.S. Varadhan. Asymptotic evaluation of certain markov process expectations for large time. iv. Communications Pure and Applied Mathematics, 36(2):183-212, March 1983. ISSN 0010-3640. doi: 10.1002/cpa.33160360004
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. 19999, 2020.
Jiawei Du, Jia Guo, Weihang Zhang, Shengzhu Yang, Hanruo Liu, Huiqi Li, and Ningli Wang- Ret-clip: A retinal image foundation model pre- pre-trained with clinical diagnostic reports. In In- ternational Conference on Medical Image Computing and Conputer-Assister Intervention, pp- 709-719. Springer, 2024.
12

---

Published as a conference paper at ICLR 2026
Huihui Fang, Fei Li, Junde Wu, Huazhu Fu, Xu Sun, Jose Ignacio Orlando, Hrvoje
Bogunovic, Xiulan Zhang. and Yanwu Xu. Open Fundus Photograph Dataset with
Pathologic Myopia Recognition and Anatomical Structure Annotation, 2024a. URL
wttphaaaw...omaat.clleeeeee/e/ee http:///eee...aaaea..///////ooaa Burrous_hotogrrpphhoo
Sataaaptp_aatte_________ wtceeastt_wtttt___ee http://wwttaaahhhooooooooooo Myoppiiiii__M__aaa__a. htg.://oooogoo..ooooooo.aao.ooo. R_eooogiiiconnooonono..ooo and Anatomical Streall_tuuuuuuee htt.:euuree.nmoottttt/o///223311
Yuxin Fang, Wen Wang, Binhui Xie, Quan Sun, Ledell Wu, Xinggang Wang, Tiejun Huang, Xinlong Wang, and Yue Cao. Eva: Exploring the limits of masked visual representation learning at scale. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 19358-19368 2023.
Yuxin Fang. Quan Sun, Xinggang Wang, Tiejun Huang, Xinlong Wang, and Yue Cao. Eva-02: A visual representation for neon genesis. Image and Vision Computing. 149:105171, 2024b.
Damian JJ Farnell, Fraser N Hatfield, Paul Knox, Michael Reakes, Stan Spencer, David Parry, and Simon Harding. Enhancement of blood vessels in digital fundus photographs via the application of multiscale line operators. Journal of the Franklin institute, 345(7):748-777) 765,--65555 2008.
Luca Giancardo, Fabrice Meriaudeau, Thomas P Karnowski, Yaqin Li, Seema Garg, Kenneth W Tobin Jr, and Edward Chaum. Exudate-based diabetic macular edema detection in fundus images using publicly available datasets. Medical image analysis, 16(1):216-25 62666,2266222 2012.
Tianjiao Guo, Jie Yang, and Qi Yu. Diabetic retinopathy lesion segmentation using deep multi-scale framework. Biomedical Signal Processing and Control, 88: 105050, 2024.
Bruce C Hansen and Robert Hess. Structural sparseness and spatial phase alignment in natural scenes. Journal of the Optical Society of America A, 24(7):1873-1885 2007.
Taimur Hassan, Muhammad Usman Akram, Muhammad Furqan Masood, and Ubaidullah Yasin. Deep structure tensor graph search framework for automated extraction and characterization of retinal layers and fluid pathology in retinal sd-oct scans. Computers in biology and medicine, 105: 112-124, 2019.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked au- toencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16000-16009, 2022.
Danny Hernandez, Jared Kaplan, Tom Henighan, and Sam McCandlish. Scaling laws for transfer. arXiv preprint arXiv: 102.0229 2021.
Carlos Hernandez-Matas, Xenophon Zabulis, Areti Triantafyllou, Panagiota Anyfanti, Stella Douma, and Antonis Argyros. Fire: fundus image registration dataset. Modeling and Arti- ficial Intelligence in Ophthalmology, 1(4):16-28, 2017.
Joel Hestness, Sharan Narang, Newsha Ardalani, Gregory Diamos, Heewoo Jun, Hassan Kianinejad, Md Mostofa Ali Patwary, Yang Yang, and Yanqi Zhou. Deep learning scaling is predictable, empirically. arXiv preprint
2019.
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Train- ing compute-optimal large language models. arXiv preprint arXiv:2203.15556 2022.
Tao Huang, Yanxiang Ma, Shan You, and Chang Xu. Learning mask invariant mutual information for masked image modeling. In The Thirteenth International Conference on Learning Represen- tations, 2025. URL htttttss:///peeeveieeeeee
net/ htt.:/ooommmmddooaaaaaa
Kai Jin, Xingru Huang, Jingxing Zhou, Yunxiang Li, Yan Yan, Yibao Sun, Qianni Zhang, Yaqi Wang, and Juan Ye. Fives: A fundus image dataset for artificial intelligence based vessel segmentation. Scientific data, 9(1):475, 2022.
Kai Jin, Zhiyuan Gao, Xiaoyu Jiang, Yaqi Wang, Xiaoyu Ma, Yunxiang Li, and Juan Ye. Mshf: A multi multi-source heterogeneous fundus (mshf) dataset for image quality assessment. Scientific data, 10(1) 2086) 2023.
13

---

Published as a conference paper at ICLR 2026
Lawrence F Jindra. Early glaucoma detection by fourier transform analysis of digitized eye fundus images, August 1993. US Patent 5,233,517.
Kaggle. Diabetic retinopathy detection, 2015.
URL
Blabetic-retinopattyyettttt
Kaggle. Cataract dataset, 2020. URL https:/ www. htt.g/l...cc//aattttts////gggg
cataractdatasset
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.0836 2020.
Tomi Kauppi, Valentina Kalesnykiene, Koni-Kristin Kamarainen, Lasse Lensu, liris Sorri, Asta Raninen, Raija Voutilainen, Hannu Uusitalo, Heikki Kälviäinen, and Juhani Pietilä. The diaretdb1 diabetic retinopathy database and evaluation protocol. In BMVC, volume pp. 10. Citeseer, 2007.
Diederik Kingma, Max Welling, et al. Auto-encoding variational bayes, 2013.
Durk P Kingma, Shakir Mohamed, Danilo Jimenez Rezende, and Max Welling. Semi-supervised learning with deep generative models. Advances in neural information processing systems, 27, 2014.
Xiangwen Kong and Xiangyu Zhang. Understanding masked image modeling via learning occlusion invariant feature. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6241-6251, 2023.
Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representations revisited. In International conference on machine learning, pp. 3519- 3529. PMIR, 2019.
Oleksandr Kovalyk, Juan Korales-Sánchees Rafael Verdii-Monedero, Inmaculada Bellés-Navame Ana Calazón-Cabanes. and José-Luis Sancho-Gómez. Papila: Dataset with fundus images and clinical data both eyes of the same patient for glaucoma assessment. Scientific Data, 91):291, 2022.
JR Harish Kumar, Chandra Sekhar Seelamantula, JH Gagan, Yogish S Kamath, Neetha IR Kuzhup- pilly, U Vivekanand, Preeti Gupta, and Shilpa Patil. Cháksu: A glaucoma specific fundus image database. Scientific data, 10(1):70, 2023.
Kundan Kumar, Debashisa Samal, and Suraj. Automated retinal vessel segmentation based on mor- phological preprocessing and 2d 2d-gabor wavelets. In Advanced Computing and Intelligent Engi- neering: Proceedings of ICACIE 2018, Volume 1, pp. 41 411-423. Springer, 2020.
Liu Li, Mai Xu, Xiaofei Wang, Lai Jiang, and Hanruo Liu. Attention based glaucoma detection: A large-scale database and cnn model. In Proceedings of the IEEF/CVE conference computer vision and pattern recognition, pp. 10571-10580, 2019a.
Tao Li, Yinggi Gao, Kai Wang, Song Guo, Hanruo Liu, and Hong Kang. Diagnostic assessment of deep learning algorithms for diabetic retinopathy screening. Information Sciences, 501:511-522, 2019b.
Li Lin, Meng Li, Yijin Huang, Pujin Cheng, Honghui Xia, Kai Wang, Jin Yuan, and Xiaoying Tang. The sustech-sysu dataset automated exudate detection and diabetic retinopathy grading. Sci- entific Data, 7(1):409, 2020.
Yuxin Lin, Wei Wang, Xiaoling Luo, Zhihao Wu. Chengliang Liu, Jie Wen, and Yong Xu. Deep hier- archies and invariant iisease-indicatte feature learning for computer aided diagnosis of multiple fundus diseases. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 5325-5333, 2025.
14

---

Published as a conference paper at ICLR 2026
Ruhan Liu, Xiangning Wang, Qiang Wu, Ling Dai, Xi Fang. Tao Yan, Jaemin Son, Shiqi Tang, Jiang Li, Zijian Gao, et al. Deepdrid: Diabetic retinopathy grading and image quality estimation challenge. Patterns, 3(6), 2022.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017.
Yeganeh Madadi, Hina Raja, Koenraad A. Vermeer, Hans G. Lemij, Xiaoqin Huang. Eunjin Kim, Seunghoon Lee, Gitaek Kwon, Hyunwoo Kim, Jaeyoung Kim, Adrian Galdran, Miguel A. González. Ballester, Dan Presil, Kristhian Aguilar, Victor Cavalcante, Celso Carvalho, Waldir Sabino, Mateus Oliveira, Hui Lin, Charilaos Apostolidis, Aggelos K. Katsaggelos, Tomasz Kubrak, A. Casado-Garéıa, J. Heras, M. Ortega, L. Ramos, Philippe Zhang, Yihao Li, Jing Zhang, Weili Jiang, Pierre-Henri Conze, Mathieu Lamard, Gwenolé Quellee, Mostafa El Habib Daho, Madukuri Shaurya, Anumeha Varma, Monika Agrawal, and Siamak Yousefi. Justraigs: Justified referral ai glaucoma screening challenge. IEEE Transactions on Medical Imaging, pp. 1-1, 2025. doi: 10. 109/T00.222...3998827
Karthik Maggie and Sohier Dane. Aptos 2019 blindness detection, 2019.
URL https://
kaggle. cwm
9-11l-1l110 plindness: detection
Henry Mann and Donald R Whitney. On a test of whether one of two random variables is stochas- tically larger than the other. The annals of mathematical statistics, pp. 50 50-60, 1947.
Max Marion, Ahmet Üstün, Luiza Pozzobon, Alex Wang, Marzieh Fadaee, and Sara Hooker. When less is more: Investigating data pruning for pretraining Ilms at scale. CoRR, 2023.
Baharan Mirzasoleiman, Jeff Bilmes, and Jure Leskovec. Coresets for data-efficient training of machine learning models. In International Conference on Machine Learning, pp. 6950-6960.
PMLR, 2020.
Luis Filipe Nakayama, Mariana Goncalves, L Zago Ribeiro, Helen Santos, Daniel Ferraz, Fernando Malerbi, Leo Anthony Celi, and Caio Regatieri. A brazilian multilabel ophthalmological dataset (brset). PhysioNet https://doi. org/10, 13026:2, 2023.
Meindert Niemeijer, Bram Van Ginneken, Michael J Cree, Atsushi Mizutani, Gwénolé Quellec, Clara Sánchez, Bob Zhang, Roberto Hornero, Mathieu Lamard, Chisako Muramatsu, et al. Retinopathy online challenge: automatic detection of microaneurysms in digital color fundus pho- tographs. IEEE transactions on medical imaging, 29(1):185 195, 2009.
NIHDS-PKU. Competition ocular disease intelligent recognition, 2019. URL https://
odi r20.9.011,.0 0ww.9..raddddoomm chanadd--chllllen.oog...
Alexander Ze Hwan Ooi, Zunaina Embong, Aini Ismafairus Abd Hamid, Rafidah Zainon, Shir Li Wang, Theam Foo Ng, Rostam Affendi Hamzah, Soo Siang Teoh, and Haidi Ibrahim. Interactive blood vessel segmentation from retinal fundus image based on canny edge detector. Sensors, 21 (19):6380, 2021.
A Oppenheim, Jae Lim, Gary Kopec, and SC Pohlig, Phase in speech and pictures. In ICASSP'79. IEEE International Conference on Acoustics, Speech, and Signal Processing, volume 4, pp. 632- 637. IEEE, 1979.
Alan V Oppenheim and Jae S Lim. The importance of phase in signals. Proceedings of the IEEE, 59,,5,52-541, 1981.
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniee, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprini arXiv:2304.0711 2023.
José Ignacio Orlando, Huazhu Fu, João Barbosa Breda, Karel Van Keer, Deepti Bathula, Andrés Diaz-Pinto, Ruogu Fang, Pheng-Ann Heng, Jeyoung Kim, JoonHo Lee, al. Refuge challenge: A unified framework for evaluating automated methods for glaucoma assessment from fundus photographs. Medical image analysis, 59:101570, 2020a.
15

---

Published as a conference paper at ICLR 2026
José Ignacio Orlando, João Barbosa-Bress Karel Van Keer, Matthew B. Blaschko, Pablo Javier Blanco, and Carlos Alberto Bulant. LES-AV dataset, 2020b. URL https://Eigsharre. com/ werticles/dataeeee LES dataset/11857655
Samiksha Pachade, Prasanna Porwal, Dhanshree Thulkar, Manesh Kokare, Girish Deshmukh, Vivek Sahasrabuddhe, Luca Giancardo, Gwenolé Quellec, and Fabrice Mériaudeau. Retinal fundus multi-disease image dataset (rfmid): A dataset for multi-disease detection research. Data, 6(2):
14, 2021.
Sachin Panchal, Ankita Naik, Manesh Kokare, Samiksha Pachade, Rushikesh Naigaonkar, Prerana Phadnis, and Archana Bhange. Retinal fundus multi-diseas image dataset (rfmid) 2.0: a dataset of frequently and rarely identified diseases. Data, 8(2):29, 2023.
Samantha K Paul, Ian Pan, and Warren M Sobol. Efficient labeling of retinal fundus photographs using deep active learning. Journal of Medical Imaging, (000060001-064000 2022.
Leon N Piotrowski and Fergus W Campbell. demonstration of the visual importance and flexibility of apatial-frequeen amplitude and phase. Perception, 11(3):337-366, 1982.
Ramon Pires, Herbert F Jelinek, Jacques Wainer, Eduardo Valle, and Anderson Rocha. Advancing vigual--viuuawworrd representations for lesion classification in retinal images. PloS one, 9(6): c96814, 2014.
Natasa Popovic, Stela Vujosevic, Miroslav Radunović, Miodrag Radunović, and Tomo Popovic. Trend database: Retinal images of healthy young subjects visualized by a portable digital non- mydriatic fundus camera. PLoS One, 16(7)ee0254918, 2021.
Prasanna Porwal, Samiksha Pachade, Manesh Kokare, Girish Deshmukh, Jaemin Son, Woong Bac, Lihong Liu, Jianzong Wang, Xinhui Liu, Liangxin Gao, et al. Idrid: Diabetic retinopathy- segmentation and grading challenge. Medical image analysis, 59:101561, 2020.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- 8748-8763. PmLR, 2021.
Edgar Riba, Dmytro Mishkin, Daniel Ponsa, Ethan Rublee, and Gary Bradski. Kornia: open source differentiable computer vision library for pytorch. In Proceedings the IEEE/CVF Winter Conference Applications of Computer Vision, pp. 3674-3683, 2020.
Claude E Shannon. A mathematical theory of communication. The Bell system technical journal, 27(3):379-42) 1948.
Leon Sick, Dominik Engel, Pedro Hermosilla, and Timo Ropinski. Attention-guided masked autoen- coders for learning image representations. In 2025 JEFE/CVF Winter Conference on Applications of Computer Vision ( (WACV), pp. 836-846. IEFF, 2025.
Julio Sidva-Rodriguez, Hadi Chakor, Riadh Kobbi, Jose Dolz, and Ismail Ben Ayed. A founda- tion sanguage-image model of the retina (flair): Encoding expert knowledge in text supervision. Medical Image Analysis, 99:103357, 2025.
Raghavendra Singh. Leveraging perceptual scores for dataset pruning in computer vision tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp-
2024.
Hidenori Takahashi, Hironobu Tampo, Yusuke Arai, Yuji Inoue, and Hidetoshi Kawashima. Ap- plying artificial intelligence to disease staging: Deep learning for improved staging of diabetic retinopathy. PloS one, 1216)901777900 2017.
Naftali Tishby and Noga Zaslavsky. Deep learning and the information bottleneck principle. In 2015 ieee information theory workshop (itw), pp. 1-5. leee, 2015.
Naftali Tishby, Fernando C Pereira, and William Bialek. The information bottleneck method. arXiv preprint physics/0004057, 2000.
16

---

Published as a conference paper at ICLR 2026
Haoran Wang, Qiuye Jin, Shiman Li, Siyu Liu, Manning Wang. and Zhijian Song. A comprehensive survey on deep active learning in medical image analysis. Medical Image Analysis, 95: 103201, 2024a.
Wenxuan Wang, Jing Wang, Chen Chen, Jianbo Jiao, Yuanxiu Cai, Shanshan Song, and Jiangyun Li. Fremim: Fourier transform meets masked image modeling for medical image segmentation. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp. 7860- 7870, 2024b.
Zifeng Wang, Zhenbang Wu, Dinesh Agarwal, and Jimeng Sun. Medclip: Contrastive learning from unpaired medical images and text. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Conference on Empirical Methods in Natural Language Process- ing, volume 2022, pp. 3876, 2022.
Shuhei Watanabe. Tree-structured parzen estimator: Understanding its algorithm components and their roles for better empirical performance. arXiv preprint arXiv:2304. 1/27, 2023.
Qijie Wei, Xirong Li, Weihong Yu, Xiao Zhang, Yongpeng Zhang, Bojie Hu, Bin Mo, Di Gong, Ning Chen, Dayong Ding, and Youxin Chen. Learn to segment retinal lesions and beyond. In International Conference on Pattern Recognition (ICPR), 2020.
Ruiqi Wu, Chenran Zhang, Jianle Zhang, Yi Zhou, Tao Zhou, and Huazhu Fu, Mm-retinal: Enowledge-enhanced foundational pretraining with fundus image-text expertise. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 722-732. Springer, 2024.
Jiahao Xie, Wei Li, Xiaohang Zhan, Ziwei Liu, Yew Soon Ong, and Chen Change Loy. Masked frequency modeling for self-supervisee visual pre-training. arXiv preprint arXiv:2206.07706, 2022.
Chang Yu, Qian Ma, Jing Li, Qiuyang Zhang, Jin Yao, Biao Yan, and Zhenhua Wang. Ff-resnet-dr model: deep learning model for diabetic retinopathy grading by frequency domain attention. Electronic Research Archive, 33(2):725-743 2025.
Kai Yu, Yang Zhou, Yang Bai, Zhi Da Soh, Xinxing Xu, Rick Siow Mong Goh, Ching- Yu Cheng, and Yong Liu. Urfound: Towards universal retinal foundation models via knowledge-guided masked modeling. In International Conference on Medical Image Computing and Computer- Assisted Intervention, pp. 753-762. Springer, 2024.
Jiong Zhang, Behdad Dashthozorg. Erik Bekkers, Josien PW Pluim, Remco Duits, and Bart M ter Haar Romeny. Robust retinal vessel segmentation via locally adaptive derivative frames in orientation scores. IEEE transactions on medical imaging, 35(12):2631-2444 2016.
Songming Zhang, Yuxiao Luo, Ziyu Lyu, and Xiaofeng Chen. Shiftkd: Benchmarking knowledge distillation under distribution shift. Neural Networks, 192:107838, 2025.
Xugang Zhang, Yanfeng Kuang, and Junping Yao. Detection of microaneurysms in color fundus images based on local fourier transform. Biomedical Signal Processing and Control, 76: 103648, 2022.
Zhuo Zhang, Feng Shou Yin, Jiang Liu, Wing Kee Wong, Ngan Meng Tan, Beng Hai Lee, Jun Cheng. and Tien Yin Wong. Origa-light: An online retinal fundus image database for glau- coma analysis and research. In 2010 Annual international conference of the IEEE engineering in medicine and biology, pp. 3065-3068. IEEE, 2010.
Y. Zhou, B. Wang, L. Huang, Cui, and L. Shao. A benchmark for studying diabetic retinopathy: Segmentation, grading, and transferability. IEEE Transactions on Medical Imaging, 40(3):818- 828. 2021. doi: 10. 1110/10M...0000000
Yukun Zhou, Mark A Chia, Siegfried K Wagner, Murat S Ayhan, Dominic Williamson, Robbert R Struyven, Timing Liu, Moucheng Xu, Mateo G Lozano, Peter Woodward-Court, et al. A founda- tion model for generalizable disease detection from retinal images. Nature, 622(7981):156-16) 2023.
17

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 FUNDUS IMAGES CONTAIN LESS HIGH-FREQUENCY CONTENT
As illustrated in Figure natural images typically exhibit broad frequeney spectrum in which edges, textures, and object boundaries contribute substantial high-frequency (HF) energy. By con- trast, fundus photographs are markedly www-freqency--dome dominated: most pixels belong smooth background regions, while clinically meaningful cructures-hemorrhee the optic disc, and vessels-are sparse and concentrated HF bands. We substantiate this observa- tion with the prrcentiil--hresholle analysis described below.
drusen, hard exudates,
Quantitative Method: 75th-Percentile Threshold Let (x) denote the per-pixel HF magnitude map of image x (see Sec. A.2). We define dataset-level threshold from ImageNet-1K (Deng al.
2009) as
T quantile, 75 (HF (ImageNeett-)1))
For any image with N pixels, we then compute the fraction of pixels exceeding this reference threshold:
R(:;T) N li x
Thus, R(x; 7) measures how many pixels a are "HF-active" relative to the ImageNet-derived reference distribution.
Quantitative Results: Histogram Analysis Figure depicts the empirical distribution of R(æ; 7) for fundus images and ImageNer-1K, K. The fundus distribution concentrates near zero, whereas Ima- geNet exhibits a broader, right-shifted distribution, indicating substantially larger fraction of HF- active pixels. Mann-Whitney U test (Mann & Whitney 1947) indicates highly significant dif- ference (p = 4.75 x 10-12), confirming that fundus photographs contain markedly less HF con- tent. These results quantitatively corroborate the qualitative patterns Figure [ clinically relevant structures in fundus images are sparse and localized within HF regions, while the overall image is dominated by low-frequency background.
18

---

Published as a conference paper at ICLR 2026
A.2 HIGH-FREQUERT EXTRACTION
To isolate diagnostically relevant high-frequency signals, we first extract and normalize the green channel, then generate a soft field-of-view mask to attenuate edge artifacts. We apply this mask before Gaussian blurring so that only the central retinal region contributes to the frequency analysis. After suppressing low-frequency background with the blur, we transform the result the Fourier domain and filter it using a Butterworth high-pass filter whose cutoff and order were tuned on held-out set of fundus images. The dataset used for this adjustment, which included annotations of the ground truth vessel and the lesion, was strictly excluded from both pretraining and evaluation. Inverting the filtered spectrum back to the spatial domain produces a high-pass filtered (HPF) image, which we re-mask and normalize to eliminate any residual boundary effects.
Foft-FOV Mask Generation To smoothly attenuate high-frequency artifacts at the field-of-view (FoV) boundaries and enhance lesion-related high-frequency signals in retinal fundus images, we compute a soft seld-of-view (Soft-FOX mask. Given an input image I E xxx we first form a grayscale image by channel averaging:
C
Yn,w =
Joh.w
(8)
C c=1
We define a threshold
T,
Tiov max h h,w
(9)
and generate binary mask
[1, Yn,w > Tsx
B
(10)
10, otherwise.
To soften the edges, convolve B with separable 2D Gaussian kernel
Ko,, (z. 8) = ki. (x) ko. (v),
ko,(t) (t) = V2 o,
1
expl cxp(-t/(20?)).
applied horizontally and then vertically:
B (B * ko.) tu Ko.-
(11)
Finally, we clamp to [0. 1] to obtain the Soft-FOV mask S E [0, 1)x
Sn, min(max (B,,, 0), 1).
(12)
High-pass Filtering After obtaining S, we extract high-frequency components from the green channel of the input, denoted 7(0) HH which maximizes vessel/lesion contrast (Biswas al. 2022 Ooi et al. 2021 Kumar et al. 2020 We perform min-max normalization (with e > 0 for stability):
(0)
(g)
h.30 miny ae Ih'w' we
7la)
E [0, 1].
(13)
h,w
maxp', vo 1n miny, W +
We then apply the Soft-FOV mask and Gaussian blur (std. Th, kernel radius rT):
Iulur Gon.ra (7(0) (SS) Hxw
(14)
The 2D discrete Fourier transform (DFT)
#-1W-1
F(u,e) v) = Σ
Istur (h, w) expl- 12 u + =r)]]
(15)
ソ=0
Let the radial distance from the spectrum center be
V(u
H²² 2
+
1 W
(16)
19

---

Published as a conference paper at ICLR 2026
A Butterworth high-pass filter of order and cutoff Do is
HBw (u, v)
(17)
2m
+ D(u,o)
Applying the filter in the frequency domain and inverting gives the raw high-pass response:
Fur v) HBw (u, v) F(u, v),
(18)
HhP(h, (h, w) = F - (Fur))( w)) ER"
(19)
To remove background and boundary responses, we binarize the Soft-FOV: Su, w) Sh,, w) > 3)
(20)
and obtain the final high-frequency map without additional normalization:
Hr h,w? 00
S(h, > 8,
Hhf
Hap @ Sth:
Hhf h,w
=
(21)
otherwise.
High-frequency Token Masking Our Vision Transformer (ViT) backbone partitions the input into P x P patches, each mapped to token (Dosovitskiy et al. 2020 To identify tokens enriched with higu-frequency content, we construct token mask from H GR" #
First, average Hhf within each non-overlapping patch:
P-1 P-1
Ag,
1
Σ
Σ
Huf uP
PP4j u == u= 1, 0,
W 1.
(22)
p2
2=
j=0
W
Then, rank {Au,u} and mark the top The % of patches to form M (0. P:
f1, Au, is in the top rar% of {Aur)s
Mu.v
{0,
(23)
otherwise.
The resulting high-frequency token mask M delineates token positions carrying abundant high- frequency information.
Table 5: Optimized hyperparameters for the Soft-FOV mask and high-frequencr (HF) extraction. Values were selected by maximizing the Dice score between the HF maps and available lesion/vessel ground-truth (GT) masks on held-out development data.
Notation
Description
Value
Do
Butterworth cutoff frequency
14.0470
Butterworth filter order
2
Tlov
Soft-FOV threshold
0.0869
da
Soft-FOV Gaussian sigma
10. 1332
Boundary cutoff
0,6701
an
Gaussian blur sigma
0.5185
Th
Gaussian blur radius
ThE
High-frequency token masking ratio (fraction)
0.25
Hyperparameter Search We optimized the Soft-FOV parameters {Tioe: J. }. the HF filtering pa- rameters (øh, The Do, n, B), and the HF token masking ratio The using the tree-structured Parzen estimator (TPE) [Watanabe] 2023). The search comprised 10,000 iterations on IDRiD (Porwal et al.) 2020) and FIVES (Jin et al. 2022) images that were excluded from both pretraining and down- stream evaluation. IDRiD provides ground truth (GT) masks for hemorrhages, hard exudates, cotton wool patches, and microaneurysms, while FIVES provides GT vessel masks. For each candidate configuration, generated HF maps and scored them against the corresponding GT masks using the Dice coefficient; the configuration achieving the highest mean Dice was selected. rebpot] the resulting hyperparameters.
20

---

Published as a conference paper at ICLR 2026
Qualitative Observations Filter Generalization
Although tuned on fundus data, the filter yields plausible HF representations on ImageNet images. As illustrated in Figure edges and fine textures in natural images are preserved in the extracted HF maps, indicating that the filter captures domain aogain-agnostic HF cues. This behavior supports the use of single parameterization for cross-doman comparisons and suggests favorable generalization beyond the fundus domain.
21

---

Published as a conference paper at ICLR 2026
A.3 PROOF OF THEOREM 1
Setup Let X = [x.]i=1 denote the sequence of patch tokens with visible/masked split Xy and X M and let Z be the encoder representation. Assume reconstruction model with isotropic Gaussian likelihood
Px(X Z) o X =ff(2))
and conditional factorization over masked patches given Z.
Step 1: Upper bounding the conditional mutual information by a conditional entropy By definition,
[[x:: XM Z) aM Z) ( Xv, 2). Since conditional entropy is nonnegative, ((XM M Xv. 2) 2 0, it follows that
T(Xv; XM 2) S ((MM Z).
Step 2: Evaluation of H(XM
Mr Z) under the Gaussian decoder Using the assumed likelihood and the conditional factorization over i E M.
H(XM Z) - Eo (x flog Z)] Bp. (m:12)/ logl-llogplee p -o Z)]
iEM
MD
log(2 (2TT +
| - 2:ll?
2
20
iEM
where M [M and D P2C denote, respectively, the number masked patches and their dimensionality.
Step 3: Identification of an affine relation to the MAE reconstruction loss Define the mean- squared reconstruction loss over masked patches
Lree
x2 - 2.11 Σ SE
M
Then
M
((X Z) a free + const,
a
20
0.
Conclusion
Combining the previous steps yields the affine upper bound I(Xv; () I Cree + const.
Since >> 0 is fixed for given 2², any minimizer of Lroe with respect to (0, Φ) is minimizer of the right-hand side, and hence
min Live 0.6
min T(Xv; XM Z).
8.6
Remarks (1) The result relies on the isotropic Gaussian likelihood with fixed variance; more gen- erally, any Exriavariance quadratic negative log- likelihood induces the same monotone relation. (2) If o² is learned, an explicit control (e.g., regularization or variance constraints) is required to keep the affine coefficient well-defined and to prevent trivial solutions.
22

---

Published as a conference paper at ICLR 2026
A.4 PROOF OF THEOREM 2
Setup Let Z. g be the context representation with ((; s, and let Z. - vv(xv) be the student representation. Assume that training enforces (1) mutual-information alignment between Z and Ze and (2) capacity matching so that the entropy of Zg does not exceed that of Ze by more than small margin.
Step 1: Propagation of the teacher bound to the visible part By the chain rule.
T(x; Zo) = T(Xv; 2 + T(XM:: Ze Xv).
whence
[[Xv; Z.) - I(X; Ze) E.
Step 2: Control of the student-context gap by alignment The identity
I(xv; Z.) - [(Xv; Z.) = I(Xv; Ze Z.) [(Xv; Za Zo) together with nonnegativity of mutual information yields
[[(vv Z.) I(Xv; Ze) max I(Xy: Ze Z.), I(Xv; Z, Ze) }.
Using [(A; B C) s H(B C)2
[1(v;; Z.) [(Xv; Za) S max HZe Zz), H(Z, Ze) } òalign.
Step 3: Capacity matching If. in addition, the entropies satisfy (H().) H(Z)) S 8cap. the asymmetry between the two conditional bounds above is uniformly controlled. Set 5 = max (Galigns )
Conclusion Combining the previous steps,
T(Xv; 2.) s T(Xv: Zo) + +
Under perfect alignment and exact capacity matching, i.e., 8 + 0, it follows that I(X;; Z.) I e.
Remarks Alignment refers the requirement that Z, and Ze be mutually predictable, equiva- lently that both HiZ, Zz) and H Z.) be small. Capacity matching refers to the requirement that the entropy of Zg be close to that of Zz, which prevents Z, from encoding additional information beyond what is present in Z
23

---

Published as a conference paper at ICLR 2026
A.5 IMPLEMENTATION DETAILS
Data preparation All models were pretrained on 256,044 fundus images collected from 39 pub- licly available datasets (see Appendix A.7 A.7 for details). All images are intensity-normalized to [0, 1], and rectangular inputs are zero-padded to preserve the aspect ratio. Images were resized to 224 224, and standardized using the ImageNet mean and standard deviation. Pre-training was performed for 100 epochs, and the last checkpoint was used for downstream tasks. Data augmentation was applied with Kornia (Riba al. 2020), including random rotation (=110), random resized cropping with a scale range of [0.2, 1.0]. random horizontal flipping, and color jittering with brightness, contrast, and saturation factors of 0.3.
VIT backbone with four register tokens, initialized from Di- nov2 weights (Oquab et al. 2023), and used patch size 14 to match the Dinov2 configuration. The decoder comprised eight Transformer layers. For latent features, ViT-based encoders used the [CLS] token for both Z and Ze. For HighFreqMI, the trainable encoder's 's latent representation Z was linearly projected into the latent space of the context features. Models were trained with input resolutions of 224 x 224.
Architecture The encoder
Optimization and training schedule We used Adam (Loshchilov Hutter 2017) with B1 0.9. B2 0.95, and a batch size of 768. The learning rate was set x 10 for the encoder/decoder and the HighFreqMI projection heads, following a cosine decay schedule after a 10-epoch warm- up. The encoder/decoder weight decay pretraining, the masking ratio was fixed at 80%. For the HighFreqMI objective, the model was trained with the MAE objective alone for the first 40 epochs, after which high-frequency alignment with the exponential moving-average (EMA) context encoder was enabled. The EMA momentum was cosine-scheduled from 0.994 to 1.0.
cosine-scheduled. During masked autoencoder (MAE)
24

---

Published as a conference paper at ICLR 2026
A.6 MODELS
Table 6 summarizes three pretraining paradigms considered in this work: (1) a CLIP-style image- text model (RET-CLIP), (2) masked image modeling (MIM) baselines (RETFound, UrFound), and (3) our RetMAE. RET-CLIP optimizes contrastive alignment between retinal images and associated text, whereas the MIM baselines pretrain VIT backbones by reconstructing masked patches. Ret- MAE retains the MAE backbone and masking scheme but augments the reconstruction objective with complementary mntual-information (MI) regularizers that suppress low-frequency redundancy and emphasize clinically salient high-frequency content, yielding frequency-balance retinal repre- sentations. All models use comparable ViT architectures and input resolutions; training follows each method's standard protocol without manual labels or architectural modifications.
Table 6: Summary of pretraining strategies. Comparison of model architecture, parameter count, input resolution, and use of text and model supervision across methods.
Method
Arch.
Params.
Res.
Text Sup.
Model Sup.
Contrastive Image Pre-Training
RET-CLIP (Du et al.
2024
ViT-B/16
86M
224
X
Masked Image Modeling
UrFound (Yu et al. 2024
ViT-B/16
86M
224
RETFound Zhou al.
2023
VIT-L/16
305M
224
x
X
Ours
RetMAE
ViT-B/14
86M
224
25

---

Published as a conference paper at ICLR 2026
A.7 DATASETS
Pretraining Datasets Table enumerates the 39 publicly available fundus datasets used for Ret- MAE pretraining together with their training-image counts. Spanning broad range of clinical con- ditions (from diabetic retinopathy to glaucoma). these sources collectively contribute 256,097 im- ages. To avoid leakage across sources, we performed image-level deduplication and retained only unique samples; consequently, our totals may differ slightly from those reported in the original re- leases. This large, diverse corpus enables learning robust retinal representations without manual labels.
Table 7: Number of pretraining images per dataset. 39 public fundus datasets were used.
Dataset
images
Dataset
# images
1000fundus (Cen et al.
2021
996
AGAR300
(Derwin 2020
26
ARIA [arnelll el al. 2008
143
AVRDB Akram el al. 2020)
99
Benitez Bemitez el al. 22011
1,406
BRSET Nakayama el al. 2023
16,265
Cataract Kaggle 2020
601
DeepDRiD Liu al. 2022
2,000
DiaRetDBT (Kauppi ct
2007
117
DiaRetDB2 Guo et
2024
28
DRI-2 2 (Pires
[2014
1,567
drimdb (Chakraborty 2024
194
DRD (Kaggic 20155
88,702
DRIONS-DB (Carmona ct al.
2008
110
FGADR (Zhou et al.
2021
1,828
FIRE Hernandez-Mattee Matas ct al.
2017
124
FUND-OCT (Hassan al.
2019
163
G1020 (Bajwa al. 2020
1,020
HEI-MED (Gancardo ct
2012
169
HRF (Budar et al. 2013
79
IOSTAR (Zhang el al. 2016
JICH (Takahashi et al. 2017
9,939
JustRAIG'S (Madadi et al. 2025
101,423
LAG (L cl al.
2019a
4,854
LES Orlando et al. 2020b)
22
MSHI (Jan al. 2023
500
ODIR (NTHDS-PKK 2019
6.996
OLA-DDK (Li al.
10
9,504
ORIGA (Zhang
2010
650
PALM (Fang et al.
2024a
1,174
PAPILA (Kovalyk et al. 2202
488
RC-RGB-MA (Das Dtsozoozggg et
2018
242
REFUGE (Orlando ct al.
2020a)
1,200
RetinalLesion (Wei ct al 2020)
1,593
RIDB (Abdul Salam
2020
100
RIGA (Almazroa el al. 2018
270
ROC NNemeejer 2009
100
SUSTech-SYSU Lin
2020
1,218
TREND (Popovic et
104
Total
256,044
Evaluation Datasets Table 8 summarizes the downstream classification benchmarks, all disjoint from the pretraining corpus. The in-domain datasets (e.g.. IDRiD, RFMiD, CHAKSU) are used to evaluate diabetic retinopathy (DR), age-related macular degeneration (AMD), and glaucoma (GL) detection, while APTOS is reserved exclusively for out-of-distribution (OOD) testing. For cach dataset, labels are mapped to task-specific binaries (e.g.. referable vs. non-referable DR) follow- ing the dataset's official taxonomy.
Table 8: Evaluation datasets. These sets were not used for pretraining.
Dataset
Lesions
Train
Val
Test
Classification
IDRiD
Porwal
2020
DR
408
102
RFMiD
Pachade ct al. 2022
DR, AMD
2,560
640
CHAKS
(Kumar ct al
202.3
GL
1,009
336
APTOS
Maggic & Dane
20T9
DR
3,394
Data Splits For each in-domain benchmark, adopt the official train/test split. Within the train- ing portion, 20% of the data is held out for validation and the remaining 80% is used for training; model selection and early stopping rely solely this validation set. We report test performance the official test split and assess generalization on the OOD set (APTOS). All methods use identical folds and preprocessing ensure a fair comparison.
26

---

Published as a conference paper at ICLR 2026
A.8 DATA SCALING, REDUNDANCY, AND FUNDUS STATISTICS
Figure plots downstream maero-AUROC on five supervised retinal benchmarks as a function of pretraining set size for a fixed encoder (ViT-Base/16) and fixed training budget. In this setting, RetMAE already surpasses much larger retinal foundation models with substantially fewer pretrain- ing images: with only ~1% of the corpus (=2.6k images), exceeds RETFound trained on 904k images; with 5% (< 12.8 images), it also outperforms UrFound trained on 187k images. The appar- ent plateau beyond 112.8k images might therefore seem surprising when viewed through the lens of natural-image foundation models.
Fundus as a low-entropy, highly redundant Domain As illustrated in Figure ! the vast major- ity of pixels in a fundus photograph belong relatively homogeneous background dominated by low-frequency content (retinal surface, illumination, and overall color tone), while clinically infor- mative structures occupy only small fraction of the field of view. Our high-frequency extraction suppresses this background and concentrates the signal into sparse, local high-frequency patterns perspective,
such as microaneurysmss hemorrhages, and exudates. From an coormation-theoeete this implies that the intrinsic entropy of the domain is relatively low and the dataset is highly redun- dant: many images share very similar low-frequency backgrounds, and the diagnostic information is concentrated in comparatively small set of high-frequency deviations. RetMAE is explicitly de- signed to focus on these high-frequency signals through HighFreqMI. Once the pretraining corpus is large enough to cover the diverse lesion and vessel patterns present in the population, additional images tend to be incrementally redundant under a fixed backbone and training schedule. In such regime, it is natural for downstream performance curves approach a ceiling with substantially fewer images than in unconstrained natural-image corpora.
Connection to subset selection, pruning, and scaling laws Our observations align with prior work showing that, in redundant datasets, carefully selected subsets can match or even outperform full-data training. Coreset and nubset-selection methods such as CRAIG demonstrate that models trained on representative subsets achieve performance comparable to full-data training while using significantly fewer examples and updates (Mirzasoleiman et al.) 2020). In large language models, data-pruning studies report that retaining only 30-50% of the pretraining corpus (ranked by simple quality metrics such as perplexity) can preserve or improve downstream performance compared to using all data (Marion et al. 2023). In computer vision, pruning strategies that prioritize images with higher intrinsic perceptual complexity (e.g., bits-per-pixel-buee based entropy scores) have been classification and segmentation tasks (Singh) [024). In medical imaging, deep active learning incluus-iclludin work on retinal fundus photographs find that actively selected subsets can reach or exceed the performance of models trained on all available images while greatly reducing labeling and training cost (Wang et al.) 2024a Paul et al., 2022). Taken together, these results support the view that, in a low-entropy, highly structured domain like fundus imaging, the effective number of distinct, task-relevant patterns is much smaller than the raw image count. Once these patterns are well covered, simply adding more similar images yields diminishing returns under a fixed-capacity encoder and fixed compute. This perspective is also consistent with neural scaling laws, which model performance as a power-law function model size. data, and compute (Hestness et al. 2019 Kaplan et al.) 2020 Hernandez et al. [22 Hoffmann ct al. 2022 Dehghani et al. 2023): continued gains typically require joint scaling of model capacity, data diversity, and the number of optimization steps. Our experiments intentionally fix the backbone and training budget to isolate the effect of our MI-based objective on sample efficiency; exploring joint capacity- cota-cpputtee scaling for RetMAE on more heterogeneous multi-center retinal and non-retinal datasets is an important direction for future work.
shown to match full-dataset performance
27

---

Published as a conference paper at ICLR 2026
A.9 ADDITIONAL RESULTS
Ablation on loss components All AUROC/AUPRC
Table 9: Ablation of loss components.
values in Table | are macro-averages across five bench- marks. The reconstruction-only baseline weakest. In- troducing Lumi yields large gains (AUROC +0.094, AUPRC +0.128), indicating that recovering high- frequency cues is the primary lever. Crucially, Lumi complementary: when added to the auxiliary set- ting. it provides further improvements (AUROC +0.018, AUPRC +0.050). Collectively, these results show that HighFreqMI increases high-frequency suffi- ciency and effectively improves retinal MAE variants.
Cree
Laux
Lhmi
AUROC AUPRC
0.685
0.486
0.779
0.614
0.923
0.799
0.941
0.849
Loss-weight ablation To assess the sensitivity of RetMAE to the relative weighting of the reconstruction, HighFreqMI, and auxiliary losses, we conducted a loss-weight ablation in which we varied one coefficient at time while fixing Arec = and disabling the remaining non-varied term (i.e., setting its coefficient to 0). For each configuration, we pre- trained the model on fundus images and evaluated the frozen encoder via linear probing. reporting the macro-averaged AU- ROC across five benchmarks (IDRiD, RFMiD-DR, RFMiD- AMD, CHAKSU, and APTOS), Table IO summarizes the sults. In both cases, performance is relatively stable across broad range of weights, with mild optimum around Aumi 0.1 for the HighFreqMI term and similarly flat region for auxiliary loss. Motivated by these observations, we adopt Aree = 1, Anmi 0.1, and λaux = 0.01 as our default configu- ration in the main experiments, which provides a robust trade- off between reconstruction, high-frequency regularization, and auxiliary alignment.
Table 10: Sensitivity of RetMAE to the HighFreq and auxiliary loss weights.
Abni
Aaux
AUROC
0
0.685
1.0
0
0.738
0.1
0.779
0.01
0
0.732
0
1.0
0.927
0
0.1
0.926
0
0.01
0.932
0.1
0.01
0.941
Additional AUPRC results Table 11 reports linear probing performance in terms of AUPRC across same five benchmarks. RetMAF attains the best macro-average AUPRC (0.849) among all methods with auxiliary losses, and achieves the top AUPRC on APTOS (0.960), further con- firming its strong OOD generalization. Compared to image-only MAE variants (MAE, RETFound, UrFound), RetMAE consistently improves AUPRC, indicating that MI-based emphasis high- frequency retinal structure yields more discriminative features. Moreover, with auxiliary losses, RetMAE slightly surpasses RET-CLIP in macro-average AUPRC, reinforcing conclusion that explicit high-fie sigh-freyeency alignment, rather than language supervision alone, is the principal driver of the gains observed in both AUROC and AUPRC.
Table 11: Linear probing performance (AUPRC). Columns marked are out-of-distribution test sets. AVG the macro-average across datasets. Values in light gray denote evaluation datasets seen during pretraining. Auxiliary loss: / indicates the of auxiliary signals beyond images (c.g.. text guidance or a retina-informed off-the-shelf encoder); indicates image-only self-supervised pretraining.
Method
Auxiliary loss
IDRiD
REMiD (DR)
REMID (AMD)
CHAKSU
APTOS'
AVG
MAE
X
0.874
0.396
0.191
0.116
0.855
0.486
RETFound
X
0.878
0.515
0.140
0.142
0.732
0.481
RetMAE
0.916
0.679
0.381
0.194
0.899
0.614
UrFound
0.919
0.870
0.601
0.243
0.936
0.714
MAE
0.949
0.850
0.588
0.686
0.921
0.799
RET-CLIP
0.957
0.900
0.605
0.797
0.952
0.842
ReMMAR
0.959
0.857
0.759
0.711
0.960
0.849
28

---

Published as a conference paper at ICLR 2026
Multi-disease evaluation To assess perfor- mance in more clinically realistic, multi-disease settings, we additionally evaluate RetMAE on two multi-disease benchmarks, FIVES (Jin et al. 2022) and RFMiD2 (Panchal et al.) (023). FIVES comprises four diagnostic categories (AMD, DR. glaucoma, and normal), which already span le- sions with distinct spatial and frequency charac- teristies (e.g., microaneurysmss, hemorrhags, and exudates in DR versus oetic-nerve cupping and nerve-fiber-layer defects in glaucoma), RFMiD2 is more challenging: it provides over 40 expert-defined retinal disease labels, including vascu- lar occlusions, macular edema, neovascularization, optic-nerve anomalies, inflammatory conditions, myopic degeneration, tessellation, pigment-epithelium changes, and others, and individual images often carry multiple labels simultaneously. This multi-label, multi-disease structure more faithfully reflects real clinical scenarios, where overlapping disease signatures are common rather than iso- lated.
Figure 7: Linear probing AUROC multi- disease fundus benchmarks.
Method
FIVES
RFMiD2
AVG
RETFound RET-CLIP ReMMA 0.922
0.837
0.806
0.822
0.943
0.808
0.876
0.884
0.903
Table summarizes linear probing AUROC on these benchmarks. RetMAE achieves the best macro- average performance (AVG 0.903), improving over the self-supervised RETFound baseline from 0.837 to 0.922 on FIVES and from 0.806 to 0.884 on RFMiD2 (AVG 0. 0.822 - 0.903). On FIVES, RET-CLIP attains the highest AUROC (0.943), with RetMAE achieving competitive second-best score (0.922); on RFMiD2, however, RetMAE clearly outperforms both RETFound and RET-CLIP (0.884 vs. 0.808). Importantly, ReMMA also surpasses NET-CLIP-whiee leverages language su- pervision-ing terms of the overall average AUROC (0.903 vs. 0.876). These results indicate that our dopein--pecific high-frequency regularizer is beneficial not only for comparatively simple binary classification tasks, but also for multi-disease, multi-label settings that better capture the clinical complexity and practical value of real-worke fundus imaging.
29

---

Published as a conference paper at ICLR 2026
Pretraining-efficiene per benchmark Figure shows per-dataset AUROC function of pre- training size. We construct nested subsets at {1. 5, 10, 25, 50, 75, 100%% of the full pretraining set each subset, and eval-
(~ {2.6, 12.8, 25.6, 64, 128, 192, 256 k images), pretrain RetMAB uate with a linear probe on IDRiD, RFMiD (DR/AMD), CHAKSU, and APTOS. RetMAFrencin is highly data-efficient: with only 1% of data attains macro-average AUROC of 0.741, exceeding RETFound (0.690); with 5% it reaches 0.925, surpassing UrFound (0. (0.855). Most gains accrue by 10-25% (0.940 and 0.938), with diminishing returns thereafter; the best macro-average is 0.941 at 100%. Scaling behavior is task-dependent: RFMiD-ANE -MD improves steadily with more data (0.836-x0.980) RFMiD-DD D-DDDDDRRDD peaks near 50% (0.966), while CHAKSU and the OOD APTOS largely saturate by 10% (0.932 and 0.954) and vary only slightly beyond that. Per-dataset highlights include IDRiD 0.717 - 0.910 from 1% to 100%: RFMiD-DR 0.806 + 0.966 (peak at 50%); RFMiD-AMD 0.836 - 0.980; CHAKSU 0.693 + 0.932 by 10%; and APTOS 0.655 - 0.954 by 10%. These trends align with our frequency-orient established, additional fundus images primarily add low-frequency background redundancy, yield- ing modest gains, whereas tasks driven by richer high-frequency structure (e.g., AMD) benefit more from scale.
view: once frequency frequency-balanced features are
ReMMAE
RETFound (904k)
UrFound (187k)
IDRiD
RFMID (DR)
RFMID (AMD)
0,953
0.53
7.74
0.736
0.6
0.6
0.6
0.4
2.6
12.8
25.6
04
128
192
256
0.4
2.6
12.8
25.6
64
128
192
256
0.4
2.6
12.3
25.6
64
128
192
256
CHAKSU
APTOS
Datasct Average
0.923
AUROC
0.8
0.706
0,690
0.6
0.6
01464
0.4
2.6
12.8
25.6
64
128
192
250
0.4
2.6
12.8
25.6
64
125
192
256
0.4
2.6
12.8
25.6
64
128
192
256
Number Pretraining Data
Figure 8: Pretraining data efficiency of RetMAE. For all datasets considered, AUROC improves as pretraining size increases. RetMAE achieves the target performance with far fewer images than RET'oound and UrFound, and consistently outperforms MAE-based retinal baselines.
30

---

Published as a conference paper at ICLR 2026
CKA and linear-probe performance across MAE token subsets Table 12 reports, for each sub- set, CKA computed with respect to the full input embedding and per-dataset linear-probe AUROC on five benchmarks (IDRiD, RFMiD-DR, RFMiD-AMD, CHAKSU, APTOS).
We observe three regularities. (1) 25% masked (which retains 75% of tokens) closely matches full in AUROC while achieving near-unity CKA (0.996), suggesting substantial redundancy in the MAE representations. (2) low-freq. only attains high alignment (CKA 0.990) yet weak diagnostic sig- nal-yielding the column minima IDRiD (0.662), REMiD-DR (0.667), and APTOS (0.718)-and remaining below full on RFMiD-AMD (0.778 vs. 0.793). (3) high-freq. only, which keeps only 25% of tokens, shows the lowest aligmment full (CKA 0.164) yet the strongest AUROC on IDRiD (0.737), RFMiD-DR (0.792). RFMiD-AMD (0.867), and CHAKSU (0.439), while remaining com- petitive on the out-of-distributioo APTOS set (0.802 vs. 0.812 for full).
Overall, the results indicate that MAE representations emphasize low-frequency background struc- ture (high CKA) with limited diagnostic utility, whereas small subset of kens-despile low alignment to the full-input umbedding-ccpuuee clinically salient information and yields superior linear-probe performance.
Table 12: CKA and lincar-probing performance across token subsets. figh-frequency tokens yield the strongest diagnostic performance despite low representational alignment with the full input. The best value in each column shown in blue and the lowest in red.
Subset
CKA
IDRiD
RFMiD (DR)
RFMID (AMD)
CHAKSU
APTOS
full
Baseline
0.726
0.721
0.793
0.371
0.812
25% masked low-freq. only
0.996
0.727
0.725
0.794
0.380
0.806
0.990
0.662
0.667
0.778
0.379
0.718
75% masked
0.890
0.668
0.677
0.768
0.396
0.725
high-freq. only
0.792
0.867
0.802
31

---

Published as a conference paper at ICLR 2026
High-Frequency decodability of patch tokens and class-to-attention per image For each pre- trained encoder, we regressed the patch-level HF targets (Eq. 22) from frozen patch tokens using ridge regression and computed per-image R², summarizing both the distribution and pooled (over- all) R² across images. The per-image distributions are visualized in Fig. 9a (left panel of Fig. ( RetMAE exhibits uniformly high decodability of HF signal: overall R 0.9909, with mean per- image 0.9896 + 0.0032 and a very tight range [0.9854, 0.9925]; the corresponding significance test strongly rejects Ho: R22 0 (p 1.1x10 16). UrFound is distant second (overall Rt² 0.8353; mean 0.8185 + 0.0671; range [0.7219, 0.8772); p 6.3x 10-?) )), showing both a lower central ten- dency and broader dispersion than RetMAE. RETFound and RET-CLIP yield substantially lower and statistically non-significant overall I?2 (0.6620 and 0.6329), consistent with weaker linear de- codability of HF content from their patch embeddings. Quantitatively, RetMAE's advantage over the next best model amounts to AR R 0.156 at the overall level, while its markedly narrower per-image spread indicates that HF information is recoverable consistently across images rather than being driven by subset of casy cases. Taken together, these outcomes corroborate the main-text claim that RetMAE preserves HF information across depth and images, providing linearly decod- able access to diagnostically salient structure.
We repeated the analysis using class-to-patch attention features to assess HF decodability from attention-derived representations. The corresponding per-image R² distributions shown in Fig. 9b (right panel of Fig. 9). Absolute R² values are lower-a natural consequence of what is being regressed: class-to-patcho attention provides allocation weights rather than feature vectors, is spatially smoothed by softmax and head averaging, and aggregates cues not specific to HF con- tent-yet the ranking remains consistent. RetMAE attains the highest overall R (0.3868; mean per-image 0.3243 = 0.2296; range [0.0728, 0.6244);
1.1 10x100),, followed by RET-CLIP (0.2872), UrFound (0.2760), and RETFound (0. (40)); all four are significant at p < 0.05. Notably, baseline models exhibit broader and occasionally negative per-image Rt values (e.g.. minima below zero for RET-CLIP and RETFound), indicating poor linear recoverability of HF targets from their class-attentional structure, whereas RetMAE's distribution is shifted upward with positive lower bound. These trends align with the frequeney-oriented analyses in Sec. 6.3] the PCA visualization of class-to-patch attention (Fig- shows sharper, anatomy-consisten chromatic separation for Ret- MAE, providing qualitative counterpart to the elevated HF decodability observed here. Together with the patch-token results above, this supports the view that RetMAE learns frequency-aware, diagnostically informative embeddings whose HF components remain linearly accessible.
Par-inage R distributionn
Por Por-image ge distribution
1.0
0.5
0.0
0.6
-0.5
RE RFund
UiFoand
RET-CLiP
ReMAA
RETFomd
UrFoadd
RET-CUP
ReMAB
Mode]
Model
(a) Patch tokens.
(b) Class-to-attention features.
Figure 9: Per-image R² violin plots for frequency decodability. Left: patch-token analysis; right:
class-to-ane toas-to-attention analysis.
32

---

Published as a conference paper at ICLR 2026
Computational complexity We quantify the additional computational cost introduced by our high-frequency regularization. All measurements conducted on a Tesla V100 GPU using a ViT- Base/16 backbone with input resolution 224 x 224, batch size 16, and averaged over 100 itera- tions, Inference time is measured using CUDA events (torch. eerrchhcuda...... Event) for accurate GPU- side timing without CPU-GPU synchronization overhead, and floating point operations per second (FLOPs) are estimated with fvcore's PlopCountAnalysi for each component. Our method introduces two additional components top of the baseline MAF forward pass: (1) high-frequency component extraction and encoding, where high-frequency patches are processed by an EMA en- coder, and (2) the HighFreqMI loss, which estimates mutual information between the main encoder latent and the fighffrequency context encoder latent via a lightweight critic. Table summarizes the resulting overhead.
As shown in Table 13 the overall overhead minimal: the proposed regularization adds less than 4% to the baseline inference time (1.88 ms over 52.48 ms) and less than 0.25% to the FLOPs. This effi- ciency is primarily due to the fact that the high-frequency encoder processes only a subset of patches (high-frequency regions), and the MINF-based mutual information estimator relies lightweight critic network with small number of parameters. Consequently, our high-frequency regularization yields substantial gains in representation quality for fundus imaging tasks while remaining highly practical for real-world deployment.
Table 13: Computational overhead of the proposed high-frequency reg- ularization on V with input resolution 224 Inference time measured per forward pass on a Tesla V100 GPU. Percentages are reported relative to the baseline MAE
Component
Inference time (ms)
FLOPs (GFLOPs)
Baseline
52.48 (100.00%)
69.87 (100.00%)
Higg-freyuency component
1.56 (2.97%)
0.105 (0.15%)
HighFreqMI loss Total overhead
0.32 (0.61 (%)
0.050 (0.07%)
1.88 (3.58%)
0.156 (0.22%)
33

---

Published as a conference paper at ICLR 2026
Large lesions and retinal detachment We additionally visualize class-to-patch attention for le- sions that are not purely high-frequency, including retinal traction detachment and large preretinal hemorrhages (Fig. 10p In the cases of retinal detachment in panels (a)-b) and a large preretinal hemorrhage in panel (c), the PCA-projected class-token attention remains well aligned with the le- sion, with strong responses along the detachment margins and hemorrhage boundaries. This behav- ior is consistent with our frequency-basee based view: even when the pathological region covers broad area, the transition zones at the lesion boundary still correspond to high-frequency structure, and RetMAE continues emphasize these regions. the extreme case in panel (d), where pathology occupies almost the entire field-of-vier and thus behaves effectively as a low-frequency signal, the class token no longer attends uniformly across the full lesion interior; however, it still concentrates on boundaries and locations where the fundus signal changes abruptly. Clinically, lesion discrimina- tion fundus photography largely driven by such sharp intensity and texture changes relative to the surrounding background neighboring structures, accurately attending these boundaries is more important than uniformly covering the entire lesion area.
Retinal traction detachment
Preretinal hemorrhage
Input
High-Irequenn
Visualization
PCA
(b)
Figure 10: Attention on large lesions and retinal detachment. PCA- PoD-ppooeeted class-token attention maps for (a)-(b) retinal traction detachment, (c) a large preretinal hemorrhage, and (d) an eye with near-global pathology. In (a)-(c), the class-token attention remains well aligned with the lesion, with strong responses along the detachment margins and hemorrhage boundaries. In the extreme case in (d), where pathology occupies almost the entire field of view and thus behaves effectively as low-frequency signal, the class token does not attend uniformly across the lesion interior, but still concentrates on boundaries and locations where the fundus signal changes abruptly.
34

---

Published as a conference paper at ICLR 2026
Additional PCA visualizations of class-to-patch attention the PCA visualization of ftasstto-pachh attention.
We present additional examples of
Input
High-Freq.
RetMAE RET-CLIP RETFound UrFound
Figure 11: Example 1 of PCA visualization of class-to-patch attention.
35

---

Published as a conference paper at ICLR 2026
Input
High-Freq.
RetMAE
RET-CLIP RETFound
UrFound
Figure 12: Example 2 of PCA visualization of tass-tto-pacch attention.
36

---

Published as a conference paper at ICLR 2026
Input
High-Freq.
RetMAE
RET-CLIP RETFound
UrFound
Figure 13: Example 3 of PCA visualization of class-to-patch attention.
37

---

Published as a conference paper at ICLR 2026
Input
High-Freq.
RetMAE
RET-CLIP RETFound
UrFound
Figure 14: Example 4 of PCA visualization of class-to-patch attention.
38


Reviews:
[{"review_id": "kzlLGsluKs", "reviewer": "Reviewer_Rstj", "summary": "This paper proposes a frequency-aware masked autoencoder (MAE) for unsupervised pretraining on retinal images, called RetMAE. This is accomplished by including a high-frequency regularization term in the objective function that reduces low-frequency redundancy. The authors main claim is that the diagnostic information in retina images are encoded in high frequencies, and thus better representing these areas yields better downstream accuracy. There is a section on evaluating the frequency bias in MAE representations applied to fundus images in which the paper presents centered kernel alignment (CKA) and linear probing as evidence. The experimental section utilizes five publicly available datasets and compares against two other MAE-based approaches as well as a vision-language baseline. \n\nAlthough the paper doesn't contribute a significant new algorithmic framework, its approach in providing frequency-based context latents to guide the representation learning paradigm for applications in which frequency bias is an impediment could be a valuable contribution. There are, however, a few areas of both theory and practice that need clarification. \n\nOn theory:\n\n1- What is the purpose of using a variational autoencoder with a fixed variance Gaussian mapping? From theorem 1, it reduces the reconstruction error to minimizing the conditional in eq 2. However, it is not clear if this enforced constraint is warranted. Is this constraint enforcing any aspect of the regularization framework?\n2- The choice of using $\\mathcal L_{MINE}$ as the context-alignment training objective is not quite clear. In other words, why does estimating MINE maximize the conditional?\n\nOn Application:\n1- Does this framework extend beyond retinal fundus images? Could other factors than frequency bias be incorporated in the regularization objective?\n2- How much computational complexity is added to the problem by incorporating the proposed high-frequency regularization?\n3- How does the pretrained encoder fare in a formal classification tasks rather than the employed linear probing?\n\nAdditional suggestions:\n\n1) use the figure in appendix A instead of Figure 1 in the main paper. The figure from the appendix better justifies the frequency bias of retinal fundus images as compared to natural images, e.g. ImageNet.\n2) CKA and its relevance to the claimed frequency bias should be explained more clearly.", "questions": "Provided in the summary.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The approach in utilizing a bias term as regularization to improve representation learning in certain application is interesting.\nThis approach could be potentially significant for applications that are not based on natural images.", "weaknesses": "Better discussion is needed to connect the theoretical aspect of the work (MI) with the practical tools utilized (MINE estimation).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3LIzLwLGvy", "reviewer": "Reviewer_d9g5", "summary": "The presented work postulates that there is a discrepancy between the most salient visual features in medical images and those learned by current representation learning techniques. In particular, the authors claim that the prevalent masked autoencoder disregards high-frequency features related to retinal pathology in color fundus photographs. To demonstrate this, they rank image patches by the amount of high-frequency content. Subsequently, they show that patches with reduced high-frequency content shape latent space structure while providing substantially less information for downstream tasks. In response to this observation, the authors propose RetMAE, an extension of the loss function of the established masked autoencoder. RetMAE regularizes the latent space by maximizing its mutual information with embeddings of patches with increased high-frequency content. In experiments using four ophthalmological datasets, the authors show that RetMAE outperforms various baselines that rely on a basic masked autoencoder. This effect persists when auxiliary signals, such as text or a pre-trained vision encoder, are included as learning signal, albeit in a diminished capacity.", "questions": "- At the moment, the performance of the proposed method is only quantified via linear probing using the latent representations. I believe that full fine-tuning should also be conducted considering that the ultimate downstream performance matters most in applied domains such as medical image analysis.\n\n- Considering that most ophthalmological foundation models make their training code and weights public, I believe that the authors should strongly consider doing the same.\n\n- Several core concepts of the paper are only very briefly introduced or require consultation of the supplementary material. I suggest that the section on the interpretation of the masked image modeling objective through the lens of a Lagragian is slightly extended so that it can be understood without consulting previous works by Huang et al. and Tishby and Zaslasky. Similarly, the frequency score calculation should be briefly introduced in the main manuscript considering its vital importance, instead of only being introduced in the supplementary material.\n\n- Additionally, I struggled with the notation on several occasions. Already in the very first mathematical paragraph, the variable $N$ is overloaded, $D$ not introduced, and mutual information $I$ is not defined. Later, the use of $X$ varies to signal whether it denotes input or decoded image tokens. The authors should carefully parse the manuscript once again, aiming to improve the clarity of its mathematical passages.\n\n- On a minor note, the acronym CKA is not introduced at its first appearance in the introduction section.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The work’s main motivation that current pre-training paradigms for vision transformers result in suboptimal feature extractors when applied to medical images is very interesting. The authors convincingly support this hypothesis in a set of initial experiments (Figure 1, Section 4, Supplementary Material A1), showing that salient image features differ in natural and medical images, and that masked image modeling has an inductive bias towards low-frequency features.\n\n- The provided solution to this problem is theoretically well founded and experimentally shown to improve performance. As such, it has the potential to benefit the large scientific community in the field of medical image analysis.\n\n-  The clinical application of ophthalmological image analysis is very well selected. Many retinal diseases manifest as small pathologies, resulting in high-frequency image features. Furthermore, there are several prominent works that have used large-scale pre-training of masked autoencoders to derive ophthalmological foundation models.\n\n- The paper is clearly structured, nicely illustrated and generally well written. Additionally, the authors include extensive supplementary material that provides in-depth technical detail about the method and experimental setup.", "weaknesses": "- The motivating hypothesis that medical images contain more salient high-frequency image features is only explored for color fundus photographs. The importance and reach of the work would substantially increase if similar findings were shown for other types of data. Similarly, the efficiency of the proposed solution is only demonstrated for color fundus photographs so that it is unclear whether the proposed method seamlessly translates to other settings or requires extensive hyperparameter tuning for both the extraction of high-frequency information and the loss weighting.\n\n- The proposed solution is highly complex. In particular, it relies on estimation of mutual information via an Donsker-Varadhan estimator, which is known to numerically unstable (Belghazi, Mohamed Ishmael, et al. \"Mutual information neural estimation.\" International conference on machine learning. PMLR, 2018). I could envision that conceptionally much simpler solutions exist that emphasize high-frequency features. For example, one could adjust the masking scheme to prioritize patches with increased high-frequency content (similar to Xie, Jiahao, et al. \"Masked Frequency Modeling for Self-Supervised Visual Pre-Training.\" The Eleventh International Conference on Learning Representations). Alternatively, one could provide the high-pass-filtered image as additional input (Wang, Wenxuan, et al. \"Fremim: Fourier transform meets masked image modeling for medical image segmentation.\" Proceedings of the IEEE/CVF winter conference on applications of computer vision. 2024). The authors should discuss the aforementioned works in more detail and include them as baselines.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "posXvg3iyF", "reviewer": "Reviewer_ULHB", "summary": "This paper focuses on the high-frequency information in fundus images that is clinically relevant. The authors introduced an information-theoretic auxiliary supervision into the MAE pretraining paradigm to guide the encoder toward clinically important regions, without requiring architectural modifications. The overall logic—from problem identification to the proposed solution and experimental validation—is clear and well-structured.", "questions": "1. This paper introduces an additional high-frequency contextual feature constraint into the latent space of MAE. Some previous studies have imposed constraints directly on the masking strategy (e.g., image-entropy-based masking). I encourage the authors to discuss this line of work to further highlight the value of their proposed method.\n\n2. According to Figure 3, the performance of RetMAE appears to saturate after pretraining on approximately 12.8k images. Increasing the data size beyond this point seems to yield no significant improvement. The authors attribute this to saturation of model capacity and computation, which is a reasonable explanation. However, given that 12.8k is far smaller than the typical data scale used for foundation model pretraining—and that the employed encoder architecture has been shown in other domains to effectively utilize much larger datasets—this phenomenon remains concerning. The authors should provide a more convincing explanation for this observation.\n\n3. Not all retinal lesions exhibit high-frequency characteristics—for example, large hemorrhages or retinal detachments. I would like to see visualizations of such cases to better understand how the proposed method behaves under these conditions.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The paper presents a clear and coherent logical flow from motivation and problem formulation to analysis, proposed solution, and experimental validation, making it easy to follow and understand.\n\n2. It offers new and interesting insights into model training for ophthalmic applications.\n\n3. The theoretical derivation of the proposed method is sound, and the approach itself is easy to reproduce.", "weaknesses": "1. Limited novelty. The inspiration of this work appears to be directly derived from Huang, Tao, et al. “Learning Mask Invariant Mutual Information for Masked Image Modeling.” arXiv preprint arXiv:2502.19718 (2025). Although this paper is cited, I would still like the authors to explicitly clarify which parts of the current method are independently proposed.\n\n2. I have concerns regarding the generalizability of the proposed approach.\n(i) The low-pass filtering property originates from the ViT architecture itself, and this phenomenon is not unique to MAE.\n(ii) The application scenario in this work is limited to color fundus photography. From the perspective of developing a robust retinal foundation model, MAE is not the only viable paradigm even within the image modality. For example, VisionFM, which follows the iBOT framework, also builds a powerful image encoder. From the standpoint of understanding the mechanism of MAE, this paper does not provide new insights. The authors need to further elaborate on the substantive contribution of their work.\n\n3. The performance evidence is limited. The chosen downstream tasks are relatively few and of low difficulty (e.g., binary classification of DR, glaucoma, and AMD). Considering that the motivation of this work focuses on clinically relevant high-frequency lesions, the authors are encouraged to validate their method on more challenging tasks to substantiate its claimed clinical value.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "atZDOUiv5C", "reviewer": "Reviewer_xA7m", "summary": "This paper proposes a frequency-balanced masked autoencoder framework (RetMAE) that enhances retinal representation learning by introducing a high-frequency mutual information regularizer to emphasize clinically critical high-frequency structures while suppressing redundant low-frequency content.", "questions": "Were all evaluations in Table 1 conducted on the standard MAE model?", "limitations": "", "rating": 2, "confidence": 5, "soundness": 1, "presentation": 3, "contribution": 2, "strengths": "This paper does present a clear motivation that high-frequency structures are clinically important in retinal imaging, and it attempts to incorporate this insight through a mutual-information-based regularizer.  And the presentation is good.", "weaknesses": "(1) According to Figure 2, the loss L_hmi proposed in this paper and the loss L_rec in the original MAE appear to optimize the latent feature in two fundamentally different directions. Specifically, L_rec aims to optimize 𝑧 such that it can reconstruct the full Image from Image_mask1. In contrast, L_hmi optimizes 𝑧 to enable the reconstruction (or transformation) of Image_mask2 from Image_mask1, where mask2 is generated through the high-frequency masking strategy proposed by the authors. This is clearly contradictory and constitutes the most significant issue.\n(2) Were all evaluations in Table 1 conducted on the standard MAE model? The results shown in this table seem to indicate that the standard MAE already has a strong ability to represent high-frequency information, which contradicts the authors’ claim of “under-encoding high-frequency diagnostic structures.” For example, in the T_low row of Table 1, even after masking 25% of high-frequency information, the CKA value remains as high as 0.990, indicating that the model still retains stable reconstruction capability for high-frequency components. Conversely, in the T_high row, when high-frequency information is used as input, the model shows low CKA because it cannot reconstruct the full image, which is expected. However, the AUROC increases, demonstrating that high-frequency information is highly discriminative; when low-frequency, lesion-irrelevant content is removed, the model’s prediction accuracy improves. Therefore, the evidence presented in Table 1 may actually support the importance of high-frequency features rather than demonstrating the insufficiency of MAE in encoding them. I believe the authors have not conducted a sufficiently thorough investigation in this aspect.\n(3) As highlighted in Comment (1), there is a potential conflict between the two loss terms. The authors should explicitly report the training weights assigned to each loss or provide a sensitivity analysis (e.g., a performance graph under different loss weight configurations) to demonstrate the impact of the loss balance on model performance.\n(4) The innovation of the proposed method appears to be limited, as it essentially adds a frequency-based loss on top of MAE, while the use of high-frequency representations to capture lesion-related features in retinal images has already been explored in numerous prior studies.\n\nIf the authors can provide a clear theoretical justification or empirical evidence resolving the contradictions I raised—particularly regarding the compatibility of the two loss objectives and the interpretation of Table 1—I am willing to reconsider my rating and increase my score accordingly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)