NeurIPS 2025

Ditch the Denoiser: Emergence of Noise Robustness in
Self-Supervised Learning from Data Curriculum
Wenquan Lu1
Jiaqi Zhang1
Hugues Van Assel2
Randall Balestriero1
1Brown University
2Genentech
{wenquan_lu, jiaqi_zhang6, randall_balestriero}@brown.edu
van_assel.hugues@gene.com
Abstract
Self-Supervised Learning (SSL) has become a powerful solution to extract rich
representations from unlabeled data. Yet, SSL research is mostly focused on
clean, curated and high-quality datasets. As a result, applying SSL on noisy data
remains a challenge, despite being crucial to applications such as astrophysics,
medical imaging, geophysics or finance. In this work, we present a fully self-
supervised framework that enables noise-robust representation learning without
requiring a denoiser at inference or downstream fine-tuning. Our method first
trains an SSL denoiser on noisy data, then uses it to construct a denoised-to-
noisy data curriculum (i.e., training first on denoised, then noisy samples) for
pretraining a SSL backbone (e.g., DINOv2), combined with a teacher-guided
regularization that anchors noisy embeddings to their denoised counterparts. This
process encourages the model to internalize noise robustness. Notably, the denoiser
can be discarded after pretraining, simplifying deployment. On ImageNet-1k with
ViT-B under extreme Gaussian noise (σ = 255, SNR = 0.72 dB), our method
improves linear probing accuracy by 4.8% over DINOv2, demonstrating that
denoiser-free robustness can emerge from noise-aware pretraining. The code is
available at https://github.com/wenquanlu/noisy_dinov2.
1
Introduction
Self-supervised methods like DINOv2 [31, 7, 16, 9] have demonstrated remarkable success by
leveraging unlabeled data to learn visual representations able to solve many tasks in zero or few
shot manners [3]. However, the performances of those methods heavily rely on the availability
of clean and high-quality datasets. As a result, some prior works have focused on building data
curation ...