ICML 2025

One-Step Generalization Ratio Guided Optimization for Domain Generalization
Sumin Cho * 1 Dongwon Kim * 1 Kwangsu Kim 1
Abstract
Domain Generalization (DG) aims to train models
that generalize to unseen target domains but often
overfit to domain-specific features, known as un-
desired correlations. Gradient-based DG methods
typically guide gradients in a dominant direction
but often inadvertently reinforce spurious corre-
lations. Recent work has employed dropout to
regularize overconfident parameters, but has not
explicitly adjusted gradient alignment or ensured
balanced parameter updates. We propose GENIE
(Generalization-ENhancing Iterative Equalizer), a
novel optimizer that leverages the One-Step Gen-
eralization Ratio (OSGR) to quantify each param-
eter’s contribution to loss reduction and assess
gradient alignment. By dynamically equalizing
OSGR via a preconditioning factor, GENIE pre-
vents a small subset of parameters from domi-
nating optimization, thereby promoting domain-
invariant feature learning. Theoretically, GENIE
balances convergence contribution and gradient
alignment among parameters, achieving higher
OSGR while retaining SGD’s convergence rate.
Empirically, it outperforms existing optimizers
and enhances performance when integrated with
various DG and single-DG methods.
1. Introduction
Deep neural networks (DNNs) achieve high accuracy when
training and test data share a similar distribution. How-
ever, in real-world applications, data distributions often shift,
causing performance degradation(Muandet et al., 2013). Do-
main Generalization (DG) addresses this issue by training
models to generalize to out-of-distribution data from un-
seen domains. The main challenge is to prevent overfitting
*Equal contribution
1Department of Computer Science and
Engineering, University of Sungkyunkwan, Suwon, Korea.
Correspondence
to:
Sumin
Cho
<jsm0707@skku.edu>,
Dongwon
Kim
<kdwaha@skku.edu>,
Kwangsu
Kim
<kim.kwangsu@skku.edu>.
Proceedings of the 42 nd Internat...