ICML 2025

Curvature Enhanced Data Augmentation for Regression
Ilya Kaufman 1 Omri Azencot 1
Abstract
Deep learning models with a large number of pa-
rameters, often referred to as over-parameterized
models, have achieved exceptional performance
across various tasks.
Despite concerns about
overfitting, these models frequently generalize
well to unseen data, thanks to effective regular-
ization techniques, with data augmentation be-
ing among the most widely used. While data
augmentation has shown great success in clas-
sification tasks using label-preserving transfor-
mations, its application in regression problems
has received less attention. Recently, a novel
manifold learning approach for generating syn-
thetic data was proposed, utilizing a first-order
approximation of the data manifold. Building on
this foundation, we present a theoretical frame-
work and practical tools for approximating and
sampling general data manifolds. Furthermore,
we introduce the Curvature-Enhanced Manifold
Sampling (CEMS) method for regression tasks.
CEMS leverages a second-order representation
of the data manifold to enable efficient sampling
and reconstruction of new data points. Exten-
sive evaluations across multiple datasets and com-
parisons with state-of-the-art methods demon-
strate that CEMS delivers superior performance
in both in-distribution and out-of-distribution sce-
narios, while introducing only minimal compu-
tational overhead. Code is available at https:
//github.com/azencot-group/CEMS.
1. Introduction
Deep neural networks have demonstrated remarkable per-
formance across a wide range of applications in various
fields (Krizhevsky et al., 2012; Long et al., 2015; Mnih
et al., 2015; Noh et al., 2015; Vinyals et al., 2015; He
1Department of Computer Science, Ben-Gurion University of
the Negev, Beer-Sheva, Israel. Correspondence to: Ilya Kaufman
<ilyakau@post.bgu.ac.il>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada. PMLR 267, 2025. Copyri...