Improve Representation for Imbalanced Regression through Geometric
  Constraints
Authors: name: Zijian Dong; name: Yilei Wu; name: Chongyao Chen; name: Yingtian Zou; name: Yichi Zhang; name: Juan Helen Zhou
Published: 2025-03-02 12:31:34

Abstract: In representation learning, uniformity refers to the uniform feature
distribution in the latent space (i.e., unit hypersphere). Previous work has
shown that improving uniformity contributes to the learning of
under-represented classes. However, most of the previous work focused on
classification; the representation space of imbalanced regression remains
unexplored. Classification-based methods are not suitable for regression tasks
because they cluster features into distinct groups without considering the
continuous and ordered nature essential for regression. In a geometric aspect,
we uniquely focus on ensuring uniformity in the latent space for imbalanced
regression through two key losses: enveloping and homogeneity. The enveloping
loss encourages the induced trace to uniformly occupy the surface of a
hypersphere, while the homogeneity loss ensures smoothness, with
representations evenly spaced at consistent intervals. Our method integrates
these geometric principles into the data representations via a Surrogate-driven
Representation Learning (SRL) framework. Experiments with real-world regression
and operator learning tasks highlight the importance of uniformity in
imbalanced regression and validate the efficacy of our geometry-based loss
functions.