Improve Representation for Imbalanced Regression through Geometric
  Constraints
Authors: name: Zijian Dong, name: Yilei Wu, name: Chongyao Chen, name: Yingtian Zou, name: Yichi Zhang, name: Juan Helen Zhou

In representation learning, uniformity refers to the uniform feature
distribution in the latent space (i.e., unit hypersphere). Previous work has
shown that improving uniformity contributes to the learning of
under-represented classes. However, most of the previous work focused on
classification; the representation space of imbalanced regression remains
unexplored. Classification-based methods are not suitable for regression tasks
because they cluster features into distinct groups without considering the
continuous and ordered nature essential for regression. In a geometric aspect,
we uniquely focus on ensuring uniformity in the latent space for imbalanced
regression through two key losses: enveloping and homogeneity. The enveloping
loss encourages the induced trace to uniformly occupy the surface of a
hypersphere, while the homogeneity loss ensures smoothness, with
representations evenly spaced at consistent intervals. Our method integrates
these geometric principles into the data representations via a Surrogate-driven
Representation Learning (SRL) framework. Experiments with real-world regression
and operator learning tasks highlight the importance of uniformity in
imbalanced regression and validate the efficacy of our geometry-based loss
functions.