ICLR 2025

Published as a conference paper at ICLR 2025
DISENTANGLED REPRESENTATION LEARNING WITH
THE GROMOV-MONGE GAP
Theo Uscidda1,2∗
Luca Eyring2,3,4,∗
Karsten Roth2,4,5
Fabian Theis2,3,4
Zeynep Akata2,3,4,†
Marco Cuturi1,6,†
1CREST-ENSAE
2Helmholtz Munich
3TU Munich
4Munich Center of Machine Learning
5Tubingen AI Center
6Apple
theo.uscidda@ensae.fr
luca.eyring@tum.de
ABSTRACT
Learning disentangled representations from unlabelled data is a fundamental chal-
lenge in machine learning. Solving it may unlock other problems, such as general-
ization, interpretability, or fairness. Although remarkably challenging to solve in
theory, disentanglement is often achieved in practice through prior matching. Fur-
thermore, recent works have shown that prior matching approaches can be enhanced
by leveraging geometrical considerations, e.g., by learning representations that
preserve geometric features of the data, such as distances or angles between points.
However, matching the prior while preserving geometric features is challenging, as
a mapping that fully preserves these features while aligning the data distribution
with the prior does not exist in general. To address these challenges, we introduce a
novel approach to disentangled representation learning based on quadratic optimal
transport. We formulate the problem using Gromov-Monge maps that transport one
distribution onto another with minimal distortion of predefined geometric features,
preserving them as much as can be achieved. To compute such maps, we propose
the Gromov-Monge-Gap (GMG), a regularizer quantifying whether a map moves
a reference distribution with minimal geometry distortion. We demonstrate the
effectiveness of our approach for disentanglement across four standard benchmarks,
outperforming other methods leveraging geometric considerations.
1
INTRODUCTION
Learning low-dimensional representations of high-dimensional data is a fundamental challenge
in unsupervised deep learning (Bengio et al., 2014). Emphasis is put...