ICML 2025

Compositional Generalization via Forced Rendering of Disentangled Latents
Qiyao Liang* 1 Daoyuan Qian* 2 Liu Ziyin 1 3 Ila Fiete 1
Abstract
Composition—the ability to generate myriad vari-
ations from finite means—is believed to underlie
powerful generalization. However, compositional
generalization remains a key challenge for deep
learning. A widely held assumption is that learn-
ing disentangled (factorized) representations nat-
urally supports this kind of extrapolation. Yet,
empirical results are mixed, with many genera-
tive models failing to recognize and compose fac-
tors to generate out-of-distribution (OOD) sam-
ples. In this work, we investigate a controlled 2D
Gaussian “bump” generation task with fully dis-
entangled (x, y) inputs, demonstrating that stan-
dard generative architectures still fail in OOD
regions when training with partial data, by re-
entangling latent representations in subsequent
layers. By examining the model’s learned kernels
and manifold geometry, we show that this failure
reflects a “memorization” strategy for generation
via data superposition rather than via composition
of the true factorized features. We show that when
models are forced—through architectural modi-
fications with regularization or curated training
data—to render the disentangled latents into the
full-dimensional representational (pixel) space,
they can be highly data-efficient and effective at
composing in OOD regions. These findings un-
derscore that disentangled latents in an abstract
representation are insufficient and show that if
models can represent disentangled factors directly
in the output representational space, it can achieve
robust compositional generalization.
*Equal contribution
1Massachusetts Institute of Technology,
Cambridge MA, USA 02139 2University of Cambridge, Cam-
bridge CB2 1EW, U.K 3NTT Research. Correspondence to: Qiyao
Liang <qiyao@mit.edu>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada. PMLR 267, ...