  We propose a novel approach to disentangle the generative factors of
variation underlying a given set of observations. Our method builds upon the
idea that the (unknown) low-dimensional manifold underlying the data space can
be explicitly modeled as a product of submanifolds. This gives rise to a new
definition of disentanglement, and to a novel weakly-supervised algorithm for
recovering the unknown explanatory factors behind the data. At training time,
our algorithm only requires pairs of non i.i.d. data samples whose elements
share at least one, possibly multidimensional, generative factor of variation.
We require no knowledge on the nature of these transformations, and do not make
any limiting assumption on the properties of each subspace. Our approach is
easy to implement, and can be successfully applied to different kinds of data
(from images to 3D surfaces) undergoing arbitrary transformations. In addition
to standard synthetic benchmarks, we showcase our method in challenging
real-world applications, where we compare favorably with the state of the art.
