A Category-theoretical Meta-analysis of Deﬁnitions of Disentanglement

TITLE:TL;DR:Disentanglement is a Product Morphism.
1, 22, 1

Yivan Zhang
Masashi Sugiyama
1
2
The University of Tokyo
RIKEN AIP
What is disentanglement?

sourtaste?

?

sweet

(a) Composition(b) Supervision

Product: core of disentanglement

Equivariant maps

Set: category of sets (objects) and functions (morphisms)
Let’s consider Y : factors, X: observations, and Z: codes.

[S, Set]: category of functors and natural transformations
˚ aDisentanglement: f : X → Z is a morphism to a product.

˚ˆˇ(c) Generalization
functionsetset

A uniﬁed deﬁnition?
What do direct product and independent random variables
have in common?
Can we deﬁne disentanglement using only functions?
What are the deﬁning properties of disentanglement?
Category theory provides a suitable abstraction to identify,
formalize, and organize common patterns, mathematicallyrigorous diagrammatic reasoning, and generality to tackle
increasingly complex machine learning problems.
AA×B

f2

p2

B
fB

(a) A commutative diagram ofa morphism C → A × B
to a cartesian productC(b) A string diagram ofa morphism C → A ⊗ B
to a monoidal productbZ1aZf :“ϕ˚ ˆ ˇϕˇZpaˆbqZ“aZ ˆbZ

XZ2bZ

Algebra action = functor from a single-object category
Equivariant map = natural transformationsour
sweet..
.

Probabilistic models capture the relations and uncertainty ofvariables. A disentangled encoder should satisfy certain
statistical independence conditions [Higgins et al., 2017,Chen et al., 2018, Suter et al., 2019].

p1

set

(red, triangle)(blue, triangle)(green, circle)(red, circle)

equivariance ⇝ naturalityStochastic maps

triangle

A function C → A × B
to a Cartesian productis just a pair of functions
C → A and C → B.

Statistical deﬁnitions

AfunctionpaˆbqXaˆbϕ˚bXGroup actions capture the transformations and symmetries[Cohen and Welling, 2014]. A disentangled encoder should beequivariant to group actions of a direct product of groups[Higgins et al., 2018].

⟨f1,f2⟩aX

identity function

Algebraic deﬁnitions

f1ˇ

composite function

Colorful and tasty candies!
We only need to taste a handful of candies to ﬁnd out therelationship between their color, shape, and taste. We canuse this knowledge to predict the taste of other candies.
Can a neural network do this?

CarXiv:2305.06886
https://yivan.xyz
yivanzhang@ms.k.u-tokyo.ac.jp

A function A × B → C
from a Cartesian productcan depend on both inputs.
Can it depend on only one?

circleStoch: category of measurable spaces and stochastic maps
Z1 Z2
Z1
Z2
Z1 Z2
“mmm“

m1m2

Y1

Y2

Modularity: a code encodes only one factor
Modularity: m : Y → Z := f ◦ g is a product of morphisms.
redY

p1

(red, circle)

p2

circleY1

p1ggX
fZY2gm1h

Y1 × Y2

p2

m2fcolor„ ȷp1

m1fshape„ ȷ „ ȷl(, △ )lp2

Xf1f„ ȷl△l

Z1

m2

p1

f

Z1 × Z2

f2

p2

Z2

When is A × B → C just A → C?
We can use exponential objects and pullbacks.

Informativeness: codes encode factors faithfully
Informativeness: m : Y → Z is a split monomorphism.
m : Y → Z has a retraction h : Z → Y , s.t. h ◦ m = idY .
We should disentangle modularity and informativeness!

Y1

Y2

Y1

Y2

Joint distributions are monoidal products, not cartesian.
We can use copy & delete in a Markov category [Fritz, 2020].
probability & statistics ⇝ Markov category
Next steps?
Disentanglement metrics (enriched category theory?)[Zhang and Sugiyama, 2023]
Analyses on functor categories and Markov categories
More structures and operations beyond product!
References
Ricky TQ Chen, Xuechen Li, Roger B Grosse, and David K Duvenaud. Isolating sources of disentanglement in variationalautoencoders. In NeurIPS, 2018.
Taco Cohen and Max Welling. Learning the irreducible representations of commutative lie groups. In ICML, 2014.
Tobias Fritz. A synthetic approach to markov kernels, conditional independence and theorems on sufﬁcient statistics. Advancesin Mathematics, 2020.
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander
Lerchner. beta-VAE: Learning basic visual concepts with a constrained variational framework. In ICLR, 2017.
Irina Higgins, David Amos, David Pfau, Sebastien Racaniere, Loic Matthey, Danilo Rezende, and Alexander Lerchner. Towards adeﬁnition of disentangled representations. arXiv:1812.02230, 2018.
Raphael Suter, Djordje Miladinovic, Bernhard Schölkopf, and Stefan Bauer. Robustly disentangled causal mechanisms: Validating
deep representations for interventional robustness. In ICML, 2019.
Yivan Zhang and Masashi Sugiyama. Enriching disentanglement: Deﬁnitions to metrics. arXiv:2305.11512, 2023.