Counterfactuals uncover the modular structure of deep generative models
Authors: Michel Besserve
Arash Mehrjou
Rémy Sun
Bernhard Schölkopf
Venue:  (2020)
Keywords: generative models
causality
counterfactuals
representation learning
disentanglement
generalization
unsupervised learning

Abstract: Deep generative models can emulate the perceptual properties of complex image datasets, providing a latent representation of the data. However, manipulating such representation to perform meaningful and controllable transformations in the data space remains challenging without some form of supervision. While previous work has focused on exploiting statistical independence to \textit{disentangle} latent factors, we argue that such requirement can be advantageously relaxed and propose instead a non-statistical framework that relies on identifying a modular organization of the network, based on counterfactual manipulations. Our experiments support that modularity between groups of channels is achieved to a certain degree on a variety of generative models. This allowed the design of targeted interventions on complex image datasets, opening the way to applications such as computationally efficient style transfer and the automated assessment of robustness to contextual changes in pattern recognition systems.

Published as a conference paper at ICLR 2020
COUNTERFACTUALS UNCOVER THE MODULAR STRUCTURE OF DEEP GENERATIVE MODELS
Michel Besserve!l2.. Arash Mehrjou 133 Rémy Sun 1.3 Bernhard Schölkopf"
1. MPI for Intelligent Systems, Tübingen, Germany.
2. MPI for Biological Cyhernetics, Tübingen, Germany.
Department of Computer Science, ETH Zürich, Switzerland. ENS Rennes, France.
ABSTRACT
Deep generative models can emulate the perceptual properties of complex image datasets, providing latent representation of the data. However, manipulating such representation to perform meaningful and controllable transformations in the data space remains challenging without some form of supervision. While previous work has focused on exploiting statistical independence to disentangle latent factors, we argue that such requirement can be advantageously relaxed and propose instead a non-statistical framework that relies on identifying modular organization of the network, based on counterfactual manipulations. Our experiments support that modularity between groups of channels is achieved to a certain degree variety of generative models. This allowed the design of targeted interventions on complex image datasets, opening the way to applications such computationally efficient style transfer and the automated assessment of robustness to contextual changes pattern recognition systems.
Original
Counterfactual
Caurcal Inderr serventtonn
Figure 1: Counterfactual manipulation of samples of BigGan trained on ImageNet (see section 4.2
INTRODUCTION
Deep generative models, by learning non-linear function mapping a latent space to space of observations, have proven successful at designing realistic images in a variety of complex domains (objects, animals, human faces, interior scenes). In particular, two kinds of approaches emerged as state-of-the-ar (SOTA): Generative Adversarial Networks (GAN) (Goodfellow et al. 2014), and Variational Autoencoders (VAE) (Kingma & Welling 2013 Rezende al. 2014).
Efforts have been made to have such models produce disentangled latent representations that can control interpretable properties of images (Kulkarni et al. 2015 Higgins al., 2017). However, the resulting models are not necessarily mechanistic (or causal) in the sense that interpretable properties of image cannot be ascribed to particular part, module, of the network architecture. Gaining access to a modular organization of generative models would benefit the interpretability and allow extrapolations, such generating an object in a background that was not previously associated with this object, as illustrated in preview of our experimental results in Fig. [
Such extrapolations are an integral part of human representational capabilities (consider common expressions such as "like an elephant in china shop") and consistent with the modular organization
"mi chel. besserve@tueb ngen..nen .egee.mppg... mpg.de

---

Published as a conference paper at ICLR 2020
of its visual system, comprising specialized regions encoding objects, faces and places (see e.g. Grill
Spector & Malach (2004). Extrapolations moreover likely support adaptability to environmental changes and robust decision making (Dvornik et al.) 2018). How to leverage trained deep generative architectures to perform such extrapolations is an open problem, largely due to the non-linearities and high dimensionality that prevent interpretability of computations performed in successive layers.
In this paper, we propose a causal framework to explore modularity, which relates to the causal principle of Independent Mechanisms, stating that the causal mechanisms contributing to the overall generating process do not influence nor inform each other (Peters et al. 2017)." We study the effect of direct interventions in the network from the point of view that the mechanisms involved in generating data can be modified individually without affecting each other. This principle can be applied to generative models to assess how well they capture a causal mechanism (Besserve et al. 2018a). Causality allows to assay how an outcome would have changed. had some variables taken different values, referred to as counterfactual (Pearl) 2009 Imbens& & Rubin 2013). We use counterfactuals to assess the role of specific internal variables in the overall functioning of trained deep generative models, along with a rigorous definition of disentanglement in a causal framework. Then, we analyze this disentanglement in implemented models based on unsupervised counterfactual manipulations. We show empirically how VAES and GANs trained on image databases exhibit modularity of their hidden units, encoding different features and allowing counterfactual editing of generated images.
Related work. Our work relates to the interpretability of convolutional neural networks, which has been intensively investigated in discriminative architectures (Zeiler & Fergus| 2014 Dosovitskly &
Brox 2016, Fong & Vedaldi 2017 Zhang et al., 2017b a). Generative models require a different approach, as the downstream effect of changes in intermediate representations are high dimensional.
InfoGANs. 3-VAEs and other works (Chen et al. 2016 Mathieu et al. 2016 Kulkarni et al. 2015
Higgins et al. 2017) address supervised or unsupervised disentanglement of latent variables related to what we formalize as extrinsic disentanglement of transformations acting on data points. We introduce the novel concept of intrinsic disentanglement to uncover the internal organization of networks, arguing that many interesting transformations are statistically dependent and are thus unlikely to be disentangled in the latent space. This relates to Bau et al. (2018) who proposed a framework based on interventions on internal variables of a GAN which, in contrast to our fully unsupervised approach, requires semantic information. Higgins et al. (2018) suggest a definition of disentanglement based on group representation theory. Compared to this proposal, our approach (introduced independently in (Besserve et al.] 2018b)) is more flexible as it applies to arbitrary continuous transformations, free from the strong requirements of representation theory (see Appendix F). Finally, an interventional approach to disentanglent aemeetentttet has also be taken by [Suter et al. (2018), who focuses on extrinsic disentanglement in a classical graphical model setting and develop measures of interventional robustness based on labeled data.
2 FROM DISENTANGLEMENT TO COUNTERFACTUALS AND BACK
We introduce a general framework to formulate precisely the notion of disentanglement and bridge it to causal concepts. This theory section will be presented informally to ease high level understanding. Readers interested in the mathematical aspects can refer to Appendix A where we provide all details.
2.1 A CAUSAL GENERATIVE MODEL (CGM) FRAMEWORK
We consider generative model M that implements function gM, which maps latent space Z to manifold VM where the learned data points live, embedded in ambient Euclidean space y (Fig. 2a). A sample from the model is generated by drawing realization = from a prior latent variable distribution with mutually independent components, fully supported in Z. We will use the term representation to designate mapping from Vns to some representation space R (we also call r(y) the representation of a point y E M. In particular, we will assume (see Def. 7 and Prop. 4 in Appendix A) that gM is (left)-invertible, such that 9M is representation of the data, called the latent representation.
Assuming the generative model is implemented by a non-recurrent neural network, we can use a causal graphical model representation of the entailed computational graph implementing the mapping gM through a succession of operations (called functional assignments in causal language), as illustrated
Note that this is not statisticol independence; the quantities transformed by the mechanisms of course do influence each other and be statistically dependent.
2

---

Published as a conference paper at ICLR 2020
Lotent representation
Statistical independence
Z
T
inpul
21
id
1a
Endogehauu worlables
1d
2-t
M
MM
9M
V2
1d
Internal IM representation
1i
laver
Ouequt
9M
9M
Support of data distribution
V M
y
T(Y)
Data ambient space
Independence of mechanisms
y
T
y
(b)
(c)
(d)
Figure 2: (a) Illustration of the generative mapping and a disentangled transformation. (b) Causal graph of example CGM showing different types of independence between nodes. (c) Commutative diagram showing sparse transformation in latent space associated to a disentangled transformation
T. (d) Illustration of intrinsic disentanglement with & [2].
in Fig. 2b that we will call Causal Generative Model (CGM). In addition the latent representation, we can then choose a collection of possibly nulti-dimensional endogenous (internal) variables represented by nodes in the causal graph, such that the mapping gM is computed by composing the endogenous variable assignment UM with the endogenous mapping 9M according to the diagram
Z "M VM
9M
VM
A paradigmatic choice for these variables is the collection of output activation maps of each channel in one hidden layer of a convolutional neural network, as illustrated in Fig. 2b) As for the latent case, use mild conditions that guarantee ΩM to left-invertible, defining the internal representation of the network (see Def. [ and Prop. 4 Appendix A). Given the typical choice of dimensions for latent and endogenous variables, the V are also constrained take values in subsets V of smaller dimension than their Euclidean ambient space D* As detailed in Appendix A, we introduce the endogenous image sets of the form V M {o ILse Vk: v (vu (Z))NEE, z E Z for subset of variables indexed by E (amounting to VM when E includes all endogenous variables).
This CGM framework allows defining counterfactuals in the network following Pearl (2014).
Definition 1 (Unit level counterfactual, informal). Given CGM M, the interventional model Mr. is obtained by replacing assignments of the subset variables & and by the vector of assignments h. Then for any latent input z, called unit, the unit-level counterfactual is the output of Mr
Y 9MA (2).
Def. is also in line with the concept of potential outcome (Imbens & Rubin 2015). Importantly counterfactuals induce a transformation of the output of the generative model.
Definition (Counterfactual mapping). Given an embedded CGM, we call the transformation
Y y gad (
A the h-coonteefaactull mapping. We say faithful to M whenever Y [Vm] C VMM-
We introduce faithfulness of counterfactual mapping to account for the fact that not all interventions on internal variables will result in an output that could have been generated by the original model. In the context of generative model, non-faithful counterfactuals generate examples that leave the support of the distribution learned from data, possibly resulting in an artifactual output (assigning a large value neuron may saturate downstream neurons), allowing extrapolation unseen data.
3

---

Published as a conference paper at ICLR 2020
2.2 UNSUPERVISED DISENTANGLEMENT FROM STATISTICAL TO CAUSAL PRINCIPLES
The classical notion of disentangled representation (e.g. Bengio et al. (2033) Kulkarni et al. (2015) posits individual latent variables "sparsely encode real-world transformations" Although the concept of real-world transformations remains elusive, this insight, agnostic to statistical concepts, has driven supervised upproaches to disentangling representations, where relevant transformations are well-identified and manipulated explicitly using appropriate datasets and training procedures.
In contrast, unsupervised learning approaches to disentanglement need learn such real-world transformations from unlabeled data. In order to address this challenge, SOTA approaches seek to encode such transformations by changes in individual latent factors, and resort to a statistical notion of disentanglement enforcing conditional independence between latent factors (Higgins et al. (017).
This statistical approach leads to several issues:
Independence is not necessary. The i.i.d. constraints on the latent variables' prior impose statistical independence between disentangled factors for the data distribution. This is unlikely for many relevant properties, counfounded by factors of the true data generating mechanisms (e.g.
skin and hair color).
Independence is not sufficient to specify a disentangled representation, such that the problem remains ill posed (Locatello et al. 2018). As a consequence, finding an appropriate inductive bias to learn representation that benefits downstream tasks remains open question.
To date, SOTA unsupervised approaches are mostly demonstrated on synthetic datasets, and beyond MNIST disentangling complex real world data has been limited to the well-calibrated CelebA dataset. On complex real-world datasets. disentangled generative models exhibit visual sample quality far below non-disentangled SOTA (e.g. BigGAN exploited our work (Brock et al. 2018)
We propose an statissiit definition of disentanglement by first phrasing mathematically the transformation-baee insights (Bengio et al.] 2013 Kulkarni et al.) 2015). Consider a transformation acting on the data manifold Var- As illustrated by the commutative diagram of Fig. [cc disentan- glement of such property then amounts to having T correspond to transformation T' of the latent space that would act only on a single variable 2k. using transformation f. leaving the other latent variables available to encode other properties. More explicitly we have
Tz)) (21, 2k,, 2 + (Z1,.., f(x),, .., =k), and (=)) = gM (T))).)
It is then natural qualify two transformations Ti and T2 disentangled (from each other), whenever they modify different components of the latent representation (see Def. TO in Appendix F). This amounts saying that the transformations follow the causal principle of independent mechanisms (Peters et al. 2017 Parascandolo et al. 20188)
Due to the fact that it relies on transformations of the latent representation, that are exogenous the CGM, we call this notion extrinsic disentanglement This "functional" definition has the benefit of being agnostic the the subjective choice of the property disentangle, and to the statistical notion of independence. However, we can readily notice that, if applied to the latent space (where components are i.i.d. distributed), this functional notion of disentangled transformation still entails statistical independence between disentangled factors. We thus need to exploit different representation to uncover possibly statistically related properties, but disentangled in the sense of our definition.
2.3 DISENTANGLING BY MANIPULATING INTERNAL REPRESENTATIONS
As illustrated in the CGM of Fig. 2b in contrast latent variables, properties encoded by endogenous variables of the graphical model are not necessarily statistically independent due to common latent causes (2j and 22 in the figure), but may still reflect interesting properties of the data that can be intervened on independently. following the principle of independence of mechanisms. We thus extend our definition of disentanglement to allow transformations of the internal variables of the network as follows.
Definition 3 (Intrinsic disentanglement, informal). In a CGM M, a transformation T YM > Ya is intrinsically disentangled with respect to subset E of endogenous variables, if there is transformation T' acting on the internal representation space such that for any endogenous value v
Tiği (v)) = -M (x))
4

---

Published as a conference paper at ICLR 2020
samsle
sample 2;
Original
Orginal 2
0(2))
Dly Ilybrid sample
(a)
(b)
Figure 3: Generation of influence maps. (a) Principle of sample hybridization through counterfac- tuals. (b) Examples of EIM generated by VAE on the CelebA dataset (lighter pixel indicate larger variance and thus stronger influence of the perturbations on that pixel).
where (v) only affects the variables indexed by
Fig. 2d illustrates this second notion of disentanglement, where the split node indicates that the value of V/2 is computed as in the original CGM (Fig. 2b) before applying transformation T to the outcome. While the above definition applies to single transformation, straightforward extensions of this concept to families transformations are provided in Appendix F, Faithful counterfactuals represent examples of disentangled transformations:
Proposition 1 (Counterfactuals and disentanglement, informal). Consider an intervention on subset E, its associated counterfactual mapping is faithful if and only if it is disentangled. For interventions on variables that remain within the support of the original marginal distribution, it is sufficient that & and its complement & do not have common latent ancestors.
This indicates that finding faithful counterfactuals can be used to learn disentangled transformations.
3 FINDING MODULARITY IN DEEP GENERATIVE MODELS
3.1 DEFINING MODULARITY
Building on Sec. 2 define modularity structural property of the internal representation, allow- ing (with the immediately following Prop. 2 to implement arbitrary disentangled transformations.
Definition (Modularity). subset of endogenous variables & is called modular whenever Var is the Cartesian product of S by V M-
Proposition 2 (Modularity implies disentanglement). If E is modular, then any transformation applied to it staying within its input domain is disentangled.
The proof is a natural extension the proof of Proposition]] [ Both the Definition and the Proposition have trivial extensions to multiple modules (along the line described in Appendix F). While we have founded this framework on a functional definition of disentanglement that applies to transformations, the link made here with an structural property of the trained network allows to define disentangled representation as follows.
Definition 5 (Disentangled representation). A disentangled representation is a partition of the endogenous variables in several modules, such that VM the Cartesian product of the corresponding endogenous image sets.
Then any transformation applied to a given module leads to valid transformation in the data space (it is relatively disentangled following Def. T0 Appendix F).
5

---

Published as a conference paper at ICLR 2020
Interestingly, disentangled representation is associated to partition of the considered set of endogenous variables into modules. This extra requirement was not considered in classical approaches to disentanglement as it assumed that each single scalar variables could be considered as an independent module. Our framework provides an insight relevant to artificial and biological systems: as the activity of multiple neurons can be strongly tied together, the concept of representation may not be meaningful at the "atomic" level of single neurons, but require to group them into modules forming "mesoscopic" level, which each group can be intervened on independently.
3.2 HYBRIDIZATION AS A DISENTANGLED TRANSFORMATION
As stated in Sec. 2 functional definition of disentanglement, leaves unanswered how to find relevant transformations. Prop. | and provide the following hints: (1) Once a modular structure is found in the network, broad class of disentangled transformations
available, (2) Transformations that stay within their input domain are good candidates of disentanglement, (3) Counterfactual interventions implicitly define transformations. We follow these guidelines by assigning constant value du to subset of endogenous variables & to define counterfactuals (i.e. h is a constant function), aiming for faithful ones by constraining vo to belong to V To avoid characterizing V we rely on sampling from the (joint) marginal distribution of the variables in E.
To illustrate the procedure, we consider a standard feed-forward multilayer neural network and choose endogenous variables to be the collection of all output activations of channels of given layer f. Let & be a subset of these channels, the hybridization procedure, illustrated in Fig. [3 goes as follows. We take two independent examples of the latent variable ž and ø2, that will generate two original examples of the output (y1, y2) = (g (z1). 9M (Zz)) (that we call Original and Original 2). We also memorize the tuple (x2)) gathering values of variables indexed by & when generating Original 2, and o( 2(21 the tuple of values taken by all other endogenous variables on this layer, but when generating Original Assuming the choice of identifies a modular structure, 0(2)) and v (22) would encode different aspects of their corresponding generated images, such that one can generate hybrid example mixing these features by assigning the collection of output values layer with the concatenated tuple (ő(z1), vz22)) and feeding it to the downstream part of the generator network.
3.3 MEASURING CAUSAL EFFECTS
The above counterfactual hybridization framework allows assessing how a given module & affeets the output of the generator. For this purpose we quantify its causal effect by repetitively generating pairs (2), 22) from the latent space, where both vectors are sampled independently each other. We then generate and collect hybrid outputs following the above described procedure for batch of samples and use them to estimate an influence map as the mean absolute effect:
(E) = (Z,
y (e( (z1) Y(z))
(2)
where Y(z)) = g3a (z1) is the non- -on-enneree output of the generator for latent input 21. In eq.
2
unit-level causal effect in the potential outcome framework (Imbens & Rubin 2015), and taking the expectation is analogous to computing the average trealment effect. Our approach has however two specificities: (1) we take the absolute value of the unit-level causal effects, as their sign may not be consistent across units, (2) the result is averaged over all interventions corresponding to possible values of z2.
the difference inside the absolute value can be interpreted
While LM has the same dimension as the output image, we then average it across color channels to get a single grayscale heat-map pixel map. We also define a scalar quantity to quantify the magnitude of the causal effect, the individual inffuence of module E, by averaging IM across output pixels.
3.4 UNSUPERVISED DETECTION OF MODULES AND COUNTERFACTUAL IMAGES
challenge with the hybridization approach is to select the subsets E to intervene on, especially with networks containing large amount of units channels per layer. We use a fine to coarse approach to extract such groups, that we will describe in the context of convolutional layers. First, we estimate elementary infueece maps (EIM) associated to each individual output channel of each convolutional layer of the network (i.e. we set E = {c} in eq. 2 Then influence maps are grouped by similarity to define modules coarser scale, as will describe in detail below.

---

Published as a conference paper at ICLR 2020
Representative EIMs for channels of convolutional layers of a VAE trained on the CelebA face dataset (see result section) are shown in Supplementary Fig. 3b and suggest channels are functionally segregated, with for example some influencing finer face feature (eyes, mouth.... and others affecting the background of the image or the hair. This supports the idea that individual channels can be grouped into modules that are mostly dedicated one particular aspect of the output.
In order to achieve this grouping in an unsupervised way, we perform clustering of channels using their EIMs as feature vectors follows. We first pre-process each influence map by: (1) performing local averaging with a small rectangular sliding window to smooth the maps spatially, (2) thresholding the resulting maps at the 75% percentile of the distribution of values over the image to get binary image. After flattening mmgge dimensions, we get a (channel x pixels) matrix S which is then fed to a Non-negative Matrix Factorization (NMF) algorithm with manually selected rank K, leading to the factorization S = WH. From the two resulting factor matrices, we get the K cluster template patterns (by reshaping each rows of H to image dimensions), and the weights representing the contribution of each of these pattern individual maps (encoded in W). Each influence map is then ascribed a cluster based on which template pattern contributes to it with maximum weight. The choice of NMF is justified by its success in isolating meaningful parts of images in different components (Lee & Seung 1999). However, we also compared our approach to the classical k-means clustering algorithm (see Supplementary Fig. 9.. In order to further justify our NMF based approach, also introduce a toy generative model.
Model 1. Consider Z a vector of K i.i.d. uniformly distributed RVs. Assume a neural network with one hidden layers composed of vector variables Vi such that
= S(
with H& R", > 1 and S strictly increasing activation function applied entry-wise to the components of each vector (e.g. a leaky ReL.U). These endogenous variables are mapped to the output
K
Y W k=1
with matrices W E RRmx,, m. > nK. Assume additionally the following random choice for the model parameters: (1) all coefficients of H&? 's are sampled i.i.d. from an arbitrary distribution that has density with respect to the Lebesgue measure, (2) there exists K sets of indices I over [1, m] each containing at least one element 44 Iş such that for all j * k. ls & I (3) For given column of W coefficient in Iç are sampled ii.dd from an arbitrary distribution that has a density with respect to the Lebesgue measure, while the remaining coefficients are set to zero.
The specific condition on the T 's enforced in (2) encodes the assumption that there is an area in the image that is only influenced by one of the modules. For example, assuming simple background/object module pair, it encodes that the borders of the image never belong to the object while the center of the image never belong background. For this model, we get the following identifiability result.
Proposition 3. For M with probability I we have:
(I) The partition of the hidden layer entailed by the K vectors (Vi) corresponds to a disentangled representation (i.e. each vector is modular relatively to the others). (2) Assume influence maps 1M (i, k) of each component in each vector V₁, are known and build the (m nK) binary matrix B by concatenating binary column vectors B (M, ki) > O. Non-negative matrix factorization of B is unique (up to trivial transformations) and identifies the subsets of endogenous variables associated each Vt.
This justifies the use of NMF on thresholded version of the matrix gathering all EIMs (to generate a binary matrix summarizing their significant influences on each output pixel). Moreover, the application of the sliding window is justified in order enforce the similarity between the influence maps belonging to the same module, reflected by the condition on identical support I, for all columns of W in Model and favoring low-rank matrix factorization.
7

---

Published as a conference paper at ICLR 2020
Cluster templates
Influence maps
Orginal
Hyönd
Ongral 2
Clusery 1
Cluterz 2
cluserr 3
Figure 4: Left: Clustering of influence maps for BEGAN trained the CelebA dataset (see text). Center: example EIM of each cluster. Right: samples of the hybridization procedure using as module all channels of the intermediate layer belonging to the cluster of corresponding row. See Fig. 12 for additional samples.
4 EXPERIMENTS
4.1 DCGAN, 3- VAE AND BEGAN ON THE CELEBA DATASET
We first investigated modularity of genrative models trained on the CelebFaces Attributes Dataset (CelebA) et al. 2015) and used basic architecture: plain 8-VAE? (Higgins et al. 2017). We ran the full procedure described in Sec. 3 comprised of EIM calculations, clustering channels into modules, and hybridization of generator samples using these modules. Hybridization procedures were performed by intervening on the output of the intermediate convolutional layer (indicated in Supplemental Fig. 7). The results are summarized in Supplemental Fig. S We observed empirically that setting the number of clusters to 3 leads consistently to highly interpretable cluster templates as illustrated in the figure, with one cluster associated to the background, one the face and one to the hair. This observation was confirmed by running the following cluster stability analysis: we partition random the influence maps in 3 subsets, and two thirds of the data, both runs overlapping only on one third. The obtained clusters were then matched in order maximize the label consistency (the proportion of influence maps assigned the same label by both runs) on the overlapping subset, and this maximum consistency was used to assess robustness of the clustering across the number of clusters. The consistency results are provided in Supplemental Fig. | and show 3 clusters is a reasonable choice as consistency is large (> 90%) and drops considerably for 4 clusters. Moreover, these results also show that the NMF-based clustering outperforms clustering with the more standard k-means algorithm. In addition, we also assessed the robustness the clustering by looking at the cosine distance between the templates associated to matching clusters, averaged across clusters. The results, also provided in Supplemental Fig, 9 are consistent with the above analysis with an average cosine similarity of .9 (scalar product between the normalized feature vectors) achieved with clusters (maximum similarity is 1 for perfectly identical templates). Exemplary influence maps shown Supplemental Fig. 8 (center panel) reflect also our general observation: some maps may spread over image locations reflecting different clusters.
this partition to the clustering twice
Interestingly, applying the hybridization procedure to the resulting 3 modules obtained by clustering leads replacement of the features associated to the module intervene on, shown in Sup- plemental Fig. g (center panel), while respecting the overall structure of the image (no discontinuity introduced). For example, on the middle row we see the facial features of the Original 2 samples are inserted in the Original I image (shown on the left), while preserving the hair.
ht httpss//githus http://wwwb.coooeese////eee
8

---

Published as a conference paper at ICLR 2020
While the 3-VAE is designed for extrinsic disentanglement. further work has shown that it can prove suboptimal with respect to other approaches (Chen et al. 2018 Locatello et al. 2018 suggesting further work could investigate whether better extrinsic disentanglement could also favor intrinsic disentanglement. It is however important to investigate intrinsic disentanglement in models for which (extrinsic) disentanglement is not enforced explicitly. This is in particular the case of most GAN-like architectures, who typically outperform VAE-like approaches in terms of sample quality in complex image datasets. Interestingly, the above results could also be reproduced in the official tensorlayer DCGAN implementation? 2 equipped with a similar architecture (see Appendix E). This suggests that our approach can be applied to models that have not been optimized for disentanglement. After these experiments with basic models, we used an implementation" of the Boundary Equilibrium GAN (BEGAN) (Berthelot et al. 2017). which used to set a milestone in visual quality for higher resolution face images. Most likely due to the increase in the number of layers, we observed that obtaining counterfactuals with noticeable effects required interventions on channels from the same cluster in two successive layers (see Appendix D). The results shown in Fig. + obtained by intervening layers 5 and 6, reveal a clear selective transfer of features from Original 2 to Original I. As the model was trained on face images eropped with tighter frame than for the above models, leaving little room for the hair and background, we observe only one module associated to these features (Fig. 4 middle row) showing clear hair transfer. The remaining two modules are now encoding different aspects of face features: eye contour/mouthhnose for the top row and eyelids/face shape for the bottom row module. We further evaluated the relative quality of the counterfactual images with respect to the original generated images using the Frechet Inception Distance (FID) (Heusel et al. 2017] (Table in the appendix), supporting that the hybridization procedure only mildly affects the image quality, in comparison to the original samples.
4.2 BIGGAN ON THE IMAGENET DATASET
In order to check whether our approach could scale to high resolution generative models, and generalize to complex image datasets containing variety of objects, we used the BigGAN-deep architecture (Brock et al. 30. pretrained on the ImageNet datasets. This is a conditional GAN architecture comprising 12 so-called Gblocks, each containing cascade of 4 convolutional layers (see Appendix C for details). Each Gblock also receives direct input from the latent variables and the class label, and bypassed by skip connection. We then checked that we were able to generate hybrids by mixing the features of different classes. As for the case of BEGAN, intervening on two successive layers within a Gblock was more effective to generate counterfactuals (examples are provided for the 7th Gblock). Examples provided in Fig. ( (cock-os)r)))))) show that it is possible to generate high quality counterfactuals with modified background while keeping a very similar object in the foreground. In more challenging situation, with objects of different nature (Koala-teddy bear on the same figure). meaningful combinations of each original samples are still generated: c.g. a teddy bear in tree (bottom row), "teddy-koala" merging teddy texture with the color of a koala on a uniform indoor background with a wooden texture (top row).
In order to investigate how the generated counterfactual images can be used to probe and improve the robustness of classifiers to contextual changes, we compared the ability of several SOTA pretrained (nttps://etsub ddvr] see Appendix C for details) recognize
classifier available on Tensorflow-hub one of the original classes. Fig. @]shows the average recognition rate of the most recognized original class (teddy-bear or koala), as function of layer depth tends overall increase. We first observe that high recognition rates are in line with the small pixel distance between hybrids and original when intervening at layers closest to the output (right panel). Interestingly, at intermediate blocks 5-6, there is a clear contrast between classifiers, with the Inception resnet performing better than the others. Interestingly, examples of non-consensual classification results in Supplementary Table together with the associated hybrids (Supplementary Fig. 17) suggest different SOTA classifiers rely on different aspects of the image content to take their decision (e.g. background versus object).
httes:/ http://www...ooccccaatteeeeeeees
httpss httpss//githae thub. hwt.g/thhhh......eeeoeeeeeee cww// /INEGAN tenxor fl
httpes/ Facb. dey
/biggan beaggggnnneeee deep- 256/1
https// how. mmage- net..ette norg
9

---

Published as a conference paper at ICLR 2020
Original
Hybrid
Original 2
Original 1
Hybrid
Original
Figure 5: Examples of BigGAN hybridizations across classes. Left: ostrich-cock, right: koala-teddy, for additional samples and Fig. 14 for entropy analysis.
See Figs. 3
Hybrid recognition rate of clozest orignal class (%)
Average absolute pix. dist. of bybrid to clasest original
duntr
rusee
0.40
0.20
Block nbr. Ifron latent to imboc))
Block nbr (fro latent imader
Figure 6: Analysis of classifier robustness counterfactual changes (bars indicate standard error).
CONCLUSION
We introduced a mathematical definition of disentanglement, related to the causal notion of counterfactual and used it for the unsupervised characterization of the representation encoded by different groups of channels in deep generative architectures. We found evidence for interpretable modules of internal variables in four different generative models trained on two complex real world datasets. Our framework opens a way to a better understanding of complex generative architectures and applications such as the style transfer (Gatys al., (015)) of controllable properties of generated images at low computational cost (no further optimization required), and the automated assessment of robustness of object recognition systems to contextual changes. From a broader perspective, this research direction contributes to better exploitation of deep neural networks obtained by costly and highly energy-consuming training procedures, by (1) enhancing their interpretability and (2) allowing them to be used for tasks their where not trained for. This offers a perspective how more sustainable research in Artificial Intelligence could be fostered in the future.
4.3 APPENDICES AND IMPLEMENTATIONS
Appendices and supplementary Figures and Tables are available online with the arXiv version of this paper http::///axxvvoor//aa/aaa 03253). Implementations are available on the companion website
mpg. wttp:mppppppp.eeeeeeeeeeeppeeepe de/besserve
deepcounterfactuu...
REFERENCES
M. A. Armstrong. Basic topology. Springer Science & Business Media, 2013.
D. Bau, J.-Y. Zhu, H. Strobelt, B. Zhou, J. B. Tenenbaum, W. T. Freeman, and A. Torralba. Gan dissection: Visualizing and understanding generative adversarial networks. arXiv preprint arXiv: 2018
Y Bengio, A Courville, and P Vincent. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 3598) 2013.
10

---

Published as a conference paper at ICLR 2020
D. Berthelot, T. Schumm, and L. Metz. Began: Boundary equilibrium generative adversarial networks. 2017.
arXiv preprint
M. Besserve, N. Shajarisales. B. Schölkopf, and D. Janzing. Group invariance principles for causal generative models. In AISTATS, 2018a.
M. Besserve, R. Sun, and B. Schölkopf. Intrinsic disentanglement: an invariance view for deep generative models. In Workshop on Theoretical Foundations and Applications of Deep Generative Models at ICML, 2018b.
A. Brock, J. Donahue, and K. Simonyan. Large scale GAN training for high fidelity natural image synthesis. arXiv preprint arXiv: 1809. 1096, 2018.
Tian Qi Chen, Xuechen Li, Roger B Grosse, and David K Duvenaud. Isolating sources of disentan- glement in variational autoencoders. In Advances in Neural Information Processing Systems, pp. 2610-2620, 2018.
X. Chen, Y. Duan, R. Houthooft, J. Schulman, I. Sutskever, and P. Abbeel. Infogan: Interpretable representation learning by information maximizing generative adversarial nets. In Advances in Neural Information Processing Systems, pp. 2172-2180, 2016.
M. Diop, A. Larue, S. Miron, and D. Brie. A post-nonlinear mixture model approach to binary matrix factorization. In 25th European Signal Processing Conference (EUSIPCO), pp. 321-325, IEEE, 2017.
A. Dosovitskiy and T. Brox. Inverting visual representations with convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4829-4837, 2016.
N. Dvornik, J. Mairal, and C. Schmid. Modeling visual context is key augmenting object detection datasets. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 364-380, 2018.
R. C. Fong and A. Vedaldi. Interpretable explanations of black boxes by meaningful perturbation. arXiv preprint arXiv: 1704:0370 2017.
L. A. Gatys, A. S. Ecker, and M. Bethge. A neural algorithm of artistic style. arXiv preprint arXiv: 150..0/500005700 2015.
I. Goodfellow, J. Pouget-Abadic M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In Advances in neural information processing systems, pp. 2672-2688 2014.
K. Grill- and R. Malach. The human visual cortex. Annu. Rev. Neurosci., 27:649-677, 2004.
M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, G. Klambauer, and S. Hochreiter. Gans trained by two time-scale update rule converge to nash equilibrium. arXiv preprint 1:Xiv:776.008000 2017.
I. Higgins, L. Matthey, A. Pal, C. Burgess, Glorot, M. Botvinick, Mohamed, and A. Lerchner. beta- VAE: Learning basic visual concepts with constrained variational framework. In ICLR 2017, 2017.
I. Higgins, D. Amos, D. Pfau, S. Racaniere, L. Matthey, D. Rezende, and A. Lerchner. Towards a definition of disentangled representations. arXiv preprint arXiv: 2018.
G. W. Imbens and D. B. Rubin. Causal inference in statistics, social, and biomedical sciences. Cambridge University Press, 2015.
D. P. Kingma and M. Welling. Auto-encoding variational bayes. arXiv preprint arXiv: 1312,61 2013.
T. D. Kulkarni, W. F. Whitney, P. Kohli, and J. Tenenbaum. Deep convolutional inverse graphics network. In NIPS, pp. 2539-2547, 2015.

---

Published as a conference paper at ICLR 2020
D. D. Lee and H. S. Seung. Learning the parts of objects by non-negative matrix factorization. Nature, 401(6755) 1999.
Z. Liu, P. Luo, X. Wang, and X. Tang. Deep learning face attributes in the wild. In Proceedings of International Conference Computer Vision (ICCV), 2015.
Locatello, S. Bauer, M. Lucic, Gelly, B. Schölkopf, and O. Bachem. Challenging com- mon assumptions in the unsupervised learning of disentangled representations. arXiv preprint arXiv: 2018.
M. F. Mathieu, Zhao, A. Ramesh, P. Sprechmann, and Y. LeCun. Disentangling factors of variation in deep representation using adversarial training. In Advances in Neural Information Processing Systems, pp. 5041-5049, 2016.
G. Parascandolo, N. Kilbertus, M. Mojas-Carulle and B. Schölkopf. Learning independent causal mechanisms. In ICML, pp. 4036-4044, 2018.
Pearl. Causality, Cambridge university press, second edition, 2009.
Pearl. The causal foundations of structural equation modeling. In Rick Hoyle (ed.), Handbook of Structural Equation Modeling. chapter 5. Guilford Press, 2014.
J. Peters, D. Janzing, and B. Schölkopf. Elements of Causal Inference Foundations and Learning Algorithms. MIT Press, 2017.
A. Radford, L. Metz, and S. Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint 15 2015.
D. J. Rezende, S. Mohamed, D. Wierstra. Stochastic backpropagation and approximate inference in deep generative models. arXiv preprint arXiv: 2014.
M. S. M. Sajjadi, O. Bachem, M. Lucic, O. Bousquet, and S. Gelly. Assessing generative models via precision and recall. arXiv preprint 1eXXiv:800600000 2018.
R. Suter, D. Miladinović, B. Schölkopf, and S. Bauer. Robustly disentangled causal mechanisms: Validating deep representations for interventional robustness. arXiv preprint 181 11////111000000 1.00007, 2018.
M. D. Zeiler and R. Fergus. Visualizing and understanding convolutional networks. In European conference on computer vision, pp. 818-833. Springer, 2014.
Q. Zhang, R. Cao, Y. Wu, and S. Zhu. Growing interpretable part graphs on convnets via multi-shot learning. In AAAI, pp. 2898-2906, 2017a.
Q. Zhang, W. Wang, and S. Zhu. Examining CNN representations with respect dataset bias. arXiv preprint 1rXiv:1710.1051 2017b.
12

---

Published as a conference paper at ICLR 2020
Supplementary information
APPENDIX A: FORMAL DEFINITIONS AND RESULTS FOR CAUSAL GENERATIVE MODELS (SECTION 2)
We rely on the assumption that a trained generator architecture can be exploited as a mechanistic model, such that parts of this model can be manipulated independently. A mathematical representation of such models can be given using structural causal models (SCMs, that rely on structural equations (SEs) of the form Y = f(x1, X2, XN,c). c). denoting the assignment of a value to variable Y, computed from the values of other variables X₂ in the system under consideration, and of putative exogenous influences imposed by factors outside the system under study, As in the above equation, we will use uppercase letters to indicate variables being the outcome of a structural assignment, while specific values taken by them will be lower case. SEs stay valid even if right- side variables undergo a change due to interventions (Pearl, 2009 Peters et al., 2017 e.g.), and can model the operations performed in computational graphs of modern neural network implementations. Such graphs then depict SCMs made of interdependent modules, for which assignments' dependencies are represented by directed acyclic graph G. Without loss of generality, we introduce a Causal Generative Model (CGM) M capturing the computational relations between a selected subset of variables comprising: (1) the input latent variables {= (2) the generator's output Y (typically multi- endogenous (internal) variables forming an intermediate representation such that the generator's output can be decomposed into two successive steps as } + {Vr} + Y. In feed-forward neural network, one Ve may for instance represent one channel of the output of a convolutional layer (e.g. after application of the ReLU non- -on-lincar.
dimensional), and (3) collection of possibly
Definition 6 (Causal Generative Model (CGM)). Given K real-valued latent variables = (zz) taking arbitrary values on domain Z = II Zk. where all Z 's are closed intervals, the CGM M = G(Z,,,
S, 9) comprises a directed acyelic graph G and a set S of N deterministic continuous structural equations that assign:
N endogenous variables {Vx = fa((aa) taking values in Euclidean spaces (V*))==1.)
based on their endogenous or latent parents Pay in G,7 one output Y == fg taking values in Euclidean space y, parents pa,, 9 being endogenous. Moreover, z are the only sources and Y is the only sink.
The graph of an example CGM is exemplified on Fig. 2b consisting of 3 endogenous variables, 2 latent inputs and the output. This aligns with the definition of a deterministic structural causal model by Pearl [2009] chapter 7), once our latent variables are identified with exogenous ones. CGMs have however specificities reflecting the structure of models encountered in practice. For instance, variable assignments may may not involve latent/exogenous variables in their right-hand side, which is unusual in causal inference. This allows modeling feed-forward networks consisting in a first layer receiving latent inputs followed by cascade of deterministic operations in downstream layers. The above definition guaranties several basic properties found in the computational graph of existing generative networks: (I) all endogenous variables V&8 are unambiguously assigned once is chosen, (2) the output Y is unambiguously assigned once either z is chosen, or, alternatively, if an appropriate subset of V 's, such as Pay. is assigned. This allows us to introduce several useful mappings.
In an ideal case, while the support of the latent distribution covers the whole latent space Z, internal variables and outputs typically live on manifolds of smaller dimension than their am- bient space. These can be defined as the images" of Z by operations of the graph: the out- put image Y'M = Y(Z], the endogenous images 1 Vi[Z] for single variable, v = u E I D*: v = (va(=))aEE, z E 2 for subset of variables indexed by E, and VM when & includes all endogenous variables.
7 "A a parent of (child) B whenever there is arrow A B.
8 "Sources are parentless nodes, sinks are childless.
An image f[A] is the subset of outputs of for subset A of input values.
13

---

Published as a conference paper at ICLR 2020
Functions assigning Y from latent variables and from endogenous variables, respectively,
9M 2
VM- (()
VM
VM. Y(v),
and GM
and we call them latent and endogenous mappings, respectively. Given the typical choice of dimen- sions for latent and endogenous variables, the VE's and Y are constrained to take values in subsets of their cuclidean ambient space. We will assume that JM and gm define proper embeddings, in particular implying that they are both invertible. We call CGM satisfying these assumptions embedded CGM.
With this vocabulary we can for example verify the example of Fig. 2b contains exactly two layers (in green). Note IM and ğas are well defined because the output can be unambiguously computed from their inputs by successive assignments along G, and are both surjective due to appropriate choices for domains and codomains. All defined image sets V Vas ...) constrained by the parameters of M, and are typically not easy to characterize. For example V M is likely a strict subset of the Cartesian product II. V/.-
Importantly, the image set VMM of a trained model is of particular significance, as it should approximate model. Learning the generator parameters such
best the support of the data distribution we that VM precisely matches the support of the target data distribution is arguably a major goal for generative models (see e.g. Sajjadi et al. (2018)).
As we will manipulate properties of the output, we restrict ourselves to transformations that respect the topology of Vas. and use embeddings as the basic structure for it, allowing inversion of 9M- Definition (Embedded CGMs). If X Y is a continuous injective function with continuous inverse f : [[X] + Y, we call f an embedding of X in Y. We say that a CGM M is embedded if 9M and 9M are respective embeddings of Z and V Vs in y
Since Definition 6 imposes continuous structural equations, which is satisfied for all operations in standard generative models, injectivity of gM is the key additional requirement for embedded CGMs.
Proposition 4. If Z of CGM M is compact (all Z 's are bounded), then M is embedded if and only 9M is injective.
Proof is provided in Appendix B. This implies that generative models based on uniformly distributed latent variables (the case of many GANs), provided they are injective, are embedded CGMs. While VAEs' latent space is typically not compact (due to the use of normally distributed latent variables), we argue that restricting it to a product of compact intervals (covering most of the probability mass) will result in an embedded CGM that approximates the original one for most samples.
Based on this precise framework, we can now provide the formal definitions and results described informally in main text.
The CGM framework allows defining counterfactuals in the network following Pearl 2014).
Definition 8 (Unit level counterfactual). Given CGM M, for subset of endogenous variables & = {ei,.., ), and assignment h of these variables, we define the interventional CGM Mn. obtained by replacing structural assignments for Vie by assignments (Ve, == hy(2))eaeE- Then for given value of the latent variables, called unit, the unit-level counterfactual is the output of Ma: Y (z) 9as. (z).
Definition 8 also in line with the concept of potential outcome (Imbens & Rubin 2015). Importantly, conterfactuals induce transformation of the output of the generative model.
Definition 2 (Counterfactual mapping). Given an embedded CGM, we call the continuous map
Y Pn:ymN Yλ (9Ml (v))
E the (E, cocootttefaactual mapping. We say it is faithful to M whenever Y, [Dm]] C VM--
Our approach then relates counterfactuals to a form of disentanglement allowing transformations of the internal variables of the network as follows.
14

---

Published as a conference paper at ICLR 2020
Definition 3 (Intrinsic disentanglement). In CGM M. endomorphism T Vas - VMs is intrinsi- cally disentangled with respect to a subset of endogenous variables, if exists a transformation T of endogenous variables such that for any latent E Z, leading to the tuple of values V
T(Y())) = (T))
where (v) only affects the variables indexed by E.
In this definition, (v)) corresponds to the unambiguous assignment of Y based on endogenous values. Fig. 2d illustrate this second notion of disentanglement, where the split node indicates that the value of V2 is computed as in the original CGM (Fig. 2b) before applying transformation T to the outcome.
Intrinsic disentanglement relates to causal interpretation of the generative model's structure in the sense that expresses a form of robustness to perturbation of one of its subsystems. Counterfactuals represent examples of such perturbations, and such, may be disentangled given their faithfulness. Proposition (Counterfactuuls and disentanglement, formal). For an embedded CGM and A continuous assignment h, the (E, ho)couunerfuuctual mapping Y,, is faithful if and only if it is intrinsically disentangled with respect to subset &, Moreover, if h[2] C V Ma it is sufficient that o.E.
and & do not have common latent ancestors for Y to be faithful.
The proof is provided in Appendix B.
APPENDIX B: ADDITIONAL DETAILS FOR SECTION 2 AND 3
TOPOLOGICAL CONCEPTS
Continuity. We recall the classical definition of continuity of f Y. Given the respective topologies and of domain and codomain (the sets of all open sets), is continuous whenever for all ry, J- (A) E TX
Euclidean topology. For defining continuity between Euclidean spaces, we rely on the Euclidean (or standard) topology naturally induced by the metrie: as set is open if and only if contains and open ball around each if its points.
Subset topology. When restricting the domain or codomain of a mapping to a subset A of the Fuclidean space, we rely on the subspace topology, that consists in the intersection of A will all open sets.
PROOF OF PROPOSITION
Following result stated in Armstrong (2013), since Z is compact and the codomain of gM is Hausdorff (because Fuclidean). then a continuous (by definition) and injective gM is an embedding. In addition, 9M injective implies Thu 's are injective on their respective domains V Moreover, the V 's being image of compact Z by a continuous mapping (by the CGM definition), they are compact, such that the respective gh are also embeddings.
PROOF PROPOSITION
Part 1: Proof of the equivalence between faithful and disentangled. One conditional is trivial: if a transformation is disentangled, it is by definition an endomorphism of VMM so the counterfactual mapping must be faithful.
A
For the second conditional, let us assume a faithful Y, and denote the (unambiguous) map from V the output
y Y(v)
YM
Note the mapping # + (w)) differs from Dis because its domain is restricted the image set V
15

---

Published as a conference paper at ICLR 2020
This map differs from gMs due to its broader domain and codomain, such that it is neither necessarily an injection nor a surjection, but they coincide on the image VM- We can first notice (using Definition & and the embedding property) that the counterfactual mapping can be decomposed
Y h YM o T" (gM))" where T' v Ve Ü v)
v x V (ö, h)
A Since Y, h. is faithful, it is then endomorphism of it Vas (continuity comes form the composition of continuous functions), as required by the definition of disentanglement.
For any E V, consider then the quantity
E (Y())) = =Y (w))
using the above decomposition, we rewrite it as
E (Y YM T' (îM) o (v) YM o (v));
where T is transformation that only affects endogenous variables in E, demonstrating that E
Y h (Y()) is disentangled with respect to E.
Part 2: Sufficient condition. This a direct application the following Proposition 5 after observing that our case, the endomorphism T required in this proposition is the constant function with value h.
Proposition For embedded CGM M, if a subset & of endogenous variables does not share common latent ancestors" with the reminder of endogenous variables E, then any endomorphism T : + V leads to transformation
T T: 9M T' (9M)
V V
v x D
with TV:
+
(ö, v)
(ö, T (v))
such that T is disentangled with respect & in M.
Proof of Proposition 5 The absence of common latent ancestor between & and & ensures that values in both subsets are unambiguously assigned by non-overlapping subsets of latent variables, A and B respectively, such that write
(V ViE) - (fA(ZA), fB(=B))
This implies that the image set of this layer fully covers the Cartesian product of the image sets of the two subsets of variables, i.e. VM v x y and guaranties that T' and endomorphism of V for any choice of endomorphism T This further implies T is well defined and an endomorphism. L
MM
PROOF SKETCH FOR PROPOSITION
3
Due to the i..d. assumption for components of Z and the structure following the sufficient condition of Prop. is clear that the subsets of endogenous variables associated to each V₂ are modular and the associated partition of the hidden layer is disentangled representation. The choice of increasing dimensions as well as the i..d. sampling of the model parameters from a distribution with a density make sure the resulting mapping is injective (and hence follow the embedded CGM assumptions of Def. 7 and that counterfactual hybridization of any component of V/, will result in an influence map whose support covers exactly Ig. Finally, the conditions on the I and the thresholding approach guaranties rank K binary factorization of the matrix B, with one factor gathering the indicator vectors associated to each V2 and the uniqueness this factorization is guaranteed by classical NMF identifiability results, e.g. following (Diop et al., (017)) Theorem III.1].
A ancestor of B whenever there is directed path A .. - ....-
the graph
16

---

Published as a conference paper at ICLR 2020
APPENDIX C: ARCHITECTURE DETAILS
VANILLA 3-VAE AND DCGAN
The 3-VAE architecture is presented in Supplementary Fig. 7 and is very similar the DCGAN architecture. Hyperparameters for both structures are specified in Table
BEGAN CELEBA
We used the method proposed in Berthelot et al. 2017) for CelebA dataset. We used the pre-trained model with the same architecture as was used in the paper. It consists of three blocks of convolutional layers each followed by an upsampling layer. The filter size of convolutional layers is (3.3) 3) all over the generator. There is also skip connections in the model that is argued to increase the sharpness of images. Consult (Berthelot et al., 2017, Figure 1) for architectural details.
BIGGAN-DEEP ARCHITECTURE DETAILS
The pretrained model taken from Tensorflow-hub (actpas//tthcb.das we summarize below the main aspects of the architectures. We used the BigGan-deep architecture of Brock et al. (2018) as a pre-trained model on 256x256 ImageNet. We did retrain the model. The architecture consists of several ResBlocks which are the building block of the generator. Each ResBlock contains CetinNorm-RLLU-OOE Layers followed by upsampling transformations and augmented with skip connections that bring fresh signal from the input to every ResBlock. Consult Brock et al. (018) for architectural details.
CLASSIFIERS ARCHITECTURE DETAILS
All pretrained models are taken from Tensorflow-hub (sstpe///tthub.der main aspects of the architectures.
we summarize below the
INCEPTION RESNET. V2
Inception ResNet V2 is neural network architecture for image classification, originally published by
Christian Szegedy, Sergey loffe, Vincent Vanhoucke, Alex Alemi: "Inception-v4,
Inception-Ress ResNeet and the Impact of Residual Connections Learning". 2016.
INCEPTION_V3
Inception V3 is neural network architecture for image classification, originally published by
Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, Zbigniew Wojna: "Rethinking the Inception Architecture for Computer Vision", 2015.
NASNET_LARGE
NASNet-A a family of convolutional neural networks for image classification. The architecture of its convolutional cells (or layers) has been found by Neural Architecture Search (NAS). NAS and NASNet were originally published by
Barret Zoph, Quoe V. Le: "Neural Architecture Search with Reinforcement Learn ing". 2017. Barret Zoph, Vijay Vasudevan, Jonathon Shlens, Quoc V. Le: "Learning Transferable Architectures for Scalable Image Recognition", 2017.
NASNets come in various sizes. This TF-Hub module uses the TF-Slim implementation lasnet of NASNet-A for ImageNet that uses 18 Normal Cells, starting with 168 convolutional filters (after the "emageNet stem"). It has input size of 331x331 pixels.
17

---

Published as a conference paper at ICLR 2020
RESNET- RESNET-V2.50
ResNet V2 is family of network architectures for image classification with a variable number of layers. It builds on the ResNet architecture originally published by
Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun: "Deep Residual Learning for Image Recognition", 2015.
The full preactivation 'V2' variant of ResNet used in this module was introduced by
Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun: "Identity Mappings in Deep Residual Networks", 2016.
APPENDIX D: IMPLEMENTATION DETAILS.
The file influence.py provided
the
link htt=p:://www.drobbox dwwwwwdrobb...oo com/ sh/
4 htnnjittmba2soq/// AAAA 6 6900c (200009224400 (4440aaad=1 contains key elements of
the code for counterfactual analysis of generative models.
APPENDIX E: ADDITIONAL RESULTS
INFLUENCE MAP CLUSTERING AND HIBRIIIIZAIIN IN DCGANS
We replicated the above approach for GANs on the CelebA dataset. The result shown in Supplemental Fig. summarize the main differences. First, the use of three clusters seemed again optimal according to the stability of the obtained cluster templates. However, we observed that the eyes and mouth location were associated with the top of the head in one cluster, while the rest of the face and the sides of the image (including hair and background) respectively form the two remaining clusters. In this sense, the GAN clusters are less aligned with high level concepts reflecting the causal structure of these images. However, such clustering still allows a good visual quality of hybrid samples.
APPENDIX F: ADDITIONAL DISCUSSION
EXTENSION AND RELATION TO OTHER FRAMEWORKS
While defined disentanglement of transformations, this concept has been classically attributed to a representation, or factors of variation. Focusing on a transformation aligns to our aim of providing an agnostic definition, in the sense that it only relies the generator architecture and given transformation, but neither on data, nor properties that are not stated explicitly in the definition. In contrast, disentangled representation is usually understood as intervening on meaningfu//ntterrecabble features in the image, which may be subjective, or at least referring to some external knowledge. Alternatively, one might expect that what should be disentangled a property or a factor of variation (say "hair color"). We argue that we can state property is disentangled by the generalizing our notion of disentanglement to a family of transformations as follows.
Definition (Disentangled family). family of transformations (possibly parametric) disentangled if all members are disentangled with respect to the same E.
Then property may be disentangled if the class of all transformations changing "only" the value of this specific property is disentangled according to Definition 9 Finally, one might also expect that several transformations (or families of transformations) should be disentangled with respect to each other, e.g. allowing state that hair color should be disentangled from hair length. This relative disentangiement is easily defined based on our original definition.
Definition 10 (Relative disentanglement). The families of functions (Fi)ieL..N are called jointly disentangled whenever they are disentangled with respect to non-overlg subsets of variables (EI)EEL.N within a given layer. 12
or within the latent space for extrinsic disentanglement
18

---

Published as a conference paper at ICLR 2020
We argue that the notion introduced in Higgins et al. (2018) be framed special case of Definition I0
SUPPLEMENTARY FIGURES AND TABLES
Architecture
VAE CelebA 4/664,64,3221663) (8.16,32.64)
GAN CelebA 4/(128,64,32,
Nb. of deconv. layers/channels of generator
Size of activation maps of generator
(4,8,16,32)
Latent space
128
150
Optimization algorithm
Adam (8 0.5)
Adam (3 0.5)
Minimized objective
VAE loss (Gaussian posteriors)
GAN loss
batch size
64
64
Beta parameter
0.0005
NA
Table
Table FID analysis of BEGAN hybrids. Distance between different pairs of classes (R: Real data, G: Generated data, Ck: Hybrids by intervention on cluster k). The distances are is computed for balanced number of examples (10k) for each class and normalized by the FID between the real data and the generated data. It can be seen in the table that Hybrids have small distance to the generated and also to each other. This can be interpreted as closeness of the distribution of Hybrids to that of generated data suggesting that Hybridization produces visually plausible images.
G
0
CO
1.022
0.036
0
C1
1.143
0.093
0.096
C2
1.158
0.097
0.099
0.045
R
G
CO
C1
C2
Decoder
Encoder Image
Image level
19
Fine level 22
Intermediate level
Coarse level
Latent var.
FC
Posterior
FC
Figure 7: VAE architecture. FC indicates fully connected layer, is 100-dimensional isotropic Gaussian vector, horizontal dimensions indicate the number of channels of each layer. The output image size is 64 64 (or 32 32 for cifar pixels and these dimensions drop by factor from layer to layer. (reproduced from RRafford et al. 2015)
19

---

Published as a conference paper at ICLR 2020
Cluster bemplittes
infuuence made
Urgin
Hybrid
Unignal
lastr laye22 layer
Cueer
Artign Amm.g--eeoeeie
Chndar
0.-
Clasee
layer2
bytt
1ayer3
Nember charnen modile
Figure 8: Left: Clustering of influence maps for VAE trained the CelebA dataset (see text)... Center: samples of the hybridization procedure using as module all channels of the intermediate layer belonging the cluster of corresponding row, top insets indicate the original ImageNet pictures to produce both samples by feeding them to the VAE's encoder. Right: magnitude of causal effects. (top: average influence modules derived from clustering as a function of the number of clusters; bottom:
individual influence of each modules, as function of the number of channels they contain, dashed line indicate linear regression).
0.5
63-
Coster
0.85
87
083
03-
070
Number clusters
Number clusters
Figure 9: Label consistency (left) and cosine similarity (right) of the clustering of influence maps for the NMF and k-means algorithm. Errrrars indicate standard deviation across 20 repetitions.
20

---

Published as a conference paper at ICLR 2020
templates
Influence meps
Orginal
Hybrid
Orignal 2
Figure 10: Clustering of influence maps and generation of hybrid samples for a VAE trained on the CelebA dataset (see text).
Cluster templates
Infuence maps
Orgina
Hybrid
Orgnal
Figure 11: Clustering of influence maps and generation of hybrid samples for GAN trained on the CelebA dataset (see text).
21

---

Published as a conference paper at ICLR 2020
Original
Original
Hybrids
Modules
Latent mixing
Original 2
Original
Hybrids
Modules
Latent mixing
Original
Original
Hybrids
Modules
Latent mixing
Figure 12: Larger collection of hybrids for the BEGAN. Modules are fixed and extracted by the NMF algorithm using clusters. For each three panel, the "Original 2" sample used to perform the intervention is fixed (show in the top-left corner) while Original I varies along the columns.
22

---

Published as a conference paper at ICLR 2020
Original 1
Original
Gblock 6
Gblock 4
Hybrids
Original 2 Original
Original Original
Gblock 7
Gblock
Hybrid
2
Original 2
Original
Figure 13: Larger collection of hybrids for the BIGAN, between classes "teddy" and "koala". Each panel corresponds to intervening on a different Gblock (from 4 7). Modules are fixed and extracted by the NMF algorithm using 3 clusters. Each row of hybrids corresponds to interventions on one of the extracted module, numbered on the right-hand side.
23

---

Published as a conference paper at ICLR 2020
Gblock 4
Gblock
Gblock 6
Gblook
Modulo
50
20
Modile 2
50-
100
200
Module
30 25 20
Figure 14: Histogram of the entropy of the resnet_v2.55 classifier logits corresponding to the
experiment of Fig. 13 Columns indicate intervened Gblock (from 4 to 7), rows indicate module (as ordered in Fig. 13p. The entropy is computed using the probabilistic output for the 10 classes receiving top ranking across all hybrids, normalized provide total probability of In particular for Gblock (left column) we can see that the module with poorer quality leads to larger entropy values. Interestingly, entropy values are also much smaller for hybrids based interventions the (more abstract level) Gblock number 4. Overall, the results suggests that object texture, which is well rendered in hybrids generated from Gblock 4, is key information for the classifier's decision.
24

---

Published as a conference paper at ICLR 2020
Original
Original
Gblock
Gblock
Spuads
Original Original
Original
Gblock 7
Gblock
50/0/H
Original
Original
Figure 15: Larger collection of hybrids for the BIGAN, between classes "cock" and "ostrich". Each panel corresponds intervening on a different Gblock (from 4 7). Modules are fixed and extracted by the NMF algorithm using 3 clusters. Each row of hybrids corresponds to interventions on one of the extracted module, numbered on the right-hand side.
Gblock
Gblock 5
Gloock 6
Gblock 7
100
250
Modole
0.2
2.0
1.0
300
Module
100
2.00
0.5
0.00
6.75
0.5
0.5
8-2
200
150
53
Figure 16: Histogram of the entropy of the resnet_v2_50 classifier logits corresponding to the Columns indicate intervened Gblock (from 4 to 7), rows indicate module The entropy is computed using the probabilistic output for the 10 classes receiving top ranking across all hybrids, normalized to provide a total probability 1. Interestingly, large entropy is obtained for first module of Gblock 5 (middle column), consistent with the fact the intervention generates hybrid bird, mixing shape properties of both cock and ostrich.
experiment Fig. 15
(as ordered in Fig. 15
25

---

Published as a conference paper at ICLR 2020
Model
resnet.v2.50
inception_v3 koala
nasnet large
inception inception.resmet..v resseetve
Left Image
Output
koala
koala
koala
Model
resnet_v2.55 -veet_22500
inception_v3
nasnet_large
resnet_v2 teddy
Middle Image
Output
koala
koala
teddy
Model
resnet.v2.55
inception_v3 teddy
nasnet.aare
inception_resnet.e
Right Image
Output
koala
teddy
koala
Table 3: The classification outcome of several discriminative models for three randomly chosen koala+teddy hybrids (see Figure. [7]. The purpose of this experiment is to investigate the use of the proposed intervention procedure for assessing robustness of classifiers. As can be seen in the following images, the resultant hybrids are roughly teddy bear in koala context. An ideal classifier must be sensitive to the object present the scene not the contextual information. teddy bear must still be classified as a teddy bear even if it appears on tree which is the koala environment in most of the koala images in the ImageNet dataset. It can seen the following table that nasnet_larg is more robust to the change of context compared to other classifiers.
Figure 17: Three koala+teddy hybrids as the inputs to the classifiers of Table.
26


Reviews:
[{"review_id": "rkeRx2DpFB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper presents a means to uncover the modular structure of deep generative models of images using counterfactuals and presenting evidence for the fact that there are interpretable modules within current popular generative models. The paper is extremely well written with a good balance between mathematical notation and intuitive explanations. I think this paper should certainly be accepted as it provides an interesting and rigorous tool to understand the behavior and properties of deep generative.\n\nI have a few questions and comments:\n\n1) How early does this sort of modularity arise over the course of training? Does it vary for GANs versus beta-vae like models?\n\n2) I think it would be good to contrast this approach with ones that also have latent mixing strategies such as [1, 2] - these do not uncover latent modular structure, but can also produce hybridized images via linear latent space mixing unlike counterfactual assignment of latent vectors like in your work.\n\n3) From what I gather, this approach cannot produce hybridizations between two explicitly provided samples from a dataset, since the models you consider do not have an inference network like in BiGAN or ALI to get the corresponding z vector for a specific image. If this were available, it would be interesting to study influence maps estimated by taking the expectation over pairs of images rather than z-space vectors. \n\n4) Can we quantify modularity or the extent of disentanglement of internal representations under such a framework?\n\nReferences\n\n[1] Manifold Mixup - https://arxiv.org/pdf/1806.05236.pdf\n[2] Adversarial Mixup Resynthesis - https://arxiv.org/pdf/1903.02709v3.pdf\n\nMinor:\n\nProposition 2 - typo - “then and transformation applied to it”", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "BJxcC6MWjH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\n    Authors propose a method to analyze a trained generator (a neural network) from a trained generative model for a distribution of images and produce a family of modules (subsets of neurons) that control certain aspects of the image (upon intervention) such that the different modules identified are \"distentangled\" in the sense of a lot of work in this area attempting to find latent controllable factors.\n\nThis paper has many original ideas and substantial novelty. One of the prime ones is to focus on existing generators and identify subsets of neurons \"inside\" the generator box that can be manipulated producing meaningful changes in the image. All existing works attempt to constrain the latent space in an appropriate way during training so as to produce different latent factors that manipulate different aspects of the image. Authors point out that disentangled latents cannot be statistically independent. This is very important (and motivates them to group neurons in the model that are statistically dependent) and I cannot agree more with the authors on this point.\n\nTransformation of the image is defined to be intrinsically disentangled if the same transformation can be produced in the image by intervening (transforming) only a small subset of neurons.\n\nAuthors assume that the map from z (latents) to image space (g_M) and map from neurons space to image space induced by the generator (\\tilde{g}_M) are invertible .\n\nAuthors define a class of manipulations of certain subsets of neurons through counterfactuals. Setting the latents to be the same (unique) z that gave rise to the image Y, just intervene on a susbet E of neurons and set them to h (from the valid range of values it can take) - the result of this intervention for this unit (z) produces a counterfactual image corresponding to y. Now, if the counterfactuals corresponding to the manifold of images map back into the manifold, then it is called a faithful counterfactual.\n\nAuthors show a very interesting result - Transformation is disentangled if and only if it is produced by a faithful counterfactual mapping. There is an additional result relating modularity of a subset of neurons to disentanglement.\n\nThen authors produce a concrete algorithm inspired by faithful counterfactual mapping as follows. The authors take two images and two latents z_1 and z_2 that produced them. Then they identify a specific neuron under z_1 , then swap that neuron with the value of the neuron from z_2 with everything else remaining the same under z_1. Then they look at the difference in the images and average across all channels. Then they average over all pairs z_1 and z_2 - this representative \"image with one channel\" is the average effect of manipulating that neuron.\n\nThen, using nonnegative matrix factorization on the thresholded versions of the average effects of all neurons in a layer, they cluster neurons and then they actually form modules. So now each module can be manipulated together when two images are taken together and neurons in the modules are swapped from one into the other.\n\nPros:\n  I really like the key idea of this paper - counterfactually manipulating neurons using its values from that of some other image and observing differences and clustering them to find similar neurons which could be manipulated as a bunch. Some of the results (although seems handpicked) seem pretty good given this is the first work to group neurons inside a pre-trained generator. I have not seen any work on \"intrinsic disentanglement\" before. This also has additional implications for people interested in GANs. The fact that pre-trained GANs can lead to counterfactually \"realistic\" images when \"concepts\" (clustered neurons of this method)  are swapped shows that GANs really do learn something non-trivial about images. \n\nI highly recommend this paper for acceptance. However I would like the authors to address some of my concerns. \n\nCons:\na)  Typo in Proposition 2 - \"then and transformation..\" - and should not be there.\nb) v_0 is used in some cases - h(z) is used in some places and h (being a constant vector) is used in some places to define counterfactual mapping. Its pretty confusing to read some of the proofs. Is it possible to uniformly define it throughout with constant h and then of course only during experiments and hybridization replace h by v(z_2) (the v from image to be swapped with).\nc) Section 3.4 - first paragraph - last line. \"Influence maps are grouped by similarity to define modules..\" - This is vague. Does this refer to the procedure in the third paragraph of the same section? If so this line could be make to point to this more precise description that comes later.\nd) So there are many synthetic experiments where the same pair of images is used to produce many counterfactual images by hybridizing under manipulation of different modules (returned by the clustering algorithm)\nWhat will be useful is for one VAE or GAN ,demonstrate the changes on multiple pairs of images. It seems the image pair is handpicked along with the clustered modules. It would be good to fix the modules and illustrate changes for multiple images using the same set of modules.\n\ne) It seems that the clustered influence maps are simplistic - representing face region, hair region and background. However, ideally we would like to find out - makeup or not, gender , hair color, bangs or not - these are various attributes of the celebA dataset that u get annotated.\nIs it possible to correlate obtained modules/influence maps with these annotations already available to find finer concepts that are represented by these annotations ? \n\nOne way to find out is - take a module or a neuron - manipulate it by hybridizing and produce a hybrid image from a pair of original images, build a classifier for one of these annotated labels (attributes) on the CelebA dataset .Then do inference on the hybridized image versus the original two images - if the prediction confidence of the attribute classifier changed across original images and also between the hybrid and one of the original- we know that module/neuron is highly correlated with this attribute. One could build a confusion matrix (sort of) between annotated attributes and all elementary (neurons or modules). This could easily tell us if indeed modules/neurons are capturing all attributes we would like. \n\n\n\n\n\n\n\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "BJlEdHcYqH", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The ideas presented in the paper are interesting and original. Whereas the theory presented has a lot of potential, it seems that the clarity of the paper could be greatly improved, in particular I would have liked more of the formal theory to be included in the body of the paper instead of relying only on the appendix. This especially matters since the theoretical aspect is a key contribution of the paper and the experimental section remains on the light side (it presents mostly examples and lacks more extensive results). \n\nI find the introduction of the proposed definition of disentanglement in sections 2.2/2.3 confusing. The authors first define “extrinsic” disentanglement of a transformation in the data space as corresponding to a transformation of one dimension only in the latent space. In section 2.3 a transformation is called “intrinsically” disentangled if it corresponds to a transformation of a subset of variables in the space of endogenous variables. It should be made clearer from the start that disentanglement is here only a property of the transformations and that the authors are not trying to define a disentangled internal representation. Further, some important questions like how to choose the reference endogenous variables and how to choose the subset E are left entirely to the experiments section. The definition of disentanglement proposed is however tied to these choices and a quick discussion would be helpful. \n\nWhereas the theory from section 2 seems precise and formal (at least in the appendix, although I did not check all the proofs), the procedure to identify modules comes with no guarantees and relies on several choices: local averaging, thresholding, nbr of clusters (the choice of this one is in my opinion well justified). Taking that into account a more extensive experimental validation would be needed to demonstrate that modules can be reliably identified. The results presented on CelebA and ImageNet are interesting, in particular using different models is a good idea, however the evaluation relies mostly on a few cases or examples and I would have liked to see more quantitative results, e.g. like in Figure 8 Appendix F. \n\nNote on related work:\nIt has been shown (Isolating Sources of Disentanglement in VAEs by Duvenaud et al., Disentangling by Factorising by Kim et al., Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations by Bachem et al.) that Beta-VAE is far from optimal for “extrinsic” disentanglement, the text in section 4.1 should take these results into account. It would also be interesting to contrast with the following paper: Robustly Disentangled Causal Mechanisms: Validating Deep Representations for Interventional Robustness by Bauer et al. which (whilst doing something pretty different) also treats of causality and disentanglement. \n\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper provides a fresh application of tools from causality theory to investigate modularity and disentanglement in learned deep generative models. It also goes one step further towards making these models more transparent by studying their internal components. While there is still margin for improving the experiments, I believe this paper is a timely contribution to the ICLR/ML community.
This paper has high-variance in the reviewer scores. But I believe the authors did a good job with the revision and rebuttal. I recommend acceptance.
Decision: Accept (Poster)