Tinkering with black boxes: counterfactuals uncover modularity in generative models

1 INTRODUCTION . Deep generative models have proven powerful in learning to design realistic images in a variety of complex domains ( handwritten digits , human faces , interior scenes ) . Complex neural architectures are now used to learn complex empirical data distributions by designing a non-linear function mapping a latent space to the space observations . In particular , two distinct approaches have recently emerged as state of the art : Generative Adversarial Networks ( GANs ) ( Goodfellow et al. , 2014 ) , and Variational Autoencoders ( VAEs ) ( Kingma and Welling , 2013 ; Rezende et al. , 2014 ) . Such architectures relate to a question that work in Neuroscience and Computer vision have long since tried to address : the relation between an observed scene and its high level internal representation . This has been framed using two objects : ( 1 ) the mapping of a 3D scene to its perceived ( 2D ) image , called forward optics , ( 2 ) the converse mapping , called inverse optics , ( see e.g . ( Kawato et al. , 1993 ) ) . Many computer vision algorithms have relied on inverse graphics approaches that model both forward and inverse optics simultaneously ( Kulkarni et al. , 2015 ) . In recent years , emphasis has been put on producing compact descriptions of the scene in terms of high level features reflecting a disentangled latent representation that can be mapped back to the image . However , the fundamental asymmetry between the structure of original forward and inverse optics mappings has received less attention . A key difference is that forward optics can be concisely described with a restricted set of equations taking into account physical parameters of the scene , while inverse optics does not have an explicit form and relies heavily on prior assumptions to be solved numerically . The simplicity of the forward optics may allow an agent to efficiently manipulate and update internal representations , for instance to plan interactions with the outside world , following a predictive coding principle ( Rao and Ballard , 1999 ) . This supports that modularity of generative models should be assessed and enforced in order to understand and manipulate representations . Achieving this aim for deep architectures is challenging , because they mostly behave as black boxes , making it difficult for users to interact with the generative process . Indeed , we can act on how the network is trained ( e.g . the optimized objective ) , what it learns to generate , but not on how the learned generative process operates . For example , to use a face generator to create a face combining the eyes of one generated face with remaining features of another one may be achieved by either additional training or complex manipulation of the network ’ s input or output . Directly influencing the generative process learned by the network on the other hand is made difficult due to the complexity of the function class entailed by the networks ’ non-linearities and high dimensional parameter space . To grasp the properties of such a system , a possible approach is to intervene on parts of the architecture that implements the generative function . Ideally , the effect of such interventions on the output would be interpretable . This suggests we should uncover a modular structure in those architectures , such that each part of a network can be assigned a specific function . In this paper , we propose that modularity can be quantified and exploited in a causal framework to infer whether modules within the architecture can be further disentangled . This hypothesis relies on the general principle of Independence of Mechanisms stating that the various mechanisms involved in generating the observed data can be modified individually without affecting each other ( Peters et al. , 2017 ) . It has been recently demonstrated that this principle can be applied to generative models encountered in machine learning Besserve et al . ( 2018 ) . One key aspect of causality frameworks is to allow evaluating with counterfactuals how the outcome of a observed system would have changed , provided some variables would have taken different values . We use such counterfactuals to assess the role of specific internal variables in the overall functioning of trained deep generative models and uncover the modular structure of these systems . We start by introducing this perspective formally with the notion of intrinsic disentanglement , and show that it extends the classical notion of disentangled representation investigated in the deep learning literature . Then , we introduce tools to analyze this disentanglement in existing systems . We show empirically how VAEs and GANs trained on a human face dataset express a form of modularity with intermediate activation maps responsible for encoding different parts of the generated images . Related work . The issue of interpretability in convolutional neural networks has been the topic of intensive investigation . Most of that research however has focused on discriminative neural networks , not generative ones . In the discriminative case , efforts have been made to find optimal activation patterns for filters ( Zeiler and Fergus ( 2014 ) , Dosovitskiy and Brox ( 2016 ) ) , to find correlation between intermediate feature space and data features ( Fong and Vedaldi ( 2017 ) , Zhang et al . ( 2017b ) ) or to disentangle patterns detected by various filters to compute an explanatory graph Zhang et al . ( 2017a ) . Furthermore , explicitly enforcing modularity in networks has been tried recently with Capsule networks architectures ( Sabour et al . ( 2017 ) ) , although Capsule network explicitly separate the architecture in different modules before training . A more detailed overview can found in review Zhang and Zhu ( 2018 ) . It is important to emphasize discriminative and generative processes differ significantly , and working on generative processes allows to directly observe the effect of changes in intermediate representations on the generated picture rather than having to correlate it back input images . The recent InfoGAN network ( Chen et al . ( 2016 ) ) and other works ( Mathieu et al . ( 2016 ) ; Kulkarni et al . ( 2015 ) ; Higgins et al . ( 2017 ) ) in disentanglement of latent variables in generative models can be seen as what we define as extrinsic disentanglement . As such , we believe our intrinsic disentanglement perspective should be complementary with such approaches and are not in direct competition . Finally our approach relates to modularity and invariance principles formulated in the field of causality , in particular as formalized by Besserve et al . ( 2018 ) . 2 MODULARITY AS INTRINSIC DISENTANGLEMENT . 2.1 CAUSAL GENERATIVE MODELS . In our aim to isolate a modular functional structure in deep networks , we first introduce a general formalism to perform interventions inside a generative network . We will rely on the notion of causal generative models to represent any latent variable model used to fit observational data . Causality entails the idea that discovered relationships between variables have some degree of robustness to perturbations of the system under consideration and as a consequence allows predicting interventions and counterfactuals . Causal models can be described based on Structural Equations ( SEs ) of the form Y : = f ( X1 , X2 , · · · , XN , ) , expressing the assignment of a value to variable Y based on the values of other variables Xk , with possibly additional exogenous effects accounted for through the random variable . This expression thus stays valid if something selectively changes on the right hand side variables , and accounts for the robustness or invariance to interventions and counterfactuals expected from causal models as opposed to purely probabilistic ones ( see for example Peters et al . ( 2017 ) ; Pearl ( 2009 ) ) . Such SEs can be combined to build a Structural Causal Model made of interdependent modules to represent a more complex system , for which dependencies between variables can be represented by a directed acyclic graph G. Let us use such structural model to represent our generator : Definition 1 ( Causal Generative Model ( CGM ) ) . A causal generative model M = G ( PZ , S , G ) consists of a distribution PZ over K latent variables Z = ( Zk ) that accounts for exogenous effects , a collection S of structural equations assigning endogenous random variables V = ( Vk ) and an output I based on values of their endogenous or latent parents Pak in the directed acyclic graph G. We assume I has no latent parent , such that it is assigned by at least two deterministic mappings : one using latent variables and one using endogenous variables I = gM ( Z ) = g̃M ( V ) . We assume PZ is such that the Zk ’ s take their respective values on intervals Zk ⊂ R , on which they have a non-vanishing density with respect to the Lebesgue measure , and are jointly independent . We define the image of M , IM = gM ( ∏ k=1 .. K Zk ) , i.e . the set of all possible objects generated by M . The graphical representation of a CGM is exemplified on Fig . 1a . This definition essentially aligns with the classical definition of a probabilistic Structural Causal Model ( SCM ) ( see e.g . Pearl ( 2009 ) ; Peters et al . ( 2017 ) ) and describes rigorously the functional structure of feed-forward generative models . Note that following Pearl ( 2009 , chapter 7 ) , we can exploit the fact that SCMs can been first defined as deterministic ( when exogenous variables are fixed ) , from which we can derive the mappings gM and g̃M . Simply combining this deterministic object with a probability distribution on exogenous variables leads to the classical probabilistic SCM definition . CGMs however have a few specificities with respect to classical SCMs . First , the latent variables correspond to the so-called exogenous variables in structural equations , and contrary to classical causal inference settings , these variables are presumably “ observed ” since a practitioner can simply access any variables pertaining to the generative model she uses . Second , the classically called “ observed variables ” , corresponding to the nodes of the graphical model , consist of two subsets : the output I ( that we consider here as a single multidimensional variable ) and the endogenous variables which are essentially internal variables corresponding to intermediate computations between the latent input and the output . We leave open the granularity of the partitioning of internal values of the network into endogenous variables : one variable Vi may for example represent the scalar activation of one single neuron , or one ( multivariate ) channel ( for example in the context of convolutional layers : a 2D activation map ) . Finally , the image set IM of the model is of particular importance for applications of generative models , as it should approximate at best the support of the data distribution we want to model . For example , if we want to generate images of human faces , IM certainly should not cover the space of all possible RGB images , but live on a complicated subset of it ( possibly with a manifold structure ) . Correctly fitting the generator parameters such that IM precisely matches the support of target distribution can be seen as the key objective of such models ( see e.g . Sajjadi et al . ( 2018 ) ) . One benefit of the causal framework is to be able to define interventions and counterfactuals . We will use the following definition . Definition 2 ( Interventional CGM and unit level counterfactual ) . Consider a CGM M = G ( PZ , S , G ) . For a value v0 of a subset of endogenous variables V|E indexed by E = { e1 , .. , en } , we define the interventional CGM Mv0 = G ( PZ , S v0 , Gv0 ) obtained by replacing the structural assignments for V|E by the constant assignments { Vek : = vk } ek∈E . Then for a given value z of the latent variables called unit , the unit-level counterfactual output is the deterministic output of Mv0 for input latent variable Z = z IEv ( z ) = gMv0 ( z ) . We will call the ( E , v0 ) -counterfactual the mapping IEv0 : gM ( z ) 7→ I E v0 ( z ) We say IEv0 is faithful to M if for all z , I E v ( z ) belongs to IM . This essentially corresponds to the definition of counterfactual that can be found in ( Pearl , 2014 ) , with notations adapted to the context of CGMs defined above . Note that we restrict for simplicity interventional CGMs to assigning constant values to endogenous variables , however interventions and their ensuing counterfactuals can be generalized to more complex assignments without major difficulty ( see e.g . Peters et al . ( 2017 ) ) . This definition is also in line with the concept of potential outcome ( see e.g . Imbens and Rubin ( 2015 ) ) . In addition , Pearl ( 2009 , chapter 7 ) ) provided a detailed discussion regarding this connection between SCM and potential outcome frameworks . Faithfulness is a property we introduce in the context of generative models to take into account that not all interventions on internal variables of a generative model will result in an output that likely belongs to the learned data distribution . For example , assigning a large value to a neuron may saturate the non-linearity of many downstream neurons , resulting in an artifactual output . For such reason , we will repetitively need to restrict a function ’ s output values to be in the same set as their inputs ( i.e . their codomain is included in their domain ) , and we will call endomorphism such function .

Summary: This paper proposes to examine generative adversarial networks by using counterfactual reasoning. The authors propose to examine modularity through the lens of interventions on the generative networks. After observing that the nodes within the generative network obey a deterministic relationship, they propose a proxy for intervention which takes samples and creates “hybrid” samples by replacing the activation output of one sample with the others. Given the vast number of nodes that exist within a generative network, the authors propose a heuristic for choosing the nodes to perturb.