GAN Dissection: Visualizing and Understanding Generative Adversarial Networks

Generative Adversarial Networks ( GANs ) have recently achieved impressive results for many real-world applications , and many GAN variants have emerged with improvements in sample quality and training stability . However , they have not been well visualized or understood . How does a GAN represent our visual world internally ? What causes the artifacts in GAN results ? How do architectural choices affect GAN learning ? Answering such questions could enable us to develop new insights and better models . In this work , we present an analytic framework to visualize and understand GANs at the unit- , object- , and scene-level . We first identify a group of interpretable units that are closely related to object concepts using a segmentation-based network dissection method . Then , we quantify the causal effect of interpretable units by measuring the ability of interventions to control objects in the output . We examine the contextual relationship between these units and their surroundings by inserting the discovered object concepts into new images . We show several practical applications enabled by our framework , from comparing internal representations across different layers , models , and datasets , to improving GANs by locating and removing artifact-causing units , to interactively manipulating objects in a scene . We provide open source interpretation tools to help researchers and practitioners better understand their GAN models . 1 INTRODUCTION . Generative Adversarial Networks ( GANs ) ( Goodfellow et al. , 2014 ) have been able to produce photorealistic images , often indistinguishable from real images . This remarkable ability has powered many real-world applications ranging from visual recognition ( Wang et al. , 2017 ) , to image manipulation ( Isola et al. , 2017 ; Zhu et al. , 2017 ) , to video prediction ( Mathieu et al. , 2016 ) . Since their invention in 2014 , many GAN variants have been proposed ( Radford et al. , 2016 ; Zhang et al. , 2018a ) , often producing more realistic and diverse samples with better training stability . Despite this tremendous success , many questions remain to be answered . For example , to produce a church image ( Figure 1a ) , what knowledge does a GAN need to learn ? Alternatively , when a GAN sometimes produces terribly unrealistic images ( Figure 1f ) , what causes the mistakes ? Why does one GAN variant work better than another ? What fundamental differences are encoded in their weights ? In this work , we study the internal representations of GANs . To a human observer , a well-trained GAN appears to have learned facts about the objects in the image : for example , a door can appear on a building but not on a tree . We wish to understand how a GAN represents such structure . Do the objects emerge as pure pixel patterns without any explicit representation of objects such as doors and trees , or does the GAN contain internal variables that correspond to the objects that humans perceive ? If the GAN does contain variables for doors and trees , do those variables cause the generation of those objects , or do they merely correlate ? How are relationships between objects represented ? We present a general method for visualizing and understanding GANs at different levels of abstraction , from each neuron , to each object , to the contextual relationship between different objects . We first identify a group of interpretable units that are related to object concepts ( Figure 1b ) . These units ’ featuremaps closely match the semantic segmentation of a particular object class ( e.g. , trees ) . Second , we directly intervene within the network to identify sets of units that cause a type of objects to disappear ( Figure 1c ) or appear ( Figure 1d ) . We quantify the causal effect of these units using a standard causality metric . Finally , we examine the contextual relationship between these causal object units and the background . We study where we can insert object concepts in new images and how this intervention interacts with other objects in the image ( Figure 1d ) . To our knowledge , our work provides the first systematic analysis for understanding the internal representations of GANs . Finally , we show several practical applications enabled by this analytic framework , from comparing internal representations across different layers , GAN variants and datasets ; to debugging and improving GANs by locating and ablating “ artifact ” units ( Figure 1e ) ; to understanding contextual relationships between objects in scenes ; to manipulating images with interactive object-level control . 2 RELATED WORK . Generative Adversarial Networks . The quality and diversity of results from GANs ( Goodfellow et al. , 2014 ) has continued to improve , from generating simple digits and faces ( Goodfellow et al. , 2014 ) , to synthesizing natural scene images ( Radford et al. , 2016 ; Denton et al. , 2015 ) , to generating 1k photorealistic portraits ( Karras et al. , 2018 ) , to producing one thousand object classes ( Miyato et al. , 2018 ; Zhang et al. , 2018a ) . GANs have also enabled applications such as visual recognition ( Wang et al. , 2017 ; Hoffman et al. , 2018 ) , image manipulation ( Isola et al. , 2017 ; Zhu et al. , 2017 ) , and video generation ( Mathieu et al. , 2016 ; Wang et al. , 2018 ) . Despite the successes , little work has been done to visualize what GANs have learned . Prior work ( Radford et al. , 2016 ; Zhu et al. , 2016 ; Brock et al. , 2017 ) manipulates latent vectors and observes how the results change accordingly . Visualizing deep neural networks . A CNN can be visualized by reconstructing salient image features ( Simonyan et al. , 2014 ; Mahendran & Vedaldi , 2015 ) or by mining patches that maximize hidden layers ’ activations ( Zeiler & Fergus , 2014 ) ; or we can synthesize input images to invert a feature layer ( Dosovitskiy & Brox , 2016 ) . Alternately , we can identify the semantics of each unit ( Zhou et al. , 2015 ; Bau et al. , 2017 ; Zhou et al. , 2018a ) by measuring agreement between unit activations and object segmentation masks , or by training a network to increase interpretability of such units ( Zhang et al. , 2018b ) . Visualization of an RNN has also revealed interpretable units that track long-range dependencies ( Karpathy et al. , 2016 ; Strobelt et al. , 2018 ) . Most previous work on network visualization has focused on networks trained for classification ; our work explores deep generative models trained for image generation . Understanding neural representation in biology . Studies of biological neural networks find evidence of both local representations in which individual neurons are selective for meaningful concepts ( Quiroga , 2012 ) , as well as distributed representations in which individual neurons are essentially meaningless ( Yuste , 2015 ) . Computational models of biological learning ( Bowers et al. , 2016 ; Dasgupta et al. , 2018 ) find sparse and local representations can aid generalization to novel stimuli . Explaining the decisions of deep neural networks . Individual network decisions can be explained using informative heatmaps ( Zhou et al. , 2018b ; 2016 ; Selvaraju et al. , 2017 ) or by scoring salience ( Simonyan et al. , 2014 ; Bach et al. , 2015 ; Sundararajan et al. , 2017 ; Lundberg & Lee , 2017 ) . Such analyses reveals which inputs contribute most to a categorical prediction by a network . Recent work has also studied the contribution of feature vectors ( Kim et al. , 2017 ; Zhou et al. , 2018b ) or individual channels ( Olah et al. , 2018 ) to a final prediction , and Morcos et al . ( 2018 ) has examined the effect of individual units by ablating them . Those methods explain discriminative classifiers . Our method aims to explain how an image can be generated by a network , which is much less explored . 3 METHOD . Our goal is to analyze how objects such as trees are encoded by the internal representations of a GAN generator G : z→ x . Here z ∈ R|z| denotes a latent vector sampled from a low-dimensional distribution , and x ∈ RH×W×3 denotes an H ×W generated image . We use representation to describe the tensor r output from a particular layer of the generator G , where the generator creates an image x from random z through a composition of layers : r = h ( z ) and x = f ( r ) = f ( h ( z ) ) = G ( z ) . Since r has all the data necessary to produce the image x = f ( r ) , r certainly contains the information to deduce the presence of any visible class c in the image . Therefore the question we ask is not whether information about c is present in r — it is — but how such information is encoded in r. In particular , for any class from a universe of concepts c ∈ C , we seek to understand whether r explicitly represents c in some way where it is possible to factor r at locations P into two components rU , P = ( rU , P , rU , P ) , ( 1 ) where the generation of the object c at locations P depends mainly on the units rU , P , and is insensitive to the other units rU , P . Here we refer to each channel of the featuremap as a unit : U denotes the set of unit indices of interest and U is its complement ; we will write U and P to refer to the entire set of units and featuremap pixels in r. We study the structure of r in two phases : • Dissection : starting with a large dictionary of object classes , we identify the classes that have an explicit representation in r by measuring the agreement between individual units of r and every class c ( Figure 1b ) . • Intervention : for the represented classes identified through dissection , we identify causal sets of units and measure causal effects between units and object classes by forcing sets of units on and off ( Figure 1c , d ) . 3.1 CHARACTERIZING UNITS BY DISSECTION . We first focus on individual units of the representation . Recall that ru , P is the one-channel h × w featuremap of unit u in a convolutional generator , where h× w is typically smaller than the image size . We want to know if a specific unit ru , P encodes a semantic class such as a “ tree ” . For image classification networks , Bau et al . ( 2017 ) has observed that many units can approximately locate emergent object classes when the units are upsampled and thresholded . In that spirit , we select a universe of concepts c ∈ C for which we have a semantic segmentation sc ( x ) for each class . Then we quantify the spatial agreement between the unit u ’ s thresholded featuremap and a concept c ’ s segmentation with the following intersection-over-union ( IoU ) measure : IoUu , c ≡ Ez ∣∣∣ ( r↑u , P > tu , c ) ∧ sc ( x ) ∣∣∣ Ez ∣∣∣ ( r↑u , P > tu , c ) ∨ sc ( x ) ∣∣∣ , where tu , c = argmaxt I ( r↑u , P > t ; sc ( x ) ) H ( r↑u , P > t , sc ( x ) ) , ( 2 ) where ∧ and ∨ denote intersection and union operations , and x = G ( z ) denotes the image generated from z . The one-channel feature map ru , P slices the entire featuremap r = h ( z ) at unit u . As shown in Figure 2a , we upsample ru , P to the output image resolution as r ↑ u , P . ( r ↑ u , P > tu , c ) produces a binary mask by thresholding the r↑u , P at a fixed level tu , c . sc ( x ) is a binary mask where each pixel indicates the presence of class c in the generated image x . The threshold tu , c is chosen to be informative as possible by maximizing the information quality ratio I/H ( using a separate validation set ) , that is , it maximizes the portion of the joint entropy H which is mutual information I ( Wijaya et al. , 2017 ) . We can use IoUu , c to rank the concepts related to each unit and label each unit with the concept that matches it best . Figure 3 shows examples of interpretable units with high IoUu , c . They are not the only units to match tables and sofas : layer3 of the dining room generator has 31 units ( of 512 ) that match tables and table parts , and layer4 of the living room generator has 65 ( of 512 ) sofa units . Once we have identified an object class that a set of units match closely , we next ask : which units are responsible for triggering the rendering of that object ? A unit that correlates highly with an output object might not actually cause that output . Furthermore , any output will jointly depend on several parts of the representation . We need a way to identify combinations of units that cause an object .

Summary: This work proposes a novel analytic framework exploited on a semantic segmentation model to visualize GANs at unit (feature map) level. The authors show that some GAN representations can be interpreted, correlate with the parsing result from the semantic segmentation model but as variables that have a causal effect on the synthesis of semantic objects in the output. This framework could allow to detect and remove the artifacts to improve the quality of the generated images.