Designing Counterfactual Generators using Deep Model Inversion

1 Introduction . With the growing need for deploying deep black-box models into critical decision-making , there is an increased emphasis on explainability methods that can reveal intricate relationships between data signatures ( e.g. , image features ) and predictions . In this context , the so-called counterfactual ( CF ) explanations [ 1 ] that synthesize small , interpretable changes to a given image while producing desired changes in model predictions to support user-specified hypotheses ( e.g. , progressive change in predictions ) have become popular . Though counterfactual explanations provide more flexibility over conventional techniques , such as feature importance estimation [ 2 , 3 , 4 , 5 , 6 ] , by exploring the vicinity of a query image , an important requirement to produce meaningful counterfactuals is to produce discernible local perturbations ( for easy interpretability ) while being realistic ( close to the underlying data manifold ) . Consequently , existing approaches rely extensively on pre-trained generative models to synthesize plausible counterfactuals [ 1 , 7 , 8 , 9 , 10 ] . By design , this ultimately restricts their utility to scenarios where one can not access the training data or pre-trained generative models , for example , due to privacy requirements commonly encountered in many practical applications . In this paper , we focus on the problem where we have access only to trained deep classifiers and not the actual training data or generative models . Synthesizing images from the underlying data distribution 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . by inverting a deep model , while not requiring access to training data , is a well investigated topic of research . For e.g. , Deep Dream [ 11 ] synthesizes class-conditioned images by manipulating a noisy image directly in the space of pixels ( or more formally Image Space Optimization ( ISO ) ) constrained by image priors such as total variation [ 12 ] to regularize this ill-posed inversion . However , Deep Dream is known to produce images that look unrealistic , often very different from the training images , thus limiting their use in practice . Consequently , Yin et al. , proposed DeepInversion [ 13 ] that performs image synthesis in the latent space of a pre-trained classifier ( Latent Space Optimization ( LSO ) ) and leverages layer-specific statistics ( from batchnorm [ 14 ] ) to constrain the images to be consistent with the training data distribution . This was showed to produce higher-quality images , particularly in the context of performing knowledge distillation [ 15 ] using the synthesized images . Proposed Work . In contrast , this work aims to develop a deep model inversion approach that generates counterfactual explanations by exploring the vicinity of a given query image , instead of synthesizing an arbitrary realization from the entire image distribution . As illustrated in the example in Figure 1 , existing deep inversion methods are ineffective when natively adopted for counterfactual generation . Due to use of weak priors , and the severely ill-posed nature of the problem , it introduces irrelevant pixel manipulations that easily satisfy the desired change in prediction . Hence , we propose DISC ( Deep Inversion for Synthesizing Counterfactuals ) that improves upon conventional deep model inversion by utilizing : ( i ) stronger image priors through the use of deep image priors [ 16 ] ( DIP ) and implicit neural representations [ 17 ] ( INR ) ; ( ii ) a novel manifold consistency objective that ensures the counterfactual remains close to the underlying manifold ; and ( iii ) a progressive optimization strategy to effectively introduce discernible , yet meaningful , changes to the query image . From Figure 1 , we find that our approach produces meaningful image manipulations , in order to change the prediction to the smiling class , while other deep inversion strategies can not . Using empirical studies , we show that DISC consistently produces visually meaningful explanations , and that the counterfactuals from DISC are effective at learning model decision boundaries and are robust to unknown test-time corruptions . Our Contributions . 1 . A general framework to produce counterfactuals on-the-fly using deep model inversion ; 2 . Novel objectives to ensure consistency to the data manifold . We explore two different strategies based on direct error prediction [ 18 , 19 ] and deterministic uncertainty estimation [ 20 ] ; 3 . A progressive optimization strategy to introduce discernible changes to a given query image , while satisfying the manifold consistency requirement ; 4 . A classifier discrepancy metric to evaluate the quality of counterfactuals ; 5 . Empirical studies using natural image and medical image classifiers to demonstrate the effectiveness of DISC over a variety of baselines and ablations . 2 Related Work . Image Synthesis from Classifiers . Inverting a pre-trained deep model is a popular strategy for image synthesis in scenarios where there is no access to the underlying training data . While deep model inversion-based methods such as Deep Dream [ 11 ] and DeepInversion [ 13 ] have been successful in generating class-conditioned images , there have been other extensions to such approaches . For example , Dosovitskiy et al . [ 21 ] proposed to invert representations of a pre-trained CNN to obtain insights about what a deep classifier has learned . On similar lines , Mahendran et al . [ 12 , 22 ] addressed the problem of pre-image recovery , which in essence attempts to recover an arbitrarily encoded representation ( in the latent space of a classifier ) to a realization on the ( unknown ) image manifold and to enable model diagnosis . Ulyanov et al . [ 16 ] improved upon this ill-posed inversion by utilizing a strong image prior in the form of deep image priors ( DIP ) . They also explored the related problem of activation-maximization [ 16 ] , where the goal is to generate an image that maximizes the activation of a given output neuron in a pre-trained classifier , and demonstrated the effectiveness of DIP . Despite the effectiveness of the deep model inversion methods for image synthesis , we find that such methods can not be natively adopted for CF generation and are insufficient for producing meaningful pixel manipulations to a given query image . Counterfactual Generation . Existing methods extensively rely on generative models to provide counterfactuals that explain the decisions of a black-box model . Examples including CounteRGAN [ 23 ] , Counterfactual Generative Networks ( CGNs ) [ 24 ] and the methods reported in [ 7 , 8 ] , have clearly demonstrated the use of generative models in synthesizing high-quality CFs for any user-specified hypothesis on the predictions . However , this requirement of access to training data or generative models can be infeasible in practical scenarios , for e.g. , restrictions arising due to privacy requirements . In contrast , our approach formulates the problem of counterfactual generation using deep model inversion , and can produce meaningful counterfactuals using only the trained classifier . 3 Proposed Approach . In this section , we describe our approach for counterfactual generation that improves upon deep model inversion , and introduce a new classifier discrepancy metric for evaluating CFs . There are four key components that are critical to designing classifier-based counterfactual generators : ( i ) choice of metric for semantics preservation ; ( ii ) choice of image priors to regularize image synthesis ; ( iii ) manifold consistency to ensure that the synthesized counterfactual lies close to the true data manifold ; and ( iv ) progressive optimization strategy to introduce gradual meaningful changes to a query image . Figures 2a and 2b provide overview of DISC implemented with deep image prior and INR-based for regularizing the counterfactual optimization process . In its simplest form , for a given query x , a counterfactual explanation can be obtained as follows : arg min x̄ d ( x̄ , x ) s.t . F ( x̄ ) = ȳ , x̄ ∈M ( x ) , ( 1 ) where x̄ = C ( x ) is a counterfactual explanation for x , F is a pre-trained classifier model , M denotes the data manifold and ȳ = y + δ is the desired change in the prediction . The metric d ( . , . ) measures the discrepancy between the query image and the counterfactual ( i.e. , semantics preservation ) . 3.1 Choice of Metric for Semantics Preservation . We can measure the discrepancy between a query and its CF explanation , d ( x̄ , x ) , in the pixel space or in the latent space of the classifier . We now describe the high-level formulation of deep model inversion-based CF generation . a . Image Space Optimization ( ISO ) . ISO for counterfactual generation involves the ill-posed optimization of an input x̄ directly in the pixel space to generate an image semantically similar to the query x , while being consistent with a user-hypothesis on the prediction , ȳ . Strategies such as Deep Dream [ 11 ] perform ISO to synthesize artistic variations of images . Mathematically , arg min x̄ d ( x̄ , x ) +R ( x̄ ) s.t . F ( x̄ ) = ȳ . ( 2 ) whereR ( x̄ ) is a suitable image prior to regularize the optimization . b . Latent Space Optimization ( LSO ) . LSO refers to the ill-posed problem of inverting an arbitrarily encoded representation from the latent space of a deep classifier to a realization on the ( unknown ) image manifold . Let Ψl ( . ) denote the lth differentiable layer of the deep classifier . Then , counterfactual generation using LSO can be mathematically formulated as arg min x̄ ∑ l d ( Ψl ( x̄ ) , Ψl ( x ) ) +R ( x̄ ) s.t . F ( x̄ ) = ȳ . ( 3 ) Approaches such as DeepInversion [ 13 ] perform LSO for conditional image synthesis ( though with distribution-level comparison instead of our sample-level comparison ) and achieve visually superior images when compared to ISO approaches . 3.2 Choice of Image Priors . As observed from ( 2 ) and ( 3 ) , the choice of the regularizer or image prior R ( . ) is central towards regularizing and tractably solve this challenging inverse problem . A variety of image priors have been proposed in the literature , and we investigate the following in this work . a . Total Variation + ` 2 . Total variation [ 22 ] ( TV ) is a popular regularizer that encourages images to contain piece-wise constant patches while the ` 2 norm regularizes the range and energy of the image to remain within a given interval . The TV norm and the ` 2 norm are given by : RTV ( x̄ ) = ∑ i , j √ ( x̄i , j+1 − x̄i , j ) 2 + ( x̄i+1 , j − x̄i , j ) 2 ; R ` 2 ( x̄ ) = √∑ i , j ‖x̄i , j‖2 ( 4 ) b . Deep Image Priors ( DIP ) . A Deep Image Prior ( DIP ) [ 16 ] leverages the structure of an untrained , carefully tailored convolutional neural network ( e.g. , U-Net [ 25 ] ) to generate images and solve a variety of ill-posed restoration tasks in computer vision . DIP has been found to produce high-quality reconstructions , based on the key insight that the structure of the network itself can act as a regularizer . Consequently , the synthesized image is re-parameterized in terms of the weights θ of the prior model fθ i.e x̄ = fθ ( z ) . c. Implicit Neural Representations ( INR ) . We also considered an alternative approach based on INR , which provide a cheap and convenient way to learn a continuous mapping from the image coordinates to the pixel values ( RGB ) . While they have been found to be effective for image/volume rendering and designing generative models [ 26 ] , we explore their use in deep model inversion ( details in appendix ) . We build upon two key results to design our INR-based counterfactual generators : ( i ) Fourier mapping : Based on NTK ( neural tangent kernel ) theory , [ 27 ] showed that using Fourier mapping can recover high-frequency features in low-dimensional coordinate-based image reconstruction . Hence , we use a Fourier feature mapping z to featurize 2−D input coordinates v ∈ [ 0 , 1 ] 2 before passing them through a coordinate-based MLP : z ( v ) = [ a1 cos ( 2πb T 1 v ) , a1 sin ( 2πb T 1 v ) , · · · ] . Using a set of randomly chosen sinusoids , this maps the input points to the surface of a highdimensional hypersphere . Training the MLP network on these embedded points corresponds to kernel regression with the stationary composed NTK hNTK ◦ hz , where hNTK denotes the neural tangent kernel corresponding to the MLP ; ( ii ) SIREN activation : In [ 17 ] , Sitzmann et al . showed that periodic activation functions are better suited for recovering natural images and their derivatives , when compared to standard activation functions . More specifically , SIREN uses a sinusoid activation Φ ( x ) = sin ( Wx + b ) . We find that using both a Fourier mapping coupled with SIREN activation leads to a very strong image prior .

Summary: In this paper authors propose to generate counterfactuals from a query image and a given trained deep classifier. One is interested to find out a semantically reasonable alternative explanation, where image would be classified from class y to y'. This is all performed without the help of an explicit generator (such as GAN).