Influence Estimation for Generative Adversarial Networks

1 INTRODUCTION . Generative adversarial networks ( GANs ) proposed by Goodfellow et al . ( 2014 ) are a powerful subclass of generative model , which is successfully applied to a number of image generation tasks ( Antoniou et al. , 2017 ; Ledig et al. , 2017 ; Wu et al. , 2016 ) . The expansion of the applications of GANs makes improvements in the generative performance of models increasingly crucial . An effective approach for improving machine learning models is to identify training instances that harm the model performance . Traditionally , statisticians manually screen a dataset for harmful instances , which misguide a model into producing biased predictions . Recent influence estimation methods ( Khanna et al. , 2019 ; Hara et al. , 2019 ) automated the screening of datasets for deep learning settings , in which the sizes of both datasets and data dimensions are too large for users to manually determine the harmful instances . Influence estimation measures the effect of removing an individual training instance on a model ’ s prediction without the computationally prohibitive cost of model retraining . The recent studies identified harmful instances by estimating how the loss value changes if each training instance is removed from the dataset . Although previous studies have succeeded in identifying the harmful instances in supervised settings , the extension of their approaches to GAN is non-trivial . Previous approaches require that ( i ) the existence or absence of a training instance directly affects a loss value , and that ( ii ) the decrease in the loss value represents the harmfulness of the removed training instance . In GAN training , however , neither of the requirements is satisfied . ( i ) As training instances are only fed into the discriminator , they only indirectly affect the generator ’ s loss , and ( ii ) the changes in the losses of GAN do not necessarily capture how the removed instances harm the generative performance . This is because the ability of the loss to evaluate the generator is highly dependent on the performance of the discriminator . To this end , ( i ) we propose an influence estimation method that uses the Jacobian of the gradient of the discriminator ’ s loss with respect to the generator ’ s parameters ( and vice versa ) , which traces how the absence of an instance in the discriminator ’ s training affects the generator ’ s parameters . In addition , ( ii ) we propose a novel evaluation scheme to judge if an instance is harmful or not on the basis of influence on GAN evaluation metric , that is , how a GAN evaluation metric ( e.g. , inception score ( Salimans et al. , 2016 ) ) changes if a given training instance is removed from the dataset . We identify harmful instances by estimating the influence on GAN evaluation metric by leveraging our influence estimation method . We verified that the proposed influence estimation method correctly estimated the influence on GAN evaluation metrics across different settings of the dataset , model architecture , and GAN evaluation metrics . We also demonstrated that removing harmful instances , which were identified by the proposed method , effectively improved various GAN evaluation metrics.1 Our contributions are summarized as follows : • We propose an influence estimation method that uses the Jacobian of the gradient of the discriminator ’ s loss with respect to the generator ’ s parameters ( and vice versa ) , which traces how the absence of an instance in the discriminator ’ s training affects the generator ’ s parameters . • We propose a novel evaluation scheme to judge if an instance is harmful or not on the basis of influence on GAN evaluation metrics rather than that on the loss value , and to leverage the proposed influence estimation method to identify harmful instances . • We experimentally verified that our influence estimation method correctly inferred the influence on GAN evaluation metrics . Further , we demonstrated that the removal of the harmful instances suggested by the proposed method effectively improved the generative performance with respect to various GAN evaluation metrics . 2 PRELIMINARIES . Notation For column vectors a , b ∈ Rp , we denote the inner product by 〈a , b〉 = ∑p i=1 aibi . For a function f ( a ) , we denote its gradient with respect to a by∇af ( a ) . We denote the identity matrix of size p by Ip , the zero vector of length p by 0p , and the ones vector of length p by 1p . Generative Adversarial Networks ( GAN ) For simplicity , we consider an unconditional GAN that consists of the generator G : Rdz → Rdx and the discriminator D : Rdx → R , where dz and dx are the number of dimensions of latent variable z ∼ p ( z ) and data point x ∼ p ( x ) , respectively . The parameters of generator θG ∈ RdG and discriminator θD ∈ RdD are learned though the adversarial training ; G tries to sample realistic data whileD tries to identify whether the data is real or generated . Formulation of GAN Objectives For the generality , we adopt the formulation of Gidel et al . ( 2019 ) in which G and D try to minimize LG and LD , respectively , to obtain the following Nash equilibrium ( θ∗G , θ ∗ D ) : θ∗G ∈ arg min θG LG ( θG , θ ∗ D ) and θ ∗ D ∈ arg min θD LD ( θ ∗ G , θD ) . ( 1 ) For the latter part of this paper , we use a coupled parameter vector θ : = ( θG , θD ) > ∈ Rdθ=dG+dD when we refer to the whole parameters of GAN . In this paper , we assume that LG and LD have the following forms2 : LG ( θ ) : = Ez∼p ( z ) [ fG ( z ; θ ) ] , LD ( θ ) : = Ez∼p ( z ) [ f [ z ] D ( z ; θ ) ] + Ex∼p ( x ) [ f [ x ] D ( x ; θ ) ] . ( 2 ) 1Code is at https : //github.com/hitachi-rd-cv/influence-estimation-for-gans 2This covers the common settings of GAN objectives : the non-zero-sum game proposed by Goodfellow et al . ( 2014 ) , Wasserstein distance ( Arjovsky et al. , 2017 ) , and the least squares loss ( Mao et al. , 2017 ) . We can recover the original minimax objective by taking fG ( z ; θ ) = log ( 1−DθD ( GθG ( z ) ) ) , f [ z ] D = −fG , and f [ x ] D ( x ; θ ) = − logDθD ( x ) . Adversarial SGD ( ASGD ) To make our derivation easier to understand , we newly formulate the parameter update of a GAN trained by stochastic gradient descent , which we call adversarial SGD ( ASGD ) . For simplicity , this paper considers simultaneous training , in which the generator and the discriminator are simultaneously updated at a single step . We denote the dataset by Dx : = { xn ∼ p ( x ) } Nn=1 , which consists of N data points . Let St ⊂ { 1 , . . . , N } be a set of sample indices at the t-th step . We assume that the mini-batch of the t-th step consists of instances { xi } i∈St and a set of latent variables Zt = { z [ t ] l ∼ p ( z ) } |St| l=1 , which are sampled independently at each step t. We denote the mean of LG and LD across the mini-batch by L G ( Z ; θ ) : = 1|Z| ∑ z∈Z fG ( z ; θ ) and LD ( S , Z ; θ ) : = 1|Z| ( ∑ z∈Z f [ z ] D ( z ; θ ) + ∑ i∈S f [ x ] D ( xi ; θ ) ) , respectively . The t-th step of ASGD updates the coupled parameters by θ [ t+1 ] = θ [ t ] −Btg ( St , Zt ; θ [ t ] ) , where Bt : = ( η [ t ] G IdG O O η [ t ] D IdD ) ∈ Rdθ×dθ , g ( S , Z ; θ ) : = ( ∇θGL G ( Z ; θ ) ∇θDLD ( S , Z ; θ ) ) ∈ Rdθ . ( 3 ) η [ t ] G ∈ R+ and η [ t ] D ∈ R+ are the learning rates of the t-th step for θG and θD , respectively . 3 PROPOSED METHOD . This section explains the two main contributions of our paper : the influence estimation method for GANs that predicts how the removal of a training instance changes the output of the generator and the discriminator ( Section 3.1 ) , and two important parts of our instance evaluation scheme , that are , the definition of influence on GAN evaluation metric and its estimation algorithm ( Section 3.2 ) . 3.1 INFLUENCE ESTIMATION FOR GAN . We refer to influence estimation as the estimation of changes in a model ’ s output under a training instance ’ s absence . As the model ’ s output changes through the changes in the model ’ s parameters , we start with the definition of ASGD-Influence , which represents the changes in parameters , and then formulate its estimator . ASGD-Influence ASGD-Influence is defined on the basis of the following counterfactual ASGD . Let θ [ t ] −j denote the parameters at t-th step trained without using j-th training instance . Counterfactual ASGD starts optimization from θ [ 1 ] −j = θ [ 1 ] and updates the parameters of the t-th step by θ [ t+1 ] −j = θ [ t ] −j−Btg ( St \ { j } , Zt ; θ [ t ] −j ) . We define ASGD-Influence ∆θ−j as the parameter differ- ence between counterfactual ASGD and ASGD at the final step t = T , namely ∆θ−j : = θ [ T ] −j−θ [ T ] . Estimator of ASGD-Influence Our estimator uses an approximation of the mean of the gradient . Let ( ∇θGL G ( Z ; θ ) , ∇θDLD ( S , Z ; θ ) ) > be the joint gradient vector of the mini-batch . We introduce the Jacobian of the joint gradient vector of the t-th mini-batch with respect to θ : Jt : = ( J [ t ] GG J [ t ] GD J [ t ] DG J [ t ] DD ) = ( ∇2θGL G ( Zt ; θ [ t ] ) ∇θD∇θGL G ( Zt ; θ [ t ] ) ∇θG∇θDLD ( St , Zt ; θ [ t ] ) ∇2θDLD ( St , Zt ; θ [ t ] ) ) . ( 4 ) When we assume both LG ( θ ) and LG ( θ ) are second-order differentiable with respect to θ , the first-order Taylor approximation gives g ( St , Zt ; θ [ t ] −j ) − g ( St , Zt ; θ [ t ] ) ≈ Jt ( θ [ t ] −j − θ [ t ] ) . With this approximation , we have θ [ t+1 ] −j − θ [ t+1 ] = ( θ [ t ] −j − θ [ t ] ) −Bt ( g ( St , Zt ; θ [ t ] −j ) − g ( St , Zt ; θ [ t ] ) ) ≈ ( Idθ −BtJt ) ( θ [ t ] −j − θ [ t ] ) , ∀j 6∈ St. ( 5 ) For simplicity , we first focus on 1-epoch ASGD in which each instance appears only once . Let π ( j ) be the step where the j-th instance is used . Considering the absence of ∇θDf [ x ] D ( xj ; θ [ π ( j ) ] ) in the π ( j ) -th step of counterfactual ASGD , we have θ [ π ( j ) +1 ] −j − θ [ π ( j ) +1 ] = η [ π ( j ) ] D |Sπ ( j ) | ( 0dG , ∇θDf [ x ] D ( xj ; θ [ π ( j ) ] ) ) > . By denoting Zt : = Idθ − BtJt and recursively applying the approximation ( 5 ) , we obtain ∆θ−j ≈ η [ π ( j ) ] D |Sπ ( j ) | ZT−1ZT−2 · · ·Zπ ( j ) +1 ( 0dG ∇θDf [ x ] D ( xj ; θ [ π ( j ) ] ) ) . ( 6 ) For the practical situation of K-epoch ASGD , in which the j-th instance is sampled K times at t = π1 ( j ) , . . . , πK ( j ) , the estimator of the ASGD-Influence is given by ∆θ̂−j : = K∑ k=1 T−πk ( j ) −1∏ s=1 ZT−s η [ πk ( j ) ] D |Sπk ( j ) | ( 0dG ∇θDf [ x ] D ( xj ; θ [ πk ( j ) ] ) ) . ( 7 ) Linear Influence To estimate the influence on outputs , we introduce linear influence L [ T ] −j ( u ) : = 〈u , ∆θ−j〉 of a given query vector u ∈ Rdθ . If we take u = ∇θfG ( z ; θ [ T ] ) , the linear influence approximates the influence on the generator ’ s loss L [ T ] −j ( u ) ≈ fG ( z ; θ [ T ] −j ) − fG ( z ; θ [ T ] ) . Let ( u [ t ] > G ∈ RdG , u [ t ] > D ∈ RdD ) : = u > ZT−1ZT−2 · · ·Zt+1 . The linear influence of the j-th instance is approximated by the proposed estimator : L [ T ] −j ( u ) ≈ 〈 u , ∆θ̂−j 〉 = K∑ k=1 η [ πk ( j ) ] D |Sπk ( j ) | 〈 u [ πk ( j ) ] D , ∇θDf [ x ] D ( xj ; θ [ πk ( j ) ] ) 〉 . ( 8 ) The estimation algorithm consists of two phases ; training phase performs K-epoch ASGD by storing information A [ t ] ← ( St , η [ t ] G , η [ t ] D , θ [ t ] , Zt ) and inference phase calculates ( 8 ) using A [ 1 ] , . . . , A [ T−1 ] . See Appendix A for the detailed algorithm .

Summary: The paper presents an influence estimation method for GANs. It discusses why previous approaches on influence estimation cannot be easily extended to GANs. It proposes to use Jacobian of the gradient of discriminator’s loss with respect to the generator’s parameters to learn how absence of an instance in the discriminator’s training affects the generator’s parameters. The authors evaluate whether an instance is harmful based on its influence on GAN evaluation metrics. They show that removing these harmful instances improves performance of GANs on MNIST with respect to three metrics: Inception Score, FID and Average Log Likelihood (ALL).