Understanding Instance-based Interpretability of Variational Auto-Encoders

1 Introduction . Instance-based interpretation methods have been popular for supervised learning as they help explain why a model makes a certain prediction and hence have many applications [ Barshan et al. , 2020 , Basu et al. , 2020 , Chen et al. , 2020 , Ghorbani and Zou , 2019 , Hara et al. , 2019 , Harutyunyan et al. , 2021 , Koh and Liang , 2017 , Koh et al. , 2019 , Pruthi et al. , 2020 , Yeh et al. , 2018 , Yoon et al. , 2020 ] . For a classifier and a test sample z , an instance-based interpretation ranks all training samples x according to an interpretability score between x and z . Samples with high ( low ) scores are considered positively ( negatively ) important for the prediction of z . However , in the literature of unsupervised learning especially generative models , instance-based interpretations are much less understood . In this work , we investigate instance-based interpretation methods for unsupervised learning based on influence functions [ Cook and Weisberg , 1980 , Koh and Liang , 2017 ] . In particular , we theoretically analyze certain classical non-parametric and parametric methods . Then , we look at a canonical deep generative model , variational auto-encoders ( VAE , [ Higgins et al. , 2016 , Kingma and Welling , 2013 ] ) , and explore some of the applications . The first challenge is framing the counter-factual question for unsupervised learning . For instancebased interpretability in supervised learning , the counter-factual question is `` which training samples are most responsible for the prediction of a test sample ? '' – which heavily relies on the label information . However , there is no label in unsupervised learning . In this work , we frame the counterfactual question for unsupervised learning as `` which training samples are most responsible for increasing the likelihood ( or reducing the loss when likelihood is not available ) of a test sample ? '' We show that influence functions can answer this counter-factual question . Then , we examine influence functions for several classical unsupervised learning methods . We present theory and intuitions on how influence functions relate to likelihood and pairwise distances . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . The second challenge is how to compute influence functions in VAE . The first difficulty here is that the VAE loss of a test sample involves an expectation over the encoder , so the actual influence function can not be precisely computed . To deal with this problem , we use the empirical average of influence functions , and prove a concentration bound of the empirical average under mild conditions . Another difficulty is computation . The influence function involves inverting the Hessian of the loss with respect to all parameters , which involves massive computation for big neural networks with millions of parameters . To deal with this problem , we adapt a first-order estimate of the influence function called TracIn [ Pruthi et al. , 2020 ] to VAE . We call our method VAE-TracIn . It is fast because ( i ) it does not involve the Hessian , and ( ii ) it can accelerate computation with only a few checkpoints . We begin with a sanity check that examines whether training samples have the highest influences over themselves , and show VAE-TracIn passes it . We then evaluate VAE-TracIn on several real world datasets . We find high ( low ) self influence training samples have large ( small ) losses . Intuitively , high self influence samples are hard to recognize or visually high-contrast , while low self influence ones share similar shapes or background . These findings lead to an application on unsupervised data cleaning , as high self influence samples are likely to be outside the data manifold . We then provide quantitative and visual analysis on influences over test data . We call high and low influence samples proponents and opponents , respectively . 1 We find in certain cases both proponents and opponents are similar samples from the same class , while in other cases proponents have large norms . We consider VAE-TracIn as a general-purpose tool that can potentially help understand many aspects in the unsupervised setting , including ( i ) detecting underlying memorization , bias or bugs [ Feldman and Zhang , 2020 ] in unsupervised learning , ( ii ) performing data deletion [ Asokan and Seelamantula , 2020 , Izzo et al. , 2021 ] in generative models , and ( iii ) examining training data without label information . We make the following contributions in this paper . • We formally frame instance-based interpretations for unsupervised learning . • We examine influence functions for several classical unsupervised learning methods . • We present VAE-TracIn , an instance-based interpretation method for VAE . We provide both theoretical and empirical justification to VAE-TracIn . • We evaluate VAE-TracIn on several real world datasets . We provide extensive quantitative analysis and visualization , as well as an application on unsupervised data cleaning . 1.1 Related Work . There are two lines of research on instance-based interpretation methods for supervised learning . The first line of research frames the following counter-factual question : which training samples are most responsible for the prediction of a particular test sample z ? This is answered by designing an interpretability score that measures the importance of training samples over z and selecting those with the highest scores . Many scores and their approximations have been proposed [ Barshan et al. , 2020 , Basu et al. , 2020 , Chen et al. , 2020 , Hara et al. , 2019 , Koh and Liang , 2017 , Koh et al. , 2019 , Pruthi et al. , 2020 , Yeh et al. , 2018 ] . Specifically , Koh and Liang [ 2017 ] introduce the influence function ( IF ) based on the terminology in robust statistics [ Cook and Weisberg , 1980 ] . The intuition is removing an important training sample of z should result in a huge increase of its test loss . Because the IF is hard to compute , Pruthi et al . [ 2020 ] propose TracIn , a fast first-order approximation to IF . Our paper extends the counter-factual question to unsupervised learning where there is no label . We ask : which training samples are most responsible for increasing the likelihood ( or reducing the loss ) of a test sample ? In this paper , we propose VAE-TracIn , an instance-based interpretation method for VAE [ Higgins et al. , 2016 , Kingma and Welling , 2013 ] based on TracIn and IF . The second line of research considers a different counter-factual question : which training samples are most responsible for the overall performance of the model ( e.g . accuracy ) ? This is answered by designing an interpretability score for each training sample . Again many scores have been proposed [ Ghorbani and Zou , 2019 , Harutyunyan et al. , 2021 , Yoon et al. , 2020 ] . Terashita et al . [ 2021 ] extend this framework to a specific unsupervised model called generative adversarial networks [ Goodfellow 1There are different names in the literature , such as helpful/harmful samples [ Koh and Liang , 2017 ] , excitatory/inhibitory points [ Yeh et al. , 2018 ] , and proponents/opponents [ Pruthi et al. , 2020 ] . et al. , 2014 ] . They measure influences of samples on several evaluation metrics , and discard samples that harm these metrics . Our paper is orthogonal to these works . The instance-based interpretation methods lead to many applications in various areas including adversarial learning , data cleaning , prototype selection , data summarization , and outlier detection [ Barshan et al. , 2020 , Basu et al. , 2020 , Chen et al. , 2020 , Feldman and Zhang , 2020 , Ghorbani and Zou , 2019 , Hara et al. , 2019 , Harutyunyan et al. , 2021 , Khanna et al. , 2019 , Koh and Liang , 2017 , Pruthi et al. , 2020 , Suzuki et al. , 2021 , Ting and Brochu , 2018 , Ye et al. , 2021 , Yeh et al. , 2018 , Yoon et al. , 2020 ] . In this paper , we apply the proposed VAE-TracIn to an unsupervised data cleaning task . Prior works on interpreting generative models analyze their latent space via measuring disentanglement , explaining and visualizing representations , or analysis in an interactive interface [ Alvarez-Melis and Jaakkola , 2018 , Bengio et al. , 2013 , Chen et al. , 2016 , Desjardins et al. , 2012 , Kim and Mnih , 2018 , Olah et al. , 2017 , 2018 , Ross et al. , 2021 ] . These latent space analysis are complementary to the instance-based interpretation methods in this paper . 2 Instance-based Interpretations . Let X = { xi } Ni=1 ∈ Rd be the training set . Let A be an algorithm that takes X as input and outputs a model that describes the distribution of X . A can be a density estimator or a generative model . Let L ( X ; A ) = 1N ∑N i=1 ` ( xi ; A ( X ) ) be a loss function . Then , the influence function of a training data xi over a test data z ∈ Rd is the loss of z computed from the model trained without xi minus that computed from the model trained with xi . If the difference is large , then xi should be very influential for z . Formally , the influence function is defined below . Definition 1 ( Influence functions [ Koh and Liang , 2017 ] ) . Let X−i = X \ { xi } . Then , the influence of xi over z is defined as IFX , A ( xi , z ) = ` ( z ; A ( X−i ) ) − ` ( z ; A ( X ) ) . If IFX , A ( xi , z ) > 0 , we say xi is a proponent of z ; otherwise , we say xi is an opponent of z . For big models A such as deep neural networks , doing retraining and obtaining A ( X−i ) can be expensive . The following TracIn score is a fast approximation to IF . Definition 2 ( TracIn scores [ Pruthi et al. , 2020 ] ) . SupposeA ( X ) is obtained by minimizing L ( X ; A ) via stochastic gradient descent . Let { θ [ c ] } Cc=1 be C checkpoints during the training procedure . Then , the estimated influence of xi over z is defined as TracInX , A ( xi , z ) = ∑C c=1∇ ` ( xi ; θ [ c ] ) > ∇ ` ( z ; θ [ c ] ) . We are also interested in the influence of a training sample over itself . Formally , we define this quantity as the self influence of x , or Self-IFX , A ( x ) = IFX , A ( x , x ) . In supervised learning , self influences provide rich information about memorization properties of training samples . Intuitively , high self influence samples are atypical , ambiguous or mislabeled , while low self influence samples are typical [ Feldman and Zhang , 2020 ] . 3 Influence Functions for Classical Unsupervised Learning . In this section , we analyze influence functions for unsupervised learning . The goal is to provide intuition on what influence functions should tell us in the unsupervised setting . Specifically , we look at three classical methods : the non-parametric k-nearest-neighbor ( k-NN ) density estimator , the non-parametric kernel density estimator ( KDE ) , and the parametric Gaussian mixture models ( GMM ) . We let the loss function ` to be the negative log-likelihood : ` ( z ) = − log p ( z ) . The k-Nearest-Neighbor ( k-NN ) density estimator . The k-NN density estimator is defined as pkNN ( x ; X ) = k/ ( NVdRk ( x ; X ) d ) , where Rk ( x ; X ) is the distance between x and its k-th nearest neighbor in X and Vd is the volume of the unit ball in Rd . Then , we have the following influence function for the k-NN density estimator : IFX , kNN ( xi , z ) = log N − 1 N + { d log Rk+1 ( z ; X ) Rk ( z ; X ) ‖xi − z‖ ≤ Rk ( z ; X ) 0 otherwise . ( 1 ) See Appendix A.1.1 for proof . Note , when z is fixed , there are only two possible values for training data influences : log N−1N and log N−1 N + d log Rk+1 ( z ; X ) Rk ( z ; X ) . As for Self-IFX , kNN ( xi ) , samples with the largest self influences are those with the largest Rk+1 ( xi ; X ) Rk ( xi ; X ) . Intuitively , these samples belong to a cluster of size exactly k , and the cluster is far away from other samples . Kernel Density Estimators ( KDE ) . The KDE is defined as pKDE ( x ; X ) = 1N ∑N i=1Kσ ( x − xi ) , where Kσ is the Gaussian N ( 0 , σ2I ) . Then , we have the following influence function for KDE : IFX , KDE ( xi , z ) = log N − 1 N + log ( 1 + 1 NKσ ( z − xi ) pKDE ( z ; X ) − 1NKσ ( z − xi ) ) . ( 2 ) See Appendix A.1.2 for proof . For a fixed z , an xi with larger ‖z − xi‖ has a higher influence over z . Therefore , the strongest proponents of z are those closest to z in the ` 2 distance , and the strongest opponents are the farthest . As for Self-IFX , KDE ( xi ) , samples with the largest self influences are those with the least likelihood pKDE ( xi ; X ) . Intuitively , these samples locate at very sparse regions and have few nearby samples . On the other hand , samples with the largest likelihood pKDE ( xi ; X ) , or those in the high density area , have the least self influences . Gaussian Mixture Models ( GMM ) . As there is no closed-form expression for general GMM , we make the following well-separation assumption to simplify the problem . Assumption 1 . X = ⋃K−1 k=0 Xk where each Xk is a cluster . We assume these clusters are wellseparated : min { ‖x− x′‖ : x ∈ Xk , x′ ∈ Xk′ } max { ‖x− x′‖ : x , x′ ∈ Xk } . Let |Xk| = Nk and N = ∑K−1 k=0 Nk . For x ∈ Rd , let k = arg mini d ( x , Xi ) . Then , we define the well-separated spherical GMM ( WS-GMM ) of K mixtures as pWS-GMM ( x ) = NkN N ( x ; µk , σ 2 kI ) , where the parameters are given by the maximum likelihood estimates µk = 1 Nk ∑ x∈Xk x , σ2k = 1 Nkd ∑ x∈Xk ‖x− µk‖2 = 1 Nkd ∑ x∈Xk x > x− 1 d µ > k µk . ( 3 ) For conciseness , we let test sample z from cluster zero : z ∈ conv ( X0 ) . Then , we have the following influence function for WS-GMM . If xi /∈ X0 , IFX , WS-GMM ( xi , z ) = − 1N +O ( N−2 ) . Otherwise , IFX , WS-GMM ( xi , z ) = d+ 2 2N0 + 1 2N0σ20 ( ‖z − µ0‖2 σ20 − ‖z − xi‖2 ) − 1 N +O ( N−20 ) . ( 4 ) See Appendix A.1.3 for proof . A surprising finding is that some xi ∈ X0 may have very negative influences over z ( i.e . strong opponents of z are from the same class ) . This happens with high probability if ‖z − xi‖2 ' ( 1 + σ20 ) d + 2σ20 for large dimension d. Next , we compute the self influence of an xi ∈ Xk . According to ( 4 ) , Self-IFX , WS-GMM ( xi ) = d+ 2 2Nk + ‖xi − µk‖2 2Nkσ4k − 1 N +O ( N−2k ) . ( 5 ) Within each cluster Xk , samples far away to the cluster center µk have large self influences and vice versa . Across the entire dataset , samples in cluster Xk whose Nk or σk is small tend to have large self influences , which is very different from k-NN or KDE .

Summary: The paper investigates influence functions for several classical unsupervised learning methods and VAE. Influence functions in unsupervised learning can reveal the most responsible training samples that increase the likelihood (or reduce the loss) of a particular test sample. A particular usage of the proposed approach is for data cleaning. The authors demonstrated their approach on the MNIST and CIFAR-10 datasets.