Understanding Instance-based Interpretability of Variational Auto-Encoders

1 Introduction . Instance-based interpretation methods have been popular for supervised learning as they help explain why a model makes a certain prediction and hence have many applications [ Barshan et al. , 2020 , Basu et al. , 2020 , Chen et al. , 2020 , Ghorbani and Zou , 2019 , Hara et al. , 2019 , Harutyunyan et al. , 2021 , Koh and Liang , 2017 , Koh et al. , 2019 , Pruthi et al. , 2020 , Yeh et al. , 2018 , Yoon et al. , 2020 ] . For a classifier and a test sample z , an instance-based interpretation ranks all training samples x according to an interpretability score between x and z . Samples with high ( low ) scores are considered positively ( negatively ) important for the prediction of z . However , in the literature of unsupervised learning especially generative models , instance-based interpretations are much less understood . In this work , we investigate instance-based interpretation methods for unsupervised learning based on influence functions [ Cook and Weisberg , 1980 , Koh and Liang , 2017 ] . In particular , we theoretically analyze certain classical non-parametric and parametric methods . Then , we look at a canonical deep generative model , variational auto-encoders ( VAE , [ Higgins et al. , 2016 , Kingma and Welling , 2013 ] ) , and explore some of the applications . The first challenge is framing the counter-factual question for unsupervised learning . For instancebased interpretability in supervised learning , the counter-factual question is `` which training samples are most responsible for the prediction of a test sample ? '' – which heavily relies on the label information . However , there is no label in unsupervised learning . In this work , we frame the counterfactual question for unsupervised learning as `` which training samples are most responsible for increasing the likelihood ( or reducing the loss when likelihood is not available ) of a test sample ? '' We show that influence functions can answer this counter-factual question . Then , we examine influence functions for several classical unsupervised learning methods . We present theory and intuitions on how influence functions relate to likelihood and pairwise distances . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . The second challenge is how to compute influence functions in VAE . The first difficulty here is that the VAE loss of a test sample involves an expectation over the encoder , so the actual influence function can not be precisely computed . To deal with this problem , we use the empirical average of influence functions , and prove a concentration bound of the empirical average under mild conditions . Another difficulty is computation . The influence function involves inverting the Hessian of the loss with respect to all parameters , which involves massive computation for big neural networks with millions of parameters . To deal with this problem , we adapt a first-order estimate of the influence function called TracIn [ Pruthi et al. , 2020 ] to VAE . We call our method VAE-TracIn . It is fast because ( i ) it does not involve the Hessian , and ( ii ) it can accelerate computation with only a few checkpoints . We begin with a sanity check that examines whether training samples have the highest influences over themselves , and show VAE-TracIn passes it . We then evaluate VAE-TracIn on several real world datasets . We find high ( low ) self influence training samples have large ( small ) losses . Intuitively , high self influence samples are hard to recognize or visually high-contrast , while low self influence ones share similar shapes or background . These findings lead to an application on unsupervised data cleaning , as high self influence samples are likely to be outside the data manifold . We then provide quantitative and visual analysis on influences over test data . We call high and low influence samples proponents and opponents , respectively . 1 We find in certain cases both proponents and opponents are similar samples from the same class , while in other cases proponents have large norms . We consider VAE-TracIn as a general-purpose tool that can potentially help understand many aspects in the unsupervised setting , including ( i ) detecting underlying memorization , bias or bugs [ Feldman and Zhang , 2020 ] in unsupervised learning , ( ii ) performing data deletion [ Asokan and Seelamantula , 2020 , Izzo et al. , 2021 ] in generative models , and ( iii ) examining training data without label information . We make the following contributions in this paper . • We formally frame instance-based interpretations for unsupervised learning . • We examine influence functions for several classical unsupervised learning methods . • We present VAE-TracIn , an instance-based interpretation method for VAE . We provide both theoretical and empirical justification to VAE-TracIn . • We evaluate VAE-TracIn on several real world datasets . We provide extensive quantitative analysis and visualization , as well as an application on unsupervised data cleaning . 1.1 Related Work . There are two lines of research on instance-based interpretation methods for supervised learning . The first line of research frames the following counter-factual question : which training samples are most responsible for the prediction of a particular test sample z ? This is answered by designing an interpretability score that measures the importance of training samples over z and selecting those with the highest scores . Many scores and their approximations have been proposed [ Barshan et al. , 2020 , Basu et al. , 2020 , Chen et al. , 2020 , Hara et al. , 2019 , Koh and Liang , 2017 , Koh et al. , 2019 , Pruthi et al. , 2020 , Yeh et al. , 2018 ] . Specifically , Koh and Liang [ 2017 ] introduce the influence function ( IF ) based on the terminology in robust statistics [ Cook and Weisberg , 1980 ] . The intuition is removing an important training sample of z should result in a huge increase of its test loss . Because the IF is hard to compute , Pruthi et al . [ 2020 ] propose TracIn , a fast first-order approximation to IF . Our paper extends the counter-factual question to unsupervised learning where there is no label . We ask : which training samples are most responsible for increasing the likelihood ( or reducing the loss ) of a test sample ? In this paper , we propose VAE-TracIn , an instance-based interpretation method for VAE [ Higgins et al. , 2016 , Kingma and Welling , 2013 ] based on TracIn and IF . The second line of research considers a different counter-factual question : which training samples are most responsible for the overall performance of the model ( e.g . accuracy ) ? This is answered by designing an interpretability score for each training sample . Again many scores have been proposed [ Ghorbani and Zou , 2019 , Harutyunyan et al. , 2021 , Yoon et al. , 2020 ] . Terashita et al . [ 2021 ] extend this framework to a specific unsupervised model called generative adversarial networks [ Goodfellow 1There are different names in the literature , such as helpful/harmful samples [ Koh and Liang , 2017 ] , excitatory/inhibitory points [ Yeh et al. , 2018 ] , and proponents/opponents [ Pruthi et al. , 2020 ] . et al. , 2014 ] . They measure influences of samples on several evaluation metrics , and discard samples that harm these metrics . Our paper is orthogonal to these works . The instance-based interpretation methods lead to many applications in various areas including adversarial learning , data cleaning , prototype selection , data summarization , and outlier detection [ Barshan et al. , 2020 , Basu et al. , 2020 , Chen et al. , 2020 , Feldman and Zhang , 2020 , Ghorbani and Zou , 2019 , Hara et al. , 2019 , Harutyunyan et al. , 2021 , Khanna et al. , 2019 , Koh and Liang , 2017 , Pruthi et al. , 2020 , Suzuki et al. , 2021 , Ting and Brochu , 2018 , Ye et al. , 2021 , Yeh et al. , 2018 , Yoon et al. , 2020 ] . In this paper , we apply the proposed VAE-TracIn to an unsupervised data cleaning task . Prior works on interpreting generative models analyze their latent space via measuring disentanglement , explaining and visualizing representations , or analysis in an interactive interface [ Alvarez-Melis and Jaakkola , 2018 , Bengio et al. , 2013 , Chen et al. , 2016 , Desjardins et al. , 2012 , Kim and Mnih , 2018 , Olah et al. , 2017 , 2018 , Ross et al. , 2021 ] . These latent space analysis are complementary to the instance-based interpretation methods in this paper . 2 Instance-based Interpretations . Let X = { xi } Ni=1 ∈ Rd be the training set . Let A be an algorithm that takes X as input and outputs a model that describes the distribution of X . A can be a density estimator or a generative model . Let L ( X ; A ) = 1N ∑N i=1 ` ( xi ; A ( X ) ) be a loss function . Then , the influence function of a training data xi over a test data z ∈ Rd is the loss of z computed from the model trained without xi minus that computed from the model trained with xi . If the difference is large , then xi should be very influential for z . Formally , the influence function is defined below . Definition 1 ( Influence functions [ Koh and Liang , 2017 ] ) . Let X−i = X \ { xi } . Then , the influence of xi over z is defined as IFX , A ( xi , z ) = ` ( z ; A ( X−i ) ) − ` ( z ; A ( X ) ) . If IFX , A ( xi , z ) > 0 , we say xi is a proponent of z ; otherwise , we say xi is an opponent of z . For big models A such as deep neural networks , doing retraining and obtaining A ( X−i ) can be expensive . The following TracIn score is a fast approximation to IF . Definition 2 ( TracIn scores [ Pruthi et al. , 2020 ] ) . SupposeA ( X ) is obtained by minimizing L ( X ; A ) via stochastic gradient descent . Let { θ [ c ] } Cc=1 be C checkpoints during the training procedure . Then , the estimated influence of xi over z is defined as TracInX , A ( xi , z ) = ∑C c=1∇ ` ( xi ; θ [ c ] ) > ∇ ` ( z ; θ [ c ] ) . We are also interested in the influence of a training sample over itself . Formally , we define this quantity as the self influence of x , or Self-IFX , A ( x ) = IFX , A ( x , x ) . In supervised learning , self influences provide rich information about memorization properties of training samples . Intuitively , high self influence samples are atypical , ambiguous or mislabeled , while low self influence samples are typical [ Feldman and Zhang , 2020 ] . 3 Influence Functions for Classical Unsupervised Learning . In this section , we analyze influence functions for unsupervised learning . The goal is to provide intuition on what influence functions should tell us in the unsupervised setting . Specifically , we look at three classical methods : the non-parametric k-nearest-neighbor ( k-NN ) density estimator , the non-parametric kernel density estimator ( KDE ) , and the parametric Gaussian mixture models ( GMM ) . We let the loss function ` to be the negative log-likelihood : ` ( z ) = − log p ( z ) . The k-Nearest-Neighbor ( k-NN ) density estimator . The k-NN density estimator is defined as pkNN ( x ; X ) = k/ ( NVdRk ( x ; X ) d ) , where Rk ( x ; X ) is the distance between x and its k-th nearest neighbor in X and Vd is the volume of the unit ball in Rd . Then , we have the following influence function for the k-NN density estimator : IFX , kNN ( xi , z ) = log N − 1 N + { d log Rk+1 ( z ; X ) Rk ( z ; X ) ‖xi − z‖ ≤ Rk ( z ; X ) 0 otherwise . ( 1 ) See Appendix A.1.1 for proof . Note , when z is fixed , there are only two possible values for training data influences : log N−1N and log N−1 N + d log Rk+1 ( z ; X ) Rk ( z ; X ) . As for Self-IFX , kNN ( xi ) , samples with the largest self influences are those with the largest Rk+1 ( xi ; X ) Rk ( xi ; X ) . Intuitively , these samples belong to a cluster of size exactly k , and the cluster is far away from other samples . Kernel Density Estimators ( KDE ) . The KDE is defined as pKDE ( x ; X ) = 1N ∑N i=1Kσ ( x − xi ) , where Kσ is the Gaussian N ( 0 , σ2I ) . Then , we have the following influence function for KDE : IFX , KDE ( xi , z ) = log N − 1 N + log ( 1 + 1 NKσ ( z − xi ) pKDE ( z ; X ) − 1NKσ ( z − xi ) ) . ( 2 ) See Appendix A.1.2 for proof . For a fixed z , an xi with larger ‖z − xi‖ has a higher influence over z . Therefore , the strongest proponents of z are those closest to z in the ` 2 distance , and the strongest opponents are the farthest . As for Self-IFX , KDE ( xi ) , samples with the largest self influences are those with the least likelihood pKDE ( xi ; X ) . Intuitively , these samples locate at very sparse regions and have few nearby samples . On the other hand , samples with the largest likelihood pKDE ( xi ; X ) , or those in the high density area , have the least self influences . Gaussian Mixture Models ( GMM ) . As there is no closed-form expression for general GMM , we make the following well-separation assumption to simplify the problem . Assumption 1 . X = ⋃K−1 k=0 Xk where each Xk is a cluster . We assume these clusters are wellseparated : min { ‖x− x′‖ : x ∈ Xk , x′ ∈ Xk′ } max { ‖x− x′‖ : x , x′ ∈ Xk } . Let |Xk| = Nk and N = ∑K−1 k=0 Nk . For x ∈ Rd , let k = arg mini d ( x , Xi ) . Then , we define the well-separated spherical GMM ( WS-GMM ) of K mixtures as pWS-GMM ( x ) = NkN N ( x ; µk , σ 2 kI ) , where the parameters are given by the maximum likelihood estimates µk = 1 Nk ∑ x∈Xk x , σ2k = 1 Nkd ∑ x∈Xk ‖x− µk‖2 = 1 Nkd ∑ x∈Xk x > x− 1 d µ > k µk . ( 3 ) For conciseness , we let test sample z from cluster zero : z ∈ conv ( X0 ) . Then , we have the following influence function for WS-GMM . If xi /∈ X0 , IFX , WS-GMM ( xi , z ) = − 1N +O ( N−2 ) . Otherwise , IFX , WS-GMM ( xi , z ) = d+ 2 2N0 + 1 2N0σ20 ( ‖z − µ0‖2 σ20 − ‖z − xi‖2 ) − 1 N +O ( N−20 ) . ( 4 ) See Appendix A.1.3 for proof . A surprising finding is that some xi ∈ X0 may have very negative influences over z ( i.e . strong opponents of z are from the same class ) . This happens with high probability if ‖z − xi‖2 ' ( 1 + σ20 ) d + 2σ20 for large dimension d. Next , we compute the self influence of an xi ∈ Xk . According to ( 4 ) , Self-IFX , WS-GMM ( xi ) = d+ 2 2Nk + ‖xi − µk‖2 2Nkσ4k − 1 N +O ( N−2k ) . ( 5 ) Within each cluster Xk , samples far away to the cluster center µk have large self influences and vice versa . Across the entire dataset , samples in cluster Xk whose Nk or σk is small tend to have large self influences , which is very different from k-NN or KDE .

Summary: This paper provides the mathematical frame for influence functions (IFs) for several unsupervised machine learning approaches such as kNN, KDE, and GMM, and proposes a way to empirically compute VAE's IF, and performs several experiments to describe the rationale for using IF for unsupervised tasks. Their approach, VAE-TracIn, can answer which training samples are most responsible for increasing the likelihood of itself (i.e. self-influences) and/or a test sample. The authors claim that the self-influences can be used to discard undesired training samples before training and provide several experimental results showing that IFs for a test sample can identify most and least responsible training samples.