FastSHAP: Real-Time Shapley Value Estimation

1 INTRODUCTION . With the proliferation of black-box models , Shapley values ( Shapley , 1953 ) have emerged as a popular explanation approach due to their strong theoretical properties ( Lipovetsky and Conklin , 2001 ; Štrumbelj and Kononenko , 2014 ; Datta et al. , 2016 ; Lundberg and Lee , 2017 ) . In practice , however , Shapley value-based explanations are known to have high computational complexity , with an exact calculation requiring an exponential number of model evaluations ( Van den Broeck et al. , 2021 ) . Speed becomes a critical issue as models increase in size and dimensionality , and for the largest models in fields such as computer vision and natural language processing , there is an unmet need for significantly faster Shapley value approximations that maintain high accuracy . Recent work has addressed the computational challenges with Shapley values using two main approaches . First , many works have proposed stochastic estimators ( Castro et al. , 2009 ; Štrumbelj and Kononenko , 2014 ; Lundberg and Lee , 2017 ; Covert et al. , 2020 ) that rely on sampling either feature subsets or permutations ; though often consistent , these estimators require many model evaluations and impose an undesirable trade-off between run-time and accuracy . Second , some works have proposed model-specific approximations , e.g. , for trees ( Lundberg et al. , 2020 ) or neural networks ( Shrikumar et al. , 2017 ; Chen et al. , 2018b ; Ancona et al. , 2019 ; Wang et al. , 2021 ) ; while generally faster , these approaches can still require many model evaluations , often induce bias , and typically lack flexibility regarding the handling held-out features—a subject of ongoing debate in the field ( Aas et al. , 2019 ; Janzing et al. , 2020 ; Frye et al. , 2020 ; Covert et al. , 2021 ) . Here , we introduce a new approach for efficient Shapley value estimation : to achieve the fastest possible run-time , we propose learning a separate explainer model that outputs precise Shapley value estimates in a single forward pass . Naïvely , such a learning-based approach would seem to require a large training set of ground truth Shapley values , which would be computationally intractable . Instead , our approach trains an explainer model by minimizing an objective function inspired by the Shapley value ’ s weighted least squares characterization ( Charnes et al. , 1988 ) , which enables efficient gradient-based optimization . Our contributions . We introduce FastSHAP , an amortized approach for generating real-time Shapley value explanations.1 We derive an objective function from the Shapley value ’ s weighted least ∗Equal contribution 1https : //git.io/JCqFV ( PyTorch ) , https : //git.io/JCqbP ( TensorFlow ) squares characterization and investigate several ways to reduce gradient variance during training . Our experiments show that FastSHAP provides accurate Shapley value estimates with an ordersof-magnitude speedup relative to non-amortized estimation approaches . Finally , we also find that FastSHAP generates high-quality image explanations ( fig . 1 ) that outperform gradient-based methods ( e.g. , IntGrad and GradCAM ) on quantitative inclusion and exclusion metrics . 2 BACKGROUND . In this section , we introduce notation used throughout the paper and provide an overview of Shapley values and their weighted least squares characterization . Let x ∈ X be a random vector consisting of d features , or x = ( x1 , . . . , xd ) , and let y ∈ Y = { 1 , . . . , K } be the response variable for a classification problem . We use s ∈ { 0 , 1 } d to denote subsets of the indices { 1 , . . . , d } and define xs : = { xi } i : si=1 . The symbols x , y , s are random variables and x , y , s denote possible values . We use 1 and 0 to denote vectors of ones and zeros in Rd , so that 1⊤s is a subset ’ s cardinality , and we use ei to denote the ith standard basis vector . Finally , f ( x ; η ) : X 7→ ∆K−1 is a model that outputs a probability distribution over y given x , and fy ( x ; η ) is the probability for the yth class . 2.1 SHAPLEY VALUES . Shapley values were originally developed as a credit allocation technique in cooperative game theory ( Shapley , 1953 ) , but they have since been adopted to explain predictions from black-box machine learning models ( Štrumbelj and Kononenko , 2014 ; Datta et al. , 2016 ; Lundberg and Lee , 2017 ) . For any value function ( or set function ) v : 2d 7→ R , the Shapley values ϕ ( v ) ∈ Rd , or ϕi ( v ) ∈ R for each feature i = 1 , . . . , d , are given by the formula ϕi ( v ) = 1 d ∑ si ̸=1 ( d− 1 1⊤s ) −1 ( v ( s+ ei ) − v ( s ) ) . ( 1 ) The difference v ( s + ei ) − v ( s ) represents the ith feature ’ s contribution to the subset s , and the summation represents a weighted average across all subsets that do not include i . In the model explanation context , the value function is chosen to represent how an individual prediction varies as different subsets of features are removed . For example , given an input-output pair ( x , y ) , the prediction for the yth class can be represented by a value function vx , y defined as vx , y ( s ) = link ( E p ( x1−s ) [ fy ( xs , x1−s ; η ) ] ) , ( 2 ) where the held out features x1−s are marginalized out using their joint marginal distribution p ( x1−s ) , and a link function ( e.g. , logit ) is applied to the model output . Recent work has debated the properties of different value function formulations , particularly the choice of how to remove features ( Aas et al. , 2019 ; Janzing et al. , 2020 ; Frye et al. , 2020 ; Covert et al. , 2021 ) . However , regardless of the formulation , this approach to model explanation enjoys several useful theoretical properties due to its use of Shapley values : for example , the attributions are zero for irrelevant features , and they are guaranteed to sum to the model ’ s prediction . We direct readers to prior work for a detailed discussion of these properties ( Lundberg and Lee , 2017 ; Covert et al. , 2021 ) . Unfortunately , Shapley values also introduce computational challenges : the summation in eq . ( 1 ) involves an exponential number of subsets , which makes it infeasible to calculate for large d. Fast approximations are therefore required in practice , as we discuss next . 2.2 KERNELSHAP . KernelSHAP ( Lundberg and Lee , 2017 ) is a popular Shapley value implementation that relies on an alternative Shapley value interpretation . Given a value function vx , y ( s ) , eq . ( 1 ) shows that the values ϕ ( vx , y ) are the features ’ weighted average contributions ; equivalently , their weighted least squares characterization says that they are the solution to an optimization problem over ϕx , y ∈ Rd , ϕ ( vx , y ) = argmin ϕx , y E p ( s ) [ ( vx , y ( s ) − vx , y ( 0 ) − s⊤ϕx , y ) 2 ] ( 3 ) s.t . 1⊤ϕx , y = vx , y ( 1 ) − vx , y ( 0 ) , ( Efficiency constraint ) where the distribution p ( s ) is defined as p ( s ) ∝ d− 1 ( d 1⊤s ) · 1⊤s · ( d− 1⊤s ) ( Shapley kernel ) for s such that 0 < 1⊤s < d ( Charnes et al. , 1988 ) . Based on this view of the Shapley value , Lundberg and Lee ( 2017 ) introduced KernelSHAP , a stochastic estimator that solves an approximate version of eq . ( 3 ) given some number of subsets sampled from p ( s ) . Although the estimator is consistent and empirically unbiased ( Covert and Lee , 2021 ) , KernelSHAP often requires many samples to achieve an accurate estimate , and it must solve eq . ( 3 ) separately for each input-output pair ( x , y ) . As a result , it is unacceptably slow for some use cases , particularly in settings with large , high-dimensional models . Our approach builds on KernelSHAP , leveraging the Shapley value ’ s weighted least squares characterization to design a faster , amortized estimation approach . 3 FASTSHAP . We now introduce FastSHAP , a method that amortizes the cost of generating Shapley values across many data samples . FastSHAP has two main advantages over existing approaches : ( 1 ) it avoids solving separate optimization problems for each input to be explained , and ( 2 ) it can use similar data points to efficiently learn the Shapley value function ϕ ( vx , y ) . 3.1 AMORTIZING SHAPLEY VALUES . In our approach , we propose generating Shapley value explanations using a learned parametric function ϕfast ( x , y ; θ ) : X × Y 7→ Rd . Once trained , the parametric function can generate explanations in a single forward pass , providing a significant speedup over methods that approximate Shapley values separately for each sample ( x , y ) . Rather than using a dataset of ground truth Shapley values for training , we train ϕfast ( x , y ; θ ) by penalizing its predictions according to the weighted least squares objective in eq . ( 3 ) , or by minimizing the following loss , L ( θ ) = E p ( x ) E Unif ( y ) E p ( s ) [ ( vx , y ( s ) − vx , y ( 0 ) − s⊤ϕfast ( x , y ; θ ) ) 2 ] , ( 4 ) where Unif ( y ) represents a uniform distribution over classes . If the model ’ s predictions are forced to satisfy the Efficiency constraint , then given a large enough dataset and a sufficiently expressive model class for ϕfast , the global optimizer ϕfast ( x , y ; θ∗ ) is a function that outputs exact Shapley values ( see proof in appendix A ) . Formally , the global optimizer satisfies the following : ϕfast ( x , y ; θ ∗ ) = ϕ ( vx , y ) ∀ x ∈ X , y ∈ Y . ( 5 ) We explore two approaches to address the efficiency requirement . First , we can enforce efficiency by adjusting the Shapley value predictions using their additive efficient normalization ( Ruiz et al. , 1998 ) , which applies the following operation to the model ’ s outputs : ϕefffast ( x , y ; θ ) = ϕfast ( x , y ; θ ) + 1 d ( vx , y ( 1 ) − vx , y ( 0 ) − 1⊤ϕfast ( x , y ; θ ) ) ︸ ︷︷ ︸ Efficiency gap . ( 6 ) The normalization step can be applied at inference time and optionally during training ; in appendix B , we show that this step is guaranteed to make the estimates closer to the true Shapley values . Second , we can relax the efficiency property by augmenting L ( θ ) with a penalty on the efficiency gap ( see eq . ( 6 ) ) ; the penalty requires a parameter γ > 0 , and as we set γ → ∞ we can guarantee that efficiency holds ( see appendix A ) . Algorithm 1 summarizes our training approach . Empirical considerations . Optimizing L ( θ ) using a single set of samples ( x , y , s ) is problematic because of high variance in the gradients , which can lead to poor optimization . We therefore consider several steps to reduce gradient variance . First , as is conventional in deep learning , we minibatch across multiple samples from p ( x ) . Next , when possible , we calculate the loss jointly across all classes y ∈ { 1 , . . . , K } . Then , we experiment with using multiple samples s ∼ p ( s ) for each input sample x . Finally , we explore paired sampling , where each sample s is paired with its complement 1−s , which has been shown to reduce KernelSHAP ’ s variance ( Covert and Lee , 2021 ) . Appendix C shows proofs that these steps are guaranteed to reduce gradient variance , and ablation experiments in appendix D demonstrate their improvement on FastSHAP ’ s accuracy .

Summary: This paper proposes FastSHAP to efficiently estimate the Shapley value in a single forward pass using a learned explainer model. Since there is no label to train the Shapley value estimator, stochastic gradient optimization using a weighted least squares-like objective function is applied to train FastSHAP. The experimental results and the deployment efficiency shows the superiority of the presented method.