Title: One Sample Fits All: Approximating All Probabilistic Values Simultaneously and Efficiently
Venue: NeurIPS 2024
Accepted: yes
Keywords: Beta Shapley values

weighted Banzhaf values

approximation

datamodels
Abstract: The concept of probabilistic values, such as Beta Shapley values and weighted Banzhaf values, has gained recent attention in applications like feature attribution and data valuation. However, exact computation of these values is often exponentially expensive, necessitating approximation techniques. Prior research has shown that the choice of probabilistic values significantly impacts downstream performance, with no universally superior option. Consequently, one may have to approximate multiple candidates and select the best-performing one. Although there have been many efforts to develop efficient estimators, none  are intended to approximate all probabilistic values both simultaneously and efficiently. In this work, we embark on the first exploration of achieving this goal. Adhering to the principle of maximum sample reuse and avoiding amplifying factors, we propose a one-sample-fits-all framework parameterized by a sampling vector to approximate  intermediate terms that can be converted to any probabilistic value. Leveraging the concept of $ (\epsilon, \delta) $-approximation, we theoretically identify a key formula that effectively determines the convergence rate of our framework. By optimizing the sampling vector using this formula, we obtain i) a one-for-all estimator that achieves the currently best time complexity for all probabilistic values on average, and ii) a faster generic estimator with the sampling vector optimally tuned for each probabilistic value. Particularly, our one-for-all estimator achieves the fastest convergence rate on Beta Shapley values, including the well-known Shapley value, both theoretically and empirically. Finally, we establish a connection between probabilistic values and the least square regression used in (regularized) datamodels, showing that our one-for-all estimator can solve a family of datamodels simultaneously. Our code is available at https://github.com/watml/one-for-all.

One Sample Fits All: Approximating All Probabilistic Values Simultaneously and Efficiently

Abstract

The concept of probabilistic values, such as Beta Shapley values and weighted Banzhaf values, has gained recent attention in applications like feature attribution and data valuation. However, exact computation of these values is often exponentially expensive, necessitating approximation techniques. Prior research has shown that the choice of probabilistic values significantly impacts downstream performance, with no universally superior option. Consequently, one may have to approximate multiple candidates and select the best-performing one. Although there have been many efforts to develop efficient estimators, none are intended to approximate all probabilistic values both simultaneously and efficiently. In this work, we embark on the first exploration of achieving this goal. Adhering to the principle of maximum sample reuse and avoiding amplifying factors, we propose a one-sample-fits-all framework parameterized by a sampling vector to approximate intermediate terms that can be converted to any probabilistic value. Leveraging the concept of ( ϵ, δ ) -approximation, we theoretically identify a key formula that effectively determines the convergence rate of our framework. By optimizing the sampling vector using this formula, we obtain i) a one-for-all estimator that achieves the currently best time complexity for all probabilistic values on average, and ii) a faster generic estimator with the sampling vector optimally tuned for each probabilistic value. Particularly, our one-for-all estimator achieves the fastest convergence rate on Beta Shapley values, including the well-known Shapley value, both theoretically and empirically. Finally, we establish a connection between probabilistic values and the least square regression used in (regularized) datamodels, showing that our one-for-all estimator can solve a family of datamodels simultaneously. Our code is available at ❤/a116/a116♣/a115✿✴✴❣✐/a116❤✉❜✳❝♦♠✴✇❛/a116♠❧✴♦♥❡✲❢♦/a114✲❛❧❧ .

1 Introduction

The problem of attribution is central in many aspects of machine learning (Rozemberczki et al. 2022). Examples include data valuation (Ghorbani and Zou 2019), feature attribution (Lundberg and Lee 2017), multi-agent reinforcement learning (Wang et al. 2022), data attribution (Ilyas et al. 2022), and the list goes on. One popular methodology is to leverage the concept of probabilistic values, which is uniquely characterized by the axioms of linearity, null, monotonicity and symmetry in cooperative game theory (Weber 1988). Recent studies demonstrate that downstream performance employing this concept relies on the choice of probabilistic values, and the best one varies (Kwon and Zou 2022b; Li and Yu 2023). Therefore, practitioners may resort to approximating multiple candidates of probabilistic values and then select the best-performing one (Kwon and Zou 2022b).

Yaoliang Yu

②❛♦❧✐❛♥❣✳②✉❅✉✇❛/a116❡/a114❧♦♦✳❝❛

In general, probabilistic values can only be approximated as they require exponentially many utility evaluations to compute exactly. Thus far, there has been a line of works devoted to designing efficient estimators for the Shapley value (e.g., Covert and Lee 2021; Jia et al. 2019; Kolpaczki et al. 2024; Zhang et al. 2023), while Li and Yu (2023) and Wang and Jia (2023b) proposed efficient estimators specific to weighted Banzhaf values. Despite recent progress in research on generic estimators for approximating any probabilistic value (Li and Yu 2024; Lin et al. 2022), none of them can approximate all probabilistic values simultaneously and efficiently . Overall, there is a strong demand for efficient one-for-all estimators, the possibilities of which will be explored in this work.

To sum up, we propose a O ne-sampleF itsA ll (OFA) framework parameterized by a sampling vector to approximate intermediate terms that can be converted to any probabilistic value. Particularly, our framework i) adheres to the principle of maximum sample reuse and ii) does not include amplifying factors in the conversion. These two properties are considered indispensable as we observe that i) the empirical fastest estimators designed for the Shapley value or weighted Banzhaf values all follow she principle of maximum sample reuse and ii) amplifying factors could deteriorate the convergence rates of estimators. Then, using the concept of ( ϵ, δ ) -approximation, i.e., P ( ∥ ˆ ϕ -ϕ ∥ 2 ≥ ϵ ) ≤ δ where ϕ refers to some probabilistic value and ˆ ϕ is its estimate, we theoretically identify a formula from our framework that effectively determines the corresponding convergence rate, through which the sampling vector can be optimized. Specifically, we deduce i) an efficient one-for-all estimator (OFA-A) while optimizing the formula for all probabilistic values on A verage and ii) a faster generic estimator (OFA-S) while the optimization is done for each S pecific probabilistic value. The results of our convergence analysis are summarized as follows:

(i) Our OFA-A achieves the convergence rate O ( n log n ) for all probabilistic values on average. Notably, O ( n log n ) is the currently-known best time complexity for some probabilistic values.

(ii) For Beta Shapley values parameterized by α, β ≥ 1 (Kwon and Zou 2022a), our OFA-A estimator requires O ( n log n ) utility evaluations to achieve an ( ϵ, δ ) -approximation simultaneously . Note that α = β = 1 corresponds to the commonly-used Shapley value (Shapley 1953). For the Shapley value, the previous best convergence rate is O ( n (log n ) 2 ) , 1 achieved by the group testing estimator (Wang and Jia 2023a, Theorem 6); however, we note that in our experiments the previous best-performing estimator is the complement estimator (Zhang et al. 2023), whose convergence rate is unknown. For Beta Shapley values with ( α = 1 , β > 1) or ( α > 1 , β = 1) , the previous best estimator requires O ( n (log n ) 3 ) utility evaluations instead (Li and Yu 2024, Proposition 4 and Remark 3).

(iii) For weighted Banzhaf values parameterized by 0 < w < 1 , the time complexity of our OFA-A is O ( n 3 2 log n ) , not rivaling the previous best convergence rate O ( n log n ) achieved by the estimator exclusive to weighted Banzhaf values (Li and Yu 2023, Proposition 2). Nevertheless, our OFA-S achieves the convergence rate of O ( n log n ) for both Beta Shapley values (with α, β ≥ 1 ) and weighted Banzhaf values.

In our experiments, the empirical convergence rates align well with the theoretical ones derived using the concept of ( ϵ, δ ) -approximation. Additionally, we establish a connection between probabilistic values and the least square regressions employed in datamodels (Ilyas et al. 2022), demonstrating that our OFA-A estimator can solve a family of datamodels simultaneously if it is the distances between feature coordinates that matter. This condition is met while using datamodels to detect similar training examples to a given target. Furthermore, we also identify a group of regularized datamodels that our OFA-A estimator can solve simultaneously without this condition.

2 Preliminaries

Let n be the number of players and [ n ] := { 1 , 2 , . . . , n } be the set of all players. In data valuation (feature attribution, respectively), n refers to the number of training data (features, respectively). For simplicity, we write S \ i and S ∪ i instead of S \{ i } and S ∪{ i } , respectively. Meanwhile, (lowercase) s denotes the cardinality of the set (uppercase) S . Then, each probabilistic value can be written as

where U : 2 [ n ] → R is a utility function and p ∈ R n is a non-negative vector such that ∑ n s =1 ( n -1 s -1 ) p s = 1 . Take data valuation as an example, U ( S ) may measure the performance of models trained on S ⊆ [ n ] , with which ϕ i ( U ) can be interpreted as the contribution of the i -th data point to the performance of models trained on [ n ] .

If there exists a (Borel) probability measure µ on the closed interval [0 , 1] such that p s = ∫ 1 0 w s -1 (1 -w ) n -s d µ ( w ) , then the resulting probabilistic value is referred to as a semi-value (Dubey et al. 1981). If µ represents a Dirac delta distribution δ a , the corresponding probabilistic value is referred to as the weighted Banzhaf value parameterized by a , or WBa . For Beta Shapley values, denoted by Beta ( α, β ) , µ ( A ) = ∫ A w β -1 (1 -w ) α -1 d w . In practice, the considered range of α or β is [1 , ∞ ) (Kwon and Zou 2022a,b). Particularly, Beta (1 , 1) , whose µ is the uniform distribution (over [0 , 1] ), corresponds to the Shapley value.

We will use the standard notion of ( ϵ, δ ) -approximation to analyze a (randomized) estimate ˆ ϕ of some probabilistic value ϕ .

Definition 1. We say a (randomized) estimate ˆ ϕ achieves an ( ϵ, δ ) -approximation of ϕ if P ( ∥ ˆ ϕ -ϕ ∥ 2 ≥ ϵ ) ≤ δ .

For instance, Wang and Jia (2023b, Theorem 4.9) proved that their proposed estimator requires O ( n ϵ 2 log n δ ) utility evaluations to achieve an ( ϵ, δ ) -approximation for WB0 . 5 , provided that ∥ U ∥ ∞ ≤ 1 . When ϵ and δ are considered fixed constants, we then simply say the estimator converges at O ( n log n ) rate.

3 Motivations

One-For-All Estimators In this paper, an estimator is referred to as one-for-all if it is capable of sampling subsets O nce to approximate A ll probabilistic values.

Though existing estimators are not designed to approximate all probabilistic values simultaneously, some of them can be easily modified for this end by using the weighted sampling technique. Take the sampling lift (SL) estimator (Moehle et al. 2022) as an example, its approximation is based on

If we fix the probability of sampling S to be the one, denoted by q ∈ R n , for the Shapley value, there is

which is the weighted sampling lift (WSL) estimator employed by Kwon and Zou (2022a). Therefore, we can store the accumulated results { U ( S ∪ i ) -U ( S ) } separately for each subset size of S so that they can be reweighted to be any probabilistic value.

The Effect of Amplifying Factors However, the scalars { p s +1 q s +1 } potentially introduce a nonnegligible factor into the theoretical convergence rate. To demonstrate, we take the WSL estimator as an example. In this case, ˆ ϕ i = 1 T ∑ T t =1 X t where { X t } T t =1 are i.i.d. random variable such that P ( X t = p s +1 q s +1 ( U ( S ∪ i ) -U ( S ))) = q s +1 and thus E [ X t ] = ϕ i . Assume that ∥ U ∥ ∞ ≤ 1 , by the Hoeffding's inequality, P ( | ˆ ϕ i -ϕ i | ≥ ϵ ) ≤ 2 exp ( -Tϵ 2 8 C 2 ) where C = max 1 ≤ k ≤ n p k q k . By solving 2 exp ( -Tϵ 2 8 C 2 ) ≤ δ , we eventually obtain T ≥ 8 C 2 ϵ 2 log 2 δ and therefore the convergence rate of ˆ ϕ i is O ( C 2 ϵ 2 log 2 δ ) . Consequently, if C →∞ as n →∞ , this theoretical convergence rate deteriorates asymptotically. For the Banzhaf value, p k = 1 2 n -1 ; since q k = ( k -1)!( n -k )! n ! , if k = n +1 2 , there is p k q k ∈ Θ( n 1 2 ) by the Stirling's approximation d ! ≃ √ d ( d e ) d . Therefore, C 2 introduces a factor of Θ( n ) into the theoretical convergence rate, though the derived formula may not be tight. If we switch the roles of p and q , the introduced factor C 2 is as worst as Θ(2 2 n ) . To generalize this idea, a formula is said to contain an amplifying factor if it involves γ · U ( S ) such that γ →∞ as n →∞ .

Regarding this, we notice that Kwon and Zou (2022b) resort to a one-for-all estimator based on

where m s = ( n -1 s -1 ) p s and each expectation is taken over the corresponding uniform distribution. We refer to this estimator as weightedSHAP in this work. As can be verified, Eq. (2) does not contain any amplifying factors, i.e., each m s does not grow as n →∞ .

The Principle of Maximum Sample Reuse However, estimators designed according to Eqs. (1) and (2) are not expected to be efficient as it does not obey the principle of maximum sample reuse. Precisely, an estimator adheres to the principle of maximum sample reuse if each sampled subset is used to update all estimates { ˆ ϕ i } i ∈ [ n ] . As analyzed by Zhang et al. (2023, Section 4.2), estimators based on sampled marginal contributions { U ( S ∪ i ) -U ( S ) } are impossible to meet the principle of maximum sample reuse. By contrast, we observe that the SHAP-IQ estimator proposed by Fumagalli et al. (2024) can also be adopted for this end, which employs the formula

where m s = ( n -1 s -1 ) p s , H = ∑ n -1 j =1 1 j , and P ( S ) ∝ ( n -2 s -1 ) -1 . In particular, SHAP-IQ is equal to the unbiased KernelSHAP estimator (Covert and Lee 2021) for the Shapley value; see (Fumagalli et al. 2024, Theorem 4.5). Although SHAP-IQ follows the principle of maximum sample reuse, it is apparent that Eq. (3) contains an amplifying factor H ∈ Θ(log n ) . Meanwhile, there is another line of research in quest of efficient estimators for the Shapley value by reducing the variance via the stratified sampling technique (Burgess and Chapman 2021; Castro et al. 2017; Maleki et al. 2013; Wu et al. 2023). However, such a technique also does not verify the principle of maximum sample reuse.

Empirical Evidence For convenience, we formally define the two aforementioned desirable properties for estimators to possess as P1: The underlying formula contains no amplifying factors and P2: Each sampled subset is used to update all the estimates { ˆ ϕ i } n i =1 . In Figure 1, we provide some experiment results while setting n = 24 to support our aforementioned informal analysis. Precisely, we implement six one-for-all estimators by combining the weighted sampling technique and the previous estimators. Some of our observations are:

(i) On WB-0.5, weightedSHAP, which satisfies P1 but not P2 , is empirically not comparable to MSR-Banzhaf that possesses both P1 and P2 . This observation supports the necessity of P2 .

(ii) On WB-0.5, SHAP-IQ sticks to P2 but not P1 . It is clear that SHAP-IQ also performs significantly worse than MSR-Banzhaf, which highlights the role of P1 .

(iii) The sudden rises of relative differences stem from the existence of significantly large amplifying factors. For WSL-Banzhaf on Beta (1 , 1) , the scalar for U ( i ) -U ( ∅ ) is as large as 2 n n !

In Table 1, we summarize the previous estimators in terms of P1 and P2 , and defer the technical details to Appendix D. Notably, the complement estimator is empirically the best for the Shapley value, while it is MSR for weighted Banzhaf values; both of them follow P1 and P2 .

0.04

0.02

0.0

0.02

0.04

0.04

0.02

0.0

0.02

0.04

OFA-A (ours)

OFA-S (ours)

WSL-Banzhaf

WSL-Shapley

ARM-Banzhaf

ARM-Shapley

MSR-Banzhaf

SHAP-IQ

weightedSHAP

permutation-Shapley

0

5000

10000

15000

20000

#utility evaluations per player

10

2

10

1

10 0

10 1

10 2

10 3

relative difference

0

5000

10000

15000

20000

#utility evaluations per player

10

1

10 0

10 1

10 2

relative difference

0

5000

10000

15000

20000

#utility evaluations per player

10

2

10

1

10 0

10 1

10 2

relative difference

Beta

(4

,

1)

Beta

(1

,

1)

Beta

(1

,

4)

0

5000

10000

15000

20000

#utility evaluations per player

10

2

10

1

10 0

10 1

10 2

relative difference

0

5000

10000

15000

20000

#utility evaluations per player

10

1

10 0

relative difference

0

5000

10000

15000

20000

#utility evaluations per player

10

1

10 0

10 1

10 2

relative difference

0

.

2

0

.

5

0

.

8

4 Main Results

The framework we propose is built upon

where m s = ( n -1 s -1 ) p s and each expectation is taken over the corresponding uniform distribution. For simplicity, we write ϕ + i,s = E i ∈ R,r = s [ U ( R )] and ϕ -i,s -1 = E i ̸∈ R,r = s -1 [ U ( R )] . Clearly, there is no amplifying factors in Eq. (4). Meanwhile, the structure of Eq. (4) suits the principle of maximum sample reuse. Since { ϕ + i,k } k =1 ,n -1 ,n and { ϕ -i,k } k =0 , 1 ,n -1 can be calculated exactly using 2 n +2 utility evaluations of U , our focus is to efficiently approximate { ϕ + i,s , ϕ -i,s } 2 ≤ s ≤ n -2 . The proposed framework is presented in Algorithm 1; q j is the probability of drawing a subset of [ n ] with size j +1 . To facilitate the choice of the sampling vector q ∈ R n -3 appearing in Algorithm 1, our first step is to theoretically ascertain a key formula that effectively determines the convergence rate of Algorithm 1.

Theorem 1. Assume i) ∥ U ∥ ∞ ≤ u and ii) 0 < ϵ ≤ √ 2 D ( m , q ) γ ( q ) 2 u 2 . For ˆ ϕ in Algorithm 1, it requires 4 nu 2 D ( m , q ) ϵ 2 log 8 n 2 δ evaluations of U to achieve P ( ∥ ˆ ϕ -ϕ ∥ 2 ≥ ϵ ) ≤ δ where

We remark that D ( m , q ) is jointly convex in m and q . The second assumption in Theorem 1 can be removed if we pre-allocate the number of sampled subsets for each ϕ + i,s or ϕ -i,s and draw subsets in a predetermined order; see the proof in Appendix A for details. Precisely, let T + i,s be the number of subsets for estimating ϕ + i,s , and define T -i,s similarly; then the pre-allocated numbers are T + i,s ≈ s · q s -1 n T and T -i,s ≈ ( n -s ) q s -1 n T , which are the expected values of T + i,s and T + i,s while using Algorithm 1; T refers to the total number of sampled subsets. By Theorem 1, the convergence rate of Algorithm 1 is O ( D ( m , q ) · n log n ) , and thus achieving the currently best convergence rate O ( n log n ) requires D ( m , q ) ∈ O (1) .

Algorithm 1: The O ne-SampleF itsA ll (OFA) Framework

4.1 A One-For-All Estimator

To obtain our one-for-all estimator, our goal is to find a q OFA-A such that D ( m , q OFA-A ) ∈ O (1) for as many m as possible. To this end, we define q OFA-A to be the uniquely optimal solution to

where ∆ = { m ∈ R n | m s ≥ 0 and ∑ n s =1 m s = 1 } and ν is the uniform distribution on ∆ . In our work, our OFA-A estimator refers to the use of q OFA-A in Algorithm 1.

Proposition 1. q OFA-A s -1 ∝ 1 √ ( s )( n -s ) and D ( q OFA-A ) ∈ O (1) . In other words, our OFA-A estimator achieves the convergence rate of O ( n log n ) simultaneously for all probabilistic values on average.

Our next proposition provides a condition on µ for semi-values such that our OFA-A estimator achieves the convergence rate of O ( n log n ) . In other words, we explicitly identify a subfamily of semi-values for which our OFA-A estimator achieves the currently best time complexity simultaneously.

Proposition 2. Our OFA-A estimator achieves the convergence rate of O ( n log n ) simultaneously for all semi-values whose probability density functions exist and are bounded. Particularly, Beta Shapley values with α, β ≥ 1 all satisfy this condition.

To our knowledge, the previous theoretically-fastest estimator for the Shapley value is demonstrated by Wang and Jia (2023a, Theorem 6) as O ( n (log n ) 2 ) . By contrast, our OFA-A estimator achieves the convergence rate of O ( n log n ) . Meanwhile, it also surpasses the previous best time complexity for Beta Shapley values with ( α = 1 , β > 1) or ( α > 1 , β = 1) , which is O ( n (log n ) 3 ) (Li and Yu 2024, Proposition 4 and Remark 3). Remarkably, our OFA-A estimator enjoys this fastest convergence rate simultaneously for a broad subfamily of probabilistic values .

Proposition 3. If p s = a s -1 (1 -a ) n -s with 0 < a < 1 , which corresponds to the weighted Banzhaf value parameterized by w , then D ( m , q OFA-A ) ∈ O ( n 1 2 ) . In other words, the OFA estimator achieves the convergence rate of O ( n 3 2 log n ) simultaneously for all WBa with 0 < a < 1 .

The previous best convergence rate for weighted Banzhaf values is O ( n log n ) (Li and Yu 2023, Proposition 2), ours is slower by a factor of n 1 2 . Nevertheless, we will demonstrate that our generic estimator, which is expected to be faster than our OFA-A estimator, achieves the best convergence rate for all weighted Banzhaf values.

4.2 A Faster Generic Estimator

Our faster generic estimator (OFA-S) is obtained via optimizing q for each S pecific m . Precisely, for each m , we have

which can be obtained using the Cauchy-Schwarz inequality. For the Shapley value, q OFA-S = q OFA-A . Our next proposition specifies a sufficient condition for semi-values such that D ( m , q OFA-S ) ∈ O (1) .

Proposition 4. For semi-values, D ( m , q OFA-S ) ∈ O (1) if i) µ has a bounded probability density function or ii) ∫ (0 , 1) 1 w (1 -w ) d µ ( w ) < ∞ . Particularly, this condition covers all weighted Banzhaf values and Beta Shapley values with α, β ≥ 1 .

All in all, we demonstrate that by sticking to the principle of maximum sample reuse and avoiding any amplifying factors, we are able to establish a generic estimator that achieves the currently best convergence rate for any previously-studied semi-value.

4.3 A Connection between Probabilistic Values and Datamodels

A datamodel, proposed by Ilyas et al. (2022), is to learn an easy-to-interpret surrogate to represent a model output distribution related to a specific test example. In this circumstance, the set of players [ n ] is identified with all the available training data. Precisely, the feature coordinates θ ∗ ∈ R n imputed to every data point in [ n ] is defined to be the uniquely optimal solution (together with a bias b ∗ ∈ R ) to the optimization problem

where η ∈ R n +1 is non-negative and ∑ n s =0 η s +1 > 0 . The weight vector η can be scaled such that the objective in the problem (6) can be treated as an expectation, and thus the objective can be approximated through sampling a sufficient number of subsets, upon which an estimate of ( θ ∗ , b ∗ ) can be obtained. We show below that θ ∗ to a family of such least square regressions can be cast as some probabilistic values if it is the pairwise differences θ ∗ j -θ ∗ k (for every j, k ∈ [ n ] ) that matter.

Theorem 2. Let ( b ∗ , θ ∗ ) be the uniquely optimal solution to the problem (6) where η s = p s -1 + p s for 2 ≤ s ≤ n . Then, there is

In other words, θ ∗ = ϕ + c for some constant c 1 ; 1 ∈ R n is the all-one vector. When using datamodels to detect similar training examples to a given target, what matters is the relative order of components in θ ∗ . Meanwhile, Ilyas et al. (2022) showed that the corresponding performance depends on the choice of the weight vector η . Therefore, our OFA-A estimator serves as a sufficient proxy for a range of { θ ∗ } and would facilitate the fine-tuning of η when using datamodels to detect similar training examples.

When θ ∗ Can Be Recovered From ϕ Interestingly, for specific choices of p ∈ R n and η ∈ R n +1 , it holds that θ = ϕ . Theorem 2 can be seen as an extension to the previous result stated in the below.

Proposition 5 (Marichal and Mathonet 2011, Proposition 4) . Suppose 0 < a < 1 is given, if p j = a j -1 (1 -a ) n -j for 1 ≤ j ≤ n and η k = a k -2 (1 -a ) n -k for 1 ≤ k ≤ n +1 , which leads to η s = p s -1 + p s for 2 ≤ s ≤ n , there is

It is worth pointing out that ϕ in Proposition 5 is exactly the weighted Banzhaf value parameterized by a , i.e., WBa . Even more, under the same setting, we can even solve a group of datamodels with ℓ 1 or ℓ 2 regularization simultaneously.

Corollary 1. Under the setting of Proposition 5, let θ ∗ be the uniquely optimal solution to

where λ > 0 , the following are true about the relation between θ ∗ and ϕ :

All operations are element-wise.

This corollary is immediate by combining Proposition 5, and Theorem 2.2 by Saunshi et al. (2022). We comment that replacing x i by 2 x i -1 , i.e., mapping 0 and 1 into -1 and 1 , respectively, in ϕ { i } ( x ) used by Saunshi et al. (2022) yields v { i } ( x ) used by Marichal and Mathonet (2011). A remarkable implication of the combination of Corollary 1 and our proposed OFA-A estimator is that we can solve a group of regularized datamodels covered by the problem (7) simultaneously ! For example, the coefficient λ can be finetuned by running Algorithm 1 just once.

5 Experiments

In this section, we are to verify i) the simultaneous efficiency of our OFA-A estimator and ii) the faster convergence rate of our OFA-S estimator compared with the considered baselines and our OFA-A estimator. Particularly, if D ( m , q ) is effective in determining the convergence rate of Algorithm 1, our OFA-S estimator is expected to be faster than our OFA-A estimator. All the experiments are conducted using CPUs.

We use two types of utility functions for this end: i) following the experiment settings of (Li and Yu 2024), U ( S ) is set to be the cross-entropy of LeNets trained on S on the classification datasets FMNIST, MNIST and iris; to obtain the exact values, the number of training data n is set to be 24 ; ii) U is defined to be the sum of unanimity (SOU) games, i.e., U ( S ) = ∑ d j =1 α j 1 S j ⊆ S where each ∅ ⊊ S j ⊊ [ n ] is randomly sampled, for which each semi-value can be computed by ϕ i = ∑ d j =1 α j ∫ [0 , 1] w s j -1 d µ ( w ) ; specifically, we set n ∈ { 64 , 128 , 256 } with d = n 2 , which implies that the implemented SOU games require n 2 utility evaluations to compute semi-values exactly. The random seed inside each utility function is fixed as 2024 , and thus each U is deterministic.

For the simplicity of presenting our empirical results, we use the area under the convergence curve (AUCC) to assess the convergence quality of estimators, and thus the smaller the better. For n = 24 , the value of each player is approximated using 20 , 000 utility evaluations, and we compute the AUCCs as 1 100 ∑ 100 j =1 ∥ ˆ ϕ (200 j ) -ϕ ∥ 2 ∥ ϕ ∥ 2 where ˆ ϕ (200 j ) refers to the estimate using 200 j utility evaluations for each player. For n ∈ { 64 , 128 , 256 } , the value of each player is approximated using 2 , 000 utility evaluations, and the corresponding AUCCs are calculated as 1 100 ∑ 100 j =1 ∥ ˆ ϕ (20 j ) -ϕ ∥ 2 ∥ ϕ ∥ 2 . All the AUCCs are reported with standard deviation using 30 different random seeds from { 0 , 1 , 2 , . . . , 29 } .

Verification of Our OFA-A Estimator For our OFA-A estimator where we substitute q OFA-A , which is defined in Proposition 1, into Algorithm 1, we choose the baselines according to Figure 1. The selected baselines include WSL-Shapley (Kwon and Zou 2022a), SHAP-IQ (Fumagalli et al. 2024), weightedSHAP (Kwon and Zou 2022b) and permutation-Shapley (Castro et al. 2009). The corresponding results are reported in Figure 2. Overall, our OFA-A estimator performs the best on all the employed 18 probabilistic values, which verify the simultaneous efficiency of our OFA-A estimator.

0.04

0.04

0.02

0.0

0.02

0.04

0.04

0.02

0.0

0.02

OFA-A (ours)

WSL-Shapley

SHAP-IQ

weightedSHAP

permutation-Shapley

(16, 1)

(4, 1)

(1, 1)

(1, 4)

(1, 16)

(16, 4)

(2, 2)

(8, 8)

(4, 16)

Beta Shapley values

10

1

AUCC

(16, 1)

(4, 1)

(1, 1)

(1, 4)

(1, 16)

(16, 4)

(2, 2)

(8, 8)

(4, 16)

Beta Shapley values

10

1

AUCC

(16, 1)

(4, 1)

(1, 1)

(1, 4)

(1, 16)

(16, 4)

(2, 2)

(8, 8)

(4, 16)

Beta Shapley values

10

2

10

1

AUCC

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

weighted Banzhaf values

10

1

AUCC

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

weighted Banzhaf values

10

1

10 0

AUCC

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

weighted Banzhaf values

10

1

AUCC

n

= 24

)

MNIST (

n

= 24

)

FMNIST (

n

= 24

)

(16, 1)

(4, 1)

(1, 1)

(1, 4)

(1, 16)

(16, 4)

(2, 2)

(8, 8)

(4, 16)

Beta Shapley values

10

1

AUCC

(16, 1)

(4, 1)

(1, 1)

(1, 4)

(1, 16)

(16, 4)

(2, 2)

(8, 8)

(4, 16)

Beta Shapley values

10

1

AUCC

(16, 1)

(4, 1)

(1, 1)

(1, 4)

(1, 16)

(16, 4)

(2, 2)

(8, 8)

(4, 16)

Beta Shapley values

10

1

10 0

AUCC

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

weighted Banzhaf values

10

1

2×10 1

3×10 1

4×10 1

6×10 1

AUCC

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

weighted Banzhaf values

2×10 1

3×10 1

4×10 1

6×10 1

AUCC

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

weighted Banzhaf values

10 0

2×10 1

3×10 1

4×10 1

6×10 1

AUCC

SOU (

n

= 64

)

SOU (

n

= 128

)

SOU (

n

= 256

)

Verification of Our OFA-S Estimator Next, we verify the faster convergence rate of our OFAS estimator, using q OFA-S as defined in Eq. (5). The baselines we employ include: (unbiased) kernelSHAP (Covert and Lee 2021; Lundberg and Lee 2017), GELS and GELS-Shapley (Li and Yu 2024), ARM (Kolpaczki et al. 2024; Li and Yu 2024), complement (Zhang et al. 2023), group testing (Jia et al. 2019; Wang and Jia 2023a), AME (Lin et al. 2022), MSR (Wang and Jia 2023b) and sampling lift (Moehle et al. 2022). Note that not all the baselines are designed for all the probabilistic values we employ. For example, the complement estimator only works for Beta (1 , 1) , i.e., the Shapley value. According to (Li and Yu 2024, Remark 9), we implement the paired sampling technique for (unbiased) kernelSHAP and group testing. The corresponding results are presented in Figure 3.

First, our OFA-S estimator is indeed faster than our OFA-A estimator, which aligns exactly with our theory; in other words, it implies that our proposed D ( m , p ) indeed determines the convergence rate of our Algorithm 1. Second, our OFA-S estimator always performs the best except on the SOU games which require only n 2 utility evaluation to get the exact values; by contrast, the utility function defined using the classification datasets require 2 n utility evaluations instead. Third, our proposed estimator is consistently the fastest on the commonly-used Beta (1 , 1) , i.e., the Shapley value; note that q OFA-A = q OFA-S for the Shapley value; therefore, our proposed estimator achieves the currently best convergence rate both empirically and theoretically.

0.04

0.04

0.02

0.0

0.02

0.04

0.02

0.0

0.02

0.04

OFA-A (ours)

OFA-S (ours)

GELS

GELS-Shapley

sampling lift

kernelSHAP

unbiased kernelSHAP

group testing

ARM

complement

AME

MSR

(16, 1)

(4, 1)

(1, 1)

(1, 4)

(1, 16)

(16, 4)

(2, 2)

(8, 8)

(4, 16)

Beta Shapley values

10

2

10

1

AUCC

(16, 1)

(4, 1)

(1, 1)

(1, 4)

(1, 16)

(16, 4)

(2, 2)

(8, 8)

(4, 16)

Beta Shapley values

10

1

10 0

10 1

AUCC

(16, 1)

(4, 1)

(1, 1)

(1, 4)

(1, 16)

(16, 4)

(2, 2)

(8, 8)

(4, 16)

Beta Shapley values

10

2

10

1

10 0

10 1

AUCC

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

weighted Banzhaf values

10

1

AUCC

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

weighted Banzhaf values

10

1

10 0

AUCC

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

weighted Banzhaf values

10

2

10

1

10 0

AUCC

n

= 24

)

MNIST (

n

= 24

)

FMNIST (

n

= 24

)

(16, 1)

(4, 1)

(1, 1)

(1, 4)

(1, 16)

(16, 4)

(2, 2)

(8, 8)

(4, 16)

Beta Shapley values

10

1

AUCC

(16, 1)

(4, 1)

(1, 1)

(1, 4)

(1, 16)

(16, 4)

(2, 2)

(8, 8)

(4, 16)

Beta Shapley values

10

1

AUCC

(16, 1)

(4, 1)

(1, 1)

(1, 4)

(1, 16)

(16, 4)

(2, 2)

(8, 8)

(4, 16)

Beta Shapley values

10

1

AUCC

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

weighted Banzhaf values

10

1

AUCC

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

weighted Banzhaf values

10

1

2×10 2

3×10 2

4×10 2

6×10 2

AUCC

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

weighted Banzhaf values

10

1

AUCC

SOU (

n

= 64

)

SOU (

n

= 128

)

SOU (

n

= 256

)

6 Conclusion

In this work, we propose a framework, termed OFA, that i) adheres to the principle of maximum sample reuse and ii) contains no amplifying factors for the goal of optimizing all probabilistic values simultaneously and efficiently. Particularly, our OFA framework is parameterized by a sampling vector q ∈ R n -3 . To gain insights, we theoretically develop a key formula D ( m , q ) concerning this framework that effectively determines the corresponding convergence rate. By optimizing q in D ( m , q ) for all probabilistic values on average, we obtain our one-for-all estimator that can theoretically approximate all probabilistic values simultaneously with the currently best convergence rate O ( n log n ) on average. Meanwhile, we propose a faster generic estimator by optimizing q for each specific probabilistic value, and we demonstrate that our generic estimate enjoys the best convergence rate for all previously-studied probabilistic values. All of our theoretical findings are verified in our experiments. Finally, we establish a connection between probabilistic values and the least square regressions used in datamodels, showing that our OFA-A estimator is capable of solving a family of (regularized) datamodels simultaneously.

References

Burgess, M. A. and A. C. Chapman (2021). 'Approximating the Shapley Value Using Stratified Empirical Bernstein Sampling'. In: IJCAI , pp. 73-81.

Castro, J., D. Gómez, E. Molina, and J. Tejada (2017). 'Improving Polynomial Estimation of the Shapley Value by Stratified Random Sampling with Optimum Allocation'. Computers & Operations Research , vol. 82, pp. 180-188.

Castro, J., D. Gómez, and J. Tejada (2009). 'Polynomial Calculation of the Shapley Value Based on Sampling'. Computers & Operations Research , vol. 36, no. 5, pp. 1726-1730.

Covert, I. and S.-I. Lee (2021). 'Improving KernelSHAP: Practical Shapley Value Estimation Using Linear Regression'. In: International Conference on Artificial Intelligence and Statistics , pp. 3457-3465.

Dubey, P., A. Neyman, and R. J. Weber (1981). 'Value Theory without Efficiency'. Mathematics of Operations Research , vol. 6, no. 1, pp. 122-128.

Fumagalli, F., M. Muschalik, P. Kolpaczki, E. Hüllermeier, and B. Hammer (2024). 'SHAP-IQ: Unified Approximation of Any-Order Shapley Interactions'. In: Advances in Neural Information Processing Systems . Vol. 36.

Ghorbani, A. and J. Y. Zou (2019). 'Data Shapley: Equitable Valuation of Data for Machine Learning'. In: International Conference on Machine Learning , pp. 2242-2251.

Hammer, P. L. and R. Holzman (1992). 'Approximations of Pseudo-Boolean Functions; Applications to Game Theory'. Zeitschrift für Operations Research , vol. 36, no. 1, pp. 3-21.

Ilyas, A., S. M. Park, L. Engstrom, G. Leclerc, and A. Madry (2022). 'Datamodels: Predicting Predictions from Training Data'. In: Proceedings of the 39th International Conference on Machine Learning .

Jia, R. et al. (2019). 'Towards Efficient Data Valuation Based on the Shapley Value'. In: The 22nd International Conference on Artificial Intelligence and Statistics , pp. 1167-1176.

Kolpaczki, P., V. Bengs, M. Muschalik, and E. Hüllermeier (2024). 'Approximating the Shapley Value without Marginal Contributions'. In: Proceedings of the AAAI Conference on Artificial Intelligence . Vol. 38. 12, pp. 13246-13255.

Kwon, Y. and J. Y. Zou (2022a). 'Beta Shapley: a Unified and Noise-reduced Data Valuation Framework for Machine Learning'. In: International Conference on Artificial Intelligence and Statistics , pp. 8780-8802.

-(2022b). 'WeightedSHAP: Analyzing and Improving Shapley Based Feature Attributions'. In: Advances in Neural Information Processing Systems . Vol. 35, pp. 34363-34376.

Li, W. and Y. Yu (2023). 'Robust Data Valuation with Weighted Banzhaf Values'. In: Advances in Neural Information Processing Systems . Vol. 36.

-(2024). 'Faster Approximation of Probabilistic and Distributional Values via Least Squares'. In: The Twelfth International Conference on Learning Representations .

Lin, J., A. Zhang, M. Lécuyer, J. Li, A. Panda, and S. Sen (2022). 'Measuring the Effect of Training Data on Deep Learning Predictions via Randomized Experiments'. In: International Conference on Machine Learning , pp. 13468-13504.

Lundberg, S. M. and S.-I. Lee (2017). 'A Unified Approach to Interpreting Model Predictions'. In: Advances in Neural Information Processing Systems . Vol. 30.

Maleki, S., L. Tran-Thanh, G. Hines, T. Rahwan, and A. Rogers (2013). 'Bounding the Estimation Error of Sampling-Based Shapley Value Approximation'. arXiv preprint arXiv:1306.4265.

Marichal, J.-L. and P. Mathonet (2011). 'Weighted Banzhaf Power and Interaction Indexes Through Weighted Approximations of Games'. European Journal of Operational Research , vol. 211, no. 2, pp. 352-358.

Moehle, N., S. Boyd, and A. Ang (2022). 'Portfolio Performance Attribution via Shapley Value'. Journal Of Investment Management , vol. 20, no. 3, pp. 33-52.

Musco, C. and R. T. Witter (2024). 'Provably Accurate Shapley Value Estimation via Leverage Score Sampling'. arXiv preprint arXiv:2410.01917 .

Rozemberczki, B., L. Watson, P. Bayer, H.-T. Yang, O. Kiss, S. Nilsson, and R. Sarkar (2022). 'The Shapley Value in Machine Learning'. In: The 31st International Joint Conference on Artificial Intelligence and the 25th European Conference on Artificial Intelligence , pp. 5572-5579.

Ruiz, L. M., F. Valenciano, and J. M. Zarzuelo (1998). 'The Family of Least Square Values for Transferable Utility Games'. Games and Economic Behavior , vol. 24, no. 1-2, pp. 109-130.

Saunshi, N., A. Gupta, M. Braverman, and S. Arora (2022). 'Understanding Influence Functions and Datamodels via Harmonic Analysis'. In: The Eleventh International Conference on Learning Representations .

Shapley, L. S. (1953). 'A Value for N-Person Games'. Annals of Mathematics Studies , vol. 28, pp. 307-317.

Wang, J. T. and R. Jia (2023a). 'A Note on 'Towards Efficient Data Valuation Based on the Shapley Value''. arXiv preprint arXiv:2302.11431 .

-(2023b). 'Data Banzhaf: A Robust Data Valuation Framework for Machine Learning'. In: International Conference on Artificial Intelligence and Statistics , pp. 6388-6421.

Wang, J., Y. Zhang, Y. Gu, and T.-K. Kim (2022). 'SHAQ: Incorporating Shapley Value Theory into Multi-Agent Q-Learning'. In: Advances in Neural Information Processing Systems . Vol. 35, pp. 5941-5954.

Weber, R. J. (1988). 'Probabilistic Values for Games'. In: The Shapley Value. Essays in Honor of Lloyd S. Shapley , pp. 101-119.

Wu, M., R. Jia, C. Lin, W. Huang, and X. Chang (2023). 'Variance Reduced Shapley Value Estimation for Trustworthy Data Valuation'. Computers & Operations Research , vol. 159, p. 106305.

Zhang, J., Q. Sun, J. Liu, L. Xiong, J. Pei, and K. Ren (2023). 'Efficient Sampling Approaches to Shapley Value Approximation'. Proceedings of the ACM on Management of Data , vol. 1, no. 1, pp. 1-24.

A Proof of Theorem 1

Theorem 1. Assume i) ∥ U ∥ ∞ ≤ u and ii) 0 < ϵ ≤ √ 2 D ( m , q ) γ ( q ) 2 u 2 . For ˆ ϕ in Algorithm 1, it requires 4 nu 2 D ( m , q ) ϵ 2 log 8 n 2 δ evaluations of U to achieve P ( ∥ ˆ ϕ -ϕ ∥ 2 ≥ ϵ ) ≤ δ where

Proof. Following Algorithm 1, let { S t } T t =1 be T independent random subsets. Define

where s = 2 , 3 , . . . , n -2 . Then, we have

Define r + i,s = T + i,s T and r -i,s = T -i,s T . In particular, both J i ∈ S t , | S t | = s K and J i ̸∈ S t , | S t | = s K are Bernoulli random variables with

Additionally, R and τ are defined to be vectors in R 2 n -6 such that R 2 k -1 = r + i,k +1 , R 2 k = r -i,k +1 , τ 2 k -1 = q k · ( k +1) n and τ 2 k = q k · ( n -k -1) n for k ∈ [ n -3] . Note that R is a random vector. By Hoeffding's inequality,

where ω > 0 , and thus

Denote the event { ∑ n -2 s =2 [ m s ( ˆ ϕ + i,s -ϕ + i,s ) + m s +1 ( ϕ -i,s -ˆ ϕ -i,s )] ≥ ϵ } by E i where ϵ > 0 . Let C be the set that contains all possible configurations C ∈ { 0 , 1 } (2 n -6) × T such that C1 T T = R and 1 ⊤ T C = 1 ⊤ 2 n -6 , i.e., C j,k = 0 indicates that the k -th subset is sampled from { R ⊆ [ n ] | r = ( j +3) / 2 and i ∈ R } if j is odd and { R ⊆ [ n ] | r = ( j +2) / 2 and i ̸∈ R } otherwise. Then,

Observe that C can be divided into two separate groups C <ω and C ≥ ω such that

Therefore,

For simplicity, we write P C <ω ( E i ) instead of P ( E i | C <ω ) . Additionally, we assume ω < γ ( q ) 2 so that neither T + i,s nor T -i,s is zero when conditioned on any C <ω . By the Chernoff bound, for any

λ > 0 , there is

where the equality is due to the independence that stems from the independence of random subsets and that the configuration is fixed. Moreover,

where { S + i,s,j } 1 ≤ j ≤ T + i,s is obtained by ordering { S t | | S t | = s and i ∈ S t } . In a similar fashion, we have

which leads to

Therefore,

where ˆ D = ∑ n -2 s =2 ( T T + i,s m 2 s + T T -i,s m 2 s +1 ) . Next, we aim to show that | ˆ D -D ( m , q ) | ≤ D ( m , q ) . Observe that

and since | r j -τ j | < ω ,

Since γ ( q ) ≤ τ j and ω ≤ γ ( q ) 2 ,

1

-

τ

j

ω

=

τ

j

τ

j

-

ω

·

1

τ

j

=

1

1

-

ω

τ

j

·

1

τ

j

2

τ

j

.

As a result, we have | ˆ D -D ( m , q ) | ≤ D ( m , q ) , and thus

Combining Eqs. (8) and (9) yields

Choosing λ > 0 that minimizes the upper bound yields

Solving the equation -Tϵ 2 4 u 2 D ( m , q ) = -2 Tω 2 yields ω = √ ϵ 2 8 D ( m , q ) u 2 , which gives

Particularly, to meet the assumption ω ≤ γ ( q ) 2 , we have to have ϵ ≤ √ 2 D ( m , q ) γ ( q ) 2 u 2 . To conclude, provided that ϵ ≤ √ 2 D ( m , q ) γ ( q ) 2 u 2 , we have

Similarly, there is

and thus

Eventually, we have

Solving δ ≥ 8 n 2 exp( -Tϵ 2 4 nD ( m , q ) u 2 ) yields T ≥ 4 nD ( m , q ) u 2 ϵ 2 log 8 n 2 δ . Note the assumption ϵ ≤ √ 2 D ( m , q ) γ ( q ) 2 u 2 can be removed if the configuration is fixed with T + i,s ≈ s · q s -1 n T and T -i,s ≈ ( n -s ) q s -1 n T .

B Proofs of Propositions

Proposition 1. q OFA-A s -1 ∝ 1 √ ( s )( n -s ) and D ( q OFA-A ) ∈ O (1) . In other words, our OFA-A estimator achieves the convergence rate of O ( n log n ) simultaneously for all probabilistic values on average.

Proof. Let Λ = { x ∈ R n -1 | 0 ≤ ∑ n -1 j =1 x j ≤ L n } where L n = n 1 2( n -1) , and a smooth homeomorphism f : Λ → ∆ is defined by letting

In other words, both f and f -1 are C ∞ . Since the volume of ∆ is n 1 2 ( n -1)! , there is

Note that √ det ( Df ( x ) ⊤ Df ( x )) = 1 for every x ∈ Λ . With Λ = { y ∈ R n -1 | 0 ≤ ∑ n -1 j =1 y j ≤ 1 } , we have

For simplicity, assume that n = 4 , notice that

Therefore, which leads to

Since D ( q ) is convex in q , q OFA-A can be directly obtained using the KKT conditions, which is

Therefore, we have

Since lim n →∞ ( n -1)!( n -1) 2 ∏ n -1 k =1 (2+ k ) = 2Γ(3) , when n is sufficiently large, there is

Proposition 2. Our OFA-A estimator achieves the convergence rate of O ( n log n ) simultaneously for all semi-values whose probability density functions exist and are bounded. Particularly, Beta Shapley values with α, β ≥ 1 all satisfy this condition.

Proof. Let ϕ be a semi-value such that p s = ∫ 1 0 w s -1 (1 -w ) n -s d µ ( w ) = ∫ 1 0 w s -1 (1 -w ) n -s p µ ( w )d w such that p µ ( w ) ≤ B for every w ∈ [0 , 1] . Particularly, we have

Therefore,

Proposition 3. If p s = a s -1 (1 -a ) n -s with 0 < a < 1 , which corresponds to the weighted Banzhaf value parameterized by w , then D ( m , q OFA-A ) ∈ O ( n 1 2 ) . In other words, the OFA estimator achieves the convergence rate of O ( n 3 2 log n ) simultaneously for all WBa with 0 < a < 1 .

Proof. With q OFA-A s -1 ∝ 1 √ s ( n -s ) , we have

Then,

and thus

Since

there is

Using the identity ∑ m j =0 ( m j ) 2 ( x + y ) 2 j ( x -y ) 2( m -j ) = ∑ m j =0 ( 2 j j )( 2( m -j ) m -j ) x 2 j y 2( m -j ) , there is

For every k ≥ 1 , ( 2 k k ) ≈ 2 2 k √ k using the Stirling's approximation, and thus

where z = (2 a -1) 2 < 1 . Therefore, we obtain ∑ n -2 s =2 (( n s ) a s (1 -a ) n -s ) 2 ≤ O ( n -1 2 ) , which eventually leads to

Proposition 4. For semi-values, D ( m , q OFA-S ) ∈ O (1) if i) µ has a bounded probability density function or ii) ∫ (0 , 1) 1 w (1 -w ) d µ ( w ) < ∞ . Particularly, this condition covers all weighted Banzhaf values and Beta Shapley values with α, β ≥ 1 .

̸

Proof. If µ ( { 0 } ) = 0 ( µ ( { 1 } ) = 0 , respectively), its induced marginal contributions all reside in ϕ + i, 1 and ϕ -i, 0 ( ϕ + i,n and ϕ -i,n -1 , respectively), which is computed exactly using Algorithm 1. Therefore, W.L.O.G., we assume that µ ((0 , 1)) = 1 .

̸

Suffice it to show that if ∫ 1 0 1 w (1 -w ) d µ ( w ) < ∞ , there is

Specifically,

Since √ s n (1 -w ) + √ n -s n w ≤ 2 , we have

C Proof of Theorem 2

To prove this theorem, we first state useful definitions and lemmas.

Definition 2 (Semi Inner Product) . Let V is a real linear space. A semi inner product ⟨· , ·⟩ on V satisfies, for every x, y, z ∈ V and every α ∈ R , i) ⟨ x, y ⟩ = ⟨ y, x ⟩ , ii) ⟨ αx, y ⟩ = α ⟨ x, y ⟩ , iii) ⟨ x + y, z ⟩ = ⟨ x, z ⟩ + ⟨ y, z ⟩ , and iv) ⟨ x, x ⟩ ≥ 0 . In addition, we write ∥ x ∥ = √ ⟨ x, x ⟩ for every x ∈ V .

Lemma 1. Let a semi inner product on a linear space V be given, and A ⊆ V is some affine space. For the following optimization problem

where p ∈ V , x ∗ is optimal if and only if

Proof. Suppose x ∗ verifies Eq. (10), for every y ∈ A ,

Next, suppose x ∗ is optimal, and for the sake of contradiction, assume that there is some y ∈ A such that ⟨ x ∗ -p, y -x ∗ ⟩ ̸ = 0 . Write z = y -x ∗ , for t ∈ R

Since ⟨ x ∗ -p, z ⟩ ̸ = 0 , there exists some t o ∈ R such that t 2 ∥ z ∥ 2 +2 t ⟨ x ∗ -p, z ⟩ < 0 , and thus ∥ x ∗ + t o z -p ∥ 2 < ∥ x ∗ -p ∥ 2 , a contradiction.

Definition 3 (Projection Induced by a Semi Inner Product) . Given a semi inner product on a linear space V , the set of all optimal solutions to the problem

where A ⊆ V is an affine space and p ∈ V , is denoted by Proj A ( { p } ) . To account for the possibility that there are multiple optimal solutions, we extend the definition by letting Proj A ( S ) = ⋃ p ∈ S Proj A ( { p } ) .

Lemma 2. Let V be a linear space with a semi inner product. Suppose there are two affine spaces B ⊆ A , for every p ∈ V , there is

Proof. We rephrase Lemma 1 to ease the proof. For each affine space A ⊆ V , define L A = Aq for some q ∈ A . Note that the resulting L A is independent of the choice of q ∈ A and it is a subspace in V . Therefore, Eq. (10) is equivalent to

Suppose x ∈ Proj B (Proj A ( { p } )) , by Lemma 1, there exists y ∈ Proj A ( { p } ) such that

Therefore,

Lemma 3 (Ruiz et al. 1998, Theorem 12) . Let v ∗ be the uniquely optimal solution to

where η s = p s -1 + p s for 2 ≤ s ≤ n . Then, there is

Recall that the problem (6) is

and our goal is to prove that

which together with Lemma 3 is sufficient to complete our proof.

Theorem 2. Let ( b ∗ , θ ∗ ) be the uniquely optimal solution to the problem (6) where η s = p s -1 + p s for 2 ≤ s ≤ n . Then, there is

Proof. The first part of our proof was inspired by (Hammer and Holzman 1992, Lemma 2.9). Let G = { U : 2 [ n ] → R } , AG = { U ∈ G | U ( S ) = a 0 + ∑ i ∈ S a i for every S ⊆ [ n ] } and A U = { g ∈ AG | U ([ n ]) = g ([ n ]) and U ( ∅ ) = g ( ∅ ) } . Note that G is a linear space and the other two are affine spaces with A U ⊆ AG . For clarity, each game in AG is written as [ a 0 , a ] where a ∈ R n .

A semi inner product on G can be defined by letting ⟨ g 1 , g 2 ⟩ = ∑ S ⊆ [ n ] η s +1 · g 1 ( S ) g 2 ( S ) for every g 1 , g 2 ∈ G . Then, [ b ∗ , θ ∗ ] is the projection of U onto AG , whereas [ U ( ∅ ) , v ∗ ] is the projection of U onto A U where v ∗ is the uniquely optimal solution to the problem (11).

By Lemma 2, there is Proj A U (Proj AG ( { U } )) ⊆ Proj A U ( { U } ) . Moreover, the uniqueness in problem (11) implies that Proj A U ( { U } ) = { [ U ( ∅ ) , v ∗ ] } , and thus

Since [ b ∗ , θ ∗ ] ∈ Proj AG ( { U } ) , the equality Proj A U ( { [ b ∗ , θ ∗ ] } ) = { [ U ( ∅ ) , v ∗ ] } means that [ U ∅ , v ∗ ] is the uniquely optimal solution to the problem

̸

Pick i, j ∈ [ n ] such that i = j , and define an additive game e i ∈ AG by letting e i ( S ) = 1 if i ∈ S and 0 otherwise, e j is defined similarly. Consider the problem

Note that [ U ( ∅ ) , v ∗ ] + t ( e i -e j ) ∈ A U for every t ∈ R , and the uniqueness to the problem (12) suggests that t ∗ = 0 is the uniquely optimal solution to the problem (13). Removing all constant terms in the problem (13) yields an equivalent problem

Write g = [ U ( ∅ ) , v ∗ ] -[ b ∗ , θ ∗ ] , since this problem is convex, letting the derivative equal 0 leads to

Write g = [ g 0 , g ] where g 0 = U ( ∅ ) -b ∗ and g = v ∗ -θ ∗ , there is

̸

̸

Similarly, we have ∑ S ⊆ [ n ] : i ̸∈ S,j ∈ S η s +1 · g ( S ) = α ( g 0 + g j ) + β ∑ 1 ≤ k ≤ n : k = i,j g k , and therefore

Since α > 0 , we eventually get g i = g j . In other words, v ∗ i -θ ∗ i = v ∗ j -θ ∗ j . Because i and j are chosen arbitrarily, our proof is completed.

To be self-contained, we also prove that the problem (6) has only one optimal solution provided that η s = p s -1 + p s for 2 ≤ s ≤ n . W.L.O.G., assume ∑ S ⊆ [ n ] η s +1 = 1 . By letting the derivative of the

problem (6) equal 0 , we have Ax = b

Left multiplying A with some row operation matrix R gives

̸

It is sufficient to prove that the bottom-right n × n submatrix of RA is invertible. Suffice it to show κ -τ = 0 and κ +( n -1) τ -nκ 2 = 0 . Using ( n s ) = ( n -1 s ) + ( n -1 s -1 ) , we have

̸

Using n ( n -1 s -1 ) = s ( n s ) , we have

Let γ = 1 -η 1 and ζ s = η s +1 /γ for every s ∈ [ n ] , there is

If η 1 > 0 , the last inequality is strict as γ < 1 . Otherwise, the first inequality is strict as Var[ s ] = E [ s 2 ] -E [ s ] 2 > 0 .

D Overview of Estimators

Recall that each probabilistic value is defined to be, for every i ∈ [ n ] ,

where p ∈ R n is a non-negative vector with ∑ n s =1 ( n -1 s -1 ) p s = 1 . If p s = ∫ 1 0 w s -1 (1 -w ) n -s d µ ( w ) for some probability measure µ on the closed interval [0 , 1] , the induced ϕ is referred to as a semi-value.

The Sampling Lift Estimator (Moehle et al. 2022) The sampling lift estimator is based on

The sampling procedure is: i) sample a subset size s ∈ [ n ] with P ( s ) = ( n -1 s -1 ) p s , and then ii) sample a subset S uniformly from { R ⊆ [ n ] \ i | r = s -1 } . For semi-values such that p s = ∫ 1 0 w s -1 (1 -w ) n -s d µ ( w ) where µ is a probability measure on the closed interval [0 , 1] , there is an alternative: i) sample a w ∈ [0 , 1] according to µ , and then sample a subset S ⊆ [ n ] \ i by incorporating each player in [ n ] \ i with probability w . With a sequence of sampled subsets { S j } T j =1 , the i -th estimate is ˆ ϕ i = 1 T ∑ T j =1 ( U ( S j ∪ i ) -U ( S j )) .

The Weighted Sampling Lift Estimator (Kwon and Zou 2022a) The formula it is built upon is

Note that setting p = q in Eq. (14) leads to the Shapley value. The sampling procedure is: i) sample a w uniformly from [0 , 1] , and then ii) sample a subset S ⊆ [ n ] \ i by incorporating each player in [ n ] \ i with probability w . Then, the i -th estimate is ˆ ϕ i = 1 T ∑ T j =1 p s j +1 q s j +1 ( U ( S j ∪ i ) -U ( S j )) .

The KernelSHAP Estimator (Lundberg and Lee 2017) This estimator is specific to the Shapley value. It employs the fact that the Shapley value ϕ Shap i is the uniquely optimal solution to

Note that the weights can be scaled so that the objective is an expectation. A sequence of subsets { S j } T j =1 where ∅ ⊊ S j ⊊ [ n ] is sampled according to P ( S ) ∝ ( n -2 s -1 ) -1 . Then, we have an approximate problem as

the uniquely optimal solution of which is treated as the estimates, i.e.,

Specifically, 1 S j ∈ { 0 , 1 } n such that its i -th entry is 1 if and only if i ∈ S j .

The Unbiased KernelSHAP Estimator (Covert and Lee 2021) The uniquely optimal solution ϕ Shap to the problem (15) is

This estimator employs the fact that A ij = 1 2 if i = j and 1 n ( n -1) ∑ n -1 s =2 s -1 n -s ∑ n -1 s =1 1 s ( n -s ) otherwise. In other words, the estimates of this estimator is

Particularly { S j } T j =1 where ∅ ⊊ S j ⊊ [ n ] are sampled using P ( S ) ∝ ( n -2 s -1 ) -1 .

Recently, Fumagalli et al. (2024) proved that Eq. (16) can be simplified as

The ARM Estimator (Kolpaczki et al. 2024) This estimator is designed according to

where P + ( S ) ∝ p s for every ∅ ⊊ S ⊆ [ n ] and P -( S ) ∝ p s +1 for every S ⊊ [ n ] (Li and Yu 2024, Proposition 8). A sequence of subsets { S j } T j =1 are sampled using P + and P -alternatively, T T

i.e., { S 2 k -1 } 2 k =1 are sampled independently according to P + , whereas { S 2 k } 2 k =1 are sampled independently using P -. Then, the i -th estimate is

where T + i = ∑ T 2 k =1 1 i ∈ S 2 k -1 and T -i = ∑ T 2 k =1 1 i ̸∈ S 2 k .

The AME Estimator (Lin et al. 2022) This estimator is restricted to a sub-family of semi-values that satisfy ∫ 1 0 1 w (1 -w ) d µ ( w ) < ∞ . For such a semi-value ϕ , it can be cast as a uniquely optimal solution to

where X ∈ R n and Y are random variables. The sampling procedure is: i) sample a w ∈ (0 , 1) using µ , ii) sample a subset S by incorporating each player with probability w , and then iii) Y = U ( S ) and X = X ( S ) such that X i = 1 w · C if i ∈ S and -1 (1 -w ) C otherwise where C = ∫ 1 0 1 w (1 -w ) d µ ( w ) . With a sequence of subsets { ( w j , S j ) } T j =1 , the uniquely optimal solution to the approximate problem

is taken as the induced estimates, which is ˆ ϕ = ( A ⊤ A ) -1 A ⊤ b where the j -th row of A is X ( S j ) ⊤ and b j = U ( S j ) .

One Way to Improve the AME Estimator The limitation of the AME estimator is that it only applies to semi-values that satisfy ∫ 1 0 1 w (1 -w ) d µ ( w ) < ∞ . Meanwhile, another potential drawback is its need to compute the inverse of A ⊤ A , though it can be circumvented by solving the approximate problem using gradients. In this work, we make a small improvement to the AME estimator by extending its applicability to all semi-values, removing ( A ⊤ A ) -1 in the approximate formula and providing a more direct analysis of its convergence rate in terms of ( ϵ, δ ) -approximation.

Our improvement begins with the observation that E [ XX ⊤ ] = I , suggesting that ( 1 T A ⊤ A ) -1 → I by the law of large numbers and thus ( A ⊤ A ) -1 is redundant. Its removal leads to a simplified formula:

We comment that the following proposition is complementary to (Lin et al. 2022, Proposition 3.3) that claims a similar result.

Proposition 6. Assume that i) ∥ U ∥ ∞ ≤ u and ii) µ ([ A,B ]) = 1 for some 0 < A < B < 1 , the improved AME estimator requires 2 nu 2 C 2 ϵ 2 log 2 n δ utility evaluations of U to achieve P ( ∥ ˆ ϕ -ϕ ∥ 2 ≥ ϵ ) ≤ δ where C = 1 min( A, 1 -B ) .

Proof. Notice that ˆ ϕ i = 1 T ∑ T j =1 Z j where { Z j } T j =1 are i.i.d. random variables with E [ Z j ] = ϕ i . By the Hoeffding's inequality, there is

where C = 1 min( A, 1 -B ) . Then,

Remark 1. Notice that the improved AME estimator requires that µ ( { 0 , 1 } ) = 0 . Nevertheless, semi-values are additively decomposable on µ and the part related to µ ( { 0 , 1 } ) can be computed exactly in linear time. Therefore, it is fair to conclude that the improved AME estimator applies to all semi-values.

The MSR Estimator (Wang and Jia 2023b) The methodology of this estimator is limited to weighted Banzhaf values parameterized with 0 < a < 1 (Wang and Jia 2023b, Appendix C.2). Precisely, p s = a s -1 (1 -a ) n -s . Each subset is sampled by incorporating each player with probability a , and then the i -th estimate is

where T + i = ∑ T j =1 1 i ∈ S j and T -i = ∑ T j =1 1 i ̸∈ S j .

The GELS Estimator (Li and Yu 2024) This estimator is established using the fact that ϕ i = v ∗ i -v ∗ n +1 where v ∗ ∈ R n +1 is the uniquely optimal solution to

The subsets { S j } T j =1 where ∅ ⊊ S j ⊊ [ n + 1] are sampled using P ( S ) ∝ p s , and then the i -th estimate is

where ˆ v k = 1 T k ∑ T j =1 U ( S j ∩ [ n ]) 1 k ∈ S j and T k = ∑ T j =1 1 k ∈ S j .

The Complement Estimator (Zhang et al. 2023) The complement estimator is specific to the Shapley value using the fact that

The sequence of subsets { S j } T j =1 is sampled using i) sample a subset size s ∈ [ n ] uniformly, and then sample a subset S uniformly from { R ⊆ [ n ] | r = s } . Then, the i -th estimate is

The Group Testing Estimator (Jia et al. 2019) We introduce the improved version presented by Wang and Jia (2023a). Note that this estimator is specific to the Shapley value. A sequence of subsets { S j } T j =1 are independently sampled according to: i) sample a subset size s ∈ [ n ] using P ( s ) ∝ 1 s ( n +1 -s ) , and then ii) sample a subset S uniformly from { R ⊆ [ n +1] | r = s } . Then, the i -th estimate is

The Permutation Estimator (Castro et al. 2009) This estimator is specific to the Shapley value, using the formula

where Π contains all permutations of [ n ] and P i ( π ) is the subset that contains all players preceding i in π . Thus, it samples a sequence of permutations { π j } T j =1 from Π uniformly with replacement, and then the i -th estimate is ˆ ϕ Shap i = 1 T ∑ T j =1 ( U ( P i ( π j ) ∪ i ) -U ( P i ( π j )) ) .

The WeightedSHAP Estimator (Kwon and Zou 2022b) As mentioned in the main paper, it is based on

where m s = ( n -1 s -1 ) p s . For each player i ∈ [ n ] , it samples a sequence of permutations { π j } T j =1 of [ n ] \ i . Then, the corresponding estimate is ˆ ϕ i = ∑ n s =1 m s ˆ ϕ i,s where ˆ ϕ i,k = 1 T ∑ T j =1 ( U ( S k ( π j ) ∪ i ) -U ( S k ( π j )) ) and S k ( π j ) is the subset that contains the first k -1 players in π j .

The SHAP-IQ Estimator (Fumagalli et al. 2024) Recall that its underlying formula is

where m s = ( n -1 s -1 ) p s , H = ∑ n -1 j =1 1 j , and P ( S ) ∝ ( n -2 s -1 ) -1 . Therefore, a sequence of subsets { S j } T j =1 where ∅ ⊊ S j ⊊ [ n ] is sampled using P ( S ) ∝ ( n -2 s -1 ) -1 , and the i -th estimate is

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Our claimed theories are presented in Section 4 and are empirically verified in Section 5.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: Proposition 3 demonstrates that our OFA-A estimator does not rival the previously best estimator for weighted Banzhaf values in terms of convergence rate, which is a price to pay for using a fixed sampling scheme for all probabilistic values.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We have provided detailed proofs in the Appendices A, B and C.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Our experiment settings are stated in Section 5, and our method is presented in Algorithm 1.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The datasets we used are from open resources, and our code will be released on a github repo.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( ❤/a116/a116♣/a115✿✴✴♥✐♣/a115✳❝❝✴ ♣✉❜❧✐❝✴❣✉✐❞❡/a115✴❈♦❞❡❙✉❜♠✐/a115/a115✐♦♥/a80♦❧✐❝② ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( ❤/a116/a116♣/a115✿ ✴✴♥✐♣/a115✳❝❝✴♣✉❜❧✐❝✴❣✉✐❞❡/a115✴❈♦❞❡❙✉❜♠✐/a115/a115✐♦♥/a80♦❧✐❝② ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Our experiment settings are stated in Section 5.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: Our experiment results in Section 5 are all reported with standard deviation using 30 random seeds.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: It is stated in Section 5.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics ❤/a116/a116♣/a115✿✴✴♥❡✉/a114✐♣/a115✳❝❝✴♣✉❜❧✐❝✴❊/a116❤✐❝/a115●✉✐❞❡❧✐♥❡/a115 ?

Answer: [Yes]

Justification: We have complied with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: Our work focuses on the convergence rate of estimators for probabilistic values that do not appear to have any significant societal impact.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: Our work focuses on the convergence rate of estimators for probabilistic values that do not appear to pose any risk for misuse.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: It is stated in Section 5.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, ♣❛♣❡/a114/a115✇✐/a116❤❝♦❞❡✳❝♦♠✴❞❛/a116❛/a115❡/a116/a115 has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: We do not introduce any new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: Our work does not involve human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: Our work does not involve human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
comments: SUMMARY:
This paper studies efficient estimation of probabilistic values. It proposes an algorithm which can approximate any probabilistic value with average convergence $O(n \log n)$. It also proposes an improved algorithm for specific cases.

STRENGTHS:
This paper provides a solid contribution over previous work.

The paper is well written and explains the main ideas clearly.

WEAKNESSES:
If I’m understanding correctly, the algorithm approximates “any” instead of “all” probabilistic values, I.e. it’s a generic algorithm which can approximate any probabilistic value, but it can’t approximate all values simultaneously (one needs to rerun the algorithm for different values). 

How valuable is the average convergence rate? It’s likely that the easy ones contribute more, and the average rate for the cases of interest is worse.

QUESTIONS:
See weaknesses.

LIMITATIONS:
See weaknesses.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-05

comments: SUMMARY:
The paper discusses a novel framework for efficiently approximating probabilistic values, such as Beta Shapley values and weighted Banzhaf values. These values are computationally expensive to calculate exactly, calling for approximation techniques.

Specifically, they propose the One-sample-Fits-All Framework (OFA), which adheres to the principle of maximum sample reuse and not increasing variance. Besides, two variants, OFA-A (and OFA-S) optimize for all probabilistic values on average (and specific probabilistic values), respectively.

Overall, they provide an efficient solution for approximating multiple probabilistic values simultaneously, with theoretical and practical implications.

STRENGTHS:
1. The quick approximation of probabilistic values is important.

2. The writing flow is clear and easy to follow (especially Sec. 3).

3. The theoretical part seems reasonable.

WEAKNESSES:
1. The variances (and/or biases) of the estimations are not thoroughly examined; instead, they only consider whether the variances will increase based on the range/value of $m_s$.

2. The authors emphasize "the principle of maximum sample reuse" all the time, and treat it as a key principle. However, they do not provide a formal definition of this principle.

3. Similarly, a formal definition of "one-for-all estimators" is missing.

4. Line 95 mentions that an existing work already achieves $O(n/\epsilon^2 \log (n/\sigma))$, while the proposed OFA-A is $O(n \log (n))$. The significance of the contribution is unclear.

5. What does $q_s$ in line 159 mean?

QUESTIONS:
See the weaknesses part.

LIMITATIONS:
The limitations are not discussed explicitly in the main text.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-08

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a novel framework called One-Sample-Fits-All (OFA) for efficiently approximating probabilistic values used in feature attribution and data valuation, such as Beta Shapley values and weighted Banzhaf values. The framework maximizes sample reuse and avoids variance amplification, making it capable of approximating all probabilistic values simultaneously. The authors leverage the concept of \n(ϵ,δ)-approximation to derive a formula that determines the convergence rate, which they use to optimize the sampling vector. The proposed framework includes two estimators: OFA-A, which is optimized for all probabilistic values on average, and OFA-S, which is fine-tuned for each specific probabilistic value. Empirical results demonstrate that OFA-A achieves the fastest known convergence rate for Beta Shapley values and competitive rates for weighted Banzhaf values.\n\nSTRENGTHS:\n1. The paper introduces a novel one-sample-fits-all framework that can approximate a wide range of probabilistic values efficiently, addressing a significant gap in the literature. The use of (ϵ,δ)-approximation to derive convergence rates and optimize the sampling vector adds strong theoretical backing to the proposed method.\n\n2. The OFA-A estimator achieves the best known time complexity for certain probabilistic values, indicating its computational efficiency.\n\nWEAKNESSES:\n1. The empirical results are demonstrated on a classification task using LeNet and small datasets like MNIST. \n\n2. The paper does not discuss much about the implementation details - the theoretical concepts and optimization processes might be complex for practitioners without a strong background in cooperative game theory and advanced statistical methods.\n\nQUESTIONS:\nThe paper does provide an extensive theoretical proof regarding scalability of the pipeline, but there is limited explorations in terms of  emperical results for scalable real life datasets and problems. It would be nice to see some comments on that. While the scalability is theoretically supported, further empirical validation on larger datasets would strengthen the claim.\n\nLIMITATIONS:\nYes the authors have indicated the weakness of their proposed method", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 1, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-21"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies efficient estimators for probabilistic values, with applications in data valuation and feature attribution. Since the computation of probabilistic values requires an exponential number of utility function evaluations, efficient estimation of probabilistic values is necessary. Existing approaches for this either contain amplifying scalers that degrade the convergence rate or fail to perform maximum sample re-use (i.e. utility evaluations are not used to update the probabilistic value estimates for all players). The paper proposes an algorithm that solves both the above challenges at once, thus providing a one-for-all estimator that achieves superior time complexity on average. The theoretical results are supported by several experimental results that validate improvements in speed of convergence as compared to the previous methods.\n\nSTRENGTHS:\nThe paper provides a simple algorithm that updates the probabilistic values of all n players at once, thus improving our ability to compute multiple probabilistic values and allowing us to evaluate the best one for downstream applications. Since the exponential number of utility evaluations hinders the ability to perform this evaluation, this paper could be an important step towards understanding how to improve the time complexity of such methods. The paper carefully compares existing methods under the axes of amplifying scalers and maximum sample reuse, providing an intuitive goal for improved estimation algorithms. The experimental assessment supports the theory developed in the paper regarding (\\epsilon, \\delta) convergence rates.\n\nWEAKNESSES:\nThe proposed algorithm depends on obtaining a good sampling vector q. Although the paper addresses this in sections 4.1 and 4.2, it would be helpful to discuss this further (e.g., challenges in solving the minimization problem in 4.1, alternatives to the faster generic estimator in 4.2, etc). \n\nThe experimental results vary the size of the dataset up to 256. In real-world data valuation tasks with larger sample sizes, would it be harder to scale this method despite beating all the baselines? \n\nIt would be helpful to discuss the limitations of the proposed approach both in terms of theoretical analysis and applicability to tasks like data valuation (It does talk about the limitations under proposition 3, but it would be helpful to have a separate discussion on limitations)\n\nQUESTIONS:\nPlease see the weaknesses\n\nLIMITATIONS:\nAlthough broader societal impacts may not apply to this paper, this work should have a section on the limitations of the proposed algorithm in a separate section in addition to the brief discussion in Proposition 3.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-26"}