E NERGY -B ASED L EARNING FOR C OOPERATIVE G AMES , WITH A PPLICATIONS TO V ALUATION P ROBLEMS IN M ACHINE L EARNING
Anonymous authors
Paper under double-blind review
A BSTRACT
Valuation problems, such as feature interpretation, data valuation and model valuation for ensembles, become increasingly more important in many machine learning applications.
Such problems are commonly addressed via well-known game-theoretic criteria, such as the Shapley value or Banzhaf value.
In this work, we present a novel energy-based treatment for cooperative games, with a theoretical justiﬁcation by the maximum entropy principle.
Surprisingly, by conducting mean-ﬁeld variational inference of the energy-based model, we recover classical game-theoretic valuation criteria through conducting one-step ﬁxed point iteration for maximizing the ELBO objective.
This observation also veriﬁes the rationality of existing criteria, as they are all attempting to decouple the correlations among players.
By running the ﬁxed point iteration for multiple steps, we achieve a trajectory of the variational valuations, among which we deﬁne the valuation with the best conceivable decoupling error as the Variational Index .
We prove that under uniform initializations, these variational valuations all satisfy a set of game-theoretic axioms.
We empirically demonstrate that the proposed Variational Index enjoys lower decoupling error and better valuation performance on certain synthetic and real-world valuation problems.
I NTRODUCTION
Valuation problems are becoming increasingly more signiﬁcant in various machine learning applications, ranging from feature interpretation (Lundberg and Lee, 2017), data valuation (Ghorbani and Zou, 2019; Sim et al., 2020) and model valuation for ensembles (Rozemberczki and Sarkar, 2021).
They are often formulated as an importance assignment problem in cooperative games.
A cooperative game ( N, F ( S )) consists of a grand coalition N = { 1 , ..., n } of n players and a value function (a.k.a.
characteristic function) F ( S ) : 2 N → R describing the collective payoff of a coalition/cooperation S .
A fundamental problem in cooperative game theory is to assign an importance vector (solution concept) φ ( F ) ∈ R n to n players.
In this paper, we explore a probabilistic treatment of cooperative games ( N, F ( S )) .
Such a treatment makes it possible to conduct learning and inference in a uniﬁed manner, and will yield connections with classical valuation criteria.
Concretely, we seek a probability distribution over coalitions p ( S = S ) 1 , measuring the odds that a speciﬁc coalition S happens.
Generally, we consider distributions where the probability of a coalition p ( S ) grows monotonically with the payoff F ( S ) .
Among all the possible probability mass functions (pmfs), how should we construct the proper p ( S ) ?
We advocate to choose the pmf with the maximum entropy H ( p ) .
This principle makes sense since maximizing the entropy minimizes the amount of prior information built into the distribution.
In other words, it amounts to assume nothing about what is unknown, i.e., choosing the most “uniform” distribution.
Now ﬁnding a proper p ( S ) becomes the following constrained optimization problem: suppose each coalition S is associated with a payoff F ( S ) with probability p ( S ) .
We would like to maximize the entropy H ( p ) =
− (cid:80) S ⊆
N p ( S ) log p ( S ) , subject to the constraints that (cid:80) S p ( S )
= 1 , p ( S ) ≥ 0 and (cid:80) S p ( S ) F ( S ) =
µ (i.e., the average payoff is known as µ ).
Solving this optimization problem (derivation in Appendix A), we reach the maximum entropy distribution :
[Equation]
where T > 0 is the temperature.
This is an energy-based model (EBM (LeCun et al., 2006)) with − F ( S ) as the energy function.
The above energy-based treatment admits two beneﬁts: i) Where supervision is available, it enables learnable value functions F ( S ) through efﬁcient training techniques of energy-based learning, such as noise contrastive estimation (Gutmann and Hyvärinen, 2010) and score matching (Hyvärinen, 2005).
ii) Approximate inference techniques such as variational inference or sampling can be adopted to solve the valuation problem.
Speciﬁcally, it enables to perform mean-ﬁeld variational inference where parameters of the inferred surrogate distribution can be used as principled player valuations.
Below, we explore mean-ﬁeld variational inference for the energy-based formulation.
Perhaps surprisingly, by conducting only one-step ﬁxed point iteration for maximizing the mean-ﬁeld (ELBO) objective, we recover classical valuation criteria, such as the Shapley value (Shapley, 1953) and the Banzhaf value (Penrose, 1946; Banzhaf III, 1964).
This observation also veriﬁes the rationality of existing criteria, as they are all trying to decouple the correlations among players through the mean-ﬁeld approach.
By running the ﬁxed point iteration for multiple steps, we achieve a trajectory of valuations, among which we deﬁne the valuation with the best conceivable decoupling error as the
Variational Index .
Our major contributions can be summarized as below:
i)
We present a theoretically justiﬁed energy-based treatment for cooperative games.
Through mean ﬁeld inference, we provide a uniﬁed perspective on popular game-theoretic criteria.
This provides an alternative motivation of existing criteria via a decoupling perspective, i.e., decoupling correlations among n players through the mean-ﬁeld approach.
ii)
In pursuit of better decoupling performance, we propose to run ﬁxed point iteration for multiple steps, which generates a trajectory of valuations.
Under uniform initializations, they all satisfy a set of game-theoretic axioms, which are required for being suitable valuation criteria.
We deﬁne the valuation with the best conceivable decoupling error as the Variational Index.
iii)
Synthetic and real-world experiments demonstrate intriguing properties of the proposed Variational Index, including lower decoupling error and better valuation performance.
P RELIMINARIES AND B ACKGROUND
Notation.
We assume e i ∈ R n being the standard i th basis vector and use boldface letters x ∈ R N and x ∈ R n interchangebly to indicate an n -dimensional vector, where x i is the i th entry of x .
By default, f ( · ) is used to denote a continuous function, and F ( · ) to represent a set function.
For a differentiable function f ( · ) , ∇ f ( · ) denotes its gradient.
x | x i ← k is the operation of setting the i th element of x to k , while keeping all other elements unchanged, i.e., x | x i ← k
= x − x i e i
+ k e i .
For two sets S and T , S + T and S − T represent set union and set difference, respectively.
| S | is the cardinality of S .
i is used to denote the singleton { i } with a bit abuse of notation.
Existing valuation criteria.
Various valuation criteria have been proposed from the area of cooperative games, amongst them the most famous ones are the Shapley value (Shapley, 1953) and
the
Banzhaf value, which is extended from the Banzhaf power index (Penrose, 1946; Banzhaf III, 1964).
For the Shapley value, the importance assigned to player i is:
[Equation]
One can see that it gives less weight to n/ 2 -sized coalitions.
The Banzhaf value assigns the following importance to player i :
[Equation]
which uses uniform weights for all the coalitions.
See Greco et al.
for a comparison of them.
Valuation problems in machine learning.
Currently most classes of valuation problems (Lundberg and Lee, 2017; Ghorbani and Zou, 2019; Sim et al., 2020;
Rozemberczki and Sarkar, 2021) use
Shapley value as the valuation criterion.
Along with the rapid progress of model interpretation in the past decades (Zeiler and Fergus, 2014;
Ribeiro et al., 2016; Lundberg and Lee, 2017; Sundararajan et al., 2017; Petsiuk et al., 2018; Wang et al., 2021a), attribution-based interpretation aims to assign importance to the features for a speciﬁc data instance ( x ∈ R N , y ) given a black-box model M .
Here each feature maps to a player in the game ( N, F ( S )) , and the value function F ( S ) is usually the model response, such as the predicted probability for classiﬁcation problems, when feeding a subset
S of features to M .
The data valuation problem (Ghorbani and Zou, 2019) tries to assign values to the samples in the training dataset N = { ( x i , y i ) }
n 1 for general supervised machine learning: one training sample corresponds to one player, and the value function F ( S ) indicates the predictor performance on some test dataset given access to only a subset of the training samples in S .
Model valuation in ensembles (Rozemberczki and Sarkar, 2021) measures importance of individual models in an ensemble in order to correctly label data points from a dataset, where each pre-trained model maps to a player and the value function measures the predictive performance of subsets of models.
R ELATED W ORK
Energy-based modeling.
Energy based learning (LeCun et al., 2006) is a classical learning framework that uses an energy function E ( x ) to measure the quality of a data point x .
Energy based models have been applied to different domains, such as data generation (Deng et al., 2020), out-of-distribution detection (Liu et al., 2020), reinforcement learning (Haarnoja et al., 2017), memory modeling (Bartunov et al., 2019), discriminative learning (Grathwohl et al., 2019; Gustafsson et al., 2020) and biologically-plausible training (Scellier and Bengio, 2017).
Energy based learning admits principled training methods, such as contrastive divergence (Hinton, 2002), noise contrastive estimation (Gutmann and Hyvärinen, 2010) and score matching (Hyvärinen, 2005).
For approximate inference, sampling based approaches are mainly MCMC-style algorithms, such as stochastic gradient Langevin dynamics (Welling and Teh, 2011).
For a wide class of EBMs with submodular or supermodular energies (Djolonga and Krause, 2014), there exist provable mean ﬁeld inference algorithms with constant factor approximation guarantees (Bian et al., 2019).
Shapley values in machine learning.
Shapley values have been extensively used for valuation problems in machine learning, including attribution-based interpretation (Lipovetsky and Conklin, 2001;
Cohen et al., 2007; Strumbelj and Kononenko, 2010; Owen, 2014; Datta et al., 2016; Lundberg and Lee, 2017; Chen et al., 2018; Lundberg et al., 2018; Kumar et al., 2020; Williamson and Feng, 2020; Covert et al., 2020b; Wang et al., 2021b), data valuation (Ghorbani and Zou, 2019; Jia et al., 2019b;a; Wang et al., 2020; Fan et al., 2021), collaborative machine learning (Sim et al., 2020) and recently, model valuation in ensembles (Rozemberczki and Sarkar, 2021).
For a detailed overview of papers using Shapley values for feature interpretation, please see Covert et al.
(2020a) and the references therein.
To alleviate the exponential computational cost of exact evaluation, various methods have been proposed to approximate Shapley values in polynomial time (Ancona et al., 2017; 2019).
Owen (1972) proposes the multilinear extension purely as a representation of cooperative games and Okhrati and Lipani (2021) use it to develop sampling algorithms for Shapley values.
V ALUATION FOR C OOPERATIVE G AMES : A D ECOUPLING P ERSPECTIVE
In the introduction we have veriﬁed that under the setting of cooperative games, the Boltzmann distribution (see Eq.
(1)) achieves the maximum entropy among all of the pmf functionals.
One can naturally view the importance assignment problem of cooperative games as a decoupling problem :
The n players in a game ( N, F ( S )) might be arbitrarily correlated in a very complicated manner.
However, in order to assign each of them an individual importance value, we have to decouple their interactions, which can be viewed as a way to simplify their correlations.
We therefore consider a surrogate distribution q ( S ; x ) governed by parameters in x . q has to be simple, given our intention to decouple the correlations among the n players.
A natural choice is to restrain q ( S ; x ) to be fully factorizable, which leads to a mean-ﬁeld approximation of p ( S ) .
The simplest form of q ( S ; x ) would be a n independent Bernoulli distribution, i.e., q ( S ; x ) :
= (cid:81)
i ∈
S x i (cid:81) j/ ∈ S (1 − x j ) , x ∈ [0 , 1] n .
Given a divergence measure D ( ·(cid:107)· ) for probability distributions, we can deﬁne the best conceivable decoupling error to be the divergence between p and the best possible q .
Deﬁnition 1 (Best Conceivable Decoupling Error) .
Considering a cooperative game ( N, F ( S )) , and given a divergence measure D ( ·(cid:107)· ) for probability distributions, the decoupling error is deﬁned as the divergence between q and p : D ( q (cid:107) p ) , and the best conceivable decoupling error is deﬁned as the divergence between the best possible q and p :
[Equation]
Note that the best conceivable decoupling error D ∗ is closely related to the intrinsic coupling amongst n players: if all the players are already independent with each other, then D ∗ could be zero.
4. M EAN F IELD O BJECTIVE FOR EBM S
If we consider the decoupling error D ( q (cid:107) p ) to be the Kullback-Leibler divergence between q and p , then we recover the mean ﬁeld approach 2 .
Given the EBM formulation in Eq.
(1), the classical mean-ﬁeld inference approach aims to approximate p ( S ) by a fully factorized product distribution q ( S ; x ) :
= (cid:81)
i ∈
S x i (cid:81) j/ ∈ S (1 − x j ) , x ∈ [0 , 1] n , by minimizing the distance measured w.r.t.
the
Kullback-Leibler divergence between q and p .
Since KL ( q (cid:107) p ) is non-negative, we have:
[Equation]
In Eq.
(6) we plug in the EBM formulation that log p ( S ) =
F ( S ) T − log Z. Then one can get log Z ≥
[Equation]
where H ( · ) is the entropy, ELBO stands for the evidence lower bound , and
[Equation]
is the multilinear extension of F ( S )
(Owen, 1972; Calinescu et al., 2007).
Note that the multilinear extension plays a central role in modern combinatorial optimizaiton techniques (Feige et al., 2011), especially for guaranteed submodular maximization problems (Krause and Golovin, 2014).
Maximizing ( ELBO ) in Eq.
(8) amounts to minimizing the Kullback-Leibler divergence between q and p .
If one solves this optimization problem to optimality, one can obtain the q ( S ; x ∗ ) with the best conceivable decoupling error.
Here x ∗ i describes the odds that player i shall participate in the game, so it can be naturally used to deﬁne the importance score of each player.
Deﬁnition 2 (Variational Index of Cooperative Games) .
Consider a cooperative game ( N, F ( S )) and its mean ﬁeld approximation.
Let x ∗ be the variational marginals with the best conceivable decoupling error, we deﬁne s ∗ :=
Tσ − 1 ( x ∗ ) to be the variational index of the game.
Formally,
[Equation]
where x ∗ can be obtained by maximizing the ELBO objective in Eq.
(8) , and σ − 1 ( · ) is the inverse of the sigmoid function, i.e. σ − 1 ( x ) =
log x 1 − x .
For a vector it is applied element-wise.
4. A LGORITHMS FOR C ALCULATING THE V ARIATIONAL I NDEX
Equilibrium condition.
For coordinate i , the partial derivative of the multilinear extension is ∇ i f F mt ( x ) , and for the entropy term, it is ∇ i H ( q ( S ; x ))
= log 1 − x i x i .
By setting the partial derivative of ELBO in Eq.
(8) to be 0, we have the equilibrium condition:
[Equation]
where σ is the sigmoid function.
This equilibrium condition implies that one cannot change the value assigned to any player in order to further improve the overall decoupling performance.
It also implies the ﬁxed point iteration x i ← σ ( ∇ i f F mt ( x ) /T ) .
When updating each coordinate sequentially, we recover the classic naive mean ﬁeld algorithm as shown in Appendix C.
Instead, here we suggest to use the full-gradient method shown in Alg. 1 for maximizing the ELBO objective.
As we will see later, the resultant valuations satisfy certain game-theoretic axioms.
It needs an initial marginal vector x 0 ∈ [0 , 1] n and the number of epochs K .
After K steps of ﬁxed point iteration, it returns the estimated marginal x K .
Algorithm
Input: A cooperative game ( N, F ( S )) with n players.
Initial marginals x 0 ← x ∈ [0 , 1] n .
#epochs K .
Output: Marginals after K steps of iteration: x K 1 for k = 1 → K do
[Equation]
In case Alg.
1 solves the optimization problem to optimality, we obtain the Variational Index.
However, maximizing ELBO is in general a non-convex/non-concave problem, and hence one can only ensure reaching a stationary solution.
Below, when we say Variational Index, we therefore refer to its approximation obtained via Alg. 1 by default.
Meanwhile, the MFI ( x ; K ) subroutine also deﬁnes a series of marginals, which enjoy interesting properties as we show in the next part.
So we deﬁne variational valuations through intermediate solutions of MFI ( x ; K ) .
Deﬁnition 3 ( K -Step Variational Values) .
Considering a cooperative game ( N, F ( S )) and its mean ﬁeld approximation by Alg. 1, we deﬁne the K -Step Variational Values initialized at x as:
[Equation]
[Equation]
where σ − 1 () is the inverse of the sigmoid function ( σ − 1 ( x )
= log x
1 − x ).
Notice when running more steps, the K -Step variational value will be more close to the
Variational Index.
The gradient ∇ f F mt ( x ) itself is deﬁned with respect to an exponential sum via the multilinear extension.
Next we show how it can be approximated via principled sampling methods.
Sampling methods for estimating the partial derivative.
The partial derivative follows,
[Equation]
[Equation]
All of the variational criteria are based on the calculation of the partial derivative ∇ i f F mt ( x ) , which can be approximated by Monte Carlo sampling since ∇ i f F mt ( x ) =
E S ∼ q ( S ;( x | x i ← 0))
[ F ( S + i ) − F ( S )] : we ﬁrst sample m coalitions S k , k = 1 , ..., m from the surrogate distribution q ( S ; ( x | x i ← 0)) , then approximate the expectation by the average 1 m (cid:80) mk =1 [ F ( S k + i ) − F ( S k )] .
According to the
Chernoff-Hoeffding bound (Hoeffding, 1963), the approximation will be arbitrarily close to the true value with increasingly more samples: With probability at least 1 − exp( − m(cid:15) 2 / 2) , it holds that | 1 m (cid:80) mk =1 [ F ( S k + i ) − F
( S k )] − ∇ i f F mt ( x ) | ≤ (cid:15) max S | F ( S + i ) − F ( S ) | , for all (cid:15) > 0 .
Roles of the initializer x 0 and the temperature T .
This can be understood in the following respects: 1)
The initializer x 0 represents the initial credit assignments to the n players, so it denotes the prior knowledge/initial belief of the contributions of the players; 2) If one just runs Alg. 1 for one step, x 0 matters a lot to the output.
However, if one runs Alg. 1 for many steps, x k will converge to the stationary points of the ELBO objective.
Empirically, it will take around 5 ∼ 10 steps to converge.
The temperature T controls the “spreading” of credits/importance assigned to the players: A higher
T leads to more ﬂat assignments, and a lower T leads to more concentrated assignments.
Computational efﬁciency of calculating Variational Index and variational values.
Alg.
1 calculates the K -step variational values, 1-step variatioinal value has the same computational cost as that of Shapley value and Banzhaf value, since they all need to evaluate the gradient of multilinear extension.
Sampling methods could help with approximating all of the three criteria when there are a large number of players.
Variational Index can be approximated by K -step variational value, the number K depends on when Alg. 1 will converge.
One can easily show that, under the setting of maximizing ELBO, x k will converge to some stationary point x ∗ , based on the analysis of mean ﬁeld approximation in Wainwright and Jordan (2008).
We have also empirically veriﬁed the convergence rate of Alg. 1 in Sec.
5.3, and ﬁnd that it always converges with 5 to 10 steps.
So the computational cost is roughly similar as that of Shapley value and Banzhaf value.
4. R ECOVERING C LASSICAL C RITERIA
Perhaps surprisingly, it is possible to recover classical valuation criteria via the K -step variational values as in Def. 3.
Firstly, for Banzhaf value, by comparing with Eq.
(13) it reads,
Ba
[Equation]
which is the 1 -step variational value initialied at 0 .
5 ∗ 1 .
We can also recover the Shapley value through its connection to the multilinear extension (Owen, 1972; Grabisch et al., 2000):
[Equation]
where the integration denotes integrating the partial-derivative of the multilinear extension along the main diagonal of the unit hypercube.
A self-contained proof is given in Appendix D.
These insights offer a novel, uniﬁed interpretation of the two classical valuation indices: both the
Shapley value and Banzhaf value can be viewed as approximating the variational index by running one step of ﬁxed point iteration for the decoupling (ELBO) objective.
Speciﬁcally, for the Banzhaf value, it initializes x at 0 .
5 ∗ 1 , and runs one step of ﬁxed point iteration.
For the Shapley value, it also performs a one-step ﬁxed point approximation.
However, instead of starting at a single initial point, it averages over all possible initializations through the line integration in Eq.
Relation to probabilistic values.
Probabilistic values for games (Weber, 1988; Monderer and Samet, 2002) capture a class of solution concepts, where the value of each player is given by some averaging of the player’s marginal contributions to coalitions, and the weights depend on the coalitions only.
According to (Monderer and Samet, 2002, Equation (3.1)), a solution φ is called a probabilistic value, if for each player i , there exists a probability p i ∈ ∆( C i ) , such that φ i is the expected marginal contribution of i w.r.t.
p i .
Namely, φ i = (cid:80) S ∈ C
i p i ( S )
[ F ( S + i ) − F ( S )] , where C i is the set of all subsets of N − i , and ∆( C i ) is the set of all probability measures on C i .
One can easily see that, for any ﬁxed x , 1-step variational value in Def.
3 is a probabilistic value with p
i ( S ) =
q ( S ; ( x | x i ← 0)) , where q ( S ; x ) is the surrogate distribution in our EBM framework.
4. A XIOMATISATION OF K -S TEP V ARIATIONAL V ALUES
Our EBM framework introduces a series of variational values controlled by T and the running step number K .
We now establish that the variational values Tσ − 1 ( MFI ( x ; K )) in Def.
3 satisfy certain game-theoretic axioms (see Appendix B for deﬁnitions of ﬁve common axioms: Null player,
[Figure] [Figure]
Symmetry, Marginalism, Additivity and Efﬁciency).
We prove that all the variational values in the trajectory satisfy three fundamental axioms: null player, marginalism and symmetry.
Detailed proof is deferred to Appendix E. Note that it might be very difﬁcult or even impossible to ﬁnd equivalent axiomatisations of the series of variational values, which will be left for future work.
Meanwhile, our methods incur a decoupling and fairness tradeoff by tuning the hyperparameters K and T .
Theorem 1 (Axiomatisation of K -Step Variational Values of Def. 3) .
If initialized uniformly, i.e., x 0 = x 1 , x ∈ [0 , 1] , all the variational values in the trajectory Tσ − 1 ( MFI ( x ; k )) , k = 1 , 2 , 3 ... satisfy the null player, marginalism and symmetry axioms.
According to Theorem 1, our proposed K -step variational values satisfy the minimal set of axioms in order to being an appropriate valuation criterion.
Note that speciﬁc realizations of the K -step variational values could also satisfy more axioms, for example, the 1 -step variational value initialied at 0 .
5 ∗ 1 also satisﬁes the additivity axiom.
Furthermore, we have the following observations:
Satisfying more axioms is not essential for valuation problems.
Notably, in cooperative game theory, one line of work is to seek for solution concepts that would satisfy more axioms.
However, for valuation problems in machine learning, this is arguably not essential.
For example, similar as what Ridaoui et al.
(2018) argues, efﬁciency does not make sense for certain games.
We give a simple illustration in Appendix F, which further shows that whether more axioms shall be considered really depends on the speciﬁc scenario being modeled, which will be left for important future work.
E MPIRICAL S TUDIES
Throughout the experiments, we are trying to understand the following: 1) Would the proposed Variational Index have lower decoupling error compared to others?
Could the proposed
Variational Index gain beneﬁts compared to the classical valuation criteria for valuation problems?
Since we are mainly comparing the quality of different criteria, it is necessary to rule out the inﬂuence of approximation errors when estimating their values.
So we focus on small-sized problems where one can compute the exact values of these criteria in a reasonable time.
Usually this requires the number of players to be no more than 25.
Meanwhile, we have also conducted experiments with a larger number of players in Appendix G.5, in order to show the efﬁciency of sampling methods.
We choose T empirically from the values of 0.1, 0.2, 0.5, 1.0.
We choose K such that Alg.
1 would converge.
Usually, it takes around 5 to 10 steps to converge.
We give all players a fair start, so xwas intialized to be 0 .
5 × 1 .
Code will be released soon.
We ﬁrst conduct synthetic experiments on submodular games (details defered to Appendix G.1), in order to verify the quality of solutions in terms of the true marginals p ( i ∈ S ) .
One can conclude
[Figure]
that Variational Index obtains better performance in terms of MSE and Spearman’s rank correlation compared to the one-point solutions (Shapley value and Banzhaf value) in all experiments.
More experimental results on data point and feature valuations are deferred to Appendix G.
5. E XPERIMENTS ON D ATA V ALUATIONS
We follow the setting of Ghorbani and Zou (2019) and reuse the code of https://github.com/ amiratag/DataShapley .
We conduct data removal: training samples are sorted according to the valuations returned by different criteria, and then samples are removed in that order to check how much the test accuracy drops.
Intuitively, the best criteria would induce the fastest drop of performance.
We experiment with the following datasets: a) Synthetic datasets similar as that of
Ghorbani and Zou (2019); b) The breast cancer dataset, which is a binary classiﬁcation dataset with 569 samples; c)
The digits dataset, that is a 10-class classiﬁcation dataset with 1797 samples.
The above two datasets are both from UCI Machine Learning repository ( https://archive.
ics.uci.edu/ml/index.php ).
Speciﬁcally, we cluster data points into groups and studied two settings: 1)
Grouping the samples randomly; 2) Clustering the samples with the k-means algorithm.
For simplicity, we always use equal group sizes.
The data point removal corresponds to singleton groups.
Fig.
1 shows the results.
One can observe that in certain situations the Variational Index achieves the fastest drop rate.
It always achieves the lowest decoupling error (as shown in the legends in each of the ﬁgures).
Sometimes Variational Index and Banzhaf show similar performance.
We expect that this is because the Banzhaf value is a one-step approximation of Variational Index, and for the speciﬁc problem considered, the ranking of the solutions does not change after one-step of ﬁxed point iteration.
There are also situations that the rankings of the three criteria are not very distinguishable, however, the speciﬁc values are also very different since the decoupling error differs.
5. E XPERIMENTS ON F EATURE V ALUATIONS /A TTRIBUTIONS
We follow the setting of Lundberg and Lee (2017) and reuse the code of https://github.com/ slundberg/shap with an MIT License.
We train classiﬁers on the Adult dataset 3 , which predicts whether an adult’s income exceeds 50k dollar per year based on census data.
It has 48,842 instances and 14 features such as age, workclass, occupation, sex and capital gain (12 of them used).
Feature removal results.
This experiment follows a similar fashion as the data removal experiment: we remove the features one by one according to the order deﬁned by the returned criterion, then observe the change of predicted probabilities.
Fig.
2 reports the behavior of the three criteria.
The ﬁrst row shows the results from an xgboost classiﬁer (accuracy: 0.893), second row a logistic regression 3 https://archive.ics.uci.edu/ml/datasets/adult
[Figure] [Figure]
[Figure]
classiﬁer (accuracy: 0.842), third row a multi-layer perceptron (accuracy: 0.861).
For the dropping results, Variational Index usually induces the fastest drop, and it always enjoys the smallest decoupling error, as expected from its mean-ﬁeld nature.
From the waterfall plots, one can see that the three criteria indeed produce different rankings of the features.
Take the ﬁrst row for example.
All criteria put “Capital Loss” and “Relationship” as the ﬁrst two features.
However, the remaining features have different ranking: Variational Index and Banzhaf indicate that “Marital Status” should be ranked third, while Shapley ranks it in the fourth position.
It is hard to tell which ranking is the best because: 1)
There is no golden standard to determine the true ranking of features; 2)
Even if there exists a ground truth ranking of some “perfect model”, the trained xgboost model here might not be able to reproduce it, since it might not be aligned with the “perfect model”.
The feature removal experiments nevertheless demonstrate the intriguing behavior of Variational Index.
Average results.
We further provide the bar plots and averaged ranking across the adult datasets in
Fig.
From the bar plots one can see that different criterion has slightly different values for each feature on average.
Average rankings in the table demonstrate the difference: The three methods do not agree on the colored features, for example, “Age”, “Education-Num” and “Captical Loss”.
5. E MPIRICAL C ONVERGENCE R ESULTS OF A LG .
Table 1 shows convergence results of Alg. 1 on feature and data valuation experiments.
The value in the cells are the stepwise difference of x k , (cid:107) x k − x k − 1 (cid:107) 2 n , which is a classical criterion to measure the convergence of iterative algorithms.
One can clearly see that Alg.
1 converges in 5 to 10 iterations.
[Table] [Table]
D ISCUSSIONS AND F UTURE W ORK .
We have presented an energy-based treatment of cooperative games, in order to improve the valuation problem.
It is very worthwhile to explore more in the following directions: 1) Choosing the temperature T .
The temperature controls the level of fairness since when T → ∞ , all players have equal importance, when T → 0 , a player has either 0 or 1 importance.
Perhaps one can use an annealing-style algorithm in order to control the fairness level: starting with a high temperature and gradually decreasing it, one can obtain a series of importance values under different fairness levels.
Given the probabilistic treatment of cooperative games, one can naturally add priors over the players, in order to encode more domain knowledge.
It may also make sense to consider conditioning and marginalization in light of practical applications.
It’s very interesting to explore the interaction of a group of players in the energy-based framework, which would result in an “interactive” index among size- k coalitions.
E THICS S TATEMENT AND B ROADER I MPACT
Besides the valuation problems explored in this work, cooperative game theory has already been applied to a wide range of disciplines, to name a few, economics, political science, sociology, biology, so this work could potentially contribute to broader domains as well.
Meanwhile we have to be aware of possible negative societal impacts, including: 1) negative side effects of the technology itself, for example, possible unemployment issue due to the reduced amount of the need of valuations by human beings; 2) applications in negative downstream tasks, for instance, the data point valuation technique could make it easier to conduct underground transactions of private data.
R EPRODUCIBILITY S TATEMENT
All the datasets are publicly available as described in the main text.
In order to ensure reproducibility, we have made the efforts in the following respects: 1) Provide a sampled code as the supplementary material.
2) Provide self-contained proofs of the main claims in Appendices D and E; 3) Provide more details on experimental conﬁgurations and experimental results in Appendix G
Appendix C ONTENTS
A Derivations of the Maximum Entropy Distribution
B Common Axioms of Valuation Criteria
C
The Naive Mean Field Algorithm
D Proof of Recovering Classical Criteria
E Proof of Theorem
F Miscellaneous Results in Sec.
G More Conﬁguration Details and Experimental Results
G.1 Synthetic Experiments on Submodular (FLID) Games . . . . . . . . . . . . . . . .
G.2 Details of the Models for Feature Valuations . . . . . . . . . . . . . . . . . . . . .
G.3 More Results on Feature Valuations . . . . . . . . . . . . . . . . . . . . . . . . .
G.4 More Average Results on Feature Valuations . . . . . . . . . . . . . . . . . . . . .
G.5 Experiments with More Players . . . . . . . . . . . . . . . . . . . . . . . . . . . .
G.6 Effect of Number of Samples in MCMC Sampling . . . . . . . . . . . . . . . . .
A D ERIVATIONS OF THE M AXIMUM E NTROPY D ISTRIBUTION
One may wonder why do we need energy-based treatment of valuation problems in machine learning?
Speciﬁcally, under the setting of cooperative games ( N, F ( S )) , why we have to take the exponential form of EBM p ( S )
∝ exp( F ( S ) /T ) ?
Because one may also formulate it as something else, say, p ( S )
∝ (1 + | F ( S ) | ) ?
In one word, because EBM is the maximum entropy distribution.
Being a maximum entropy distribution means minimizing the amount of prior information built into the distribution.
Another lens to understand it is that: Since the distribution with the maximum entropy is the one that makes the fewest assumptions about the true distribution of data, the principle of maximum entropy can be seen as an application of Occam’s razor.
Meanwhile, many physical systems tend to move towards maximal entropy conﬁgurations over time (Jaynes, 1957a;b).
In the following we will give a derivation to show that p ( S ) ∝ exp( F ( S ) /T ) is indeed the maximum entropy distribution for a cooperative game.
The derivation bellow is closely following Jaynes (1957a;b) for statistical mechanics.
Suppose each coalition S is associated with a payoff F ( S ) with probability p ( S ) .
We would like to maximize the entropy H ( p ) =
− (cid:80) S ⊆
N p ( S ) log p ( S ) , subject to the constraints that (cid:80) S p ( S )
= 1 , p ( S ) ≥ 0 and (cid:80) S p ( S ) F ( S ) =
µ (i.e., the average payoff is known as µ ).
Writing down the Lagrangian
[Equation]
Setting
[Equation]
we get:
[Equation]
is the log-partition function.
So,
[Equation]
Note that the maximum value of the entropy is
[Equation]
So one can get,
[Equation]
which deﬁnes the inverse temperature.
So we reach the exponential form of p ( S ) as:
[Equation]
B C OMMON A XIOMS OF V ALUATION C RITERIA
Following the deﬁnitions in Covert et al.
(2020a), the ﬁve common axioms are listed as bellow.
φ i ( F ) denotes the value assigned to player i in the game ( N, F ( S )) .
Note that the notions might be slightly different in classical literature of game theory.
Null player:
For a player i in the cooperative game ( N, F ( S )) , if F ( S + i ) = F ( S ) holds for all
S ⊆
N − i , then its value should be φ i ( F ) = 0 .
Symmetry:
For any two players i, j in the cooperative game ( N, F ( S )) , if F ( S + i ) =
F ( S + j ) holds for all S ⊆
N −
i − j , then it holds that φ i ( F ) =
φ j ( F ) .
Marginalism: For two games ( N, F ( S )) and ( N, G ( S )) where all players have identical marginal contributions, the players obtain equal valuations: F ( S + i ) − F ( S )
=
G ( S + i ) − G ( S ) holds for all ( i, S ) , then it holds that φ i ( F ) =
φ i ( G ) .
Additivity: For two games ( N, F ( S )) and ( N, G ( S )) , if they are combined, the total contribution of a player is equal to the sum of its individual contributions on each game: φ i ( F + G ) =
φ i ( F )+ φ
i ( G ) .
Efﬁciency: The values add up to the difference in value between the grand coalition and the empty coalition: (cid:80) i ∈ N φ
i ( F ) =
F ( N ) − F ( ∅ ) .
Remark
(The notion of “marginalism”) .
Speciﬁcally, the marginalism axiom was ﬁrst mentioned by (Young, 1985, Equation 7 on page 70), where it was called the “independence” condition; Following (Chun, 1989, page 121), it was formally called the “marginality”.
This axiom requires a player’s payoffs to depend only on his own marginal contributions – whenever they remain unchanged, his payoffs should be unaffected.
v σ
[Equation]
v i f F mt
− x ∗
[Equation]
The naive mean ﬁeld algorithm is one of the most classical algorithm for mean ﬁeld inference.
It is summarized in Alg. 2.
D P ROOF OF R ECOVERING C LASSICAL C RITERIA
For Banzhaf value, by comparing its deﬁnition in Eq.
(2) with Eq.
(13) it reads,
Ba
[Equation] [Equation]
which is the 1-step variational value initialied at 0 .
5 ∗ 1 .
For Shapley value, according to Grabisch et al. (2000), here we prove a stronger conclusion regarding the generalization of Shapley value: Shapley interaction index, which is deﬁned for any coalition S :
[Equation]
[Equation]
Given Hammer and Rudeanu (2012), we have a second form of the multilinear extension as:
[Equation] [Equation]
where a ( S ) :
= (cid:80) T ⊆ S ( − 1) | S |−| T | F ( T ) is the Mobius transform of F ( S ) .
Then, one can show the S -derivative of f F mt ( x ) is (suppose S
= { i 1 , ..., i | S | } ),
[Equation]
So,
[Equation] [Equation]
Then it holds,
[Equation] [Equation]
[Equation]
According to Grabisch (1997), we have Sh S
= (cid:80) T ⊇ S a ( T )( | T | − | S | + 1) − 1 , then we reach the conclusion: [Equation]
[Equation]
When | S | = 1 , we recover the conclusion for Shapley value.
E P ROOF OF T HEOREM
Theorem 1 (Axiomatisation of K -Step Variational Values of Def. 3) .
If initialized uniformly, i.e., x 0 = x 1 , x ∈ [0 , 1] , all the variational values in the trajectory Tσ − 1 ( MFI ( x ; k )) , k = 1 , 2 , 3 ... satisfy the null player, marginalism and symmetry axioms.
Proof of Theorem 1.
In step k , we know that the value to player i is:
[Equation]
For the null player property, since F ( S + i ) =
F ( S ) always holds, it is easy to see that
Tσ − 1 ( MFI ( x ; k ))
i = 0 holds for all
i ∈ N .
Now we will show that the symmetry property holds.
The value to player i (cid:48) is:
[Equation]
Now let us compare different terms in the summands of Eq.
(36) and Eq.
We try to match the summands one by one.
There are two situations:
Situation I: For any S ⊆ N − i − i (cid:48) , we choose S (cid:48)
=
S .
In this case we have F ( S + i ) − F ( S )
=
F ( S (cid:48)
+ i (cid:48) )
− F ( S (cid:48) ) .
For the products of x we have:
[Equation]
We know that x i (cid:48)
=
x i holds from step 0, by simple induction, we know that x
i (cid:48)
=
x i holds for step k as well.
So in this situation, the summands equal to each other.
Situation II: For any S
=
A
+ i (cid:48) , we choose S (cid:48)
=
A + i , where A ⊆ N − i − i (cid:48) .
In this case, it still holds that F ( S + i ) − F ( S ) =
F ( S (cid:48)
+ i (cid:48) )
− F ( S (cid:48) ) .
For the products of x we have:
[Equation]
Again, by the simple induction, we know that x i (cid:48)
=
x i holds for step k .
The above two situations ﬁnishes the proof of symmetry.
For the marginalisim axiom, one can see that the update step for the two games are identical, and it is easy to deduce that they produce exactly the same trajectories, given that they have the same initializations.
F M ISCELLANEOUS R ESULTS IN S EC .
Gradient of entropy in Sec.
4.2 Note that q is a fully factorized product distribution q ( S ; x ) :=
(cid:81)
i ∈
S x i (cid:81) j/ ∈ S (1 − x j ) , x ∈ [0 , 1] n , so its entropy H ( q ( S ; x )) can be written as the sum of entropy of n independent Bernoulli distributions.
And the entropy of one Bernoulli distribution with parameter x i is
[Equation]
So we have,
[Equation]
Satisfying more axioms is not essential for valuation problems.
Notably, in cooperative game theory, one line of work is to seek for solution concepts that would satisfy more axioms.
However, for valuation problems in machine learning, this is arguably not essential.
For example, similar as what Ridaoui et al.
(2018) argues, efﬁciency does not make sense for certain games.
For a simple illustration, let us consider a voting game from a classiﬁcation model with 3 binary features x ∈ { 0 , 1 } 3 with weights w =
[2 , 1 , 1] (cid:62) :
f ( x ) :
= 1 { w (cid:62) x ≥ 3 } .
Now we are trying to ﬁnd the valuation of each feature in N = { x 1 , x 2 , x 3 } .
Naturally, the value function in the corresponding voting game shall be F ( S ) =
f ( x S ) where x S means setting the coordinates of x inside S to be 1 while leaving others to be 0.
In this game let us count how many times each feature could ﬂip the classiﬁcation result: for feature x 1 , there are three situations: F ( { 1 , 2 } ) − F ( { 2 } ) , F ( { 1 , 3 } ) −
F ( { 3 } ) and F ( { 1 , 2 , 3 } ) − F ( { 2 , 3 } ) ; for feature x 2 , there are one situation: F ( { 1 , 2 } ) − F ( { 1 } ) ; for feature x 3 , there are one situation: F ( { 1 , 3 } ) − F ( { 1 } ) .
Then the voting power (or valuation) of each feature shall follows a 3 : 1 : 1 ratio.
By simple calculations, one can see that the Banzhaf values of the three features are 34 , 14 , 14 , which is consistent with the ratio of the expected voting power.
However, the Shapley values of them are 46 , 16 , 16 , which is not consistent due to satisfying the efﬁciency axiom.
By the above example we are trying to explain that for valuation problems, satisfying more axioms is not necessary, sometimes even does not make sense.
Whether more axioms shall be considered and which sets of them shall be added really depend on the speciﬁc scenario, which will be left for important future work.
G M ORE C ONFIGURATION D ETAILS AND E XPERIMENTAL R ESULTS G. S YNTHETIC E XPERIMENTS ON S UBMODULAR (FLID) G AMES
Here we deﬁne a synthetic game with the value function as a FLID (Tschiatschek et al., 2016) objective F ( S ) , which is a diversity boosting model satisfying the submodularity property.
We know that its multilinear extension admits polynomial time algorithms (Bian et al., 2019).
Let
W ∈ R | N |× D
+ be the weights, each row corresponds to the latent representation of an item, with D as the dimensionality.
Then
[Equation]
which models both coverage and diversity, and u (cid:48)
i =
u
i − (cid:80) D d =1 W i,d .
In order to test the performance of the proposed variational objectives, we consider small synthetic games with 6, 8 and 10 players such that the ground truth marginals can be computed exhaustively.
We would like to compare with the true marginals p ( i ∈ S ) since they represent the probability that player i participates in all coalitions, which is hard to compute in general.
The distance to the true marginals is also a natural measure of the decoupling error as deﬁned in Def. 1.
We apply a sigmoid function to Shapley value and Banzhaf value in order to translate them to probabilities. We calculate the mean squared error
[Equation]
since we would like to clearly show positive players (marginal > 0.5) and negative players (marginal < 0.
(MSE) and Spearman’s rank correlation ( ρ ) to the ground truth marginals p ( i ∈ S ) and report them in the ﬁgure legend.
Fig.
4 collects the ﬁgures, one can see that the Variational Index clearly obtains better performance in terms of MSE and Spearman’s rank correlation compared to the one-point solutions (Shapley value and Banzhaf value) in all experiments.
G. D ETAILS OF THE M ODELS FOR F EATURE V ALUATIONS
For xgboost, the train accuracy is 0.8934307914376094, the speciﬁc conﬁguration with the xgboost package is:
XGBClassifier ( base_score =0.5 , booster = ’ gbtree ’ , colsample_bylevel =1 , colsample_bynode
=1 , colsample_bytree =1 , gamma=0 , gpu_id
=−1, importance
[Table]
, random
[Table]
[Table]
We used the sklearn package for the logistic regression and MLP classiﬁers.
For logistic regression, the accuracy is 0.8418967476428857, the speciﬁc conﬁguration is:
LogisticRegression ( random_state =0 , solver =" l i b l i n e a r " , C=0.5)
For the MLP, its
G. M ORE R ESULTS ON F EATURE V ALUATIONS
In this part we provide more results on feature removal in Figures 5 and 6.
Fig.
5 shows similar behavior as that shown in the main text.
Fig.
6 provides not very distinguishable results of the three criteria.
G. M ORE A VERAGE R ESULTS ON F EATURE V ALUATIONS
We provide additional statistical results with the MLP model (Fig.
8) and logistic regression model (Fig. 9).
Meanwhile, we also put the full statistics on the xgboost model in Fig.
7 since in the main text the table of the samples with “False” grouthtruth label is skipped due to space limit.
From Fig.one can observe that Variational Index induces different rankings of the features compared to Shapley and Banzhaf: Variational Index ranks “Marital Status” the second, while Shapley and Banzhaf put it in the third location.
It is also very interesting to see that the logistic regression model (with the lowest training accuracy among the three models, shown in Fig. 9) provides different ranking for the ﬁrst two features compared to MLP and xgboost.
For the samples with “True” groundtruth labels, “Education-Num” is the ﬁrst important feature for the logistic regression model, while “Captical Gain” was ranked ﬁrst for the
MLP and xgboost.
G. E XPERIMENTS WITH M ORE P LAYERS
Furthermore, we experiment with a bit larger number of players ( n = 80 ) using MCMC sampling to approximate the partial derivative in Eq.
The sampling based approximation works pretty fast.
Table 2 shows the top 15 ranked players returned by our method, Shapley value and Banzhaf value for a synthetic data valuation problem.
Note that the ranking of Variational Index is more similar to that of Banzhaf value than that of Shapley value.
[Figure]
[Figure]
𝑇 =
[Figure]
𝑇 =
[Figure]
𝑇 =
[Figure]
G. E FFECT OF N UMBER OF S AMPLES IN MCMC S AMPLING
Here we illustrate the accuracy sampling tradeoff when estimating the gradient of multilinear extension using MCMC sampling.
The results is shown in Fig.
One can observe that with more samples, Alg. 1 will converge faster (in fewer number of epochs).
[Figure]
[Figure] [Figure]
[Figure] [Figure]
[Figure] [Figure]
[Table]
Variational Index
Banzhaf value
Shaplay Value
[Figure]
[Figure] [Figure]
[Figure] [Figure]