Explaining Preferences with Shapley Values
Authors: Robert Hu
Siu Lun Chau
Jaime Ferrando Huertas
Dino Sejdinovic
Venue: NeurIPS 2022 Accept (2022)
Keywords: Interpretability
Preference Learning
Kernel
Shapley Values
RKHS

Abstract: While preference modelling is becoming one of the pillars of machine learning, the problem of preference explanation remains challenging and underexplored. In this paper, we propose \textsc{Pref-SHAP}, a Shapley value-based model explanation framework for pairwise comparison data. We derive the appropriate value functions for preference models and further extend the framework to model and explain \emph{context specific} information, such as the surface type in a tennis game. To demonstrate the utility of \textsc{Pref-SHAP}, we apply our method to a variety of synthetic and real-world datasets and show that richer and more insightful explanations can be obtained over the baseline.

Explaining Preferences with Shapley Values
Robert Hu*t
Siu Lun Chau" Department Statistics
Jaime Ferrando Huertas Shaped New York
Amazon London
University of Oxford
Dino Sejdinovic School Computer and Mathematical Sciences University of Adelaide
Abstract
While preference modelling is becoming of the pillars of machine learning, the problem of preference explanation remains challenging and underexplored. In this paper. we propose PREF-SHAP. Shapley value-based model explanation frame- work for pairwise comparison data. We derive the appropriate value functions for preference models and further extend the framework model and explain context specific information, such as the surface type in tennis game. To demonstrate the utility of PREF-SHAP, we apply our method variety of synthetic and real-world datasets and show that richer and more insightful explanations can be obtained over the baseline.
1 Introduction
Preference learning [1] is a classical problem in machine learning, where one is interested in learning the order relations on collection of data items. Preference learning algorithms 2 often assume that there latent utility function f : X + dictating the outcome of preferences, where X' denotes the domain of item covariates. An explicit feedback such as item ratings or rankings from recommender systems can be treated noisy evaluations of f, whereas pairwise comparison data (also known as duelling data) arising from, c.g., sports match outcomes
7 can be used to
implicitly infer f. i.. item (4) is preferred over (beats) item (r) when (xx f(((*)). As shown by Kahneman and Tversky (8), humans often struggle with evaluating absolute quantities when it comes to eliciting preferences, but are broadly capable of evaluating relative differences, core observation often exploited in preference learning. Motivated by such, this work will focus on
explaining preferences inferred using duelling data.
Explaining preference models is crucial when they applied in areas such as recommendation systems (9]. finance [10, and sports science [11] for the practitioner to trust, debug and understand [12. However, despite its importance, no prior work has studied this problem to the best of our knowledge. While one may suggest applying existing explainability tools such LIME [13), or SHAP [14] learned utility function f. reason that this approach only explains the utility but not the mechanism of eliciting preferences itself. We highlight the important differences between these viewpoints in our numerical experiments. Moreover, the utility-based model places strong rankability assumption on the underlying preferences, meaning that if we f()(0) < fx((r)), then < is a total order on all the items. However, as Pahikkala et al. 15 and Chau et al. [16] have discussed, there are many departures from rankability in practice, e.g. we might easily see a preference of over B, B over C, but C' over - conforming the rock-paper-scisssors relation. Such inconsistent preferences under frequent study in social
the value of their findings
define ( I (r)
'Equal contribution, order decided by coinflip Work mainly done while the authors were with the Department of Statistics, University of Oxford
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

choice theory [7 18 and are of wider interest in both healthcare 19 and retail [20] where data are both large and noisy.
To move beyond the rankability assumption, we will utilise the Generalised Preferential Kernel from [6] model the underlying preferences, and develop PREF-SHAP, novel Shapley value [2 -aaed explainability toolbox, to explain the inferred preferences, Our contributions can be summarised as follows:
1. We propose PREF-SHAP, novel Shapley value-based explainability algorithm, to explain preferences based on duelling data.
2. We empirically demonstrate that PREF-SHAP gives more informative explanations com- pared to the naive approach of applying SHAP to the inferred utility function f.
3. We release high-performant implementation of PREF-SHAP at 22
2
Background materials
We will first give a brief overview of preference learning and Shapley Additive Explana- tions (SHAP) [14. which are the two core concepts of our contribution, PREF-SHAP, described in
Section 3
Notation Scalars are denoted by lower case letters, while vectors and matrices are denoted by bold lower case and upper case letters, respectively. Random variables are denoted by upper case letters. X Idd denotes the item space with d features and y {-1, 1, 1} is the binary preference outcome space We let k : X' x X + R be kernel function and Hi the corresponding reproducing kernel Hilbert space (RKHS).
2.1 Preference Learning
In this section, we will introduce the two approaches model preferences from duelling data, namely the utility based approuch and the more general approach from Chau et al. [16]. Formally, a preference feedback is denoted as duelling, when a pair of items (x(1), (())) E x X' given to a user, and binary outcome y E y telling us whether x()) (() won the duel, observed. general, we observe
m binary preferences among n items, giving the data D (y, x x (u, x)
j=1
We also use X E Ruxd to denote the full item covariate matrix.
Utility-based Preference model (UPM) The following likelihood model is often used 2 to model duelling feedback using latent utility function f:
xx()
= (xx)x) (x( (x())))
where is the logistic CDF, i.e. (2) (1 exp(-2)) Maximum likelihood approaches then deployed to learn the latent utility function f. Consequently, preferences between items can be inferred accordingly from f = {f(x))²? i.e. x; is average preferred over x, if f, f,.
Albeit elegant, there are several drawbacks to this approach in modelling preferences. As mentioned, using a one-dimensional vector f to derive preferences assumes that the items {x: are perfectly rankable, i.e. there a total ordering on X' which the true preferences are consistent with. This is strong assumption that often does not hold in practice. For example, it is well studied that cognitive biases often lead to inconsistent human preferences behavioural economics [8]. Moreover, the
ranking community has also challenged this assumption by devising rankability metrics 23 to test this restrictive assumption in practice.
Generalised Preference Model (GPM) Chau et al. [16 proposed to model preference directly using a more general X' x X - that captures the preference within any pair items, using the likelihood
p v x x()) = (09 (x(0) (19(x ((0,,(')))
(2)
We note that has to be a skew-symmetric function to ensure the natural property ply x((), x(r)) =
1 - p④y x (r), x x()). The utility based approach can be obtained as special case of this model, i.e.
'Thus. we do not model 'draws' in match outcomes, but the model can be straightforwarddy extended to include them by specifying the appropriate likelihood function.

---

We propose that when one is interested in modelling (and thus explaining) pairwise preferences, we should consider the preference function g directly instead of explaining preferences based on a restrictive utility model f.
by setting =
-
We follow Chau et al. [6]] approach model g pon-parametrically using kernel methods (25
We assume as a function lives in the following RKHS of skew-symmette functions: given kernel
k X' x X' + I defined on the item space X, the generalised preferential kernel kg on X' X' is constructed as follows:
kg ((x)
( - (x (0)
(xxx)) 1s x
This kernel allows us to model the similarity across pairs of items. Moreover, if & is a universal kernel (26), then kg also satisfies the corresponding notion of universality, meaning that the corresponding RKHS Hkg is rich enough to approximate any bounded continuous skew-symmetric function arbitrarily well [16] Theorem. 1]. To infer g 'Hgg using likelihood (2) one simply runs
kernel logistic regression with data y labels and (x (()) inputs. We will refer to this approach as the Generalised Preference Model (GPM).
We emphasize that explaining GPM allows us to specifically explain inconsistent prefer- ences, which in contrast to explaining rank allows us infer preferences even when transitivity is violated. Such insights can be of great importance in broader contexts such as decision theory [27 and utility theory (28) where transitivity does not hold.
Incorporating context variables. Besides item-level covariates x E X, when there exist additional
context covariates u U S Red that describe the context in which a specific pairwise comparison is made, they can be incorporated into the kernel design discussed in Chau et al. [16] Appendix. BJ. Examples of such context covariates could be court type when a tennis match is conducted, or where different user compares two clothing items in e-commerce. Considering the enriched dataset { (1), (E)
x3 (r) }
D =
now model the preference incorporating the context as:
p(y u. xxx()x(() the context-specific preference function IV u x X' X- I can be learnt non parametrically with the following kernel,
(90 (u.x(), xxx())x('))) Now, given kernel ky defined on the context space U.
1.(0) (u, , xx'))
= bi (u;, uz) KE (xx(), x (x59) (15,xx)))) xx)))
We refer this approach as the Context speecttcc Generalised Preference Model (C-GPM).
2.2 Shapley Additive Explanations (SHAP)
To explain preferences, we will utilise the popular SHAP (SHapley Additive exPlanations) paradigm, which is based on the concept of Shapley values (SV). SV [2] were originally proposed credit allocation scheme for group of d players in the context of cooperative games, which are characterised by a value function value for player j in game v is defined as:
[0, 1]dd R that measures utility of subsets of players. Formally, the Shapley
(v(()=
Σ SCOW/
where Ω = (1,....., d) is the set of players of the game. Given value function the Shapley values are proven to be the only credit allocation scheme that satisfies particular set of favourable and fair game theoretical axioms, commonly known as efficiency, null player property, symmetry and additivity (21) Strumbelj and Kononenko (29 later connect Shapley values to the field of explainable machine learning by drawing an analogy between model fitting and cooperative game. Given a specific data point, by considering its features as players participating in a game that measures features' utilities, the Shapley values obtained can be treated as local feature importance scores. Such games are typically defined through the value functions defined below.
Definition 2.1 (Value functions). Let X be random variable on X C Rd and f X 18 a model from hypothesis space H. The value function v X x [0, 1]e x H is given by
Vx.s(f) = (xr [f([xs, xs=}) X5= xxs]
where is an appropriate reference distribution, X is the subvector of X corresponding to the feature set S, S is the complement of the feature set S and (Xs, Xs=) = X denotes the concatenation of Xs and Xse.
3

---

In other words, given a data point the utility of the feature subset S is defined as the impact on the model prediction, after "removing" the contribution from S via integration with respect to the reference distribution r. These seemovll-bssed strategies are common in the explainability literature [30]_ Nonetheless, the correct choice of the reference distribution has been a long-standing debate [31] Janzing et al. [32] argued from causality perspective that the feature marginal dis-
tribution should be used as the reference distribution, i.c. r r(Xs= Xs = xs) = p(Xs*) where p is the data distribution. On the other hand, Frye et al. [33] disagreed by pointing out these "marginal" value functions ignore feature correlations and lead to unintelligible explanations in higher-dimensional data, and they instead advocate the use of conditional distribution reference, Le. r(Xs= s Xs = Ø5) = p(Xg= [Xg = zs). Thus, there is no consensus and in fact, Chen et al. 31 took neutral stand and argued the choice depends the application hand. This also leads to design of value functions for specific problems, e.g. improving local estimation [4 incorporating
causal knowledge [35 36 and modelling structured data [37]. In this paper, will design an appropriate value function for preference learning and show that naive application of the existing value function to preference learning will lead to unintuitive results.
Shapley value estimation. Given a data point x and a model f. estimating Shapley values consist of two main steps: Firstly, for each feature subset S C 22 estimate the value function (S) either by Monte Carlo sampling from the reference distributions r. or by utilissig a model specific structure to speed up the estimation such as in LINEARSHAP [29], DEEPSHAP [14), TREESHAP (38), and RKHH-SHAA [1]. The former sampling procedure is straightforward when r is the marginal distribution, but computationally heavy and difficult when is the conditional distribution, as it involves estimating an exponential number of conditional densities (39] Finally, after estimating
the value functions, one compute the Shapley values based on Eq. 3or by utilising the efficient weighted least square approach proposed by Lundberg and Lee 14].
Estimating value functions when f E Ha.- We give review to the recently introduced RKHS- SHAP algorithm proposed by Chau et al. [[2] as it is another core component for PREF-SHAP. RKHS-SHAP is a SV estimation method for functions in given RKHS. It circumvents the need for any density estimation and utilises the arsenal of kernel mean embeddings [40] to estimate the value functions non- non-porannnn parametricallly Assume & takes a product kernel structure across dimensions, then for any f E H.. by applying the reproducing property [25], the value function can be decomposed as:
Vx,s (f) = (S, [x [b. ((Xs, h-) Xs = xsl) Hh
(5)
(5, kxg M+(Xs (x=xx))
(6)
where kxs is the product of kernels belonging to the feature set S, and Mr(Xg=lXs=xxs f kxge r(Xs= Xs xs)dX5= is the kernel mean embedding 40) of the reference distribu- tion r. Depending on the choice of the reference distribution, one recovers either the standard kernel mean embedding or the conditional mean embedding. This allows us to arrive at a closed form expression of the value function and circumvents the need for fitting an exponential number of conditional densities.
3 Proposed method: PREF-SHAP
In this section, we will present PREF-SHAP, new Shapley explainability toolbox designed to explain preferences by attributing contribution scores over item-level and context-level covariates for our preference models. Recall the likelihood model for C-GPM from Sec. 2.1
p p(u |v| xx,x()),x()))
(y9u (u.x((),x)) (x()))))))))))))))))))))
where gr is the context- incluxt-nncluued preference function that denotes the strength of preference of item x!!) over item (r) under context u. As there are two distinct sets of covariates present, will propose two different value functions to capture the influences from items and context variables respectively, and show how they could be estimated non-parametrically using tools from the kernel methods literature, as in RKHS-SHAP.
3.1 Preferential value function for items
To explain a general preference model g : X x X - R. we propose the following preferential value function for items.

---

Definition 3.1 (Preferential value function for items). Given a preference function g E H. pair of items (x(*), x(") E X x X' to compare, we define the preferential value function for items as ylni); X' x X' x [0, 1 x H - I such that: v (pr) x x s (g) Eng [o(tx X J. (X x
x (0)
= .
(0)
x
(8)
where expectation is taken over the reference q
(2 X
X
( X X X =x x
(2)
We note that v/Pr) is also applicable to the context-specific preference models. For example, applying p(Pr) to 9u := (u, allows one quantify the item covariate's 's influences under specific context u, while applying w(Pr) g = Ep(() 190 quantifies the average influence from each of the item covariates instead.
Similar to standard value functions, the influence of feature set S shared by the items x(9, x(() is measured as the impact on the preference model after "removing" contributions from features in S" via integration with respect to some reference distribution r. Similar to 9, this value function is skew-symmette in its first two arguments, i.e. ((pr)(((5 (x(x)xx(r) x(r),S,g) -y(pr) (x(1), x(9), S, g).
This is justified, since features that "encourage" preference of x(1) over x(r) should naturally be the ones that "discourage" preference of x over x(() to ensure consistency. In this paper, we assume the items are i.i.d sampled from some distribution p. and we utilise the observational data
distribution as reference as in 33 i.c. we take (x X (r)
5 IX
(r) x!") to be x
p (x!] _ x (6) ( x =x²²). Although we decide here to use the observational
distribution as the reference, the corresponding estimation procedure follows analogously if one instead uses the marginal distribution approach in Janzing et
332
Problems with direct application of SHAP to preference model g A naive way of explaining with SHAP general preference model g which assumes no rankability would require concatenation of the items' covariates. Namely, we would set (x((), x()) Rdd and then apply SHAP to the function gz) directly, now giving 2d Shapley values for each observed preference, i.c. two Shapley values for each feature. Not only does this approach require to consider a larger number of feature coalitions during computation (squaring the original amount), but it also ignores that x() and x(1) in fact consist of the same features, leading to inconsistent explanations, i.e. that the same feature in xf)) and x() has different influence, hence giving different explanations simply due to the ordering of items. We illustrate these pitfalls of such a naive approach in Appendix B
Empirical estimation of the preferential value function y (pr)
x(()xx()) x (9) While the preferential
value function is general in the sense that it could be applied to any preference function g. we divert our attention to functions in Hkg. where kg is the generalised preferential kernel introduced in 2.c This allows us adapt the recently introduced RKHS-SHAP to our settings, and we can thus circumvent learning an exponential number of conditional densities as in [33). In the following segment, we prove the existence of the Riesz representation of the preferential value functional, a necessary step to adapt the SKHS-SHAA framework to our setting.
Proposition 3.1 (Preferential value functional for items). Let k be a product kernel on X, i.e. kxx(() x(r))= = II R(() (a((),,'00) Assume k(i) are bounded for all j. then the Riesz representa-
(P)
tion of the functional x(t,,, exists and takes the form:
(p) x (00)))
(x(x!0, 60
(0)()(())))))) (((x()) - ((x(()) 8 K(X(9) .S)
/2
where K(x,, S) = ks( xs) @ MX and ks() = 2 Ojes (,, is the sub- sub-product kernel defined analogously as Xg.
All proofs are included in the appendix. By representing the functionals as elements in the corre- sponding RKHS, we can now estimate the value function non-parametrically using kernel mean embeddings.
Proposition 3.2 Normmmrrcmttric Estimation). Given و (() ()), test items x! (0) x("), the preferential value function at test items x x! for coalition S
ET (x)) ),),)), datasets
5

---

Table 1: summary of how our preference value functions can tackle different explanation tasks
Candidare
Explanarion of interest
Value function
Preference function
x x0,,x(7 x(()
Which item features contributed most to this duel?
(pr)
x
xidis
g. By (U, --))
x(O
Which item features contributed most to x(O's matches?
tt Ei=
& x
(Pr)
9. By (gv --)
u, x.x(f x0) xx())
Which context features contributed most to this duel?
uux0 x
90
Which context features contributed most average?
u.x x x
90
and preference function can be estimated as
(1) x(0) (9) =a (T(X))
X OoiX
)) - -I(! (xx))xx))) ( r(x xx(X))xx)))) x(2)). (2)
where r(X' X5
(1) (6)) = K x x0 & x
X K Xx x
10: Kx = Kx + nAI, a
Xge
{a m and À > is regularisation parameter.
3.2 Preferential value function for contexts
The influence an individual context feature in U has on a C-GPM function gu can be measured by the following value function.
Proposition 3.3 (Preferential value function for contexts). Given a preference function gu E Heg. denote Ω (l....,''), d'), then the utility of context features S S 52' on (1, x(ry is mea- uxxi1 s sg = Elgu ((us, Us= }, xx))) Ug = usl where the expectation taken over the observational distribution of U. Now, given a test triplet (u, xxx)) ((),,, if gu =ーz= aghh (u,, ),))).))) the non-parametric estimator is:
sured by (PP)
gl0o) 1,((0 x(r), (āv) = a ((KUg. @ Ugre (Kug, + mXT) KU Kugr.ug)
④ (x00)
where x(1) = (Kx0,xx x00 ② K xx) x @ K Kx)x)))
Analogously, the average influence of specific context feature can be computed by taking an average over all pairs of matches, i.e. by using a modified value function m
プ (PE)) u.x
(6) xx .s (āv). We
summarise different ways to modify the proposed preferential value functions to interrogate the preference models in Table [
Computational complexity of PREF-SHAP When computing GPM, it is fundamentally a kernel ridge regression (KRR), which naïvely has complexity ( There exists a multitude of approximation techniques for KRR, the most common type being the Nyström approximation 41. For all our experiments, we use FALKON 42 a large-scale library for solving kernel logistic regression using preconditioned conjugate gradient descent and Nyström approximations. FALKON has a computational complexity of O(n which effectively becomes the complexity for GPM when using FALKON. As the value function for GPM requires estimating conditional mean embeddings, which in turn also are KRR's, s, one can appeal to FALKON again to reduce complexity to (n). We summarize the procedure of PREF-SHAP in Algorithm [ We further detail computational details pertaining to computing coalitions S and batched conjugate gradient descent (BatchedCGD) in Appendix
4 Experiments
The main focus of our experiments is to illustrate the difference between explaining GPM (PREF- SHAP) and applying SHAP to UPM, thus highlighting the difference in explaining the mechanism of eliciting preferences and explaining the utility. When we explain UPM, we first explain how items (1), (r) affect their utilities f(x()), f((())). Explaining the utility corresponds to calculating the value functions of the utilities 12(0) ((f) and (2(r), s s(f). f). By linearity of SHAP values [ and the simple structure relating preference and utilities in UPM, we can explain UPM by subtracting the
Shapley values of ( with x(r). However, this type of explanation is only correct when data rankable, which seldom happens in practice, thus motivating PREF-SHAP.

---

Algorithm 1 PREF-SHAP
Input:
Solution a, datasets X ( X, U, test items x(1) x((), u, batch size ny number of coalition samples n.s context-specific flag eflg
1:
Compute effective dimension der := Number of features with variance greater than
2: Compute coalitions S = {Si,.... Sng J. form binary matrix Z E {0, (0,1)nse 1p from S, and
compute weights W = [W1.... Wnsl with w;
d-1
(1
3: for batch S S do
4: if cfig Take S, S of ( x X, x else Take S of U,u
5: if cflg Compute , [K X f0 K Xx (o) else UK mXT) Kuggrugr using
xsix))
BatchedCGD
6: if cflg Compute ((P)
x(1) (g) else i (1))))) (ōv)
7: end for
8: if cflg Set V2 V = (b(1)
x(1)xx() ((), S, (g) b= else Vx = (uxx(()) x ( (00))
B
9: Calculate Shapley values __ (Z wZ) Z Wv.
10: return B_
We apply PREF-SHAP to unrankable synthetic and real-world datasets to connect theory with practice. We split data, i.c matches with their outcomes, into train (80%), validation (10%), and test (10%) and explain the model kernels are selected using gradient descent, based on the proposed method in [43]. We first generate synthetic duelling data where performance be compared against ground truth, demonstrate that PREF-SHAP capable identifying the relevant features.
random subset of the data. The hyperparameters for the
Synthetic data
We first consider synthetic experiment with unrankable duelling data. We generate
the items by first sampling 1000 item covariates [x:[0] .rABB. .AAC ,I:BO =: P: E RA1 N 4). We associate each item with a cluster membership E (A, B, C), where the assignment is randomly chosen for each item with equal probability. We then form the full item covariate by concatenating P: with one-hot encoded o; x; [p;, one_hot(c:)], 40000 matches between randomly chosen pairs of items are conducted by the following mechanism: match outcomes are decided based on the underlying cluster membership of the items. For example, if an item from cluster A competes against an item from cluster B, the winner decided by their inter-cluster covariate A: i.e. 3 j ry 2A AAB When the match is between members of the same cluster, it is dictated by the maximum among the within-cluster variable, i.e. max(x;)) "i
[0] ). See Fig. 1 for an illustration. As no clusters have any advantage over the others, the data not rankable, and we expect the inter-cluster covariates AAB A-C ,,BC to have similar explanations average, but significantly different from each other when we examine local explanations.
We consider both global and grouped-local explanations of the synthetic dataset Figure aa and Figure [ respectively the global explanations, we explain all matches regardless of the cluster membership, while in the grouped- local explanations we only explain matches between items from A against items from B. For more grouped-local explanations on different cluster pairs, we refer to appendix
A.
DC
A
A
B
B
Interpreting the simulation explanations. The bechive plots showcase the recovered PREF-SHAP values, where the bar plots demonstrated the average PREE-SHAP values for each feature. The colour in the bechive plots indicates the magnitude of the difference between the corresponding features of the winner and of the loser in that match. For example, red point in a bechive
An illus- tration of our sim- ulation: each edge corresponds to the variable that dic- tates the comparison based on the colour.
Figure I:
(d) plot for feature indicates that the difference Fwinner Tooser large.
(d)
Fig. iillssraaes the explanation results for the global synthetic experiments. We see that PREF-SHAP identified the within-cluster variable n!!l as the most important, which is consequence of the fact that the largest number of matches are played between the items of the same cluster (cf. Fig. [ where there three blue lines and two lines of each of the other colours). The three inter-cluster
7

---

Table Dataset summary
Dataset
Matches
Niems
Contex
D
comtinsous
Doinary
D
D Colneer
Synthetic 40000
1000
4
3
Chameleon 106
35
Pokémon
60000
800
7
0
Tennis
95359
3483
4114 (tournaments)
4
7
6
which, by symmetry, should be the case. Furthermore, the correct battle mechanism is captured by PREE-SHAP but not UPM, as we see that the large PREF-SHAP values for each feature are red in the beehive plot. This indicates that items with larger value are more likely to win against items with lower value in the corresponding features. In contrast, SHAP for UPM does not recover this insight.
variables contributed similarly according
The explanations for the matches between items from against items from B, are shown in Fig 3 Here, xAB is correctly picked as the relevant feature in these matches with PREF-SHAP, but not with SHAP for UPM. We see again that there is clear tendency that large PREF-SH SHAP values are red for feature x showing that PREF-SHAP once again captures the designed gaming mechanism whicc is not the case in SHAP for UPM. Intuitively, even though SHAP for UPM allows local explanations, it does so based on a global utility, which fails completely in a non-rankable case.
PREF-SHAP
SHAP for UPM
PREF- SHAP
SHAP for UPM
-0.15
1n.A
10.13
+0:15
A
44.
+0.15
-0.88
2 In
15C
+0.1
300
x-C
mrann|Phet-si SHPP vetumn)
10
Meon value
SHAP Impact preference
SHAP value (impact model output)
Figure 2: Bar and Beehive plots for global explanations on the synthetic dataset.
PREF-SHAP
SHAP for UPM
PREF-SHAP
SHAP for UPM
102
+0:15
H0.13
4/0
+0.39
6/6
10.18
I
+0.17
24
SHW values) Inpact on preference
Prot-SHAP values
meant SHAP vatuej]
SHAP value (impact model output)
Figure 3: Bar and Beehive plots for grouped local explanations on the synthetic dataset (Cluster A vs B)).
Real-world explanations For our real-world datasets, we consider publicly available datasets Chameleon, Pokémon and Tennis. We provide descriptive statistics of these datasets in Table 5 and give their brief descriptions below. Appendix contains further large scale experiments on an additional dataset consisting of user-item interactions on a fashion retail website.
The Chameleon dataset 44 considers 106 contests between 35 male dwarf chameleons. Physical traits of the chameleons are measured such as the height of their casque, length of their jaw, body mass etc. According to [4]] they fitted linear Bradley Terry model and examined the coefficients to deduce that casque height (ch.res) and relative area of the flank patch pprop.patch) positively affected the fighting ability the most. The Pokémon dataset considers 60000 Pokémon battles among 800 Pokémon. Pokémon have different characteristics such as attack power, speed, health etc. The Pokémon further has at least one different type such Electric, Water, Fire, etc. Certain types have advantages and disadvantages against each other, for instance, fire Pokémon are weak to water-based attacks (receiving twice the damage) and as a result have a disadvantage against water Pokémon.
The Tennis dataset considers professional tennis matches between 1991 and 2017 in all major tournaments each year. The data is provided publicly by ATP World Tour [5]. Features such as birthyear, weight, height etc are included about each tennis player together with context details the match such as the court being indoor or outdoor and what surface the match is being played on.
The above datasets are not rankable, and we validate this claim by comparing GPM performance against UPM in Table 4 together with the estimated rankability measure SpecR proposed in (2 for each dataset. SpecR measures the similarity of the data to a complete dominance graph (i.e. rankable data). It takes values between 0 and with values close to I being evidence in support of rankability.
8

---

Table 3: GPM vs UPM. Mean and standard deviations of performance averaged over 5 runs.
Synthetic
Chameleon
Pokémon
Tennis
GPM
UPM
GPM
UPM
GPM
UPM
C-GPM 0.58 0.13
UPM
Test AUC SpecR
0.98
0.01
1.02
10.0
0.86
0.8210.00
0.5210.02
0.09
0.24
0.20
For the Tennis data where there are additional relationships with the context (tournaments), we estimate the average SpecR of each tournament. Both the superior performance of GPM over UPM and the low SpecR measures suggest that the datasets are generally not rankable, which points to limitations of explaining preferences via utility- modeling.
Explaining Pokémon battles. We first consider standard dueling data for explaining preferences. We explain the learned preferences and learned differences in utilities on the Pokémon dataset in Figure In this dataset, we have summed the Shapley values for Type features.
PXEF-SHAP
SHAP for UPM
PREF-SHAP
SHAP for UPM
Deferon 12.05
Allach
1ppc
Speial
1/p0
legendar
50
20
Speed
H0:05
+0.05
Sp. Alz
Legendayy
10:04
Legendery 5p //k
117 +0:25
25
Direense
10.54
IP
5p.
(haer AA sahamji
SHAP inpectoo preferance
SHAP value IMsacE modol (UEOL)
Figure 4: Bar and Bechive plots for the Pokémon dataset. PREF-SHAP captures that both speed and type matter, while SHAP for UPM only captures the type importance.
We see that explaining general preferences provides further insight than just explaining the difference in utility functions. In particular, SHAP for UPM does not capture the additional importance of Speed in winning battles. As higher (more red) values of differences in
Water
Fire
+0.2
Fighting
-0.17
speed have positive impact on the outcome, we con- clude that having higher speed than your opponent is advantageous besides a type advantage. This insight is aligned with the "Sweeper" strategy [46], where one would employ a leading Pokémon with very high speed and attack to attempt downing the opponent before they can strike back. In Fig. [5] we PREF-SHAL can also capture the correct type advantage/disadd disadvantages among the Pokémon, but not
speed x wwnnner
spedd I lossrr
Ghost
+0.16
meent]PPat SHOP valuesil Impact proferenic
Ghest
.0.19
Water
+0. 15
Are
+0.14
Righting
+0,
mean[[SHAP value])
SHAP for UPM.
Figure 5: Explaining matches between 4 types of Pokémon, among them only fire and water has type disadvan- tagc/advantage against each other. PREF-SHAP (top) correctly identifies that fire and water are the most important, while water and fire are not deemed most important by SHAP for UPM.
Explaining Chameleon contests. We find that UPM's explana- tions are more aligned with 44'ss findings (prop.path and ch. res are the most important features), which is unsurprising since the Bradlcy Terry model used in [44] is also a utility based model. However, since GPM gives a much better predictive performance than UPM (Test AUC 0.92 v.s. 0.80), we believe PREF-SHAP's explanations are also insightful. In fact, PREF-SHAM discovers that having larger jaw sizes (iLres) than your opponent have significant negative ef- fect on match outcome, a previously undiscovered mechanism from [44]. We verify this finding in Appendix B by applying PREF-SHAP to GPM trained on multiple folds of the Chameleon dataset and consistently find that high values of the jaw size (jl.res) variable have a negative impact on the
important,
outcome.
PREF SHAP
SHAP for UPM
PREF SHAP
SHAP for UPM
antp-38tch
p00 $ims stp mann
props patch
masos
+0.3
SVL
H02
mask
5VL
5vi.
15.2
+2.24
prop mah
SVL
dires
meendiFre 5-15 voluee)
1ET pact ON proference
SHAP VAlUA ORFPOCt modul (ACOLI)
Figure 6: Bar and Beehive plots for the Chameleon dataset
Explaining Tennis matches. We now consider preference learning with context covariates and explain both item characteristics and context covariates in Figure
/ In terms of based inference,
PREF-SHAP finds that being older than your opponent (z
Toser
yob
0 Bluc), physically

---

heavier, and taller than your opponent positively impacts the chances of winning. We also find that debuting earlier as a professional tennis player than your opponent positively impacts your chances of winning. This is not surprising as debuting earlier may be indicative of promising young talent. Across all competitions, there appear to be no significant patterns environment effects.
Tennis players
Environment
Tennis players
Environment
year
37
sitt year ig tdggh
Clay Gorpet
Lom
Onier 20
+8.5
+0.34
Outtoer
In000r
Gress
12.06
+0.00
MC.00 landed iniardedd
QU5603r
Meant St-APvaluss))
trpect prefosric
Inpect preference
Impact preference
Figure 7: Item and context-specific Pref-SHAP values for the Tennis dataset
Explaining Djokovic's losses In plot Figure we locally explain all Novak Djokovic's losses in his professional career. Novak Djokovic is regarded as one the greatest tennis players of all time, understanding his weakness could serve practical demonstration of the utility of PREF-SHAP.
Djokovic
Environment
Djokovic
Environment
#0.35
Hard
Clty
10.15
Hard
Indoor
+0.2
peo hekger 1t-nonddd
Ouidus
+0.13
Qued00r Carpet
Handod
Othur 131
+0.00
Grecs
H0.1
mreat Inghrt peefewece
Iepact on insferencc
Irpect proference
Impact
Pref SHAP values
Figure 8: Local explanations of Djokovic losses
While the results take similar shape to the global explanations, Djokovic remarkably seems to be weaker to players shorter than him, contrary to the general advantage being taller. Besides this, Djokovic seems to be weaker clay courts and when playing indoors.
Conclusion
In this work, proposed PREF-SHAP to explain preference learning for pairwise comparison data. We proposed the appropriate value function for preference explanations and demonstrated the pathologies of the naive concatenation approach in Appendix [ Experiments demonstrated that PREF-SHAP recovers richer explanations than utility-based approaches, showcasing the ability of PREF-SHAP in interpreting the mechanism of preference elicitation.
Acknowledgments
The authors sincerely thank Oscar Clivio for the helpful comments and feedback. SLC is supported by the EPSRC and MRC through the Ox WaSP CDT programme EP/L016710// DS supported in part by Tencent AI Lab and in part by the Alan Turing Institute (EP/N510129/1),
10

---

References
[1] Johannes Fürnkranz and Byke Hüllermeier. Pairwise preference learning and ranking. In Nada Lavrač, Dragan Gamberger, Hendrik Blockeel, and Ljupčo Todorovski, editors, Machine Learning: ECML 2003, pages 145-156, Berlin, Heidelberg, 2003. Springer Berlin Heidelberg. ISBN 978-3-540-39857-88
[2] Ralph Allan Bradley and Milton E. Terry. Rank analysis of incomplete block designs: The method of paired comparisons. Biometrika, 39(3/4):324 345, 1952.
[3] Louis Thurstone. law of comparative judgment. Psychological review, 101(2):26,, 1994.
[4] Wei Chu and Zoubin Ghahramani. Preference learning with Gaussian processes. In Proceedings of the 22nd International Conference on Machine Learning, pages 137-144, 2005.
[5] Javier González, Zhenwen Dai, Andreas Damianou, and Neil D Lawrence. Preferential Bayesian optimization. In Proceedings of the 34th International Conference on Machine Learning, pages 1282-1291, 2017.
[6] Manuela Cattelan, Cristiano Varin, and David Firth. Dynamic bradley-terry modelling of sports tournaments. Journal of the Royal Statistical Society: Series C (Applied Statistics), 62(1): 135-150, 2013.
[7] Siu Lun Chau, Mihai Cucuringu, and Dino Sejdinovic. Spectral ranking with covariates. arXiv preprint arXiv:2005.00 04035, 2020.
[8] Daniel Kahneman and Amos Tversky. On the interpretation of intuitive probability: reply to Jonathan Cohen. Cognition, 7(4):409-41) 411, 1979.
[9] Neil Houlsby, Ferenc Huszar, Zoubin Ghahramani, and Jose M HemándezzLobato. Collabora- tive gaussian processes for preference learning. In Advances in neural information processing systems, pages 1096-21044 2012.
[10] Stefanos Bennett, Mihai Cucuringu, and Gesine Reinert. Lead-lag detection and network clustering for multivariate time series with an application to the us equity market. arXiv preprint arXiv:2201.08283 2022.
[11] Devi M Stuart-Fox, David Firth, Adnan Moussalli, and Martin J Whiting. Multiple signals in chameleon contests: designing and analysing animal contests as tournament. Animal Behaviour, 71(6):1263-1271) 2006.
[12] Siu Lun Chau, Javier Gonzalez, and Dino Sejdinovic. RKHS-SHAP: Shapley values for kernel
methods. arXiv preprint 10.iv:2/ 2021.
[13] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. "why should trust you?": Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13-17, 2016, pages 1135 144, 2016.
[14] Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In Advances in neural information processing systems, pages 4765-4774, 2017.
[15] Tapio Pahikkala, Willem Waegeman, Evgeni Tsivtsivadze, Tapio Salakoski, and Bernard De Baets. Learning intransitive reciprocal relations with kernel methods. European Journal of Operational Research, 206(3):676-685, 2010.
[16] Siu Lun Chau, Javier Gonzalez, and Dino Sejdinovic. Learning inconsistent preferences with gaussian processes. International Conference on Artificial Intelligence and Statistics, 2022.
[17] Christian List. Social Choice Theory. In Edward N. Zalta, editor, The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, Spring 2022 edition, 2022.
[18] William V Gehrlein. Condorcet's paradox. Theory and Decision, 15(2):161-197, 1983.
11

---

[19] Rosy Tsopra, Yean-Baptise Lamy, and Karima Sedki. Using preference learning for detecting inconsistencies in clinical practice guidelines: Methods and application to antibiotherapy. Artificial Intelligence in Medicine, 89, 04 2.18. doi: 10.1016/j.artmed.2200 201 8.04.013.
[20] Yifan Feng, René Caldentey, and Christopher Ryan. Robust learning of consumer preferences. Operations Research, 12 2021. doi: 10. 107.28/0
[21] Lloyd S Shapley. A value for n-person games. Contributions to the Theory of Games, 2(28): 307-317, 1953.
[22] Code for Pref-SHAR https://iiitbb.comm//HHfff//fF
[23] Paul Anderson, Timothy Chartier, and Amy Langville. The rankability of data. SIAM Journal on Mathematics of Data Science, 1(1):121-443, 2019.
[24] Thomas R Cameron, Amy N Langville, and Heather C Smith. On the graph laplacian and the rankability of data. Linear Algebra and its Applications, 588:81-100, 2020.
[25] Vern I Paulsen and Mrinal Raghupathi. An introduction the theory of reproducing kernel Hilbert spaces, volume 152. Cambridge university press, 2016.
[26] Bharath K Sriperumbudur, Kenji Fukumizu, and Gert RG Lanckriet. Universality, characteristic kernels and RKHS embedding of measures. Journal of Machine Learning Research, 12(Jul): 2389-2410, 2011.
[27] Paul Anand. Are the preference axioms really rational? Theory and Decision, 2392):189-214, Sep 1987. ISSN 1573-7187. doi: 10. 00.1007/BF00126305. URL httpt:t////ii. or.p//..111100000. 10. 1007/ BF00126305
[28] Robert J. Aumann. Utility theory without the completeness axiom: A correction. Econometrica. 32(1/2):210211 212, 1964. ISSN 00129682, 14680262. URL httptt//:///w... http::/www..oooooooo//oooooooooe stable/
1913746]
[29] Erik Štrumbelj and Igor Kononenko. Explaining prediction models and individual predictions with feature contributions. Knowledge and information systems, 41(3):647-665, 2014.
[30] Ian Covert, Scott Lundberg, and Su-In Lee. Explaining by removing: A unified framework for model explanation. Journal of Machine Learning Research, 22(209):1-90, 2021.
[31] Hugh Chen, Joseph D Janizek, Scott Lundberg, and Su-In Lee. True the model true to the data? arXiv preprint arXiv:2006.16234, 2020.
[32] Dominik Janzing, Lenon Minorics, and Patrick Blöbaum. Feature relevance quantification in explainable ai: A causal problem. In International Conference Artificial Intelligence and Statistics, pages 2907-2916 2020.
[33] Christopher Frye, Damien de Mijolla, Laurence Cowton, Megan Stanley, and Ilya Feige. Shapley-based explainability on the data manifold. arXiv preprint arXiv:2006.01272, 2020.
[34] Sahra Ghalebikesabi, Lucile Ter-Minassian, Karla DiazOrdaz, and Chris C Holmes. On locality of local explanation models. Advances in Neural Information Processing Systems, 34, 2021.
[35] Christopher Frye, Ilya Feige, and Colin Rowat. Asymmetric shapley values: incorporating causal knowledge into model-agnostic explainability. arXiv preprint arXiv:1910.06358, 2019.
[36] Tom Heskes, Evi Sijben, Ioan Gabriel Bucur, and Tom Claassen. Causal shapley values: Exploiting causal knowledge to explain individual predictions of complex models. Advances in neural information processing systems, 33:4778-477 2020.
[37] Alexandre Duval and Fragkiskos D Malliaros. Graphsvx: Shapley value explanations for graph neural networks. In Joint European Conference on Machine Learning and Knowledge Discovery Databases, pages 302-318. Springer, 2021.
[38] Scott M Lundberg, Gabriel G Erion, and Su-In Lee. Consistent individualized feature attribution for tree ensembles. arXiv preprint arXiv:1802.03e6 2018.
12

---

[39] Chih-Kuai Kuan Yeh, Kuan- Yun Lee, Frederick Liu, and Pradeep Ravikumar. Threading the needle of on and off off-manifold value functions for shapley explanations. arXiv preprint drXiv:2202.11919 2022.
[40] Krikamol Muandet, Kenji Fukumizu, Bharath Sriperumbudur, and Bernhard Schölkopf. Kernel mean embedding of distributions: review and beyond. arXiv preprint arXiv: 1.X.::660.00552.
2016.
[41] Petros Drineas and Michael W. Mahoney. On the nystrom method for approximating gram matrix for improved kernel-based learning. Journal of Machine Learning Research, 6(72): 2153-2175, 2005. URL http://jmlr... www.:oowrp/p..eaeeeee.o http:///ml..oog/pppper/////////
dr drineao05a... html
[42] Giacomo Meanti, Luigi Carratino, Lorenzo Rosasco, and Alessandro Rudi. Kernel methods through the roof: handling billions of points efficiently. ArXiv, 0at/22006.13550 2020.
[43] Giacomo Meanti, Luigi Carratino, Ernesto De Vito, and Lorenzo Rosasco. Efficient hyperpa- rameter tuning for large scale kernel ridge regression, 2022. httpps//ss////xii..... orggzii.i.rrrr/ooooooooo abs/
2201. 06314
[44] Devi Stuart-Fox, David Firth, Adnan Moussalli, and Martin Whiting. Multiple signals in chamelcon contests: Designing and analysing animal contests as a tournament. Animal Be- haviour, 71:1263-1271 06 2006. doi: 10.1016j.abbehxx2255..7000
[45] Tennis dataset. 2022.
[46] Sweeper Strategy Description.
httrriiiivv_aaattinnee Loapiannnhhee_oo_____________ "Job_ Shob_System"
13

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes]
3. If you experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either the supplemental material or as URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits. hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (e.g.. with respect the random seed after running experi- ments multiple times)? [N/A]
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, or cloud provider)? See Appendix A Yes]]
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you're rou're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
14


Reviews:
[{"review_id": "nz6jH_PYjU", "reviewer": "Reviewer_TiH3", "summary": "In the setting of explainable artificial intelligence, this paper investigates the challenging problem of explaining predictions made by preference models without strong rankability assumptions. To this point, one cannot exploit standard explainability tools for utility functions. So, this study instead considers the recent (possibly contextured) Generalized Preference Model proposed by Chau et al. (2022) that exploits kernels for capturing the likelihood function on pairs of items. By coupling this model with the paradigm of Shapley values, the authors derive a preferential value function for dueling items. Based on the existence of Rietz representation of the corresponding functional, this value function admits an elegant closed-form that can be computed efficiently. Experiments performed on a wide range of preference tasks corroborate the benefits of this Pref-SHAP approach, especially in comparison with a naive application of SHAP in pairwise preference explanation.  \n", "questions": "No real questions, but see the comment above about the runtime complexity.\n", "limitations": "The Generalized Preference Model examined in this paper for inferring explanations makes no assumption about item rankability and takes only a few assumptions about the non-parametric function $g$ used in the model and its kernelization. The authors go even further by inferring explanations in the contextual setting. So, I did not find any real limitations in this framework.\n", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Overall the paper is very well-written and well-motivated. As many technical tools are introduced for deriving, in an efficient way, relevant explanations for general preference models, the paper is very dense. But all concepts are introduced parsimoniously, with a pedagogical effort made for explaining difficult parts. As far as I could check, the technical results look sound. Finally, the experiments performed on various tasks clearly highlight the practical utility of Pref-SHAP. In a nutshell, this is a good paper that shall pave the way for future research on explaining predictive preferences.\n\nI found no real weaknesses in this paper. Just one point: it would be informative for the reader to give some bounds on the runtime complexity for estimating the value functions derived in Propositions 3.2 & 3.3.  \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "aj96iePqNlt", "reviewer": "Reviewer_wsLb", "summary": "This paper proposes a method to explain attributions of inputs in a preference model. Specifically, the authors apply Shapley values to the preference model. To this end, the authors design the utility function for Shapley values on generalized preference models based on kernel functions. The proposed method can estimate attributions of both inputs and context variables.", "questions": "- I am confused about the claim that computing two Shapley values for the same feature in $x^{(l)}$ and $x^{(r)}$ leads to inconsistency. Although $x^{(l)}$ and $x^{(r)}$ consist of the same features, their feature values are different. Therefore, each feature value can be assigned a Shapley value. A simple example is that the Shapley value of $x_i^{(l)}=-x_i^{(r)}$.\n- Some references for tables are incorrect. “Table 5” in Line289 should be “Table 2”, and “Table 4” in Line 308 should be “Table 3”.\n", "limitations": "The authors do not address the limitations of their work. ", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "[Strengths]\n+ The paper is well-motivated and well-structured.\n+ The authors do not directly apply Shapley values to preference models, but consider properties of preference models and propose an efficient way to compute Shapley values.\n\n\n[Weaknesses]\n- Although the proposed Pref-SHAP uses kernel functions to compute the value function $v$, its essence is actually the same as SHAP. The only difference is the implementation/computation of the value function, i.e., the model output when only given a subset of input variables. Some benefits of Pref-SHAP over SHAP on UPM actually come from advantages of GPM, rather than the explanation method.\n- What is the advantage of computing $v$ by using kernel functions over directly computing the expected probability of the model? If using kernel functions is more efficient, then it would be better to compare the computational cost of the two implementations.\n- In the third row of Table 2 for the candidate $x^{(l)}$, why not only modify $x^{(l)}$ in the computation of Shapley values without changing $x^{(r)}$?\n- The comparison of local explanations in Figure 3 seems unfair. I think that one_hot$(c_i)$ should remain unchanged when computing $v(S)$ on GPM and UPM. In this way, the clusters of pairs are constant and explanations are restricted in the specified condition.\n- For experiments on realistic datasets, how to judge which explanations are correct? For example, in Figure 6, both Pref-SHAP and SHAP for UPM show a chaotic pattern in beehive plots. Besides, differences in results between Pref-SHAP and SHAP for UPM may not stem from the explanation methods, but come from the difference between models, because UPM cannot perform well for unrankable tasks and may lead to strange explanations. It would be better if the authors compare the proposed method with other explanation methods, e.g., Integrate Gradient, on the same model.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "FoTx82LVwh-", "reviewer": "Reviewer_jJ4D", "summary": "The paper seeks to `explain' the outcomes of binary contests (whether between two tennis players or elements of a consumer's choice set).  By imposing restrictions on the class of preference orderings considered, the paper is able to derive a ``closed form expression of the value function'' underlying the Shapley value, reducing the computational cost of determining the Shapley value. ", "questions": "1. I would like to know what the generalization from UPM to GPM allows, in terms of utility theory.  I'm not familiar with the SOTA in this area, but wonder, for example, how GPM relates to e.g. Deaton and Muellbauer's `almost ideal demand system'.\n\n1. line 193's ``inconsistent explanations'' makes me wonder what is being assumed about cross-partial elasticities (in the language of economic theory): is it assumed that features do not interact (i.e. neither substitutes nor complements)?  If so, that is a very strong assumption. ", "limitations": "No concerns.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Originality**\n\nThe paper seems original to me, although in a somewhat complex way:\n- a standard argument for Shapley value's use is that it is model agnostic.  From this point of view, knowing the underlying model defeats some of its motivation.\n- utility functions (cardinal, ordinal, stochastic), preference orderings and the relationships between them have been studied for decades within the economics literature, as have (to a lesser extent) `contest success functions'.  The paper does not give indications of understanding that, perhaps making it seem more original than it is.\n\n**Quality**\n\nNo concerns.\n\n**Clarity**\n\nWell presented.\n- line 40: it could make sense to explain what the authors have in mind by ``to explain''.\n- is the \"unrankable\" problem the same one that led to the use of stochastic utility in economic theory (q.v. work by e.g. Pattanaik, Barbera)?\n- line 91: the standard approach in economic theory is to see a `good' as a complete description, thus including what are called `context variables' here.  It would be useful if the authors commented on why they do _not_ follow that approach.\n\n**Significance**\n\nTheoretically, I feel that the paper's detachment from the extensive microeconomic theory literature limits it.  (This said, I have not seen the question of applying Shapley values applied to consumer models.)\n\nThus, I needed to be convinced by the Experiments.  While I understand that they are `toy examples', it may still be that they trivialize the material (e.g. who cares which Pokeman wins a match, or Djokovic's performance on clay?).  Personally, I found myself wanting to know e.g. what this can say about inter-state wars (e.g. using https://correlatesofwar.org/data-sets/COW-war/dyadic-inter-state-war-dataset-1).  This both seems orders of magnitude more important, and might inform a topical issue, Russia's invasion of Ukraine.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "459MSw9TPpS", "reviewer": "Reviewer_eoAJ", "summary": "The paper adapts the SHAP value estimation to the object-ranking task of the preference learning setting. Thereby, the authors present a novel way of computing SHAP values which the authors call PREF-SHAP. The authors present and evaluate the efficacy of PREF-SHAP with one synthetic dataset and three (plus one in the appendix) real-world ranking datasets. Moreover, the authors present their PREF-SHAP methodology formally by presenting a value function for the object ranking task of preference learning. The authors formally prove their methodology under the assumption that the ranking model under investigation is a kernel-based Generalised Preference Model. The proofs can be found in the appendix. Moreover, the authors publish their implementation online for open access.", "questions": "# Questions\n- You frame your Pref-SHAP method to be the solution for all Preference learning/ranking machine learning models. However, you primarily apply RKHS-SHAP (or any SHAP estimation approach) to the object-ranking task on your mentioned kernel-based Generalised Preference Models. \n\t- Can you outline in what way your method is applicable to other types of preference models ($g(.)$) or tasks? \n\t- Why did you focus solely on kernel-based Generalised Preference Models? The sentence \"While the ... \" (lines 196 to 199) is not very informative for this big reduction of the problem. \n- Is it possible to calculate and add the ground-truth Shapley values for your synthetic dataset and a \"perfect model\"?\n\n# Suggestions\n- Specify your method's main limitations and future work potential.\n- To save some space, I would move some of the Pref-SHAP plots of section 4 into the appendix, as they are quite repetitive and take up a lot of space that could be better used for a thorough discussion of the limitations of your work or on a different kind of evaluation. I would keep the Pokemon and/or Tennis examples. These are great.\n- There exists a paper from econometrics that first made the connection of calculating Shapley Values via the least-squares formulation. (Also mentioned by Lundberg and Lee in their SHAP paper). In line 143 of your manuscript, the proper citation could be added: Charnes, A., Golany, B., Keane, M., & Rousseau, J. (1987). Extremal Principle Solutions of Games in Characteristic Function Form: Core, Chebychev and Shapley Value Generalizations. In J. P. Ancot, A. J. H. Hallett, J. K. Sengupta, & G. K. Kadekodi (Eds.), Advanced Studies in Theoretical and Applied Econometrics. Econometrics of Planning and Efficiency (Vol. 11, pp. 123–133). Springer Netherlands. https://doi.org/10.1007/978-94-009-3677-5_7\n- I suggest, adding a different focus to your evaluations: How do users interpret your explanations? How does RKHS-SHAP compare against regular KernelSHAP on the same setting (value function)", "limitations": "# Limitations\n- The authors do not specify the limitations of their work. \n- There is no discussion about the ethical implications of this work apart from the motivation. As I do not see big ethical concerns with such XAI works (rather the opposite) I don't think that this is a problem.\n\n- The main limitation of the work is the more narrow application domain of object-ranking using a specific model type (kernel-based Generalised Preference Models). This is not clearly specified in the beginning.\n\n- The evaluation of the method is done in a way that is clearly favoring the newly proposed method and no comparison is done to existing approximations (mainly KernelSHAP) in the same value-function setting (see the point in the \"Quality\" section). \n\n- The proposed Pref-SHAP method is evaluated on a functional level without conducting any human experiments. For this work, a human experiment is not essential, as the work is rather foundational. However, as it is one of the first XAI + Preference Learning works, it is very interesting to see how people would interpret the output of Pref-SHAP, as it is unintuitive to do so even for a person who is quite familiar with SHAP.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Originality\nThe proposed Pref-SHAP is a natural extension of the already existing SHAP explanation framework. Pref-SHAP mainly builds on already existing work of Shapley Value approximation. The presented work relies heavily on a newly introduced RKHS-SHAP approximation method, which in itself is very new and, as of now, is still under review. Moreover, the work focuses heavily on a newly proposed model type of kernel-based Generalised Preference Models and the theoretical results are solely assuming such a model. Hence, the originality of the proposed work is rather incremental. (Application of SHAP to a new problem domain and reduction of problem space to analyze the solution formally) \n\nThat said, the presented work is to the best of my knowledge the first work linking a subset of preference learning and Shapley values explanations.\n\n# Quality\nIn general, the quality of the work is high. The method is presented formally. The conducted empirical datasets are well selected and illustrate how SHAP on ranking functions can be used. As there exists no SOTA and, thus, it is sufficient to compare against a sensible baseline. The theoretical analysis is straightforward and interesting when the model function is kernel-based.\n\nI understand the main contribution of the proposed work to be the value function in Definition 3.1 and that you show the results of evaluating the value function on a specific model instantiation (kernel-based Generalised Preference Models). The value function follows quite naturally from applying SHAP to the ranking domain and is presented very well. However, the authors do not give a valid motivation for why they chose to only focus on these specific kernel-based model functions. The only mention of this is the sentence: \"While the ... \" (lines 196 to 199), which does not include proper reasoning. Propositions 3.1 to 3.3 follow quite naturally from instantiating the model function $g(.)$ with a kernel-based GPM. Still, the theoretical results are interesting. Especially by framing the SHAP approximation problem as a kernel and utilizing RKHS (one of the main contribution of RKHS-SHAP, as I understand the paper).\n\nI see a problem with the evaluation of PREF-SHAP and the chosen baseline. Of course, it is interesting to see what would happen if you were to concatenate the features of two instances and apply KernelSHAP rather naively. However, I argue that it is pretty obvious that KernelSHAP would not really work there, because it assumes feature independence, and the features are the same, and, thus, are dependent. A better comparison would be to group the same features of two instances together and remove both features simultaneously. Of course sampling from the conditional distribution is more problematic (solved by RKHS-SHAP, I figure) but in the removing-with-marginal case (after Covert et al. 2021, [28] in your work) this would be no problem and straightforward. I would be very interested, in why you chose to use these kernel-based GPMs and would be more interested in a thorough investigation of common Shapley value approximation techniques for different models and model types with your new value function (Definition 3.1). \n\n# Clarity\nIn general, I enjoy the presentation of the work.\n- I like the motivation and introduction giving a good overview into the field of preference learning for scholars not familiar with it. The same is true for the Shapley section. However, I suggest being more specific about what part of preference learning you are addressing (mainly object-ranking).\n- I like the discussion of choosing marginal vs. conditional distribution for \"feature removal\".\n- In particular, I like Table 1: This gives a good overview of the aforementioned preference learning / SHAP description\n- Figure 1 is a great and small illustration of how the synthetic dataset was created and for what purpose it was implemented.\n- In general, the selection of empirical test datasets suits the work greatly. The Pokémon dataset is clear to understand. The Tennis matches are a good example of a \"real\" real-world dataset and the focus on Djokovic's losses is a humorous and interesting illustration of your PREF-SHAP method\n- The writing of the paper is good and it flows well. (Some hiccups along the way and the Propositions are a bit under-discussed)\n\n### Minor Clarity Remarks:\n- Coming from a Feature Importance background, I had problems with the formal notation of denoting the item index in the superscript in braces (l and r: $\\boldsymbol{x}^{(l)}, \\boldsymbol{x}^{(r)}$) as this representation is sometimes used to denote the feature column of the input space. Make this clear again in your paragraph introducing your notation.\n- All Propositions (3.1 to 3.3) are hard to understand at first (though easier when following the proofs in the appendix). I suggest adding a better natural text description to the formulation to make it clearer.\n- The contribution statement should be more specific and classify the main contribution more precisely (focus on kernel-based Generalised Preference Models)\n\n- Minor Remarks: \n\t- Appendix A: I don't think that you wanted to say that your algorithm is \"embarrassingly\" parallel (probably you meant \"embracing\").\n\t- Proposition 3.3 (Line 223-224) is overflowing horizontally. (the same is true for some expressions in the appendix ... there this is no problem)\n\t- Sentence \"An explicit...\" (line 14-17) is very hard to read. The i.e. part through me off. Split this maybe in two.\n\n# Significance\nThe significance of the work is high. Preference learning is a big part of machine learning research. Moreover, explainability and trustworthiness are also extremely important. Explainability is especially important for ranking problems, as search engines or recommender systems influence people on a daily basis. Being able to explain such systems is an important research question.\n\nTo the best of my knowledge, there exists no similar work of Shapley Value + any Preference Learning. As such, an application of the SHAP method to Preference Learning is a significant contribution to the community.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept