On the Impact of the Utility in Semivalue-based Data Valuation
Authors: Mélissa Tamine
Benjamin Heymann
Maxime Vono
Patrick Loiseau
Venue: ICLR 2026 Poster (2026)
Keywords: Data valuation
Semivalue
Utility
Robustness
Primary area: interpretability and explainable AI
TL;DR: Building on a geometric perspective (the spatial signature), we propose a methodology centered on an explicit robustness metric that informs practitioners whether and by how much their data valuation results will shift as the utility changes.

Abstract: Semivalue–based data valuation uses cooperative‐game theory intuitions to assign each data point a value reflecting its contribution to a downstream task. Still, those values depend on the practitioner’s choice of utility, raising the question: *How robust is semivalue-based data valuation to changes in the utility?* This issue is critical when the utility is set as a trade‐off between several criteria and when practitioners must select among multiple equally valid utilities. We address this by introducing the notion of a dataset’s *spatial signature*: given a semivalue, we embed each data point into a lower-dimensional space in which any utility becomes a linear functional, making the data valuation framework amenable to a simpler geometric picture. Building on this, we propose a practical methodology centered on an explicit robustness metric that informs practitioners whether and by how much their data valuation results will shift as the utility changes. We validate this approach across diverse datasets and semivalues, demonstrating strong agreement with rank‐correlation analyses and offering analytical insight into how choosing a semivalue can amplify or diminish robustness.

Published as a conference paper at ICLR 2026
ON THE IMPACT OF THE UTILITY IN SEMIVALUE-BASED DATA VALUATION
Mélissa Tamine
Benjamin Heymann
Criteo AI lab, FairPlay joint team, France CREST, ENSAE, Institut Polytechnique de Paris com
Critco Al lab, FairPlay joint team, France b wmy.nmmnncc teo.
m. we.tamnnneeee nee
Maxime Vono
Patrick Loiseau
Criteo AI lab, FairPlay joint team, France m. wwnoonnoeeooooooo er www.errrit.....oo.. com
Inria, FairPlay joint team, France patrick. loiseau@inria fr
ABSTRACT
Semivalue based data valuation uses cooperative- theory intuitions to assign each data point a value reflecting its contribution to downstream task. Still, those values depend robust is semivalue-bur -ased data valuation to changes in the utility? This issue critical when the utility is set as a trade-off between several criteria and when practitioners must select among multiple equally valid utilities. We address this by introducing the notion of dataset's spatial signature: given semivalue, we embed each data point into a diver-dimensional space in which any utility becomes linear functional, making the data valuation framework amenable geometric picture. Building on this, we propose a practical methodology centered an explicit robustness metric that informs practitioners whether and by how much their data valuation results will shift as the utility changes. We validate this approach across diverse datasets and semivalues, demonstrating strong agreement with rank-corres correlatooon analyses and offering analytical insight into how choosing a semivalue can amplify or diminish robustness.
the practitioner's choice of utility, raising the question: How
simpler
INTRODUCTION
Supervised machine learning (ML) relies on data, but real-world datasets often suffer from noise and biases as they are collected from multiple sources and are subject measurement and annotation errors (Northcutt et al., 2021). Such variability can impact learning outcomes, highlighting the need for systematic methods evaluate data quality. In response, data valuation has emerged as growing research field that aims to quantify individual data points' contribution to downstream ML task, helping identify informative samples and mitigate the impact of low-quality data. popular approach to tackling the data valuation problem is to adopt a cooperative game-theoretic perspective, where each data point is modeled as player in a coalitional game, and the usefulness of any data subset is measured by utility function. This approach leverages game theory solution concepts called semivalues (Dubey et al., 1981), which input data and utility to assign an importance score to each data point, thereby inducing ranking of points in the order of their contribution to the ML task (Ghorbani & Zou, 2019; Kwon & Zou, 2022; Wang & Jia, 2023; Jia et al., 2019; Wang et al., 2024a).
Motivation. When computing semivalues, the utility typically selected by the practitioner to reflect the downstream task. In some contexts, this choice is obvious. For example, when fine-tuning a large language model (L.LM), one might balance two competing objectives: helpfulness (how well the model follows user instructions) and harmlessness (its tendency to refuse or safely complete unsafe requests) (Bai et al., 2022a;b). If the practitioner then asks, "Which training examples most contributed to my desired helpfulness haallnsss-harmlessness balance?", the only sensible utility for semivalue- based data valuation is this composite trade-off itself. By contrast, in more open-ended tasks, the
"Corresponding author.

---

Published as a conference paper at ICLR 2026
utility can be genuinely ambiguous. Imagine training a dog cat image classifier and asking, "Which data points contributed most to overall performance?" Then, accuracy, precision, recall, FI, AUROC, balanced accuracy, and many others are all defensible choices. However, none is uniquely dictated by the task.
These two example settings respectively motivate two general scenarios: (1) the utility trade-off scenario, where the utility is a convex combination of fixed criteria, with a tunable weight v, and (2) the multiple-valid-uuee metrics, none of which being uniquely dictated by the task. In both cases, we argue that data valuation practitioners are well advised to ask themselves:
scenario, where the utility must be chosen among several equally defensible
How robust are my data valuation results to the utility choice?
In what follows, we explain why.
Utility trade-off scenario: Anticipating costly re-training. In scenarios where the utility itself is a trade-off, e.g., when fine-tuning an LLM by combining helpfulness and harmlessness into a single objective parameterized by v, practitioners often rely on data values to identify the k most valuable training examples and train next models on this smaller subset to reduce computational cost as is common practice when using data valuation (Ghorbani & Zou, 2019; Jiang et al., 2023). However, if the top-k set shifts dramatically with changes in v, as priorities between harmlessness and helpfulness evolve, practitioners risk repeated, costly re-training. Quantifying robustness to utility choice makes this risk explicit, alerting practitioners up front to whether data valuation is safe, one-time investment or whether they must plan for ongoing computational overhead as their utility rrfeeoof evolves.
Multiple-valid-utitty scenario: Detecting when data valuation fails as a heuristic. In many real-world tasks, like the earlier dog vs. cat classifier example, practitioners must select utility from several valid options, none uniquely dictated by the problem. Now, one would "morally" expect their induced orderings of points to be consistent. After all, each utility is valid measure for the same task. It may be hard to think that a data point deemed highly important under accuracy would suddenly vanish from the top tier under Fl-score, or vice versa. In practice, however, we observe precisely such discrepancies, depending on both the training dataset and the semivalue. We compute data values under both accuracy and Fi-score on several public datasets, using three popular semivalues: Shapley (Ghorbani & Zou, 2019), (4, 1) 1)-Beta Shapley (Kwon & Zou, 2022). and Banzhaf (Wang & Jia, 2023) (see Appendix A. for experimental details). Table 1 reports the Kendall rank correlation between the two score sets for each combination of dataset and semivalue. Low correlations reveal cases where rankings change substantially depending on whether accuracy or F1-score is used as the utility ! And because utility is inherently better, practitioner has no principled way to choose between the data values ranking produced under accuracy versus Fl-score (or any other valid utility). In such settings, arbitrary utility choices can drive the ordering of data points in entirely different directions: the context (dataset + semivalue) is therefore not "lataavaluatiinaalee, and semivalue-bart valuation fails as a reliable heuristic. By contrast, if rankings remain consistent across all valid utilities for the task, data values truly capture the underlying importance ordering of points. Therefore, knowing how robust the scores ranking is the utility choice enables a practitioner to determine data valuation can be trusted as meaningful heuristic in that context, or whether is too sensitive to utility to provide reliable guidance.
data
whether semivalue-baste
In this paper, we propose methodology that enables data valuation practitioners to assess how robust their semivalue-basse data valuation results are to the utility choice, in both scenarios. We summarize our main contributions as follows.
1. Unified geometric modeling of the two scenarios. We observe that the same geometric representation can capture both scenarios. In this representation, we can, given a semivalue, embed each training data point into lower-dimensional space (we call the set of embedded points the dataset's spatial signature) where any utility becomes a linear functional, making the data valuation framework amenable to a simpler geometric interpretation.
2. A robustness metric derived from the geometric representation. Building on the notion of spatial signature, we introduce metric that practitioners can compute to quantify how
We extend these experiments to additional classification utilities and rank correlation metrics for complete- ness (see Appendices A.2 and A.3), and observe variability as well.
2

---

Published as a conference paper at ICLR 2026
Table 1: Mean Kendall rank correlation (# standard error) between data values computed with accuracy versus Fi-score. For each semivalue and dataset, we approximate data values 5 times via Monte Carlo sampling. Standard errors reflect the variability across these trials.
Dataset
Semivalue
Shapley
(4. 1) Beta Shapley
Banzhal
BREAST
0.95 (0.003)
0.95 (0.003)
0.97 (0.008)
TITANIC
-0.19 (0.007)
-0.17 (0.01)
0.94 (0.003)
CREDIT
-0.47 (0.01)
-0.44 (0.02)
0.87 (0.01)
HEART
0.64 (0.006)
0.68 (0.004)
0.96 (0. 003)
WIND
(0.008)
(0.008)
0.99 (0.002)
CPU
0.59 (0.02)
0.62 (0.02)
0.86 (0.007)
2DPLANES
0.38 (0.01)
0.44 (0.01)
(0.03)
POL
0.67 (0.02)
0.77 (0.01)
0.40 (0.04)
robust the data values' orderings are to the utility choice, providing a practical methodology for assessing the robustness of data valuation results.
3. Empirical evaluation of robustness and insights. We compute the robustness metric across multiple public datasets and semivalues and find results consistent with our rank-correlation experiments: contexts with low rank correlation exhibit a low robustness score, and vice versa. Moreover, we observe that Banzhaf consistently achieves higher robustness scores than other semivalues, phenomenon for which we provide analytical insights.
Related works. Our focus on the robustness semivalue-based data valuation to the utility choice differs from most prior work, which has concentrated on defining and efficiently computing data valuation scores. The Shapley value (Shapley, 1953; Ghorbani & Zou, 2019), in particular, has been widely studied as data valuation method because it uniquely satisfies four key axioms: linearity, dummy player, symmetry, and efficiency. Alternative approaches have emerged by relaxing some of these axioms. Relaxing efficiency gives rise to the semivalue family (Dubey al., 1981), which encompasses
(Koh & Liang, 2017), Beta Shapley (Kwon & Zou, 2022), and Data Banzhaf (Wang & Jia, 2023), while relaxing linearity leads the Least Core (Yan & Procaccia, 2021). Extensions such as Distributional Shapley (Ghorbani et al., 2020; Kwon et al., 2021) further adapt the framework to handle underlying data distributions instead of fixed dataset. On the algorithmic front, exact semivalue computation is often intractable, as each semivalue requires training models over all possible data subsets, whose number grows exponentially with the dataset size. Consequently, rich literature on approximation methods has emerged to make data valuation practical at scale (Mann & Shapley, 1960; Maleki, 2015; Jia et al., 2019; Ghorbani & Zou, 2019; Wang et al., 2024a; Garrido Lucero et al., 2024). By contrast, when and why data valuation scores remain consistent across different utilities has received far less attention. Prior work has explored related robustmess questions in special cases: Wang & Jia (2023) examines how semivalue- when the utility function is corrupted by inherent randomness in the learning algorithm, and Wang al. (2024b) studies how different choices of utility affect the reliability of Data Shapley for data- subset selection. Our proposed methodology broadens this scope by quantifying when data valuation remains robust shifts in the utility function. Specifically, we independently observe the same sensitivity to utility specification as in Dichl & Wilson (2025), which argues that manivalue-based data valuation can be arbitrary and gameable under utility underspecification. While this work focuses on exposing this vulnerability and its implications, we develop a practical geometric framework that quantifies the sensitivity itself and measures impact on ranking stability.
valuations fluctuate
Notations. We set N N (0). For n N we denote [n] := (1, n... For dataset D. we denote as 2nd its powerset, i.e., the set of all possible subsets of D. including the empty set and D itself. For d E N we denote X' C Id and y C RR as an input space and an output space, respectively.
BACKGROUND
The data valuation problem involves a dataset of interest D {21 =(x,, v1) (e)ee where for any i [n) each s; X' is feature vector and y: ) is the corresponding label. Data valuation aims to
3

---

Published as a conference paper at ICLR 2026
assign a scalar score to each data point in D, quantifying its contribution downstream ML task. These scores will be referred to as data values.
Utility functions. Most data valuation methods rely on utility functions to compute data values. utility is set function 2 - IR that maps any subset S of the training set D score indicating its usefulness for performing the considered task. Formally, this can be expressed as u(S) perf (er(()())) where A learning algorithm that takes subset S input and returns a trained model, and perf is a metric used to evaluate the model's ability to perform the task on hold-out test set. For convenience. we interchangeably refer the utility u and the performance metric perf as inherently depends on perf.
Semivalues. The most popular data valuation methods assign value score to each data point in 'D using solution concepts from cooperative game theory, known as semivalues (Dubey et al., 1981). The collection of methods that fall under this category is referred to semivalue- data valuation. They rely on the notion of marginal contribution. Formally, for any i, [n], let D) denote the set of all subsets of 'D of size - 1 that exclude Then, the marginal contribution of 2; with respect to samples of size 1 is defined as
(,(
Σ SCD)
(()
The marginal contribution 4y(,;i u) considers all possible subsets S E : with the same cardinality j - and measures the average changes u when datum of interest 2; is removed from S U {2;}.
Each semivalue-based method is characterized by weight vector := (Wi,..., w.,) and assigns a score px2;; w, u) to each data point 2; E 'D by computing a weighted average of its marginal contributions { (Ay u) [e(j)- Specifically,
((2 w. u)
Σ w (
Below, we define the weights for three commonly used semivalue-based methods. Their differences in weighting schemes have geometric implications discussed in Section 4.1.
Definition 2.1. Data Shapley (Ghorbani & Zou, 2019) is derived from the Shapley value (Shapley, 1953), a solution concept from cooperative game theory that fairly allocates the total gains generated by coalition of players based on their contributions. In the context of data valuation, Data Shapley takes a simple average all the contributions so that its corresponding weight vector Watap (Wbea.)) is such that for all j E [n.]. Wahap.j 2
Definition 2.2. (a, Be-tt Shapley (Kwon & Zou, 2022) extends Data Shapley by introducing
tunable parameters (a, 3) TR?2, which control the emphasis placed marginal contributions from smaller or larger subsets. The corresponding weight vector wheta = ( is such that for all
j E [n], Wheta.j (3-1) Beta()+8-in-sta) where Beta (a, 3) ((a)T(B)TT(( + () and I is Beta(a.
the Gamma function.
Definition 2.3. Data Banzhaf (Wang Jia. 2023) derived from the Banzhaf (Banzhaf, 1965), a cooperative game theory concept originally introduced to measure player's 's influence in weighted voting systems. Data Banzhaf weight's 's vector Whanzul (wbanchat,j ) is such that for all [n].
Whanzhaf,j
These semivalue- based methods satisfy fundamental axioms (Dubey et al., 1981) that ensure desirable properties in data valuation. In particular, any semivalue d(.,,,)) satisfy the linear- ity axiom which states that for any di, ar E IR, and any utility u, e, 6(2g0, ws a u + azv) = on ((2;; us w. u) (20(2;,,,))
4

---

Published as a conference paper at ICLR 2026
3 A METHODOLOGY TO ASSESS DATA VALUATION ROBUSTNESS TO THE UTILITY CHOICE
We now turn to the two scenarios introduced in Section 1, namely the utility trade-off scenario and the multiple-valid-ue -utlliy scenario, and show how both admit a common geometric formalization. In what follows, we let D { be the dataset that the practitioner seeks to score and rank by order of importance, and we let w be the chosen semivalue weight vector, so that each datum score is given by ((2; w, as defined in (1). We start by giving formal definition of each scenario.
Utility trade-off scenario. In this scenario, the practitioner defines utility as a convex combination of multiple fixed criteria. In the simplest case where considers only two fixed criteria u and u (e.g. helpfulness vs. harmlessness when fine-tuning an LLM),, the utility is
Uy = -v + (1 v)u
V E [0, 1],
where the scalar v is explicitly chosen by the practitioner (based operational priorities) to set the desired trade-off between = and u² Note that this choice naturally extends to K fixed criteria u², ..... k by taking - P.u..
By semivalue linearity, each data point's score under uy is (az; un) = vo(z w + (1 - 11--)0(z;w)) u
Multiple-valid-utiit scenario. In this scenario, there is no single correct utility: practitioners must choose among several equally defensible performance metrics. In the common case of binary classification, might measure model quality with accuracy, Fl-score, negative log-loss: each valid but may yield different data valuation results; see Table 1. Almost all of these utilities admit finear-faactioal form (Koyejo al., 2014) in
test-set statisties: the empirical true- positive rate X(5) = + E [gs (x²) = 1, Uj = 1] and the empirical positive-prediction rate (S) i Dj= (gs((1)) 1J, where gs = A(S) is the classifier trained on S. Specifically, they can be written
Co + c: ((5) c2)
u(S) do d,X(s) + d2y(S))
(2)
with coefficients (c., d.) determined by the chosen utility (see Table 22). Any linear-fractional utility of the form (2) with do # admits the first-order expansion at (Λ, 7) = (0,
u(S)
co do
+
do codi X(S) + c2do cod2
(S) + (I(A((),
df
df
Thus, first order, u is affine in (A, 7) and validate this surrogate empirically (see Appendix B. Thus, by linearity of the semivalue and the fact that constants vanish, for each 24, is reasonable to consider that
6(23w, cido codi dg
6(2,,, A) c2do cod2
df
Remark. The multiple-valid utility scenario also extends to multiclass classification metrics with
Ek= km ag for K > 2 (see Appendix C.5 for details).
K
3.1 A UNIFIED GEOMETRIC MODELING OF THE TWO SCENARIOS
Both scenarios can be unified by observing that the practitioner's utility lies in two-dimensional family spanned by the two fixed base utilities in and u2. Concretely, we consider
n, (S) on (S) + 02 u2()), a = (an, E IR2.
so that varying the utility means moving in this two-dimensional parameter space. In the utility trade-off scenario restricted to two fixed criteria, (μ1, 42) = (u~, u²), and (a1, a2) = (v, 1 v/) ranges over [0, 1 In the multiple-valid-unee -uility scenario for binary classification, (u1, u2) = Y), and (a), 012) E 2 In either case, the objective the same: to quantify robustness, i.e., how stable the ranking of semivalue scores {ō(2) vn)] } as we change a.
With this unified view in hand, we have the following proposition, which can be extended to the general case U₂x Σk=1 04.24-- A detailed extension is provided in Appendix B.2.
5

---

Published as a conference paper at ICLR 2026
Proposition 3.1. Let D be any dataset of size n and let w R" be semivalue weight vector. Then
there exists a map &u.D : DD - R2 such that for every utility ua = 0r U1 + a2t22, 6(2; W. Ua) =
(Vw,p(z), for any E D. We call S. = ((w, 2 E D) the spatial signature of under semivalue w.
Consequently, ranking the data points in D by uh is equivalent to sorting their projections onto the vector a:
6(2;;, Ua) ( ua)
(Wu,D(2s). a) > (Wu,p(zy), a).
Moreover, since scaling a by any positive constant does not change the sign of (1), (Zj), a)), any two utilities un and un whose coefficient vectors point in the same direction induce identical rankings. Thus, each utility in the parametric family can be uniquely identified by its normalized vector = ||| E S', with ā ranging over the unit circle S' Consequently, ranking stability to the utility choice reduces to analyzing how the projections order of (=),, a) e D} C R2 changes as we rotate the unit-vector ā around S Figure illustrate the geometric mapping hand.
00.000
(a) Shapley
(b) (4, 1)-Beta Shapley
(c) Banzhaf
Figure/ 1: Spatial signature of the WIND dataset for three semivalues (a) Shapley, (b) (4, 1)-Beta Shapley, and (c) Banzhaf. Each cross marks the embedding (x) of a data point (with 21 = X. W2 = 7), the dashed circle is the unit circle S and the filled dot indicates one utility direction a.
Figure shows that, under Banzhaf, the points lie almost exactly on a single line through the origin, much more so than under Shapley or (4. (4. 1e-Beta Shapley. This near-collinearity persists across all datasets used in the experiments (see Appendix D.I). In Proposition 3.3 and Section 1, we give insight into how this geometric property directly leads to Banzhaf's higher robustness.
3.2 A ROBUSTNESS METRIC DERIVED FROM THE GEOMETRIC REPRESENTATION
Building on the geometric mapping of semivalue-based data valuation proposed in Section 3.1, a natural way to quantify how robust semivalue scores ranking is changes in the utility is to ask how far on the unit circle one must rotate from a given utility direction before the induced ordering undergoes a significant change?
Formally, let ag be the starting utility direction, whose semivalue scores induce reference ranking of the data points. We say that two points 2; and 2, experience a pairwise swap when their order under new direction ā is opposite to their order under āg. We then aim to define robustness as the smallest geodesic distance S' that one must travel from āy before p pairwise swaps have occurred.
To make this concrete, we express the required geodesic distance in closed form by characterizing the critical angles on S at which pairwise swaps occur. For each unordered pair (i,j), Way = (=,)(=;) Vu, D and observe that the condition (a, vtj) 0 defines two antipodal "cut" points on the unit circle: H., {a E S" (a, miy) = 0}. Across all (*) = N pairs, these give 2N cuts, whose polar angles we list in ascending order as
0 < 02 ... 02N < 2n,
and then wrap around by setting 02N~1 = 0, + 2T. The open ares between successive cuts are
A& (Ox, 0x41) of length d Oc+1 Oz: 1,..., 2N that M d 2n. These open

---

Published as a conference paper at ICLR 2026
ares partition S' into ranking regions, meaning that the induced semivalue ordering is identical for every utility direction ā E Ag. Figure 2 illustrates two example spatial signatures and their induced ranking regions. We view these arcs cyclically by taking indices modulo 2N.. Now let our reference direction êr have polar angle yo E (0x; 0x+1). To induce p swaps, one must cross distinct arcs: counterclockwise this is Sx (p) = Ei=1 A(k+s) mod while clockwise it is S (p) = Ei=1 A(k-t) mod Writing t = Po 8x E (0, >s), the minimal geodesic distance from dy to achieve swaps is²
Pp(ão) = min (p) ,, i, (T) + +1..
We now define our robustness metric based on Pp.
Hu
S
Figure 2: Ranking regions induced by utilities on the unit circle S for two example spatial signatures. Each colored are on the unit circle corresponds to one of the open ares Aç. Within any single are, the projection order (and hence the data-point ranking) remains unchanged.
Definition 3.2 (Robustness metric Rp). Let Sw.D = (2e. be the spatial signature for dataset D under semivalue weights w. For ā E S¹, let (ā() denote the minimal geodesic distance on S one must travel from to incur p (2) pairwise swaps in the induced ranking. Define the average p-swaps distance En-Unit(s")))A [Ap(aa) Rp E [0, 1)
2x Joth (t)td. Then the robustness metric
Ro = En-
Ba-tnir(s)) (pp(a))
max Ea-Unit(S) Ip
T/4
Su.D
where the denominator //4 is the maximum possible value of Ba(py(ā)) which occurs precisely when all embedded points (2;) collinear
Concretely, given spatial signature, the p-robustness metric Rp of this signature is the normalized average minimal angular distance one must rotate on the unit circle force exactly p pairs of points to swap in order in the induced ranking.
Interpretation. Rp close to 1 means that one can rotate ā significantly without flipping more than p pairs, so the ranking is stable. R2 close means that even tiny rotation will likely flip p pairs. Moreover, if there are no tied ranks, Rp captures how far in expectation one must move from utility direction before the Kendall rank correlation degrades by 2p/ (²) (see Appendix B.4 for details).
Computation. We derive a closed-fome expression for En- that computes exactly in On l) time (see Appendix B.5). In contrast, semivalue approximation methods based on Monte Carlo sampling require Onn log n) model trainings to estimate the data values (Jia et al., 2019). Therefore, in practice, once the semivalue scores are in hand, the additional cost of computing Rp is negligible compared to the heavy model-training overhead, making this robustness metric an affordable add-on any data valuation pipeline.
All cut-angles, lengths,,,,, and resulting geodesic distance Pp are entirely determined hy the spatial signature S. For brevity, we omit the explicit dependence on it from our notations.
Proof of this claim is given in Appendix B.6.
7

---

Published as a conference paper at ICLR 2026
Extension to K3 > 2. The robustness metric Ry extends naturally K 2 base utilities, where utility directions ā lic on the unit sphere SK-1, While no closed-form exists for E[pp] in this case, it can be efficiently approximated via Monte Carlo sampling. Appendix B.5 provides convergence guarantees.
3.3 SPATIAL ALIGNMENT AND THE ROBUSTNESS OF SEMIVALUES
The robustness metric Rp (Definition 3.2) measures the stability of the data-value ranking as the utility varies. It increases with the collinearity of the spatial signature Su.D = (tw, : E "D} C RR, which is captured by the Pearson correlation between the two coordinate score vectors for base utilities in and w2- In Proposition 3.3, we express this correlation directly in terms of marginal contributions, and characterize how depends on semivalue weights under mild assumptions.
Let (ua)) = (0(2): ma), d(zn; w. (n)) E R for a E [1,2]. For n, w E R", write i = n Eg 14, Var(v) = A. E. 2 and Cov(U, w) 1 n
E, () w). We study
Cov(o(u o(uz))
Corr (((m), (Var(p(u))) (u( Var Var( (6(u)))
Proposition 3.3 (Utility aligmment and semivalue weights). Let u1 u2 be two base utilities and ((n)), 6(u2) E T their semivalue score vectors. If for all j # k the murginal-contrbbuion vectors (u)))) == (Ai(z1, A,,...,,,,nsss and Ae(u2) = (Au(z1, u2)..... .., As(Zn, W2))
uncorrelated across points, then
Σラ of (A)(
Corr(d(u). o(uz))
Ej= 1 w Var ر, v Ej= w/ Var (y(22)
Defining the size-j alignment factor
T3 = (ov(A,( Ay(w2) Corr (A, A (ar())))))))
then the correlation increases as the semivalue weights {wy} concentrate on sizes j where r. is large.
The proof is given in Appendix B.7.
4 EMPIRICAL EVALUATION OF ROBUSTNESS AND DISCUSSION
4.1 MULTIPLE-VALID UTILITY SCENARIO
In this section, we empirically validate the p-robustness metric R, in the muliipee-valdd-ttine scenario. We evaluate R. for three semivalues, Shapley, (4, 1)-Beta Shapley, and Banzhaf, on several public binary classification datasets. The results in Figure 3 (detailed in Table 7) closely track Section I's correlation experiments reported in Table I: datasets and semivalues that exhibit low rank correlations between different utilities also show low Rp. and vice versa.
We also observe that across practically every dataset and choice of p. using the Banzhaf weights achieves the highest Rg. This makes sense geometrically: Figure I and the analogous plots for the other datasets in Appendix D.I show that the Banzhaf weighting scheme tends to collinearize the spatial signature, i.e., push the points closer to a common line through the origin. And since the maximum possible average swap-distance occurs when all embedded points are collinear, this near-collinearity explains why Banzhaf yields the greatest robustness to utility shifts. This observation aligns with prior empirical findings (Wang & Jia, 2023; Li & Yu, 2023), which reported that Banzhaf scores tend to vary less than other semivalues under changing conditions.
These geometric insights are made rigorous by Proposition 3.3, applied to the correlation between the semivalue vectors for À and 7, i.c., Corr(o(A), Φ(7)). says that under mild assumption on cross- size correlations of marginal contributions (empirically verified notably; see Appendix A.5), this correlation decomposes into a weighted average of size-specific alignment factors Tj. with weights wf. Figure plots the normalized ry versus coalition size j and overlays the Shapley, (4, 1)-Beta, and Banzhaf weight profiles. On BREAST, Tý is uniformly high
BREAST and TITANIC
8

---

Published as a conference paper at ICLR 2026
breast
ttanic
credit
heart
Shuplor
0.8
0.2
100
2000
1500
500
1000
1500
500
1000
2500
500
1000
1500
F
wind
cpu
2dplanes
pol
0.6
Figure 3: Mcan p-robustness Rp (error bars standard errors over 5 Monte Carlo approximations) plotted against p E (500, 1000, 1500) for each dataset and semivalue. Each plot corresponds to one dataset, with Shapley (blue), (4, 1--Beta Shapley (pink), and Banzhaf (green) curves. Higher Rp indicates greater ranking stability under utility shifts.
across j. so all three semivalues yield similar collinearity, which is consistent with the overlapping robustness curves in Figure 3. On TITANIC, rý peaks at intermediate j and decays at the extremes; because Banzhaf concentrates weight in this middle region, attains a larger weighted average (hence higher overall correlation), explaining why its robustness curve sits well above Shapley and (4, 1)-Beta in Figure 3.
Breast
Ttanic
0.12
Shapley
Beta (4.1)
0.10
Bandhaf
(mean SE
0.6
0.08
3
0.06
0.4
Figure 4: Mean (normalized) ry (error bars = standard emrors over 5 semivalue approximations) for BREAST (blue) and TITANIC (red) vs. coalition size with semivalue weights overlaid.
Additional robustness experiments, including comparison to top-k stability metrics and extensions to K base utilities, are reported in Appendices A.7 and A.9 and discussed in Appendix A.
4.2 UTILITY TRADE-OFF SCENARIO
We also evaluate RD in the utility trade-off scenario, where utility is defined convex combination of competing criteria. Specifically, we consider utilities of the form u, = /u1 + (1 - v/)uz with [0, 1], and analyze how semivalue-basse Banzhaf) evolve as y varies. We run this on regression datasets (DIABETES, CALIFORNIA HOUSING, AMES) for utility pairs MSE/MAF, MSE/R2 and MAE/R2, and multiclass classification datasets
rankings (using Shapley, (4, 1e-a Shapley, and

---

Published as a conference paper at ICLR 2026
(DIGITS, WINE, IRIS) for utility pairs Accuracy/macro-FF. Accuracylmacro-Reese Recall, and macro- F1/macro-Recall Across all settings, Banzhaf achieves the highest Rp. indicating more stable rankings. These results consistent with the ones obtained in multiple-valid utility scenario (see Section 4. The data sources are given in Appendix A.1 while full results with experimental settings reported in Tables 9, 10, 11, 12, 13, and 14. Additional experiments for the case K 2 base utilities are detailed in Appendix A.6.2 and are discussed in Appendix A.
5
CONCLUSION
This work studies the robustness of semivalue-based data valuation methods under utility shifts in two scenarios where it matters, by introducing unified geometric view via the spatial signature and parametric robustness measure Ry- This yields practical way to quantify the stability of data-value rankings as the utility varies.
Limitation. While the framework is general, our analysis of the multiple-valid-uiitty sce- nario focuses on binary classification metrics in the linear-fractional family and on subset of multiclass metrics. Non-linear-fractional binary metrics (e.g., negative log-loss) and regression utilities fall outside our scope this scenario.
Future works. By revealing cases in which semivalue-based data valuation fails pro- duce reliable scores, we aim to encourage future research to assess whether these methods genuinely solve the problem they claim to address.
REPRODUCIBILITY STATEMENT
The full codebase is publicly available at attps://gittuub..mm/mmmmiiinn www.:/ooieeneettteeeeeeeeo lissa/ utility- criility-ipacc..o .orpact.gitt. reproduces all tables and figures in the paper (with scripts gener- ate them). Full experimental protocols, including datasets, pre-processing, byperparameters, and compute settings, are documented in Appendix and are cross-referenced at the relevant points in the main text. All missing proofs and supporting theoretical results are given in Appendix B, where assumptions are stated, and derivations are provided.
ACKNOWLEDGMENTS
This work was partially supported by the French National Research Agency (ANR) through grants ANR-20-CE23-0007 and ANR-23-CE23-0002 and through the PEPR IA FOUNDRY project (ANR- 23-PEIA-0003).
REFERENCES
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint A.Xi::2244008666 2022a.
Yuntao Bai, Sauray Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, al. Constitutional ai: Harmlessness from ai feedhack. arXiv preprint arXiv:22 2022b.
J.F. Banzhaf. Weighted voting doesn't 't work: A mathematical analysis. Rutgers Law Review, 19(2): 317-343, 1965.
Louise Bloch and Christoph Friedrich. Data analysis with shapley values for automatic subject selection in alzheimer's disease data sets using interpretable machine learning. Alzheimer's s Research Therapy, 13, 09 2021.
SHC Choi, Sung- Cha, and Charles Tappert. survey of binary similarity and distance measures. Syst. Cybern. Inf., 11 2009.
10

---

Published as a conference paper at ICLR 2026
Dean De Cock. Ames, iowa: Alternative to the boston housing data set. Journal of Statistics Education, 19(3):1-12, 2011.
Hannah Diehl and Ashia C Wilson. Semivalue-based data valuation is arbitrary and gameable. arXiv preprint arXiv:2506. 12619, 2025.
Dheeru Dua and Casey Graff. Digits dataset. UCI Machine Learning Repository, 2019.
Pradeep Dubey, Abraham Neyman, and Robert James Weber. Value theory without efficiency. Mathematics of Operations Research, 6(1):122-128, 1981.
Bradley Efron, Trevor Hastie, lain Johnstone, and Robert Tibshirani. Least angle regression. The Annals of Statistics, 32(2), April 2004.
R. A. Fisher. The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7(2): 179-188, 1936.
Felipe Garrido Lucero, Benjamin Heymann, Maxime Vono, Patrick Loiseau, and Vianney Perchet. Du-shapley: A shapley value proxy for efficient dataset valuation. Advances in Neural Information Processing Systems, 37:1973-2000 2024.
Amirata Ghorbani and James Zou. Data shapley: Equitable valuation of data for machine learning. In International conference on machine learning, pp. 2242-2251. PMLR, 2019.
Amirata Ghorbani, Michael Kim, and James Zou. A distributional framework for data valuation. In International Conference on Machine Learning, pp. 3535-3544, PMLR, 2020.
Ruoxi Jia, David Dao, Boxin Wang, Frances Ann Hubis, Nick Hynes, Nezihe Merve Gürel, Bo Li, Ce Zhang, Dawn Song. and Costas J Spanos. Towards efficient data valuation based on the shapley value. In The 22nd international conference artificial intelligence and statistics, pp. 1167-1176. PMLR, 2019.
Kevin Jiang, Weixin Liang, James Y Zou, and Yongchan Kwon. Opendataval: a unified benchmark for data valuation. Advances Neural Information Processing Systems, 36:28624 36:28624-28647, 2023.
R. Kelley Pace and Ronald Barry. Sparse spatial autoregressions. Statistics & Probability Letters, 33 (3):291-297, 1997.
Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In International conference on machine learning, pp. 1885-1894, PMLR, 2017.
Oluwasanmi Koyejo, Nagarajan Natarajan, Pradeep Ravikumar, and Inderjit S Dhillon. Consis- tent binary classification with generalized performance metrics. In Advances in Neural Information Processing Systems, volume 27, 2014.
Yongchan Kwon and James Zou. Beta shapley: unified and noise-reduced data valuation framework for machine learning. Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, volume 151, pp.
2022.
Yongchan Kwon, Manuel Rivas, and James Zou. Efficient computation and analysis of dis- tributional shapley values. In Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, volume 130 of Proceedings of Machine Learning Research, pp. 793-801, 2021.
Weida Li and Yaoliang Yu. Robust data valuation with weighted banzhaf values. In Advances in 2023.
Neural Information Processing Systems, volume 36, pp.
Sasan Maleki. Addressing the computational issues of the shapley value with applications in the smart grid. In University of Southampton. Physical Sciences and Engineering. Doctoral Thesis, 115pp. 2015.
Irwin Mann and Lloyd S. Shapley. Values of Large Games, IV: Evaluating the Electoral College by Montecarlo Techniques. Rand Corporation, 1960.

---

Published as a conference paper at ICLR 2026
Curtis Northcutt, Anish Athalye, and Jonas Mueller. Pervasive label emrors in test sets destabilize machine learning benchmarks. In Proceedings of the Neural Information Processing Systems Track Datasets and Benchmarks, volume 1, 2021.
Konstantin D. Pandl, Fabian Feiland, Scott Thiebes, and Ali Sunyaev. Trustworthy machine learning for health care: scalable data valuation with the shapley value. In Proceedings of the Conference on Health, Inference, and Learning, pp. 21-57,00111
Andrea Dal Pozzolo, Olivier Caelen, Reid A. Johnson, and Gianluca Bontempi. Calibrating prob- ability with undersampling for unbalanced classification. In 2015 IEEE Symposium Series Computational Intelligence, pp. 159-166, 2015.
Lloyd S Shapley. A value for n-person games. In Harold W. Kuhn and Albert W. Tucker (eds.). Contributions to the Theory of Games I1, pp. 307-317. Princeton University Press, Princeton, 1953.
Richard Stanley. An introduction to hyperplane arrangements, pp. 389-496. MIT, 10 2007.
Siyi Tang, Amirata Ghorbani, Rikiya Yamashita, Sameer Rehman, Jared A Dunnmon, James Zou, and Daniel I. Rubin. Data valuation for medical imaging using shapley value and application to large-scale chest x-ray dataset. Scientific reports, 11(1):8366, 2021.
Dootika Vats and Christina Knudson. Revisiting the gelman-ruie rubbin diagnostic. Statistical Science, 36 (4):518-522) 2021.
Jiachen T Wang and Ruoxi Jia. Data banzhaf: A robust data valuation framework for machine learning. In International conference on artificial intelligence and statistics, pp. 6388-6421. PMLR, 2023.
Jiachen T Wang, Prateek Mittal, and Ruoxi Jia. Efficient data shapley for weighted nearest neighbor algorithms. In International Conference Artificial Intelligence and Statistics, pp. 2557-2565. PMLR, 2024a.
Jiachen T. Wang, Tianji Yang, James Zou, Yongchan Kwon, and Ruoxi Jia. Rethinking data shapley for data selection tasks: Misleads and merits. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 52033- 52063, 21-27 Jul 2024b.
Tom Yan and Ariel Procaccia. you like shapley then you'll Ill love the core. AAAI Conference on Artificial Intelligence, 2021.
T. Zaslavsky. Facing up to Arrangements: Face-Count Formulas for Partitions of Space by Hyper- planes: Face-count Formulas for Partitions of Space by Hyperplanes. American Mathematical Society: Memoirs of the American Mathematical Society. American Mathematical Society, 1975.
Kaiping Zheng, Horng-Reey Chua, Melanie Herschel, H. V. Jagadish, Beng Chin Ooi, and James Wei Luen Yip. Exploiting negative samples: catalyst for cohort discovery in healthcare analytics. In Proceedings of the 41st International Conference on Machine Learning, 2024.
12

---

Published as a conference paper at ICLR 2026
A
ADDITIONAL SETTINGS & EXPERIMENTS
For the reader's convenience, we first outline the main points covered in this section.
Appendix A.1: Experiment settings for empirical results in the main text.
Appendix A.2: Additional results on rank correlation for more binary classification metrics.
Appendix A.3: Additional results rank correlation using the Spearman rank correlation.
Appendix A.4: Table for Rp results in Figure 3.
Appendix A.5: Empirical verification of the assumption of Proposition 3.3.
Appendix A.6: Results for the utility-traee-off scenario summarized in Section 4.2 and extension K base utilities.
Appendix A.7: Results for the multiple-valid utility scenario extended to K > 2 base utilities.
Appendix A.8: What if we A varies instead of perf?
Appendix A.9: Empirical link between the robustness metric Rp and top-k stability metrics (overlap @ and Jaccard k).
Appendix A.10: Overall discussion about empirical robustness results.
A.1 EXPERIMENT SETTINGS FOR EMPIRICAL RESULTS IN THE MAIN TEXT
In this section, we describe experimental protocol for estimating semivalue scores, which serve to obtain all the tables and figures included in this paper.
Datasets. Table summarizes the datasets used in our experiments, of which are standard benchmarks in the data valuation literature (Ghorbani & Zou, 2019; Kwon & Zou, 2022; Jia et al., 2019; Wang & Jia, 2023; Jiang et al., 2023). Due the computational cost of repeated model retraining in our experiments, we select a subset of 100 instances for training and 50 instances for testing from each classification dataset and 300 instances for training and 100 instances for testing from regression datasets.
Table A summary of datasets used in experiments.
Dataset
Source
BREAST
Stttp::/www..org
TITANIC
https:
CREDIT
Pozzolo et al. (2015)
HEART
WIND
CPU
https: http::/www.ggennlccrr//////
2DPLANES
https:/
POL
SttEps://www.opeee http://wwoommoooo///////////////
DIABETES
Efron et al. (2004)
CALIFORNIA HOUSSNG
Kelley Pace & Barry (1997)
AMES
De Cock (2011)
IRIS
Fisher (1936)
WINE
https: .iw..eee................o....eee wus.uiiiii...................... e edu/mm/ http////daatasts//nnna
DIGITS
Dua & Graff (2019)
Because our primary objective is to measure how changing the utility alone affects semivalue rankings, must eliminate any other sources of variation, such as different train/test splits, model initialization, or Monte Carlo sampling noise, that could confound our results. To this end, we enforce two strict controls for semivalue scores computation across utilities:
A fixed learning context (A, Duaa). 2. Aligned sampling for semivalue approximations.
13

---

Published as a conference paper at ICLR 2026
FIXED LEARNING CONTEXT. As outlined in Section 2, utility function is defined as:
(()) = (A()) Diea);
where A is learning algorithm that outputs a model trained on dataset S, and perf evaluates the model on a test set Diss. The learning algorithm A specifies the model class, objective function, optimization procedure, and hyperparameters (e.g., learning rate, weight initialization).
By fixing (A, Diest), we ensure that swapping between two utilitiis, say, accuracy versus Fl-score, amounts solely to changing the performance metric perf. Consequently, any shift in the semivalue scores' ranking (measured by rank correlation metries) can only be attributed to the utility choice.
Controlling for sampling noise in semivalue estimates. The above discussion assumes access to exact semivalue scores, but in practice, we approximate them via Monte Carlo permutation sampling, which injects random noise into each run. Without accounting for this sampling variability, differences in semivalue scores' rankings could reflect estimator noise rather than genuine sensitivity to the utility.
To enforce this, we introduce aligned sampling alongside the fixed learning context (A, Drest). Aligned sampling consists of pre-generating single pool of random permutations (or sampling seeds) and reusing those same permutations when estimating semivalues for each utility. By sharing both the model-training environment and the permutation draws, ensure that any differences in resulting rankings are driven solely by the change in utility.
Fixed set of permutations. Let P { T2...., denote a fixed set m random permu- tations of the data points in D. We apply this exact set of permutations across multiple utilities {u1, U2,..., ux) such that (.) = ((rf Drsu] with fixed (A, Diea) for all k: E [K].
For given performance metric perfx and the set of permutations P, we estimate the marginal contributions ( (z:; u=)] for each data point E 'D with respect to the utility such as
Â (2 (x) = (ux (S) U (z+)) - up ())
where m the number of permutations used, T. denotes the s-th permutation and ST represents the subset of data points of size j 1 that precedes 2; in the order defined by permutation T..
Determining the number of permutations m. The number of permutations m used in the marginal contribution estimator is determined based maximum limit and a convergence criterion applied across all utilities uj...., UK. Formally,
m max mMm,n min (Mmax m.comm)) ))
where miii is a predefined minimum number of permutations avoid starting convergence checks prematurely, mmax is a predefined maximum number of permutations set to control computational feasibility, Moony the smallest number of permutations required for the Gelman- Rubin (GR) (Vats & Knudson, 2021) statistic to converge across all utility functions U1,...; ug- MK.. Using the Gelman-Rubin statistic as convergence criterion follows established practices in the literature (Jiang al., 2023; Kwon & Zou, 2022).
For each data point 2,, the GR statistic RG computed for every 100 permutations across all utilities. The sampling process halts when the maximum GR statistic across all data points and all utilities falls below a threshold, indicating convergence. We adopt the conventional threshold of 1.05 for GR convergence, consistent with prior studies in data valuation (Jiang et al., 2023).
In this framework, the GR statistic, R. is used to assess the convergence of marginal contribution estimates for each data point 2; across C' independent chains of sampled permutations under each utility Uy. The GR statistic evaluates the agreement between chains by comparing the variability within each chain to the variability across the chains, with convergence indicated when R approaches 1. Specifically, to compute the GR statistic for each data point z; under utility ug. we determine
1. The within-chain variance W which captures the variability of marginal contributions for 2; within each chain. Specifically, if there are independent chains, W is calculated the
14

---

Published as a conference paper at ICLR 2026
average of the sample variances within each chain
C
W C Σ s
where si._ is the sample variance of marginal contributions for 24 within chain c. This term reflects the dispersion of estimates within each chain,
2. And the between-chain variance Be which measures the variability between the mean marginal contributions across the chains. It indicates how much the chains differ from each other. The between-chain variance is defined as
B = - E (Ae(z; A (2s ux))²
where (z; un) ) is the mean marginal contribution for 2, in chain e, and A(2; ux) is the overall mean across all chains
C
A (1)
Σ Ac(=;u)) ux)-
The term B quantifies the extent of disagreement among the chain means.
Combining both / and B/, the GR statistic R2 for data point 2; under utility ug is defined as:
ss
B
R
W
Intra-permutation truncation. Building on existing literature (Ghorhani & Zou, 2019; Jiang et al., 2023), we further improve computational efficiency by implementing perra--ermutationn truncation criterion that restricts coalition growth once contributions stabilize. Given a random permutation T, E the marginal contribution for each data point ã (the j-th point in the permutation T,) is calculated incrementally as the coalition size j increases from 1 to n. However, instead of expanding the coalition size through all n elements, the algorithm stops increasing j when the marginal contributions become stable based on relative change threshold.
For each step i [n] within permutation, the relative change V/ the utility up is calculated as: uu ((2r.) Di=1 (ZTT.11)) ug (1=m,.))
Wk ((2x,,1/=>
where {z+., 1 represents the coalition formed by the first data points in TT.. This measures the relative change in the utility when adding the next data point to the coalition. The truncation criterion stops increasing the coalition size at the smallest value j satisfying the following condition:
= min {j E [n) ( S j : Vi < 10-*}| 2 10].
This means that the coalition jize is fixed at the smallest j for which there are least 10 prior values of Vi (for < j) that are smaller than a threshold of 10- This condition ensures that the utility ug has stabilized, indicating convergence within the permutation. This intra-permn ptraa-permuttionn truncation reduces computational cost by avoiding unnecessary calculations once marginal contributions stabilize.
Aggregating marginal contributions for semivalues estimation. Once the marginal contributions have been estimated consistently across all permutations and utilities, they are aggregated compute various semivalues, such as the Shapley, Banzhaf, and (4, 1)-Beta Shapley values. Each semivalue method applies a specific weighting scheme (see Definition 2.1, 2.2, 2.3) to the marginal contributions reflect the intended measure of data point importance.
For data point z; under utility uk. its approximated data value ò( 21; w, uy) is computed by applying a weighting scheme w to the marginal contributions across coalition sizes
H(z w. up) = Wi Áy(2; ux),
where ( U&) is the estimated marginal contribution for coalition size j - 1, and Wj is the weight assigned coalition size 1.
15

---

Published as a conference paper at ICLR 2026
Learning algorithm A. For binary classification experiments, A is a logistic- classifier (binary cross-entropy loss) trained via L-BFGS with l regularization (A 1.0). For multiclass classification experiments, A is feed-forward MLP (ReLU hidden layers, softmax output) trained with cross-entropy via L-BFGS and &2 regularization (A = 1.0). For regression experiments, A is linear ridge model (squared-error loss, & regularization À = 1.0) trained with L-BFGS. We initialize all weights from N((,, 1) with fixed random seed, disable early stopping, and fix the maximum number of training epochs 100. The optimizer's step size is 1.0.
Decision-threshold calibration for binary classification. Because we compare multiple binary classification utilities (accuracy, Fi-score, etc.), using fixed probability cutoff (e.g., 0.5) can unfairly favor some metries over others, especially under class imbalance. To ensure that differences in semivalue scores' rankings arise from the utility definition (and not an arbitrary threshold), we calibrate the decision boundary to the empirical class prevalence. Concretely, if p is the fraction of positive labels in the training set, we set the cutoff at the (1 p)-quat of the model's predicted probabilities. This way, each trained model makes exactly p% positive predictions, aligning base-rate assumptions across utilities and isolating the effect of the performance metric itself.
Computational resources and runtime. All experiments (8-core CPU) with 16 GB RAM) without parallelization. full semivalue estimation, consisting of independent Monte Carlo approximations, for one dataset of 100 data points takes approximately 15 minutes.
single machine (Apple M1
A.2 ADDITIONAL. RESULTS RANK CORRELATION FOR MORE BINARY CLASSIFICATION METRICS
In Table 1, we compare semivalue score rankings under accuracy versus Fi-score. Here, we broaden this analysis to include other widely used binary classification utilities (recall, negative log-loss, and arithmetic mean). Tables 3 and 4 show that ranking variability persists across datasets and semivalue choices when using these additional metrics.
Table 3: Mean Kendall rank correlations (standard error in parentheses rounded to one significant figure for clarity) between accuracy (acc) and negative log-loss (n11), and between Fl-score (f1) and negative log loss, for three semivalues (Shapley, Beta (4,1), Banzhaf). Values are averaged over 5 estimations.
Dataset
Shapley
(4.1)-Beta Shapley
Banzhaf
acc-nll
fi-nll
acc-nll
fi-n
acc-nll
f1-nl1
BREAST
-0.59 (0.02)
-0.60 (0.02)
-0.65 (0.01)
-0.66 (0.01)
0.18 (0.01)
0.18 (0.01)
TITANIC
-0.53 (0.01)
0.54 (0.01)
(0.00)
-0.61 (0.01)
(0.02)
-0.07 (0.01)
CREDIT
(0.02)
-0.43 (0.01)
(0.01)
-0.49 (0.01)
0.38 (0.01)
0.28 (0.03)
HEART
(0.02)
0.01 (0.02)
(0.02)
-0.17 (0.03)
-0.07 (0.01)
-0.05 (0.01)
WIND
0.67 (0.02)
0.74 (0.02)
0.73 (0.01)
(..60)
(..40)
CPU
0.55 (0.01)
0.68 (0.01)
0.59 (0.01)
0.69 (0.01)
-0.03
0.52 (0.01)
2DPLANES
0.22 (0.02)
0.98 (0.01)
0.41 (0.01)
0.98 (0.01)
-0.03 (0.01)
0.18 (0.01)
Pol
0.58 (0.01)
0.79
0.81 (0.01)
-0.01 (0.02)
0.13 (0.02)
A.3 ADDITIONAL RESULTS ON RANK CORRELATION USING THE SPEARMAN RANK CORRELATION
For completeness, we re-evaluate all of our pairwise semivalue ranking comparisons using Spearman rank correlation instead of Kendall rank correlation. As shown in Tables 5 and 6, datasets and semivalues that exhibit low Kendall correlations between different utilities also yield low Spearman correlations, and vice versa.
16

---

Published as a conference paper at ICLR 2026
Table 4: Mean Kendall rank correlations (standard error in parentheses rounded to one significant figure for clarity) between recall (rec) and accuracy (acc) for three semivalues (Shapley, Beta (4,1), Banzhaf). Values are averaged over 5 estimations.
Dalaset
Shapley
(4.1)-Bett Shupley
Banzhal
acc-am
scc-rec
Sm-rec
Scc-am
acc-rec
am-rec
DCC-am
acc-rec
am-rec
BREAST
0.93 (0.01)
0.98 (0.01)
0.92 (0.01)
0.94 (0.01)
0,98 (0.01)
0.92 (0.01)
0.82 (0.03)
0.99 (0.01)
0.81 (0.03)
TITANIC
(0.05(0.04)
0.77 (0.02)
-0.05 (0.05)
-0.27 (0.03)
0.62 (0,04)
0.08 (0.05)
0.46 (0.02)
0.81 (0.01)
0.65 (0.01)
CREDIT
-0.31 (0.01)
0.07 (0.01)
0.60 (0.02)
.0.31 (0.02)
(0.04)
0.62 (0.01)
0.35 (0.01)
0.58 (0.01)
(0.6)
HEART
0.19
0.98 (0.01)
0.18 (0.02)
0.22 (0.01)
0.98 (0.01)
(.00
0.61 (0.01)
0.98 (0.01)
0.59 (0.02)
WIND
0.08 (0.03)
0.98 (0.01)
0.07 (0.03)
(0.02)
(0.01)
0.08 (0.04)
(0.01)
0.98 (0.01)
(.75)
CPU
0.19 (0.04)
0.75 (0.02)
0.18 (0.01)
0.22 (0.08)
0.78 (0.02)
0.22 (0.02)
0.79 (0.01)
0.95 (0.01)
0.86 (0.01)
2DPLANES
0.31 (0.02)
0.99 (0.01)
0.31 (0.02)
0.33 (0.02)
0.99 (0.01)
0.33 (0.02)
0.037 (0.01)
0.99 (0.01)
0.37 (0.01)
PoL.
0,56 (0.01)
0.73 (0.01)
0.29 (0.01)
0.56 (0.01)
0.79 (0.01)
(0.01)
0.67 (0.01)
0.69 (0.01) 0.36 (0.01)
Table 5: Mean Spearman rank correlations (standard error in parentheses rounded to one significant figure for clarity) between accuracy (acc) and negative log-loss (n11), and between F1-score (f and negative log loss, for three semivalues (Shapley, Beta (4,1), Banzhaf). Values are averaged over 5 estimations.
Dataset
Shapley
(4.1)-Beta Shupley
Banzhaf
E1
a11
[l m111
acc
n11
f1 m111
acc E1
ecc-nil m11
fl alll
BREAST
0.99 (0.01)
-0.76 (0.02)
-0.78 (0.02)
0.99 (0.01)
-0882 (0.01)
-0.83 (0.01)
0.98 (0.01)
0.22 (0.01)
0.23 (0.01)
TTEANIC
-0.20 (0.D1)
-0.71 (0.01)
0.74 (0.01)
-0.18 (0.01)
-0.79 (0.01)
(0.01)
0.95 (0.01)
0.18 (0.02)
-0.20 (0.01)
CREDIT
0.50(00.02)
(0.02)
-0.61 (0.02)
(0.01)
(0.01)
(0.02)
0,90 (0.01)
0.53 (0.01)
(0.03)
HEART
0.71 (0.01)
-0.04 (0.92)
0.03 (0.03)
0.67 (0.01)
40.28 (0.04)
-0.23 (0.04)
0.96 (0.01)
-0.10 (0.02)
40.08 (0.02)
WIND
0.85(0:01)
0.84 (0.01)
0.86
0,85 (0.01)
(0.01)
0.97 (0.01)
0.34 (0.01)
0.62 (001))
CPU
0.47 (0.02)
0.73 (0.01)
0.85 (0.01)
0.45 (0.0))
(001)
0.86 (0.01)
0.87 (0.01)
-0.71 (0.01)
0.70
2DPLANES
0.24 (0.01)
0.33 (0.02)
0.99 (0.01)
0.28 (0.02)
0.58 (0.01)
0.99 (0.01)
0.75 (0.01)
-0.04 (0.02)
0,24 (0.05)
POL.
0,70 (0.01)
0.77 (0.01)
0.92 (0.01)
0.69 (0.01)
0.90 (0.01)
0.93 (0.01)
0.53 (0.01)
-0.01 (0.03)
0.21 (0.02)
Table 6: Mean Spearman rank correlations (standard error in parentheses rounded to one significant figure for clarity) between recall (rec) and accuracy (acc) for three semivalues (Shapley, Beta (4,1), Banzhaf). Values are averaged over 5 estimations.
Datasel
Shapley
(4.i)-Bela Shapley
Banzhaf
6ccmmmm
aco-rec
acc-amt
aco-rec
am-rec
acc-
acc-rec
aft-rrc
BREAST
(0.0))
0.99 (0.01)
0.99 (0.01)
(0.01)
0.99 (0.01)
0.99 (0.01)
0.90 (0.02)
0.99 (0.01)
0.89 (0.03)
TITANIC
0.37 (0.05)
0.91 (0.02)
(0.08)
.0.37 (0.04)
0.89 (0.02)
0.10 (0.08)
(0.03)
(0.01)
(0.01)
CREDIT
-0.45 (0.01)
0.09 (0.02)
0.79 (0.01)
-0.40 (0.03)
0.11 (0:02)
0.83 (0.01)
0.5 (0.01)
0.75 (0.01)
0.92 (0.01)
HEART
0.29 (0.02)
0.99 (0.01)
0.27 (0.02)
0.28 (0.02)
0.89 (0.02)
0.27 (0.01)
0.80 (0.02)
0.99 (0.01)
(0.02)
WIND
0.12
0.99 (0.01)
0.11 (0.04)
0.12
0.97 (0.02)
0.10
0.92 (0.01)
0.99 (0.01)
0.92 (0.01)
0.27 (0.01)
(0.00)
0.27 (0.01)
0.27 (0.02)
0.92 (0.03)
0.31 (0.03)
0.93 (0.01)
0.99 (0.01)
(0.01)
2DPLANES
0.44 (0.03)
0.99 (0:01)
0.44 (0.03)
0.47 (0.03)
0.99 (0:01)
0.47 (0.03)
0.52 (0.01)
0.99 (0.01)
0.52 (0.01)
POL
0.75 (0.01)
(0.01)
0.42 (0.01)
0.74 (0.01)
0.93 (0.01)
0.48 (0.02)
0.85 (0.01)
0.87 (0.01)
0.52 (0.01)
A.4 TABLE FOR R2 RESULTS IN FIGURE 3
In support of Figure 3 displayed in Section 4, Table 7 below reports the mean and standard error of the p-robustness metric Rp for E (500, 1000, 1500} on each dataset and semivalue.
A.5 EMPIRICAL VERIFICATION OF THE ASSUMPTION OF PROPOSITION 3.3
In this section, verify empirically that the assumption of Proposition 3.3 holds for the two datasets we take as examples in Figure 4, namely BREAST and TITANIC. For (u,, = (A, we compute the cross-size covariance matrix
Ey = Cov (A))))))), Ax())),
j, k E {1, (1......1) n):
using the same Monte Carlo runs as for the semivalues. We then check that off diagonal terms are negligible compared to the diagonal by computing two metrics:
Σx
Sayua M2
8 (Corr(((u1), (((22) - (orr) [Corr( (())))
غ = max )
jk
and
Sun u2 jj
17

---

Published as a conference paper at ICLR 2026
Table Mean p-robustness Rg (standard error in parentheses) for p {500, 1000, 1500} estimated over 5 Monte Carlo trials (each trial corresponding to approximating the semivalue scores). Boldface marks the semivalue with the highest Rp for each dataset and p. Higher Rp indicates greater stability of the induced ranking under utility shifts.
Rom
Bysse
Reso1
Stapley
(4ett Shapley
Bauzhaf
Shapley
(4.1) Stapley
Banzhaf
Stapley
y-Rta Shupley
Banzhof
BREAST
(0.004)
(0.004)
(0.0044)
(0,003)
(0.02)
0.79 (0.00%)
(0.002)
0.71
TTTANIC
0.058 (0.001)
0.058 (0.001)
0.44 (0.03)
0.27 (0.004)
0.28 (0.004)
(0.01)
0.48 (0.004)
0.48 (0.004)
0.89 (9.007)
CRUDIT
0.084 (0.005)
0,091 (0.IK5)
(.82 (0.07)
(136 (0.01)
0.38 (0.011
0.97 (0.01)
0.58 (0.01)
0.60 (0.01)
1.99 (0.002)
HEART
0.36 (OLME3)
0.36 (0.005)
0.53 (0.01)
0.54 (0.00)
1154 (0.009)
0.57 (0.0077
0.73 (0.005))
0.73 (00855)
1L93 (0,0055
WIND
0.23 (0.00)
0.21 (0.01)
0.58 (0.01)
0.64 (0.01)
0.62 (0.009)
0.88 (0.005)
0.79 (0.055)
0.77 (0.007)
0.94 (0,004)
CPU
0.11 (0.0033
0.12 (0.003)
0.38 (0.02)
0.45 (0.009)
0.45 (0.009)
0.77 (0.009)
0.66 (0.009)
0.67 (0.009)
0.58 (0.004)
ZOPLANES
0,090 (0.031)
11084 (0,0022
0.19 (6.012)
0138 (0.0144)
0136 (0.096)
0.59 (0.02)
1.59 (0.0044
0.57 (0.885)
0.75 (0,01)
Pot.
0,050 (0.003)
0.09 (0.003)
0.19 (0.01)
0.39 (0.005)
0.42 (0.007)
0.50 (0.01)
0.60 (0.005)
0.63 (0.005)
0.70 (0.01)
where Corrdiag keeps only the diagonal entries Su142 On BREAST and TITANIC, we find & < 0.12 meaning that, row-wise, the total magnitude off-diagonal covariances Σnッj Surwa is at most Lik
12% of the corresponding diagonal term Euy wa, i.e., off difgonal cross size effects are negligible. Moreover, find that 8 < 7% showing that using only the diagonal of Eeioa reproduces the full correlation within few percent, which is exactly what one would expect if Cov (uI), (u2)) R 0 for j f k. Exact means + 95% CIs are reported in Table 8.
Table 8: Verification of the cross-size size independence assumption (Proposition 3.3): 8 max, Enzj [S 1/$ (smaller is better) and 8 := [Crr(((uu)) o(u2)) (orTiia(ó(U))))) ((uz)) Corr(((u), o(22)) (smaller is better). Mean + 95% CI over R=5
seeds.
Dataset
é (mean 95% CT)
8 (mean + 95% CT)
BREAST
0.08 1 0.03
0.03 1 0.01
TITANIC
0.10 +1: 0.02
0.05 1 0.02
A.6 RESULTS FOR THE utility-trade-off SCENARIO SUMMARIZED IN SECTION 4.2 AND EXTENSION > BASE UTILITIES
In this section, we evaluate robustness in the utility trade-off setting for regression, binary classifica- tion, and multiclass classification.
A. CASE WHERE K = 2 BASE UTILITIES
In this setting, the utility is convex combination of two task-relevant metrics,
U //1 + (1 v)u2,
V E [0, 1].
We consider the following utility pairs:
Regression. MSE/MAE (Table 9), MSE/R² (Table 10), and MAE/R² (Table 11). Multiclass classification. Accuracy/macro-FI (Table 12), Accuracy/macro-Recall (Table 13), and Macro-Fl/mmacoo-Ret (Table 14).
For each pair, we compute semivalue-based rankings (Shapley, (4, ()etta Shapley, Banzhaf) and evaluate robustness along the convex path using Rs00-
A.6.2 CASE WHERR K > 2 BASE UTILITIES
To study trade-offs beyond pairs of utilities, we consider K 3 base utilities simultaneously, which allows us to visualize corresponding spatial signatures in 3D (see Figures 12-19 in Appendix D.2). In this setting, the utility is therefore convex combination of three task-relevant metrics,
u, V1 U1 + V2tl2 1333s (21, 22, 2?
18

---

Published as a conference paper at ICLR 2026
Table 9: Regression. Robustness scores R300 along the MSE-MAE convex path. Semivalues are approximated over 5 runs using a linear regression model trained with L-BFGS. Datasets: DIABETES (n 442, d 10). CALIFORNIA HOUSING (n 20,640, = 8), AMES HOUSING (n 2,930, d = 10); each subsampled to 300 training points. Reoo is reported as mean + standard error across the 5 semivalue approximations.
Dataset
Semivalue
R500 (mean + SE)
DIABETES
Shapley
0.99 + 0.01
(-etaa Shapley
0.99 + 0.01
Banzhaf
0.99 + 0.01
CALIFORNIA
Shapley
0.72 + 0.01
(4,1)-Bets Shapley
0.71 + 0.01
Banzhaf
0.75 + 0.01
AMES
Shapley
0.99 + 0.01
(4,1)-Beta Shapley
0.99 + 0.01
Banzhaf
0.99 + 0.01
Table 10: Regression. Robustness scores Rs00 along the MSE-R² convex path, reported as mean + standard error across 5 semivalue approximations.
Dataset
Semivalue
R500 (mean + SE)
DIABETES
Shapley
0.89 + 0.01
(-et)aBeta Shapley
0.89 + 0.02
Banzhaf
0.91 + 0.01
CALIFORNIA
Shapley
0.70 + 0.01
(4.1) (4.1)-Beta Shapley
0.67 + 0.01
Banzhaf
0.81 + 0.01
AMES
Shapley
0.99 + 0.01
(4.1) Shapley
0.99 1 0.01
Banzhaf
0.99 + 0.01
Table Regression, Robustness scores R500 along the MAE-R2 convex path. Mean standard error over the approximations.
Dataset
Semivalue
R500 (mean + sc)
DIABETES
Shapley
0.90 1 0.02
(4,1)-Beta Shapley
0.89 + 0.02
Banzhaf
0.94 + 0.01
CALIFORNIA
Shapley
0.66 + 0.01
(4,1)-Beta Shapley
0.65 + 0.01
Banzhaf
0.72 + 0.02
AMES
Shapley
0.98 + 0.01
(4.1)-Beta Shapley
0.98 + 0.01
Banzhaf
0.98 1 0.01
where 2 denotes the standard 2-simplex. Specifically, we consider the following utility triplets:
Binary classification. Accuracy F1, and Recall (Table 15)
Regression. MSE, MAE, and R² (Table 16)
Multiclass classification. macro-F1, macro- Recall, and Accuracy (Table 17)
For each task, compute the 3D spatial signatures Sw,D R and then approximate Rs00 using the sampling scheme of Remark B.6 with m = 1000 Monte Carlo sampling. Examples of 3D spatial signatures are given in Appendix D.2.
19

---

Published as a conference paper at ICLR 2026
Table 12: Multiclass. Robustness scores R500 along the Accuracy-macro-F] convex path. Semi- values are approximated over 5 runs using MLP (SGD, fixed seeds). Datasets: DIGITS (n = 1,797, d 64, 10 classes). WINE (n 178, d 13, 3 classes), IRIS (n 150, 4,3 classes); each subsampled to 100 training points. Mean + standard error over the 5 approximations.
Dataset
Semivalue
Rsc0 (mean + se)
DIGITS
Shapley
0.78 + 0.03
(4,1)-Beta Shapley
0.75 + 0.04
Banzhaf
0.82 + 0.04
WINE
Shapley
0.64 + 0.05
(4.1)-Beta Shapley
0.61 + 0.05
Banzhaf
0.68 1 0.04
IRIS
Shapley
0.56 0.06
(4.1)-Beta Shapley
0.53 + 0.05
Banzhaf
0.60 + 0.06
Table 13: Multiclass. Mean robustness Rsoo along the Accuracy-macro Recall convex path (mean + SE over 5 semivalue approximations).
Dataset
Semivalue
Rs00 (mean + SE)
DIGITS
Shapley
0.70 1 0.01
(4,1)-Bett Shapley
0.68 1 0.01
Banzhaf
0.76 1 0.03
WINE
Shapley (4,1)-Beta Shapley
0.60 + 0.01
0.57 1 0.01
Banzhaf
0.63 = 0.04
IRIS
Shapley
0.52 + 0.02
(4, (-Btaa Shapley
0.50 + 0.03
Banzhaf
0.56 + 0.03
A.7 RESULTS FOR THE MULTIPLE-VALE UTILITY SCENARIO EXTENDED TO K >> 2 BASE UTILITIES
We extend the multiple valid scenario to K > 2 base utilities in the multiclass classification setting by using the analytical decomposition derived in Appendix C.5. In fact, each multiclass utility can be written as va = - aglle, where u. is a class-wise utility for class c. Particularly, in consider the per-class precision for u, that is defined in Appendix C.5. Hence K C, the number of classes. We evaluate this setting on the three multiclass datasets used for the utility trade-off scenario experiments (namely DIGITS, WINE, and IRIS) and approximate R800 by sampling directions a E SC-1 and applying the approximation scheme of Remark B.6 with m 1000 Monte Carlo sampling. The results are given Table 18.
experiment,
A.8 WHAT IF WE A VARIES INSTEAD OF PERE?
Since u perf A, one can alter the utility either by changing the algorithm or by changing the performance metric perf. Our main study held A fixed and varied perf. To illustrate the effect of A, we run an additional experiment with fixed metric (Accuracy) and two learning algorithms: (i) logistic regression trained with L-BFGS and (ii) multilayer perceptron (MLP) trained with SGD (introducing randomness via initialization and optimization). Table reports the mean Spearman rank correlation (with standard error) between semivalue-baad based rankings obtained across multiple runs with the two algorithms, for each semivalue and dataset.
These results show that rankings vary with the learning algorithm, though not strongly when changing the performance metric (cf. Table in the main paper). We also observe smaller standard emors for Banzhaf than for Shapley (4, -Beta, suggesting Banzhaf rankings less sensitive
20

---

Published as a conference paper at ICLR 2026
Table 14: Multiclass. Mean robustness R500 along the macro Fl-macro Recall convex path (mean + SE over 5 semivalue approximations).
Dataset
Semivalue
R500 (mean + SE)
DIGITS
Shapley
0.71 + 0.01
(4,1)-Beth Shapley
0.71 1 0.01
Banzhal
0.75 1 0.03
WINE
Shapley
0.67 1 0.02
(4,1) Shapley
0.68 + 0.01
Banzhaf
0.77 + 0.04
IRIS
Shapley
0.80 + 0.02
(4,t)-Beta Shapley
0.78 + 0.02
Banzhaf
0.83 = 0.05
Table 15: Binary. Mean robustness Rseo along the Accuracy-FI-Recht convex path (mean + SE over semivalue approximations) for binary classification datasets used in Figure
Dataset
Semivalue
R500 (mean 1 SE)
BREAST
Shapley
0.30 + 0.01
(4.1)-Beta Shapley
0.33 1 0.01
Banzhaf
0.34 0.06
TITANIC
Shapley
0.11 0.01
(4.1)-Beta Shapley
0.11 0.01
Banzhaf
0.13 1: 0.03
CREDIT
Shapley
0.15
(4,1)-Beta Shapley
0.16 0.02
Banzhaf
0.11
HEART
Shapley
0.16 0.01
(4,1)-Beta Shapley
0.16 0.01
Banzhaf
0.83 0.04
WIND
Shapley
0.33 + 0.01
(4,1)-Beta Shapley
0.31 0.02
Banzhaf
0.58 - 0.01
CPU
Shapley
0.21 0.01
(4,1)-Beta Shapley
0.22 = 0.01
Banzhaf
0.29 + 0.03
2DPLANES
Shapley
0.16 1 0.01
(4.1)- Shapley
0.18 + 0.01
Banzhaf
0.22 0.03
POL
Shapley
0.21 1 0.01
(4,1)- Shapley
0.24 0.02
Banzhaf
0.39 0.04
the randomness in the MLP, which aligns with prior analytical and empirical findings (Wang & Jia, 2023; Li Yu, 2023).
A.9
EMPIRICAL LINK BETWEEN THE ROBUSTNESS METRIC Rp AND TOP- STABILITY METRICS (OVERLAP@)) & AND JACCARD@
Appendix B.8 establishes an analytical link between our robustness metric Rp and top-k stability metrics (overlap@/ & and Jaccard k). We complement this analysis with empirical study that directly relates Rd to these metrics, in the same spirit as our comparison between Rp and rank correlation measures (Kendall and Spearman) in Section 4.1.
We consider the family of binary classification utilities spanned by base utilities (A,7) used
21

---

Published as a conference paper at ICLR 2026
Table 16: Regression, Mean robustness scores R600 (# standard errors) along the MSE-MAE- R
convex path. Semivalues are approximated over 5 runs using linear regression model trained with L-BFGS. Datasets: DIABETES (n 442, = 10), CALIFORNIA HIOSSING (n = 20,640, 8). AMES HOUSING (n 2,930, d 10); each subsampled to 300 training points. Raoo is reported as mean standard error across the semivalue approximations.
Dataset
Semivalue
REao (mean SE)
DIABETES
Shapley
0.83 + 0.01
(4,1)-Beta Shapley
0.82 + 0.02
Banzhaf
0.85 + 0.01
CALIFORNIA
Shapley (4,1) Shapley
0.69 + 0.01
0.68 + 0.01
Banzhaf
0.86 1 0.03
AMES
Shapley
0.89 + 0.02
(4,1)-Beta Shapley
0.89 1 0.02
Banzhaf
0.91 1 0.03
Table 17: Multiclass. Mean robustness scores Rsoo (#t standard errors) along the Accuracy-macnt F1-macro- Recall convex path. Semivalues approximated over 5 runs using MLP (SGD, fixed seeds). Datasets: DIGITS (n = 1,797, 64, 10 classes), WINE (n 178, d = 13, classes), IRIS (n 150, d 4, classes); each subsampled to 100 training points. Mean + standard error over the 5 approximations.
Dataset
Semivalue
Rsc0 (mean +: SE)
DIGITS
Shapley
0.63 + 0.03
(4,1)-Beta Shapley
0.61 + 0.03
Banzhaf
0.78 + 0.04
WINE
Shapley
0.44 + 0.01
(4,1) Shapley
0.42 + 0.02
Banzhaf
0.69 1 0.03
IRIS
Shaplcy
0.63 + 0.01
(4,1)-Beta Shapley
0.60 1 0.01
Banzhaf
0.66 1 0.02
to compute Rp in Section 4.1. For one pair of distinct metrics in this family, precisely Accuracy vs. F1 score, we compute the rankings induced by the corresponding semivalue and, for different values of k, we evaluate the associated top-k overlap & and Jaccard @ k between the two rankings. This yields, for each dataset and semivalue, a collection of top-k stability scores that summarize how sen- sitive top-k selections are to switching between these utilities. We then compare these top-k stability scores given Table 20 with robustness scores Ro plotted in Figure and summarized Table 7.
Across most settings, we observe the same qualitative pattern as with Kendall and Spear- correlation metrics: semivalues that achieve larger R, for given utility family (typically Banzhaf) also exhibit higher overlap@k l and Jaccard when comparing the induced rankings under different utilities in that family. In other words, methods that are more robust according Rp are also those whose top-k selections change the least when moving between these equally valid utility choices.
A. OVERALL DISCUSSION ABOUT EMPIRICAL ROBUSTNESS RESULTS
The empirical study presented across the paper provides comprehensive validation of the robustness metric R. introduced in Section 3.2. The empirical results consistently suggest that Rp captures stability phenomena observed when varying the utility across the two scenarios studied: the multiple- valid- -uliitu scenario (Sections 4.1, A.9.A.7) and the utility-trade-op off scenario (Sections 4.2, A.6). We summarize the main conclusions below.
22

---

Published as a conference paper at ICLR 2026
Table 18: Mean robustness scores R500 (# standard errors) for the multiple-valid-uiilee scenario with K > base utilities in the multiclass classification setting, using the class-wise precision decomposition described in Appendix C.5. Semivalues are approximated over 5 runs using an MLP (SGD, fixed seeds). Datasets: DIGITS (n 1,797, d = 64, 10 classes), WINE (n 178, d 13,3 classes), IRIS (n 150, = 4, 3 classes); each subsampled to 100 training points. Mean + standard error over the 5 approximations.
Dataset
Semivalue
Rs00 (mean = SE)
DIGITS
Shapley
0.17 0.03
(4,1)-Beta Shapley
0.19 + 0,02 0.57 + 0.04
Banzhaf
WINE
Shapley
0.21 + 0.01
(4.1)-Beta Shapley
0.20 + 0.01
Banzhaf
0.68 + 0.02
IRIS
Shapley
0.65 + 0.02
(4.1)-Beta Shapley
0.64 + 0.01
Banzhaf
0.70 = 0.03
Table 19: Spearman rank correlation (mean + standard error) between semivalue rankings computed with logistic regression model and MLP, using accuracy the metric. Results are averaged over 5 runs (varying both the MLP imtialization/pptimzzzioon and the semivalue approximation).
Dataset
Semivalue
Spearman (mean + se)
BREAST
Shapley
0.62 + 0.21
(4.1)-Beta Shapley
0.67 0.18
Banzhaf
0.67 = 0.05
TITANIC
Shapley
0.71 0.13
(4.1)-Beta Shapley
0.71 +: 0.07
Banzhaf
0.80 +: 0.03
HEART
Shapley
0.65 1 0.21
(4.1)-Beta Shapley
0.62 1 0.22
Banzhaf
0.93 + 0.07
WIND
Shapley
0.82 + 0.11
(4.1)-Beta Shapley
0.87 + 0.10
Banzhaf
0.85 + 0.03
(1) Agreement of RD with rank- -based stability metries. The robustness metric Rp demonstrates consistency with traditional rank correlation measures (Kendall and Spearman) across all experiments. In the multiple-vos valid-utilin scenario for binary classification (Section 4.1, Tables and 7), datasets exhibiting low rank correlations between accuracy and F1-score consistently show correspondingly low RG values. This alignment extends to top-k stability metrics (Table 20), where higher Rt values correlate with greater overlap@* k and Jaccard@ @ scores when switching between equally valid utilities. This suggests that Rp captures meaningful ranking stability that aligns with practitioner intuition while offering a geometric interpretation.
(2) Banzhaf's consistent robustness advantage. In all binary classification experiments (Tables and 15), multiclass experiments (Tables 12-14 and 17) and regression experiments (Tables 9-11 and 16), the same trend emerges: the Banzhaf value attains very often the highest robustness. The 2D spatial signatures in Appendix D.I provide a geometric insight for this phenomenon: Banzhaf weights concentrate on intermediate coalition sizes, where the empirical alignment factors r, largest (Figure 4), yielding embeddings that are nearly collinear in TR² and thus achieve maximal Ry. In contrast, Shapley and (4, 1)-Bcta Shapley, whose weights are either uniform or emphasize small coalition sizes, produce less aligned 2D signatures and therefore lower Rpp
(3) Extension to > base utilities. The robustness metric naturally extends to higher- dimensional utility families. Our trilit--rade--fff experiments for binary classification with three base
23

---

Published as a conference paper at ICLR 2026
Table 20: Top-k stability metrics (Accuracy vs. FI). Mean standard errors for (0,,20, 50). Higher values indicate greater robustness of top-k selections under utility shifts.
Overlap@k k: 20
Jaccard
Dataset
Semivalue
= 10
k: == 50
h: = 10
k = 20
k = 50
Shapley
0.75 + 0.04
0.62 0.03
0.45 0.03
0.61 0.04
0.47 + 0.08
0.32 + 0.03
BREAST
Beta
0.76 + 0.04
0.63 = 0.03
0.46 1 0.03
0.62 0.04
0.48 + 0.03
0.33 1 0.03
Banzhaf
0.82 1 0.04
0.70 = 0.04
0.52 + 0.03
0.71 + 0.04
0.57 + 0.04
0.40 + 0.03
Shapley
0.25 + 0.08
0.18 = 0.02
0.12 + 0.02
0.15 = 0.03
0.10 0.02
0.07 = 0.02
TITANIC
Beta
0.28 +: 0.03
0.20 = 0.02
0.14 + 0.02
0.17 = 0.03
0.12 0.02
0.08 + 0.02
Banzhaf
0.88 +: 0.03
0.76 = 0.04
0.59 + 0.03
0.79 0.03
0.64 0.04
0.48 + 0.03
Shapley
0.18 + 0.03
0.13 0.03
0.09 + 0.02
0.10 0.03
0.07 107
0.05 - 0.02
CREDIT
Beta
0.22 + 0.03
0.16 0.03
0.11 + 0.02
0.13 0.03
0.09 + 0.02
0.06 0.02
Banzhaf
0.91 + 0.02
0.89 = 0.03
0.76 0.03
0.89 0.02
0.80 0.03
0.61 + 0.03
Shapley
0.65 0.03
0.56 0.03
0,41 0.02
0.50 0.03
0.41
0.29 0.02
HEART
Beta
0.68 + 0.08
0.59 = 0.03
0.43 0.02
0.53 0.03
0.44
0.31 + 0.02
Banzhaf
0.91 + 0.03
0.83 + 0.03
0.65 + 0.03
0.84 0.03
0.72 0.03
0.55 + 0.03
Shapley
0.78 + 0.08
0.68 0.03
0.49 0.03
0.65 0.03
0.54
0.37 + 0.03
WIND
Beta
0.79 + 0.08
0.69 = 0.03
0.50 + 0.03
0.66 = 0.03
0.55 0.55
0.38 + 0.03
Banzhaf
0.92 + 0.03
0.85 = 0.03
0.70 + 0.03
0.86 0.03
0.75
0.60 0.03
Shapley
0.58 + 0.08
0.46 = 0.03
0.33 + 0.03
0.42 = 0.03
0.32 0.00
0.22 = 0.02
CPU
Beta
0.61 + 0.03
0.49 0.03
0.35 = 0.03
0.45 0.03
0.34
0.23 0.02
Banzhaf
0.85 + 0.03
0.78 0.04
0.61 0.03
0.76 0.03
0.65
0.49 0.03
Shapley
0.52 0.03
0.41 = 0.03
0.29 0.03
0.36 0.03
0.27
0.18 = 0.02
2DPLANES
Beta
0.46 0.03
0.32 0.03
0.41 0.03
0.00
0.20 0.02
Banzhaf
0.74 0.04
0.64 = 0.04
0.47 - 0.04
0.60 0.04
0.35 - 0.03
Shapley
0.72 0.03
0.61 0.03
0.43 0.02
0.57 0.03
0.00
0.31 0.02
POL
Beta
0.78 + 0.03
0.68 = 0.03
0.49 - 0.02
0.65 0.03
0.003
0.35 + 0.02
Banzhaf
0.48 + 0.04
0.41 = 0.04
0.29 + 0.03
0.33 0.04
0.27 0.04
0.18 + 0.03
utilitics (Table 15) confirm that the geometric insights from R² hold R3: semivalues with spatial signatures concentrated along dominant axis (Figures 12-19 achieve higher robustness, when the utility varies over the simplex A rather than a line segment. These findings align with the geometric interpretation of Proposition 3.1 (and its K-ddmmnssoonal extension) and the lanking-region counts in Corollary B.4: when the spatial signature is collinear (has dominant direction), the number of ranking regions minimized, leading the maximum average angular distance swap and, consequently, the highest robustness.
24

---

Published as a conference paper at ICLR 2026
B ADDITIONAL PROOFS & DERIVATIONS
For the reader's convenience, we first outline the main points covered in this section.
Appendix B.1: First-order approximation of the utility the multiple-valid-utiiln scenario for binary classification and empirical validation.
Appendix B.2: Proof of Proposition 3.1 and its extension K' base utilities.
Appendix B.3: Ranking region counts for specific cases of spatial signature.
Appendix B.4: Link between the robustness metric Rp and the Kendall rank correlation.
Appendix B.5: Closed-form for [a(pp(a))-
Appendix B.6: Maximum average p-swaps distance occurs under collinearity of the spatial signature.
Appendix B.7: Proof of proposition 3.3.
Appendix B.8: Link between the robustness metric Rp and top-k stability metrics (overlap@a @ and Jaccard@k).
B. FIRST-ORDER APPROXIMATION THE UTILITY IN THE multiple-valid-uriityy SCENARIO FOR BINARY CLASSIFICATION AND EMPIRICAL VALIDATION
This section justifies the approximation used in Section 3, where a freet--raatinaa utility function replaced by its affine surrogate.
Formally, we state in Section that any frcetr-factional utility u of the form equation 2 with do * admits first-order (Taylor-Young Young) expansion around (X, 7) (0, 0) of the form
cu Cids do cod1 X(S) + do Cu d2 (S) + o (
u(S) do
di
di
where (eo: C c22 do, d d2) are real coefficients which specify the particular utility.
The proof is direct Taylor expansion of u at (A, 7) = (0, 0). followed by an empirical validation of the affine surrogate by inspecting discordance rates.
Proof. Define (AA,)) 7) = co + c and D(A,7) = do + d1 X + d2 so that u(S)
(()(S).((5) with
N(A.7)
f(A,)) = D(A,))
Assuming dg * 0 (i.e., the denominator does not vanish at (0, 0)). the first-order Taylor expansion of around (0, 0) is
0f
0f
((A7)=((0.0))
A+
r o(I(AA,)))))
ax
(0.0)
dy
(
(0.0)
Concretely,
Co
0f
Ci do codi
0f
c2do cod2
f(0,0) 0) do
0 (0.0)
dj
Oy (0.0)
dg
Moreover, since all norms are equivalent in R?, the Euclidean norm [I(A, (()l is equivalent to the infinity norm max(A, 7). This concludes the proof.
To verify that the affine surrogate faithfully preserves the true utility's induced ordering, we compare de datasets introduced in Section A. and for each of the three
rankings under u and under its first-order proxy â 20 + dido-cadi X cado y. For each of the eight public semivalues (Shapley, (4, 1)-Beta Shapley, and Banzhaf), proceed follows:
of
25

---

Published as a conference paper at ICLR 2026
1. Exact ranking. Compute semivalue scores by using the exact Aheer-frractoons utility u, then sort the resulting scores to obtain a reference ranking r of the n data points.
2. Affine surrogate ranking. Replace the utility with its first-order affine approximation around (A.7) = (0, 0) denoted as ü, compute semivalue scores, and sort to obtain an approximate ranking
3. Discordance measurement. For each pair of rankings (r, f). count the number d of discordant pairs (i.e., pairs of points ordered differently between the two rankings), and record the proportion d/N, where N (2)
4. Repetition and averaging Repeat steps 1-3 several times, each time using an independent Monte Carlo approximation of the semivalue scores, capture sampling variability.
Table 21 reports, for each dataset and semivalue, the average proportion of discordant pairs (# standard error) between rankings obtained with the exact fnnaarfractional utility and its first-order affine proxy, for both Fl-score and Jaccard coefficient (see Table 22 for their definitions). In all experiments, the sum of the mean discordance rate and standard error never exceeds 2.3%.
These discordance rates, at most few percent of all (%) pairs, confirm that, in practice, the omitted higher-order terms of the utility have only a minor effect on the induced semivalue ranking. Conse- quently, using the affine surrogate instead of the exact linear-fractional form is reasonably justified whenever one's primary interest lies in the ordering of data values rather than their precise numerical magnitudes.
Table 21: Proportion of discordant pairs (# standard error) between rankings induced by the exact finato-fractionl utility and its first-order affine surrogate ú, for F1-score and Jaccard utilities. Values are computed over N (2)) 1225 pairs and averaged over Monte Carlo trials.
Dataset
Fl -score
Jaccard
Shapley
(4.1)-Beta Shapley
Banzhaf
Shapley
(4.1)-Beta Shapley
Banzhaf
BREAST
0.8% (0.1%)
0.8% (0.2%)
0.9% (0.1%)
0.7% (0.1%)
0.9% (0.1%)
0.9% (0.2%)
TITANIC
1.3% (0.3%)
1.3% (0.3%)
0.8% (0.3%)
1.6% (0.4%)
1.3% (0.3%)
0.7% (0.1%)
CREDIT
1.5% (0.5%)
1.6% (0.2%)
1.0% (0.1%)
1.5% (0.3%)
1.7%()..%))
0.7% (0.3%)
HEART
1.0% (0.1%)
0.8% (0.1%)
0.8% (0.1%)
1.2% (0.2%)
1.1% (0.3%)
0.7% (0.2%)
WIND
1.0% (0.2%)
0.8% (0.1%)
0.8% (0.2%)
0.9%(..1%)
1.2% (0.4%)
1.0% (0.4%)
CPU
1.6% (0.5%)
1.2% (0.2%)
0.7% (0.1%)
1.3% (0.2%)
1.3% (0.2%)
0.9% (0.1%)
2DPLANES
1.7% (0.1%)
(09%)).1%)
0.8% (0.1%)
103%(..1%%
(0.2%)
1.1% (0.4%)
POL.
1.8% (0.4%)
2.0% (0.2%)
1.5% (0.5%)
2.1% (0.2%)
2.0% (0.2%)
1.6% (0.5%)
B.2 PROOF OF PROPOSITION 3.1 AND ITS EXTENSION TO K BASE UTILITIES
This section provides the formal proof of Proposition B.I, which generalizes Proposition 3.1. It states that the semivalue score of any data point under utility that is linear combination of base utilities can be written as an inner product in IR This result forms the backbone of the geometric perspective developed in Section
Proposition B.1 (Extension of Proposition 3.1 to K > 2 base utilities). Let 'D be any dataset of size n and let w E R be a semivalue weight vector. Then there exists a map thw,D : D - R& such that for every utility th = Ek= 0'g o(2; w. Ma) = (Vw,p(=), for any E D. We call Sw,D (()) E D) the spatial signature of D under semivalue w.
The proof is a straightforward application of semivalue linearity. The main contribution is the geometric interpretation of semivalue vectors as projections.
Proof. For each data point z E DD let its semivalue characterized by w be denoted by so(:; w; u.) when the utility is ua. Under the standard linearity property of semivalues, the following linear decomposition holds:
K
o (zii, (2;w,ua) =0(=;u,
ano(z;w (x)
k=1
26

---

Published as a conference paper at ICLR 2026
So if we define for each 2,
(z) = (d(2 uI),..., ((s,,,,, RR
then by definition of the scalar (inner) product in IR @ ua) (VW, (2).
In the main text, we focus on the case where K = 2, i.c. utilities correspond to directions on the unit circle St. Proposition B.1 shows that the same reasoning carries over to any finite family of K base utilities: data points embed as (2 E RK and ranking by a convex combination Uo k== 0g is equivalent sorting the inner products (z), a). Since only the direction of a matters, each utility is identified with point ā l//|| on the unit sphere Sk-1, Thus, for general K, robustness to utility choice reduces to studying how the ordering of these projections varies as moves over S
B.3 RANKING REGIONS COUNTS FOR SPECIFIC CASES OF SPATIAL SIGNATURES
This section formalizes the notion of ranking regions, which play a central role in the robustness analysis developed in Section 3. We begin by considering the hyperplane arrangement induced by all pairwise differences between embedded data points in the spatial signature. This arrangement partitions space into connected components, referred to as regions in the theory of hyperplane arrangements (see Definition B.2). In our context, each such region corresponds to a set of utility directions under which the ordering of data points remains constant. We refer to these as ranking regions.
Definition B.2 (Region of a hyperplane arrangement), Let V be finite arrangement of hyperplanes in a real vector space V. The regions of A are the connected components of
U H.
HEA
Each region is the interior of a (possibly unbounded) polyhedral cone and homeomorphic to V. We denote the number of such regions by r(A).
We now specialize Definition B.2 to our data valuation setting. Let D = { 2n be a dataset and let (2,) R denote the embedding of each point under semivalue weighting w. For each pair i < J. we define
Hij - (a (a,
(a, (l,) - ((,)())))) = =0
Each set Hij is defined the kernel of the linear functional + (a, tw, (zz)). Since (2) + (zz))(z)) for + j (unless the data points are embedded identically), the difference vector (2,) - Vi p(z)) E R is nonzero. Therefore, this kernel is a linear subspace of codimension one in R which, by definition, origin a = 0g; it is thus central hyperplane by definition.
hyperplane. Moreover, each H: contains the
Each hyperplane Hij is the set of utility directions that assign equal projection scores to points zi and zj- The finite arrangement Aw,D = {H, 1 < < S n) then induces a collection of regions in the sense of Definition B.2, partitioning R into open cones such that, in each region, the relative ordering of projected values (a, D.. and (a,, (2,)) remains the same for all i < Therefore, each region determines unique ordering of the embedded points, corresponding to a distinct way of ranking the data points of 'D based on utility direction. To study robustness with respect to directional changes, project this arrangement onto the unit sphere SK Since all hyperplanes are central, their intersection with the sphere produces great spheres, and the resulting decomposition of S consists of spherical connected regions over which the ranking of the data points remains invariant, We refer to these regions as ranking regions. Formally, a ranking region is a connected component of SK 1 U.zz (Hi Sk-11)
We now study how the number of such ranking regions depends the geometry of the spatial signature. In particular, using Proposition B.3, we provide an explicit count of ranking regions two specific geometric configurations of the embedded points.
27

---

Published as a conference paper at ICLR 2026
Proposition B.3 (Regions counts). Let A (Hi...., Hm ) be arrangement of m central (i.e.,
origin-passing) hyperplanes in a real vector space of dimension K.
1. If no K hyperplanes in the arrangement intersect in a common subspace of dimension greater than zero (in particular, not line), then the number of regions into which A partitions V is
K
r(A) =
Σ k=
m 7
2. If all hyperplanes coincide (i.e., H1 = mm) then the number of regions is:
r(A) = 2.
Proof. Let A {H,,..., Hm) be an arrangement of m hyperplanes in a real vector space of dimension K.
Suppose no K hyperplanes in the arrangement intersect in a common subspace of dimension greater than zero (in particular, not in line).
Choose any hyperplane E .., and define two affine hyperplanes H+andd H parallel to H and opposite sides of the origin, such that the origin lies strictly between them.
Each the remaining m - hyperplanes A intersects H+ in hyperplane of dimension hyperplanes in H (which is space of dimension K - 1). By Proposition 2.4 in Stanley (2007) (derived from Zaslavsky's work Zaslavsky (1975)), the number of regions induced by this non-central* arrangement in H is:
K - and these intersections form an arrangement of
Σ
(
These regions correspond exactly the regions of V Une A H that lie entirely on one side of H. By symmetry, the same number regions lies the opposite side (on HT). Therefore, the total number of regions for the whole arrangement is:
((A) = 2 Σ
m
2. Suppose that all hyperplanes in the arrangement coincide, i.e., H, some hyperplane H C V. Then
H = for
-
HEA
and the complement V UHE E consists of exactly two connected open sets: the two half-spaces determined by H. Therefore, the number of regions is
r(A) = 2.
We now apply Proposition B.3 to the arrangement Aw. formed by the hyperplanes H. defined from pairwise differences of embedded points the spatial signature Sw, Since each H.y is a central hyperplane in K the arrangement Aw. partitions the space into open polyhedral cones, whose connected components are the regions of the arrangement. Each of these cones intersects the unit sphere KK unique open subset, yielding spherical partition of SK-1 Therefore, the number of ranking regions on SK is equal to the number of regions of the central hyperplane arrangement K and can be computed directly using Proposition B.3.
Corollary B.4 provides the number of ranking regions for two specific geometric configurations of spatial signature.
*innee the 1 hyperplanes do not all pass through same point H
28

---

Published as a conference paper at ICLR 2026
Corollary B.4 (Ranking regions counts). Let D f21,..., denote the spatial signature of point 2, under semivalue weighting w. For each pair < j. define the hyperplane
2n | be a dataset and let (2s) RR*
Hi - fa== (a, (w,z2)) - (w.)) = 0}.
Since there are (2) pairs (i,j), the arrangement Au,D (Ha
< n} consists of N = (2) central hyperplanes RK Let r(), denote the number of connected regions in the complement of this arrangement. Then
1. no K hyperplanes Hij intersect in common subspace of dimension greater than zero. the number of ranking regions is
r(Au,)) = Σ
K-1
N 1
where N =
12
k
k=0
2. If all embedded points (zz) lie line in I then all hyperplanes Hig coincide and
r(Au, D) = 2.
Figure 2 illustrates these two specific geometric configurations on the circle S' (corresponding to the case K 2). In both cases, the observed number of ranking regions coincides with the counts given by Corollary B.4.
B.4 LINK BETWEEN THE ROBUSTNESS METRIC Rp AND THE KENDALL RANK CORRELATION
there are no tied ranks, the Kendall rank correlation between two orderings of points is defined as - 1 2D N. where D is the number of discordant pairs and N (2) is the total number of pairs. Since crossing one ranking region swaps exactly one pair, each such swap increases D by one and thus decreases by 2/N. Consequently, p swaps lower the correlation from
38 Therefore, R. captures how far in expectation one must move from a utility direction before the Kendall rank correlation degrades by at least 2p/N.
However, this statement only holds in the setting where no ties occur. In practical scenarios involving ties, the degradation in T can be either smaller or larger than what Rd would suggest. The purpose of this subsection is to explain why worse- cersethhn-eeye expected degradation possible, which is the main risk when interpreting RS through the lens Kendall correlation in practice.
The Kendall rank correlation between rankings A and B is defined
C
VNN (N- tA) (N -N-tB)
where e is the number of concordant pairs, d is the number of discordant pairs (c and d count only untied pairs), (2) the total number of pairs, LA (resp. (B) the number of tied pairs in ranking A (resp. B).
Performing pairwise swaps among tied items can amplify the degradation of T beyond the idcalized 2p/N amount (derived under the no-ties assumption) due to two effects:
Resolving ties i.e.. decreasing A or t. increases the factors N or N - tg and thus the denominator. For fixed numerator d, this directly reduces the magnitude of т Critically, even as e d decreases (due to increased discordance), the growing denominator further exacerbates the decline.
Swapping two items within a block of k tied points can order up to (t) formerly tied pairs at once. If these newly ordered pairs are discordant, a single swap increases d by up to (+). rather than just
Consequently, when many tied groups exist, one might observe after p swaps,
2p
AT
N
i.e., larger drop in rank correlation than in the no-ties case.
29

---

Published as a conference paper at ICLR 2026
B.5 CLOSED-FORM FOR Ea [a))]
This section provides the derivation of a closed-form expression for Elpp], introduced in Section 3, which quantifies how far, average, one must rotate the utility direction the sphere before p pairwise ranking swaps occur. In Section 3, we describe how this quantity captures the local stability of the ranking induced by the spatial signature. Here, we formally compute this quantity in the case where K = 2 (i.e., in the case where the utilities we consider can be written as a linear combination of two bass--tilities). We also show that the closed-form expression derived in the K = 2 case can be computed in Onn" log n) which no closed-form is available, and describe how Elpp] can be approximated via Monte Carlo sampling.
time. Finally, we briefly discuss the higher- nigher-dimensional case K > 2, for
Recall that Pp(ā) measures the minimal geodesic distance one must rotate a utility direction à E SK-1 before the ranking of points in D = {2s JEjn] changes by p pairwise swaps. Each pair of points (23, 2j) defines cuts on S which are utility directions along which the scores of 2, and 2y are equal. These cuts partition S into (ranking) regions where the ranking of points remains fixed. In what follows, we focus on the case K 2, where utility directions lie on the unit circle S! and Pp (a) can be treated function of the angle associated with ā E St.
We parametrize the unit circle S by the angle E 10, 2 writing @({)) = (cos pp sin E St. Since Pp depends only on this angle, we abbreviate Pp (a(()) by Pp({p) Equivalently,
2T / Pr(())dp
1
2x
E En-Unit(S)) [pp(a)] En--nii (v))
Recall from Section 3 that the 2NV emphcut angles 01 S .... 02N < 2 partition the interval [0, 2T| into ares of lengths 2 0x+1 0& (with 02N+1 01 + 2T). Within each arc, the ranking remains fixed, and crossing into the next are incurs exactly one additional swap.
For yp E (0k, 1k+1), the function Pp({)) equals the minimum of the clockwise and counterclockwise distances to the p-th next cut:
0p(x)) min ( (p) (p 02). S (p) + (vp 0z)),
where the quantities St (p) and Sj (p), recalled from Section 3, are defined as
(t) = Σ A(k+)) mod 2N. SE(P) =
A mod 2N-
==1
Hence, the average value of Pp can be written as
Ea(pp(a)] = _ h0-0
2x Po(x)de
2T
1 2N
Σ
Po(p)dip
1
2N
min (p) t, ST (p) + fJdt,
2T
Oz
2n
where we set / = 9 - 0g [0, Au] as local coordinate that measures the angular distance from the left endpoint of the k-th arc.
The expression inside the integral reflects the shortest of two angular paths along the circle from the start of the k-th arc: one going clockwise (of length St (p) t) and one counterclockwise (of length ST (p) + f). These two expressions intersect at
St (p)
S (p)
Intuitively,
If 17 < 0, even at / 0. the clockwise path is already shorter, so pp(() = S (p) / for all E (0, Au).
If 2 Ag. the counterelockwise path is shorter throughout the entire are, (t. = Sp (p) +t for all & E [0, Ax).
30

---

Published as a conference paper at ICLR 2026
1f0 < Ak. then for t ty the counterclockwise path is shorter, and for t tg the clockwise path is shorter.
We therefore split S po(t)dt - min (p) - St (p) + tJdt into the three cases:
1. 1f t I O, we have
J min - (,5,,), +tddt =
/
d
1 (s - t)dt =
j
2
2. If 1/ 2 Ac. we have
L. min
1,0)-1,5pp (p) - 1]dl / Ae
(s) + t)dt ST(Dde =
jo
3. 1f0 4 < An. we have
% min (p) - L, 51 (p) + 1jdl = 1"h (5% (p) + +]dd + (hi
(5 /
-idl
-5 + (12) + + (p) - -
2 (A - (11)
Putting these three cases together and summing over yields a piecewise-defined expression for the integral each are. Precisely, plugging these into the expression for [ā]) in Eq. equation we finally obtain the closed-form
2N
E. [ap(aa)
Σ Tk.
2TT
where 4s denotes the value of the integral over the k-th arc, defined as
Au
T =
min (p) 1, S, (p) t)dt,
and can be computed using the following case distinction
(p) 2
if f S if t 2 Ak. if < < A
1/ = (p) + 222 S (p) + = + (p) - -2) - 10 (t
with o (p)
previously defined.
Remark B.5 (Computational cost). The closed-form expression for E[pp] can be computed in O(n log n) time. First, computing the 2N = (n²²) cut angles defined by all unordered pairs of points (2) requires O(n²) time, since each involves simple trigonometric operation in R². Sorting these angles define the are intervals costs O(n² log n)). Once sorted, the distances S (p) and ST (p) to the p-th next and previous cuts can be computed efficiently for all using sliding windows indexing O(n2) time. The final step, i.e., evaluating the integral over each of the 2.V ares, also takes Onn²) time. Thus, the total computational cost is O(n² log n). dominated by the sorting step.
Remark B.6 (Case K 2). The above closed-form derivation relies the fact that utilities correspond to angles on S (i.e., = 2). When > 2, utilities lie on the higher-dimensional sphere -K In that setting, incur p swaps, but the integral Ba-Uni(( [āp admits no simple closed-fome expression. In practice, one must approximate it numerically by Monte Carlo sampling. Specifically, for any unit vector ā E SK-1 each pair (i,j) defines great-sphere
still define (ā(ā) the minimal geodesic distance on SK-1 to
Hi (B gK-1 (3,0g) 0} Vij ) (x2)).
31

---

Published as a conference paper at ICLR 2026
The shortest geodesie distance from ā that cut is given in closed form by
def (ā) arcsin (ā, /t//|/ubll>\
Thus, by getting all N (")) distances {d.z) sorting them, and picking the p-th smallest, obtain Py(ā). Repeating for many independent Unif(SK-1) gives a Monte Carlo estimate of the average.
Let Am == 1 2 (a) with i..dd draws ( ~ Unif(Sk-1), and let == Es [pp(a)] Since
0 S Pp(@) I 2 Hoeffding's inequality gives, for any 8 E (0, 1),
IP (iim 1l
log(2/6) 2m
I 6.
Equivalently, to guarantee
Im ml S with probability least 1 8, it suffices that
2
log 8
B.6 MAXIMUM AVERAGE P-SWAPS DISTANCE OCCURS UNDER COLLINEARITY THE SPATIAL SIGNATURE
This section provides the theoretical justification for the claim made in Section 3 that the average distance (alp is maximized when the spatial signature is collinear, and equals π //4 in this case.
Recall that the spatial signature is the set of embedded vectors
S (t), E TR D),
where thu (z) reflects the contribution of each data point D family of K base utilities, weighted by the semivalue coefficients This embedding allows utility directions ā SK-1 to induce rankings via projection. The quantity (ā) measures the minimal geodesic distance on the sphere 5 one must rotate ā before the ranking changes by pairwise swaps. We focus here on the case K 2, where utility directions lie the unit circle S and show that the maximum of Elpp] is achieved when all embedded points lie on a common line through the origin. This derivation provides an upper bound for [[pp] and motivates the normalization in the robustness metric Ri (see Definition 3.2).
Each pair of points (Z); 2z) induces cul the circle St, namely the two antipodal points where (a,, (2, - (w. = O. When all embedded points (22) lie on single line through the origin, Corollary B.4 states that there is exactly one cut (of multiplicity N = (*) which splits S into two open ares, each of length æ. Within either are, no swaps occur until one crosses that cut, at which point all N pairs flip simultaneously. Concretely, for any direction angle f E [0, x Pp corresponds to the shortest angular distance this cut, either clockwise or counterclockwise, and is thus given by
Pp( min -0,00,00000000 0},
Hence,
Exle,(a))
f"m 2T
pr(v)de
/ 10, 0)d0 2x
_
1
π 4
2
Now, any deviation from perfect collinearity introduces distinct cuts, which only further subdivide those two r-length ares into shorter pieces. Shorter maximal arc-lengths imply a smaller average distance to the nearest swap, for every spatial signature and every 1 & N.
Bspp]
with equality if and only if the signature is exactly collinear.
Remark B.7 (Case > 2). For K 2, perfect collinearity of the spatial signature still maximizes the average p-swap distance, but the value max E, [pp(ā)] must be evaluated numerically since for Sw.
K the distribution of angular distances from a uniformly random point on the sphere to fixed
great great sub-sphere no longer admits simple elementary integral like for K 2.
32

---

Published as a conference paper at ICLR 2026
Remark B.8 (Lower bounds for Ealpp(ā))). Trivially, since Pp(ā)
for all a,
Ea (a)] 2 O, vp
2
instead we assume the spatial signature to such that all N = (2) cuts on S are distinct, then Proposition B.3 states that there exactly 2.N positive-length easy to see that considering all ways to choose { summing to 2n, the configuration Aς- = x/N for all & minimizes the average Pp- Concretely, for AE= x/N we find
of total length 2x. In this case,
Blpl
1 2T
2N
R/N
P
1)
(w 2) 2
Hence, under the distinet cuts assumption,
B[pp(()) 2 p 1 N'
with equality exactly when the 2NV cuts are perfectly equally spaced.
In this special setting, where all (2) cuts on S are distinct, one could alternatively define the robustness metric as
(S Ex -p (p- 2 E [0,1].
7 2)
However, since in practice cannot detect a priori that this condition cuts holds, instead the general robustness metric Rp as given in Definition 3.2.
B.7 PROOF OF PROPOSITION 3.3
In this section, we provide the detailed proof of Proposition 3.3. Recall that for any utility u,
((2,w,)) Σ Wy uz)
Ø=1
where A, u) is the marginal contribution of 2; with respect to coalitions size j - By definition of the covariance,
(ov()(A),(1)) 1 n
Σ (d(=sw) ((()) (((a)))) o(7)),
where o(-) denotes the mean over i. Using billinearity of covariance, we get
Cov(o(A),
wywx Cov(A,(A), Ae(T)).
j=l k=
where
M...., (y(A) (A) A), .., M,....,,((E))))) (,(iiW,))) and 4y(7) = (A) Ayerr,
Under the assumption Cov (O,(() An(7)) = 0 for all j + k. only the A terms remain, giving
Cov()(A). Σ w ((11
Similarly,
Var (o())) = Σ w/ Var (A)))) Var(d(1)) = -
w/ Var (A,())))))
j=1
j=1
33

---

Published as a conference paper at ICLR 2026
By the definition of Pearson correlation,
n wj (2)
Corr(6(A),((7))
wj (a)(
wj Var(A,
with Cov (oy(A)()),AA(1)) = Corr(A, becomes
A, ( (ar(A, Var (y Then, the correlation
COIr(A (A), As(r))) Var( (ar(A (2)) Var(4,(7))
Corr(((A), = =>u² w/
Var(A,
w Var(A,(1))
Var (A)) Var (A, can be understood as the effective alignment of marginal contributions coalition size j across the two utilities À and y. Specifically, Corr(A (A), (n)) measures how similarly data points' marginal contributions at size j - 1 move under À compared to under and (Vr((A; Var(A, down-weights sizes j where marginal contributions are nearly constant (and thus uninformative) for either utility.
Each term r == (o)
B.8 LINK BETWEEN THE ROBUSTNESS METRIC Rp AND TOP-K STABILITY METRICS (OVERLAP@) k AND JACCARD@ k)
In this section, we give an analytical link between p and top-k overlap/laccard stability metries which definitions provided in Appendix C.6.
Let D be training set of size Let and u' be two utilities (linear combinations of base utilities) that induce rankings æ and n' on DD and assume и and n differ by at most swaps. Let
(kk
((k) ((k)
oo S() ,ي)
denote the top-k sets under u and u/ respectively. We then recall from Appendix C.6 the definitions of top-k overlap@ @ k and Jaccard@k:
- Top-k overlap@k:
Overlap@kku,,u)) |s n
s( [0,
E
k
Jaccard
|sth)) (kk wl
Jaccard@k(u, u')
10.
( U (())
Since both sets have cardinality k, this simplifies to
| n (s(k) (k) Jaccard@k(u, w') 2k 1S (((1))
Now let A = s(k) and B = ((k) Let L be the number of points that leave the top- set when moving from to B; then JAnBI = L. and the symmetric difference has size |AAB = 2L. Now, data point can enter or leave the top-k set only if its rank crosses the boundary between positions and + 1. This can happen only when we perform swap involving the items currently at ranks & and & + 1. Each such boundary swap change the membership of most the two swapped items. Let be the number of boundary swaps among the swaps. Then at most 2b distinct items change membership. Since b < p, most 2p distinet items can change membership. Because
34

---

Published as a conference paper at ICLR 2026
|AAB| 2L, get 2L I 2p, hence L S Therefore, JAn k: - L2 p. This yields the
following deterministic bounds (for p I k; otherwise they become vacuous):
Overlap@k(u,ul) u') AAUB
p
k
K
and since |AUB| = |A| + B JAn B = 2k IAn B). we have |AUB| S 2 (k: p) = k k.
A M B k k-p
Jaccard@kku,u)
A JAUB B
k+p
Now, by Definition 3.2, for each utility direction ā E SK-1, (a) is the minimal geodesic distance such that moving by (a) the sphere produces exactly p swaps in the ranking induced by a. So, for a fixed and k, larger Rp x [ā)] means that one must move farther in average in the utility space before reaching regime where top-k overlap/Jaccard can be low these bounds.
35

---

Published as a conference paper at ICLR 2026
C
ADDITIONAL DEFINITIONS
For the reader's 's convenience, we first outline the main points covered in this section.
Appendix C.I: Some definitions of linear fractional utilities.
Appendix C.2: Rank correlation metrics (Kendall & Spearman).
Appendix C.3: Axioms satisfied by semivalues.
Appendix C.4: Applications of semivalue- data valuation methods.
Appendix C.5: Extension of the multiple-valide utility scenario to multiclass classification
metrics.
Appendix C.6: Top-k stability metrics (overlap@A @k & Jaccard @ k))
C.1 SOME DEFINITIONS OF LINEAR FRACTIONAL UTILITIES
Below, we give the concrete coefficients (co, C1 02) and (do, d1, d2) for several commonly used finear-facctioal performance metries. Each of these metries can expressed in the form
Co c1)(S) + c2>(S)
u(S) = do + di ((5) + d2)(55)
as recalled from equation 2.
Table 22: Some examples of linear fractional utilities. For more examples, see Choi al. (2009). We set
- [J = 1J. the proportion of positive labels in Deest-
m
Utility
(co, 02,
(do, d2)
Accuracy
(1 (1-x22, 2,
(1,0,
F3-score
(0,1+
(,11,011)
Jaccard
(0,1,0)
(T, 1)
A.--meauree
( I-x 157
2
(1,0,0)
C.2 RANK CORRELATION METRICS (KENDALL & SPEARMAN)
Let X = (I1, .g,..., and Y (y1..... Un) be two real-valued score vectors on the same items. And let Tx and Ty be their induced rankings. Rank correlations measure monotonic relationships between relative ordering xx and Ny.
Definition C.1 (Kendall rank correlation). Define the set of all pairs of distinct indices P = {(i,))
1 I j S n). For each (i. P, call the pair concordant if (xr x1)(1;; yÿ) 0, discordant if (x; 23 yi) < O, and tie in X (resp. Y) if T; Iz (resp. y: yj))
Let e the number of concordant pairs, the number discordant pairs, and (x (resp. ly) the number of ties in X (resp. Y). Then, the Kendall rank correlation 7 is
d
(C))
- tx
ty
which simplify to if there are no ties (1x = by 0).
Definition C.2 (Spearman rank correlation). Let T (i) be the rank of x, in X and likewise (i) for Y. Define the -ank-differences d, xx (i) Ty (i). The Spearman rank correlation is the Pearson correlation of the ranked vectors:
Σ (x - xx) (i) Ty)
8
T
) (TY (i) )2
36

---

Published as a conference paper at ICLR 2026
where x 2 (i( and my
Ty (i). If there ties, it simplifies to
i=
E dc n nnn - 1)
Both metrics lie in [-1, with +1 indicating perfect agreement and -1 perfect reversal.
C.3 AXIOMS SATISFIED BY SEMIVALUES
Semivalues as defined in equation satisfy fundamental axioms that ensure desirable properties in data valuation. We formally state these axioms in the following. Let o(,, w; be semivalue-based data valuation method defined by a weight vector w and let u and v be utility functions. Then, ø satisfies the following axioms:
1. Dummy, (f((S((=)) u(S) for S ç 22 and some e R, then (,=)w,))
c,
2. Symmetry. If (5(S (z1}) w {z)}) for all S S DM 2j}, then ((ziw 6(2z;,,u) u).
3. Linearity. For any a1, 02 E IR, p ar + a-v)) = a o10 u) + a20(2; v).
While all semivalues satisfy the above axioms, Data Shapley uniquely also guarantees efficiency: EEED ((2, w. u) = =
C.4 APPLICATIONS OF SEMIVALUE-BASEE BASED DATA VALUATION METHODS
In practice, semivalue-bart based methods are mostly applied to perform data cleaning or data subser selection (Tang et al., 2021; ;; Pandl et al., 2021; Bloch & Friedrich, 2021; Zheng et al., 2024). Both tasks involve ranking data points according to their assigned values.
Data cleaning. Data cleaning aims improve dataset quality by identifying and removing noisy or low-quality data points. Since semivalue-basee methods quantify each point's contribution to downstream task, low-valued points natural candidates for removal. Specifically, common approach is to remove points that fall into the set Nr, defined as the subset of data points with the lowest values (Ghorbani & Zou, 2019). Formally, NV, (z; D ((z;; u, w) S T), where 7 is threshold determined through domain knowledge or empirical evaluation.
Data subset selection. Data subset selection involves choosing the optimal training set from available samples to maximize final model performance. Since semivalues measure data quality, prioritizing data points with the highest values is a natural approach. Consequently, a common practice in the literature (Wang & Jia, 2023; Jiang et al.. 2023; Wang al., 2024b) is selecting, of data points with top-k data values, i.e., s(k) A(
given size budget k, the subset ((k)
arg maxsgp.js = Exes 0(2;,,,0) M(u.w))
C.5 EXTENSION OF THE multiple-valid utility SCENARIO TO MULTICLASS METRICS
Let y = (1,..., K} the class set and gs be the model trained on S. For each class k, define the one-vs-rest confusion counts the test set:
TP; # = k; gs(r) = k), FN₂ # k. g3(x) k), FPx = (y * k. gs(x) k),
and the class supports nu TPx FN (true instances of class k) and fu = TPp + FPx (predicted as class k).
Recall.
The per-class recalls form the K-vector
TPR
TPk
r(S) =
E [0, 1]
T*(S)= = TPR + PN Tk
37

---

Published as a conference paper at ICLR 2026
Any average recall can be written dot product with weight vector Ax Σx wk = 1}:
{w E R
(S) = (we, r(S)).
Two common choices immediate:
macro-recall: "aarro 1,
wgt weighted-reeall: w
Σ
1==1 T4
Thus macro- and weighted-recat are the same linear functional applied to the per-class recall basis r(S) with different w.
Precision and Fj.
Analogously, define the per-class precisions
TPR
TP₁
P(S) = (.)))),..PP(((S)) (S(S): = TP + +PR
and per-class Fi's
(S)(S)r)(S) fx(S) == (S( + (S) (with fu = 0f pk + + = 0),
f(S) = (f1...., fk).
Macro/weighted versions are again linear averages over the same class basis:
prec = (w, P(S)) (S) (w, f(S)), w E AK.
with u maccro and nst defined as above.
Implication for our framework. Let the class-wise utilities be ( == (S) (or uf pree (S) (S) u (S) = fa(()) Then any macro/weighted multiclass metric convex combination
(S)
Wk (R))
E AK-
By linearity of semivalues,
@ ws, uw)
Σ wy d (z;;w,u))
so the spatial signature lives in R to all convex mixtures w E ÁK is therefore K K-utility instance and Rp is computed via the Monte Carlo procedure on SK-1 described in Appendix B.5.
k with coordinates given by the class-wise utilities. Robustness
C.6 TOP-k STABILITY METRICS (OVERLAP@ & & JACCARD@K)
Given two rankings и and n' over the same dataset {-1, Zn J. we denote by Top; the set of the & highest-ranked points under Two standard metries then commonly used.
Top-k overlap@k. The overlap@k between π and is defined as
Overlap@k(T, m') (T) n (Top 1].
It measures the fraction of items remaining in the top-k set when switching from ranking to another.
Top-k Jaccard@a The Jaccard@k similarity normalizes the overlap by the size of the union of the two sets:
(oop (T) n Top (T')
Jaccard@k(T, x')
[Top, (T) U Top, (T') E [0, 1].
It provides scale-free measure of agreement, where 1 indicates identical top-k selections.
38

---

Published as a conference paper at ICLR 2026
D ADDITIONAL FIGURES
D.1 ADDITIONAL FIGURES FOR K BASE UTILITIES
In Section we plot the spatial signatures for the wIND dataset (Figure 1) to illustrate the geometric
mapping at the heart of our framework. Figures 5, 6, 7, 8, 9, 10 and 11 present the analogous plots for the remaining binary datasets introduced in Table
44.44423
(a) Shapley
(b) (1, 1)-Beta Shapley
Banzhaf
Figure 5: Spatial signature of the BREAST dataset for three semivalues (a) Shapley, (b) (4, 1)-Beta Shapley, and (c) Banzhaf. Each cross marks the embedding (z) of data point (with #1 U2 ?), the dashed circle is the unit circle S and the filled dot indicates utility direction ā.
(a) Shapley
(b) (4, 1)-Beta Shapley
(c) Banzhaf
Figure 6: Spatial signature of the TITANIC dataset for three semivalues (a) Shapley, (b) (4, 1)-Beta Shapley, and (c) Banzhaf. Each cross marks the embedding (u, of a data point (with и1 = X, U2 = 7), the dashed circle is the unit circle S and the filled dot indicates one utility direction a.
D.2 ADDITIONAL. FIGURES FOR K BASE UTILITIES
In this section. we visualize, in three dimensions, the spatial signatures associated with the utility trade-off experiments for binary classification with K base utilities (Accuracy, F1, Recall) described in Appendix 6.6.2. These plots (corresponding to Figures 12, 13, 14, 15, 16, 17, 18 and 19) provide the geometric counterpart of the robustness scores reported in Table 15.
39

---

Published as a conference paper at ICLR 2026
0.80
502
-023
34-0
(a) Shapley
(b) (4, 1)-Beta Shapley
Banzhaf
Figure 7: Spatial signature of the CREDIT dataset for three semivalues (a) Shapley, (b) (4, 1 Shapley, and (c) Banzhaf. Each cross marks the embedding (2) data point (with #1 U2 = ?), the dashed circle is the unit circle 8 and the filled dot indicates utility direction á.
(a) Shapley
(b) (4, 1)-Beta Shapley
(c) Banzhaf
Figure 8: Spatial signature of the HEART dataset three semivalues (a) Shapley, (b) (4, 1)-Beta
Shapley, and (c) Banzhaf. Each cross marks the embedding (2) of data point (with #1 = X. U2 = 7), the dashed circle is the unit circle S and the filled dot indicates one utility direction a.
(a) Shapley
(b) (4, 1)- Shapley
(c) Banzhaf
Figure 9: Spatial signature of the CPU dataset for three semivalues (a) Shapley, (b) (4, -eta
Shapley, and (c) Banzhaf. Each cross marks the embedding (z) of a data point (with # = X, U2 = 7), the dashed circle is the unit circle S! and the filled dot indicates one utility direction a.
40

---

Published as a conference paper at ICLR 2026
-201
(a) Shapley
(b) (4, 1)-Beta Shapley
Banzhaf
Figure 10: Spatial signature of the 2DPLANES dataset for three semivalues (a) Shapley, (b) (4, 1)-Beta Shapley, and (c) Banzhaf. Each cross marks the embedding pw, (z) of data point (with #1 u2 ?), the dashed circle is the unit circle 8 and the filled dot indicates utility direction á.
Paggh Aic whatt
#6 AM.
(a) Shapley
(b) (4, 1)-Beta Shapley
(c) Banzhaf
Figure 11: Spatial signature of the dataset for three semivalues (a) Shapley, (b) (4, 1)-Beta
Shapley, and (c) Banzhaf. Each cross marks the embedding (2) of a data point (with #1 = A. U2 = 7), the dashed circle is the unit circle S and the filled dot indicates one utility direction a.
(a) Shapley
(b) (4, -)-B Shapley
(c) Banzhaf
Figure 12: Spatial signature of the BREAST dataset for three semivalues (a) Shapley, (b) (4, Beta Shapley, and (c) Banzhaf. Each blue points marks the embedding (w, (z) of a data point E D (with in = accuracy, U2 = fl, Us recall). The represented sphere is S
41

---

Published as a conference paper at ICLR 2026
(a) Shaplcy
(b) (4, 1)-Beta Shapley
(c) Banzhaf
Figure 13: Spatial signature of the TITANIC dataset for three semivalues (a) Shapley, (b) (4, 1e-a Shapley, and (c) Banzhaf. Each blue points marks the embedding (w, of a data point & E D (with i1 = accuracy, U2 = fl, Uy = recall). The represented sphere is S
(a) Shapley
(b) (4, 1) -)-Beta Shapley
(c) Banzhaf
Figure 14: Spatial signature the CREDIT dataset for three semivalues (a) Shapley, (b) (4, Beta Shapley, and (c) Banzhaf. Each blue points marks the embedding (2) of data point D (with accuracy, W22 f1, Ug recall). The represented sphere is S
(a) Shapley
(b) (4, Be-tt Shapley
(c) Banzhaf
Figure 15: Spatial signature of the HEART dataset for three semivalues (a) Shapley, (b) (4, -eta Shapley, and (c) Banzhaf. Each blue points marks the embedding Dz) of a data point E D (with in accuracy, W2 = fl, ug recall). The represented sphere is S
42

---

Published as a conference paper at ICLR 2026
be
(a) Shapley
(4, 1)-Beta Shapley
Banzhaf
Figure 16: Spatial signature of the WIND dataset for three semivalues (a) Shapley, (b) (4, -eta Shapley, and (c) Banzhaf. Each blue points marks the embedding (), of data point D (with #1 accuracy, U2 Ug recall). The represented sphere is S
(a) Shapley
(b) (4, 1) Beta Shapley
Banzhaf
Figure 17: Spatial signature of the CPU dataset for three semivalues (a) Shapley, (b) (4, Beta Shapley, and (c) Banzhaf. Each blue points marks the embedding (2) of data point D (with #1 accuracy, U2 fl, us recall). The represented sphere is S
(a) Shapley
(b) (4, 1)- Shapley
(c) Banzhaf
Figure 18: Spatial signature of the 2DPLANES dataset for three semivalues (a) Shapley, (b) (4, Beta Shapley, and (c) Banzhaf. Each blue points marks the embedding (w, )) of a data point 2 E D (with in = accuracy, U2 = fl, Us recall). The represented sphere is S
43

---

Published as a conference paper at ICLR 2026
(a) Shapley
(b) (4, 1) 1)-Beta Shapley
(c) Banzhaf
Figure 19: Spatial signature of the POL dataset three semivalues (a) Shapley, (b) (4, 1)-Beta Shapley, and (c) Banzhaf. Each blue points marks the embedding (2) of data point D (with accuracy, W22 = f1, u3 recall). The represented sphere is S
44


Reviews:
[{"review_id": "iDZIYjwzro", "reviewer": "Reviewer_MwCd", "summary": "This paper provides an in-depth study of stability of semi-value based data valuation methods when there are shifts in utility. The authors  introduce the concept of spatial signature, where a training data point is projected on to a low-dimensional space. The authors assess the robustness of semi-value using a geometric representation of the utility space, specifically a unit circle, such that the metric quantifies how much utility direction must be rotated before the ranking undergoes a significant change. The authors use a geodesic angular distance over the unit circle, and the robustness metric Rp is measured as normalized minimal geodesic distance of rotation needed to achieve p swaps in ranking. The paper offers an evaluation of their metric against data valuation methods Data Shapley, Data Banzhaf and Beta Shapley.", "questions": "1. I’d be interested in the authors' perspective on whether the robustness metric reflects an inherent property of the data valuation framework, or if it is more influenced by the dataset and the choice of utility function. For example, in imbalanced binary classification tasks, the rankings generated by different utility functions (e.g., F1 score vs. accuracy) can vary substantially. How does this variation impact the stability of data rankings and the robustness metric? And if it is affected by dataset and utility functions, then in what setting do the authors advice using this robustness metric in practice ?\n\n2. How do the authors propose choosing p (especially as a factor of the size of the training dataset) for a given dataset?", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The introduction of a novel geometric perspective on and a formal robustness metric for semi-value based data valuation offers a way to benchmark data valuation frameworks and test their stability. In real-world datasets, the utility function can change anytime, reinforcing the importance of this metric. \n2.  The authors provide a rigorous evaluation of their metric on existing data valuation methods and utility functions. \n3. The paper also offers insights into why some valuation methods tend to collinearize the spatial signature and in doing so achieve higher robustness to utility shifts. \n4. The robustness metric proposed is shown to be computationally efficient via the closed-form expression for the expected minimal geodesic distance needed to induce p pairwise swaps in ranking.", "weaknesses": "1. The paper focuses on utility functions specific to binary classification and raises the question of whether the findings will translate to other utility functions / learning tasks. Similarly the evaluations focus on two utility systems, and multi-utility systems are yet to be explored.\n2. Benchmarking the stability of data valuation methods is not new. The key contribution of the paper is the geometric perspective - and this paper would be strengthened by additional experiments that show the true value of this geometric nature of the stability problem. While this paper discusses collinearity of spatial signatures leading to better robustness, the experimental validations focus on the robustness metric and not the geometric properties (eg. how the spatial signatures are distributed for a given valuation framework / dataset or the practical impact of this collinearity in geometric structure).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7qJRnG9Vai", "reviewer": "Reviewer_X13z", "summary": "This paper investigates how sensitive semivalue-based data valuation scores are to the practitioner’s choice of utility function. It introduces the dataset spatial signature, an embedding in ℝ² where any candidate utility function corresponds to a linear functional. In this space, ranking data points with a semivalue reduces to projecting the dataset signature onto a direction that represents the chosen utility.", "questions": "1. Currently, empirical studies of multiple-valid-utility systems are mainly conducted in binary classification. It is also recommended to conduct multiple-valid comparisons of \"multiple equally-valid indicators\" in multi-class classification, rather than just trade-offs (e.g., macro-F1 vs macro-Recall vs Accuracy). 2. While $R_p$ provides a theoretically elegant proxy for stability under utility shifts, the paper would be significantly strengthened by connecting $R_p$ to downstream selection stability. It is recommended to report the relationship between top-k overlap@k/Jaccard@k and $R_p$.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is clearly written and easy to follow.\n\nIt presents an elegant and intuitive geometric representation that translates variations in utility functions into simple linear projections.\n\nThe proposed method is validated across a wide range of datasets and semivalues (Shapley, Beta Shapley, and Banzhaf).\n\nThe authors have thoroughly addressed the major limitations of the previous version by adding regression and multiclass tasks, introducing a formal robustness metric, and conducting experiments to assess algorithmic modifications.", "weaknesses": "I have reviewed this paper before, and the authors have resolved most of my confusion, but I still have a remaining question.\n\nThe experiments on multiple-valid-utility scenarios remain somewhat limited and would benefit from further expansion.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ufrHEG6yWf", "reviewer": "Reviewer_t7pG", "summary": "The authors propose a unified geometric framework based on the spatial signature and also the robustness metric, designed to help data valuation practitioners assess the sensitivity of semivalue-based data valuation results to the choice of utility in both the multiple-valid utility and utility trade-off settings. They evaluate the effectiveness of the proposed metric through experiments on multiple real-world datasets, considering both the multiple-valid-utility and utility trade-off scenarios.", "questions": "How does embedding the data point to a low dimensional space affects its value and ranking?\n\nIs there a specific reason why authors typically restrict the use of regression problems to utility trade-off scenarios, rather than including them in multiple-utility analyses? \n\nWhy do Tables 9–14 only consider p=500?\n\nHow distinct are the observed patterns with the robustness metric from those obtained using Kendall or Spearman correlations? In what ways is the robustness metric a great addition to the data valuation practitioner’s toolbox?\n\nIn what ways do dataset characteristics, data preprocessing choices, and algorithmic choices, e.g., chosen classification/regression algorithms, affect the results? \n\nWhy restrict the multiple-valid-utility scenario to binary classification problems?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Authors present another perspective on how the choice of the utility affects the resulting data values and rankings during data valuation.  \n\nTo mitigate the noise and randomness introduced by Monte Carlo sampling, they propose the use of aligned sampling.\n\nThe theory and empirical setups are well-written and easy to follow. \n\nAuthors conduct multiple experiments to assess the efficacy of their metric, and also make their code available.", "weaknesses": "Restricting the utility to a 2D space spanned by two fixed base utilities u1 and u2 (and extension to class-wise utilities ) makes the robustness metric hard to scale.\n\nIt's possible that some information is lost when embedding data to a low dimensional space. It's unclear to me how this affects the results.\n\nThe data values from Monte Carlo approximations in regression problems tend to be unreliable due to greater variability in utility estimates when sample sizes are small. So, is there a specific reason why authors restricted the use of regression problems to utility trade-off scenarios, rather than including them in multiple-utility analyses?\n\nThere appears to be a misalignment between the claims made in the introduction and the empirical setup. In particular, the treatment of utility trade-offs in the experiments does not accurately reflect the notion of trade-off presented in the introduction.\n\nThere are very few results presented for the utility trade-off scenario, with none in the main paper and few in the appendix. Also, the results relegated to the appendix are neither discussed nor explained. For instance, why do Tables 9–14 only consider p=500? And why are only regression tasks used?\n\nWhile the robustness metric could, in principle, address the question “How robust are my data valuation results to the choice of utility?”, it is unclear how distinct the observed patterns are from those obtained using Kendall or Spearman correlations. For instance, based on the results for both Kendall and the robustness metric, the cases where Banzhaf outperforms Shapley appear identical. The authors note that, in the absence of ties, the robustness metric “captures how far, in expectation, one must move from a utility direction before the Kendall rank correlation degrades by at least 2p/N.” While this is insightful, the results presented in the tables and figures reveal only broad trends, which seem largely similar across both metrics.\n\n\n\nThe authors briefly mention in the appendix that “one can alter the utility either by changing the algorithm A or by changing the performance metric,” an important consideration also discussed in several of the cited works. In my opinion, this is a strong consideration, and seeing how the robustness metric performs across datasets would have been helpful. However, the results presented in Table 15 are reported using Spearman correlation rather than the proposed robustness metric. Also, these results are not discussed, and key details of the experimental setup are not given, such as the number of runs. \n\n\n\nThe authors should consider adding a discussion section. Overall, the experimental results are insufficiently discussed, making it difficult to highlight the main contributions of the proposed metric, both theoretical and empirical.\n\n\n\nMinor.\nThe references section needs a second look. Cite the published versions of the papers where they exist and not their arXivs. E.g., Jiachen T. Wang, Tianji Yang, James Zou, Yongchan Kwon, and Ruoxi Jia. Rethinking data shapley\nfor data selection tasks: Misleads and merits, ICML’24", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)