Model Shapley: Equitable Model Valuation with Black-box Access
Authors: Xinyi Xu
Thanh Lam
Chuan-Sheng Foo
Bryan Kian Hsiang Low
Venue: NeurIPS 2023 poster (2023)
Keywords: Model Valuation
Dirichlet Abstraction
Shapley Value
TL;DR: We propose model Shapley, an equitable model valuation method for machine learning models with only black-box access.

Abstract: Valuation methods of data and machine learning (ML) models are essential to the establishment of AI marketplaces. Importantly, certain practical considerations (e.g., operational constraints, legal restrictions) favor the use of model valuation over data valuation. Also, existing marketplaces that involve trading of pre-trained ML models call for an equitable model valuation method to price them. In particular, we investigate the black-box access setting which allows querying a model (to observe predictions) without disclosing model-specific information (e.g., architecture and parameters). By exploiting a Dirichlet abstraction of a model’s predictions, we propose a novel and equitable model valuation method called model Shapley. We also leverage a Lipschitz continuity of model Shapley to design a learning approach for predicting the model Shapley values (MSVs) of many vendors’ models (e.g., 150) in a large-scale marketplace. We perform extensive empirical validation on the effectiveness of model Shapley using various real-world datasets and heterogeneous model types.

Model Shapley: Equitable Model Valuation with Black-box Access
Xinyi Xu Thanh Lam', Chuan-Sheng Foo", Bryan Kian Hsiang Low
Dept. of Computer Science, National University of Singapore, Republic of Singapore
Inst. for Infocomm Research; Centre for Frontier AI Research, A STAR, Republic of Singapore xinyi xu@u, nus edu, lamchi thanh19970gmail. com foo www.ccuunnsseengee . edu. sg, lowkh@comp nus edu.sg
Abstract
Valuation methods of data and machine learning (ML) models are essential to the establishment of AI marketplaces. Importantly, certain practical considerations (e.g., operational constraints, legal restrictions) favor the use model valuation over data valuation. Also, existing marketplaces that involve trading of pre-trained ML models call for an equitable model valuation method to price them. partie- ular, we investigate the bluck-box access setting which allows querying a model (to observe predictions) without disclosing model-specific information (c.g., ar- chitecture and parameters). By exploiting Dirichlet abstraction of model's predictions, we propose a novel and equitable model valuation method called model Shapley. We also leverage Lipschitz continuity of model Shapley design a learning approach for predicting the model Shapley values (MSVs) of many vendors' models (e.g., 150) in large-scale marketplace. We perform extensive empirical validation on the effectiveness of model Shapley using various real-world datasets and heterogeneous model types.
1 Introduction
Data valuation methods have important roles in extensive applications such dataset pricing in data marketplace []] evaluating the contributions of data from multiple collaborating parties [71], and identifying valuable data filtering out less valuable data) for training higher-quality machine learning (ML) models [20). However, the following practical considerations favor the use of model valuation over data valuation: (A) Data valuation can be operationally infeasible due the character- istics of training data, such as being massively distributed over millions of sources [73], enormous in size (e-g., 400 billion byte-pair-encote
tokens for a language model (7), and/or transient (e.g., the data for online learning are not stored persistently [14)). In contrast, trained model is not distributed by nature (and hence does not need be aggregated from distributed sources), often much smaller in size than the training data (e.g.. less than 1%) and usually stored persistently for inference.
(B) Data privacy regulations (e.g., GDPR) can legal impediment to data valuation in designing fair payments for ML-baaed collaborations (e.g., in medicine (16) or cyber-defense (25) because the centralization of (possibly) private data, which seems necessary for data valuation [39], is prohibited by such regulations. In contrast, the ML models trained private data without centralization [40] can be available for valuation. (C) Existing marketplaces that involve trading of pre-traines ML models (e.g., AWS marketplace, Modzy) call for equitable model valuation to price them. These practical considerations motivate the use of equitable model valuation over data valuation.
'The GPT-3 language model contains 175 billion parameters trained 45 TB of text data 77] It is of size 1.75 10 (2 x 2) x 350 GB (at 16 float precision) which less than 1% training data size.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

For model valuation, practitioners (e.g., model vendors or clinicians who use the trained models) would likely prefer their models to be examined via only black-box access i.e., by querying the model with input for the corresponding predictions without observing its internal mechanism [8. It does not disclose the proprietary model information (for model vendors) nor the sensitive information contained in the model (for clinicians), and provides an added advantage of the model valuation being model-aggostic since no model-specific information is used. However, these make model valuation challenging by restricting the available model information to only selected query set (e.g.. a validation set) with the observed predictions. In contrast, with white-box access, there is more model information available (e.g., informatiin-thhorrtiic criteria for probabilistic models [71] norms of the parameters for deep neural networks (22)) to assess model complexity or certain analytical properties (e.g., uniqueness of an optimal model in logistic regression).
Intuitively, the value of model depends on its intended task: For example, model trained to classify MNIST digits [45] is not very valuable to a clinician trying to classify diagnostic scans. This dependency is useful in practice for selecting the most 'valuable' model for the desired task (i.e., query set). We refer to task and query set interchangeably hereafter. While the predictive accuracy of a model on the task provides an intuitive value [66], it can be too reductive: Suppose that two models M, and M2 have identical predictive accuracies on the task but M, (M₃)) makes highly (barely) certain predictions, i.e., M, (M₂') predicts the true class with over 90% (barely over 50%) probability. Intuitively, the values for M, and M, should not be the same, which cannot be achieved with accuracy alone, hence suggesting that additional model information beyond accuracy is required. (1) What then should be suitable abstraction of model w.E.. task, for model valuation?
Satisfying certain equitability properties model valuation is imperative in the application of model pricing to ensuring fair market. For instance, consistent valuation of identical models is important as it is unfair to price them differently otherwise. Furthermore, the market economy dictates that the value of model depends on other available models (e.g., more available substitutes cause depreciation), which is important to guarantee a fair market by preventing price fixing (i.e., an exploitative pricing scheme often made illegal by anti-trust laws). (2) How then should model valuation be formulated to satisfy these equitability properties?
In particular, the Shapley value is shown to satisfy these equitability properties but raises another practical challenge that computing it exactly incurs O(2N) ) time where N the number of models in a marketplace. So, for a given computational budget, there is fundamental ceiling to the size of the marketplace such that including more models causes the marketplace to be unable to determine their values (within reasonable time). (3) How can the desirable equitability properties of the Shapley value still be exploited without imposing a significant restriction on the size of the marketplace?
This paper presents a novel model-agnostic valuation framework to tackle the above challenges. For (1), we use an insight that the predictive pattern of a model (w.r.t. task) is an suitable abstraction for valuation, especially since the black-bor -bok- access rules out other model information. Specifically, we use a Dirichlet distribution to approximate model's predictive pattern/distribution w.r.t. a task, which we call the Dirichlet abstraction of this model to encode both its predictive accuracy and certainty (Sec. 2}. We describe how to adjust the level of abstraction to trade off the amount of model information (i.e., higher abstraction level) for the availability of a smaller query set. Then, for (2), we observe that ensuring equitability requires a similarity measure of the models (e.g., to ensure identical models are valued consistently or to identify substitutes). We exploit the ability of the Dirichlet abstractions to preserve the similarity between models for proposing model Shapley as an equitable model valuation (Sec. 3 As an illustration, identical models produce identical Dirichlet abstractions which result in equal model Shapley values (MSVs). To address (3), based on the Dirichlet abstraction, we leverage Lipschitz continuity of model Shapley to justify and propose a learning approach of training a model appraiser (i.c., a regression learner) on a small subset of models (and their MSVs) for predicting other models' MSVs to validate model Shapley's practical feasibility in a large-scale marketplace (Sec. as empirically verified on real-world datasets and up to 150 heterogeneous models. We empirically validate that better predictive accuracy (e.g., F1 score) due to better training data, more suitable model types, and/or higher predictive certainty result in higher Ms,s, and demonstrate use case for identifying valuable subset of models from the marketplace to construct a larger learner (e.g.. random forests) (Sec.
'This is different from black-bor box models (e.g., deep neural networks) which are difficult to interpret or explain even with access to the internal mechanism such architecture and parameters.
2

---

2 Dirichlet Abstraction of a Model
We give some preliminaries the Dirichlet distribution and Hellinger distance below
Definition 1 (Dirichlet distribution (63 The probability density function of a C-dimensional
Dirichlet random variables Z ~ Dar(a)) with parameters = [a1,.... ac] E (0, oc) is p(2; a) =
I .4 (orTR/(([Eee [ r(ax)] / EE } where is the gamma function.
Definition 2 (Hellinger distance (27). The Hellinger distance between distributions (whose proba- bility density functions are) p and is du (p. == [1 f Vp(x) g(z) dz]
2.1 Dirichlet Abstraction and MLE
We will now describe an abstraction of a model via a Dirichlet approximation to the model's predictive pattern over some task and a maximum likelihood estimation (MLE) for Each (learnt) C-way classification model M; X + ACC) is mapping from the input space X' to the (C- (C'11) probability simplex (CC) (i.e., space of C-dimensional probability vectors). Denote random variable X ~ Px whose support supp X)) X is the input space, then its distribution Px induces a predictive distribution (M,(X) over A (C)). Concretely, Px is represented by a task/query set D = {ay E X. E (I....,Cl)) C)));=...... (1, is realization of (M,(X)
where each 2, is realization of X Px, so that
Dirichlet abstraction. Note that the predictive distribution PM.(X) (induced by Px) (i) has the exactly same support to that of C-dimensional Dirichlet distribution, so (M,(X) can be mathematically modeled using a Dirichlet distribution; and (ii) can be statistically modeled using a Dirichlet distribution because the predictive pattern of classification model can be (statistically) characterized well with a Dirichlet distribution (68] Informally, the "shape" of predictive distribution of classification model is similar to that of Dirichlet distribution. Hence, let (M,(X) Q = Dir(o.), whose parameters o; can be learnt using MLE (described later) and refer to a or Q as M,'s Dirichlet abstraction. In words, we abstract the predictive distribution (x)( of the model M, induced by Px into Dirichlet distribution, hence the name Dirichlet abstraction. Note that the notational dependence on Px (or D) is suppressed when the context is clear.
The proposed Dirichlet abstraction offers several important advantages (further elaborated in Sec.B) (a) By design, this formulation replaces the heterogeneity in models with the homogeneity their Dirichlet abstractions, and allows the subsequently proposed model valuation to be applicable to models of different types: The respective Dirichlet abstractions of multi-layer perceptron and a logistic regression can be compared directly. (b) Q: encodes the predictive accuracy and certainty of M. through a theoretical connection between the Hellinger distance du of two Dirichlet abstractions and the cross entropy of M; (formalized by Proposition 2 (c) Importantly for model valuation, an appealing analytic property of the Dirichlet distribution is that du of two Dirichlet abstractions can be evaluated in closed-for -ooee-fommmaa and incurs ((1) computational complexity.
MLE. We adopt the MLE approximation a using the predictions of M. on D since the log- likelihood function is concave with a unique maximizer and can thus be efficiently optimized (57). Since (I,)) denotes a realized predictive probability vector of (X) ~ Dir(as), we use the ob- served sufficient statistics log h D
= log (z, (i.e., with an element-wiss log operation)
to derive the log-likelihood as follows
36
F(a, ha) == D G(a)) (((a)) + De (ax - 1) log Tu.
where G(a.) log I (E an) Σx log [(ax). From (). Tú arises as alternative (to ax)) abstraction of M, w.r.t. D. Hence, compare az and h, theoretically (Proposition 3 in App. and empirically (Sec. 4.2)
Additional discussion on the choice (e.g., comparison with the Chernoff distance) and suitability of Hellinger distance is provided in App.[A
3

---

2.2 Class-specific Dirichlet Abstraction
Since the Dirichlet abstraction Qi of M; w.I.t some task D does not explicitly account for the class information in D (i.e., the true classification label y of each data zz), models with certain (different) predictive patterns can be indistinguishable. This can be problematic because the Dirichlet abstractions of optimal model and another model with zero predictive accuracy but specific predictive pattern can identical, making it difficult to distinguish between these two models.
Suppose that there are C = 3 classes in balanced query set D (i.e., equal data size for each class). For some M; (with optimal predictive accuracy), artificially construct M2 to have the same Dirichlet abstraction, but zero predictive accuracy in the following way: Since D is balanced, group the data into triplets ((2)11 22,2, 25,.)]j=1..... 1/////// for the input data from 3 classes. Next, define (Z,,e) == (T mod 8)+1) for e 1,2,3. Intuitively, for each prediction (i.e., a input
3- 3-dimensional probability vector in a 2-simplex) by M, M, makes an identical one but data from a wrong class, i.e. we 'shift' M,'s 's predictions by one class to construct the predictions of My. Obviously, the predictive accuracy of M, is zero, but its predictions (ín aggregation) are identical to those of M, which means h; = f2 in ( hence resulting in Q: = Qv. In Fig. Plots 1 and 2 are the predictions of M, and M, respectively, on D, which are (visually) indistinguishable. Plots 3 and are samples from Q and Q2 respectively, which are also (visually) indistinguishable. The implication is that, in this case (of M, and such specially constructed M,),, ), the highest level of abstraction (i.c., using the entire query set D to construct Dirichlet abstractions Q, and Q2)) ) is not effective distinguish between i and i. Hence, we adopt a lower level of abstraction, descrihed next.
The remedy the so-called class-specific Dirichlet abstraction: Partition the query set D _ Dx where D contains only data from the k-th class. The Dirichlet abstraction D, of M, on D2 is called the class-specific Dirichlet abstraction w.r.t. class k. Based on the example above, we verify using small experiment Over the entire D. we obtain Or; Og = [0.5040, 0.5339, 0.5306]. Restricting to query set DD Trom class 1 only, gives D,DD: = [21.8601, 2.2005, 2. 2.0215] and ag. [1.5817, 1.7576, 19.4037]. which are clearly different. In Fig. I Plots 5 and 6 are the predictions of M, and Mg. respectively, on D1 while plots 7 and 8 are samples from QuP, and Q D,, We see that M, is clearly different from My, and Qs, D, is clearly different from Qv, demonstrating the effectiveness of a lower level of abstraction via the class-specific Dirichlet abstraction.
2
5
6
Figure 1: The triangles denote the 2-simplex and each dot is a 3-dimensional probability vector. Plots 1-4 (5-8) use D (Di) as query set. Plots 1&5 (2&6) show predictions of M, (Mp). Plots 3&7 (4&+8) show randomly drawn samples from Dirichlet abstraction Q2 (Qu). For example, plot 4 shows random samples from Q7 w.r.t. D, while plot shows predictions of M; w.r.t. Dj.
Trade-off between level of Dirichlet abstraction and size of query set. Note that we need not stop at partitioning D at the class level. For instance, more refined partition can be first according to the classes and then certain groups of input fcature values. Doing so produces Dirichlet abstractions with a lower level of abstraction (i.e., less abstract and containing more model information), but also requires a larger query set D to begin with so that each smaller partitioned query set contains sufficient data (for predictions) to obtain accurate MLE estimate For extremely refined partition of 'D where each partitioned query set is very small (e.g., size S 5), the obtained MLE estimates and the corresponding Dirichlet abstraction can be inaccurate and thus not useful. It is thus important to find a suitable trade-off between the abstraction level vs. query set size. In particular, in Sec. 5 we observe that partitioning D according the classes provides such suitable trade-off:: E.g., it can distinguish models with almost equal predictive accuracy due to the high class imbalance
'On balanced query set of size D 900, Mx,x) == Gr[ = c) + ce]um12.3 with c. 21(0, 0.2) and normalizing constant G,, i.e., M, always makes the correct classification but with a small additive and independent uniform noise fc. 'The work of 33) shows sample complexity of [2) being polynomial in C and additive error.

---

in the data but different F1 scores. Since the Dirichlet abstraction and the class-specific version are both Dirichlet distributions, and share the same theoretical properties (e.g.. enabling a closed-form expression of the Hellinger distance), subsequently, we refer to Dirichlet abstractions generally (i.e., omitting the specific dependence on 'D or Dz), unless otherwise specified.
Equitable Valuation via Model Shapley
We discuss and formalize several equitability properties to derive a general formulation called model Shapley to satisfy them. Then, we will specify the characteristic function and a precision-weighted fusion to encode a model's predictive accuracy and certainty into its MSV.
3.1 Equitability Properties and Model Shapley
Consider N models in a marketplace and denote M,'s equitable value by di. (PI) If a model M; has not been queried at all, then its value is indeterminate and we set o; = 0 by default. (22) If two models M, and Mr give identical predictions (over the task), then their values are equal, i.e.
Φ; = te. (P3 If some buyer is interested in multiple tasks simultaneously and a model (from some vendor) performs very well on only one of the tasks, then is unfair for the vendor set the value/price solely based on this performance. Instead, an equitable value should be based on the model's joint performance on these tasks (e.g., a linear combination according to the buyer's interests in these tasks). (P4) The existence of perfect substitutes depreciates the value of model. (PI-PP4) are useful for equitable valuation in ML model marketplaces [5]]6]] 666. To formalize these properties, some notations are needed: Let v: 2 + denote characteristic function (specified later) s.t. v(C) quantifies the value of a collection C C [N] = (1.....N of models to capture the dependence of M,'s value on other existing models My (e.g., substitutes). Denote M, value by
Φ + Φ (e(N)) which is fully specified (up to linear scaling) by the properties:
P1 Null player: (VC C [M] \ {i} V(U (i)) - v(C) = 0) - =0
P2 Symmetry: (VC C [N] (1,7)
= - -
P3 Linearity: Vy,7' E R (VDUD == "D + - S. (DU'') = (D( + (:(D")
P4 Diminished marginal utility: Add perfect substitute (i.e., duplicate/copy) M. of M, to the pool of N models already containing M, and denote the new pool by [N'] = [N] {ic]. Denote the value of M, w.r.t. [N] by Φ; and w.r.t. (N') by &/. Then, of Φ1-
Proposition 1. Properties P1 P2 and P3 fully specify ((-) up to a linear scaling Z: Φ, = L==ZEccIIVe we [m(Cu (i)) - v()) where we |C]l (N |C| 1)/NN.
(2)
Its proof follows directly from 200 Proposition 2.1]. Since 2 coincides with the Shapley value [69]. we refer to & as model Shapley and o; as the model Shapley value (MSV). Note that P3 requires two distinct tasks D and D' to distinguish between the MSVs of M, w.r.t. these two tasks. P4 additionally requires v to be conditionally redundant copy M., (conditioned benefit of adding M. Intuitively, as adding M, is already sufficient for the desired task, subsequently adding M., does not yield (as much) extra benefit (24). In contrast, [51] also adopts (2) but assumes v already exists while we explicitly design v to encode predictive accuracy and certainty:
5 in App. A): The benefit of a model M; already being added) is not more than the initial
(CC) -d( Q*)
where Qe is the precision-weighter fusion of the Dirichlet abstractions in C. and d(-, -) is distribu- tional distance measure between two Dirichlet abstractions. Recall from See. 2 that the predictive distribution of M, is represented in its Dirichlet abstraction (visualized in Fig. []. In particular, the more accurate M, is, the closer (distributionally) Q; to the Dirichlet abstraction Q" of an expert (i.e., optimal classifier). Hence, a (high) similarity between Q and Q" can suggest M, 's (high) predictive accuracy. Specifically. Q" implemented as follows: For an input data-label pair (zz, Uj), ), take the one-hot encoded vector EyG E 10 of yj- add an independent uniform noise Ej [0, 0.01J (to avoid degeneracy during MLE), normalize it to sum to 1 to yield Ey,.e,
$98. 22% of the data are from only 3 out of the 23 classes. VvC S [N]] \. (i) v(Cu v(C) 2 vCC ie) VCU (i)) given that i $ [N].
5

---

'prediction' of the expert, and solve ① using these "predictions' over D. The predictive certainty of M, is encoded in the precision la,)1 of its Dirichlet abstraction, which is then used a weight to fuse the individual Dirichlet abstractions Q: in C to obtain Qc:
Definition 3 (Prictsion-weighheed Fusion). Let the random vector [B1...., 3x0l ~
Bir([[a]11).... (amll)) be independent of M,(x) for all i C [N] where Then, the Drecision-weigheee fusion Qu is the distribution of E (,)
|C|.
In Definition 3, the weight Br on (x) is large if |ay]1 is large, i.e., Q; has high precision/M, has a high predictive certainty. Definition 3 has an important implication: Qe is a fully specified
Dirichlet, i.e., Qc Dir(1_ Oii,1,..... En i=1 oz,c)) (Lemma 3 in App. A with four advantages: (A) Q and Qe are both Dirichlets so that a single d (e.g., dy) in 3 can be used for both singleton and non-singleton C's. Interestingly, gives a perspective that each M, lives as Q: in a metric space w.r.t. du (Fig. [ in App.
C. (B) We can theoretically justify learning model Shapley (Theorem []. (C) (3) using du can be evaluated in closed form with (1) time, which is important since (2) requires ( evaluations of (C) for different C's. (D) An alternative to specify v(C) is linear combination of the performance of M,, Vi C vC). However, it is unclear what the weights in
this linear combination should be. In contrast, Definition 3 'automatically" resolves this issue by
fusing each Qi according to its precision |ox:1 into Qc.
Interpreting MSV. Note that (2) is an average (over all possible C S [NT {i}) of how much M; (or, more precisely, Q:) improves the distributional similarity between Qe and Q" (i.e., the expert) after i joins C. Both the predictive accuracy and certainty of M, can affect (2). To see this, a high predictive accuracy of M, implies that Q; is (distributionally) close to Q" a high predictive certainty of M, ensures its weight 3; is large when fused into Qc, so the predictive certainty can amplify M,'s effect in bringing Qe close to Q" (if Q; is close to Q*). ). Hence, a model M, with a high predictive accuracy and certainty is likely to have a high MSV d; When considering several models jointly (66], we can use 6; to indicate how well (on average) M, combines with other models (i.c., whether M, joining C' leads to a performance improvement),
verified in Sec. 5 In contrast to the work of [66], which supports up to N = 32 simpler binary classifiers, our approach -more scalable and general- supports up to N 150 C-way classifiers (Sec. 4.2)
3.2 Connection to Cross-entropy and Other Model Evaluation Criteria
We first formalize the connection between cross-entropy (CE) and our approach that uses the Hellinger distance, and then use this connection to extend our approach to other evaluation criteria such as adversarial robustness 43
distributional robustness (67.. and algorithmic fairness in MI. [56).
As a distributional distance measure, CE can be used specify (3) because the CE loss is used to evaluate the performance of a model. Then, specifying (3) with CE or the Hellinger distance (as proposed in Sec. 4 can be connected in how they evaluate the model performance.
Proposition 2. Let VCE(C) == -CE(Qc, Q*) (C) = -dH((c Q*) and H(-)) be differential entropy. Then,
(CB(C) - (C) if HQ (Qc. Q*) 2 0, and VCE(C) 2 consto. x where const, [log(1/ (og((/qquin qmin 1)] 24min) with Juin = min, ((2) for density 9-
I(C)
Proposition shows that (C) has upper and lower bounds that are monotonic in 1/i providing some justification for using UH (instead of PCE). Note that while Proposition 2 utiliize in due to a
key Lemma 2 (in App. A). we adopt ²и == -da (Sec. D as du satisfies the triangle inequality (for Theorem D. Moreover, Proposition Saniiioo@@ccomn confirms that en encodes the predictive accuracy and certainty of model since ICE encodes the predictive accuracy and certainty (see the example in App. AA)
Interestingly, this connection to CE enables the extension to adversarial robustness, distributional robustness and algorithmic fairness, with different practical motivations. For instance, adversarial robustness (in model valuation) important to application scenarios where the model can encounter adversarial attacks in deployment 43 Formally, the respective objective functions objective,,,, 43 objective 667 Equation 5] and objectivego
[56) Definition 2] can be achieved from suitable definition of using du (precise definitions and full deviations are deferred to App. A as summarized
Note that CE(p, q) = p(x) log g(z) dæ for two distributions with densities p, g-

---

in [ We highlight that this illustrates the potential generality of MSVs using the Hellinger distance w.r.t. Dirichlet abstractions (i.e., using (C) in 2 and defer the formal treatment of such theoretical connections to future work. A question one might ask is that: (How) can multiple such linearly combine selected
evaluation criteria be combined? The answer is yes, by leveraging evaluation criteria, as discussed in App. A
Table 1: Extension to other evaluation criteria for model valuation. The notation dependence on the query set is made explicit. Dadv, Delean denote the query sets containing adversarial and non- adversarial (clean) training examples. {Dg)gee is collection of query sets where Dg contains training examples from a particular "group" /couup"/ataaaaaa distribution. D Dimprox contain positive training examples under the protected and unprotected groups, respectively.
Criteria
Query sets
Choices of
objective udv objectivepRo objectivero
Dadv: Deken (Dg D poot D
(d+(Q, Q"; (aav)) + v dH (Q, Q": :D:aan))))) Delea))) maxgeg du Q;; P2) - -Idn Q": D :poot du( (Q, Q" D
unprot
Pumport)
4 Learning Model Shapley
To address challenge (3) in Sec. [ we propose learning approach to train a model appraiser (i.e., a regression learner) from the MSVs of small subset of models for predicting MSVs of the remaining models (further elaborated in App. C.. we can learn a good appraiser with only 20% of all models (empirically verified), then the marketplace size can (theoretically) quintuple. To justify this learning approach, we derive a Lipschitz continuity of model Shapley, which is also empirically verified using 5 rerl-world datasets and various model types. Next, implement this learning approach by training a Gaussian process regression (as the model appraiser) on a subset (from 5% to 50% in size) of 150 model-MSV pairs and examine its predictive performance on the rest. Our implementation is available at /tttts://tgtthii cow
SShapeey
4.1 Lipschitz Continuity of Model Shapley
We derive Lipschitz continuity of the model Shapley function & [N] + R: The difference between the MSVs of two models M,, M2 (i.e., inputs to b) bounded by the distance du Qv)) between them, multiplied by constant factor.
Theorem 1 (Lipschitz Continuity). Let d = Hp in 3). (i, dn( (Quui, Qcue) < du (x))
Then, Vi E [[]] (VC S [N]] Zdn (Qe, Qv).
|0b
Its proof is in App. A Theorem T states that the difference in MSVs of two models is bounded by the Hellinger distance between their Dirichlet abstractions, and the constant Z from (2) is the Lipschitz constant. This is based on a simple each fused with common Qc. the resulting similarity is higher (i.c., smaller dh) since Qoui and Qoui' have Qe in common (see Proposition 4 in App. A). Moreover, Table and F App. A) empirically verify Theorem T Then, Theorem D provides theoretical justification for the learning approach because guarantees that similar inputs (i.e., small du (Qi, Q2 )) imply similar outputs (i.e., small jé;- (v.) namely, the model Shapley function is well-behaved w.r.t. its inputs, and hence learnable. This reasoning is applied to justify learning the data Shapley value [21]
condition: When Q and Or are
4.2 Empirical Learning Performance via Gaussian Process Regression (GPR)
To exploit the Lipschitz continuity (i.c., Theorem D. we adopt the Gaussian process regression (GPR) due to a uniform error bound of GPR on Lipschitz continuous functions [47). Our implementation trains a GPR (as the model appraiser) the MSVs of subset of N 150 models and examines its predictive performance on the remaining ones.
Regression setting. We train N = 150 independent models on MNIST (CIFAR-10): 50 of logistic regression (LR), multi-layer perceptron (MIP), and convolutional neural network (CNN) each (ResNet-18 18, SqueezeNet, and DenseNet-121 each). For simplicity, use the test set without
7

---

partitioning as the query set D. For each M, we obtain h2 via its predictions on D and solve (I) to obtain a; as imput features for separate regressions. For the regression labels, as calculating @; exactly incurs ((2150) time, we use the (e = 0.1, = apppoooproinnnioonn
[53] Φ; as the average of 3745 Monte-Carlo samples. This results in two sets {or; di) and (hr, Φi) of model-MSV pairs of size 150 each. We train GPR random subset of 150 model-MSV pairs to learn predict the MSV on the remaining pairs. In GPR, we use the squared exponential kernel (() (the lengthscale a is learnt) where d(i, d') == du (Q., Q2)) for {as, di and d(i,;") = 1h hul
for (h2,0) ):)
High regression performance verifies learnability. We examine the test performance using two error metries: mean- mean-squares squared error (MSE) and maximum error (MaxE) w.r.t varied training ratios from 5% to 50,, in Fig. 2 In particular, results for training ratio of 20% are in Table 2 We observe that even using only 20% of model-MSV pairs for training, the learning is effective (i.e., low test errors), which shows its feasibility in a large-scale model marketplace. This can be attributed to the learnability justified by Theorem D and the uniform error bound of GPR (47). In addition, learning on ar is more effective than learning on h; (as Table 2 and Fig. Fig..2ssen show higher errors for the latter), since the average operation to get h; loses some model information.
Table 2: Top (bottom) are results on MNIST (CIFAR-10) for the training ratio of 20%. Average (std. error) over 10 random train-test splits.
MSE
MaxE 3.53e (99.7e
1.59e-9(6.96-7 (6.96 1.59e
h
03.1ee
1.59e (2.66
Q;
1.79e
$5.2e -6
9.05e $3.9e ") 3.23e 222.4
h4
3.05e
(4.5e-9
0.0015
0.0002
ai
0.015
Max error
0.04
Max error
MSE
hi
MSE
0.0010
aj
0.0001
0.010
0.0005
h;
0.02
0.005
0.0000
0.0000
0.25 0.50
0.25 0.50
Training ratio
Training ratio
Figure 2: Average (std. errors) of test performance vs. training ratios for MNIST & CIFAR-10 10 over 10 random train-test splits for each training ratio. Dashed (solid) lines follow the left (right) axis. Colors indicate a; (blue) or hy (orange). To elaborate, at training ratio of 5%, the GPR is trained on a random subset of 5% of the total 150 model-MSV pairs and its test performance on the remaining 95% of the model-MSV pairs is reported.
5 MSV vs. Common Evaluation Criteria
This work is motivated by the lack of a standardized model valuation, but there are different evaluation criteria useful in different scenarios ge. accuracy, Fl score, predictive certainty). Interestingly, we find that MSV can produce consistent model values with these criteria. Additionally, we evaluate the utility of MSV directly in a use case where buyer wishes to purchase multiple models with black-box access to build larger learner (e.g., random forest or voting classifier) [33 66) and show that MSV can be used to effectively identify the most 'valuable' models for this purpose.
MSV vs. predictive performance. Fig. 3 (left) compares the MSVs of different model types (independently trained on the same data) for MNIST: Here D consists of misclassified data (from the original test set) of all the models to highlight the difference in their predictive accuracies. Without
8

---

needing to partition D. we observe that CNNs significantly ouiperform both MLPs and LRs (in terms of accuracy) and have the highest MSVs. This is expected, since CNNs are more capable of performing well in image-based tasks, and hence the MSVs for CNNs are correspondingly higher. Then, we examine predictive certainty. We use the same CNN model type independently trained on the same MNIST data (i.e., their accuracies are essentially equal), but artificially increase the predictive certainty for some: We multiply the highest probability of (x₃)) by a factor of [1, 5, 10] and then normalize the resulting vector to sum to I without affecting the predicted class/accuracy. Fig. 3 (right) shows that models with higher predictive certainty have higher MSVs, confirmed by additional results CIFAR- 10, MedNIST, and DrugRe in App. These results confirm our intuition that a model with high predictive accuracy and certainty is likely to have a high MSV.
0.02
0.000
100
75
Accuracy
-0.005-
Accuracy
0.01
MSS
50
MSV
-0.010
50
0.00
MSV
25
MSV
-0.01
Accuracy
Accuracy
-0.01
CNN MLP LR
1 5 10
Figure 3: More suitable model types (left)/ighhr predictive certainty (right) lead higher MSVs.
Next, we train 3 groups of 3 LR models independently with varying sizes of training data from 0.001 0.01 and 0.1 of the entire KDD99 dataset containing highly imbalanced data; the top 3 classes out of 23 constitute 98. 22% of total data. Intuitively, the models trained on more data should perform better and thus be more valuable, but due to the class imbalance. Fig. 4 (left) shows difficulties in differentiating these models based on their accuracies (or MSVs w.r.t. the entire/unpartitioned query set). Intuitively, to differentiate them, we need a lower level (i.e., more refined) Dirichlet abstraction, namely the slass-ppecific Dirichlet abstractions: Partition the entire query set according to the
C' 23 classes with 7 == /Dx]- Define (Q,, Qu: (Pelk=t....))
EE Yke du (Qr, Qv )
to leverage P3 to compute Φ) ((=,,,,.,....cll = Ei- Thk (Dx)) (right of Fig. 4p. Then we can that MSVs are indeed consistent with F1 score (a criterion especially suited for imbalanced data) without explicitly using F1 score in the computation. In addition, for KDD, due to the high class imbalance, there are classes with extremely small data size (i.e., 5) and our calculation of du(Q;, Qr: (Dx)) naturally suppresses their effect (possibly inaccurate Q:) via % == [Dx]- However, in practice, it should be noted that partitioning the query set to obtain lower level of Dirichlet abstraction should be considered w.r.t. the size of available query set (Sec. ( In other words, obtain lower level of Dirichlet abstraction (and thus more refined representation). it incurs a higher cost from collecting a larger query set. In our experiments, we find that the size of each partitioned query set D; should contain at least 102 samples (e.g., for KDD most classes have least or close to 102 samples).
0.015
100
0.6
0.02
Accuracy
F1 score
0.010-
-0.4
ASW
MSV
50
0.01
0.005
MSV
MSV
0.2
Accuracy
F1 score
0.000
0
0.00
0.0
0.0010.01 0.1
0.001
0.01
0.1
Figure 4: (D) (left) and (x({a)) (right) vs. the sizes of training data (as a proportion to the full dataset).
Identifying valuable models to purehase. For a more end-to-end use case (instead of a single evaluation criterion), we evaluate the MSVs for up to 50 models and the performance of larger

---

learner by including subset of these models based on their MSVs in a highest/lowest- -frst sequence in Fig. 5 The larger learner is random forests (voting classifier) and models are decision trees (LeNets (46) for Breast Cancer (CIFAR- 10). As the test accuracy of highest MSV-first increases more quickly (orange line), it verifies our previous comment on models that perform well when combined with other models are likely to have high MSVs. This characteristic offers some practical utility. If a buyer looking to purchase models from a marketplace [66], then following the highest MSV sequence, the buyer only needs to purchase subset of 15 (left of Fig. 5 25 (right of Fig. (5) out all 50 available models, thus saving cost. More results on CIFAR- --0 with ResNet- are App. C
Test Accuracy
Test. Accuracy
0.95
0.44
Max accuracy
Max accuracy
0.42
Lowest first
Lowest first
0.90
Highest first
Highest first
0.40
20
40
20
40
No. models
No. models
Figure 5: Test accuracy vs. number of models Breast Cancer dataset (left) and CIFAR-10 (right).
6 Related Work
The work of [66] investigates the binary classification setting and is not suitable for empirical comparison as we consider problems with multiple classes. The works of [1 [1] approach the design of model marketplace from an economics perspective by addressing issues like arbitrage (e.g., via horizontal/vertical pricing). contrast, we formalize the value of a model via what has learned w.r.t. task. The black-box access setting is appealing in model marketplace as it
accommodates different model types. Some existing methods 8 29 30 33 44 focus on how to learn fused model from several trained models (possibly with black-box access) instead of how to value these models. We highlight that we design the fusion (Definition B to leverage its analytic properties in Theorem [ The approach of learning the Shapley value arises data valuation problems but has not been considered in model valuation. Interestingly, we can draw parallels between Theorem ] and [19] Theorem 2.8]. For brevity, include more extensive contrasting comparison with data valuation in App.
7 Discussion and Future Work
We exploit a Dirichlet abstraction of classification models with only black-box access for proposing a novel equitable model valuation called the model Shapley. We discuss that choosing suitable level of the Dirichlet abstraction can improve how accurately MSV reflects a model's predictive performance and empirically show that using the partitioned query sets (according to the classes) can provide a suitable trade-off between the level of abstraction and the size of the available query set. MSV behaves consistently (in our experiments) with some common model evaluation criteria (i.e., predictive accuracy and certainty, Fl score) and can be extended to more sophisticated criteria. This implics MSV can potentially help unify existing evaluation criteria to provide simplified model valuation in practice, without needing to explicitly perform separate evaluations.
For future work, it is interesting to explore how model valuation can help address the practical considerations encountered existing data valuation methods [70] nique to existing collaborative (learning) frameworks which require valuation of models, both non- non-parametric ones 62
78
80 and apply this tech-
Moreover, a more detailed investigation into satisfying the equitability of Shapley value [59] and its trade-off with the
72
74 81
and parameterized ones [18
49
computational cost 82 of practical interest, such as by applying more sophisticated analyses [47
or methods [9]
31
32
52
for proposed Gaussian process regression learning approach.
10

---

Acknowledgments and Disclosure of Funding
This research/project is supported by the National Research Foundation Singapore and DSO National Laboratories under the Al Singapore Programme (AISG Award No: ASS22-RP-00200013 Xinyi Xu supported by the Institute for Infocomm Research of Agency for Science, Technology and Research (A STAR).
References
[1] A. Agarwal, M. Dahleh, and T. Sarkar. A marketplace for data: An algorithmic solution. In Proc. ACM-EC, 2019.
[2] L. Agussurja, X. Xu, and B. K. H. Low. On the convergence of the shapley value in parametric Bayesian learning games. In Proc. ICML, pages 180-196, 2022.
[3] S. Arora, R. Ge, Y. Halpern, D. Mimno, A. Moitra, D. Sontag, Y. Wu, and M. Zhu. A practical algorithm for topic modeling with provable guarantees. In Proc. ICML, pages 280-288, 2013.
[4] M. Basseville. Distance measures for signal processing and pattern recognition. Signal Processing, 18(4)3349-369, 1989.
[5] A. Blackard and D. Dean. Comparative accuracies of artificial neural networks and discriminant analysis in predicting forest cover types from cartographic variables. Computers and Electronics in Agriculure, 24(3):131-157) 1999.
[6] S. Boyd, S. P. Boyd, and L. Vandenberghe. Convex optimization. Cambridge Univ. Press, 2004.
[7] Brown, B. Mann, N. Ryder, M. Subbiah,
Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei. Language models are few-shot learners. In Proc. NeurIPS, 2020.
[8] H. Chang, V. Shejwalkar, R. Shokri, and A. Houmansadr. Cronus: Robust and heterogeneous collaborative learning with black-box knowledge transfer. In Proceedings of the 1st NeurIPS Workshop on New Frontiers in Federated Learning (NFFL 2021), 2021.
[9] Chen, N. Cao, K. H. Low, R. Ouyang, C. K.-Y. Tan, and Jaillet. Parallel gaussian process regression with low-rank covariance matrix approximations, In Proc. UAI, page 152-161, 2013,
[10] L. Chen, P. Koutris, and A. Kumar. Model-based pricing: Do not pay for more than what you learn! In Proc. ACM SIGMOD Workshop on Data Management for End-to-end Machine Learning, 2017.
[11] L. Chen, P. Koutris, and A. Kumar, Towards model-based pricing for machine learning in a data marketplace. In Proc. ACM SIGMOD, page 1535-1552, 2019.
[12] H. Chernoff. A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations. The Annals of Mathematical Statistics, 23(4):493-507 1952.
[13] Z. Cong, X. Luo, Pei, Zhu, and Y. Zhang. Data pricing in machine learning pipelines. Knowledge and Information Systems, 64:1417-1455, 2022.
[14] A. S. Das, M. Datar, A. Garg, and S. Rajaram. Google personalization: Scalable online collaborative filtering. In Proc. wwW, 2007.
[15] P. Devijver and J. Kittler. Pattern Recognition: A Statistical Approach. Prentice-Hall London, 1982.
[16] J. M. Drazen, S. Morrissey, D. Malina, M. B. Hamel, and E. W. Campion. The importance and the complexities of data sharing. New England Journal of Medicine, 375(12): 2016.

---

[17] M. Ester, H.-P. Kriegel, J. Sander, and X. Xu. A density-based algorithm for discovering clusters large spatial databases with noise. In Proc. KDD, 1996.
[18] X. Fan, Y. Ma, Z. Dai, W. Jing, Tan, and B. K. H. Low. Fault-tolerant federated reinforce- ment learning with theoretical guarantee. In Proc. NeurIPS, 2021.
[19] A. Ghorbani, M. Kim, and J. Zou. A distributional framework for data valuation. Proc. ICML. pages 3535-3544, 2020.
[20] A. Ghorbani and J. Zou. Data Shapley: Equitable valuation of data for machine learning. In Proc. ICML, pages 2242-2251, 2019.
[21] A. Ghorbani, J. Zou, and A. Esteva. Data Shapley valuation for efficient batch active learning,
2021.
[22] I. Goodfellow, Y. Bengio, and A. Courville. Deep Learning. MIT Press, 2016.
[23] F. Gräßer, S. Kallumadi, H. Malberg, and S. Zaunseder. Aspect-based sentiment analysis of drug reviews applying cross-domain and cross-date learning. In Proc. International Conference on Digital Health, page 121-125, 2018.
[24] D. Han, M. Wooldridge, A. Rogers, S. Tople, O. Ohrimenko, and S. Tschiatschek. Replication- robust payoff- allocation for machine learning data markets. Journal of IEEE Transactions on Artificial Intelligence, 2022.
[25] M. F. Haque and R. Krishnan. Toward automated cyber defense with secure sharing of structured cyber threat intelligence. Information Systems Frontiers, 23(4):883-896, 2021.
[26] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proc. CVPR, 2016.
[27] E. Hellinger. Neue begründung der theorie quadratischer formen von unendlichvielen veränderlichen. Journal für die reine und angewandte Mathematik, page 210-271, 1909.
[28] S. Hettich and S. D. Bay. KDD Cup 1999 data data set. huttps://arciiv/ ics. uci edu/ m1/ mol/date /dtaaaatsss/////////a /kdd+ cupptccu++c99c00..
1999.
[29] N. Hoang, T. Lam, B. K. H. Low, and Jaillet. Learning task-agnostic embedding of multiple black-box experts for multi-task model fusion. In Proc. ICML, pages 4282-4292, 2020.
[30] Q M. Hoang, T. N. Hoang, B. K. H. Low, and C. Kingsford. Collective model fusion for multiple black-box experts. In Proc. ICML., pages 4857-4867, 2019.
[31] Q. M. Hoang, T. N. Hoang, and K. H. Low. A generalized stochastic variational Bayesian hyperparameter learning framework for sparse spectrum Gaussian process regression. In Proc. AAAI, volume 31, 2017.
[32] N. Hoang, Q M. Hoang, and B. K. H. Low. A unifying framework of anytime sparse gaussian process regression models with stochastic variational inference for big data. In Proc. ICML, volume 37, pages 578. 2015.
[33] T. N. Hoang, S. Hong, C. Xiao, B. K. H. Low, and J. Sun. AID; Active distillation machine to leverage pre-trained black-box models in private data settings. In Proc. WWW, pages 3569-3581, 2021.
[34] H. Homei. Randomly weighted averages: A multivariate case, 2016.
[35] G. Huang, Z. Liu, L. van der Maaten, and K. Q. Weinberger. Densely connected convolutional networks. In Proc. CVPR, 2017.
[36] J. Huang. Maximum likelihood estimation of dirichlet distribution parameters. CMU Technique report, 18, 2005.
[37] F. N. landola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer. Squeezenet: Alexnet-level accuracy with 50x fewer parameters and; 0.5 mb model size. arXiv preprint arXiv.1602.07360. 2016.
12

---

[38] H. Jeffreys, An invariant form for the prior probability in estimation problems. Proceedings of the Royal Society of London. Series A, Mathematical and Physical Sciences, 186(1007):4534455 1946.
[39] R. Jia, D. Dao, B. Wang, F. A. Hubis, N. Hynes, N. M. Gürel, B. Li, C. Zhang, D. Song. and C. J. Spanos. Towards efficient data valuation based on the shapley value. In Proc. AISTATS, pages 1167-1176, 2019.
[40] G. A. Kaissis, M. R. Makowski, D. Rückert, and R. F. Braren. Secure, preercciigeeeiiingggg and federated machine learning in medical imaging. Nature Machine Intelligence, 2:305-31 2020.
[41] A. Krizhevsky. Learning multiple layers of features from tiny images. Master's thesis, Depart- ment of Computer Science, University of Toronto, 2009.
[42] S. Kullback and R. A. Leibler. On information and sufficiency. The Annals of Mathematical Statistics, 22(1):79- 86, 1951.
[43] A. Kurakin, I. Goodfellow, and S. Bengio. Adversarial machine learning at scale. In Proc. ICLR, 2017.
[44] T. C. Lam. N. Hoang, B. K. H. Low, and P. Jaillet. Model fusion for personalized learning, In Proc. ICML, pages 5948-5958, 2021.
[45] Y. LeCun, B. Boser, Denker, D. Henderson, R. Howard, W. Hubbard, and L. Jackel. Hand- written digit recognition with prokapropagttton network. In Proc. NeurIPS, 1990.
[46] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324 1998.
[47] A. Lederer, J. Umlauft, and S. Hirche. Uniform error bounds for Gaussian process regression with application to safe control. In Proc. NeurIPS, 2019.
[48] J. Lin. On the Dirichlet distribution. 2016. Master's thesis, Department of Mathematics and Statistics, Queen's University.
[49] X. Lin, X. Xu, S.-K. Ng, C.-S. Foo, and B. K. H. Low. Fair yet asymptotically equal collaborative learning. In Proc. ICML, pages 21223-21259, 2023.
[50] T. Lissack and K.-S. Fu. Error estimation in pattern recognition via l, distttteece between posterior density functions. IEEE Transactions Information Theory, 22(1):34-45, 1976.
[51] J. Liu, J. Lou, J. Liu, L. Xiong, J. Pei, and J. Sun. Dealer: An end-to-end model marketplace with differential privacy. Proc. VLDB Endow., 14(6):957-69, 2021.
[52] B. K. H. Low, J. Yu, J. Chen, and P. Jaillet. Parallel gaussian process regression for big data: Low-rank representation meets markov approximation. In Proc. AAAI, page 2821-28277 2015. 2015.
[53] S. Malcki, L. Tran-Thanke G. Hines, T. Rahwan, and A. Rogers. Bounding the estimation error of sampling-bart based Shapley value approximation with/without stratifying, 2013.
[54] A. Malinin. Uncertainty estimation in deep learning with application to spoken language assessment. PhD thesis, Department of Engineering, University of Cambridge, 2019.
[55] A. Malinin and M. Gales. Predictive uncertainty estimation via prior networks. In Proc. NeurlPS, 2018.
[56] N. Mchrabi, F. Morstatter, N. Saxena, K. Lerman, and A. Galstyan. A survey on bias and fairness in machine learning, ACM Computing Surveys, 54(6), 2021.
[57] T. Minka. Estimating dirichlet distribution, 2000.
[58] MONAI Consortium. MONAI: Medical open network for Al. https://github. com/
Project-KONAI/MDNAN 2020. Version 0.6.0.
[59] Q. P. Nguyen, B. K. H. Low, and P. Jaillet. Trade-off between payoff and model rewards in shaplcy-fair collaborative machine learning. In Proc. NeurIPS, pages 30542-30553, 2022.
13

---

[60] E. Patrick and F. Fischer. Nonparametric feature selection. JEEE Transactions on Information Theory, (555)5777-584 1969.
[61] Y. Polyanskiy. Lecture notes on /-iivergencess 2019. Lecture notes, httpp::/peoppppe lids mit. wtt.ee//////.hooapaagaaaaaaaaaaa
fdiv. pdd
[62] R. Qiao, X. Xu, and B. K. H. Low. Collaborative causal inference with fair incentives. In Proc. ICML, pages 28300-28320, 2023.
[63] T. W. Rauber, T. Braun, and K. Berns. Probabilistic distance measures of the Dirichlet and Beta distributions. Pattern Recognition, 41(2):33-6455, 2008.
[64] S. Romano, N. X. Vinh, J. Bailey, and K. Verspoor. Adjusting for chance clustering comparison measures. Journal of Machine Learning Research, (71):4635-4666 2016.
[65] G. Ronning. Maximum likelihood estimation of Dirichlet distributions. Journal of Statistical Computation and Simulation, 3244) 1989.
[66] B. Rozemberezki and R. Sarkar. The Shapley value of classifiers in ensemble games. In Proc. CIKM, pages 1 1558-1567, 2021.
[67] S. Sagawa, W. Koh, T. B. Hashimoto, and P. Liang. Distributionally robust neural networks. In Proc. ICLR, 2020.
[68] M. Sensoy, L. Kaplan, and M. Kandemir. Evidential deep learning to quantify classification uncertainty, In Proc. NeurIPS, 2018.
[69] L. S. Shapley, A value for n-person games. In H. W. Kuhn and A. W. Tucker, editors, Contributions to the Theory of Games, volume 2, pages 307-317. Princeton University Press, 1953.
[70] R. H. I.. Sim, X. Xu, and B. K. H. Low. Data valuation in machine learning: "ingredients", strategies, and open challenges. Proc. IJCAI-22, pages 5607-5614, 2022. Survey Track.
[71] R. H. L. Sim, Y. Zhang, M. C. Chan, and B. K. H. Low. Collaborative machine learning with incentive-aware model rewards. In Proc. ICML, 2020.
[72] R. H. L. Sim, Y. Zhang, T. N. Hoang, X. Xu, B. K. H. Low, and P. Jaillet. Incentives in private collaborative machine learning. Proc. NeurlPS, 2023.
[73] K. Singhal, H. Sidahmed, Z. Garrett, S. Wu, J. K. Rush, and S. Prakash. Federated reconstruction: Partially local federated learning. In Proc. NeurIPS, 2021.
[74] S. S. Tay, X. Xu, C. S. Foo, and B. K. H. Low. Incentivizing collaboration in machine learning via synthetic data rewards. In Proc. AAAI, volume 36, pages
2022.
[75] C. Tosh and S. Dasgupta. The relative complexity of maximum likelihood estimation, MAP estimation, and sampling. In Proc. COLT, pages 2993-3035, 2019
[76] T. Wang, Y. Yang, and R. Jia. Learaailiiyy of learning performance and its application to data valuation, 2021.
[77] L. Watson, R. Andreeva, H.-T. Yang, and R. Sarkar. Differentially private Shapley values for data evaluation, 2022.
[78] Z. Wu, Y. Shu, and B. K. H. Low. DAVINZ: Data valuation using deep neural networks at initialization. In Proc. ICML, pages 24150-24176, 2022.
[79] X. Xu, L. Lyu, X. Ma. C. Miao, C. S. Foo, and H. Low. Gradient driven rewards to guarantee fairness in collaborative machine learning. In Proc. NeurlPS, 2021.
[80] X. Xu, Z. Wu, C.-S. Foo, and B. K. H. Low. Validation free and replication robust volume-based data valuation. In Proc. NeurIPS, 2021.
[81] X. Xu, Z. Wu, A. Verma, C. Foo, and B. K. H. Low. Fair: Fair collaborative active learning with individual rationality for scientific discovery. In Proc. AISTATS, pages 4033-4057, 2023.
[82] Z. Zhou, X. Xu, R. H. L. Sim, and C.-S. B. Low. Probably approximate Shapley fairness with applications in machine learning. In Proc. AAAI, 2023.
14

---

A Theoretical Discussion
A.1 Results Related to Maximum Likelihood Estimation (MLE)
log-liielihood function F (.. For completeness, we provide the full log-likelihood function [36 57 65] using the predictions as follows: To avoid notational overload. let y (x,)) denote the predictive probability vector of M, on æj and its dependence on is suppressed. Similarly, we suppress the dependence a since the context is clear. Then,
F(a, (y))j=...... log pogp((yshhe f=. = loo II p(usla)
D
Oske
log
Uj
ar-1
I(ax))
km
= log
Σ
log ((a)) + -au log hi
where log hi, the k-th component of log h; Since the final expression the above log-likelihond function F only depends the log-predictions (log Vj )===,..,, through the observed sufficient statistics log h4 (i.e., with an element-wist log operation), {y, b.
directly use h; in T instead of
Similar predictions between models imply similar MLE approximations of their Dirichlet abstractions. Intuitively, two models produce similar predictions if and only if their Dirichlet abstractions are similar. The result below precisely formalizes this intuition by exploiting the analytic tractability the log-likelihood function
I which concave with a unique maximizer:
Proposition 3. Suppose that the observed sufficient statisties log h4 (log hς) based on the predictions of M, (M,)) on D and the MLE approximation of its Dirichlet abstraction o; (Ot) are given. Then,
[h: hv I
a: az.
Proof. From 1.
F(ar, = D x [G(a) + log hi a T - log hi ]
argmax F(a,, hv) argmax G(o:) + log his T
a
argmax G(a) + log hy a - log a + +log h T
a
argmax F(ax, har) + D (log h log h a
(4)
Note that the RHS expression of 4 to be maximized remains concave since F(a, hwy) is con- cave in a (36) and the D (log h, - log hy) T term is linear in a. If h, he 1 = O. then the D (log h4 log hr) a term in 4 becomes 0. So, a; = argmax, F(a, hT) = argmax, F(a, ) ay since F is concave with a unique maximizer. Therefore, lh, hgl1 = a; =ap.
From Lemma below, log h4 = (bas,))) ((1))))) and log hv. = (ay ((ar (1) where ag (ay, is the k-th component of as (axp) and &b is the digamma function It follows immediately that
h - hy.e = exp(v((a).)) ((ass|)) exp(e((v,,) - ((arl))),
so a = =0, = h hrh 0.
Importantly, both abstractions able preserve the similarity between M, and M,, via either the Hellinger distance du(a, avy) (further discussed in Sec. 4.1) or the & distance [hg h1
9hhe digamma function (xx) = d/dr(In I'(x)) is monotonically increasing function that converges to In x 1/2x for any > 0.
15

---

(Proposition abbove) where a lower & or dи distance is equivalent to a higher similarity, We also empirically compare a,, h; in Sec. 4.2 S.c. and find that in general a; better.
Lemma 1. Suppose that the observed sufficient statistics log h, based the predictions of M, D and the MLE approximation of its Dirichlet abstraction a are given. Then,
log ha,k <b(aa,,)) ((((11))
Proof. The partial derivative of F(xx, h4) w.r.t Or can be explicitly derived as follows:
( ha)
C
D
16 Σ
ak
vb((a))))))))))))()))) + log hi,k
Oag
For concave optimization problem (i.c., maximizing the log-likelihood) subject to a non-negative orthant constraint (i.e., 0), the work of [6] has shown that the optimality conditions can be expressed as
VF(a:)))) = and 0 aF h.) =0
for I,....C. C.
dak
The last condition is known as the complementarity. Focusing on the complementarity condition w.r.t. the k-th components of a; and h; gives
(F(a, h4) 0.
0i,k 0
V
dock
Since ai,& > 0 (Definition
(F(a,h.) D (vb(lasii) (((n,))) lolgohhhh ha,x)
Oan
which is simplified to
& ((aa)))) v(ax,)) + log hi =
and thus,
log h +((ux) - +b
A.2 Results Related to Dirichlet Distribution and Distributional Distance
Suitable measures of distance between Dirichlet distributions. We highlight the challenges in applying the distance measures beside the Hellinger distance in App. Specifically, the Hellinger distance provides unique and desirable theoretical properties (e.g., satisfying the triangle inequality) which we exploit (c.g.. derive Theorem I
In our context, we seek a distance measure that is well-defined between two Dirichlet distributions, be evaluated in closed form, and has analytic properties. The work of 63 has painstakingly compared several probabilistic distance measures including the Balllack-Leiibe
(KL) and symmetric KL divergences [8 422. Patrick-Fischer distance [60]. generalized Matusita distance 4 Lissack Fu
distance (50), Kolmogorov [15] distance, Chernoff distance [12), and Hellinger distance (27). and made the following observations: The KL and symmetric KI. divergences and the Patrick-Fischer distance can encounter cases where the distance becomes undefined. The generalized Matusita distance, Lissack-Fu distance, and Kolmogorov distance all lack anti-derivative. So, consider and compare the remaining two options: Chernoff and Hellinger distances, which are connected as follows.
Chernoff vs. Hellinger distances. As mentioned earlier, the Chernoff distance du is the other theoretically appealing choice for Dirichlet distributions (63). so discuss its connection with the Hellinger distance as follows. We start by recalling the definition for Chernoff distance and derive an analytic connection between them.
Definition 4 (Chernoff distance [2].. The Chernoff distance between two distributions p and is
de(p. q; A) = - (x: g² (x>)(x) dx where À E (0, 1). Note that de(p, qi À = 1/2) is symmetric in and q.
16

---

For two continuous distributions p. q. their Hellinger distance du(p. q) and their Chernoff distance
(with = 1/2) dc(p: qi À 1/2) are connected via the Bhattacharyya coefficient (C(p, q) = Vp(x)g(z)da E [0, 1] as follows.
1 - (p.()))) = = 1/2))
by substituting the equalities
du(p.g) q) = v -C(pp.)
and
dc(p,q;>== 1/2) In(BB(pp)))
As result, or equivalently.
du(p. = -1-expll exp-exp(de((p
giA 1/2)),
(5)
dc(p, 1/2) 11- (p.
(6)
From (q..(6), de is monotonic w.r.t. dH. but de has logarithmic dependence which results in it not being a proper metric (i.e., does not satisfy the triangle inequality) unlike dμ We specifically leverage the triangle inequality to prove Theorem I which seems difficullt to do for the Chernoff distance.
Interestingly, this logarithmic dependence turns out to be practical advantage where du can run the risk of numerical overflow [3). Our clustering experiment of the models (App. runs into the issue of numerical overflow the Hellinger distance and we resort to the Chernoff distance. Our empirical results (App. C also support this that using de can obtain better results than using dH.
Proof of Proposition 2
Continuing from our discussion in Sec. 3 VCE PCB (C) S (C(CC follows from Lemma 2 below by requiring sufficient condition the differential entropy of Qc. as stated in Proposition 10 On the other hand, constructing a lower bound of VCu (CE(C) via 2g is less direct, Deriving such lower bound (Proposition involves exploiting a property specific to Dirichlet distributions that the differential entropy is non-positive and bounded from above.
be from the upper bound on CE using da in Lemma 2
2
By substituting p Qe and q = Q into the first inequality of Lemma 2 2dh (c. Q**) < CE Cc @+) H (QQ.). Using ((Qe) + (Qe, Q*) 0, it follows that
Proof of Proposition
di (C.. Q""
CE Qc Q*) ). so (C) < (C) by plugging the definitions of vCr(C)
-CE(Qr, Q²) and D (C) = df Q*).
Next, the differential entropy of Dirichlet distribution p parameterized by a 48 Table 2.1] is
H(p) = G((a) + EE
k= C) s E ax) - (ax 1) vaas))
which is maximized at 1c
155 54 and its maximum value is therefore - log I(C). Substituting
( < ((C) into the second inequality of Lemma 2 gives
CE(p, q) I const, x (1 - di g)1²) log [(C)
By substituting p Qc and q Q" into the above and plugging in the definitions of UCB(C) -CE(Qe, Q") and (C) = d3 Qe, (*),
(C) < consto x (1- (1=11+ log I(C),
which can be rearranged to complete the proof.
Lemma 2. For any two continuous distributions p and q.
2d (p. q) < CE(p, q) (()) I const, x (1 [1-[1- [1-11- dr 9)]²)
where (p) = - f p(x) log (()) dr is the differential entropy of du(p, 4) E [0, 1] is the Hellinger distance. and const, = [log(1 Amin 1)]/(L 29min) with Qmin == min; (22).
Note ((() 00c Q*).
17

---

Proof. Firstly,
CE(p, = - f plx) log q(x) dr
{ dki (p, q) + (p)
where (p. q) == f p(x) log(p(z)/g(z)) dr is the Kullback-Leibe distance. Next, we derive lower bound of dKL (p, q) in terms of d)(p.q):
(p,(), (p.
(8)
To prove (8)
((x)
dxr (p. 9) = p(x)) log
dr
(xx)
Vp(x) dæ
2 p(x) log
v (xx
= 2 /D(()) p(z))
V 7(x)
-log
dæ
Vp(x)
22 / p(a)
v /q(z)
dx
v Vp(x)
/ {p(s)) (p(o)+p(a)))) p(xx) 2/ 2Vp(zz) (((z=(/()))) ( dx
1+ / (f(LO) - 2/ p(x) 2V(((=) /q(z)) dr / d())d de / (p(lo) 2Vp(z)) ((z)) dx
/ (ala) +(()) p(x) 2Vv(z)) ((x)) dr (Vo(e) Vv(a)) dx
().
where the inequality is due to - logg 2 1 for all 2 2 0 by setting x vilz)/ ((x//Vp(a) Moreover, have an upper bound of dku (p. q) from [6]] Equation 7.27]:
pp(z)
2 0.
(x, S const, (1 -1-pppp))))
(9)
Lastly, substituting (p. g) CE(p. (p)) from 7 into { and ( completes the proof.
An example on how cross-entropy loss encodes the predictive accuracy and certainty. The CE loss is used to construct upper and lower bounds for our proposed method (i.c., v in Eq. B Hence, proposed method also encodes the predictive accuracy and certainty, exemplified below.
Recall that the CE loss of C-dimensional predicted probability vector w.r.t. the one-hot encoded true label
C
Σ Uk x In(ūx). A=1
W.l.o.g,, assume that y/1 1 (i.c., the correct class is the first class).
For two predictions [0.9, 0.1, 0, ........0 0] vs. vs. 0.9, .., 0]. The CE losses are 0.105 and 2.30, respectively. Note that the first prediction is correct while the second in incorrect and that both predictions are "equally certain".. Hence, a higher predictive accuracy implies lower CE.
18

---

2. For two predictions [0.9, 0.511, respectively. Note that both predictions are correct while the first prediction "more certain". Hence, a higher predictive certainty implies lower CE, if the prediction is correct.
vs. 0.4, ........ OJ. The CE losses are 0.105 and
Extension other model evaluation criteria. addition to predictive accuracy and certainty, there are other possible desirable criteria for model valuation such as adversarial robustness 43] distributional robustness [67]. and algorithmic fairness (i.e., by removing prediction bias) in ML [56]. Put differently, it is possible to devise other model valuations based on adversarial robustness, distributional robustness, or the algorithmic fairness of a model. Interestingly, since these more sophisticated criteria all utilize the same building blocks (i.e., the predictive accuracy and certainty of models on carefully selected query sets), our proposed approach can subsume these different criteria, described below and summarized in Table [ To be little technical, the high-level idea is that, since these criteria leverage the CE loss fcE in very specific ways, and we have derived the relationship between du and CE above (Lemma 2). approach can be extended to incorporate these criteria through careful choices of v.
For instance, the work of 43 explicitly defines "adversarial" training examples Daav to be distin- guished from "clean" training examples Datenn and evaluates model's performance (i.e., a linear combination of the CE losses w.r.t. the adversarial and clean training examples separately) as follows:
objective == (M; + v lcs (M; Peteun)
for some weight v > O where we suppress the constants that linearly depend on the sizes of D Dadv and Detean for simplicity and (cr(M;; D) is the common CE loss incurred by the predictions of M on query set 'D.
The work of [67 Equation 5] gives the group-adjusted distributionally robust optimization (DRO) estimator where each "group" E 9 contains training examples from a possibly different data distribution. Effectively, the optimizer minimizes the maximum CE loss over different groups/query sets s.t. each query set is a dataset from a possibly different data distribution:
objectivepRo
max (cr:
SEG
where, for simplicity, group size-dependent constant and a model capacity-dependent constant are ignored.
The work of [56] presents a number of different definitions of faimess in ML to cater to different situations. In general, these definitions each describe particular way for the model to make classifications w.r.t. specific conditions on (the features of) the data in order to be fair. For simplicity. we illustrate with equal opportunity (EO) [56] Definition 2] which "means that the probability of a person in positive class being assigned to a positive outcome should be equal for both protected and unprotected (female and male) group members." " To relate this to CE, we can define two query sets: Proot containing positive training examples under the protected group and Dit containing positive training examples under the unprotected group. To achieve equal opportunity, the average CE losses on both query sets should be (approximately) equal (i.e., both groups have equal true positive rates) or, equivalently, the difference in the CE losses should be small:
objectiveço = |<EE p(ttt)) (ME Dit Pitppeot
Table 1 ggies the specific definitions of query sets with the corresponding (possible) choices of the characteristic function adapted from the above-mentioned minimization objectives by replacing (cs(M; D) with d(Q, @*; D) and adding negation since these are minimization objectives.
Firstly, we show the original implemennationn/Fomullion using the CE loss fCE 43 67 56) can be reformulated using the CE between Dirichlet abstractions. We provide the reformulation of objectivenRO-
max CE(Q. D)) gEG
by replacing fçF with CE on the respective Dirichlet abstraction Q (for M) and an expert Q* (from the test set) and omit the explicit derivations of the other two for brevity.
Such reformulation enabled by the connection between CE and ler: The CE loss (cr(M; D) implicitly assumes expert who provides the correct labels to compute the loss on the query set D.
19

---

On the other hand, CE(Q, ;"; explicitly uses the expert and measures the cross-entropy between the two Dirichlet abstractions (i.e., one for the model M and the other for the expert) on the query set D. Therefore, if a model M makes predictions similar to the expert on fixed query set DD then both (M; D) and CE(Q, : are small (i.e., optimum 0). Next, inspired by the relationship between Hp and CE in Proposition 2 the expressions in CE are reformulated using dy in [ Note that Proposition 2 provides us with the intuition and is not used exactly. Hence, our approach of decoupling the query set(s) from the model valuation makes it general enough to subsume these more sophisticated criteria (of model's performance) for model valuation. A question one might ask is that: (How) can multiple such evaluation criteria be combined? The answer yes, by leveraging (P3) to linearly combine selected evaluation criteria, as discussed next.
Combining multiple evaluation criteria. Specifically, for a user (c.g., potential buyer of the model) who knows the relative importance of several different criteria (Formalized by the specific query sets such as in Table D. then the user can specify the weights to achieve desirable trade-off. This is because different users might have different preferences and there is no me--siz--its--l solution. To elaborate, suppose the user only cares about whether the model makes accurate predictions but not at all about adversarial robustness because the user intends to deploy in a controlled and safe environment, then the task constructed for adversarial robustness is not very relevant to this user. In contrast, the user does care about the adversarial robustness (which, is often at trade-off against pure predictive performance), then the user can set the weights between the two tasks according to their preferences.
On the other hand, if the trade-off's of the tasks are unknown, for instance the objectives are very complex, then uncovering the relationship between tasks (which are potentially trade-offs of each other) is useful. Specifically, the approach to obtain the connections in Table is useful. For instance, predictive accuracy and adversarial robustness are trade-offs of each other since the objective of adversarial robustness "balances" between the clean and adversarial cross entropy (CE) losses. Upon identifying this theoretical connection, the user can then specify the weight between the two accordingly.
Remark Note that our discussion on how to extend and combine multiple model evaluation criteria aims to provide the technical tools for doing so, instead of identifying how a buyer or seller should use them. To elaborate, the buyer can use our method to combine several evaluation criteria based on known preferences of the relative importance of these criteria. Our discussion does not aim to guide the buyer in identifying such preferences or consider the potential asymmetry of information in a marketplace where only the buyer (or the seller) knows such preferences, how the other party should react, We believe these are further and interesting questions for future exploration.
Other useful technical results.
Lemma 3 (Precision-weiee))
fusion preserves Dirichlet). The precision fusion in Defi- nition follows a Dirichlet distribution: Qc = Dir( 0,,1...., >n= (s,cc) [34 Theorem 2.1]. Lemma 4 (Bhattacharyya coefficient between Dirichlet distributions 63 ). Let a and a' denote the C-dimensional parameters specifying the two Dirichlet distributions p and q. respectively. Then,
the Bhattacharyya coefficient is
I I((ax + a4)/2)
BC(p, q)
IE (ax + 22)/2)
VILE_ ((an) I(a))
Lemma 5. The Hellinger distance between two Dirichlet distributions p and q parameterized by the respective and a' is
1/2
Ie T( + at)/2) r(Ee- (ax a4)/2)
v((a))) I((a'))
du(p. = /2 x
Th I((x) I(aa)
which follows directly from an equivalent definition of du(p, q) v2 V V- BC(p,q)
A.3 Lipschitz Continuity of Model Shapley
We provide the proof of Theorem I here and describe a sufficient condition for fusion (i.e., Defini-
tion 3) to increase similarity.
20

---

Further elaboration on Lipschitz continuity. We adopt the following general definition for Lipschitz continuity: For two metric spaces (X, dx) and (Y, dx), function f : X H Y is L
Lipschitz continuous if there exists a constant T,g > 0 s.t. Van, 22 E X
dy (x(((z)), S (zd
In our formulation, for the model Shapley function + the input space is the set [N] (or more precisely the set (Q; í E [N]] of Dirichlet abstractions) of the N models and the metric is the Hellinger distance du, which is proper metric for probability distributions (in this case Dirichlet distributions); the oulput space R (i.e., for model Shapley values) and the metric the absolute difference (i.e.. 1 - Φv| for two inputs Q., Q.)..
In summary, recall that Φ; Φ(2, v, [M))(())) in Sec. 3 the Lipschitz continuity of 4 is w.r.t. its first argument (i.e., i), when is defined as in 3) and the set [Mi)e[j] of models is fixed (i.e., correspondingly the set [Qi]ie is also fixed). For brevity, we suppress the notational dependence on the latter arguments and write as d, = ((;; [N] + R where as mentioned above, the metric for the input space is defined d(i, f') = du Q Q+) for i, E [N] and the metric for the outputs is the absolute difference 16: - Øv/
Proof of [ We follow an idea that the similarity between the Dirichlet abstractions Q; and Qv will lead to small difference in their expected marginal contributions Φb and de when fused with a common Qe for any C C [N] (. Consequently, we can apply Lemma 6 in App. A.4
Firstly, from vC( - dh Qc, Q*)) in (3)
(n U (i)) (C U (f)) = du (QCu())) Q*) + du du(Quu(r), Q*).
Then, using the property of triangle inequality of dh, it follows that
du "*) + du da(Quu(r)) Q*) z du du(QCu()). (ou())))cc()))))))))))))))))())) S du(Q: Qx)
where the last incquality is due to the fusion- iicoon-icreasss--e -amrraass-similaaitty condition stated in Theorem I (and examined below by Proposition
4 The final result can be obtained applying Lemma 6 Note that d(i, t') == du (Q:, ) ) < 1 (since the Hellinger distance is upper bounded by 1), so the condition in Lemma 6 is satisfied with L 1. In other words, for a different distance (other than the Hellinger distance), a different value of may be necessary. The constant Z is directly inherited to be the Lipschitz constant.
Proposition 4 (Fusion increases similarity). Suppose that a, and ov parameterize Qi and Qr. respectively. Then,
Yvk E [C]
0i,k + OR.k
((asa))
2 vb (larl1
|aili + |as-11 v((aas11)
2
ai, + av'k - ((a,,)
2 v ( | + +(araarr)) - ((arl)))
A vb
2
du (Q(Cuu))) Qeu(rn) S dH(Q,, Qr) where va is the digamma function.
Proof of Proposition f It can be observed from Lemma 5 that du(p, increases iff BC(p,q)
(Lemma decreases. Then, it equivalent to show that
BC(Qcu()) (cctt))) 2 BC(Q,QQ).
It can also be observed that BC can be viewed differentiable function taking in 2C' parameters (i.c., ag and ax)). Consider w.l.o.g. its partial derivative w.r.t. 0s,k and w.r.t ay for is = 1,...,C: C:
aBC
Oz.k + 02
1b |as/1 + |as |1) ) + (((asa)))
coeff x ((an)) +
Dog
2
èBC coeff
[v((ay,)++ +
ai,k + av.k
1/
las12 |arl1
+ +((ar1))
Doy.
2
2
21

---

where
v ((las)))((ar)) I + ay,x)/2) >0 0
coeff
2 II I(as,s) p(ay,)) r( (ar, + +ar,)
due to the positivity of I' over the positive domain
Now, Lemma 3 iimplies that the fusion always increases ai Since a denotes the k-th component of a; and (iC denotes the k-th component of acu(i), (cu(f), 2 ai,-- So, if OBC/da, 2 0 and /BC/Oay, 2 O for k = 1....; C, then the resulting BC increases (or, equivalently, du decreases) after fusion.
Since coeff >
Oii,k + asyr,)
v |ass1 + |aarl1
ÕBC 2 0
-v((ss)) +
+ vlla;11)
2
2
2
aas,k
Oi,k + Ovik
assli
+ ||
QBC
-b((gx,) + 1/
+ ((1xx111
20 day jk
2 0
2
2
for k = L,..., C. So, the final result follows.
Let Daum Ei=1 ol(anx + x22)) (v(a;,)) + ((ap,)))) and D4. C[ (cwl1)/2) - (+(la,, 1) + (1ayy 1)).2]. The sufficient condition for Proposition 4 implies that Deam 2 Dj.. Intuitively, sum is a sum of dimension-/class- -wise differences between Q and Or and it is large if every pair of a4,& and ay different and relatively small (i.c., low concentration for dimension/class k). aam is likely large if the dimension C' is large as there are more pairs of as, and 0yy,k whose difference contributes towards D On the other hand, DJ-1 is measure of the difference in the overall precisions of Qi and Qr. D1.1 is large if ax)1 and |acel | are different and have small values. D is likely small if C is large. As C increases. the precisions |ayl and |ay l will increase, which will cause oh ((aa)) + ov 22) and (1<((a;11) + ((ax/11))/ to be very close due to the converging behavior of vb
The condition Dum 2 DJ.1 says that if the class wise difference between Q, and Qr outweighs the difference in their precisions, then fusion increases similarity. Intuitively, if the 'shapes' of Q, and Qr are very different, then fusing each with a common distribution Qe "evens out' the difference in their shapes and increases the similarity. In particular, if C is large (i.e., a high-dimensional classification task), then the condition is more likely to be satisfied.
Empirical verification of Theorem Specifically, we verify whether a small du ( Qe) leads to a small (6, Φel via the Pearson coefficient between da(Qu. Qe) and (6, Φr over all i, and visualizing the corresponding heatmaps (Fig. 6 The setting as follows. We investigate 5 real-world datasets (and various ML models). including MNIST, CIFAR-10 10 [T]. two medical datasets: a drug reviews dataset that classifies the type of prescribed medicine based on the text reviews (DrugRe) (23 and a medical imaging dataset that classifies the medical department from the diagnostic scans (MedNIST) [58), and a cyher-threat detection dataset that classifies network intrusion based on input features such as IP addresses and network communication protocol (KDD999) (28). Recall these are some of the highlighted application domains of model valuation (i.e., medicine and cyber-defense) in Sec. Query set D is the respective test set of each dataset without partitioning.
We adopt grouping paradigm where the grouped models have the same model type (but undergo independent training with the same data) to ensure some similarity within each group. So, we can verify whether the models within the same group have similar MSVs. To see why models of the same type can produce similar Dirichlet abstractions, we provide clustering result of C₁ (Fig. I in App. C. Specifically, for MNIST, we implement model types: logistic regression (LR), multilayer perceptron (MIP),, and 2-layer convolutional neural network (CNN). For CIFAR- 10, we utilize 3 known model types with pre-trained weights: ResNet-18 (26). SqueezeNet (37, and DenseNet- 121 (35. For DrugRe, we use CNN and a bi-directional long-short term memory (BiLSTM)
network. More details on other datasets are in App. C
"Thee code for verifying this partial derivative using an automatic differentiation package is included in the supplementary material.
22

---

The high Pearson coefficients in Table 3 provide
provide some verification for Theorem [ The matched color intensities in the heatmaps in Fig. confirm that similar models have similar MSVs. Left (right) of Fig. Gis is w.r.t. general (class-specific) Dirichlet abstractions.
86420
0.75
86 1622
0.1
86420
0.10
86420
0.05
0..0
0.05
0.25
0.0
0.00
0 02468
024 68
02468
02468
Figure 6: Left two plots are heatmaps for du (Q, Q2) and |0: del. Right two plots are heatmaps
for du(Q,, Qvi (Dr}) and (ox((D) - or ((D)))]
Table 3:
Pearson coefficient between du Qi, Qex) and lob - Φv using the test set as a single query set. A high coefficient verifies Theorem
MNIST CIFAR-10
MedNIST
KDD
DrugRe
0.974
0.996
0.974
0.999
0.955
A.4 Model Shapley Value
Closed-form expression of and the resulting computational complexity. Recall that an advan- tage of the combined choices of Definition 3 and the Hellinger distance du (described in Sec. B an available closed form evaluation of du between two Dirichlet abstractions, which has a constant time computational cost (i.e., (1)). This is important as dh is used to define the characteristic function (3) for model Shapley 2 Specifically, note that the definition of model Shapley (2) requires evaluating the characteristic function v for an exponential number of times due to the summation over all possible subsets C S [N]] \ {i}. In other words, even with a characteristic function that can be evaluated in constant time (c.g., dz). the computational complexity of MSV is still at least exponential in N, which is almost intractable for large N: if evaluating the characteristic function has a higher computational complexity, then the computational complexity of MSV would be even more intractable.
Similarity- -bounded difference in SVs. The following lemma provides a general result for bound- ing the difference in two Shapley values (not necessarily MSVs) and is used in the proof of Theorem may be of independent interest.
Lemma 6 (Similarity-boundee difference in the Shapley values). For all i, 8' E [N],
(vC = [M] (i,d)} |M[C u (1)) v(C u (FD) S Ld(i, - 1é: dvl S ZLd(i, F')
where L 2 is a constant and d(i,)') )) is some distance measure between i and i' and is the linear
scaling as in ②
Proof of Lemma way as follows:
6
The difference (Øb; Ki can bounded by enumerating the coalitions a paired
10: - Φrl5 S
Σ CCINIVAL)
Σ
Zwelu(C') - v(C)
(e,e") CCINT,6) C'C(N). cu(t)=c'
where we = IC|1(N - |C] 1)!/N! and we have multiplied the constant Z into the separate summations. Note that this enumeration (considering both summations) exhausts (w.l.o.g. from the viewpoint of i) C C [N] {i} in the calculation of 6, The first summation enumerates all C [N] \ (i, so the remaining C to consider for i's marginal contributions as in (2) are the that include i' but not i (considered in the second summation). The summands in the second summation are in fact also in the form of (CCU (1)) - v(Cu (t)) for some C C [N] \ (i, i). Note that in the second summation,
= C'L
23

---

so C" (d) = } Consequently, let C = C' \ {i). Then, v(C))~v(C]-= pu(cuccu u(Cu
To complete the final step, first consider the simpler case of 2 = 1 and the coefficient we is defined in way such that 6b satisfies efficiency [69] (i.e., LegiNTyo we 1), the condition
VC C [M] { p(C (i)) r(C (i - Ld(i, 1') can be used to bound the overall sum of the RHS
1 1, - Ld(i,
More generally for Z + 1, it can be directly multiplied to the RHS as follows
1 Φ2 - L F')
since every term is multiplied by the same constant in the above two summations.
Proposition (Diminishing Model Shapley Value due to Substitutes). According to the definitions
in (P4,, the (of) denotes the model Shapley value of M. w.r.t [N] ([N' = [N] U (ie) and
M. M, is a perfect substitute/identicl duplicate/copy. Then,
(AC C (N]V (C) - -C)) 2 vCC (i, - (Cu (i))) = of S Φ:
Proof of Proposition The inequality o; 2 df is shown by examining the pairwise difference over their respective summands in the summation of model Shapley.
For di. di = EccInNe 5e where the summand sc == (c, we is as in and MC,(C) = v(C U {i}) v(C) the marginal contribution of i w.r.t. C.
For &r di = -EecInma s'c where the summand c'=Cu(b)
s'c = IC|I(N + 11 C - 1)!
MC,(C)+ + I(N + C2' 1)! (C,(
(N + 1)!
(N 1)!
of is obtained by observing that adding ic to [N] means additionally enumerating all the C C [N] (0) but added with ic. as shown above.
As |C"|
C1 + 1.
S'c
WC
[NN (C]) MC,(C) + (((()))CCC
N +
We
[[N (CI) MC, + ([C) + 1) MC,(C)
+
we
[NN + (C)
N ++
se
where the inequality is due to the conditionally redundant property of 12 Since Φi and of enumerate the same summation and individual summands have s'c S sc: it follows that of S dr.
Remedy for duplication from a dishonest seller. While the marginal utility of each duplicate model decreases, the combined utility of all the duplicated models may be higher than if there is only one such model. Hence, a dishonest seller might exploit this by duplicating model to receive higher combined utility. As hypothetical example to illustrate this: two models M., M2 from vendors i respectively, each have values 0.5, then the model seller decides to fraudulently duplicate model M, to another M. Although the value for M, depreciates, such duplication can lead to higher value for vendor i, namely the value of M, and M. combined might be higher than if only M, is present.
We note that our proposed approach can be adapted to address this issue relatively easily, by sub- stituting our proposed in Eq. (3) into the variant of the Shapley value (24] Theorem 4.5), which importantly continues to satisfy the properties PI (P2 and (P3) (24.. However, we highlight that (the robustness to) such duplication is beyond the scope of this work.
Ourr definition of conditional redundancy is a weaker version of (24] Assumption 2] which stipulates the benefit of copy M., (conditioned model M, already being added) is exactly
24

---

B Additional Literature Review
B.1 Relation to Data Valuation and its Design Approach
Data valuation, originating from the motivating application scenario of AI marketplaces [ 24). studies how to determine the intrinsic worth of data, often in the context of ML. Intuitively, these marketplaces treat data as commodities for trading. pricing mechanism (i.e.. a valuation function) is necessary. There have been some works exploring data valuation [10 20 39 [0). by leveraging ML principles and assumptions. For instance, data are more valuable training on the data produces an ML model with higher performance (i.e., more accurate). In addition to the application scenario of Al marketplaces, the value of data can also be/has been used in interpretable ML (20], data sharing [16] and collaborative MI. 766 79.. However, as motivated in Sec. [] there are various practical scenarios where data valuation is difficult. Hence, we explore an alternative by shifting our focus onto the ML
models in these scenarios to consider model valuation.
In terms of the design approach, although the existing data valuation works have different technical perspectives and thus different solutions, they often adopt common first- first-principle approach, i.e., data which can produce more accurate models are more valuable. This approach provides an interpretation for the valuation function the values assigned to the data. While it may seem counter- -ounter-intuitive to market economies where the price/value can naturally arise from the demand and supply, this approach is sensible because the lack of effective demand (i.e., the willingness and ability of buyers to purchase goods/data/MI. model at different prices). To elaborate, effective demand requires the buyers to have some intrinsic valuation function for the data/ML model which helps determine the quantity the buyers are willing to purchase at some fixed price. In contrast to the more conventional goods, the market for data/ML model is relatively niche in the sense that even the buyers themselves do not already have good intrinsic valuation function for the data/ML. model. As result, the buyers are unable to specify the effective demand, which makes it difficult for the price/valuation to arise naturally from the demand and supply in a market. To this end, the (proposed) valuation methods (i.e., existing data valuation methods and the model valuation method in this paper) aim to fill in this gap by explicitly designing such a valuation function where the value is determined through the utility of the data/MI. model in the context of MI. (e-g.. predictive performance). In this vein, the existing data valuation methods and our proposed model valuation method share common perspective of designing the valuation function reflect the utility of the data/ML model in terms of some performance in the ML context. An added benefit of this design approach is the interpretability of the value. To see this, suppose in the marketplace (c.g., AWS marketplace), an auditor questions the basis for certain pricing of some ML model, our proposed valuation function can provide some insight to that question and can potentially be used by regulators to oversee the ML model marketplaces.
In terms of the practical setting, data valuation can be viewed as (mostly) white-box, (i.e., the actual data are used as the input to the designed valuation function and thus completely observed) 13 Intuitively, in order to determine the value of some data, the valuation function must "see" the data. In this regard, this setting for data valuation leads a relatively straightforward formal representation of the data, which is the data. In contrast, model valuation can encounter additional practical difficulties which make the formal representation of ML much less straightforward. Similar to in data valuation, we might want to use the model itself as its formal representation (e.g.. the parameters of the parametric models). However, the so-called black- box access which is particularly appealing in model valuation (Sec.[], excludes the choice of using the model itself (e.g., the parameters). Then, it becomes unclear what the formal representation of a model to use under this black-box box access setting. In other words, for model valuation, precisely what is the input to the valuation function?
To briefly summarize the comparison between data and model valuation: In light of the practical obstacles of applying data valuation, we explore the alternative of model valuation. We adopt similar design approach to those adopted by existing data valuation works to explicitly design a model valuation function that reflects the utility of the ML model/data in terms of performance in the ML context, To address the additional challenges due to the black-box access setting (which is not encountered in data valuation), we propose a novel formal representation of an ML model (for classification).
13 Althhoggh there is some preliminary work on using some noisy version of data for valuation 77 so it is not completely white-box. in general there is some form of access to the data or its statistic for valuation.
25

---

B.2 Model Evaluation Criteria and Model Valuation
Model evaluation eriteria. The value of an ML model depends on its utility/perfommacc in the ML context. But the performance is a multi-faceter concept because there are different evaluation criteria, motivated by and suitable in different scenarios. For instance, one of the most commonly used evaluation criteria is the predictive accuracy (i.e., the proportion of correct predictions of the model). Another useful criterion is the predictive certainty (i.e., the certainty with which the model makes the predictions), as illustrated in Sec. T Furthermore, there are other sophisticated and practically important evaluation criteria such as fairness in prediction [56], robustness to adversaries [43] and robustness to distributional shifts in data [67. In this regard, one approach is to design model valuation bespoke to each of these criteria separately. However, it is more scalable and appealing to leverage common theoretical connection among these criteria to design more general model valuation that can be specified to different criteria as needed.
Model valuation. The work of [66] investigates the binary classification setting and is not suitable for empirical comparison as we consider problems with multiple classes. The works of [T] [1 13 51
approach the design of a model marketplace from an economics perspective by addressing issues like arbitrage (e.g.. via horizontal/vertical pricing). contrast, we formalize the value of a model via what it has learned w.r.t. a task. In addition, the black-box access setting is appealing in model marketplace as it accommodates different model types. Some existing methods [8] 30 (33 focus on how to learn fused model from several trained models with black-box access instead of how to value these models.
C Additional Experiments
C.1 Additional Experiment Settings
Licenses of used datasets and computational resources. MNIST [45] Creative Commons
Attribution-Share Alike 3.0. CIFAR-10 [41]: The MIT License (MIT). MedNIST (58): Apache
License 2.0. DrugRe (23 Apache License 2.0 KDD99 28] Apache License 2.0.
We perform our experiments on server with Intel(R) Xeon(R) Gold 6226R CPU @2.90GHz and four NVIDIA GeForce RTX 3080's. s. As in our experiments, we use the pre-trained weights for the models (where available) instead of training from scratch. This is because models, instead of focusing on the training procedure. As a result, the usage of GPUs is moderate (mainly for performing inference the trained models, typically within 2 hours depending on the complexity of the models).
method is w.r.t. trained
Multiple independent training for robustness of results. We train a particular setting multiple (i.e., 3) independent times and compare the results from different settings to ensure the robustness of results. For instance, Fig. examines the effect of training data ratio on MSVs. For each particular training data ratio, we perform independent training using the same model type and set of hyperparameters so that the plotted results are robust to randomness in the training (via stochastic gradient descent).
C.2 Additional Discussion and Experiment Details for Learning MSV
Learning approach. Conventionally, the model Shapley value (MSV) of each model M, is calculated (for sufficiently small N) or approximated (for larger N, e.g., larger than 30). The computational complexity of exact calculation scales exponentially in the number of models (i.e., O(2*)) )), and that of approximation scales polynomial in N (53) depending on the approximation requirement (i.c., a better approximation with smaller error would incur a higher computational cost). Furthermore, each of these computational complexities has to be multiplied by the number N of models, since the calculation or approximation is performed for each model individually. Our learning approach aims to reduce the number of models for which calculation approximation is performed, following the steps (i) perform calculation or approximation (e.g.. Monte Carlo) for a subset (of size Z) of all N models individually; (ii) use the obtained Z model-MSV (or -ppro-aaee MSV) pairs to fit a regression learner (e.g., Gaussian process regression); (iii) use the regression learner to predict the MSV for the remaining (N Z) models.
We highlight that this approach does not aim to reduce the computational complexity of the MSV of model. Instead, this approach aims to reduce the total computational complexity of obtaining the
26

---

MSVs of N models, by a factor of N/Z (e.g.. if Z = 30 for N = 150 models such as in Table 2 the total computational complexity is reduced to 1/5th). Importantly, our learning approach is parallel to the prior and existing efforts that aim to reduce the computational complexity of the MSV of model. It means that if more efficient approximation (than the oft-used Monte Carlo) to the MSV of model is proposed (in the future), it can be directly integrated with our learning approach in step (i) described above, namely replacing Monte Carlo.
Why Gaussian process regression. There are several reasons that we adopt the Gaussian process regression (GPR) as the specific choice for the learnigg approach: (i) GPR is a kernel-based method, which can exploit a suitably defined distance function between inputs. The Hellinger distance between two Dirichlet abstractions, or the E distance between the observed sufficient statistics h, of two models are both such suitable distance functions between the inputs (i.e., models). Hence, we adopt GPR to exploit the squared exponential kernel exp(-d(i, (2a))) on the similarity measure between models (where the lengthscale o is learned). Specifically, for {as, k.). d(i. z') == du(Qi, Qr). while for { }. d(i, i') == 1/, h,11- (ii) While the exact MSVs satisfy (PI)-(P4). the predicted MSVs are not guaranteed to satisfy these properties. Fortunately, the predicted MSVs have a bounded error to the exact MSVs, then these properties can be approximately satisfied [82]. In particular, GPR has such an error guarantee 47). The result 47 Theorem 3.1] requires the function to be learnt to be Lipschitz continuous (which we derive in Theorem []. and also the kernel to be Lipschitz continuous, so we adopt the squared exponential kernel, which is Lipschitz continuous (47 Moreover, note that the result [47] Theorem 3.1] is w.r.t. continuous input space (i.e., a subset of R for some c)), but the error guarantee depends on the Lipschitz continuity only through the metric between two inputs (i.e., l2 norm of the input vectors). This is to say, their result can be adapted to our setting (where the input space is not continuous, but discrete): In our formulation for & the metric between two inputs is the Hellinger distance (i.e., du (Q., Q ). It is thus an appealing future direction provide a formal guarantee based on these two theoretical results.
C.3 Additional Results for MSV vs. Predictive Accuracy/Certainty
Varying size of training data. We use the same model type and vary the size of training data. For MNIST, we use a CNN, CIFAR-10, we use ResNet-18. 18, for MedNIST we use a specific architecture called MedNet [T4] and for DrugRe we use a CNN for text. Fig. 7 shows more training data generally lead to (better trained models, and thus) higher MSVs. Although in some cases the MSVs are negative, it can be mitigated (if necessary) by exploiting the linearity property of MSV to linear translate all MSVs by a positive amount. For instance, if only ranking of the models is needed, then negative values are acceptable as long as the ordering is correct.
0.00
0.050
Acsuracy
77
Aecuracy
-0.05
75
MSV
0.025
MSV
-0.05
MSV
-0.10
MSV
0.00
50
MSV
-0.01
MSV
0.000
Accuracy
Accuracy
-0.15
Accuracy
Accuracy
-0.00
0.01
0.1
0.01
0.1
0.01
0.1
0.01
0.1
Figure 7: From left to right: MNIST, CIFAR-10. 10, McdNIST and DrugRe. For CIFAR-10 10 and MedNIST, D is the original test set without partitioning and for MNIST and DrugRe we partition the original test set according to classes.
Varying model type. We vary the model types and train them independently on the same data. We train each model independently for 3 copies for robustness of results. For KDD99, the model types are CNN, MLP and LR. For CIFAR-10 100, the model types are ResNet-18, SqueezeNet and DenseNet-121. 121. For MedNIST, the model types are ResNet- 18, MedNet, and tiny CNN. For DrugRe, the model types are CNN for text and BiLSTM. Note that these are the model types used in the empirical verification of the generalized symmetry result in Sec. 4.1
Fig. g shows the following, For KDD99, MLP performs the worst (very negative MSVs) while LR performs the best. For CI DenseNet-121 outperforms the rest while SqueezeNet
ipynb
27

---

the worst. For MedNIST, the bespoke MedNet performs the best while the tiny CNN performs the worst. For DrugRe, BiLSTM outperforms CNN for text.
100
30
MSV
0.00
Acsuracy
MSV
0.000
40
Accuracy
MSV
-0.02
Accuracy
0.00
Accuraey
-50
-0.01
-0.02
Accuracy
MSV
Accuracy
20
-0.05
Accuracy
MIV
0.02
Accuracy
-10
CNNMLP LR
Res
Sqz
Den
Med Res
Tny
CNN BILSTM
Figure 8: From left to right: KDD99, CIFAR-10, MedNIST and DrugRe. For KDD99, MedNIST and DrugRe D is the original test set without partitioning and for CIFAR-10 we specifically use the misclassified input data from the original test set and perform partitioning according to classes.
Varying predictive certainty. We use the same model type (trained on the same data) and only vary the predictive certainty. The model type is LR for KDD, DenseNet for CIFAR-10 10, MedNet for MedNIST and CNN (for text) for DrugRe.
Fig. 9 shows increasing predictive certainty (while maintaining the predictive accuracy) improves MSVs.
0.025
100
0.00
0.000
0.00
-30
Accuraty
60
Accuracy
-0.01
MSV
-0.025
50
MSV
-0.11
-40
MSV
-0.002
MSV
-0.0
-20
Accuracy
-0.00
-0.03
Accuracy
0
0.02
Figure 9: From left to right: KDD99, CIFAR-10, MedNIST and DrugRe. For all datasets DD the original test set without partitioning.
C.4 Additional Results for Identifying Valuable Models for Larger Learner
We perform additional experiments on Cov'ype 5 MNIST and 100. For CovType, the models used are decision trees of depths at most and the larger learner is random forest. For MNIST (CIFAR-100) the models used are LeNet-5 [6] (Resnet- 18) and the larger learner is a voting classifier. The total number of models is 50 for CovType and CIFAR-10, and 25 for CIFAR-100.
Fig. 10 shows for all three datasets, MSVs effectively identify the valuable subset of models since following the highest-first sequence increases the test accuracy more quickly. In particular, identify overfitting for CovType and MNIST since the max attained accuracy occurs before all the models are included.
Test Acouracy
Test Accuracy
0.97
Test Test. Acouracy
0.60
0.55
Max accuracy
0.97
Max accuracy
Max accuracy
Lowest first
Lowest first
0.35
Lowest first
Highest first
Highest first
Highest first
0.50
0.96
0.30
40
40
10 No. models
No. models
No. models
Figure 10: From left right: CovType, MNIST, and CIFAR-100. Test accuracy vs. number of included models.
C.5 Empirical Advantage of the Chernoff Distance over the Hellinger Distance
Additional performance metrics of learning MSV. While using du produces high regression performance of learning MSV (i.e., low test errors), we find that the performance may be somewhat limiting when we apply other performance metries such coefficient of determinant (R²) and
28

---

explained variance (ex Var). In particular, we find thet de outperforms th on these two metrics, shown in Table
Table Average (standard errors) of test performance (over 10 random trials) on 80% of data after training on 20% of data for MNIST (top two rows) and CIFAR-10 (bottom two rows). Left (right) two columns correspond to using du (dc). For both metrics, higher better (optimal is 1).
R22
Var
R²2
Ex
0;; 0.29(6.72e
0.201(2.90e ")
0.93(2e
0.92(3e
-4.04(0.462) h2
-3.66e((1.33)
0.85(7e
0.44(3e 2)
04 0.73(2.00e 3)
0.67(3.766 ?)
0.93(2e 3)
0.91(3e
h,
-8.69(1.21)
(2.47)
0.80(1e 3)
0.76(2e 3)
Notice in Table 4 the right two columns (dc) outperform the left two columns (dn). We hypothesize that this can to the logarithmic dependence in de resulting a more "linear" relationship between Q; and db.. For Dirichlet distributions (i.e., Dirichlet abstractions) over a C-dimensional space (C' > 1), the product of the pdf (appears in both de and dH) over the space may not be well-behaved, and the integral of this product makes it more intractable (possibly due to the curse of dimensionality). The logarithmic operation in de can help mitigate this, resulting in the better regression performance using GPR, in terms of ex Var and Rta.
C.6 Additional Benefit of A Lower Level of Dirichlet Abstraction
Recall the example in Sec. 2 that using lower level of Dirichlet abstraction by partitioning D according to the classes allows us to correctly differentiate M. from its 'shifted' version My. In- tuitively, partitioning D according to the classes improves the similarity measure between M; and Mr (via some distributional distance) and we hypothesize that it can also improve the general- ized symmetry, which exploits the similarity measure between two models (e.g., via dC(Qs, Qs))). We verify this on MNIST and CIFAR-10 by partitioning the respective query sets D (i.c., test set) according to C 10 classes s.t. query set Dj is from class k and Y
[Dxl. We
compute de Qr: (Dx)k=L....e) = D Ei= Th (Dx), as shown in the right two plots of Fig. Note that we
Σk=1 7K d Q.) and apply (3) to compute
use the Chernoff distance (instead of the Hellinger distance) due to its numerical stability, and will elaborate later on this point.
The setting for this experiment is as follows, we utilize the MNIST and CIFAR-IO datasets respectively. For each dataset, we train N = 150 models with 3 different model types (50 of each). Specifically, for MNIST, we train 50 of LR, MLP and CNN while for CIFAR-10, we train 50 of ResNet- 18, SqueezeNet and DenseNet-1211 121. As in Sec. 4.1 we evaluate the performance via the Pearson corre- lation coefficient between de (Q Qui (Dx) k= 1...... and ( or ((Px))
in Table 5
Table 5: Comparison of Pearson coefficients between using single query set D vs. partitioned query sets } for MNIST (top two rows) and CIFAR- 10 (bottom two rows).
Pearson \.
60
90
120
150
(D)
0.9493
0.9378
0.9146
0.9011
(((())
0.9440
0.9362
0.9326
0.9274
(D) 0.9958
0.9955
0.9934
0.9898
6 ([Dx))
0.9961
0.9949
0.9936
0.9924
Results in Table TJonfr] our hypothesis that lower level of Dirichlet abstraction can lead to better observation of the symmetry result. This because a lower level of Dirichlet abstraction provides more refined representation of each model (i.e., w.r.t. individual classes instead of overall). In particular, we note the performance improvement for a larger N is important so that model Shapley is learnable to be adopted in a large-scale marketplace.
29

---

Numerical overflow prevents the Hellinger distance from performing well. Due to the numerical
overflow issue of du: If the argument r(-) (e.g., the denominator in ((a) in Definition I
approaches 171 .61 614499 in double precision floating point numbers, from 633 Recall Definition
fuses models in a coalition C _ [N] to obtain Qc = Dir(E i= Oi,1,...., Σ as,C) where |C] = n.
The summation Σi=1 01 can lead to overflow when n (i.e., CD) large, which tends to happen when N (the total number of models is large) because the calculation of MSV Φ; requires the enumeration
of S [N] \. {i} to obtain the respective Qe. For instance
150, then the largest C S [N] {i) & for the class k is on average larger than 171.614479/ /149 x 1.15, then it leads to numerical overflow for (-)) (and thus Hellinger distance).
contains 149 models and if their respective parameter &
Note that the Chernoff distance avoids this by combining the logarithmic operation with the (-)): in implementation In (()) can computed directly instead of first computing T()) and then the logarithmic operation.
C.7 Same Model Types Produce Distributionally Close Dirichlet Abstractions
We perform clustering (using de as the distance measure instead of dy because shows better visual illustration and the cluserting results) to illustrate the effects of model types on the similarity between the resulting Dirichlet abstractions using the default test set as the query set without perform partitioning according to classes. Specifically, for the N = 150 models trained on MNIST (50 CNNs, 50 MLPs and 50 LRs), we apply the density-based spatial clustering of applications with noise (DBSCAN) [77. because it is suitable for data which contain clusters of similar density (in our case, 3 clusters each of size 50). Fig. I1 shows good separation of different model types, supported by additional quantitative indices: homogeneity (1.000), completeness (0.971) and V-measure (0.985). Homogeneity measures the degree to which a single cluster contains only members a single class. Completeness measures the degree to which the all the members of single class correctly classified into the same cluster. V-measure is a harmonic mean of both homogeneity and completeness. These high values suggest the Dirichlet abstractions contain sufficient model information to clearly distinguish between these model types. As a result, the clustering performance is quite good via the adjusted random index (ARI) (0.990 and optimal is 1.0), which measures the correctness of the given clustering via its match with the ground truth. We consider ARI because it is suitable for large equal- sized clusters [64], which is this case here: equal-sized clusters of size 50. This result confirms the effectiveness of our approach to "convert" the heterogeneous models into homogeneous Dirichlet abstractions which live in the metric space (with metric dh, though highlight the experiments performed w.r.t. de due to practical concerns). In particular, the numerical stability issue of dи [63 prevented us from obtaining meaningful clustering results using dy. Moreover, this result motivates our grouping paradigm used in our experiments to verify the generalized symmetry.
Estimated mumber of clusters: 3
CNN
0.08
MLP
LR
0.06
0.04
0.02
0.00
0.000
0.025
0.050
0.075
0.100
Figure 11: DBSCAN clustering of N 150 models trained on MNIST based on the Chernoff distance de Definition
30


Reviews:
[{"review_id": "eZSu7QCsyq", "reviewer": "Reviewer_tCDm", "summary": "The paper considers the problem of assigning a value to a ML model (e.g., in a marketplace with multiple models). The proposed idea is to estimate (via MLE) the Dirichlet abstraction of a model (potentially conditioned on the class) and to compute the Shapley value of a game where the value function is the Hellinger distance between the Dirichlet abstraction of a model that fuses the models in the coalition and an (almost) optimal model. The paper shows that this proposal is well-behaved and allows for learning the Shapley value to reduce the complexity (the number of Shapley values to compute). The paper also shows a number of experiments on standard datasets (MNIST, CIFAR et al.) and compares to other more standard quality metrics. ", "questions": "[Please also see the weaknesses part where I put questions as well (probably the most important); I am putting here whatever I have not already written above.]\n\n- The Dirichlet abstraction allows handling multiple classes nicely, but it was not clear to me what would simplify or get more standard if we have binary classification. \n\n- I understand why it helps (l. 210-220) with the Shapley computation, but it would be good to better explain why doing the fusion as it is done makes sense from the prediction perspective. \n\n- For clarification: the Dirichlet abstraction (e.g., l. 100) is not allowed to depend on x?\n\n- Is it possible to extend the framework to assign different weights to different types of errors?\n\n- It may be good to add a bit more background on the Dirichlet distribution and Hellinger distance (in App), just enough to help follow the rest of the paper for those who are not familiar with these.\n\n- Fig 1: although I understand it doesn't matter for the point the paper makes there, I find that plots 1-2 aren't similar to plots 3-4. Is this normal?", "limitations": "See above.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The problem of valuing a model with black-box access (i.e., we can access predictions but not the model itself) is interesting because of many considerations that restrict sharing data and model parameters. The paper proposes an original solution in considering the Dirichlet abstraction of the model to compute metrics. This indeed helps because then models fusion as well as Hellinger distance can be computed easily. In a sense, the framework itself is the main contribution more than the theoretical results (which is totally fine). \n\nOne of the advantages of the Dirichlet abstraction framework is that it handles easily and naturally multiple classes, as opposed to binary classification which is much more standard in that type of literature. \n\nI find the paper well written and easy to follow (up to the issue of Figures to understand numerical experiments, which I discuss later). ", "weaknesses": "I had a doubt about the Shapley value: Shapley assumes an axiom of efficiency. Here, $Z$ seems free. So $\\phi_i$ will be the Shapley value only for a particular value of $Z$. Does it have any impact? \nMore generally, this should be clarified. For instance, Thm 1 clearly depends on $Z$ and I have not seen that said anywhere. In the proof, it looks like $Z$ is taken to be $1$, is that the case? Does it correspond to Shapley then?\n\nThe condition in Prop 4 (App A), which implies that fusion increases similarity, is not too explicit. Is there a nice intuitive interpretation of it? And does it hold (theoretically) for the models used in the experiments for instance?\n\nThe notation can be improved in some places. For instance, l. 208, $n$ is the size of a subset that depends on $\\mathcal{C}$ whereas $n$ is almost always the size of the full set $N$. Also the fact that $C$ is the dimension of the label whereas $\\mathcal{C}$ is a subset of $N$ is confusing. \n\nThe definition of $\\mathbb{Q}^*$ poses some questions. It is in a sense supposed to represent an optimal model, but then some noise is added with range $[0, 0.01]$ to avoid degeneracy in MLE. So it means that theoretically, there could be a better model with a non-zero distance and hence strictly speaking the Hellinger distance is not decreasing with the model quality. Is it correct? If so, isn't there a nicer way to define $\\mathbb{Q}^*$ theoretically?\n \nOn the learning of the Shapley value: the proposal is to compute the Shapley value for a subset of models (say, 20 out the 150 models) and to use that to infer the value of others because the Shapley value has some regularity wrt the Dirichlet abstraction. This is nice, but poses two questions: \na- is it obvious that one can just compute the Shapley values of a subset of models? Normally, computing the Shapley value requires computing the marginal increment to some subcoalitions (even if not all, using MC). Here, we cannot do it for coalitions that aren't in the subset of 20, so how do we do? This requires clarification. Perhaps the special form of the game allows that but it needs to be clarified. \nb- in the experiments (l. 305-312), my understanding is that the paper computes the Shapley value of all 150 models (used as ground-truth), and then uses only a subset of them to infer the others. That by-passes the issue I mentioned just above... but then it does not allow to get the reduced complexity in practice so the issue stands completely. I think a better experiment would be to compute the Shapley value of the 20 models using whatever solution the authors came up with to my point a- above (which needs to be clarified), and then using it to infer the SV of the others, comparing it to the case where one can indeed compute all 150 SVs.\n\nFigures are really too small. I understand the page limit but it is excessive. Some of them are literally unreadable (like the plot is hidden behind the legend, e.g., Fig 3 left, for MLP). Also in many of them, I could not understand what the x-axis is (mostly because it was not written).\n\n\nUPDATE AFTER REBUTTAL: I raise my score to 7, due to a better understanding, in particular of the experiments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer tCDm for taking the time to review and providing such detailed feedback and comments, and for finding our paper \"interesting\" and \"well written\". We wish to address the feedback and questions as follows.\n\nW1.\n> So $\\phi\\_i$ will be the Shapley value only for a particular value of $Z$. Does it have any impact? More generally, this should be clarified. ... Thm 1 clearly depends on $Z$ and I have not seen that said anywhere.\n\n__Theorem 1 holds regardless of the value of $Z$__, by identifying the suitable constant $L$ in Lemma 6 in App. A.4, though the bound can become looser for a larger $Z$ (due to a large $L$ in Lemma 6). \n\n> ... it looks like $Z$ is taken to be $1$ , is that the case? Does it correspond to Shapley then?\n\nW.l.o.g., we set $Z=1$ and it does recover the original Shapley value.\n\nW2.\n> The condition in Prop 4 (App A), ... Is there a nice intuitive interpretation of it?\n\n__If the \"shapes\" of $\\mathcal{Q}\\_i$ and $\\mathcal{Q}\\_{i'}$ are very different, then fusing each to a common $\\mathcal{Q}\\_{\\mathcal{C}}$ increases the resulting similarity as it \"evens out\" their difference (lines 732-736).__ This is further elaborated in the lines 724-731, 732-736 of App. A.3.\n\n> And does it hold (theoretically) ... ?\n\nOur empirical verification of Theorem 1 (which requires Prop. 4) observes that the the models which are similar do lead to similar MSVs in lines 278-299 and Figure 2 and Table 2. Theoretical verifictaion is an interesting future exploration.\n\nW3.\n> notation can be improved in some places\n\nWe clarify that $N$ denotes an integer, while $[N]$ is the set $\\{1,\\ldots, N\\}$. We thank the reviewer for raising this and will revise our notations.\n\n\nW4.\n> there could be a better model ... Is it correct? If so, isn't there a nicer way to define theoretically $\\mathcal{Q}^\\*$?\n\nWe first highlight that __our theoretical results do _not_ require this definition of $\\mathcal{Q}^\\*$__. Then,\nthough there is a possibly better hypothetical model with a non-zero distance, empirically we find our definition effective, possibly because __such hypothetical models are very rare__. Nevertheless, exploring alternative definitions of $\\mathcal{Q}^\\*$ is an interesting direction. \n\nW5.\n> a- is it obvious that one can just compute the Shapley values of a subset of models?\n\n__We are considering a game of $150$ models to only obtain the MSVs of $20$ models__, instead of a game of only $20$ models. This elaborated further in App. C.2 (lines 881-888).\n\n> b- ... my understanding is that the paper computes the Shapley value of all 150 models (used as ground-truth), and then uses only a subset of them to infer the others.\n\nThis is the correct understanding of the learning approach, which only sees a subset of the ground-truths as the training data. The __remaining unseen subset is used only for evaluation__.\n\n> a better experiment would be to compute the Shapley value of the 20 models ... compute all 150 SVs.\n\n__This is indeed our expeirment setting__. Table 3 shows comparison between the predicted MSVs and the obtained ground-truths to have low prediction errors, the saving in computational cost is from __only needing to obtain MSVs for a subset of models__, further elaborated in App. C.4 (lines 869-880).\n\nW6.\n> I could not understand what the x-axis is\n\nWe will increase our figure size in our revision, and clarify here: the x-axis (and y-axis) in Fig.2 is the model indices $i$; the x-axis Fig. 3 (left) is the model types, the x-axis Fig. 3 (right) is the predictive certainty (see lines 337-339); the x-axis in Fig. 4 is the size of training data (see its caption, and lines 340-341).\n\nQ1.\n> but it was not clear to me what would simplify or get more standard if we have binary classification.\n\nTheoretically, $C>2$ or $C=2$ does _not_ make a significant difference, as the intended design. In contrast, [55] have specifically exploited the simpler structure of $C=2$ and thus it seems difficult to extend their method to $C>2$. For practice, a smaller $C$ means fewer class-specific query sets to collect (lines 357-358).\n\nQ2.\n> ... why doing the fusion as it is done makes sense from the prediction perspective.\n\nWe wish to highligh that the __fused Dirichlet abstraction $\\mathcal{Q}\\_{\\mathcal{C}}$ is _not_ a predictive model itself__ (i.e., it can not be used to produce predictions by taking as input queries), and it is __primarily designed to be amenable to valuation.__ \n\nQ3.\n> ... the Dirichlet abstraction (e.g., l. 100) is not allowed to depend on $x$?\n    \n**A Dirichlet abstraction is a Dirichlet distribution, so it does not depend on $x$**. For a model $\\mathbf{M}\\_i: \\mathcal{X} \\mapsto \\triangle(C)$, for a random variable $X \\sim P_X$ whose $\\text{supp}(X) = \\mathcal{X}$ (and $P_X$ represents the task), $\\mathbf{M}\\_i(X)$ is a distribution over $\\triangle(C)$, and this is represented using the Dirichlet abstraction (lines 101-102).\n\n\nQ4.\n>  ... extend the framework to assign different weights to different types of errors?\n\nWe consider the \"types of errors\" to be different model evaluation criteria, and a user __can easily linearly combine different model evaluation criteria__ (Sec. 3.2 and Table 1), by exploiting P3 (linearity) and implementing the selected criterion from Table 1. \n\nQ5.\n> ... a bit more background on the Dirichlet distribution and Hellinger distance.\n\nApp. A.2 includes this background on Dirichlet distribution and Hellinger distance, and we will incorporate the suggestion in our revision.\n\n\nQ6.\n> Fig 1: ... I find that plots 1-2 aren't similar to plots 3-4. Is this normal?\n\nYes, plots 1-2 show the probability vectors while plots 3-4 show the densities of the learnt Dirichlet abstractions from these \nprobability vectors. \n\nWe thank Reviewer tCDm for the detailed review and questions, and hope our response has helped clarified the questions and  raised your opinion of our work. We are happy to provide further elaboration in discussion (due to character limit of rebuttal).\n"}, {"review_id": "k0DebHBsIm", "reviewer": "Reviewer_RTGV", "summary": "This paper introduces a novel approach to model comparison and valuation, utilizing a method known as Dirichlet Abstraction. The fundamental idea is to abstract the predictive behavior of different models via a Dirichlet distribution. This abstraction allows the comparison of diverse models on an equal footing.\n\nThree main challenges in model valuation are identified in the paper: developing a suitable abstraction for model valuation relative to a task, satisfying equitability properties in model valuation, and exploiting the equitability properties of the Shapley value in a large marketplace.\n\nTo address these challenges, the authors introduce an innovative approach to model valuation, utilizing a Dirichlet distribution to approximate a model’s predictive pattern or distribution with respect to a task. This abstraction, termed the Dirichlet abstraction, incorporates both the model's predictive accuracy and certainty. The authors then propose using the model Shapley value as an equitable valuation method, leveraging the Dirichlet abstractions' ability to preserve similarity between models. To address the computational challenge of Shapley value in a large marketplace, the paper suggests a learning approach for training a model appraiser. This model appraiser, trained on a small subset of models and their model Shapley values (MSVs), can predict other models’ MSVs, thus validating model Shapley’s practical feasibility in a large-scale marketplace.\n\nThe paper's empirical validation, performed on real-world datasets and with up to 150 heterogeneous models, confirms that higher predictive accuracy, more suitable model types, and higher predictive certainty correlate with higher model Shapley values. The authors also provide a use case for identifying a valuable subset of models from the marketplace to construct a more complex learner.", "questions": "How does the proposed technique compare with existing model valuation techniques, in terms of accuracy, computational efficiency, and other relevant metrics? It would be beneficial if the authors could provide a comparative study in this regard.\n\nHow does the method perform when the available data is sparse or unbalanced across classes? Could the authors elaborate on how their approach handles diverse model architectures, especially those with characteristics that cannot be adequately captured by a Dirichlet distribution?\n\nCould the authors elaborate on how to choose an optimal trade-off between the level of Dirichlet abstraction and the size of the query set? A concrete set of guidelines or a decision-making framework would be helpful for practitioners seeking to apply this method.\n\nSome of the assumptions in the paper (like model homogeneity post-abstraction) might not hold in all practical scenarios. Could the authors elaborate on the implications if these assumptions are violated and how such situations could be handled?", "limitations": "The authors have not explicitly addressed the potential limitations and broader societal impacts of their work.\n\nThe authors could provide more discussion on the potential limitations of their work. For instance, they might discuss the sensitivity of the model's valuation to the choice of the task or query set, and the limitations of the Dirichlet abstraction in approximating a model's predictive pattern or distribution.\n\nIn terms of broader societal impacts, the authors might consider discussing how their proposed model valuation framework could potentially affect the machine learning marketplace. For instance, could this valuation approach inadvertently favor certain types of models or tasks over others, or influence the development and use of machine learning models in ways that could have unintended consequences? They could also consider the potential impacts on data privacy, given that model valuation, unlike data valuation, does not require centralization of potentially private data.", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper presents a novel approach to model comparison and valuation using Dirichlet abstraction, offering a creative combination of existing statistical concepts applied in a new manner. The concept of utilizing Dirichlet distributions to abstract predictive behaviors of models is quite innovative. Furthermore, the introduction of class-specific Dirichlet abstraction is an original refinement to their method, preserving more detailed information about the model's behavior.\n\nThe quality of the research seems to be high, given the soundness of the mathematical framework used in the paper and the logical structure of the presented methodology. The authors provide a theoretical background to support their claims and supplement it with visual aids, offering a detailed explanation of how their method works. The decision to use Hellinger distance to measure the dissimilarity between two probability distributions shows a thoughtful choice in ensuring computational efficiency.\n\nThe ability to compare different types of machine learning models in a unified framework is an important advancement in the field. This has potential implications for model selection in various machine learning tasks, enabling a more flexible, efficient, and fair comparison of models. Moreover, the efficient computation of the Hellinger distance between Dirichlet distributions has implications in other fields where these statistical concepts are used. Overall, the paper presents a valuable addition to the machine learning and statistical literature.", "weaknesses": "While the paper does present a novel approach to model valuation, there are several areas where it could be improved.\n\nThe paper assumes the availability of a large query set for accurate MLE estimation. It should address situations where the available data is sparse or not balanced across classes. A detailed discussion or a possible solution for handling these scenarios would strengthen the paper.\n\nWhile the authors discuss the trade-off between the abstraction level vs. query set size, there is a lack of clear guidelines or a framework for determining the optimal trade-off. This could lead to difficulties in implementing the proposed method in real-world applications. Adding a more formal discussion or a proposed methodology to address this trade-off would be beneficial.\n\nThe approach assumes homogeneity across models after the Dirichlet abstraction, which might not always be the case in practical scenarios. Some models might have specific characteristics that cannot be captured by the Dirichlet distribution. More discussion on how such cases can be handled would improve the paper.\n\nThe paper lacks a comparison with other existing model valuation methods. It would be beneficial to include an evaluation of how the proposed method fares against existing methods in terms of accuracy, efficiency, and robustness.\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer RTGV for taking the time to review our paper and providing very detailed feedback and comments, especially saying that our work presents \"a novel approach\", \"is quite innovate\" and \"a valuable addition to the machine learning and statistical literature\". We wish to address the feedback and questions as follows. \n\nW1.\n> ... It should address situations where the available data is sparse or not balanced across classes. A detailed discussion or a possible solution ... .\n\n__We address a highly imbalanced case in our experiments, i.e., KDD99 dataset, using the class-specific Dirichlet abstractions weighted by the size of each class-specific query set__ (lines 345-348). Our results show that this approach is _effective and necessary_. Figure 4 (left) under the non-class-specific approach is unable to distinguish models trained on different amounts of training data (with 100-fold difference in size) while Figure 4 (right) under the class-specific approach is able to do so. We will make this discussion more explicit in our revision.\n\nW2.\n> ... there is a lack of clear guidelines or a framework for determining the optimal trade-off. ... Adding a more formal discussion or a proposed methodology to address this trade-off would be beneficial.\n\n__Adopting the highest level of abstraction can already be quite effective__ (shown in our experiments for MNIST, CIFAR-10, and two real-world datasets MedNIST and DrugRe) if the query set is _not highly imbalanced_. \nFor a highly class-imbalanced query set (such as KDD99 in our experiments), __adopting the class-specific Dirichlet abstractions weighted by the size of the class-specific query set is effective__ (lines 345-348). We will include this in our revision and believe that a more extensive and formal framework is a very useful future direction.\n \nW3.\n\n> ... assumes homogeneity across models after the Dirichlet abstraction, ... \n\n__All models (regardless of architectures) for the same learning task__ (i.e., same feature and label spaces) __can be represented via their corresponding Dirichlet abstractions, which are homogeneous by design, not assumption__. As an example, the Dirichlet abstractions of a logistic regession and a CNN are both Dirichlet distributions, but with possibly different parameters. \n\n> Some models might have specific characteristics that cannot be captured by the Dirichlet distribution.\n\n[2] provides a polynomial sample complexity of the query set w.r.t. the error and a model's actual predictive distribution. It is an interesting future direction to study its implications for model valuation.\n\n\nW4.\n> ... a comparison with other existing model valuation methods\n\n__There are limited existing methods applicable for comparison__: The closest work is [55], but it is restricted to the binary classification, thus not applicable to $C$-way classification (in lines 230-231 and 370-371). We include an additional discussion on other works from an economics viewpoint (without a machine learning focus) in App. B.2 (lines 844-851). \n\nHence, __we investigate some intuitive baselines where our proposed approach produces consistent results to these intuitive baselines__ (i.e., predictive accuracy in Figure 3, F$1$ score in Figure 4 right, training data sizes in Figure 4 left). A further discussion is included in App. B.2 (lines 833-843). Moreover, we have also __derived theoretical connections between our approach to some sophisticated criteria such as fairness, and robustness__ (Sec. 3.2 and Table 1).\n\nQ1.\n> How does the method perform when the available data is sparse or unbalanced across classes? \n\nSee response for W1.\n\n> ... how their approach handles diverse model architectures, especially those with characteristics that cannot be adequately captured by a Dirichlet distribution?\n\nSee response for W3.\n    \n> ... how to choose an optimal trade-off between the level of Dirichlet abstraction and the size of the query set?\n\nSee response for W2.\n\n> Some of the assumptions ... (like model homogeneity post-abstraction) might not hold ... . Could the authors elaborate on the implications if these assumptions are violated and how such situations could be handled?\n\n- On \"model homogeneity\", see response for W3.\n\n- A key assumption (for Theorem 1) is fusion-inreases-similarity (lines 272-273). __We specifically derive a sufficient condition for identifying when this assumption holds__ (Prop. 4 in App. A.3) and provide an interpretation.\n\nL1.\n> ... the sensitivity of the model's valuation ..., and the limitations of the Dirichlet abstraction in approximating a model's predictive pattern or distribution.\n\nWe thank the reviewer for the suggestion and will incorporate this in our revision.\n\nL2.\n> ... could this valuation approach inadvertently favor certain types of models or tasks over others, or influence the development and use of machine learning models in ways that could have unintended consequences?\n\n__This is an important motivation to consider a diverse range of evaluation criteria, from the more intuitive predictive accuracy to the more sophisticated ones such as fairness and robustness__. We are happy to provide further elaboration on this, due to the character limit of the rebuttal.\n\nL3.\n> They could also consider the potential impacts on data privacy, given that model valuation, unlike data valuation, does not require centralization of potentially private data.\n\nThe __privacy regulation on data is an important motivation of our work__ (lines 26-30) and is further discussed in App. B. 1.\n\nWe thank the reviewer for the detailed feedback and for the suggestions, and will incorporate these discussions in our revision. We hope our response clarified the questions and has helped improve your opinion of our work. We are happy to provide further clarifications or elaboration."}, {"review_id": "54fHDSQIFX", "reviewer": "Reviewer_bude", "summary": "\nThe authors represent the model's predictive accuracy and certainty with Dritchlet abstractions and formalize model Shapley values, which measure the value of models to a given task defined by a query set. \nThey evaluate their work on MNIST, CIFA-10, DrugRe, and medNIST datasets.", "questions": "\nFor the rebuttal, the authors should respond to all the questions highlighted in the weaknesses subsection.", "limitations": "\nThe authors do not mention any limitations of their work.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The authors lay out an interesting problem, with clear background and supporting statements. \n\n- The authors formulate a solid, well-thought-out solution to the problem.\n\n- The theoretical statements are sound, and the results of the empirical analyses are clearly explained. \n\n- The paper is well structured and written, thus facilitating a good read. \n\n", "weaknesses": "\nP4: Diminished marginal utility \n- While it might be the case that the marginal utility of duplicate models reduces, the model sellers might negatively affect the marketplace with untruthful model presentations. Say, for example, two models (A, B), each have values 0.5, then if one model seller decides to fraudulently duplicate model A, the value although depreciates, might assign a high value collectively to A (2/3).\n\nQuery set \n- The authors say they have black-box access to the models. This is somewhat okay with image data, where one doesn't need explicit knowledge of the key features. For example, if I'm interested in testing the value of cat/dog classification models, I will bring cat/do images in the query set. However, for things like loan worthiness, with black box access, how does one construct the query set?\n- What is the upper and lower bound on the query set size before one reverse engineer the model? \n\nP3\n- Multiple complementary tasks might be a fair way to measure value of models with p3 assumption. How about if these tasks are tradeoffs of each other? \n\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer bude for taking the time to review our paper, and for appreciating our studied problem (\"lay out an interesting problem\"), proposed approach (\"a solid, well-thought-out solution\"), presented results (\"theoretical statements are sound, and the results of the empirical analyses are clearly explained\") and our writing (\"paper is well structured and written, thus facilitating a good read\"). We wish to address the questions as follows. \n\nW1.\n> P4: Diminished marginal utility\n\n__Our proposed approach can be adapted to address this issue relatively easily__, by substituting our proposed $\\nu$ in Equ.(3) into the variant of the Shapley value [Theorem 4.5, 21], which importantly continues to satisfy the properties P1, P2 and P3 [21]. However, we wish to highlight that (the robustness to) such duplication is beyond the scope of this work and we will include this discussion in our revision.\n\n\nW2.\n> However, for things like loan worthiness, with black box access, how does one construct the query set?\n\n__Formally, there is no difference between such different types of data__, as we have also considered non-image data, specifically tabular data (i.e., KDD99) and text-based data (i.e., DrugRe) in our experiments. \nThe key requirement is that the __input and output specificaions of the model, have to match the input and output specifications of the collected data in the query set__. We believe this is a reasonable practical requirement because (i) without knowing the correct input of a model, a user cannot use the model for predictions; (ii) the output is $C$-dimensional for a $C$-way classification model.\n\nTo illustrate, for an image task, the black-box model (which could be logistic regression or a convolutional neural network), takes as input an image and returns as output a predicted probability vector. Similarly for other forms of data (e.g., tabular), the model takes as input a data point (i.e., a row in the tabular data) and returns as output a predicted probability vector.\n\n> What is the upper and lower bound on the query set size before one reverse engineer the model?\n\nTo the best of our knowledge, no known theoretical bounds are available to completely reverse engineer the model _only_ based on quries; __our approach is empirically shown to work with query sets which are smaller than an existing reverse-enginnering attack that requires additional assumptions _not_ satisfied in our setting__. Indeed, this is a main motivation for adopting the black-box (access) model in our setting as in lines 36-37.\n\nTo elaborate, (Oh et al., 2018) requires (i) knowing something about the model (to be reverse engineered), such as \"a diverse set of white-box models ... that are expected to be similar to the target black box at least to a certain extent\"; (ii) hundreds to thousands of queries, to construct a model that _only predicts similarly_, __not__ the black-box model itself. In our setting, (i) is not satisfied; moreover, in our experiments, we find that query sets of size as small as $100$ are sufficient, which is much smaller than that used in (Oh et al., 2018).\n\nW3. \n\n> Multiple complementary tasks might be a fair way to measure value of models with p3 assumption. How about if these tasks are tradeoffs of each other?\n\n__If the user knows the importance of these tasks, then the user can specify the weights to achieve a desirable tradeoff.__ This is because different users might have different preferences and there is no one-size-fits-all solution. To elaborate, suppose the user _only_ cares about whether the model makes accurate predictions but not at all about adversarial robustness because the user intends to deploy it in a controlled and safe environment, then the task constructed for adversarial robustness is not very relevant to this user. In contrast, if the user does care about the adversarial robustness (which, is often at trade-off against pure predictive performance), then the user can set the weights between the two tasks according to their preferences.\n\nOn the other hand, if the tradeoffs of the tasks are unknown, for instance the objectives are very complex, then uncovering the relationship between tasks (which are potentially tradeoffs of each other) is useful. Specifically, the approach to obtain the connections in Table 1 is useful. For instance, predictive accuracy and adversarial robustness are tradeoffs of each other since the objective of adversarial robustness \"balances\" between the clean and adversarial cross entropy (CE) losses. Upon identifying this theoretical connection, the user can then specify the weight between the two accordingly. We will include this discussion in our revision.\n\n*References.*\nOh, Seong Joon and Augustin, Max and Schiele, Bernt and Fritz, Mario. Towards Reverse-Engineering Black-Box Neural Networks. In ICLR, 2018.\n\nWe thank Review bude for the positive feedback and the comments. We hope that our response has clarified the questions, and has helped raise your opinion of our work."}, {"review_id": "O9zNZhG8X7", "reviewer": "Reviewer_Coej", "summary": "This paper studies an abstract problem of model valuation.\nThey propose a notion of valuing models, called the *Model Shapley Value*, based on the classic notion of Shapley Value from the literature on cooperative games.\nAdditionally, the work proposes an abstraction for models, a *Dirichlet abstraction*, that is meant to enable efficient comparison of different models.\nThey propose a method of learning to valuate models, and evaluate their method empirically.", "questions": "Many of my questions are listed above.\n- What is the task at hand?\n- What is the key goal of this paper?\n- What are the key contributions to achieving this goal?\n\nI believe that there is probably something interesting happening in this paper, but I am not able to evaluate due to my lack of clarity on the basic objects studied.", "limitations": "Adequately addressed.", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 1, "contribution": 3, "strengths": "The problem of valuing different predictive models is well-motivated.  This paper nicely motivates the study of valuation of models, rather than simply data, on a number of axes.  The reliance on Shapley value is standard in the ML literature, but also well-motivated, due to the strong theoretical guarantees and study of Shapley.", "weaknesses": "Major concern:  I find the presentation of the paper to be a substantial barrier to understanding and evaluating the work.\n\nConcretely, the task at hand of model valuation is never actually described formally.\n- What is the task?  What is the input and output?\n- What is the trivial solution?\n- Why is the proposed solution better?\n\nUnfortunately, I do not understand the description of Dirichlet abstractions.\n- Again, what is the point of the abstraction?  We are representing a model as a Dirichlet probability distribution?  Why?\n- What is $\\mathbf{M}_i(x)$?  Specifically, what is $x$ here?  Is this the induced distribution on predictions given a randomly sampled input $x_j$?\n- Why can this be viewed as a Dirchlet distribution?  I see the citation, but it feels like a substantial enough point that it should be justified and motivated in the text of this paper.\n- \"$\\mathbb{Q}_i$ encodes the predictive accuracy and certainty of $\\mathbf{M}_i$ through a theoretical connection...\" << Is this a formal statement?  How does this work?  How is it justified?\n\n\nMy lack of understanding of the basic building blocks behind this paper makes it difficult for me to evaluate the theory or experiments appropriately.  I've spent a decent amount of time trying to understand, and it is still not clear to me what is happening.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Review Coej for reviewing our paper and for finding our problem \"well-motivated\" and the usage of the Shapley value also \"well-motiavated\".\n\nWe wish to address the questions as follows.\n\nW1.\n\n> What is the task?  What is the key goal of this paper?\n\nAs pointed out by the reviewer:\n> This paper nicely motivates the study of valuation of models,\n\n__The task or key goal is to study the valuation of models, and proposes a suitable method for model valuation__, requiring us to resolve three main technical challenges as in Sec. 1:\n\n- (i) how to represent a model (which we call an abstraction) such that it is amenable to valuation? (lines 44-54);\n    \n- (ii) how to ensure equitability properties are satisfied, so as to ensure the fairness of a model marketplace? (lines 54-60);\n\n- (iii) in leveraging the Shaley value to satisfy these equitability properties, a computational challenge arises, how then to resolve this challenge? (lines 61 - 66).\n\n> What are the key contributions ... ?\n\n__Resolving the above three challenges constitutes the key contributions, as summarized in lines 67-86 of Sec. 1.__ To elaborate, as the reviewer pointed out, the Dirichlet abstraction is proposed to resolve (i) as a suitable model abstraction, that is amenable to valuation. Then, the Shapley value formulation is adopted via designing a specific function $\\nu$ (in Equ. (3) in line 195) to satisfy these equitability properties and resolve (ii); and a learning approach, by leveraging the analytic properties of the the proposed function $\\nu$ and Dirichlet abstractions, is proposed to resolve (iii).\n\n> What is the input and output?\n\nThe formal problem is described in Sec. 3.1. The input is $N$ models $\\\\{ \\mathbf{M}\\_1,\\ldots , \\mathbf{M}\\_i, \\ldots , \\mathbf{M}\\_{N} \\\\}$ with only black-box access.\n\nThe desired output consists of  __the definition of $\\phi\\_i$ for each model, which satisfies the equitability properties P1-P4__ (lines 181-184) and  __a computationally feasible implemetation to obtain $\\phi\\_i$__.\n\n> What is the trivial solution?\n\n__A trivial solution is to use the predictive accuracy of a model as its value (lines 47-48)__.\n\n> Why is the proposed solution better?\n\n__Lines 48-53 highlight why the trivial solution is too reductive and thus missing other key aspects of the model__, such as its predictive certainty. Our proposed solution is shown to be consistent with both the predictive accuracy (Figure 3 left) and certainty (Figure 3 right). Moreover, our proposed solution is extendable to more sophisticated evaluation criteria (Sec. 3.2 and Table 1).\n\nW2.\n\n> Again, what is the point of the abstraction?\n\n__The abstraction is meant to be formal representation of the model__. Recall that we consider the black-box access setting (motivated in lines 33-38), which means only queries and the model predictions are observed. In this way, without a formal representation, it is difficult to provide a formal treatment of the problem of model valuation.\n\n> We are representing a model as a Dirichlet probability distribution? Why?\n\nAs pointed out by the reviewer, \"Additionally, the work proposes an abstraction for models, a Dirichlet abstraction, that is meant to enable efficient comparison of different models.\"\nThere are two main reasons: (i) __to enable efficient comparison__ by using the closed-form expressions of the Hellinger distance between Dirichlet distributions; (ii) __to provide theoretical analysis__ by leveraging the analytic properties of the Hellinger distance between Dirichlet distributions, as in Proposition 2 and Theorem 1.\n    \n> What is $\\mathbf{M}_i(x)$?\n\nA model $\\mathbf{M}\\_i$ is a mapping, namely $\\mathbf{M}\\_i: \\mathcal{X} \\mapsto \\triangle(C)$.\n\n> Specifically, what is $x$ here? Is this the induced distribution on predictions given a randomly sampled input $x_j$? \n\n$x \\in \\mathcal{X}$ is a specific feature vector, viewed a a realization of a random variable $X \\sim P_X$ whose $\\text{supp}(X) = \\mathcal{X}$ (and $P_X$ is represented empirically by the task). Then, $\\mathbf{M}\\_i(X)$ is an induced distribution (by $P_X$) over $\\triangle(C)$.\n\n> Why can this be viewed as a Dirchlet distribution?  ...\n\nBecause (i) __the support of $\\mathbf{M}\\_i(X)$ matches _exactly_ to that of a Dirichlet distribution__; (ii) from a statistical viewpoint, __the Dirichlet distribution is a suitable modeling choice for the distribution $\\mathbf{M}\\_i(X)$ [57]__. We will make this point explicit in our revision.\n\n> \"$\\mathbb{Q}_i$ encodes the predictive accuracy and certainty of $\\mathbf{M}_i$ through a theoretical connection...\" << Is this a formal statement? How does this work? How is it justified?\n\n__The formal result is Proposition 2__. The insight is that the cross entropy (CE) loss of a model encodes the predictive accuracy and certainty (see below). The CE loss is used to construct upper and lower bounds for our proposed method (i.e., $\\nu$ in Equ.(3)). Hence, our proposed method also encodes the predictive accuracy and certainty.\n\nRecall the CE loss of a $C$-dimensional predicted probability vector $\\hat{y}$ w.r.t. the one-hot encoded true label $y$: $-\\sum_{k=1}^C y_k \\times \\ln(\\hat{y}_k) $. \n\nW.l.o.g., assume that $y_{1}=1$ (i.e., the correct class is the first class).\n\n- For two predictions $[0.9,0.1, 0,\\ldots,0]$ vs.$[0.1, 0.9,0, \\ldots, 0]$. The CE losses are $0.105$ and $2.30$, respectively. Note that the first prediction is correct while the second in incorrect and that both predictions are \"equally certain\". Hence, __A higher predictive accuacy implies a lower CE__.\n\n- For two predictions $[0.9,0.1, 0,\\ldots,0]$ vs.$[0.6, 0.4,0, \\ldots, 0]$.  The CE losses are $0.105$ and $0.511$, respectively. Note that both predictions are correct while the first prediction is \"more certain\". Hence, __A higher predictive certainty implies a lower CE__, if the prediction is correct.\n\nWe hope our clarifications have addressed your questions and helped improve your opinon of our work."}]

Author rebuttal:
We thank all the reviewers for taking the time to review our paper and providng the detailed comments and positive feedback:

- Our studied problem is interesting and well-motivated (Reviewers tCDm, bude & Coej);

- Our approach is novel and creative, and the quality of our research is high (Reviewer RTGV); our solution is solid and well-thought-out, and our theoretical statements are sound (Reviewer bude); our approach using the Shapley value is well-motivated (Reviewer Coej);

- Our paper is well structured and written (Reviewers tCDm & bude);

- Our paper presents a valuable addition to the machine learning and statistical literature (Reviewer RTGV).

In our prepared response to your feedback and questions, with main points summarized below, we have
- Provided additional clarifications on some theoretical results: 
    
    - On the effect of $Z$ in Equ.(2), condition in Prop. 4, definition of $\mathcal{Q}^*$ and our proposed learning approach (Reviewer tCDm);
    - On the homogeneity of Dirichlet abstractions and other assumptions (Reviewer RTGV);
    - On the implications of P4 and P3 (Reviewer bude);  
    - On the formal problem setting, theoretical motivation and justification of the proposed Dirichlet abstraction, to highlight our main contributions (Reviewer Coej).

- Provided discussion from a practical viewpoint: 

    - On the application to the simpler binary classification, and extension to combining different errors (Reviewer tCDm);
    - On the preparation of the query set (Reviewers RTGV & bude);
    - On the trade-off between query set size and abstraction level, and dealing with an unbalanced query set with very sparse data for some classes (Reviewer RTGV); 
    - On the bounds of query set size for potential reverse-enginnering (Reviewer bude)  .

We thank all the reviewers for reviewing our paper and their reviews, and hope that our response has clarified your questions and helped raise your opinions of our work. We are happy to provide further clarifications during the discussion period.

Meta-review: All of the reviewers agree that the paper proposes an interesting well-motivated problem and offers a good initial solution that may be of interest to the community. The reviewers also found the Dirichlet abstraction proposed in the paper a novel and effective contribution. However, the reviewers identified several opportunities to improve the paper. The authors clarified several questions around the theoretical results, and discussed the practicality of the method, which should be added to the paper to improve its exposition. One of the reviewers astutely pointed out a subtle distinction between black-box access (claimed in the paper) and grey-box access (implicitly required to construct common query sets for a task). These nuances are important to clarify, even in a first paper on a proposed problem formulation.
Decision: Accept (poster)