SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Xinyi Xu†§, Thanh Lam†, Chuan-Sheng Foo§, Bryan Kian Hsiang Low†
Dept. of Computer Science, National University of Singapore, Republic of Singapore†
Inst. for Infocomm Research; Centre for Frontier AI Research, A\*STAR, Republic of Singapore
xinyi.xu@u.nus.edu, lamchithanh1997@gmail.com
foo_chuan_sheng@i2r.a-star.edu.sg, lowkh@comp.nus.edu.sg
Valuation methods of data and machine learning (ML) models are essential to the establishment of AI marketplaces. Importantly, certain practical considerations (e.g., operational constraints, legal restrictions) favor the use of model valuation over data valuation. Also, existing marketplaces that involve trading of pre-trained ML models call for an equitable model valuation method to price them. In particular, we investigate the black-box access setting which allows querying a model (to observe predictions) without disclosing model-specific information (e.g., architecture and parameters). By exploiting a Dirichlet abstraction of a model's predictions, we propose a novel and equitable model valuation method called model Shapley. We also leverage a Lipschitz continuity of model Shapley to design a learning approach for predicting the model Shapley values (MSVs) of many vendors' models (e.g., 150) in a large-scale marketplace. We perform extensive empirical validation on the effectiveness of model Shapley using various real-world datasets and heterogeneous model types.
Data valuation methods have important roles in extensive applications such as dataset pricing in a data marketplace [1], evaluating the contributions of data from multiple collaborating parties [71], and identifying valuable data (or filtering out less valuable data) for training higher-quality machine learning (ML) models [20]. However, the following practical considerations favor the use of model valuation over data valuation: (A) Data valuation can be operationally infeasible due to the characteristics of training data, such as being massively distributed over millions of sources [73], enormous in size (e.g., 400 billion byte-pair-encoded tokens for a language model [7]), and/or transient (e.g., the data for online learning are not stored persistently [14]). In contrast, a trained model is not distributed by nature (and hence does not need to be aggregated from distributed sources), often much smaller in size than the training data (e.g., less than  $1\%$ ),<sup>1</sup> and usually stored persistently for inference. (B) Data privacy regulations (e.g., GDPR) can be a legal impediment to data valuation in designing fair payments for ML-based collaborations (e.g., in medicine [16] or cyber-defense [25]) because the centralization of (possibly) private data, which seems necessary for data valuation [39], is prohibited by such regulations. In contrast, the ML models trained on private data without centralization [40] can be available for valuation. (C) Existing marketplaces that involve trading of pre-trained ML models (e.g., AWS marketplace, Modzy) call for an equitable model valuation to price them. These practical considerations motivate the use of equitable model valuation over data valuation.
<sup>1</sup>The GPT-3 language model contains 175 billion parameters trained on 45 TB of text data [7]. It is of size  $1.75 \times 10^{11} \times (2 \times 2) \times 10^{-9} = 350$  GB (at 16 bit float precision) which is less than  $1\%$  of training data size.
For model valuation, practitioners (e.g., model vendors or clinicians who use the trained models) would likely prefer their models to be examined via only black-box access,[2] i.e., by querying the model with input for the corresponding predictions without observing its internal mechanism [8]. It does not disclose the proprietary model information (for model vendors) nor the sensitive information contained in the model (for clinicians), and provides an added advantage of the model valuation being model-agnostic since no model-specific information is used. However, these make model valuation challenging by restricting the available model information to only a selected query set (e.g., a validation set) with the observed predictions. In contrast, with white-box access, there is more model information available (e.g., information-theoretic criteria for probabilistic models [71] or norms of the parameters for deep neural networks [22]) to assess model complexity or certain analytical properties (e.g., uniqueness of an optimal model in logistic regression).
Intuitively, the value of a model depends on its intended task: For example, a model trained to classify MNIST digits [45] is not very valuable to a clinician trying to classify diagnostic scans. This dependency is useful in practice for selecting the most 'valuable' model for the desired task (i.e., as a query set). We refer to task and query set interchangeably hereafter. While the predictive accuracy of a model on the task provides an intuitive value [66], it can be too reductive: Suppose that two models  $\mathbf{M}_i$  and  $\mathbf{M}_{i'}$  have identical predictive accuracies on the task but  $\mathbf{M}_i(\mathbf{M}_{i'})$  makes highly (barely) certain predictions, i.e.,  $\mathbf{M}_i(\mathbf{M}_{i'})$  predicts the true class with over  $90\%$  (barely over  $50\%$ ) probability. Intuitively, the values for  $\mathbf{M}_i$  and  $\mathbf{M}_{i'}$  should not be the same, which cannot be achieved with accuracy alone, hence suggesting that additional model information beyond accuracy is required. (1) What then should be a suitable abstraction of a model w.r.t. a task, for model valuation?
Satisfying certain equitability properties in model valuation is imperative in the application of model pricing to ensuring a fair market. For instance, consistent valuation of identical models is important as it is unfair to price them differently otherwise. Furthermore, the market economy dictates that the value of a model depends on other available models (e.g., more available substitutes cause depreciation), which is important to guarantee a fair market by preventing price fixing (i.e., an exploitative pricing scheme often made illegal by anti-trust laws). (2) How then should model valuation be formulated to satisfy these equitability properties?
In particular, the Shapley value is shown to satisfy these equitability properties but raises another practical challenge that computing it exactly incurs  $\mathcal{O}(2^{N})$  time where  $N$  is the number of models in a marketplace. So, for a given computational budget, there is a fundamental ceiling to the size of the marketplace such that including more models causes the marketplace to be unable to determine their values (within reasonable time). (3) How can the desirable equitability properties of the Shapley value still be exploited without imposing a significant restriction on the size of the marketplace?
This paper presents a novel model-agnostic valuation framework to tackle the above challenges. For (1), we use an insight that the predictive pattern of a model (w.r.t. a task) is an suitable abstraction for valuation, especially since the black-box access rules out other model information. Specifically, we use a Dirichlet distribution to approximate a model's predictive pattern/distribution w.r.t. a task, which we call the Dirichlet abstraction of this model to encode both its predictive accuracy and certainty (Sec. 2). We describe how to adjust the level of abstraction to trade off the amount of model information (i.e., higher abstraction level) for the availability of a smaller query set. Then, for (2), we observe that ensuring equitability requires a similarity measure of the models (e.g., to ensure identical models are valued consistently or to identify substitutes). We exploit the ability of the Dirichlet abstractions to preserve the similarity between models for proposing model Shapley as an equitable model valuation (Sec. 3). As an illustration, identical models produce identical Dirichlet abstractions which result in equal model Shapley values (MSVs). To address (3), based on the Dirichlet abstraction, we leverage a Lipschitz continuity of model Shapley to justify and propose a learning approach of training a model appraiser (i.e., a regression learner) on a small subset of models (and their MSVs) for predicting other models' MSVs to validate model Shapley's practical feasibility in a large-scale marketplace (Sec. 4), as empirically verified on real-world datasets and up to 150 heterogeneous models. We empirically validate that better predictive accuracy (e.g., F1 score) due to better training data, more suitable model types, and/or higher predictive certainty result in higher MSVs, and demonstrate a use case for identifying a valuable subset of models from the marketplace to construct a larger learner (e.g., random forests) (Sec. 5).
2This is different from black-box models (e.g., deep neural networks) which are difficult to interpret or explain even with access to the internal mechanism such as architecture and parameters.
2 Dirichlet Abstraction of a Model
We give some preliminaries on the Dirichlet distribution and Hellinger distance below:3
Definition 1 (Dirichlet distribution [63]). The probability density function of a  $C$ -dimensional Dirichlet random variables  $Z \sim \mathrm{Dir}(\pmb{\alpha})$  with parameters  $\pmb{\alpha} = [\alpha_{1}, \dots, \alpha_{C}] \in (0, \infty)^{C}$  is  $p(z; \pmb{\alpha}) = \prod_{k=1}^{C} z_{k}^{\alpha_{k}-1} / \left\{\left[\prod_{k=1}^{C} \Gamma(\alpha_{k})\right] / \Gamma\left(\sum_{k=1}^{C} \alpha_{k}\right)\right\}$  where  $\Gamma$  is the gamma function.
Definition 2 (Hellinger distance [27]). The Hellinger distance between distributions (whose probability density functions are)  $p$  and  $q$  is  $d_{\mathrm{H}}(p,q)\coloneqq [1 - \int \sqrt{p(x)q(x)}\mathrm{d}x]^{1 / 2}$ .
2.1 Dirichlet Abstraction and MLE
We will now describe an abstraction of a model via a Dirichlet approximation to the model's predictive pattern over some task and a maximum likelihood estimation (MLE) for it. Each (learnt)  $C$ -way classification model  $\mathbf{M}_i: \mathcal{X} \mapsto \triangle(C)$  is a mapping from the input space  $\mathcal{X}$  to the  $(C-1)$ -probability simplex  $\triangle(C)$  (i.e., space of  $C$ -dimensional probability vectors). Denote a random variable  $X \sim P_X$  whose support  $\operatorname{supp}(X) = \mathcal{X}$  is the input space, then its distribution  $P_X$  induces a predictive distribution  $P_{\mathbf{M}_i(X)}$  over  $\triangle(C)$ . Concretely,  $P_X$  is represented by a task/query set  $\mathcal{D} := \{(x_j \in \mathcal{X}, y_j \in \{1, \ldots, C\})\}_{j=1,\ldots,D}$  where each  $x_j$  is a realization of  $X \sim P_X$ , so that  $\mathbf{M}_i(x_j)$  is a realization of  $P_{\mathbf{M}_i(X)}$ .
Dirichlet abstraction. Note that the predictive distribution  $P_{\mathbf{M}_i(X)}$  (induced by  $P_X$ ) (i) has the exactly same support to that of a  $C$ -dimensional Dirichlet distribution, so  $P_{\mathbf{M}_i(X)}$  can be mathematically modeled using a Dirichlet distribution; and (ii) can be statistically modeled using a Dirichlet distribution because the predictive pattern of a classification model can be (statistically) characterized well with a Dirichlet distribution [68]. Informally, the "shape" of predictive distribution of a classification model is similar to that of a Dirichlet distribution. Hence, we let  $P_{\mathbf{M}_i(X)} = \mathbb{Q}_i := \mathrm{Dir}(\pmb{\alpha}_i)$ , whose parameters  $\pmb{\alpha}_i$  can be learnt using MLE (described later) and refer to  $\pmb{\alpha}_i$  or  $\mathbb{Q}_i$  as  $\mathbf{M}_i$ 's Dirichlet abstraction. In words, we abstract the predictive distribution  $P_{\mathbf{M}_i(X)}$  of the model  $\mathbf{M}_i$  induced by  $P_X$  into a Dirichlet distribution, hence the name Dirichlet abstraction. Note that the notational dependence on  $P_X$  (or  $\mathcal{D}$ ) is suppressed when the context is clear.
The proposed Dirichlet abstraction offers several important advantages (further elaborated in Sec. 3): (a) By design, this formulation replaces the heterogeneity in models with the homogeneity in their Dirichlet abstractions, and allows the subsequently proposed model valuation to be applicable to models of different types: The respective Dirichlet abstractions of a multi-layer perceptron and a logistic regression can be compared directly. (b)  $\mathbb{Q}_i$  encodes the predictive accuracy and certainty of  $\mathbf{M}_i$  through a theoretical connection between the Hellinger distance  $d_{\mathrm{H}}$  of two Dirichlet abstractions and the cross entropy of  $\mathbf{M}_i$  (formalized by Proposition 2). (c) Importantly for model valuation, an appealing analytic property of the Dirichlet distribution is that  $d_{\mathrm{H}}$  of two Dirichlet abstractions can be evaluated in closed-form and incurs  $\mathcal{O}(1)$  computational complexity.
MLE. We adopt the MLE approximation of  $\alpha_{i}$  using the predictions of  $\mathbf{M}_i$  on  $\mathcal{D}$  since the log-likelihood function is concave with a unique maximizer and can thus be efficiently optimized [57]. Since  $\mathbf{M}_i(x_j)$  denotes a realized predictive probability vector of  $\mathbf{M}_i(X) \sim \mathrm{Dir}(\alpha_i)$ , we use the observed sufficient statistics  $\log \bar{h}_i \coloneqq D^{-1} \sum_{j=1}^{D} \log \mathbf{M}_i(x_j)$  (i.e., with an element-wise log operation) to derive the log-likelihood as follows [36]:
$$
F (\boldsymbol {\alpha}, \bar {h} _ {i}) := D \left[ G (\boldsymbol {\alpha}) + \sum_ {k = 1} ^ {C} \left(\alpha_ {k} - 1\right) \log \bar {h} _ {i, k} \right] \tag {1}
$$
where  $G(\alpha) \coloneqq \log \Gamma\left(\sum_{k} \alpha_{k}\right) - \sum_{k} \log \Gamma(\alpha_{k})$ . From (1),  $\bar{h}_{i}$  arises as an alternative (to  $\alpha_{i}$ ) abstraction of  $\mathbf{M}_{i}$  w.r.t.  $\mathcal{D}$ . Hence, we compare  $\alpha_{i}$  and  $\bar{h}_{i}$  theoretically (Proposition 3 in App. A) and empirically (Sec. 4.2).
3Additional discussion on the choice (e.g., a comparison with the Chernoff distance) and suitability of Hellinger distance is provided in App. A.
2.2 Class-specific Dirichlet Abstraction
Since the Dirichlet abstraction  $\mathbb{Q}_i$  of  $\mathbf{M}_i$  w.r.t. some task  $\mathcal{D}$  does not explicitly account for the class information in  $\mathcal{D}$  (i.e., the true classification label  $y_{j}$  of each data  $x_{j}$ ), models with certain (different) predictive patterns can be indistinguishable. This can be problematic because the Dirichlet abstractions of an optimal model and another model with zero predictive accuracy but a specific predictive pattern can identical, making it difficult to distinguish between these two models.
Suppose that there are  $C = 3$  classes in a balanced query set  $\mathcal{D}$  (i.e., equal data size for each class). For some  $\mathbf{M}_i$  (with optimal predictive accuracy), artificially construct  $\mathbf{M}_{i'}$  to have the same Dirichlet abstraction, but zero predictive accuracy in the following way: Since  $\mathcal{D}$  is balanced, group the data into triplets  $\{(x_{j,1}, x_{j,2}, x_{j,3})\}_{j=1,\dots,D/3}$  for the input data from 3 classes. Next, define  $\mathbf{M}_{i'}(x_{j,c}) := \mathbf{M}_i(x_{j,(c \bmod 3)+1})$  for  $c = 1,2,3$ . Intuitively, for each prediction (i.e., a 3-dimensional probability vector in a 2-simplex) by  $\mathbf{M}_i$ ,  $\mathbf{M}_{i'}$  makes an identical one but on an input data from a wrong class, i.e., we 'shift'  $\mathbf{M}_i$ 's predictions by one class to construct the predictions of  $\mathbf{M}_{i'}$ . Obviously, the predictive accuracy of  $\mathbf{M}_{i'}$  is zero, but its predictions (in aggregation) are identical to those of  $\mathbf{M}_i$ , which means  $\bar{h}_i = \bar{h}_{i'}$  in (1), hence resulting in  $\mathbb{Q}_i = \mathbb{Q}_{i'}$ . In Fig. 1: Plots 1 and 2 are the predictions of  $\mathbf{M}_i$  and  $\mathbf{M}_{i'}$  respectively, on  $\mathcal{D}$ , which are (visually) indistinguishable. Plots 3 and 4 are samples from  $\mathbb{Q}_i$  and  $\mathbb{Q}_{i'}$  respectively, which are also (visually) indistinguishable. The implication is that, in this case (of  $\mathbf{M}_i$  and such specially constructed  $\mathbf{M}_{i'}$ ), the highest level of abstraction (i.e., using the entire query set  $\mathcal{D}$  to construct Dirichlet abstractions  $\mathbb{Q}_i$  and  $\mathbb{Q}_{i'}$ ) is not effective to distinguish between  $i$  and  $i'$ . Hence, we adopt a lower level of abstraction, described next.
The remedy is the so-called class-specific Dirichlet abstraction: Partition the query set  $\mathcal{D} = \cup_{k=1}^{C} \mathcal{D}_k$  where  $\mathcal{D}_k$  contains only data from the  $k$ -th class. The Dirichlet abstraction  $\alpha_{i,\mathcal{D}_k}$  of  $\mathbf{M}_i$  on  $\mathcal{D}_k$  is called the class-specific Dirichlet abstraction w.r.t. class  $k$ . Based on the example above, we verify using a small experiment. Over the entire  $\mathcal{D}$ , we obtain  $\alpha_i = \alpha_{i'} = [0.5040, 0.5339, 0.5306]$ . Restricting to query set  $\mathcal{D}_1$  from class 1 only, gives  $\alpha_{i,\mathcal{D}_1} = [21.8601, 2.2005, 2.0215]$  and  $\alpha_{i',\mathcal{D}_1} = [1.5817, 1.7576, 19.4037]$ , which are clearly different. In Fig. 1: Plots 5 and 6 are the predictions of  $\mathbf{M}_i$  and  $\mathbf{M}_{i'}$ , respectively, on  $\mathcal{D}_1$  while plots 7 and 8 are samples from  $\mathbb{Q}_{i,\mathcal{D}_1}$  and  $\mathbb{Q}_{i',\mathcal{D}_1}$ . We see that  $\mathbf{M}_i$  is clearly different from  $\mathbf{M}_{i'}$ , and  $\mathbb{Q}_{i,\mathcal{D}_1}$  is clearly different from  $\mathbb{Q}_{i',\mathcal{D}_1}$ , demonstrating the effectiveness of a lower level of abstraction via the class-specific Dirichlet abstraction.
Figure 1: The triangles denote the 2-simplex and each dot is a 3-dimensional probability vector. Plots 1-4 (5-8) use  $\mathcal{D}$  ( $\mathcal{D}_1$ ) as query set. Plots 1&5 (2&6) show predictions of  $\mathbf{M}_i$  ( $\mathbf{M}_{i'}$ ). Plots 3&7 (4&8) show randomly drawn samples from Dirichlet abstraction  $\mathbb{Q}_i$  ( $\mathbb{Q}_{i'}$ ). For example, plot 4 shows random samples from  $\mathbb{Q}_{i'}$  w.r.t.  $\mathcal{D}$ , while plot 5 shows predictions of  $\mathbf{M}_i$  w.r.t.  $\mathcal{D}_1$ .
Trade-off between level of Dirichlet abstraction and size of query set. Note that we need not stop at partitioning  $\mathcal{D}$  at the class level. For instance, a more refined partition can be first according to the classes and then certain groups of input feature values. Doing so produces Dirichlet abstractions with a lower level of abstraction (i.e., less abstract and containing more model information), but also requires a larger query set  $\mathcal{D}$  to begin with so that each smaller partitioned query set contains sufficient data (for predictions) to obtain an accurate MLE estimate. For an extremely refined partition of  $\mathcal{D}$  where each partitioned query set is very small (e.g., size  $\leq 5$ ), the obtained MLE estimates and the corresponding Dirichlet abstraction can be inaccurate and thus not useful. It is thus important to find a suitable trade-off between the abstraction level vs. query set size. In particular, in Sec. 5, we observe that partitioning  $\mathcal{D}$  according to the classes provides such a suitable trade-off: E.g., it can distinguish models with almost equal predictive accuracy due to the high class imbalance
<sup>4</sup>On a balanced query set of size  $D = 900$ ,  $\mathbf{M}_i(x_j) \coloneqq G_j[\mathbb{1}(y_j = c) + \epsilon_c]_{c=1,2,3}$  with  $\epsilon_c \sim \mathcal{U}(0,0.2)$  and normalizing constant  $G_j$ , i.e.,  $\mathbf{M}_i$  always makes the correct classification but with a small additive and independent uniform noise  $\epsilon_c$ .
The work of [3] shows a sample complexity of  $|\mathcal{D}|$  being polynomial in  $C$  and an additive error.
in the data<sup>6</sup> but different F1 scores. Since the Dirichlet abstraction and the class-specific version are both Dirichlet distributions, and share the same theoretical properties (e.g., enabling a closed-form expression of the Hellinger distance), subsequently, we refer to Dirichlet abstractions generally (i.e., omitting the specific dependence on  $\mathcal{D}$  or  $\mathcal{D}_k$ ), unless otherwise specified.
3 Equitable Valuation via Model Shapley
We discuss and formalize several equitability properties to derive a general formulation called model Shapley to satisfy them. Then, we will specify the characteristic function and a precision-weighted fusion to encode a model's predictive accuracy and certainty into its MSV.
3.1 Equitability Properties and Model Shapley
Consider  $N$  models in a marketplace and denote  $\mathbf{M}_i$ 's equitable value by  $\phi_i$ . (P1) If a model  $\mathbf{M}_i$  has not been queried at all, then its value is indeterminate and we set  $\phi_i = 0$  by default. (P2) If two models  $\mathbf{M}_i$  and  $\mathbf{M}_{i'}$  give identical predictions (over the task), then their values are equal, i.e.,  $\phi_i = \phi_{i'}$ . (P3) If some buyer is interested in multiple tasks simultaneously and a model (from some vendor) performs very well on only one of the tasks, then it is unfair for the vendor to set the value/price solely based on this performance. Instead, an equitable value should be based on the model's joint performance on these tasks (e.g., a linear combination according to the buyer's interests in these tasks). (P4) The existence of perfect substitutes depreciates the value of a model. (P1)-(P4) are useful for equitable valuation in ML model marketplaces [51, 66]. To formalize these properties, some notations are needed: Let  $\nu : 2^{[N]} \mapsto \mathbb{R}$  denote a characteristic function (specified later) s.t.  $\nu(\mathcal{C})$  quantifies the value of a collection  $\mathcal{C} \subseteq [N] := \{1, \dots, N\}$  of models to capture the dependence of  $\mathbf{M}_i$ 's value on other existing models  $\mathbf{M}_{i'}$  (e.g., substitutes). Denote  $\mathbf{M}_i$ 's value by  $\phi_i \gets \Phi(i, \nu, \{\mathbf{M}_i\}_{i \in [N]})$  which is fully specified (up to linear scaling) by the properties:
P1 Null player:  $(\forall \mathcal{C} \subseteq [N] \setminus \{i\} \nu(\mathcal{C} \cup \{i\}) - \nu(\mathcal{C}) = 0) \Rightarrow \phi_i = 0$
P2 Symmetry:  $(\forall \mathcal{C} \subseteq [N] \setminus \{i, i'\}) \nu(\mathcal{C} \cup \{i\}) = \nu(\mathcal{C} \cup \{i'\})) \Rightarrow \phi_i = \phi_{i'}$ .
P3 Linearity:  $\forall \gamma, \gamma' \in \mathbb{R}$  ( $\nu_{\mathcal{D} \cup \mathcal{D}'} \coloneqq \gamma \nu_{\mathcal{D}} + \gamma' \nu_{\mathcal{D}'}$ )  $\Rightarrow \phi_i(\mathcal{D} \cup \mathcal{D}') = \gamma \phi_i(\mathcal{D}) + \gamma' \phi_i(\mathcal{D}')$ .
P4 Diminished marginal utility: Add a perfect substitute (i.e., duplicate/copy)  $\mathbf{M}_{i_c}$  of  $\mathbf{M}_i$  to the pool of  $N$  models already containing  $\mathbf{M}_i$  and denote the new pool by  $[N'] := [N] \cup \{i_c\}$ . Denote the value of  $\mathbf{M}_i$  w.r.t.  $[N]$  by  $\phi_i$  and w.r.t.  $[N']$  by  $\phi_i'$ . Then,  $\phi_i' \leq \phi_i$ .
Proposition 1. Properties P1, P2, and P3 fully specify  $\Phi(\cdot)$  up to a linear scaling  $Z$ :
$$
\phi_ {i} := Z \sum_ {\mathcal {C} \subseteq [ N ] \backslash \{i \}} \omega_ {\mathcal {C}} \left[ \nu (\mathcal {C} \cup \{i \}) - \nu (\mathcal {C}) \right] \quad \text {w h e r e} \omega_ {\mathcal {C}} := | \mathcal {C} |! \times (N - | \mathcal {C} | - 1)! / N!. \tag {2}
$$