Rashomon Capacity: A Metric for Predictive Multiplicity in Classification
Abstract
Predictive multiplicity occurs when classification models with statistically indistinguishable performances assign conflicting predictions to individual samples.
When used for decision-making in applications of consequence (e.g., lending, education, criminal justice), models developed without regard for predictive multiplicity may result in unjustified and arbitrary decisions for specific individuals.
We introduce a new metric, called Rashomon Capacity, to measure predictive multiplicity in probabilistic classification.
Prior metrics for predictive multiplicity focus on classifiers that output thresholded (i.e., 0-1) predicted classes.
In contrast, Rashomon Capacity applies to probabilistic classifiers, capturing more nuanced score variations for individual samples.
We provide a rigorous derivation for Rashomon Capacity, argue its intuitive appeal, and demonstrate how to estimate it in practice.
We show that
Rashomon Capacity yields principled strategies for disclosing conflicting models to stakeholders.
Our numerical experiments illustrate how Rashomon Capacity captures predictive multiplicity in various datasets and learning models, including neural networks.
The tools introduced in this paper can help data scientists measure and report predictive multiplicity prior to model deployment.
Introduction
Rashomon effect , introduced by Breiman [1], describes the phenomenon where a multitude of distinct predictive models achieve similar training or test loss.
Breiman reported observing the Rashomon effect in several model classes, including linear regression, decision trees, and small neural networks.
In a foresighted experiment, Breiman noted that, when retraining a neural network 100 times on three-dimensional data with different random initializations, he “ found 32 distinct minima, each of which gave a different picture, and having about equal test set error ” [1, Section 8].
The set of almost-equally performing models for a given learning problem is called the Rashomon set [2,3].
We focus on a facet of the Rashomon effect in classification problems called predictive multiplicity .
Predictive multiplicity occurs when competing models in the Rashomon set assign conflicting predictions to individual samples [4].
Fig.
1 presents an updated version of Breiman’s neural network experiment and illustrates predictive multiplicity in three classification tasks with different data domains and neural network architectures.
Here, models that achieve statistically-indistinguishable performance on a test set assign wildly different predictions to an input sample.
If predictive multiplicity is not accounted for, the output for this sample may ultimately depend on arbitrary choices made during training (e.g., parameter initialization).
Predictive multiplicity captures the potential individual-level harm introduced by an arbitrary choice of a single model in the Rashomon set.
When such a model is used to support automated decisionmaking in sectors dominated by a few companies or Government—labeled Algorithmic Leviathans in [5, Section 3]—predictive multiplicity can lead to unjustified and systemic exclusion of individuals from critical opportunities.
For example, an algorithm used for lending may deny a loan to a specific 36th Conference on Neural Information Processing Systems (NeurIPS 2022).
[Figure]
applicant.
However, during model development, there may have been a competing model which performs equally well on average, yet would have approved the loan for this individual.
As another example, Governments are increasingly turning to algorithms for grading exams that grant access to higher-level education (see, e.g., UK [6] and Brazil [7]).
Here, again, accounting for predictive multiplicity is critical: an arbitrary choice of a single model in the Rashomon set may lead to an unwarranted restriction of educational opportunities to an individual student.
We introduce new methods for measuring and reporting predictive multiplicity in probabilistic classification.
First, we postulate several properties that a predictive multiplicity metric must satisfy to simplify its interpretation by stakeholders.
We then provide a new predictive multiplicity metric called Rashomon Capacity .
Unlike prior metrics restricted to thresholded scores (i.e., decisions),
Rashomon Capacity captures more nuanced score variations among models in the Rashomon set.
Communicating score variations among models in the Rashomon set helps stakeholders understand whether a model’s output depends on randomness in training.
We show that Rashomon Capacity of an input sample can be entirely captured by at most c models in the Rashomon set, where c is the number of predicted classes, regardless of the size of the Rashomon set .
Remarkably, the computation of Rashomon Capacity also sheds light on a strategy for resolving predictive multiplicity.
Instead of releasing a single model, we provide a greedy algorithm for identifying a subset of models in the Rashomon set that captures most of the score variations across a dataset.
These models can be communicated to a stakeholder, empowering them to decide how to resolve conflicting scores via, e.g., randomization [5] and bagging [1].
In summary, our main contributions include:
We introduce a new score-based metric for quantifying predictive multiplicty in probabilistic classifiers called Rashomon Capacity.
Rashomon Capacity can be applied to both raw and thresholded (i.e. 0-1) scores.
We systematically discuss the challenges of computing a multiplicity metric, and demonstrate how Rashomon Capacity can be computed in practice.
We describe a methodology for reporting predictive multiplicity in probabilistic classification using Rashomon Capacity, with examples on different datasets and models.
We advocate that predictive multiplicity must be reported to stakeholders in, for example, model cards [13].
We propose a procedure for resolving predictive multiplicity in probabilistic classifiers.
Even though the Rashomon set may span a large (potentially uncountable) number of models, we show
that the score variation for a sample is fully captured by a small subset of models in the Rashomon set.
Communicating these predictions to stakeholders can empower them to decide how to resolve predictive multiplicity.
Omitted proofs, additional explanations and discussions, details on experiment setups and training, and additional experiments are included in Supplementary Materials (SM).
Code to reproduce our experiments are available at https://github.com/HsiangHsu/rashomon-capacity.
Background and related work
Notation.
We consider a dataset D
= { ( x i , y i ) } ni
=1 , e.g., a training or test set, for a classification task with c classes/labels.
Each sample pair ( x i , y i ) is drawn i.i.d.
from P X,Y with support X × ∆ c .
Here, ∆ c ≜ { ( r 1 , · · · , r c ) ∈ [0 , 1] c ; (cid:80) ci =1 r i = 1 } denotes the c -dimensional probability simplex.
Let e k be a length- c indicator vector with one in the k th position and zero elsewhere, i.e., [ e k ] k = 1 , and [ e k ] j = 0 ∀ j ̸ = k , where [ · ] j denotes the j th entry of a vector.
Each y i is one-hot encoded, i.e., y i ∈ { e k } ck
=1 . 1 ( · ) denotes the indicator function.
We denote by H a hypothesis space, i.e., a set of candidate probabilistic classifier is parameterized by θ ∈ Θ ⊆
R
d that approximate P Y | X = x
i , i.e., H ≜ { h θ : X → ∆ c : θ ∈ Θ } .
The loss function used to evaluate model performance is denoted by ℓ : ∆ c × ∆ c → R + (e.g., cross-entropy) and
L ( h θ ) ≜
E P X,Y [ ℓ ( h θ ( X ) , Y )]
the population risk.
As usual, the population risk is approximated by the empirical risk ˆ L ( h θ ) ≜ 1 n (cid:80) ni
=1 ℓ ( h θ ( x i ) , y i ) .
Rashomon set, Rashomon ratio, and pattern Rashomon ratio.
We define the Rashomon set as the set of all models in the hypothesis space that yield similar average loss.
Formally, given a
Rashomon parameter ϵ ≥ 0 , the Rashomon set is defined as an ϵ -level set [3]:
[Equation]
Note that the Rashomon set is determined by the hypothesis space H , the Rashomon parameter ϵ , and also implicitly by the data distribution due to the evaluation of L ( h θ ) .
The cardinality |R ( H , ϵ ) | or the volume 1 vol ( R ( H , ϵ )) of the Rashomon set (depending on whether the Rashomon set has finite elements) can be used to quantify the size of the Rashomon set.
Given R ( H , ϵ ) , the Rashomon ratio [3, Defn. 2] is defined as ˆ R ( H , ϵ ) ≜ vol ( R ( H ,ϵ )) vol ( H ) .
ˆ R ( H , ϵ ) represents the fraction of models in the hypothesis space that fit the data about equally well.
A large Rashomon ratio indicates high multiplicity.
Moreover, models with various desirable properties, such as better generalizability, can often exist inside a large Rashomon set.
Similar to the Rashomon ratio, pattern Rashomon ratio [3, Defn. 12] is defined as the ratio of the count of all possible binary predicted classes given by the functions in the Rashomon set to that given by the functions in the hypothesis space.
The complexity of computing pattern Rashomon ratio grows exponentially with the number of samples, yielding an “expensive” metric for predictive multiplicity when applied to large datasets.
Ambiguity and discrepancy.
Instead of characterizing multiplicity in the hypothesis/ parameter space, Marx et al.
[4] measures multiplicity in terms of the thresholded outputs (i.e., predicted classes) of a classifier, and proposed two metrics: ambiguity and discrepancy .
Ambiguity is the proportion of samples in a dataset that can be assigned conflicting predictions by competing classifiers in the
Rashomon set.
Discrepancy is the maximum number of predictions that could change in a dataset if we were to switch between models within the Rashomon set.
More precisely, given a base model ˆ h , the ambiguity α ϵ (ˆ h ) and the discrepancy δ ϵ (ˆ h ) are respectively defined as
[Equation]
For linear classifiers, both quantities in (2) can be estimated by mixed integer programming [4,
Section 3].
A small ϵ could still lead to a large ambiguity, see Section SM.
2.2 for a discussion.
Measuring predictive multiplicity of probabilistic classifiers
The metrics in (2) for predictive multiplicity are based on predicted classes.
Thus, they require finding the arg max or thresholding the scores at the output of a classifier.
In probabilistic classification, thresholding may mask similar predictions produced by competing models and artificially increase multiplicity: output scores can be almost equal across different classes, yet the (thresholded) predicted classes can be very different.
For example, two scores [0 . 49 , 0 . 51] and [0 . 51 , 0 . 49] for a binary classification problem can lead to entirely different predicted classes after thresholding—1 and 0, respectively—and ultimately overestimate predictive multiplicity (see Fig.
2 (Left) for another multi-class example).
In fact, predictive multiplicity metrics based on predicted classes may yield multiplicity even for a single fixed model when, for example, the threshold criteria for output scores is changed.
This subtle, yet important difference motivates us to reconsider existing metrics and introduce a new predictive multiplicity metric that is applicable to both output scores and decisions (cf.
Fig.
2 (Right) for an overview).
We begin this section by first outlining desirable properties of predictive multiplicity metrics for probabilistic classifiers.
These properties motivate our definition of Rashomon Capacity and provide guidelines for the creation of new multiplicity metrics in future research.
Motivated by the potential individual-level harm incurred by an arbitrary choice of model in the Rashomon set, we focus on persample multiplicity metrics.
We formally define Rashomon Capacity in terms of the KL-divergence between the output scores of classifiers in the Rashomon set.
We then use Rashomon Capacity to define a predictive multiplicity metric for individual samples in a dataset.
We present these definitions in the ideal case where the population loss can be computed exactly, and discuss empirical approximation in Section 4.
For more extensions and geometric interpretations of Rashomon Capacity, see Section SM.
2.3 and SM.
2.4 respectively.
3.1 Properties of multiplicity metrics for probabilistic classifiers
Consider a fixed distribution P X,Y and a corresponding Rashomon set R ( H , ϵ ) for a classification problem with c classes.
We collect all possible output scores for a sample x i ∈ D and define the ϵ -multiplicity set as
[Equation]
Let m ( · ) be a measure of predictive multiplicity, and m ( M ϵ ( x i )) be the predictive multiplicity of sample x i .
Which properties should m ( · ) have?
Ideally, we expect m ( M ϵ ( x i )) to be a bounded value in [1 , c ] , since at least one class is assigned to sample x i , and at most c different classes could be assigned to x i .
Moreover, if m ( M ϵ ( x i )) =
1 (i.e., a predictive multiplicity of 1), then one would expect that only one score is produced for x i and, thus, all predictions in M ϵ ( x i ) are exactly the same.
Similarly, if m ( M ϵ ( x i ))
=
c (i.e., predictive multiplicity equal to the number of classes), then there must exist c models { h 1 , · · · , h c } ⊆ R ( H , ϵ ) such that h j ( x i ) =
e j .
In other words, each of the c classes can be assigned to the sample, yielding a predictive multiplicity of c .
Finally, m ( M ϵ ( x i )) should be monotonic in M ϵ ( x i ) , i.e., if M ϵ ( x i ) ⊆
M ′ ϵ
( x i ) , then m ( M ϵ ( x i ))
≤ m ( M ′ ϵ ( x i )) .
We summarize these desirable properties of predictive multiplicity metrics in the following definition.
Definition 1.
Let P c be the power set 2 of the probability simplex ∆ c , we say that a function m : P c
→ R is a predictive multiplicity metric 3 if for any A , B ∈ P c
1. 1 ≤ m ( A ) ≤ c ; 2.
m ( A )
=
if and only if |A| ≤ 1 ; 3.
m ( A ) =
c if and only if e k ∈ A for k ∈ [ c ] , i.e., A contains the corner points of ∆ c ; 4.
m ( A ) ≤ m ( B )
if A ⊆ B .
We introduce next a predictive multiplicity metric called Rashomon Capacity that satisfies all properties above.
In the rest of the paper, when the ϵ -multiplicity set M ϵ ( · ) is clear from context, we use m ( x i ) as shorthand for m ( M ϵ ( x i )) .
3.2 Rashomon Capacity
Our goal is to quantify predictive multiplicity in terms of the score variations assigned to each sample
x i in a dataset D , given a Rashomon set R ( H , ϵ ) and the corresponding ϵ -multiplicity set M ϵ ( x i ) .
Note that an element in M ϵ ( x i ) is a probability distribution over c classes.
Thus, it is natural to adopt divergence measures for distributions to capture the “variation” of scores in M ϵ ( x i ) .
From a geometric viewpoint, a larger spread in scores indicates a greater amount of predictive multiplicity for a given sample x i .
Assume a probability (or “weight”) distribution P M across models in R ( H , ϵ ) (and therefore each score in M ϵ ( x i ) ), where M denotes the random variable of selecting/sampling the models in the Rashomon set.
Intuitively, if P M assigns mass 1 to a single model and 0 to all other models in the Rashomon set, then the output of only one model is considered.
Alternatively, if P M is the uniform distribution, then the outputs of every model in the set are equally weighed.
Given a divergence measure between distributions d ( ·∥· ) , we quantify the spread of the scores in M ϵ ( x i ) by
[Equation]
Here, the minimizing q acts as a “center of gravity” or “centroid” for the outputs of the classifiers in the
Rashomon set for a chosen distribution P M across models.
Analogously, the quantity ρ ( M ϵ ( x i ) , P M ) can be understood as a measure of “spread” or “inertia” across model outputs.
We select the distribution P M that results in the largest spread in scores:
[Equation]
The missing element is the choice of divergence measure d ( ·∥· ) .
A natural candidate is crossentropy (or log-loss)
d ( h ( x i ) ∥ q ) =
− h
( x i )
⊤ log q , since this is the standard loss used for training and evaluating probabilistic classifiers.
Alas, the minimal cross-entropy min q − h ( x i )
⊤ log q = h
( x i )
⊤ log h ( x i ) is not 0 and depends on h ( x i ) .
Consequently, if one were to choose d ( ·∥· ) to be cross-entropy, the minimum value of (5) would not be consistent and would depend on x i — even when the outputs across all models in the Rashomon set match!
Thus, we shift cross-entropy by its minimum − h ( x i )
⊤ log h ( x i ) .
This results in KL-divergence as our divergence measure of choice:
[Equation]
= − h
( x i )
⊤ log q
[Equation]
h ( x i )
⊤ log h ( x i ) .
Putting it all together, we next formally define the spread in scores measured using KL-divergence as Rashomon Capacity .
Definition 2.
Given a sample x
i and the corresponding ϵ -multiplicity set M ϵ ( x i ) , the Rashomon
Capacity 4 is defined as
[Equation]
The exponent C ( M ϵ ( x i )) is ubiquitous in information theory; in fact, C ( M ϵ ( x i )) is the channel capacity [14] of a channel P Y | M whose rows are the entries of M ϵ ( x i ) .
This connection motivates the name “Rashomon Capacity” and is useful for proving that m C ( x i ) is indeed a predictive multiplicity metric, as stated in the next proposition.
Proposition 1.
The function m C ( · ) = 2 C ( M ϵ ( · )) :
X → [1 , c ] satisfies all properties of a predictive multiplicity metric in Definition 1.
In contrast, ambiguity and discrepancy in (2) do not satisfy the properties of a predictive multiplicity metric outlined in Definition 1.
An interesting connection between ambiguity and Rashomon Capacity is that ambiguity measures the fraction of samples in a dataset with non-zero Rashomon Capacity.
In addition, Rashomon Capacity is fundamentally different from the size of a Rashomon set, in the sense that a larger Rashomon set does not necessarily lead to a larger Rashomon Capacity.
Using a binary classification problem as an example, consider two Rashomon sets with scores
R 1 ( H , ϵ ) =
{ h 1 , h 2 , h 3 } , h 1 ( x i ) =
55] , h 2 ( x i ) =
50] , h 3 ( x i ) =
R 2 ( H , ϵ ) =
{ h 1 , h 2 } , h 1 ( x i ) =
15] , h 2 ( x i ) =
R 2 ( H , ϵ ) has a larger Rashomon Capacity than R 1 ( H , ϵ ) , albeit |R 2 ( H , ϵ ) | = 2 < |R 1 ( H , ϵ ) | = 3 .
3.3 Rashomon Capacity in score and decision domains
Rashomon Capacity is defined in terms of the raw outputs in ∆ c of a probabilistic classifier; therefore it can also be evaluated with decisions, since a decision, after one-hot encoding, still lies in the probability simplex (at a vertex).
This allows Rashomon Capacity to provide a more nuanced view of score variation, and can be used to identify the number of “conflicting” classes in the predictions produced by models in the Rashomon set (i.e., Rashomon Capacity is between 1 and c ).
Taking a ternary classification as an example, for three score vectors [0 . 49 , 0 . 51 , 0] and [0 . 51 , 0 . 49 , 0] , the Rashomon Capacity of the raw scores for these samples is close to 1.
For the thresholded scores ( [0 , 1 , 0] and [1 , 0 , 0] ) the Rashomon Capacity is now 2.
Since Rashomon Capacity is between 1 and 3, this indicates that the confusion is between 2 classes instead of 3.
In contrast, prior metrics that also operate on thresholded scores, such as ambiguity and discrepancy (2), only capture agreement among predictions.
In this sense, score-level metrics could potentially provide a finer characterization of predictive multiplicity.
This does not mean that multiplicity should only be reported in the score domain.
On the contrary, our suggestion is that multiplicity should be measured at both the score and threshold levels and reported to stakeholders, as scores and decisions paint different pictures on how models conflict.
Computational challenges of multiplicity metrics
The computation of any multiplicity metric requires an approximate characterization of the Rashomon set—even for simple hypothesis spaces such as logistic regression.
For instance, the computation of the Rashomon ratio involves estimating vol ( R ( H , ϵ )) , which is a level set estimation problem [15], and is computationally infeasible 5 when the hypothesis space H is large [16].
For a logistic regression, the exact form of the pattern Rashomon ratio is not tractable due to the non-linearity of the maximum likelihood ratio [17].
Similarly, the computation of ambiguity/discrepancy requires solving an optimization over the Rashomon set and can be computationally burdensome since 0-1 loss is not differentiable.
Given a dataset and a hypothesis space, there are two core challenges in computing any predictive multiplicity metric (see Definition 1).
The first challenge is selecting an appropriate Rashomon parameter ϵ , since the smallest achievable test loss L ( h θ ) is unknown and only empirically approximated using a dataset with finite samples.
The second challenge is approximating the Rashomon set without exhaustively searching the hypothesis space 6 .
Next, we discuss strategies for addressing these two challenges.
Selection of the Rashomon parameter ϵ .
The value of ϵ can be set relative to the performance of a reference model.
A natural choice of the reference model is the empirical risk minimizer, i.e., h θ ∗ , where θ ∗ ∈ arg min θ ∈ Θ ˆ L ( h θ ) (see Section 2 for notation).
Here, we can set ϵ
=
ˆ L ( h θ ∗ ) + ϵ ′ with ϵ ′ ≥ 0 , i.e., the Rashomon parameter depends on the minimum empirical loss.
The parameter ϵ ′ , in turn, can be selected in terms of the upper boundary of a confidence interval around the empirical minimum.
For example, the confidence interval can be estimated via bootstrapping (see
Fig.
SM.
4.6 for bootstrapped loss intervals).
Naturally, ϵ ′ depends on the size of the test set used for evaluating h ˆ θ —when the dataset has n samples, usually ϵ ′ will be of the order
O (1 / √ n ) [19].
If more samples are available to evaluate model performance (rendering narrower confidence intervals), this parameter’s value will decrease.
The discussion above motivates the one-sided definition for the Rashomon set in (1).
If one were able to find the unique classifier that outperforms all others in terms of population loss, then it is justified to use this classifier in practice.
In fact, this point is eloquently made in [5], which argues that there is no (moral) harm in using the most accurate classifier, i.e., selecting a classifier that has a provable smaller average loss (or higher accuracy) than another.
Existing works on multiplicity, e.g., [3] and [4], also adopt a one-sided Rashomon set definition.
The Rashomon subset.
With limited computational power and memory, exploring the Rashomon set, i.e., searching the hypothesis space H to find all models with test losses smaller than ϵ , is challenging.
For several hypothesis spaces (e.g., neural networks), we are only able to acquire a small number of models in the Rashomon set.
We denote the collection of these models as a Rashomon subset (cid:101) R ( H , ϵ ) .
The Rashomon subset can be used to approximate the “true” Rashomon set R ( H , ϵ ) in the evaluation of multiplicity metrics.
We can construct a Rashomon subset with K models by choosing the
Rashomon parameter ϵ
in terms of a reference model h θ ∗ , i.e.,
[Equation]
For example, the Rashomon ratio can be approximated as K/ vol ( H ) .
Similarly, we can optimize for ambiguity/discrepancy in (2) over h θ i ∈ (cid:101)
R
( H , ϵ ′ ) .
Finally, we can approximate the ϵ ′ -multiplicity subset (cid:102) M ϵ
( x i ) ≜ (cid:110)
h
θ i ( x i ); h ∈ (cid:101)
R ( H , ϵ ′ )
(cid:111) in (3) for Rashomon Capacity.
In this sense, evaluating multiplicity metrics boils down to finding a Rashomon subset (cid:101) R ( H , ϵ ′ ) and computing the metric for that set.
This is an approximation of the true multiplicity, yet becomes more accurate with an increasing K .
Next, we provide an algorithm based on model weight perturbation to find a Rashomon subset for estimating Rashomon Capacity for any differentiable model.
Note that, unless the Rashomon set can be fully characterized, all estimates of multiplicity metrics are underestimates based on approximating the true Rashomon set by a Rashomon subset.
We emphasize that identifying and disclosing predictive multiplicity — even if an underestimate — is still critical for models deployed in applications of individual-level consequence (e.g., healthcare, education, lending), and is better than the current practice reporting no multiplicity at all.
Computing Rashomon Capacity.
The definition of Rashomon Capacity in (6) does not assume a finite cardinality of the Rashomon set.
Remarkably, even when the Rashomon set has infinite cardinality, the value of Rashomon Capacity of a sample can be recovered by considering only a small number of models in the Rashomon set.
In fact, for each sample x i , there exists a ϵ -multiplicity subset of at most c models that fully captures the variation in scores.
This statement is formalized by the next proposition, which can be proven by applying Carathéodory’s theorem [20].
Proposition 2.
For each sample x i ∈ D , there exists a ϵ -multiplicity subset (cid:102) M ϵ
( x i ) ⊆
M ϵ ( x i ) with | (cid:102)
M ϵ ( x i ) | ≤ c that fully captures the spread in scores for x i across the Rashomon set, i.e.,
C ( (cid:102)
M ϵ ( x i ))
=
C ( M ϵ ( x i )) .
In particular, there are at most c models in a Rashomon subset (cid:101) R ( H , ϵ ) whose output scores yield the same Rashomon Capacity for x i as the entire Rashomon set.
Proposition 2 implies that, for each sample, there exists a Rashomon subset with c models that captures all score variations.
In other words, the value of Rashomon Capacity remains the same regardless if we measure multiplicity on this subset or on the entire Rashomon set.
This allows us to circumvent the task of characterizing the entire Rashomon set (which has potentially infinite models), and focus on identifying c models per sample that maximize score variations while still satisfying a target loss constraint.
We describe next a method (described in detail in Algorithm SM. 2) based on weight perturbation that obtains c models in the Rashomon subset for each sample.
Given a sample x i , we obtain models with output predictions p k by approximately solving the following optimization problem which maximizes the output score for class k :
[Equation]
To solve (9), for each k , we set the objective to be min θ ∈ Θ − [ h θ ( x i )]
k , compute the gradients, and update the parameter θ until L ( h θ ) >
ϵ .
Given a pre-trained model in the Rashomon set, (9) can be viewed as an adversarial weight perturbations ( AWP ) technique to explore the Rashomon set [21,22] (see Section SM.
2.6 for exact weight perturbation on logistic regression).
With the discrete ϵ -multiplicity subset obtained by solving (9), Rashomon Capacity can be computed by standard procedures such as the Blahut–Arimoto (BA) algorithm [23, 24].
The BA algorithm is a class of iterative algorithms for numerically computing discrete channel capacity (or more generally, the rate-distortion function), see Section SM.
2.5 for more details.
Note that AWP may still underestimate the true Rashomon Capacity, yet it greatly improves the estimates compared to prior work and is less computationally intensive than sampling, as shown in the next section.
The procedure in (9) reveals a desirable property of a Rashomon subset: it should include models with significant score variations.
Similarly, a desirable Rashomon subset for accurately evaluating ambiguity/discrepancy is one with models that have most score disagreement.
This property also explains why collecting a Rashomon subset by straightforwardly sampling models [3] in the
Rashomon set could be inefficient (cf.
Algorithm SM.
1) since the randomly sampled models would not necessarily have a significant score disagreement/variation.
In fact, the sampling strategy could require a significant amount of models; for example in [3, Section 6], a Rashomon subset consisting of 250k decision trees.
Empirical study
We illustrate how to measure, report, and potentially resolve predictive multiplicity of probabilistic classifiers using Rashomon Capacity on UCI Adult [25], COMPAS [26], HSLS [27], and CIFARdatasets [8].
UCI Adult and COMPAS are two binary classification datasets on income and recidivism prediction, respectively, and are widely used in fairness research [28].
The HSLS is an education dataset, collected from high school students in the USA, whose features include student and parent information (see Section SM.
3.1 for details).
We created a binary label Y from students’ 9 th -grade math test scores (i.e., top 50% vs. bottom 50%).
We select the first three datasets to illustrate the effect of predictive multiplicity on individuals.
Finally, we include the CIFAR-10 dataset to demonstrate how to report Rashomon Capacity in multi-class classification.
For the classifiers, we adopt feed-forward neural networks for the first three datasets, and a convolutional neural network VGG16 [9] for CIFAR-10.
For more information on the datasets, neural network architectures, and training details, see Section SM. 3.2.
To the best of the authors’ knowledge, this is the first time that predictive multiplicity metrics are formally measured and reported in neural networks 7 .
All numbers reported are evaluated on the test set.
5.1 Measuring and reporting predictive multiplicity via Rashomon Capacity
We evaluate two methods, sampling with different weight initialization seeds [3] and AWP (9), and report the Rashomon Capacity in Fig.
For sampling , we construct a Rashomon subset (cid:101) R ( H , ϵ ′ ) with 100 models by different random initialization (cf.
(8) with K = 100 ).
For AWP , the ϵ ′ -multiplicity subsets (cid:102) M ϵ
( x i ) of each sample x i is constructed by the p 1 , · · · , p c obtained from (9).
For example, when ϵ ′ = 0 .
01 , the models in the Rashomon set (cid:101) R ( H , 0 . 01) achieves small and statistically indistinguishable test losses from each other; however, the Rashomon Capacity is non-zero for a significant fraction of samples.
In particular, we showcase the average top 1% and top
[Figure]
5% of the Rashomon Capacity from all samples for different ϵ ′ , and the cumulative distribution of the
Rashomon Capacity across the samples.
As the Rashomon parameter increases, both sampling and
AWP lead to higher Rashomon Capacity since the Rashomon set gets larger.
Observe that the increase of Rashomon Capacity when ϵ ′ varies from 0 .
01 to 0 .
02 is different for UCI Adult and COMPAS datasets, since the choice of ϵ ′ is data-dependent.
The AWP (9) achieves higher Rashomon Capacity than random sampling as AWP intentionally explores the Rashomon set that maximizes the scores variations.
It is important to keep in perspective that each sample in the high-Rashomon Capacity tail displayed in Fig.
3 corresponds to an individual who receives conflicting predictions.
In applications such as criminal justice and education, conflicting predictions for even one individual should be reported in, e.g., model cards [13].
For the experiments in Figure 3, the estimated Rashomon ratio are almost all zeros since the hypothesis space, parameterized by millions of parameters, is always significantly larger.
In SM, we further report (i) other strategies to explore the Rashomon set in Section SM.
4.2, (ii) Rashomon Capacity evaluated with decisions instead of scores in Section SM.
4.3, and (iii) other metrics of multiplicity such as ambiguity/discrepancy in Section SM.
5.2 Resolving predictive multiplicity by greedy model selection
Predictive multiplicity could be resolved by, for example, selecting a subset of models (potentially of size one in the ideal case) and releasing the scores to a stakeholder.
We propose a greedy model selection procedure to select a subset of competing classifiers for resolving predictive multiplicity.
Given R competing classifiers, the goal is to select r models ( r < R ) that result in distributions of the Rashomon Capacity similar to that of the original R models.
Starting from a dataset D and a
Rashomon subset (cid:101) R ( H , ϵ ′ ) , this can be implemented by (i) initializing a set A of models by randomly selecting a model in (cid:101) R ( H , ϵ ′ ) , (ii) growing A by adding one model from (cid:101) R ( H , ϵ ′ ) that maximizes the average Rashomon Capacity across D , and (iii) stopping until there are r models in A .
This greedy model selection is inspired by Property 4 (monotonicity) in Definition 1, since including the models to the set A does not reduce capacity.
In Fig.
4, we models from the Rashomon sets for
UCI Adult, COMPAS, HSLS and CIFAR-10 datasets respectively.
Here, the hypothesis space are feed-forward neural networks (see details in Section SM. 3.2).
Observe that only a small subset of the sampled models, selected by the greedy model selection procedure, is required to recover the distribution of the Rashomon Capacity.
On COMPAS dataset, the 10 models obtained by the greedy model selection procedure capture the Rashomon Capacity computed with the original 163 models, i.e., these 10 models display most of the score variations.
In Section SM.
4.1, we provide a worked-out example with the COMPAS dataset on how Rashomon
Capacity can be used to identify high-multiplicity samples.
We observe patterns in sex and prior [Figure] convictions leading to samples having high Rashomon Capacity.
We further discuss promising strategies, e.g., ensemble methods, model calibration and weight regularization to resolve multiplicity, in Sections SM.
4.5 and SM.
We observe that the ensemble method could lead to a smaller
Rashomon Capacity, and is a viable strategy for resolving multiplicity in small models, but may be infeasible for large, computationally expensive models.
In addition, weight regularization for logistic regression (e.g., LASSO or ridge penalties) could also reduce Rashomon Capacity.
On the other hand, a perfectly calibrated classifier does not necessarily resolve multiplicity—a classifier whose predicted classes matches the true classes “on average” across samples does not necessarily translate to a consistent set of predictions for a single target sample across equally calibrated classifiers.
Final remarks
Limitations.
The AWP (9), despite being more efficient than random sampling, still requires re-training/perturbing a significant amount of models, and is computationally burdensome when scaling up to large datasets with millions of samples.
Rashomon Capacity in certain cases may seem small for already significant score variations across classes, due to the convexity of KL-divergence.
Indeed, this issue will occur for any strictly convex measure of divergence in (4).
We provide a further discussion of how to interpret the numerical values of Rashomon Capacity in Section SM.
Future directions.
First, overcoming the computational bottleneck to efficiently explore the
Rashomon set is an impactful direction for optimization techniques.
Second, Rashomon Capacity could be generalized to other probability divergences, e.g., f -divergences [29], Rényi divergence [30], or Wasserstein distance [31].
This generalization could potentially provide further operational significance and tunability for measuring multiplicity, as long as the conditions in Definition 1 are satisfied.
Third, ensemble methods could be a promising strategy to reduce predictive multiplicity that worth studying.
Broader impacts.
The Rashomon effect impacts model selection [32–35], explainability [36], and fairness [37].
Rudin et al.
[32] suggested that, given the choice of competing models, machine learning practitioners should select interpretable models a priori , rather than selecting a black-box model with conjectural explanations post-training.
Hancox-Li [33] and D’Amour et al.
[34] argued that epistemic patterns, e.g., causality, should be reflected when selecting models the Rashomon set.
Black et al.
[35] further studied multiplicity in the context of the conventional bias-variance trade-off analysis.
Competing models in the Rashomon set may not only render conflicting explanations for predictions [36] and measures of feature importance [2], but also have inconsistent performance across population sub-groups.
Consequently, the arbitrary choice of a single model may result in unnecessary and discriminatory bias against vulnerable population groups [37].
See Section SM.
for further discussion, including connections with individual fairness [38].
Acknowledgement
This material is based upon work supported by the National Science Foundation under grants
CAREER 1845852, IIS 1926925, and FAI 2040880, and by Meta Ph.D. fellowship.