Data-IQ: Characterizing subgroups with heterogeneous outcomes in tabular data
Authors: Nabeel Seedat
Jonathan Crabbé
Ioana Bica
Mihaela van der Schaar
Venue: NeurIPS 2022 Accept (2022)
Keywords: data-centric AI
data characterization
data quality

Abstract: High model performance, on average, can hide that models may systematically underperform on subgroups of the data. We consider the tabular setting, which surfaces the unique issue of outcome heterogeneity - this is prevalent in areas such as healthcare, where patients with similar features can have different outcomes, thus making reliable predictions challenging. To tackle this, we propose Data-IQ, a framework to systematically stratify examples into subgroups with respect to their outcomes. We do this by analyzing the behavior of individual examples during training, based on their predictive confidence and, importantly, the aleatoric (data) uncertainty. Capturing the aleatoric uncertainty permits a principled characterization and then subsequent stratification of data examples into three distinct subgroups (Easy, Ambiguous, Hard). We experimentally demonstrate the benefits of Data-IQ on four real-world medical datasets. We show that Data-IQ's characterization of examples is most robust to variation across similarly performant (yet different models), compared to baselines. Since Data-IQ can be used with any ML model (including neural networks, gradient boosting etc.), this property ensures consistency of data characterization, while allowing flexible model selection. Taking this a step further, we demonstrate that the subgroups enable us to construct new approaches to both feature acquisition and dataset selection. Furthermore, we highlight how the subgroups can inform reliable model usage, noting the significant impact of the Ambiguous subgroup on model generalization.

Data-IQ: Characterizing subgroups with heterogeneous outcomes in tabular data
Nabeel Seedat University of Cambridge ns7410cam ac uk
Jonathan Crabbé University of Cambridge jc21330cam. ac.
Ioana Bica University of Oxford The Alan Turing Institute ioana. bicadeng. uk
Mihaela van der Schaar University of Cambridge The Alan Turing Institute
UCLA mv4720cam ac. uk
Abstract
High model performance, on average, can hide that models may systematically underperform on subgroups of the data. We consider the tabular setting, which surfaces the unique issue of outcome heterogeneity this is prevalent areas such as healthcare, where patients with similar features can have different outcomes, thus making reliable predictions challenging. To tackle this, we propose Data-IQ, framework to systematically stratify examples into subgroups with respect their outcomes. We do this by analyzing the behavior of individual examples during training, based on their predictive confidence and, importantly, the aleatoric (data) uncertainty. Capturing the aleatoric uncertainty permits a principled char- acterization and then subsequent stratification of data examples into three distinct subgroups (Easy, Ambiguous, Hard). We experimentally demonstrate the ben- efits of Data-IQ on four real-world medical datasets. We show that Data-IQ's characterization of examples is most robust to variation across similarly performant (yet different) models, compared to baselines. Since Data-IQ be used with any ML model (including neural networks, gradient boosting etc.), this property en- sures consistency of data characterization, while allowing flexible model selection. Taking this step further, we demonstrate that the subgroups enable us to construct new approaches both feature acquisition and dataset selection. Furthermore, we highlight how the subgroups can inform reliable model usage, noting the significant impact of the Ambiguous subgroup on model generalization.
Introduction
Most machine learning models optimized using empirical risk minimization (ERM), to maximize average performance during training [1]. However, in real-world settings, while models may perform well average, they might underperform specific subgroups of data [2-4]. Most of the current literature has focused on this problem computer vision, where the underperforming subgroups are typically associated with data examples that have spurious correlations [1. 5] mislabelling [6].
this paper, we focus on tabular data, the most ubiquitous format in medicine and finance, where data based on relational databases [7, 8). Specific to the tabular setting, we formalize understudied source of underperformance, namely heterogeneity of outcomes. This phenomenon is vital in healthcare, where patients with similar features can have different outcomes [9-11]. For example, [12] showed that prognostic models for risk prediction perform well on average, but underperform on
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

(i) Model parameters diff. training stages
(ii) Per-example training dynamics
(iIl) Characterize subgroups
Confidennce
Confdence
Riz
TRAINING
Model cheokpoinns
NARO
D train
Dtrain
aleatoric (data) uncertainty
01, 02.00..0.0 05
E
Epochs (NnoEsimmooors (OBT)
Pla, - +lz.59)
Characterize test data
DEPLOYMENT
Intent
AMBIGUOUS
ML Model
Unveliable
Pical
Project embedding of training data
Intent
REMAINDER
model deployment
Subgroup informed
Figure 1: Data-IQ IQ systematically characterizes data into subgroups using (i) any ML model. trained in stages (epochs/iterations) & can he cheekpointed. At training time, Data-IQ (ii) leverages the model's checkpoints analyze the training behavior of individual examples and (iii) characterizes each example based on its aleatoric (data) uncertainty& & prediction confidence. (iv) At deployment, the subgroups inform model usage by embedding the training set in a representation space and characterizing the new data points in this representation space.
specific cancer types due heterogeneity of risk (outcome). Prior works have audited subgroups belonging to sensitive attributes (e.g. demographics, race or gender), as it is well-known that ML models generally underperform on these subgroups [13, 14]. However, this approach is limiting, as needs the sensitive attributes to be specified, and it also does not capture the case where complex feature interactions may lead to underperformance.
We take a different approach to automatically stratify data into subgroups, usable with any MI. model trained in stages (epochs//terationns) e.g., neural networks, gradient boosting etc. Specifically, we study the behavior of individual examples during training, called training dynamics. This allows us to formalize that examples can lie on the spectrum from easy to hard predict. More coneretely, let's consider the task of patient mortality prediction. Based on their features, sicker patients more often have a mortality event. Thus, they are easy to learn for any model and will be predicted correctly with high confidence (Easy). However, subgroup might have a heterogeneous outcome: survival despite their poor prognosis. This heterogeneity could result from randomness. making practically impossible for a model to learn. These examples will be predicted incorrectly yet with high confidence (or equivalently have low confidence for the correct class) (Hard). In tabular data, there are also examples with inherent ambiguity where the predicted probability for the correct class remains low. They appear where the current features are insufficient to distinguish the example correctly, regardless of the model used [15, 16] (Ambiguous). These subgroups naturally arise in real data; see Fig. (ii).
Identifying these subgroups is practically valuable, as improving accuracy and robustness often depends on the data's characteristics and quality [17-20]. As mentioned in [20, 21], the "data" work often undervalued as merely operational. yet failing account for it can have immense practical harm [12, 20]. Consequently, our goal is to build systematic framework with the following desired properties (P1-P4), motivated by the considerations of practitioners at various stages of the ML pipeline. In satisfying P1-P4, we seek to address the "dire need for an ML-aware data quality that is not only principled, but also practical for larger collection (...) .) of ML. models" [19]:
(P1) Robust data characterization: the characterization of data examples should be robust, such that it is consistent across similar performing models, that have different (P2) Principled data collection: the characterization should be informative and actionable, providing practitioners insights that enable both quantitative feature collection and selection between datasets.
(P3) Reliable model deployment: the characterization should enable reliable model usage, both by unmasking unreliable subgroups using the subgroups to tailor the data for better performance. (P4) Plug & play: the characterization should be applicable to a variety of ML models widely used on tabular data, including neural networks, gradient boosting (and variants) etc.
To fulfill PI-P4, we propose Data-IQ. a systematic framework that characterizes examples based on the inherent qualities (IQ) of the data; at both training and deployment time. As outlined in 1i Data-IQ leverages confidence and the "data-centric A/" spirit focused on the data: aleatoric uncertainty (i.e. uncertainty inherent to the data). This permits Data-IQ to provide ML-aware data quality that principled and practical for a variety of ML. models, making the following contributions:

---

Contributions: ! Data-IQ models the aleatoric (data) uncertainty, which permits subgroup identifi- cation that is most robust to variation across different yet similar performing modeessparrmmetteizztions compared to other baselines, i.e. P/. ② Data-IQ aids with principled data collection P2 in two ways: Firstly, it permits to quantify the value of an acquired feature by measuring how the feature reduces the aleatoric uncertainty of the example. This information enables more principled approach to feature acquisition. Secondly, it permits to compare datasets based on the proportion of ambiguous examples. We demonstrate that the proportions link to how well model trained with the dataset generalizes. 3 Experimentally, the subgroups identified by Data-IQ can inform reliable model deployment, i.e. P3. We highlight cases, where assessment average might mask unreliable performance, including data sculpting, model robustness, and uncertainty estimation methods. 4 Data-IQ by construction is "plug-and paaeddpla"" i... P4 with any ML model that can be checkpointed, granting practitioners flexibility to apply Data-IQ to their model of choice.
2 Related work
This paper primarily engages with the literature on data characterization and contributes to the nascent area of data-centric AI [22, 23J. An extended discussion of related work is found in Appendix A.
Data characterization. The literature to characterize data samples has used a myriad of different metrics. However, their goals have typically been different, such as spurious correlation or misla- belling, compared to Data-IQ, whose goal is to characterize subgroups with respect to the outcome predictions. Furthermore, none of these methods completely addresses all the desired properties (PI-P4). The closest to our work on data quality Data Maps [24]. A key contrast Data-IQ is that Data Maps use confidence and prediction variability to flag instances. In Sec. 3, we show that this prediction variability corresponds to the model uncertainty (i.e. epistemic uncertainty). Alternatively, Data-IQ takes a different and more principled approach, capturing the inherent data uncertainty (known as aleatoric uncertainty) [25]. Epistemic uncertainty is reducible by collecting more data. In comparison, alcatoric uncertainty is irreducible even with more samples. This is due to the fact that it captures properties inherent to the data [25-27]: only better features can reduce the alcatoric uncertainty [27]. Later in Fig. 4, we show real data that capturing the alcatoric uncertainty allows Data-IQ to be more robust to variation across different models, compared to Data Maps (P/). This allows practitioners characterize their data in such way that the insights more consistent. We further show theoretically in Sec. 3.3, why the characterization by Data-IQ indeed provides a more principled definition for Ambiguous examples, compared to Data Maps.
Besides Data Maps, other related methods address specific computer vision problems: identifying mislabelled images using area under the margin (AUM) [6], gradient norm to identify "important examples" " to aid pruning during training [28]. or underperformance due to spurious image correlations [1]. The tabular setting considered in this paper requires new methods, due to the specific problem of heterogeneous outcomes for examples with similar features (i.e. "feature collision"". The ambiguity in the tabular, "feature collision" sense, is different or non-existent in modalities such images.
The assessment of data quality is critical but often overlooked problem in ML [20]. While the focus in ML is typically on optimizing models, the task of ensuring high quality data (or even improving one's data) can be equally valuable to improving performance [17, 20]. Even when it is considered, the process of assessing datasets is adhoc or artisinal [20, 22, 29, 30]. The recent growth of the data-centric AI space aims to build systematic tools for "data collection, labeling, and quality monitoring processes for datasets to be used in machine learning" [23, 30J. Data-IQ contributes to this nascent body of work, specifically around ML-aware data quality monitoring [19].
Data-Centric AI.
Formulation
This section gives a detailed formulation of Data-IQ and motivates our proposed example stratification that uses aleatoric uncertainty and confidence. We then describe how Data-IQ stratifies examples into subgroups at both training and testing time. Finally, we show Data-IQ's formulation permits usage with any ML model trained in stages, e.g. neural networks, GBDTs etc, unlike other approaches.
3

---

3.1 Preliminaries
We consider the typical supervised learning setting, where the aim is to assign an input a X C R to class y E ) C N. We have dataset DD with N E N' examples, i.e. D = {(x", n²) n E [N]} drawn IID from an unknown distribution. Our goal is then to learn a model fo : X + V, parameter- ized by 0 E O Typically, the parameters 0 are learned to minimize empirical risk, by minimizing the average training loss i.. () = +/4 -i= ( ys; O). with a loss function X x x x + Rt.
This brings us to the essence of the problem: "not all examples are created equally". c.g. patients with similar features might have heterogeneous outcomes, reflected in their labels y being different. These correspond to subgroups within Drain on which a predictive model might systematically underperform. We formalize this concept of hidden heterogeneous subgroups by assigning to each example x" a hidden subgroup label g" E EG, where G = [Easy, Ambiguous, H Hord). Before giving precise description of how those group labels are assigned, it useful to detail the context. Several works have established that the training dynamics of a model, contains signal about the quality of the data itself [31-33]. For instance, it takes more epochs/iterations for a model to assign the correct label to noisier/more difficult training examples. With Data-IQ, we build on those observations and assign a label g" to each example x" by studying its training dynamic, which is then used to estimate the aleatoric uncertainty and predictive confidence of each example. The following sections detail how this is done and how this contrasts with existing approaches.
3.2 Uncertainty decomposition during training
Recall that practitioners desire flexibility in the choice of the model. Hence, we focus on any ML model that is trained in stages and can be checkpointed during training, fo = y parameterized by 0 and on a given example from the training set (z. y) E Dursin- Assume that the model fo corresponds to a conditional categorical distribution, assigning a probability to each class given the input æ: fx( = P(Y X x, ú 8). During iterative training, the model parameters 0 vary. where over E E No epochs/iterations, these parameters take E different values at each checkpoint, i.e. 0,, 01.02... .2....,. gg. Since our analysis relies on the model's training dynamics, we want to take those different parameters into account. For the sake of notation, we introduce a random variable 9 that has an empirical distribution over this set of parameters captured through the training process & ~ Peemp Pomp (10e e [E]}). The variability of the model's 'ss parameters at training time is then reflected by the variance Vo [-].
The uncertainty we model is based on the random variable Y X = x that represents the possible labels given the imput xr. Since the ground-truth label y is available for training examples, we would like to distinguish between 2 cases: ! the predicted label corresponds to the ground-truth label Y = y and 2 the predicted label different from the ground-trutt label Y * y.. To this end, we introduce binary random variable Y that is set to one when the predicted label equals the ground-truth label (Ý if Y y) and that is zero otherwise (Y if * y). As discussed carlier, we are interested the uncertainty on the predictive random variable Y X x. This uncertainty is modeled by the variance u(x) Vrix ( x = =). We will now show that this quantity can be evaluated with the model predictions.
We start by noting that the definition of Y implies that Y X = x. 1 = 0 is a Bernoulli random variable with parameter' P(z, 0) P(Y = y X = x, 2 0) [fa( From this observation, we can decompose v(x) with the law of total variance and make each term explicit:
v(x) = Vo Ex,
Mix
xz 00
+ E. V
[ = xr I
Bernouli: P(x, 0)
Bernoulli Pix, 0)(1 P(x,, 0))
(1)
Vo [P(x, 0)
+ Es [P(x, 0) - P(x, 0))]
Epistemic uncertainty: (x)
Alcatoric uncertainty: (az)
In Eq. (1), we have split the overall uncertainty into two components: epistemic and aleatoric uncertainty. This type of decomposition is similar to those in the context of Bayesian neural
In this case, [f]()]] denotes the component y of the probability vector fo (x:).

---

networks [34, 35]. To understand the distinction between uncertainties, it is useful to closely examine the variances in the first equality of Eq. (1). For epistemic uncertainty Veps variance is evaluated on the model parumeters 9. Hence, epistemic uncertainty originates from the fact that a model's predictions oscillate when we change its parameters. For the aleatoric uncertainty Val, the variance evaluated on the predicted label Y X, 9. Hence, the variability originates from the inability to predict the correct label with high confidence. While existing works use epistemic uncertainty to stratify examples, we argue that aleatoric uncertainty is better principled choice to capture the inherent data uncertainty.
3.3 Stratification based on data uncertainty
We now explain how the above notion of uncertainty permits to assign group label g E g to each training example First, we use the empirical distribution @ ~ Pamp ( (0 E [E]}) to explicitly
write the two types of uncertainties in Eq. (1),where P(x) = 1/E Σe 0.x,
1 E
1 Σ -((()) E
Uep (x)
va(
F. P(x, 0.)(1 - P(x,
Σ
(2)
Stratification at training time. Before giving a precise definition of the group labels, let us give an intuitive definition for each group. ①I Easy: examples that have low data uncertainty that the model can correctly predict with high confidence, 2 Ambiguous: examples that have high data uncertainty, hence the model is unable to predict with confidence and ( Hard: examples that have low data uncertainty that the model is unable to predict (i.e. predicted incorrectly yet with high confidence or equivalently have low confidence for the correct class). We note that we need the model's 's prediction for the ground-truth class to delineate Easy and Hard examples, In practice, we use the model's s average confidence for the ground-trui class P(x) defined previously for this purpose. We make use of this concept to detail how labels are assigned to training examples (I, y) E Dimin:
Easy
if P(x)) > Cup A (x)) < P50 [vai( if P(x) S Clow A val(x) < Poo [Val (Drnin))
g(x,, D )
Hard
(3)
Ambiguous
otherwise
where Cup and Cjow are upper and lower confidence threshold resp. and Pr the n-th percentile. We provide a practical method to set Cup and Clow, applicable to any dataset in Appendix A.
In contrast to Data-IQ which uses Aleatoric uncer- tainty (z); Data Maps [24] identifies ambiguous training examples (ar, y) E Dirain as those with high epistemic uncertainty Cep (2). We consider a typical scenario to see how this characterization might cause problems - illustrated in Fig. 2. Consider an exam- ple a in which the model cannot classify confidently during the entire training P(x, 0.) = 0.5 Ve E [E].
Data
Data maps [23]
Tia Bag Eo PIx dl
Ba [P x xx *]]
Variabllty due Inhcrrent inablity to predict with curtairty (Rikked the data!
Variabiity
presictions model)
Low Val High Upp x
HighU/]
Low Dap X
In this case, the epistemic uncertainty Vep(2) van-
Low U41
ishes, as the prediction is consistently unconfident (i.e. low variability of the model predictions). This implies that Data Maps would consider this example as non-ambiguous, despite the ambiguous model pre- diction for this example. This problem can be traced back to the definition of epistemic uncertainty, which measures the sensitivity of a model prediction with respect to the model's parameters.
High Vep
10.0
15.0
17. 20.0
Eaoch
Figure 2: Example outlining the differences be- tween vaa (Data-IQ) and vep (Data Maps) & show- ing the type of uncertainty matters.
more principled definition for ambiguous examples should capture examples for which the model cannot predict the appropriate label with high confidence (i.e. data uncertainty). This is precisely what the aleatoric uncertainty (2)) captures (Data-IQ). Furthermore, it is easy to verify that the previous example Pzr, fe) = 0.5 Ve E [E] maximizes the aleatoric uncertainty (see Fig. 2). Since high aleatoric uncertainty captures ambiguous examples for various values of the model's parameters, we believe that it better reflects the inherent quality of the data. In that sense, we expect this quantity to be more stable and robust to variation for different MI. model parameters/architeture changes (PI). We experimentally validate the consistency in Sec. 4.
5

---

Stratification at inference time. Most previous methods are only applicable at training time. To address this limitation and improve the practical utility of our method, we also stratify examples into subgroups at deployment time. However, if we try to apply the above stratification for incoming data at deployment time, we face a problem: P(x)) requires the ground-truth class y.
For this reason, we follow an alternative approach based on representation learning that does not require access to ground-trutt labels. The idea the following: we construct low-dimensional UMAP embedding [36] h X + H of the training set's examples Terain E Drrin- In doing this, we note two things (see Appendix C): ! Ambiguous examples have distinctive features and are clustered in embedding space. Thus, it is possible to distinguish the Ambig examples using the embedding. 2 It is not possible to reliably distinguish Easy examples from Hard examples based on the embedding, because Hard examples are a minority with outcome randomness that have similar features, as the Easy examples. Combining these observations, we note it is possible to identify Ambiguous test examples. This label is assigned by computing the related embedding ((tres) and comparing this embedding to the nearest neighbor embedding from the training set, i.e. d[h(Zrest) ), h( ]] VIraain E D For models like neural networks with an implicit representation space, the same analysis can be done using the model's representation space.
3.4 Using Data-IQ with variety of models, beyond Neural Networks (P4)
The baseline methods discussed are primarily applicable only to neural networks. However, practically in tabular settings (e.g. healthcare/finacee etc). practitioners often use other highly performant iterative learning algorithms such as Gradient Boost Decision Trees (GBDTs) or variants [7, 37]. Data-IQ's formulation by construction is naturally adaptable any ML model trained in stages, that can be checkpointed. This satisfies P4, which allows practitioners the flexibility to use Data-IQ with their application-spectee model of choice. Appendix A provides guidelines, space and time considerations, as well as, discussing the specifics of how Data-IQ is easily adapted, for example to GBDTs.
4 Experiments
This section presents a detailed empirical evaluation demonstrating that Data-IQ 23 satisfies (P1) Robust data characterization, (P2) Principled data collection and (P3) Reliable Model Deployment, introduced in 1ec. Recall that (P4) Plug and play is satisfied by construction of Data-IQ.
Datasets. We conduct experiments on four real-world medical datasets, with diverse characteristics (different sizes, binary/multiclass, varying degrees of task difficulty etc) and highlight real-world applicability with heterogeneous patient outcomes: (1) Covid- 19 dataset of Brazilian patients [38], (2) Prostate cancer datasets from both the US [39] and UK [40], (3) Support dataset of seriously ill hospitalized adults [41], (4) Fetal state dataset cardiotocography [42]. We describe the datasets in greater detail in Appendix B, along with further experimental details. We observe similar performance across different datasets, but given the space limitations, we rypically show pertinent results for a single dataset, and include results for the other datasets in Appendix C.
4.1 (P1) Robust data characterization
Robustness to variation. As per Pl, we desire that Data-IQ identifies subgroups in manner robust variation across differ- ent models. This would allow a practitioner to obtain consistent insights about their data even when using different model ar- phitectures/prameeerizziioos When com- paring the different methods from Sec. 2, we note that each method has its own specific metric used to characterize examples (see Appendix B). To assess robustness to variation, we compare the consistency of the different characterization metrics, evaluated on models with different archieeuuee/parameerritiiooo All models are trained to convergence, with early stopping on validation set.
Comreattion
Dala-IO
Data Mops
AUN
Coutu
0.4
Support
Danasot
Figure 3: Robustness to variation across models based on Spearman correlation, where Data-IQ has the highest correla- tion (i.c. consistency) across all datasets.
2 http:///gthubbcoomseedddaabbbb/d

---

Quantitatively, we compute the Spearman rank correlation between all model combinations, see Fig. We observe that Data-IQ is the most consistent and robust to variation across different models, having the highest score on all datasets, satisfying P/. Further, the baseline methods themselves are also not consistent in performance ordering across datasets, which is undesirable. Ultimately, the robustness means practitioners can feel confident in the consistency of data insights, derived using Data-IQ.
0.55
4021314 Lrceniarry
(a) Data-IQ
To further compare Data-IQ and Data Maps[24], we examine 3 distinct models that achieve similar performance on the Covid-19 [38] tabular dataset, and we produce a characterization of the training set using each model in Fig. 4. We note that Data Maps groups can be recovered from (3) by replacing the aleatoric uncertainty Ua1 from Data-IQ with its epistemic counterpart Vep- The y-axis is the same for both methods and corresponds to P(r). The x-axis corresponds to (x) for Data-IQ and to Vep (x) for Data Maps. Each model assigned a color in Fig. We note three things ! Data-IQ's 's characterization of the data is significantly more stable across models. (2 Linked to the points in Sec.3, Data Map's 's high and low confidence examples in fact have high epistemic uncertainty Uep. which can lead to incorrect conclusions when attempting to use Data Maps to characterize data. 3 Data-IQ always distributes the data around a bell shape, which standardizes its interpretation. We provide theoretical analysis to explain this bell shape observation in Appendix A.
0.00 Ver stary (Friteei incersiityl
0.00
0.30
6.38 5223
(b) Data Maps [24]
Figure 4: Data-IQ's robust- ness to variation across diff. models (i.e.colors)
Neural Networks vs Other model classes. Data-IQ can be used with any ML model trained in stages linked to P4: Plug and Play. Methods such XGBoost, LightGBM and CatBoost methods are widely used by practitioners on tabular data, often more so than neural networks [7]. Ideally, based on Pl, we desire that the characterization of examples be consistent for similar performing models, irrespective of whether the model is neural network or an XGBoost model.
Data-IQ:
To assess the robustness of both Data-IQ and Data Maps, we train neural network, XG- Boost, LightGBM and Cat- Boost models to achieve the same performance and then per- form the characterization for all models. We can clearly see in Fig. 5 (Support) that Data-IQ has a similar characterization across all four models. Contrast- ingly, for Data Maps, the char- acterizations are significantly different for the different model classes. The implication of this result that by Data-IQ capturing the uncertainty inherent to the data (aleatoric uncertainty), it leads to more consistent and stable characterizations of the data itself. Especially, this highlights that Data-IQ characterizes the data in manner that is not sensitive to the choice of model when compared to Data Maps. For more, see Appendix C.
06
38
M - 6:73
CAl 1.75
M-t
01
CKT 3
630
376
511
B3C
2.30
635
09
620 am
613
615 133 021 3.3 E3
(a) Data-IQ
(b) Data Maps
Figure 5: NN XGBoost: Data-IQ is more consistent
Data insights from subgroups. Given the distinct differences between the subgroups, we seek to understand what factors make these subgroups different and how they can provide insight into the dataset. Such insights are especially useful in clinical settings. Results for the prostate cancer dataset are illustrated in Fig. (with other datasets in Appendix C). To visualize the different groups of patients within each subgroup, we cluster each subgroup (Easy, Ambiguous and Hard) using Gaussian Mixture Model (GMM) similar to [5], selecting the optimal clusters based on the Silhouette score. We assess cluster quality vs alternatives in Appendix C.
In general, across datasets. the subgroups are: (1) Easy: Severe patients with a death outcome, and less severe patients with a survival outcome. (2) Ambiguous: Patients with similar features, but different outcomes. This could suggest that the features, we have hand are insufficient to separate the differences in outcomes. (3) Hard: Severe patients with a survival outcome, and less severe patients with death outcome, i.e. opposite outcomes as expected due to randomness in the outcomes.
7

---

Less Severe flo PSA, stage Severe (high PSA, stage mortality
Patients similer fostures BUT different outcomes
Severe (high stage survive
Less Severe (low PSA, stage 2) mortality
155
EASY Figure 6: Comparing subgroups identified by Data-IQ (descriptions above). Colors represent the GMM clusters.
(b) AMBIGUOUS
(c) HARD
4.2 (P2) Principled data collection
Principled feature acquisition. As per Sec.4.1, the Ambiguous sub- group has examples with similar fea- tures, yet different outcomes. Recall that this case of ambiguity in the tab- ular setting is very different from am- biguity in other modalities, such as images. The ambiguity is due to in- sufficient features to adequately sep- arate the examples. We link this to the concept that the Ambiguous sub- group has high aleatoric uncertainty that is irreducible, even if we collect more data examples. Rather, aleatoric uncertainty can only be reduced by ac- quiring better features [27]. We lever- age this idea and show that Data-IQ's example characterization provides principled approach to assessing the benefit of acquiring specific feature. This is different from feature selection, where all features are present and we select the most "im- portant feature". Additionally, this is different from active learning which quantifies the value of acquiring examples, not features.
(a) Data-IQ subgroup ambiguity pro- portion reduced informative features are acquired.
(b) Data-IQ aleatoric uncertainty mains stable for Ambiguous, reducees for others as features are acquired.
(c) Data Maps subgroup proportions largely unaffected features acquired.
(d) Data Maps variability increases subgroups features acquired.
Figure 7: Quantifying the value of feature acquisition based on change in ambiguity. Only Data-IQ captures this.
With the above in mind, a valuable feature should decrease the example ambiguity (i.e. aleatoric uncertainty). Hence, decrease in the proportion of Ambiguous examples can serve as a proxy for the feature's potential value to the dataset. Understanding the value of features is useful in settings aequisition
such as healthcare, where feature acquisition comes a cost. To showcase the potential, we construct
semi- semi-synthetic experiment, where we rank sort the features based on correlation with the target.
We then train different models, where we sequentially "acquire" features of inereasing value (based on correlation). Fig. 7. shows results for the Support dataset. For Data-IQ, Fig 7 (a) shows that as we acquire "valuable" features, the proportion of the Ambiguous subgroup drops, whilst the Easy subgroup increases, with significant changes for the important features. This shows that Data-IQ's subgroup characterization can be used to quantify a feature's 's value, by its ability to decrease ambiguity. In contrast, Data Maps, Fig (c), shows minimal response to feature acquisition, suggesting it may not be sensitive enough to capture the feature's 's value.
Further, for Data-IQ we see that the examples that remain as Ambiguous, after features are collected, maintain a consistent alcatoric uncertainty. This is desired as it demonstrates for those examples which remain Ambiguous, that indeed the features collected are not informative enough to reduce their inherent (alcatoric) uncertainty, i.e. those remaining still need better features (see Fig (b)). While for Data Maps the added features, in fact increase the variability for all subgroups making it harder to stratify (see Fig (d)). This links to the fact that Data Maps subgroups can't capture the value of the acquired features. We further show experimentally in Appendix C that for Ambiguous examples, is not simply a case of increasing the size of the dataset (i.e. more examples). In fact, this can increase the proportion of Ambiguous examples due to the increased probability of feature collisions as the dataset size increases. Ultimately, this motivates the usefulness of principled feature acquisition (which Data-IQ can guide), as way to decrease dataset ambiguity.
8

---

Principled dataset comparison. Extending beyond feature-level, an understudied scenario involves systematically selecting between entire datasets two cases: (1) purchasing data from data markets [43-45] and (2) organizations where the data is siloed, with lengthy access processes [46, 47]. In both cases, synthetic versions of the real dataset has begun to be used [46]. For now, we ignore privacy concerns, and focus on data fidelity and quality, which compares the real and synthetic datasets using statistical measures. However, as per [48], the conclusions can vary across different metrics. In practice, competing "synthetic" datasets can be generated by different ML models or vendors. Thus, while they model the same underlying distribution, depending on the process used, one version might be superior. We now ask whether Data-IQ could permit us systematically select between synthetic datasets? We consider the setting where the real data is not accessible. Hence, we can't 't use existing evaluation metrics, yet still wish to compare the synthetic datasets (e.g. comparing vendors).
We simulate this scenario by generating synthetic data using 2 different models, representing 2 syn- thetic data vendors: (V1) CTGAN [49] and (V2) Gaussian Copula [50]. We then characterize the dataset subgroup proportions using Data-IQ. We hypothesize that datasets with greater proportions of Easy examples generalize better. As is common, we validate fidelity by training with synthetic data
Table I: Comparison of accuracy performance rank and (dataset quality). Synthetic dataset quality (t easy) the best real data test per- formance.
w/ better
Dataset
(V1) CTGAN
(V2) Gaussian Copula
Prostate
Rank (63% Easy)
Rank (30% Easy)
Covid
Rank (70% Easy)
(63% Easy)
and testing with real data [51, 52], where the best
Support
Rank (59% Easy)
Rank 2 (38% Easy)
Fetal
Rank 2 (40% Easy) Rank 1 (51% Easy)
fidelity data produces the best model performance on real data (test set). Table shows that datasets with the highest quality as measured by Data-IQ indeed produces the best performance real test data (i.e. Rank 1). Further, it shows that the same "vendor" does not always produce the best dataset, highlighting the value of comparative assessment. Ultimately, these two aspects demonstrate that when the real data is unavailable, Data-IQ is a useful tool in the hands of practitioners wishing to assess data quality, especially when selecting between different datasets.
4.3 (P3) Reliable Model Deployment
Tramt US: Test
Trant UIc Teat
Less is more: data sculpting based on subgroups. What role do the data subgroups, specifically Ambiguous examples, play in ensuring model generalization? Using multi-country prostate can- cer data, we train a baseline model on US data (SEER) and assess generalization when deployed on patients in the UK (CUTRACT) and vice versa. In Fig. 8, we see that test time generalization per- formance, monotonically increases as we decrease the proportion of Ambiguous training data (see Appendix C. for absolute num- bers). Ultimately, this illustrates the value of sculpting the training dataset, by removing ambiguous examples, way to improve the reliability of a deployed model.
Accueatr
0.60
training
Figure 8: Performance improve- ment via data sculpting based on subgroups
Group-DRO: Not a silver bullet when used in tabular settings. Once subgroups of underper- formance are identified, it is generally assumed that methods such as Group Distributionally Ro- bust Optimization (DRO) [53] can be applied to improve model performance and robustness. We compare Group-DRO with groups identified by Data-IQ, and as baselines: George [S] and Just-Train-Twice (JTT) [1]. As per the previ- ous experiments, the largest underperforming group (in proportion) is the Ambiguous exam- ples. Similar to the literature, we evaluate the performance change from baseline model, af- ter Group-DRe DRO is applied.
Table 2: Comparison of different model improve- ment/rohustness techniques
Dattsset
Group
Baseline
Goup-DRR (Data-1Q)
Grog-DRO (Gooogge)
Overall
0.837
0J540
0.565
0.723
Prostate
Amhigwos
0.740
0.741
0.598
0.538
The
0,935
0.935
0,535
0.896
Overall
0.73)
07332
0.687
0.435
Covid
Ambiguous
0.629
0.633
0.609
0.477
The
0.832
0.832
0.766
0.438
Overall
0.734
0.734
0.660
0.611
Suppert
Amhignous
(1621
0.622
0.576
0.521
The Rest
0,858
0.858
0,754
0.711
Overall
6X31
Feaal
Ambiguuus
0.575
0.701
0.695
0.698
The Kest
0.570
0.974
0,970
0,970
The results in Table 2 show that Group-DRO using Data-IQ's groups both improves overall perfor- mance and improves performance on the Ambiguous group. Whilst, for the other baselines, the performance actually degrades.
9

---

Nevertheless, while using Data-IQ can boost performance, it is evident that simply applying Group- DRO is not a silver bullet to equalize subgroup performance, given the sometimes small improvement. The rationale is our tabular setting is different, from the spurious correlation setting in computer vision where Group-DRO typically shines. Ultimately, we believe, hased on the feature acquisition results, that in tabular settings, practitioners would be well served acquiring better features to improve performance and reduce ambiguity.
Subgroup-informed usage of uncertainty estimation. Uncertainty estimation methods are essen- tial in safety-critical areas such as healthcare [54], yet are typically assessed on average. We ask the question: since subgroups have different performance properties, are uncertainty estimates equally reliable for each subgroup? As done in the literature [55, 56], if an uncertainty estimate is reliable and informative of predictive performance, it can be used to defer "uncertain examples". This is done by rank sorting examples based on uncertainty and thresholding proportions of tolerated uncertainty [55-58]. Ideally, as the threshold proportion of examples increases (i.e. inclusion of more uncertain examples), we should see a monotonic decrease of accuracy. We assess this by training a Bayesian Neural Network (BNN)[59] to obtain uncertainty estimates. We then compute the performance across different threshold proportions т E {0.1, as is commonly done, across the entire dataset (i.e. average).
} for the Ambiguuus subgroup specifically, and
Amiaguat
0.8
Overa Ulng!
Fig. 9 shows a specific example, wherein average examples exhibit the monotonic decreasing relationship, expected. However, the Ambiguous examples categorized by Data-IQ. contrary to expecta- tions, show an increase in accuracy more uncertain examples are included. This suggests that uncertainty estimates in this case are not as informative of predictive performance for the Ambiguous examples. This shows the potential for practitioners to use Data-IQ deployment time understand which examples require auditing before deferring, as the "average" monotonic decreasing behavior, based on the uncertainty estimates, may not always hold. Ultimately, the result further highlights how subgroup characterization via Data-IQ could assist practitioners in unmasking unreliable performance, not evident on average.
Noto: Dkevpottdde Mtnohhe
0.6
60
Figure 9: Data-IQ subgroups unmask unreliable prediction de- ferral by uncertainty methods
5 Discussion
In this paper, we introduce Data-IQ, a systematic framework that can be used with any MI, model with checkpoints, to characterize examples into subgroups with respect to the outcome. Through several experiments, we demonstrate that the usage of aleatoric uncertainty, which captures properties more inherent to the data, is indeed more principled, being more robust to variation across models and/or parameterizations. Data-IQ's consistency is unmatched by any compared baseline. Data-IQ should not automate and replace the intuition of data scientist, Rather, we have demonstrated, Data-IQ should serve as a systematic "data-centric ML" tool that assists and empowers data scientists with the "data" work at training time, whilst also guiding reliable model usage at deployment time.
Data-IQ beyond tabular settings. The main paper has primarily assessed the utility of Data-IQ in the tabular setting. That said, in Appendices C.7 and C.8, we evaluate the utility of Data-IQ on text data (NLP) and images (computer vision) respectively.
Limitations and future opportunities. ! While Data-IQ characterizes examples; the current for- mulation does not allow us understand which attributes are responsible for the characterization per example. This would be an interesting extension around dataset explainability, allowing practitioners to better probe their data. 2 In high-stakes settings such healthcare, to mitigate possible adverse effects (e.g. difficulty of Easy vs Hard), Data-IQ should be used with "human-in inmaan-in-ie in-the-loop",. allowing experts to complement and validate findings with domain knowledge.
Acknowledgments
The authors are grateful to Zhaozhi Qian, Yuchao Qin, Evgeny Saveliev and the anonymous NeurlPS reviewers for their useful comments & feedback. Nabeel Seedat is supported by the Cystic Fibrosis Trust, Jonathan Crabbe by Aviva, Ioana Bica by the Alan Turing Institute, EPSRC grant EP/N510129/1 and Mihacla van der Schaar by the Office of Naval Research (ONR), NSF 1722516.
10

---

References
[1] Evan Z Liu, Behzad Haghgoo, Annie Chen, Aditi Raghunathan. Pang Wei Koh, Shiori Sagawa, Perey Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. In International Conference on Machine Learning, pages 6781-6792.
PMLR, 2021.
[2] Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers generalize to imagenet? In International Conference on Machine Learning, pages 5389-5400. PMLR, 2019.
[3] John C Duchi, Tatsunori Hashimoto, and Hongseok Namkoong. Distributionally robust losses against mixture covariate shifts.
[4] Tatsunori Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. Fairness without demographics in repeated loss minimization. In International Conference on Machine Learning, pages 1929-1938. PMLR, 2018.
[5] Nimit Sohoni, Jared Dunnmon, Geoffrey Angus, Albert Gu, and Christopher Ré. No subclass left behind: Fine- aene-grained robustness in coarse coarse-grained classification problems. Advances in Neural Information Processing Systems, 0es11933--13555 2020.
[6] Geoff Pleiss, Tianyi Zhang, Ethan Elenberg, and Kilian Q Weinberger. Identifying mislabeled data using the area under the margin ranking. Advances in Neural Information Processing Systems, 33:17044-17056 2020.
[7] Vadim Borisov, Tobias Leemann, Kathrin SeBler, Johannes Haug, Martin Pawelczyk, and Gjergji Kasneci. Deep neural networks and tabular data: A survey. arXiv preprint arXiv:2110.01@6 2021.
[8] Jinsung Yoon. Yao Zhang, James Jordon, and Mihaela van der Schaar. Vime: Extending the success of self-and semi-supervised learning tabular domain. Advances in Neural Information Processing Systems, 33, 2020.
[9] Zepeng Huo, Lida Zhang, Rohan Khera, Shuai Huang, Xiaoning Qian, Zhangyang Wang, and Bobak J Mortazavi. Sparse gated mixture-of-experts to separate and interpret patient heterogeneity in ehr data. In 2021 IEEE EMBS International Conference on Biomedical and Health Informatics (BHI), pages 1-4. IEEE, 2021.
[10) Gerald Gartlehner, Suzanne I. West, Alyssa J Mansfield, Charles Poole, Elizabeth Tant, Linda J Lux, and Kathleen N Lohr. Clinical heterogeneity in systematic reviews and health technology assessments: synthesis of guidance documents and the literature. International Journal of Technology Assessment Health Care, 28(1)336-43, 2012.
[11] Jinsung Yoon, Camelia Davtyan, and Mihaela van der Schaar. Discovery and clinical decision support for personalized healthcare. IEEE journal of biomedical and health informatics, 21(4):1133 1145, 2016.
[12] Eun Jeong Oh, Ravi B Parikh, Corey Chivers, and Jinbo Chen. Two-stage approaches to accounting for patient heterogeneity in machine learning risk prediction models in oncology.
JCO Clinical Cancer Informatics, 5:1055-1023, 2021.
[13] Weijie Jiang and Zachary A Pardos. Towards equity and algorithmic fairness in student grade prediction. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pages 2021.
[14] Irene Y Chen, Emma Pierson, Sherri Rose, Shalmali Joshi, Kadija Ferryman, and Marzyeh Ghassemi. Ethical machine learning in healthcare. Annual Review of Biomedical Data Science, 4:123-144, 2021.
[15] Tin Kam Ho and Mitra Basu. Complexity measures of supervised classification problems. IEEE Iransactions on pattern analysis and machine intelligence, 24(3):289-300, 2002.
[16] Ana C. Lorena, Luis P. F. Garcia, Jens Lehmann, Marcilio C. P. Souto, and Tin Kam Ho. How complex is your classification problem? a survey on measuring classification complexity. ACM Comput. Surv. 52(5). sep 2019.
[17] Abhinav Jain, Hima Patel, Lokesh Nagalapatri, Nitin Gupta, Sameep Mehta, Shanmukha Guttula, Shashank Mujumdar, Shazia Afzal, Ruhi Sharma Mittal, and Vitobha Munigala. Overview and importance of data quality for machine learning tasks. In Proceedings of the 26th ACM
11

---

SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 3561-3562 2020.
[18] Nitin Gupta, Hima Patel, Shazia Afzal, Naveen Panwar, Ruhi Sharma Mittal, Shanmukha Guttula, Abhinav Jain, Lokesh Nagalapatti, Sameep Mchta, Sandeep Hans, et al. Data quality toolkit: Automatic assessment of data quality and remediation for machine learning datasets. arXiv preprint arXiv: 208.0::90080033333 2021.
[19] Cedric Renggli, Luka Rimanic, Nezihe Merve Gürel, Bojan Karlas, Wentao Wu, and Ce Zhang. data quality-driven view of mlops. IEEE Data Engineering Bulletin, 2021.
[20] Nithya Sambasivan, Shivani Kapania, Hannah Highfill, Diana Akrong, Praveen Paritosh, and Lora M Aroyo. "everyone wants to do the model work, not the data work": Data cascades in high-stakes ai. In proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pages 1-15, 2021.
[21] Kiri Wagstaff. Machine learning that matters. arXiv preprint arXiv: 1206.4666, 2012.
[22] Andrew Ng, Lora Aroyo, Cody Coleman, Greg Diamos, Vijay Janapa Reddi, Joaquin Van- schoren, Carole-Jean Wu, and Sharon Zhou. Neurips data-centric ai workshop. 2021.
[23] Neoklis Polyzotis and Matei Zaharia. What can data-centric ai learn from data and ml engineer- ing? arXiv preprint arXiv22112.06433 2021.
[24] Swabha Swayamdipta. Roy Schwartz, Nicholas Lourie, Yizhong Wang. Hannaneh Hajishirzi, Noah A Smith, and Yejin Choi. Dataset cartography: Mapping and diagnosing datasets with training dynamics. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9275-9293 2020.
[25] Alex Kendall and Yarin Gal. What uncertainties do we need in bayesian deep learning for computer vision? Advances in neural information processing systems, 30, 2017.
[26] Armen Der Kiureghian and Ove Ditlevsen. Aleatory or epistemic? does matter? Structural safety, (152::00-111,, 2009.
[27] Efstratios Gavves. Bayesian deep learning, lecture 10, university of amsterdam, November 2019.
[28] Mansheej Paul, Surya Ganguli, and Gintare Karolina Dziugaite. Deep learning on a data diet: Finding important examples early in training. Advances in Neural Information Processing Systems, 34, 2021.
[29] Sezal Chug, Priya Kaushal, Ponnurangam Kumaraguru, and Tavpritesh Sethi. Statistical learning to operationalize a domain agnostic data quality scoring. arXiv preprint arXiv:2108.088 2021.
[30] Nabeel Seedat, Jonathan Crabbe, and Mihaela van der Schaar. Data-suite: Data-centric iden- tification of in-distribution incongruous examples. In International Conference on Machine Learning. PMLR, 2022.
[31] Devansh Arpit, Stanistaw Jastrzębski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S Kanwal, Tegan Maharaj. Asja Fischer, Aaron Courville, Yoshua Bengio, et al. A closer look at memorization in deep networks. In International conference on machine learning, pages 233-242. PMLR, 2017.
[32] Sanjeev Arora, Simon Du, Wei Hu, Zhiyuan Li, and Ruosong Wang. Fine grained analysis of op- timization and generalization for overparameterized two-layer neural networks. In International Conference on Machine Learning, pages 322-332. PMLR, 2019.
[33] Yuanzhi Li, Tengyu Ma, and Hongyang R Zhang. Learning over-parame two-layer neural networks beyond ntk. In Conference on learning theory, pages 2613-2682. PMLR, 2020.
[34] Yongchan Kwon, Joong-Ho Ho Won, Beom Joon Kim, and Myunghee Cho Paik. Uncertainty quantification using bayesian neural networks in classification: Application to biomedical image segmentation. Computational Statistics & Data Analysis, 142: 1068 16, 2020.
[35] Nabeel Seedat and Christopher Kanan. Towards calibrated and scalable uncertainty representa- tions for neural networks. arXiv preprint arXiv: 1tt:::9911.000,, 2019.
[36] Leland MeInnes, John Healy, Nathaniel Saul, and Lukas Großherger. Umap: Uniform manifold approximation and projection. Journal of Open Source Software, 3(29):861, 2018.
12

---

[37] Ravid Shwartz-Ziv and Amitai Armon. Tabular data: Deep learning is not all you need. Information Fusion, 81:84-90, 2022.
[38] Pedro Baqui, Ioana Bica, Valerio Marra, Ari Ercole, and Mihaela van Der Schaar. Ethnic and regional variations in hospital mortality from covid-19 in brazil: a cross-sectional observational study. The Lancer Global Health, 8(8)xe1018 1026, 2020.
[39] Máire A Duggan, William F Anderson, Sean Altekruse, Lynne Penberthy, and Mark E Sher- man. The surveillance, epidemiology and end results (seer) program and pathology: towards strengthening the critical relationship. The American journal of surgical pathology. 40(12):e94,
2016.
[40] CUTRACT Prostate Cancer UK. Prostate cancer uk.
[41]
William A Knaus, Frank E Harrell, Joanne Lynn, Lee Goldman, Russell S Phillips, Alfred F Connors, Neal V Dawson, William Fulkerson, Robert M Califf, Norman Desbiens, et al. The support prognostic model: Objective estimates of survival for seriously ill hospitalized adults. Annals of internal medicine, 122(3):191-203, 1995.
[42] Diogo Ayres-de Campos. Joao Bernardes, Antonio Garrido, Joaquim Marques-de Sa, and Luis Pereira-Leite. Sisporto 2.0: program for automated analysis of cardiotocograms. Journal of Maternal-Fetal Medicine, 95) 1-3318, 2000.
[43] Pantelis Koutroumpis, Aija Leiponen, and Llewellyn DW Thomas. Markets for data. Industrial and Corporate Change, 2963)6645-660, 2020.
[44] Ramesh Raskar, Praneeth Vepakomma, Tristan Swedish, and Aalekh Sharan. Data markets to support ai for all: Pricing, valuation and governance. arXiv preprint 1905.09900040 2019.
[45] Mohammad Rasouli and Michael I Jordan. Data sharing markets. arXiv preprint arXiv:2107.088: 00,60000,, 2021.
[46] Khaled El Emam, Lucy Mosquera, and Richard Hoptroff. Practical synthetic data generation: balancing privacy and the broad availability of data. O'Reilly Media, 2020.
[47] Andre Goncalves, Priyadip Ray, Braden Soper, Jennifer Stevens, Linda Coyle, and Ana Paula Sales. Generation and evaluation of synthetic patient data. BMC medical research methodology. 20(1):1-40, 2020.
[48] Ahmed Alaa, Boris van Breugel, Evgeny Saveliev, and Mihaela van der Schaar. How faithful is your synthetic data? sample-level metrics for evaluating and auditing generative models, 2022.
[49] Lei Xu, Maria Skoularidou, Alfredo Cuesta-Infante, and Kalyan Veeramachaneni. Modeling tabular data using conditional gan. Advances in Neural Information Processing Systems, 32, 2019.
[50] Zheng Li, Yue Zhao, and Jialin Fu. Syne: A copula based framework for generating synthetic data from aggregated sources. In 2020 International Conference on Data Mining Workshops (ICDMW), pages 571-578. IEEE, 2020.
[51] Cristóbal Esteban, Stephanie L Hyland, and Gunnar Rätsch. Real-valued (medical) time series generation with recurrent conditional gans. arXiv preprint arXiv:1706. 02666,0263333 2017.
[52) Michael Platzer and Thomas Reutterer. Holdout-based empirical assessment of mixed-type synthetic data. Frontiers in big Data page 43, 2021.
[53] Shiori Sagawa, Pang Wei Koh, Tatsunori Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint
2019.
[54] Nabeel Seedat. Mcu-net: A framework towards uncertainty representations for decision support system patient referrals healthcare contexts. arXiv preprint arXiv:2007.0399 2020.
[55] Christian Leibig, Vaneeda Allken, Murat Seçkin Ayhan, Philipp Berens, and Siegfried Wahl. Leveraging uncertainty information from deep neural networks for disease detection. Scientific reports, 7(1):1114, 2017.
[56] Tong Xia, Jing Han, Lorena Qendro, Ting Dang, and Cecilia Mascolo. Uncertainty-aware 1ovid-19 detection from imbalanced sound data. arXiv preprint arXiv:2104.02005, 2021.
[57] Ranganath Krishnan and Omesh Tickoo. Improving model calibration with accuracy versus uncertainty optimization. Advances in Neural Information Processing Systems, 33:18237-18248, 2020.
13

---

[58] Jishnu Mukhoti and Yarin Gal. Evaluating bayesian deep learning methods for semantic segmentation. arXiv preprint arXiv:1811.12709 2018.
[59] Soumya Ghosh, Jiayu Yao, and Finale Doshi-Velez. Structured variational learning of bayesian neural networks with horseshoe priors. In International Conference on Machine Learning, pages 1744-1753. PMLR, 2018.
[60] Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In International conference on machine learning, puges 1885-1894. PMLR, 2017.
[61] Amirata Ghorbani and James Zou. Data shapley: Equitable valuation of data for machine learning. In International Conference on Machine Learning, pages 2242-2251. PMLR, 2019,
[62] Gaojie Jin, Xinping Yi, Liang Zhang, Lijun Zhang, Sven Schewe, and Xiaowei Huang. How does weight correlation affect generalisation ability of deep neural networks? Advances in Neural Information Processing Systems, 33:21346-21356, 2020.
14

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes] Please refer to our "contributions" paragraph in Section 1. Furthermore, all claims made about Data-IQ are verified in Section 4 and in the supplementary material.
(b) Did you describe the limitations of your work? [Yes] See Section 3.3 and Section 5.
(c) Did you discuss any potential negative societal impacts of your work? [Yes] Data-IQ provides practioners with a tool to help them with their "data" work and to better understand the characteristics of their data. In fact, Data-IQ may help reveal the negative impacts of ML models. However, as discussed Section 5, expert knowledge should still be used to validate our characterization.
(d) Have you read the ethics review guidelines and ensured that your paper conforms them? [Yes] We have carefully read the ethics review guideline and confirm that our paper respects the guidelines.
2. If you including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] See Section 3 and Appendix of the supplementary material
(b) Did you include complete proofs of all theoretical results? [Yes] See Section 3 and Appendix A of the supplementary material
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or URL)? [Yes] See footnotes and 3
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Appendix B, detailing all relevant information
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes] Included as relevant to the experiments in Section 4 and Appendix C
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Appendix
4. If you using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] See citations [38], [39]. [40], [41].[42]. We cite these in the main paper and supplementary material
(b) Did you mention the license of the assets? [Yes] See Appendix B of the supplementary
material.
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
We only use existing assets
(d) Did you discuss whether and how consent obtained from people whose data you's 're using/curating? [Yes] It is discussed in detail in the publications by the dataset creators, which we have cited
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [Yes] See Appendix B of the supplementary material.
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
15


Reviews:
[{"review_id": "rwvdYkP7cF", "reviewer": "Reviewer_WioB", "summary": "The paper proposes a framework to identify examples in the datasets that a classifier tends to distinguish correctly, wrongly, and randomly.  The paper decomposes the accuracy of a given example into a summation of epistemic uncertainty and aleatoric uncertainty. The paper then uses aleatoric uncertainty to classify examples into the three aforementioned categories, as opposed to existing work that uses epistemic uncertainty as the criterion.", "questions": "* In equation (2), is the estimate based on the empirical process a good estimate for the two uncertainty quantities given that the samples in the empirical process might be correlated with each other rather than iid?", "limitations": "The paper discusses the limitation of the proposed method. Finding the attributes that are responsible for the hardness of the data points seems to be particularly interesting and relevant for the interpretability of machine learning algorithms.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n* the paper deals with a significant problem towards a better understanding of how difficult a data point is to classifiers in terms of classify it accurately.\n* The proposed approach that decomposes the accuracy of a given example into a summation of epistemic uncertainty and aleatoric uncertainty is clean and informative of the difficulty of classifying a given example.\n* The presentation of the paper is clear. Experiments are illustrative.\n\nWeakness:\n* Given the generality of the proposed method, the paper may consider evaluating the proposed method beyond tabular data.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "omLYyd9Oloz", "reviewer": "Reviewer_6nr2", "summary": "This paper proposed Data-IQ, a framework compatible with any machine learning (classification) models whose training is conducted in stages (iterations/epochs), to assess the quality of data samples in tabular format by categorizing them into Easy, Ambiguous, Hard groups based on the aleatoric uncertainty of the data samples. The paper demonstrated the utility of Data-IQ in guiding feature acquisition, comparing training datasets and improving model generalization through experiments and analysis on multiple real-world datasets.", "questions": "1. The aleatoric uncertainty is quantified as the average variance over all training epochs. I am wondering how the distribution of the variance of a single data sample over the training epochs looks like? If the distribution of one training example is skewed, e.g. the variance is high at the beginning but it becomes much lower as the model parameters are refined during training, and the variance of another training example remains at a medium level during training, these two training examples might end up with similar average uncertainty but the training dynamics are very different. In this case, would the average variance still be a good metric to evaluate the uncertainty of an individual data sample?\n\n\n2. The evaluation are mainly conducted on neural networks, which demonstrates that Data-IQ is robust to different parametrizations of the same model type. Would it be possible to compare the results across different model types? Without comparison across model types, e.g. neural networks v.s. gradient boosting, the claim that Data-IQ is robust to model variation is less convincing as the aleatoric uncertainty might still be pertinent to the model class (though not to different parameters under the same model type).", "limitations": "The authors have addressed the limitations and I think another potential limitation is how robust Data-IQ will be across different model types as stated in Q2 in **Questions**.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Originality:** the idea of using prediction outcomes to quantify the uncertainty of data samples is not new, e.g. in some active learning approaches the next queried data point would be the sample with predicted probability close to 0.5 as it is the most uncertain. The novelty of Data-IQ lies in that it quantifies the aleatoric uncertainty of data samples by evaluating the variance of the prediction outcome which intends to capture the inherent uncertainty of the data samples rather than the uncertainty with regard to the model parameters. Therefore, Data-IQ is capable of providing guidance to improve data quality in new directions with regard to the dataset itself, e.g. new feature acquisition.\n\n**Quality:** the evaluation of the variance of the prediction outcome is straightforward, by noting that the classification outcome is a Bernoulli random variable, the variance is $p(x, \\theta_e)(1 - p(x, \\theta_e))$ in which $p(x, \\theta_e)$ is the predicted probability using the model parameters $\\theta_e$ at training epoch $e$. Then the aleatoric uncertainty is quantified as the average variance over all training epochs and categorization of samples is based on thresholds chosen by heuristics. The analysis of the categorization on real-world datasets is quite comprehensive, e.g. how to use Data-IQ to assess the quality of a new feature and how the Ambiguous samples will impact model generalization etc., but with regard to methodological perspectives, the technical novelty / contribution is not significant.\n\n**Clarity:** the paper is well written and easy to follow. All the figures and tables are very clear.\n\n**Significance:**  I think the research topic in this paper is of high practical value as the assessment of data quality (uncertainty) is very important for training high-quality machine learning models and obtaining reliable predictions on unseen data. However, as stated in \n**Quality**, the methodological / theoretical novelty of the method is not very high.  ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "A7QuW0heDWt", "reviewer": "Reviewer_2wVN", "summary": "The authors proposed a framework of data characterisation and data sculpting during model training based on the outcome and feature characteristics of the subgroups within a dataset. The authors separate the subgroups into the Easy, Ambiguous and Hard groups. Via a set a of experiments on different datasets, they showed that the the framework was able to identify the uncertainties due to the data characteristics, insights regarding the subgroup feature characteristics, and can potentially improve model generalisability by excluding Ambiguous group from training.", "questions": "1. To my understanding, the \"Hard\" group are sample with similar feature as the \"Easy\" group, but opposite outcomes. How is this different from the \"Ambiguous\" group, which includes samples that have similar features but different outcomes? \n2. Regarding the data insights, are the findings clinical meaning full? Are their any evidences (related biomedical researches), can be referenced?\n3. The improved in Table 2 is marginal. Also the comparison involves two factors: Data-IQ and modelling method Group-DOR. Could the authors show a baseline model + Data-IQ?", "limitations": "The authors discussed the limitations of the method in the section 5. However, the discussion was limited, maybe due to the length of the paper. I personally would be interested to understand how the framework may influence the inclusivity of the dataset used for model training.", "rating": 3, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The authors provides a different perspective, which puts the characteristics of data in the centre, to analyse the factors that influences the performance of a model. They provide a tool, Data-IQ, to help the researcher understand data better before diving into modelling process. To me, it is in innovative and should be encouraged. However, what the authors propose might not be sufficiently mature and can be used as framework for assessing datasets and guide model training process by the community. Regarding the writing, the paper was trying to put too much ideas into the limited length, including variation vs robustness, subgroup characterisation, model generalisability, etc.. It would be much clearer if there could focus on one single point, e.g. data sculpting based on the identified subgroups.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "9OQvSU_McKV", "reviewer": "Reviewer_3GB7", "summary": "The authors present an approach that given a model can identify 3 general sub-groups in the dataset that differentiates the model performance across the dataset. In particular they analyzed tabular data to find such sub-groups that are useful for meaningful data-driven AI and satisfies requirements from an MLOps perspective. They conducted experiments and demonstrated various aspects of the proposed method. Overall, their proposed method aims at informing reliable model usage. ", "questions": "Some questions for the authors:\n- while the authors rely on the training dataset performance, its not evident that their results and/or formulation is dependent on it. For example, the results in the supplementary section shows how the $v_{al}$ stabilizes after a few epochs. One can also draw corollary from the steady state dynamics for state space model and so on. Equation 1 also approximates the aleatoric uncertainty as an expectation. One can argue that for converged models, the steady state value may be enough. In contrast would a burn-in period have an impact on this estimate?\n- It seems that the entire formulation is true while comparing models of similar performance. How does this formulation play into Eqns (1) and (2).\n- Section 4.3: How many ambiguous datapoints were in each dataset? The absolute number is important while analyzing the importance of the effect of removing such datapoints on model performance", "limitations": "N/A", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The are several interesting aspects of this paper\n\n- the proposed method of identifying the sub-groups using the training performance of the model using the data uncertainty aspect is while simple, is intuitive and more importantly the presented analyses supports the importance of this formulation. \n- The identified 4 paradigms for a useful sub-group discovery is interesting, especially from a MLops perspective. The paper is masterfully presented that describes and motivates each of these paradigm and analyzes how the proposed method addresses each of these requirements. Especially, the  plug and play aspect of this method is very appealing. The results presented in the appendix comparing the classical GBT models is also interesting (perhaps could be included in the main paper). \n- The insights about the importance of the `Ambiguous` groups is perhaps the most interesting aspect of the paper. Pertinently the novel usage of this identified sub-group for aspects such as identifying the most important features to comparing the `value` of a dataset is interesting. \n\nWhile the paper itself is quite interesting, there are a few aspects to improve upon:\n- the Figures are very low resolution and hinders readability. While acknowledging the space limitations, some of the sections/paragraph could be moved to supplementary section. For example, Figure 4 and the corresponding paragraph comparing agains DataMaps is quite intuitive and may not need to be part of the main paper. The y-axis is closely related to the metric for $v_{al}$ and the shape is quite expected. Similarly, Fig 8 can be argued to be intuitive and a direct consequence of the selected measure.\n- Table 1 and the corresponding paragraph may need further context. The discussion ignores the context around the fidelity of the synthetic data to actual ground truth. E.g. guarding against trivially Easy data\n- Finally, while the paper is quite well written, the actual method to calculate the Easy, Ambiguous, and Hard groups is not presented properly - it should be present at least as an algorithm in the supplementary section. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "3cUX0WGCHa", "reviewer": "Reviewer_YLpH", "summary": "The authors make a strong case that population level metrics for model performance may not be representative of model performance on subgroups. The authors proposed `DataIQ` a framework to phenotype and subgroup patients based on model uncertainity on an individual level. The authors apply their proposed technique on multiple real world healthcare datasets and demonstrate ability of the model to 1) identify robust data 2) Data collection 3) Model deployment. ", "questions": "- Have the authors considered other potential sources of incorporating randomness for the aleatoric uncertainty they are trying to capture over the distribution of $v$, for example perhaps the use of a bayesian prior on $v$ or dropout masks? \n", "limitations": "As indicated above the distribution of $v$ is intimately tied to the model choice and the optimization approach. Even when the model class is the same, the hyper parameter choices for the optimizer like step size, momentum, might lead to vastly different distributions in the weights. \n\nSince the authors propose to use the estimated groups as 'protected groups', it would be welcome to see more extensive discussion of how these choices effect recovery of the estimated groups. ", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Overall I like this paper, even though there are a few weaknesses. \n\n# Strengths\n\n- The paper describes provides several reasons to model model uncertainty and conducts several ablation experiments to justify each use case. \n- The multiple ablation studies, including the change in estimated uncertainty with addition of features (Figure 6)  as well as domain shift (Figure 7) are welcome. \n- The comparison of output of the model along a with Group DRO objective to improve subgroup level performance adds to the completeness of an already exhaustive set of experiments.\n\n# Weaknesses\n\n- The way the paper proposes to estimate the variance around the model outcomes is by assuming the model parameters at the end of each epoch to be sampled from a distribution. While interesting such a choice is not characteristic of an empirical distribution in the strictest sense. In this case the model parameters are going to be highly correlated which would break the assumption of sampling from an IID distribution.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept