Context-Aware Testing: A New Paradigm for Model Testing with Large Language Models
Authors: Paulius Rauba
Nabeel Seedat
Max Ruiz Luyten
Mihaela van der Schaar
Venue: NeurIPS 2024 poster (2024)
Keywords: model testing
tabular data
large language models
Primary area: evaluation
TL;DR: We introduce context-aware testing (CAT) which uses context to guide the search for meaningful model failures

Abstract: The predominant *de facto* paradigm of testing ML models relies on either using only held-out data to compute aggregate evaluation metrics or by assessing the performance on different subgroups.  However, such *data-only testing* methods operate under the restrictive assumption that the available empirical data is the sole input for testing ML models, disregarding valuable contextual information that could guide model testing. In this paper, we challenge the go-to approach of *data-only testing* and introduce *Context-Aware Testing* (CAT) which uses context as an inductive bias to guide the search for meaningful model failures. We instantiate the first CAT system, *SMART Testing*, which employs large language models to hypothesize relevant and likely failures, which are evaluated on data using a *self-falsification mechanism*. Through empirical evaluations in diverse settings, we show that SMART automatically identifies more relevant and impactful failures than alternatives, demonstrating the potential of CAT as a testing paradigm.

Context-Aware Testing: A New Paradigm for Model Testing with Large Language Models
Paulius Rauba" University of Cambridge pr5010cam ac uk
Nabeel Seedat" University of Cambridge ns7410cam. ac uk
Max Ruiz Luyten University of Cambridge mr 1 16cam. ac. uk
Mihaela van der Schaar University of Cambridge
mv4726cam ac uk
Abstract
The predominant de facto paradigm of testing ML models relies either using only held-out data to compute aggregate evaluation metrics or by assessing the performance different subgroups. However, such data-only lesting methods operate under the restrictive assumption that the available empirical data is the sole input for testing ML models, disregarding valuable contextual information that could guide model testing. In this paper, we challenge the go-to approach of data-only testing and introduce context-aware testing (CAT) which uses context as inductive bias to guide the search for meaningful model failures. We instantiate the first CAT system, SMART Testing, which employs large language models hypothesize relevant and likely failures, which are evaluated on data using self. falsification mechanism. Through empirical evaluations diverse settings, we show that SMART automatically identifies more relevant and impactful failures than alternatives, demonstrating the potential of CAT as testing paradigm.
Introduction
The ability rigorously test and validate machine learning (ML) models is erucial for their reliable deployment in real-world applications. Despite solid aggregate performance, ML models are un- reliable in variety of real-world scenarios, such
different subgroups [1-6], or encountering data deviating from its training distribution [7-10], often resulting in significant financial or societal consequences. These failures point deficiencies in the way test such MI. models. To understand this in greater detail, let's address two questions: how are we currently testing ML models and can we do better?
How are we testing? The predominant de facto paradigm of testing ML models relies on using only held-out data to evaluate the model either average by assessing performance different subgroups within the dataset. Such data-only methods optimize a given objective function to find subgroups (or slices of data) within the dataset where trained model underperforms relative to aggregate performance. We refer to this relative underperformance as a model failure. However, data-only methods operate under the restrictive assumption that the available empirical data is the sole input for testing MI. model performance. In practice, this almost always violated. It is common to have a priori knowledge of where models are likely to fail, given the problem context.
The restrictive assumption of data-only methods comes at a significant cost. Specifically, data-only methods, which have been the go-to approach for ML testing, required to iterate and test the
Equal Contribution
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

performance across a large number of possible subgroups. Each subgroup test is equivalent to evaluating separate hypothesis about the model's performance on that specific slice of data. This raises subtle but important problem-fromd the perspective of hypothesis testing, such methods implicitly test multiple hypotheses (Sec. 3.1). The more subgroups we test, the more hypotheses we implicitly evaluate. Consequently, this results in critical problems associated with multiple testing: (i) high false positive and (ii) falh negative rates (Sec. 3.2). A third complementary challenge is that each data slice tested is (iii) not necessarily practically meaningful. The practical implications of these drawbacks are quite concerning- hinder our ability to accurately identify where the model performs poorly, thereby undermining the reliability of our model evaluations.
Can we do better? To address these three challenges, we propose to loosen the restrictive assumption of reliance only on data and propose new paradigm of testing called Context-Aware Testing (CAT) to offer an alternative to the dominant data-only view (Sec. 3.3). CAT provides a principled approach to incorporate external knowledge- or context- to the ML testing process. With the view that each evaluated data slice corresponds to an implicit hypothesis test, we propose that MI. evaluation on observational datasets can be achieved via context-guided sampling mechanism (Definition 1). This mechanism is a sampling procedure that uses context as an inductive bias to prioritize specific data slices to test for which have a higher chance of surfacing meaningful model failures. Therefore, CAT fundamentally helps to answer the question of "what should we test for?".
Let's consider an example of building an ML model to predict prostate cancer [11]. Data-oni -nlly methods employ a search procedure over the dataset to find divergence across large number of possible feature combinations which may lead to (i) high false positive rates by identifying spurious underperforming subgroups (e.g. based on eye color or patient ID); (ii) high false negative rates by failing to identify true underperforming subgroups due to the large number of combinations tested and applied testing corrections; and (iii) testing subgroups which are not practically meaningful (e.g. interaction between eye color and height). In contrast, CAT-based approach would define and target task-relevant subgroups, limiting the number of tests conducted with better false positive control and greater statistical power. As we empirically show in Sec. 5, obtaining many false positives and false negatives is overwhelmingly common in current testing practices.
In bringing CAT to reality, we develop the framework called SMART' 2 Testing, which performs automated MI. model evaluation by actively identifying potential failure cases (Sec. 4). SMART uses large language models (LL.Ms) to generate contextually relevant failure hypotheses to test. We further introduce falssfictsii mechanism, to automatically validate the generated failure hypotheses using the available data, allowing efficient pruning of spurious hypotheses. Finally, SMART generates comprehensive model reports that provide insights into the identified failure modes, their impact, and potential root causes, enabling stakeholders to make informed decisions.
Contributions. ! We identify critical gaps in predominant data-only ML testing, illustrating they miss important dimensions (Sec. 3). 2 We formalize the Context-Aware Testing paradigm, providing a principled framework incorporate context in addition to data into the testing process, which is then used to guide the generation of targeted tests (Sec. 3). 3 We build the first context- aware testing system, SMART Testing, which employs LLMs hypothesize likely and relevant model failures and empirically refutes them with data using novel self falsificalin mechanism (Sec. 4). ④ We demonstrate the value of context for effective testing, challenging the de facto data-only paradigm by showing how SMART identifies impactful model failures while avoiding false positives across diverse settings, when compared to data-only testing. Additionally, SMART identifies failures on important societal groups and generates comprehensive model reports (Sec. 5).
2 Related work
To highlight the need for SMART Testing, we contrast it with other ML testing paradigms specifically Data-only testing methods which address the same testing problem as SMART. We provide an overview in Table and an extended discussion in Appendix A.
Data-only testing methods [12-15]: Address the question: "what should we test?". Data-only methods search the data to find "slices" where the model's predictions underperform compared to average performance, deeming those slices model failures. Although automated, data-only
systtmmttic modular, automated, requirrementssresppssiie transfrrbble

---

approaches operate only on raw data without accounting for the problem context. Consequently, they must search across a large space potential failures, usually covering all subsets of features and their distinct values. While an exhaustive search may seem beneficial, as we show in Sec. 3, the reality that performing many tests on finite dataset risks discovering slices where model failure irrelevant or due to random variability, i.e. the multiple testing problem. SMART Testing addresses this challenge by prioritizing relevant and likely model failures through contextual awareness.
Orthogonal testing dimensions: While SMART addresses what to test, several other dimensions of model testing exist that are orthogonal to our approach (detailed in Appendix A). (i) Behavioral Testing [16. 17, 5] evaluates model behaviors (i.e. responses to data) by operationalizing tests along pre-defined dimensions (often defined by humans) - rather than discovering the test dimensions. SMART fundamentally differs by addressing the core issue of "what to test". (ii) Software functional testing [18, 19), aims to primarily test functional correctness (e.g. input-output functionality such as monotonicity), rather than testing for failures. In addition, test cases are either pre-specified or specified with an approximation of the underlying model to probe for functional correctness.
3 A context-aware testing framework for ML
The prevailing paradigm for testing ML models relies on data-only methods which exclusively use data to surface model failures. In this section, we explore the limitations of data-only methods by viewing ML testing as multiple hypothesis testing problem. We explore why data-only methods are uniquely prone to finding false positive and false negative model failures. To address this, we introduce a new paradigm of testing called context-aware testing which relies on external knowledge, or context, as an inductive bias to better identify where models fail.
3.1 A multiple hypothesis testing view of MI. evaluation
Preliminaries. Denote the feature space by X' and the label space by yM and P the joint probability distribution over X x V. We wish to test a fixed, trained black-box model f X' + V, using finite dataset D C X' x )' usually split into Dtrain = {II; " and Drest {(z;, })) Mi We assume the existence of a loss function é: y x ) + which measures the discrepancy between the model's 's prediction and the true labels point-wise.
The primary goal of ML testing is to identify meaningful failure modee--subgron (data slices) of the data distribution where the model's performance is significantly worse than its average behavior. Formally, let S X x y denote data slice and let Ps = P(-1((,,) E 8) be the conditional distribution induced by S. We aim to identify slices where the slice-specific expected loss As ( [((f(x), y)] significantly exceeds the population-level expected loss ppp B(a,)>~ ( (x))
Testing ML models is a multiple hypothesis testing problem. We are interested in identifying failure modes that generalize beyond the training dataset. We can interpret the empirical dataset D as a sample from a broader distribution P, and our goal is to make an inferential claim on the performance on the data slices with respect to P. Suppose we have a candidate data slice S C D. {(((x), y) and compare it
We can evaluate the empirical slice-specific loss as شss = 1 to the empirical loss over the entire dataset Å/P = |D] -(=0)) {(I(x), To make an inferential claim about the model's performance on the data slice S w.r.t. P. we can follow the frequentist testing paradigm and formulate a hypothesis test where we evaluate whether the performance is significantly different. Therefore, Ho μs = up and alternative hypothesis H₁ us * μDD, where js (x.,y)>PP [ [x( v)] and nD = (aa,y)> [(((xx)))))))))))))))))))))))))) y)] denote the true slice-specific and population-level losses. In practice, this evaluation could be done by running an appropriate frequentist statistical test and evaluating whether p < for each slice, given some pre-defined a.
However, realistic testing scenarios, we evaluate the model's performance not just on a single slice but on large collection of candidates {S,) This amounts to conducting many simultaneous hypothesis tests. Accounting for multiple testing is important. naive testing procedure that does not adjust for multiplicity could surface a large number of spurious failure modes simply by chance
'while technically any subset of the dataset can be considered data slice, we are practically interested in meaningful failures, such model failures vulnerable occi-econnoiic groups
3

---

(Type I error). Conversely, controlling the false discovery rate [20] may involve adjusting the per-test significance threshold a' < a. potentially sacrificing power to detect true failures (Type II error).
? The multiple hypothesis testing viewpoint reveals a key challenge in ML model evaluation: To reliably surface meaningful failures, we require principled procedure for generating a relatively small number of promising hypotheses (candidate slices) to test.
3.2 The failures of data-only testing
Existing ML testing methodologies are data-only in that they only use the available empirical data to test for model failures and vary in their optimization objective [14, 12, 13, 21, 15]. However, data-only methods operate under the restrictive assumption that the available empirical data the sole input for testing ML model performance. In practice, this is almost never the case. It is common to have an a priori understanding of where models are likely to fail given the data distribution, model class, training algorithm, and deployment context. This restrictive assumption results in three challenges: (i) High false positive rate: data-only methods search over a large space of data slices and each evaluation amounts to implicit hypothesis test (Sec. 3.1). Therefore, the probability of observing a false failure increases with every test performed. > (ii) High false negative rate: The naive testing procedure can be made robust by correcting for the number of tests performed which reduces the statistical power to detect true failures. A (iii) Lack of meaningful failures: Data-only methods are fundamentally limited by the fact that not all statistically significant slices practically meaningful. We empirically validate these claims in Sec. 5.
@ The core limitation of data-only testing is the lack of a principled failure mode discovery procedure that can incorporate prior knowledge to guide the search for meaningful errors.
3.3 Formulating context-aware testing
To address the limitations of data-only testing, we introduce context-aware testing (CAT), a princi- pled framework for identifying meaningful model failures using context. This could be the context implicitly encoded in the dataset (i.e. via meaningful feature names) or available external input (i.e. external contextual knowledge, such as a string of input information from a human). Let denote the space of all possible contextual information and E C be specific external input. Our core insight is that we can use C as an inductive bias to select which slices to test for.
Definition 1 (Context-Aware Testing). Let X, V, P, f. D (:) and & be defined in the standard supervised learning setup. Let C be space of contexts.
Context-aware testing is defined by two procedures:
1. A condext-guided slice sampling mechanism π C (X x x X such that x (c, D, m) = {SI..... Sm.... where e is used as an inductive bias for function a to prioritize slices likely to contain meaningful failures, and E N are the number of slices evaluate.
2. A multiple hypothesis testing procedure: VS, E <lc, D.m). test Ho pss, = PDD vs. [e([(x),v),, -Fe,,))-ss,11UU
H1 ps, # pDD, where ps, = B(
pD E (x))
A meaningful failure is characterized by statistical significance and practical relevance.
The targeted sampling mechanism æ uses the context as an inductive bias to prioritize testing of slices that are (i) more relevant to the deployment context, and (ii) more likely to exhibit significant performance gaps.
This principled slice selection offers several key advantages over data-only methods: (i) Improved false positive control: by limiting the number of tests conducted to m, CAT controls the risk of spurious discoveries that arise when naively testing all possible slices. (ūi Improved true positive rate: The targeted selection of slices likely to contain failures maintains test power by avoiding the need to aggressively correct for multiple testing. > (iil) Meaningful failures: context- context-guided guided sampling identifies failures that are both statistically significant and practically relevant.

---

? Context- aware testing overcomes the limitations of data-only methods by employing a principled, context- -aitext-guiedd slice sampling mechanism и to prioritize the discovery of meaningful model failures.
The core technical challenge in realizing CAT is the development of an effective context-conditionll sampling algorithm æ. In the following section, we propose a concrete instantiation of using large language model to generate plausible failure modes and guide testing of ML models.
SMART Testing
Ampar
Hypothasi genaration (Slep f)
Operatonaissttor & Fastticttion (Smp 3)
Destuation [See
Opertienal
Ingerbois
Cata
phon
Operasonaltas
Model Peport
SMART Tester
System
aupu
Figure 1: Overview of the SMART Testing Framework showing the four steps. All steps automatically executed by an LLM.
We now instantiate the CAT framework outlined in Sec. 3.3 with method called SMART testing (systematic, modular, automated, requirementt-respoiiive and transferable). SMART generates relevant and likely hypotheses about potential model failures and empirically evaluates these hy- potheses on available data. SMART follows a four-step procedure: (1) Hypothesis generation; (2) Operationalization; (3) Self-falsification; (4) Reporting. Table I provides an early illustration of what practically means to generate hypotheses and test them with SMART, The procedure below details how the four steps are implemented.
Table I: Example hypotheses on model failure, justifications, and operationalizations generated by the SMART framework on a healtheare dataset. The p-values show whether the model's performance significantly differs from average performance with Accc measuring the effect size.
Hypothesix
Intifictiion
Operaiicalization
pollu:
| Ace)
Evidence
underpesfoons on diserly pa- neres
Elaerly patiznia may lone more cumples kolth sitations to agconcheal
age >=
0.000
0.99
Supporial
comochidines, which could make predicionss less accurare.
underperfoms on patients with muttiple conoebidices.
The prosence of multiple comortutitis could complicase the prodiction model due to inseractions between differeut Zeath conditions.
comme hidice >= 0.900
0,270
Not supported
anderperfiens on paticess undes pring conurvativn montagennn
Conervvtiv managemest night be chose for paticuss with more comples low profosable csen, which could load wene peofictive gerformenze.
treotinE conservaivee
0.000
02000
Supported
Step 1: Hypothesis Generation. Recall from See. 3.3, we wish to define a sampling mechanism I to sample slices S which are both relevant and have a high relative likelihood of failure where should be both contextually-aware and able to integrate requirements to guide sampling.
We posit that LLMs have the potential to satisfy these properties due to the following capabilities: Contex- tual understanding: LLMs have been pretrained with a vast corpus of information and hence have extensive prior knowledge around different contexts and settings (22 25]. Integrate requirements: LLMs are adept at integrating requirements additional information about the problem via natural language [26, 24). Hypothe- sis proposers: In proposing likely failure modes, L.MSs have also been shown to be "phenomenonal hypothesis proposers" [27].
Process input sequences (c,,C)
Generate hypothesss (H, J.) ((.)
H
Hg
EED -Co
LLM
Hx
Operationatize hypotheses
Dets Space D
Figure 2: SMART uses an LLM to integrate context C. and data context D... Relevant and likely failure hypotheses are then generated by LLM (i.e. sampling mechanism). The hypothe- ses are then operationalized in DD and evaluated. In contrast, data-only methods are not guided by context and requirements, searching more exhaustively in D for divergent slices.
An LLM defined as a probabilistic mapping (: > P(E), where Σ denotes the vocabulary space, and P(E) represents the probability distributions over Σ. The model processes input sequences s E Σ', each con- catenation of tokens representing external (contextual) input C. (which can be null) and dataset contextualiza- tion CD formalized as s = (Ce: Cp). We extract the contextualized description CD from the dataset D using
5

---

an extractor function E D + CD, which captures essential dataset characteristics (e.g. feature relationships and high-level dataset information). Additionally, we highlight the the LLM will implicitly extract context based on the context encoded in the dataset (e-g. via meaningful feature names). Based on input 8, the L.LM predicts a distribution over Σ from which hypotheses of model failure and corresponding justifications are sampled.
As depicted in Fig. 2, given C, and Cp. we sample the N most likely hypotheses of failures, H = 1 Hl....., ,, and corresponding justifications J =
to provide ex- plainability. This process is formalized using the LLM's 's mapping as follows:
J4) ((s), where
CD), vi E (1.2,....
Step 2: Operationalization. The process of operationalizing each hypothesis HI, E H. involves translating its natural language expression into a form that can directly operate on the training dataset Dirain (an example is provided in Table 1). To achieve this, we define an interpreter function I: H - {0, 11.d that maps each natural language hypothesis H;
corresponding binary-valued function gi X' + (0. 1) on the feature space X, where gi g;(x) = 1 if x satisfies the criteria and (x(x) = otherwise. Each function 9 induces a data slice S; C D urain consisting of data points that satisfy the criteria of hypothesis H,, such that S. {(x, v) E 'Drain 9:(x)) 1.. Therefore, each hypothesis, after operationalization, corresponds to a specific slice that is being tested on. Steps and 2 serve to practically instantiate from Sec. 3.3.
Step 3: self-falsification We introduce a novel self- falsification mechanism to empirically evaluate (or
Hypothesis H generation
H evalusaion on data
refute) the generated hypotheses". Specifically, for each feasible hypothesis H; E H. we attempt to fal- sify the hypothesis with observed empirical data 5. This involves evaluating the model I over the data slice S; operationalized from H,. We then assess whether the slice performance on f has a signifi- cant deviation from the model's overall performance. For instance, in Table 1, this is done by computing |Acc| and the p-value. The significance of this de- viation is determined through frequentist statistical testing, i.e. when p < a for any a which might also be adjusted for multiple hypothesis testing. This step effectively "reshuffles/reranks" the hypotheses based their likelihood the observed data. For example, when benchmarking we select the top n hypotheses based on statistical significance: H" = arg min,, {p: < a). N;EHT
D Diriio
Ded.
Dusi
H
J1
91
St
p <
H2
J2
S2
Jn
74
Figure A self-falsification module within the SMART framework. A hypothesis generator gen- erates plausible hypotheses and justifications for when the model might fail. This is operationalized with d; and tested against the empirical data.
Remark: As shown in Fig. 1, can exclude the self oofffaalsiiication from SMART in cases of small-sample sizes. We denote this ablation of SMART as SM ARTNST-
Step 4. Model Performance Evaluation and Reporting. Finally, SMART automatically generates a report of the overall performance of the model under varying conditions generated by the LLM, including summary report, a complete summary of the tests carried out, intermediate and final results, and potential failure modes of the ML model.
@ SMART is a tabular CAT method which (i) directly searches for model failures, sampling targeted tests (Sec. 3.3) and (ii) incorporates data, and context into ML testing.
Practical use of SMART.
We highlight the practical use of SMART testing and emphasize SMART's s case of use and minimal input requirements needed. In particular, shown in the example below, users do not need any prior knowledge to use SMART. Rather, we make use of the context inherently encoded in the dataset, feature names and task.
*self- Self--aasiiication is rooted scientific philosophy and the seminal ideas of Popper [28] who proposed the principle of falsification the ability to refute hypotheses with empirical observations- as driving science. We assume data for feff-falsificatin is available. If not, discovered slices could guide new data collection.

---

import SMART
Instantiate SMART
=oodeltteteer SMART (MARt(''pt-45 g4t
Give desired context (this can be left as an empty string)
context - "Find where a model fails for the cancer prediction task." Load ML model
model XCBoost
detreei X. describe
Test the model 10 model_tester. (X, context, dataset_ siasset_descrsption, model)
Code Listing Use of SMART Testing
Illustrating SMART use cases
We now quantitatively evaluate SMART Testing and demonstrate the viability of this new ML testing paradigm (i.e. CAT), in contrast to data-only testing. Table 2 summarizes our experiments and takeaways.
Baselines. We compare SMART with a variety of data-only testing baselines namely: Autostrat [14), Slicefinder [12], Sliceline [13]. Pysubgroup [21] and Divexplorer [15]. We also evaluate SM ART (i.e. no self-falsification) as an ablation. Given space limits, we exemplify the LLM in SMART by GPT-4 [29] and our ML model to audit is logistic regression. We investigate other LLMs the Appendix D.3 and other tabular models in Appendix D.4, observing similar results. We provide additional experimental details in Appendix C.
Table 2: Summary of experiments and takeaways.
Experiment
inclevant
caterrfoming SMART decwnrs dicee winh perforaane dierpprccto medels.
SMART conisienlly wvidd inclevan
methed.
Assos shility steriity sigeniiant modil fillnn
Tubic
Meaure ENR slettfying unkeesffennng sibgrougs
SMART achieves the linest false ueguive rues settings.
Tikee 5
Asses robustees to pocanial LLM bizes
SMART dflectichs niigge hise deaiifinn undeoefooniise subgrougs
Praslani shilny caiely tcaing ropimmnnes
crisii mist sopurmmets whik muimanny antidical signltcancs.
Figur
Joses sanille
feature detection
SMART comipently avoid incievir festues reguidess surple
sices
Asaza
sample exutent fillures
parfommnce
SMART mesisaally ostperionn data mahhoh
all sample
Tible 12
Evalune teodacy fag
SMART avads qurious fultures, unlisz data only metheds.
App D3
Compare perticu mance of GP133 and GPT-4 SMART
SMART ming bott versices octperfom beactrafk methods
13,
Evalaaic SMART acrows differen
modeh
larger pefformane diserpuacio acnes typothesic
models
saing
Compar
tygothens open-weight models
SMART rpra-ariglt models for cggilicntt overap-11 hygtheees
Tikle 22
Bvalste SMART
SMART ideddizs lange pefoemance dicrepancios aznsa vasioss models
App. D.10
Showowe SMART's practiod surput
SMART pramasss compoheemite moddd reprrts pronsdng clar pulliice carch hypethoek.
5.1 Robustness to False Positives
Goal. We aim to underscore the role of contextual awareness in preventing false positives in model testing. In particular, consider the scenario when dealing with tabular data that may contain many irrelevant or uninformative features [30], persisting even post-feature selection [31, 32). We contrast SMART which explicitly accounts for context, to data-only approaches which are context-unaware and can only operate on the numerical data.
Setup. We fit predictive model to the training dataset, varying the number of irrelevant, syntheti- cally generated features contained in the dataset. The irrelevant features are drawn from different distributions. We then quantify the proportion of conditions in the identified slices that falsely include the irrelevant synthetic features. We evaluate using five real-world tabular datasets spanning diverse domains, namely finance, healthcare, criminal justice and education: loan, breast cancer, diabetes, COMPAS recidivism [33] and OULAD eduction [34]. These datasets have varying characteristics, from sample size to number of features and representative of different contexts pertinent to tabular MI., to demonstrate the effectiveness of SMART across various real-world contexts
Analysis. Fig. 4 shows the proportions of irrelevant features included in slices for increasing numbers of irrelevant features. Data-only methods are unaware of context and are shown to spuriously include high proportions of irrelevant features in their slices; i.e. false positives (FPs). Additionally, these false discoveries increase for data-only methods as the number of irrelevant features increases.
*Code can be found at: settps:/ggihhe com pauut/pallaia http:pplluuusss con/vvandersccaarla////i Testing
Testing https://github
7

---

Peronn Indoraat folwees
Loan Defoult
AlAozrac Breast Cancer
A
COMPAS
Education
Siccinn
Diabetes
Roo
Rumber TIme wwand hearsees
Number erelesan: Teatures
Number Intevvat
Number
meleevnt Seatures
Number irslevent Teatures
Figure 4: Contextuul-awwrreees in SMART reduces FPs, i.e reducing the proportion of irrelevant features in slices. SMART is not sensitive to the number of irrelevant features, unlike data-only methods. J. is better
Consequently, the sensitivity to FPs of data-only methods risks that slices identified are neither empirically relevant nor meaningful. In contrast, SMART, by virtue of contexual-awareness when generating hypotheses is not sensitive to extraneous and contextually irrelevant features. Importantly, SMART also maintains its robustness to FPs even with an increasing number of irrelevant features.
Remark. We also assess sensitivity to the number of data samples. Fig. 9, Appendix D.6 shows that SMART remains robust variations irrespective of sample size. In contrast, data-only methods have variable performance with false discoveries sensitive to the sample size.
Avoiding non-existent failure slices. It is important that model testing does not flag failures when there are none. In Appendix D.5, we demonstrate that SMART's contextual awareness means that it is resistant to spurious failure slices that have no underlying relationships. In contrast, data-only approaches implicitly assume the existence of problematic slices, which we empirically show makes them prone to spuriously flagging non-existent failure slices.
Takeaway 1. SMART's contextual awareness ensures testing relevance, thereby reducing false positives across different scenarios, in contrast to data-only methods.
5.2 Targeting model failures
Goal. We assess whether the identified failure slices persist when evaluated on new, unseen data across different tabular models. This evaluates the generalization of the identified model failures across multiple different ML models.
Table 3: Identifying slices with the highest performance discrepancies. We show differences accuracies (cAAc between the top identified divergent slice and average perfor- mance across four classifiers (over runs). f is better.
Setup. We use the prostate cancer dataset [35] and aim discover slices indicative of underperformance. Thereafter, we assess generalizability of the identified failures across four different MI models (logis- tie regression, SVM, XGBoost and MLP)) Specifically, we compare the top identified slice from each method and compute the ab- solute difference between the accuracies of that slice and the remainder of the dataset. We posit that testing framework should be able to identify slices with perforeeff discrepan- cies on unseen data across multiple ML models.
RegreticR
SVM
XGBChestßer MLPClasiller
Ausostrat PSG
0.24 0.02
0.24 : 0.02
0.09 1: 0.09
0.24 1: 0.02
0.11
PSGLA
0.23 1.01
0:23 0001
0.31 0.07
0.23 + 0.01
Diverplorzr
0.05
0.11
0:00 0.13
0.15
0.02 : 0.06
Sherfinder
0.01 + 0.00
0.01 = 0.000
0.00 + 0.00
0.01 + 0.01
Sliceline
0.26 10..1.000 0,06
0,26 - 0.06
0.18 1 0.09
0.26 1 0.06
SMARTVAP
0 0.01
0,37 001
0:09 3 0.0
0.17 4 0.0
SMART
0.37
Analysis. Table 3 shows that SMART slices exhibit the greatest discrepancies in model accuracies on unseen test data across different MI. models indicating the discovered failure modes are generalizable. We find that SMART surfaces slices with greater performance differences compared to SMART (ablation without self falifiiisiiia,ionn)) highlighting the importance of the introduced self- falsification mechanism. In contrast, data-only methods fail to identify slices where the performance discrepancy is as large as SMART. This limitation can be attributed to the tendency of data-only methods to overfit the training data, leading to high false discovery rates.
Takeaway SMART discovers failure slices where the model substantially underperforms, gen- eralizing to unseen test data across different ML models. In contrast, data-only approaches fail to find slices where the difference in accuracies is large, highlighting the lack of generalizability and reliability of their findings.
5.3 Robustness to False Negatives
In our setup, false negatives (FNs) are directly tied to true positives (TPs). i.e. the more true positives we find, the fewer FNs we miss. Across multiple experiments (Fig. 4, Table 10, Table 3. Table 16), we consistently show that SMART identifies TPs at substantially higher rates than data-only. For example, in Table 16, SMART identifies an average of 9,6 out of 10 subgroups where the ML
8

---

model significantly underperforns. The fact that data-driven methods discover fewer such subgroups implies that they are missing the ones SMART uncovers.
Goal. That said, we conduct an additional experiment to directly assess the false negative rate, wherein can control issues (as FNs naturally unspecified in real data).
Setup. We simulate a dataset to predict recidivism (Y) based on five covariates: gender, race, age, income, and education. log( (0X ) = aj - Xog + 6gCnncome + 5gGeeducation)) + where ~ P(Y+))
(Ho, (o). We then train predictor function f on the data and symthetically introduce underperfor- mance on certain corrupted subgroups. For an individual i, if they belong to corrupted subgroup j. the prediction Y; is equal to ((.) with probability - p. and a random prediction sampled from Bernoulli distribution with probability p. prediction is simply f(X.). Finally, we measure how often each testing method identifies that the model f underperforms on subgroup.
If the individual does not belong to subgroup j. the
Table 4: False Negative Rate (FNR) for different meth- ods various settings. J is better.
Analysis. Table 4 demonstrates results where we synthetically manipulate/corrupt the performance of an MI. model on a sin- gle subgroup (n=1), two subgroups (n=2), and three subgroups (n=3) out of a total of five. The results show the average of 20 runs, where the corrupted groups are ran- domly selected within each run. We find that SMART consistently is least susceptible to false negatives across all corrupted variables, when compared to data-only methods which struggle especially once more than one variable is corrupted. This serves to corroborate our earlier results.
FNR (n=1)
FNR (n=2)
FNR (n=3)
Autostrat
0.75 + 0.44
0.88 = 0.22
0.92 = 0.15
pysubgroup beam
0.65 : 0.49
0.75 0.26
0.68 0.17
pysubgroup_apring
0.65 + 0.49
0.75 = 0.26
0.68 0.17
Divexplorer Sliceline
0.05 + 0. 22
0.40 = 0.35
0.72 0.31
0.25 + 0.44
0.50 = 0.36
0.70 0.28
SMART
0.00 0.00
0.05
0.15
0.38 0.27
Takeaway 3. SMART is less prone to FNs, compared to data-only methods across all settings.
5.4 Assessing and mitigating potential LLM challenges and biases.
Background. In many settings, we want SMART to be part of human-in-the- model evaluation, particularly to address challenges with LLMs, such as biases or missing dimensions. Let us first discuss how SMART addresses some issues by design and then perform an experimental assessment. Using data to mitigate LLM challenges: We use data in two ways (i) Data usage in the generation of hypotheses: Before generating explicit hypotheses of where the model is likely to fail, we provide the LLM with additional information about the data description and model failures of the training dataset. The hypotheses sampled are therefore reflective of the inductive bias of the LLM as well as being conditioned on the data itself; (ii) Data usage in falsifying hypotheses: core to SMART is the fafffalsifiiation mechanism where we iteratively generate hypotheses and test them on a validation dataset. Data is therefore used to filter out hypotheses which are not supported by the data. Hence, even if "incorrect" hypotheses about group failures are proposed, this step ensures they are discarded. SMART provides clear and transparent testing: This is done in two ways: (i) SMART's model reports: that document specific failure cases with natural language justifications (see Appendix 10.10 for examples). Automatically generated reports can be a useful tool for humans-in intte-----tt experts to audit and validate the testing process, such as evaluating whether tests should be added or removed. For example, domain expert (e.g. a clinician) could review the report to assess whether the identified failure modes are truly relevant and concerning that specific context. (ii) Tests include justifications: The justifications for the tests allow human users to inspect the model's 's testing procedures, understand the reasons, and audit for biases or missed dimensions.
Goal. Going beyond the mitigation strategies by design, we also assess SMART's robustness to prior biases. Specifically, assess common ethnicity related biases of LLMs.
Setup. We use the same data generat- Table 5: Proportion of times the corrupted minority subgroup ing process as Sec. 5.3.
is correctly identified as the top underperforming subgroup.
Corrupted White
Corrupted black
We train a predictor function f and simulate a scenario where we inten- tionally corrupt a model's predictions
Punite
Fiank
Pusite
Placc
0.01
178::00.00
0.15:2
0.16 0.01
:t
DA12
0.88 0.03
0.08 0.03
0.30 9.13
3.87 + 0:03
0.05
0.98 0.01
0.00 1: 0.00
0.00 0.00
0.98 : (1.01
for a proportion r of minority sub- group ("white" or "black" ethnicity).
0.10
0.98 4 0.01
0.00 0.00
0.00 1: 0.00
0.99 0.01
0.01
0.00
0.00 0.00
0.30
1.00 0.00
0.00 :: 0.00
0.00 : 0.00
1,00 4: 0.00
0.50
1.00 0.00
0.00 0:00::0.000 0.00
0.00 0.00
1.00 - 0.00

---

ranges from O (no corruption) to (completely random predictions for the subgroup). SMART aims to identify the top underperforming subgroup without bias based on historical pattern.. We measure the proportion of times the top subgroup contains the "white" (Pwhite) or "black" (Putack) minority, averaged over 20 runs and seeds, separately corrupting "white" and "black" ethnicities.
Analysis. We show in Table 5 that SMART is able to identify where models underperform even in scenarios such as ethnicity bias, where LLMs exhibit prior biases from the training dataset. This links to the above discussion that SMART mitigates such biases by design both using the training dataset to guide hypothesis generation and using the ealffaalsiicationn mechanism to empirically evaluate hypotheses (and discard those that aren't 't reflective of the data).
Takeaway 4. SMART mitigates potential biases in the LLM, both by using the real data to guide hypothesis generation, as well as using the seff-faliiication mechanism to filter spurious hypotheses.
6 Discussion and limitations
Responding to recent calls for better model testing [36, 6]. we formalize Context-Aware Testing: a new testing paradigm, actively seeking out relevant and likely model failures based on contextual awareness - going beyond data alone. We develop SMART Testing, using LLMs to hypothesize likely and relevant model failures providing improved and automated testing of tabular ML models, compared to data-only methods in various scenarios.
Model reports. SMART produces comprehensive and automated model reports documenting failure cases and justifications, thereby providing data scientists, MI. engineers and stakeholders increased visibility into model failures. We provide an example SMART report in Appendix 10100
Practical considerations. Given the potential utility of SMART we highlight the following five practical considerations: HHypothesis generation. While CAT offers principled framework for context-guided testing, LLMs present challenges in hypothesis generation. Although SMART has mechanisms to address these (see Sec. 5.4), it cannot guarantee the absence of biases. Use with small datasets. SMART may be limited in some cases by insufficient real data to operationalize and test hypotheses, suggesting future work could explore targeted data collection or synthetic data generation to enhance testing. Extensions to other modalities. SMART formalized to test tabular MI. models, due to the interpretable and structured features in tabular data to guide hypothesis generation. While extensions to other modalities such as image and text is beyond the scope of this work, this is promising future research direction that would require addressing the lack of explicitly interpretable features possibly via external metadata and developing new ways to operationalize hypotheses on unstructured data. That said, tabular data is ubiquitous in real-world applications [37, 38] with approximately 79% of data scientists working on tabular problems daily, vastly surpassing other modalities [39, 40). This highlights the immediate impact and relevance Aurpasi Need for interpretable/eeamnngiii feature names. Feature names play an important role in finding model failures (see Appendix Appendix D.9). The need for interrreaable/menninggh feature names (e.g. column labels such as sex, age, race etc) as a source of context is similar to human requirements of interpretable feature names understand what the data refers to. While feature names are typically the de facto in research and industry datasets, in the rare occasions they are not, this will affect the performance of SMART. Cost of SMART. SMART extremely accessible and cheap to use, approximately 10 USD for 5 hypotheses and <0.5 USD for 100 hypotheses for state-of-the-art models (see Appendix D.7).
of S.ART.
Broader impact. Better testing practices can help ensure models are reliable, safe, and beneficial before being deployed in real-world applications. We hope that our work can help mitigate model testing risks in real-world applications as well as spur new testing regimes which are context aware.
Acknowledgments and Disclosure of Funding
We would like thank the reviewers, Fergus Imrie, Nicolas Astorga, Kasia Kobalezyk, Tennison Liu and Andrew Rashbass for their helpful feedback. PR is supported by GSK, ML is supported by AstraZeneca, NS by the Cystic Fibrosis Trust. This work was supported by Microsoft's 's Accelerate Foundation Models Academic Research initiative.
10

---

References
[1] Luke Oakden-Rayner, Jared Dunnmon, Gustavo Carneiro, and Christopher Ré. Hidden strat- ification causes clinically meaningful failures in machine learning for medical imaging. In Proceedings of the ACM Conference on Health, Inference, and Learning, pages 151-159, 2020.
[2] Harini Suresh, Jen J Gong, and John V Guttag. Learning tasks for multitask learning: Heteroge- nous patient populations in the ICU, In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 802-810, 2018.
[3] Karan Goel, Albert Gu, Yixuan Li, and Christopher Re. Model patching: Closing the sub- group performance gap with data augmentation. In International Conference on Learning Representations, 2020.
[4] Angel Alexander Cabrera, Minsuk Kahng, Fred Hohman, Jamie Morgenstern, and Duen Horng Chau. Discovery of intersectional bias in machine learning using automatic subgroup generation. In ICLR Debugging Machine Learning Models Workshop, 2019.
[5] Boris van Breugel, Nabeel Seedat, Fergus Imrie, and Mihaela van der Schaar. Can you rely on your model evaluation? improving model evaluation with synthetic test data. Advances in Neural Information Processing Systems, 36, 2024.
[6] Nabeel Seedat, Fergus Imrie, and Mihaela van der Schaar. Navigating data-centric artificial intelligence with DC-Check: Advances, challenges, and opportunities. JEEE Transactions on Artificial Intelligence, 2023.
[7] Oleg S Pianykh, Georg Langs, Marc Dewey, Dieter R Enzmann, Christian J Herold, Stefan O Schoenberg, and James A Brink. Continuous learning AI in radiology: Implementation principles and early applications. Radiology, 297(1):6-14) 14, 2020.
[8] Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga. Richard Lanas Phillips, Irena Gao, Tony Lee, Etienne David, Ian Stavness. Wei Guo. Berton A. Earnshaw. Imran S. Haque, Sara Beery, Jure Leskovec, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, and Percy Liang. WILDS: A benehmark of n--the-wild distribution shifts. In International Conference on Machine Learning, pages 5637-5664. PMLR, 2021.
[9] Kayur Patel, James Fogarty, James A Landay, and Beverly Harrison. Investigating statistical machine learning as a tool for software development. In Proceedings of the SIGCHI Conference on Human Factors Computing Systems, pages 667-676, 2008
[10] Lea Goetz, Nabeel Seedat, Robert Vandersluis, and Mihaela van der Schaar. Generalization key challenge for responsible ai in patient-facing clinical applications. npj Digital Medicine, (1):126, 2024.
[11] Máire A Duggan, William F Anderson, Sean Altekruse, Lynne Penberthy, and Mark E Sherman. The surveillance, epidemiology and end results (SEER) program and pathology: towards strengthening the critical relationship. The American Journal of Surgical Pathology, 40(12)e994 2016.
[12] Yeounoh Chung, Tim Kraska, Neoklis Polyzotis, Ki Hyun Tae, and Steven Euijong Whang. Slice finder: Automated data slicing for model validation. In 2019 IEEE 35th International Conference on Data Engineering (ICDE), pages 1550-1553. IEBE, 2019.
[13] Svetlana Sagadeeva and Matthias Boehm. Sliceline: Fast, tinear-algebral slice finding for ml model debugging. In Proceedings of the 2021 International Conference on Management of Data, pages 2290-2299 2021.
[14] Adebayo Oshingbesan, Winslow Georgos Omondi, Girmaw Abebe Tadesse, Celia Cintas, and Skyler Speakman. Beyond protected attributes: Disciplined detection of systematic deviations in data. In Workshop on Trustworthy and Socially Responsible Machine Learning, NewrIPS
2022, 2022.
11

---

[15] Eliana Pastor, Luca De Alfaro, and Elena Baralis. Looking for trouble: Analyzing classifier behavior via pattern divergence. In Proceedings of the 2021 International Conference on Management of Data, pages 1400-1412, 2021.
[16] Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh. Beyond accuracy: Behavioral testing of nip models with checklist. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4902 4902-4912, 2020.
[17] Paul Röttger. Bertie Vidgen, Dong Nguyen, Zeerak Waseem, Helen Margetts, Janet Pierre- humbert, et Hatecheck: Functional tests for hate speech detection models. In Proceedings of the 59th Anmual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume Long Papers), page 41. Association for Computational Linguistics, 2021.
[18] Maria Christakis, Hasan Ferit Eniser, Jörg Hoffmann, Adish Singla, and Valentin Wüstholz. Specifying and testing k-safety properties for machine-learning models. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, pages 4748-4757, 2023.
[19] Arnab Sharma and Heike Wehrheim. Higher income, larger loan? monotonicity testing of machine learning models. In Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis, pages 200-210, 2020.
[20] Yoav Benjamini and Yosef Hochberg. Controlling the false discovery rate: practical and powerful approach to multiple testing. Journal of the Royal statistical society; series B (Methodological), 57(1))289-300, 1995.
[21] Florian Lemmerich and Martin Becker. pysubgroup: Easy-to-use subgroup discovery in python. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 658-662, 2018.
[22] Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmano, et al. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311 023111, 2022.
[23] Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et Large language models encode clinical knowledge. Nature, pages 1-9, 2023.
[24] Nabeel Seedat, Nicolas Huynh, Boris van Breugel, and Mihaela van der Schaar. Curated LLM: Synergy of LLMs and data curation for tabular augmentation in low-data regimes. In Forty-first International Conference on Machine Learning. 2024.
[25] Nicolas Astorga, Tennison Liu, Nabeel Seedat, and Mihaela van der Schaar. Partially observable cost-aware active-learning with large language models. Advances in Neural Information Processing Systems, 38, 2024.
[26] Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V Le, Denny Zhou, and Xinyun Chen. Large language models optimizers. arXiv preprint arXiv:2309.03:409, 2023.
[27] Linlu Qiu, Liwei Jiang, Ximing Lu, Melanie Sclar, Valentina Pyatkin, Chandra Bhagavatula, Bailin Wang, Yoon Kim, Yejin Choi, Nouha Dziri, et al. Phenomenal yet puzzling: Testing inductive reasoning capabilities of language models with hypothesis refinement. arXiv preprint arXiv:2310.08559, 2023.
[28] Karl R Popper. Science as falsification. Conjectures and refutations, 1(1963):33-3) 1963.
[29] OpenAl. Gpt-4 technical report, 2023.
[30] Md Rezaul Karim, Md Shajalal, Alexander GraB, Till Döhmen, Sisay Adugna Chala, Alexander Boden, Christian Beecks, and Stefan Decker. Interpreting black-box machine learning models for high dimensional datasets. In 2023 IEEE 10th International Conference on Data Science and Advanced Analytics (DSAA), pages 1-10. IEEE, 2023.
12

---

[31] Danilo Vasconcellos Vargas, Hirotaka Takano, and Junichi Murata. Contingency training. In The SICE Annual Conference 2013, pages 1361-1366. IEEE, 2013.
[32] Léo Grinsztajn, Edouard Oyallon, and Gaël Varoquaux. Why do tree-based models still outperform deep learning typical tabular data? Advances in Neural Information Processing Systems, 35:507-5200 2022.
[33] Julia Angwin, Jeff Larson, Lauren Kirchner, and Surya Mattu. Machine bias. ProPublica: htpps//wwwppopuiiicccoraddaadaid aisss-aasessmettt----iriimmeee 2016.
May
[34] Jakub Kuzilek, Martin Hlosta, and Zdenek Zdrahal. Open university learning analytics dataset. Scientific data, 4(1):1-8, 2017.
[35] Prostate Cancer UK PCUK. Cutract. htpps:/ppoosuteeeneeer...rgg 2019.
[36] Negar Rostamzadeh, Ben Hutchinson, Christina Greer, and Vinodkumar Prabhakaran. Think- ing beyond distributions in testing machine learned models. In NeurIPS 2021 Workshop Distribution Shifts: Connecting Methods and Applications. 2021.
[37] Vadim Borisov, Tobias Leemann, Kathrin Seßler, Johannes Haug, Martin Pawelczyk, and Gjergii Kasneci. Deep neural networks and tabular data: A survey. arXiv preprint arXiv:2110001880 2021.
[38] Ravid Shwartz-Ziv and Amitai Armon. Tabular data: Deep learning is not all you need. Information Fusion, 81:84 90, 2022.
[39] Kaggle. Kaggle machine learning and data science survey, 2017. URL httpsttss////w.kkee com com/datasets//ean htwpiaassstssggggl/laaaee
su17
[40] Lasse Hansen, Nabeel Seedat, Mihacla van der Schaar, and Andrija Petrovic. thetic tabular data generation through data-centric ai: A comprehensive benchmark. Advances in Neural Information Processing Systems, 37:33781-3388 2023.
Reimagining syn-
[41] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations, 2018.
[42] Greg d'Eon, Jason d' d'Eon, James R Wright, and Kevin Leyton- Leyton-Brown. The spotlight: A general method for discovering systematic errors in deep learning models. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 1962-1981, 2022.
[43] Michael P Kim, Amirata Ghorbani, and James Zou. Multiaccuracy: Black-box post-processing for fairness in classification. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pages 247-254, 2019.
[44] Sahil Singla, Besmira Nushi, Shital Shah, Ece Kamar, and Eric Horvitz. Understanding failures of deep networks via robust feature extraction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12853 12862, 2021.
[45] Sabri Eyuboglu, Maya Varma, Khaled Saab, Tean-Benoil Delbrouck, Christopher Lee-Messer, Jared Dunnmon, James Zou, and Christopher Re. Domino: Discovering systematic errors with cross-modal embeddings. arXiv preprint arXiv:2203. 14960, 2022.
[46] Shreya Shankar, Labib Fawaz, Karl Gyllstrom, and Aditya Parameswaran. Automatic and precise data validation for machine learning. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pages 1198-2207, 2023.
[47] Neoklis Polyzotis, Martin Zinkevich, Sudip Roy, Eric Breck, and Steven Whang. Data validation for machine learning. Proceedings of machine learning and systems, 1:334-341 2019.
[48] Gary S Collins, Johannes B Reitsma, Douglas G Altman, and Karel GM Moons. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (tripod) the tripod statement. Circulation, 131(2):211-213 2015.
13

---

Appendix: Context-aware testing: a new paradigm for testing with large language models
Table of Contents
A Extended related work
15
A.1 Enhanced overview of relevant literature
15
A.2 Components of the ML Testing pipeline
17
A.3 Comparison of the features of slice discovery methods
17
SMART Details
Contextual understanding
B.2
Operationalizing variables
................................
18
B.3
Feasibility checks
20
B.4 Data adjustment queries
21
B.5 Requirements, automation, and outputs
21
B.6 Moving outside of IID data
22
B.7 SMART and multiple testing
22
Benchmarks & Experimental Details
23
C.I Datasets
23
C.2 Baselines
................................
23
C,3 LLM prompts
23
C33.1 Generic templates...
23
C.3.2
Example prompts: OULAD Education.
24
Experimental details
29
C.4.1
Context-awand aware testing (Sec. 5.1.)
29
C.4.2
Requirements
Requirements-coontrrandd constrained testing (Sce. 5.2.)
29
C.4.3
Targeting model failures (Sec. 5.3.)
30
C.4.4
Adaptive testing for deployment environment (Sec. 5.4.)
30
C.4.5
Discovery of socictally important groups and failure understanding (Sec.
5.5.)
30
D Additional experiments
31
D.1
Requirements--oostriaee testing
31
D.2
Adaptive testing for deployment environment
32
Effects LLMs
................................
33
D.3.1
Comparison over identified divergent slices
................................
33
D.3.2
Performance across different models
................................
33
D.3.3
Hypothesis generation.
34
D.4
Effects of different tabular machine learning models
35
D.5 On the inductive biases of MI. model testing
36
D.6 Context aware sensitivity
37
D.7 Cost SMART
38
D.8 SMART with open-weight models
38
D.9 Understanding the importance of feature
38
D.10 Example of model report
40
14

---

A Extended related work
A.1 Enhanced overview of relevant literature
This work primarily engages with works on ML model testing. Consequently, we detail different paradigms of ML model testing next, which we summarize in Table 6 and Figure 5 showing that none of the existing paradigms satisfy all the properties for ML testing neither terms of automation nor from the perspective of relevance as they are unable to incorporate context and/or requirements.
Table 6: Comparison of ML testing paradigms in terms of how tests are defined, the type of search space (i.e. relevance based on context and requirements), sensitivity to multiple testing and automation to enable scalability.
Seurch space
Paradigm
Objjectve
Test definition
Context-aware
Integrate Requirements
Autoonated
testng
Outpons
Examples
Average testing
Overall perfommance
N.A.
Behavioral
Test expected behavior
Pre-detined (Human)
N.A.
N.A.
dimencional scoce
17,51 [14, 15, 131
Dats-only
Mdnityy divergost groups
Search
X
Miltidiiennionll secce
Mulhi dimensiimal suoce justifications report
SMART (Oues)
Probe for model fuilare
Search
Existing paradigms in ML model testing. The ML community has predominatly approached ML model evaluation/testing via the use of held-out test datasets. On the basis of the test dataset a single performance metric (accuracy, AUC etc) is computed. This single average evaluation may mask nuances of the model's performance along various dimensions. One approach to address this is to create better benchmark datasets when evaluating models on common benchmarking tasks. For example, manual corruptions like Imagenet-C [41] or by collecting additional real data such as the Wilds benchmark [8]. Benchmark datasets are iotr-intensive to collect and their utility is limited to the specific benchmark tasks.
What if we want evaluate models not confined to benchmarking tasks? To test in cases beyond benchmark tasks, the community has proposed trying to find slices or regions wherein the model fails (i.e. via stress tests). As mentioned Sec. this could be Behavioral testing: which requires human expertise and intuition to define the test scenarios (c.g. Checklist [16], HateCheck [17] or 3S-Testing [5]) - which not automated and does not scale. Moreover, runs high risk of overlooking critical weaknesses due to human cognitive biases. Data-only testing: which does not account for context or requirements and searches exhaustively (e.g. Autostrat [14], SliceFinder [12] or DivExplorer [15]). This may slices focus on arbitrary, less important, or unrealistic/implausbbbe scenarios that are unlikely to be seen reality. Moreover, we run the risk of the multiple testing problem, where by virtue of the large number of tests evaluated, we might discover a divergent group by chance.
Hypothesis-driven ML model testing. In contrast, hypothesis driven approach to ML model testing brings about the falsifiability approach widely adopted in science. begins with the formulation of specific, testable hypotheses based on theoretical understanding, context/domain knowledge, and the intended application of the model (i.e. requirements). The concept of hypothesis-driven ML model testing is deeply rooted in work by Popper [28] who proposed the principle of falsifiability as driver of scientific progress. The progression of knowledge hinges on the formulation and rigorous testing of hypotheses which can either be falsified or supported by empirical evidence. In the context of hypotheses in MI model testing, testable statements about the model's performance under various conditions, fairnsss, and robustness can improve our understanding of the model's performance. By rigorously testing these hypotheses, we can uncover the strengths and limitations of MI. models.
Contrast to data-only methods on unstructured data.
In this paper, we have discussed data-only methods for slice discovery or blindspot discovery applicable to structured data. Specifically, we focus on tabular data where metadata in the form of column names is explicitly encoded into the data. The data-only approaches covered in this work directly search over the raw feature space to identify slices with similar attributes wherein the model would exhibit underperforming predictions.
For completeness, we contrast to data-only approaches often applicable to data without explicit structure and metadata (images, text, audio). There have been numerous methods including [42-45]
15

---

Low
Med
High
dxploy adapribiliiy
Automation & Adaptabilit
AutoMI
Desired ML testing framework
High
paradigma gms
Average group performance testing
Data-driven testing paradigms
Med
Autousted
No testing
Non-expert testing
Subject matter experts testing
Low
Testing Relevance
Requirementt-drive contest-awant curated search space
Figure 5: Contrasting paradigms of MI. model testing along (i) Testing relevance which accounts for requirements and/or context and (ii) Degree of automation and adaptability when carrying out testing. We desire new paradigm of ML testing to address both.
to identify slices of unstructured data with systematic failures. They all follow a similar pattern: (1) the data is embedded (often with pre-trained) model into representation space and (2) the underperforming slices are identified by clustering on the raw embedding space or after dimensionality reduction. We then need to post-hoc interpret the clusters in order to understand what they represent. This contrasts the tabular data setting where we find cohesive groups of features that provide an explicit interpretation. Moreover, in tabular regimes, we do not have access to pre-trained models in the same way as for domains such as images or text.
Specifically, to context-aware testing we also note that tabular data inherently includes context through interretableemmaningee feature names and metadata. This context not naturally present in images (which are tensors of pixel intensities). Hence, extending CAT to other domains like images text would require incorporating metadata provide necessary context which is often unavailable. In addition, one would need to develop new ways to operationalize hypotheses unstructured data.
Contrast to software testing.
Going beyond ML model testing, the idea of testing is also prevalent in software systems. For example, unit tests of functions in codebase. We highlight that software testing of functional input-output correctness also an area proposed software testing, which could also theoretically be applied to ML systems. That said, we contrast between SMART and these paradigms in Table 7 below, demonstrating that we tackle a different testing problem.
Table 7: Comparison of SMART testing with software testing works
Criteria
SMART
Christakis al. [18] Pre-Specifted dimen-
Sharma [19]
How SMART different?
Test genera-
L.M.-
Approximates the
SMART uses the LLM lomatically hypothesize the
tion
based test
sions of functional
black-bux model test with a white-box decision tree model. SMT solvers vio- lations monotonic- ity properties i.c. in- put increaset=output increase
correctness (k-safety
test cases & exccute them
propertie)
What is the focus of testing
Identify model fail- Assesses func- ures
SMART failures encompass broader range of possibili- ties
Are tests context- aware?
Yes
No
SMART uniquely incorpo- context into testing. leading to more reul- istic assessments.
Additionally, further orthogonal area to ML model testing is that of data validation [46, 47). In contrast, to testing an MI. model for failures, data validation aims to test input data pipelines for data quality problems drift.
16

---

A.2
Components of the ML Testing pipeline
Step Evaluation report ("How does model perform?" )
Step I: Test Search
Step Operationalize ("How on data?")
('What lest?")
Apprsuches:
Approaches: SMART Testing: Sufficient data or Behaviona testing: Insufficient data
Approuches: SMART Testing
(i) SMART Testing: Hypothesis generation (ii) Data-only
search
(o.g. Breugel [5)
Figure 6: Components of ML Testing - (i) Test search, (ii) Operationalize, (iii) Evaluation report, with example approaches for each component.
Test search: First, we need to decide "what to test". This is a search problem to identify test dimensions. This could either be done via SMART which is targeted (via context and requirements) or data-only methods (where the space is larger). Alternatively, human experts available, humans could define the test dimensions.
Operationalize:
Second, need operationalize the test and address the challenge of "how to carry out this test on data". If we have sufficient data SMART could operationalize the tests on the data via interpreter. Alternatively, if we don't have sufficient data to the test then SMART could be augmented by behavioral testing approaches such as van Breugel al. [5] which once the test has been defined synthetic data to augment small subgroups/slices.
Evaluation report: Third, we carry out the test and evaluate the model to answer the question, "how does the model perform'. SMART can be used to produce comprehensive report of failures and justifications an automated manner.
A.3 Comparison of the features of slice discovery methods
Even as we zoom into the task of discovering slices where the model might underperform, we observe that SMART has features which are not supported by most of other discovery methods. We exemplify some of these features in Table 8
Table Comparison of slice discovery methods
Criteria
SliceFinder Pysubgroup
DivExplorer
Autostrat
SMART Testing
Integrates custom domain knowledge
X
X
X
X
Constant slice discovery time
X
x
x
x
Performance is resistant to irrelevant data
X
X
X
X
Can capture rure slices
X
X
X
X
Inherently supports logical ORs Resistant to overfitting the training set
X
X
X
X
X
X
X
17

---

B SMART Details
We present block diagram of the key components of SMART Testing in Figure In addition, for each component we provide additional motivations and technical details not covered in the main paper. We further provide more technical information on certain implementation details.
Contextual understanding
Requirement-drivn
Automation
Contextual output
Evaluation
Contestual
Teat
Summary report
Inowlodge
Teal
Teat
Input data
Evaluation fremework
ML
Recuirements @ Context
Evaluation
Focus cases
Context informs model lesting
Evaluation incorporates
The process manusl input
describas cases clearly
Figure 7: A strong machine learning testing framework should incorporate textual understanding during testing. should meet requirements, should automated, and provide contextual reports with an emphasis on model failures.
B.1 Contextual understanding
core component of SMART is leveraging LLMs to strategically identify what test for tabular ML models. This process is anchored on the premise that LLMs can effectively navigate the space of potential model fallure slices, harnessing their contextual understanding pinpoint slices where the model is most susceptible to failure. The operation of LLMs within the SMART framework involves three critical inputs:
1. Context (C): A string describing the overarching scenario and the task at hand.
2. Dataset Information (D.): Extracted from the training data, this includes a description of the observations where the model did fail (j = y), or where failed ÿ * y). This could be characterized as string with description of the covariates of each group (e.g., mean, median, mode, and a textual description of the distribution).
Utilizing these inputs, the LLM generates set of hypotheses (H) and corresponding justifications (T) regarding potential model failures. The framework also incorporates self-refine mechanism to enhance hypothesis generation. This mechanism iteratively refines hypotheses based on the observed or provided context (C) and data (D.), re-ranking them by their likelihood. The self-refine mechanism introduced in order to generate hypotheses that more likely to target specific model failures.
B.2 Operationalizing variables
Once the hypotheses have been proposed, is important to operationalize them. This operationaliza- tion can be achieved through one of two methods:
Hypothesizing possible operationalizations (c.g., age > 70 way to operationalize "elderly people"). In this case, the previous interactions and information is provided to the LLM, together with the relevant description the data. is then asked provide possible ways to operationalize specific hypothesis. This is done by passing an "operationalization prompt" which contains the aforementioned information.
LLM-based Operationalization:
18

---

The following is an example of an operationalization prompt in Python:
1
The following are hypotheses about which people within dataset the model might underperform Propose apecific ranges for each hypothesis. Hypotheses (hypotheses).
4
Dataset information: {context). [context_target)
7
The dataset contains (len (unique columns. The columns are {', poin(unqque values k))s items
))). The values
are (str
TASK: Propose specific variable ranges for each hypothesis such that they are clearly operationalizable and defined. Use this format: Hypothesis: <>; Operationalization <>
Code Listing Operationalization (LLM knowledge): General template
This is then converted into specific operationalization for each hypothesis using an external compiler which maps the strings to function.
2. Data-driven Operationalization: Utilizing training data identify optimal splits for given covariates (e.g., age > 82.32 as condition to split data based on the "age" covariate). This method receives the covariate (or of covariates) as an input and returns the optimal split. The optimal split is defined the split which can identify two groups which have the largest absolute difference in accuracies given some requirements (e.g. minimum or maximum group size in each group). This splitting can be done by any black box splitting function which takes set of inputs and returns splitting mechanism.
Black-box splitting function for data-driven operationalization. While any black box splitting function could perform on the data, we provide details on the specific function used in SMART.
The function implemented is based on a decision tree model, which can be either a regressor or classifier, depending the nature of the outcome variable. In all experiments, we perform classification-basedd tasks.
The function operates by fitting decision tree the data and recursively traversing the tree to find the split that yields the largest absolute difference the outcome variable's mean value between two slices. The split must also satisfy group size constraints, specified as minimum and maximum group sizes. The traversal process evaluates each potential split, calculating the mean outcome for each slice and the discrepancy between these means. The optimal split is the that maximizes this discrepancy while adhering to the group size requirements.
The function returns query string that represents this optimal split. This string can then be used to segment the dataset into the identified slices for further analysis or testing. The following pseudocode for the splitting algorithm.
19

---

Algorithm Optimal splitting mechanism within the SMART framework
Data: data features, outcome, min main_group_sizee max naaz_group_size
Result: Optimal query string for data split
Function GetDptiimalSllitQuuer (data.frame, features, outcome, min cain__roup__sii_ max
mmux_grouppsize))
Validate input features and outcome in dataframe Determine the type of decision tree model based on oulcome type Fit the decision tree model to dalaframe using fealures and outcome Initialize an empty list conditions for tracking split conditions return TraverseTree (root, 0, conditions)
Function TreverseTree (node, depth, condilions):
node is leaf then
Calculate the discrepancy in outcome between the two slices return the condition and discrepancy if group size constraints are met
end
7
Determine left and right conditions based on the threshold at node return TraverseTree (nodelleff the right conditions, depending on which has the greater discrepancy
depth + 1, conditions U (left_condition) //
This splitting mechanism used for continuous features. separate black-box splitting mechanism developed for categorical features based iterating different permutations of these features and evaluating them that helps with variable selection.
In practice, employ the LLM-based operationalization for the SMART ablation and the data- driven operationalization within the original SMART framework. However, we highlight that this is design choice that have found works in practice; there is nothing stopping from using any operationalization framework within the main SMART framework.
B.3 Feasibility checks
SMART is built on many modules which can be toggled on or off. One of such modules is a module called "feasibility check" which evaluates whether there any hypotheses which should be tested in the first place. The experiments presented in Sec. 5.1 highlight the importance of being able to identify when relationship between covariates exist.
The feasibility check contains three steps. First, an LLM is queried to evaluate whether any relation- ships could exist between the covariates and outcome variable. For instance, this could be whether a relationship could exist between loan default and the annual rainfall in a given region. Second, the answer is self-refined. This helps evaluate feasibility because, in practice, initial responses tend to be over-optiiistic (such as hypothesizing that annual rainfall is associated with loan default via geographic proxy). This self-refinee relf-refinement helps critically evaluate the previous answer. The number of steps in the self-refinement process a hyperparameter. Lastly, the answer (whether or a relationship could exist between the two variables and, hence, should be inspected) is converted to boolean value via an external function.
The following is pscudocode which implements the feasibility module.
20

---

Algorithm 2 Feasibility for evaluating slices
Input:
unigue_ mmigue_values, contert, contert ttrgget system_ n n_refine
Output: bessbbilittbbooleen booleaanresppooseeeeeccceeee Function
wwwuuuuuuuuuuuuss..eeeaa contert, contert -tarrrttttrrtt,,, system_message
m_refine):
// Construct the feasibility task prompt
lask < "Evaluate subgroups for model performance" lask +- laskU "Context: U context U
"Target: context target task taskU "Columns: join(unique jointumiquu_uuuueskkeessss
// Get initial feasibility response
10
feasibility_ response GetLLMResponse (task, jystem__essane / / Refine the answer feasiblittyrresposs
(umiqqu values, contert, contert_targe
Setsbblity_responsee system_message, n_refine) // Convert boolean
12
boolean_lase
"Based on analysis, provide yes/no answer" booleun_lask
boolean boolean_lasku Jasku "Analysis:" " feaaiibiltty
13
www__boooolleeeaeeeeee responsee +- GetLLMResponse (hooleam_lash
B.4 Data adjustment queries
Given that SMART operates with an LLM, sometimes the framework outputs proposals which do not operationalize the data well. For instance, even if column "age" is categorical variable, SMART might propose to operationalize the hypothesis "elderly people" as age 72 which would cause eπrоr.
To avoid this, we implement an additional data adjustment module which can handle such cases. catches the error and re-prompts the L.M to find group which could be operationalized given the data structure, and does so iteratively until such a group is found.
The following is pseudocode function that explains how data adjustment is performed.
Algorithm 3 Pseudocode for adjusting subgroup/slice queries
Data: X (DataFrame), n_subgrouus (integer)
Result: Adjusted subgroup queries in the dataset
Function Ad jdjustuubrouupuuer (X, r_subbrouus))
15
for each subgroup condition do if ChecckuurryExiisteneee (X, condition) then UpdateeSubgroupConiittiio cen.ooonniiiioiooniiioiiiiiiiioi (condition)
16
17
18
else
// Condition yields rows, adjust it
19
adj usseed condition GetLLMResponse (condition) // Update the condition
in the subgroup Up (udjussted end
20
end
return
B.5 Requirements, automation, and outputs
Requirements. SMART can natively integrate user requirements into its framework. This done by inputting requirements string and concatenating together with the context. Some requirements are directly integrated into the framework itself (e.g. functionalities for determining the minimum or maximum sample size of data split).
Automation. Fig. showcases pipeline of SMART and which components are automated. SMART is developed using an "sklearn" style fit, predict framework. The fit method automatically performs a feasibility check, generates hypotheses, justifications, operationalizes them, performs self-falsification using empirical data, re-ranks the hypotheses, and saves all intermediate results. The
21

---

method can then automatically be used on any piece of data to evaluate whether it underperforms on the groups that have been found underperform.
Outputs. In addition to SMART outputting subgroups/slices or a scalar number, it can output a model report. An example report is provided in D.100. We note that this report is simply example which employs both the findings of the fitting procedure and an additional LLM to summarize the outputs. More fine-grained outputs can be constructed.
B.6 Moving outside of IID data
In the paper, we propose that SMART can move outside of IID data and generalize better in the presence of covariate shift (refer Sec. D.2). We highlight that this is done by performing analysis on the original data and using an LLM propose possible hypotheses and splits for a different target domain where data is present (hence, operationalizing only using the LLM with access to previous operationalizations) The following is an example prompt which is designed to do this.
You have access the following information.
Dataset information: (context), (context_target)
The dataset contains (len (unique, (uniqueevaluuss)) columns. The columns C, ".join (unique_valll values keys())) The values {str( unique_ values. val.eesiitess()) it)ms
However, you no longer working with the same data just described. Rather, this is the context: (nev_context).
10
These are the hypotheses: (self. hedd.uudddeeed.potheeee.
11
TASK: Propose specific variable ranges for each hypothesis such that they are clearly operationalizable and defined. Use this format: Hypothesis: <>; Operationalizattion: <>.
13
Code Listing 3: Operationalization (LLM knowledge): General template
B.7 SMART and multiple testing
The reason why SMART performs testing via hypothesis generation is because testing for all slices is equivalent to generating and testing hypothesis the data. Here, we outline in greater detail how hypothesis generation connected multiple testing.
As outlined the main manuscript, searching for f failures may bring up the challenge multiple hypothesis testing. Specifically, when we evaluate the failure rate of model / across different slices
S, C D, we are testing the null hypothesis ((() ..s. = pp against the alternative H1() jus. * μpp Then, the probability of making Type error increases with each test. This drastically inflates the family-wise error rate (FWER). For instance, assuming that each slice is independent, the probability of making number of slices tested. While this can be addressed by adjusting for multiple testing, we run into the trade-off between the FWER control and statistical power. As employ statistical correction methods to account for Type errors, we increase the probability of Type II errors.
more Type I errors across all tests is given by 1 - (1 a)",, where is the total
22

---

C Benchmarks & Experimental Details
We summarize all experimental details, datasets and benchmarks.
Code can be found at: https://gitthb.. com/ github. httpa///aaaeeeeaaaaaaaa//AAATA1.
cww/paul palplllluuarr wawaaaabbb/MMARTTTeaaa or https://
C.1 Datasets
We summarize the different datasets we use in this paper in Table 9, The datasets vary in number of samples, number of features and domain.
Table 9: Summary of the datasets used.
Name
samples
features
Domain
Task
Loan
5000
8
Finance
Loan default
Breast
116
8
Healthcaree/Mdiceee
Breast cancer
Compas [33]
4743
8
Criminal justice
Criminal recidivism
OULAD [34]
5000
8
Education
Exam score
Diabetes
768
8
Hcalthcare/Medicinn
Diabetes label
SEER [35]
2000
12
Healthcare/MMdicineee
Prostate cancer morality
CUTRACT [35]
20000
12
Healthcare/Mediiinne
Prostate cancer morality
Note that, depending on the experiment, we might vary the sample size or features used. This can be because of the added synthetic variables or certain slice discovery algorithms taking too long run.
C.2 Baselines
We compare SMART testing with the following data-only baselines Section 5,
Autostrat [14]. We use the implementation from
Pysubgroup [21]. We use implementation from
Divexplorer [15]. We use the implementation from
Slicefinder [12]. We use the implementation from
Sliceline [13]. We use the implementation from
C.3
LLM prompts
C.3.1 Generic templates.
Construutt the task description for the LLM."HI
task your task is to propose possible hypotheses to which subgroups within the dataset might have worse predictive societal bias the
performance than on average because dataset, insufficient data, other relationships, or others. The subgroups might be based any of the provided characteristica, well any combination of such characteristics.
Dataset information: (context). (context_targes
The dataset contains (len (unique (unique_values)) values)) columns. The columns are O jojoinnniqueee... values. k)))
23

---

Task:
Create {n} hypotheses to which subgroups within the dataset than average because of societal
the model will perform Important: Your hypothesis can contain biases or other reasons. either one variable or two variables in the condition. Therefore, your goal is to find discrepancies in the model's performance, not the underlying data outcomes. Justify why you think that for each the {n} hypotheses. Format of the output: Hypothesis: <>; Justification: <>.
10
Code Listing 4: Generic hypothesis prompt
The following hypotheses about which people within dataset the model might underperform on. Propose specific ranges for each hypothesis. Hypotheses: (hypotheses).
TASK:
return dictionary that contains inder number the key and the column value the value. If there multiple columns in that hypothesis, return them in a list. There are the column names : C', '.join (unique_values keys
Code Listing Generic operationalization prompt
C.3.2 Example prompts: OULAD Education.
INPUT TEXT
Your task is to propose possible hypotheses as to which subgroups within the dataset might have worse predictive performance than average because of societal bias in the dataset, insufficient data, other relationships, or others. The subgroups might based on any of the provided characteristics, as well as on any combination of auch characteristics.
Dataset information:
Open University Learning Analytics Dataset (OULAD) contains data about courses, students and their interactions with Virtual Learning Environment (VLE) for seven selected courses (called modules). Presentations of courses start in February and October they marked by B and J respectively. The dataset consists tables connected using unique identifiers. All tables are stored the cav format.
The task is to predict the students who are likely to fail withdraw from course.
10
The dataset contains 14 columns. The columns are gender, region, edghestt iad_ age_ num num_of_prewt pun_of_prrev_ttemmttr studied_credits disability, test, group_0, group_1, group_2, group_3, group_4. The values are dict_items ([('gender', [MM, 'F "]), ('region') ['South Region', 'Scotland', Yorkshire Region', West Midlands Region', 'East Anglian Region', 'East Midlands Region', 'South East Region', 'Wales', 'North Western Region', London Region', South West Region', 'Ireland', 'North Region']), Slaghest__ducattion
['over Than Level', "A Level or Equivalent', 'HE qualification', 'No Formal quals', 'Post Graduate qualification ('imd_band', [770-0%', +40-50% '0-10%' '10-20', '50-60%> '20-30%', '60-70%', '80-90%' 190-100%', +30-40%,, nan)), ('age_band', ['0-35', 135-55', >55<=!)), ( [0, 1, 2, 3, 5, 4, 6)), ('studied_credits ('min': 30, 'mean': 79. 79.77122804326149.
'max": 655)), ('disability
24

---

;, UN,, "y"),, ('test', [False, Truel), ('group_0', [False, True
1), ('group_ _1, [True, Falsel), ('group_2', [True, Falsel), (* group_3', [False, Truel), ('group_4', [False, True])])
12
13 Task: Create 5 hypotheses as to which subgroups within the dataset the model will perform worse than average because of societal biases other reasons. Therefore, your goal to find discrepancies in the model's performance, not the underlying data outcomes. Justify why you think that. Format of the output: Hypothesis: <>; Justification: <>.
15
OUTPUT TEXT 17 Hypothesis 1: Students with disabilities will have worse predictive
16
performance than average.
Justification: Students with disabilities may face additional
challenges in their learning process, which could affect their performance in the course. They may require additional support or accommodations that are not adequately captured in the dataset, leading lower predictive performance for this subgroup.
19
20 Hypothesis 2: Students from lower socioeconomic backgrounds (indicated by ind_ band) will have worse predictive performance than average. Justification: Students from loser sociceconomic backgrounds may have limited access to resources and support systems that necessary for successful completion of the course. This lack resources and support may not be adequately captured in the dataset, leading to lover predictive performance for this subgroup.
23 Hypothesis Students with lower levels of education (indicated by highest_sducation) will have worse predictive performance than average.
Justification: Students with lower levels of education may have less prior knowledge and skills that are necessary for success the course. This lack foundational knowledge may not adequately captured in the dataset, leading to lower predictive performance for this subgroup.
Hypothesis 4: Studenta who have attempted the apterptee
multiple times indicated by num.of_prev.attemptt will have worse predictive performance than average.
Justification. Students who have attempted the course multiple times may have encountered difficulties or challenges in their previous attempts that hindered their success. These challenges may not adequately captured in the dataset, leading to lower predictive performance for this subgroup.
certain regions (indicated by region) will have worse predictive performance than average. 30 Justification: Students in certain regions may have different educational systems, resources, and support networks that can impact their performance may be adequately captured in the dataset, leading lower predictive performance for subgroups based on region.
Hypothesis 5: Students
the
These regional differences
Code Listing 6: Hypothesis generation: OULAD Dataset
BBB
INPUT TEXT
The following hypotheses about which people within dataset the model might underperform on. Propose specific ranges for each hypothesis. Hypotheses: Hypothesis 1: Students with disabilities will have worse predictive performance than average.
25

---

Justification: Students with disabilities may face additional
challenges in their learning process, which could affect their performance in the course. They may require additional support or accommodations that are not adequately captured in the dataset, leading to lower predictive performance for this subgroup.
Hypothesis 2: Studenta from lower socioeconomic backgrounds (indicated by ind_band) will have worse predictive performance than average. 10 Justification: Students from lower socioeconomic backgrounds may have limited access to resources and support systems that necessary for successful completion of the course. This lack resources and support may not be adequately captured in the dataset, leading to lover predictive performance for this subgroup.
12 Hypothesis 3: Students with lower levels of education (indicated by highest_education) will have worse predictive performance than average.
13 Justification: Students with lower levels of education may have less prior knowledge and skills that are necessary for success in the course. This lack foundational knowledge may not adequately captured in the dataset, leading to lower predictive performance for this subgroup.
Hypothesis 4: Students who have attempted the course multiple times indicated by nun.of_prev_ateeptaa will have worse predictive performance than average.
16 Justification: Students who have attempted the course multiple times may have encountered difficulties attempts that hindered their success. These challenges may not adequately captured in the dataset, leading to lower predictive performance for this subgroup.
challenges in their previous
17
Hypothesis 5: Students in certain regions (indicated by region) will have worse predictive performance than average.
19 Justification: Students in certain regions may have different educational systema, resources, and support networks that can impact their performance in the course. These regional differences may be adequately captured in the dataset, leading lower predictive performance for subgroups based on region..
20
Dataset information:
Open University Learning Analytics Dataset (OULAD) contains data about courses, students and their interactions with Virtual Learning Environment (VLE) for seven selected courses (called modules). Presentations of courses start in February and October they are marked by B and J respectively. The dataset consists of tables connected using unique identifiers. All tables stored in the cav format.
23
24 The task is to predict the students who are likely to fail or withdraw from course.
25
26
27
The dataset contains 14 columns. The columns are gender, region, highest_education ind_band, age_band, r_m_of_prev.attempta studied_credits, disability, test, group_0, group_1, group_2, group_3, group_4. The values are dict_items ([('gender', [M,, oF ')), ('region', ['South Region', 'Scotland', 'Yokkshire Region', West Midlands Region', 'East Anglian Region', 'East Midlands Region', 'oouth East Region', 'Wales', 'North Western Region', London Region', 'oouth Weat Region', 'Ireland', 'North Region ("highest_educcae education.,, ['Lover Than Level', 'Á Level or Equivalent', 'HE Qualification', "No Formal quals', 'Post Graduate Qualification ))), ( ('ndd' [770-80%,, 240-50%, '0-10%
26

---

'10-20', '50-60% '20-30%', 160-70%' 180-9%%' 190-100% +30-40%', nanl), ('age_band', ['0-35', 135-55', >55<=']), ( pun_of_prev_attempte
[0, 1, 2, 3, 5, 4, 6]), ('stadie__creditss. ('min': 30, 'mean': 19.77122804321499
'max': 655)), ('disability P'N, 'y"),, ('test', [False, Truel), ('group_0', [False, True 1), ('group_13 [True, Falsel), ('group_2', [True, Falsel), (* group_3', [False, Truel), ('group_4', [Falae, True])])
28
29
TASK: Propose apecific variable ranges for each hypothesis such that they are clearly operationalizable and defined. Use this format: Hypothesis: Operationalization: <>,
30 w..
31
32
OUTPUT TEXT Hypothesis 1: Students with disabilities will have worse predictive performance than average.
Operationalization:: Students with disabilities can identified by the "disability" variable in the dataset. To test this hypothesia, we can define the range for students with disabilities as those who have value of "Y" in the "disability" column.
34
35
36 Hypothesis 2: Students from lower socioeconomic backgrounds (indicated by ind bad_)and) will have worse predictive performance than average. Operationalization: Socioeconomic background can be measured using the " "imd_band" variable in the dataset. To test this hypothesis, can define the range for students from lower socioeconomic backgrounds as those who have an "ind_band" value in the range of "0-30%".
39 Hypothesis 3: Students with lower levels of education (indicated by highest_education will have worse predictive performance than average Operationalization: Education level can be determined using the highest_oducation variable in the dataset. To test this hypothesis, can define the range for students with lower levels of education as those who have a value of "Lover Than A Level" or "No Formal quals" in the "highest__ducation"
column.
Hypothesis 4: Students who have attempted the indicated by num.of_prev.sttemptt) will have worse predictive performance than average
multiple times
Operationalization: Previous attempts can be measured using the variable in the dataset. To test this hypothesis, can define the range for students who have attempted the course multiple times as those who have a value column.
wmm_of_prev_attemee
greater than in the "nun.of._rev_attem
Hypothesis 5: Studenta in certain regions (indicated by region) will have worse predictive performance than average. 16 Operationalization Region can be determined using the "region"
variable in the dataset. To test this hypothesis, we can define the range for students in certain regions those who belong the regions of "North Region" "Wales".
Code Listing Operationalization (LLM knowledge): OULAD Dataset
INPUT TEXT
The following groups that defined based on the dataset.
Convert them into Python dictionary format. Each group should represented as a key-value pair in the dictionary, where the key is an index (0 to 4), and the value is a string representing the group using Python syntax and logical operators. For multiple
27

---

conditions, Python's logical 'and? ('&2)) 'or' (II)).
Ensure the format is valid Python dictionary.
Examples:
Single Condition: (0: XX > 45
Multiple Conditions: (1: *X > 45) and (Y 20)
10 Groups to summarize: Hypothesis Students with disabilities will have worse predictive performance than average.
Operationalization:: Students with disabilities can be identified by the "disability" variable in the dataset. To test this hypothesis, we can define the range for students with disabilities as those who have a value of "y" in the "disability" column.
13
Hypothesis 2: Students from lower socioeconomic backgrounds (indicated by ind_band) will have worse predictive performance than average. 15 Operationalization: Socioecononic background can be measured using the " "ind_band" variable in the dataset. To test this hypothesis, we define the range for students from lower socioeconomic backgrounds those who have an "ind_band" value in the range "0-30%".
16
Hypothesis 3: Students with lower levels of education (indicated by bighest_education) will have average.
predictive performance than
18 Operationalization: Education level can be determined using the hi
variable in the dataset. To test this hypothesis, can define the range for students with lower levels education those who have value "Lower Than Level " or "No Formal quals" the "highest_.duucation column.
19
Hypothesis 4: Students who have attempted the course multiple times ( indicated by mu..o..prev_ttemmp))) will have performance than average.
predictive
21
Operationalizatioon Previous attempts can be measured using the num of pr.._fpreeaateettee variable in the dataset. To test this hypothesis, can define the range for students who have attempted the course multiple times those who have value column.
greater than in the "num_of aum_of..prev_atemmpst
22
Hypothesis 5: Students in certain regions (indicated by region) will have worse predictive performance than average.
24 Operationalization: Region can be determined using the "region" variable in the dataset. To test this hypothesis, we can define the range for students in certain regions as those who belong to the regions of "North Region" " "Wales".
Column names: gender, region, digheat__ducationn ofm_of oum_of__rev__teee pre.....eeeeteettteeeeeeeee group_1, group_2, group_3, group_4
imd_band, age_band, studied_credite disability, test, group_0,
26
Column values: dict_items ([(''enddrr)))"')) [MM, "F*)), ('region', ('Suuth Region', 'Scotland', 'Yoksshire Region', 'West Midlands Region', East Anglian Region', 'East Kidlands Region', 'South East Region', 'Waes'' 'North Western Region', 'London Region', 'oouth West Region', 'Ireland', 'North Region Lover Than A Level', 'A Level Equivalent', 'HE Qualification', oNo Formal quals', 'Post Graduate Qualification )]), ('idd_band', ['70-80%' '40-50%' '0-10%' '10-20', '50-60%' '20-30%' '60-70%', '80-90%', 190-1007', '30-40%' nan)), C'ag_bband" [0, 1, 2, 3, 5, 4, 6)), ( ('sdits,, ('min': 30, 'mean':
saighest_education' [
[*0-35*, 335-55' '55<=?]), ('nom_of.provattoopp))
79.77122804326149. 'max': 655)), ('disability', DP, "Y'),, ( test', [False, Truel), ('group_0', [False, Truel), ('group_1', [ True, Falsel), ('group_22 2', [True, Falsel), ('group_3', [False, True)), ('group_4", [False, Truel)])
28

---

29
OUTPUT TEXT
30
31
0: 'disability = "Y",,
'imd_band in ["0-30%"]',
"dighest__ducatton in ["Lower Than A Level", "No Formal quals"]
34
3: ofum_of_prev_ateen > 1',
35
4: 'region in ["North Region", "Wales"]"
Code Listing 8: Interpreter: OULAD Dataset
C.4
Experimental details
We note that all experiments were performed using two compute resources: a with NVIDIA RTX A4000 GPU and 18-Core Intel Core 19-10980XE, as well as an Apple MI Pro 32GB RAM. We exemplify SMART Testing using GPT-4 [29] as the LLM but run further experiments to the sensitivity to the type of language model in Appendix D.3.
C.4.1
Context-aware testing (Sec. 5.1.)
Goal. We aim underscore the role of context in MI model testing to prevent false positives, especially when dealing with tabular data where data may contain many irrelevant or uninformative features [30], persisting even post-feature selection [31, 32]. We contrast SMART which explicitly accounts for context, in contrast to data-only approaches which are context-unaware only operating on the data.
Setup. We fit a predictive model to the training dataset, varying the number of irrelevant, synthetically generated features contained in the dataset where irrelevant features drawn from different distributions. We then quantify the proportion of conditions the identified slices that falsely include the irrelevant synthetic synthetically features.
Because different methods are sensitive to different types of irrelevant features, we developed data generating processes that encompasses many types of variables. Over many runs, different data-only methods pick up on some of these variables, showcasing that all methods are susceptible to randomly sampled irrelevant features in the dataset.
Sampling mechanism. To evaluate the impact of irrelevant features, we enrich the dataset by adding synthetic categorical variables. The number of new variables is equal to the number of existing features the dataset. For each new variable 24, we determine its type by sampling from a Bernoulli distribution with probability 0.5. If the sampled value is 0, is Bernoulli variable with success probability 0.1; otherwise, z; a categorical variable with four categories, following a predefined probability distribution (e.g.. (0.1, 0.3, 0.4, 0.2))):
Type(x;) (0r5o
(ernoulli(O.1)
if Tx:) = 0, if Type(z.) 1.
24 C, 0.4
We that there are synthetic data generating processes that completely break other data-only methods. As an example, creating unique ID column for each sample breaks the Autostrat algorithm. all the subgroups/slices identified the unique IDs. The data generating process employed in our experiment reflects broad variety of commonly encountered DGPs.
C.4.2 Requirements.constrainel testing (Sec. 5.2.)
We whether each of the methods fulfil three requirements.
The first requirement involved the use of the variable "age" in each detected slice. This was passed input to SMART. The other methods do not accept context input and, therefore, it was not
29

---

possible to fulfil these requirements. The numbers provided for other methods are simply how often they fulfilled the requirements by chance.
The second and third requirements involved obtaining a minimum and maximum sample size. This input SMART and the ablated SMART version. Based this, SMART changed its hyperparameter within its function which asks to indicate minimum and maximum sample size for the discovered slices, As with the previous experiments, this was adjusted for the other groups because they do not take textual input.
was passed
C.4.3 Targeting model failures (Sec. 5.3.)
In order to evaluate the targeting of model failures, we try four different tabular models with pre- specified hyperparameters. We find discrepant slices on the training dataset and evaluate them on the testing dataset.
C.4.4 Adaptive testing for deployment environment (Sec. 5.4.)
The goal of the adaptive testing experiment is to understand the extent to which SMART, as well as other data-only methods, can use data in a source domain to generalize to new, target domain where a covariate shift has been detected. To this end, the datasets provided have a known covariate shift and can be evaluated.
Each method was trained on the UK dataset and the discovered slices were evaluated on the US dataset. No additional context provided to data-only methods since they do not accept any text context as inputs.
In contrast, we have provided SMART with the previously discovered slices and hypotheses, and have asked to re-evaluate these hypotheses the context of the US dataset. Specifically. SMART re-hypothesized possible model failures for the US market but used the UK data to operationalize the variables. The ablated version, SM ARTN SF. achieved the best overall performance on the US market. The ablated version (i) did not have access to the UK data and the failures of the models; and (ii) operationalized each covariate using the LLM alone (refer to Sec. B.2 for a discussion on operationalizations with different SMART versions). This provides evidence that, in the presence of covariate shift, using inductive knowledge or domain expertise might be more useful to finding meaningful model failures.
C.4.5 Discovery of societally important groups and failure understanding (Sec. 5.5.)
As discussed in the main paper, SMART provides both possible hypotheses and justifications for model failures which can be evaluated using simple "fit" method. Furthermore, SMART prioritizes meaningful data slices which are of societal importance. Such slices can be inspected for any data input.
30

---

D Additional experiments
D.1 Requiremeent--costtriindd testing
Goal. Requirements are a crucial, yet neglected part of ML model testing, such as verifying performance on societally relevant dimensions or verifying specific aspects meet compliance requirements. However, no previous testing framework has incorporated the notion of satisfying requirements when defining the test slices. This experiment illustrates how SMART integrates requirements provided in natural language, which then influences the hypotheses generated to satisfy testing requirements.
Setup. We cover three real-world requirements that end-users might have: one based on demographics and two based sample size. Requirement I: Each of the top 10 identified unique slices should involve the age of a person. Requirement 2: The sample size of the top 10 identified unique slices must have at least 150 observations. Requirement The sample size the top 10 identified unique slices has to be small, within 10 and 150 observations. We exemplify the experiment using a real-world prostate cancer dataset from the UK [35], as healthcare often mandates certain testing requirements (e.g. Collins et al. [48]).
Analysis. Table 10 shows that SMART which directly integrates requirements (via natural language), satisfies the requirements greater number of times compared to data-only methods, which only satisfies requirements by chance (hence the low number of times). Beyond satisfying requirements, the SMART slices also represent model failures that almost always have statistically significant performance differences from average when evaluated test data. Finally, while S.M ART (ablation without celf-falsification) can satisfy requirements, the number of statistically significant slices lower than SMART, thus underscoring the value of our self-falsification mechanism. That said, SM ASF
still outperforms data-only baselines.
Table 10: Requirement satisfaction showing how many times the top 10 generated slices satisfied the requirements (Req) and how many of these slices had statistically significantly different performance from average (Sig) on a testing dataset. Maximum is 10. is better.
R1: Age
R2: Min sample size
R3: Max sample size
Req
Sig
Req
Sig
Req
Sig
Autostrat
0,00 + 0.00
0.00
+ 0.00
1.00
+ 0.00
1.00
+ 0.00
0.00 + 0.00
0.00 + 0.00
PSG_B _
2.00 + 0:00
2.00
0.00
2.00
0.00
2.00
+ 0.00
0.00 : 0.00
0.00 = 0.00
PSG
2.00 + 0.00
2.00
0.00
2.00
0.00 0.00
2.00
+ 0.00
0
0.00 = 0.00
Divexplorer
3.20 2.11
0.95
1.56
0.00 0.00
0.00
+ 0.00
2.70
2.57
1.05 - 1.75
Slicefinder
4.50 + 1.72
1.75
0.99
4.55 + 1.28
3.05 + 0.80
3.20 1.33
1.10 0.94
Sliceline
1.20 + 0.40
1.20
+ 0.40
1.35 + 0.48
1,35 + 0.48
0.15 + 0.48
0.15 - 0.48
SMART NSF
9.90 + 0.30
5.65
0.65
6.00
0.00
6.00
+ 0.00
4.15 = 0.65
2.60 0.86
SMART
9.70
+ 0.56
9.50 1 0.59
9.85 1 0.36
9.80 1 0.51
8.15 1 1.11
6.10 + 1.70
Takeaway 2. SMART, unlike data-only methods, identifies slices that have significant performance differences, whilst also satisfying requirements important dimension not even considered by previous testing methods.
31

---

D.2 Adaptive testing for a deployment environment
Goal. Deploying an ML model often entails going beyond IID, such as a different deployment environment. We consider the case of deploying a model different country where there is covariate shift 12 and evaluate the capabilities of testing frameworks to adapt across the different environment and identify model failures.
Setup. We use real-world prostate cancer datasets from different country's cancer registries with known distribution shifts: SEER (US) [11] and CUTRACT (UK) [35]. We train predictor f on UK data, while our target deployment environment is the US.
Identifying model failures. Table 11 shows that SMART better tests models at deployment time using the information provided. SMART identifies a much greater number of statistically significant model failures (almost all possible), both within the same environment (UK) and when shifting to a different one (US), even after adjusting for multiple comparisons using Bonferroni correction.
Analysis.
Table 11:
Number of slices identified (out maximum of 10) that had significantly divergent performance from average (higher is better). S_a counts the number of significantly divergent groups
Sample size sensitivity. We also assess sensitivity to sample size, see Fig. Both variants of SMART shown consistently outperform data-only counterparts in identifying much greater number of significant model failures. Within domain (UK):
0.05; S a/n applies the Bonferroni correction. is better.
D
D
Sa
S
S.
S
S.
S
expected, we find that for lower sample sizes, SM SR (without the celf-falsification mechanism) is su- perior, however, given enough data we then find that SMART benefits from the self-falsification.
PSG
2,00 0.00
0.00
2.00
0.00
000
2.00
0.00
0.00
2.00
0.00
1.75
0.43
1.50
0.50
PSG
2,00 1: 0.00
2.00
1 0.00
2.00
1
0.00
2,00
1
0.00
1.75
1 0.43
1.45
0.50
Divesploeer
1,05
1.58
0.45
0.9
2,00
2.28
0,3
1.33
3.90
+ 2.7
2.75
2.21
Slicetineer
0.96
0.46
Slediine
1.00
SMART
5.30
0
SMART
9.00
0.49
9.25
0.54
9.45
0.50
8,85
0.57
8,85
0.79
8.35
0.79
Deployment environment (US): we find that self-falsification similarly requires sufficient samples; which we note expected behavior. Interestingly, in the deployment setting (US), SM SMARTNSE SSF generally identifies the greatest number of significant failures (almost all possible) across different sample sizes. This suggests that under covariate shift, using inductive knowledge (via the LLM) or domain expertise might be more useful to find meaningful model failures.
UK Test Results
US Results
slices discovered
10.0
10.0
7.5
7.5
5.0
5.0
2.5
2.5
0.0
0.0
10
20
50
100
200
500
10
20
50
100
200
500
Sample Size
Sample Size
Autostrat
PSG A
Slicefinder Sliceline
SMART_NS NSF
PSG_B
Divexplorer
SMART
Figure 8: Number of significant groups discovered (out of total of 10) based on the training dataset size. SMART can operate under any sample size; self- celf-falsification mechanism requires a larger sample size falsify hypotheses. better.
Overall, the results highlight the flexibility of SMART to handle different scenarios and sample sizes. From a practical perspective, while both SMART variants outperform data-only methods, the implication is that there is nuance in using different SMART variants for different scenarios.
Takeaway 4, SMART identifies significant divergent failure slices in deployment setting, outperforming data-only methods across environments and sample sizes.
((X) changes, while pXY[X) remains the same
32

---

D.3 Effects of LLMs
In this section, we provide more experimental details which compare the effectiveness of two GPT models, GPT3.5 and GPT4. We highlight that the goal is not to exhaustively test the framework with every LLM. Rather, the goal is to showcase that SMART is feasible with at least the capabiliitess of GPT-4. We provide this section as way to measure the sensitivity of the model's performance with lower LLMs but highlight that we do not recommend using it with smaller LLMs, especially LLMs with fewer than 7B parameters.
D.3.1 Comparison over identified divergent slices
The following table reproduces the experiment from Sec. D.2 by directly comparing two models GPT3.5 and GPT4. The setup is the same as the original experiment.
Table 12: Number of slices identified (out of maximum of 10) that had significantly divergent performance from average (higher is better). Sa counts the number of significantly divergent groups 0.05; Sa/n applies the Bonferroni correction.
DLR
DeN
D
5a/e
8.fn
5./4
Autostrat
1.00 =: 0.00
1.00 :: 0.00
1,00 =: 0.00
1,00 = 0.00
0.95 = 0.22
0.95
0.22
PSGP
2.00 + 0.000
2,00 0.00
2,00 = 0.000
2,00 0.80
1.75 + 0.44
1.50
01.50
2,00
Divesplorer
1.65 1.88
0.45: 0.92
2,00 2.28
0.5 1.53
3.90 - 2.57
2.75
2:21
Slicelinder
3.65 + 0.96
2.75 0.62
3.85
1.11
2,70 0.46
6.80 + 1.83
5.95
1.02
Slaceline
1,00 2: 0.00
1,00 3: 0.00
1,35 0:48
1.35 0:48
1,35 0.48
1.35 2 048
NSF
8.00
SMART_GPTS
9.60 1: 0.49
9.25 0.54
9.45 0.50
8,85 0.57
8.85 0.79
8.35 0.79
SMART MARR__SF GPP35
8.20 = 0.60
7.15 0.65
8,20 = 0.75
6.95 0.67
9.25 = 0.62
8.25 0:43
SM GPT35
10.00
0.00
0.43
The table provides a measure of the model's performance on the training dataset from the same environment (DUK) ), the testing dataset from the same environment (DUK), ) and a different deployment train: environment (Des).
Takeaway. Both GPT3.5 and GPT4 provide strong increases over benchmark methods with little variability between the two LLMs. One of the possible reasons why that the hypothesis space of possible model failures is somewhat limited. This can be seen by the similar hypotheses that are generated by both GPT models.
D.3.2 Performance across different models
In this section, we vary different tabular machine learning model types and identify how well the ablated and original SMART, identified with GPT3.5 and GPT4 identify slices with large performance discrepancies.
Table 13:
The differences in accuracies between the top slice identified for each method on a testing dataset. The p-value computes the p-value associated with the difference in the accuracy. For the accuracy, higher values imply a greater ability to detect divergent slices (hence, higher is better). For the v-value, lower is better. Averages +- standard deviations are shown across 5 runs with random seeds and data splits
Logistic Regression
SVM
XGBoost
Multi-laer Percoption
Accc
p-value
Accc
p-value
Aoce
p-value
[cc
p-value
SMART NSF GPT3.5
0.23 + 0.3
0.00 0.00
0.23 0:03
0.00 0.00
0.12 + 0.6
014 0.27
0.23 # 0.03
0.00 0.30
SMART GPT33
+
047
$ 004
0.07
0.00
SMARI_NSP GPT4
0.10 4 00l
0.00 0.00
0.10
0:01
0,00
0.00
0.8 4 duu
025 4 0.42
0.10
4: 0.01
0,00 0,00
SMART.GFY GPT4
0.40 1 6.02
0.00 0.00
0.40
0.02
0.00 0.00
0.29 1: 0.06
0.00 : 0.00
0.40 1: 0.02
0.00 2: 0.00
SMART with deep learning models. SMART's targeted sampling of hypotheses, is entirely independent of the downstream model used, i.e. SMART's context-guided slice sampling mechanism is used to generate hypotheses independently of the downstream model.
33

---

We extend our analysis with Logistic Regression, SVM. XGBoost, and MLP to further include two tabular deep learning method: TabPFN and TabNet. As shown in Table 14, across all models SMART is the best at finding subgroups where the models are least reliable.
Table 14: Identifying slices with the highest performance discrepancies. We show differences in accuracies (oA.)ce)) between the top identified divergent slice and average performance across two offe-of-the-ae deep learning classifiers (over runs) on the SEER dataset. 1 is better. 0.00 implies the evaluation method does not support model.
Evaluation Method
Classifier
Autostrat
PSG [
PSG_A
Divexplorer
Slicefinder
Sliceline
SMART
TubPFNClassifee
0.20 0.10
0.19 0.05
0.18
0.05
0.00
0.00
0.00 0.00
0.23 0.05
0.28 0.17
TabNet
0.04
0.04
0.04
0.08
Takeaway. GPT4 adds additional value when it comes to identifying slices with performance discrepancies. However, GPT3.5 is still able find significant performance discrepancies across divergent slices, competitive and in many cases exceeding the performance of other methods (refer to Sec. D.4).
D.3.3 Hypothesis generation
A key component which determines the quality of the SMART method is the ability to navigate the search space by generating hypotheses. Therefore, we overview the kinds of hypotheses which generated and how they differ based on the type of LLM used.
The table below provides example of the top 10 hypotheses and justifications generated by GPT3.5 and GPT4 for the prostate cancer example in the UK, where the goal is to find specific slices where the model is likely to fail, and propose justifications for them.
Key takeaway. Many of the hypotheses for both models are similar. The underlying difference is in the ordering and justification of the hypotheses.
34

---

Table 15: Comparison of Hypotheses and Justifications generated by GPT4 and GPT3,5 for a breast cancer prediction task. The hypothesis target likely model failures.
GPT38
Hypullois
Fustifiatioo
Hyssillesss
Jusiiiicttion
Parients with lower
The model might perform wome younger prosuie
Subgroup of parients with older (shove 80) will have predictive performunce due likelitond systems, leading higher mortality rates. Higher Gleosan sccees indicate more iggressive advanced prestute cancer, which is generally asso- cissed with poinr progrness and hightr moraaity mice.
Older is ofhen asseciated with increosed risk of comorbidties weaker iumune systems,, which can negeively impuat the progresi produtic see
common
oldkr mes, these night insufficiont for yourger patients.
Patients with higher commtiiliies
model might perform worse porients multile consotidttles us complesity of their health condition might noe be fully
Suhgroup of pationes with higher Gleason sccees or 5i have waese prodictive performance due to more aggnosive and advenood cincer, leading to higher mociality rales.
captared the medel.
Patients who under-
The model might perform wome for these pe- tiess as the etfectiveness of conservative man- agenent night vary groely among indiviannls and this weichillty night not be well coptarcal by the model.
Subgroup patients with stage 4 cancer will have wosse predictive pesformaace due to the advanced stnge of the discsoer indicating mehsttvis and lim- icca iecamment options.
Singe 4 prostate cancer chzracterized by the spread of cancer to distant ocpass, indicating a more sdvanced and aggnosive discase. The limited most- mest opsints soiilble for stng: centrr can scsult wone predictive performance.
went consevvative monsgrmmnt
Paticnts with lower prostute specilic arrigan lovels
The model night pertorm wane fox these pa- tiens lower PSA levels might be ussociated with corfy-ttage persosic cancer, which might be harder predict
comsehidiies odhen have com-- ples health conditios that cat inguxt the eftective- prodonn cancer incatment and monagement leading
Pations with
Subgroup of patients with multiple comoebdities (e.g. 3 4) will have wore predictive performance due the presence of other health comuiiiine that complicate the trestmem management
lack symptoms.
predictive performance.
Patients with higher Gllessan
model might pertiom for po- hightr Gleason indicate aggressive cances, which might progress us peedictably.
Subgroup of putiens who received conservativ printry insttment option have wore predictive performance to the ponce tial dela in receiving more aggressive treatments.
Conservue management involve wuchhrdl carveillnce,
mutagrment
wailng
delayed inination more aggressive neatments.
delay treammen may worse predictive performance.
Patients in siago 4
The model night perform wene thos pi- tients as stage 4 prostate cancer is advanced and might progress impredictably.
Primcry bermone theragy is often uscil control the growth of prostate cancer by reducing the levels of male hormones (indregens) Pocents who do noc receive this therapy may have uncontzolled cancer growth mathing in wene prodictive per perfomunes. Suhy of paicons dho not undergo nafical n-- diothragy will have worse predective periormance duc to the poscatial lack of localized carcer contral.
Sungroup of patients who not renrive primory
hornoze therapy will have worse predictive perfor monce due the potencal look of hormonal contool over the cancer grow th.
Patients who under- went radical prosta- lectonty
The medel night perform wenz for those go- nees as the cunoomes of ridical prostatectonyy night vayy greatly among individuals, and this variahilit might not well captored the model.
Radicai cadinthraapy is tncetmentt opricn that sime to destroy cancer cells in the prosione gland. Pa tiems who do not undergo this tecatment may have higher likelitund of localined canver progrosion. londing
prodictive per furmnnce.
Patients with lower Glesson
The model might perform wome for po ticsts
Radical prottzectomy involves surgical removal of prostate gland. Parean who undeggo
Subgroup
undergo radical jenstataaton have prodictive perfur- the potenial preseice residual cells the prostate gland.
who
indieste gressive canser, which might poogress prodict.
procedure may residual proviats which contribute wane predictiva performtncc.
Patients stage
model might perform wome these po- tienss stage I prostate cancer is cazly-atage and might be harder to prodict due lack of sympooms.
Subgroup patients higher prostate specific antigen (PSA) levels (ahove 50) will lave wune prodictiee performance due indication mon: advanced cancer.
Higher PSA advanced prosase cancer. Patienss hipher PSA leoels may have a woree progracis higher man- tality ries, leading to wone predictive performance. Subgroup of porients with a combitution older age (shove 70), higher Gleason scort or 5), and eags canarr will love the wont prodictive perfor- monce dun to the asmulative effect adhannal sys. agarescive cancer, and metastzsis.
are associaned more
10 Patients who under-
The model might perform wome for these po: ticns as the cuicomes of prinsey hommone therapy might vary greatly among individuals, this weishilit might mot be well caphered by the model.
The combination of older age, higher Gleason scores, and stage 4 cancer reperseuts a subgroup with mahhiple nepative progroatic factors, The as- miltaive efficct thoo factors is likely io moult in the work predictive performance.
went pimury hur-
mone cherapy
D.4 Effects of different tabular machine learning models
The primary task of the SMART method is evaluate a given, trained machine learning model. Thus far, we have been using a logistic regression model as the basis for evaluation in the main experiments However, the results are not sensitive to the type of the tabular model. Therefore, in this section, we provide additional experiments where we vary the tabular model for the task. Specifically, we consider the following models initialized with their default hyperparameters for evaluation: Logistic Regression, Support Vector Machines, Boosting (implemented with XGBoost) and a multi-layer perceptron with 2 hidden layers and RELU activation functions in the hidden layers.
Goal. The primary goal is to understand whether the framework generalizes to other models which operate under different mapping mechanisms (e.g. logistic regression, which is linear model, compared to a tree-based model).
Setup. Given that not all the slice discovery or model evaluation algorithms output multiple slices, we constrain the evaluation to only focus on a single slice which might have discrepant performance. We use the UK prostate cancer dataset and evaluate the discrepancy of the top identified slice relative to the average across all identified model types. The discrepancy is calculated as the absolute differences of the average performance between the two groups, as well as the p-value associated with the difference. The results show the average performance +- standard deviation of 5 random splits. A higher absolute difference indicates that the model fails on one of the slices more than average.
Discretizing the inputs. Many of the discovery methods, however, operate only on categorical data. The previously used dataset, however, has three continuous variables: age, possate--ppciffe antigen, and comorbidities. We therefore also assess the quality of these methods to discover slices on the testing dataset when these three variables are discretized into 10 bins each. The following
35

---

Table 16: The differences in accuracies between the top slice identified for each method on a testing dataset. The p-value computes the p-value associated with the difference the accuracy. For the accuracy, higher values imply a greater ability to detect divergent slices (hence, higher is better). For the p- p-value, lower is better. Averages +- standard deviations are shown across 5 runs with random seeds and data splits.
Logistic Regression
SVM
XGBoost
Multi layer Perceptron
|A4cc]
p-value
|A.co]]
p-value
AAoc]
p-value
[AAcc]]
p-value
Autostrat
0.24 + 0.02
0.00 + 0.00
0.24 + 0.02
0.00 0.00
0.09 + 0.09
0.33 = 0,46
0.24 + 0.02
0.00
pysubgroup, beam
0.23 + 0.01
0.00 - 0.00
0.23
0.00 0.00
0.11 = 0.07
0.18 0.40
0.23 - 0.01
0:00
pysubbrouup_e seeubrroupapriortt
0.23 + 0.01
0.00 0.00
0.20
0.00 0.00
0.11 0.07
0.19 0.42
0.23 0.01
0.00 0.00
Divexplorer
0.05 + 0.11
0.81 + 0.43
0.09
0.61 0.53
0.14 0.15
0.47=0.4
0.02 + 0.05
0.86 0.32
Slicefinder
0.01 +
0.00 1 0.00
0.01 0.00
0.00
0.00 0.00
0.00
0.01 0.01
0.00 0.00
Sliceline
0,26 1: 0.06
0.00 0.00
0.26 0.06
0.00 1 0.00
0.18 1 0.09
0,00 = 0.01
0.26 1 0.06
0.00 1 0.00
SM SMART NSF
0.17 0.01
0.00 0.00
0.01
0.00
0.09 = 0.05
0.07 0.16
0.17 = 0.01
0.00
SMART
0.37 1: 0.03
0.00
1 0.00
0.37 0.08
0.00 0.00
0.26 4: 0.06
0,00 = 0.00
0.37 + 0.03
0.00 1 0.00
table reports the performance of all the methods (note: we did not discretize the dataset for SMART because it can work natively continuous data).
Table 17:
The differences in accuracies between the top slice identified for each method on a testing dataset when the datasets continuous features are discretized.
Logistic Regression
SVM
XGBoost
Multi-layer Perceptron
AAcc
p-value
A Acel
p-value
Acc
p-value
|
p-value
Autostrat
0.02 0.02
0.38 0.27
0.02 : 0.02
0.45 + 0.37
0.02 0.02
0.52 0.32
0.02 0.02
0.43 : 0.33
PSG B
0.01 1 0.01
0.56 1: 0.29
0.01 1 0.01
0.62 1 0.38
0.01 = 0.01
0.55 1 0.25
0.01 1 0.01
0.57 1 0,23
PSG_A
0.01 0.01
0.55 0.29
0.01 +:
0.55 0.226
0.01 0.01
Divexplorer
0,10 + 0.08
0.47 + 0.28
0.11+0.10
0.41 + 0.36
0.13 0.11
0.38 1.38
0.13 -13=00110
0.38 + 0.37
Slicefinder
0.00 1: 0.00
0.00 1: 0.00
0.00 1: 0.00
0.00 1: 0.00
0.00 :: 0.00
0.00 0.00
0.00 1: 0.00
0.00 1: 0.00
Sliceline
0.09 0.01
0.05 0.06
0.08 + 0.02
0.14 + 0.09
0.11 0.07
0.25 0.24
0.09 0.01
0.06 + 0.08
ART NSF
0.17 0.01
0.00
1. 0.00
0.17 1: 0.01
0.00 1 0.00
0.09 1 0.05
0.07 0.16
0.17 1. 0.01
0.00 1: 0.00
SMART
0.37 0.03
0.00 0.00
0.37
+
0.26 0.06
0.00 0.00
0.37 0.03
Takeaway. SMART is able to consistently identify the greatest performing slices across a number of different tabular models.
D.5 On the inductive biases of MI. model testing
Goal. We further observe that data-only testing methods implicitly assume the existence of slices with discrepancies in performance. While indeed ML models do fail fail-it- it is equally as problematic
to highlight failures where there are none.
Setup. To evaluate this we propose a fully synthetic setup. Here, both the dependent vari- able (Y) and the independent variables (X) are sampled from a predefined random distribu- tion. Specifically, we predict loan default (Y' E {0, 1}) based on a set of independent variables X = (NNuss MMret, Arainfall F Footor, Pssason) which are conceptually and empirically independent of the outcome of interest, Ideally, if we account for context we should be able identify these disparate features should not influence the account and hence without prior relationships we should not flag spurious slices.
We compare the data-only methods to SMART under three different data generating processes (scenarios) that capture diverse underlying dynamics denoted as Suniform: Sakewed, and Sintenationss where each DGP has a focus on uniform, skewed, and interactive effects, respectively.
The first scenario is given by the variables sampled from the following DGPs:
36

---

Nuuns
Uniform {1, 1,499)
M pref
Uniform
Arainfal
Uniform (20000, 99999),
Foolrr
Uniform(1,6) 6):
Peasonn
Uniform (0, 3},
Y Uniform
The second scenario given by the variables sampled from the following DGPs:
Nunss Uniform (1, 1,499),
Mpret Binomial [1, 0.5),
rainfall Categorical 0.3, 0.4, 0.2),
F Feolor Binomial [1, 0.1),
P season
Binomial (1, 0.05).
Y Uniform (0, 1).
The third scenario given by the variables sampled from the following DGPs:
N Nruns
Uniform [1, 499).
Mpref Binomial(1 (1,0.),
Arninfall (0 0.4,
Amusic hap Mpret X Arainfall
Arn hap Nmns x Mpret,
Uniffrm((0,) 1}.
Analysis. Table 18 shows the number of slices spuriously discovered, while Table 19 outlines the number of conditions within the slices. We can clearly see the pitfalls of data-only approaches which detect slices which in reality have no relation one another often surfacing few conditions per group which suggests they arise by chance. The rationale for this failure is simply because data-only approaches do not and cannot reason about the features and/or understand context and simply aim to find slices with discrepancies in performance - which of course could arise by chance. In contrast, we see that SMART by virtue of awwreesss can avoid surfacing groups which in reality have no relationships.
Table 18: Number of discovered slices synthetic dataset with prior relationships in three data generating process scenarios. slices capped at most 20. Average of 50 runs standard deviations is shown.
Method
Suniform
S
Simeractions
Autostrat
1.00 .0
1.00 +
1.00 +
PSG_B B
20.00 + 0
20.00 + .0
20.00 + 0
PSG_A
20.00 + .0
20.00 .0
20.00 + _0
divexplorer
20.00 +
20.00 .
20.00 + .0
slicefinder
20.00 + .0
20.00 + .
20.00 + .0
SMART
0.00
0.00 .0
0,00
D.6 Context aware sensitivity
We provide an additional experiment where we vary the sample size the training dataset and observe how that affects the number of slices discovered for each method. We show that the SMART not affected by irrelevant features regardless of the sample size of the training dataset. The result shown in Figure 9,
37

---

Table 19: Number of conditions per discovered slice (false positives) in three data generating process scenarios. Average of 50 runs +- standard deviations is shown. Lower is better.
Method
uuifform
Sakewed
S interactions
Autostrat
2.17 0.46
1.73 + 1.01
1.13 + 0.35
PSG_B
1.03 = 0.18
1.90 0.71
1.27 + 0.45
PSG_A
1.03 = 0.18
1.90 = 0.71
1.27 + 0.45
divexplorer
2.10 = 0.31
2.87 = 0.68
2.40 + 0.50
slicefinder
1.40 = 0.50
1.50 = 0.57
1.00 + 0.00
SMART
0.00 0.00
0.00 0.00
0.00 + 0.00
Poreent indorait Tortrto fotures
Default
Diveapio lorst
Breast Cancer
Auccstre
PSG A
COMPAS
PSC
SMART
Education
nicefindrr
Diabetes
300
530
Semple
Sample
Sample
Figure 9: Proportion of irrelevant features (y) for each slice discovery method, based on the sample size. Lower is better.
D.7 Cost of SMART
We assess the cost of LLM hypothesis generation and scalability to larger datasets. Specifically, demonstrate not only that SMART is cheap but also easily scalable to large datasets
Scalability: SMART's scalability depends on the number of hypotheses generated. not dataset size (unlike data-only methods). This allows SMART to easily scale to arbitrarily large datasets.
In practical terms, cost then also scales primarily with the number of hypotheses generated, not dataset size. We provide rough estimate based on token counts of input and outputs for 2 datasets (SEER and OULAD) in Table 20. This would be less than 0.1 USD for 5 hypotheses and less than 0.5 USD for 100 hypotheses for state-of-the-art Table 1o:rodels-MMARR (USD) for different GPT LLMs and different numbers of hypotheses generated. The cost is estimated based on token counts. Note: GPT-4o models are post our paper and are even cheaper.
Cost Analysis:
Model
Cost SEER
Cost SEER
Cost OULAD
Cost SEER
Hypothesis (USD)
100 Hypolhesis (USD)
Hypothhsss (USD)
100 Hypethesis (USD)
GPT4
0.017
0.249
0.022
0.316
GPT-3.5
0.004
0.050
0,005
0.064
4PT-44 Mini
0.0003
0.005
0.0005
0.006
GPI-4o
0.008
0.125
0.011
0.158
D.8 SMART with open-weight models
SMART ideally should be used with the most capable LLM possible.Thar That said, we assess the differences in hypotheses between open-weight models and GPT-4.
We assess Mistral-7b, Qwen- Llama-3-8b. Llama-70b, where for the OULAD and SEER datasets we generate 5 hypotheses and assess overlap to the hypotheses generated by GPT-4. This is presented in Table 21.
To summarize, the overlap between open-source models and GPT-4 is between 60-80%. We find that open-source models propose similar hypotheses, but they are not replacements for more capable models. This highlights that less capable models might propose similar hypotheses, yet they still catch fewer model failures.
D.9 Understanding the importance of feature names
SMART uses the implicit context encoded in the interpretable feature names as source of contextual information to guide hypothesis generation. For instance, in medical dataset, features with names
38

---

Table 21:
Comparison hypotheses by GPT-4 and overlap w/ open-weight models
Datasel
Factors (GPT Hypotheses)
Mistral-7b
Llama 3-86
Qwen 1,3-7b
Lluma 70b
SEER Dataset Oulad Dataset
Disability
IMD hand
Age
Number of previous attempts Test (boolcan)
Oulad Overlap Percentage
60%
70%
60%
60%
Age
Prostato-apecffic antigen (PSA) Comorhidities
(sonservative manugement)
Cancer stage
SEER Overlap Percentage
like age, sex, or patient covariate features provide context to guide LLM hypothesis generation. This contrasts with data-only approaches which only use the numerical data values alone and ignore the context surrounding the feature names.
We aim to assess the sensitivity to interpretable feature names to provide guidance on the use of SMART. First, we perform qualitative study where we limit the data schema by hiding the feature names (such that they become uninformative) and inspect the hypotheses and justifications generated. We find that in the limited-schene schema case, SMART generates hypotheses based on inferences about the feature information (e.g. "the model might fail on feature_4 4 if feature_ represents gender") In contrast, informative names guide meaningful hypothesis generation. Such hypotheses and justifications illustrated in Table 22.
Second, we evaluate whether limiting the data schema by hiding some feature names and leaving minimal external context affects detection rates of model failures. We compare two versions of SMART, original and with corrupted feature labels, in identifying data slices with high performance discrepancies from average (Fig. 10. We find that across two real-world private datasets, hiding the feature names hinders model evaluation. This highlights that feature names play an important role in finding model failures.
These results highlight while SMART does not rely on any additional feature descriptions, feature names play an important role in finding model failures, just as any human requires interpretable feature names understand the data. That said, feature names (e-g. column labels such sex, age, race etc) are present in almost all tabular datasets both in the research field and in the real world where data is stored in SQL tables with column names.
Comparison of Base and Corrupt Values
CUTRACT
SEER
Difference in Acc.
0.06
Difference in Acc.
0.06
0.02
0.02
0.00
0.00
Base
Corrupt
Base
Corrupt
Figure 10: Identifying the importance of feature names source of information for context-aware testing. We compare two versions of SMART CUTRACT and SEER datasets. The first version, "basc", is the normal SMART method. The second version, "corrupt", has feature names changed to uninformative ones. The goal to identify slices with the highest performance discrepancies between the average & subgroup prediction, calculated as the difference in their respective accuracies (y-axis). We see showing feature names enables to find subgroups that have higher performance discrepancy from baseline performance than without feature names. better.
39

---

Table 22: Example hypotheses and justifications when dataset column names hidden. The hypotheses and justifications for features that not encode meaningful information lose practical meaning. Context-award aware testing benefits from rich feature names that can be used to guide meaningful hypothesis generation.
type
Hypothesis
Base
The model will perform worse on predicting the rist
Age is signncamt foctor many diseases, and older indisidaiis often have more complex bealt: situttonss with muligle comotiiiiies. This compleaity could make more dificclt for the model accunsely predict discasc risk.
of dicsse for older individuals.
model perfoens wons predicting of individuals with Gleason scoces.
The Cleason scree ar system used detemmise appres prstute cuncer, Lower acorcs prodec due
les aggresaive cancer, which may more
progression les obvious symptoons.
Comppadl
('1: [faatee_01) 0'))
fcsiane reproaants chanaciristic sach mex eiliicity, the mudel night perforn wone for corbin salgnmups dus uncicaal hines. For inatores if the cheeet hisual unnorh parlictbe
ciliiilt the medel's protictions
other
might
Comppoed
(22 (Teature 41))
teatura represents a chameneritic such as gender, the model might pertoers wome for centain sitgrorps due societal biases. For esumple, if the dutuset is blased bowards one gender, the model's prodictions for the other pender night be less accuniee.
D.10 Example of a model report
We show an example of a model report which is automatically generated by SMART. A model report includes a summary of key hypotheses, justifications, sources, and summary findings written report. A model report fully satisfies the requirements of the user.
In this case, the model is generated to directly match latex output requirements. It is provided in the shaded box below as part of the US prostate cancer (SEER) [11] evaluation.
A report on the performance of the model has been concluded. The following the hypotheses, their justifications tested on the model with their conclusions on whether the hypothesis was supported. Hypothesis
Justification
Operationalization
Hypothesis Supported
The model may per- form worse for older patients
Older patients may have morbidities and complex health sit- uations that are not fully captured by the dataset. Additionally, soci- etal biases may lead to less aggres- sive treatment options being pursued for older patients, which could affect the model's 's predictions.
age
Yes
The model may perform worse for patients with lower prostate-specific antigen levels The model may per- form worse for pa- tients who have un- dergone conservative management
Lower levels of prostate-specific antigen may be associated with car- lier stages of prostate cancer, which may be harder to predict due less data and less obvious symptoms. Conservative management is a less aggressive form of treatment, which may be chosen due to variety of factors not captured the dataset, such as patient preference other hcalth considcrations. This could introduce additional complexity into the model's 'ss predictions.
prostate_specck speciiffcccceeece fieeeiiffeeaae anttigen < 10 Yes
treatment_conee consecratteee
Yes
_management ==
The model may per- form worse for pa- tients with higher number of comorbidi- tics
Patients with more comorbidities complex health situ-
comorbidities
may have ations that not fully captured by the dataset. Additionally, these pa- tients may be more likely to die from causes other than prostate cancer, which could confuse the model's predictions.
The model may per- forn worse for pa- tients with a higher Gleason score
A higher Gleason score indicates more aggressive cancer, which may be harder to predict due to its rapid progression and the potential other health factors to influence out- comes.
gleason_sccme seceooo_sccoree.. ==
No
À further analysis identified slice. The following are two tables which summarize a part this information:
the training and testing datasets to understand the effects of each
40

---

group_Siee support
p_
num_criteria outcome_ diff
accuracy diff
HO
4915.00
0.31
0.00
1.00
0.23
0.05
H1
7749.00
0.48
0.00
1.00
0.26
0.01
1456.00
0.09
0.00
1.00
0.06
0.09
H3
574.00
0.04
0.94
1.00
0.00
H4
94.00
0.01
0.14
1.00
0,36
0.09
odds_ratio outcome
odds ratioaattoooooo acc
lift outcome
lict_acc
weighted relative__
weighted relative__eeeee acc
relgatee_relative_r
H0
0.82
1.39
1.46
0.94
0.07
-0.02
H1
0.96
1.12
0.48
0.99
-0.13
-0.01
0,99
1.11
0.89
0.01
-0.01
H3
0.85
1.00
-0.00
-0.00
H4
0.48
0.55
1.72
1.10
0.00
0.00
Recommendations:
1. The model appears be less reliable for older patients (age 75), patients with lower levels of prostate-specific antigen (< 10), and those who have undergone conservative management treatment. It also shows decreased performance for patients with more than two comorbidities and those with a Gleason score of 4. However, the model is more reliable when these conditions are not met.
2. Before deploying the model, the end user should be aware of the following: The model's performance may be compromised for older patients and those with multiple comorbidities. Consider additional validation or alternative models for these groups. Patients with lower prostate-specific antigen levels and those who have undergone conservative management treat- ment may also experience less accurate predictions. Additional clinical insights may be needed for these cases. Although the model shows decreased performance for patients with Gleason score of 4, this group is relatively small, the impact overall model performance may be limited. However, caution should be exercised when interpreting results for these patients. Remember, these recommendations are based on the training and test datasets from the UK. If deploying in a different geographical context, consider revalidating the model with local data.
Definitions of the metrics:
Group Size:
group_size |slice|
Support:
|lice
support
ddtaseet
Number of Criteria:
num_criteria Count("and") +
Outcome Difference:
outcome_diff wwrg__uucome_ddaaee rrs_ouuccme__sseee slice
Accuracy Difference:
accuracy_diff _atasst accuracy_slice
Odds Ratio (Outcome):
(1 P1)
odds hatttoouutoooeeeeeeaaaaaoaaoeeee
Po po)
Odds Ratio (Accuracy):
Pi (1 Pt)
odds ratio acc
po - po)
(where pr and Po are accuracies in the slice and the rest of the dataset, respectively)
41

---

Lift (Outcome):
lift outcome P1
p
Lift (Accuracy):
P1
lift_ acc
P
(where p1 is accuracy in the slice and p is accuracy in the entire dataset)
Weighted Relative Outcome:
weighted relative_ outcome support diff outcomes
Weighted Relative Accuracy:
weighted_relative__eo relative_accuracyy accuracy support diff accuracy
42

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract accurately reflects the claims made in the paper. Our paper introduces the new paradigm of context-aware testing which we discuss Sec. 3. We introduce our context-aware testing instantiation SMART in Sec. 4. We experimentally show how under variety different experimental conditions SMART outperforms data-only benchmarks in Sec. 5. We therefore believe the provided evidence fully supports the claims that context-aware testing is new alternative to testing ML models relative to data-only methods.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We discuss the limitations of our work in Sec. 6. We also provide ways of addressing these limitations in Sec. 5.4 with an extended discussion on possible biases, miti- gation strategies, and best practices, including how SMART provides unique opportunities to address such challenges with features such as model reports or transparent hypothesis testing and operationalization.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they seale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
43

---

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: The paper makes a claim that context can be used to guide the search for relevant and meaningful model failures via a context-guided slice sampling mechanism which uses context to prioritize likely slices. This is defined and discussed in Sec. 3.3.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can appear in either the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The paper provides the key experimental setups in the main experimental section (Sec. 5) together with additional experimental details in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
44

---

(c) If the contribution is a new model (e.g.. a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We provide details about the algorithms and data in Appendix C, Sec. B and Sec. D. Code can be found at: attps://githul com/paul http/paaliussraaa wwuuluusuauaa//aaa Testing or https://githh:: com cew.///aaneerrccaaaaaa///AATTT Testing
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.ct
www.iiiissiicccioiioooccc for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
/ /nips. wwtppcccpcccceeecccccccccccccclt httpi////uudee//euuuuuiiiiii cettSuummssioooooo...
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: All the details on the experiments are either provided within Section 5, with full details provided in Appendix D.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
45

---

Answer: [Yes]
Justification: Error bars (standard deviation) are included as relevant over multiple seeds for the experiments in Section 5 and additional experiments in Appendix D.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by emror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
All the compute details on the experiments are provided in Appendix C.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https :ttps:///eeurrr /wwp/eeuurpp.uee. wwwerrppsccppppeee htt.aiiieeecccscccieeeeeeeeee
Answer: [Yes]
Justification:
We have read the code of ethics do not violate any of the dimensions.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
46

---

Question:
Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Broader impacts of testing and implications outlined in Sec and 6. We discuss potential impacts and mitigations in Sec 5.4 and Sec
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Not applicable - our paper presents a new method for reliable and safe ML model testing.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (c.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Appendix provides details and citations for all assets (data and baselines) used in the paper.
47

---

Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
The paper does not produce new assets such datasets, but uses existing benchmarks/datasets.
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
We do not have crowdsourcing experiments or research with humans.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
48

---

Answer: [NA]
Justification: We do not have crowdsourcing experiments or research with humans that would need an IRB.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as institution conducting the review.
49


Reviews:
[{"review_id": "UNz5J8vMAy", "reviewer": "Reviewer_6t3M", "summary": "This paper introduces context-aware testing (CAT), which is a novel tabular model testing paradigm that uses context as an inductive bias to guide the search for meaningful model failures, and build a CAT system named SMART Testing. Detailedly, SMART includes four steps: (1) use an LLM to generate hypotheses of failures based on a combination of contextual information and dataset contextualization, (2) transform natural-language-based hypotheses into a form that can directly operate on the training dataset, (3) discard “incorrect” hypotheses by testing out each hypothesis through a self-falsification mechanism, and (4) automatically generate a report of the overall performance of the model under varying hypotheses. Evaluation shows that SMART is more robust than data-only methods in terms of false positives and false negatives, and can target model failures more accurately.", "questions": "- What is the cost of using GPT-4 to generate hypotheses in SMART and is it scalable to larger evaluation benchmarks?\n- Since the idea of SMART is general, is it generalizable to other domains apart from tabular testing, such as testing NLP models?", "limitations": "- It would be good to discuss the potential future directions of removing biases from SMART.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper explores an important and interesting research direction.\n- The improvements achieved by SMART shown in the evaluation are significant.", "weaknesses": "- There is a lack of systematic comparison between the practicality of testing subgroups found by SMART and other data-only methods. While the paper includes some examples of the reports generated by SMART, it would be better for authors to provide both qualitative and quantitative comparison between SMART and data-only methods in terms of the practicality of testing subgroups.\n- The tabular models included in the evaluation are outdated. In the paper, SMART testing is only applied to evaluate the performance of simple machine learning models such as Logistic Regression and SVM. It would be better to include more up-to-date deep learning models, such as transformer-based models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear R-6t3M,\n\nThank you for your thoughtful comments to improve the paper. We provide answers (A)-(F) & highlight updates to the paper.\n\n---\n## (A) Highlighting our systematic comparison of SMART vs data-only\n\nWe appreciate your suggestions and believe there may be a misunderstanding regarding the extent of our comparisons. We wish to clarify we provide in total **11 quantitative** and **2 qualitative experiments** (**see global response table above**), as well as **five new additional insight experiments** in the response pdf. \n\n**UPDATE:** To help clarify, we add a **new summary table (Global response table**) with the main takeaway and references to the relevant sections, tables and figures. \n\n\nWe hope this addresses your concerns and demonstrates the thoroughness of our comparisons.\n\n---\n## (B) Clarifying tabular models used\n\nWe thank you for your comment on the choice of downstream tabular models.  We clarify that a core contribution of SMART is its targeted sampling of hypotheses, which is entirely independent of the model used. As detailed in **Section 3.3 and Section 4**, SMART's context-guided slice sampling mechanism $\\pi$ is used to generate hypotheses independently of the downstream model. This is why the specific tabular model used is not the primary focus of our paper.\n\nDespite this, we demonstrate SMART's effectiveness across multiple models, including Logistic Regression, SVM, XGBoost, and MLP, showing the applicability of our findings.\n\nWhile we understand the reviewer's concern about \"outdated\" models, we argue that our chosen models remain highly relevant for tabular data:\n   - Interpretability: Models like logistic regression offer crucial interpretability in domains such as healthcare and finance and are still widely used in industry.\n   - Performance: Recent work has shown that traditional ML models like tree-based boosting methods (XGBoost) often outperform deep learning approaches on typical tabular data [Grinsztajn et al., NeurIPS 2022] and we would argue that tabular deep learning has not been universally adopted for tabular data.\n\nThat said, we have also run additional results with two tabular deep learning method: **TabPFN and TabNet (Table 4 in the response pdf)**. Across all the methods, SMART is the best at finding subgroups where the models are least reliable.\n\n\n---\n## \\(C) Cost of LLM hypothesis generation & scalability to larger datasets\n\nWe clarify the cost and scalability of SMART — demonstrating not only that SMART is cheap but also easily scalable to large datasets\n\n- Scalability: SMART's scalability depends on the number of hypotheses generated, not dataset size (unlike data-only methods). This allows SMART to easily scale to arbitrarily large datasets.\n- Cost Analysis: In practical terms, cost then also scales primarily with the number of hypotheses generated, not dataset size.  We provide a rough estimate based on token counts of input and outputs for 2 datasets (SEER and OULAD). This would be $<0.10$ USD for 5 hypotheses and $<0.50$ USD for 100 hypotheses for state-of-the-art models.\n\n| Model       | Cost SEER 5 Hypothesis (USD) | Cost SEER 100 Hypothesis (USD) | Cost OULAD 5 Hypothesis (USD) | Cost SEER 100 Hypothesis (USD) |\n|----|-----|----------|------|------|\n| GPT-4       | 0.017                  | 0.249                    | 0.022                   | 0.316                    |\n| GPT-3.5     | 0.004                  | 0.050                    | 0.005                   | 0.064                    |\n| GPT-4o Mini | 0.0003                  | 0.005                    | 0.0005                   | 0.006                    |\n| GPT-4o      | 0.008                  | 0.125                    | 0.011                   | 0.158                    |\n\n\n\n\n---\n## (D) SMART beyond tabular data (e.g. NLP)\n\nWe appreciate your interest in the generalizability of SMART. Our current focus is on tabular data due to its importance in high-stakes domains like healthcare and finance,  as well as,  context being naturally encoded in tabular data which is used to guide the targeted sampling of failures. Extending SMART to other domains like NLP requires non-trivial adaptation, warranting dedicated investigation.\n\n\nApplying SMART to text would require addressing the lack of explicitly interpretable features and developing new ways to operationalize hypotheses on unstructured data. In contrast, tabular data offers inherent context due to the interpretability of features, which is crucial for context-aware hypothesis generation and operationalization.  \n\n**UPDATE:** We'll use the extra camera-ready page to highlight this as an interesting future avenue for extending the context-aware testing paradigm.\n\n---\n## (E) Discussion on future bias mitigation when using SMART\n\nWe agree with you that is important to look at directions for removing biases from SMART. Our discussion on precisely this issue can be found in **Sec 5.4**. We realize that the title of the section might not fully convey this and hence will rename it (see update below).\n\nTo summarize, in **Sec 5.4** we have outlined several strategies to mitigate biases inherent in LLMs. \n\n- (a) Using the training dataset to guide hypothesis generation.\n- (b) Use our self-falsification mechanism to empirically test hypotheses against the actual data, thereby filtering out spurious hypotheses.\n- \\(c) Transparent Reporting: SMART generates comprehensive model reports (example in Appendix D.7) that provide clear justifications for each hypothesis, allowing for human oversight and intervention if needed.\n\n**UPDATE:** Rename section 5.4 as “Assessing and mitigating potential LLM challenges and biases” to better flag this.\n\n---\n## Thank you\n\nThank you for your engagement --- we believe these changes should improve the paper's contribution and clarity. Given these changes, we hope you might consider revising your assessment of the paper's impact and the corresponding evaluation for NeurIPS.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BPB1HKpmxZ", "reviewer": "Reviewer_bfyv", "summary": "The paper offers a multiple hypothesis testing view of ML evaluation (in regard to test slice\nfinding). Authors identify problems with data-only approaches (like high amounts of false positive\nand false negative model failure triggers). The paper proposes SMART, a context-aware LLM based ML\nmodel testing methodology to mitigate said problems. SMART uses LLMs for hypothesis generation (data\nslices where model could potentially underperform). The method is empirically evaluated and compared\nto a set of data-only baselines, showing less false-positives and recovering more model failure\nmodes.", "questions": "- Is it possible to run a method when the data schema is limited (e.g. some feature names are unknown, and the context is rather shallow in general)?\n- Could you provide experiments on more real world datasets with real known model failure cases? The\n  TableShift [3] benchmark could help with such an experiment as it already has subsets (OOD eval sets)\n  that could serve as ground-truth problematic data slices.\n- In what form the code would be available? I suggest making a package with said methodology to increase adoption.\n- Are open-weights free LLMs able to generate sufficiently high quality hypothesis for the method to work?\n\nReferences:\n- [1] Shankar, Shreya, et al. \"Automatic and precise data validation for machine learning.\" Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. 2023.\n- [2] Polyzotis N. et al. Data validation for machine learning //Proceedings of machine learning and systems. – 2019. – Т. 1. – С. 334-347.\n- [3] Gardner, Josh, Zoran Popovic, and Ludwig Schmidt. \"Benchmarking distribution shift in tabular data with tableshift.\" Advances in Neural Information Processing Systems 36 (2024).", "limitations": "Limitations are adequately addressed", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The paper is well structured and well written, it is easy to follow and is enjoyable to read. It\ntackles an important problem in model testing at an interesting angle, identifies an issue with\nexisting (and common) methodology. The paper presents a novel, clever and well motivated solution\nthat incorporates strong sides of LLMs (contextualization, hypothesis generation) with strong sides\nof prior data-based solutions. The discussion on mitigating potential LLM challenges is important\nand well considered.", "weaknesses": "1. *It is unclear how the method degrades with less informative context*.\n   - Most experiments validate the method in conditions well suited for it. e.g. easily\n     distinguishible irrelevant features (could you clarify what are the feature names in the\n     experiment, is it truly the case that the LLM could easily differentiate them from potentially\n     useful features?)\n3. *Some details of the method are missing from the main text*\n   - What are the prompts (better to include some information on this in the main text, I had questions when exploring the experimental results)\n   - Not sure from which subset of the data the slices are selected (it is intuitive that slices are\n     subsets of the test data, but if I'm not mistaken it is not clearly defined in the text).\n4. *Some related work may be missing*. I think that the subject of ML model testing in industry and production scenarios might be\n   related, while it is not discussed in the paper [1, 2].", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer bfyv,\n\nThank you for taking the time to carefully review our work. We provide answers below (A-F) and highlight our paper updates.\n\n---\n## (A) Method degradation with less informative context\n\nWe appreciate your interest in SMART's performance under varying context quality. We've developed the context-aware framework primarily for the tabular domain because feature names encode useful information for reasoning about potential failures. Therefore, we *do not* recommend using CAT when feature names do not encode useful information. \n\nWe address your concern in two ways. First, we perform a qualitative study where we limit the data schema by hiding the feature names and inspect the hypotheses and justifications generated. We find that in the limited-schema case, SMART generates hypotheses based on inferences about the feature information (e.g. \"the model might fail on feature_4 if feature_4 represents gender\"). In contrast, informative names guide meaningful hypothesis generation. Such hypotheses and justifications are illustrated in **Table 3 in the response pdf**.\n\nSecond, we evaluate whether limiting the data schema by hiding some feature names and leaving minimal external context affects detection rates of model failures. We compare two versions of SMART, original and with corrupted feature labels, in identifying data slices with high performance discrepancies from average (**Fig. 1 in the response pdf**). We find that across two real-world private datasets, hiding the feature names hinders model evaluation. This highlights that feature names play an important role in finding model failures.\n\n**UPDATE**: We'll discuss SMART's sensitivity to feature names in the appendix.\n\n---\n## (B) Moving content from the appendix to the main text\n\nWe agree with you that highlighting the specific prompt structure used would improve readability and we'll add more discussion in the camera-ready version. \n\nWe will also clarify our data subset selection methods (slices are always selected based on train data and evaluated on test data).\n\n---\n## \\(C) Related work\n\nWe appreciate you bringing attention to relevant related work. Both works [1] and [2] differ from our approach in that they focus on data validation techniques for general ML pipelines, while our context-aware testing framework specifically aims to use context to guide the search for model failures. \n\n**UPDATE**: We will cite these works in the revision and provide a discussion of how context-aware testing differs from existing data validation techniques used in industry.\n\n---\n## (D) Comparison on TableShift benchmark\n\nWhile TableShift [3] is a useful benchmark, it's not directly applicable to SMART. TableShift primarily focuses on OOD-detection. In contrast, SMART searches for model failures within the existing data distribution (recall that the context-guided sampling mechanism in Def. 1 samples from the existing data distribution).\n\nThat said, we agree that addressing dataset shift would be a valuable extension of context-aware testing. We'll mention this as future work, highlighting TableShift as a potential benchmark for extending context-aware testing to the OOD domain.\n\n---\n## (E) Code release \n\nWe expect to release the code in the form of a package mirroring your suggestion. We demonstrate the usage capabilities here and will expand it in the appendix.\n\n```python\nimport SMART\n\n# Load SMART\nmodel_tester = SMART('gpt-4') # Instantiates smart\n\n# Give desired context\ncontext = \"\"\"Find where a model fails. This is a prostate cancer prediction task.\"\"\"\n\n# Load a desired ML model\nmodel = XGBoost()\n\n# Test the model\nmodel_tester.fit(X_train, context, model) \n```\n\nOnce fitted, many attributes can be accessed, such as found subgroups ```model_tester.subgroups```, generated hypotheses ```model_tester.hypotheses```, justifications ```model_tester.justifications```, model report ```model_tester.model_report```, and more. We can also evaluate it: ```print(model_tester.top_underperforming_subgroup)```\n\n```python=\nOverall accuracy of the model: 85.1%\nAccuracy on 'Age > 75' subgroup: 55.2%\nDiscrepancy of: 29.9%\n```\n\n**UPDATE:** Add code usage to appendix.\n\n---\n## (F) Open-weight models for hypothesis generation\n\nBefore jumping in, we wish to highlight **Appendix D.3**, which notes our goal is not to exhaustively test the SMART framework with every LLM. Rather, the goal is to showcase that SMART is feasible with at least the capabilities of GPT-4. While we assessed SMART's performance with different LLMs, we caution against using it with less capable models.\n\nThat said, based on your comment we have conducted an experiment with Mistral-7b, Qwen-1.5-7b, Llama-3-8b, Llama-70b, where for the OULAD and SEER datasets we generate 5 hypotheses and assess overlap to the hypotheses generated by GPT-4. This is presented in **Table 2 in the response pdf**.\n\nTo summarize, the overlap between open-source models and GPT-4 is between 60-80%. We find that open-source models propose similar hypotheses, but they are not replacements for more capable models. This corresponds to our findings in **Appendix D.6** --- less capable models might propose similar hypotheses, yet they still catch fewer model failures. \n\n**Closed-source models are cheap.** In case your worry is that closed-source models are expensive to run, we conduct an additional evaluation calculating the cost of running SMART with closed-source models (**Table 1 in the response pdf**). The cost does not scale with dataset size but with the number of tests conducted. We find that generating 100 hypotheses with a powerful closed-source model (GPT-4o-mini) costs less than 1 cent and using state-of-the-art models (GPT-4) costs about 0.30 USD.\n\n---\n## Thank you\n\nThank you for your engagement. Given these changes, we hope you might consider revising your assessment of the paper's impact.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "n7QXkz2lF1", "reviewer": "Reviewer_H6db", "summary": "The paper introduces Context-Aware Testing (CAT), a new method for testing machine learning (ML) models. Current ML testing methods rely only on data, which often leads to high false positive and false negative rates and misses meaningful failures. CAT improves this by adding external knowledge or context to the testing process. The paper presents SMART Testing, the first version of CAT, which uses large language models (LLMs) to identify potential failures and a self-falsification mechanism to test them. The results show that SMART Testing finds more relevant and impactful failures than traditional data-only methods.", "questions": "1. How can we identify the most relevant contextual information for specific applications? How do we know that one context information is more valuable than another?\n2. Are there alternative methods or models that could complement or substitute LLMs for hypothesis generation in CAT?\n3. Are there any strategies to mitigate biases inherent in LLMs when using them for hypothesis generation in CAT?\n4. Can the SMART Testing framework be adapted to handle non-tabular data such as images?", "limitations": "1. The paper mainly concentrates on tabular data and does not investigate how CAT could apply to other data types like images.\n2. The reliance on LLMs for hypothesis generation in CAT poses a constraint, particularly in scenarios where LLMs are not accessible or applicable. This dependency may restrict the method's usability and applicability across diverse settings and applications.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-written and easy to follow.\n2. The concept of Context-Aware Testing represents a significant advancement in ML testing, offering a new perspective that goes beyond traditional data-only methods.\n3. The SMART Testing framework effectively uses large language models (LLMs) and self-falsification, which demonstrates the practical application of CAT.\n4. The paper presents valuable empirical evaluations that demonstrate the effectiveness of SMART Testing in identifying significant model failures.", "weaknesses": "1. The effectiveness of CAT depends on the quality and relevance of the contextual information. While context information may be available, effectively using it to generate meaningful hypotheses and justifications is not guaranteed, and there are cases where CAT may not be effective.\n2. Using LLMs for hypothesis generation could introduce biases from the data used to train these models, potentially impacting the reliability of the testing results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear R-H6db,\n\nThank you for your thoughtful comments to improve the paper. We provide answers (A)-(E) & highlight updates to the paper\n\n---\n\n# (A) Clarifying contextual information\n\nWe'd like to clarify that when we refer to \"context\", we mean the *relevant background knowledge of a system (e.g. an LLM) that informs the generation of targeted hypotheses*.\n\n(i) Dependence on context: To clarify, we don't _require_ external contextual information or human expert input. For tabular data, context is inherently encoded in the feature names. Our focus on tabular data leverages this built-in context. For instance, in a medical dataset, features like age, sex, or patient features provide context to guide LLM hypothesis generation. This contrasts with data-only approaches which only use the numerical data values alone and ignore the context surrounding the feature names.\n\n(ii) Identifying relevant context: We _do not_ require manual selection of contextual information. The relevance or value of the contextual information is automatically determined by the LLM: (a) when proposing hypotheses based on tabular data's inherent context and (b) via our self-falsification mechanism which empirically evaluates hypotheses and prunes those not supported by data.\n\n**UPDATE**: We will update Sec. 4 using the extra camera ready page to clarify this concept of context and how it is used.\n\nFinally, to highlight the minimal input needed, we thought it would be useful to show SMART's ease of use. i.e. additional contextual info input is not necessary but possible. If not provided the LLM leverages the context inherently encoded in the dataset `X`.\n\n\n```python\nimport SMART\n\n# Instantiate SMART\nmodel_tester = SMART('gpt-4') \n\n# Give desired context (this can be left as an empty string)\ncontext = \"\"\"Find where a model fails. This is a prostate cancer prediction task.\"\"\"\n\n# Load ML model\nmodel = XGBoost()\ndataset_description = X.describe()\n\n# Test the model\nmodel_tester.fit(X, context, dataset_description, model) \n```\n\n**UPDATE**: include the code snippet in the revision to show usage of SMART.\n\n---\n# \\(B) Alternatives to the LLM for hypothesis generation\n\nTo the best of our knowledge, the current main alternative for generating targeted hypotheses would be human input. However, this lacks SMART's automation and scalability. This underscores both the novelty of our LLM-based approach for targeted model testing. Additionally, this highlights exciting research opportunities to develop alternative targeted samplers for hypothesis generation within the proposed CAT framework.\n\n--\n# (C) LLM bias & mitigation strategies\n\nWe agree with you that is important to look at directions for removing biases from SMART. Our discussion on precisely this issue can be found in **Sec 5.4**. We realize that the title of the section might not fully convey this and hence will rename it (see update below).\n\nTo summarize, in **Sec 5.4** we have outlined several strategies to mitigate biases inherent in LLMs. \n\n- (a) Using the training dataset to guide hypothesis generation.\n- (b) Use our self-falsification mechanism to empirically test hypotheses against the actual data, thereby filtering out spurious hypotheses.\n- \\(c) Transparent Reporting: SMART generates comprehensive model reports (example in Appendix D.7) that provide clear justifications for each hypothesis, allowing for human oversight and intervention if needed.\n\nEmpirically, Sec. 5.4 (Table 4) demonstrates SMART's ability to correctly identify underperforming subgroups, even in scenarios where LLMs might have prior biases (e.g., ethnicity-related biases).\n\n**UPDATE:** Rename section 5.4 as “Assessing and mitigating potential LLM challenges and biases” to better flag this.\n\n---\n# (D) CAT beyond tabular data - images\n\nOur focus on tabular data is intentional due to its importance in many high-stakes domains such as healthcare, finance, and criminal justice, making ML testing particularly impactful for these critical applications by addressing a significant gap in current data-only ML testing methodologies. \n\nAdditionally, tabular data inherently includes context through feature names and metadata. This context is not naturally present in images (tensor of pixel intensities).  As discussed in Appendix A, extending CAT to other domains like images would require incorporating metadata to provide necessary context — which is often unavailable.\n\nWhile extending CAT to other domains is an interesting future direction, it's non-trivial. We hope future work can build on our work and adapt the context-aware paradigm to images.\n\n**UPDATE:** add this discussion to our discussion in Appendix A.\n\n---\n# (E) Constraint on LLM: might not be accessible or applicable in all scenarios\n\nWe wish to clarify the value of SMART even with possible constraints. First, LLMs are becoming increasingly accessible, especially from a cost perspective. As shown in **Table 1 (Response pdf)** it costs less than 0.10 USD for 5 hypotheses and less than 0.50 USD for 100 hypotheses for state-of-the-art LLMs. \n\nSecond, we show that SMART outperforms data-only methods even when using cheaper, less capable LLMs like GPT-3.5 (**Appendix D.3**) which are widely available to the public.\n\nThird, research done within the ML community is not always accessible or applicable (as illustrated by the vast research agenda of deep learning which presupposes access to expensive GPUs). Similarly, we see our research as being important in pushing the frontier of ML testing despite presupossing access to an LLM.\n\nFinally, given the stark differences in the effectiveness of data-only and context-aware methods (refer to the global response table), we think it is extremely important for the research community to have access to improved testing methods for more reliable ML testing in high-stakes domains.\n\n---\n# Thank you\n\nThank you for your engagement. Given these changes, we hope you might consider revising your assessment of the paper's impact.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cVu5UEOsWh", "reviewer": "Reviewer_ctad", "summary": "This paper introduces Context-Aware Testing (CAT), a novel approach that uses context as an inductive bias to guide the search for meaningful model failures. Unlike previous methods that rely solely on data to find slices where a model's predictions underperform compared to average performance, CAT addresses the limitations of these data-only approaches. Traditional methods often fail to consider the problem context and can lead to multiple testing problems. To overcome these issues, the authors propose loosening the restrictive assumption of relying solely on data. They introduce CAT as a new testing paradigm that incorporates context to provide a more comprehensive evaluation of model performance. Additionally, the paper presents SMART Testing, which employs large language models (LLMs) to hypothesize relevant and likely failures. These hypotheses are then evaluated on data using a self-falsification mechanism, allowing for the automatic identification of more relevant and impactful failures compared to traditional methods.", "questions": "- Can you provide examples for step 1 like in Table 1? What specifically constitutes contextual information and dataset contextualization?\n- In Section 5.1, how are the descriptions for synthetic features provided in SMART testing? If the context corresponds to \"it is synthetic\" or \"it is independent from the other,\" I think the results are too trivial.\n- If there are no accurate descriptions related to the features or datasets, how is the method applicable?", "limitations": "- As mentioned in the weaknesses, the reason why LLM is needed should be elaborated. As the authors mentioned in the manuscript, using LLMs could even hinder the total procedure of SMART testing if there is no clear reason for their use.\n- While the motivation is quite practical, rather than research-friendly, which is valuable, it would be beneficial to provide \"practical\" examples to illustrate the application of the proposed methods.", "rating": 3, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "The topic is highly interesting, and I agree with the authors' motivation that diverse perspective testing is necessary for practical deployment in real-world cases. These dimensions can improve the practical utility of tabular prediction models. In particular, the core limitation of data-only testing—the lack of a principled failure mode discovery procedure that can incorporate prior knowledge to guide the search for meaningful errors—is also persuasive.", "weaknesses": "Despite the importance of the topic and motivation, it is difficult to agree that the suggested method fully addresses all the issues. Most importantly, in the first step of hypothesis generation, we need to define the contextual information and dataset contextualization. If my understanding is correct, I am quite confused about how this contextual information differs from rule-based methods defined by human experts to use contextual information to find specific data slices. For example, in Table 1, if we have contextual information about age or comorbidities, we can directly utilize that information to find data slices without using LLMs or the operationalization step. How are these different, and how are steps 1 and 2 beneficial? I feel that utilizing LLMs is quite redundant for practical use. Additionally, too much prior knowledge is needed to utilize SMART testing, raising concerns about its practicality.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear R-ctad,\n\nThank you for the feedback which helped to clarify our paper. We'll address misunderstandings about contextual information and LLM necessity.\n\nWe provide grouped answers A-D & highlight paper updates.\n\n---\n# (A) Clarification on contextual information  \n\nWe apologize for any confusion caused by our use of the term 'context'. To clarify, we use 'context' in two distinct ways: (a) external input by a human (not strictly necessary for the framework) and (b) external knowledge (such as relevant background knowledge, domain understanding) within an external system (LLM). While we made a small footnote about this on page 5, we agree this is not enough and could have resulted in confusion about the nature of our work. \n\n| Aspect | Context as input (external input) | Context as external knowledge |\n-|-|-\n| Definition | Additional problem/dataset information in the form of a textual string | Relevant background knowledge of a system (e.g. an LLM) that informs the generation of targeted hypotheses |\n| Source | User input (optional) | An external system (e.g., GPT-4) which includes an understanding of the relationships between dataset features and their meaning|\n| Example | `info = \"this is a dataset on heart disease prediction\"` | Use of a language model (e.g., GPT-4) to generate hypotheses or justifications by understanding of the data |\n| Required? | Not necessary (can be left blank) | Required for CAT to work |\n\n\"Context-aware testing\" refers to external knowledge use, not user input. SMART uses tabular context (which is inherently encoded in the feature names) for hypothesis generation without needing human input. For example, **Table 1** in the paper showcases the hypotheses that were generated by SMART leveraging the feature names without any external human effort.\n\n## Need for prior knowledge & practical example.\n\nFurther, based on your suggestion to include a 'practical' example to illustrate the application of the proposed method, we thought it would be useful to show how easy it is to use SMART as well as the minimal contextual information and dataset contextualization needed. \n\nAs shown below, users do not need any prior knowledge to use SMART. Rather, we make use of the context inherently encoded in the dataset, feature names and task.\n\n```python\nimport SMART\nmodel_tester = SMART('gpt-4')\n\n# Optional external input (can be left blank)\ncontext = \"Find where a model fails. This is a prostate cancer prediction task.\"\n\nmodel = XGBoost()\ndescription_dataset = X_train.describe()\n\nmodel_tester.fit(X_train, context, description_dataset, model)\n\nprint(model_tester.subgroups)\n```\n\nThis implements SMART with just a few lines, requiring no additional human effort beyond providing the dataset and model.  In our implementations, we typically provide generic, easily accessible context that does not vary across tasks (see **Appendix C.3**. for examples).\n\n**UPDATE**: Add table to Sec. 3.3 to clarify the two uses of context and rename the *context input* as *external input* which we hope will disambiguate the two meanings. \n\n---\n# (B) Difference from Rule-Based Methods\n\nCAT and SMART fundamentally differ from rule-based methods:\n\n- No human input required: Unlike rule-based methods, CAT does not need experts to define rules based on domain knowledge.\n- Automated hypothesis generation: SMART uses feature names as implicit context to guide hypothesis generation automatically.\n- Scalability: This approach can be applied to any tabular dataset without human intervention.\n\n**UPDATE**: Add a contrast to rule-based in Sec. 3.2\n\n---\n# \\(C) Why we need an LLM in SMART\n\nWe wish to avoid using humans for large-scale testing because human intervention is expensive, time-consuming and often unavailable. Instead, we desire an *automated* approach to testing ML models. For this, we suggest to employ LLMs. The use of an LLM is crucial because (as per L177-186):\n\n- Contextual understanding: LLMs can interpret feature names and generate relevant hypotheses without human input\n- Targeted sampling: LLMs limit the number of tests, focusing on relevant model failures without testing all possible combinations\n\n**Benefits**. We show that by using LLMs to search for model failures, we are able to test ML models better. SMART finds data slices where models are unreliable much more often than data-only methods (see Sec. 5). Therefore, SMART offers a solution when human experts are unavailable or too expensive.\n\nUltimately, by using LLMs to *automatically* generate context-aware hypotheses, we can test models for model failures without succumbing to the issues that data-only testing methods face (see Sec. 3.2).\n\n**UPDATE**: Enhance current description around L174-186.\n\n---\n# (D) Questions\nQ1: What constitutes contextual info? \n\nA1: See response (A) clarifying contextual information. \n\nQ2: How are the synthetic features provided in SMART testing in Sec. 5.1?\n\nA2: We add new random features to the dataset which have uninformative names, such as \"feature1\", \"feature2\". We show that data-only methods will test for *everything* which results in systematic errors. SMART avoids this by selectively (and automatically) testing for *important* data slices.\n\nQ3: How is the method applicable if there are no descriptions of features?\n\nA3: See response \\(C): (a) feature names provide sufficient context for hypothesis generation; (b) generic prompts are used that do not change across tasks (see **Appendix C.3** for examples). There is no need to provide additional descriptions.\n\nQ4: How are steps (1) and (2) in SMART beneficial if we already have contextual information about age and comorbidities?\n\nA4: They are beneficial because we do not assume prior access to any knowledge (see response (A)). These hypotheses are automatically generated by the system purely from tabular feature names & generic external input about the task.\n\n---\n# Thank you\n\nGiven our clarifications, we hope you consider revising the paper's evaluation ☺️", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank the reviewers for their insightful and positive feedback! 

We are encouraged that the reviewers found our work on model testing "important" (**R-6t3M, R-bfyv**) and "interesting" (**R-6t3M, R-bfyv, R-ctad**). They agreed our motivation for diverse perspective testing is "necessary for practical deployment" (**R-ctad**). Our Context-Aware Testing concept, which incorporates prior knowledge to guide the search for meaningful errors, was deemed "persuasive" and recognized as "a significant advancement in ML testing" (**R-H6db**) by "identifying an issue with existing (and common) methodology" (**R-bfyv**) and addressing the "core limitation of data-only testing" (**R-ctad**). The reviewers appreciated our "novel, clever and well motivated solution" (**R-bfyv**), which is the SMART testing LLM instantiation deemed to effectively demonstrate "the practical application of CAT" (**R-H6db**). Our "valuable" (**R-H6db**) empirical evaluations showed the effectiveness of SMART Testing in identifying significant model failures (**R-H6db**), demonstrating "significant" improvements (**R-6t3M**), complemented by our "important and well considered" discussion on mitigating potential LLM challenges (**R-bfyv**).


---
# Summary of experiments and findings
We'd like to highlight the experiments we've conducted as a summary. We hope these will clearly showcase the value of context-aware testing.


| Type | Figure/Table | Purpose | Finding |
|------|--------------|---------|---------|
| Quantitative | Figure 4 | Assess robustness to irrelevant features | SMART consistently avoids irrelevant features, outperforming data-only methods. |
| Quantitative | Table 9 | Evaluate ability to satisfy testing requirements | SMART satisfies most requirements while maintaining statistical significance. |
| Quantitative | Table 2 | Assess ability to identify significant model failures | SMART discovers slices with larger performance discrepancies across models. |
| Quantitative | Table 3 | Measure false negative rates in identifying underperforming subgroups | SMART achieves the lowest false negative rates in all settings. |
| Quantitative | Table 4 | Assess robustness to potential LLM biases | SMART effectively mitigates biases in identifying underperforming subgroups. |
| Quantitative | Figure 8 | Assess how sample size affects irrelevant feature detection | SMART consistently avoids irrelevant features regardless of sample size. |
| Quantitative | Table 10 | Evaluate performance in different deployment environments | SMART identifies more significant failure slices in new environments. |
| Quantitative | Figure 9 | Assess impact of sample size on performance | SMART consistently outperforms data-only methods across all sample sizes. |
| Quantitative | Table 11 | Evaluate tendency to flag non-existent failures | SMART avoids spurious failures, unlike data-only methods. |
| Quantitative | Table 6 | Compare performance of GPT-3.5 and GPT-4 in SMART | Both GPT versions in SMART outperform benchmark methods. |
| Quantitative | Table 12 | Evaluate SMART across different tabular ML models | SMART identifies larger performance discrepancies across various models. |
| Qualitative | Table 7 | Compare hypotheses generated by GPT-3.5 and GPT-4 | Both GPT versions generate similar, relevant failure hypotheses. |
| Qualitative | Appendix D.7 | Showcase SMART's practical output | SMART generates comprehensive model reports, providing clear justifications for each hypothesis. |

---
# Response pdf
In addition to the above experiments, we provide new insights based on our discussions. In the response pdf, we provide the following:

- **Table 1. Cost of SMART (USD)**. We quantify the usage of smart in USD for different closed-source GPTs. SMART is extremely accessible and cheap to use. Generating 100 hypotheses with GPT-4o Mini costs less than 1 cent.

- **Table 2. Comparison hypotheses between GPT-4 and open-weight models**. We provide insights into whether smaller open-weight models can be used as an alternative to more capable closed-source models. While we do not recommend this, we find significant overlap between smaller and larger models.

- **Table 3. Example hypotheses and justifications when dataset column names are hidden**. This table showcases that SMART cannot generate meaningful hypotheses if the feature names are not interpretable.

- **Table 4. Identifying slices with the highest performance discrepancies across two deep learning tabular classifiers**. We additionally run how SMART compares to other approaches for two state-of-the-art deep learning classifiers.

- **Figure 1. Identifying the importance of feature names as a source of information**. We show that the ability to detect model failures is smaller if column names are hidden.

---
# Thank you
The review has been extremely productive. We thank everyone for their help in shaping the paper to be in better form.

Meta-review: This work introduces a method for formulating and testing hypotheses around when and why a statistical model performs poorly on a given dataset. An LLM is employed to both formulate plausible hypotheses around potential failure modes and select datapoints that are relevant to these hypotheses, followed by an optional set of statistical tests to confirm or reject generated hypotheses (referred to as self-falsification) (Sec. 4). The resulting method outperforms various prior methods that do not incorporate a broader semantic context on discovering impactful features (Sec. 5.2) and is more robust (Sec. 5.1 & 5.3).

The authors communicated extensively with the reviewers to clarify concerns, which led to most reviewers agreeing to accept the work.  Specifically, most reviewers agreed that this work provides a novel method that offers significant improvements. The main concern from the dissenting reviewer is that the proposed method requires meaningful feature names to be present in the tabular data that it operates on. While this is a valid limitation that should be noted in the work, many tabular datasets, including the six real-world datasets considered in this work (five in Sec. 5.1, one in 5.2) do have descriptive feature names. The results generally demonstrate that this technique can yield nontrivial gains in those settings. Additionally, concerns around the level of technical contribution were surfaced by the dissenting reviewer. However, the majority of reviewers express positive views on the level of technical novelty. The combination of LLM-based hypothesis generation and statistical verification is particularly effective (Tab. 2) and novel in this context.

Given this, the recommendation is to accept this work on the ground of its strong results and nontrivial technical novelty. The authors are encouraged to address all the noted concerns, including the method's reliance on meaningful feature names noted by reviewer ctad.
Decision: Accept (poster)