Context-Aware Testing: A New Paradigm for Model Testing with Large Language Models

Authors: paulius  rauba
nabeel  seedat
max ruiz luyten
mihaela  van der schaar
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 18:09:50

Abstract: The predominant *de facto* paradigm of testing ML models relies on either using only held-out data to compute aggregate evaluation metrics or by assessing the performance on different subgroups.  However, such *data-only testing* methods operate under the restrictive assumption that the available empirical data is the sole input for testing ML models, disregarding valuable contextual information that could guide model testing. In this paper, we challenge the go-to approach of *data-only testing* and introduce *Context-Aware Testing* (CAT) which uses context as an inductive bias to guide the search for meaningful model failures. We instantiate the first CAT system, *SMART Testing*, which employs large language models to hypothesize relevant and likely failures, which are evaluated on data using a *self-falsification mechanism*. Through empirical evaluations in diverse settings, we show that SMART automatically identifies more relevant and impactful failures than alternatives, demonstrating the potential of CAT as a testing paradigm.

Main paper: 
 introduction: 
 The ability to rigorously test and validate machine learning (ML) models is crucial for their reliable deployment in real-world applications. Despite solid aggregate performance, ML models are unreliable in a variety of real-world scenarios, such as on different subgroups [1] [2] [3] [4] [5] [6] , or encountering data deviating from its training distribution [7] [8] [9] [10] , often resulting in significant financial or societal consequences. These failures point to deficiencies in the way we test such ML models. To understand this in greater detail, let's address two questions: how are we currently testing ML models and can we do better?How are we testing? The predominant de facto paradigm of testing ML models relies on using only held-out data to evaluate the model either on average or by assessing performance on different subgroups within the dataset. Such data-only methods optimize a given objective function to find subgroups (or slices of data) within the dataset where a trained model underperforms relative to aggregate performance. We refer to this relative underperformance as a model failure. However, data-only methods operate under the restrictive assumption that the available empirical data is the sole input for testing ML model performance. In practice, this is almost always violated. It is common to have a priori knowledge of where models are likely to fail, given the problem context.The restrictive assumption of data-only methods comes at a significant cost. Specifically, data-only methods, which have been the go-to approach for ML testing, are required to iterate and test the approaches operate only on raw data without accounting for the problem context. Consequently, they must search across a large space of potential failures, usually covering all subsets of features and their distinct values. While an exhaustive search may seem beneficial, as we show in Sec. 3, the reality is that performing many tests on a finite dataset risks discovering slices where model failure is irrelevant or due to random variability, i.e. the multiple testing problem. SMART Testing addresses this challenge by prioritizing relevant and likely model failures through contextual awareness.Orthogonal testing dimensions: While SMART addresses what to test, several other dimensions of model testing exist that are orthogonal to our approach (detailed in Appendix A). (i) Behavioral Testing [16, 17, 5] evaluates model behaviors (i.e. responses to data) by operationalizing tests along pre-defined dimensions (often defined by humans) -rather than discovering the test dimensions. SMART fundamentally differs by addressing the core issue of "what to test". (ii) Software functional testing [18, 19] , aims to primarily test functional correctness (e.g. input-output functionality such as monotonicity), rather than testing for failures. In addition, test cases are either pre-specified or specified with an approximation of the underlying model to probe for functional correctness. 
data-driven operationalization:: 
 Utilizing training data to identify optimal splits for given covariates (e.g., age >= 82.32 as a condition to split data based on the "age" covariate). This method receives the covariate (or set of covariates) as an input and returns the optimal split. The optimal split is defined as the split which can identify two groups which have the largest absolute difference in accuracies given some requirements (e.g. a minimum or maximum group size in each group). This splitting can be done by any black box splitting function which takes in a set of inputs and returns a splitting mechanism.Black-box splitting function for data-driven operationalization. While any black-box splitting function could perform on the data, we provide details on the specific function used in SMART.The function implemented is based on a decision tree model, which can be either a regressor or a classifier, depending on the nature of the outcome variable. In all our experiments, we perform classification-based tasks.The function operates by fitting a decision tree to the data and recursively traversing the tree to find the split that yields the largest absolute difference in the outcome variable's mean value between two slices. The split must also satisfy group size constraints, specified as minimum and maximum group sizes. The traversal process evaluates each potential split, calculating the mean outcome for each slice and the discrepancy between these means. The optimal split is the one that maximizes this discrepancy while adhering to the group size requirements.The function returns a query string that represents this optimal split. This string can then be used to segment the dataset into the identified slices for further analysis or testing. The following is pseudocode for the splitting algorithm. In practice, we employ the LLM-based operationalization for the SMART ablation and the datadriven operationalization within the original SMART framework. However, we highlight that this is a design choice that we have found works in practice; there is nothing stopping from using any operationalization framework within the main SMART framework.SMART is built on many modules which can be toggled on or off. One of such modules is a module called "feasibility check" which evaluates whether there are any hypotheses which should be tested in the first place. The experiments presented in Sec. 5.1 highlight the importance of being able to identify when no relationship between covariates exist. The feasibility check contains three steps. First, an LLM is queried to evaluate whether any relationships could exist between the covariates and an outcome variable. For instance, this could be whether a relationship could exist between loan default and the annual rainfall in a given region. Second, the answer is self-refined. This helps to evaluate feasibility because, in practice, initial responses tend to be over-optimistic (such as hypothesizing that annual rainfall is associated with loan default via a geographic proxy). This self-refinement helps to critically evaluate the previous answer. The number of steps in the self-refinement process is a hyperparameter. Lastly, the answer (whether or not a relationship could exist between the two variables and, hence, should be inspected) is converted to a boolean value via an external function.The following is pseudocode which implements the feasibility module. Given that SMART operates with an LLM, sometimes the framework outputs proposals which do not operationalize on the data well. For instance, even if a column "age" is a categorical variable, SMART might propose to operationalize the hypothesis "elderly people" as age > 72 which would cause an error.To avoid this, we implement an additional data adjustment module which can handle such cases. It catches the error and re-prompts the LLM to find a group which could be operationalized given the data structure, and does so iteratively until such a group is found.The following is a pseudocode function that explains how data adjustment is performed. Requirements. SMART can natively integrate user requirements into its framework. This is done by inputting requirements as a string and concatenating it together with the context. Some requirements are directly integrated into the framework itself (e.g. functionalities for determining the minimum or maximum sample size of a data split).Automation. Fig. 1 showcases the pipeline of SMART and which components are automated. SMART is developed using an "sklearn" style fit, predict framework. The fit method automatically performs a feasibility check, generates hypotheses, justifications, operationalizes them, performs self-falsification using empirical data, re-ranks the hypotheses, and saves all intermediate results. The method can then automatically be used on any piece of data to evaluate whether it underperforms on the groups that have been found to underperform.Outputs. In addition to SMART outputting subgroups/slices or a scalar number, it can output a model report. An example report is provided in D.10. We note that this report is simply an example which employs both the findings of the fitting procedure and an additional LLM to summarize the outputs. More fine-grained outputs can be constructed.In the paper, we propose that SMART can move outside of IID data and generalize better in the presence of covariate shift (refer to Sec. D.2). We highlight that this is done by performing an analysis on the original data and using an LLM propose possible hypotheses and splits for a different target domain where no data is present (hence, operationalizing only using the LLM with access to previous operationalizations). The following is an example prompt which is designed to do this. TASK : Propose specific variable ranges for each hypothesis such that they are clearly operationalizable and defined . Use this format : Hypothesis : < >; Operationalization : < >."""Code Listing 3: Operationalization (LLM knowledge): General templateThe reason why SMART performs testing via hypothesis generation is because testing for all slices is equivalent to generating and testing a hypothesis on the data. Here, we outline in greater detail how hypothesis generation is connected to multiple testing.As outlined in the main manuscript, searching for f failures may bring up the challenge of multiple hypothesis testing. Specifically, when we evaluate the failure rate of model f across different slices S i ⊆ D, we are testing the null hypothesisH (i) 0 : µ Si = µ D against the alternative H (i) 1 : µ Si ̸ = µ D .Then, the probability of making a Type I error increases with each test. This drastically inflates the family-wise error rate (FWER). For instance, assuming that each slice is independent, the probability of making one or more Type I errors across all tests is given by 1 -(1 -α) m , where m is the total number of slices tested. While this can be addressed by adjusting for multiple testing, we run into the trade-off between the FWER control and statistical power. As we employ statistical correction methods to account for Type I errors, we increase the probability of Type II errors.Task : Create { n } hypotheses as to which subgroups within the dataset the model will perform worse than on average because of societal biases or other reasons . Important : Your hypothesis can contain either one variable or two variables in the condition . Therefore , your goal is to find discrepancies in the model 's performance , not the underlying data outcomes . Justify why you think that for each of the { n } hypotheses . Format of the output : Hypothesis : < >; Justification : < >.Code Listing 4: Generic hypothesis prompt """ The following are hypotheses about which people within a dataset the model might underperform on . Propose specific ranges for each hypothesis . Hypotheses : { hypotheses }. TASK : return a dictionary that contains an index number as the key and the column value as the value . If there are multiple columns in that hypothesis , return them in a list . There are the column names : { ' , '. join ( unique_values . keys () ) }. """ """ ----------INPUT TEXT --------------Your task is to propose possible hypotheses as to which subgroups within the dataset might have worse predictive performance than on average because of societal bias in the dataset , insufficient data , other relationships , or others . The subgroups might be based on any of the provided characteristics , as well as on any combination of such characteristics .Open University Learning Analytics Dataset ( OULAD ) contains data about courses , students and their interactions with Virtual Learning Environment ( VLE ) for seven selected courses ( called modules ) . Presentations of courses start in February and October -they are marked by B and J respectively . The dataset consists of tables connected using unique identifiers . All tables are stored in the csv format .The task is to predict the students who are likely to fail or withdraw from a course . Task : Create 5 hypotheses as to which subgroups within the dataset the model will perform worse than on average because of societal biases or other reasons . Therefore , your goal is to find discrepancies in the model 's performance , not the underlying data outcomes . Justify why you think that . Format of the output : Hypothesis : < >; Justification : < >. """ ----------OUTPUT TEXT --------------Hypothesis 1: Students with disabilities will have worse predictive performance than average . Justification : Students with disabilities may face additional challenges in their learning process , which could affect their performance in the course . They may require additional support or accommodations that are not adequately captured in the dataset , leading to lower predictive performance for this subgroup .Hypothesis 2: Students from lower socioeconomic backgrounds ( indicated by imd_band ) will have worse predictive performance than average . Justification : Students from lower socioeconomic backgrounds may have limited access to resources and support systems that are necessary for successful completion of the course . This lack of resources and support may not be adequately captured in the dataset , leading to lower predictive performance for this subgroup .Hypothesis 3: Students with lower levels of education ( indicated by highest_education ) will have worse predictive performance than average . Justification : Students with lower levels of education may have less prior knowledge and skills that are necessary for success in the course . This lack of foundational knowledge may not be adequately captured in the dataset , leading to lower predictive performance for this subgroup .Hypothesis 4: Students who have attempted the course multiple times ( indicated by num_of_prev_attempts ) will have worse predictive performance than average . Justification : Students who have attempted the course multiple times may have encountered difficulties or challenges in their previous attempts that hindered their success . These challenges may not be adequately captured in the dataset , leading to lower predictive performance for this subgroup .Hypothesis 5: Students in certain regions ( indicated by region ) will have worse predictive performance than average . Justification : Students in certain regions may have different educational systems , resources , and support networks that can impact their performance in the course . These regional differences may not be adequately captured in the dataset , leading to lower predictive performance for subgroups based on region .The following are hypotheses about which people within a dataset the model might underperform on . Propose specific ranges for each hypothesis . Hypotheses : Hypothesis 1:Students with disabilities will have worse predictive performance than average .Justification : Students with disabilities may face additional challenges in their learning process , which could affect their performance in the course . They may require additional support or accommodations that are not adequately captured in the dataset , leading to lower predictive performance for this subgroup .Hypothesis 2: Students from lower socioeconomic backgrounds ( indicated by imd_band ) will have worse predictive performance than average . Justification : Students from lower socioeconomic backgrounds may have limited access to resources and support systems that are necessary for successful completion of the course . This lack of resources and support may not be adequately captured in the dataset , leading to lower predictive performance for this subgroup .Hypothesis 3: Students with lower levels of education ( indicated by highest_education ) will have worse predictive performance than average . Justification : Students with lower levels of education may have less prior knowledge and skills that are necessary for success in the course . This lack of foundational knowledge may not be adequately captured in the dataset , leading to lower predictive performance for this subgroup .Hypothesis 4: Students who have attempted the course multiple times ( indicated by num_of_prev_attempts ) will have worse predictive performance than average . Justification : Students who have attempted the course multiple times may have encountered difficulties or challenges in their previous attempts that hindered their success . These challenges may not be adequately captured in the dataset , leading to lower predictive performance for this subgroup .Hypothesis 5: Students in certain regions ( indicated by region ) will have worse predictive performance than average . Justification : Students in certain regions may have different educational systems , resources , and support networks that can impact their performance in the course . These regional differences may not be adequately captured in the dataset , leading to lower predictive performance for subgroups based on region ..Open University Learning Analytics Dataset ( OULAD ) contains data about courses , students and their interactions with Virtual Learning Environment ( VLE ) for seven selected courses ( called modules ) . Presentations of courses start in February and October -they are marked by B and J respectively . The dataset consists of tables connected using unique identifiers . All tables are stored in the csv format . . The task is to predict the students who are likely to fail or withdraw from a course . TASK : Propose specific variable ranges for each hypothesis such that they are clearly operationalizable and defined . Use this format : Hypothesis : < >; Operationalization : < >. """ ----------OUTPUT TEXT --------------Hypothesis 1: Students with disabilities will have worse predictive performance than average . Op erationalization : Students with disabilities can be identified by the " disability " variable in the dataset . To test this hypothesis , we can define the range for students with disabilities as those who have a value of " Y " in the " disability " column .Hypothesis 2: Students from lower socioeconomic backgrounds ( indicated by imd_band ) will have worse predictive performance than average . Op erationalization : Socioeconomic background can be measured using the " imd_band " variable in the dataset . To test this hypothesis , we can define the range for students from lower socioeconomic backgrounds as those who have an " imd_band " value in the range of " 0 -30% " . Hypothesis 3: Students with lower levels of education ( indicated by highest_education ) will have worse predictive performance than average . Op erationalization : Education level can be determined using the " highest_education " variable in the dataset . To test this hypothesis , we can define the range for students with lower levels of education as those who have a value of " Lower Than A Level " or " No Formal quals " in the " highest_education " column .Hypothesis 4: Students who have attempted the course multiple times ( indicated by num_of_prev_attempts ) will have worse predictive performance than average . Op erationalization : Previous attempts can be measured using the " num_of_prev_attempts " variable in the dataset . To test this hypothesis , we can define the range for students who have attempted the course multiple times as those who have a value greater than 1 in the " num_of_prev_attempts " column .Hypothesis 5: Students in certain regions ( indicated by region ) will have worse predictive performance than average . Op erationalization : Region can be determined using the " region " variable in the dataset . To test this hypothesis , we can define the range for students in certain regions as those who belong to the regions of " North Region " or " Wales " .Code Listing 7: Operationalization (LLM knowledge): OULAD Dataset""" ----------INPUT TEXT --------------The following are groups that are defined based on the dataset . Convert them into a Python dictionary format . Each group should be represented as a key -value pair in the dictionary , where the key is an index (0 to 4) , and the value is a string representing the group using Python syntax and logical operators . For multiple conditions , use Python 's logical ' and ' ( '&& ') or ' or ' ( '|| ') . Ensure the format is a valid Python dictionary .-Single Condition : {0: 'X > 45 '} -Multiple Conditions : {1: '( X > 45) and ( Y < 20) '} Groups to summarize : Hypothesis 1: Students with disabilities will have worse predictive performance than average .Op erationalization : Students with disabilities can be identified by the " disability " variable in the dataset . To test this hypothesis , we can define the range for students with disabilities as those who have a value of " Y " in the " disability " column .Hypothesis 2: Students from lower socioeconomic backgrounds ( indicated by imd_band ) will have worse predictive performance than average . Op erationalization : Socioeconomic background can be measured using the " imd_band " variable in the dataset . To test this hypothesis , we can define the range for students from lower socioeconomic backgrounds as those who have an " imd_band " value in the range of "0 -30%". Hypothesis 3: Students with lower levels of education ( indicated by highest_education ) will have worse predictive performance than average . Op erationalization : Education level can be determined using the " highest_education " variable in the dataset . To test this hypothesis , we can define the range for students with lower levels of education as those who have a value of " Lower Than A Level " or " No Formal quals " in the " highest_education " column .Hypothesis 4: Students who have attempted the course multiple times ( indicated by num_of_prev_attempts ) will have worse predictive performance than average . Op erationalization : Previous attempts can be measured using the " num_of_prev_attempts " variable in the dataset . To test this hypothesis , we can define the range for students who have attempted the course multiple times as those who have a value greater than 1 in the " num_of_prev_attempts " column .Hypothesis 5: Students in certain regions ( indicated by region ) will have worse predictive performance than average . Op erationalization : Region can be determined using the " region " variable in the dataset . To test this hypothesis , we can define the range for students in certain regions as those who belong to the regions of " North Region " or " Wales ". ----------OUTPUT TEXT --------------{ 0: ' disability == " Y " ' , 1: ' imd_band in ["0 -30%"] ' , 2: ' highest_education in [" Lower Than A Level " , " No Formal quals "] ' , 3: ' num_of_prev_attempts > 1 ' , 4: ' region in [" North Region " , " Wales "] ' } Code Listing 8: Interpreter: OULAD DatasetWe note that all experiments were performed using two compute resources: a server with NVIDIA RTX A4000 GPU and 18-Core Intel Core i9-10980XE, as well as an Apple M1 Pro 32GB RAM. We exemplify SMART Testing using GPT-4 [29] as the LLM but run further experiments to test the sensitivity to the type of language model in Appendix D. Goal. We aim to underscore the role of context in ML model testing to prevent false positives, especially when dealing with tabular data where data may contain many irrelevant or uninformative features [30] , persisting even post-feature selection [31, 32] . We contrast SMART which explicitly accounts for context, in contrast to data-only approaches which are context-unaware only operating on the data.Setup. We fit a predictive model to the training dataset, varying the number of irrelevant, synthetically generated features contained in the dataset -where irrelevant features are drawn from different distributions. We then quantify the proportion of conditions in the identified slices that falsely include the irrelevant synthetic synthetically features.Because different methods are sensitive to different types of irrelevant features, we developed a data generating processes that encompasses many types of variables. Over many runs, different data-only methods pick up on some of these variables, showcasing that all methods are susceptible to randomly sampled irrelevant features in the dataset.Sampling mechanism. To evaluate the impact of irrelevant features, we enrich the dataset by adding synthetic categorical variables. The number of new variables is equal to the number of existing features in the dataset. For each new variable x i , we determine its type by sampling from a Bernoulli distribution with probability 0.5. If the sampled value is 0, x i is a Bernoulli variable with success probability 0.1; otherwise, x i is a categorical variable with four categories, following a predefined probability distribution (e.g., {0.1, 0.3, 0.4, 0.2}):Type(x i ) ∼ Bernoulli(0.5),x i ∼ Bernoulli(0.1) if Type(x i ) = 0, Categorical(0.1, 0.3, 0.4, 0.2) if Type(x i ) = 1.We note that there are synthetic data generating processes that completely break other data-only methods. As an example, creating a unique ID column for each sample breaks the Autostrat algorithm, as all the subgroups/slices identified are the unique IDs. The data generating process employed in our experiment reflects a broad variety of commonly encountered DGPs.We test whether each of the methods can fulfil three requirements.The first requirement involved the use of the variable "age" in each detected slice. This was passed as an input to SMART. The other methods do not accept context as input and, therefore, it was not possible to fulfil these requirements. The numbers provided for other methods are simply how often they fulfilled the requirements by chance.The second and third requirements involved obtaining a minimum and maximum sample size. This was passed as an input to SMART and the ablated SMART version. Based on this, SMART changed its hyperparameter within its function which asks to indicate a minimum and maximum sample size for the discovered slices. As with the previous experiments, this was not adjusted for the other groups because they do not take textual input. The goal of the adaptive testing experiment is to understand the extent to which SMART, as well as other data-only methods, can use data in a source domain to generalize to a new, target domain where a covariate shift has been detected. To this end, the datasets provided have a known covariate shift and can be evaluated.Each method was trained on the UK dataset and the discovered slices were evaluated on the US dataset. No additional context was provided to data-only methods since they do not accept any text or context as inputs.In contrast, we have provided SMART with the previously discovered slices and hypotheses, and have asked to re-evaluate these hypotheses in the context of the US dataset. Specifically, SMART re-hypothesized possible model failures for the US market but used the UK data to operationalize the variables. The ablated version, SM ART N SF , achieved the best overall performance on the US market. The ablated version (i) did not have access to the UK data and the failures of the models; and (ii) operationalized each covariate using the LLM alone (refer to Sec. B.2 for a discussion on operationalizations with different SMART versions). This provides evidence that, in the presence of covariate shift, using inductive knowledge or domain expertise might be more useful to finding meaningful model failures.C.4.5 Discovery of societally important groups and failure understanding (Sec. 5.5.)As discussed in the main paper, SMART provides both possible hypotheses and justifications for model failures which can be evaluated using a simple "fit" method. Furthermore, SMART prioritizes meaningful data slices which are of societal importance. Such slices can be inspected for any data input.Goal. Requirements are a crucial, yet neglected part of ML model testing, such as verifying performance on societally relevant dimensions or verifying specific aspects to meet compliance requirements. However, no previous testing framework has incorporated the notion of satisfying requirements when defining the test slices. This experiment illustrates how SMART integrates requirements provided in natural language, which then influences the hypotheses generated to satisfy testing requirements.Setup. We cover three real-world requirements that end-users might have: one based on demographics and two based on sample size. Requirement 1: Each of the top 10 identified unique slices should involve the age of a person. Requirement 2: The sample size of the top 10 identified unique slices must have at least 150 observations. Requirement 3: The sample size of the top 10 identified unique slices has to be small, within 10 and 150 observations. We exemplify the experiment using a real-world prostate cancer dataset from the UK [35] , as healthcare often mandates certain testing requirements (e.g. Collins et al. [48] ).Analysis. Table 10 shows that SMART which directly integrates requirements (via natural language), satisfies the requirements a greater number of times compared to data-only methods, which only satisfies requirements by chance (hence the low number of times). Beyond satisfying requirements, the SMART slices also represent model failures that almost always have statistically significant performance differences from average when evaluated on test data. Finally, while SM ART N SF (ablation without self-falsification) can satisfy requirements, the number of statistically significant slices is lower than SMART, thus underscoring the value of our self-falsification mechanism. That said, SM ART N SF still outperforms data-only baselines.Table 10 : Requirement satisfaction showing how many times the top 10 generated slices satisfied the requirements (Req) and how many of these slices had statistically significantly different performance from average (Sig ) on a testing dataset. Maximum is 10. ↑ is better. Autostrat 0.00 ± 0.00 0.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 0.00 ± 0.00 0.00 ± 0.00 PSG_B 2.00 ± 0.00 2.00 ± 0.00 2.00 ± 0.00 2.00 ± 0.00 0.00 ± 0.00 0.00 ± 0.00 PSG_A 2.00 ± 0.00 2.00 ± 0.00 2.00 ± 0.00 2.00 ± 0.00 0.00 ± 0.00 0.00 ± 0.00 Divexplorer 3.20 ± 2.11 0.95 ± 1.56 0.00 ± 0.00 0.00 ± 0.00 Takeaway 2. SMART, unlike data-only methods, identifies slices that have significant performance differences, whilst also satisfying requirements -an important dimension not even considered by previous testing methods.Goal. Deploying an ML model often entails going beyond IID, such as a different deployment environment. We consider the case of deploying a model to a different country where there is a covariate shift 12 and evaluate the capabilities of testing frameworks to adapt across the different environment and identify model failures.Setup. We use real-world prostate cancer datasets from different country's cancer registries with known distribution shifts: SEER (US) [11] and CUTRACT (UK) [35] . We train predictor f on UK data, while our target deployment environment is the US.Analysis. ▶ Identifying model failures. Table 11 shows that SMART better tests models at deployment time using the information provided. SMART identifies a much greater number of statistically significant model failures (almost all possible), both within the same environment (UK) and when shifting to a different one (US), even after adjusting for multiple comparisons using Bonferroni correction. Table 11 : Number of slices identified (out of a maximum of 10) that had significantly divergent performance from average (higher is better). S_α counts the number of significantly divergent groups at α = 0.05; S_α/n applies the Bonferroni correction. ↑ is better.D UK train D UK test D US test Sα S α/n Sα S α/n Sα S α/nAutostrat 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 0.95 ± 0.22 0.95 ± 0.22 PSG B 2.00 ± 0.00 2.00 ± 0.00 2.00 ± 0.00 2.00 ± 0.00 1.75 ± 0.43 1.50 ± 0.50 PSG A 2.00 ± 0.00 2.00 ± 0.00 2.00 ± 0.00 2.00 ± 0.00 1.75 ± 0. as expected, we find that for lower sample sizes, SM ART N SF (without the self-falsification mechanism) is superior, however, given enough data we then find that SMART benefits from the self-falsification.Deployment environment (US): we find that self-falsification similarly requires sufficient samples; which we note is expected behavior. Interestingly, in the deployment setting (US), SM ART N SF generally identifies the greatest number of significant failures (almost all possible) across different sample sizes. This suggests that under covariate shift, using inductive knowledge (via the LLM) or domain expertise might be more useful to find meaningful model failures. Overall, the results highlight the flexibility of SMART to handle different scenarios and sample sizes. From a practical perspective, while both SMART variants outperform data-only methods, the implication is that there is nuance in using different SMART variants for different scenarios.Takeaway 4. SMART identifies more significant divergent failure slices in a deployment setting, outperforming data-only methods across environments and sample sizes. The model might perform worse for younger patients as prostate cancer is more common in older men, and there might be insufficient data for younger patients.Subgroup of patients with older age (above 80) will have worse predictive performance due to higher likelihood of comorbidities and weaker immune systems, leading to higher mortality rates.Older age is often associated with increased risk of comorbidities and weaker immune systems, which can negatively impact the prognosis of prostate cancer. 2Patients with higher comorbiditiesThe model might perform worse for patients with multiple comorbidities as the complexity of their health condition might not be fully captured by the model.Higher Gleason scores indicate more aggressive and advanced prostate cancer, which is generally associated with poorer prognosis and higher mortality rates.Subgroup of patients with higher Gleason scores (4 or 5) will have worse predictive performance due to more aggressive and advanced cancer, leading to higher mortality rates. 3Patients who underwent conservative managementThe model might perform worse for these patients as the effectiveness of conservative management might vary greatly among individuals, and this variability might not be well captured by the model.Subgroup of patients with stage 4 cancer will have worse predictive performance due to the advanced stage of the disease, indicating metastasis and limited treatment options.Stage 4 prostate cancer is characterized by the spread of cancer to distant organs, indicating a more advanced and aggressive disease. The limited treatment options available for stage 4 cancer can result in worse predictive performance. 4Patients with lower prostate-specific antigen levelsThe model might perform worse for these patients as lower PSA levels might be associated with early-stage prostate cancer, which might be harder to predict due to lack of symptoms.Patients with multiple comorbidities often have complex health conditions that can impact the effectiveness of prostate cancer treatment and management, leading to worse predictive performance.Subgroup of patients with multiple comorbidities (e.g., 3 or 4) will have worse predictive performance due to the presence of other health conditions that may complicate the treatment and management of prostate cancer. 5Patients with higher Gleason scoresThe model might perform worse for these patients as higher Gleason scores indicate more aggressive cancer, which might progress unpredictably.Subgroup of patients who received conservative management as their primary treatment option will have worse predictive performance due to the potential delay in receiving more aggressive treatments.Conservative management may involve watchful waiting or active surveillance, which can result in delayed initiation of more aggressive treatments. This delay in treatment may lead to worse predictive performance. 6Patients in stage 4The model might perform worse for these patients as stage 4 prostate cancer is advanced and might progress unpredictably.Primary hormone therapy is often used to control the growth of prostate cancer by reducing the levels of male hormones (androgens). Patients who do not receive this therapy may have uncontrolled cancer growth, resulting in worse predictive performance.Subgroup of patients who did not receive primary hormone therapy will have worse predictive performance due to the potential lack of hormonal control over the cancer growth. Subgroup of patients who did not undergo radical prostatectomy will have worse predictive performance due to the potential presence of residual cancer cells in the prostate gland.9Patients in stage 1The model might perform worse for these patients as stage 1 prostate cancer is early-stage and might be harder to predict due to lack of symptoms.Subgroup of patients with higher prostate-specific antigen (PSA) levels (above 50) will have worse predictive performance due to the indication of more advanced cancer.Higher PSA levels are often associated with more advanced prostate cancer. Patients with higher PSA levels may have a worse prognosis and higher mortality rates, leading to worse predictive performance. 10Patients who underwent primary hormone therapyThe model might perform worse for these patients as the outcomes of primary hormone therapy might vary greatly among individuals, and this variability might not be well captured by the model.The combination of older age, higher Gleason scores, and stage 4 cancer represents a subgroup with multiple negative prognostic factors. The cumulative effect of these factors is likely to result in the worst predictive performance.Subgroup of patients with a combination of older age (above 70), higher Gleason scores (4 or 5), and stage 4 cancer will have the worst predictive performance due to the cumulative effect of advanced age, aggressive cancer, and metastasis.The primary task of the SMART method is to evaluate a given, trained machine learning model. Thus far, we have been using a logistic regression model as the basis for evaluation in the main experiments. However, the results are not sensitive to the type of the tabular model. Therefore, in this section, we provide additional experiments where we vary the tabular model for the task. Specifically, we consider the following models initialized with their default hyperparameters for evaluation: Logistic Regression, Support Vector Machines, Boosting (implemented with XGBoost) and a multi-layer perceptron with 2 hidden layers and RELU activation functions in the hidden layers.Goal. The primary goal is to understand whether the framework generalizes to other models which operate under different mapping mechanisms (e.g. a logistic regression, which is a linear model, compared to a tree-based model).Setup. Given that not all the slice discovery or model evaluation algorithms output multiple slices, we constrain the evaluation to only focus on a single slice which might have discrepant performance. We use the UK prostate cancer dataset and evaluate the discrepancy of the top identified slice relative to the average across all identified model types. The discrepancy is calculated as the absolute differences of the average performance between the two groups, as well as the p-value associated with the difference. The results show the average performance +-standard deviation of 5 random splits. A higher absolute difference indicates that the model fails on one of the slices more than average.Discretizing the inputs. Many of the discovery methods, however, operate only on categorical data. The previously used dataset, however, has three continuous variables: age, prostate-specific antigen, and comorbidities. We therefore also assess the quality of these methods to discover slices on the testing dataset when these three variables are discretized into 10 bins each. The following Table 16 : The differences in accuracies between the top slice identified for each method on a testing dataset. The p-value computes the p-value associated with the difference in the accuracy. For the accuracy, higher values imply a greater ability to detect divergent slices (hence, higher is better). For the p-value, lower is better. Averages +-standard deviations are shown across 5 runs with random seeds and data splits. Autostrat 0.24 ± 0.02 0.00 ± 0.00 0.24 ± 0.02 0.00 ± 0.00 0.09 ± 0.09 0.33 ± 0.46 0.24 ± 0.02 0.00 ± 0.00 pysubgroup_beam 0.23 ± 0.01 0.00 ± 0.00 0.23 ± 0.01 0.00 ± 0.00 0.11 ± 0.07 0.18 ± 0.40 0.23 ± 0.01 0.00 ± 0.00 pysubgroup_apriori 0.23 ± 0.01 0.00 ± 0.00 0.23 ± 0.01 0.00 ± 0.00 0.11 ± 0.07 0.19 ± 0.42 0.23 ± 0.01 0.00 ± 0.00 Divexplorer 0.05 ± 0.11 0.81 ± 0.43 0.09 ± 0.13 0.61 ± 0.53 0.14 ± 0.15 0.47 ± 0.49 0.02 ± 0.05 0.86 ± 0.32 Slicefinder 0.01 ± 0.00 0.00 ± 0.00 0.01 ± 0.00 0.00 ± 0.00 0.00 ± 0.00 0.00 ± 0.00 0.01 ± 0.01 0.00 ± 0.00 Sliceline 0.26 ± 0.06 0.00 ± 0.00 0.26 ± 0.06 0.00 ± 0.00 0.18 ± 0.09 0.00 ± 0.01 0.26 ± 0.06 0.00 ± 0.00 SM ART _N SF 0.17 ± 0.01 0.00 ± 0.00 0.17 ± 0.01 0.00 ± 0.00 0.09 ± 0.05 0.07 ± 0.16 0.17 ± 0.01 0.00 ± 0.00 SMART 0.37 ± 0.03 0.00 ± 0.00 0.37 ± 0.03 0.00 ± 0.00 0.26 ± 0.06 0.00 ± 0.00 0.37 ± 0.03 0.00 ± 0.00 table reports the performance of all the methods (note: we did not discretize the dataset for SMART because it can work natively on continuous data). 0.00 ± 0.00 0.00 ± 0.00 0.00 ± 0.00 0.00 ± 0.00 0.00 ± 0.00 0.00 ± 0.00 0.00 ± 0.00 0.00 ± 0.00 Sliceline 0.09 ± 0.01 0.05 ± 0.06 0.08 ± 0.02 0.14 ± 0.09 0.11 ± 0.07 0.25 ± 0.24 0.09 ± 0.01 0.06 ± 0.08 SM ART _N SF 0.17 ± 0.01 0.00 ± 0.00 0.17 ± 0.01 0.00 ± 0.00 0.09 ± 0.05 0.07 ± 0.16 0.17 ± 0.01 0.00 ± 0.00 SMART 0.37 ± 0.03 0.00 ± 0.00 0.37 ± 0.03 0.00 ± 0.00 0.26 ± 0.06 0.00 ± 0.00 0.37 ± 0.03 0.00 ± 0.00Takeaway. SMART is able to consistently identify the greatest performing slices across a number of different tabular models.Goal. We further observe that data-only testing methods implicitly assume the existence of slices with discrepancies in performance. While indeed ML models do fail -it is equally as problematic to highlight failures where there are none.Setup. To evaluate this we propose a fully synthetic setup. Here, both the dependent variable (Y ) and the independent variables (X) are sampled from a predefined random distribution. Specifically, we predict loan default (Y ∈ {0, 1}) based on a set of independent variables X = {N runs , M pref , A rainfall , F color , P season }, which are conceptually and empirically independent of the outcome of interest. Ideally, if we account for context we should be able to identify these disparate features should not influence the account and hence without prior relationships we should not flag spurious slices.We compare the data-only methods to SMART under three different data generating processes (scenarios) that capture diverse underlying dynamics denoted as S uniform , S skewed , and S interactions , where each DGP has a focus on uniform, skewed, and interactive effects, respectively.The first scenario is given by the variables sampled from the following DGPs:N runs ∼ Uniform{1, 499}, M pref ∼ Uniform{1, 5}, A rainfall ∼ Uniform{20000, 99999}, F color ∼ Uniform{1, 6}, P season ∼ Uniform{0, 3}, Y ∼ Uniform{0, 1}.The second scenario is given by the variables sampled from the following DGPs:N runs ∼ Uniform{1, 499}, M pref ∼ Binomial(1, 0.5), A rainfall ∼ Categorical(0.1, 0.3, 0.4, 0.2), F color ∼ Binomial(1, 0.1), P season ∼ Binomial(1, 0.05), Y ∼ Uniform{0, 1}.The third scenario is given by the variables sampled from the following DGPs:N runs ∼ Uniform{1, 499}, M pref ∼ Binomial(1, 0.5), A rainfall ∼ Categorical(0.1, 0.3, 0.4, 0.2), A music_hap = M pref × A rainfall , A run_hap = N runs × M pref , Y ∼ Uniform{0, 1}.Analysis. Table 18 shows the number of slices spuriously discovered, while Table 19 outlines the number of conditions within the slices. We can clearly see the pitfalls of data-only approaches which detect slices which in reality have no relation to one another -often surfacing few conditions per group which suggests they arise by chance. The rationale for this failure is simply because data-only approaches do not and cannot reason about the features and/or understand context and simply aim to find slices with discrepancies in performance -which of course could arise by chance. In contrast, we see that SMART by virtue of context-awareness can avoid surfacing groups -which in reality have no relationships. We provide an additional experiment where we vary the sample size in the training dataset and observe how that affects the number of slices discovered for each method. We show that the SMART is not affected by irrelevant features regardless of the sample size of the training dataset. The result is shown in Figure 9 . We assess the cost of LLM hypothesis generation and scalability to larger datasets. Specifically, we demonstrate not only that SMART is cheap but also easily scalable to large datasets• Scalability: SMART's scalability depends on the number of hypotheses generated, not dataset size (unlike data-only methods). This allows SMART to easily scale to arbitrarily large datasets.• Cost Analysis: In practical terms, cost then also scales primarily with the number of hypotheses generated, not dataset size. We provide a rough estimate based on token counts of input and outputs for 2 datasets (SEER and OULAD) in Table 20 . This would be less than 0.1 USD for 5 hypotheses and less than 0.5 USD for 100 hypotheses for state-of-the-art models. SMART ideally should be used with the most capable LLM possible.That said, we assess the differences in hypotheses between open-weight models and GPT-4.We assess Mistral-7b, Qwen-1.5-7b, Llama-3-8b, Llama-70b, where for the OULAD and SEER datasets we generate 5 hypotheses and assess overlap to the hypotheses generated by GPT-4. This is presented in Table 21 .To summarize, the overlap between open-source models and GPT-4 is between 60-80%. We find that open-source models propose similar hypotheses, but they are not replacements for more capable models. This highlights that less capable models might propose similar hypotheses, yet they still catch fewer model failures.SMART uses the implicit context encoded in the interpretable feature names as a source of contextual information to guide hypothesis generation. For instance, in a medical dataset, features with names We aim to assess the sensitivity to interpretable feature names to provide guidance on the use of SMART. First, we perform a qualitative study where we limit the data schema by hiding the feature names (such that they become uninformative) and inspect the hypotheses and justifications generated. We find that in the limited-schema case, SMART generates hypotheses based on inferences about the feature information (e.g. "the model might fail on feature_4 if feature_4 represents gender").In contrast, informative names guide meaningful hypothesis generation. Such hypotheses and justifications are illustrated in Table 22 .Second, we evaluate whether limiting the data schema by hiding some feature names and leaving minimal external context affects detection rates of model failures. We compare two versions of SMART, original and with corrupted feature labels, in identifying data slices with high performance discrepancies from average (Fig. 10 . We find that across two real-world private datasets, hiding the feature names hinders model evaluation. This highlights that feature names play an important role in finding model failures. We compare two versions of SMART on CUTRACT and SEER datasets. The first version, "base", is the normal SMART method. The second version, "corrupt", has feature names changed to uninformative ones. The goal is to identify slices with the highest performance discrepancies between the average & subgroup prediction, calculated as the difference in their respective accuracies (y-axis). We see showing feature names enables to find subgroups that have a higher performance discrepancy from baseline performance than without feature names. ↑ is better.Table 22 : Example hypotheses and justifications when dataset column names are hidden. The hypotheses and justifications for features that do not encode meaningful information lose practical meaning. Context-aware testing benefits from rich feature names that can be used to guide meaningful hypothesis generation.Base The model will perform worse on predicting the risk of disease for older individuals.Age is a significant factor in many diseases, and older individuals often have more complex health situations with multiple comorbidities. This complexity could make it more difficult for the model to accurately predict disease risk.The model will perform worse on predicting the risk of disease for individuals with lower Gleason scores.The Gleason score is a grading system used to determine the aggressiveness of prostate cancer. Lower scores indicate less aggressive cancer, which may be more difficult to predict due to its slower progression and less obvious symptoms. We show an example of a model report which is automatically generated by SMART. A model report includes a summary of key hypotheses, justifications, sources, and summary findings as a written report. A model report fully satisfies the requirements of the user.In this case, the model is generated to directly match latex output requirements. It is provided in the shaded box below as a part of the US prostate cancer (SEER) [11] evaluation.A report on the performance of the model has been concluded. The following are the hypotheses, their justifications tested on the model with their conclusions on whether the hypothesis was supported. The model may perform worse for patients with a higher number of comorbidities Patients with more comorbidities may have more complex health situations that are not fully captured by the dataset. Additionally, these patients may be more likely to die from causes other than prostate cancer, which could confuse the model's predictions.The model may perform worse for patients with a higher Gleason score A higher Gleason score indicates more aggressive cancer, which may be harder to predict due to its rapid progression and the potential for other health factors to influence outcomes.A further analysis was run on the training and testing datasets to understand the effects of each identified slice. The following are two tables which summarize a part of this information: Recommendations:1. The model appears to be less reliable for older patients (age > 75), patients with lower levels of prostate-specific antigen (< 10), and those who have undergone conservative management treatment. It also shows decreased performance for patients with more than two comorbidities and those with a Gleason score of 4. However, the model is more reliable when these conditions are not met.2. Before deploying the model, the end user should be aware of the following: -The model's performance may be compromised for older patients and those with multiple comorbidities. Consider additional validation or alternative models for these groups. -Patients with lower prostate-specific antigen levels and those who have undergone conservative management treatment may also experience less accurate predictions. Additional clinical insights may be needed for these cases. -Although the model shows decreased performance for patients with a Gleason score of 4, this group is relatively small, so the impact on overall model performance may be limited. However, caution should be exercised when interpreting results for these patients. Justification: The abstract accurately reflects the claims made in the paper. Our paper introduces the new paradigm of context-aware testing which we discuss in Sec. 3. We introduce our context-aware testing instantiation SMART in Sec. 4. We experimentally show how under a variety of different experimental conditions SMART outperforms data-only benchmarks in Sec. 5. We therefore believe the provided evidence fully supports the claims that context-aware testing is a new alternative to testing ML models relative to data-only methods.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: We discuss the limitations of our work in Sec. 6. We also provide ways of addressing these limitations in Sec. 5.4 with an extended discussion on possible biases, mitigation strategies, and best practices, including how SMART provides unique opportunities to address such challenges with features such as model reports or transparent hypothesis testing and operationalization.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.• While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
a context-aware testing framework for ml: 
 The prevailing paradigm for testing ML models relies on data-only methods which exclusively use data to surface model failures. In this section, we explore the limitations of data-only methods by viewing ML testing as a multiple hypothesis testing problem. We explore why data-only methods are uniquely prone to finding false positive and false negative model failures. To address this, we introduce a new paradigm of testing called context-aware testing which relies on external knowledge, or context, as an inductive bias to better identify where models fail.Preliminaries. Denote the feature space by X and the label space by Y, and P the joint probability distribution over X × Y. We wish to test a fixed, trained black-box model f : X → Y, using a finite dataset D ⊂ X × Y usually split into D train = {(x i , y i )} Ntrain i=1 and D test = {(x i , y i )} Ntest i=1 . We assume the existence of a loss function ℓ : Y × Y → R which measures the discrepancy between the model's prediction and the true labels point-wise.The primary goal of ML testing is to identify meaningful failure modes-subgroups (data slices) of the data distribution where the model's performance is significantly worse than its average behavior. Formally, let S ⊆ X × Y denote a data slice and let P S = P(•|(x, y) ∈ S) be the conditional distribution induced by S. We aim to identify slices where the slice-specific expected loss µ S = E (x,y)∼P S [ℓ(f (x), y)] significantly exceeds the population-level expected loss µ P = E (x,y)∼P [ℓ(f (x), y)] 3 . Testing ML models is a multiple hypothesis testing problem. We are interested in identifying failure modes that generalize beyond the training dataset. We can interpret the empirical dataset D as a sample from a broader distribution P, and our goal is to make an inferential claim on the performance on the data slices with respect to P. Suppose we have a candidate data slice Ŝ ⊆ D . We can evaluate the empirical slice-specific loss as μS = | Ŝ| -1 (x,y)∈ Ŝ ℓ(f (x), y) and compare it to the empirical loss over the entire dataset μD = |D| -1 (x,y)∈D ℓ(f (x), y). To make an inferential claim about the model's performance on the data slice S w.r.t. P, we can follow the frequentist testing paradigm and formulate a hypothesis test where we evaluate whether the performance is significantly different. Therefore, H 0 : µ S = µ D and alternative hypothesis H 1 : µ S ̸ = µ D , where µ S = E (x,y)∼P S [ℓ(f (x), y)] and µ D = E (x,y)∼P [ℓ(f (x), y)] denote the true slice-specific and population-level losses. In practice, this evaluation could be done by running an appropriate frequentist statistical test and evaluating whether p < α for each slice, given some pre-defined α.However, in realistic testing scenarios, we evaluate the model's performance not just on a single slice but on a large collection of candidates {S j } m j=1 . This amounts to conducting many simultaneous hypothesis tests. Accounting for multiple testing is important. A naive testing procedure that does not adjust for multiplicity could surface a large number of spurious failure modes simply by chance (Type I error). Conversely, controlling the false discovery rate [20] may involve adjusting the per-test significance threshold to α ′ ≪ α, potentially sacrificing power to detect true failures (Type II error).The multiple hypothesis testing viewpoint reveals a key challenge in ML model evaluation: To reliably surface meaningful failures, we require a principled procedure for generating a relatively small number of promising hypotheses (candidate slices) to test.Existing ML testing methodologies are data-only in that they only use the available empirical data to test for model failures and vary in their optimization objective [14, 12, 13, 21, 15] . However, data-only methods operate under the restrictive assumption that the available empirical data is the sole input for testing ML model performance. In practice, this is almost never the case. It is common to have an a priori understanding of where models are likely to fail given the data distribution, model class, training algorithm, and deployment context. This restrictive assumption results in three challenges: ▶ (i) High false positive rate: data-only methods search over a large space of data slices and each evaluation amounts to an implicit hypothesis test (Sec. 3.1) . Therefore, the probability of observing a false failure increases with every test performed. ▶ (ii) High false negative rate: The naive testing procedure can be made robust by correcting for the number of tests performed which reduces the statistical power to detect true failures. ▶ (iii) Lack of meaningful failures: Data-only methods are fundamentally limited by the fact that not all statistically significant slices are practically meaningful. We empirically validate these claims in Sec. 5 .The core limitation of data-only testing is the lack of a principled failure mode discovery procedure that can incorporate prior knowledge to guide the search for meaningful errors.To address the limitations of data-only testing, we introduce context-aware testing (CAT), a principled framework for identifying meaningful model failures using context. This could be the context implicitly encoded in the dataset (i.e. via meaningful feature names) or available external input (i.e. external contextual knowledge, such as a string of input information from a human). Let C denote the space of all possible contextual information and c ∈ C be specific external input. Our core insight is that we can use C as an inductive bias to select which slices to test for. Definition 1 (Context-Aware Testing). Let X , Y, P, f , D = {(x i , y i )} N i=1 , and ℓ be defined as in the standard supervised learning setup. Let C be a space of contexts.Context-aware testing is defined by two procedures:1. A context-guided slice sampling mechanism π : C × (X × Y) × N → 2 X ×Y such that π(c, D, m) = {S 1 , . . . , S m }, where c is used as an inductive bias for function π to prioritize slices likely to contain meaningful failures, and m ∈ N are the number of slices to evaluate.2. A multiple hypothesis testing procedure:∀S i ∈ π(c, D, m), test H 0 : µ Si = µ D vs. H 1 : µ Si ̸ = µ D , where µ Si = E (x,y)∼P S i [ℓ(f (x), y)], µ D = E (x,y)∼P [ℓ(f (x), y)]A meaningful failure is characterized by statistical significance and practical relevance.The targeted sampling mechanism π uses the context C as an inductive bias to prioritize testing of slices that are (i) more relevant to the deployment context, and (ii) more likely to exhibit significant performance gaps.This principled slice selection offers several key advantages over data-only methods: ▶ (i) Improved false positive control: by limiting the number of tests conducted to m, CAT controls the risk of spurious discoveries that arise when naively testing all possible slices. ▶ (ii) Improved true positive rate: The targeted selection of slices likely to contain failures maintains test power by avoiding the need to aggressively correct for multiple testing. ▶ (iii) Meaningful failures: context-guided sampling identifies failures that are both statistically significant and practically relevant.Context-aware testing overcomes the limitations of data-only methods by employing a principled, context-guided slice sampling mechanism π to prioritize the discovery of meaningful model failures.The core technical challenge in realizing CAT is the development of an effective context-conditional sampling algorithm π. In the following section, we propose a concrete instantiation of π using a large language model to generate plausible failure modes and guide testing of ML models.LLM hypothesis generationTesting feasible givenPrompt Conservative management might be chosen for patients with more complex or less predictable cases, which could lead to worse predictive performance.Notreatment_conservative _management == 1 0.000 0.200 SupportedStep 1: Hypothesis Generation. Recall from Sec. 3.3, we wish to define a sampling mechanism π to sample slices S which are both relevant and have a high relative likelihood of failure -where π should be both contextually-aware and able to integrate requirements to guide sampling.Data Space... We posit that LLMs have the potential to satisfy these properties due to the following capabilities: ▶ Contextual understanding: LLMs have been pretrained with a vast corpus of information and hence have extensive prior knowledge around different contexts and settings [22] [23] [24] [25] . ▶ Integrate requirements: LLMs are adept at integrating requirements or additional information about the problem via natural language [26, 24] . ▶ Hypothesis proposers: In proposing likely failure modes, LLMs have also been shown to be "phenomenonal hypothesis proposers" [27] .Operationalize hypotheses LLMAn LLM is defined as a probabilistic mapping l : Σ * → P (Σ), where Σ denotes the vocabulary space, and P (Σ) represents the probability distributions over Σ. The model processes input sequences s ∈ Σ * , each a concatenation of tokens representing external (contextual) input C e (which can be null) and dataset contextualization C D is formalized as s = (C e , C D ). We extract the contextualized description C D from the dataset D using an extractor function E : D → C D , which captures essential dataset characteristics (e.g. feature relationships and high-level dataset information). Additionally, we highlight the the LLM will implicitly extract context based on the context encoded in the dataset (e.g. via meaningful feature names). Based on input s, the LLM predicts a distribution over Σ from which hypotheses of model failure and corresponding justifications are sampled.As depicted in Fig. 2 , given C e and C D , we sample the N most likely hypotheses of failures, H = {H 1 , H 2 , . . . , H N }, and corresponding justifications J = {J 1 , J 2 , . . . , J N }, to provide explainability. This process is formalized using the LLM's mapping l as follows:(H i , J i ) ∼ l(s), where s = (C e , C D ), ∀i ∈ {1, 2, . . . , N }.Step 2: Operationalization. The process of operationalizing each hypothesis H i ∈ H involves translating its natural language expression into a form that can directly operate on the training dataset D train (an example is provided in Table 1 ). To achieve this, we define an interpreter function I : H → {0, 1} X that maps each natural language hypothesis H i to a corresponding binary-valued function g i : X → {0, 1} on the feature space X , where g i (x) = 1 if x satisfies the criteria and g i (x) = 0 otherwise. Each function g i induces a data slice S i ⊆ D train consisting of data points that satisfy the criteria of hypothesis H i , such that Step 3: Self-falsification We introduce a novel selffalsification mechanism to empirically evaluate (or refute) the generated hypotheses 4 . Specifically, for each feasible hypothesis H i ∈ H, we attempt to falsify the hypothesis with observed empirical data 5 . This involves evaluating the model f over the data slice S i operationalized from H i . We then assess whether the slice performance on f has a significant deviation from the model's overall performance.S i = {(x, y) ∈ D train : g i (x) = 1}.For instance, in Table 1 , this is done by computing |∆Acc| and the p-value. The significance of this deviation is determined through frequentist statistical testing, i.e. when p < α for any α which might also be adjusted for multiple hypothesis testing. This step effectively "reshuffles/reranks" the hypotheses based on their likelihood on the observed data. For example, when benchmarking we select the top n hypotheses based on statistical significance:H * = arg min n hi∈H T {p i < α}.Remark: As shown in Fig. 1 , we can exclude the self-falsification from SMART in cases of small-sample sizes. We denote this ablation of SMART as SM ART N SF .Step 4. Model Performance Evaluation and Reporting. Finally, SMART automatically generates a report of the overall performance of the model under varying conditions generated by the LLM, including a summary report, a complete summary of the tests carried out, intermediate and final results, and potential failure modes of the ML model.SMART is a tabular CAT method which (i) directly searches for model failures, sampling targeted tests (Sec. 3.3) and (ii) incorporates data, and context into ML testing.We highlight the practical use of SMART testing and emphasize SMART's ease of use and minimal input requirements needed. In particular, as shown in the example below, users do not need any prior knowledge to use SMART. Rather, we make use of the context inherently encoded in the dataset, feature names and task. Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [Yes] Justification: The paper makes a claim that context can be used to guide the search for relevant and meaningful model failures via a context-guided slice sampling mechanism which uses context to prioritize likely slices. This is defined and discussed in Sec. 3.3. Guidelines:• The answer NA means that the paper does not include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced. • All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can appear in either the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced. 
experimental result reproducibility: 
 Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)? Answer: [Yes] Justification: The paper provides the key experimental setups in the main experimental section (Sec. 5) together with additional experimental details in the appendix. Guidelines:• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. systematic, modular, automated, requirements-responsive, transferablewhile technically any subset of the dataset can be considered a data slice, we are practically interested in meaningful failures, such as model failures on vulnerable socio-economic groups.self-falsification is rooted in scientific philosophy and the seminal ideas of Popper[28] who proposed the principle of falsification-the ability to refute hypotheses with empirical observations-as driving science.We assume data for self-falsification is available. If not, discovered slices could guide new data collection.Code can be found at: https://github.com/pauliusrauba/SMART_Testing or https://github. com/vanderschaarlab/SMART_Testinghttps://github.com/Adebayo-Oshingbesan/BPS_Autostrat_Code/https://github.com/flemmerich/pysubgrouphttps://github.com/elianap/divexplorerhttps://github.com/yeounoh/slicefinderhttps://github.com/DataDome/slicelinep(X) changes, while p(Y |X) remains the same 
illustrating smart use cases: 
 We now quantitatively evaluate SMART Testing 6 and demonstrate the viability of this new ML testing paradigm (i.e. CAT), in contrast to data-only testing. Table 2 summarizes our experiments and takeaways.Baselines. We compare SMART with a variety of data-only testing baselines namely: Autostrat [14] , Slicefinder [12] , Sliceline [13] , Pysubgroup [21] and Divexplorer [15] . We also evaluate SM ART N SF (i.e. no self-falsification) as an ablation. Given space limits, we exemplify the LLM in SMART by GPT-4 [29] and our ML model to audit is logistic regression. We investigate other LLMs in the Appendix D.3 and other tabular models in Appendix D.4, observing similar results. We provide additional experimental details in Appendix C. Assess robustness to irrelevant features SMART consistently avoids irrelevant features, outperforming data-only methods. Table 3 Assess ability to identify significant model failures SMART discovers slices with larger performance discrepancies across models. Table 4 Measure FNR in identifying underperforming subgroups SMART achieves the lowest false negative rates in all settings. Table 5 Assess robustness to potential LLM biases SMART effectively mitigates biases in identifying underperforming subgroups. Table 10 Evaluate ability to satisfy testing requirements SMART satisfies most requirements while maintaining statistical significance. Figure 8 Assess how sample size affects irrelevant feature detection SMART consistently avoids irrelevant features regardless of sample size. Table 11 Evaluate performance in different deployment environments SMART identifies more significant failure slices in new environments. Figure 9 Assess impact of sample size on performance SMART consistently outperforms data-only methods across all sample sizes. Table 12 Evaluate tendency to flag non-existent failures SMART avoids spurious failures, unlike data-only methods. App. D. 3 Compare performance of GPT-3.5 and GPT-4 in SMART SMART using both GPT versions outperform benchmark methods. Table 13 , 14 Evaluate SMART across different tabular ML models SMART identifies larger performance discrepancies across various models. Table 20 Evaluate SMART the cost of using SMART SMART is cost efficient to generate hypothesis less than 0.5 USD. Table 21 Compare the overlap of hypotheses w/ open-weight models SMART using open-weight models has a significant overlap of hypotheses. Table 22 Evaluate SMART' SMART identifies larger performance discrepancies across various models. App. D.10 Showcase SMART's practical output SMART generates comprehensive model reports, providing clear justifications for each hypothesis.Goal. We aim to underscore the role of contextual awareness in preventing false positives in model testing. In particular, we consider the scenario when dealing with tabular data that may contain many irrelevant or uninformative features [30] , persisting even post-feature selection [31, 32] . We contrast SMART which explicitly accounts for context, to data-only approaches which are context-unaware and can only operate on the numerical data.Setup. We fit a predictive model to the training dataset, varying the number of irrelevant, synthetically generated features contained in the dataset. The irrelevant features are drawn from different distributions. We then quantify the proportion of conditions in the identified slices that falsely include the irrelevant synthetic features. We evaluate using five real-world tabular datasets spanning diverse domains, namely finance, healthcare, criminal justice and education: loan, breast cancer, diabetes, COMPAS recidivism [33] and OULAD eduction [34] . These datasets have varying characteristics, from sample size to number of features and are representative of different contexts pertinent to tabular ML, to demonstrate the effectiveness of SMART across various real-world contexts Analysis. Fig. 4 shows the proportions of irrelevant features included in slices for increasing numbers of irrelevant features. Data-only methods are unaware of context and are shown to spuriously include high proportions of irrelevant features in their slices; i.e. false positives (FPs). Additionally, these false discoveries increase for data-only methods as the number of irrelevant features increases. Consequently, the sensitivity to FPs of data-only methods risks that slices identified are neither empirically relevant nor meaningful. In contrast, SMART, by virtue of contexual-awareness when generating hypotheses is not sensitive to extraneous and contextually irrelevant features. Importantly, SMART also maintains its robustness to FPs even with an increasing number of irrelevant features.Remark. We also assess sensitivity to the number of data samples. Fig. 9 , Appendix D.6 shows that SMART remains robust to variations irrespective of sample size. In contrast, data-only methods have variable performance with false discoveries sensitive to the sample size.Avoiding non-existent failure slices. It is important that model testing does not flag failures when there are none. In Appendix D.5, we demonstrate that SMART's contextual awareness means that it is resistant to spurious failure slices that have no underlying relationships. In contrast, data-only approaches implicitly assume the existence of problematic slices, which we empirically show makes them prone to spuriously flagging non-existent failure slices.Takeaway 1. SMART's contextual awareness ensures testing relevance, thereby reducing false positives across different scenarios, in contrast to data-only methods.Goal. We assess whether the identified failure slices persist when evaluated on new, unseen data across different tabular models. This evaluates the generalization of the identified model failures across multiple different ML models. Setup. We use the prostate cancer dataset [35] and aim to discover slices indicative of underperformance. Thereafter, we assess generalizability of the identified failures across four different ML models (logistic regression, SVM, XGBoost and MLP). Specifically, we compare the top identified slice from each method and compute the absolute difference between the accuracies of that slice and the remainder of the dataset. We posit that a testing framework should be able to identify slices with high-performance discrepancies on unseen data across multiple ML models.Analysis. Table 3 shows that SMART slices exhibit the greatest discrepancies in model accuracies on unseen test data across different ML models -indicating the discovered failure modes are generalizable. We find that SMART surfaces slices with greater performance differences compared to SM ART N SF (ablation without self-falsification), highlighting the importance of the introduced selffalsification mechanism. In contrast, data-only methods fail to identify slices where the performance discrepancy is as large as SMART. This limitation can be attributed to the tendency of data-only methods to overfit the training data, leading to high false discovery rates.Takeaway 2. SMART discovers failure slices where the model substantially underperforms, generalizing to unseen test data across different ML models. In contrast, data-only approaches fail to find slices where the difference in accuracies is as large, highlighting the lack of generalizability and reliability of their findings.In our setup, false negatives (FNs) are directly tied to true positives (TPs). i.e. the more true positives we find, the fewer FNs we miss. Across multiple experiments (Fig. 4 , Table 10 , Table 3 , Table 16 ), we consistently show that SMART identifies TPs at substantially higher rates than data-only. For example, in Table 16 , SMART identifies an average of 9.6 out of 10 subgroups where the ML model significantly underperforms. The fact that data-driven methods discover fewer such subgroups implies that they are missing the ones SMART uncovers.Goal. That said, we conduct an additional experiment to directly assess the false negative rate, wherein we can control issues (as FNs are naturally unspecified in real data).Setup. We simulate a dataset to predict recidivism (Y ) based on five covariates: gender, race, age, income, and education. log( P (Yi|Xi=j) P (Yi̸ =j) ) = α j -(δ 1 X age + δ 2 C income + δ 3 C education ) + ϵ, where ϵ ∼ N (µ 0 , σ 0 ). We then train a predictor function f on the data and synthetically introduce underperformance on certain corrupted subgroups. For an individual i, if they belong to corrupted subgroup j, the prediction Ŷi is equal to f (X i ) with probability 1 -p, and a random prediction sampled from a Bernoulli distribution with probability p. If the individual does not belong to subgroup j, the prediction is simply f (X i ). Finally, we measure how often each testing method identifies that the model f underperforms on a subgroup. The justifications for the tests allow human users to inspect the model's testing procedures, understand the reasons, and audit for biases or missed dimensions.Goal. Going beyond the mitigation strategies by design, we also assess SMART's robustness to prior biases. Specifically, we assess common ethnicity related biases of LLMs. 0.98 ± 0.01 0.00 ± 0.00 0.00 ± 0.00 0.98 ± 0.01 0.10 0.98 ± 0.01 0.00 ± 0.00 0.00 ± 0.00 0.99 ± 0.01 0.20 0.98 ± 0.01 0.00 ± 0.00 0.00 ± 0.00 1.00 ± 0.00 0.301.00 ± 0.00 0.00 ± 0.00 0.00 ± 0.00 1.00 ± 0.00 0.50 1.00 ± 0.00 0.00 ± 0.00 0.00 ± 0.00 1.00 ± 0.00Setup. We use the same data generating process as Sec. 5.3.We train a predictor function f and simulate a scenario where we intentionally corrupt a model's predictions for a proportion τ of a minority subgroup ("white" or "black" ethnicity).τ ranges from 0 (no corruption) to 1 (completely random predictions for the subgroup). SMART aims to identify the top underperforming subgroup without bias based on historical patterns. We measure the proportion of times the top subgroup contains the "white" (P white ) or "black" (P black ) minority, averaged over 20 runs and 5 seeds, separately corrupting "white" and "black" ethnicities.Analysis. We show in Table 5 that SMART is able to identify where models underperform even in scenarios such as ethnicity bias, where LLMs exhibit prior biases from the training dataset. This links to the above discussion that SMART mitigates such biases by design both using the training dataset to guide hypothesis generation and using the self-falsification mechanism to empirically evaluate hypotheses (and discard those that aren't reflective of the data).Takeaway 4. SMART mitigates potential biases in the LLM, both by using the real data to guide hypothesis generation, as well as using the self-falsification mechanism to filter spurious hypotheses. 
discussion and limitations: 
 Responding to recent calls for better model testing [36, 6] , we formalize Context-Aware Testing; a new testing paradigm, actively seeking out relevant and likely model failures based on contextual awareness -going beyond data alone. We develop SMART Testing, using LLMs to hypothesize likely and relevant model failures providing improved and automated testing of tabular ML models, compared to data-only methods in various scenarios.Model reports. SMART produces comprehensive and automated model reports documenting failure cases and justifications, thereby providing data scientists, ML engineers and stakeholders increased visibility into model failures. We provide an example SMART report in Appendix D.10.Practical considerations. Given the potential utility of SMART we highlight the following five practical considerations: ▶Hypothesis generation. While CAT offers a principled framework for context-guided testing, LLMs present challenges in hypothesis generation. Although SMART has mechanisms to address these (see Sec. 5.4), it cannot guarantee the absence of biases. ▶Use with small datasets. SMART may be limited in some cases by insufficient real data to operationalize and test hypotheses, suggesting future work could explore targeted data collection or synthetic data generation to enhance testing. ▶ Extensions to other modalities. SMART is formalized to test tabular ML models, due to the interpretable and structured features in tabular data to guide hypothesis generation. While extensions to other modalities such as image and text is beyond the scope of this work, this is a promising future research direction that would require addressing the lack of explicitly interpretable features possibly via external metadata and developing new ways to operationalize hypotheses on unstructured data. That said, tabular data is ubiquitous in real-world applications [37, 38] with approximately 79% of data scientists working on tabular problems daily, vastly surpassing other modalities [39, 40] . This highlights the immediate impact and relevance of SMART. ▶ Need for interpretable/meaningful feature names. Feature names play an important role in finding model failures (see Appendix Appendix D.9). The need for interpretable/meaningful feature names (e.g. column labels such as sex, age, race etc) as a source of context is similar to human requirements of interpretable feature names to understand what the data refers to. While feature names are typically the de facto in research and industry datasets, in the rare occasions they are not, this will affect the performance of SMART. ▶ Cost of SMART. SMART is extremely accessible and cheap to use, approximately <0.10 USD for 5 hypotheses and <0.5 USD for 100 hypotheses for state-of-the-art models (see Appendix D.7).Broader impact. Better testing practices can help ensure models are reliable, safe, and beneficial before being deployed in real-world applications. We hope that our work can help mitigate model testing risks in real-world applications as well as spur new testing regimes which are context aware.Appendix: Context-aware testing: a new paradigm for testing with large language models This work primarily engages with works on ML model testing. Consequently, we detail different paradigms of ML model testing next, which we summarize in Table 6 and Figure 5 showing that none of the existing paradigms satisfy all the properties for ML testing neither in terms of automation nor from the perspective of relevance as they are unable to incorporate context and/or requirements. Existing paradigms in ML model testing. The ML community has predominatly approached ML model evaluation/testing via the use of held-out test datasets. On the basis of the test dataset a single performance metric (accuracy, AUC etc) is computed. This single average evaluation may mask nuances of the model's performance along various dimensions. One approach to address this is to create better benchmark datasets when evaluating models on common benchmarking tasks. For example, manual corruptions like Imagenet-C [41] or by collecting additional real data such as the Wilds benchmark [8] . Benchmark datasets are labor-intensive to collect and their utility is limited to the specific benchmark tasks.What if we want to evaluate models not confined to benchmarking tasks? To test in cases beyond benchmark tasks, the community has proposed trying to find slices or regions wherein the model fails (i.e. via stress tests). As mentioned in Sec. 1, this could be ▶ Behavioral testing: which requires human expertise and intuition to define the test scenarios (e.g. Checklist [16] , HateCheck [17] or 3S-Testing [5] ) -which is not automated and does not scale. Moreover, it runs a high risk of overlooking critical weaknesses due to human cognitive biases. ▶ Data-only testing: which does not account for context or requirements and searches exhaustively (e.g. Autostrat [14] , SliceFinder [12] or DivExplorer [15] ). This may slices focus on arbitrary, less important, or unrealistic/implausible scenarios that are unlikely to be seen in reality. Moreover, we run the risk of the multiple testing problem, where by virtue of the large number of tests evaluated, we might discover a divergent group by chance.Hypothesis-driven ML model testing. In contrast, a hypothesis-driven approach to ML model testing brings about the falsifiability approach widely adopted in science. It begins with the formulation of specific, testable hypotheses based on theoretical understanding, context/domain knowledge, and the intended application of the model (i.e. requirements). The concept of hypothesis-driven ML model testing is deeply rooted in work by Popper [28] who proposed the principle of falsifiability as a driver of scientific progress. The progression of knowledge hinges on the formulation and rigorous testing of hypotheses which can either be falsified or supported by empirical evidence. In the context of hypotheses in ML model testing, testable statements about the model's performance under various conditions, fairness, and robustness can improve our understanding of the model's performance. By rigorously testing these hypotheses, we can uncover the strengths and limitations of ML models.Contrast to data-only methods on unstructured data.In this paper, we have discussed data-only methods for slice discovery or blindspot discovery applicable to structured data. Specifically, we focus on tabular data where metadata in the form of column names is explicitly encoded into the data. The data-only approaches covered in this work directly search over the raw feature space to identify slices with similar attributes wherein the model would exhibit underperforming predictions.For completeness, we contrast to data-only approaches often applicable to data without explicit structure and metadata (images, text, audio). There have been numerous methods including [42] [43] [44] [45] Figure 5 : Contrasting paradigms of ML model testing along (i) Testing relevance which accounts for requirements and/or context and (ii) Degree of automation and adaptability when carrying out testing. We desire a new paradigm of ML testing to address both.to identify slices of unstructured data with systematic failures. They all follow a similar pattern:(1) the data is embedded (often with a pre-trained) model into a representation space and (2) the underperforming slices are identified by clustering on the raw embedding space or after dimensionality reduction. We then need to post-hoc interpret the clusters in order to understand what they represent. This contrasts the tabular data setting where we find cohesive groups of features that provide an explicit interpretation. Moreover, in tabular regimes, we do not have access to pre-trained models in the same way as for domains such as images or text.Specifically, to context-aware testing we also note that tabular data inherently includes context through interpretable/meaningful feature names and metadata. This context is not naturally present in images (which are tensors of pixel intensities). Hence, extending CAT to other domains like images or text would require incorporating metadata to provide necessary context -which is often unavailable. In addition, one would need to develop new ways to operationalize hypotheses on unstructured data.Contrast to software testing.Going beyond ML model testing, the idea of testing is also prevalent in software systems. For example, unit tests of functions in a codebase. We highlight that software testing of functional input-output correctness is also an area proposed in software testing, which could also theoretically be applied to ML systems. That said, we contrast between SMART and these paradigms in Table 7 below, demonstrating that we tackle a different testing problem. Additionally, a further orthogonal area to ML model testing is that of data validation [46, 47] . In contrast, to testing an ML model for failures, data validation aims to test input data pipelines for data quality problems or drift.Step "how does the model perform'. SMART can be used to produce a comprehensive report of failures and justifications in an automated manner.Even as we zoom into the task of discovering slices where the model might underperform, we observe that SMART has features which are not supported by most of other discovery methods. We exemplify some of these features in ✗ ✗ ✗ ✗ ✔ Constant slice discovery time ✗ ✗ ✗ ✗ ✔ Performance is resistant to irrelevant data ✗ ✗ ✗ ✗ ✔ Can capture rare slices ✗ ✗ ✗ ✗ ✔ Inherently supports logical ORs ✗ ✗ ✗ ✔ ✔ Resistant to overfitting the training set ✗ ✗ ✗ ✗ ✔We present a block diagram of the key components of SMART Testing in Figure 7 . In addition, for each component we provide additional motivations and technical details not covered in the main paper. We further provide more technical information on certain implementation details. Utilizing these inputs, the LLM generates a set of hypotheses (H) and corresponding justifications (J ) regarding potential model failures. The framework also incorporates a self-refine mechanism to enhance hypothesis generation. This mechanism iteratively refines hypotheses based on the observed or provided context (C) and data (D c ), re-ranking them by their likelihood. The self-refine mechanism is introduced in order to generate hypotheses that are more likely to target specific model failures.Once the hypotheses have been proposed, it is important to operationalize them. This operationalization can be achieved through one of two methods:1. LLM-based Operationalization: Hypothesizing possible operationalizations (e.g., age > 70 as a way to operationalize "elderly people"). In this case, the previous interactions and information is provided to the LLM, together with the relevant description of the data. It is then asked to provide possible ways to operationalize a specific hypothesis. This is done by passing an "operationalization prompt" which contains the aforementioned information.The following is an example of an operationalization prompt in Python:1 f """ 2The following are hypotheses about which people within a dataset the model might underperform on .Propose specific ranges for each hypothesis . Hypotheses : { hypotheses }. TASK : Propose specific variable ranges for each hypothesis such that they are clearly operationalizable and defined . Use this format : Hypothesis : < >; Operationalization : < >.""" 11 Code Listing 2: Operationalization (LLM knowledge): General template This is then converted into a specific operationalization for each hypothesis using an external compiler which maps the strings to a function. 
 
 Appendix:  


Decision: This work introduces a method for formulating and testing hypotheses around when and why a statistical model performs poorly on a given dataset. An LLM is employed to both formulate plausible hypotheses around potential failure modes and select datapoints that are relevant to these hypotheses, followed by an optional set of statistical tests to confirm or reject generated hypotheses (referred to as self-falsification) (Sec. 4). The resulting method outperforms various prior methods that do not incorporate a broader semantic context on discovering impactful features (Sec. 5.2) and is more robust (Sec. 5.1 & 5.3).

The authors communicated extensively with the reviewers to clarify concerns, which led to most reviewers agreeing to accept the work.  Specifically, most reviewers agreed that this work provides a novel method that offers significant improvements. The main concern from the dissenting reviewer is that the proposed method requires meaningful feature names to be present in the tabular data that it operates on. While this is a valid limitation that should be noted in the work, many tabular datasets, including the six real-world datasets considered in this work (five in Sec. 5.1, one in 5.2) do have descriptive feature names. The results generally demonstrate that this technique can yield nontrivial gains in those settings. Additionally, concerns around the level of technical contribution were surfaced by the dissenting reviewer. However, the majority of reviewers express positive views on the level of technical novelty. The combination of LLM-based hypothesis generation and statistical verification is particularly effective (Tab. 2) and novel in this context.

Given this, the recommendation is to accept this work on the ground of its strong results and nontrivial technical novelty. The authors are encouraged to address all the noted concerns, including the method's reliance on meaningful feature names noted by reviewer ctad.

Reviews: review: paper_summary: paper_summary: This paper introduces context-aware testing (CAT), which is a novel tabular model testing paradigm that uses context as an inductive bias to guide the search for meaningful model failures, and build a CAT system named SMART Testing. Detailedly, SMART includes four steps: (1) use an LLM to generate hypotheses of failures based on a combination of contextual information and dataset contextualization, (2) transform natural-language-based hypotheses into a form that can directly operate on the training dataset, (3) discard “incorrect” hypotheses by testing out each hypothesis through a self-falsification mechanism, and (4) automatically generate a report of the overall performance of the model under varying hypotheses. Evaluation shows that SMART is more robust than data-only methods in terms of false positives and false negatives, and can target model failures more accurately.
strength_weakness: strength_weakness: - There is a lack of systematic comparison between the practicality of testing subgroups found by SMART and other data-only methods. While the paper includes some examples of the reports generated by SMART, it would be better for authors to provide both qualitative and quantitative comparison between SMART and data-only methods in terms of the practicality of testing subgroups.
- The tabular models included in the evaluation are outdated. In the paper, SMART testing is only applied to evaluate the performance of simple machine learning models such as Logistic Regression and SVM. It would be better to include more up-to-date deep learning models, such as transformer-based models.
questions: questions: - What is the cost of using GPT-4 to generate hypotheses in SMART and is it scalable to larger evaluation benchmarks?
- Since the idea of SMART is general, is it generalizable to other domains apart from tabular testing, such as testing NLP models?
limitations: limitations: - It would be good to discuss the potential future directions of removing biases from SMART.
paper_summary: paper_summary: The paper offers a multiple hypothesis testing view of ML evaluation (in regard to test slice
finding). Authors identify problems with data-only approaches (like high amounts of false positive
and false negative model failure triggers). The paper proposes SMART, a context-aware LLM based ML
model testing methodology to mitigate said problems. SMART uses LLMs for hypothesis generation (data
slices where model could potentially underperform). The method is empirically evaluated and compared
to a set of data-only baselines, showing less false-positives and recovering more model failure
modes.
strength_weakness: strength_weakness: 1. *It is unclear how the method degrades with less informative context*.
   - Most experiments validate the method in conditions well suited for it. e.g. easily
     distinguishible irrelevant features (could you clarify what are the feature names in the
     experiment, is it truly the case that the LLM could easily differentiate them from potentially
     useful features?)
3. *Some details of the method are missing from the main text*
   - What are the prompts (better to include some information on this in the main text, I had questions when exploring the experimental results)
   - Not sure from which subset of the data the slices are selected (it is intuitive that slices are
     subsets of the test data, but if I'm not mistaken it is not clearly defined in the text).
4. *Some related work may be missing*. I think that the subject of ML model testing in industry and production scenarios might be
   related, while it is not discussed in the paper [1, 2].
questions: questions: - Is it possible to run a method when the data schema is limited (e.g. some feature names are unknown, and the context is rather shallow in general)?
- Could you provide experiments on more real world datasets with real known model failure cases? The
  TableShift [3] benchmark could help with such an experiment as it already has subsets (OOD eval sets)
  that could serve as ground-truth problematic data slices.
- In what form the code would be available? I suggest making a package with said methodology to increase adoption.
- Are open-weights free LLMs able to generate sufficiently high quality hypothesis for the method to work?

References:
- [1] Shankar, Shreya, et al. "Automatic and precise data validation for machine learning." Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. 2023.
- [2] Polyzotis N. et al. Data validation for machine learning //Proceedings of machine learning and systems. – 2019. – Т. 1. – С. 334-347.
- [3] Gardner, Josh, Zoran Popovic, and Ludwig Schmidt. "Benchmarking distribution shift in tabular data with tableshift." Advances in Neural Information Processing Systems 36 (2024).
limitations: limitations: Limitations are adequately addressed
paper_summary: paper_summary: The paper introduces Context-Aware Testing (CAT), a new method for testing machine learning (ML) models. Current ML testing methods rely only on data, which often leads to high false positive and false negative rates and misses meaningful failures. CAT improves this by adding external knowledge or context to the testing process. The paper presents SMART Testing, the first version of CAT, which uses large language models (LLMs) to identify potential failures and a self-falsification mechanism to test them. The results show that SMART Testing finds more relevant and impactful failures than traditional data-only methods.
strength_weakness: strength_weakness: 1. The effectiveness of CAT depends on the quality and relevance of the contextual information. While context information may be available, effectively using it to generate meaningful hypotheses and justifications is not guaranteed, and there are cases where CAT may not be effective.
2. Using LLMs for hypothesis generation could introduce biases from the data used to train these models, potentially impacting the reliability of the testing results.
questions: questions: 1. How can we identify the most relevant contextual information for specific applications? How do we know that one context information is more valuable than another?
2. Are there alternative methods or models that could complement or substitute LLMs for hypothesis generation in CAT?
3. Are there any strategies to mitigate biases inherent in LLMs when using them for hypothesis generation in CAT?
4. Can the SMART Testing framework be adapted to handle non-tabular data such as images?
limitations: limitations: 1. The paper mainly concentrates on tabular data and does not investigate how CAT could apply to other data types like images.
2. The reliance on LLMs for hypothesis generation in CAT poses a constraint, particularly in scenarios where LLMs are not accessible or applicable. This dependency may restrict the method's usability and applicability across diverse settings and applications.
paper_summary: paper_summary: This paper introduces Context-Aware Testing (CAT), a novel approach that uses context as an inductive bias to guide the search for meaningful model failures. Unlike previous methods that rely solely on data to find slices where a model's predictions underperform compared to average performance, CAT addresses the limitations of these data-only approaches. Traditional methods often fail to consider the problem context and can lead to multiple testing problems. To overcome these issues, the authors propose loosening the restrictive assumption of relying solely on data. They introduce CAT as a new testing paradigm that incorporates context to provide a more comprehensive evaluation of model performance. Additionally, the paper presents SMART Testing, which employs large language models (LLMs) to hypothesize relevant and likely failures. These hypotheses are then evaluated on data using a self-falsification mechanism, allowing for the automatic identification of more relevant and impactful failures compared to traditional methods.
strength_weakness: strength_weakness: Despite the importance of the topic and motivation, it is difficult to agree that the suggested method fully addresses all the issues. Most importantly, in the first step of hypothesis generation, we need to define the contextual information and dataset contextualization. If my understanding is correct, I am quite confused about how this contextual information differs from rule-based methods defined by human experts to use contextual information to find specific data slices. For example, in Table 1, if we have contextual information about age or comorbidities, we can directly utilize that information to find data slices without using LLMs or the operationalization step. How are these different, and how are steps 1 and 2 beneficial? I feel that utilizing LLMs is quite redundant for practical use. Additionally, too much prior knowledge is needed to utilize SMART testing, raising concerns about its practicality.
questions: questions: - Can you provide examples for step 1 like in Table 1? What specifically constitutes contextual information and dataset contextualization?
- In Section 5.1, how are the descriptions for synthetic features provided in SMART testing? If the context corresponds to "it is synthetic" or "it is independent from the other," I think the results are too trivial.
- If there are no accurate descriptions related to the features or datasets, how is the method applicable?
limitations: limitations: - As mentioned in the weaknesses, the reason why LLM is needed should be elaborated. As the authors mentioned in the manuscript, using LLMs could even hinder the total procedure of SMART testing if there is no clear reason for their use.
- While the motivation is quite practical, rather than research-friendly, which is valuable, it would be beneficial to provide "practical" examples to illustrate the application of the proposed methods.

Comments: title: Thanks for the score increase & the engagement
comment: Thanks for your continued engagement. We're glad we've addressed prior concerns and reached a common understanding on some points. We clarify the remaining comments, which we hope persuades you to increase your score.

**a) Should we be using SMART without clear feature names?**

We're glad that we're in agreement that we also do not recommend using context-aware testing, and SMART specifically without feature names. While we see this reliance on the contextual nature of feature names as a benefit (by the nature of context-aware testing), we acknowledge we can be more explicit about this. We will expand the discussion section that data-only methods should still be the go-to testing approach in the _rare cases in practice_ where feature names are uninformative. 

**b) What about when feature names change?**

Regarding your OpenML benchmark example, SMART can handle variations in feature names as long as they remain informative (e.g., "age" vs "patient_age"). We have consistently demonstrated in the paper that SMART can generate meaningful hypotheses with diverse column names. Thus, changing the name would not affect the reliability. That said, we have another safety mechanism in place --- our *self-falsification mechanism* (L209-224) which uses empirical data to evaluate each hypothesis. *Finally, to clarify, in real applications in both research and industry, we find that the models are tested against a single dataset version containing informative feature names.*

**c) Is this a critical limitation in deep learning tabular problems?**

You also mention that this is a limitation with "LLMs in tabular deep learning problems". This is the first time you have raised this concern. However, this critique is simply not true. As discussed, both our framework context-aware testing and our method SMART specifically are model-agnostic (discussed in L140-162). We give both theoretical and empirical reasons for this.

- **We show theoretically why our framework is model agnostic**. We develop context-aware testing because we show that the alternative --- data-only testing--- implictly falls into the 'multiple testing problem' (Sec. 3.1.). This issue of ML testing does not depend on the underlying model. We address these theoretical issues of data-only testing methods (Sec. 3.2.) *even in deep learning models*.
- **We provide empirical results to support this claim with deep learning methods**. In addition to our original evaluation which included MLPs (**Table 12 - main paper**), we have updated with two new deep learning tabular models (**Table 4 - response pdf**).

**d) What is the novelty of our study?**

We're quite surprised to see your comment that our novelty is limited. 

In your 'summary' section of the original review, you summarized the following (taken as quotes):
- Unlike previous methods that rely solely on data to find slices where a model's predictions underperform compared to average performance, CAT addresses the limitations of these data-only approaches.
- Traditional methods often fail to consider the problem context and can lead to multiple testing problems.
- To overcome these issues, the authors propose loosening the restrictive assumption of relying solely on data

In addition, you highlighted our contributions in the strengths:
- 'the core limitation of data-only testing—the lack of a principled failure mode discovery procedure that can incorporate prior knowledge to guide the search for meaningful errors—is also persuasive.'

We are concerned that there may be some **misalignment between your original review and your current comments**.

To wrap up the discussion around our contributions, it seems that you acknowledge that we:
- **Identify a fundamental limitation in testing methods** which rely only on data (Sec. 3.1 in the paper).
- **Provide a theoretical mechanism explaining why data-only testing falls short**, relating it to the foundational view in statistics of multiple model testing in frequentist statistics. (Sec. 3.2)
- **Proposing a new paradigm which avoids the theoretical issues**: selectively testing the most relevant tests instead of testing for everything --- and falling into the multiple hypothesis testing problem again; (Sec. 3.3)
- **Building a concrete system - SMART** which is an auditable, easy-to-use, automated system as an alternative to data-only testing (Sec. 4)
- **Showing that this works empirically** on over 12 quantitative & 5 qualitative evaluations. (Sec. 5)

Therefore, we respectfully disagree with your most recent comment that our work "merely introduces an LLM to identify model failures" --- which also overlooks our clarification in our previous **Answer 3**

**e) Finding consensus**.

We hope our responses and clarifications addressed your main questions and concerns? If so, we kindly ask that you consider increasing your score to better reflect this. If not, please let us know what specific unadressed issues remain, and we'll do our utmost to resolve them.
Thank you for the reply. As the authors replied, my main point is that "SMART should not be used in settings with no feature names".
This implies that results could vary significantly when feature names are corrupted or deleted, even if the dataset and the fitted model remain exactly the same. Contrary to the authors' claim that "Feature names (e.g. column labels such as sex, age, race etc) are present in almost all tabular datasets both in the research field and in the real world where data is stored in SQL tables with column names", this is not an uncommon scenario in practical use cases. For example, in one of the largest tabular benchmarks, OpenML, it is easy to find multiple versions of the exact same datasets with different feature names.

I would like to emphasize that this is a critical limitation when introducing LLMs in tabular deep learning problems. The authors should address this issue prominently in the manuscript, and the ablation study (as conducted in the rebuttal) should be included in the paper.

Even if this critical aspect is resolved, I still have concerns about the novelty of this study, as it merely introduces an LLM to identify model failures without sophisticated design or sufficient insights.
Dear reviewer H6db,

We are glad our response was helpful and would like to thank you for raising your score! 

We really appreciate your feedback and the time you put into engaging with our work, which has helped us improve the clarity of the paper ☺ We will make changes to the revised paper to reflect our discussion. Thanks again!

Regards

Paper Authors
Thank you for your response. Your answers have addressed my concerns, and I would like to raise my score from 5 to 6.
Many thanks for the continued engagement to improve the paper. We'll put everything we've talked above into a single answer which can hopefully address this concern (such that no one is left with the impression that our paper claims "LLMs + context" magically works" ☺).

**Q1: How can the role of context in SMART be more accurately communicated?**

**A1**: To summarize and wrap up the changes we're making on context (valuable discussion -- thanks again):
- Emphasize that context-aware testing is a general framework and SMART is one implementation.
-  **Change the way we use the word context**. Clarify that "context" in CAT refers to two aspects: (i) Minimal task description (to be renamed to external input); (ii) The LLM's inductive biases ('knowledge') used for hypothesis generation
- **We rely on both context and data**. Emphasize that SMART *does not* rely solely on context, but on a combination of hypothesis generation and data-driven evaluation (which forms a significant part ouf our evaluation framework).
- **Context-aware testing avoids the multiple testing problem prevalent in data-only testing**. Link clearly the context-aware mechanism with the problem with data-only testing to better motivate this design choice, explaining that context helps to avoid the multiple hypothesis testing problem implicit in current data-only ML testing (Sec. 3.1.).
- **Highlight ablation study**. Show with an ablation study that not using data results in much worse performance --- a context-guided sampling mechanism (LLM) is not sufficient (ablation study already performed -- **Appendix D.3.**).

**Q2: How do we address the black-box nature of LLMs?**

**A2**: While we answer this concern throughout the paper in separate sections, we'll use the additional paper in the camera-ready version to summarize everything into a single paragraph:

- Using LLMs offers concrete insights into which hypotheses are tested and why (Table 1 in our paper). This is not the case for data-only methods which test for everything, resulting in the issues described in  Sec 3.1-3.2.
- The testing procedure is fully auditable via model reports (unlike data-only methods), described in Appendix D.7.
- Our *self-falsification* module uses empirical data to validate the hypotheses on actual data.


**Q3: Can we strengthen scientific insights?**

**A3**: We firmly believe that strong foundational statistical intuition which explains complicated phenomena are extremely valuable as scientific insights. That's why we employ a foundational view in statistics --- multiple hypothesis testing --- to explain for the first time: _why_ and _when_ data-only methods fail in finding ML model failures (Sections 3.1-3.2).

We'll add more statistical intuition why using a context-aware sampling mechanism (e.g. an LLM) resolves the problem of multiple hypothesis testing in ML evaluation (to add after line 165). We'll further link it to our experimental confirmation of these insights, where we indeed show that our theoretical analysis conforms with our experiments on false positives in Sec. 5.1. or false negatives in Sec. 5.2.


---
These should fully clarify the (i) meaning of context; (ii) value of context; (iii) mechanism why it works, in addition to the practical benefits.

Given the above outlined, we trust that future readers will find our work's contributions and scientific rigor to be more apparent. Thank you for engagegment and with the above in mind we hope you reassess the paper's impact on the field of ML testing---and the associated score for NeurIPS. ☺
Thank you for your efforts in developing SMART and for your thoughtful engagement in the rebuttal process.

Based on my latest understanding of the context, I believe it’s crucial to reconsider how the role of context in SMART is communicated. On one hand, overestimating the role of context could be misleading, particularly given the black-box nature of LLMs, where it’s not clear how they actually utilize contextual information. On the other hand, simply stating that "LLMs + context" magically works might understate the significance of your contributions, potentially giving the impression that the work lacks strong scientific insights.

I feel that the current version of the paper doesn’t fully address these concerns. Could you find a more balanced way to present this aspect of your work before the end of the discussion period? I would be inclined to raise my score if these issues are addressed effectively.
Dear reviewer ctad,

We thank you once more for the exchange we had. We hope it has been clarified that there is no need to provide feature descriptions, that our method is practically easy to use, and that SMART is cheap to use (testing a thousand hypotheses is cheaper than a morning espresso cup ☺️).

Have our responses and clarifications addressed your main questions and concerns? If so, we kindly ask that you consider increasing your score from to better reflect this. If not, please let us know what specific unadressed issues remain, and we'll promptly provide additional information to resolve them.

We appreciate your reconsideration and look forward to your feedback.
Dear reviewer H6db,

Thank you for the engaging exchange we had that helped to clarify our work. As the discussion period is coming to a close, we'd like to follow up on your prior concerns.

Have our responses and clarifications addressed your main questions and concerns? If so, we kindly ask that you consider increasing your score from a 5 to better reflect this. If not, please let us know what specific issues remain, and we'll promptly provide additional information to resolve them.

We appreciate your reconsideration and look forward to your feedback.
Dear reviewer 6t3M,

We are glad our response was helpful and would like to thank you for raising your score! Thanks again for your time and suggestions which have helped us to improve the paper.

Regards

Paper Authors
Thank you for your detailed rebuttal. While I still think it is important to extend SMART to scenarios beyond tabular data for more comprehensive evaluation, I would like to adjust my rating from 5 to 6.
Dear reviewer H6db,

We're happy we've addressed your concerns --- thank you for helping us improve the paper. We believe our framework as well as our practical method (SMART) will be used by researchers and practitioners who regularly test ML models. 

Given these changes, we hope you consider revising your assessment of the paper's impact and the corresponding evaluation for NeurIPS. ☺️
Thank you very much for your rebuttal. Your answers have adequately addressed my concerns.
We're happy we've clarified the word context in our conversation. Your understanding about "context" is exactly correct, and we'll change our writeup to better mimic our discussion here (thank you for this valuable discussion). 

---
**Question 1. Gap between description and SMART implementation**. Great catch --- you're entirely right and we missed this detail ourselves. We now see where the confusion comes from. 

To clarify (what you've already point out), the context **c** should not be a vector of contexts but rather a single piece of contextual information. 

While the definition of the input space for $\pi$ in line 148 is correct in that $\pi$ takes a single piece of context (a text string), this is not obvious from the description and writeup. Furthermore, we agree with you that it should be clearer that the sampling mechanism $\pi$ is a function (e.g. an LLM) that is using its inductive bias to map inputs to outputs.

**Actions taken**: (1) As stated before, we will change the word `context` to *external information* where we indeed mean external information, such as the nature of the task; (2) We will adjust the definition to not be a vector of contexts. (3) We will give more focus on the fact that the sampling mechanism $\pi$ itself is a context-aware mechanism, i.e. uses its inductive bias to generate meaningful model failures.

This discussion has been extremely fruitful. 

---
**Question 2. Clarification on hypothesis evaluation**.

When we say we sample *n* most likely hypotheses, we mean that we generate *n* hypotheses within the LLM by explicitly stating how many hypotheses and justifications to generate within the input prompt. 

However, as you correctly pointed out, this does not necessarily imply that the first hypothesis is more likely to find a model failure than the second one. At this point, the *n* generated hypotheses are just proposals. Therefore, it's not clear whether they actually contain subsets of data where the model fails the most or not.

This is why we also have steps 2-3 of our SMART method. 

Step 2: we operationalize each hypothesis (convert a textual description to a clear criterion, e.g. "elder people" -> `age > 70` (lines 200-208 in the paper). 

Step 3: self-falsification. Here, we perform hypothesis testing on a validation dataset to evaluate each hypothesis on actual data and compute the model's performance on that specific subset of the data which has been hypothesized. Therefore, for each hypothesis, we compute the difference between the aggregate performance and the performance on that subset of data.

With this, we can answer your question:

**Question:** Which criteria do we use to evaluate which hypotheses are more likely to lead to model failure than others? **Answer**: We use the performance discrepancy (e.g. difference between accuracies) between the model overall and that hypothesized slice of data on a validation dataset. Then, SMART returns the slices of data which have largest performance discrepancies. This is explained in lines 221-224.

---
**Worked out example for Question 2.**
We thought it might be useful to give a simple toy example of SMART. 

**Step 1.** Suppose SMART generates *n=3* hypotheses (adapted from Appendix D.3.):

- H1: The model will perform worse for younger people as there is insufficient data for young people
- H2: The model will perform worse for patients with multiple comorbidities due to the complexity of the disease
- H3: The model will perform worse for patients with stage 4 cancer because it might progress unpredictibly.

**Step 2.** We can operationalize each hypothesis (done automatically):
- H1: `'age < 35'`
- H2: `'comorbidities >= 2'`
- H3: `'cancer_stage == 4'`

**Step 3.** We then use these definitions to test the model on a validation dataset (done automatically). Suppose the overall accuracy on a validation dataset is 90\%. Then suppose for these subgroups, we obtain:

- H1: `'age < 35'`. Accuracy: 70%. Lower by: 20%. p-value: 0.02
- H2: `'comorbidities >= 2'`. Accuracy: 90%. Lower by: 0%. p-value: 1
- H3: `'cancer_stage == 4'`. Accuracy: 50%. Lower by: 40%. p-value: 0.00

Then, we find that people with ``cancer_stage == 4`` is the data slice with the highest performance discrepancy, followed by ``age < 35``. 

We note that this is done automatically as a part of SMART.

---
We genuinely appreciate the time you put into understanding our work and helping us improve. We hope we have addressed your concerns.
Dear Reviewer ctad

Thank you for your response. We feel there has been a major misunderstanding. Please find answers to your questions below, many of which have _already_ been answered in our earlier response or in the paper itself, which we re-clarify below.

---
**Comment 1**: Limited Applicability Without Feature Descriptions: The effectiveness of your approach is significantly limited when feature descriptions are unavailable.

**Answer 1**. There seems to be a misunderstanding, as this is *not true*. We have answered this question in **Response (D) A3** -- we do not require feature descriptions. Our inputs, apart from a one-sentence string describing the nature of the task, are equivalent to the inputs of data-only testing (despite showing superior performance). 

To put this misclarification to an end: *Our method does not rely on any additional feature descriptions*. We *do* require interpretable feature names, just as any human requires interpretable feature names to understand what they are working with. Feature names (e.g. column labels such as sex, age, race etc) are present in almost all tabular datasets both in the research field and in the real world where data is stored in SQL tables with column names. 

*This means that the utility of this method directly extends to all the users who work with such data*.

---

**Comment 2**: Cost of Utilizing LLMs: While LLMs are powerful, their use can be costly, both in terms of computation and resources. 

**Answer 2**. There seems to be a misunderstanding about the costs associated with LLMs and the computational requirements. It is *not true* that our model is costly of computationally expensive. We refer the reviewer to our **global response (and our response pdf)** where we address the exact point on cost showing SMART is cheap to use and hence very practical. For your convenience, see the snippet below

*Table 1. Cost of SMART (USD). We quantify the usage of smart in USD for different closed-source GPTs. SMART is extremely accessible and cheap to use. Generating 100 hypotheses with GPT-4o Mini costs less than 1 cent.* (GPT-4 < 0.3).

Finally, given the stark differences in the effectiveness SMART vs data-only, we think it is vital for the community (both research and industry)  to have access to improved testing methods for more reliable ML testing.

---

**Comment 3**. From a research perspective, the novelty of the approach seems constrained. The example provided in the rebuttal merely demonstrates using an LLM to identify failure cases by inputting a dataset description. 

**Answer 3**. We'd like to address the misunderstanding you got from our rebuttal example. The code snippet we provided demonstrates how easy SMART is to use and that it requires minimal prior knowledge from the user. **This is a benefit --- we want research code to be usable by practitioners**. The ease of use does not negate the technical novelty of our approach. 

To re-clarify, the core technical novelties of SMART from our paper:

- Reframing model testing as a frequentist hypothesis testing problem and giving a statistical explanation of the mechanisms why current methods fail at testing ML models
- New paradigm for ML testing: using context as an inductive bias to guide the search for meaningful model failures. 
- Showing that with context-aware testing, we can define and target task-relevant subgroups, limiting the number of tests conducted with better false positive control and greater statistical power.

It seems you have not acknowledged the reframing part of model testing as our contribution, even though it forms a significant part of our paper. 

Additionally, our model reports provide insights to practitioners about the tests.

---
**Comment 4**:  For instance, in the OpenML benchmark, datasets with the same data but different IDs can have vastly different descriptions. Can we truly trust these outcomes?

**Answer 4**. You're correct that different datasets might have different feature names. That's okay. 
- SMART doesn't rely solely on feature names. It also leverages feature names and basic statistics of the data, which are consistent across different descriptions of the same dataset.
- Self-falsification: While SMART uses descriptions to generate hypotheses, they are validated with actual data via our self-falsification mechanism. This ensures that regardless of the description, only hypotheses supported by the data are retained.

We also ensure trustworthiness via the transparency and auditability provided by the model reports.

**Comment 5**: Removing feature names hurts performance.

We agree that removing feature names would affect SMART — as we’ve shown in **Fig 1 (Response pdf)**.  This highlights that feature names play an important role in finding model failures. Hence, SMART should not be used in settings with no feature names (i.e. we provide guidance to practitioners).

---
*We hope these help clarify things.  Please let us know.*
Thank you for your efforts in clarifying your work. From what I understand, the context information in SMART is either provided through a context variable (e.g., "context = this is a loan prediction task") or implicitly inferred by the LLM from feature names. The approach using the context variable only requires providing the necessary background information of the task to the "context" variable, without the need for skillful or careful wording. I hope I have correctly understood the concept of context information in your framework.

Based on our discussion, I notice a gap between the description of context-aware testing in Section 3.3 and what is actually implemented in SMART. From line 143, it appears that context information c = {c1,…, ck} must consist of k carefully chosen context elements, and that the sampling mechanism \pi(c, D, m) (line 149) would be sensitive to the choice of c. However, it seems that c is either a simple sentence providing background information or something inferred by the LLM itself. I’m concerned that readers may not clearly grasp this distinction when reading Section 3.3.

Another gap I see between your explanation of the SMART methodology and its actual implementation is in Step 1 of Section 4. Line 197 states, “we sample the N most likely hypotheses of failures, H = {H1,…, HN}.” My concern here is how you determine that the N hypotheses selected are indeed the N most likely hypotheses of failures. Since all N hypotheses are generated by the LLM, do you require the LLM to generate more than N hypotheses and then select the most likely ones? What criteria are used to evaluate which hypotheses are more likely to lead to failures than others? In my view, the N hypotheses generated by the LLM are merely proposed hypotheses, and it’s unclear whether better ones exist in terms of their likelihood to trigger model failures.
Thank you for your detailed rebuttal. After careful consideration, I still believe that my initial assessment is accurate. The primary concern is that the proposed method seems more suited for research purposes rather than practical use, which is crucial for testing methodologies.

- Limited Applicability Without Feature Descriptions: The effectiveness of your approach is significantly limited when feature descriptions are unavailable. This restricts its utility in practical scenarios where such descriptions might not be provided.

- Cost of Utilizing LLMs: While LLMs are powerful, their use can be costly, both in terms of computation and resources. This raises concerns about the practicality of applying your method in real-world settings.

Moreover, from a research perspective, the novelty of the approach seems constrained. The example provided in the rebuttal merely demonstrates using an LLM to identify failure cases by inputting a dataset description. This does not appear to offer substantial insights for practitioners.

For instance, in the OpenML benchmark, datasets with the same data but different IDs can have vastly different descriptions. While the model fitting results would be identical, the CAT outcomes could vary significantly based on the description. Additionally, simply removing feature names could lead to different results despite the data being the same. Can we truly trust these outcomes?

I was hoping to see a response addressing these specific concerns, but I did not find a satisfactory explanation in the rebuttal.
Thank you for your engagement again.

We attempted to answer this in our response.

*Identifying relevant context: We do not require manual selection of contextual information. The relevance or value of the contextual information is automatically determined by the LLM: (a) when proposing hypotheses based on tabular data's inherent context and (b) via our self-falsification mechanism which empirically evaluates hypotheses and prunes those not supported by data.*

To address your follow-up, we answer each question in turn. We hope this helps clarify the use of contextual information.

**Q1: How do we determine if one piece of contextual information is more valuable than another?**

**A1**: In practice, there is no need to compare or determine different pieces of contextual information. We consistently find that any description of the task in free-word form is sufficient to outperform all current ML model testing practices. This is true across the seven datasets (Appendix D.2, D6), five ML models (Appendix D.4) and five different LLMs tested (Table 2 in response pdf), as well as consistent across different sample sizes (Fig 9). However, feature names that are informative are required for this (we do not recommend using SMART in case feature names are uninformative). We have shown that if the feature names are not informative (e.g. `feature_1` instead of `age`), SMART does not consider these features. We consider this a benefit by providing robustness to irrelevant features (Fig 4.)

**Q2: Is it important to identify and prioritize more valuable context, or does it not matter how precisely the context information is defined for the LLM?**

**A2**: To echo our previous response (A1), the specific wording of the task does not matter. In fact, in many cases it can be left empty if the target value is clearly encoded (e.g. ``default`` instead of``y`` for a loan default prediction task) in the name. Why is this the case? The primary benefit from our method comes from testing only a few, selected subgroups instead of exhaustively testing for everything. This requires generating a few hypotheses which we find are not sensitive to the exact wording of the ``context`` variable. For instance, changing the word from ``context = this is a loan prediction task`` to ``context = this task is about loan prediction`` does not affect its output. As per before, no other context is strictly required for our method, and the vast majority of our 12 quantitative experiments (refer to global response) do not use any more context than in the example provided (except when this is required to show some property or feature of SMART).

**Q3: Could you comment on whether the quality of the context information affects the effectiveness of your context-aware testing approach?**

**A3:** That largely depends on what is meant by "quality of the context". In case you are talking about the context variable description (e.g. ``context = 'this is a loan prediction task')`` being actively misleading, then we do not expect SMART to perform well. However, we do not recommend using this method with actively misleading prompts. If by "quality of context" you mean whether the feature names are interpretable, then we show in **Table 3 and Figure 1 in the response pdf** that SMART cannot generate meaningful hypotheses if the feature names are not interpretable. This is expected --- as the generation of meaningful hypotheses relies on having access to feature names. To give more intuition as to why the specific context does not matter, we provide concrete model reports that SMART generates (**Appendix D.7.**) which can showcases what tests were performed. Lastly, this can be used to inspect which piece of context was and was not meaningful in generating the hypotheses.

---
Fundamentally, we find that adding only minimal context in any free-form text, as long as the task description is clear, outperforms any existing ML testing method to-date. This is consistent across 12 quantitative experiments (refer to global response).

We apologize this was not clear at the beginning.

**Actions taken**: In response to your questions, we will add a broader discussion of the role context plays in the introduction and method sections.

Does this better address your questions and help to clarity the paper's contribution?
Regarding my question about how we determine that one piece of contextual information is more valuable than another: Is it important to identify and prioritize more valuable context, or does it not matter how precisely the context information is defined for the LLM? Could you comment on whether the quality of the context information affects the effectiveness of your context-aware testing approach?
Dear reviewer H6db,

Thank you for engaging with our work.

To clarify, we use 'context' in two distinct ways: (a) external input by a human (not strictly necessary but usually helpful for the framework for specifying the overall task) and (b) external knowledge (such as relevant background knowledge, domain understanding) within an external system (LLM). While we made a small footnote about this on page 5, we agree this is not enough and could have resulted in confusion about the nature of our work and will clarify this confusion.

SMART uses an LLM to generate hypotheses about what to test for, *assuming that the feature names indeed encode clear information*, such as age.

In your illustrative example where we have the same covariates but different meanings of the target label, it would be necessary to add external information/context about the nature of the task as a short input. The short description of the task is always easily available info when testing a model.

**Example when we are predicting whether to give bank loans**

```python
import SMART

# Instantiate SMART
model_tester = SMART('gpt-4') 

# Give desired context <--- need to specify this in your example
context = """Find where a model fails. This is a prediction task about whether to give bank clients bank loan or not."""

# Load ML model
model = XGBoost()
dataset_description = X.describe()

# Test the model
model_tester.fit(X, context, dataset_description, model) 
```

Notice the variable ``context`` is used as an input to SMART which provides required information about the nature of the task. In case this task is about job promotion within the upcoming five years, this would have to be changed as follows:

```context = """Find where a model fails. This is a dataset about whether people get promoted within a 5-year period. """```

You are correct that it would not be possible to identify what the task is if the target label (`y`) is not labeled in a clear manner (e.g. `5_year_job_promotion`) or if this is not given any additional context, such as the example perovided in the variable `context`. 


When we say that there is no need for manual human testing, we mean that there is no need to provide explicit information about *what to test for* and the only human input is the variable `context` which provides required information. Within SMART, we use a prompt template (provided in **Appendix C.3.**) that passes information about the context task, description of covariates, and feature names, which generates hypotheses about where the model is likely to fail which are then evaluated.

**UPDATE**: To address this confusion, we will rename the context input as external input which we hope will disambiguate the two meanings.

Does this help to clarify your question?
Thank you for your clarification. After reviewing your answer (A), I remain unclear about how the context information is generated. You mentioned that 'context is inherently encoded in the feature names' and that no external contextual information or human input is required. Does this imply that the LLM can automatically infer context purely from feature names?

Consider two identical tabular datasets with the same feature names—such as name, age, gender, occupation, and income—but with different prediction objectives: one for deciding bank loans and the other for predicting job promotions within five years. Both have binary outputs (yes or no). How would the LLM infer different context in these two different scenarios?
Dear Reviewer bfyv,

Thank you for your thoughtful consideration of our paper and your positive response to our rebuttal. 
We appreciate your recommendation for acceptance and, if you feel it is appropriate, would be grateful if you could consider whether our paper might align more closely with the criteria for a score of 7 rather than 6.

We believe our work meets the criteria for a score of 7 for these reasons:

- High Impact: Our work improves ML testing by addressing limitations found in all of the current data-only methods. More effective ML testing enabled by SMART is impactful to improve the reliability, safety, and trustworthiness of ML models across various applications and industries.

- Potential for Wide Adoption: Our implementation of SMART as an accessible Python package (mirroring your suggestion) has the potential for widespread adoption and impact within the ML community. 

- Thorough Evaluation: Our paper presents a comprehensive evaluation, including 11 quantitative and 2 qualitative experiments. We've also conducted 5 new experiments, providing additional insights into SMART's performance.

We see that these points align with the criteria for a score of 7: "Technically solid paper, with high impact on at least one sub-area of AI or moderate-to-high impact on more than one area of AI, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations."

We respectfully ask you to consider revising your assessment based on these alignments. 

Regardless of your decision, we're deeply appreciative of your support and recommendation for acceptance. Your feedback has really helped us to improve the paper!

Thank you again for your time and consideration.
Dear reviewer ctad,

We are sincerely grateful for your time and energy in the review process.

We hope that our responses have been helpful. Given the limited time left in the discussion period, please let us know if there are any leftover concerns and if there was anything else we could do to address any further questions or comments :)

Thank you!

Paper authors
Dear reviewer H6db

We are sincerely grateful for your time and energy in the review process.

We hope that our responses have been helpful. Given the limited time left in the discussion period, please let us know if there are any leftover concerns and if there was anything else we could do to address any further questions or comments :)

Thank you!

Paper authors
Dear reviewer 6t3M

We are sincerely grateful for your time and energy in the review process.

We hope that our responses have been helpful. Given the limited time left in the discussion period, please let us know if there are any leftover concerns and if there was anything else we could do to address any further questions or comments.

Thank you!

Paper authors
Thanks for the detailed response with clarifications and additional experiments. I believe my concerns and questions are adequately addressed. I intent to keep the score, as I already recommend acceptance.