Domino: Discovering Systematic Errors with Cross-Modal Embeddings

1 INTRODUCTION . Machine learning models often make systematic errors on important subsets ( or slices ) of data1 . For instance , models trained to detect collapsed lungs in chest X-rays have been shown to make predictions based on the presence of chest drains , a device typically used during treatment ( OakdenRayner et al. , 2019 ) . As a result , these models frequently make prediction errors on cases without chest drains , a critical data slice where false negative predictions could be life-threatening . Similar performance gaps across slices have been observed in radiograph classification ( Badgeley et al. , 2019a ; Zech et al. , 2018a ; DeGrave et al. , 2021a ) , melanoma detection ( Winkler et al. , 2019a ) , natural language processing ( Orr et al. , 2020 ; Goel et al. , 2021 ) , and object detection ( de Vries et al. , 2019b ) , among others . If underperforming slices can be accurately identified and labeled , we can then improve model robustness by either updating the training dataset or using robust optimization techniques ( Zhang et al. , 2018 ; Sagawa et al. , 2020 ) . However , identifying underperforming slices is difficult in practice . When working with highdimensional inputs ( e.g . images , time-series data , video ) , slices are often “ hidden ” , meaning that they can not easily be extracted from the inputs and are not annotated in metadata ( Oakden-Rayner et al. , 2019 ) . For instance , in the collapsed lung example above , the absence of chest drains is challenging to identify from raw image data and may not be explicitly labeled in metadata . In this setting , we must perform slice discovery : the task of mining unstructured input data for semantically meaningful subgroups on which the model performs poorly . 1We define a data slice as a group of data examples united by a shared attribute or characteristic . In modern machine learning workflows , practitioners commonly attempt slice discovery with a combination of feature-based interpretability methods ( e.g . GradCAM , LIME ) and manual inspection ( Selvaraju et al. , 2017 ; Ribeiro et al. , 2016 ) . However , these approaches are time-consuming and susceptible to confirmation bias ( Adebayo et al. , 2018 ) . As a result , recent works have proposed automated slice discovery methods ( SDMs ) , which utilize learned input representations to identify semantically meaningful slices where the model makes prediction errors ( d ’ Eon et al. , 2021 ; Yeh et al. , 2020 ; Sohoni et al. , 2020 ; Kim et al. , 2018 ) . An ideal SDM should automatically identify data slices that fulfill two desiderata : ( a ) slices should contain examples on which the model underperforms , or has a high error rate and ( b ) slices should contain examples that are coherent , or align closely with a human-understandable concept . An SDM that is able to reliably satisfy these desiderata across a wide range of settings has yet to be demonstrated for two reasons : Issue 1 : No quantitative evaluation framework exists for measuring performance of SDMs with respect to these desiderata . Existing SDM evaluations are either qualitative ( d ’ Eon et al. , 2021 ) , performed on purely synthetic data ( Yeh et al. , 2020 ) , or consider only a small selection of tasks and slices ( Sohoni et al. , 2020 ) . A comprehensive evaluation framework should be quantitative , use realistic data , cover a broad range of contexts , and evaluate both underperformance and coherence . Currently , no datasets or frameworks exist to support such an evaluation , making it difficult to evaluate the tradeoffs among prior SDMs . Issue 2 : Prior qualitative evaluations have demonstrated that existing SDMs often identify slices that are incoherent . A practically useful SDM should discover coherent slices that are understandable by a domain expert . For example , in the chest x-ray setting described earlier , the slice “ patients without chest drains ” is meaningful to a physician . Slice coherence has previously been evaluated qualitatively by requiring users to manually inspect examples and identify common attributes ( d ’ Eon et al. , 2021 ; Yeh et al. , 2020 ) . Such evaluations have shown that discovered slices often do not align with concepts understandable to a domain expert . Additionally , even if slices do align well with concepts , it may be difficult for humans to identify the shared concept . Thus , an ideal SDM would not only output coherent slices , but also identify the concept connecting examples in each slice . In this work , we address both of these issues by ( 1 ) developing a framework to quantitatively evaluate the effectiveness of slice discovery methods at scale and ( 2 ) leveraging this framework to demonstrate that a powerful class of recently-developed cross-modal embeddings can be used to create an SDM that satisfies the above desiderata . Our approach – Domino – identifies coherent slices and generates automated slice descriptions . After formally describing the slice discovery problem in Section 2 , we introduce an evaluation framework for rigorously assessing SDM performance in Section 3 . We curate a set of 1235 slice discovery settings , each consisting of a real-world classification dataset , a trained model , and one or more “ ground truth ” slices corresponding to a meaningful concept in the domain . During evaluation , the SDM is provided with the dataset and the model , and we measure if the labeled slices can be successfully identified . We find that existing methods identify “ ground truth ” slices in no more than 23 % of these settings . Motivated by the recent development of large cross-modal representation learning approaches ( e.g . CLIP ) that embed inputs and text in the same latent representation space , in Section 4 we present Domino , a novel SDM that uses cross-modal embeddings to identify coherent slices . Crossmodal representations incorporate semantic meaning from text into input embeddings , which we demonstrate can improve slice coherence and enable the generation of automated slice descriptions . Domino embeds inputs alongside natural language with cross-modal representations , identifies coherent slices with an error-aware Gaussian mixture model , and generates natural language descriptions for discovered slices . In Section 5 , we use our evaluation framework to show that Domino identifies 36 % of the “ ground truth ” coherent slices across three input domains ( natural images , medical images , and time-series ) – a 12 percentage-point improvement over existing methods . 2 RELATED WORK . Slice performance gaps . Machine learning models are often limited by the presence of underperforming slices , and in Section A.1 , we provide a survey of underperforming slices reported by prior studies across a range of application domains and slice types . Oakden-Rayner et al . ( 2019 ) referred to this problem as “ hidden stratification ” and motivated the need for slice discovery techniques . Slice discovery . Prior work on slice discovery has predominantly focused on structured input datasets ( e.g . tabular data ) , where slicing can generally be performed with predicates ( e.g . nationality = Peruvian ) ( Chung et al. , 2019 ; Sagadeeva & Boehm , 2021 ) . The slice discovery problem becomes particularly complex when input data lacks explicit structure ( e.g . images , audio , etc . ) , and three recent studies present methods for performing slice discovery in this unstructured setting ( d ’ Eon et al. , 2021 ; Sohoni et al. , 2020 ; Kim et al. , 2018 ) . The proposed SDMs follow two steps : ( 1 ) embed input data in a representation space and ( 2 ) identify underperforming slices using clustering or dimensionality reduction techniques2 . These SDMs are typically evaluated by measuring performance over a limited number of slice settings or by performing qualitative assessments . The trade-offs between these SDMs have not been systematically compared , and as a result , the conditions under which these SDMs succeed at identifying coherent slices remain unclear . Benchmark datasets for machine learning robustness . Recently , several benchmark datasets have been proposed for evaluating the performance of models on dataset shifts . These benchmarks are valuable because they provide labels specifying important slices of data . However , these datasets do not suffice for systematic evaluations of SDMs because they either only annotate a small number of slices ( Koh et al. , 2021 ) or do not provide pretrained models that are known to underperform on the slices ( He et al. , 2021 ; Khosla et al. , 2011 ; Hendrycks & Dietterich , 2019 ; Liang & Zou , 2021 ) . Cross-modal embeddings . Cross-modal representation learning approaches , which embed input data and text in the same representation space , yield powerful embeddings that have contributed to large performance improvements across information retrieval and classification tasks . Often trained with contrastive learning techniques , cross-modal models generate semantically meaningful input representations that have been shown to be highly effective on zero-shot classification tasks ( Radford et al. , 2021 ) . Cross-modal models that have inspired our work include CLIP for natural images ( Radford et al. , 2021 ) , ConVIRT for medical images ( Zhang et al. , 2020 ) , and WikiSatNet ( Uzkent et al. , 2019 ) for satellite imagery . 3 SLICE DISCOVERY PRELIMINARIES . In this section , we formally define the slice discovery problem . Consider a standard classification setting with input X ∈ X ( e.g . an image , time-series , or graph ) and label Y ∈ Y = { 1 , 2 , ... , C } over C classes . Additionally , assume there exists a set of k slices that partition the data into coherent ( potentially overlapping ) subgroups , where each subgroup captures a distinct concept or attribute that would be familiar to a domain expert . For each input , we represent slice membership as S = { S ( j ) } kj=1 ∈ { 0 , 1 } k. As an example , in the collapsed lung scenario presented in Section 1 , X 2Additional implementation details are provided in Section A.2.2 . represents chest X-rays , Y is a binary label indicating the presence of collapsed lungs , and S = { S ( 1 ) , S ( 2 ) } represents two slices : one consisting of normal radiographs with chest drains and the other consisting of collapsed lungs without chest drains . The slices , inputs , and labels vary jointly according to a probability distribution P ( X , Y , S ) over X × Y × { 0 , 1 } k. We assume that training , validation and test data are drawn independently and identically from this distribution . For some application-dependent value of , a model hθ : X → Y exhibits degraded performance with respect to a slice S ( j ) and metric ` : Y × Y → R if EX , Y |S ( j ) =1 [ ` ( hθ ( X ) , Y ) ] < EX , Y |S ( j ) =0 [ ` ( hθ ( X ) , Y ) ] − . Assuming that a trained classifier hθ : X → Y exhibits degraded performance on each of the k slices in S , we define the slice discovery problem as follows : • Inputs : a trained classifier hθ and labeled datasetD = { ( xi , yi ) } ni=1 drawn from P ( X , Y ) . • Output : a set of k̂ slicing functions Ψ = { ψ ( j ) : X × Y → { 0 , 1 } } k̂j=1 that partition the data into k̂ subgroups . We consider an output to be successful if , for each ground truth slice S ( u ) , a slicing function ψ ( v ) predicts S ( u ) with precision above some threshold β : ∀u ∈ [ k ] . ∃v ∈ [ k̂ ] . P ( S ( u ) = 1|ψ ( v ) ( X , Y ) = 1 ) > β . A slice discovery method ( SDM ) , M ( D , hθ ) → Ψ , aims to solve the slice discovery problem .

Summary: Recent studies have proposed automated slice discovery methods (SDMs), which leverage learned model representations to mine input data for slices, or important subgroups of data, on which a model performs poorly.  An ideal SDM should automatically identify:  1. Slices that contain examples on which the model underperforms, or has a high error rate. 2. Slices that contain examples that are coherent, or align closely with a human-understandable concept.  This is difficult because: 1: No quantitative evaluation framework exists for measuring performance of SDMs; Existing SDM evaluations are either qualitative, performed on synthetic data, or consider only a small subset. 2. Prior qualitative evaluations have demonstrated that existing SDMs often identify slices that are incoherent, even though they may satisfy the first ideal case.  Domino: The authors preset Domino, an SDM that leverages cross-modal embeddings and a novel error-aware mixture model to discover and describe coherent slices using natural language descriptions. The proposed method could also quantitatively compare SDMs, which has not been done before.  Step 1. Embed: Encode inputs in a cross-modal embedding space with a function.  Step 2. Slice:  identify underperforming regions in the cross-modal embedding space using an error-aware mixture model fit on the input embeddings, model predictions, true class labels using expectation maximization. I.e. input embeddings, class labels, and model predictions as independent based on slice.   Step 3. Describe: Use the text embedding function ψtext learned in step (1) to generate a set of k natural language descriptions of the discovered slice.  Evaluation Approach of 3 popular slide types: Rare slice: To generate settings with rare slices, Construct a skewed dataset such that for a given class label Y, elements in subclass C occur with proportion α, where 0.01 < α < 0.1. Correlation slice: Construct a dataset such that a linear correlation α exists between the target variable and other class labels, where 0.2 < α < 0.8. Noisy label slice. Construct dataset such that for each given class label Y, the elements in subclass C exhibit label noise with probability α, where 0.01 < α < 0.3.  Experiments show that when cross-modal embeddings are provided as input, the error-aware mixture model often outperforms previously-designed SDMs.