Domino: Discovering Systematic Errors with Cross-Modal Embeddings

1 INTRODUCTION . Machine learning models often make systematic errors on important subsets ( or slices ) of data1 . For instance , models trained to detect collapsed lungs in chest X-rays have been shown to make predictions based on the presence of chest drains , a device typically used during treatment ( OakdenRayner et al. , 2019 ) . As a result , these models frequently make prediction errors on cases without chest drains , a critical data slice where false negative predictions could be life-threatening . Similar performance gaps across slices have been observed in radiograph classification ( Badgeley et al. , 2019a ; Zech et al. , 2018a ; DeGrave et al. , 2021a ) , melanoma detection ( Winkler et al. , 2019a ) , natural language processing ( Orr et al. , 2020 ; Goel et al. , 2021 ) , and object detection ( de Vries et al. , 2019b ) , among others . If underperforming slices can be accurately identified and labeled , we can then improve model robustness by either updating the training dataset or using robust optimization techniques ( Zhang et al. , 2018 ; Sagawa et al. , 2020 ) . However , identifying underperforming slices is difficult in practice . When working with highdimensional inputs ( e.g . images , time-series data , video ) , slices are often “ hidden ” , meaning that they can not easily be extracted from the inputs and are not annotated in metadata ( Oakden-Rayner et al. , 2019 ) . For instance , in the collapsed lung example above , the absence of chest drains is challenging to identify from raw image data and may not be explicitly labeled in metadata . In this setting , we must perform slice discovery : the task of mining unstructured input data for semantically meaningful subgroups on which the model performs poorly . 1We define a data slice as a group of data examples united by a shared attribute or characteristic . In modern machine learning workflows , practitioners commonly attempt slice discovery with a combination of feature-based interpretability methods ( e.g . GradCAM , LIME ) and manual inspection ( Selvaraju et al. , 2017 ; Ribeiro et al. , 2016 ) . However , these approaches are time-consuming and susceptible to confirmation bias ( Adebayo et al. , 2018 ) . As a result , recent works have proposed automated slice discovery methods ( SDMs ) , which utilize learned input representations to identify semantically meaningful slices where the model makes prediction errors ( d ’ Eon et al. , 2021 ; Yeh et al. , 2020 ; Sohoni et al. , 2020 ; Kim et al. , 2018 ) . An ideal SDM should automatically identify data slices that fulfill two desiderata : ( a ) slices should contain examples on which the model underperforms , or has a high error rate and ( b ) slices should contain examples that are coherent , or align closely with a human-understandable concept . An SDM that is able to reliably satisfy these desiderata across a wide range of settings has yet to be demonstrated for two reasons : Issue 1 : No quantitative evaluation framework exists for measuring performance of SDMs with respect to these desiderata . Existing SDM evaluations are either qualitative ( d ’ Eon et al. , 2021 ) , performed on purely synthetic data ( Yeh et al. , 2020 ) , or consider only a small selection of tasks and slices ( Sohoni et al. , 2020 ) . A comprehensive evaluation framework should be quantitative , use realistic data , cover a broad range of contexts , and evaluate both underperformance and coherence . Currently , no datasets or frameworks exist to support such an evaluation , making it difficult to evaluate the tradeoffs among prior SDMs . Issue 2 : Prior qualitative evaluations have demonstrated that existing SDMs often identify slices that are incoherent . A practically useful SDM should discover coherent slices that are understandable by a domain expert . For example , in the chest x-ray setting described earlier , the slice “ patients without chest drains ” is meaningful to a physician . Slice coherence has previously been evaluated qualitatively by requiring users to manually inspect examples and identify common attributes ( d ’ Eon et al. , 2021 ; Yeh et al. , 2020 ) . Such evaluations have shown that discovered slices often do not align with concepts understandable to a domain expert . Additionally , even if slices do align well with concepts , it may be difficult for humans to identify the shared concept . Thus , an ideal SDM would not only output coherent slices , but also identify the concept connecting examples in each slice . In this work , we address both of these issues by ( 1 ) developing a framework to quantitatively evaluate the effectiveness of slice discovery methods at scale and ( 2 ) leveraging this framework to demonstrate that a powerful class of recently-developed cross-modal embeddings can be used to create an SDM that satisfies the above desiderata . Our approach – Domino – identifies coherent slices and generates automated slice descriptions . After formally describing the slice discovery problem in Section 2 , we introduce an evaluation framework for rigorously assessing SDM performance in Section 3 . We curate a set of 1235 slice discovery settings , each consisting of a real-world classification dataset , a trained model , and one or more “ ground truth ” slices corresponding to a meaningful concept in the domain . During evaluation , the SDM is provided with the dataset and the model , and we measure if the labeled slices can be successfully identified . We find that existing methods identify “ ground truth ” slices in no more than 23 % of these settings . Motivated by the recent development of large cross-modal representation learning approaches ( e.g . CLIP ) that embed inputs and text in the same latent representation space , in Section 4 we present Domino , a novel SDM that uses cross-modal embeddings to identify coherent slices . Crossmodal representations incorporate semantic meaning from text into input embeddings , which we demonstrate can improve slice coherence and enable the generation of automated slice descriptions . Domino embeds inputs alongside natural language with cross-modal representations , identifies coherent slices with an error-aware Gaussian mixture model , and generates natural language descriptions for discovered slices . In Section 5 , we use our evaluation framework to show that Domino identifies 36 % of the “ ground truth ” coherent slices across three input domains ( natural images , medical images , and time-series ) – a 12 percentage-point improvement over existing methods . 2 RELATED WORK . Slice performance gaps . Machine learning models are often limited by the presence of underperforming slices , and in Section A.1 , we provide a survey of underperforming slices reported by prior studies across a range of application domains and slice types . Oakden-Rayner et al . ( 2019 ) referred to this problem as “ hidden stratification ” and motivated the need for slice discovery techniques . Slice discovery . Prior work on slice discovery has predominantly focused on structured input datasets ( e.g . tabular data ) , where slicing can generally be performed with predicates ( e.g . nationality = Peruvian ) ( Chung et al. , 2019 ; Sagadeeva & Boehm , 2021 ) . The slice discovery problem becomes particularly complex when input data lacks explicit structure ( e.g . images , audio , etc . ) , and three recent studies present methods for performing slice discovery in this unstructured setting ( d ’ Eon et al. , 2021 ; Sohoni et al. , 2020 ; Kim et al. , 2018 ) . The proposed SDMs follow two steps : ( 1 ) embed input data in a representation space and ( 2 ) identify underperforming slices using clustering or dimensionality reduction techniques2 . These SDMs are typically evaluated by measuring performance over a limited number of slice settings or by performing qualitative assessments . The trade-offs between these SDMs have not been systematically compared , and as a result , the conditions under which these SDMs succeed at identifying coherent slices remain unclear . Benchmark datasets for machine learning robustness . Recently , several benchmark datasets have been proposed for evaluating the performance of models on dataset shifts . These benchmarks are valuable because they provide labels specifying important slices of data . However , these datasets do not suffice for systematic evaluations of SDMs because they either only annotate a small number of slices ( Koh et al. , 2021 ) or do not provide pretrained models that are known to underperform on the slices ( He et al. , 2021 ; Khosla et al. , 2011 ; Hendrycks & Dietterich , 2019 ; Liang & Zou , 2021 ) . Cross-modal embeddings . Cross-modal representation learning approaches , which embed input data and text in the same representation space , yield powerful embeddings that have contributed to large performance improvements across information retrieval and classification tasks . Often trained with contrastive learning techniques , cross-modal models generate semantically meaningful input representations that have been shown to be highly effective on zero-shot classification tasks ( Radford et al. , 2021 ) . Cross-modal models that have inspired our work include CLIP for natural images ( Radford et al. , 2021 ) , ConVIRT for medical images ( Zhang et al. , 2020 ) , and WikiSatNet ( Uzkent et al. , 2019 ) for satellite imagery . 3 SLICE DISCOVERY PRELIMINARIES . In this section , we formally define the slice discovery problem . Consider a standard classification setting with input X ∈ X ( e.g . an image , time-series , or graph ) and label Y ∈ Y = { 1 , 2 , ... , C } over C classes . Additionally , assume there exists a set of k slices that partition the data into coherent ( potentially overlapping ) subgroups , where each subgroup captures a distinct concept or attribute that would be familiar to a domain expert . For each input , we represent slice membership as S = { S ( j ) } kj=1 ∈ { 0 , 1 } k. As an example , in the collapsed lung scenario presented in Section 1 , X 2Additional implementation details are provided in Section A.2.2 . represents chest X-rays , Y is a binary label indicating the presence of collapsed lungs , and S = { S ( 1 ) , S ( 2 ) } represents two slices : one consisting of normal radiographs with chest drains and the other consisting of collapsed lungs without chest drains . The slices , inputs , and labels vary jointly according to a probability distribution P ( X , Y , S ) over X × Y × { 0 , 1 } k. We assume that training , validation and test data are drawn independently and identically from this distribution . For some application-dependent value of , a model hθ : X → Y exhibits degraded performance with respect to a slice S ( j ) and metric ` : Y × Y → R if EX , Y |S ( j ) =1 [ ` ( hθ ( X ) , Y ) ] < EX , Y |S ( j ) =0 [ ` ( hθ ( X ) , Y ) ] − . Assuming that a trained classifier hθ : X → Y exhibits degraded performance on each of the k slices in S , we define the slice discovery problem as follows : • Inputs : a trained classifier hθ and labeled datasetD = { ( xi , yi ) } ni=1 drawn from P ( X , Y ) . • Output : a set of k̂ slicing functions Ψ = { ψ ( j ) : X × Y → { 0 , 1 } } k̂j=1 that partition the data into k̂ subgroups . We consider an output to be successful if , for each ground truth slice S ( u ) , a slicing function ψ ( v ) predicts S ( u ) with precision above some threshold β : ∀u ∈ [ k ] . ∃v ∈ [ k̂ ] . P ( S ( u ) = 1|ψ ( v ) ( X , Y ) = 1 ) > β . A slice discovery method ( SDM ) , M ( D , hθ ) → Ψ , aims to solve the slice discovery problem .

Summary: The paper propose a framework for identifying on which subsets of data machine learning models make systematic errors. The problem is cast in two parts: (1) identify a model that can be identify a subset of data and predict degraded performance of the machine learning model for this subset and (2) ensure that the identified subset is "coherent". The framework is evaluated on a number of classification tasks in computer vision and medicine.