Model ChangeLists: Characterizing Changes in ML Prediction APIs
Authors: Sabri Eyuboglu
Karan Goel
Arjun D Desai
Lingjiao Chen
Mathew Monfort
Matei Zaharia
Christopher Re
James Zou
Venue: Submitted to ICLR 2023 (2023)
Keywords: model evaluation
model comparison
machine learning as a service
robustness

Abstract: Updates to Machine Learning as a Service (MLaaS) APIs may affect downstream systems that depend on their predictions. However, performance changes introduced by these updates are poorly documented by providers and seldom studied in the literature. As a result, users are left wondering: do model updates introduce subtle performance changes that could adversely affect my system? Ideally, users would have access to a detailed ChangeList specifying the slices of data where model performance has improved and degraded since the update. But, producing a ChangeList is challenging because it requires (1) discovering slices in the absence of detailed annotations or metadata, (2) accurately attributing coherent concepts to the discovered slices, and (3) communicating them to the user in a digestable manner. We introduce Mocha, an interactive framework for building, verifying and releasing ChangeLists that addresses these challenges. Using it, we perform a large-scale analysis of three real-world MLaaS API updates. We produce a ChangeList for each, identifying over 100 coherent data slices on which the model’s performance changed significantly. Notably, we find 63 instances where an update improves performance globally, but hurts performance on a coherent slice – a phenomenon not previously documented at scale in the literature. These findings underscore the importance of producing a detailed ChangeList when the model behind an API is updated.

Under review as a conference paper at ICLR 2023
MODEL CHANGELISTS: CHARACTERIZING CHANGES IN ML PREDICTION APIS
Anonymous authors Paper under double-blind review
ABSTRACT
Updates to Machine Learning Service (MLaaS) APIs may affect downstream systems that depend on their predictions. However, performance changes intro- duced by these updates poorly documented by providers and seldom studied in the literature. As a result, users are left wondering: do model updates intro- duce subtle performance changes that could adversely affect my system? Ide- ally. users would have access to detailed ChangeLi specifying the slices of data where model performance has improved and degraded since the update. But, producing a changeLise st is challenging because it requires (1) discovering slices in the absence of detailed annotations metadata, (2) accurately attribut- ing coherent concepts to the discovered slices, and (3) communicating them to the user digestable manner. We introduce Mocha, an interactive framework for building, verifying and releasing ChangeL. sts that addresses these challenges. Using it, we perform large-scale analysis of three real-worle MILaaS API up- dates. We produce a ChangeLi st for each, identifying over 100 coherent data slices on which the model's performance changed significantly. Notably, we find 63 instances where an update improves performance globally, but hurts perfor- mance on a coherent slice - a phenomenon not previously documented at scale the literature. These findings underscore the importance of producing a detailed ChangeList when the model behind an API is updated.
INTRODUCTION
Modern software systems often depend on Machine Learning as a Service (MLaaS) APIs developed by cloud providers (e.g. AWS, GCP, Azure) or research organizations (e.g. OpenAl, HuggingFace). The models behind these APIs are periodically updated and new versions are released. However, to the user, how a new update will affect the workings their broader system is typically unclear. Consider, for example, a newspaper that uses image tagging API to source archival photos for retrospective stories (Greenfield) 2018). Updates to the underlying model could lead to unexpected changes in the workflow of photo editors and journalists who rely the system.
MLaaS providers rarely provide transparent evaluations of their updates, and those that do focus on global metries and vague notions of improvement. Release notes from cloud providers like Amazon, Google and Microsoft for their APIs are terse and provide little information e.g. Microsoft's Vision API (Feb '22 update) noting "general performance and AI quality improvements" (Microsoft b).
These release notes tell an incomplete story: saying that one model improves on another obscures the fact that models may perform very differently fine-grained slices of data (Ribeiro al. 2020] de Vries et al.) (201)). Returning to the newspaper described above, performance after the update may improve globally, while still deteriorating on historic photos the kind of photos commonly found in the newspaper's archives. Without more detailed evaluations, users are left wondering:
Do model updates introduce subtle changes that could adversely affect my system?
While many studies include detailed comparisons of MLaaS APIs (Buolamwini Gebru 2018 Goel et al. 2021a|b b] Ribeiro et al. 2020 Qi et al. 2020), they lack comparisons of the same API before and after an update. Recent work shows that API updates can lead to performance drops on benchmarks (Chen al. 2021), but the analysis is limited simple tasks and global measurements.

---

Under review as a conference paper at ICLR 2023
horse person field
Discovery
Attrioution
Release
Nov 20
Black and white
ChangeList
Black and white
142
horse
+1.2%
Feb 22
person
Torse in Held
Horse in field
214
field
white
-7.6%
How explain model updates users?
Mocha creates ChangeLists describe changes due model updates
Overview of Mocha. (left) An MLaaS API updates and changes predictions for down- stream users; (right) Mocha is an interactive framework for building, verifying and releasing Changeli sts to explain model updates using slices of data.
Figure 1:
Answering this question would be easier if providers released detailed reports specifying the slices of data where performance has changed. We formalize this using the notion ChangeList List. Ideally, Changelist is interactive, allowing a user to explore how the model's behavior has changed on the slices most important to their system. For the example above, the newspaper should be able draw conclusions like: "the updated API detects objects in historic photos with 10% lower recall". Such conclusions would inform decisions around whether or not to integrate the update. However, producing comprehensive Changeli is difficult due to 3 main challenges:
1. For complex data like images, the set of slices that partition the data is extremely large and unknown a priori. How can we gather coherent slices that explain the change?
When interpreting slices, we typically attribute concepts (e.g. historic) to them. However, the slice was discovered automatically, it may not align perfectly with concept, leading false conclusions about performance on the concept. How do we quickly perform accurate attribution?
The number of slices with significant changes can be very large, and not all changes will be relevant to all users. How do we help users surface slices most important to their system?
To address these challenges we introduce Mocha, an interactive framework for building, verifying and releasing ChangeLists for model updates. Mocha consists of three phases:
1. Discovery:
First, we adapt recently proposed slice discovery method (Eyuboglu et al., 2022) to gather slices for the ChangeL st in Mocha. We use cross-modal embeddings and a simple mixture model identify slices of data where the models differ. Mocha also supports manual slicing over metadata, and can incorporate slices from any method of slice discovery.
2. Attribution:
Next, we ascribe concepts to the discovered slices. Via an interactive process termed micro-labeling, we verify the accuracy of the attributions and dynamically correct them. Cross-modal embeddings (e.g. CLIP) are used to guide an importance sampler (Owen) 2013) that surfaces a small number of examples for labeling. Labeled examples are then used estimate the precision and recall of the user attributions, and to update slices to reflect label feedback.
3. Release:
Finally, to help users understand model updates, we release the ChangeLis Lis in the Mocha web interface. The slices in the Changelist are indexed by modal embeddings, and are therefore easily searchable by text or image. Further, if the ChangeLiss st is missing slices important to the user, they can initiate discovery and attribution to edit the ChangeLi st.
While Mocha can be used to prepare ChangeLists for any pair of models, we focus particu- larly on demonstrating its application to the challenging real-world problem of documenting MLaaS APIs. We use Mocha to study updates to three image tagging APIs with the HAPI database (Chen) et )..) which gathers predictions on the same test examples before and after an update. We produce one ChangeLi per API update, with findings from our study of ChangeLis below:
The ChangeLists include over 100 coherent slices on which the model's performance changed significantly. These slices were annotated in the dataset and were discovered by Mocha.
There are 63 slices in the ChangeLists which API update introduced statistically significant degradation in performance. For example, between 2020 and 2022, the accuracy of a model from Google Cloud Vision the task of tagging "people" degraded by 17. 17.7%-poim for
2

---

Under review as a conference paper at ICLR 2023
black and white images. This phenomenon, where an update improves performance globally but hurts performance on a coherent slice, has not been documented at this scale in the literature.
Our findings underscore the importance of releasing ChangeLists, and recommend that providers and research organizations release detailed ChangeLists alongside model updates to help users make informed decisions around integrating updates into downstream systems.
1.1 PRIOR WORK
Evaluating MLaaS APIs. A growing number of publications include evaluations of MLaaS APls. Some evaluate single API in depth (Hosseini al. 2017). Others compare several different APIs on the same task Yao et al. 2017 Reis et al. [2018] Hosseini et al. 2019). For example, Chen et al. compare APIs from different providers and demonstrate that performance varies by class (Chen iet al. 2020). Several studies discuss significant racial disparities in the performance of MLaaS APIs (Buolamwini & Gebru, 2018 Koenecke et al., 2020). More generally, evaluation frameworks like Checklist and RobustnessCiym applied to MLaaS APIs (Ribeiro et al. 2020) Goel et al.) (021b) demonstrate an array of vulnerabilities not discemible with standard evaluations. While some of these studies compare APIs from different providers, few compare different versions of an API from the same API. Recently, Chen et al. (202T) showed that the accuracy of ML APIs sometimes changes after an update. This analysis, which most similar to our own, is limited to simple classification tasks and does not consider error consistency or slice-level differences in performance.
Comparing Machine Learning Models. Prior studies have compared machine learning models by measuring the consistency of the errors made by different image classifiers (Geirhos et al. 2020
2021 Gontijo-Lopes al., Mania et al., 2019 Fort et al. 2019). For example, Mania MMaiia et al. 2019)
measure the consistency of errors made by different ImageNet classifiers with the same accuracy, showing that error consistency is significantly higher than would be expected if predictions from different models were independent. Building on this, recent work explores how differences in model initialization and architecture affect the consistency of errors (Gontijo-Lopes et al.; Fort al. 2019)
Instead of using a fixed set of test inputs, others generate new inputs where models disagree (Li ct al. 2021 Xie et al., 2019) Pei et al.) 2017) or compare outputs of explanation methods (Jia al. 2021
MEASURING GLOBAL CHANGES IN REAL API UPDATES
We first introduce change metrics, summary statistics that describe the effect of a model update on performance. Our metrics measure (1) the performance shift due to the update and (2) the inconsis- tency of this shift across the data. We use these metries provide a new perspective in the analysis of three real API updates, which motivates our proposal to build ChargeLi st in Section 3
Preliminaries. Consider supervised learning setup where each example (X, Y) is composed of an impul E X (e.g. an image) and a target Y E y (e.g. a binary label). Assume we have a loss function (or point-wise metric) 8 y x y + R. Additionally, we have black-box access to two models trained for this task u!'l, [2]2] X' - y - e.g. these models could serve predictions for the same MLaaS API at different points in time: p(4] before an update and e! after. To compare the models, collect their predictions gا"l ():))) on dataset D ((x;, v.)) PPX,Y) Y).
Change Metries. We define two change metries in terms of D (x)) ((bb)) the difference in loss between the models,
1. Performance Shift (on average, did the model improve as a result of the update?). This metric estimates the expected difference in losses,
#e (T))""((x)))
=ED()
(1)
Positive values of μ indicate that the model improved after the update.
2. Performance Inconsistency (is the shift in performance inconsistent across the dataset?). This metric estimates the standard deviation of the difference between the losses,
of [ar (X),)) l( ()), Var[D]. Larger values of o; indicate that the models frequently disagree, and the shift is not consistent. This metric is inspired by prior consistency metrics (Geirhos al. 2020 Gontijo-Lopps et al.).
(2)
3

---

Under review as a conference paper at ICLR 2023
API Update
Global Metrics
Slice Metrics
Precision:
slices
Microsoft Computer Vislon
0.012
Nov
Recall: (1 0.045
0.166
Most improvedt
Street name
0.55
A F1: 0.069
Most Degraded:
Bathrooms
visible
0.391
Precision: & 0.004
42 slices
31 slices
Google Cloud Vision Nov 2020 + Feb 2022
μc 0.007
Recall: ( 0.027
0.172
Most improved:
Motorcycle
I::
Ft: 0.04
Most Degraded:
Beds in hotel rooms
j 4 0.674
Precision: & 0.004
47 slices
à 16 slices
EveryPixel Nov 2020 Feb 2022
μc 0.005
Recall:
0.186
Mest improved:
Cats near windows and doors [ 0.922
Most Degraded:
Sinks near cats
Iu: 4 0.758
Figure 2: Overview of Updates. We study three real-world API updates: Microsoft (a) Computer Vision (top), Google Cloud Vision (middle), and |EveyPPiel] Image Recognition (bottom). We provide background information on the update (left), global metrics computed on the entire dataset (middle, see Section 2). and summary of the slices discovered by Mocha (right, see Section 4).
Example: zero-one loss. For the special case of the zero-one loss, coi : x y - {0, 1}. we have
n Mo1 - vi) [ vil),
1 n
: P
Observe that μoi is simply the difference in accuracy between the models, while oy measures the disagreement between the models that is left over after accounting for some of the performance shift. The maximum do1 1 occurs when both models have the same accuracy but disagree everywhere.
Discussion. These metrics allow us to measure when users should be cautious in using an updated model. With positive performance shift and no inconsistency, model updates can be integrated by users safely. However, high shift inconsistency even absent performance shift is concerning, since the update may disproportionately hurt performance on data important to the user's application.
Global Analysis of APIs. We analyze global performance changes across three updates to real measuring fot change metrics alongside changes in recall, precision, and Fl-score. We briefly discuss the task, data, and APIs next (full details in Appendix (.4).
MLaaS APIs:
Task (Image Tagging). In image tagging, the input X is an image and category (e.g. horse) pair and the target Y E {0, 1} is a binary label indicating whether an object of the category is in the image.
Dataset (LVIS). We use the Large Vocabulary Instance Segmentation (LVIS) dataset, relabeling of the Common Objects in Context (COCO) images (Gupta et al. the breadth of categories output by image tagging APIs (n 1,577,603 6003 across .2003 categories).
2019 Lin et al. 2014) that reflects
APIs. We consider three object detection APIs: Microsofi Computer Vision API, Google Cloud Vision API, and EveryPixel Image Keywording Service The predictions are sourced from History of APIs (Chen et al.), a longitudinal database of API predictions. We additionally process the API outputs to map to labels in LVIS (details Appendix(A.. (A.)..
Results. From Nov '20 to Feb '22, Google and Microsoft saw accuracy shifts Moo of +0.7% and +1.2% respectively (+4.0% and +5.9% F1), while EveryPixel saw small degradation in fol of -0.5% (-2.9% Fl). However, these shifts tell only partial story: all three updates exhibit non- zero shift inconsistency (aor > 0.15). To put this into context, the predictions of the Google API (001 0.172 and ao YY 0.326) changed on 10 + % of positive examples between versions. This highlights that the API's behavior changes in ways unexplained by global change metrics.
CHARACTERIZING MODEL UPDATES IN DETAIL
These findings highlight the importance producing a more detailed, fine-grained understanding of model updates. This motivates our key proposal: the introduction of a ChangeList (Section 3.1 to explain the observed performance shift and shift inconsistency using the change in fine-graned data slices, and interactive framework called Mocha to build Changel.i st (Section 3.2)
4

---

Under review as a conference paper at ICLR 2023
3.1 CHANGELISTS: A FINI-GR CHARACTERIZATION OF MODEL UPDATES
For user, the decision on whether to use an updated model requires understanding the data exam- ples that account for performance shift and inconsistency. Users seek explanations that focus on the data important to their application. We formalize this relationship through slices of data important to users, and define ChangeLists in terms of these slices in order reflect user needs.
Slices (S). A slice is a subset of data examples that share something in common e.g. in object recognition, the set of images with dim lighting constitutes slice. Formally, we represent a slice with a random variable S {0, 1] and a set of slices with S { E {0, 13*k with joint distribution P(X, S, over inputs, targets and slices. Each example has a realization of the slice random variables {o()) ) If a = 1, then example (I,, M1) is in slice $(5), In practice, datasets do not include realizations for all possible slices, e.g. including annotations for dim lighting. Intuitively, we would like a Changelist to present users with slices alongside duman-rredatee descriptions and metrics quantifying how their performance has changed. We define these next.
Slice Attributions (A). Define random variable A E {0, 1} represent text attribution for slice (6) (e.g. "dim lighting") ") with attributions A {40)) corresponding to S and example level attribute realizations {a {a())
1, then example (ara, y1) satisfies attribution A Typically, these realizations are unknown, and only the text attribution AG) will be given.
af Ji=1- If
Slice Change Metrics (M). Given a slice St) denote change metrics My o for loss functions (5)
(5) (5)
l - by with the set of change metrics for S denoted by M [ t 3-... Mi, oz 1==1-
( G)
We are now ready to define a ChangeList using these concepts.
Definition (ChangeList e). Given dataset D and models (01,v(2) Changelist is a collec- tion of slices along with their corresponding descriptions A and change metrics M.
Given this definition, we ask: what constitutes good ChangeList? We discuss several criteria that we expect will be desired by users of ChangeLists. These desiderata are not exhaustive, and we expect more to emerge as ChangeLists are adopted into wider practice.
1. Diversity (of S). Different users have different slices of interest e.g. decorators may tag images of homes, while doctors analyzing patient behavior may tag hospital images. ChangeLists should contain a diversity of slices to reflect this.
Coverage (of og with S). The slices S should together explain the inconsistency dg. The explanatory power of S can be measured by the coefficient of determination f2 =1 - BD
f(S))²), where f(s) = + bl's is function fit by performing linear regression of D on S. Alignment (of A with S). Attributions should align with the examples in each slice, and users should be able to read these attributions to understand the content of each slice.
4. Relevance (of M). Change metrics reported in the ChangeLiss st should be chosen be rele- vant to the tasks for which the models are to be used.
5. Navigability (of S). Users of ChangeListe sts should be able to search over information in the ChangeList, including global change metrics, slices, and attributions.
6. Editability (of C). Finally, users would ideally benefit from the ability to modify a released Changelist st to meet their needs e.g. by interactively adding new slices of interest.
We emphasize that it difficult create a perfect ChangeLi st since every relevant slice cannot be anticipated ahead of time. Our interactive framework Mocha is designed to help model providers systematically build ChangeLists, while remaining flexible enough addresses these concerns.
3.2 MOCHA: AN INTERACTIVE FRAMEWORK FOR GENERATING CHANGELISTS
Mocha provides an interactive interface (Fig.) 3h for building, releasing and reading ChangeLists. The process of building ChangeLists using Mocha is split into 3 phases:
1. Discovery (Section 3.2.1. A. To discover coherent slices that explain the shift inconsistency di, adapt the Domino (Eyuboglu al., 2022) slice discovery method to our model comparison
5

---

Under review as a conference paper at ICLR 2023
a Nocha Changetiit
Siice Focua Viea
Search
Active Stice
Search Cranpies
Bext
3J6547
Sort Champlee
0.01
0.15
1.68M
Postire
reppatie
6%%%%
Con
Figure 3: A ChangeList produced using Mocha. (a) ChangeList View shows data slices where model performance has improved and degraded. Users can navigate the slices either by issuing search queries or by sorting size and performance shift. (b) Slice Focus View shows examples in the currently selected slice. During the attribution of discovered slices, it also provides rapid labeling tools needed for labeling importance weighted samples (see Section 3.2.2 The implementation of the Mocha GUT and back-end is written in Python using a common data-wrangling library.
setting. Additionally, we use Domino to generate text descriptions for each discovered slice 5(,, which serve as initial attributions A Mocha users can manually define slices using interactive tools for search, filtering and labeling, well as add slices generated by any methods or sources.
2. Attribution (Section 3.2.2). Once slices are discovered, these initial slice attributions can be up- dated using interactive slice inspection in Mocha. A key problem is estimating the alignment of slice S with its attribution A, while collecting attribute realizations {a: examples labeled by the user. Mocha proposes micro-labeling: an importance sampling proce- dure driven by CLIPP to find the most relevant examples for estimating alignment, coupled with an interface to rapidly label their attribution realizations. Slices with poor attribution alignment can also be updated to improve alignment using fast training procedure.
few informative
3. Release (Section 3.2.3).
Finally, once a ChangeList is finalized, it can be released as an interactive web application presented in the Mocha interface. Users search and sort the ChangeList by the attributions and change metrics in order to read it quickly, and can continue to edit the ChangeLi st using Mocha at any time. Mocha also provides semantic text search over slices, using CLIP to find slices with similar image prototype or text attribution embeddings.
3.2.1 DISCOVERY
Slices are often sourced from metadata extracted programmatically from the inputs (Goel et al., 2021b). When working with complex data types (e.g. images), many important slices are not anno- tated in metadata and cannot easily be extracted programmatically. The limited slices available are insufficient to explain the shift inconsistency, and we must turn to slice discovery.
Slice discovery for model comparison is the task of mining unstructured inputs X for coherent slices that explain the shift inconsistency de. Recent work has explored slice discovery frameworks for error analysis on single model (Eyuboglu et al. 2022 Singla et al.) 2021 Sohomi et al. 2020;
d'Hon et al. 2022 Yeh et al. 2020, Kim et al.). Below we adapt the Domino framework (Eyuboglu al. 2022 to our new task of explaining model differences in terms of unknown, unlabeled slices.
Domino takes as input trained models yl w X - y' and a labeled dataset D = ((x1, vi) ) P(X, Y), and outputs slicing functions V = (106) X x ) [0, 1} that partition the data into slices S == L (X,Y) E [o, 1)a. Domino proceeds in 3 steps: (1) embed the dataset, (2) slice the resulting representation space, and (3) describe the discovered slices with natural language.

---

Under review as a conference paper at ICLR 2023
Embed. We embed the dataset D using encoder Pmnut X' 2 (2 E R.), which yields embeddings Z = {2g := (x:)( for each example. Following Eyuboglu et al. (002)), we use CLIP (Radford et al. 2021), cross-modal foundation model, as encoder.
Slice. We discover slices by fitting a k-component mixture model to the embeddings Z and model losses elt), g|2), For mixture St), we assume Z ()S)) varies as multivariate Normal with diagonal covariance. The distribution of losses depends on the loss function t. In the zero-one loss case, we assume 601 (Sw) _ vary as categoricals. We then optimize the log- bog-likelihood with expec- tation maximization. Like Domino, we use a hyperparameter ry to balance the contribution of the embeddings and losses to the log-likelihood - higher y trades-off coherence for explanatory power.
Finally, to help users interpret discovered slices, we describe slices in natural language. We source candidate natural language phrases using a large, generative language model. We then identify descriptions (5)) which are closest in embedding space to the centroid of the each slice sto).
Describe.
For details on Domino, we refer the reader to Eyuboglu et al. (2022). Once the discovery phase is complete, the Mocha interface (Fig. (3) displays all discovered and user-specified slices (see Ap- pendix 1.4.1 for manual slicing), along with identified attributions and change metrics. Users can continue to perform discovery at any time in order to add additional slices to the ChangeLi ist.
3.2.2 ATTRIBUTION
The goal of the attribution phase is to help users verify and edit discovered slices, while communi- cating the contents of each slice accurately. Mocha enables users to interactively (1) edit machine attributions by inspecting examples, align them with the slice; (2) estimate aligmment between the slice and its attribution; (3) update problematic slices that are poorly aligned with their attributions.
Edit Descriptions. When users interpret discovered slices, they typically attribute succinct con- cepts to slices e.g. "subjects wearing sunglasses" if most images in a slice show a person wearing sunglasses. This attribution allows them to draw conclusions such as "the model update improved by x% accuracy on subjects wearing sunglasses". Mocha provides interactive components (Fig. 3 discussed in Section 3.2.3) to quickly inspect slices in order to edit machine-generated descriptions. After editing, each slice has a single, textual attribution.
Estimate Alignment. Next, we want to determine the alignment of the slice S with its attribution A using precision P and recall R. Mcasuring alignment lets users decide if a slice should be kept in the ChangeList, st. updated to improve alignment, or simply deleted. High precision implies that most slice examples satisfy the attribution i.e. A = 1, while with high recall, most examples that satisfy the attribution in the dataset are in the slice. Unfortunately, calculating P, R. requires exhaustively labeling the unknown attribute realizations {a=)% for each example (i.e. labeling whether each example satisfies the slice, See B.I, which is intractable to do for every slice.
Under a small labeling budget, we can only sample a few examples for labeling to estimate P, R. While we can estimate P using simple random sampling (see Appendix A.2). naively estimating recall can have high variance (Owen 2013 Kossen et al. 2021). since the number of false negatives (examples with A = outside the slice) is frequently small relative to the dataset size n.
The key problem is how construct proposal distribution q that upweights and samples "enough" false negatives to perform estimation via a procedure such as importance sampling (Owen) 2013). Our insight is to use CLIP (or any cross-modal foundation model) construct one or more proposal distributions qt. by ranking examples in terms of their similarity to the text attribution A. The advantage of using CLIP in this way that it can provide an informative ordering of the examples in response to the wide range of (arbitrarily written) user attributions. A description of our estimation procedure is provided in Appendix A.2 Once the precision and recall are estimated, the user can apply a consistent decision rule (e.g. minimum threshold on lower confidence bounds) in order to decide if the slice is satisfactory. If not satisfactory, the user can update it, which we discuss next.
Update Slices. For a slice S = (X,YY with poor alignment with its attribution A, Mocha users can update the slice by training new slicing function ('X,, Y), using logistic regression CLIP embeddings (ignoring the label Y). Ideally, to improve alignment with A, & should be aligned with A on the dataset DD i.e. is a good classifier of the attribution realizations {a: )
7

---

Under review as a conference paper at ICLR 2023
The main challenge is specifying labels for training vi, the {a,} are either unknown, partially known for previously labeled examples. We use a simple procedure to address this: use attribu- tion labels if available (optionally with additional labeling), otherwise use the original slice labels (1b y:)) This updates e conservatively by matching & where necessary, and allows Mocha to systematically improve the slice attribution alignment when a slice is updated. Note that for sta- tistical validity we ensure no overlap between examples used for training or alignment estimation.
3.2.3 INTERACTION WITH MOCHA AND RELEASING CHANGELISTS
Finally, we provide an overview of how producers and users can interact with the Mocha interface (Fig 3 and discuss the process of releasing ChangeLists in Mocha.
Interaction. Mocha contains several components to help users build and release ChangeLists,
ChangeList View. (Fig. The left panel L shows the current Changelist. The ChangeList is displayed as barplot against a chosen change metric, with the slice title and size annotated (Dc). The user can select any slice for drilldown in the Slice Focus View. Users can sort slices in the ChangeIrist st with change metries (BB), or using text-based semantic search to order slices with the most similar image prototype or slice name embedding first (QA).
Slice Focus View. (Fig. 6 The right panel 2 displays information about the slice selected in the ChangeList, including tools for performing attribution and navigating the slice. Slice summary statistics are shown along with the ability to edit its name and description (QA). The gallery (FF) enables quick inspection of slice examples, including example selection to display additional metadata. The gallery can be configured to view more or less examples at glance, and can be sorted and filtered by slice, user and task labels, or any metadata (QD, They can also be sorted by semantic similarity to a text search, implemented using CLIP (QC). During at- tribution, the corresponding component (QB) guides the user through slice updates via training, and attribute quality estimation. The user first passes through an (optional) slice update where the slicing function is retrained using user provided labels. Then, for precision (and recall) estima- tion, the gallery displays the samples to be labeled for the estimation procedure of Section 3.2.2 and provides keyboard and mouse shortcuts for rapidly selecting and labeling examples. Once estimates are calculated, they are displayed in the same attribution component.
Releasing ChangeLists. After multiple rounds of discovery and attribution, ChangeList can be released directly in the interactive Mocha web interface (Fig. 33) Users can navigate slices in released ChangeL i sts using the ChangeList View, and easily search for and drilldown into slices relevant to them. A key advantage of this method of release is that users can directly edit ChangeLists by performing additional rounds of discovery and attribution in Mocha.
DEMONSTRATING MOCHA ON REAL-WORLD API UPDATES
In this section, we discuss our takeaways from applying Mocha to three recent API updates and generating ChangeLi st for each. First, we provide overview statistics summarizing the changes documented in the ChangeLi st Next, we dive into each API detail, highlighting noteworthy changes, focusing those where slice performance goes in the opposite direction as it does globally.
Overview of ChangeLists. Mocha enabled us to rapidly discover over 100 slices across three (j) ~ E[D] and
real API updates. For each discovered slice 5ti), we compute the accuracy shift Pol test the hypothessts that the difference in accuracy D is symmetric about zero using the Wilcoxon signed-rank test. On 103 slices, we find that at least one of the API's performance changed signif- icantly (using the Bonferonni correction for multiple hypothesis testing. a 0 Among these, 63 instances of an API's performance degraded significantly and on 52, the performance degraded by more than 5%-ppinns. This phenomenon, where an update improves performance globally, but hurts performance
coherent slice, has not been documented at this scale in the literature.
In Section 2 we motivate the need for ChangeLists by showing our updates introduced incon- sistency that was not captured by the performance shift. We can quantify how much of the incon- sistency is "explained" by our slices with the coefficient of determination r². Our Change achieve quite different 22 each update: 16.7% on EveryPixel, 12.4% Google, and 5.3% on
8

---

Under review as a conference paper at ICLR 2023
Mirrosoft Computer
Google
EveryPiorl
Stap dper (131)
Trin and bus wher (368)
inter (107)
Tann and bus
Lampa how reams
Gea cungag
pepple (792)
Nepple rost
(545)
Presi autiin
Krytoari their pessle (176)
Multile ever (001)
Thaficl iggt up diss (137)
Compucens tome wort (-])
Orid play scorts (48)
High
Bross Hize (10)
tliss statppais 11300
horson using election es or
dking parann (135)
Cate knytoards (377)
Cate on bugpage (k
0.0
02
0.00
0.25
0.50
0.95
0.0
0.25
0.50 5.75
Rertroems vettlel.
Feds hotd mome
Sitte near (87)
Comruber or home wok 1-22
Step (100
cedtoort zoemed
Suffbcedds on shis and
Snonteard airoore (257)
tow brocest at the farm or 1-]
Dags in the rouse (61)
Skotcboar is boe (753-
People nox lacorrs (545)
Cads near ndows and ceers
Car near sases (540h-
Benn riditt (220))
Snewcoars mountrr
cats in fook of the TV (83:
Cars near mosaroodes 1:000-
Sapacord er laptoo with oat
050
ACOUNCC
Chonge
Figure 4: Changes discovered by Mocha. For each API update, we show the ten slices where the API performances increased (top) or decreased the most (botlom). The x-axis shows the change in accuracy. The y-axis shows the name ascribed to the slice in attribution and its size in parentheses.
Microsoft. These low ,² values highlight the difficulty of collecting a comprehensive set of slices and the importance of interactive ChangeLists that allow users to find additional slices.
The performance shift and inconsistency statistics above are defined in terms of a point-wise metric (i.e. accuracy), but also compute precision, recall and F1. Of the 63 that showed statistically significant performance shifts, 74.38% also saw changes in Fi-score greater than 5% and 61.1% saw changes in Fl greater than 10%. Comparing precision and recall also provides insight into whether performance shifts are due to an increase in false positives or false negatives. Among the statistically significant performance shifts, 76% exhibited a change in recall greater than 5% and 10.3% exhibited a change in precision greater than 5%. These statistics are summarized in Figure
ChangeList (Google Cloud Vision). Google's API is used in diverse settings ranging from historic photos classification in newspaper archives [Greenfield] 2018) to managing visual assets in cloud storage (Kus) (017). Even though the API improved on average after the update, it is important to identify fine-grained slices where performance has degraded. In Figure we show 10 slices where performance degraded. Notably, the API's accuracy in detecting stop signs decreased by over 60%-points, a finding with potential safety implications. Post update, the accuracy of the API's "person" " tag drops by 20.8%-points if the person is skiing or snowboarding. If they are playing baseball, accuracy drops by 40.9%-points, and if the photo is in black and white it is 17. 17.7%-points. This last slice may be of particular interest to a newspaper using the API on archival photos.
ChangeList (Microsoft Computer Vision). Like Google, Microsoft's 's API is used in diverse settings and backs mobile applications and other intelligent software systems (Microsoft (). Across the entire dataset, Microsoft's API improved significantly (+4.00% and +5.9% F1). However, our ChangeLi st includes 14 slices on which the performance drops significantly. For example, accu- racy in tagging "horses" degrades by more than 10%-points in old, black and white photos.
ChangeList (EveryPixel Image Recognition). Unlike the other APIs, the average model per- formance degraded slightly between updates (-0.5% accuracy and -2.9 Fl). Still, we were able to find data slices where the API improved. Notably, after the update, its "cat" detection improved across a broad set of contexts: near windows and doors, in the bathroom, and on or near keyboards, In contrast, the Microsoft API, which improved globally, exhibited significantly degraded perfor- mance on "cat" detection after the update.
CONCLUSION
MLaaS APIs are frequently updated with new versions, but providers rarely document how the API's behavior has changed. As result, it is unclear to users how this affects downstream systems that depend on the API. In this work, we present Mocha, a framework for producing a ChangeList a detailed report that highlights changes in performance using fine- fine-grained slices of data. We hope that it is a step towards a future where providers include changelogs with every model update.
9

---

Under review as a conference paper at ICLR 2023
ETHICAL CONSIDERATIONS
We highlight limitations with our proposed framework, which should be considered before use. These limitations also present potential avenues for future work. Our approach relies heavily on general-purposs pretrained models. In specialized domains (e,g, medicine, law), such models may be readily available, limiting the applicability of our work. Furthermore, these models commonly exhibit social biases. For example, CLIP has been shown to encode harmful stereotypes in its representations Agarwal et al. (02T). Because we use CLIP embeddings both identify slices and generate natural language descriptions, it is possible that Cha igeList st. may include offensive and harmful descriptions or groupings of data. As a result, it is critical that each ChangeLiss st be carefully reviewed prior to release. There is also risk that ChangeList may fail to document an important change in model behavior, potentially giving users a false sense of security. Future work should explore techniques for measuring the completeness of ChangeList.
REPRODUCIBILIIY STATEMENT
We plan release and open-source implementation of Mocha, including the ChangeLista stss that we discussed in Section 4 as well as all metadata associated with the generation of these ChangeLi sts (outputs of slice discovery, labels collected during attribution, estimation of align- ment metries).
REFERENCES
Sandhini Agarwal, Gretchen Krueger, Jack Clark, Alec Radford, Jong Wook Kim, and Miles Brundage. Eval- uating clip: towards characterization of broader capabilities and downstream implications. arXiv preprint arXiv:2108.0 02818, 2021.
Monica F Bugallo, Victor Elvira, Luca Martino, David Luengo, Joaquin Miguez, and Petar M Djuric. Adaptive importance sampling: The past, the present, and future. IEEE Signal Processing Magazine, 34(4)660-79 2017.
Joy Buolamwini and Timnit Gebru. Gender shades: classification. In Conference on fairness, accountability and transparency, 77-91. PMLR, 2018.
Intersectional accuracy disparitics in commercial gender
Lingjiao Chen, Zhihua Jin, Sabri Eyuboglu, Christopher Re, Matei Zaharia, and James Y Zou. Hapi: A large- scale longitudinal dataset of commercial ml api predictions. Advances in Neural Information Processing Systems Datasets and Benchmarks.
Lingjia Chen, Matei Zaharia, and James Y Zou. Frugalml: How to use ml prediction apis more accurately and cheaply. Advances in Neural Information Processing Systems, 33:10685- 10696, 2020.
Lingjiao Chen, Tracy Cai, Matei Zaharia, and James 7ou. Did the model change? efficiently assessing machine learning api shifts. arXiv preprint arXiv:2107. 14200, 2021.
James H Martin Daniel Jurafsky. Word senses and wordnet. In Speech and Language Processing, pp. 10. 2021.
Terrance de Vries, Ishan Misra, Changhan Wang, and Laurens van der Maaten. Does object recognition work for everyone? In Proceedings the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR) Workshops, June 2019.
Greg d' d'Eon, Jason d' d'Eon, James Wright, and Kevin Leyton-Brown. The Spotlight: A General Method for Discovering Systematic Errors in Deep Learning Models. AAAI, July 2022.
Bradley Efron and Robert Tibshirani. An introduction the bootstrap. CRC press, 1994.
EveryPixel. Everypixel (EPixel) Image Tagging APL attps:://bbo..evrry..eet com/ ap1
Sabri Eyuboglu, Maya Varma, Khaled Saab, Jean-Benoit Delbrouck, Christopher Lee-Messer, Jared Dunnmon, James Zou, and Christopher Ré. Domino: Discovering systematic errors with cross-modal embeddings. In International Conference on Learning Representations, March 2022.
Christiane Fellbaum. WordNet: An Electronic Lexical Database. Bradford Books, 1998.
10

---

Under review as a conference paper at ICLR 2023
Stanislav Fort, Huiyi Hu, and Balaji Lakshminarayanan. Deep ensembles: A loss landscape perspective. arXiv preprint arXiv:1912.027555 2019.
Robert Geirhos. Kristof Meding, and Felix A Wichmann. Beyond accuracy: quantifying trial-by ba---b-ttral behaviour of cnns and humans by measuring error consistency. Advances in Neural Information Processing Systems, 33:13890-13900 2020.
Robert Geirhos, Kantharaju Narayanappa, Benjamin Mitzkus, Tizian Thieringer, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Partial success in closing the gap between human and machine vision. Advances in Neural Information Processing Systems, 34, 2021.
Karan Goel, Laurel Orr, Nazneen Fatema Rajani, Jesse Vig, and Christopher Ré. Goodwill hunting: Analyzing and repurposing off-the-shelf named entity linking systems. In Proceedings of the 2021 Conference of the North American Chapter the Association for Computational Linguistics: Human Language Technologies: Industry Pupers, pp. 205-213,22 2021a.
Karan Goel, Nazneen Fatema Rajani, Jesse Vig. Zachary Taschdjian, Mohit Bansal, and Christopher Ré. Robustness gym: Unifying the nlp evaluation landscape. In Proceedings of the 2021 Conference of the North American Chapter the Association for Computational Linguistics: Human Language Technologies: Demonstrations, pp. 42-55, 2021b.
Raphael Gontijo-Lopes, Yann Dauphin, and Ekin D Cubuk. No one representation to rule them all: Overlapping features of training methods. International Conference Learning Representations.
Google. Google Vision APL.
comp vision
Sam
Greenfield.
Picture what the cloud can do: How the new york times using google cloud to find untold stories in
millions of archived photos.
hawp://ooo..oootttttaaaaeeeeeaae learniing how te--the-nnw--ooee times- stg.aa/ooooo.....e-eooooooooooeo -od
2018.
Agrim Gupta, Piotr Dollar, and Ross Girshick. Lvis: A dataset for large vocabulary instance segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5356-5364, 2019.
Hossein Hosseini, Baicen Xiao, and Radha Poovendran. Google's cloud vision API is not robust to noise.
In Xuewen Chen, Bo Luo, Feng Luo, Vasile Palade, and M. Arif Wani (eds.), 16th IEEE International Conference on Machine Learning and Applications, ICMLA 2017, Cancun, Mexico, December 18-21, 2017, doi: 10. N..1099/CMLA.2011 172. URL https:://oi. 1109/
pp. 101-105. IEEE, 2017.
ICMLA 2017.00122
Hossein Hosseini, Baicen Xiao, and Radha Poovendran. Studying the live cross-platform circulation of im- ages with computer vision API: An experiment based on sports media event. International Journal of Communication, 13:1825-1845 2019. 2019.
Hengrui Jia, Hongyu Chen, Jonas Guan, Ali Shahin Shamsabadi, and Nicolas Papernot. A zest of lime: Towards sichitettureiindependdee model distances. In International Conference on Learning Representations, 2021.
Michael Kim, Amirata Ghorbani, and James Zou. Multiaccuracy: Black-box post- pest-processing for faimess in classification. In A/ES 2019.
Allison Koenecke, Andrew Nam, Emily Lake, Joe Nudell, Minnie Quartey, Zion Mengesha, Connor Toups, John R Rickford, Dan Jurafsky, and Sharad Goel. Racial disparities automated speech recognition. Pro- ceedings of the National Academy of Sciences, 117(14):68847689, 2020.
Jannik Kossen, Sebastian Farquhar, Yarin Gal, and Tom Rainforth. Active testing: Sample-effcient model evaluation. In International Conference on Machine Learning, pp. 5753-5763, PMLR, 2021.
Ben Kus. Box: Bringing image recognition and ocr to cloud content management, 2017.
Yuanchun Li, Ziqi Zhang, Bingyan Liu, Ziyue Yang, and Yunxin Liu. Modeldiff: testing-hased dnn similarity comparison for model reuse detection. In Proceddings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis. pp. 139-151, 2021.
Tsung-Yi Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitmick. Microsoft COCO: common objects in context. In David Fleet, Tomás Pajdla, Bernt Schiele, and Tinne Tuytelaars (eds.), Computer Vision ECCV 2014 13th European Conference, Zurich, Switzerland, September 6- 12, 2014, Proceedings, Part V. volume 8693 of Lecture Notes in Computer Science, pp. 740-755. Springer, 2014. doi: 10.1007/978-3-319 1-0-----9---------- 1 _48. URI wtt.ttttpa... //o .org/ 10. 1000000000011 1177//98--0 3- 10602- 48

---

Under review as a conference paper at ICLR 2023
Horia Mania, John Miller, Ludwig Schmidt, Moritz Hardt, and Benjamin Recht. Model similarity mitigates test set overuse, Advances in Neural Information Processing Systems, 32, 2019.
Neil G Marchant and Benjamin IP Rubinstein. Needle in haystack: Label-efficient evaluation under extreme class imbalance. Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery Data Mining, pp. 1180-1190, 2021.
GA MeIntyre. A method for unbiased selective sampling, using ranked sets, Australian of agricultural research, 3(4):385-338 1952.
Microsoft. Microsoft computer vision API aattps://azure
coqnitive- ctoerv.ces/commutee-i111.. a. AAcessed d 2c1-2020).
Microsoft. Microsoft Release Notes.
httpss//dccs.mlersottt
cognitive- wtw.:vvwwwevvies/oeoeeeeeeeeeeee
Art B. Owen. Monte Carlo theory, methods and examples. 2013.
Van L Parsons. Stratified sampling. Wiley StatsRef: Statistics Reference Online, pp. 1-11, 2014.
Kexin Pei, Yinzhi Cao, Junfeng Yang, and Suman Jana. Deepxplore: Automated whitebox testing of deep
learning systems. In proceedings of the 26th Symposium on Operating Systems Principles, pp. 1-18, 2017.
Fait Poms, Vishnu Sarukkai, Ravi Teja Mullapudi, Nimit S Sohoni, William R Mark, Deva Ramanan, and Kayvon Fatahalian. Low-shot validation: Active importance sampling for estimating classifier performance on rare categories. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.
10705-10714. 2021.
Haode Qi, Lin Pan, Atin Sood, Abhishek Shah, Ladislav Kunc, Mo Yu, and Saloni Potdar. Benchmarking commercial intent detection services with pravtice-driven evaluations. arXiv preprint arXiv:2012.03929, 2020.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sas- try, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International Conference Machine Learning, pp. 8748 8763. PMLR, 2021.
Arsénio Reis, Dennis Paulino, Vitor Filipe, and João Barroso. Using online artificial vision services to assist the blind an assessment microsoft cognitive services and google cloud vision. In Álvaro Rocha, Hoj- jat Adeli, Luis Paulo Reis, and Sandra Costanzo (eds.). Trends and Advances in Information Syxtems and Technologies- Volume 2 [WorldCIST) 18, Naples, Italy March 27-29, 2018J. volume 746 of Advances in doi: 10.1007997833319--7722222 10. 1001/ 91779978-3222 7771122-77772222
Intelligent Systems and Computing, pp. 174-184. Springer, 2018.
URL httptht/t///.o.... wtw..rrgl1..o
Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh. Beyond accuracy: Behavioral testing of nip models with checklist. arXiv preprint arXiv:2005.0411 18. 2020.
Sahil Singla, Besmira Nushi, Shital Shah, Ece Kamar, and Eric Horvitz. Understanding failures of deep net- works via robust feature extraction. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12853-12866 2021.
Nimit Sohoni, Jared Dunnmon. Geoffrey Angus, Albert Gu, and Christopher Ré. No subclass left behind: Fine- Fine-graine grained robustness in coarse- coarse-grained classification problems. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (cds.), Advances in Neural Information Processing Systems, volume 33, pp. 19339- 19352. Curran Associates, Inc., 2020. URL https: http:ss:///ccceedneseee www.nneuriiccmm wttp:r/pppp.pppppp//e/eeep file/e0 68861 3958a1 9e08 7el7711233222 435533355555bb444 Paper.
Xiaofei Xie, Lei Ma, Haijun Wang, Yuekang Li, Yang Liu, and Xiaohong Li. Diffchaser: Detecting disagree- ments for deep neural networks. In IJCAI, pp. 5772-5778, 2019.
Yuanshun Yao, Zhujun Xiao, Bolun Wang, Bimal Viswanath, Haitao Zheng, and Ben Y. Zhao. Complexity performance: empirical analysis of machine learning a service. In Steve Uhlig and Olaf Maennel (eds.). Proceedings of the 2017 Internet Measurement Conference, IMC 2017, London, United Kingdom, November 1-3, 2017 pp. 384-397. ACM, 2017. doi: 10. 10.1145//131366531117777 URL htt.http///// http://ooooooo.oooooo 1145/3131363 3131372
httpas://dooooo//////// orgg
Chih-Kuan Yeh, Been Kim, Sercan Arik, Chun-Liang Li, Tomas Pfister, and Pradeep Ravikumar. On completeness-wware concept-based explanations in deep neural networks. Advances in Neural Information Processing Systems, 33:20554-20565, 2020.
12

---

Under review as a conference paper at ICLR 2023
A APPENDIX
A.1 DISCOVERY
In this section, provide details on the slice discovery techniques used in this work. In general, we follow closely the approach Eyuboglu et al. (2022).
We discover slices by fitting k-component mixture model the embeddings Z and model losses e For mixture 8(3), we assume Z|SO) varies multivariate Normal with diagonal covari- ance. The distribution of losses depends on the loss function f. In the zero-one loss case, we assume gfl Is(),, ( (S() vary categoricals. The log-likelihood over the validation dataset is given as follows and mazimied using inpectationmmaximizatooo
log
(3)
Like Domino, we use hyperparameter "y to balance the contribution of the embeddings and losses to the log-likelihood higher ? trades-off coherence for explanatory power. A slice is coherent the examples in it share something common. A set of slices have explanatory power if membership those slices can explain the shift inconsistency.
A.2 ATTRIBUTION: ALIGNMENT ESTIMATION
We would like to estimate the precision and recall with only small amount of labeling effort:
s;a;
s;a,
R
8
E= 4
Consider one of the slices S that was discovered in the first phase of Mocha (Section 3.2.1 Because S= (X, Y), we can compute the realizations of the slice variable {s, (1,,,, V/ ) across full dataset. On the other hand, we cannot access any of the realizations of the attributions {a:}, since they are unknown.
Estimating Precision. We estimate precision directly using the standard approach of Monte Carlo estimation with simple random sampling (SRS). To estimate precision P, we first sam- ple np examples to label with their attribution realizations {a,}, and then compute the estimator P P= E. [[;=1] We standard bootstrap procedure compute confidence interval around up
the estimated precision (Efron & Tibshirani 1994).
Estimating Recall. Unfortunately, estimating recall efficiently is difficult since the number of false negatives (i.e. examples with A 1 that lie outside the slice) be small relative the size of the dataset, making SRS an inefficient method with high variance in this setting. Beyond SRS, there are many approaches to sampling and estimation with small sample size including stratified sam- pling (Parsons) 2014), importance sampling (Owen) 2013), ranked set sampling (McIntyre) 1952) and others, as well as adaptive variants (Bugallo et al. (011). Estimating recall with limited la- bels has also recently received more attention in the machine learning community, particularly with adaptive approaches (Kossen al. 2021] Marchant & Rubinstein [201 Pomm et al. (02T))
Among these approaches, importance sampling is a strong and reliable baseline, and we leave the exploration of adaptive methods to future work. For simplicity, we reduce recall estimation two step process: (1) using mixture importance sampling (Owen) (003)) to estimate the proportion Q of examples with the attribution A 1 in the complement of the slice; and (2) using plug-in estimator for recall with the estimates for precision P and proportion Q. Formally, we define Q.
Q Σ (1 8:)a;
$t)
In the first step, we mixture importance sampling ie. simple variant of the Horvitz-Thomppe Thompson estimator that is unbiased (Owen, (2013). Key to this method is the choice the proposal distribu- tions 9i which upweight samples that are likely to be useful for estimation (Owen 2013). Indeed,
13

---

Under review as a conference paper at ICLR 2023
the key problem is how to construct proposal distributions that sample "enough" false negatives for estimation via importance sampling (Owen 2013). Good choices for the qi (i.e. those that lead to low variance estimates) would put higher weight the less prevalent samples with A 1, and lower weight on those with A = 0. At first glance, this appears impossible without labeling the attribute realizations {az)- While prior work has studied the estimation of classifier recall with lim- ited labels (Kossen et al.) 2021 Marchant & Rubinstein 2022]] Poms et al.) 202I), these all reuse the classifier being evaluated construct proposal distribution. We do not have classifier for arbitrary (user constructed) text attributions A in our setting.
Instead, we propose a procedure that relies on a flexible method to construct proposal distributions. Our insight is to use CLIP (or any cross mrosl foundation model) to construct one or more proposal distributions Te. by ranking examples
terms of their similarity to the text attribution A. The advantage of using CLIP in this way is that it can provide an informative ordering of the examples in response to the wide range of (arbitrarily written) user attributions. This turn leads to proposal distributions that are more likely to appropriately upweight samples that correspond to the concept A, which may have been arbitrarily selected by the user.
In detail, each example x, in the population is assigned a score Xı based on inner-product search with respect to text queries E [d] written by the user. Here, the user will write text queries that they think align with the attribution A. The similarity score A serves useful proxy for whether the example satisfies the attribution A, and the ranking of examples by Aay should correlate with the attribution realizations. Then, we construct proposal distribution qi from each set of scores by first min-max scaling the scores, and then powering them in order to skew the distribution i.e.
(Aey go (1z) x ( (Ae (Aiy-ming Ast))
(maxx Are mming A.x.) ) for an exponent r. This serves to create a proposal distribution that assigns very low probability examples that have the lowest scores.
Once the proposal distributions q are created, we construct a mixture distribution 1a = E. 0g with Σ, 0. = 1 (by default, we use the uniform mixture a; = J). We sample ni examples from the mixture distribution with corresponding weights wij (with wi = Σ, arwws)) and user provided at- tribution realizations aj. We can then estimate the proportion of examples Q in the slice complement with A= = 1, as well as the recall R using an (unbiased) plug-in estimator,
[|a==]]
(n n2))
My
Ei 8
R
nQ
(n 1a-na n.,)
P + n
We provide confidence intervals for recall by rumning a standard bootstrap procedure independently for both and Q, and combine these independent estimates to get bootstrapped estimates for recall. We then output the appropriate quantiles corresponding to the required confidence level.
Once the precision and recall are estimated, the user can apply a consistent decision rule (e.g. a minimum threshold on lower confidence bounds) in order decide if the slice is satisfactory. If not satisfactory, the user can update it, as discussed in Section 3.2.2
A.3 EXTENDED DESCRIPTION OF THE LONGITUDINAL DATABASE OF API PREDICTIONS
A.3.1 IMAGE RECOGNITION
Task (Image Tagging). In image tagging, the input X an image and category (e.g. horse) pair and the target Y E {0, }} is a binary label indicating whether an object in the category is in the image. We consider the point-wise zero-one loss E(y, إنر = [v j We also report other metrics that are not wise: recall, precision, and F1 Fs-ccre.
Dataset. We use the Large Vocabulary Instance Segmentation (LVIS) dataset, a relabeling of the original Common Objects in Context (COCO) images (Gupta et al. 2019 Lin et al. 2014). The dataset has 1,577,60 .603 examples. LVIS labels have two advantages over the original COCO labels. (1) LVIS includes over 1,203 categories (compared to the 80 in COCO), which better reflects the breadth of categories output by modern image tagging APIs. (2) LVIS provides negative sets, set of images for each category where no instance of the category appears. This allows us to measure both the precision and F1-score of the APIs, while still using long-tail set of categories.
14

---

Under review as a conference paper at ICLR 2023
Table 1: Estimates of precision and recall for measuring attribution alignment with slices found across all 3 API updates. Slices such as "stop signs" and "skateboard wheel" have low recall, so they may be rejected for inclusion in a final ChangeList, while all other slices have both precision and recall above 0.7.
name
count
recall
precision
Snowboard airborne Stop signs
257
0.971 (0.94, 1.00)
0.995 (0.98, 1.00)
103
0.631 (0.54, 0.76)
0.947 (0.89, 0,99)
Street name signs
277
0.789 (0.73, 0.85)
0.986 (0.97, 1.00)
Cats in the bathroom
119
0.889 (0.83, 0.95)
1.000 (1.00, 1.00)
Dogs in the house
561
0.801 (0.75, 0.85)
0.980 (0.97, 0.99)
Horses in old photos
101
0.974 (0.92, 1.00)
1.000 (1.00, 1.00)
Horses in rural settings
446
0.913 (0.87, 0.96)
0.930 (0.91, 0.96) Y
Surfboards on lakes and rivers
35
1.000 (1.00, 1.00)
1.000 (1.00. 1.00)
Surfboards in the ocean Surfboards away from water
614
0.859 (0.82, 0.92)
1.000 (1.00, 1.00)
173
0.959 (0.89, 1.00)
1.000 (1.00. 1.00)
Motorcycle wheel
134
0.795 (0.73, 0.86)
1.000 1.00)
Train and bus wheel
349
0.936 (0.89, 0.98)
1.000 (1.00, 1.00)
Airplane wheel
187
0,981 (0.96, 1.00)
1.000 (1.00, 1.00)
Skateboard wheel
186
0.663 (0.62, 0.72)
1.000 (1.00, 1.00)
Skiing person
135
0.957 (0.91, 0.99)
0.981 (0.95, 1.00)
Moohe Changetist
1A
Search Blice
Acthv
Search
to maiched...
apainal
Image piito.
Image
calig elscrrnncc
1B
Asscripilen
Sort Siice
+ C
COMMA A4
6_6847
0.176
179
1C
Sort Examptee
Incoratranng
Siice
Excrples
Acouncy
Figure 5: Mocha ChangeList View. The Mocha ChangeList panel consolidates information about the current ChangeLi st. Users can search for slices using text-based semantic queries, which match slices with the most similar image prototype or slice name (component 1A). Slices can also be ordered by associated metadata, such as change in performance or number of examples in the slice (component IB). The barplot summarizes changes in a user-selected metric across the different slices (component IC).
APIs. We consider three object detection APIs: Google's AutoML Vision Object Detection API (Google), EveryPixel's Image Keywording Service EveryPixel, and Microsoft Computer Vi- sion Image Understanding API (Microsoft. a). The predictions are sourced from History of APIs (HAPI), longitudinal database of API predictions (che). We use the raw outputs of the APIs and perform our own preprocessing that maps the labels output by the APIs to those in LVIS (see Section A.4 for details).
15

---

Under review as a conference paper at ICLR 2023
ChangeLaet
2C
Search Sliee
Active Sice
Search Ceamples
Nesial aecoooaa realing
ceST op
Filter Seamles
2D
slolk 5047
6544
0.176
179
Hew
Attibutioon
0.01
0.16
13
1.58M
Peslive
Man
Regertie Undienteer
Examplos
Figure 6: Slice Focus View. The slice focus vie enables granular inspection of different examples in the slice. Users can visualize characteristics of the selected (active) slice (component 2A) and manually label different attributes in the dataset (component 2B). Users can also search for examples that match unstructured text queries (component 2C) and filter and sort examples by existing or generated metadata (components 2D,2E), All examples are ordered in the gallery, which enables efficient data scrubbing.
Reconciling labels. The label set output by image recognition APIs will not necessarily match that of the evaluation dataset. For example, LVIS includes labels for 1,723 different object categories, while the 2020 version of the Google API output over 7, 7,462 different object categories Gupta et al. (219)). In order to evaluate an API's performance on a dataset, we must first reconcile the two category sets. If an API outputs category not in the LVIS vocabulary (e.g. "toboggan"), to map it to a more general category in the LVIS vocabulary (e.g. "sled"). To do so, we leverage the WordNet lexical database Fellbaum (1998), collecting for each category in LVIS all words with more specific meaning (i.e. its hyponyms). We find the hyponyms of a category using the following procedure:
want
1. For each category in evaluation dataset, get the corresponding WordNet synsets. (LVIS categories are already based on WordNet synsets.)
2. For each synset compute all (direct and indirect) hyponyms.
3. For each hyponym collect all of its lemmas and filter them down only include those whose most common noun word sense (based on WordNet sense ordering, see Daniel Ju- rafsky (202T) is the hyponym. This gives us mapping from each lemma list of categories.
Note: This serves to filter out improbable mappings. For example, the synset "mouse.n.02" defined as "a person who is quiet timid" and hyponym of "person". However, this leads to an odd mapping: prediction of "mouse" is mapped the category "person". To avoid this, we only consider the most common noun word sense of lemma, which for "mouse" describes rodent.
4. For each lemma select the category with the highest path similarity between its synset and the lemma's hyponym. This gives us a mapping from each lemma to a single category.
Using the resulting mapping. the set of categories used in the dataset. If predicted category is not in the mapping, we ignore it. This means that each predicted category is mapped most one dataset category. If the object
then "translate" the set of categories predicted by the API to
16

---

Under review as a conference paper at ICLR 2023
Mocta Chanpocitt
Sioe Focus Vie
Artivo
pigel
Hage
Sort Sloce
Hot cata
0_5547
Scrt Examples
Preduie
1
Ne onte ats
jeakk wwwptinappppsaaaec slpir
B
Fesmplex
20
grow 20000
Figure 7: Slice Refinement and Coherence Statisties. The attribution panel (A) provides an fast zero-one data labeling interface, which allows users to efficiently refine slices, bootstrap refiner models, and compute coherence metrics (c.g. precision, recall) on the chosen slice. Users batch select examples in the gallery (B) and select one of three options label: 1) positive, negative, 3) unlabeled (i.e. erase).
belongs in both specific and more general categories (e.g. "canoe" and "boat") the API is expected to output both the specific and general categories, This is based on the recommended evaluation approach provided by the LVIS authors Gupta et al. 2019).
A.4 EXTENDED DESCRIPTION OF McCIAA
A.4.1 MANUALLY GATHERING SLICES
Manually gathering slices is a critical process for refining outputs of slice discovery methods (SDMs) and for creating slices that were not automatically discovered. However, a manual step requires scalable data exploration, which is difficult to do with large datasets. In Mocha, users can rapidly scrub through data the gallery and label examples to assign them to the appropriate slice (Fig. Users also create their slices and label examples that part of that slice.
Mocha also leverages image-text foundation models, like CLIP, to perform similarity search be- tween image examples and text queries (Fig. (8) Similarity search can reduce the burden of having to scrub through large datasets when the attributes of interest are not labeled. Similarity searches also be used to find semantically meaningful groups of images, which can expedite manual slice discovery workflows.
17

---

Under review as a conference paper at ICLR 2023
Sioce
Active Sice
Search Coamples
Nenio using
Filter Ccamples
6_5547
0:176
179
Amribatton
0.01
0.16
13
1.68M
Puddis Repathe Uniibatee
Examples
1:0 of 2001
C CUMM 3
Figure 8: Searching for Examples. Mocha supports semantic similarity search between images and unstructured using image-text foundation models, like CLIP. Based the user search query (A), images in the selected slice (or, if no slice is selected, entire dataset) are sorted by their similarity to the query (B).
18


Reviews:
[{"review_id": "Dl2yZHlNpuz", "reviewer": "Reviewer_C4kp", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper provides an interactive framework called Mocha for an end-user to view and edit change lists.  This notion of change lists are also one of the key contributions of this paper to explain performance shift and shift inconsistency by analyzing changes in fine grained data slices. The paper leverages techniques primarily motivated from Domino (Eyuboglu et al. 2022) and discovers the slices using a mixture model. Attributions are inferred using importance sampling driven by CLIP followed by alignment specific sanity checks. The paper presents empirical results comparing production models from Microsoft and Google to view these change lists.", "summary_of_the_review": "Overall, I find this paper to be a good read in terms of providing a framework to drive value for end users. My major concerns are on (a) technical novelty of individual components and (b) user effectiveness studies to assess how viable is it to use Mocha.  Also, I feel this paper should be more well placed rather than being considered in core technical track of ICLR as it might have good value as a demo or industry track paper at ICLR or other similar top tier conferences.", "strength_and_weaknesses": "**Strengths**\n\n 1) The paper addresses an important problem of trying to understand how model updates impact different data slices. In particular, the issue of global improvement in model performance at the risk of lowered performance in certain subgroups is very important to know when deploying production model changes.\n\n2) Paper knits all components together in an interactive framework and shows results using production models from Microsoft and Google.\n\n**Weakness**\n\n1) I believe one of the paper's biggest flaws is the lack of technical novelty. While I do acknowledge that the overall framework drives value, the individual components leverage existing prior art such as Domino (Eyuboglu et al. 2022), Mixture models and importance sampling procedures making incremental technical improvements. I am also not quite sure if this paper fits in within the core technical track of a conference like ICLR. It might be a very good fit for UI, demo or Industry track.\n\n2) I also do believe that some user studies are also important to assess how user friendly Mocha is. Overall the paper does present this as a framework which can potentially overwhelm the user due to its multiple technical intricacies.  Ideally, Mocha should be useful for data scientists and business intelligence teams also and not just be applicable to core AI scientists to drive real value. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "6fbRw4to0g", "reviewer": "Reviewer_Q1WT", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a user interface for analyzing differences between ML models (ostensibly public cloud models, but it could be anything).  While the idea of segmenting model input for the purpose of model understanding is interesting, this paper is basically describing a fairly primitive GUI for the (already published, Eyuboglu et al 2022) method that does the heavy lifting.  So most of the core novelty has already been described.\n\nQuite frankly, this paper comes across as a UI paper written by a ML person, but doesn’t score well in either category.  I can’t see its relevance to ICLR.  This paper might be a better fit for a visualization conference, but even there, the concepts behind these visualizations seem relatively standard, so it is hard for me to see what contribution is being made to the visualization literature.", "summary_of_the_review": "I think the paper has low novelty and is a bad fit for ICLR.", "strength_and_weaknesses": "Strengths:\n+ Blackbox understanding of ML models is an interesting and growing problem\n+ The Domino method this builds on is interesting\n\nWeaknesses:\n- Very low novelty\n- No user studies to validate Mocha’s effectiveness\n- No comparison with existing ML explanation tools\n- Insufficient comparison of design tradeoffs \n- Even ignoring all the problems with this paper, I can’t believe that it serves a purpose in its current form.  Basically the granularity of the provided information just seems wrong.  The authors are able to cherry pick one useful example (black and white imagery classification got worse), but in general the slice labelings seem all over the place.  In practice, a real user of these APIs likely has labeled images they care about, and would generally prefer to measure the ML system’s performance on their task directly.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "3qNgInoilx", "reviewer": "Reviewer_LdES", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper tackles an interesting problem of explaining changes between two different deep models in terms of detailed, fine-grained changes in performance across different subsets of data, called slices. This is done by first identifying distinct subsets of data that correspond to some coherent concept, these subsets are then automatically annotated with natural language descriptions which the users can edit/modify. The differences between the two models in then described in terms of \"Changelists\" which is a collections of slices of data along with annotations and performance metrics on these slices. \n\n", "summary_of_the_review": "My final recommendation is a weak accept. I think the concerns regarding clarity should be addressed before acceptance. ", "strength_and_weaknesses": "Strengths:\n\n1. A very interesting problem with practical implications.\n2. Well-written paper and easy to follow modulo some aspects which are mentioned in weaknesses.\n3. Experiments on different publicly available deep learning APIs show the usefulness of the proposed approach.\n\nWeaknesses:\n1. I think the description of attribute text isn't very clear. What are the instance level attributions a_i? Are these the generated descriptions from a generative model which were deemed closest in embedding space to the centroids (as described in Section 3.2.1)? The paper mentions that users can modify this attribution. Do we have a single text attribution for each slice or multiple? I think adding some examples of attributions more specifically where it is described \n\n2. It would help to write out the loss used in discovering the slices using EM instead of just referring to Domino (perhaps in the appendix). For instance this line is unclear \"Like Domino, we use a hyperparameter γ to balance the contribution of the embeddings and losses to the log-likelihood – higher γ trades-off coherence for explanatory power.\" What does explanatory power mean? what does coherence mean?\n\n3. Similarly in the last paragraph on page 7, \"For a slice S = ψ(X, Y ) with poor alignment with its attribution A, Mocha users can update the slice by training a new slicing function ψ ̃(X, Y ), using logistic regression on CLIP embeddings.\". It is not clear what does this precisely mean. CLIP embeddings encode image X to some latent space Z, how does Y play a role? does the logistic regression classifier take as input both Z and Y? I think these details are important in fully understanding the implementation in this paper.\n\n4. The estimation of Recall for attribution and slice alignment using importance sampling isn't very clear. The paper proposes to use a mixture distribution with one q_i per user generated text-description. My main confusion is this, does the user write multiple text annotations for each attribute or we have one annotation for an attribute? This goes back to my first point. Describing the attributes more clearly is necessary for better readability of the paper in my opinion. \n\n5. The formula for \\hat{Q} is not clear. The paper claims we do not have access to instance level attribute realizations for every image in the dataset. So, importance sampling is used to get n_Q samples and then manually annotate their attribute realizations. However, the formula seems to sum over all n samples in the dataset in the numerator. Is this a typo?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "3AlOeyRvGsS", "reviewer": "Reviewer_NHrB", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper provides a tool for understanding the performance changes between different updates of ML as service applications. The developed tool is interactive and helps the users discover meaningful coherent different data slices where the performance is changing on, called ChangeLists. The model is built on CLIP and Domino, and provides a web based interface that could be very helpful for users of MLaaS to figure out if updates should be adapted based on their specific application.  ", "summary_of_the_review": "While I agree with the paper contribution being significant, I think this is more a demo paper and/or visualization paper. The novelty on the methodology side is limited and it is more a clever usage of prior methods/models (Domino/Clip). I am happy if the paper is accepted since it probably will be useful but I am not sure if it passes the novelty bar of ICLR. ", "strength_and_weaknesses": "Very well motivated and useful paper, with sound methodology, and could have impact in practice.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject