SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Rare case discovery. There are 118 problems in the rare case discovery task. In our experiment, we only use 99 problems for evaluation, as we filter out the problems where the given rare cases are not "cases" of the label. For example, in rare cases "menu" and "fare" for the label "food". In this experiment, we evaluate if HiBug can identify and describe the rare case. This task is also challenging since HiBug does not know the name of potential rare cases beforehand.
In  $66.6\%$  of the problems, HiBug find the name of the rare case or a WordNet synonym [Miller, 1998]. In  $80.8\%$  of the problems, HiBug finds the name or a sub-case of the target rare case. For example, HiBug finds "latte" but the name of the rare case is "coffee". For comparison, domino recovers ground truth bug slices with a mean precision-at-10 of  $60\%$  and gives the correct description for a data slice by  $39\%$ .
4.2 What is the quality of discovered bugs?
Experiment setup. HiBug produces data slices by grouping data with similar attribute values. We evaluate the discovered data slices of HiBug in three different settings. Basic experiment settings are present in Table2. We present the running outputs of HiBug in the appendix.
Lipstick wearing classification (lipstick for short): Firstly, we consider a setting where the model predicts incorrectly because of spurious correlation on the feature. We build a ResNet18 model to classify the usage of lipstick based on the CelebA [Liu et al., 2015] dataset. The reliance of features ("male" and "female") is injected by training with biased data. Retail product classification (rpc for short): We also consider a real-life classification task proposed in [Wei et al., 2019], which is a 200-classes classification on retail products. ImageNet10 classification: Finally, we apply our methods to a ten-classification task on ImageNet [Yang et al., 2022].
Results. We define data slices with low performance in the validation set as bug slices. Figure 3 shows the number of data slices under different validation accuracy thresholds. Although these slices are identified by the model's validation accuracy, their low-performing patterns are generalizable. To confirm this, we selected data that match the attribute values of the slices outside the validation and training sets (as shown in Table 3). We then verified the model's accuracy on this data. The results demonstrate that the model's accuracy on the bug slices is significantly lower than the overall accuracy. It is also worth noting that these data slices are coherent. A visualization of the data in a slice and the description of data in Figure4.
Table 3: This table shows the generalization ability of the error slices discovered by HiBug. For example, slices in column "Model Acc.-0.1" has a validation accuracy lower than or equal to the original model's accuracy - 0.1. We select the data matching these descriptions of data slices from the data pool. The number is the model's accuracy on the selected data.
Dataset	All slices	Model Acc.	Model Acc.- 0.1	Model Acc.- 0.2
Lipstick	0.907	0.842	0.819	0.804
RPC	0.778	0.668	0.590	0.539
ImageNet10	0.873	0.849	0.823	0.774
Validation Accuracy: 0.60
Gender: Male
Hair Color: Brown
Emotion: Pensive
Validation Accuracy: 0.81
Validation Accuracy: 0.71
Figure 4: Visualization of data slices generated by HiBug.
Color: Black
Number: One
Background: Tree
With Human: No
Coarse class : Animal
Validation Accuracy: 0.76
Color: White
Background: Street
With Human: Yes
Coarse class : Vehicle
4.3 Can we improve the model with the discovered bugs? And How?
Experiment setup. In continuation of the previous experiment, we extend our evaluation to ascertain whether the attribute values of bug slices can be further employed for data selection and data generation, ultimately contributing to model enhancement.
Data selection. Following the data selection method described in section3.3, HiBug selects a certain amount of data from the unlabeled data pool within a predetermined budget. The labels of this selected data are subsequently revealed, and the model is retrained with the original training set augmented by the selected data. We employ random selection and Corset [Sener and Savarese, 2017] — a representative active learning method — as baselines for comparison. The performance of each method is evaluated based on the error rate of the retrained models on the test set. We select 8,000, 500, and 1,000 new data points for the lipstick, RPC, and ImageNet10 tasks, respectively. As demonstrated in Table 4, our method consistently surpasses the baseline performance.
Data generation. The attribute values of bug slices produced by HiBug are represented in human-
Figure 5: Image generated by class name(left) and images generated by prompts from HiBug (right).
Table 4: The model's error rate after retraining the original train set and data selected by different methods. We provide two results of the lipstick experiment – training from scratch or fine-tuning pretrained model. "Original" denotes the error rate of the model trained with original train set.
Method	Original	Random	Coreset	HiBug
Lipstick scratch	9.3%	8.9%	8.9%	8.3%
Lipstick pretrain	9.0%	8.2%	8.1%	7.1%
RPC	22.2%	9.6%	7.8%	6.0%
ImageNet10	12.7%	10.8%	10.3%	9.8%
Table 5: Evaluation of generated data. "Class name" denotes data generated by prompts with class name only. "HiBug" represents data generated by prompts with class name and attribute values. In the first row, we present the original model's error rate on the generated data. In the second row, we present the model's error rate on test set after retraining with original train set and generated images.
Number of images	1000	5000	10000
Prompt	Class name	HiBug	Class name	HiBug	Class name	HiBug
Error rate on generated data	16.5%	37.2%	16.6%	35.9%	16.6%	35.5%
Retrain error rate on test set	10.9%	10.7%	9.6%	9.1%	8.8%	8.0%
understandable terms. This facilitates downstream data generation for model enhancement, given that contemporary generative model, such as stable diffusion, also use language terms as input.
In our work, we create new data using prompts that encompass the class name (e.g., 'dog' or 'cat') and the attribute values of top low-performing data slices. For comparison, we also generate images based on prompts containing only the class name. This experiment is confined to ImageNet10 due to the limitations of the stable diffusion model in producing images satisfying the given class name in other settings.
Figure5 shows some examples of generated data. We observe that the images produced with prompts from HiBug exhibit greater diversity than those generated from class names alone. As indicated in Table5, when the model is tested on the generated data, the images created with a prompt from HiBug exhibit a higher error rate, suggesting that they represent corner cases for the model. Additionally, the model's performance improves when retrained with the combination of the generated data and the original training set. It's worth noting that a model trained with 10,000 generated data points outperforms a model trained with 1,000 selected data points from our data selection experiment. This finding indicates the potential of utilizing generated data for model improvement in future studies.
Time cost. In our experiments, we utilized an Intel(R) Xeon(R) Silver 4310 CPU @ 2.10GHz and an NVIDIA GeForce RTX 3090 GPU. The primary contributor to the computational cost in our framework, HiBug, is attribute assignment, which employs large vision-language models and scales linearly with the number of attributes. For a comprehensive study of the time cost of this component, we refer to [Li et al., 2022].
Time complexity of bug discovery. For the bug discovery of attribute combinations, we only focus on finding low-performance combinations. Its computational complexity is O(D), where D is the number of data in the dataset. Specifically, in the bug discovery process, every data has been assigned to an attribute combination. Firstly, we create a dictionary storing the number of correct data and the total number of data for every attribute combination. Then, we go through all the data in the dataset, collecting the data's correctness into the dictionary. Finally, we calculate the accuracy for each attribute combination in the dictionary while outputting those with low performance. Although the number of attribute combinations grows exponentially with the number of attributes, it is upper-bounded by the number of data in the dataset. Therefore, the time cost of this process can be viewed as 2D, which is O(D). In our experiment of bug discovery with 4536 combinations across a dataset of 20k images, the total execution time was a mere 1.06 seconds.
In summary, we present HiBug, an automated, interpretable model debugging framework that overcomes the limitations of traditional bug discovery methods by reducing human intervention and increasing bug coverage. HiBug employs pre-trained models like chatGPT to propose human-understandable attributes for specific computer vision tasks. Using vision-language models, it efficiently identifies shared visual attributes in underperforming data slices. This process aids in uncovering rare training data instances and spurious correlations, allowing for data-driven model enhancement. Experimental results underscore HiBug's effectiveness, marking a significant advancement in interpretable model debugging.
Despite the progress made in our study, certain limitations persist. When the model's task is uncommon, the pre-trained language model may fail to yield relevant visual attributes. Furthermore, during the data generation phase, the generative model may encounter difficulties in creating images that align with the class name and our attribute values. Addressing these issues will be a core focus of our future work.
Impact of Components in HiBug. In this paper, we choose BLIP and ChatGPT as the main components of HiBug. We acknowledge that while these large models have demonstrated strong ability, they can introduce bias. Please note that we rely on these models for attribute name and attribute value proposal and assignment. Even if they have some bias (e.g., they may neglect some kinds of attributes), we can still discover a lot of interpretable bugs (as evidenced by the experimental results). Data selection/generation can then help to repair the discovered bugs. However, due to the bias, some attributes remain uncovered and these bugs may not be found. Corresponding data cannot be generated for repair.
To address this problem, we provide a user interface that visualizes the entire process, which is presented in the Appendix. Users can make slight interventions to modify attributes and attribute values if necessary. Meanwhile, HiBug is highly flexible and modular. Therefore, designers can update any component in HiBug with a better counterpart.
Social Impact. In terms of potential applications and societal impacts, we want to emphasize that HiBug is rooted in industry engagement. We investigated the common debug flow in the industry and discussed it with a number of machine learning (ML) developers. The typical flow is to go through all failure cases and try to summarize failure patterns by ML developers themselves with ad-hoc solutions. After that, they collect new data related to failure patterns and re-train the model. This process requires intensive human efforts. Our original idea of HiBug is to build an AI assistant, like a co-pilot, to facilitate this process.
This work is supported by the General Research Fund of Hong Kong Research Grants Council (Grant No. 14203521), CUHK SSFCRS Project 3136023, the National Natural Science Foundation of China (Grant No. 62306093), and the Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies (Grant No. 2022B1212010005).
Greg d'Eon, Jason d'Eon, James R Wright, and Kevin Leyton-Brown. The spotlight: A general method for discovering systematic errors in deep learning models. In 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 1962-1981, 2022.
Sabri Eyuboglu, Maya Varma, Khaled Saab, Jean-Benoit Delbrouck, Christopher Lee-Messer, Jared Dunnmon, James Zou, and Christopher Ré. Domino: Discovering systematic errors with cross-modal embeddings. arXiv preprint arXiv:2203.14960, 2022.
Yang Feng, Qingkai Shi, Xinyu Gao, Jun Wan, Chunrong Fang, and Zhenyu Chen. Deepgini: prioritizing massive tests to enhance the robustness of deep neural networks. In ISSTA '20: 29th
7.1 Model repair by HiBug
We elaborate on the details of data selection methods and data generation methods of HiBug.
Data selection. Our data selection method aims to identify crucial data for model repair from the unlabeled data pool. To achieve effective repair, we require the selected samples to exhibit both high error rates and diversity.
To accomplish this objective, our first step is to identify data slices with high error rates. Given a predetermined budget and an unlabeled data pool, we assign each data point in the unlabeled pool to its respective data slice based on attribute values. However, a challenge arises because the labels for these data points are unavailable. Therefore, we turn to the validation set, which already possesses known labels. By leveraging the labels in the validation set, we can calculate the failures and the error rate associated with each data slice. This information allows us to estimate the error rates of the unlabeled data slices.
Once we have obtained the error rate for each slice, we rank all data slices according to their error rates and select those with high error rates. This process is carried out iteratively, starting with the data slice that exhibits the highest error rate. We accumulate the number of failures associated with the selected slices as the iterations progress. The iteration continues until the accumulated number of failures equals or exceeds a threshold defined by the hyper-parameter  $\beta$  multiplied by  $D_F$ . Here,  $\beta$  represents the percentage of failures we aim to address, while  $D_F$  corresponds to the total number of failures observed in the validation set.
After identifying slices with high error rates, we distribute the budget among the data slices proportionally to the number of failures they encompass. This allocation ensures that data slices with a higher number of failures receive a larger share of the budget, while also considering the diversity factor. Finally, we select unlabeled data from each data slice based on the budget allocated to that particular slice. During this selection process, we prioritize data points with lower confidence to maximize the potential for model improvement. In our experiments, we set the hyper-parameter  $\beta$  to a fixed value of 0.9.
Data generation. HiBug can generate more related data (e.g., data with failure patterns) that can further improve the model performance. For data generation, we employ a straightforward approach. Firstly, we collect the attribute values of the top 200 data slices with the highest validation error rates for each label. Next, we construct a prompt that incorporates label name and attribute variables:
$$
^ {\prime \prime} A p h o o f ^ {*} 1 ^ {*} 2 ^ {*} l a b e l ^ {*} 3, i n a ^ {*} 4 b a c k g r o u n d ^ {\prime \prime} \tag {2}
$$
During the data generation process, the variables *1, *2, *3, *4 are replaced by the values of attributes, such as "several" "black" "with a person" and "snow". The variable *label is substituted with the corresponding label name, such as "dog" or "cat."
We show the effectiveness of data generation with HiBug in ImageNet10. In this experiment, we generate a total of 10,000 data points, with 1,000 data points generated for each label. We down-sample the overall generated data to obtain results for 1,000 data points and 5,000 data points, as presented in Table 5. For comparison, we use a baseline method "Class name" that uses the following prompt, where the variable *label will be substituted with the corresponding label name during generation.
$$
" A \text {p h o t o} * \text {l a b e l .}" \tag {3}
$$
7.2 Introduction to the user interface
We offer a user interface for HiBug to facilitate more effective debugging. The user interface provides two primary functionalities. Firstly, users can easily view the distribution of attributes. For each attribute name, a histogram representing the various attribute values can be plotted, as demonstrated in Figure 6 (a). It's worth noting that we also provide additional flexibility by allowing users to input attribute names during the attribute proposal stage. This feature can be particularly useful when users
(a) Edit and display attributes
(b) Display data
Figure 6: In (a), we show that users can view the distribution of attributes by histogram (left tab), and propose or edit attributes (right tab). In (b), we show that users can view the distribution of features by scatter plot (left tab), and attributes of a specific data point in the scatter (right tab).
GPT
Figure 7: Attribute names proposed by chatGPT. The value of binary attributes are "yes" or "no".
Non-binary attribute:
1. Hair color 2. Eye color
3. Skin tone 4. Nose shape
5. Gender 6. Face shape
7. Hair texture 8. Age
9. Height 10. Body type
Binary attribute :
1. Wearing glasses 2. Wearing makeup
3. Wearing a hat 4. Has a beard
5. Wearing nail polish 6. Has tattoos
7. Has piercings 8. Wearing a watch
9. Has freckles 10. Wearing tie
Non-binary attribute:
1.Item shape 2.Item layout
3. Item size 4. Item material
5. Item color 6. Item pattern
7. Item texture 8. Number of items
9. Price range 10. Item range
1. With Barcode 2. With Label Sticker
3. With Transparent Packaging 4. With Holographic
Elements 5. With Metallic Finish 6. With Clear Window or Display? 7. With Embossed or Bordered Texture
8. With Cutout or Die-Cut Design 9. With Matte on
Glossy Surface 10. With Multi-pack or Bundle Packaging
ImageNet10
Non-binary attribute:
1. Photo background 2. Object color
3.Object size 4.Object texture
5. Number of objects 6. Object position
7. Time of day 8. Perspective
9. Lighting 10. Body type
1. Has Tail 2. Has Wings
3.Has Antlers 4. Has Fins
5. Has Feathers 6. Has Trunk
7. Has Shell 8. Has Hooves
9. Has Scales 10. Has Tusk
are confident about specific tasks and seek to enhance the debugging process. Secondly, users can examine the data distributions in the embedding space. For example, we can display and analyze specific failures, as depicted in Figure 6 (b).
7.3 Experiment details
The dcbench dataset. HiBug aims to discover the spurious correlation and rare cases that exist in the models provided by the dcbench dataset. The correlation discovery task in dcbench consists of 880 problems. Each problem contains a model under test, the model's predictions on a validation set, and labels for the validation set. Each model contains some extent of spurious correlations, which are intentionally created for testing. For example, the necklace-wearing prediction of a model might correlate with gender. With HiBug, we can discover these spurious correlations and thus reduce the bias problems in machine learning models. The rare case discovery task in dcbench contains 118 models. Each model is a binary classification model. These models are created with intentionally inserted rare case problems. For instance, in the class of vehicles, they make the subclass carriages very rare, and the model prediction on carriages can be poor. Similarly, HiBug can detect these rare case problems and provide explanations.
To summarize, the dcbench contains many models with bias (cannot make proper predictions on some subset of data), and the goal of HiBug is to detect these problems, which can lead to a better machine learning model. We shall add a comprehensive description of the dataset in the revised version.
ChatGPT attribute proposal. Figure 7 shows the attribute names proposed by ChatGPT in the three experiment settings discussed in the main paper. We also provide additional flexibility by allowing users to select attribute names for HiBug to explore. We highlight the attributes selected for our experiments.
Lipstick (train from scratch)
Rare case:
Emotion: sad
Emotion: pensive
Correlation in all prediction:
Gender: female, Label: 1
Object shape: octagon
Object layout: placed scattered
Object color: red and white
Object color: blue
Object color: red
Object color: teal
Photo background: inside a house
Photo background : buildings
Photo background: bridge
Photo background: house
Photo background : snow
Table 6: Given the same attribute list for each data, HiBug can discover more error slices with a larger validation set.
Size of validation set	15k	10k	5k	1k	0.5k
Number of error slices	288	238	168	71	49
HiBug's bug discovery outputs. Figure 8 showcases the bug discovery outputs obtained by HiBug in the three experiment settings outlined in the main paper. These outputs demonstrate the effectiveness of HiBug in identifying bugs and providing valuable insights to the user.
7.4 Further ablation studies
Size of the validation set.
We conducted additional ablation studies to explore the impact of the validation set size on HiBug's performance. To quantify the error slices, we define them as data slices with validation performance lower than or equal to the overall validation performance. As presented in Table 6, we observed that HiBug can discover a greater number of error slices when the validation set size is larger. This phenomenon can be attributed to the larger dataset covering a broader range of attribute values, leading to the identification of a higher number of error slices by HiBug.
Do bugs fixed after model repair? In the lipstick experiment, the model's predictions are correlated with the gender of the person in the data. This correlation is detected in the bug discovery process of HiBug. For comparison, we use HiBug to test the model after repairing by data selection. The result is presented in Figure 9. We observe that most bugs discovered before repair disappear. This confirms again that our model repair strategy is effective.
Before model repair, Validation ACC: 0.906
Figure 8: HiBug's bug discovery outputs on three experiment settings.
Figure 9: HiBug's outputs before and after repair.
After model repair, Validation ACC: 0.915