Visual Data Diagnosis and Debiasing with Concept Graphs
Authors: Rwiddhi Chakraborty
Yinong Oliver Wang
Jialu Gao
Runkai Zheng
Cheng Zhang
Fernando De la Torre
Venue: NeurIPS 2024 poster (2024)
Keywords: responsible ai
bias
generative ai
fairness
Primary area: fairness

Abstract: The widespread success of deep learning models today is owed to the curation of extensive datasets significant in size and complexity. However, such models frequently pick up inherent biases in the data during the training process, leading to unreliable predictions. Diagnosing and debiasing datasets is thus a necessity to ensure reliable model performance. In this paper, we present ConBias, a novel framework for diagnosing and mitigating Concept co-occurrence Biases in visual datasets. ConBias represents visual datasets as knowledge graphs of concepts, enabling meticulous analysis of spurious concept co-occurrences to uncover concept imbalances across the whole dataset. Moreover, we show that by employing a novel clique-based concept balancing strategy, we can mitigate these imbalances, leading to enhanced performance on downstream tasks. Extensive experiments show that data augmentation based on a balanced concept distribution augmented by ConBias improves generalization performance across multiple datasets compared to state-of-the-art methods.

Visual Data Diagnosis and Debiasing with Concept Graphs
Rwiddhi Chakrabortyl:22 1.2- Yinong (Oliver) Wang'
Jialu Gao
Runkai Zheng'
Cheng Zhang!.
Fernando De la Torre
Carnegie Mellon University UIT The Arctic University of Norway Texas A&M University
Abstract
The widespread success of deep learning models today is owed to the curation extensive datasets sigmificant in size and complexity. However, such models frequently pick up inherent biases in the data during the training process, leading to unreliable predictions. Diagnosing and debiasing datasets is thus a necessity to ensure reliable model performance. In this paper, we present CONBTAS, novel framework for diagnosing and mitigating Concept co-occurrence Biases in visual datasets. CONBIAS represents visual datasets as knowledge graphs of concepts, enabling meticulous analysis of spurious concept co-occurrences to uncover concept imbalances across the whole dataset. Moreover, we show that by employing a novel clique-based concept balancing strategy, we can mitigate these imbalances, leading enhanced performance experiments show that data augmentation based on a balanced concept distribution augmented by CONBIAS improves generalization performance across multiple datasets compared to state-of-the-art methods."
downstream tasks. Extensive
Introduction
Over the last decade we have witnessed an unparalleled growth in the capabilities of deep learning models across wide range of tasks, such as image classification [17, 47, 71, object detection [41, 55]. semantic segmentation [20, 26, 43], and so on. More recently, with the introduction of large multi- modal models, these capabilities have improved further [25, 15]. However, such models, while demonstrating impressive performance
wide range of tasks, have been shown to be biased in their predictions [30, 13). These biases come in various forms, based in texture [14], shape [39, 32), object co- oecocurreeece [51, 52, 48], and so on. In addition to exploring model biases, dataset diagnosis, or evaluating biases directly within the dataset, particularly crucial as large datasets available today are beyond the scope of human evaluation, owing to their size and complexity. For example, ImageNet [6), widely used dataset in deep learning literature, known to have thousands of erroneous labels and lack of diversity in its class hierarchy [33, 58]. Other popular datasets such MS-COCO [23] and CelebA [27]. have problematic social biases with respect to gendered captions and prejudicial attributes of people from different races. As result, frameworks that effectively diagnose and debias these datasets are sought.
While multiple works exist in the categorization and exploration of biases in visual data [9, 90, an end-to-end pipeline incorporating both diagnosis and debiasing has received relatively scant attention. approach to
ALIA [8] is the closest and most recent work exploring such data-aagmeenttionnbaen debiasing, but it has two shortcomings first, does not diagnose the dataset which aims to debias. Without such diagnosis, it is challenging to identify the biases to be mitigated in the first place.
"Correspondence to: wwiddhi.chakrabortyeee @uit.no
"Code: https:/ggitub..om/rwwaaaaaa/
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

The Aupatto Bub prahhrraddd lclido thchers
Image Descriptions
Traditional Augmencation
Statiities
Previous Approuches
Our Approach
Tree
Waserbirds
Landbirds
Worehar
Waterbirds
Landbirds
land
Boak
land&watter
water&laadd
Original
Concept Co Co-occuteaaces
Concapt Graph
(Unbiased)
The conventional data diagnosis and augmentation pipeline begins with an original (biased) dataset. Existing methods address these biases via object frequency calibration [52]. metadata analysis [8]. or traditional augmentation techniques [60, 5J. In contrast, concepts, with orange nodes representing classes and blue nodes representing concepts, facilitating a systematic diagnosis class-concept imbalances for debiasing object co-occurrences in vision datascts.
Figure I:
framework models visual data knowledge graph of
Second, the method relies on a large language model (ChatGPT-4 [4]) to generate diverse, unbiased, in-domain descriptions. This approach is potentially confounding since there is reliable way to ensure that the biases of the large language model itself do not affect the quality of such domain descriptions. In this work, we address both these shortcomings.
We present CONBIAS, our framework for diagnosis and debiasing of visual data. Our key contribution is in representing visual dataset as knowledge graph of concepts. Analyzing this graph for imbalanced class-concept combinations leads principled diagnosis of biases present in the dataset. Once identified, we generate images to address under-represented class-concept combinations, promoting a more uniform concept distribution across classes. By using concept graph, we circumvent the reliance on a large language model to generate debiased data. Figure illustrates the core idea of our approach in contrast with existing methods. We target object co-occurrence bias, issue known confound downstream tasks [34, 10J. Object co-occurrence bias refers to any spurious correlation between label and an object causally unrelated to the label. Representing the dataset knowledge graph of object co-occurrences provides structured and controllable method to diagnose and mitigate these spurious correlations.
Our framework proceeds in three steps: (1) Concept Graph Construction: We construct a knowledge graph of concepts from the dataset. These concepts are assumed to come from dataset ground truth such as captions or segmentation masks. (2) Concept Diagnosis: This stage then analyzes the knowledge graph for concept imbalances, revealing potential biases in the original dataset. (3) Concept Debiasing: We sample imbalanced concept combinations from the knowledge graph using graph cliques, each representing class-concept combination identified as imbalanced. Finally, we generate images containing under-representte concept combinations to supplement the dataset. The image generation protocol is generic and uses an off-the-shelf inpainting process with a text- to-image generative model. This principled approach ensures that the concept distribution in augmented data is uniform and less biased. Our experiments validate this approach, showing that data augmentation based on balanced concept distribution improves generalization performance across multiple datasets compared to existing baselines. In summary, our contributions include:
We propose
concept graph-based framework to diagnose biases in visual datasets, which represents a principled approach to diagnosing datasets for biases, and to mitigating them.
Based on our graph construction and diagnosis, we propose a novel clique-based concept balancing strategy to address detected biases.
We demonstrate that balanced concept generation in data augmentation enhances classifier gener- alization across multiple datasets, over baselines.
2
Related Work
Bias discovery in deep learning models. The identification of biases in trained deep learning models has a rich history, with early works exploring the texture and shape-bias tradeoff in ImageNet-

---

pretrained ResNets [14. 32, 39]. More recently, the field of worst group robustness has emerged, aiming to generalize classifier performance across multiple groups in the data that correspond to known spurious correlations [49, 45, 24, 42]. Debiasing and concept discovery in the feature space of the learned classifier is also common [1, 54, 59]. Testing model performance sensitivity to the presence of particular attributes has also been explored [53, 36]. With the recent rise in popularity of large language models, efforts have been made to identify learned biases using off-the-shelf captioning models [56], adaptive testing [11], and language guidance [19, 37]. Traditional data augmentation approaches such as CutMix [60], and RandAug [5], are used as baselines as well. Our work intervenes on the dataset directly, instead of operating in the model feature space testing
model sensitivity. This allows for a more intuitive and principled approach to bias discovery.
Data diagnosis. Our work is placed in the context of data diagnosis, i.e. identifying biases directly from the data without using the model as a proxy. One of the early influential works expounding the importance of datasets in deep learning research was systematic review of the popular datasets in computer vision [51]. A modern appraisal categorizing more diverse types of biases in visual datasets
exists in [9]. Additionally, works investigating possible issues with dataset labels have also received
interest [33, 58J. Data diagnosis tools such as REVISE [52] compute object statistics (including co-occurrence) to generate high-level insights of the data. However, REVISE is not an end-to-end framework that at once diagnoses and debiases data. It is rather an exploratory tool for an overview of common concepts in the dataset. A more recent method, ALIA, uses a language model populate diverse descriptions of the given dataset, consequently generating images from such descriptions. A more critical look on dataset bias lies in the field of fairness, particularly with regards to societal bias [12, 16]. Finally, benchmark datasets for data diagnosis have also been proposed [29, 28].
Object co-occurrence bias in visual recognition. Objects are biased in the company they keep. This adage well known in the computer vision literature, as outlined in [34, 10]. Modern efforts to mitigate object co-occurrence bias involve feature decorrelation [48], object aware contrastive representations [31], causal interventions [38], and fusing object and contextual information via attention [2]. The common theme in tackling contextual and co-occurrence bias lies entirely in using better models (feature representations) rather than intervening in the dataset directly. We place our debiasing method along the data augmentation direction, allowing for better controllability and interpretability of the debiasing stage, rather than relying on semantic features learned by a classifier, which may be difficult for humans to interpret.
3 Approach
Figure 2 illustrates the overall pipeline of our method. In this section, we begin with the problem statement in Section 1, and move to the three major stages in our method definition. Section 3.2 describes the procedure of concept graph construction. Section 3.3 illustrates the details of concept diagnosis. Finally, Section 3.4 presents our method for concept debiasing.
3.1 Problem Statement
We are given a dataset D = [2,, v:) a set of images and their corresponding labels. We also assume access to concept set C {c1, .2................... ck that describes unique objects present in the data. An example concept set looks like the following: falley, crosswalk, downtown, .... gas station); i.e. a list of unique objects present each image in addition to the class label. Finally, we are given a classifier fo(X) parameterized by network parameters 0. The central hypothesis of this work is that the class labels exhibit co-occurring bias with the concept set C. affecting downstream task performance. In this light, we wish to generate augmented dataset Daug that is debiased with respect to the concepts and their corresponding class labels. Thus, given the new dataset D' = DU D Dang. we wish to retrain (X) in the standard classification setup:
f arg min (Cly
(1)
where L(y, fo(x)) is the cross entropy loss between the class label and classifier prediction. Our framework consists of three stages: Concept Graph Construction, Concept Diagnosis, and Concept Debiasing. Next, we provide details on each step.
3

---

Conscge
Clas-conceph oo-cccumetse
Concep-conscept co-occurience
Line width indicates Bequency
Randon Fast
Beach
Tree
Temec Farcr.
Waterbira
Eandbird
Waterbind
Tree
Landbird
2-clique
Ocean
with concepi metadata
(b) Concept Grapp
wartind bea' co-cocannte
Beachh
Discovered Blas Waterbirds
Discovered Bins
Landbird
Man
M
Inpainting Generative Model
Lake Pond Beach Ocean
Forest
3-clique
Grass
Bambuo & Troe San Tree
Ocean Rocky
leclique
(e)
Dataset diagnosis result
Clique based sampling
Overview of framework CONBIAS. (a) Given dataset and its concept metadata which contains the objects present in each image, (b) we build the concept graph using object co-occurrences. The line thickness indicates the co-occurrence frequencies of particular concepts with their respective classes. (c) Next, the clique- clique-based based sampling strategy generates under-represented class-concept combinations. which yield (d) the dataset diagnosis result. (e) Finally, with biases discovered, generate images of classes containing under-represented concept combinations the dataset with standard text-to-image generative model.
Figure 2:
3.2 Concept Graph Construction
We construct concept graph E, W)) from the data, where |V| is the node set of the graph, [E] is the edge set, and [W] is the set of weights for each edge in the graph. We first construct the node V union of the label set Y and concept set
= YUC.
Next, construct the edge set F:
E {i,,)) immge D2 such that both and j appear in D
Finally, we construct the weight set W by computing the weights wij for each edge (i,j) in G:
N
wiy = Σ I(i Dn and j E Dn),
n=1
where is the indicator function that returns if both and j are present in the nith image in D, and 0 otherwise, and N is the total number of images in the dataset.
The concept graph G encapsulates co-occurrence counts between nodes, thus providing alternative representation of the (originally visual) data. As we show in the next section, this representation helps uncover novel imbalances (bias) contained in the dataset.
3.3 Concept Graph Diagnosis
In the previous section, we define how to build the concept graph. Here, we present how leverage the concept graph for discovering co-occurrence biases. We present a principled approach to discovering concept-combinations across classes that co-occur in an imbalanced fashion.
Definition (Class Clique Sets) For each class Y; E Y, we construct a set of k-cliques using the concept graph G. The set of all possible k-cliques for class Y, is denoted as K,F:
K {( C2....., Co. Cjt
C' and j2
jaJ.
where /1..2,... are the indices concepts C. Then, Kc₃ for class Y; can successfully con- structed for à 2, ... K, where K is the size of the largest clique in G containing Y, We construct class clique sets for every class in the dataset. An illustration of concept cliques in the Waterbirds dataset that help in bias diagnosis is provided in Figure 3.

---

(Forest, Man, Woman, Barbooo
(Man, Woman, Bamboo)
(Tree, Grass)
(Man, Woman)
Figure Examples of concept clique sets Landbird class Waterbirds dataset uncovered by our diagnosis. Concepts such as Tree, Foreat, Man, Woman, Bamboo are overwhelmingly associated with this class, indicating strong co-occurrence bias. All these concepts are causally unrelated to the bird type.
Definition (Common Class Clique Sets) Given KG; for each class, we then compute the cliques common to all classes. These are the cliques of interest, whose imbalances we want to investigate:
K
Kc
where encapsulates all common cliques enumerated across the dataset for all classes. Refer to Figure for broad illustration of the k-clique set construction from the concept graph G.
Definition (Imbalanced Common Cliques) Given the set of common cliques across all classes K., compute the imbalanced class-concept combinations, i.c. the imbalanced clique set /:
[x [[FFr Picyyy arg min (Frcgy Peygy) })}, Vi,j,
where From and Fxmm indicates the co-occurrence frequency of concepts in clique m with respect to class y and y, respectively, and the arg min operator identifies the underrepresented class for the particular concept clique. Thus, each element in / is number representing the imbalance of cach common clique across all classes. For the special case where the size of clique m is 1, this equates to simply looking up the value Wij in G. For the case where the size of m 2, it is straightforward to compute the co-occurrence class Yi with respect to concepts in m:
Wij Σ I(i E Dr and Da... and K Dn);
for each image D, in the data. The set / holds rich information about the data. In addition to holding the imbalanced counts of concept combinations across all classes, the set I also holds which the underrepresented class with respect to particular concept clique.
Intuitively, concept combinations that are common across all the classes, but do not co-occur uniformly across the classes are likely biased concept combi- nations. We provide example from the Waterbirds dataset in Figure 4. The training set in Waterbirds is intention- ally biased to the background: 95% landbirds appear with land backgrounds, and 95% of waterbirds appear with wa- backgrounds. First, we find com- mon cliques of varying sizes across the classes (Landbird, Waterbird). One example of a common clique of size 3 is (Landbird, Beach, Ocean) and (Waterbird, Beach, Ocean). We compute the co-occurrence of (Landbird, Beach, Ocean) and (Waterbird, Beach, Ocean) from the extracted metadata, and the imbalance clear. Since waterbirds are far more prone to appear water, there significantly more images of waterbirds
1000
Landbird
Waterbird
855
800
Frequency
600
259
200
37
28
(Beach, Occan)
(Occan)
(Tree, Forest)
(Grass)
Water- birds dataset. We show the frequencies of concepts cliques as discovered in the dataset. We see imbalances across not only single concepts (e.g.. Ocean, Grass) but also concept combi- nations (e.g., (Beach, Ocean), (Tree, Forest)). These are the biases we to mitigate
Figure Examples of concept imbalances
downstream task.
5

---

containing concepts Beach and Ocean than landbirds, which are more prone to be in land-based environments. we look at the co-occurence of Landbird with a land-based concept such as Grass, we see the opposite imbalance. There are significantly more images of landbirds containing trees over waterbirds. Similarly, for the water-based concept of Ocean, we see strong imbalance towards the Waterbird class. In our debiasing stage, we should therefore generate more images of water- birds with tree-based backgrounds, and landbirds with beach/water- based backgrounds. Using the clique-based approach, we have successfully uncovered the known background bias in the Waterbirds dataset. This approach is generalizable to multiple classes. All we need are common cliques, and the computation of concept co-occurrences across the dataset. In this way, concept graph approach uncovers interesting concept combinations across the whole dataset that appear in an imbalanced and spurious fashion. More examples of such imbalances provided in the supplementary material.
3.4 Concept Graph Debiasing
We have, to this point, constructed a knowledge graph of the visual data, and diagnosed it for concept- based co-occurrence biases. Once the imbalanced clique set is identified in G. we debias the data by generating images containing under-represented concepts across classes.
Recall that / = (Js. Y.) inherently holds the underrepresented class Y, and the frequency Sa by which the original dataset needs to be adjusted with new images of class Y; with respect to concept clique i. Following the example in the previous section, notice that the concepts (Beach, Ocean) are significantly over-represented in the Waterbird class than Landbird. Similarly, the concept Tree is significantly over-represented with the Landbird class than the Waterbird class. As a result, we sample fü instances of these under-eepresented cliques with respect to their classes, and prompt text-to-image generative model for more images of the Waterbird class with the concept Tree. Similarly, we would prompt the model to generate images of Landbird with the concept Beach, Ocean. We generate images for all class based imbalances following this upsampling protocol.Typical prompts for our image-inpainting model would look like: An image of a ocean and a beach, An image of tree, An image of forest, etc. We use an inpainting -bseed method make sure that the original object is not modified in the image, and that the new concepts are only injected into the non-object space in the image. See the supplementary material for the generated images and the prompts.
Using this upsampling protocol, we generate set of images that leads to augmented, debiased dataset Daug- The original training data D can now be augmented using this data, and the classifier fx(X) can be retrained on the dataset D U Dang. In the next section, we conduct experiments on three datasets to demonstrate our method's significant improvements of baselines.
Note on concept set annotations. We assume the availability of reliable ground truth concept sets. Such annotations already exist for the datasets we investigate- Waterbirds, UrbanCars, and COCO-GB. We agree that unreliable ground truth concept sets would hinder generalization abilities, but this assumption is not dissimilar to the assumption of reliable ground truth labels in classification tasks. Moreover, the reliance on ground truth concept sets, sometimes referred to as concept banks, have also been considered in prior work [57]. Ground truth concept sets serve as auxiliary knowledge bases and provide human level interpretability to the task at hand.
Note computational complexity. In general, given K classes and concepts, the graph clique enumeration is expected to grow in (eepp(K + C.). However, in practice, we find that constraining clique sizes < 4 leads to interpretable bias combinations, with no significant effect of the exponential runtime.
Experiments
We validate our method on vision datatset diagnosis and debiasing across various scenarios. We begin by introducing the experimental setup including the datasets, baselines, tasks, and implementation details in Section 4. Section 4.2 presents the main results of our proposed framework, CONBIAS, compared with sfate-of-the-ar methods. Finally, Section 4.3 details ablation studies and analyses.

---

Table State-of-the-art comparison on different datasets. Results are averaged over three training runs. CB:
class balanced split. OOD: out-of-distribution split. Binary class classification accuracy is used as the metric.
Our method outperforms previous approaches across multiple datascts.
Waterbirds [45]
UrbanCars [22]
COCO-GB [50]
Method
CB
OOD
CB f
OOD 1
CB1
OOD
Baseline [17]
67.1
44.9
73.5
40.5
58.5
51.9
RandAug [5]
73.7
60.2
76.3
46.1
55.8
50.2
CutMix [60]
67.9
45.6
74.4
39.3
57.4
51.2
+ ALIA [8]
69.6
48.2
74.0
42.5
58.7
52.4
+ CONBIAS (Ours)
77.9
69.3
78.3
52.9
58.8
51.4
4.1 Setup
Datasets. We use three datasets in our work: Waterbirds [45], UrbanCars [22], and COCO-GB [50], that are commonly used in the bias mitigation domain. We tackle background bias in the Waterbirds dataset, background and co-occuring object bias in the UrbanCars dataset, and finally gender bias in COCO-GB. All the tasks are binary classification tasks. More details on the training splits and class labels provided in the supplementary material. For Waterbirds, there are 66 nodes and 865 edges in the concept graph. For UrbanCars, the graph contains 19 nodes and 106 edges. For COCO-GB, there are 81 nodes and 2326 edges in the graph.
Baseline methods. Our baselines are include vanilla Resnet-50 model pre-trained on ImageNet, two typical data augmentation based debiasing methods: (1) CutMix, a technique where we cut and paste patches between different training images to generate diverse discriminative features, and (2) RandAug, which creates random transformations on the training data during the learning phase. Finally, we compare against the recently proposed and state-of-the-ar ALIA[8], which uses large language model to generate diverse, in-distribution prompts for text-to-image generative model.
Evaluation protocols. We compute the mean test accuracy over the dlas-balanced test data and the out-of-diitribuutin (OOD) test data, similar to [8]. The class-balanced data contains an even distribution of classes and their respective spurious correlations, while the OOD data contains coun- terfactual concepts. For example, in Waterbirds dataset, for the class-balanced test data 50% images of Landbirds have Land backgrounds, while 50% images of Waterbirds have Water backgrounds. The OOD test set contains Landbirds on Water, and Waterbirds on Land. More details on the test sets are presented the supplementary material.
Implementation details. We use existing implementations to train our models. Our Base model is Resnet-50 pretrained ImageNet [17]. We generate the same number of images per data- augmentation protocol to ensure a fair comparison. For comparison with ALIA on Waterbirds, we directly use their generated dataset available here. For the other datasets, we used the existing ALIA implementation to generate the augmented data. Following previous work, we use validation loss based checkpointing
choose the best model, the Adam optimizer with learning rate of 10-3, weight decay of 10-5, and a cosine learning schedule over 100 epochs. To generate images, we use Stable Diffusion [44] with a CLIP [40]-based filtering mechanism ensure reliable image generation. Finally, we inpaint the object onto the generated image using ground truth masks (available for all datasets). All code was written in PyTorch [35].
4.2 Main Results
In Table we present the main results, averaged over three training runs. First, we note that for Waterbirds and UrbanCars, we observe significant improvements in both the Class-Balande and OOD test sets over the typical augmentation methods such as CutMix and RandAug. Second, we note the significant improvement in performance over the most recent state-of-the-re augmentation method, ALIA. Third, for COCO-GB, while we notice slightly smaller difference in the CB and OOD accuracies between our method and the baselines, our hypothesis is that this happens because of limited number of samples used for augmentation. ALIA uses confidence based filtering mechanism to remove generated samples. This leads to small final number of 260 samples be added for the
7

---

Table 2:
Benefit of graph structure CON- BIAS. Leveraging the graph structure is beneficial as opposed to simply computing single concept-class fre- queney counts on UrbanCars.
Table 3: Performance the IP2P variant of CONBIAS with respect to base, ALIA, and our original model on Waterbirds, Our method signifi- cantly improves
ALIA when using IP2P, although the best results are still achieved when us- ing the stable diffusion based inpainting protocol.
Model
CB
OOD
Base
73.4
40.4
Models
CB f
OOD
Base + ALIA
74.0
42.5
Base
67.1
44.9
Base (BG)
78.5
51.9
Base + ALIA
69.6
48.2
Base (CoOhj)
77.0
47.3
Base (Both)
78.1
51.3
Base + CONBIAS (IP2P)
72.9
60.5
Base (CONBIAS)
79.4
53.2
Basc + CONBIAS
77.9
69.3
Table Robustness our method to evaluation metrics In addition to CB and OOD performance, we also report metrics evaluating multiple shortcut mitigation. Results on UrbanCars (Average of three training runs).
Model
BG-GAP1
CoObj-GAP
BG+CoObj GAP
Basc
-11.2
-21.5
-54,8
Basc (BG)
-5.0
-19.4
-3..0
Base (CoObj)
-6.3
-19.2
-47.3
Base (Both)
-5.6
-23.2
-4.66
Base (CONBIAS)
-6.0
-18.4
41.4
retraining part. In the ablation section, we show this hypothesis to be true, and further demonstrate that on adding more images for the retraining step, we progressively increase the performance gap between our method and the baselines. These three observations taken together validate the usefulness of our approach. The next section provides additional insights on the usefulness of our method and the effect of ablating its components.
4.3 Ablations and Analyses
We further analyze our method along five axes: (1) The usefulness of the graph structure, (2) Robustness of our method to other evaluation metries, (3) The impact on CB and OOD performance by increasing the number of added samples for the retraining step, (4) The usefulness of discovered concepts by our method on the trained classifier, and (5) The impact of the generative component in our work compared to ALIA, since the latter uses InstructPix2Pix [3] while we use Stable Diffusion based inpainting protocol.
Effect of the graph structure. Recalling the definition of Class Clique Sets, in principle one could only use cliques sizes of i.e., the direct neighbors of each class node. This would be equivalent to computing the frequency of co-occurrence over a single hop neighborhood of the class node in the graph. In this ablation structure, instead of simple direct neighborhood based frequency calculation. We trained three separate models on three different types of Daug: Ours (BG), trained on images containing only background shortcuts, Ours (CoObj), images containing only the co-occurrence shortcuts, and Ours (Both), images containing both shortcuts, but not simultaneously.
show that should use larger cliques, i.e. leverage the graph
Table 2 shows the results. First, our approach of leveraging the graph structure provides improvement simply using the frequency of h-hop neighborhood. Second, note that all the methods outperform the baseline and ALIA, which shows that incorporating frequency based co-occurrences is in broader sense much more useful than relying on diverse prompts generated by ChatGPT-4, which is the approach taken by ALIA.
Robustness to evaluation metrics. The CB and OOD test accuracies test for generalization capa- bilities, but more direct evaluators of shortcut learning exist in the literature. In [22], for instance, the authors propose (i) The ID Accuracy- which is the accuracy when the test set contains common background and co-occurring objects, (ii) The BG-GAP which is the drop in ID accuracy when the test set contains common co-occurring objects, but uncommon background objects, (iii The

---

(+) Perforaance on CB
(b) Performance OOD
65
61.8
ALIA
Contias
Accuncy (%Q
60
58.8
60.2
59.5
584
60.2
59.5
57.4
57.1
53.8
ALIA
CooBiias
500
600
800
900
1000
1100
1200
1300
500
600
700
800
900
1000
1100
1200
1300
samples
of samples
Figure 5: Performance on COCO-GB. We show the accuracies on (a) Class-Balanced (CB) and (b) Out-of-Distribution (OOD) splits. We observe that increasing number of images Daug improves performance up to certain point (1000 images).
2018
Input
Base
ALIA
ConBins (ours)
Input
Base
ConBins (ours)
Figure 6: Visualization of the heatmaps for different methods. Top row: Waterbirds, Middle row: UrbanCars. Bottom row: COCO-GB. Our method enforces the hase model to focus only the relevant features in the data, and removing reliance on shortcut features, i.e. the background for Waterbirds, the background and co-occurring object for UrbanCars, and the gender for COCO-GB.
CoObj-GAr GAP, which is the drop in ID accuracy when the test set contains uncommon co-occurring objects, but common background objects, and finally (iv) The BG + CoObj GAP, which is the case when both background and co-occuringg objects are uncommon in the test set. A multiple shortcut mitigation method should minimize the BG + CoOBj GAP metric, and also make sure it does not exacerbate any shortcut that the base model relies on. In Table 4, we present results of Base, Base (BG), Base (CoObj), Base Both), and Base (CONBIAS) on these metrics for UrbanCars. We are able to post the lowest drops among all baselines on the CoObj-GAP and BG + CoOBj GAP metrics, suggesting mitigation of multiple shortcut reliance. This places our method in more realistic context, as infeasible to assume that real world data will only have a single type of bias in them.
Scaling the number of images in Dang- In Table 1, we commented on the fact that our method provides marginal improvement over the baselines in the COCO-GB dataset. Our hypothesis that this was due to the low number of images in the augmented dataset. In Figure 5, we demonstrate the impact of adding more images to D for retraining, Clearly, method benefits from this protocol, leading to significant differences over ALIA as we keep increasing the number of images. Note that, infinite enrichment is not recommended and has been found to be detrimental to classifier performance, as progressive addition of synthetic images will likely lead to addition of off-oiiiiiibttiio examples in the training data. This explains why, after inflexion point, the accuracy suffers from adding more images. Similar observations have been made in [8] and [18].
Discovered concept imbalances and feature attributions. To verify that the model indeed debiases the imbalanced concepts that our method discovers, we present GradCAM [46] attributions of the model predictions after retraining. In Figure 6, we show results on all datasets. While other methods frequently focus on the spurious feature. CONBIAS helps the model focus only on the relevant, object level features of the data.

---

The impact of the generative model. ALIA uses an InstructPix2Pix (IP2P) based generation procedure, while we use stable diffusion with mask-inpainting procedure make sure the objects remain consistent in the image. To ablate the effect of the generation, we present results of our method with IP2P as the generative model instead, on Waterbirds dataset, in Table First, we note that even with IP2P as the generative component, we are able to outperform ALIA, which suggests that it is actually the superior quality of our concept discovery method that leads to the improved results, Second, our inpainting based method outperforms our IP2P based method, which we argue is due to the objects being preserved in the generated image, as opposed to traditional image editing methods, where the object may transform arbitrarily, hurting the quality of augmented data.
5 Conclusion, Limitations, and Future Works
While CONBIAS is the first end-to-end pipeline to both diagnose and debias visual datasets, there some limitations: First, that the enumeration of cliques grows exponentially with the size of the graph. For larger real world graphs, there could be more efficient strategies to find the concept combinations. Second, in this work we focus on biases emanating out of object co-occurrences. A variety of other biases exist in vision datasets, and future work would look to address the same. We add extended section on broader impact of our work in the supplementary material. In summary, datasets in the real world are biased, and the exponential increase in dataset sizes over the past decade amplifies the challenge of investigating model and dataset biases, While both dataset and model diagnosis are exciting areas of research, an end-to-end diagnosis and debiasing pipeline such CONBIAS offers principled approach to diagnosing and debiasing visual datasets, in turn improving downstream classification performance. Our state-of-the art results open up numerous interesting possibílities for future work incorporating more novel graph structures, and diagnosis under the regime where concept sets may be wholly or partially unavailable, remain interesting directions to pursue.
Acknowledgements
This research is partially supported by grant from Apple Inc. We thank Nicholas Apostoloff, Oncel Tuzel, and Jerremy Holland for their valuable feedback on the draft of this paper. Any views, opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and should not be interpreted as reflecting the views, policies or position, either expressed or implied, of Apple Inc. The authors would also like to thank the Norwegian Research Council for funding a doctoral research visit to the Human Sensing Lab, Carnegie Mellon University.
References
[1] Md Rifat Arefin, Yan Zhang, Aristide Baratin, Francesco Locatello, Irina Rish, Dianbo Liu, and Kenji Kawaguchi. Unsupervised concept discovery mitigates spurious correlations. arXiv preprint 1rXiv:2402.13368 2024.
[2] Philipp Bomatter, Mengmi Zhang, Dimitar Karev, Spandan Madan, Claire Tseng, and Gabriel Kreiman. When pigs fly: Contextual reasoning in synthetic and natural scenes. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 255-264, 2021.
[3] Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18392-18402, 2023.
[4] Tom Brown, Benjamin Mann. Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranay Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. In NeurlPS, 2020.
[5] Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quac V Le. Randaugment: Practical automated data augmentation with reduced search space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 702-703, 2020.
[6] Jia Deng, Wei Dong. Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Patiern Recognition, pages 248-255. 255, 2009.
10

---

[7] Alexey Dosovitskiy. Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly. Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
[8] Lisa Dunlap, Alyssa Umino, Han Zhang, Jiezhi Yang. Joseph E Gonzalez, and Trevor Darrell, Diversify your vision datasets with automatic diffusion-based augmentation. Advances in Neural Information Processing Systems, 36, 2024.
[9] Simone Fabbrizzi. Symeon Pupadopoulos, Eirini Ntoutsi, and Joannis Kompatsiaris. A survey bias in visual datasets. Computer Vision and Image Understanding. 203:103552, 2022.
[10] Carolina Galleguillos, Andrew Rabinovich, and Serge Belongie. Object categorization using co-occurrence, location and appearance. In 2008 IEEE Conference on Computer Vision and Pattern Recognition, pages 1-8. IEEE, 2008.
[11] Trena Gao, Gabriel Ilharco, Scott Lundherg, and Maren Tulio Ribeiro. Adaptive testing of computer vision models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4003-4014, 2023.
[12] Noa Garcia, Yusuke Hirota, Yankun Wu, and Yuta Nakashima. Uncurated image text datasets: Shedding light on demographic bias. In Proceedings of the IEEF/CVE Conference on Computer Vision and Pottern Recognition, pages 6957-6966, 2023.
[13] Robert Geirhos, Jörn-Henrit Jacobsen, Claudio Michaclis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665-673, 673, 2020.
[14] Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Imagenet- trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In International Conference on Learning Representations, 2019.
[15] Tanmay Gupta and Aniruddha Kembhavi. Visual programming: Compositional visual reasoning without training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14953 14962, 2023.
[16] Laura Gustafson, Chloe Rolland, Nikhila Ravi, Quentin Duval, Aaron Adcock, Cheng- Fu, Melissa Hall. and Candace Ross. Facet: Faimess in computer vision evaluation benchmark. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 20370-20332 2023.
[17] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.
[18] Ruifei He, Shuyang Sun, Xin Yu, Chuhui Xue, Wenqing Zhang, Philip Torr, Song Bai, and XIAOJUAN QL Is synthetic data from generative models ready for image recognition? In The Eleventh International Conference on Learning Representations, 2022.
[19] Younghyun Kim, Sangwoo Mo, Minkyu Kim, Kyungmin Lee, Jacho Lee, and Jinwoo Shin. Bias-to-text: Debiasing unknown visual biases through language interpretation. arviv preprint 2rXiv:2301.1/104 2023.
[20] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg. Wan-Yee Yen Lo, Piotr Dollár, and Ross Girshick. Segment anything. arXiv:2334.02643, 2023.
[21] Sangjun Lee, Inwoo Hwang, Gi-Cheon Kang, and Byoung-Thi Tak Zhang. Improving robustness to texture bias via soape-focused augmentation. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 4322 1322-4330, 2022.
[22] Zhiheng Li, Ivan Evtimov, Albert Gordo, Caner Hazirbas, Tal Hassner, Cristian Canton Ferrer, Chenliang Xu, and Mark Ibrahim. whac-a-mole dilemma: Shortcuts come in multiples where mitigating one amplifies others. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 20071-20082, 2023.
[23] Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-1 2014, Proceedings, Part V 13, pages 740-755. Springer, 2014.

---

[24] Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang. and Chelsea Finn. Just train twice: Improving group robustness without training group information. In International Conference on Machine Learning, pages 6781-6792. PMLR, 2021.
[25] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jac Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024.
[26] Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303. 033.99999 2023.
[27] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference Computer Vision (ICCV), December 2015.
[28] Aengus Lynch. Gbètondji JS Dovonon, Jean Kaddour, and Ricardo Silva. Spawrious: A benchmark for fine control of spurious correlation biases. arXiv preprint arXiv:2303.cm 054700 2023.
[29] Mark Mazumder, Colhy Banbury, Xiaozhe Yan, Bojan Karlaš, William Gaviria Rojas, Sudnya Diamos, Greg Diamos, Lynn He, Alicia Parrish, Hannah Rose Kirk, et al. Dataperf: Benchmarks for data-centric ai development. Advances in Neurul Information Processing Systems, 36. 2024.
[30] Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey on bias and fairess in machine learning. ACM computing surveys (CSUR), 54(6):1335, 2021.
[31] Sangwoo Mo, Hyunwoo Kang, Kihyuk Sohn, Chun ShannLiang and Jinwoo Shin. Object-aware contrastive learning for debiased scene representation. Advances in Neural Information Processing Systems, 34: 14:121 12264, 2021.
[32] Chaithanya Kumar Mummadi, Ranjitha Subramaniam, Robin Hutmacher, Julien Vitay, Volker Fischer, and Jan Hendrik Metzen. Does enhanced shape bias improve neural network robustness to common corruptions? In International Conference on Learning Representations, 2020.
[33] Curtis Northcutt, Jiang. and Isaac Chuang. Confident learning: Estimating uncertainty in dataset labels. Journal of Artificial Intelligence Research, 10:1373-1411 2021.
[34] Aude Oliva and Antonio Torralba. The role of context in ohject recognition. Trends in cognitive sciences, 11(12):520-53 527,0-52722222 2007.
[35] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
[36] Gregory Plumb, Marco Tulio Ribeiro, and Ameet Talwalkar. Finding and fixing spurious patterns with explanations. Transactions on Machine Learning Research, 2022.
[37] Viraj Prabhu, Sriram Yenamandra, Prithvijit Chattopadhyay, and Judy Hoffmman. Lance: Stress-testing visual models by generating language-guided counterfactual images. Advances in Neural Information Processing Systems, 36, 2024.
[38] Wei Qin. Hanwang Zhang, Richang Hong, Ee Ee-Peng Lim, and Qianru Sun. Causal interventional training for image recognition. IEEE Transactions Multimedia, 2533--11004, 2023.
[39] Xinkuan Qiu, Meina Kan, Yongbin Zhou, Yanchao Bi, and Shiguang Shan. Shape-biased cnns are not always superior in out-ff-distribution robustness. In Proceedings of the IEEE/CVP Winter Conference on Applications of Computer Vision, pages 2326-2335, 2335, 2024.
[40] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748-8763. PMLR, 2021.
[41] Joseph Redmon and Ali Farhadi. Yolo9000: better, faster, stronger. In Proceedings the IEEE conference on computer vision and pattern recognition, pages 7263-7271, 2017.
[42] Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. learning to reweight examples for robust deep learning. In International conference machine learning, pages 4334-4343. PMLR, 2018.
[43] Tianhe Ren, Shilong Liu, Ailing Zeng. Jing Lin, Kunchang Li. He Cao, Jiayu Chen, Xinyu Huang. Yukang Chen, Feng Yan. Zhaoyang Zeng, Hao Zhang, Feng Li, Jie Yang, Hongyang Li, Qing Jiang, and Lei Zhang Grounded sam: Assembling open-word world models for diverse visual tasks, 2024.
12

---

[44] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolttion image synthesis with latent diffuusion models, 2021.
[45] Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks. In International Conference on Learning Representations, 2019.
[46] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, pages 618 626, 2017.
[47] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recogni- tion. Yoshua Bengio and Yann LeCun, editors, 3rd International Conference on Learning Representa- tions, ICLR 2015, San Diego CA, USA, May 7-9, 2015, Conference Truck Proceedings, 2015.
[48] Krishna Kumar Singh, Dhruv Mahajan, Kristen Grauman, Yong Jae Lee, Matt Feiszli, and Deepti Ghadi- yaram. Don't It judge an object by its context: Learning to overcome contextual bias. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pages 1 1070-11078 2020.
[49] Nimit Sohoni, Jared Dunnmon, Geoffrey Angus, Albert Gu, and Christopher Ré. No subclass left behind: Fine-grained robustness in coarse- -rriined classification problems. Advances in Neural Information Processing Systems, 33:19339 19352, 2020.
[50] Ruixiang Tang, Mengnan Du Yuening Li. Zirui Liu, Na Zou, and Xia Hu. Mitigating gender bias in captioning systems. In Proceedings of the Web Conference 2021, pages 633-645, 2021.
[51] A Torralba and AA Efros. Unbiased look at dataset bias. In Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition, pages 1521-1528 2011.
[52] Angelina Wang, Alexander Liu, Ryan Zhang, Anat Kleiman, Leslie Kim. Dora Zhao, Iroha Shirai, Arvind Narayanan, and Olga Russakovsky. Revise: A tool for measuring and mitigating bias in visual datasets. International Journal of Computer Vision, 130(7):1790- 1810, 2022.
[53] Angelina Wang and Olga Russakovsky. Overwriting pretrained bias with finetuning data. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3957-3968. 2023.
[54] Bowen Wang. Liangzhi Li, Yuta Nakashima, and Hajime Nagahara. Learning bottleneck concepts in image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10962-1099 10971, 2023.
[55] Zhenyu Wang, Yali Li. Xi Chen, Ser-Nam Lim, Antonio Torralba. Hengshuang Zhao, and Shengjin Wang. Detecting everything in the open world: Towards universal object detection. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 1 1433 11:3--14: 11443, 2023.
[56] Olivia Wiles, Isabela Albuquerque, and Sven Gowal. Discovering bugs in vision models using off off-the-shelf image generation and captioning. In NewrlPS ML Safety Workshop, 2022.
[57] Shirley Wu, Mert Yuksekgonul, Linjun Zhang, and James Zou. Discover and cure: Concept- mitigation of spurious correlation. In International Conference on Machine Learning, pages PMLR, 2023.
[58] Kaiyu Yang, Klint Qinami, Li Fei-Fei, Jia Deng, and Olga Russakovsky. Towards fairer datasets: Filtering and balaneing the distribution of the people subtree in the imagenet hierarchy. In Proceedings of the 2020 conference on fairness, accountabiliry, and transparency, pages 547-558, 2020.
[59] Sriram Yenamandra, Pratik Ramesh, Viraj Prabhu, and Judy Hoffman. Facts: First amplify correlations and then slice to discover bias. In Proceedings of the IEEF/CV International Conference Computer Vision, pages 4794-4804 2023.
[60] Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. In Proceedings of the IEEF/CVF international conference on computer vision, pages 6023-6032, 2019.
13

---

Supplementary Materials
In this supplementary materials, provide details and additional results omitted in the main text.
Section 6: Broader Impact.
Section 7: Dataset details Splits and image examples.
Section 8: Full Concept Set for each dataset.
Section 9: Dataset Imbalances.
Section 10:
The Generative Model.
Section 11:
Examples Generated Images using CONBIAS.
Section 12: Results with standard deviations.
Section 13: Compute details and runtime.
Section 14: The sampling algorithm for attribute rebalancing.
Section 15: An illustration of method diagnosing spurious correlations in ImageNet-1k
6 Broader Impact
Fairness in is rapidly gaining priority current research as models and datasets grow exponentially larger, thus making it more and more complicated to diagnose them for biases. It is imperative to focus on understanding and mitigating biases learned by models, and inherent biases in the data, to ensure reliable and transparent predictions in the real world. The advent of generative models in particular, including large language models, and image generative models, invites new questions into how to reliably regulate such technologies. These models trained datasets in the order of hundreds of billions of data points. How do we ensure that problematic aspects of the data do not pass onto the models learning from them? How do ensure that models do not generate synthetic data that is potentially harmful, misleading, and misinformative nature? How do we evaluate the quality of generated data by such models? These are the pressing questions that our research direction is interested in.
Dataset Details
We three datasets in our work Waterbirds [45], UrbanCars [22], and COCO-GB [50].
For Waterbirds, the class labels are Landbird, Waterbird. The Waterbirds dataset has the background bias, i.e. 95% images of landbirds have land- badd-basd backgrounds, and 95% images of waterbirds have water-based backgrounds. For the concept set annotations, we use the captions extracted by authors of [8] captions available here.
For UrbanCars, the class labels Urban, Country, defining the type of car. There multiple biases in UrbanCars (1) Background Bias, i.e. Urban cars appear with 95% correlation with urban backgrounds, and Country cars appear with 95% correlation with country backgrounds. (2) Co-Occurring object, i.e. Urban cars appear with 95% correlation with urban objects, and Country cars appear with 95% correlation with country objects.
For COCO-GB, the class labels are Man, Woman. The bias for the dataset are the set objects MS-COCO dataset [23]. The authors of [50] find strong bias of most objects in the data with respect to the "Man" " class, and design a secret, gender- gender-balancee balanced test set to evaluate gender bias classifiers.
7.1 Waterbirds
In Fig 7 we present examples from the Waterbirds training data. The classes are heavily biased to the backgrounds, i.e. Landbirds on Land, Waterbirds on Water.
7.2 UrbanCars
In Fig 8 we present examples from the UrbanCars training data. The classes are heavily biased to multiple shortcuts Background and Co-Occurring objects.
14

---

Figure 7: Examples of training data in Waterbirds dataset. Waterbirds (Top) are 95% biased towards water backgrounds, while Landbirds (Bottom) are 95% biased towards land backgrounds.
STOP
Figure 8: Examples of training data in UrbanCars dataset. Urban cars (Top) are 95% biased towards urban backgrounds and urban co-occurring objects. Country (Bottom) 95% biased towards country backgrounds and country co-occurring objects.
7.3 COCO-GB
In Fig 9 present examples from the COCO-GB training data. The "Man" class is known be heavily biased in MS-COCO to everyday objects.
Figure 9: Examples of training data in MS-COCO dataset. Images of men are heavily biased towards common, everyday objects, as opposed women. Authors of [50] find over a 90% in all object correlations towards men.
7.4 Splits
In Table 5 we present the train, validation, and test splits for our three datasets.
Dataset
Train
Test
Validation
Waterbirds
4795
1199
5794
UrbanCars
8000
1000
1000
COCO-GB
32582
1331
1000
Table 5: Dataset sizes for Train, Test, and Validation sets
8 Concept Sets
In Table 6 we present the full concept sets each dataset. The Waterbirds dataset has 64 unique concepts, the UrbanCars dataset has 17 unique concepts, and COCO-GB has 81 unique concepts, all
15

---

from the MS-COCO dataset. Note that both MS-COCO and UrbanCars have ground truth concepts, while for Waterbirds, use the extracted captions here.
Datasct
Concepts
Waterbirds
duck, pond, tree, grass, post, ocean, bridge, surfer, surfboard, beach, people, forest, beak, sailboat, bamboo, sunlight, boy, foot, boat, mountain,
seagull, field, rock, crab, wall, woman, cell phone,
wing, deer, leaf, backpack, hillside, statue, display, wave, lake, pen, palm tree, shirt, sign, bamboo forest, grass plant, tree branch, bushes, horse, sidewalk, parrot, sun, cup, town, snowy forest, red eye, twig, wooden fence, path, penguin, fishing rod, pelican, kayak, wine glass, lighthouse, mountain landacape, wooden path
UrbanCars
alley, crosswalk, dountown, gas station, garage-outdoor, driveway, forest road, field road, desert road, fireplug, stop sign, street sign, parking meter, traffic light, cow, horse, sheep
COCO-GB
stop sign, tie, kmife, car, bicycle, fire hydrant, cow, motorcycle, umbrella, sports ball, cat, surfboard, elephant, skateboard, skis, backpack, couch, bed, wine glass, carrot, cup, airplame, handbag, cake, cell phone, woman, refrigerator, potted plant, sandwich, vase, chair, bus, frisbee, parking meter, bench, horse, truck, snouboard, train, clock, keyboard, scissors, man, bottle, kite, traffic light, book, dining table, sheep, fork, spoon, tennis racket, dog, bowl, suitcase, boat, donut, baseball bat, orange, toothbrush, banana, oven, laptop, toilet, sink, pizza, mouse, baseball glove, tv, teddy bear, hot dog, broccoli, remote, bird, microwave, apple, zebra, bear, toaster, giraffe, hair drier
Table 6: Concepts for Waterbirds, UrbanCars, and COCO-GB datasets
Dataset Imbalances
In this section we shed more insight into what sort of concept imbalances ConBias discovers. These object level insights are also, to the best of our knowledge, the first of its kind, shedding more light on the secret co-ocurrence biases hidden in data.
9.1 Waterbirds
In addition to the main paper, we list some other category imbalances in Waterbirds in Figure 10. Some of these extreme imbalances appear in diverse 2-clique/3-clique combinations. For example, that concepts like forest, man, woman are significantly biased towards the Landbird class, while concepts like beach, man, sun, lake, mountain biased towards the Waterbird class. This the background bias that is known in the Waterbirds dataset, that ConBias successfully uncovers.
9.2 UrbanCars
In UrbanCars, the class labels (country car, urban car) are intentionally biased towards background and co-occurring objects. In Figure 11, we see that there exists an extreme imbalance betwee urban concepts such as driveway, traffic light towards urban cars, and country concepts such as forest road, field road, cow, horse towards country cars. These are exactly the background and co-occurring biases in the construction of the data, that ConBias successfully uncovers.
9.3 COCO-GB
The gender bias in COCO-GB has been extensively studied [50]. In Figure 12, we show the extreme imbalance towards specific concepts in the MS-COCO dataset. Concepts such as baseball bat, sports ball, motorcycle, truck overwhelmingly correlate with images men, which may be problematic for the classifier to learn
16

---

Landod
Waterbeed
60
50
Counts
40
30
20
The process will first transform the concept list [concept concept 2, ..., concept N] into a prompt: "a photo of concept 1, concept 2...., and concept N." The prompt is then passed into the text-to-image generation model (stable diffusion) to get the generated image as background. We apply clip-score beach, non, sun
forest, man. woman
grass, trep
lake, mountain
Categories
Figure 10: Extreme imbalance of particular concepts in Waterbirds dataset, as discovered by ConBias.
$30
Urban
500
Courtry
400
Counts
300
driveway, traffic light
field road, horae
forest road, cow
alley street sign
Categories
Figure 11: Extreme imbalance of particular concepts UrbanCars dataset, discovered by ConBias.
10 Generative Model
Here present more details of our generative model. We use Stable Diffusion based inpainting, as illustrated Figure 13. Given the prompt, we first generate an image using Stable Diffusion [44]. Next, using ground truth masks of the object, we paste the object at the foreground of the generated image. In this way, we preserve the original object in the image, which is a challenge for traditional image editing methods such InstructPix2Pix. We believe the inpainting method a more principled approach to synthetic image generation, particularly if the downstream task is classification in nature.
The generation process of single image takes the followings as input: The sampled concept combination and the class for which this concept combination needs to be generated. The output of the generative model is the final image with the specified concepts in the background and an instance of the specified category in the foreground.
filtering after the generation process to only keep the images with CLIP-scon over 0.6 make sure
17

---

600
Man
532
Womee
500
400
Counts
300
238
200
100
0
bench, bicycle
baseball bat. sports ball
motorcylk, truck
book, refrigerator. sink
Categories
Figure 12: Extreme imbalance of particular concepts in MS-COCO dataset, as discovered by ConBias.
photo band and bomboo forest Prompt
Synthetic Image
Text-s-lmage Genuration Model
Filter
Hamboo, Forest, Man
Original Image
Waserbird's
Image Foreground
Inpaint
Ground Truth Mask
Figure 13: Image generation Pipeline: Given concepts to be upsampled as discovered ConBias, sample the concept combinations and images from the class to be upsampled. We prompt Stable Diffusion for image containing such concepts. We extract the object of interest using ground truth masks, and inpaint the object over the generated image. This ensures that the object features are not harmed during generation. We use a CLIP-based scoring filter make sure the generated image contains the concepts requested in the prompt. We have found a score of 0.6 to be satisfactory as threshold.
that the generated images can accurately represent the concept list. Next, the process will sample an image of the specified category from the original dataset, and use the mask to segment out the desired object. Finally, inpainting is performed to clip the desired object as foreground onto the generated image to obtain the final image.
11 Generated Images by ConBias
In this section we present examples of synthetic data generated by ConBias for Waterbirds, UrbanCars, and COCO-GB.
18

---

11.1 Waterbirds
In Figure 14 we present diverse images generated by ConBias for the two classes of Landbird and Waterbird. Due to the bias diagnosis stage where we found the overwhelming correlation between landbirds with land based backgrounds such as tree, forest, field, grass, etc, and waterbirds with water based backgrounds such as beach, ocean, boat, etc, ConBias was automatically able to decide which concept combinations to use to generate new, debiased images.
Figure 14: (Top) Generated images of waterbirds with land-based backgrounds. (Bottom) Generated images of landbirds with water-based backgrounds, as discovered by ConBias. Note the consistency in object preservation.
11.2 UrbanCars
In Figure we present diverse images generated by ConBias for the two classes of Urban and Country cars. Due to the bias diagnosis stage, between urban cars with urban based backgrounds such as gas station, driveway, alley, etc and urban co-occurring objects such as fireplug, stop sign, etc. Similarly, for country cars, discovered bias towards country backgrounds such as desert road, field road, forest road, and, and country co-occurring objects such cow, sheep, horse. As result, ConBias helps generate urban cars with country based backgrounds and co-occurring objects, and vice versa.
able to discover the overwhelming correlation
Tto
Figure 15: (Top) Generated images of country cars with urban-based backgrounds and co-ocurring objects. (Bottom) Generated images urban cars with urban-based backgrounds and co-occurring objects, discovered by ConBias. Note the consistency in object preservation.
11.3 COCO-GB
In Figure 16 we present diverse images generated by ConBias for the two classes of Man and Woman in COCO-GB. In this dataset, were able to discover significant under-repressentaiin of women with respect to common, everyday objects in the MS-COCO dataset. Some examples include skateboard, motorcycle, car, truck, etc. These objects could have gendered assumptions and it is imperative for debiased datasets to have uniform representation across classes for such concepts.
would also like to bring to attention of our readers the successful nature of the inpainting procedure. We are able to consistently preserve the class label of interest in the synthetic images. This is imperative to ensure that the generative pipeline does not create unreasonable objects that make it infcasible for the classifier to learn.
19

---

Figure 16: Generated images of COCO-GB using everyday, common objects that are discovered to he biased towards men by ConBias. Example concepts include skateboard, motorcycle, truck, sports ball, etc. Note the consistency in object preservation.
12 Confidence Intervals
In Table 7 we present the averaged results with standard deviations over three training runs. For both Waterbirds and UrbanCars, our improvements are large and significant. For COCO-GB, while originally did not observe statistically significant results, in the main paper we showed that increasing the number of images in Dang leads to significant improvements over the baselines.
Table 7: State-of-the-art comparison on different datasets. Results are averaged over three training runs. CB: class balanced split. OOD: out-of-distribute split. Binary class classification accuracy is used as the metric. CONBIAS outperforms previous approaches across multiple datasets. Standard deviations included.
Waterbirds [45]
Urban [22]
COCO-GB [50]
Method
CB
0OD
CB
OOD
CB
OOD
Baseline [17]
67.1 + 0.5
44.9 = 0.8
73.5 = 0.6
40.5 0.8
58.5# 0.7
51.9 0.7
RandAug [5]
73.7 +
60.2
76.3 0.8 0.8
46.1 =
55.8 0.4
50.2
+ CutMix [60]
67.9 + 0.7
0.74 0.7
39.3 0.9
57.41 0.5
0.62
+ ALIA [8]
69.6 + 1.2
48.2
7..0
42.5
58.7+ 0.4
0.6
+ ConBias (ours)
77.9 0.9
6..3
78.3 0.7
52.9+ 0.7
58.8 0.6
51.4
13 Compute Details
We trained all models on single NVIDIA RTX A4000 and used PoTorc [35] for all experiments. With the early stopping cosine learning scheduled described the main paper, we observed fast training times, with 90 minutes for three runs Waterbirds and UrbanCars, and 180 minutes for three runs on COCO-GB.
14 Sampling Algorithm
The rebalance sampling algorithm receives the concept graph concept co-occurrence matrix. The algorithm iterates through all cliques with an order of decreasing clique sizes to make sure we would not double compensate for the imbalance, e.g., 3-cliques would impact the already- -reeadd-allnced 2-cliques if we operate in bottom-up fashion. For each iteration, retrieves all cliques of concepts of size k along with their corresponding frequencies with each class. The algorithm identifies the maximum co-occurrence count among all classes for each combination and checks if any class is under-represented by comparing its count with the maximum. If a class is under-represented, the algorithm computes the number of synthetic samples needed to balance the representation and adds this information the results list. This process continues for all combinations and classes until all clique sizes have been processed. The output of the algorithm is list of queries specifying the class, concept combination, and the number of samples needed balance the dataset.
15 Diagnosing ImageNet-1k
In this section we demonstrate the usefulness of ConBias in diagnosing spurious concepts in more complex dataset, i.e. ImageNet-lk. Specifically, we investigate the classes ambulance, beach wagon, sports car, limousine, minivan, jeep, convertible, cub. These are associated with the superclass of Car. This approach can used for any set of classes in the dataset. We use the open-source concept annotations available here.
20

---

Algorithm Rebalance Sampling
Input: Moce Concept occurrence matrix as a dictionary
I: resulls 6
> Initialize result list for complement samples
2: k < the maximum size of concept cliques
3: while k > 1 do
4: combos + Moce[k]
Get the concept combinations and counts of all k-cliques 5: for each concept combination E combos do
6: n4 <- number of co-occurrence between combo C' and class i m f- max((n;) for each class i do
7:
Determine the maximum frequency among the classes
8:
9:
n, < m then
class is under-represented w.r.t. combo C Compute the number of samples to generate Save generation query
10:
i. <
resulls c- resulls (i, C,
12:
end if
13:
end for
14:
end for
15:
Update Mo with the generated samples
16:
k- k- k-1
Move cliques with size smaller 1
17:
end while
Output: results
The full set of queries generate
Co-arcurrence Counts for "Tram"
Co-occurrence Counts for "Prison"
100-
125
Count
100
75
0
aniuulance
canvensilee
miteiian
linosing
beacch wagann
sperts car
Jvap
cab
Intiulance
convertible
Imousinne
Juscch eagonn
gports oar
Help
edb
Classes
(a) Imbalance for I-clique Tram
(b) Imbalance for l-clique Prison
Co-ocoumence for GGo,,
Co-ocoumence ((KK, Grilio)
175
140
150
125
100
100
25
20
antrriance
conventblee
minivan
Brosutine lino
sports car
jamp
686
ambulance
comverible
minwent
limounine
besech wappor
kepp
cab
beach
sports
Classes
Oasses
(c) Imbalance for 2-clique (Golf, Sorrel)
(d) Imbalanee lacce for 2-clique (Ski, Grille)
Figure 17: Concept imbalances for cars in ImageNet-1k
In Figure 17 notice some interesting imbalances discovered by ConBias. For I-clique concept combinations such as Tram and Prison, the dataset disproportionately contains images of cabs and ambulances respectively. For 2-elique concept combinations, the dataset disproportionately represents the jeep, convertible, and beach wagon classes. It is evident that such concepts are spurious when it comes to classifying a car type, but a strong imbalanced distribution would bias the classifier to pick up spurious features. In this way, ConBias helps uncover such biases, allowing for intervention downstream tasks.
21

---

NeurIPS Paper Checklist
490
491
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
492
493
495
486
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
497
498
You should answer [Yes], [No]. or [NA].
489
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
500
Please provide short (1-2 sentence) justification right after your answer (even for NA).
502
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
503
504
505
506
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation. While "[Yes] " is generally preferable to "[No]" it is perfectly acceptable to answer " "[No]" provided proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA]" is not grounds for rejection. While the questions are phrased in binary way, we nuanced, so please just use your best judgment and write a justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to question, in the justification please point to the section(s) where related material for the question can be found.
508
510
leekowwlddge that the true answer often
512
514
515
1. Claims
516
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
518
Answer: [Yes]
519
Justification: Table in the main paper and our extended analysis section demonstrate the usefulness of our approach.
520
Guidelines:
522
The answer NA means that the abstract and introduction do not include the claims made the paper.
523
524
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
525
526
527
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
528
$29
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2.
Limitations
532
Question: Does the paper discuss the limitations of the work performed by the authors?
533
Answer: [Yes]
534
Justification:
We discuss limitations our work in the final section of the paper.
535
Guidelines:
536
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
537
The authors encouraged to create separate "Limitations" section in their paper.
14

---

539
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
540
542
548
544
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
545
548
547
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-texx system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
549
551
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
553
554
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
555
556
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
558
560
582
3. Theory Assumptions and Proofs
583
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
584
585
Answer: [NA]
586
Justification:
There are no theoretical results or proofs in the paper.
567
Guidelines:
568
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
570
All assumptions should be clearly stated or referenced in the statement of any theorems.
572
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
573
574
575
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
576
577
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
578
579
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
581
582
Answer: [Yes]
583
Justification: We include details of the datasets in both the main paper and supplementary. The implementation of baselines rely on open source codes that are well documented. Further, we will release our own code for better reproducibility.
585
586
Guidelines:
587
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
589
15

---

591
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
592
598
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
695
587
598
589
600
801
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
603
605
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
606
607
608
(c) If the contribution is new model (e.g.. a large language model), then there should either be way to access this model for reproducing the results or a way to reproduce the model (e.g.. with an open-source dataset or instructions for how to construct the dataset).
610
612
613
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, it may be that access to the model is limited in some way (e.g.. registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
614
615
617
618
5. Open access to data and code
619
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
620
621
622
Answer: [Yes]
Justification: We attach the code in addition to the supplementary material. All datasets except UrbanCars are open source. The code to generate UrbanCars, however, is open source.
624
Guidelines:
627
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips) cc/
829
sanilmmmsioooooccc for more details.
630
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-soure benchmark).
631
632
633
634
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: / /nipp.cc.acc
636
httpe//ggiiiees/ddSuuubiiiieeee SCtd//eoooeisiiiieeeeeeea wwwssssossliiieeeeee insionicocc) for more details.
637
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
638
639
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
641
642
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
643
644
Providing as much information as possible supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
16

---

648
6.
Experimental Setting/Details
647
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
649
650
Answer: [Yes]
Justification: All training details and hyperparameters included in the main paper as well as the supplementary.
652
653
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
656
657
The full details can be provided either with the code, in appendix, or as supplemental material.
658
7. Experiment Statistical Significance
680
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
663
Justification: All results reported are the mean of three separate training runs. We include error bars in the supplementary material, and omit them from Table I in the main paper for better readability.
685
686
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
669
670
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
672
673
674
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
678
679
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
680
682
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
683
684
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
686
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
689
681
Answer: [Yes]
692
Justification: We provide details on the architecture used and the time execution in the supplementary.
694
Guidelines:
695
The answer NA means that the paper does not include experiments.
696
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
17

---

698
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
699
700
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
702
703
9.
Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: /ttt://eeeripsppp heppppic/ttiis/////iteeppp ines?
705
706
Answer: [Yes]
707
Justification:
We have reviewed and agree to the NeurIPS code of ethics.
Guidelines:
709
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
711
712
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
715
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
718
Justification:
We discuss the broader impact of our work in the supplementary.
719
Guidelines:
720
The answer NA means that there no societal impact of the work performed.
721
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
724
725
726
727
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinfornation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
728
730
732
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
735
736
737
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
739
741
742
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
744
746
Answer: [Yes]
747
Justification: We perform the necessary manual checks to ensure the safety of our data generation process.
Guidelines:
18

---

750
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
792
754
755
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
758
757
recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
759
780
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
782
Answer: [Yes]
785
Justification: All original creators/owners of assets are cited in the work. We use CC-BY 4.0 license.
786
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
770
The authors should state which version of the asset used and, possible, include URL.
771
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
774
775
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. damaattsets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
776
777
778
779
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors are encouraged to reach out to the asset's 's creators.
782
783
13. New Assets
784
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
786
Answer: [Yes]
787
Justification: We share the code to reproduce experiments in our work, and will make them open source.
789
Guidelines:
790
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
792
The paper should discuss whether and how consent obtained from people whose asset is used.
795
796
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
797
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
800
19

---

802
Answer: [NA]
803
Justification:
This work does not involve research with human subjects.
804
Guidelines:
805
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
806
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
808
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
811
15.
Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
814
815
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
817
819
Answer: [NA]
820
Justification: This work does not involve research with human subjects.
821
Guidelines:
822
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
823
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
825
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
828
830
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
20


Reviews:
[{"review_id": "FLny0SdBvU", "reviewer": "Reviewer_xMok", "summary": "This paper introduces a concept graph-based framework to diagnose and mitigate biases in visual datasets by representing datasets as knowledge graphs of object co-occurrences. The approach involves constructing a concept graph, diagnosing concept imbalances, and debiasing by generating images with under-represented concept combinations. This method enhances dataset balance and improves generalization performance across multiple datasets.", "questions": "see above", "limitations": "Yes, the authors discuss about limitations in section 5.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "\tThe paper presents a structured and controllable method for diagnosing and mitigating spurious correlations by representing datasets as knowledge graphs of object co-occurrences.\n\tExperimental results demonstrate that balanced concept generation in data augmentation enhances classifier generalization across multiple datasets, outperforming baseline methods.\n\tThis manuscript is written well, it’s easy to read and follow.", "weaknesses": "This manuscript seems to be an engineering technical report about the data augment and the technical contribution is weak for an academic paper. The method of constructing a knowledge graph to determine object co-occurrences is overly direct; statistical analysis of objects and frequency comparison can achieve similar results.\n•  What is the advanced intuition behind constructing a knowledge graph to determine object co-occurrences? How does it compare to previous methods in terms of advantages?\n•  The paper lacks details on the concept set annotations. How might the quality of these annotations affect the performance of the proposed method?\n•  Why are the results of ALIA on Table 1 difference with ALIA[1].\n[1] Lisa Dunlap, Alyssa Umino, Han Zhang, Jiezhi Yang, Joseph E Gonzalez, and Trevor Darrell. Diversify your vision datasets with automatic diffusion-based augmentation. Advances in Neural Information Processing Systems, 36, 2024\n\nThere is a typographical error on line 198: a space is missing after the period.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive feedback.\n\n**Intuition and advantage of constructing knowledge graph:** \n ConBias is not simply a statistical co-occurrence counting mechanism. In fact, in lines [87-90], we mention that there are diagnostic tools that exist today which compute such object co-occurrence statistics. The novelty of our method is in the creation of a graph that encapsulates multiple class-concept co-occurrences. The analysis of the class-concept graph cliques is crucial to the graph construction. In Table 2, we show the benefit of using the graph structure as opposed to a simple counting mechanism based on statistical co-occurrences. Additionally, our method is novel as to the best of our knowledge, no existing works leverage a concept co-occurrence based graph to simultaneously diagnose and debias datasets. The advantages over previous methods is in the controllability and interpretability of debiased data generation. Our results, as demonstrated in Table 1 and Table 4, show significant advantages in generalization over previous approaches.\n\n\n**Details of concept set annotations:**\nWe described the details for all the datasets used in Section B and Section C of the appendix and the supplementary materials. In Table A2 in appendix, we detailed the list of concept sets: the Waterbirds dataset has 64 unique concepts; the UrbanCars dataset has 17 unique concepts; the COCO-GB dataset has 81 unique concepts. All the concepts are from the MS-COCO dataset. We will incorporate these details into the main paper in the final version.\n\n\n**Effect of the quality of the annotations:**\nWe assume the concept sets to be available for graph construction. One can interpret this availability as the same as the assumption that ground truth labels are useful for classification tasks. Just as unreliable ground truth labels can hinder classifier performance, the concept sets also need to be of high quality and reliability to ensure strong generalization. We mention this assumption in the final section of the main paper, and are looking forward to future work that investigates unreliable, partially available concept sets. We will update the paper to clarify this important point raised by the reviewer.\n\n\n**Results of ALIA:**\nWe invite the reviewer to refer to Section G of the appendix, where we present the confidence intervals in addition to the averaged results in Table 1. The results presented are within the confidence intervals as presented in the ALIA paper, who also average results over three runs. \n\n**Typos in L198:**\nThank you and we will correct it in the final version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NR8iv8qwwr", "reviewer": "Reviewer_XqYX", "summary": "This paper points out inherent biases in visual dataset. To address this issue, the paper proposes a new framework called ConBias, which proceeds in three steps: (1) Concept Graph Construction, (2) Concept Diagnosis, and (3) Concept Debiasing. Using concept metadata in the dataset, concept graph is constructed. After identifying imbalanced clique sets, ConBias utilizes Stable Diffusion to generate images containing under-represented concepts across classes. The experimental results show the effectiveness of the proposed framework to mitigate dataset biases.", "questions": "1. Please describe total numbers of generated graph nodes and edges.", "limitations": "The authors address limitations and broader impact well in the paper.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper proposes a new concept graph-based framework, which is easy to construct using metadata and diagnose biases in visual datasets.\n2. A new approach to generate unbiased images using stable diffusion robustly enhances the overall performance.\n3. The experimental results across different datasets show reasonable performances compared to the state-of-the-art method.", "weaknesses": "1. The main concern is that the proposed framework should require metadata to construct a graph. It can be a problem to apply this method directly in many cases where visual datasets do not include any metadata.\n2. I do not agree that the existing approach utilizing LLMs is no reliable way, since (1) there is also a possibility that the given metadata has wrong or biased information, and (2) as recent LLMs become powerful, the quality of domain descriptions is reliable (rather automatically improving the quality as time goes by). To sum up, there is no guarantee that the proposed method is always less biased compared to the existing method.\n3. The processes to construct graph and diagnose bias do not look technically novel. The overall process is primarily based on a counting mechanism, not a learning-based method.\n4. The authors tested the proposed framework on three benchmark datasets. However, the used datasets look like toy experiments, i.e., all the tasks are binary classification tasks and easy to identify biases. I wonder that the proposed method still constructs and detects inherent, complex biases in large and multi-class tasks, e.g., ImageNet.\n5. The experimental results in the tables are quite different from the values displayed in the ALIA paper [8]. If it is true, can you explain the reasons?\n6. There are no comparisons about computational complexity between the existing methods and the proposed one. Please clarify how the costs are changed per the number of total concepts.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the valuable comments.\n\n**Metadata:**\nAs we discuss in the main paper (lines 324-325), we assume the availability of reliable ground truth concept sets. Such annotations already exist for the datasets we investigate - Waterbirds, UrbanCars, and COCO-GB. We agree that unreliable ground truth concept sets would hinder generalization abilities, but this assumption is not dissimilar to the assumption of reliable ground truth labels in classification tasks. Moreover, the reliance on ground truth concept sets, sometimes referred to as concept banks, have also been considered in [A]. Ground truth concept sets serve as auxiliary knowledge bases and provide human level interpretability to the task at hand. We look forward to using partially available concept sets in future work (lines 324-325). We will update the paper to reflect this important point raised by the reviewer.\n\n[A] Wu, Shirley, et al. \"Discover and cure: Concept-aware mitigation of spurious correlation.\" In ICML, 2023.\n\n\n**LLMs:**\nWith regard to the reviewer’s first point, we do not argue in the paper that LLMs are not reliable for metadata, and we agree with the reviewer that LLMs are useful for a variety of tasks. The metadata that we use comes from ground truth concepts that are available in all the datasets tested in our work. We do not use LLMs to extract metadata for graph construction. \nInstead, we argue that LLMs are unreliable in the generation of fair, balanced data (lines 36-39). This is an issue that has been acknowledged by the authors of ALIA as well (please refer to Section 6 in the ALIA paper). This is the issue we overcome with ConBias: Instead of using an off-the-shelf LLM to generate diverse image data, we use a controllable and interpretive knowledge graph that encodes the class-concept imbalances in the data.\n\n**Novelty:**\nThe core intuition of ConBias is that we want a controllable and interpretable way to generate debiased data. In a learning based framework, these two requirements are compromised since a gradient based optimization technique relies on parameters that may deem the model to be a black box.\n\nConBias is not simply a statistical co-occurrence counting mechanism. In fact, in lines [87-90], we mention that there are diagnostic tools that exist today that compute such object co-occurrence statistics. The novelty of our method is in the creation of a graph that encapsulates multiple class-concept co-occurrences. The analysis of the class-concept graph cliques is crucial to the graph construction. In Table 2, we show the benefit of using the graph structure as opposed to a simple counting mechanism based on statistical co-occurrences. Additionally, our method is novel as to the best of our knowledge, no existing works leverage a concept co-occurrence based graph to simultaneously diagnose and debias datasets. Our results, demonstrated in Table 1-4, confirm the usefulness of the graph clique balancing strategy. \n\nBroadly, we believe that if we wish to debias datasets, we need human-level intuition on what concepts are biased, and what concepts need to be debiased in a principled fashion, corresponding to human-level intuition. ConBias provides this intuition (please refer to Figure 3 and Figure 4 in the main paper, and Section D in the supplementary section). A black-box learning based approach, on the other hand, would hinder these capabilities. \n\n\n**Extension to multi-class tasks:**\n\nSince the reviewer raised the example of Imagenet, in the attached PDF above, we present new results on diagnosing multiple classes in Imagenet-1k. We discovered some interesting spurious correlations uncovered by ConBias. As such, ConBias can be seamlessly used for datasets of increasing complexity. We will update the paper to include these results.\n\nWaterbirds and UrbanCars, while binary, are the de facto datasets used in the literature for evaluating single-shortcut and multi-shortcut learning in classifiers. These are the most commonly used datasets for benchmarking purposes. COCO-GB is a subset of the MS-COCO dataset that includes real-world data consisting of humans and everyday objects in the wild.\n\nWe include the motivations behind the datasets and other details in Section 4.1 and Supplementary Section B. For the reviewer’s convenience, we also include three references here that may clarify the use of these datasets:\n\n- Li, Zhiheng, et al. \"A whac-a-mole dilemma: Shortcuts come in multiples where mitigating one amplifies others.\" In CVPR. 2023.\n\n- Sagawa, Shiori, et al. \"Distributionally Robust Neural Networks.\" In ICLR 2020.\n\n- Tang, Ruixiang, et al. \"Mitigating gender bias in captioning systems.\" In WWW. 2021.\n\n**Results of ALIA:**\nWe invite the reviewer to refer to Section G of the Supplementary material, where we present the confidence intervals in addition to the averaged results in Table 1. The results presented are within the confidence intervals as presented in the ALIA paper, who also average results over three runs. \n\n\n**Computational complexity:**\nWe invite the reviewer to refer to Section H in the Supplementary section for details on runtime. In general, given $K$ classes and $C$ concepts, the graph clique enumeration is expected to grow in $O(exp(|K+C|)$. However, in practice, we find that constraining clique sizes to $k<=4$ leads to interpretable bias combinations, with no significant effect of the exponential runtime. We agree that this is a heuristic, and more efficient clique enumeration methods can be developed (lines 314-315). We will update the paper to reflect this point raised by the reviewer. \n\n\n**Total number of graph nodes and edges:**\nFor Waterbirds, there are 66 nodes and 865 edges in the graph. For UrbanCars, the graph contains 19 nodes and 106 edges. For COCO-GB, there are 81 nodes and 2326 edges in the graph. We will update the paper to include these additional details.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Bd0i2AjOzn", "reviewer": "Reviewer_TVhQ", "summary": "The authors present ConBias, which is a novel approach for diagnosing and de-biasing co-occurrence bias in datasets. Unlike previous works, ConBias addresses both diagnosis and de-biasing / balancing strategy, to target the data augmentation specifically to address spurious concept co-occurrences. The main components of ConBias are as follows: \n1. New concept graph-based analysis framework which involves occurrences of common concepts and classes in the data in a graph-form to identify spurious correlations \n2. Identify main cliques that occur across all classes, and determine the frequency of occurrence with respect to each classes to find spuriously correlated concepts \n3. re-balance the dataset based on the imbalanced concept cliques. specifically, generate new backgrounds for the objects using an inpainting-based augmentation. \n\nThen they perform a thorough evaluation of their new re-balanced dataset across various evaluation protocols including the class-balanced test data and the OOD test data, as well as other metrics for shortcut learning. The results show that ConBias consistently outperforms other existing baselines. The authors also show ablations to justify their design decisions around the graph structure, as well as the generative model choice.", "questions": "Related to the preceding section, I have several inquiries for the authors:\n\n1. Could you provide insights into how this method might scale effectively in a multi-class classification scenario? What are your expectations regarding its performance scalability?\n2. Could you clarify why additional evaluation metrics were exclusively presented for UrbanCars and not for Waterbirds or Coco-GB datasets?\n3. How precisely is the upsampling conducted concerning different sets of cliques? While I understand the additional image count aligns with ALIA's methodology for fair comparison, it remains unclear how many images this comprises, how various co-occurring object sets contribute to the augmentation dataset distribution, and the criteria used in making these decisions.", "limitations": "Yes, the authors sufficiently address limitations and societal impact in the final section of their paper.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "Originality: The work demonstrates originality by introducing targeted dataset-debiasing based on concept co-occurrence diagnosis within the dataset augmentation space. Unlike previous approaches that often debias without diagnosing or only diagnose biases within datasets without suggesting effective debiasing methods directly to the data, this method focuses on improving the robustness of binary classification tasks through meticulous dataset examination.\n\nQuality: The paper presents a novel approach that consistently outperforms existing debiasing augmentation methods. The experimental setup spans multiple datasets across various domains, providing robust justification for the proposed approach. Results are clearly presented, demonstrating reasonable improvements in performance metrics.\n\nClarity: The paper is well-written and logically organized, making it easy to follow the ConBias pipeline. Graphs and illustrations are informative, enhancing understanding of each step and experimental setup. \n\nSignificance: This work introduces a debiasing method aimed at reducing overfitting to spurious correlations. This is significant as it can improve the reliability of vision tasks across various applications, especially in domains where robust performance on out-of-distribution (OOD) cases is crucial, such as healthcare diagnosis", "weaknesses": "The binary classification test bench used in the experiments may be insufficient in scope to fully demonstrate the potential of the proposed method. While the results show promise, it remains unclear if the method's performance scales effectively to multiple sets of classes, which is crucial for assessing its broader impact and significance. Moreover, in scenarios with multiple sets of classes, there exist spurious correlations across different subsets that require a more sophisticated and targeted sampling strategy than what the current approach, which extracts shared cliques across all classes, provides.\n\nIn addition, I believe further justification or benchmarking against debiasing methods in the feature space is necessary. Although this work pioneers combined diagnosis and debiasing in the dataset space, previous research has focused on diagnosing and debiasing models at training time or in feature spaces. Earlier methods required dataset bias labels, while recent advancements explore shortcut learning without explicit labels. The paper briefly mentions choosing the dataset space for enhanced interpretability but does not fully justify the computational trade-offs involved in generating balanced data as well as concept labels. The work would benefit significantly from additional benchmarking against state-of-the-art feature space analysis and debiasing methods where they are able to demonstrate improvements or providing a detailed rationale for its focus on the dataset debiasing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the encouraging review and the detailed feedback.\n\n**Extension to multi-class tasks and scalability:**\nConBias can be conveniently extended to the multi-class setup. For this, we note the definitions in line 142 and line 146. While the common clique computation remains the same, for the imbalanced clique computation, instead of taking the pairwise absolute difference between classes, we can use a different metric such as variance and entropy to estimate the degree of imbalance among multiple classes. Functionally, the imbalanced set would contain the same information, i.e. how disproportionately is one concept combination (1-clique, 2-clique, etc.) represented with respect to a particular class relative to the other classes? In addition, in the attached PDF for the rebuttal (Figure 1), we also show how ConBias can diagnose biases in a complex multi-class dataset such as Imagenet-1k. \nWe will add this discussion to the final version and thank the reviewer for the insightful question.\n\nFor scalability,  we invite the reviewer to refer to Section H in the Supplementary section for details on runtime. In general, given $K$ classes and $C$ concepts, the graph clique enumeration is expected to grow in $O(exp(|K+C|)$. However, in practice, we find that constraining clique sizes to $k<=4$ leads to interpretable bias combinations, with no significant effect of the exponential runtime. We agree that this is a heuristic, and more efficient clique enumeration methods can be developed (lines 314-315). We will update the paper to reflect this point raised by the reviewer. \n\n**Comparison to debiasing methods based on the feature space:**\nConBias is a data-centric method rather than a model-centric method. We wish to intervene in the data directly rather than using a specific model as a proxy (lines 80-83). Once the biases are diagnosed, we can controllably and reliably generate fair data that improves model debiasing and generalization capabilities on downstream tasks. \n\nA direct benchmarking against feature based methods would not be feasible since it is difficult to control for the effect of the augmented dataset. This is why our baselines, such as ALIA, also do not compare against feature-based debiasing methods. To ensure a fair comparison for data debiasing techniques, we need to ensure that the effect of adding data to the training set is adjusted for. In a pure feature debiasing based setup, this would be hard to control. However, the reviewer raises an important point about the dependence between data and feature debiasing. We agree that these two are not mutually exclusive. In fact, this is why we present results in Figure 6 where we visualize the classifier feature heatmaps after retraining. We demonstrate that the features learned by the classifier are indeed the core features, and not spurious features. On each dataset in Figure 6, we show that the classifier focuses on the object of interest (the core features), and not the background/gender/co-occurring object - which are precisely the spurious cues.\n\n\n**Evaluation metrics:**\nIn Table 4, we evaluate the multi-shortcut abilities of ConBias. Multiple shortcut mitigation is a recent robustness problem introduced by Li, Zhiheng, et al. \"A whac-a-mole dilemma: Shortcuts come in multiples where mitigating one amplifies others.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023. UrbanCars is the standard dataset for evaluating multiple shortcut robustness, with BG-Gap, CoObj-Gap, and BG+CoObj-Gap as the evaluation metrics. Both Waterbirds and COCO-GB are single shortcut datasets, and as such, these metrics do not apply to these datasets. \n\n**Upsampling:**\nThe rebalance sampling algorithm receives the concept graph as a concept co-occurrence matrix. The algorithm iterates through all cliques with an order of decreasing clique sizes to make sure we would not double compensate for the imbalance, e.g., 3-cliques would impact the already-balanced 2-cliques if we operate in a bottom-up fashion. For each iteration, it retrieves all cliques of concepts of size k along with their corresponding frequencies with each class. The algorithm identifies the maximum co-occurrence count among all classes for each combination and checks if any class is under-represented by comparing its count with the maximum. If a class is under-represented, the algorithm computes the number of synthetic samples needed to balance the representation and adds this information to the results list. This process continues for all combinations and classes until all clique sizes have been processed. The output of the algorithm is a list of queries specifying the class, concept combination, and the number of samples needed to balance the dataset. We will include a pseudo-code in the final version for better readability.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "R9zYq4aCwu", "reviewer": "Reviewer_BpQx", "summary": "The paper introduces a method for addressing the issue of inherent bias in data during the training process. This bias can lead to unreliable predictions from the model. The proposed method, called ConBias, is a new approach for identifying and mitigating Concept co-occurrence Biases in visual datasets. The method represents visual datasets as knowledge graphs of concepts, allowing for detailed analysis of misleading concept co-occurrences that reveal imbalances across the dataset. Additionally, the author demonstrates how a new clique-based concept balancing strategy can address these imbalances, resulting in improved performance on subsequent tasks. Extensive experiments show that applying data augmentation based on the proposed method leads to significant improvements in generalization performance across various datasets when compared to state-of-the-art methods.", "questions": "https://arxiv.org/abs/2310.04562 proposes \"ULTRA\" KD foundation graph pre-training. Can using this method benefit the proposed graph structure initialization?", "limitations": "The authors have properly addressed the proposed method limitations.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is well-written and explains the method thoroughly. The authors' experiments on various common datasets demonstrate the method's effectiveness and the comparisons to a prior art show that it achieves state-of-the-art performance. The paper includes a thorough analysis of the method's components, justifying its contributions.", "weaknesses": "In the paper, I missed the ablation study of the authors' knowledge graphs design, the impact of using a different number of graph layers, knowledge graph architectures, different initializations, etc.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive comments. \n\n**Clarification on the ablation studies:**\nSince the concept graph phase in ConBias is not a learning mechanism, we circumvent the need for initialization or training strategies. This is beneficial as we achieve both controllability and interpretability of the bias diagnosis process. The imbalanced cliques provide an intuitive mechanism to diagnose the spurious correlations in the data. With regard to ablation studies, since the augmented data generation phase is a learning mechanism (we use stable diffusion with a CLIP based filter), here we present ablation results (Table 3). We show that using a different generative model such as IP2P still results in significant improvements over the baselines. This result demonstrates that it is actually the bias diagnosis and clique balancing strategy of ConBias that proves to be the true novelty. \n\n\n**Graph structure initialization:**\nWe thank the reviewer for referring to the Ultra-KD paper. Broadly, such an approach excites us immensely, where a knowledge graph encapsulates deep information about the data. For instance, in lines 317 and lines 324, we mention the potential of future work in developing more novel graph structures that can capture diverse biases beyond object co-occurrences. Foundational knowledge graphs with rich relational information may prove to be the next logical step in dataset diagnosis and debiasing. We will add this discussion to the final version, and thank the reviewer for raising this idea.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank the reviewers for providing valuable feedback on our work. We are glad that they found our idea "original" (R3); our diagnosing and debiasing method "structured and controllable" (R1), "novel, significant" (R3); our graph-based framework "a new approach to generate unbiased images" (R2), "robust" (R2); our graphs and illustration "informative" (R3); our experiments and justification "robust" (R3), "thorough" (R4); our paper "well-written" (R1, R4). 

In this rebuttal, we separately address each reviewer’s remaining comments as follows. Also, the attached PDF contains results from diagnosis experiments on Imagenet-1K, as requested by reviewer R2. We will incorporate all the feedback to the best of our abilities.

R1: Reviewer **xMok**; R2: Reviewer **XqYX**; R3: Reviewer **TVhQ**; R4: Reviewer **BpQx**.

Meta-review: The reviews are mixed, but most reviewers are leaning to accept. The primary concern that remains across several reviewers is that the proposed approach relies heavily on the existence of accurate and high-quality metadata, and was only tested on less-complex datasets without realistic bias. The ACs agree that these are limitations to the general application of this approach, but there are many domains (for example many areas of science) where curated metadata is available for all datapoints. Thus, while the method proposed may not be useful for all data, the contribution would be potentially impactful for some subareas and the ACs have decided to recommend accept.
Decision: Accept (poster)