Title: ConceptScope: Characterizing Dataset Bias via Disentangled Visual Concepts
Venue: NeurIPS 2025
Accepted: yes
Keywords: dataset bias analysis

vision datasets

sparse autoencoder
Abstract: Dataset bias, where data points are skewed to certain concepts, is ubiquitous in machine learning datasets. Yet, systematically identifying these biases is challenging without costly, fine-grained attribute annotations. We present ConceptScope, a scalable and automated framework for analyzing visual datasets by discovering and quantifying human-interpretable concepts using Sparse Autoencoders trained on representations from vision foundation models. ConceptScope categorizes concepts into target, context, and bias types based on their semantic relevance and statistical correlation to class labels, enabling class-level dataset characterization, bias identification, and robustness evaluation through concept-based subgrouping. We validate that ConceptScope captures a wide range of visual concepts, including objects, textures, backgrounds, facial attributes, emotions, and actions, through comparisons with annotated datasets. Furthermore, we show that concept activations produce spatial attributions that align with semantically meaningful image regions. ConceptScope reliably detects known biases (e.g., background bias in Waterbirds) and uncovers previously unannotated ones (e.g, co-occurring objects in ImageNet), offering a practical tool for dataset auditing and model diagnostics.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces ConceptScope to detect biases in datasets by disentangling visual concepts through a sparse autoencoder. This is achieved by training a linear sparse autoencoder to reconstruct visual features for a pretrained model. The authors found that the activation of sparse codes reveals one concept, e.g., object and texture. Based on this, the normalized alignment score is proposed to distinguish concepts for target, contextual, and bias. The approach is able to discover biases in any datasets and achieves good performance in 5 benchmarks for bias discovery. ConceptScope can be also used for robustness evaluation.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n1. The proposed method for concept discovering via SAE is simple yet effective, providing insights on diagnosing biases in datasets. \n2. The metrics of detecting bias is novel. The experimental results reveal good performance in detecting biases among datasets.\n3. The paper is easy to follow. \n\nWeakness:\n1. This work only validates the concept discovering for CLIP. As we know SAM is a powerful segmentation model, how well it will be to apply SAE into SAM?\n2. The comparison to feature clustering is missing. The features can generate good segmentation mask for concepts if proper threshold is used [1]. Therefore, performing clustering can also get visual concepts. It is vital to know the advantages of sparse autoencoder comparing to clustering.\n\nLanguage Issue:\n1. In line 217, \"For each target attribute y, we image-level activation value...\"\n\n[1] Wu J, Mo S, Awais M, et al. Masked momentum contrastive learning for zero-shot semantic understanding[J]. arXiv preprint arXiv:2308.11448, 2023.\n\nQUESTIONS:\nWhat are the advantages of SAE compared to clustering in concepts discovering?\n\nHow Figure 5 diagnoses model robustness? It is reasonable to find the target with biases getting the best accuracy. The trend cannot tell me how robust the model is. What is the robustest model from 34 models?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper leverages sparse auto-encoders (Cunningham et al [7]) to discover concepts related to the image target class, the context, and potential bias. Methodologically, it follows a similar line work from Lim et al [12], which are leveraging sparse autoencoders to reveal selective remapping of concepts. The paper relies on multimodal LLM to discover concepts that emerge from a vision encoder trained on ImageNet. Evaluation assesses whether such framework can predict concepts in various datasets and discover biases present in visual datasets.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n*S1. The method enables to discover novel biases*\n* Figure 4 illustrates novel biases being discovered by the proposed method. It is valuable that the method goes beyond known biases in a dataset and enables to discover relations between concepts that can affect the object classification performance.\n\n*S2. Concepts are visually grounded*\n* Figure 3 illustrates how discovered concepts are grounded in images. This offers a better interpretability of the discovered concepts.\n\n**Weaknesses**\n\n*W1. Related work section is insufficient*\n* The paper ignores related works in bias discovery, which is a new but already rich field. It would be beneficial to discuss differences in terms of datasets, biases, and evaluation protocol. For example, here are a few:\n  * Beery et al. Recognition in Terra Incognita. ECCV 2018.\n  * Hirota et al. SANER: Annotation-free Societal Attribute Neutralizer for Debiasing CLIP. ICLR 2025.\n  * Lang et al. Explaining in Style: Training a GAN to explain a classifier in StyleSpace. ICCV 21.\n  * Li et al. Discover the Unknown Biased Attribute of an Image Classifier. ICCV21.\n  * Zhang et al. Discover and Mitigate Multiple Biased Subgroups in Image Classifiers. CVPR 24.\n\n*W2. The annotation process for concepts is unclear*\n* The paper only mentions that it \"annotates latent dimensions using multimodal LLM based on reference with localized activations\". How many concepts were annotated, and how was this decided? Are there any manual checks to verify the validity of the annotations? Can any bias be introduced in the annotation process? Is there any manual input in the annotation process, or all is done by an LLM? In the current form, it is hard to understand how concepts are discovered and reproduce the annotation process.\n\n*W3. Out-of-distribution concepts are unclear*\n* The paper is based on discovering concepts present in ImageNet. What if we want to characterize a dataset for which concepts were not discovered?\n* Furthermore, what is a sufficient set of concepts to be able to perform the different tasks assessed in the paper?\n* On a related topic, in Table 1, the performance on the DTD dataset is quite poor. What makes this dataset challenging? Similar question for the RAF-DB dataset.\n\nQUESTIONS:\nI am happy to increase my score with improvements regarding discussion and comparison with missing related work, details about how concepts are annotated and their limitations.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces ConceptScope, a novel framework for identifying and characterizing biases within image classification datasets. The approach leverages a Sparse Autoencoder (SAE) trained on visual encoder representations (e.g., from CLIP) to automatically discover a dictionary of disentangled visual concepts. The framework then categorizes these concepts for each class as \"target\" (essential to the class definition), \"contextual\" (correlated but not essential), or \"bias\" (contextual concepts that spuriously co-occur with high frequency). The authors empirically validated the approach by investigating its effectiveness in concept prediction and localization tasks in comparison to VLM-based baselines. ConceptScore was then compared to other bias detection approaches in tasks such as detecting known biases in benchmarks like CelebA and Waterbirds. Its capacity to discover novel, unannotated biases in datasets like ImageNet was also investigated. Finally, the paper demonstrates a practical application for diagnosing model robustness by partitioning test sets based on concept activation strength.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- Discovery of Novel Biases: A significant strength is the framework's demonstrated ability to uncover previously unannotated biases in widely used datasets, such as cultural biases in the \"bridegroom\" class or object co-occurrence biases in the \"necklace\" class in ImageNet. This highlights the practical utility of the tool for dataset curators and researchers. \n\n\n- Strong Empirical Performance: In the task of detecting known biases, ConceptScope achieves state-of-the-art results, outperforming several baseline methods on benchmarks like NICO++ without requiring the training of additional classifiers specifically for bias detection.\n\n- Automation and Scalability: The framework seems to be designed to be fully automatic, identifying concepts and their distributions without needing costly, fine-grained human annotations for biases.\n\nWeaknesses\n\n- Clarity and Presentation: The overall presentation of the methodology is confusing. Key definitions are either vague or circular. For example, the paper defines \"visual concepts\" as \"groups of visual concepts\" and introduces \"concept strength\" as a critical measure without defining it upfront, making it difficult to understand the logic before reaching later sections. Figure 2, which is central to the method, lacks sufficient detail, particularly regarding the role of the VLM in building the concept dictionary and the degree of manual intervention required.\n\n\n- Oversimplified Bias Definition: The framework designates a single \"bias concept\" per class based on the contextual concept with the highest activation strength. This is a restrictive definition, as a class may have multiple spurious correlations (e.g., the \"dog\" class could be biased toward \"grass,\" \"parks,\" and \"tennis balls\" simultaneously).\n\n\n- Potential for Introduced Bias: The initial step of categorizing concepts into \"target\" and \"contextual\" may itself introduce bias. For instance, the paper suggests a \"smile\" is an essential target concept for a \"happy face\". This assumption could cause the model to incorrectly penalize or misinterpret valid images of happy faces without a visible smile, thereby introducing a new layer of characterization bias.\n\n\n- Evaluation Gaps: While the method outperforms baselines, the absolute performance on concept segmentation (AUPRC of 0.344) and concept prediction (average AUPRC of 0.66) suggests that the underlying concept extraction has significant room for improvement before it can be considered \"accurate\" or \"reliable\" in a standalone capacity. Furthermore, the submission lacks a comparison of the computational cost against baselines, which is an important practical consideration.\n\nQUESTIONS:\n- Clarification on Definitions: Could you provide a more precise definition of \"concept strength\"? What is the exact threshold used to determine if a strength is \"high\" when identifying a bias concept? The paper states it is $\\mu_{c', y}​+\\sigma_{c', y}$​​ , which seems to imply that there will always be concepts identified as biased. Is this intended?\n- Handling of Multiple Biases: Is the framework fundamentally limited to identifying only one bias concept per class? How would it perform in a scenario with several distinct, equally prominent spurious concepts?\n- Supervision and Manual Effort: Could you clarify which steps of the ConceptScope pipeline are fully automated and which require manual intervention? Specifically, what is the exact process and VLM-prompting strategy used to annotate the latent concepts in the dictionary, and how robust is this to the choice of VLM?\n- Experimental Details:\n  - What population are the error bars in Figure 3 calculated over?\n  - Could you provide a brief definition or a more direct reference for the Precision@10 metric used for the bias discovery task? The paper cites a source but does not define the metric itself.\n  - Computational Cost: How does the computational cost of ConceptScope (including SAE training) compare to the baseline methods in Table 2, which require training models on biased datasets to identify failure modes?\n  - Validity of Target/Context Separation: The stratification of concepts into \"target\" and \"contextual\" seems critical. How does the framework ensure this initial step is not itself biased? For example, using CLIP similarity to measure necessity and sufficiency might inherit any biases present in CLIP's text-image embedding space.\n\nLIMITATIONS:\nI think it is important to include in the limitations that the core assumption that concepts can be cleanly separated into \"target-essential\" and \"contextual\" categories is a strong one. As raised in the weaknesses, this process may introduce its own biases by enforcing a narrow, monolithic definition of a class (e.g., \"happy face must have a smile\"), potentially causing the framework to mischaracterize the dataset's diversity.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes ConceptScore, a framework for discovering bias concepts in class-labeled image datasets. It first uses SAE and VLM to build a concept dictionary from a pretrained model and an image dataset. Then it calculates several metrics based on SAE activations to determine the set of \"target\", \"bias\", \"non-bias contextual\" concepts for a given class y. Experiments demonstrate SAE is able to accurately capture and localize concepts, and on bias discovery task (both known and \"in the wild\") it outperforms existing methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The tackled problem is very important and underaddressed, especially evidenced by recent research.\n2. The framework of conception dictionaries together with target, bias, and non-bias contextual concepts is novel and intuitive.\n3. The usage of SAE is well motivated and demonstrated to be effective.\n4. The experiments conducted are diverse, covering both the SAE part, and the whole pipeline, with different kinds of datasets and tasks.\n\nWeaknesses:\n\n1. A core contribution is the definition to distinguish the concepts into 3 categories. However, I find this part not clear, and it is a very important part to improve.\n\nThe difference between the alignment scores (equation (4)) and the bias score (equation (5)) should be clarified. Are they highly correlated? What is the difference intuitively? To the readers they all seem to rank the relevance/co-occurance of the concepts in images of class y. In figure 2 (c) confirm this impression where the target concept also ranks higher in formula (5) but its definition is from eqn (4). The scores should be explained in an intuitive manner, not just mathematical languages, especially in clarifying the differences when it is quite subtle. \n\nAlso, the threshold in both (4) and (5) require some explanation/motivation.\n\n\n\n2. The necessity and sufficient score computation uses CLIP embedding of class label y as the \"ground truth\" - masked images with retained patches that generate closer image embeddings to this y embedding are considered closer to the \"true\" class definition. This could be problematic as CLIP itself could be biased in *its* training. The CLIP text encoder may already have learned to associate beaches with turtle, thus when the image contains beach but not turtle, CLIP can still give a high score. This would disturb the calculation of the scores and make it an inaccurate estimation.\n\n\n\n3. The final and very important step of identifying the concept dictionary is using a VLM (e.g., GPT-4o) to annotate, given a few concept-masked referenced images. This step is only described in one sentence. It would be great if more details are given, e.g., the prompt used, how confident the answers are, and how well it aligns with human judgement. Some examples can be presented.\n\n4. This work tackles image datasets with classes (e.g., imagenet), but image-text datasets are gaining dominance, especially in MLLM and generative model training. The work would be more significant if it tackles image-text datasets. This is also acknowledged by authors in the limitation section.\n\nMinor:\n\nLine 202-203: above the average of that of other context concept. Should it be \"above one standard deviation to the right of the average...\"?\n\nQUESTIONS:\nSee weaknesses.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}