Title: Automated Detection of Visual Attribute Reliance with a Self-Reflective Agent
Venue: NeurIPS 2025
Accepted: yes
Keywords: Model robustness; Interpretability; Interpretability Agent; Self-reflection; Interpretability benchmarks; Spurious correlation
Abstract: When a vision model performs image recognition, which visual attributes drive its predictions? Detecting unintended reliance on specific visual features is critical for ensuring model robustness, preventing overfitting, and avoiding spurious correlations. We introduce an automated framework for detecting such dependencies in trained vision models. At the core of our method is a self-reflective agent that systematically generates and tests hypotheses about visual attributes that a model may rely on. This process is iterative: the agent refines its hypotheses based on experimental outcomes and uses a self-evaluation protocol to assess whether its findings accurately explain model behavior. When inconsistencies arise, the agent self-reflects over its findings and triggers a new cycle of experimentation. We evaluate our approach on a novel benchmark of 130 models designed to exhibit diverse visual attribute dependencies across 18 categories. Our results show that the agent's performance consistently improves with self-reflection, with a significant performance increase over non-reflective baselines. We further demonstrate that the agent identifies real-world visual attribute dependencies in state-of-the-art models, including CLIP's vision encoder and the YOLOv8 object detector.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nUnderstanding the features utilized by a visual recognition model in its prediction is necessary for detecting improper functionality. While many methods utilized predefined concepts lists and/or manual inspection, the authors propose an automated method leveraging a multimodal LLM. This method incorporates a self-reflective mechanism that separates it from other automated methods and results in improved performance in attribute reliance detection. This is achieved by setting up the LLM to iteratively generate and test hypotheses about a recognition model's reliance on a particular attribute. Additionally, the author curate a benchmark of 130 models that are diversely prepared with reliance on 18 diverse types of visual attributes based on known vulnerability of visual models (e.g., spurious correlations). The models are given predefined reliance thus giving users the ability to assess attribute detection/reliance methods. Finally, the authors demonstrate their automated pipeline's ability to uncover attribute dependencies in pretrained models such as CLIP and YOLO.\n\nSTRENGTHS AND WEAKNESSES:\n- (S) Paper organization, problem communication, and method overview are very clear and well written.\n- (S) The method successfully detects reliance attributes in the recognition models and outperforms other reliance detection methods.\n- (W) The method seems to be reliant on several hyper parameters such as:\n\t- The choice of multi-modal LLM\n\t- The choice of text-to-image model\n\t- The choice of editing method to said text-to-image model\n- (S) The addition of a benchmark of 130 models with known attribute reliance is a valuable contribute to reliance detection method, but also explainable & interpretable methods in general\n- (W) The attribute reliance of these models is simplistic as it uses an external attribute classifier (either SigLIP or FairFace) to detect the attribute and simply lowers the classification score if the attribute isn't detected. Additionally, all of the 130 models are based on Grounding DINO.\n- (W) For the \"Counterfactual demographic attributes\" described in section 4.2, it is unclear to me what the main difference to \"demographic attributes\" is.\n- (W) The evaluation criteria is entirely dependent on the LLM, so it is unclear when hypothesis testing will end and when self-reflection will end.\n\nQUESTIONS:\n- With the lack of concrete threshold / criterion for when a stable explanation is achieved, will the agent ever determine that the model doesn't rely on unintended attributes, or will it keep trying to find one? Basically, what if the model works as intended?\n- The counterfactual setting is unclear to me, could you clarify that intended functionality this experiment is measuring?\n- Is having a reliance variable always an issue? For example, when assessing whether an image includes a teacher or scientist, one cannot determine from looking at the person alone, we have to use attributes such as \"a class room setting\", \"a laboratory environment\", etc.\n\nLIMITATIONS:\n- Limited by the conceptual understanding and capability by the chosen pretrained multi-modal model\n\t- The method involves the LLM proposing candidate attributes, generating & editing the images (which would then include accurate understanding of the attributes). If the LLM is capable of accurately classifying, then why use the other model? Are there cases where the LLM fails to classify or detect well, but successfully improves the model it is diagnosing?\n- The approach depends on a several larger and more complex models to explain smaller models.\n\t- Evaluation and and effectiveness is entirely reliant on the strength of the chosen LLMs, summarizers, and image generators.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes an automatic framework for detecting which visual attributes an object recognition model uses when identifying a visual concept, especially focusing on detecting unintended attributes. The framework uses a \"self-reflective agent\" that works in two stages.\n\nFirst, in the \"Hypothesis-Testing\" stage, the agent (based on Claude-Sonnet-3.5) is provided with a subject recognition model and a target visual concept and must discover on which visual attributes on the input image the model relies on in order to recognize the target visual concept. To that end, the agent proposes test hypotheses about the attributes, generates synthetic images with and without these suspected attributes, scores the images with the subject recognition model, and refines its proposed hypotheses based on the model's responses until it reaches a \"stable explanation\". \n\nNext, in the \"Self-Reflection\" stage, another language model evaluates the explanation from the first stage. It checks if the explanation matches the recognition model’s confidence scores by again generating and scoring (with the model) more synthetic images. If inconsistencies are found, it launches a new \"Hypothesis-Testing\" stage. \n\nThe paper introduces a new benchmark of synthetic models with controlled attribute dependencies to evaluate the framework. Experiments show that the framework improves with more self-reflection stages. It is also tested on real models like CLIP and YOLOv8, successfully identifying their visual attribute dependencies.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**    \n- The paper introduces a novel agentic framework for automated interpretation of vision models (i.e., automatically detecting the visual attributes on which they rely on), addressing an important and emerging research direction in explainable AI.     \n- The proposed framework appears to be well-designed and makes (overall) sense. The results demonstrate the advantage of integrating into the framework \"self-reflection\" stages.    \n- Also, it is supported by a novel benchmark, which can be proved useful for future works in this area.     \n- The paper is generally well-written and accessible, successfully communicating its key ideas (however, as discussed below, some descriptions lack technical clarity)     \n\n**Weaknesses:**    \n- The work appears to assume that the recognition model depends on a single, \"decisive\" visual attribute (meaning the model succeeds only if the attribute is present and fails otherwise). However, in practice, recognition models might rely on multiple attributes, and their decisions may depend on a combination of features rather than a single absolute factor. For instance, a model might recognize a concept when a sufficient subset of relevant attributes is present, rather than requiring all or none.     \n- Based on the above, and given that the framework’s reliance on synthetic images, it remains unclear how well the generated explanations align with the model’s behavior on real-world images (if I understood correctly, both the proposed framework and the proposed benchmark rely on synthetic images for providing explanations and evaluating them respectively). Did the authors test whether the generated explanations (e.g., for the real models CLIP and Yolov8) match the model outcomes on real-world images instead of synthetic ones?     \n- It is not clear what is the difference between the testing phase in the \"Hypothesis-Testing\" stage and the \"Self-Reflection\" stage. Both generate synthetic images with and without the visual attribute, then score these images with the recognition model, and observe if the hypothesis matches the model's score outcomes. So, why a different \"Self-Reflection\" stage is needed is unclear.     \n- While on high-level it is easy to understand how the approach works, the main paper's description of how exactly the two main stages work is not entirely clear. For instance, it is not clear how exactly the agent determines whether a hypothesis matches the observed results in the \"Hypothesis-Testing\" and \"Self-Reflection\" stage. What is the exact criterion for determining if the hypothesis and the results match or requires further refinement?  Does the framework relies exclusive on synthetic images for generating explanations about the visual attributes?\n- The proposed framework appears to be somewhat complex.\n\nQUESTIONS:\nPlease answer my concerns/questions/concerns listed on the weaknesses section above. Specifically:\n- How do you respond on my comment about the assumption that the recognition model depends on a single, \"decisive\" visual attribute?\n- How well the detected visual attributes from the synthetic images match the model's behavior on real-world images\n- What is the difference between the testing phase in the \"Hypothesis-Testing\" stage and the \"Self-Reflection\" stage\n- Questions about workings of the proposed framework\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes an automated agent for visual attribute reliance. The idea is to use an LLM agent that can propose hypothesis, tests them and iteratively refines them based on observations. One key contribution of the method is the self-reflection stage in which the agent uses a self-evaluation tool to score it's explanation. The authors also construct a synthetic dataset to measure the effectiveness of their proposed approach.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The use of self-reflection agent is quite interesting. It is a natural extension to MAIA style approaches.\n2. The proposed synthetic dataset can be useful. The method for constructing the dataset makes sense.\n\nWeakness:\n1. Main limitation is that the contribution is limited. I feel the use of agents for visual explanations have been proposed in works like MAIA. This work just adds a self reflecting agent on top of it.\n2. The self-reflecting agent assumes text to image model is good. But text to image model can have biases in itself. They might not be able to generate the presence or negation of some attributes. T2I models have especially been shown to be poor at negations. Given this, it is hard to conclusively tell how good the self-reflection stage is.\n3. There are a lot of components cascaded in this system. So, if one stage breaks, the entire system can fail. This concerns me.\n4. The diversity of generations from T2I models can also be low. Is that a concern?\n5. The synthetic dataset is constructed with simple object classes and attributes. What about the long tail category objects?\n\nQUESTIONS:\n1. Is there any error analysis that is done to show how sensitive or robust the method is when text to image models fail to generate the presence or absence of some attributes. How much error can we tolerate.\n2. T2I models can suffer when generating some rare classes, like rare breeds of dog etc. How can the self-reflective agent work in these settings.\n3. Can the LLMs come up with diverse set of candidate hypothesis? Is there a way to quantify this?\n\nLIMITATIONS:\nI feel the contribution over MAIA is limited. The over-reliance on text to image generators is a bit concerning when they themselves can have biases or failure modes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors introduce an agentic framework for identifying visual attribute dependencies in trained vision models. The core of the contribution is a \"self-reflective agent,\" implemented with a multimodal LLM, that systematically generates and tests hypotheses about the unintended visual attributes that a model may rely on. They also introduce a new benchmark of 130 models with synthetically injected, known attribute dependencies, which they use to evaluate their framework.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nFraming model interpretability as an iterative, scientific discovery process driven by a self-reflective agent is novel and potentially impactful for practitioners. \n\nThe introduced benchmark is highly useful for the field and its construction is well described. \n\nExperiments are well designed and the framework is compared to several reasonable benchmarks.\n\n\nWeaknesses\n\nThe framework is dependent on a TTI model to generate images for hypothesis testing and self-evaluation. This introduces a major potential confounder that the paper does not adequately address. TTI models are themselves trained on large web-scale datasets and often use CLIP-like embeddings; thus, they encode their own strong societal and representational biases. For example, when the agent tests a hypothesis like \"the model relies on corporate settings for the concept *suit*,\" it generates images with and without this attribute. However, when it prompts the TTI model for \"a person in a suit *not* in a corporate setting,\" the TTI model might, due to its own biases, change other attributes—it might generate a woman instead of a man, or change the style of the suit. The agent might then observe a drop in the subject model's score and incorrectly conclude that the \"corporate setting\" was the key attribute, when in fact the reliance was on the person's gender...\n\n\nThe paper frames the discovered dependencies as \"unintended\" or \"vulnerabilities\". However, some of the examples seem to be strong, statistically valid features that are part of a concept's representation. For example, the finding that CLIP's vision encoder recognizes \"teachers\" based on \"classroom backgrounds\" is not necessarily a spurious shortcut. In many contexts, the background is a defining feature. How else would a model distinguish a teacher from  another person in a professional setting? The framework currently lacks a mechanism to differentiate between a harmful shortcut (e.g., reliance on gender for a job-related concept) and a benign, contextually relevant feature. This makes the practical significance of some findings unclear.\n\nQUESTIONS:\nSee weaknesses.\n\nMinor: What the computational cost (e.g., API calls, time) for analysing a single concept in a model?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}