Is What You Ask For What You Get? Investigating Concept Associations in Text-to-Image Models
Authors: Salma Abdel Magid
Weiwei Pan
Simon Warchol
Grace Guo
Junsik Kim
Mahia Rahman
Hanspeter Pfister
Venue: Accepted by TMLR ()

Abstract: Text-to-image (T2I) models are increasingly used in impactful real-life applications. As such, there is a growing need to audit these models to ensure that they generate desirable, task-appropriate images. However, systematically inspecting the associations between prompts and generated content in a human-understandable way remains challenging. To address this, we propose \emph{Concept2Concept}, a framework where we characterize conditional distributions of vision language models using interpretable concepts and metrics that can be defined in terms of these concepts. This characterization allows us to use our framework to audit models and prompt-datasets. To demonstrate, we investigate several case studies of conditional distributions of prompts, such as user-defined distributions or empirical, real-world distributions. Lastly, we implement Concept2Concept as an open-source interactive visualization tool to facilitate use by non-technical end-users. A demo is available at https://tinyurl.com/Concept2ConceptDemo. 

Warning: This paper contains discussions of harmful content, including CSAM and NSFW material, which may be disturbing to some readers.

Published in Transactions Machine Learning Research (05/2025)
Is What You Ask For What You Get? Investigating Concept Associations in Text-to-lmage Models
Salma Abdel Magid, Weiwei Pan, Simon Warchol, Grace Guo, Junsik Kim, Mahia Rahman, Hanspeter Pfister
Department of Compater Science
Harvard University
Reviewed OpenReview: https: o/perrvveee net. forum? id= ud=mk1YI&Vv/7
Abstract
Text-to-image (T2I) models are increasingly used in impactful real-life applications. As such, there a growing need to audit these models to ensure that they gen- erate desirable, task-appropriate images. However, systematically inspecting the associations between prompts and generated content in human-understandable way remains challenging. To address this, we propose Concept2Concepp, a frame- work where we characterive conditional distributions of vision language models using interpretable concepts and metrics that can be defined in terms of these concopts. This charactcrization allows us to use our framework to audit models and prompt-datasets. To demonstrate, we investigate several case studies of con- ditional distributions of prompts, such as user-defined distributions or empirical, real-world distributions. Lastly, we implement Concept2Concept as an open-source interactive visualization tool to facilitate by non-technical end-users. A demo is available at hntips:/tinnuurlccom Concept2ConneetiDeng
Worning: This paper contains discussions of harmful content, including CSAM and NSFW material, which may be disturbing to some readers.
1 Introduction
Text-to-image (T2T) models have become central to many real-world AT-driven applications. How- ever, the complexity these models makes it difficult to understand how they associate concepts in images with textual prompts. Existing works have shown that T2I models can resolve prompts in unexpected ways (Bianchi et al. (2023) Furthermore, the training datasets for T2T models often large, uncurated, and may contain undesirable prompt-to-image associations that models learn to internalize (Birhane et al. (2024)). Thus, without robust auditing frameworks that help us detect these undesirable associations, we risk deploying T21 models that generate unexpected and inappropriate content for given task.
However, auditing T2I models challenging because it is difficult systematically, efficiently, and intuitively explore the vast space of prompts and possible outputs. Furthermore, raw pixel values are difficult to reason about semantically. In response, previous works proposed learning mappings

---

Published in Transactions Machine Learning Research (05/2025)
from raw pixel inputs to high-level concepts. This can be achieved post-hoe (Kim et al. (2018
Zhou et al. (2018); Ghorbani et al. (2019)) ) or as an intervention during training (Koh et al. (2020
Chen et al. (2020) Although these methods were designed for classification networks, it is this general intuition which motivates our work.
In this paper, we address the challenge of auditing the generative behaviors of T21 models by proposing a framework for producing interpretable characterizations of the conditional distribution of generated images given a prompt, pimmage prompt). We do so by extracting high-level concepts from each image and summarizing p(image prompt) in terms of such concepts. Here, we define concepts category that includes objects, nouns, ideas, and labels classes identified through the open vocabulary of an object detector.
Our contributions are as follows:
(1) We propose an interpretable framework for concept-association based auditing of conditional distributions of T2T models. Specifically, in our framework: we sample images from a T2I model, given a prior distribution over prompts either a user-defined distribution or an empirical real- world distribution. Then, using fast, scalable visual grounding model, we extract concepts from generated images. We characterize the conditional distribution of the generated images by analyzing the distribution of concepts. Our framework allows users to systemically investigate associations of conditional distributions at varying levels of granularity, from broad concept trends, co-occurrences, to detailed visual features. By design, our framework utilizes visual grounding models that localize concepts images, enabling a deeper analysis of visual representations. Simple association mining metrics help uncover non-obvious concept relationships.
(2) We demonstrate range of concrete use cases for our framework by auditing models and prompt datasets. In addition to demonstrating the elfectiveness of the framework, our analysis also exposed new findings that are independently interesting. In particular, we discovered child-serual abuse material (CSAM) in a human-preferences prompt dataset used to train an actively used evaluation metric, and misaligned classes in a synthetically generated ImageNet dataset. These findings not only demonstrate the utility our framework but also contribute to the broader discourse on the safety, fairness, and alignment of T21 models.
(3) We introduce an interactive visualization tool, based loop auditing of T2T models. Our tool allows users to explore and inspect the identified concept associations. To facilitate widespread use, we provide our framework as an open-source package, enabling researchers and practitioners to easily audit their own models and datasets.
framework, for human-in-thi
2 Related Work
Qualitative Bias Auditing of T2I models. There exists a rich body of studies that have qualitatively investigated biases in T2I models, focusing on social biases related to gender, race, and other identity attributes. For example, Bianchi et al. (2023 qualitatively demonstrated biases related to basic traits, social roles, and everyday objects. Similarly, Ungless et al. (022) manually analyzed images generated by T21 models and found that certain non-cisgender identities were misrepresented, often depicted in stercotyped or sexualized ways. Through several focus groups, Mack el al. (2024) found that. T2I models repeatedly presented "reductive archetypes for different disabilities". Qualitative evaluations thus play a critical role in exposing instances where generative models can be biased. However, given the large space of possible prompts and images, instance-
2

---

Published in Transactions Machine Learning Research (05/2025)
based bias probing alone cannot paint systematic picture of how T21 models may (mis)behave in applications.
Automated Bias Auditing of T21 models. A number of works have focused on automating bias detection at scale, For example, in Cho et al. (2023), the authors measured visual reasoning skills and social biases in T2I models by using a combination of automated detectors and human evaluations to assess the representation of different genders, skin tones, and professions. Likewise, Luccioni et al. (2024) employed Visual Question Answering (VQA) models and clustcring- evaluations measure correlations between social attributes and identity characteristics. TIBET (Chinchure et al. 2023) and OpenBias (D (D'nncà et al. (2024) dynamically generate axes of bias, either based on single prompt collection of input prompts, then use VQA model to detect bias based on the generated questions. Both TIBET and OpenBias do not operate on the general concept-level since they only use a VQA model. In other words, the delection mechanism is inher- ently limited to the question being asked. We argue that using fixed detection questions renders the system somewhat closod-set. Instead of restricting the analysis to specific bias axcs and answers (which might not even apply to the prompt and image pair), we propose examining the overall distribution of open-set concepts in the generated images. This approach focuses on analyzing what is actually generated rather than speculating about potential outputs. This perspective motivated our adoption of open-vocabulary object detectors. Lastly, this class of existing methods typically requires an additional large language model to generate the bias axes (and the questions for the VQA model), thus introducing significant additional computation.
Object-Centered Auditing of T2I models. Most closely related to our work is Try Before You Bias (TBYB) (Vice al. (2023)), which proposes an object-eentered evaluation methodology quantify biases in T2I models using image captioning and set of proposed metrics. Unlike our approach, TBYB does not extract or localize objects, limiting its ability to analyze concept- to-concept relationships. It focuses primarily on summarizing bias through a single metric rather than enabling deeper exploration via concept visualizations, co-occurrences, or stability analysis. Similarly, CUPID (Zhao et al. (2024)) presents a visualization framework that allows users to discover salient styles of objects and their relationships by leveraging low-dimensional density- based embeddings. However, CUPID is designed for interactive exploration of a single prompt a time and does not scale to real-world prompt datascts. In contrast, our method is built to handle large-scale evaluations by systematically analyzing object representations across diverse prompts.
3 Concept2Concept: An Intuitive Framework for Characterizing the Conditional Distribution of T21 Models
We propose Concept&Conceet, a novel framework to provide systematic and interpretable charac- terizations of the conditional distribution of images generated by a T21 model given a prompt, p( (image prompt). We do so by first extracting high-level concepts from generated images, then characterizing the conditional distribution of these concepts given prompts, p(concept/proopp)) Figure provides an overview of Concept2Concept.
Obtaining Concept Distributions from T2I Models. We assume distribution of text prompts p(t), defined by the user or the auditing task. We empirically represent p(t) with N sampled prompts {ti] from p(t):
t p(t), for i = 1,2,..... ..... N.
(1)
3

---

Published in Transactions Machine Learning Research (05/2025)
(t)
Pa (X tt)
C,, P (CC (81,))
11 photo of psant person
pioto
fo A phott of perse
op penton
Otgeet Detecter Hopel
Aprittoit pesced
Hocel
Stop Andletn
Faqu Stateity
Cn DOGUIINECA
Instion
POATTER
Figure 1: Concept2Concept enables users to systematically analyze the conditional distributions by investigating concept frequency and stability, co-occurrences, and detailed visual features. This approach enables comprehensive insights into underlying concept associations.
For each sampled prompt 4, we approximate the conditional distribution of images given prompt t by generating K images { from the T2I model G:
Fi,k Po(E,A)))) for h 1,2,.... K.
As image distributions are difficult for humans to work with at a global level, we focus on studying the distribution of concepts in the generated images. Specifically, for each image, we are interested in C(x), the set of concepts in image In practice, Fis by applying object detector D to label and localize (e.g., bounding box) the concepts in the image C, = (zz.,.). The choice of object detector D is not fundamental to our framework and can be coplication-ssecifi. For instance, in our experiments, we utilize two distinct detectors- Flooence 2 (Xiao et al. (2023) and BLIP VQA (Li et al. (2022) -aahh offering different levels of detection capabilities. The flexibility to choose D allows us to adapt the framework to various tasks, depending on what is important to detect and at which level of gramularity. We include an ablation study on the choice of detector in the appendix section A.4 Recent large vision-language models like Florence 2 offer multiple modes including visual grounding. We consider Ci, samples from distribution Ci, p(C|ar,,) where concepts are extracted deterministically from given image Iik (i.e. p(C]EL,) is a delta distribution).
compute C(Fi,)) for each generated image
Finally, we empirically approximate two distributions of concepts the marginal distribution of concepts over the prompt distribution, p(C), and the conditional distribution of concepts given a prompt, P(C)t). p(C) captures the distribution of concepts across all prompts whereas p(C]t) captures the distribution of concepts for specific text prompt, t. Formally:
4

---

Published in Transactions Machine Learning Research (05/2025)
P(C) [NCCWD() di, p(C|() - p(Cl))o(el)) dr.
In practice, both probabilities are empirically approximated by generating K images per prompt and extracting detected concepts using an object detector. Subsequently, the proposed metrics are also approximated empirically.
Summarizing Concept Distributions. We further summarize the concept distributions p(C) and p(C|() we obtain from the T2I model to help end-users explore and discover associations between concepts in the prompt and concepts in the generated images. To this end, we number of metrics to aid in analysis of concept associations. Visual examples of these metrics
are included in Figure ! along with a pedagogical experiment in section 4.1 to illustrate how they can be leveraged for interpretable and quantitative insights.
Concept Frequency P(c). We calculate the empirical frequency of each concept e across generated images. The probability P(c) in Equation 3
is estimated by:
Σ Ic E C₁
P(c) NXK
(4)
where T[e E Ci,+] is the indicator function that equals 1 if concept e is present in Ci,e, and 0 otherwise. This identifies the dominant concepts associated with the prompt distribution T.
Concept Co- Occurrence We analyze concept co-oceurrences uncover rich associations between concepts the generated images. For each pair of concepts (c, d'), we compute their co-occurrence probability:
Σ
Tc. d E C.N)
P(c,c))
NxK
This analysis helps map the relationships between concepts present in the images. Since the num- ber of detected concepts can be large and co-occurrences grow quadratically, we can employ simple association mining metrics to identify significant and relevant co-occurrences: support, confidence, and lift. We refer the reader to the appendix for additional details.
Choosing Task-Relevan Prompt Distributions p(t). The concept distribution p(C) depends on the choice of the prompt distribution p(t). Generally, the choice of p(t) should be informed by the task, e.g. auditing models for social biases. To clairfy how Concept2Concept can be used across different auditing nceds, provide structured guide outlining how users apply our framework based their specific goals in appendix table 2 Below, consider two primary scenarios for auditing: model auditing and prompt dataset auditing.
Model Auditing. In this scenario, the prompt distribution p(t) should be uscr-defined and should capture realistic ways users may interact with the model order to understand its behaviors. Here, users may generate controlled sets of prompts, possibly including counterfactual examples, to audit how the T2I model G represents specific concepts. By carefully designing p(t), users manipulate
5

---

Published in Transactions Machine Learning Research (05/2025)
the input conditions and study the resulting concept distribution p(C) marginalized over prompts p(t). This allows for targeted analysis of the model's behavior with respect to particular concepts biases. We provide several experiments in section
Prompt Dataset Auditing. When we are trying to understand the images generated from a set of prompts, p(t) should be an empirical distribution derived from real-world prompt datasets, such as those used reinforcement learning from human feedback (RLHF). By examining the concept distribution pCC) marginalized over prompts p(t), we can surface potential issues like harmful or inappropriate content in training datasets. We provide several experiments in section 5
Experiment Setup. Ezhaustive experimental details for each experiment appendir. The first three case studies focus on model auditing, and the second set of studies focus on auditing prompt datasets. In Application 1, we use Stable Diffusion (SD) 2.1, following existing works. To provide a comprehensive analysis of T2T models and their potential biases, we also performed cross-model evaluations. Specifically, we included two newly released models Lumina Next SFT Gao et al. (2024)) and SD3 Medium (Esser al. (2024)) our experiments for section 4.3 Both models currently lead the T2T leaderboards with thousands of downloads, and SD 3 Medium, released in July 2024, recorded approximately 52K downloads in October 2024 alond] To demonstrate that our framework is model-agnostic and can be applied to closed-source models accessed via API calls, we also conduct an experiment using ChatGPT model 4o. Results from these additional experiments can be found in the appendix. In Application 4.1, the model varies because relies on a real-world human preferences dataset, where different images and prompts were generated using various models, including SD, SDXL, and Dreamlike Photoreal. In Application 4.2, the StablelmageNett dataset was generated using SD 1.4.
be found in the
4
Application 1: Auditing the Model
4.1
Case Study Small Pedagogically Designed Prompt Set
Top Concepts for Each Action
Action
0.6
jogging
P(Concegg)
running
0.4-
sprinting
0.2-
0.0
shorts
bree
elderty Labint
Nog Aoy
grass
glasses
jacket
logging
fence
girl
beard
aged
lank
shadow
track
strost
aydnoo
Sped
Toye
The
Cancept
Figure2 2: Top concepts detected by our framework. Concepts are curated highlight the effec-
tiveness of the framework for user-defined prompt distributions in Section 4.1
We first demonstrate how use Concept2Concept to probe for unexpected generation behaviors on a small, pedagogically designed prompt set. We designed prompt set that varies along a
6

---

Published in Transactions Machine Learning Research (05/2025)
tespiing
Concept
Speinting
lagging
Ranning
legginge
glocooo
Figure 3: Examples of concepts extracted from our framework along with sample co-occurrencess
social attribute of interest, age, and a second prompt set that adds an axis of variation along semantically similar words (e.g. jogging vs running). Concretely, our prompt distribution is a uni- form distribution over the set. {*A photo in {young, middle-aged, old}, and [action] takes value {jogging, spriniing.uunning)...
[age] person [action] "}, where [age] takes value
In Concept2Concept, comparing conditional concept distributions helps us identify concept associations. Figure 2 shows the conaltional concept distributions p(C(() obtain
through Concepi2Concept. By comparing these distributions, we find that the concept jogging is largely associated with the concept woman (the concept of "woman" " occurs in roughly 60% of the generations). Conversely, running is associated with man in about 80% of the generations. We are also able to discover that different attires are associated with the concept of jogging and running respectively (see Figure 3).
In Concept2Concept, visually grounding concepts helps verify that concepts are resolved as we desire. Figure provides a small example of concept co-occurrences. Even seemingly concrete concepts can be visually resolved in diverse ways. The localization of our framework is highly precise, even for small objects like g glasses which occupy only a small fraction the entire image. Using Concept2Concept, we can identify, compare, and contrast the conceptual representations in generated images resulting from different prompts. This enables us to uncover unexpected concept associations (e.g. boy and sprinting vs.
woman and jogging Additional
results, including concept stability included in A.8
4.2
Case Study 2: Reproducing Bias Probing Results from Literature
Model
Concept Frequency
U.S. Labor Bureau
% woman
% man
% woman
% man
StableBias (Luccioni et al. (2024)
31.10 %
68.90%
Ours
28.41 %
71.59%
47.03%
52.97%
TBYB
Vice et al.
2023
31.64%
68.36%
Ours
19.56 %
80.44 %
Table The average concept frequency of Woman and man framework for the StableBias (Luccioni et al. (2024) and TBYB (Vice et al. (2023) case studies. Note that these are two different case studies with different experimental settings.
generated by concept detector in our
Using our framework Concept2Concepl, we demonstrale that we can reproduce experimental results from multiple existing works on gender-based bias probing. We consider two studies, each using different probing framework: StableBias (Luccioni et al. 2024 and Try Before You Bias (TBYB)
7

---

Published in Transactions Machine Learning Research (05/2025)
(Vice et al. (0033) In both works, the authors prompt T21 model with names of professions and report the distribution of gender reprosentation (in percentagos) among the generated imagos. Our findings are summarized in Table 1 Consistent with the two existing studies, we found that the concept woman is underrepresented across most professions, with only about 30% of the images depicting the concept womann, while approximately 70% of the images depicted the concept man While we were able to reproduce similar gender distributions as StableBias, our distributions are notably different. from those reported for TBYB. We provide a discussion for this discrepancy A.8 Concept2Concept generalizes bias auditing by providing a flexible framework in which existing methods can be seen as specific instantiations. By adapting object detection approaches, framework can, for example, emulate VQA-style probing model for targeted bias analysis (e.g., gender representation) or perform a more comprchensive bias audit using general object detectors that capture broader concept relationships. This flexibility allows Concept2Concept to unify and extend prior works, demonstrating that approaches like StableBias and TBYB are specialized cases within a more generalizable and scalable methodology.
4.3 Case Study 3: Scaling Up Qualitative Studies Disability Representation
Tay Canaegs ler's Auaalay
Cerrept
36
0P-
06
Piomett Becision
Visualle
SaCrASSA: wheelchar, Inceaatn
Oasagt
Figure 4: Concept distributions for two examples of disability representation. We can leverage the concepts to alter the images in a human--nderstanddee way through simple negative prompting. We can also visualize unexpected co-occurrences of specific concepts: shorts and sticks.
We replicated and extended findings from qualitative study on disability reprosentation in T2T models, which involved focus group to evaluate the generated outputs (Mack et al. (2024) ). Our framework automates this process by conceptually quantifying how the model represents disabilities across various prompts. Concretely, Teisability = (4, = A person with [value] where [value] e {a disability, bipolar disorder, a chronic illness, cerebral palsy, a limb difference, hearing loss]. Fig- ure 4 (top left) shows that for the prompt "a person with a disability", nearly 100% of the generated images depicted a wheelchair, despite not being explicitly stated in the prompt. When analyzing specific disability-relateed prompts, the model produced similarly stereo- typical associations. For instance, the prompt "cerebral palsy" primarily generated images of young and boy while "a limb difference" (Figure bottom left) resulted in images with the concepts shorts and Toot. Individuals in the images were typically dressed in shorts to emphasize their disa Unexpectedly stick co-occurred with Shorts. We visualize this in Figure 4 (bottom right) and find
that the model produces branch-liie like sticks, perhaps to represent crutches. In the case of "chronic illness", the model often depicted people in hospital, beds, with their faces covered. Additional results and detailed experimental setup can be found in the appendix AA10
8

---

Published in Transactions Machine Learning Research (05/2025)
We demonstrate how the framework's conceptual characterization of the conditional distribution can be useful for adjusting the T2I outputs. Figure shows how prompting with concepts wish to attenuate and/or amplity. A common approach for negative prompting involves replacing the empty string in the sampling step with negative conditioning text. This modification leverages the core mechanics of Classifier-Free Guidance by substituting the unconditional prompt with meaningful negative text. Suppose we want to exclude the concept wheelchair and emphasize Jace and person in the images. Our framework, coupled with simple prompt revision, enables users to dircetly alter conceptual output distributions. This case study thus illustrates the framework's s ability to identify harmful and unexpected biases.
apply negative
5 Application 2: Auditing Prompt Datasets
Concepts Co-occurring with "girl"
Count
10a
Mean
109
Concepts Co-occurring with 'woman'
102
10a
Mean Co
Concept
Figure 5: Co-occurrences of concepts with the detected concepts and woman in 10 random samples of the Pick-A-Pic Dataset KKisttan et al. (2023) Additional results are in A.13
5.1 Case Study 4: Detecting Unexpected Issues in Pick-A-Pic
Warning: This section contains discussions of harmful contend, including CSAM and NSFW ma- terial, which may be disturbing to some readers.
The Pick-a-Pic dataset (Kirstain et al. 2023 is one of many human preferences datascts consisting of prompt-image pairs. The authors reasoned that these "human preferences datasets" are useful for realigning T21 models so that they produce output users actually want to see. Kirstain al.
Additional results including model experiments and implementation details be found in the appendix.

---

Published in Transactions Machine Learning Research (05/2025)
concept nakked
nudity
nudity
nudity
nudity
nudity
prompp
androsymous
womann reeding book the botthib
Japanese redheed woman
Alrikan women wet clothee
concept underwear
concept
pronppt locsibattion
Sypee
Hapher scaua laaton
lcation
laation
insuall Laaton
lcation
w0s Laaion
sesual latton
preteen body
blonde 188e
handsome musculer man
atractiec
on all
glrt
stamiing
girl banding
hot ady
swaring beins
Figure 6: Prompts in the Pick-a- PPc dataset that trigger the naked, underwear, and thong concept associations. None of the prompts explicitly call for nudity or bypen-sexualization (HS).
2023) trained the PickScore the Pick-a-Pic training set to learn collected human preferences. The PickScore has since been used (1) as a standalone evaluation metric to measure the quality of any given T2I model and (2) to improve T21 generations by providing a ranking of a sample of images given a prompt. It is clear that both use cases are incredibly safety-critical. We used Concept2Concept to explore concept associations Pick-a-Pir and audit the dataset for unexpected and undesirable associations. Notably, our analysis of concept associations in Pick-a-Pic revealed child sexual abuse material (CSAM), pornography, and hyper-sexualizatiio of women, girls, and children.
We draw 10 random samples of size 1K cach from the training split of the Pick-a-Pic dataset In addition to the prompls and images, each row indicates which image the user ranked higher. When sampling, we save the images that the user ranked as better for a corresponding prompt. In the case of a tie, we randomly choose one of the two images. Our analysis thus conducted on the prompts and images that would reward model for choosing similar content.
Figure 5 shows concept co-oceurrences for the concepts girl and woman In addition to the stereo- typical and non-diverse concept co-occurrence distribution, we highlight in red concepts that may warrant additional investigation or probing. We find that the concept girl co-occurs with the con- cepts young maked, mude thong. underwear, and lingerie anngeriaaoong others. Similarly, woman co-occurs with naked breast and lingeric. We investigated through concept localization and determined the input concepts (promptss) associated with the delected concepts. Figure G we show examples of this process. Notably, none of these prompts explicitly call for harmful material, yet the models output-and the users chosc-mudity, hyper-soxualization. CSAM, and porno- graphic material. For example, the top of figure shows that the prompt "Japanese redhead woman" produced naked individual. Similarly, the prompt. "An asian woman" and "Afrikan woman wet clothes" produced hyper-sexualized ([hong) and naked content. We note that hyper- sexualization when not necessarily desired or explicitly stated in the prompt is not limited to woman or girl but is also exhibited for man and boy Additional results are shown in the appendix, along with the overall top detected concepts, with confidence intervals.
httpps http://wwwgnggooooo////////////n
10

---

Published in Transactions Machine Learning Research (05/2025)
Where user may not elicit pornographic material, a T21 model will enforce it. Moreover, due to the design of the web app used to collect the dataset, users are presented with two images at time and new image is presented only when the user ranks one of the existing images. The user can only break out of the ranking if they change the prompt. Pick-a-Pic was filtered automatically using a list of NSFW keywords. This list was not released. Using our framework, we show that these problematic concepts do not necessarily occur in the prompt, but the associations are still present. As such, their filtering scheme may not be the most effective way of auditing the dataset. We emphasize that this is of high importance for several reasons. First, T2I models have been shown to memorize the original training data (Carlini et al. (2023) ); there is possibility of replicating real CSAM and pornography. Second, the fact that this harmful material is also in dataset that is used to realign and evaluate T2I models should not be understated. The human has to be in the loop, and our framework simplifies this by characterizing the distribution in terms of human-understandable concepts.
We recognize the significant effort that goes into building largo-scale datasots like Pick-a-Pic, and we appreciate the value it brings to the research community. We shared these findings with the dataset authors, including the unique IDs of the flagged rows to facilitate review. The authors responded promptly and took the dataset offline. We are interested in further collaborations to support the continued improvement of such datasets, and we hope our findings can contribute to their safe community-drivee development.
5.2 Case Study 5: Detecting Misalignment in Synthetic ImageNet
ski
redbone
turnstile
ski_ mask
REDSOTN
GANK
Identified through localization of dreadlocks and co-occurrence: dreadlocks beanie, mask, ski
Figure 7: Sample of misaligned synthetic ImageNet images detected through the conceptual char- acterization of conditional distributions through our framework. We include the first 9 images from each class. There clear misalignment in these concepts. All 100 images for these classes well other detected misaligned classes can be found in the appendix.
In this section, we demonstrate another example of auditing the prompts (and the T2I model) used to generate a synthetic ImageNetlk (Russakovsky et al. (2015)) dataset. Many works demonstrate that using synthetic ImageNet, either to augment real ImageNet or entirely replace it, boosts performance, as discussed in the related works section 2 Moreover, works also use synthetic versions of ImageNNet to evaluate other models (Bansal Grover (2023)). These are two important and safety-critical use cases of T21 model outputs, which motivate our use of the Concept2Concept framework to investigate their concept associations.
11

---

Published in Transactions Machine Learning Research (05/2025)
Following TBYB (Vice et al. (2023)), we audit the synthetic StablelmageNNet dataset (Kinakh (2022)). ). Coneretely, TstahlolmagoNet {t, "a photo of [value], realistic, high quality where [value] E (ImageNet1K Classes]. Several existing works have experimented with similar setup to this generation procedure (Bansal & Grover (2023) and [Saryildız et al. (2023) ) Using our framework, we identified misaligned concept associations in Figure 7 Through the localization and co-occurrence of the concept dreadlocks with beanie, mask and ski we found several classes had
completely misaligned images. For example, the class turnstile in real ImageNet intended
to be "A narrow, mechanical gate, with rotating arms of wood or metal..: However, the T2I model generated photos of musical band called Turnstile. Similarly, for the class redbone, the intended ImageNet class refers to "A variety or breed of American hound with a predominantly red coat..." "4 However, the model instead generated images of human individuals. Another set of issues arises in two closely related classes: ski and ski mask. First, the model did not produce ski content, and second, the model replaced it with individuals with certain skin tones and hairstyles. The issue is thus two-fold: one of prompt adherence and one of fairness. One can attribute the failure to cither a vague prompt or a poor T2I model. In any case, raises concerns regarding both the dataset and the model's accuracy and bias. It is also important to note that while this exact dataset was not published in a specific paper, the recipe for generation is replicated in other works as a comparison point (Bansal & Grover (2023); [Saryıldız et al. (2023) demoustrating that the model (1) actually learns good representations with this recipe and (2) presents an approach practitioners actively use and investigate.
6 Open Source Interactive Tool
Given the ubiquity of T2I models and, as demonstrated in the case studies, the problematic concept associations and underlying prompts they may contain, there is broad need for further analysis of these models and their corresponding datasets. To lower the technical barrier for such auditing, we propose an interactive visualization tool that integrates seamlessly into the Jupyter Notebook environment. The primary advantage of using a widget is its seamless integration within the Jupyter ecosystem, which is already a cornerstone of much of machine learning research and development. Users can investigate specific concepts, their stability, and co-occurrence with other concepts (Figure 3. Additionally, users may search for specific concepts to identify the prompts used to generale the concept, the distribution of these prompts, and localize how the concept is depicted in different images (Figure 35 Finally, users can quickly share their findings with others through web- basd versions of Jupyter, such as Colab. This helps democratize the auditing process and enables greater collaboration on what may be sensitive or harmful issues. Readers can conduct their own exploration using a demo of our tool at https:/ http://tttyuuu.rrr Concept2CoonceptDeen
7 Conclusion
We proposed an interpretability framework designed to characterize the conditional distribution of T21 models in terms of high-level concepts. The purpose of this framework is to provide users with an in-depth understanding of how T2I models interpret prompts and associate concopts in generated images. By providing in-depth analysis through metrics such as concept frequency, stability, and co-occurrence, we reveal biases, stereotypes, and harmful associations that other frameworks may overlook. We also note that our findings of misaligned classes in StablelmageNet and CSAM and pornographic material in the Pick-a-Pir Pic dataset are independently significant.
40xford Dictionary
12

---

Published in Transactions Machine Learning Research (05/2025)
References
Salma Abdel Magid, Jui-Hsien Wang, Kushal Kafle, and Hanspeter Pfister. They're all doctors: Synthesizing diverse counterfactuals to mitigate associative bias. arXio preprint arXiv:2406. 11331, 2024.
Shekoofeh Azizi, Simon Kornblith, Chitwan Saharia, Mohammad Norouzi, and David J Fleet. Syuthetic data from diffusion models improves imagenet classification. arXiv preprint
arXim:2304.08466 2023.
Hritik Bansal and Aditya Grover. Leaving reality to imagination: Robust classification via generated datasets. International Conference on Learning Representations Workshop Trustworthy and Reliable Large-Seale ML Models, 2023.
Federico Bianchi, Pratyusha Kalluri, Esin Durmus, Faisal Ladhak, Myra Cheng, Debora Nozza, Tatsunori Hashimoto, Dan Jurafsky, James Zou, and Aylin Caliskan. Easily accessible text-to- image generation amplifies demographic stercotypes at large scale. In Proceedings of the 2023 ACM Conference Fairness, Accountability, and Transporency, pp. 1493-1504, 2023.
Abeba Birhane, Sanghyun Han, Vishnu Boddeti, Sasha Luccioni, et al. Into the laion's den: Inves- tigating hate in multimodal datasets. Advances in Neural Information Processing Systems, 36, 2024.
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European conference on computer vision, pp. 213-229. Springer, 2020.
Nicolas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramer, Borja, Balle, Daphne Ippolito, and Eric Wallace. Extracting training data from diffusion models. In 32nd USENIX Security Symposium (USENIX Security 23), pp. 5253 5270, 2023.
Zhi Chen, Yijie Bei, and Cynthia Rudin. Concept whitening for interpretable image recognition. Nature Machine Intelligence, 2(12):772 782, 2020.
Aditya Chinchure, Pushkar Shukla, Gaurav Bhatt, Kiri Salij, Kartik Hosanagar, Leonid Sigal, and Matthew Turk. Tibet: Identifying and evaluating biases in text-to-image generative models. arXiv preprint 2rXim:2312.01200 2023.
Jaemin Cho, Abhay Zala, and Mohit Bansal. Dall-eval: Probing the reasoning skills and social biases of Lext-lo-image generation models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3043-3054, 2023.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pp. 248 255. leee, 2009.
Moreno D'Incå, Elia Peruzzo, Massimiliano Mancini, Dejia Xu, Vidit Goel, Xingqian Xu, Zhangyang Wang, Humphrey Shi, and Nicu Sebe. Openbias: Open-set bias detection in text-to-image gen- erative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognation, pp. 12225-12233 2024.
13

---

Published in Transactions Machine Learning Research (05/2025)
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Bocsel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-fire first International Conference Machine Learning, 2024.
Kathleen C Fraser and Svetlana Kiritchenko. Examining gender and racial bias in large vision- language models using novel dataset of parallel images. arXiv preprint (xxx:2442.007707 2024.
Peng Gao, Le Zhuo, Ziyi Lin, Chris Liu, Junsong Chen, Ruoyi Du, Enze Xie, Xu Luo, Longtian Qiu, Yuhang Zhang, et al. Lumina-t2x: Transforming text into any modality, resolution, and duration via flow-based large diffusion transformers. arXiv preprint arXit:2405005945 2024.
Amirata Ghorbani, James Wexler, James Y Zou, and Been Kim. Towards automatic concept- basod explanations. Advances in neural information processing systems, 32, 2019.
Ruifei He, Shuyang Sun, Xin Yu, Chuhui Xue, Wenqing Zhang, Philip Torr, Song Bai, and Xiaojuan Qi. Is synthetic data from generative models ready for image recognition? arXiv preprint Xrxin:2210. 0.07774, 2022.
Phillip Howard, Avinash Madasu, Tiep Le, Gustavo Lujan Moreno, Anahita Bhiwandiwallaa, and Vasudev Lal. Socialcounterfactual::: Probing and mitigating intersectional social biases in vision- language models with counterfactual examples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11975-11985, 2024.
Been Kim, Martin Wattenberg, Justin Gilmer, Carric Cai, James Wexler, Fernanda Viegas, al. Interpretability beyond leature altribution: Quantitative testing with concept activation vectors PMLR, 2018.
(tcav). In International conference machine learning, pp.
V. Kinakh. Stable imagenet-1k dataset. htttss//rrr..aggggaaa//////ttttr stable-imagenet.ix 2022. Accessed: 2024-09-30.
Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick-a-pic: An open dataset of user preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36:36652 36663, 2023.
Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In International conference on machine learning, pp. 5338 5348. PMLR, 2020.
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre- training for unified vision-language understanding and generation. Tn International conference on machine learning, pp. 12888 12900. PMLR, 2022.
Youwei Liang, Junfeng He, Gang Li, Peizhao Li, Arseniy Klimovskiy, Nicholas Carolan, Jiao Sun, Jordi Pont-Tuset, Sarah Young, Feng Yang, et al. Rich human feedback for text-to-image genera- tion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19401-19411, 2024.
Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint Xrxi::2303 05499, 2023.
14

---

Published in Transactions Machine Learning Research (05/2025)
Sasha Luceioni, Christopher Akiki, Margaret Mitchell, and Yacine Jernite, Stable bias: Evaluating socictal reprosentations in diffusion models. Advances in Neural Information Processing Systems, 36, 2024.
Kelly Avery Mack, Rida Qadri, Remi Denton, Shaun K Kane, and Cynthia Bennett. "they only care to show us the wheelchair": disability representation in text- text-to-image ai models. In Proceedings of the CHT Conference Human Factors in Computing Systems, pp. 1-23, 2024.
Matthias Minderer, Alexey Gritsenko, and Neil Houlsby. Scaling open-vocabulary object detection.
Advances in Neural Information Processing Systems, 36:72983 73007, 2023.
Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei. 2osmos-2: Grounding multimodal large language models to the world. arXiv preprint Xix:223006 14824, 2023.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115:211 252, 2015.
Mert Bülent SarıyIldı, Karteek Alahari, Diane Larlus, and Yannis Kalantidis. Fake till you make it: Learning transferable representations from synthetic imagenet clones. In Proceedings of the IEEE/CVI Conference Computer Vision and Pallern Recognilion, pp. 8011-8021, 2023.
Brandon Smith, Miguel Farinha, Siobhan Mackenzie Hall, Hannah Rose Kirk, Aleksandar Shtedrit- ski, and Max Bain. Balancing the picture: Debiasing rision-language datasets with synthetic contrast sets. arXiv preprint arXim:2305.15409 2023.
Eddie Ungless, Björn Ross, and Anne Lauscher. Stereotypes and smut: The (mis) representation of non-cisgender identities by text-to-image models. arXiv preprint arXiv:2305. 17072, 2023.
Jordan Vice, Naveed Akhtar, Richard Hartley, and Ajmal Mian. Quantifying bias in text-to-image
generative models. arXio preprint Xiv:2211 13053, 2023.
Bin Xiao, Haiping Wu, Weijian Xu, Xiyang Dai, Houdong Hu, Yumao Lu, Michael Zeng, Ce Liu, and Lu Yuan. Florence-2: Advancing unified representation for variety of vision tasks. arXiv preprint or arXhn:2311.06242 2023.
Jaazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36, 2024.
Yayan Zhao, Mingwei Li, and Matthew Berger. Cupid: Contextual understanding prompt- conditioned image distributions. In Computer Graphics Forum, pp. e15086. Wiley Online Library, 2024.
Bolei Zhou, Yiyou Sun, David Bau, and Antonio Torralba. Interpretable basis decomposition for visual explanation. In Proceedings of the European Conference on Computer Vision (ECCV), September 2018.
15

---

Published in Transactions Machine Learning Research (05/2025)
Table 2: Guidance on selecting prompt distributions based on different auditing goals.
User's Goal
Prompt Distribution
Test how model represents specific concepts in controlled way.
Small, manually designed set of prompts targeting key concepts.
Validate & supplement known blases reported in the literature.
Prompt sets from prior bias-audlitigg studies (e.g., =:30 professions from U.S,
Labor Statistics).
Supplement existing qualitative study with quantitative insights.
A targeted expansion of prompts based qualitative findings.
Audit large-scale cuman-preeerencee datasets. Audit large-scale synthetic datasets
A dataset of real-world user prompts and chosen synthetic images.
A datasct of real-world class labels used as prompts to generate training images.
A Appendix
A.1 Additional methodological details.
Due to space constraints, we also include An overview our method can be found in Figure I concept stability below.
Concepl Slabilily. To assess the variability of concept across prompls, we compute its coefficient of variation (CV)
N Σ 1=1
(C
Cc P(c)
de
(6)
N
We a threshold to focus on concepts that oceur with sufficient frequency: C= fcE C| P(c) 7). Persistent concepts those that consistently appear regardless of the prompt (small CV), while triggered concepts more sensitive to specific concepts within the prompts (large CV).
A.2 Generalization and Practical Utility
We have structured our experiments to demonstrate a range of use cases for Concept2Concept. Rather than applying the framework to a single dataset or use case, we show how it can be adapted to different auditing goals. Additionally, Concep12Concept is inherently model-agnostic and can be applied to any T2I system, making broadly useful across different architectures and datasets.
Guiding Future Users To make clcarer how Concept2Concept can be used across different auditing needs, provide structured guide outlining how users can apply our framework based their specific goals in Table 2
A.3 Ethics Statement
We recognize that the detection of concept does not imply an absolute truth. The definition of concept is subjective and can vary across different contexts, shaped by socictal, cultural, and
16

---

Published in Transactions Machine Learning Research (05/2025)
historical influences. Concepts are not neutral; they carry power dynamics that affect how they are understood and applicd.
For instance, when the framework detects woman Asian it is important to recognize that these labels are not necessarily true, as such attributes cannot be reliably inferred from visual cues alone especially in the context of synthetic images. Since these images are artificially generated by models, the concept of identity tied to real-world characteristics, such as gender or ethnicity, becomes even more ambiguous. In this sense, the labels applied to synthetic images are inherently inaccurate, as they refer to constructs rather than real individuals. However, these detections are still valuable because they help expose biases within the models and datasets. By surfacing such issues, tool provides insight into how certain concepts (epreeeedeee (ivvr)simpliffed, allowing for critical evaluation and improvement of text-to-image models.
Beyond identity-related concepts, the way certain attributes and associations are reflected in AL- generated content can also reinforce harmful stereotypes and biases. A key example is the asso- ciation of nudity and revealing clothing with specific demographics. While mudity itself may not be considered inherently problematie, becomes concerning when it is disproportionately over- represented for marginalized groups, contributing to hypersexnalized biases in AT-generated out- puts. This issue is particularly critical in datasets that are repurposed for training and evaluation pipelines- such as Pick- A-Pic and StablelmageNet- -weeee unchecked biases risk being propagated into downstream AT applications.
The broader AI research community has recognized the importance moderating Al-generated nudity, as evidenced by the prevalence of NSFW filtering both research and industry practices. Our framework does not impose normative judgments but rather provides systematie approach to characterizing and making these associations transparent, cnsuring that such biascs can be critically examined rather than silently perpetuated.
Similarly, other design choices in our work reflect inherent biases. For example, our use of U.S. labor statistics as a comparison point introduces bias by privileging a specific cultural and national framework, which is not be representative of broader, global contexts. This comparison inher- ently reflects the dominant perspective from which the data was sourced, potentially excluding or misropresenting other groups.
These biases and design choices, whether in concept detection, dataset selection, or interpretive framing, shape the outcomes of our work and how it is understood. We acknowledge that the definitions we apply and the concepts we choose to highlight are not neutral; they actively influence the narrative and meaning of our results. Therefore, we strive to remain aware of the ethical implications of our decisions and aim for transparency in acknowledging the limitations and biases inherent in our work.
A.4 Limitations
While our framework provides valuable insights into the concept associations learned by text- to-image (T21) models, it has several limitations that are important to acknowledge. First, the interprotability of the results depends heavily on the quality of the object detection model. If this model fails to accurately detect objects or introduces its own biases, the subsequent analysis can be skewed. Since our framework primarily focuses on visually identifiable features such as objects and entitics, it does not explicitly capture high-level stylistic attributes. We acknowledge that
17

---

Published in Transactions Machine Learning Research (05/2025)
capturing artistic styles (e.g., Van Gogh's distinct visual patterns) presents a different challenge, as these styles are defined by global image features rather than discrete objects. Future work could explore what visual elements (e.g., brushstroke patterns, color palettes, composition structures) contribute to the recognition of artistic styles and how they might be systematically detected.
Second, the computational complexity of analyzing co-occurrences can grow significantly with the number of detected concepts, especially in large-scale datasets or highly complex prompts. An- other important direction is to explore active mitigation strategies that go beyond prompt revision. This could include integrating the framework with model training pipelines intervene during the training process, helping to guide the model toward learning more equitable and unbiased representations.
A.5 Additional Related Works
Auditing Synthetic Data and Prompt Datasets. One important use case of synthetic data is for training backbone or foundation models. Works have demonstrated that training backbone models using synthetic ImageNet (Deng et al. (2009)) clones can achieve similar performance on specific evaluation benchmarks as compared to the real ImageNet dataset (Azizi et al. (2023) :
He ct al. (022]; Sanyildız al. (2023)). ) They can also be used to realign or mitigate bias in loundation models (Abdel Magid al. (2024); Howard al. (2024)) or evaluate vision-language
models (Fraser & Kiritchenko (2024); Smith et al. (2023)). ). In addition to training foundation models, synthetic images and their corresponding prompts are also used in reinforcement learning with human feedback (RLHF). Many datasets of real user prompts and preferences have been collected. Examples include RichHF-18K (Liang el al. (2024)), ImageReward (Xu et al. (2024)), ) and Pick-a-Pic (Kirstain et al.] (2023)). In this work, we demonstrate how to use our framework to audit synthotic datasets as well prompt datasets for RLHF alignment of T2I models. For auditing prompt datasets, we focus on StablelmageNett (Kinakh (2022) and Pick-a-Pic. The latter is used to train PickScore which is then used an evaluation metric to better align T2T models with human preferences.
A.6 Ablation Study on the Choice of Object Detector
The object detector an important design choice in our framework. We intentionally design our method so that users have the flexibility to swap detectors as their application demands. The only strong constraint on the detector we impose that should be able to localize concepts (e.g. produce bounding boxes). A second, looser constraint is that the model should generally be open-vocabulary to accommodate broad concept detection. Next, compare the scnsitivity of our framework to different object detectors.
We reproduce the results of experiment 1 in the main paper (section 4. which contains roughly 3k images, using and comparing the following 5 different detectors: Florence-2 (Xiao et al. 2023
GroundingDino (Liu et al. (2023) ), Kosmos-2 (Peng et al. (2023)) ), OwIV2 (Minderer el al. 2023 and DETR (Carion al. (2020)). ). It is important to note that GroundingDino and OwlV2 are text-
conditioned and thus require pre-computing phrases captions. For this, use BLIP
Li ct al.
18

---

Published in Transactions Machine Learning Research (05/2025)
Concept Frequency, Plconcept), Across Object Detectors
backpack
0.00%
0.00%
1.0
ball
0.00%
bench
0.59%
0.00%
..00%
0.00%
DU 15%
73:48
blue 0.00%
0.26%
0.11%
41 44%%
44.44%%
boy 0.00%
11.19%
9.04%
6.07%
12.04%
car 4.48%
0.00%
0.00%
1.26%
0.22%
cell 2.44%
0.00%
D 00%
0.00%
0.00%
city 0.00%
0.11%
0.63%
0.81%
0.33%
clock 3.07%
0.00%
0.00%
0.00%
0.00%
couple 0.00%
0.63%
D 26%
1. 78%
0.48%
dirt- 0.00%
6.85%
6.00%
1.9%%
14.04%
0.8
down 0.00%
0.00%
44%
0 0.0%%
0.00%
elderly 0.00%
0.00%
0.00%
8.00%
59.7280
face 0.00%
00%
0.07%
1 30%
90.35%
fall 0.00%
0.0%%
0.81%
0.07%
0.11%
field 0.00%
0.22%
2.19%
1.78%
4.59%
fo fettwar-
0.00%
0.00%
00%
0.00%
frisbee- 2.00%
0.00%
0. 0.00%
0.00%
0.04%
0.00%
5.56%
glove
gray 0.00%
9.3%%
green
0.22%
human 0.00%
0000%
90%
0.00%
90 33.
jacket 0.00%
04%
40%
7
7.89%
0.6
jegging 0.00%
044%
0.66%
0,22%
legging 0.00%
0.0%
0.00%
10.78%
15.19%
man 0.00%
26.26%
men 0.00%
0. 70%
0.70%
0.93%
0.5.2%
Concept
meter- 0.33%
0.00%
0 0.00%
0.00%
0.00%
navy 0.00%
0.00%
0.00%
0.19%
23.67%
orange 0.00%
0.00%
0.00%
7.44%%
6.67%
pant. 0.00%
0.00%
0.00%
11.41%
12.19%
park 0.00%
0.07%
23.11%
parking 0.33%
0.22%
0.04%
0.00%
0.00%
path 0.00%
555165
15.22%
people 0.00%
0.59%
0.00%
2.00%
0.07%
person
100
0.19%
0.07%
0.52%
21.19%
0.4
0.00%
0.00%
0.00%
0.00%
racket
0.44%
0.00%
0.00%
0.00%
0.00%
0.00%
0.00%
0.00%
0.00%
running
0.00%
0.00%
22%
0.74%
9,78%
0.00%
100
short
0.00%
00%
0 9,00%
30W
64.41%
sidewalk
0.00%
1. 52%
2. 448%
2.67%
6.33%
sign
0.96%
0 000
0.00%
0.04%
0.00%
skateboard
0.26%
0 00%
D. 00%
0.07%
0.00%
sport 2.04%
0.00%
0.00%
0.11%
0.04%
stop 0.96%
00%
0. 0.00%
0.00%
0.00%
street 0.00%
2.4%
5. 1.11
2.93%
2.96%
sun 0.00%
0.00%
0.67%
0.07%
0.00%
t-shir 0.00%
0.00%
0.00%
0.37%
18.85%
tennis 0.44%
0.26%
04%
0.00%
0.22%
tree 0.00%
0.26%
0. 266
34.96%
1.78%
truck 0.19%
0.00%
0.00%
0.04%
0.00%
two 0.00%
1.89%
0.11%
2.15%
0.00%
wearing 0.00%
0.00%
0.00%
0.04%
20.85%
white 0.00%
0.00%
0.00%
24.44%
19
0.00%
0.00%
15.93%
young
0.00%
0.15%
11.96%
28.22%
DETR
GroundingDINO
Owlv2
Kosmos-2
Florence-2
Object Detector
Figure 8: Concept frequency by object detector for experiment 1.
2022) Second, we include DETR although it is closed vocabulay detection model precisely to demonstrate its performance in comparison to open-vocabulary models.
Figure 8 shows a heatmap visualization illustrating differences in detected concepts across detectors.
Figure shows comparative analysis of the number of parameters versus unique concept count, providing insights into efficiency versus effectiveness tradc-offs. Our rosults indicate the following
19

---

Published in Transactions Machine Learning Research (05/2025)
Number of Unique Concepts vs. Model Parameters
Florence
Kosmas
Total Uniquie Cancepts
Cwly2
56-
GrourdingDINO
DETR
Number Parameters
Figure 9: Number model parameters (log scale) and the total unique concepts detected for the spocific data in experiment 1 by cach object detector.
important findings: first, Florence-2 provides the best performance (most unique detected concepts) and smallest model size (i.e. faster and more diverse inference). Second, closed vocabulary models like DETR-which was trained on the class labels of the COCO dataset- are not useful for broad concept detection. Third, although both GroundingDino and OwlV2 rely the same BLIP caption, they do exhibit clear differences in their grounding performance (as indicated by Figures g and g
Our selection of Florence-2 was informed by its strong balance between size, speed, and detection capabilities. Florence 2 offers the advantage of being smaller and faster than Kosmos-2. Addi- tionally, Florence-2 qualitatively outperforms other notable models such as CLIP, SAM, Flamingo, and even Kosmos-2, as documented in the Florence 2 paper. Models like CLIP and Flamingo are unsuitable for our framework due their inability to localize concepts, and SAM, while effective at. localization, lacks semantic labeling.
A.7 Experimental Setup
The experimental setup for each case study is detailed below.
A.8 Additional Results and Details: Toy Example
Table 3 details the experimental setup for this case study. Figure I0 shows additional visual examples of detected concepts. We note how the concept jacket can clearly manifest in different styles and colors. This visualization supports our argument for this component of our framework. Figure II shows the concept stability for each of the actions. We can clearly see which concepts generally persist for single action (holding age constant) and across actions. Moreover, we can also determine which output concepts are triggered by more of the input concepts.
20

---

Published in Transactions Machine Learning Research (05/2025)
Hyperparameter
Value
Object Detector
Florence 2
Object Detector Mode
Aaplion-groundiin
BiteDaanceSSXLLLLgghhhae 4 step model
Text-to-Image Model
T2I Model Hyperparameters
inference steps= 4; guidance scale=
Number of Images
300
a uniform distribution over the set {*A A photo [age] person [action]"] where [age] takes value in {young, middle-aged, old}, and [action] takes value in (jogging, sprinting, running).
Prompt Distribution
Table 3: Case study toy example: hyper- Syper-paramcters and their corrosponding values
shorts ponytail jacket
Figure 10: Additional examples of concepts localized by our framework.
A.9 Additional Results and Details: StableBias and TBYB
Tables and 5 detail the experimental setting for StableBias and TBYB, respectively. For the StableBias case study, the list of adjectives and professions found in Table 4 of the original work's supplementary material Luccioni et al. (2024)). The professions for TBYB are listed in Table 1 of the original work's supplementary material (Vice et al. (2023)). We note that the list of professions for the two experiments (TBYB and StableBias) are different. Second, the TBYB reports their results on a much larger set of prompts that do not include professions. We omit these as that is not the focus this case study. Moreover, we observed that due to the prompl template, many images did not render with detectable person. This is likely the reason for the discrepancy in results between our detections and theirs.
21

---

Published in Transactions Machine Learning Research (05/2025)
44
Concept stability across actions. This is an effective way of capturing how concept persists or is triggered by a specific imput concept.
Figure 11:
Hyperparameter
Value
Object Detector
BLIP VQA
Object Detector Mode
direct VQA: "What is this person's 's gender?"
Texgelo-Image Model
Stable Diffusion v2.1
T21 Model Hyperparameters
inference steps= 50; guidance scale= 7.5
Number Images
30
uniform distribution over the set
Prompt Distribution
("Portrait photo of a [adjective] [profession]
Image Size
768 768
Table 4: Case study Stable Bias: hyper-pant parameters and their corresponding values
Hyperparameter
Value
Object Detector
BLIP VQA
Object Detector Mode
direct VQA: "What is this person's gender?"
Text-lo-Image Model
Stable Diffusion v2.1
T21 Model Hyperparameters
inference steps= 50; guidance scale= 7.5
Number of Images
30
uniform distribution the set {"A A photo of [deac] [profession]] "), where [desc] takes value in {who is a good, who is a bad, that looks like a} 768 x 768
Prompt Distribution
Image Size
Table 5: Case study TBYB: hyperparameters and their corresponding values
A.10 Additional Results and Details: Disability Representation
Table f details the experimental setup for this case study. Figure 12 shows the concept stability over all disabilities and Figure 13 shows the top concopts by disability.
A.11
Cross-Modes Comparison.
A.11.1
Disability
22

---

Published in Transactions Machine Learning Research (05/2025)
Hyperparameter
Value
Object Detector
Florence
Object Detector Mode
caption Soppion-tgrounin grounding
Text-to-Inee Image Model
Stable Diffusion v2.1
T21 Model Hyperparameters
inference sleps= 50; guidance scale= 7.5
Number of Images Per Prompt
100
{"A A person with [value] where [desc]
Prompt Distribution
{a disability, bipolar disorder, cerebral palsy, limb difference, hearing loss, a chronic illness)
Image Size
768 768
Table Case study disability representation: hyper-parameters and their corresponding values
Smallest and Largest 10 Concepts by Coefficient of Variation: all disabilities
2.0
Coefficient of Variation
1.5
1.0
0.5
0.0
shirt
man
hair
face
red
hand
person
t-shirt
wall
woman
chin
white
right
expression s
eye
blue
f
glass
sidewar aak
lpp
wheel
boy
en
che
short
text
child
nose
shoe
teeth
foot
african- african-america am-ricm
girl
ear
branch
eatring
stick
Label
Figure 12: Concept stability for the disability representation case study.
23

---

Published in Transactions Machine Learning Research (05/2025)
Top 20 Concepts chronic illness
Top 20 Concepts for hearing lass
Top Concepts limb difference
0.8
0.8
0.5
Probabiity
0.6
0.5
0.3
0.2
0.2
0.2
0.1
hand
aye
hair
woman
head
mouth
L-shing
tort
chin
am
expressiion
skin
ear
person
hair
shirt
eye
man
glazs
woman
right
face
camiig
small
llo
expressioo
focus
lent
t-hitt
beard
black
person
shont
hand
foot
t-hhit
stick
branch
bandage
pant
Concepts disabilty
Top
palsy
0.6
1.0
0.5
0.8
0.4
0.3
0.3
0.2
0.2
0.1
0.0
hair
nace
tybro hand
exoression
Black
beand
0.0
whoetchale
woman
person
man
whod
k-shirg
hand
soy
pairt
stirt
boulleing
Tace
sm-lee
well
streec
arm
0.0
wheetch behint
hair
you
Concept
Concept
Concept
Figure 13: Top concepts for all disabilities.
Top-20 Concepts by Model for prompt: person with disability, photo
Model
frequency
SD21
0.6
SD3MEDIUM
0.2
lighting
woman
hand
head
fairr
eye
person
face
(-shirt
sweater
glass
man
chin
mom
floor
jean
leg
light
neck
pant
wheelchall
wheel
sidewalk
sky
building
smile
wall
street
amm
window
doar
elderly
flooring
desk
african-americc
Concept
Figure 14: Top concepts detected by our framework for the prompt: "A person with a disability, photo."
24

---

Published in Transactions Machine Learning Research (05/2025)
LUMINA
SD21
503MEDIUM
Figure 15: Random sample of generated images by each model for the prompt: "A person with a disability, photo." " Please zoom in.
25

---

Published in Transactions Machine Learning Research (05/2025)
A.11.2 Limb Difference
Top-20 Concepts by Model for prompt: A person with limb difference, photo
0.8
Model
LUMMNNA
frequency
SD3MEDIUM
hand
pecson
lighting
slin
hair
face
head
tone
mun
finger
Eshhrt
eye
leg
woman
arm
pant
tattoo
boy
portrait
body
stort
foot
stick
left
small
sloe
young
branch
floor
right
bandaage
sly
brace
legging
tank
top
Figure Top concepts detected by our framework for the prompt: "A person with a limb differ- ence, photo." "
LUMINA
SD21
Figure 17: Random sample of generated images by each model for the prompt: "A person with limb difference, photo." Please zoom in.
26

---

Published in Transactions Machine Learning Research (05/2025)
A.11.3 Hearing Loss
Top-20 Concepts by Model prompt: A person with hearing loss, photo
Model
LUMINA
SD21
Rreueno
0.5
SDJMLDIUM
0.2
pesson
woman
lighting
hair
face
man
head
t-shirr
lipht
hand
expresiinn
boy
red
Hp
bndy
mouth
chin
eyebrow
girl
ear
glass
earring
right
small
Facur
left
beard
black
earphone
aid
hearing
smile
blue
slight
white
Concept
Figure 18: Top concepts detected by our framework for the prompt: "A person with hearing loss, photo."
503MEDIUM
Figure 19: Random sample of generated images by each model for the prompt: "A person with hearing loss, photo." " Please zoom in.
27

---

Published in Transactions Machine Learning Research (05/2025)
A.11.4 Toy Example.
Top-20 Concepts by Model prompt: photo young person jogging.
1.0
Model
LUMINA
SD21
Frequency
SDJMLDIUM
0.6
0.2
halr
young
short
woman
sky
am
leg
Face
tank
top
girl
ponytall
field
sun
legging
bra
road
sport
Eshirt
grass
lence
man
arai
boy
shoe
body
Concept
Figure 20: Top concepts detected by our framework for the prompt: "A photo of young person jogging."
503MEDIUM
Random sample of generated images by each model for the prompt: "A photo of a young person jogging." " Please zoom in.
Figure 21:
A.12 Simple Experiment with Closed Source Text-to-lmage Model
In this section, we show how our method can be applied to closed source models, which are pro- sumably safety fine-tuned. Figure 23 shows 10 images generated using the interface of ChatGPT model 4o. The prompt for generation is "A person with disability, photo", ", following that in case study 3 (see section 4.3 of the main paper). Figure 22 shows the results from our method, Concept2Concepl, when applied to these images.
28

---

Published in Transactions Machine Learning Research (05/2025)
Distribution Concepts for: A with disability, phota Images generated using ChalGPT model.
bench
man
5uh
park
smile
Concent
pathway
people
face
fablet
SW SWester
grass
hair
blue
jacket
0.8
1.0
Probability
Figure 22: Top concepts detected by our framework on images generated using closed source T21 model.
Figure 23: Images generated using ChatGPT for the disability prompt.
29

---

Published in Transactions Machine Learning Research (05/2025)
A.13 Additional Results and Details Case Study: Pick-a-Pic
Table 7 shows the experimental setting for this case study. We drew 10 random samples of size 1k each from the train split of the Pick-a-Pic dataset. Each image be generated by different text-to-image model. We refer the reader to the dataset for the details of each image's generation hyper-paraen parameters.............. Figure 25 shows the top detected concepts, along with confidence intervals. Figure 24 shows prompts within the Pick-a-Pic dataset that roquest child mudity, violence, slurs, and sexually explicit material. We have hidden most of them so as to not overwhelm the reader. The dataset can be accessod https: htg.g//nngno..o///taat///////lla picckppi
Hyperparameter
Value
Object Detector
Florence
Object Detector Mode
dense region caption and detection variable
Text-lo-lmage Model
T2T Model Hyperparameters
variable
Number of Images Per Prompt
variable (typically 1)
p(() drawn from train split of Pick-a-Pic
Prompt Distribution
Table 7: Case study Pick-a-Pic: hyperparameters and their corresponding values
This section contains discussions of harmful content, including CSAM and NSFW ma- terial, which may be dislurbing to some readers.
Warning:
A.14 Additional Results and Details Case Study: StablelmageNet
Table details the experimental setting of this dalaset. haaphh/ssttal--mgneeell.ee
The dataset can be accessed at We show all 100 images for
https:/ www.kaggle.com waw.aaa/ll.ooddaatttttttttaaaaae
the misaligned classes in StablelmageNet Figures 26
Hyperparameter
Value
Object Detector
Florence 2
Object Detector Mode
caption + grounding
Text-to-Imase Model
Stable Diffusion v1.4
inference steps= 50; guidance scale=
T21 Model Hyperparameters
7.5
Number of Images Per Prompt
100
Prompt Distribution
a photo of class, realistic, high quality
Image Size
512 x 512
Table 8: Case study StablelmageNet: hyperparameters and their corresponding values
30

---

Published in Transactions on Machine Learning Research (05/2025)
Trigger warning: This next table contains discussions of harmful content, including CSAM and NSFW material, which may be disturbing to some readers.
Prompt
14 year old girl wearing almost nothing
a teen boy, full body. Bare
teen little girl shameless
CHILD NUDITY
CHILD NUDITY
CHILD NUDITY
CHILD NUDITY
CHILD NUDITY
CHILD NUDITY
VIOLENCE
CHILD NUDITY
CHILD NUDITY
CHILD NUDITY
CHILD NUDITY
CHILD NUDITY
VIOLENCE
CHILD NUDITY
VIOLENCE
CHILD NUDITY
CHILD NUDITY
CHILD NUDITY.VIOLENCE
CHILD NUDITY
CHILD NUDITY
CHILD NUDITY
CHILD NUDITY
CHILD NUDITY,VIOLENT
CHILD NUDITY
CHILD NUDITY
VIOLENCE
DEROGATORYBBLUR.HIID NUDITY
CHILD NUDITY
CHIL NUDITY
NUDITY
DEROGATORY/SLUR.CHID NUDITY
CHILD NUDITY
CHILD NUDITY
CHILD NUDITY
CHILD NUDITY
CHILD NUDITY
Figure 24: Harmful prompts in the Pick-a-Pic Pic dataset.
31

---

Published in Transactions Machine Learning Research (05/2025)
Top Detacted Concepts Across 10 Random Samples
Mean Detrctions
102
10a
Concept
The top detected concepts over 10 random samples of size 1k drawn from the Pick-a-Pae Pic dataset.
Figure 25:
A.15
Interactive Tool
Figures
35 and
show screenshots of the Concept2Concept interactive tool.
32

---

Published in Transactions Machine Learning Research (05/2025)
All 100 images for flass=755_st
skkn
LE
KSE AMOEE
KEAH
KSK
SKAI STO
MK
Baics
AK NELOONAISSICS
SKRALI
SKO Ri
AK
Figure 26
33

---

Published in Transactions Machine Learning Research (05/2025)
All 100 images for class=796_sk __ski mask.
SACK
SKKSKK I.I.Z
SACK
AR 7
SAKSS
PRKE
SKESDOK
SEL
PPS
SAPAI.
Figure 27
34

---

Published in Transactions Machine Learning Research (05/2025)
All 100 images for 870_tunsttttttee
TURTISTEE
23
ARESLE
TUTINE
Figure 28
35

---

Published in Transactions Machine Learning Research (05/2025)
All 100 images for class=017_jay
Figure 29
36

---

Published in Transactions Machine Learning Research (05/2025)
All 100 images for class=097_draaie
ORAAFCC
MGO
Figure 30
37

---

Published in Transactions Machine Learning Research (05/2025)
All 100 images for class=168 redbone. reasooe
REDEOIN
REDDEONE
REE SON
380
REID ONN.
REDDDONON
RECON
ON
REDOOONN
RED BEOOOS
REDOD RENNI
REDD CONN
REDOOIN
Figure 31
38

---

Published in Transactions Machine Learning Research (05/2025)
All 100 images for class=513_comm _orneet, horn, trumpet, trump.
Caft
Jktchi
Giil
Figure 32
39

---

Published in Transactions Machine Learning Research (05/2025)
All 100 images for class=345_od ox.
Figure 33
40

---

Published in Transactions Machine Learning Research (05/2025)
CONDEPT ANALY11S
concept: person
Concept Metric
PEROISTENCE
MOXLEWESS
Thrssood
3.4
1.2
Concept
Concept
Fitwiig for distanos greater than
Clennce
Seeft
book
tookshal
CO0020
000000003
0.00000
book
books
62
6472149
0.00021022
0.0001
Dugeston
pharmacle
76
6
0
0.0011
book
76
6090005
0.0044000
0.00277
del
0.07468
3.3004004
0.00511
page
Figure 34: Users can interactively inspect concept metrics, such as the persistence and fickleness scores, their stability, and co-occurrence with other concopts. The interactive tool also includes features for users to sort by certain metrics and filter by keywords.
41

---

Published in Transactions Machine Learning Research (05/2025)
CONDEPT INSRECTION
SEARCH
Concept Prompts
Concapt Prompte
proept
Aperson that lakko lie athiete
Apennn that losk suiad
Apesson that lokka a refuges
Aperson
Apenonn that dentel
Aerson that lodka liee sadden
Apenia
Apenon that looks
Aperson who bad lbsarie
Apinian
Apenan that loto
Apenson who good musriee
Apenoon
Apeaon who oduges
Apenio
Foom 1-10
Figure 35: The interactive tool also includes concept inspection tab that allows users to search for concepts of interest. For each concept, the tool uses visual grounding models to localize how the concept is represented in different images. Localized concepts are displayed as thumbnails. The prompts that were used to generate the concept are also displayed in a table below. Users can search for and compare two concepts in this panel.
42


Reviews:
[{"review_id": "eYxP5TOsmM", "reviewer": "Reviewer_BdUc", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**:\n- [S1] The paper presents a novel, interpretable auditing framework that rigorously defines and analyzes the conditional distribution of generated images in terms of high-level concepts.\n- [S2] The authors situate their work well within the context of existing literature, showing how Concept2Concept generalizes and extends prior bias auditing methods.\n- [S3] Multiple detailed experiments (e.g., reproducing gender bias findings, auditing disability representation, and detecting harmful content in the Pick-a-Pic dataset) effectively demonstrate the framework’s versatility and practical utility.\n- [S4] The provision of an interactive, open-source visualization tool lowers the technical barrier for auditing and encourages broader adoption and collaboration.\n\n**Weaknesses**:\n- [W1] The framework’s performance is closely tied to the reliability of off-the-shelf object detectors, which themselves may introduce biases or limitations.\n- [W2] As the number of detected concepts grows, especially with high-resolution images or large prompt spaces, the co-occurrence analysis may become computationally intensive.\n- [W3] Some experimental setups rely on carefully crafted prompt distributions that might not generalize across different use cases or more naturalistic settings.\n- [W4] Some sections, particularly those detailing hyperparameter choices and experimental setups, are dense and could benefit from clearer exposition to improve reproducibility. Certain methodological details, particularly regarding the quantitative metrics and their interpretations, could benefit from clearer explanations and additional visual aids.\n- [W5] While the paper effectively identifies problematic biases and associations, it offers only a preliminary discussion on how these findings might translate into actionable mitigation strategies.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces **Concept2Concept**, an auditing framework designed to provide interpretable characterizations of the conditional distribution of images generated by text-to-image models. The core idea is to extract high-level, human-understandable concepts from generated images using **open-vocabulary object detectors** and then analyze these via quantitative metrics such as **concept frequency, stability, and co-occurrence**. By formalizing metrics like concept frequency, stability, and co-occurrence, the authors provide a systematic method to reveal unexpected biases, misalignments, and even harmful content (e.g., CSAM, hyper-sexualization) in both model outputs and training/prompt datasets.  Furthermore, the paper contributes an open-source, interactive visualization tool is provided to facilitate human-in-the-loop auditing, making the framework accessible to non-technical end users.\n\nThe authors demonstrate the utility of their framework across several case studies:\n- Using a pedagogically designed prompt set to expose associations such as gender biases (e.g., “jogging” paired predominantly with one gender) and exploring nuanced differences in representations (e.g., attire differences between jogging and running).\n- By replicating experiments from StableBias and TBYB, the framework shows its ability to provide quantitative insights into gender representation in generated images.\n- The framework surfaces stereotypical associations (e.g., over-reliance on wheelchair imagery for disability prompts) and demonstrates how prompt revision can adjust conceptual outputs.\n- Applying the method to prompt datasets (such as Pick-a-Pic) and synthetic datasets (StableImageNet) reveals unexpected and harmful content—including the disturbing presence of CSAM and hyper-sexualization—which has critical safety implications.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "**Major revisions**:\n- [R1] Expand the discussion on the potential biases and limitations introduced by the object detectors (Florence 2, BLIP VQA) and consider suggesting complementary methods or calibration techniques.\n- [R2] Provide additional experiments or ablation studies to analyze the sensitivity of the framework with respect to key hyperparameters and the choice of object detector.\n- [R3] Incorporate experiments using more diverse, real-world prompt distributions to demonstrate the framework’s robustness and generalizability.\n- [R4] Elaborate on how the insights gained from auditing (e.g., identification of CSAM or misaligned synthetic data) can inform concrete mitigation strategies or adjustments in model training pipelines. This point is critical for strengthening the work’s practical impact.\n\n**Minor revisions**:\n- [MR1] Provide a more intuitive explanation of the key metrics (frequency, stability, co-occurrence) along with illustrative examples that guide the reader through their interpretation.\n- [MR2] Consider adding a user study or qualitative feedback from non-technical end users who have interacted with the visualization tool to further validate its usability.", "broader_impact_concerns": "The paper addresses a crucial aspect of AI ethics by exposing biases and harmful content in T2I models, which has significant societal implications. However, the detection of sensitive material such as CSAM and hyper-sexualized content—especially involving minors—necessitates a cautious approach. The authors acknowledge these ethical concerns, yet further discussion is needed on:\n- [Q1]  How can the framework and its interactive tool be safeguarded against potential misuse or misinterpretation of its findings?\n- [Q2]  What protocols should be in place to act upon the detection of harmful content, and how can these audits be integrated into existing safety pipelines without causing undue alarm?", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "CbAHmWj4s2", "reviewer": "Reviewer_rxwX", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "[Strengths]\n* **Flexible, Concept-Centered Framework**: By focusing on interpretable concepts and their distributions, the proposed framework offers a versatile tool to investigate T2I models. Each component can be picked out of the box and refined for different objectives.\n* **Scalable Approach**: Through automated analysis of large collections of generated images, the framework has strong potential to be used for industrial-scale and production settings. As the authors mentioned, given the widespread deployment of T2I models, the paper is well-timed.\n* **Empirical Findings and Insights Through Widely Used Datasets**: The case studies on the Pick-a-Pic and synthetic ImageNet datasets highlight practical usability. The paper identified problematic/undesired results that might easily remain hidden under causal inspection.\n\n[Weaknesses]\n* **Reliance on Object Detectors**: The proposed framework critically depends on external object detectors to identify and localize concepts. These “learnable” detectors themselves may exhibit biases learned during training. Also, these detectors, either with “closed” or “open” vocabulary support, would only localize the concepts/terms they learned.\n* **Manually Designed Prompt Distributions**: The framework relies on user-defined or manually curated prompts, which could unintentionally input incomplete or biased audits. This also belongs to the problem that the paper is trying to solve – A more robust approach might offer a more representative view of the model’s behavior in practical settings.\n* **Limited Breadth of Concept**: While the paper defines “concept” to include object names, nouns, labels, etc., these are somewhat primitive and cannot represent more abstract and semantically meaningful information. For example, there would be difficulties in capturing stylistic attributes, cultural references, overall tone, or mood. I acknowledge that the authors defined the “concept” as visually identifiable features, and they are very important; however, it would be beneficial to discuss this issue.\n* **Disconnection of Proposed Metrics**: Although the authors introduce “Concept Frequency”, “Concept Stability”, and “Concept Co-Occurrence”, these terms are barely used in the later context and analyses. In the case studies, the presented figures and discussions employ more generic descriptors (“probability”, “proportion of detections”, “mean co-occurrence count”) instead of explicitly referring to the introduced metrics.\n* **Text-Image Concept and Clarification**: The term “Concept2Concept (C2C)” implies an emphasis on relationships among concepts within generated images. However, much of the analysis deals with prompt-to-concepts. In Section 4.1, the paper connects the “prompt” (jogging, running, sprinting) to “concept” (woman, man, glasses, etc.). Here, “jogging” is included in the prompt and not detected by the visual detector, thus confusing to be interpreted as concepts defined in this paper. In this perspective, the sentence “concepts are resolved as we desire” can be somewhat misleading. This creates a potential gap between ‘prompt concepts’ and ‘image concepts.’ Clarifying this would help mitigate confusion. I generally understand the intention of the framework, but it should be clear; for example, one can ask whether those concepts in the prompts (which are not clearly described in the manuscript) and concepts in the generated images are assumed to be in the same set.\n* **Necessity of the Full Framework for Dataset Auditing**: In section 5.1, C2C is utilized to uncover problematic or harmful content in the Pick-a-Pic dataset. However, it seems that merely applying an object detector to the images themselves could arguably detect the same content. A more detailed explanation/analysis of why integrating the proposed C2C is necessary for the purpose would strengthen the rationale for adopting this framework.\n* **Unclear Value of Concept Distributions**: While the paper emphasizes the importance of ‘distribution-level’ analyses, it remains somewhat unclear how these distributions directly enhance the interpretability or add values to analyses. Please consider establishing a clearer link between the concept “distributions” to various tasks discussed in the paper.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces a flexible, concept-driven framework called Concept2Concept. The framework aims to audit text-to-image (T2I) models at scale. By sampling a distribution of prompts and detecting concepts in the generated images, the framework provides a tool (or, measurement) that humans can interpret the behavior of black-box T2I models.\nKey contributions include:\n* A unified T2I investigation framework including metrics.\n* Empirical case studies and findings on the widely used datasets.\n* Open-source interactive tool that can be easily used for a broader spectrum of users.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "* Please see the weaknesses above. In particular, please address the prompt/concept overlap more explicitly, and expand the discussion of the ‘concept’ scope. Please connect the proposed measurements and support the importance of 'distributions' with empirical analyses.\n* For equation (3), I think P(C) should be represented as a double integral of (t) and (x) as below. Please check and modify if necessary. $p( C ) = \\int_t \\int_x p(C|x)p_G(x|t)p(t)dxdt$\n* Inconsistent reference styles – especially for page 11.", "broader_impact_concerns": "The paper addressed ethical concerns and limitations in the Appendix. I think the paper tries to carefully handle the potential risks of the paper and its applications.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "9U9WmkgdSH", "reviewer": "Reviewer_fziz", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n+ The paper provides a general framework to analyze biases in any forms as users desire.\n+ The paper discovers and reproduces some biases existing in current T2I models regarding gender, age, etc.\n+ The authors open-source the framework.\n\nWeaknesses\n\n- Concept associations are analyzed without considering the broader context of the image or prompt. For example, detecting a \"thong\" or \"naked\" might be appropriate in some artistic or cultural contexts but is flagged as problematic.\n- The analysis relies heavily on the object detection models. Althoug the authors have mentioned in the Limitations section, they do not specify clearly how to mitigate the influence. The authors could report human evaluation on (a small portion of) images analyzed in this study. Additionally, the choice of detectors needs to be well-justified, and, adding comparison with alternative detectors will be good.\n- The detectors may not work well on non-photorealistic images, such as cartoon, abstract, surreal, or artistic images.\n- Some experiments lack generalizability. For example, section 4.1 studies only three actions on one model (SD2.1). Readers are not sure whether the conclusion can be generalized to a broader range of concepts and models.\n- The framework could be misused to identify and exploit biases in T2I models for malicious purposes, such as generating harmful content more effectively.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces Concept2Concept, a framework that audits T2I models by extracting and analyzing high-level concepts from generated images, enabling interpretable evaluations of how prompts influence outputs. Using metrics like concept frequency, stability, and co-occurrence, the framework uncovers biases and unexpected associations, such as gender stereotypes, misrepresentations of disabilities, and inappropriate content in training datasets like Pick-a-Pic. It also provides an open-source interactive tool that allows users to explore concept associations, supporting more transparent and ethical AI development.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Use multiple object detectors and compare their performance to improve reliability.\n2. Setup comprehensive and systematic studies instead of case studies.", "broader_impact_concerns": "- Potential for misuse: The framework could be misused to identify and exploit biases in T2I models, enabling malicious actors to generate harmful content more effectively.\n- Limited consideration of cultural context: The paperfocuses on Western cultural norms and datasets, limiting its applicability to other regions and demographics. Concepts like \"nudity\" or \"disability\" may be interpreted differently across cultures, yet the framework applies a one-size-fits-all approach.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors answered questions and addressed the concerns raised by the reviewers through updates in the manuscript. 
Though the presented case-studies are seen to be more of an illustrative nature rather than a comprehensive study conclusively validating the generalizability of the framework, and despite some remaining uncertainties related to the dependence on a concept detector (with potentially its own biases), a prompt distribution (may be difficult to construct) and discussion of cultural nuances in sensitivity to concepts, the proposed framework is introduced in an accessible manner and has the potential to trigger the attention of the relevant part of the community.
Decision: Accept (poster)