Kati8/ICLR_2023_2025 huggingface dataset:
authors: Sepehr Dehdashtian
authors: Gautam Sreekumar
authors: Vishnu Boddeti
reviews: 
Comments: This paper aims to measure and understand stereotypes in Text-to-Image models. 

* Measurement: it introduces metrics to quantify the presence of stereotypes in generated images by comparing them to real-world distributions, and also assess diversity along stereotypical attributes. 

* Track Origins:  The author attempts to identify hidden associations that T2I model makes with certain concepts and trace how stereotypical attributes emerge over the generation process
Strengths: 1. The paper tackles the significant problem of stereotypes in T2I models, which has important implications for ethical AI use and social impact.
2. The paper is clearly written and organized. The visual presentations are good. 
3. It proposes StOP and SPI that try to understand the origin of the stereotypes, which may offer further insights.
Weaknesses: 1. The authors emphasize that their method is innovative in focusing specifically on stereotypes rather than just bias in the abstract. However, this distinction is not fully explained in the main content, and the methodology still seems quite similar to bias detection to me.

2. The way they identify stereotypes is by prompting LLMs to generate stereotypical attributes for a given concept, but this method is already common in other papers. For example, [1] prompts LLMs to generate stereotypical / anti-stereotypical sentences for certain concepts and then identifies the stereotypical / anti-stereotypical attributes from those sentences.

3. Although the paper includes many formulas, I found that the core idea is actually quite simple, and the formulas only make it harder to follow. If I understand correctly, the main process is: identify a concept, use LLMs to generate stereotypical attributes, generate images, measure the appearance of these attributes, and compare to real-world distributions & similarities across groups. This could be communicated in just a few sentences, which would make the approach clearer.

4. I am concerned with the idea of comparing to real-world distributions. For example, men are dominant in many professions even today. In society, an educated person can be aware of stereotypes to some extent, but telling the model that it’s okay to learn patterns such as “men are more likely to be doctors” because it reflects real-world distributions doesn’t seem fully convincing to me.

[1] chatgpt based data augmentation for improved parameter-efficient debiasing of llms
Recommendation: 6
Reviewer confidence: 4
Soundness/correctness: 2
Clarity: 3
reviews: 
Comments: This paper introduces OASIS, a toolbox for auditing visual stereotypes in text-to-image (T2I) models. OASIS offers novel methods for measuring stereotypes and analyzing their origins within T2I models. OASIS stereotype scores reveal that newer T2I models produce significant stereotypical biases, potentially limiting their applicability.
Strengths: - The paper is well-written, clear, and easy to follow.
- The paper addresses a complex and socially significant problem by evaluating biases and stereotypes in T2I models. Additionally, it provides new methods to trace the origins of these stereotypes in T2I models, offering valuable insights to help mitigate these biases.
- The findings of this work highlight the need for greater inclusion and representation of underrepresented communities, particularly those from regions with limited internet presence, in the development of T2I models.
Weaknesses: - This work focuses exclusively on generated images based on nationality, which raises several concerns. For instance, in racially diverse countries like the United States, how can an image indicate that someone is a U.S. citizen without relying on stereotypes and social biases? Additionally, in regions where people from neighboring countries may share similar facial features or cultures (such as North and South Korea), how useful can OASIS be in distinguishing these cases?
- How can OASIS be useful beyond nationality domains? It is not clear in the paper how OASIS can be extended to evaluate other biases in T2I models, such as gender biases, racism, etc.
- Overall, this is a solid piece of work; however, a deeper discussion of its limitations and the specific types of stereotypes for which OASIS may or may not be effective would be beneficial.
Recommendation: 8
Reviewer confidence: 3
Soundness/correctness: 3
Clarity: 4
reviews: 
Comments: The proposed work has a novel contribution in an existing research gap. Defining, separating, and quantifying a measure for stereotypes and biases in Image generation models (from text prompt) is a strong case of Ethical Machine Learning. The authors built a foundation of definitions that allowed them to measure estimates of data and image stereotypes. Then they apply that on well-known Text to Image models to uncover the stereotypes risks of applying those models according to the proposed measures/estimates.
Strengths: Addressing a research gap
Clear mythology
Application to existing models
Weaknesses: It would have been nice if the writeup included a detailed case study of qualitative analysis of cases of stereotypes
Recommendation: 10
Reviewer confidence: 4
Soundness/correctness: 3
Clarity: 3
reviews: 
Comments: In this paper, the authors propose multiple new metrics to understand stereotyping in T2I models.  In particular, they propose a metric for stereotyping that compares the prevalence of a given attribute to a societally-anchored prior probability.  They go on to present numerous other analyses for diversity of images and measuring how differnt stereotypical concepts relate to demographics in latent space.  They show that generally newer models exhibit less stereotyping and that stereotypes arise mostly in the earlier stages of optimization.
Strengths: 1. Measuring sterotyping is an important and challenging problem for generative models.

2. The diversity of metrics seem quite interesting to get a more complete picture of how bias and stereotypes arise in these models.

3. The paper is fairly thorough in its testing.
Weaknesses: 1. The biggest weakness in my opinion is that the paper is quite hard to follow for many of the metrics. I believe this is due to making the description of the metrics unnecessarily complex, and I would encourage the authors to try to describe all of the concepts more simply.  In addition to making the paper harder to understand, it also obfuscates the insights and contributions.

2. One of the papers claimed main contributions is measuring stereotyping relative to societal priors on the distribution.  However, as the authors state in the appendix, the prior is often unknown and thus falling back to 50% is a reasonable choice.  That said, this seems to limit the contribution over prior work.

3. The paper does not validate their work with human judgements

4. More minor: the paper only presents the evaluations but do not work on mitigations.

5. The intro could be more clear and sensitive in how it characterizes what are stereotypes, why they matter, and how they define which stereotypes are important.
Recommendation: 6
Reviewer confidence: 3
Soundness/correctness: 4
Clarity: 2