reviews: 
Comments: The authors present OASIS, a set of 4 metrics/methods to analyze the presence of stereotypes in T2I models, how those stereotypes form, and the connections between them. The authors use 3 popular models for this analysis. In the paper, OASIS shows the performance of these models on various prompts (e.g. "Show me a picture of a Mexican person") and how certain stereotypes are strongly associated with the prompt, at rates greater than those stereotypes in the real world. The paper shows weaknesses in models in their attempts to prevent stereotypes such as revealing stereotypes when prompts or combined or in how they produce low-variance along the stereotype's dimension. The paper also shows how certain stereotypes are clustered together and at what stage in the image generation process these models form (which is almost always in the early stages). 

In all, OASIS provides a useful toolkit for understanding stereotypes in T2I models and highlights the importance for further work in this very important area.
Strengths: The paper was very well-written, and it was quite easy to follow. I greatly appreciated how the terminology was constantly linked to concrete examples. The general formatting was also intuitive. 

The paper did a good job of thinking about the broader context of this work, thinking a bit more critically about what phrases like biases actually mean and what we should think about when studying these models. Also incorporating real-world census data was a neat idea. 

I liked the results about how the biases are being generated early in the image-generation process.

The formulation for the stereotype score was quite intuitive and seems easy enough to compute at scale when provided with accurate descriptors from these LLMs.
Weaknesses: I feel that overall, there was a lack of bench-marking for the methods presented in this paper beyond providing a few visual examples (which are still useful!). There were a few parts of the paper where I felt this significantly 

1. Computing the stereotype score: How accurate is the probability computation? The intuition of the metrics' formulation is clear, but there is no validation that the score provides accurate results in practice. As the paper mentions, CLIP is a biased VLM on its own, so its not fair to assume that this score will work perfectly. A smaller dataset demonstrating the accuracy of the score could be quite useful. 

2. Spectral variety along a stereotype. There was a nice description of each method for measuring $\delta A$ in the appendix, but its not clear which one is used in the paper and which one is more accurate in practice. 

3. The biases of models like CLIP are well-documented (e.g. Birhana, Prabhu, & Kahembwe, 2021). If CLIP is being used extensively to identify additional biases, I think it makes sense to add more validation about how said biases may or may not be affecting the performance of OASIS. 

4. I understand the SPI claims about how certain stereotypes are intrinsically associated with a concept. However, the paper only provides a handful of examples of where this is occurring. I would appreciate a larger-scale analysis that includes something like 100 contexts generated in some way (i.e. top 100 most common ethnicities), and the average step at which each stereotype attribute occurs. This would be much more convincing than what could be cherry-picked examples. 

I think that the breadth of the experiments/validation were also a bit limited. I understand that gaining access to real-world data for a ground truth distribution is not always feasible, but consistently focusing on broad nationalities feels like it lacks nuance. 

Another broad critique is that certain figures were unclear. I'll bring up those missing details in the following sections
Recommendation: 6
Reviewer confidence: 4
Soundness/correctness: 3
Clarity: 4