TWIGMA: A dataset of AI-Generated Images with Metadata From Twitter
Authors: Yiqun T. Chen
James Zou
Venue: NeurIPS 2023 Datasets and Benchmarks Poster (2023)
Keywords: Generative AI
Diffusion model
Large-scale Twitter datasets
TL;DR: Our work investigates the themes, variations, and characteristics of AI-generated images on Twitter, revealing distinct features, lower variability compared to human images, and a longitudinal shift towards artistically sophisticated content.

Abstract: Recent progress in generative artificial intelligence (gen-AI) has enabled the generation of photo-realistic and artistically-inspiring photos at a single click, catering to millions of users online. To explore how people use gen-AI models such as DALLE and StableDiffusion, it is critical to understand the themes, contents, and variations present in the AI-generated photos. In this work, we introduce TWIGMA (TWItter Generative-ai images with MetadatA), a comprehensive dataset encompassing over 800,000 gen-AI images collected from Jan 2021 to March 2023 on Twitter, with associated metadata (e.g., tweet text, creation date, number of likes), available at https://zenodo.org/records/8031785. Through a comparative analysis of TWIGMA with natural images and human artwork, we find that gen-AI images possess distinctive characteristics and exhibit, on average, lower variability when compared to their non-gen-AI counterparts. Additionally, we find that the similarity between a gen-AI image and natural images is inversely correlated with the number of likes. Finally, we observe a longitudinal shift in the themes of AI-generated images on Twitter, with users increasingly sharing artistically sophisticated content such as intricate human portraits, whereas their interest in simple subjects such as natural scenes and animals has decreased. Our findings underscore the significance of TWIGMA as a unique data resource for studying AI-generated images.

TWIGMA: A dataset of AI-Generated Images with Metadata From Twitter
Yiqun T. Chen
James Zou Departments of Biomedical Data Science; Electrical Engineering; and Computer Science Stanford University Stanford, CA 94305
Department of Biomedical Data Science Stanford University Stanford, CA 94305 y yiqunc@stanford.com
jamesz@stanf ord. edu
Abstract
Recent progress in generative artificial intelligence (gen-Al) has enabled the gener- ation of photo-realistic and artistically-inspitiie photos a single click, catering to millions users online. To explore how people use gen-Al models such as DALLE and StableDiffusion. it is critical to understand the themes, contents, and variations present in the Al- Al-generated photos. In this work, we introduce TWIGMA (TWItter Generative-ai images with MetadatA), a comprehensive dataset encompassing over 800,000 gen-Al images collected from Jan 2021 to March 2023 on Twitter, with associated metadata (e.g., tweet text, creation date, number of likes), available https://zenod:
0oarrds//8011888 Through a comparative analysis TWIGMA with natural images and human artwork, we find that gen-Al images possess distinctive characteristies and exhibit, on average, lower variability when compared their non-gen-A AI counterparts. Additionally, we find that the similarity between a gen-Al image and natural images is inversely correlated with the number likes. Finally, we observe longitudinal shift in the themes of Al-generated images on Twitter, with users increasingly sharing artistically sophisticated content such as intricate human portraits, whereas their interest in simple subjects such as natural scenes and animals has decreased. Our findings underscore the significance TWIGMA as a unique data resource for studying Al Al-generated images.
Introduction
Recent advancements in text-to-image generation models, such DALLE 3 and StableDiffu- sion 37], have revolutionized the creation of realistic and visually captivating images. The ability to generate artistically inspiring images the click of button has attracted millions of daily active users. However, the surge in popularity has also given rise to intriguing questions about the boundaries of human and model creativity, well the diversity in topics and styles of the generated images. For instance, let's consider the StableDiffusion model: it passes the user-specified text input (known as prompts) into text encoder CLIP 32 obtain encoding that captures the essence of the prompt. Next, the model utilizes a latent diffusion model that takes the text embedding as input and stochastically generates an image that is guided by the text encoding. Therefore, the output image is influenced by both the user's input prompt and the complex diffusion process. Consequently, the stochasticity and black-box nature of the diffusion process led extensive practice and research in the field of prompt engineering, where content creators iteratively refine their text prompts to generate images that align closely with their desired targets [22] 28). Similar other emerging technologies with significant capabilities, generative image models have given rise to plethora of intricate legal and ethical challenges. These include concerns such as the proliferation of Al-generated NSFW
37th Conference on Neural Information Processing Systems (NeurIPS 2023) Track on Datasets and Benchmarks.

---

(Not-Safe-For Work, which broadly includes offensive, violent, and pornographic topics) images, the dissemination of fake news, and controversies surrounding copyright.
Given the ever-increasing popularity and controversy surrounding A Al-generated images, understand- ing their themes, content, and variations is increasingly erucial. However, existing datasets available for research purposes do not sufficiently address these specific inquiries, as many of them were created for specialized investigations (e.g., fake art detection [44 (1I) and image quality evalua- tion [40), resulting in limited content diversity. Some recent exceptions include the Kaggle dataset on Midjourney prompts 3 and DiffusionDB [52], two large-scale datasets compiled from Discord for Midjourney and StableDiffission models, respectively. However, these pioneering datasets are limited in terms of model variations, user distribution, and relatively short data collection periods.
To address these limitations, we present TWIGMA (Twitter Generative-Al Images with MetadatA) large-scale dataset encompassing 800,000 Al-generated images from diverse models. Spanning January 2021 to March 2023, TWIGMA covered an extended timeframe and included valuable metadata, such as inferred image subjects and number of likes. To the best of our knowledge, TWIGMA is the first Al-generated image dataset with a substantial time span and rich metadata, enabling analysis of temporal trends in unsupervised learning techniques and inferred image captions to understand themes of Al-generated images. This complements earlier research, which primarily concentrated on logged prompts' subjects and content [52 [53]. We demonstrate the value and the type of insights TWIGMA enables by using characterize the underlying themes and novel aspects of Al-generated images. In addition to offering opportunities for fine-tuned adaptations to generate images with high popularity on social platforms like Twitter, our dataset and accompanying analyses contribute substantial insights into the ways in which humans engage with generative models, topic of much interest to a broader scientific framework: as the prevalence of generative content continues to grow, our study could serve pilot for future endeavors aimed at comprehending and enhancing the sociological [38] and human-machine interaction 27 dimensions inherent to these data-drives gen-Al models.
image content. Moreover, we leverage
(a)
(b)
(d) TWIGMA innileeted ooe 203100 3021
Curation of
TWIGMA Dataset
containing unique gettrafisn-Al images
lduniiication
Tonesta imsge-raia pugge-ralateedee hamhraaga" trom 202200 X/1 2013/04401 (=02,255,382)
Crnaeedd 9E7A6000 7,70 Bani
Creaad 01819100 1,00
Diaadd 08939009
The L
Twests screamed for wilk, diverabble pholos (nw] (373388)
Tecning overyt
trom
200
seack GiNisieee
Screening
DiffusionDB
Auguet
Rerroved henels with duplicafed imagees (om823, tweets)
new
Creeteee ted stylel
Valooa Uhiced
contloy fial aittury
award werinn
hight araled, 280
@
prompt
gir
good
please
LiC
e
LAION-5B
Included
lest, CLI--VIT-14 embrdding
Findl date: am300 imagen.
images
col.Icc goon
S=image
-imgee
twen
det Mttillen Led
Painting
ArtBench
ast
calid. aoee
Prdeefhaion. atc orltuccent atriorr conie ali right desateery welieeeee one of ahe beah
one
SUr ce
stable
Work
day
ano
Corst
Pumer on
tstatiin
Figure 1: Creation process and content overview of TWIGMA. (a): Curation process of the TWIGMA dataset, resulting in approximately 800,000 images posted from 2021 to March 2023. (b): Steady growth in the count of tweets featuring generative AI images over time. (c): Wordcloud showcasing the prevalent keywords extracted from the tweets; popular words include new, AI, art, and prompts. (d): Schematic depiction of the data and metadata incorporated in our analyses.
The rest of the paper is organized as follows. We review image datasets and evaluation metrics for model novelty and variation in Section 2 The data collection and analysis processes are outlined in Section (3] Section presents empirical results addressing our research questions. Finally, Section discusses limitations, safety and ethics concerns, and future research directions.

---

Table I: Comparing TWIGMA with other image datasets analyzed in the paper.
Dataset
Source
# Images
# Prompts/Captie
Additional Features
TWIGMA
Twitter
800,000
Inferred RIIIP captions
Metadara (date, likes, clastering).
(This paper)
Jan, 2020 Mar. 2023
Images come from anlltiple gen-Al models. Creation date, wee hash
Diffusion Discord
1.8 unique prompis
DifiusionDB
[2)
August 2022
WikiArt, Ukiyo-c.org
60,000
Painting names
Artist names, creation time, art genre.
Arteeach[0
1400-2000
Common Crawl
5,85 Billion
Corresponding text
NSFW and watermark detection
LAION-SB 4H
Filtered image-text similarity
2
Related work
The creation of large-scale image-text datasets has rapidly evolved, transitioning from carefully annotated datasets relying human labels [16 [1] vast collections of image-text pairs gathered from the web (22 37 As text-to-image models continue to demonstrate prompts, researchers have started
Al-generated image dataset:
unprecedented capabilities in generating images based curating similar datasets featuring images generated by these models. For example, DiffusionDB and Midjourney Kaggle provide large-scale datasets (14 million and 250,000, respectively) with prompts and images generated by StableDiffusion and Midjourney, respectively. However, these pioneering, general-purpone datasets are limited in terms of style (as they originate from a single model variation), user distribution (restricted to Discord users of specific channels), and relatively short data collection periods (one month in 2022 for both datasets). Researchers have also constructed datasets of Al Al-generated images for specialized use, such as detecting generated art images [46] [1], evaluating qualities of specific contents such as human portraits 66 18). investigating safety filters and hidden vocabularies [26] (34], and evaluating potential biases 24.. Given the problem- oorieem-oriented nature, these datasets are often limited in size and skewed in themes. Recognizing the gap between existing datasets and our research questions novelty, themes, and variation of Al-generated images, we curated TWIGMA to demonstrate its potential this paper.
Novelty and variation of Al-generated images: Many text-to-image models rely continuous train- ing with substantial amount of human artistic artifacts sourced from datasets such LAION [42). which includes images from platforms such as Wikiart and Pinterest, and potentially copyrighted or proprictary contents B1
34 44
Users have also fine-tuned the open-sourced StableDiffusion model
on additional samples from specific artists or art genres, such as anime, to generate Al- Al-generated art
imitating those styles [5 g 23 While prior research has examined the reproduction of training data
in Al-generated art settings [46 51 and from the perspective of adversarial attack [7] [6), large-scale empirical results comparing AI- and con-Al--gneratee images are relatively limited. This prompts our first research question (RQ) to investigate the novelty of Al-generated images:
RQI: How distinct are the distributions of Al-generated images from those of non-Al images?
Furthermore, the stochastic nature of many eext-to-iagee models has sparked inquiries about the variability of their generated image outputs. In notable court case, a judge determined that AI- generated images do not qualify for copyright protection, citing the significant disparity between the user's intended prompts for Midjourney and the resulting visual material it produced. Recent research has explored the variation within the prompt space [2 52 53). and our second RQ builds upon these findings and zooms in a particular dimension of RQI, the variation in the image space:
RQ2: How do Al-generated image variations compare to non- non-Al-generated counterparts?
Lastly, we leverage the extended time span offered by TWIGMA to explore our final RQ:
RQ3: How do the content and theme of Al-generated images change over time?
By answering RQ1-3, our paper makes the following contributions to the literature: metrics proposed in generative model evaluation to demonstrate the distinctiveness of Al Al-generated images compared to real images, providing evidence of the novelty embedded in the underlying models. Moreover, we find that the distance between Al-generated and human images (i) correlates with the number of likes received; and (ii) can be leveraged to identify human images that served as inspiration for Al-generated creations. Secondly, introduce quantitative measures to establish
Firstly, we utilize
3

---

that (i) images from generative models exhibit less diversity than real images; and (ii) a substantial portion of variations in the image space can be attributed to the variation in user prompts. Finally, we observed a shift in the preferences for generative models and image topics among Twitter users: they are increasingly sharing artistically sophisticated or distinct content, such as intricate human portraits and anime, while interest in simpler subjects like natural scenes and animals has declined.
3 Methods
Curating the TWIGMA dataset: To curate the TWIGMA dataset, we began with the initial filtering of available tweets using hashtags commonly associated with Al Al-generated images, such as #dalle, #stablediffusior and #aiart. We then proceeded to iteratively refine the set of hashtags by incorporating new hashtags that co-occur with the existing ones and contain highly relevant tweets with Al-generated images. We evaluated the quality of these new hashtags based on the first 20 tweets returned by the hashtag search. This iterative process allowed us to create a final set of 19 hashtags, encompassing both generic community descriptions (e.g.. #a art, #generativears
and specific models (e.g., #midj ourney, #dalle2); please refer to Figure T [a) for the complete list of hashtags.
Next, utilized the official Twitter API scrape tweets containing at least one of the identified hashtags. This process encompassed the time frame from January 1, 2021 to March 31, 2023, resulting in approximately 2.2 million tweets. Out of these potentially duplicated tweets, around 1.3 million contained downloadable photos as of April 2023. To ensure data quality, we conducted two-step deduplication process. Firstly, we utilized the media ID in Twitter to retain only one image in cases where the same tweet was associated with multiple hashtags. Secondly, we computed CLIP-ViT-L-14 embeddings [32 remove images with identical embeddings. This deduplication step resulted in dataset of 623,707 tweets and 805,650 images in TWIGMA.
Furthermore, we extracted comprehensive metadata for each image in the TWIGMA dataset, in- cluding original tweet texts, engagement metrics such as likes, as well as user-related informa- tion such follower counts. Recognizing that tweet texts may always accurately describe the image content, we also generated captions for each image in TWIGMA using the BLIP wrggrecords/80333
model [19]. We release TWIGMA at gi github. 1o/TWWGMA// This dataset includes our curated metadata
and provide a
tutorial at https: http:/yyiquuhehe pertaining to the image creation date, number of likes, assigned cluster membership obtained through k-means clustering, and the inferred BLIP captions. Additionally, we provide list of Twitter IDs and the necessary code to retrieve images and metadata using the Public Twitter API.
This paper presents a series of analyses to showcase the fascinating insights that TWIGMA can offer into human-Al-generann numerous other intriguing questions can be addressed using TWIGMA.
images. Our approaches represent initial exploration effort, and
Measuring distinctiveness of Al-generated images: To enable efficient computational evaluation without relying on human ratings, we operationalized the concept of "distinct" as the difference between the distribution Q of a generative model (e.g., images generated by StableDiffusion in TWIGMA) and the distribution of real data (e.g., real images in LAION (42)); we then leverage recent advances in quantifying the difference between two continuous distributions. Our analysis incorporated a diverse range of image datasets, including Al-generated images from TWIGMA and DiffusionDB, as well human images from LAION 42) and ArtBench (20] (see details in Table [ ArtBench features 60,000 high-quality, genre-annotated images of artwork spanning ten different art genres from the 14th century to the 21st century; and LAION encompasses HiP---millarttyfiiiii image-text pairs sourced from the Common Crawl.
To investigate the novelty of Al-generated images, first employed two-dimensional UMAP 25 in the CLIP embedding space to visualize the differences in distributions between Al-generated and non- Al-generated images. Additionally, we used k-means clustering (with the efficient implementation in FAISS (13 and computed Kullback-Leible (KL) divergence among pairs of image distributions via the quantization approach outlined in Pillutla et al. [30]: our results robust to the choices of divergence measures and specific estimators thereof.
Measuring the variation of A Al-generated images: Quantifying the variation of a continuous distribution a well-studied topic across many disciplines. In this paper, we primarily drew upon existing literature that assesses diversity in natural images and generated texts: (i) pairwise distance,

---

2, 12/ - 1), where 24, Ty are embeddings of randomly sampled images; this metric is equivalent to cosine distance when I2 and zj have unit Ez norms; (ii) inverse of explained variance (IEV) by the largest singular value 2 Ta/ai, where o; denotes the ath-largest singular value of the embedding matrix; (iii) product of marginal variances (PMV) [7]
computed as Σis ||:
(MI- 8:)
1/d
where & denotes the sample standard deviation of the ith feature; and (iv) entropy estimated using a quantization approach [35]. We adjusted the original definitions slightly for some measures to ensure that a higher value always indicates greater variability in the data.
To characterize the themes of Al-generated images posted on Twitter, we first applied k-means clustering to group images in TWIGMA. Subse- quently, we employed the BLIP [99 model to caption the images in TWIGMA, which facilitated the interpretation of the obtained image clusters. To assess the quality of the theme revealed by the BLIP captions, we conducted an additional experiment with four volunteer annotators (two males, and two females; two of them have used generative text-to-image tools). The volunteers were tasked to: (1) annotate randomly- ssammlee images (10 from each identified cluster) with three relevant keywords; and (2) devise groupings for the images without knowledge of the clustering we had established.
Characterizing the themes of Al Al-generated images:
4 Results
4.1
Quantified novelty and variations of A Algenerrteed images
In this section, we present our analysis of the novelty and variations of Al-generated images. Figure 2
displays different approaches we used to compare Al-generated and non-Al-generated images. The UMAP density plot in Figure reveals minimal overlap between sampled LAION, TWIGMA, and ArtBench images, whereas the overlap between TWIGMA and DiffusionDB is substantial We note a distinct density peak of TWIGMA images absent in DiffusionDB, primarily representing a subset of images generated in specific styles (such as anime) and often of NSFW nature (see Figure 4 for more details). This distinction is further confirmed by estimated k-means clusters and KL divergence between image distributions, as shown in panels (b) and (c) of Figure [ Specifically, cluster primarily consists of LAION images, while Al-generated images are present in clusters 2-4. Additionally, the estimated KL divergence between Al-generated and non- aon-AA-ggnnen Al-generated data pairs tends to be much larger compared to the divergence between DiffusionDB and TWIGMA.
While panels (a)-(c in Figure focused on examining whether AI Al-generated and non-Al-generater images are distinct, we additionally investigated whether image distances contain useful information about an image's likability. Figure presents the analysis results for TWIGMA data (limited to the subset for which we were able to obtain likes data using Twitter API): On the left side, we present the average similarities of the five nearest neighbors in L.AION and ArtBench, determined using cosine similarities, for the images TWIGMA. On the right side, we plot the average likes for each quintile of similarities, where quintiles 1 and 5 represent the least and most similar images to LAION or ArtBench, respectively. There is some evidence suggesting that Al-generated images that are less similar to their non-Al-gent -l-genrreee counterparts receive more likes. For instance quintile images for LAION and Artbench received, on average, 30.1 (95% CI: [28.8, 31.5)) and 31.9 (95% CI: [29.7. 34.1J) likes, respectively, compared to 23.8 (95% CI: [21.9, 25.7 and 24. (95% Cl: [21.5, 26.7I) likes in quintile 5. However, it's important to note that the number of likes for a tweet is influenced by various factors, such as user's social networks 49). To account for these factors, we used the following Poisson regression that incorporates the number of followers and similarity quintiles:
(Biikss X)) = a + 83 1500005500000 followers) + 8,115,00000 followers) + Yy (Quintie i). (1)
The regression model in I shows that within the same follower count category (i.c., < 500,500 5, 000, or 5, 000 followers), compared TWIGMA images most similar to their ArtBench neighbors (quintile 5), images in quintile 1 receive, on average, 19% more likes (95% CI: [18% 19%], p<0.0001); similar trends are observed for images in quintile 2 (12% more likes; 95% CI: [11%, 12%]; p<0.0001), while the differences are less pronounced for more similar quintiles (0.8%
fewer likes in quintile 3 and 2% more likes in quintile 4). Comparable results are found for LAION neighbors as well, where images in quintiles 1 and 2 receive, on average, 10% (95% CI: 19%, 10%]. pc0.0001) and 2% more (95% Cl: [2%, 3%), p=0.0001 likes, respectively, compared to quintile images posted by accounts with the same follower categories. Upon further investigation, we observe
5

---

(a)
(b)
(c)
dhargenco
Choter Quaser 5 Chockr Cluoe
4.0%
5.%%
3.0%
THNBKK GRI GEEE Rabeecs Locoe
5:42
3:66
Krnonn datner
0.0
1.0%
36 0%
60.0%
4.10
2:40
2:1
3.18
3.14
0.13
1.0%
0.0%
10.0%
19.0%
3:46
3.99
0.22
Antioce nataa onti
LAVV
Arbern
Drwsens
(d)
Arboot LACO
Average leee by simberity reference
Menea 30
LAKON
Dit.sionda
TWIEMA
0.25
3.50
0.75
1.20
Quntis (ound bemner))
Guamllee
Guinlee
Guinti (FRc3l serviaa)
Coanne
Costra veternnos
Figure 2: Comparing Al-generated and non-AI-generated images. (a): Kernel density plot of the 2D-UMAP embedding of sandomly-samples images from TWIGMA, LAION, DiffusionDB, and ArtBench. (b): K-means clustering (with = 4) separates Al-generated and non-Al-generated images. (c): Estimated KI. divergence of pairs of image distributions using the quantization approach (with 10 clusters) outlined Section 3 Each entry the table represents the KI. divergence between the row and column distributions, c.g., the 5.42 entry is the KL divergence between LAION and ArtBench, defined as >n, PLAION (z) log (PLAION(I)) Patt//cc
(d): (Left) Average cosine similarity of the five nearest neighbors in LAION/ArtBench for TWIGMA images. (Right) Average number of likes per similarity quintile, with error bars representing 95% confidence intervals. On average, TWIGMA images least similar to LAION/ArtBenh receive the most likes.
that many of the highly-liked images that stand out from the non-Al-generatee images are often anime-style NSFW photos. This observation highlights the need for caution when interpreting the number of likes as an additional indicator of aesthetic and creative value for an image.
Next, explore image variations across distributions (see Figure 3 (b)).. On average, LAION images exhibit the highest variability, followed closely by ArtBench, DiffusionDB, and TWIGMA, which show similar variation distributions based on pairwise Euclidean distance. Notably, ArtBench shows reduced variation when conditioned on image pairs from the same artist Similarly, conditioning on prompts in DiffusionDB reduces around 50% the variation in generated images, measured by the average distance between two image embeddings; that is, the average distance between images in DiffusionDB with the same prompt is roughly half that between randomly selected images with different prompts. Additional variation metrics in Figure 3b) align with pairwise distance. The lower estimated entropy for ArtBench may be due to its smaller sample size compared to the other datasets.
Given the observation that a substantial portion of the variations in the image output space is explained by the variation in the input prompts, we further analyzed the latter in Figure c Specifically, in (c), we plotted the average pairwise distance of images with the same prompt as a function of the number of words in the prompts. Each dot represents unique prompt, and we observe that, on average, longer prompts with more details lead to reduced variations in the output images (Kendall's T: -0.1, valuues:: 0.01). Furthermore, to calibrate the variation of the prompt variations, the pairwise distance of CLIP embeddings from different text input sources with similar word counts to the DiffusionDB prompts. These sources included real image captions (55, English Haikus [.] Twitter text (after removing hyperlinks and hashtags). CNN news summarization (43) and book chapter (55]. We found that image captions exhibit similar variation to DiffusionDB prompts and Twitter texts, indicating higher variability. In contrast, as expected, texts with unified styles and themes, such as Haikus and book chapters, show significantly lower variation. Overall, our analysis suggests wide range of topics covered by the DiffusionDB prompts.
compared

---

(a)
(b)
Entropy
Cosine Delance
(d)
LAION
27.3
2.7
1.42
0.55
Iroge
Arternch
ArBexc
DifusionDD
19.6
1.82
0.48
Haroi
TWIEMA
20.0
2.03
0.49
ArBarch
Sumne wrtal
0.38
DffaseeD
Poorpts
3.2
1.7
0.29
(c)
TWIGMA
waw pernoc estroo
Twitter
oo hel
Antlent
CRR
oz
save pesegg
Baoc
Conougt
Factinoo CUP
Prinaloo dicinc
Nummbrr worth prompts
Figure 3: Comparing variations of AI Al-generated and non-Al-generated images. (a): Pairwise Euclidean distance of randomly-samples æz-normalieed image embeddings from LAION, ArtBench, DiffusionDD., TWIGMA, paintings by the same artists in ArtBench, and DiffusionDB images with identical prompts, (b): Additional variation metrics (see Section 3 for details) for the dataset mentioned in (a); larger values of these metrics correspond for the most and least variable datasets are in bold and italic, respectively. (c): Average pairwise Euclidean distance fz-normalized embeddings in DiffusionDB with identical prompts (each dot unique prompt); average, longer and more detailed prompts correspond to reduced variations in output images. (d): Similar analysis as in (a), but with sampled text embedding data from real image captions, English Haikus, DiffusionDB prompts, Tweets, CNN news, and book chapter.
more variable dataset. Metric values
4.2 Themes of Al-generated images on Twitter
Here, we examine the themes and their longitudinal changes for images in the TWIGMA dataset. In Figure
4 M[), we display the creation-date color-coded two-dimensional UMAP of TWIGMA image embeddings, suggesting a shift in the image distribution over time in our Twitter dataset. To investigate this qualitative observation further, we applied k--maans clustering to the TWIGMA data with & = 10 and plotted the change of cluster membership over time. We note substantial changes in cluster memberships: clusters 1, 3, and have experienced a steady decline over time, while clusters 8 and 9 showed a consistent increase. Here, the choice of h: = 10 was motivated by the observation Figure 3a), which indicated that TWIGMA exhibits comparable variation to ArtBench, a dataset with 10 distinct art styles (sensitivity analysis using other values of l yielded similar results).
In Figure c), we visualize each cluster along with the most frequent topics derived using BLIP captions, with emojis representing the relative trends over time. Prominent themes we observed include painting, woman, and man, aligning with the known interest of users generating detailed human portraits in various styles using text-to-image models [5]
Our findings also indicate shift in preferences for image topics among Twitter users over time: there growing interest in sharing artistically sophisticated or distinct content, such intricate human portraits, while interest in simpler themes such as natural scenes has declined. addition, clusters 5 and 8 notably contain substantial number of images with increasing popularity but also significant amounts of NSFW, pornographic, and nude content. This observation aligns with recent studies that highlight the rapid growth of online communities focused on generating such contents [14] this observation, we obtained predicted NSFW scores using a pre-trained detector
48 53
34 41. Inspired by
4 revealing a
substantial number of likely NSFW images in both TWIGMA and LAION (see Figure d)).
We observe that portrayed themes in AI Al-generated images extend beyond singular art genres and are swiftly evolving over time. For human art example, during the Renaissance period spanning centuries, the focus remained primarily on real-life human portraits and scenes from contemporary life [50]. Similarly, modern art often features recurring motifs such as abstract shapes and vivid hues [36] (akin to examples in Clusters and illustrated in Figure c). for instance). This observa- tion the fast-evolving themes in Al-generated images finds support in efforts that decentralize and

---

individualize diffusion models through LoRA updates 12 characterized by specific themes within a span of just under a year.
39]. resulting in numerous model variants
We also evaluate the concordance between themes extracted from captioning models and those pinpointed by human annotator volunteers, as shown in Figure@ec) Notably, a substantial alignment emerges between themes uncovered via unsupervised learning (middle column) and those surfaced through the annotators' keyword-basee based approach (right column), with overlapping themes highlighted in bold. While the initial annotator pool is small, feedback from the volunteers shed useful insights: for instance, the two NSFW clusters primarily comprise explicit content of females, and therefore are challenging to differentiate for the human annotators. Moreover, comparing human grouping with k-means clustering unveils that images allocated to clusters 1, 2, and 7 exhibit substantial heterogeneity, often leading to their segregation into distinct clusters when annotated by humans.
Lastly, we used cosine similarity to extract the nearest neighbors of TWIGMA images and identified pairs with a high likelihood that the neighbors from ArtBench and LAION served as potential inspirations (i.e., training data) for the text-to-image models (see Figure Ie)). The striking similarities between some of these neighbor pairs suggest the potential of leveraging a similarity approach more systematically identify such pairs.
5
Discussion
Recent advancements in generative Al have revolutionized text-to-image generation, empowering users to create millions of captivating images. Our work explores themes and variations in Al- generated images. To facilitate this investigation, we introduce TWIGMA, an extensive dataset comprising 800,000 gen-Al images, associated tweets, and metadata collected from Twitter between January 2021 and March 2023. Our analysis characterizes distinctiveness, variation, and longitudinal shift of themes of gen-Al images shared on Twitter. The analysis in this paper is not meant to be exhaustive but rather to illustrate the types of interesting questions that TWIGMA can help to answer, opening the door investigating various facets of human-Al generation.
Limitations: It is important to note the limitations in our work, primarily due to the large-scale nature of the datasets employed. One limitation stems from the scope and quality of the datasets utilized in our analysis. Although we made efforts to select representative data sources from both Al-generated and non-Al-generated image spaces, the content within the final datasets used in paper imposes certain constraints: ArtBench primarily consists of popular art genres in WikiArt (2) resulting in limited coverage of non- non-European modern arts, well works from lesser-known known independent artists. Consequently, this could lead to an underestimate of the similarity between human art images and Al Al-generated art images. Similarly, LAION data was pre-filtered based on text-image pair similarity [42), and around 10% of Twitter images were not available through the official Twitter API at the time of our study, due the deletion or removal of tweets. Additionally, is possible that a subset of images within TWIGMA are -on-ee-gge,r,,ed, as users sometimes share relevant non-Al- --gne--geeee content, such as real images closely resembling Al Al-generated outputs or screenshots from model websites or APIs. All of the aforementioned data issues could bias our findings in Section 4.1 Lastly, it essential to acknowledge that while TWIGMA benefits from an extended time frame and Twitter's ' substantial user base in comparison to prior research efforts, the images in TWIGMA only constitute a subset of those publicly shared on specific channel. When interpreting outcomes drawn from this dataset, researchers should bear in mind that the user base inherent to TWIGMA may potentially differ from their demographic of interest.
Future work: There are several promising avenues for future research. Firstly, the inclusion of more contemporary and modern art images, particularly illustration art and anime, in our analysis, along with the identification of
images in TWIGMA using state-of-the-art meth- ods 46). would strengthen the robustness of our findings. Moreover, maintaining regular updates to the TWIGMA dataset in order to consistently monitor the trajectory of themes and contents of AI -e-gererte images would provide valuable insights for the research community. Furthermore, introducing a more human- centric perspective into dataset curation and analysis could enrich down- stream insights. For instance, our current emphasis on variations in the CLIP embedding space could be complemented by exploring other dimensions of diversity and variation [24] 29]. This could entail investigating stereotypical associations between output images and societal representations within human images (c.g., whether generated NSFW images are disproportionately associated with
8

---

(a)
(b)
Outer
Quter
Dutrr
(c)
Themes via unsupervised learning
Themes via human annotators
Cluster I: Painting, white, black
Abstract, colorful,
Cluster 2: Man, cat, day
Animal, cartoon
Cluster 3: City, man, futuristic
Futuristic; building, city
Cluster 4: Man, car, large
Car, vessel
Cluster S: Woman, posing, girl
Woman, pom, anime
Cluster Painting, background, forest
Art, painting, tree
Cluster Man, woman, puinting
Profile
Cluster 8: Girl, woman, hair
Anime, pom, fantasy
Cluster Woman, dress
Woman, angel
Cluster 10: Man, woman, robot
Man, profile
Additional notes annotators the same group. addition, themes in clusters 1,2, and are less coherent and robust in comparison other clusters.
images
8, clusters often assigned
(d)
(e)
TWIGGNA
Artlench" TARON NN
TINGIMA
JAKON KN
Antienchy
TWIGMA
Artiendy LAOON
Figure 4: Themes and longitudinal trends of images in TWIGMA. (a): Two-dimensional UMAP embedding TWIGMA images, color-coded based on their creation dates on Twitter. (b): Composition of image clusters (estimated using k- 6-means clustering with 10) over time. We observe notable changes cluster membership and underlying themes of TWIGMA images from 2021 to 2023, (c): Left: Randomly sampled images from the 10 clusters identified in (b). Clusters 5 and 8 predominantly contain NSFW photos and have been pixelated accordingly. Middle: Cluster annotations derived from the most frequently occurring words in the BLIP-inferred captions. Upward, downward, and rightward arrow emojis signify rising, declining, or consistent trends over time, respectively. Right: Cluster annotations obtained by human annotators. Themes that align between BLIP and human annotations are highlighted in bold. (d): Predicted NSFW scores from a pre- trained CLIP-based NSFW detector. Noteworthy presence of NSFW photos is observed in both TWIGMA and LAION datasets. (e): TWIGMA images and their nearest neighbors in terms of image embedding cosine similarities from ArtBench LAION. These images from ArtBench and LAION serve as potential inspirations (i.e., training data) for the text-to-image model, indicating the possibility of identifying such pairs using similarity-measuree bbasdd approach.

---

one race or gender group). As another example, one potential follow-up study could leverage large pool of human annotators to more comprehensively investigate two pivotal aspects: (i) the alignment between human comprehension and categorization of TWIGMA images and scalable machine learning outputs; and (ii) the correlation between human ratings and the number of likes for an image (included as part of TWIGMA). These studies could empower the development of future generative AI models that aim to align more closely with specific human preferences and image styles.
Safety and ethical concerns: One challenging aspect of text-to-image generative models is the generation of NSFW content. Our data analysis revealed substantial subset of explicit images posted on Twitter, even among those with high likes and retweets. While some generative models like DALLE2 and StableDiffusion have built-in safety filters that block generated NSFW images, these filters can still be circumvented through prompt engineering 34 subcategory of models and online communities specifically dedicated to generating NSFW content. These observations serve as cautionary tale for large-scale studies involving Al-generated content, as NSFW content is likely to be present and popular among certain subsets of followers due to the relatively low regulation in the current landscape. Additionally, our analysis also highlighted some Al-generated images that bear striking resemblance to images found in human images and art datasets. These findings bear implications for potential copyright violations if training images were used without explicit consent, especially when the outputs are essentially reproductions minor edits of memorized training examples. Finally, there is a risk of perpetuating stereotypical representations if AI- al-generated images tend to resemble specific groups based demographic identities. Similar observations, discussions, and potential mitigations have been noted in other recent works 10 24
Moreover, there is a
6 Conclusion
Understanding themes. contents, and user interests of Al-generated images critical topic that requires data beyond the currently available prompt and output image pairs datasets. We introduced TWIGMA, an extensive dataset comprising 800,000 gen-Al images, tweets, and associated metadata from Twitter (Jan 2021-March 2023). Our contribution is two-fold: Firstly, TWIGMA enables analysis of Al-generated image content and evolution across models and time on Twitter. Additionally, our analysis highlights the distinctiveness and variation of Al-generated images when compared to non-Al -------geeeeteed content. We hope that our dataset and analysis will contribute to the broader discourse on the safety, novelty, and sociological or legal challenges posed by Al-generated images.
Acknowledgments and Disclosure of Funding:
We thank Zhi Huang for sharing their code to scrap Twitter content using the public Twitter API. YC is supported by a Stanford Data Science Postdoctoral Fellowship. JZ is supported by the National Science Foundation (CCF 1763191 and CAREER 1942926), the US National Institutes of Health (P30AG059307 and U0IMH098953) and grants from the Silicon Valley Foundation and the Chan-Zuckerberg Initiative.
References
[1] Haiku dataset. http::/langgiiiiiieooodddseseess Accessed: 2023-06-06.
[2] WikiArt. https///www.wikiart.rr/// Accessed: 2023-5-5.
[3] Midjourney user prompts. http://www...gggecoommmoseeeeees texttoimage, 2022.
Accessed: 2023-06-06.
[4] CLIP-based NSFW Detector. Detector.htpps//giitbbbbcom http://ghhaacooo.ooooovooc.....d 2022. Accessed: 2023-06-06.
[5] Waifu diffusion. hutp://hngginggacccoooomtevvrvva
2022. Accessed: 2023-6-4.
[6] Ali Borji. Generated faces the wild: Quantitative comparison of Stable Diffusion, Midjourney
and DALL-E 2. arXiv preprint arXiv:2210.00550 2022.
10

---

[7] Nicholas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramer, Borja Balle, Daphne Ippolito, and Eric Wallace. Extracting training data from diffusion models. arXiv preprint arXiv:2301. 13188, 2023.
[8] Sheng-Yer Yen Chou, Pin- Yu Chen, and Tsung-Yi Ho. How to backdoor diffusion models? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4015-4024, 2023.
[9] Jessica L Feuston and Jed R Brubaker. Putting tools in their place: The role of time and perspective Human-Al collaboration for qualitative analysis. Proc. ACM Hum. Comp-C Interact., (CSSCW2):1222 October 2021.
[10] Noa Garcia, Yusuke Hirota, Yankun Wu, and Yuta Nakashima. Uncurated image-text datasets: Shedding light on demographic bias. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6957-6966, 2023.
[11] Nathan Hodas and Kristina Lerman. The simple rules of social contagion. Scientific Reports, 4(1):1-7, March 2014.
[12] Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li. Shean Wang,
Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. June 2021.
[13] Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, 7(3):535-547, 2019.
[14] Alex Kim. Nsfw data scraper. https//ggihbbb.commaacxokkk////t scrrpe 2022, Ac- cessed: 2023-06-06.
[15] Gwanghyun Kim and Jong Chul Ye. DiffusionCLIP: Text-guided image manipulation using diffusion models. arXiv preprint arXiv:2 10.02711, 2023.
[16] Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, et al. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International Journal of Computer Vision, 123:32-73, 2017.
[17] Yi-An Lai, Xuan Zhu, Yi Zhang, and Mona Diab. Diversity, density, and homogeneity: Quantitative characteristic metrics for text collections. arXiv preprint arXiv:2003, 08529, 2020.
[18] Kimin Lee, Hao Liu, Moonkyung Ryu, Olivia Watkins, Yuqing Du, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, and Shixiang Shane Gu. Aligning text-to-image models using human feedback. arXiv preprini arXiv:2302. 12192, 2023.
[19] Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. BLIP: Bootstrapping language- image pre-training for unified vision-language understanding and generation. In International Conference on Machine Learning, pages 12888-12900. PMLR, 2022.
[20] Peiyuan Liao, Xiuyu Li, Xihui Liu, and Kurt Keutzer. The AnBench dataset: Benchmarking generative models with artworks. arXiv preprint arXiv:2206. / 1404, 2022.
[21] Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft COCO: Common objects in context. In ECCV 2014, pages 740-755, 2014.
[22] Vivian Liu and Lydia Chilton. Design guidelines for prompt engineering text-to-image generative models. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, number Article 384 in CHI '22, pages 1-23, New York, NY, USA, April 2022. Association for Computing Machinery.
[23] Haoming Lu, Hazarapet Tunanyan, Kai Wang, Shant Navasardyan, Zhangyang Wang, and Humphrey Shi. Specialist diffusion: Plug-and-Play Sample-Efficient Fine-Tuning of Text-to- Image diffusion models to learn any unseen style. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 14267-14276. openaccess thecvf.com, 2023.

---

[24] Alexandra Sasha Luccioni, Christopher Akiki, Margaret Mitchell, and Yacine Jernite. Stable bias: Analyzing societal representations in diffusion models. arXiv preprint arXiv:2303.11408. 2023.
[25] Leland MeInnes, John Healy, and James Melville. UMAP:
Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv: arXiv: 080..0022.226. 2018.
[26] Raphaël Millière. Adversarial attacks on image generation with made-up words. arXiv preprint arXiv:2208.0411 2022.
[27] Changhoon Oh, Jungwoo Song, Jinhan Choi, Seonghyeon Kim, Sungwoo Lee, and Bongwon Suh. I lead, you help but only with enough details: Understanding user experience of Co- Creation with artificial intelligence. In Proceedings of the 2018 CHI Conference Human Factors in Computing Systems, number Paper 649 in CHI '18 pages 1-13, New York, NY, USA, April 2018. Association for Computing Machinery.
[28] Jonas Oppenlaender. The creativity of text-to-image generation. In Proceedings of the 25th International Academic Mindtrek Conference, Academic Mindtrek '22, pages 192-202, New York, NY, USA, November 2022. Association for Computing Machinery.
[29] Hadas Orgad, Bahjat Kawar, and Yonatan Belinkov. Editing implicit assumptions in Text-to- Image diffusion models. arXiv preprint arXiv:2333008000 2023.
[30] Krishna Pillutla, Lang Liu, John Thickstun, Sean Welleck. Swabha Swayamdipta, Rowan Zellers, Sewoong Oh, Yejin Choi, and Zaid Harchaoui. MAUVE scores for generative models: Theory and practice. arXiv preprint 22x.i:2222.115555 2022.
[31] Giada Pistilli, Carlos Munoz Ferrandis, Yacine Jernite, and Margaret Mitchell. Stronger together: on the articulation of ethical charters, legal tools, and technical documentation in ML. arXiv preprint htt.::233...6111.. 2023.
[32] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Proceed- ings of the 38th International Conference on Machine Learning, pages
2021.
[33] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In Proceedings of the 38th International Conference on Machine Learning, volume 139, pages 8821-8831, 2021.
[34] Javier Rando, Daniel Paleka, David Lindner, Lennart Heim, and Florian Tramèr. Red-Teame Red-Teaming the Stable Diffusion safety filter. arXiv preprint arXiv:2210.04610 2022.
[35] Samuel Rhys Cox, Yunlong Wang, Ashraf Abdul, Christian von der Weth, and Brian Y. Lim. Directed diversity: Leveraging language embedding distances for collective creativity in crowd idcation. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, number Article 393 in CHI '21, pages 1-35, New York, NY, USA, May 2021. Association for Computing Machinery.
[36] Robertson, Jean, McDaniel, and Craig. Themes of contemporary art visual art after 1980. cir.nii.ac.jp 2013.
[37] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684-10695, 2022.
[38] Kevin Roose. An Al-generated picture won art prize. artists aren't happy. The New York times, 2(September), 2022.
[39] Simo Ryu. LoRA: Using low-rank adaptation to quickly fine-tune diffusion models. http://ggthbbb.omm//oooossoooooo
12

---

[40] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour. Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, and Others. Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Information Processing Systems, 2022.
[41] Patrick Schramowski, Manuel Brack, Björn Deiseroth, and Kristian Kersting. Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, 2023.
[42] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaezmarczyk, and Jenia Jitsev. LAION-5B: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 2022.
[43] Abigail See, Peter Liu, and Christopher Manning. Get to the point: Summarization with pointer-generator networks. arXiv preprint 17Xx::1704.04365 2017.
[44] Shawn Shan, Jenna Cryan, Emily Wenger, Haitao Zheng, Rana Hanocka, and Ben Y Zhao. GLAZE: Protecting artists from style mimicry by text-to-image models. arXiv preprint arXiv:2302.04222, 2023.
[45] Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018.
[46] Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein.
Diffusion art or digital forgery? investigating data replication in diffusion models. Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pages 2023.
[47] Krishna Srinivasan, Karthik Raman, Jiecao Chen, Michael Bendersky, and Mare Najork. Wit: Wikipedia-based image text dataset for multimodal multilingual machine learning. In Pro- ceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2443-2449, 2021.
[48] Michat Stypułkowski, Konstantinos Vougioukas, Sen He, Maciej Zięba, Stavros Petridis, and Maja Pantic. Diffused heads: Diffusion models beat GANs on talking-face generation. arXiv preprint arXiv:2307. 033996 2023.
[49] Romilla Syed, Maryam Rahafrooz, and Jeffrey M Keisler. What takes to get retweeted: An analysis of software vulnerability messages. Computers in Human Behavior, 80:207-215, 2018.
[50] The Editors of Encyclopedia Britannica. Renaissance art. June 2023.
[51] Yabin Wang, Zhiwu Huang, and Xiaopeng Hong. Benchmarking deepart detection. arXiv preprint 20X.4:2302.14445 2023.
[52] Zijie Wang, Evan Montoya, David Munechika, Haoyang Yang, Benjamin Hoover, and Duen Horng Chau. DiffusionDB: A large-scale prompt gallery dataset for text-to-image generative models. arXiv preprint arXiv:2210.14896 2022.
[53] Yutong Xie, Zhaoying Pan, Jinge Ma, Luo Jie, and Qiaozhu Mei. A prompt log analysis of text-to-image generation systems. In Proceedings of the ACM Web Conference 2023, pages 3892 3902. Association for Computing Machinery, 2023.
[54] Wenxuan Zhou, Bill Yuchen Lin, and Xiang Ren. IsoBN: Fine-Tuning BERT with isotropic batch normalization. Proceedings of the AAAI Conference on Artificial Intelligence, 35(16): 14621- May 2021.
[55] Yukun Zhu, Ryan Kiros, Rich Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In 2015 IEFE International Conference on Computer Vision (ICCV), pages 19-27. IEEE, December 2015.
13


Reviews:
[{"review_id": "qQKR0u14Wj", "reviewer": "Reviewer_VLov", "summary": "", "questions": "", "limitations": "The authors have addressed the limitations.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "1. The process of how to construct the dataset is detailly described.\n2. The paper is easy to read and follow.\n", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "d8jAglROYf", "reviewer": "Reviewer_Araw", "summary": "", "questions": "", "limitations": "In the \"limitations\" paragraph, it is not clear how the authors \"estimate the bias form non-AI-generated images\" to be quite small. What does \"estimate\" mean? Maybe they want to say that they don't have ways of estimating it and, therefore, they \"assume\" it? Once again, this is a strong claim that is not properly justified.", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "The longitudinal study is an important feature of this dataset, as well as the presence of metadata from the tweets. In addition, the proposed research questions are very interesting (RQ2 could be better phrased).", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "ii1j7fvRIF", "reviewer": "Reviewer_8zrT", "summary": "", "questions": "", "limitations": "The author has adequately addressed the limitations and potential negative societal impact of their work in section 5.", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "The proposed TWIGMA dataset represents a groundbreaking advancement in the domain of AI-generated text-image datasets, which help researchers to explore and analyze temporal trends in human-AI generated image content, a capability previously unattainable in the field.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "WMmalPuFGW", "reviewer": "Reviewer_vkpY", "summary": "", "questions": "", "limitations": "The paper should acknowledge that Twitter users are not necessarily representative and that TWIGMA inherits biases from the platform and its users. The paper states that previous datasets are limited in terms of user distribution, however, the paper does not evidence that it is not also limited in the same way.", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "- The clarity of writing is a strength\n- A large-scale dataset of 800k Twitter IDs is released alongside metadata, allowing data users to download the associated Tweets and images if they wish to conduct their own analyses. This respects Twitter's data use and sharing policies\n- A temporal study of the content of AI-generated images on Twitter is conducted, which is different to previous studies introducing datasets that focus on detecting fake artworks or evaluating image quality. This can be useful for studying how people's use of AI-based technologies change over time.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "ekKVUHitXQ", "reviewer": "Reviewer_s6di", "summary": "", "questions": "", "limitations": "The limitations were adequate enough, at least in my view.", "rating": 9, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "- Well-written paper.\n- Clear and concise structure.\n- A dataset that might be the first of its kind (including gen-AI images).\n- Valuable contribution in many domains dealing with gen-AI, its effects, and overall impact.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}]

Author rebuttal:

Meta-review: This paper presents a novel dataset on a timely topic and accompanying analyses of that dataset to analyze its contents. The authors have made relevant changes following reviewer feedback and have already significantly improved their submission.
One hesitation I have is given changes in Twitter's API, researchers no longer have the possibility to "hydrate" the dataset contents by adding the images themselves, which limits the directions of future work. Perhaps the authors should make the images available upon request, or explore other options of sharing them. I would like this to be reflected in future versions of the paper.
Decision: Accept (Poster)