Paper: Social Reward: Evaluating and Enhancing Generative AI through Million-User Feedback from an Online Creative Community (ICLR-2024)

Reviewer comment (Evaluation):
In the proposed social reward metric, the authors mainly focus on the creation-related metric such as the number of times an image has been reused for editing purposes by other users. However, user comments, views or likes are also important dimensions. It’s better to show whether a higher value of the proposed metric in this paper will induce a lower or higher value of the traditional metrics.

Author response:
Please accept our heartfelt appreciation for your diligent and insightful review of our manuscript. Your detailed observations and suggestions have shed light on several aspects of our paper that require further attention and refinement, and we believe they will significantly enhance the overall quality of the manuscript.  
With respect to the reasoning behind prioritizing remixes as user feedback metric (**weakness #1**) there are several considerations:  
- Since our platform revolves around creative editing and sharing images, users are naturally motivated to express themselves visually through remixes. This can be illustrated by remixes being made by our user base **25** times more than like and **40** times more than comment. Thus remix provides a much richer source of user feedback.  
- Remixes inherently involve a more active form of engagement compared to more passive actions like views, comments or likes. Users not only appreciate the image but actively participate in the creative process by remixing it, showcasing a deeper level of involvement and investment in the content thus adding a layer of quality to the engagement metric.  
- Directly using views as a metric may not accurately reflect user engagement because views only measure passive consumption, providing no indication of user interaction or appreciation. It doesn't distinguish between users who briefly glance at an image and those who actively engage with it.

Reviewer comment (Experiments):
The social reward metric in this paper is optimized using the triplet loss. As a dataset and benchmark paper, it’s recommended to compare with other existing contrastive learning methods such as NCE or infoNCE.

Author response:
**The response to the weakness #2:**  
We agree with your suggestion of comparing the model performance trained with other contrastive learning based losses. In fact, we were running experiments with different losses and observed limited impact of loss on model accuracy.  
For instance, model trained with more conventional Contrastive loss produced 67.1 % in pairwise accuracy metric on test set. Suggested by you InfoNCE yielded a higher 68.5% on the test set. But since the best performance was achieved by Triplet loss (69.7%) it was the one included in the paper.  
Model performance comparison with different loss functions will be enclosed to the paper submission as an appendix.

Reviewer comment (Experiments):
The constructed dataset consists of prompt, positive image, and negative image. It’s unclear what’s the specific threshold to decide the positive and negative image.

Author response:
**The response to the weakness #3:**  
In order to address you question better please find below a detailed description of PA Image-Social dataset collection procedure. This description will also be added to the Appendix part of our manuscript.  
- We start from discovering the most popular synthetic images in Platform A in terms of accrued remixes within a 10 month time frame. 
All images are sorted by remix number in descending order.  
- Top 1% of the images are selected as positive images (in paper this approach is called “content signal”, section 3.2). 
Additionally, in case if the image had been remixed by an influencer user it is also added to the set of positive images (in paper this approach is called “creator signal”, section 3.2).   
- Then we retrieve prompts associated with collected positive images.  
- For the obtained prompts, we retrieve all the images that received at least one remix and construct the distribution of view count before the first remix.  
- We cut that distribution by 99% percentile and use the obtained view count threshold in the next steps.  
- For the same prompts, we retrieve all the images that received exactly zero remixes (there are large amounts of “zero remix” images due to “Pareto like” nature of content diffusion in social platforms).  
- For every prompt, we filter out “zero remix” images that are below view count threshold. In such a way, only images for which there is enough evidence of negative user feedback will be labeled as such.  
- During data collection procedure some cases which cause inconsistencies are dropped, for example, we drop prompts for which there are no images other than the ones labeled as popular or there are no images that received zero remix etc.  
- Additionally, we employ in-house mature content detection models to filter out images that fall within the NSFW category.

Reviewer comment (Writing):
I failed to follow what the authors meant by referring to Pick-a-Pic as “relatively small scale of collected user preferences along with absence of ‘collective feedback’”. Please explain why.

Author response:
Under the relatively small scale of collected user preferences in Pick-a-Pic dataset we mainly meant comparison with our PA Image-Social dataset. For instance, we have about 2.5 times more images (656K vs 1.7 M), about 5 times more image pairs (615 K vs 3M) and about 2.5 more prompts (38.5 K vs 104 K). Please refer to Table 1. But most importantly is 230 times difference in the number of users involved (6.4 K vs 1.5 M). This number is critical in the light of our research goal, evaluating the performance of  text-to-image models within the context of social network popularity for creative editing, because of two reasons:  
- It is necessary for the collected human preference to be from a large group of users to be deemed representative of community scale feedback 
- The authors of  Pick-a-Pick built a web application, where users can generate images and express their preferences on them. Each prompt-image pair is getting feedback from a single user (the one who generated the image) in contrast to **”collective feedback”** (which implies multiple users’ independent feedback on the same image), that is the principle upon which PA Image-Social dataset is built.

Reviewer comment (Evaluation):
In Figure 7, it is hard to tell why Social Reward fine-tuning actually improves, as the examples might just be cherry picked and observing them doesn’t give me the intuition what Social Reward essentially improves or corrects any pattern consistently. Table 6 numbers are also marginally close to each other. Could the authors elaborate some critical win cases or “take home points”, that can be consistently observed after using Social Reward fine-tuning?

Author response:
For more visuals of generated image comparison between baseline and fine-tuned models please refer to **figure 3** of supplementary material appendix file named **“SocialReward_Appendix_ICLR_2024_Conference_Submissions.pdf”**.   
We believe numbers reported in **Table 6** showcase significant performance gain of fine tuned model over the baseline. The percentage values in the table signify the percentage of times an image generated by a fine-tuned model is favored in terms of specific metric over the one generated by baseline model for the given prompt. We can observe that with exception of CLIP alignment, all scores are favoring fine tuned versions in at least 63% of cases and up to 75% of cases.  
With respect to the consistent patterns observed  in text-to-image models fine-tuned using SocialReward here are some of them:

1. **Improved quality:**
 - Overall quality/fidelity of generated images visibly improved which can be noticed in decreased number of artifacts, distorted objects and shapes, more minimalistic and higher aesthetic quality of generated images

2. **Complete Display of Main Objects:**
  - The model consistently generates images where the main object is fully displayed, minimizing instances of partial or cropped representations. This characteristic ensures that the central focus of the image is readily available for creative manipulation.

3. **Preference for Minimal Text:**
  - There is a notable tendency for the model to avoid generating unnecessary text in the images, emphasizing a visual-first approach and compensating for the weakness of current models to generate text without artifacts.

4. **Reduced Collages and Merged Images:**
  - There is a decrease in the generation of complex image collages or the merging of multiple images. This design choice provides users with a clean canvas, facilitating easier integration of additional elements during the creative editing phase.

5. **Clarity in Backgrounds, Especially in Natural Scenes:**
  - The model consistently produces images with clear and distinct backgrounds, particularly noticeable in natural scenery. The absence of additional objects in the background enhances the usability of the generated images for creative projects, offering a blank slate for further customization.

Reviewer comment (Experiments):
The study relies entirely on self-report survey measures. Self-report can be subject to biases like social desirability bias where participants answer in a way they feel is more socially acceptable rather than reflecting their true thoughts and behaviors. How the authors think their data curation was/was not affected by the social bias?

Author response:
While social bias presents an important concern in terms of collecting candid and reliable responses in self-report surveys, we believe its impact within the scope of our study is limited.  We leverage organic user behavior as a source of our data. Users are not asked to participate in the survey, solely implicit anonymous feedback in the form of users organically choosing images for editing purposes is used.  
However, we fully share the concern of  accounting for biases in data collection procedure from such noisy source as social network and specifically address such biases as Content Exposure Time Bias, User Follower Base Bias and Prompt Bias **(section 3.2 Data Collection).**

Reviewer comment (Experiments):
(Minor) For robust user study, as a cross-sectional study collecting data at only one time point, it cannot determine causation or the direction of effects. A longitudinal design collecting multiple waves of data over time would allow for stronger conclusions about how the variables influence each other over time.

Author response:
Thank you for your very valid remark regarding the longitudinal design of data collection.  
Although our study may not establish causation or the direction of effects in the same way a classic
longitudinal design would, the extended data collection period can still provide valuable insights into how
variables may influence each other over time. Due to this consideration of the importance of having a representative dataset in temporal dimension, in the scope of this work we collected data from a prolonged period of 10 months in the scope.  
We plan to dive into a longitudinal study for user preference shift and how that might lead to our score model updating over time, as the scope of our next work.

Reviewer comment (Experiments):
The paper offers a comprehensive introduction to the metrics and evaluation methods; however, I am very curious about the PA Image-Social dataset and want to learn more about its details.

Author response:
We would like to express our sincere gratitude for your thoughtful and constructive review of our paper submission.
We appreciate the time and effort you have invested in reviewing our work. 
With respect to the question you raised, firstly, let us note that our paper indeed can greatly benefit from incorporating more information regarding PA Image-Social dataset. On the second note, here are the details you requested: 
- Popular (deemed as "positive") images constitute 8% of all images in PA Image-Social dataset. The main reason behind the smaller number of “positive” images in PA Image-Social dataset is typical “Pareto like” pattern observed in majority social networks where minority of the content gathers majority of community engagement. 
- There are 174 influencer users (users are categorized as influential in our platform if they have a certain amount of followers and are frequently posting new content) whose remixing of generated images contributed to the dataset collection.
- Engagement levels of individual users, in our case, expectedly follow power law distribution logic (pattern which usually describes user activity in online social platforms). For instance, the least active 50% of our users generate about 30% of remixes. Whereas the most active 10% generate about 40% of remixes.  
- We plan to include this information in the paper’s Appendix section.

Reviewer comment (Experiments):
I am curious about the data distribution within the collected dataset. It would be highly informative if the authors could provide insights into key aspects such as the percentage of popular images, the presence of influential users, and the overall engagement levels of individual users.

Author response:
We would like to express our sincere gratitude for your thoughtful and constructive review of our paper submission.
We appreciate the time and effort you have invested in reviewing our work. 
With respect to the question you raised, firstly, let us note that our paper indeed can greatly benefit from incorporating more information regarding PA Image-Social dataset. On the second note, here are the details you requested: 
- Popular (deemed as "positive") images constitute 8% of all images in PA Image-Social dataset. The main reason behind the smaller number of “positive” images in PA Image-Social dataset is typical “Pareto like” pattern observed in majority social networks where minority of the content gathers majority of community engagement. 
- There are 174 influencer users (users are categorized as influential in our platform if they have a certain amount of followers and are frequently posting new content) whose remixing of generated images contributed to the dataset collection.
- Engagement levels of individual users, in our case, expectedly follow power law distribution logic (pattern which usually describes user activity in online social platforms). For instance, the least active 50% of our users generate about 30% of remixes. Whereas the most active 10% generate about 40% of remixes.  
- We plan to include this information in the paper’s Appendix section.