RePALM: Popular Quote Tweet Generation via Auto-Response Augmentation (2024)

Abstract: A quote tweet allows users to share others’ content and add their comments. To help users write a quote tweet with better public engagement, we study the popular quote tweet generation task. It aims to generate quote tweets with higher popularity reflected by more likes, replies, and retweets. Despite the exceptional language generation capabilities of large language models (LLMs), limited work has examined how LLMs can learn the popularity of text to engage the public better. Consequently, we propose a novel Response-augmented PopularityAligned Language Model (RePALM) to align language generation to popularity by leveraging readers’ insights from augmented autoresponses. Here, we employ the Proximal Policy Optimization framework with a dualreward mechanism to jointly explore the quote tweet’s popularity and consistency with the auto-responses. For experiments, we gathered two datasets of quote tweets with external links and others’ tweets. Extensive results show the superiority of RePALM over advanced language models without response augmentation.

heading: 1 Introduction
text: A quote tweet is a tweet that shares a source message, e.g., an external link or another user’s tweet, by quoting them and adding accompanying comments. Its purpose is to enhance the visibility of the source message, which is beneficial to various applications, such as media broadcasts, advertisements, and social media engagement (Lin et al., 2023; Dujeancourt and Garz, 2023; Bandy and Diakopoulos, 2021). A popular quote tweet can prompt public readers to engage in discussions actively. It essentially helps broaden the dissemination of the source message, inciting a more dynamic discourse and exchange of viewpoints
**Corresponding author
among users. Previous work showed that the wording of tweets could substantially impact popularity, reflected by user replies, retweets, and likes (Tan et al., 2014).
Nevertheless, many users are not good at writing popular quote tweets. To help them better engage the public for meaningful interactions, we present a novel task of popular quote tweet generation to extensively study how NLP models can learn to generate a popular quote tweet given a source message of an external link or other users’ tweets.
Despite the recent advances of LLMs in language generation (Wei et al., 2021; Ouyang et al., 2022b), the mainstream research focuses on the writing itself. However, limited work concerns the public readers’ reactions to the text. For this reason, existing models cannot effectively understand the text’s popularity, which reflects its potential to draw public engagement. To illustrate this point, Table 1 shows a sample source message of news fol-
9566
lowed by the manually written and automatic quote tweets. We observe that the manually written reference is rich in original thoughts and opinions. On the contrary, LLaMA2-chat (Touvron et al., 2023) and ChatGPT (Ouyang et al., 2022b) summarize the news without incorporating any additional insights, thus less likely to draw engagement.
Viewing LLMs’ limitation in popularity learning, we propose a novel Response-augmented Popularity-Aligned Language Model (RePALM). RePALM learns to generate popular quote tweets by employing LLMs to predict possible reader responses, which work as a mirror to reflect public reactions for potential engagement measurements. Augmented by these (auto-)responses, RePALM is trained to align the quote tweet writing to popularity measure via reinforcement learning (RL).
Concretely, we first gather multiple LLMgenerated auto-responses and select those that best match the source message with a consistency matching method. Then, we feed a source message with its selected responses into RePALM to generate multiple quote tweets. Next, we optimize RePALM’s training process with the Proximal Policy Optimization (PPO) framework (Schulman et al., 2017) with a novel dual-reward design. Here, one reward is to predict popularity trained with a popular-unpopular quote tweet pairs; the other measures consistency between generated quote tweets and selected responses to align with public reactions. Finally, we develop a reward ranking and sampling method to select high-reward training examples to improve training effectiveness.
To the best of our knowledge, RePALM is the first model to utilize LLM-predicted auto-responses for popularity-aligned language generation. By learning from these potential responses, RePALM can effectively generate popular quote tweets that help draw public engagement. For example, as illustrated in Table 1, the output of RePALM is rich in captivating viewpoints, such as “blown away by its capabilities” and “just learn and try it.”
As a pilot study on popular quote tweet generation, we benchmark the task with two datasets: QuoteLink with tweets that quote external links and QuoteTweet with tweets that quote other users’ tweets. There are 70K pairs of popular-unpopular samples; each pair quotes the same source and is from the same author, yet one is more popular.
We further experiment with the two datasets. The main results first show that RePALM outperforms all comparison models in both automatic
measure and human evaluation. For example, RePALM achieves 23.26 in Rouge-1 metric, compared to 20.94 from ChatGLM3. Besides, the ablation study implies the positive contributions of varying RePALM modules. Then, quantitative analyses show the effectiveness of RePALM in varying scenarios. After that, we conduct a case study to interpret why RePALM can perform better. Next, we analyze the wording of quote tweets from four aspects to examine the essential features of popularity and how RePALM effectively captures them. At last, a case study interprets RePALM’s superiority. In summary, our contributions are threefold: • We present the first popular quote tweet generation study with two large-scale datasets. • We propose RePALM with dual-reward RL to exploit auto-responses to reflect public reactions for aligning language generation to popularity. • We extensively experiment with popular quote tweet generation and show RePALM’s superiority.

heading: 2 Related Work
text: Quote Tweet Generation. Although newly proposed, our task can benefit from two lines of methods: summarization and headline generation. The former (Phang et al., 2022; Lewis et al., 2020) aims to extract the salient contents from the source text, and the latter (Kanungo et al., 2021; Zhang et al., 2020a) to create a headline to summarize or quote the source. Additionally, text reframing (Ziems et al., 2022), as a technique for altering the expression or style of text, can help rewrite quote tweets. However, most methods focused on the writing without considering the popularity factors for further public engagements on social media.
Our work is further related to language generation in a broader scope. The emergence of LLMs has substantially advanced this field, especially in the zero-shot domain. Taking recent advances in LLMs, many studies have examined how to align language models with human feedback. For example, ChatGPT, a closely related model to InstructGPT (Ouyang et al., 2022b), is specifically trained to follow human instructions. LLaMA2-chat (Touvron et al., 2023) is an open-source language model that demonstrates SOTA performance in conversational abilities. Our RePALM explores aligning the language model with popularity for quote tweet generation, which has not been studied previously.
Popularity Analysis. Our task is also related to popularity prediction on social media, where
users express their preferences through replying, liking, or retweeting. The count of such behavior is usually adopted as the popularity indicator. Tan et al. (2014) analyzed the effect of wording on tweet propagation. Lamprinidis et al. (2018) used a multi-task GRU network to predict headline popularity. Kano et al. (2018) employed such popularity measure to supervise extractive summarization distantly. Gao et al. (2020) leveraged social media feedback data to build a large-scale dataset to predict popularity. Yu et al. (2024) propose popularityaligned language models to generate top-liked user replies to social media events.
Response Augmentation. Our method is inspired by previous work enriching context with augmented responses to provide readers’ views and enhance NLP training. Xu and Li (2022) borrowed human senses by retrieving responses for social media multimodal classification. Niu et al. (2023) incorporated responses to supplement image features for image aesthetics assessment. Liu et al. (2023) employed human responses for humor detection in short-form videos. However, previous related work mainly relies on existing responses, which cannot be applied in scenarios without human responses. On the contrary, we make the first efforts to utilize LLMs to simulate potential user responses automatically and enable language generation models to gain a better sense of popularity.

heading: 3 Quote Tweet Datasets
text: We collected large-scale data from Twitter for our popular quote tweet generation task. Based on the
source message types, we separated the data into two distinct datasets: QuoteLink and QuoteTweet, where the former gathering quote tweets for external links and the latter for other users’ tweets.
Data Collection. Following Nguyen et al. (2020), we first downloaded the general Twitter streams from 02/2016 to 10/2018. Then, we removed duplicate users and shortlisted the tweets from users with over 10,000 followers; the reason for that is to investigate tweets with a specific degree of visibility to measure popularity impartially. Subsequently, we separate selected tweets by the types of source messages in two datasets: one is to quote an external link attached at the end of the text, which we used for the QuoteLink dataset; the other contains tweets that quote other users’ tweets corresponding to the QuoteTweet dataset. After that, we gathered the content of these tweets with source messages and measured the number of likes, replies, and retweets to reflect popularity. Finally, we retained the tweet text in English and removed irrelevant fields, such as images and videos. More data collection details are described in Appendix A.
Tweet Pair Construction. To train models with the popularity of quote tweets, we construct popular-unpopular quote tweet pairs labeled Tweet A and Tweet B to train models with the popularity of quote tweets. We implemented four rules to construct such pairs: 1) Tweets A and B must be from the same author and quote the same source message. 2) Suggested by Tan et al. (2014), Tweet A must have at least 10 more likes, replies, or retweets than Tweet B. 3) The posting time interval between Tweet A and Tweet B must be less than 12 hours. 4) To ensure that Tweet A and B have sufficient distinctiveness for learning popularity, we used SimCSE (Gao et al., 2021) to measure the semantic similarity of the tweet pair and removed pairs whose similarity was above the median (0.53 in our datasets). For model training and testing, we randomly split each of the datasets into training (60%), validation (20%), and test (20%) sets.
Data Analysis. Table 2 shows the statistics of two datasets. We observe that in the QuoteLink dataset, the average length of tweets is generally longer than in the QuoteTweet dataset. It indicates that users may add more words and detailed information when quoting external links. For the popularity gap, popular quote tweets in both datasets have significantly higher likes, replies, and retweets than unpopular ones. It shows the datasets allow a meaningful comparison of popular-unpopular samples. Moreover, inspired by Maas et al. (2011), we are interested in analyzing how quote tweets’ popularity is related to opinions and utilize GPT4 for opinion assessment. The results show that popular quote tweets contain more opinions than unpopular ones. It highlights the possible benefits of leveraging responses that reflect public opinion for learning popularity (potential engagement).
In addition, Table 3 displays the wording differences between two datasets’ popular and unpopular quote tweets. We can observe that popular quote tweets usually exhibit more positive words, more indefinite articles, and higher readability. Popular quote tweets have a slightly higher imitation score, indicating that writing more faithfully to the source message might be more popular. In §6.4, we will discuss how models learn these wording features.

heading: 4 RePALM Framework
text: RePALM overview. To begin with, we describe our datasets as D = {si, tiu, tip}Ni=1; si stands for the source message, which could be either an external link or a tweet to quote. tiu and t i p form a pair of unpopular (negative) and popular (positive) tweets of the same quote si for the model to compare, and N is the pair number. In the following, we omit the index i for better illustration. RePALM aims to generate a popular quote tweet tp based on the source s. Its workflow is depicted in Figure 1 with four major components described as follows.

heading: 4.1 Auto-Response Generation and Selection
text: Considering the high relevance of popularity to readers’ senses, we incorporate the possible user responses into RePALM’s popularity learning process to provide readers’ views. However, when quote tweets are created, the public reactions have not yet formed, rendering the absence of actual user responses to refer to. To address this issue, we simulate potential public reactions with an LLM to help RePALM generate popular quote tweets.
Concretely, we first prompt the LLM and employ top-p sampling (Holtzman et al., 2020) to sample varying responses (to form the readers’ view from diverse angles). Then, we compute their semantic
similarity to measure the responses’ consistency to the source message. Lastly, we rank the responses based on similarity to select the most relevant ones as the auto-response for the remaining learning process. The prompt is shown in Appendix D.

heading: 4.2 Response-Aware Quote Tweet Sampling
text: After obtaining the auto-responses, we incorporate them into the quote tweet generation process with the following steps. First, we prompt the LLM to generate quote tweets by augmenting the generated response with the source message. The prompt we adopted is: "Given the news [source] and potential public reaction [human response], create a quote tweet that highlights the main point of the news while capturing the public’s response." Then, we use top-p to sample multiple quote tweets. The purpose is to diversify quote tweets to create a more comprehensive range of samples to train the reward model better (see §4.3) and improve generalization.

heading: 4.3 Dual-Reward Modeling
text: The next step is to align LLMs (with preliminary language generation capabilities) to the popularity factor. Inspired by RLHF (Ouyang et al., 2022a), we exploit the PPO framework and propose dualreward modeling for popularity alignment. The dual-reward model consists of popularity reward modeling and consistency reward modeling.
Popularity reward modeling primarily assesses how likely social media users will engage with the generated tweet. It is trained on our datasets of comparisons between quote tweets of different popularity for the same source message. Specifically, in the training phase, it takes the source message and two quote tweets as input, i.e., the popular and unpopular ones, and outputs the reward scalar for each quote tweet. We use a cross-entropy loss to optimize the popularity reward model, comparing popularity to labels. The reward difference indicates that one quote tweet will be more popular than the other. The loss function is as follows:
LpopRM (θ) = −E(s,tu,tp)∼D [log(σ(rpopθ (s, tu)− rpopθ (s, tp)))] (1)
where θ is the training parameters of the popular reward model. rpopθ (s, t) is the scalar output of the reward model for source s and quote tweet t.
Consistency reward modeling examines the consistency of the generated quote tweet to the potential response. Our intuition is that quote tweets reflecting the readers’ viewpoints are more likely to
be popular. To achieve this, we measure the semantic similarity between the auto-response and the quote tweet with unsupervised SimCSE as the auxiliary reward. The corresponding loss is rcons(s, t).
The overall reward of RePALM, denoted as r(s, t), is hence the sum of the two rewards:
r(s, t) = rpopθ (s, t) + r cons(resp, t) (2)

heading: 4.4 Training Data Sampling and Learning
text: In PPO-based popularity alignment, the training data quality is crucial, yet social media data can inevitably be noisy. Thus, inspired by Dong et al. (2023), we sample the data to shortlist those with higher model confidence (showing higher reward) for fine-tuning. Specifically, we rank the collected pairs of reward-source-tweet (r, s, t) and select the top k percent of samples with the highest rewards as our sampled training datasets DRL. After that, we adopt the PPO training function defined as:
LRL = −E(r,s,t)∼DRLr(s, t) (3)

heading: 5 Experimental Setup

heading: 5.1 Model Settings
text: We will introduce our RePALM model parameters in four parts: 1) Auto-response generation. We adopted LLaMA2 (Touvron et al., 2023) (specifically LLaMA2-chat-7b) across all experiments to generate auto-responses. This model is solely for this purpose (without involvement in the quote tweet generation). To sample diverse responses, we set the top-p to 0.7 and the temperature to 0.95. For each source message, we sample 5 responses and rank 1 by similarity to the source; 2) Quote tweet generation. Another LLaMA2 is employed for generating quote tweets. To sample diverse quote tweets for RL, we maintain the same settings as 1), i.e., top-p at 0.7, the temperature at 0.95, and set the sampling number m to 5. The maximum token generation length is set to 150. 3) Popularity reward modeling. We used a smaller-scale (114M) reward model, GPT-2 (Radford et al., 2019) with a learning rate of 2e−4, a batch size of 16, and a total of 5 training epochs; 4) PPO training process. For PPO, we set the learning rate to 2e−5, batch size to 4, and training epochs to 3. We set k to 60, i.e., select samples with rewards in the top 60% for training. LoRA (Hu et al., 2022) was used to optimize the quote tweet generation model efficiently.
For training and test, we examine the overall popularity with the sum of likes, replies, and retweets.
for all comparison models (measured by paired t-test with p-value < 0.05).

heading: 5.2 Baselines and Comparison
text: Recall in §2; we related our task to summarization, headline generation models, and open-source LLM. Our baselines were chosen accordingly. For summarization models, we utilized SOTA summarizers, 1) PEGASUS-X (Phang et al., 2022) and 2) BART-Summary (Lewis et al., 2020). Additionally, we used T5 (Chung et al., 2022) to generate headlines, denoted as 3) T5-HeadLine. For open-source LLMs, we included 4) ChatGLM3-6B (Du et al., 2022) and 5) LLaMA2 (Touvron et al., 2023). For comprehensiveness concerns, the comparison also involved our response generation module, 6) LLaMA2-Response and the fine-tuned the LLaMA2 on our datasets, 7) LLaMA2-FT. The details for baseline models are in Appendix E.

heading: 5.3 Evaluation Metrics
text: For Automatic Evaluation, we compare model outputs with popular quote tweets (as references) and evaluate the similarity with ROUGE (Lin, 2004), BLEU (Papineni et al., 2002), NIST (Lin and Hovy, 2003) and BertScore (Zhang et al., 2020b).
For Human Evaluations, we randomly sampled 100 source messages from each dataset, along with quote tweets generated by different models. We then invited 5 human raters to conduct pair-wise comparisons to select the preference between the different quote tweets considering two dimensions: consistency of a generated quote tweet to the source message, and popularity of the tweet that has the potential to engage the public. This way, we enable easier human ratings to avoid biases. The specific questions for human raters are in Appendix F.
For LLM Evaluations, we used GPT-4 to rate the
generated quote tweets on a 5-point Likert scale based on opinion and popularity. Here, opinion measures the expression of a novel viewpoint compared to the source message, considering its crucial roles in popularity (see Table 2). These two criteria are relatively subjective (unlike consistency in human evaluation), and LLM evaluation focuses on them for a more extensive and fair comparison. LLM evaluation concerns all test data in both datasets with detailed prompts in Appendix B.

heading: 6 Experimental Results

heading: 6.1 Automatic Evaluation
text: Main Result. Table 4 (top) shows the main comparison result. We draw the following observations.
(1) Generating popular tweets to quote a user’s tweet is more challenging than quoting an external link, possibly because user tweets are shorter and lack sufficient context (as shown in Table 2); our RePALM can enrich context via response augmentation and shows superiority. (2) Applying summarization or headline generation models yields subpar performance. It suggests that simply echoing key points from the source message without providing new insights might not be enough to ensure popularity. Meanwhile, the results of LLaMA2Response are unsatisfactory, indicating that a popular quote tweet entails more than just a random response. (3) Zero-shot ChatGLM3 and LLaMA2 show promising results, indicating the potential of LLMs to serve as the backbone for our task. Meanwhile, LLaMA2-FT performed worse than zeroshot LLaMA2, suggesting the benefits of comparing popular and unpopular samples in learning popularity, a relative concept. (4) Our RePALM, built
on LLMs, yielded significantly better results than baselines, showing the effectiveness of response augmentation and RL-based popularity alignment.
Ablation Study. To investigate the effects of its components further, we conducted an ablation study with response augmentation, dual-reward modeling, and reward sampling. As seen in Table 4 (bottom), all components, in general, contribute positively to the model’s performance. Notably, the model’s performance declines the most when responses are reduced, indicating the crucial role of response augmentation in popularity learning.

heading: 6.2 Human and LLM Evaluation
text: To further examine whether the output is helpful to humans, we conduct manual pair-wise evaluations to assess consistency and popularity. RePALM is compared to its backbone, LLaMA2 (also the best baseline). Besides, we experiment with the ablation (-w/o response) to examine the effects of responses. The results are shown in Table 5. RePALM’s output is preferred over 1.65 times to the comparison models, indicating the effectiveness of response augmentation and RL-based popularity alignment.
We next present the results of the LLM evaluation in Table 6. RePALM outperforms all comparison models in both criteria with the performance
gain especially large in popularity. It is possibly because the augmented responses can helpfully incorporate opinions in the output and further increase the potential to draw public engagements.

heading: 6.3 Quantitative Analysis
text: We have shown the overall superiority of RePALM. Here, we examine its results in varying scenarios.
Varying Response Length and Number. While augmented responses shows overall benefits, we quantify their effects here. The first analysis concerns the auto-response length. As shown in Figure 2(a), the score first increases to peak at length 100, then decreases with larger length. It is because augmenting too-short responses offers limited help; conversely, the too-long responses may provide redundancy information and adverse effects.
We then analyze the impact of response numbers on RePALM’s performance. Figure 2(b) RePALM the model performs best with only one response. As the number of responses increases, the performance substantially declines. It is possibly because in the current augmentation design, introducing numerous responses might confuse the model, highlighting the usefulness of response selection.
Impact of Source Message Length. We next analyze the impact of source message length. Figure 2(c) shows the results on QuoteLink; a similar trend is observed in QuoteTweet. We observe that when the source messages are very short (0-50 tokens), the augmented auto-responses could help better due to their provision of richer contexts given sparse input. With longer source massages, RePALM also maintains better results in consistency.
Impact of Reward Sample Ratio. Recall that in §4.4, we selected the top k percent data with the highest reward for training. We hence analyzed the impact of different sample ratios k on RePALM’s results. Figure 2(d) shows that the optimal ratios for QuoteLink and QuoteTweet are 60% and 80%, respectively. It is also evident that under all sample ratios, RePALM’s performance surpasses that of LLaMA2. When the sample ratio is 100% (i.e., all
samples participate in PPO training), the model’s performance decreases. It indicates that data sampling is helpful in increasing training effectiveness.
Performance on Varying Popularity Measures. The discussions above centered on overall popularity (the sum of like, reply, and retweet numbers). Here we probe into how RePALM performs on each measure. Table 7 shows the results. The three measures show similar learning difficulties and RePALM consistently outperforms LLaMA2.

heading: 6.4 In-depth Analysis of Wording
text: Recall that in Table 3, we present the differences in wording between popular and unpopular quote tweets in the dataset. In Table 9, we report the performance of different models on these metrics. Similar to popular quote tweets, the quote tweets generated by RePALM outperform in terms of the usage of emotional words, generality, and readability. Notably, the imitation metric dropped from 24.71 to
7.33 compared to LLaMA2, approaching the level of popular quote tweets. It indicates that RePALM avoids copying the source messages while staying faithful to the original text. In particular, RePLAM performs better than its ablation without augmented responses across all metrics. It implies that augmented responses help RePALM generate original contents, helpfully improving popularity.

heading: 6.5 Case Study
text: Finally, a case study in Table 9 interprets why RePALM is effective. The output of RePALM is more detailed and include a richer opinions (highlighted by the colored text), which tends to increase the likelihoods of public engagements. It is because the auto-response contains viewpoints, e.g., "have the potential to revolutionize" and "pick up and learn from it." By response augmentation in popularity alignment, RePALM captures and reflects them in the generation, resulting in better outputs.

heading: 7 Conclusion
text: We have presented the first study on popular quote tweet generation with two extensive datasets. We
have proposed a novel model RePALM to exploit augmented auto-responses to better align language generation with popularity. The experiments show RePALM outperforms advanced LLMs in our task.
Ethics Statement
In our paper, we create a large Twitter dataset for studying popular quote tweets. We carefully followed Twitter’s API guidelines to collect only public tweets and users. The data, used solely for academic research, has been anonymized to protect user privacy, including removing authors’ names and replacing specific tags like @mentions and URLs. Adhering to Twitter’s redistribution policy, we will only share this anonymized data and require researchers to agree to use it only for academic purposes, ensuring compliance with ethical standards and Twitter’s data policies.
Limitations
We list the limitations of our paper in three aspects: 1) Untrained auto-response, 2) lack of author perspective, and 3) generalization of the method.
Untrained auto-response. We understand that people often react to specific details or key information in tweets. Our auto-response generation method directly utilizes the pre-trained language model LLaMA2 without additional training. Consequently, the generated responses tend to be general, lacking in-depth understanding, and targeted responses to specific topics or details. At times, such responses fail to provide a genuine human reaction.
Lack of author perspective. In generating quote tweets, we considered the reader’s perspective by introducing human responses. However, we overlooked the writer’s perspective, such as the personal linguistic habits of users when tweeting. As mentioned in Tan et al. (2014), there is a strong connection between the popularity of a user’s tweets and their personal wording.
Generalization of the method. Our RePALM approach has been validated as effective in quote tweet generation. In future work, we aim to generalize this approach to different tasks on social media. Because we know that social media texts are short, and many tasks are related to popularity. These are precisely the two directions that our method can address.
In our study, we acknowledge the potential for our model to be misused, particularly because its ability to generate highly engaging content can lead to undesirable consequences. These include a higher proportion of politically divisive content or misinformation that is reworded to sound credible and socially acceptable. While we hope that our research can inspire technological advancements, we strongly discourage misuse or application to sensitive topics. We intend to promote positive and responsible use of this technology, ensuring it benefits society rather than causing harm.

heading: Acknowledgements
text: This work is supported by the NSFC Young Scientists Fund (Project No. 62006203), a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (Project No. PolyU/25200821), the Innovation and Technology Fund (Project No. PRP/047/22FX), and PolyU Internal Fund from RC-DSAI (Project No. 1-CE1E).

heading: A Dataset Construction Detail
text: First, we downloaded the general Twitter Stream grabbed by the Archive Team 1, containing 400M of Tweet data streamed from 02/2016 to 10/2018 on Twitter. Then, we filter out tweets posted by authors with fewer than 10,000 followers and only keep English tweets that do not contain videos or images. Following that, we have 122,269 users and 259,043 pairs of tweets that report the same source message, which is used to construct positivenegative quote tweet pairs. Finally, after applying our four rules to filter tweet pairs, we obtain the final dataset as shown in Table 2.

heading: B Prompts for LLM evaluation
text: Recently, using large models as a means of evaluation has become a trend (Zheng et al., 2023; Wang et al., 2023; Chan et al., 2023), achieving higher accuracy than humans in many tasks. Therefore, we utilize GPT-4 (Ouyang et al., 2022b) to rate the opinion and popularity of a quote tweet on a 5-point Likert scale. The prompt for assessing opinion is shown in Figure 7, and the prompt for assessing popularity is shown in Figure 8.

heading: C Wording
text: In Tables 3 and 8, following Tan et al. (2014), we analyzed the wording differences in the first 100 tokens of various quote tweets, evaluating from four perspectives:, with specific assessment methods outlined as follows: 1) We measure the Emotion by the number of positive and negative words (measured by Connotation Lexicon Feng et al. (2013)). 2) We use the number of indefinite articles (a, an) and definite articles (the) to assess Generality. 3) For Readability, we use Flesch reading ease (Flesch, 1948) and Flesch-Kincaid grade level (Kincaid et al., 1975). 4) For the evaluation of Imitation, we use the number of shared unigrams and bigrams between the quote tweet and the source message.
1https://archive.org/details/twitterstream
In Table 3, for better representation, we selected partial indicators to represent these four perspectives. Positive words serve as the evaluation basis for Emotion, indefinite articles for Generality, Flesch reading ease for Readability, and unigrams for Imitation. The complete result is presented in Table 10.

heading: D Prompt for Auto-response Sampling
text: The auto-response sampling and selection process can be formulated as follows:
Rsampled = LLM(s)
resp =MaxSim(Rsampled, s) (4)
where the SimCSE-measured cosine similarity is used to calculate the semantic similarity, which is the same model used in consistency reward modeling. MaxSim function finds the response in Rsampled that is most similar to s. Figure D shows the prompt for response generation.

heading: E Prompts for Baseline Models
text: We introduced various language models as the baselines and prompted them to generate quote tweets
by creating summaries and headlines. In this section, we present the settings for different baselines. For the PEGASUS-X and Bart-Summary models, we directly downloaded models that were fine-tuned, requiring no prompt as the output is the summary of the input. For the T5-Headline, the prompt used is shown in Figure 4. For models that directly generate quote tweets, i.e., T5, LLaMA2, and ChatGLM3, the prompts are shown in Figure 5. For the LLaMA2-Response model, the prompt used is displayed in Figure 3.

heading: F Questions for Human Evaluation
text: For human evaluation, we conduct a pair-wise comparison. Human raters are asked to select their preference for different quote tweets in the aspect of consistency and popularity. As Shown in Figure 6, the models are anonymous to the human raters.

heading: G Supplementary Experiment Results.
text: Table 11 presents the results of LLaMA2 using different training methods. These include LLaMA2FT (Response Augmented), which involves finetuning LLaMA2 with added responses; LLaMA2PPO, which fine-tunes LLaMA2 using the PPO algorithm; and LLaMA2-PPO (Response Augmented), which trains LLaMA2 with added responses using the PPO algorithm.
Table 12 presents the human evaluation results between RePALM and ChatGLM3.

heading: Future Work
text: In future work, we aim to generalize this approach to different tasks on social media. Because we know that social media texts are short, and many tasks are related to popularity. These are precisely the two directions that our method can address. In our study, we acknowledge the potential for our model to be misused, particularly because its ability to generate highly engaging content can lead to undesirable consequences. These include a higher proportion of politically divisive content or misinformation that is reworded to sound credible and socially acceptable. While we hope that our research can inspire technological advancements, we strongly discourage misuse or application to sensitive topics. We intend to promote positive and responsible use of this technology, ensuring it benefits society rather than causing harm