R e PALM : Popular Quote Tweet Generation via Auto-Response Augmentation (2024)

Abstract: A quote tweet enables users to share others’ content while adding their own commentary. In order to enhance public engagement through quote tweets, we investigate the task of generating popular quote tweets. This task aims to produce quote tweets that garner higher popularity, as indicated by increased likes, replies, and retweets. Despite the impressive language generation capabilities of large language models (LLMs), there has been limited research on how LLMs can effectively learn the popularity of text to better engage the public. Therefore, we introduce a novel approach called Response-augmented Popularity-Aligned Language Model (RePALM), which aligns language generation with popularity by leveraging insights from augmented auto-responses provided by readers. We utilize the Proximal Policy Optimization framework with a dual-reward mechanism to jointly optimize for the popularity of the quote tweet and its consistency with the auto-responses. In our experiments, we collected two datasets consisting of quote tweets containing external links and those referencing others’ tweets. Extensive results demonstrate the superiority of RePALM over advanced language models that do not incorporate response augmentation.

## introduction a quote tweet is a tweet that shares a source message, e.g., an external link or another user's tweet, by quoting them and adding accompanying comments. its purpose is to enhance the visibility of the source message, which is beneficial to various applications, such as media broadcasts, advertisements, and social media engagement @xcite @xcite . a popular quote tweet can prompt public readers to engage in discussions actively. it essentially helps broaden the dissemination of the source message, inciting a more dynamic discourse and exchange of viewpoints * *corresponding author source message: chatgpt-a silver bullet for your customer support org? language models like chatgpt can write blog posts, hold conversations, and even pass the bar. a popular quote tweet (manually written): will chatgpt replace customer support teams? at @users, we've already deployed language models like chatgpt to help support orgs like ... at scale. learn what this means for you and how your company can stay ahead. llama2-chat: pondering the future of #customersupport: will #chatgpt be the silver bullet for orgs? #ai #languagemodel chatgpt: revolutionizing customer support with chat-gpt! discover how language models like chatgpt are not just conversing and blogging, but also acing legal tests. is this the future of customer service? #chatgpt #customerservi-ceinnovation #airevolution repalm: just set up my chatgpt and i'm blown away by its capabilities! just learn and try it on your customer support team. will it replace human agents? maybe not, but it's definitely a game-changer for customer service. #chatgpt nevertheless, many users are not good at writing popular quote tweets. to help them better engage the public for meaningful interactions, we present a novel task of popular quote tweet generation to extensively study how nlp models can learn to generate a popular quote tweet given a source message of an external link or other users' tweets. despite the recent advances of llms in language generation @xcite , the mainstream research focuses on the writing itself. however, limited work concerns the public readers' reactions to the text. for this reason, existing models cannot effectively understand the text's popularity, which reflects its potential to draw public engagement. to illustrate this point, viewing llms' limitation in popularity learning, we propose a novel response-augmented popularity-aligned language model (repalm). repalm learns to generate popular quote tweets by employing llms to predict possible reader responses, which work as a mirror to reflect public reactions for potential engagement measurements. augmented by these (auto-)responses, repalm is trained to align the quote tweet writing to popularity measure via reinforcement learning (rl). concretely, we first gather multiple llmgenerated auto-responses and select those that best match the source message with a consistency matching method. then, we feed a source message with its selected responses into repalm to generate multiple quote tweets. next, we optimize repalm's training process with the proximal policy optimization (ppo) framework @xcite with a novel dual-reward design. here, one reward is to predict popularity trained with a popular-unpopular quote tweet pairs; the other measures consistency between generated quote tweets and selected responses to align with public reactions. finally, we develop a reward ranking and sampling method to select high-reward training examples to improve training effectiveness. to the best of our knowledge, repalm is the first model to utilize llm-predicted auto-responses for popularity-aligned language generation. by learning from these potential responses, repalm can effectively generate popular quote tweets that help draw public engagement. for example, as illustrated in as a pilot study on popular quote tweet generation, we benchmark the task with two datasets: quotelink with tweets that quote external links and quotetweet with tweets that quote other users' tweets. there are 70k pairs of popular-unpopular samples; each pair quotes the same source and is from the same author, yet one is more popular. we further experiment with the two datasets. the main results first show that repalm outperforms all comparison models in both automatic measure and human evaluation. for example, repalm achieves 23.26 in rouge-1 metric, compared to 20.94 from chatglm3. besides, the ablation study implies the positive contributions of varying repalm modules. then, quantitative analyses show the effectiveness of repalm in varying scenarios. after that, we conduct a case study to interpret why repalm can perform better. next, we analyze the wording of quote tweets from four aspects to examine the essential features of popularity and how repalm effectively captures them. at last, a case study interprets repalm's superiority. in summary, our contributions are threefold: our work is further related to language generation in a broader scope. the emergence of llms has substantially advanced this field, especially in the zero-shot domain. taking recent advances in llms, many studies have examined how to align language models with human feedback. for example, chatgpt, a closely related model to instruct-gpt @xcite , is specifically trained to follow human instructions. llama2-chat @xcite is an open-source language model that demonstrates sota performance in conversational abilities. our repalm explores aligning the language model with popularity for quote tweet generation, which has not been studied previously. ## quote tweet datasets we collected large-scale data from twitter for our popular quote tweet generation task. based on the source message types, we separated the data into two distinct datasets: quotelink and quotetweet, where the former gathering quote tweets for external links and the latter for other users' tweets. ## repalm framework repalm overview. to begin with, we describe our datasets as @xmath0, t i u , t i p } n @xmath1; s i stands for the source message, which could be either an external link or a tweet to quote. t i u and t i p form a pair of unpopular (negative) and popular (positive) tweets of the same quote s i for the model to compare, and n is the pair number. in the following, we omit the index i for better illustration. repalm aims to generate a popular quote tweet t p based on the source s. its workflow is depicted in figure 1 with four major components described as follows. ## conclusion we have presented the first study on popular quote tweet generation with two extensive datasets. we have proposed a novel model repalm to exploit augmented auto-responses to better align language generation with popularity. the experiments show repalm outperforms advanced llms in our task.