H onest B ait: Forward References for Attractive but Faithful Headline Generation (2023)

Abstract: Current methods for generating attractive headlines often learn directly from data, which bases attractiveness on the number of user clicks and views. Although clicks or views do reflect user interest, they can fail to reveal how much interest is raised by the writing style and how much is due to the event or topic itself. Also, such approaches can lead to harmful inventions by over-exaggerating the content, aggravating the spread of false information. In this work, we propose HonestBait, a novel framework for solving these issues from another aspect: generating headlines using forward references (FRs), a writing technique often used for clickbait. A self-verification process is included during training to avoid spurious inventions. We begin with a preliminary user study to understand how FRs affect user interest, after which we present PANCO, an innovative dataset containing pairs of fake news with verified news for attractive but faithful news headline generation. Auto matic metrics and human evaluations show that our framework yields more attractive results (+11.25% compared to human-written verified news headlines) while maintaining high veracity, which helps promote real information to fight against fake news.

## introduction fake news has become a medium by which to spread misinformation @xcite . one common way to fight against fake news is to release verified news. 2 however, as the goal of news verification is to correct misinformation, verified news headlines are often bland, making it difficult to gain the attention of users, which works against the need to alleviate the harmful impact of fake news. therefore, headlines for verified news articles should be rewritten to be more intriguing but still faithful, which is expected to pique reader interest in verified news. many studies have been conducted on generating attractive headlines @xcite , among which clickbait represents the style that generates the most reads or clicks. despite their success in attracting readers, there are several challenges in current models. first, clickbait datasets for training headline generators with sensational style transfer are commonly collected based on the amount of views or clicks, which assumes that headline popularity is always due to the writing style @xcite . however, user reading preferences could also be motivated by trending topics or major events. for instance, "flights cancelled as typhoon nears" was the most popular news on a day that a typhoon was coming. although such headlines get many views and clicks, the writing style itself is not interesting, and could end up as noise in the dataset. second, harmful "hallucinations" created by headlines exaggerated to be more sensational could distort the meaning of the original article. this is especially critical as we do not want our model itself to spread misinformation. however, as such sensational headline generation models often generate clickbait with more ambiguous words, it increases the difficulty of evaluating faithfulness by aligning title semantics with the news content. in this work, we propose making real news intriguing by learning what fake news is good at. we seek to learn what makes fake news eye-catching instead of simply mimicking the titles of fake news. quantity-wise, the many circulating fake news articles serve as learning materials by which we can learn to generate more attractive headlines; stylewise, fake news is deliberately written to attract attention. to learn such attractive writing styles, we adopt the forward-reference (fr) writing technique @xcite , which draws from psychology and journalism, and is frequently used to create attractive headlines. specifically, fr creates an information gap between readers and the news content with the headline, motivating the reader's curiosity @xcite to investigate the news content, and hence provoking the desire to click on the headline. one example is the headline "wanna be an enviable couple? 12 things a happy couple must do... it's that simple!", which drives readers to find out what those things are. here, to understand the relation between veracity, attractiveness, and fr types in news headlines, we conducted a preliminary user study to investigate the attractiveness of fake and real news, and analyzed the fr types used in headlines in terms of veracity. given these results and observations, we propose honestbait, a novel framework by which to generate attractive but faithful headlines. in this framework, we use fr to remove the need to learn directly from the click-based dataset. to ensure the faithfulness of the generated headlines, we design a lexical-bias-robust textual entailment component on the generated headline and its original content to confirm that the content infers the headline. in addition, we propose panco, an innovative dataset which consists of pairs of fake and verified news headlines, their content, and their fr types. we conduct experiments on panco and evaluate the results in terms of both automatic metrics and human evaluation. in sum, the contributions of our work are threefold: a similar concept is cataphora, in which information is forwarded as a teaser at the sentence level @xcite . @xcite investigate how editors rewrite headlines for digital platforms, and analyze the linguistic features of what makes for an attractive headline. zhang et al. (2018) address attractive headline generation as question headline generation (qhg), which assumes that interrogative sentences are more popular. although this modality is indeed a type of fr, we argue that the interrogative style may not be suitable for all kinds of headlines, especially verified news. hence in our work, we fully consider all kinds of fr which are commonly used and seen in social media and on digital platforms. sample headlines exhibiting fr techniques can be found in fig. 4 in the appendix. ## preliminary user study in this section, we investigate for a given topic which of the fake or real headlines users are more interested in, and how often forward references are found in interesting titles. accordingly, we seek to test the following two hypotheses: h1: fake news headlines motivate user reading interest more than real news headlines. h2: forward references are commonly seen/used in headlines which interest users. we conducted the user study on both chinese and english news to determine whether forward references were used across languages. for english headlines, we adopted fakenewsnet @xcite , which contains fake and real news headlines about gossip and political news from gossipcop and politifact. since the real and fake news in fak-enewsnet are not paired up, we performed topical clustering to alleviate topical bias. for chinese headlines, we directly leveraged news pairs labeled as disagreed in the wsdm fake news challenge dataset, foot_0 which contains one fake news headline and its corresponding verified news headline. we conducted the english user study using amazon mechanical turk @xcite . each pair was labeled by three turkers, whereas each chinese pair was annotated by five native speakers we recruited. to test h1, annotators chose which headline they wanted to read further, with four options: first headline, second headline, both, and none. news veracity was not revealed during the study. results show that both chinese and english readers prefer fake news headlines. for chinese headlines, 39.75% of fake titles were judged to be more interesting than the real ones, whereas only 23.60% of real titles won. for english headlines, the percentages are 34.57% and 30.33%, respectively. note that in english, we are comparing real news with fake news due to the scarcity of paired verified and fake news data, whereas in chinese, we are comparing verified news with fake news. this could be why the preference for real and fake news in english is closer than in chinese. even so, both chinese and english show with statistical significance (p-values far less than 0.05) that readers prefer fake headlines. we report the complete distribution including ties, as shown in fig. 1 . this result supports h1: fake news headlines motivate reading interest more than real news headlines. to test h2, we randomly sampled 1,000 preferred and rejected headlines, respectively, from the previous user study, and asked another set of three annotators to label the fr type. results show that 73.48% of chinese and 85.32% of english preferred headlines utilizing fr techniques (at least one fr included in the headline), whereas in rejected headlines, the ratio is 22.35% / 17.72%. this further supports h2: fr is commonly used in interesting headlines. in conclusion, we found that fake news headlines draw more reader interest, and the use of fr techniques is a key part of what makes headlines intriguing. ## methodology having motivated the use of fr, we propose hon-estbait, a novel framework which incorporates fr techniques and veracity verification. honestbait consists of two stages. in the first stage, we pretrain an fr predictor and an fr proposer ( § 4.1). both of them take verified news titles as input. the fr predictor is trained to predict which frs a verified headline contains; hence the gold label is the frs of the current input verified headline. the fr proposer, in turn, learns to predict which combination of frs the corresponding fake news exhibits; the gold label is the frs of the corresponding fake headline of the current input verified headline. the main concept in stage 1 is learning frs from fake news to provide the direction best suited to rewriting a monotonic verified headline into an interesting headline. when the fr predictor and fr proposer are ready, we proceed to stage 2 to generate attractive but faithful headlines. figure 2 depicts the overall architecture of the second stage. the input during stage 2 consists only of verified news headlines and their content for learning headline generation, where the developed fr predictor and fr proposer together provide rewards to the learning model. first, we use a sequence generator ( § 4.2) to generate headlines from the input verified news content, and utilize the fr proposer to predict which combination of fr types is best suited to rewriting the input verified news headline. during each decoding step, we use the fr predictor to predict which fr types the currently generated headline contains, and we align the prediction from the fr proposer and the fr predictor to transform the original boring verified headlines into exciting ones. this is achieved by computing the fr type reward ( § 4.3). after decoding, we make use of a faithfulness scorer ( § 4.4) and a sensationalism scorer ( § 4.5) to compute the faithfulness and sensationalism rewards by which to evaluate the generated headline; all three rewards are then combined to make the generated results attractive but faithful. during inference, given verified news headlines and their content, honestbait then generates attractive but faithful headlines using the above-mentioned components. below we describe each major component in detail. ## experiment in this section, we describe experiments conducted to evaluate honestbait. we first describe the experimental dataset and then describe the result of human evaluation, automatic metrics, a case study, and hyperparameter analyses to further demonstrate the superiority of the proposed model. ## conclusion we present honestbait, a novel framework for generating faithful but interesting headlines from a new aspect: forward references. moreover, we construct panco, a novel dataset that includes the title and content of pairs of fake and verified news, along with their forward reference types for further research. our user study shows that verified news headlines are relatively boring, and forward references are used in most headlines liked by readers. experimental results show that honestbait outperforms all baselines in both automatic and human evaluations, which demonstrates its effectiveness in generating attractive but faithful headlines. we expect honestbait to help rewrite monotonous real news headlines to increase their exposure rate to help combat fake news.