Generating User-Engaging News Headlines (2023)

Abstract: The potential choices for news article headlines are enormous, and finding the right balance between conveying the essential message and capturing the reader’s attention is key to effective headlining. However, presenting the same news headline to all readers is a suboptimal strategy, because it does not take into account the different preferences and interests of diverse readers, who may be confused about why a particular article has been recommended to them and do not see a clear connection between their interests and the recommended article. In this paper, we present a novel framework that addresses these challenges by incorporating user profiling to generate personalized headlines, and a combination of automated and human evaluation methods to determine user preference for personalized headlines. Our framework utilizes a learnable relevance function to assign personalized signature phrases to users based on their reading histories, which are then used to personalize headline generation. Through extensive evaluation, we demonstrate the effectiveness of our proposed framework in generating personalized headlines that meet the needs of a diverse audience. Our framework has the potential to improve the efficacy of news recommendations and facilitate creation of personalized content.

## introduction personalized news recommendation systems, such as google news and yahoo news, help users discover articles that align with their interests @xcite . however, these systems often present the same article headline to all users, making it difficult for them to understand the connection between their interests and the recommended article, potentially reducing the effectiveness of the recommendation system. to address this, we propose a new framework for generating personalized, engaging headlines that clearly show the connection between a user's reading history and a recommended article. our framework has the potential to improve the efficacy of personalized news recommendations, and recommendations for short videos, articles, recipes, etc. @xcite @xcite generating personalized headlines is a challenging task due to the constraints of conciseness and the need to capture the reader's attention. a personalized headline should (a) effectively convey the main message of the article and (b) provide a clear link to the user's reading history, using only about 10 words on average @xcite . there are two main challenges in this task. first, a headline that entices users to click, but only presents limited information and fails to convey the essential story, becomes clickbait rather than a useful headline @xcite . second, it is difficult to find large scale annotated datasets containing news articles, multiple personalized headlines, and associated user profiles. such a dataset would be useful in developing personalized headlines, but it is currently unattainable. the key to effective personalization is to develop a comprehensive framework that enables us to (a) understand users' interests based on their reading histories, (b) produce personalized headlines, and (c) evaluate the effectiveness of these headlines in terms of user preference. previous studies on headline generation have primarily focused on producing headlines that accurately summarize a given news article or its first sentence @xcite @xcite @xcite , but have not considered the potential benefits of personalization. in this study, we propose a pipeline that incorporates user profiling foot_0 and a comprehensive synthesis of our approach focuses on learning a relevance function that condenses a user's reading history into a collection of signature phrases. this method for user profiling is both efficient and adaptable, as the signature phrases can be easily updated as the user's interests evolve @xcite . these signature phrases are derived from news article based on the user's reading history through contrastive learning without the need for annotated data. for example, if the phrase upper east side frequently appears in the user's reading history, it could become a signature phrase for that user (figure 1 ). these signature phrases do not need to appear verbatim in the user's reading history and can indicate broader interests, e.g., if the phrases avengers and hulk appear in the user's reading history, it could indicate a love for marvel movies and marvel studios could be a signature phrase that reflects this interest. we build a synthetic dataset that trains the model to generate personalized headlines for a news article. using signature phrases, our model is able to create a connection between the recommended article and the user's interests, resulting in personalized headlines that are both engaging and anchored to the article to avoid clickbait. evaluating personalized news headlines presents unique challenges @xcite . it would be ideal to have human evaluators judge the effectiveness of system headlines. indeed, we have conducted a human evaluation in this study. however, this process is time-consuming and costly, making it impractical during the system development phase. thus, we propose a comprehensive synthesis of automated and human evaluation methods to assess headline relevance and user preference. by using signature phrases, we can synthesize user profiles of various types. we hypothesize that personalized headlines generated for these user profiles will be preferred by the same users over generic, nonpersonalized headlines according to recommenderdriven metrics @xcite . we also experiment with a variety of automatic metrics to assess headline quality in terms of informativeness, relevance to the source article, and content accuracy @xcite . in this paper, we make the following contributions: automatic headline generation has made significant progress in recent years @xcite @xcite @xcite , thanks in part to the development of large language models @xcite @xcite @xcite and the availability of benchmark news datasets such as gigaword, xsum, and newsroom @xcite @xcite . these datasets include a single headline for each news article, serving as the groundtruth for the models. in contrast to previous works, we aim to personalize headline generation to improve content recommendations, where a personalized headline should convey the main points of the article and capture the user's attention. personalization is a highly sought-after technique, and researchers have explored its use for tasks such as headline generation, dialog response generation and recipe creation @xcite @xcite @xcite . we anticipate that this technique to continue to have a significant impact. for example, when a recommender system distributes news articles or short videos, personalizing the headline can help users find a clear connection between their interests and the recommended article/video @xcite , thus improving their experience. evaluating personalized content is a largely under-explored area, partly due to the lack of ground truth for personalized content generation @xcite . without ground truth, it is challenging to apply commonly used text generation evaluation metrics such as rouge, bleu, bertscore, moverscore, bleurt, etc. @xcite @xcite @xcite . to leverage recent advances in data synthesis @xcite @xcite , we propose synthesizing user profiles of various types. we then evaluate system headlines against these profiles along multiple dimensions, including their alignment with user interests, relevance to the source article, and content accuracy. in the following, we provide details of our approach. ## our approach our goal is to generate a user-engaging headline that conveys the main idea of a given news article d for a specific user u. to achieve this, we have developed a three-step framework: (1) signature phrases identification. using a key-phrase generation module, we identify a set of candidate signature phrases from the set of candidate signature phrases, we select a subset z u d ⊆ z d that relates to user u's interests as the user signature phrases (section 3.2); ## corpora processing in this section, we describe the corpora processing step, including the creation of synthesized users and the generation of signature phrase based headlines. our data is sourced from two existing news corpora: newsroom @xcite and gigaword @xcite . the newsroom corpus contains 995,041 articleheadline pairs in its training set, 108,837 in its validation set, and 108,862 in its test set. the gigaword corpus contains 7,704,419 instances in its training set, 394,390 in its validation set, and 381,045 in its test set. for each corpus, we construct two datasets: a synthesized user dataset and a headline generation dataset. the first dataset is used for training the use signature phrase selection model (section 3.2) and evaluating the entire system, while the second dataset is used for training the signature-oriented headline generation model (section 3.3). further data statistics can be found in ## experiments we thoroughly evaluate our proposed system from different perspectives, including objective evaluation (section 5.2), subjective evaluation (section 5.3) and ablation studies (section 5.4), for personalized headline generation. ## conclusion we investigate the generation of personalized headlines tailored to various users' interests. we propose a topic-focused generation framework and methods for creating synthesized data to support the training of our framework without the need for human-annotated datasets. additionally, we explore evaluation methods that enable the automatic evaluation of the generated headlines from multiple perspectives. our experiments demonstrate the effectiveness of our proposed approaches.