M edia HG : Rethinking Eye-catchy Features in Social Media Headline Generation (2023)

Abstract: An attractive blog headline on social media platforms can immediately grab readers and trigger more clicks. However, a good headline shall not only contract the main content but also be eye-catchy with domain platform features, which are decided by the website’s users and objectives. With effective headlines, bloggers can obtain more site traffic and profits, while readers can have easier access to topics of interest. In this paper, we propose a disentanglement-based headline generation model: MediaHG (Social Media Headline Generation), which can balance the content and contextual features. Specifically, we first devise a sample module for various document views and generate the corresponding headline candidates. Then, we incorporate contrastive learning and auxiliary multi-task to choose the best domain-suitable headline, according to the disentangled budgets. Besides, our separated processing gains more flexible adaptation for other headline generation tasks with special domain features. Our model is built from the content and headlines of 70k hot posts collected from REDBook, a Chinese social media platform for daily sharing. Experimental results with language metrics ROUGE and human evaluation show the improvement in the headline generation task for the platform.

## introduction nowadays, in the midst of massive flow of information on large-scale social network sites (such as facebook and instagram), users always feel more difficult to quickly obtain the information they want. as a result, people tend to focus on more niche platforms, which are often made up of groups of users with common personal interests. the vertical platforms are not only used for easier information search but also strengthen the community of users with the same interests. users' attention is always limited to attractive headlines that can catch their eyes at first glimpse. as the headline condenses the main topic into a concise and appealing description, a good headline can trigger a high click rate. so generating better headlines is significant for media platforms to compete for attracting users' limited attention and deliver users better experiences. we conduct the research based on a vertical chinese social media platform redbook (figure 2 ) because it is widely praised by hundreds of millions of users and targeted by the same interest group. as more than 70% of users are female (an official reported data), the topics and tone of posts are feminine such as some typical words "babycare", "makeup" etc. the platform breaks down identity restrictions and allows people to share their colorful lives and experience, which are of reference value for users. we define the eye-catchy features of posts as the ability to attract users, which can be intuitively measured by the number of "likes", that is, the heat of posts on the platform. producers can obtain more traffic and profits while getting more likes from the platform, then they may be subscribed by more fans. the advertising revenue of bloggers is usually closely related to both the number and profile of their followers. so only with the help of good headlines, can the bloggers attract their target user group. by analyzing eye-catchy redbook headlines with more likes (more than 2k) on the platform, we find that both contents and style influence attractiveness. since the majority of users are women, topics of interest to women occupy a large part of the topics on the platform. accordingly, headlines with domain special topics such as "lipstick swatches" or "ootd" ("outfit of the day") are more appealing with more likes. combined with hot topics, the style of the headline also has a great impact on eye-catchy features. for example, when reporting the hot issue "lipsticks sharing", the headline "new lipsticks for winter so tender like a creamy almond peach!!" (as shown on the platform "冬季口红新品～奶茸茸的杏仁桃子好 温柔" in chinese) wins over 13,000 likes, while another headline with the same topic but a plain description "lipstick share, a new style for winter" only has 300 likes. we also find the eye-catchy style of the headlines on the platform is not that similar to the style of news media headlines, as it more closely resembles the tone of women talking and sharing with their friends. focusing on relevant works, we found that recent researches simply regard the headline generation task as a typical summarization task @xcite . they only focus on the content parallel to the given reference summary, ignoring the domain eye-catchy features. however, attractive headlinegeneration tasks have received less attention. a recent clickbait research @xcite leverages adversarial training and attractiveness scores module to guide the summarization process. another @xcite introduces a novel parametersharing scheme to disentangle the attractive style from the text. however, these previous works only concentrate on style and neglect the content importance, which also weighs in eye-catchy headline generation. disentanglement module is devised to divide the style and content into latent spaces, but a style encoder in generator training is not flexible enough @xcite . to address the headline generation issue, we pro-pose the mediahg model which disentangles the eye-catchy features as additional requirements in sequence-to-sequence training. the model is composed of a headline candidates generator and an eye-catchy headline selector. in our setting, the neural abstractive model is responsible for headline generation, capturing the main topic of the input document, while the selection module with constraint will encourage the adherence of generated headlines to domain eye-catchy features. instead of confounding eye-catchy features, we treat the content feature and style feature extraction respectively. in the generation period, we devise a random sample module with different parts of the text and generate candidate headlines responding to the content. during selection period, we leverage ranking-based contrastive learning @xcite @xcite and multi-task @xcite to select the best headlines among candidates. the selection is decided by the coordinating quality scores of stylecontent attractiveness. we will describe the specific quality metrics model in detail in the following part. therefore, candidates are assigned with probabilities according to their quality, which will further influence the generation model. in other words, the headline generation model not only generates output headlines autoregressively but also estimates the probability distribution over candidate headlines. our main contributions are listed below: we also give a new definition of domain eyecatchy headlines that is, those attractive combinations with topic and style suitable to the platform users. 2 related work headline generation text generation has undergone impressive progress in recent years @xcite ) @xcite ) @xcite ) @xcite ) @xcite , and headline generation occupies the dominance. most existing works merely focus on document summarization with extractive @xcite ) @xcite or abstractive @xcite ) @xcite models. several methods regard headline generation as a task based on length-controlled text summarization. controlling length in summaries has been addressed by encoding positional information @xcite , a length-aware attention mechanism @xcite , and a length constraint optimization @xcite . content guidance in gsum @xcite is used as the input for its sequence-to-sequence model, and shifts in guidance distribution would require further training. attractive headline generation is paid less attention by researchers. a sensation scorer @xcite is designed to judge whether a headline is attractive and guide the headline generation by reinforcement learning. also, a parameter-sharing scheme @xcite is introduced to further extract style from the text. the style-content duality is considered with vae (variational autoencoder) as a feature extractor @xcite and two disentangled space constraints in parallel tasks. differently, mediahg allows flexible shifts in various guidance without expensive retraining. disentanglement disentangling neural networks' latent space has been explored in the computer vision domain to factorize the features (such as ro-tation and color) of images @xcite (higgins et al., 2017) @xcite . compared to the computer vision field, nlp tasks mainly treat sentiment as a salient style and focus on invariant representation learning. it is used to control sentiment through training a discriminator @xcite . then, disentangled representation learning is further widely adopted in nonparallel text style transfer. for example, separate training with style-specific embeddings and style-specific embeddings are proposed @xcite . some work also focuses on disentangling syntax and semantic representations in text. vgvae @xcite trains the generative model with multiple losses that exploit aligned paraphrastic sentences and word-order information to get better syntax and semantics representations. we utilize the core principle of disentangling to separate different feature budgets. reranking candidates recent conditional generation work explores the idea of reranking candidates from different dimensions(wan et al., 2015)(mizumoto and matsumoto, 2016). different search methods have been used in neural language summarization models, such as greedy search in fac-torsum (fonseca et al., 2022) and beam search (vijayakumar et al., 2016) in simcls (liu and liu, 2021)according to a learned evaluation function. the perturb-and-select summarizer (oved and levy, 2021) performs random perturbations and uses similar ideas to generate candidates ranked according to a coherence model. unlike only intrinsic importance compared with the original document in simcls, content and contextual eye-catchy budgets are both considered in our work. ## approach in this section, we describe our approach in detail, as shown in figure 3 . inspired by fac-torsum @xcite , we treat the content importance model as sampling document views(intrinsic importance), and contextual features as additional budgets(extrinsic importance). we pre-train the headline generation model with our dataset redbook ( ## conclusion and future in this paper, we propose an eye-catchy headline generation model mediahg for vertical interest social media platforms. our research is the first one focusing on vertical interest websites. as people's interests flourish with the information gap broken down, more websites will be designed to appeal to the same interest groups. our design allows the features extractor approach to be used more flexibly with other websites' data. both automatic and human evaluation show our improvement in headline generation.