Headline Token-based Discriminative Learning for Subheading Generation in News Article (2023)

Abstract: The news subheading summarizes an article’s contents in several sentences to support the headline limited to solely conveying the main contents. So, it is necessary to generate compelling news subheadings in consideration of the structural characteristics of the news. In this paper, we propose a subheading generation model using topical headline information. We introduce a discriminative learning method that utilizes the prediction result of masked headline tokens. Experiments show that the proposed model is effective and outperforms the comparative models on three news datasets written in two languages. We also show that our model performs robustly on a small dataset and various masking ratios. Qualitative analysis and human evaluations also show that the overall quality of generated subheadings improved over the comparative models.

## introduction the news headline summarizes the article to grab the attention and interest of the readers @xcite @xcite . however, the headline is written in a brief form of short sentences with topic-related phrases @xcite , making it hard for users to grasp the entire content of the news article from the headline alone. to tackle this problem, some news vendors provide a subheading, usually located right below the headline, to convey its main content within several sentences. this component can provide a core and informative summary of a news article that cannot be conveyed by the headline alone. mainly, subheadings are written by professional news writers with concise content that corresponds to the main body of the news. recently, @xcite released xlsum, a multilingual news summary dataset, referring to subheading as a summary of the article. therefore, generating subheadings can be considered an abstractive summarization problem that needs to cap-ture the topical knowledge from the body of the article. the main approach is to add auxiliary signals to make the model aware of topical knowledge. @xcite and @xcite add an external guidance signal by lexical similarity between input text and summary. @xcite extracts the context word sequences from the reference to reflect some important phrases from the article. although these external auxiliary sources provide diverse topical signals, they are cost-intensive to heuristically manipulate and have limitations in guiding the overall topical information of the article. other approaches incorporate contrastive learning into sequence-to-sequence (seq2seq) model, allowing the model to learn topical representation of the input text @xcite @xcite . they explicitly constructs positive or negative inputs to introduce contrastive loss as an augmentation of mle training. in this work, we propose a novel framework for generating compelling news subheadings by discriminating whether each token in the reconstructed headline is the same as the token in the original headline. unlike previous approaches that use heuristically extracted topical information or positive and negative pairs, we utilize headline that fundamentally implies the topic of the entire article. we make full use of this indispensable object as a guide signal through token-based discriminative learning. we conducted comparative experiments on three datasets written in english or korean to evaluate the performance of our model and verified our model through additional qualitative analysis with human evaluations. ## datasets we used one english news summarization dataset and two korean news summarization datasets. ## proposed method as shown in figure 1 , our proposed model consists of subheading generation (bottom) and token-based discriminative learning (top) parts. the loss occurring in each part is defined as l gen and l dis , and the model is trained by minimizing the following loss: ## conclusions in this paper, we propose a novel model for generating a subheading for news article. along with token-based discriminative learning, our model can effectively utilize topical information from a headline that is essential in articles and does not require additional manipulated information. experiments on three datasets written in two different languages show the effectiveness of the proposed model. also, qualitative results and human evaluation show that the overall quality of generated subheadings is improved compared to comparative models. we expect that our model will be extended in future research to an abstractive summarization task that include both a headline and a body text, such as legal texts or papers.