Multiset Dual Summarization for Incongruent News Article Detection (2023)

Abstract: The prevalence of deceptive and incongruent news headlines has highlighted their substantial role in the propagation of fake news, exacerbating the spread of both misinformation and disinformation. Existing studies on incongruity detection primarily concentrate on estimating the similarity between the encoded representation of headlines and the encoded representation or summary representative vector of the news body. In the process of obtaining the encoded representation of the news body, researchers often consider either sequential encoding or hierarchical encoding of the news body or to acquire a summary representative vector of the news body, they explore techniques like summarization or dual summarization methods. Nevertheless, when it comes to detecting partially incongruent news, dual summarization-based methods tend to outperform hierarchical encoding-based methods. On the other hand, for datasets focused on detecting fake news, where the hierarchical structure within a news article plays a crucial role, hierarchical encoding-based methods tend to perform better than summarization-based methods. Recognizing this contradictory performance of hierarchical encoding-based and summarizationbased methods across datasets with different characteristics, we introduced a novel approach called Multiset Dual Summarization (MDS). MDS combines the strengths of both hierarchical encoding and dual summarization methods to leverage their respective advantages. We conducted experiments on datasets with diverse characteristics, and our findings demonstrate that our proposed model outperforms established state-of-the-art baseline models.

## introduction misleading and deceptive news headlines have emerged as a powerful force in the propagation of false information, resulting in a double-pronged effect that intensifies the proliferation of misinformation and disinformation @xcite . firstly, when integrated into current news stories gaining popularity, such headlines entice readers to interact with the material, thereby extending the influence of false information. secondly, they perpetuate a cycle of fake news by twisting facts and manipulating how readers perceive them, ruining the trustworthiness of reputable news outlets. these dual negative consequences significantly bolster the spread of misinformation and disinformation in the contemporary media @xcite effron and raj, 2020) 1, 2 . a news article is said to be incongruent if the news headline misrepresents its body through fabrication, manipulation, false connections 3 , or incorrect context 4 @xcite @xcite . as outlined in research studies @xcite @xcite , deceptive headlines play a significant role in vitality on social media and also influence readers' opinions @xcite . inconsistent news can adversely impact readers, leading to false beliefs and erroneous opinions 5, 6 (ecker et al., 2014, 2022; tsfati et al., 2020). once this misleading information spreads, it becomes challenging to rectify, as research @xcite has found that corrective measures may not have a substantial impact and can, in some instances, even reinforce individuals' misconceptions. consequently, the detection of deceptive and incongruent news articles @xcite @xcite @xcite has become a crucial research problem in combating the dissemination of misinformation in digital media. according to findings in the studies @xcite , there exist four primary characteristics associated with deceptive incongruent news articles: (i) the claim presented in the headline either has no relation to or directly contradicts the claim articulated in the body of the news article. (ii) news headlines and the body text pertain to the same topic or event, yet the content in the headline and the body are unrelated to each other. (iii) while both the headline and body describe a genuine event, the dates or entities mentioned in them have been manipulated. (iv) there are cases where certain paragraphs in the news body align with the headline while others do not, resulting in what is termed as "partially incongruent. in the early stages @xcite @xcite of research on detecting incongruent news, researchers employed basic n-gram features to assess the similarity between news headlines and the body of news articles. further, studies on incongruent news article detection can be grouped into three categories: similarity-based, summarization-based, and dual summarization-based. similarity-based studies can be further grouped into two categories: nonhierarchical encoding and hierarchical encodingbased methods. non-hierarchical encoding-based studies, as seen in works by @xcite and @xcite , aim to obtain sequential encoding of the news body and headline. they then estimate the similarity between the encoded representations of the headline and news body. on the other hand, hierarchical encoding-based methods, as explored in studies by, @xcite @xcite . define a news article as a hierarchical structure where the body is a collection of paragraphs, and each paragraph is a collection of sentences. they then obtain hierarchical encodings of the news and an encoding of the news headline. subsequently, these methods estimate the similarity between the encoded representation of the headline and the news body for incongruent news detection. while hierarchical encoding aids in obtaining a better encoded representation of the news body for incongruent news article detection, studies @xcite @xcite report that the aforementioned similarity-based methods often struggle to detect incongruent news in articles with larger paragraphs and sentences. to overcome the limitation of similarity-based methods, summarization-based studies @xcite @xcite ) first summarize 'the news body to generate a synthetic news headline and then estimate the similarity between the generated news headline and the actual headline to detect incongruent news. though summarization-based studies overcome the limitations of similarity-based studies, summarization-based studies fail to detect partially incongruent news articles @xcite . dual summarization-based study @xcite split the sentences of news body into two sets, positive and negative sets, based on contextual similarity of sentences and headlines. next, generate the summary of both positive and negative sets separately and estimate the similarity between the headline and summary of positive and negative for incongruent news detection. however, the dual summarization-based study @xcite splits the news articles into two sets based on the similarity between sentences and headline, which leads to a loss of hierarchical information present in a news article and also, the positive and negative sentences are biased toward headline which leads to loss of news body context information. our experimental results in in this section, we will briefly review prior research related to the detection of incongruent news articles. initial studies on incongruent news article detection mainly consider bag-of-words-based features, such as n-grams, term frequency-inverse document frequency (tf-idf), and topic modelling features @xcite @xcite . moreover, research on detecting incongruent news articles can be categorized into three main groups: similarity-based, summarization-based, and dual summarization-based methods. similarity-based studies can be further grouped into two categories: non-hierarchical encoding and hierarchical encoding-based methods. non-hierarchical encoding-based studies, as seen in works by @xcite and @xcite , aim to obtain sequential encoding of the news body and headline. they then estimate the similarity between the encoded representations of the headline and news body. on the other hand, hierarchical encoding-based methods, as explored in studies by, @xcite @xcite . define a news article as a hierarchical structure where the body is a collection of paragraphs, and each paragraph is a collection of sentences. they then obtain hierarchical encodings of the news and an encoding of the news headline. subsequently, these methods estimate the similarity between the encoded representation of the headline and the news body for incongruent news detection. while hierarchical encoding aids in obtaining a better encoded representation of the news body for incongruent news article detection, studies @xcite @xcite report that the aforementioned similarity-based methods often struggle to detect incongruent news in articles with larger paragraphs and sentences. to overcome the limitation of similarity-based methods, summarization-based studies @xcite @xcite ) first summarize 'the news body to generate a synthetic news headline and then estimate the similarity between the generated news headline and the actual headline to detect incongruent news. though summarization-based studies overcome the limitations of similarity-based studies, summarization-based studies fail to detect partially incongruent news articles @xcite . dual summarization-based study @xcite split the sentences of news body into two sets, positive and negative sets, based on contextual similarity of sentences and headlines. next, generate the summary of both positive and negative sets separately and estimate the similarity between the headline and summary of positive and negative for incongruent news detection. however, the dual summarization-based study @xcite splits the news articles into two sets based on the similarity between sentences and headline, which leads to a loss of hierarchical information present in a news article and also, the positive and negative sentences are biased toward headline which leads to loss of news body context information. motivated by the above limitations of dual summarizationbased methods, this study proposes we proposed multiset dual summarization (mds), which combines both hierarchical encoding and dual summarization to leverage the advantages of each. ## proposed method as discussed above, this study aims to combine hierarchical encoding of the news body with dual summarizations. our proposed model, mds, first obtains a hierarchical encoding of the news body, encodings for each sentence in the news body and headline. it then divides the sentences of the news body into four sets based on the similarity between the encoded representation of the headline and the encoded representation of the sentences in the news body, as well as the similarity between the hierarchical encoded representation of the news body and the encoded representation of the sentences in the news body. by following the above objective, our proposed model takes advantage of both a dual summarization approach and a hierarchical encoding-based approach for incongruent news article detection. figure 2 presents the block diagram of our proposed model, multiset dual summarization mds. given a pair of news body b and headline, h our proposed model mds first splits the news article into four sets, namely head positive h + , head negative h -, body positive set b + and body negative set b -. sentences of news body are placed into head positive h + and head negative h -based on similarity between the encoded representation of the sentence and the encoded representation of the headline. in contrast, sentences of news body are placed into body positive, b + and body negative b -sets based on the similarity sentence encoder sentence encoder sentence encoder sentence encoder paragraph encoder paragraph encoder body encoder centroid of body generator between the encoded representation of the sentence and an encoded representation of a full news body. the primary rationale for categorizing body sentences into head positive h + and head negative h -sets lies in the identification of incongruities within news articles. when a news article is partially incongruent, the sentences that align with the headline are placed in the head positive h + , while the sentences that deviate from the headline are classified in the negative set head negative h -. similarly, in the case of a fully congruent news article, the majority of the sentences in the body should belong to the head positive h + , with only a few sentences residing in the head negative h -. however, in the scenario where a news article is entirely incongruent, all the sentences in the body should contrast with the headline and, therefore, belong to the negative set head negative h -, with the exception of one or a few sentences that align with the headline and are placed in the head positive h + . similarly, in the case of congruent news articles, the sentences of the news body must be correlated with each other. accordingly, in the case of congruent news, most of the sentences will be in body positive set, b + , and only a few sentences will be body negative set b -. whereas in the case of a partially incongruent news article, one or more paragraphs of the news body will not correlate with other paragraphs of the news body. consequently, sentences which are highly similar to the encoded representation of the news body will be placed in body positive set, b + , and sentences which do not align with other paragraphs of news body, i.e. least similar to the encoded representation of news body will be placed in body negative set b -. ## similarity between headline and sentence of news body we apply bidirectional long short-term memory (bilstm) @xcite over the headline and sentences of news body to obtain encoded representations h and s i of headline h and sentence s i , respectively. subsequently, fol- fully connected layer-2 true fake sigmoid process text process text multihead attention concat no fully connected layer-1 process text process text multihead attention concat yes process text process text multihead attention concat no process text process text multihead attention concat yes centroid of body generator if x @xmath0, then the sentence s i is added to head positive set; otherwise, it is added to set head negative set h -. similarly, if v @xmath1, then the sentence s i is added to the body positive set; otherwise, it is added to set body negative set b -. next, obtain a summary representative vector p, n, c, and d and form a feature vector e and pass it to a fully connected neural network for incongruent news classification. h -. next, to split the sentences of news body into body positive set, b + and body negative set, b -, we first obtain an encoded representation of news body b applying hierarchical encoding over the news body. our hierarchical encoding of news body is similar to recursive and sequential deep hierarchical encoding (rashe) model as defined in the study (kumar et al., 2023) except treelstm is replaced by bidirectional lstm (bilstm) to encode sentences within paragraphs of news body. figure 1 presents the working of hierarchical encoder model used to encode news body. once we obtain an encoded representation b of news body by applying hierarchical encoding over news body b, next we estimate the similarity score v i between b and s i as defined below.