Clickbait Detection with Style-aware Title Modeling and Co-attention (2020)

Abstract: Clickbait is a form of web content designed to attract attention and entice users to click on specific hyperlinks. The detection of clickbaits is an important task for online platforms to improve the quality of web content and the satisfaction of users. Clickbait detection is typically formed as a binary classification task based on the title and body of a webpage, and existing methods are mainly based on the content of title and the relevance between title and body. However, these methods ignore the stylistic patterns of titles, which can provide important clues on identifying clickbaits. In addition, they do not consider the interactions between the contexts within title and body, which are very important for measuring their relevance for clickbait detection. In this paper, we propose a clickbait detection approach with style-aware title modeling and co-attention. Specifically, we use Transformers to learn content representations of title and body, and respectively compute two content-based clickbait scores for title and body based on their representations. In addition, we propose to use a character-level Transformer to learn a style-aware title representation by capturing the stylistic patterns of title, and we compute a title stylistic score based on this representation. Besides, we propose to use a co-attention network to model the relatedness between the contexts within title and body, and further enhance their representations by encoding the interaction information. We compute a title-body matching score based on the representations of title and body enhanced by their interactions. The final clickbait score is predicted by a weighted summation of the aforementioned four kinds of scores. Extensive experiments on two benchmark datasets show that our approach can effectively improve the performance of clickbait detection and consistently outperform many baseline methods.

## introduction clickbait is a type of web content that is designed to attract users' attention and further entice them to click hyperlinks to enter specific webpages, such as news articles, advertisements and videos @xcite . several illustrative examples of clickbaits are shown in fig. 1 . we can see that the title of the first clickbait is written in a sensationalized way by using words with strong emotions like "must", and the title of the second clickbait is misleading because it does not match the content of the body. clickbaits are commonly used by online publishers, because clickbaits can draw more attention to the online websites where they are displayed and improve the revenue by attracting more clicks on advertisements @xcite . however, clickbaits are deceptive to users because the main content of clickbaits is often uninformative, misleading, or even irrelevant to the title, which is extremely harmful for the reading satisfaction of users @xcite . thus, clickbait detection is an important task for online platforms to improve the quality of their web content and maintain their brand reputation by improving user experience @xcite . many methods formulate clickbait detection as a binary detection task, and they mainly focus on modeling the content of online articles and the relevance between title and body @xcite @xcite . for example, @xcite proposed to use a combination of bi-gru network and attention network to learn representations of tweets posted by users for clickbait detection. @xcite proposed a similarity-aware clickbait detection model, which learns title and body c c l 2 0 2 0 computational linguistics title 7 things you must know about exercise and weight loss covid-19 news in your area you won't believe how many beloved mom-and-pop restaurants are closing body the biggest challenge for an obese person is losing a few extra pounds. well, people sometimes let themselves eat what they like... download our app today and get what you want! consider joining this community as a helpful resource... the pandemic has caused a lot of businesses to fold, especially independent restaurants, cafes, and coffee shops. representations via an attentive bi-gru network, and measures the global and local similarities between these representations for clickbait prediction. however, in these methods the stylistic patterns of titles (e.g., capitalization) are not taken into consideration, which are useful clues for identifying clickbaits @xcite . in addition, they cannot model the interactions between the contexts in the title and body, which are important for measuring the title-body relevance for clickbait detection. our work is motivated by the following observations. first, the content of webpage title and body is important for clickbait detection. for example, in the title of the third webpage in fig. 1 , the contexts like "you won't believe" are important indications of clickbaits because they express strong emotions. in addition, the body of this webpage is short and uninformative, which also implies that this webpage is a clickbait. second, the stylistic patterns of title like the usage of numeric and capitalized characters can also provide useful clues for identifying clickbaits. for example, the title of the first webpage in fig. 1 starts with a number "7" and it uses an all-capital word "must" to attract attention, both of which are commonly used by clickbaits. therefore, modeling the stylistic patterns of title can help detect clickbaits more accurately. third, there is inherent relatedness between the contexts within the title and body of the same webpage. for example, the words "weight loss" in the title of the first webpage in fig. 1 have close relatedness with the words "losing" and "pounds" in the body. modeling these interactions are helpful for measuring the relevance between title and body more accurately. in this paper, we propose a clickbait detection approach with style-aware title modeling and co-attention (satc), which can consider the interactions between contexts within title and body as well as the stylistic patterns of title. we first use transformers to learn representations of title and body based on their content, and then compute a title content score and a body content score based on the representations of title and body, respectively. in addition, we propose to use a character-level transformer to learn a style-aware title representation by capturing the stylistic patterns in the title, and we further compute a title stylistic score based on this representation. besides, we propose to use a co-attention network to model the interactions between the contexts within title and body, and further enhance their representations by encoding their interaction information. we compute a title-body matching score based on the relevance between the interaction-enhanced representations of title and body. the final unified clickbait score is a weighted summation of the four kinds of scores, which jointly considers the content of title and body, the stylistic information of title, and the relevance between title and body. extensive experiments on two benchmark datasets show that our approach can effectively enhance the performance of clickbait detection by incorporating the stylistic patterns of title and the title-body interactions. the main contributions of this paper are summarized as follows: ## methodology in this section, we introduce our proposed clickbait detection approach with style-aware title modeling and co-attention (satc). the framework of our proposed satc approach is illustrated in fig. 2 . it consists of four core modules, i.e., a content modeling module to learn representations of title and body from their content, a style modeling module to capture the stylistic patterns in the title, an interaction modeling module to capture the interactions between the contexts within title and body, and a clickbait prediction module to compute the clickbait score. the details of each module are introduced as follows. c c l 2 0 2 0 ... ... word transformer word transformer co-attention character transformer 12 things you never ... these are the 12 things... add add attention attention attention dense dense matcher dense 𝑦 𝑠 𝑦 𝑡 𝑦 𝑏 𝑦 𝑟 title stylistic score title content score body content score matching score 𝑦 attention attention clickbait score dense title body ... ... ... ... ... 𝒆 1 𝑐 𝒆 𝑀 𝑐 𝒆 1 𝑡 𝒆 𝑁 𝑡 𝒆 1 𝑏 𝒆 𝑃 𝑏 𝒉 1 𝑡 𝒉 𝑁 𝑡 𝒉 1 𝑏 𝒉 𝑃 𝑏 𝒓 1 𝑡 𝒓 𝑁 𝑡 𝒓 1 𝑏 𝒓 𝑃 𝑏 𝒆 𝑏 𝒓 𝑏 𝒓 𝑡 𝒆 𝑐 𝒆 𝑡 figure 2: the architecture of our satc approach for clickbait detection. ## conclusion in this paper, we propose a clickbait detection approach with style-aware title modeling and co-attention, which can capture the stylistic patterns in the title and the interactions between the contexts in the title and body. we use transformers to learn content representations of title and body, and respectively compute two content-based clickbait scores for them based on their representations. in addition, we propose to apply a character-level transformer to capture the stylistic patterns of title for learning style-aware title representations, which are further used to compute a title stylistic score. besides, we propose to use a co-attention network to model the relatedness between the contexts within title and body, and further combine their original representations with the interaction information to learn interaction-enhanced title and body representations, which are further used to compute a title-body matching score. the final clickbait score is predicted by a weighted summation of the four kinds of clickbait scores. extensive experiments on two benchmark datasets show that our approach can effectively improve the performance of clickbait detection by using style-aware title modeling to capture stylistic information and co-attention networks to model title-body interactions.