Multimodal Clickbait Detection by De-confounding Biases Using Causal Representation Inference (2024)

Abstract: This paper focuses on detecting clickbait posts on the Web. These posts often use eye-catching disinformation in mixed modalities to mislead users to click for profit. That affects the user experience and thus would be blocked by content provider. To escape detection, malicious creators use tricks to add some irrelevant non-bait content into bait posts, dressing them up as legal to fool the detector. This content often has biased relations with non-bait labels, yet traditional detectors tend to make predictions based on simple co-occurrence rather than grasping inherent factors that lead to malicious behavior. This spurious bias would easily cause misjudgments. To address this problem, we propose a new debiased method based on causal inference. We first employ a set of features in multiple modalities to characterize the posts. Considering these features are often mixed up with unknown biases, we then disentangle three kinds of latent factors from them, including the invariant factor that indicates intrinsic bait intention; the causal factor which reflects deceptive patterns in a certain scenario, and non-causal noise. By eliminating the noise that causes bias, we can use invariant and causal factors to build a robust model with good generalization ability. Experiments on three popular datasets show the effectiveness of our approach.

## introduction with the rapid development of social media, people share views and advertise products by posting content on platforms @xcite like wechat, twitter, instagram, etc. to increase viewership and obtain more advertising revenue, unscrupulous creators misuse these platforms to publish deceptive, poor-quality posts. such posts are often described with a catchy thumbnail or a sensational headline. for example, the posts have a sexy thumbnail, with curiosity-inciting phrases like "you won't believe", and "x reasons why" in their headlines. that would bait readers to click on the linked articles. however, these articles would be unrelated to the thumbnails and headlines of the posts. moreover, they are often full of hoaxes, rumors, and fake news, which not only degrade users' experience but also affect the credibility of the platforms @xcite . thus, the detection of such clickbait posts has great commercial value for social media. due to the large volume of emerging posts, a manual review of the clickbait is infeasible. machine detection has become a hot topic @xcite . the detective sources can be summarized into two categories @xcite . the first is based on social behavior. since bait posts are required to spread in social networks to expand their influence, typical propagation characteristics can be observed. it can be detected based on social metadata like comments, the number of views, shares, likes, etc @xcite . that requires a rich collection of user feedback for judgment, but this feedback is often delayed or some users do not even share it. as a result, malicious posts can only be found after they have been widely spread, which is too late for online applications. another direction is to analyze the post contents. the bait posts often have certain linguistic characteristics in terms of deceptive words, syntax, subjectivity, writing style @xcite , and even punctuation (coste and @xcite . besides, there may be abnormal relations between their subparts in various modalities, such as a rumor article text with visual thumbnails of an irrelevant actor to attract clicks. early works design a set of rules to detect these features and relations, but the rules are hand-crafted and non-scalable. current mainstream methods turn to the neural model for improving scalability. they seek correlations between the post content and labels to make predictions. however, to escape detection, malicious creators would use some unrelated content to disguise a bait post as a valid one. as shown in fig. (1) , in the left post, some of the conspicuous bait thumbnail and text are replaced with unharmful content, making it look like a legal one on the right. such content may co-occur with non-bait label usually, creating spurious correlations, i.e., the unstable and confounding patterns in the feature space @xcite . that would easily cheat the model to fail to discover the hidden bait. respectively, some valid posts may be misjudged as bait simply because they contain content that co-occurs with a bait label. such biased correlations lead to high misjudgments. the bias is inadequate to differentiate by a naive encoder, which is designed for generic content understanding tasks such as text classification. the encoder focuses on general embedded representations rather than key factors that cause fraudulent behaviors @xcite . these behaviors are not static, their topics and types may change over time. the writing styles would also vary for different authors, but such disguise tricks commonly follow some styles and patterns in a particular scenario. on the other hand, the data-driven neural model relies on scarce labeled data heavily. to derive a robust model, we have to collect data on all scenarios for supervised training. such uniform data is either unavailable or costly to acquire. as a result, the statistics on limited samples are not significant, which is easy to train the model to remember some inessential spurious correlations. that would harm the model's robustness. a simple solution is to customize a representation that perceives posts' quality, but that would face tedious hand-crafted engineering. motivated by above observations, we propose a new debiased approach to detect clickbait posts. in detail, we first represent the given posts based on a set of multimodal features, including textual and visual features, linguistic and cross-modal features, as well as features of the creators' profiles. considering these mixed representations contain unknown biases, we resort to causal representation learning, which is good at eliminating non-causal spurious noise. the representations are disentangled into three key latent factors, including (1) invariant factor that indicates the inherent bait's intention and post quality; (2) causal factor in a specific scenario; and (3) non-causal noise factor. the noise like unrelated words and topics is defined under a specific scenario, and each is unique. the invariant factor is obtained by invariant risk minimization, scenarios are estimated from the data, and the causal factor is learned by contrastive learning. by removing the noise and using the remaining invariant and causal factors, we can build a robust clickbait detector. it can combat spurious bias and generalize well on newly formed bait subspecies. to facilitate training, we further develop a data augmentation technique to alleviate the labeled data scarcity problem. extensive experiments on three real-world datasets show the effectiveness of our approach. ## approach fig. (2) shows our framework with four steps. we first extract multimodal features from the posts. by causal inference, we then disentangle the invariant factors from them, and separate the remaining parts into the causal factor and non-causal factor. finally, we make the prediction based on the invariant and causal factors. next, we define some notations and elaborate on each component of our approach. ## evaluations we fully conducted experiments with qualitative and quantitative analyses to evaluate our approach. ## related work clickbait detection is a hot research topic that aims to predict misleading and sensational social posts. the predictive sources can be divided into two categories. the first one is based on social activity. malicious creators keep posting clickbait and ceaselessly evolve new subspecies to avoid detection. some researchers propose to examine this bait behavior based on the creators' account profiles @xcite and activity metadata @xcite . nevertheless, some valid creators would produce both high-quality and poor posts, which easily led to false positive predictions. on the other hand, clickbait posts often need to be widely spread on social networks to enhance their influence. these posts should have remarkable propagation characteristics, such as a unique spread path, a large number of shares but a short reading time due to dissatisfaction, etc. the propagationbased methods had been proposed @xcite , but they cannot tackle new posts due to lack of user feedback @xcite . thus, such methods often perform high accuracy, but low coverage. also, they have an alert delay, which would bring a lot of losses and could not support online applications. another direction is based on the post's content. the bait posts often have certain language characteristics, such as delusive words, fallacious phases, hot topics, raged emotions @xcite , linguistic stylometry @xcite , etc. to detect them, early works design rules based on various features, such as the semantic @xcite , or multimodal ones @xcite . however, these rules are hand-crafted and non-extensible @xcite . to tackle this issue, current research turns to data-driven neural networks. various network architectures have been proposed @xcite , such as rnn @xcite , cnn @xcite . more advanced techniques like feature attention @xcite and graph attention mechanisms @xcite have been developed. to learn representations from rich unlabeled data @xcite , some researchers explore the pre-trained language models (plms), like bert @xcite and roberta @xcite . they are fine-tuned to fit the classified tasks @xcite , so as to use their embedded semantic knowledge @xcite to facilitate detection @xcite . besides, some works point out that the clickbait was not only in the unimodal like the text @xcite but also in multimodal @xcite , such as giving an attractive image with an unrelated article. they propose to incorporate more features in multiple modalities @xcite . for model training, there are other studies on domain adaptation @xcite and data augmentation @xcite to solve the data shortage problem by knowledge transfer. differently, we found that traditional representations have spurious correlations. malicious creators would exploit this bug to yield a large number of new subspecies by rewriting the posts' content, such as adding some valid but unrelated terms or images, and concealing the bait content in a normal post, etc. in this mixed content environment, the existing model easily misjudges the bait posts due to spurious bias. we thus propose a debiased method to tackle this problem by causal inference. causality-inspired methods are a hot research topic in many tasks @xcite . to grasp the causality in the tasks, some works proposed the causal loss function @xcite , while others designed a task-related causal structure @xcite to guide decision-making @xcite . they pointed out the value of a good representation for the model's performance @xcite . some studies propose to capture the salient representations by feature engineering @xcite or noise filtering @xcite , but they ignore spurious correlations. in contrast, we propose to elicit key factors to eliminate this bias. ## conclusion this paper studied the task of detecting clickbait in multimodal social media posts. existing methods learned shallow features that introduced spurious correlations, rather than capturing inherent factors that caused clickbait. malicious creators used this spurious bias to form new bait subspecies by rewriting the posts with tricks, such as disguise with valid content, leading to misjudgment and poor robustness. to tackle this problem, we proposed a new debiased framework by causal representation inference. in detail, we first extracted multimodal features, including textual, visual, linguistic, crossmodal, and creator profile features. by causal inference with structural constraints, we then disentangle them into three latent factors, including the invariant factor that indicated inherent bait intentions, a causal factor for a certain scenario, and noise factor. based on invariant and causal factors, we can build a robust model. moreover, we propose a data augmentation technique to reduce training costs. experimental results on three popular datasets shown the effectiveness of our model.