B angla B ait: Semi-Supervised Adversarial Approach for Clickbait Detection on B angla Clickbait Dataset (2023)

Abstract: Intentionally luring readers to click on a particular content by exploiting their curiosity defines a title as clickbait. Although several studies focused on detecting clickbait titles in English articles, low-resource language like Bangla has not been given adequate attention. To tackle clickbait titles in Bangla, we have constructed the first Bangla clickbait detection dataset containing 15,056 labeled news articles and 65,406 unlabelled news articles extracted from clickbait-dense news sites. Each article has been labeled by three expert linguists and includes an article’s title, body, and other metadata. By incorporating labeled and unlabelled data, we finetune a pre-trained Bangla transformer model in an adversarial fashion using Semi-Supervised Generative Adversarial Networks (SS-GANs). The proposed model acts as a good baseline for this dataset, outperforming traditional neural network models (LSTM, GRU, CNN) and linguistic feature-based models. We expect that this dataset and the detailed analysis and comparison of these clickbait detection models will provide a fundamental basis for future research into detecting clickbait titles in Bengali articles.

## introduction due to the widespread usage of the internet, the news industry has progressively evolved into an online news industry leading to the explosion of clickbait titles in recent years. as the concept of clickbait can be hazy to grasp, the classification of clickbait is a highly subjective endeavor. @xcite suggests that clickbait titles can be roughly categorized into eight types. there are an estimated 11.4 million internet users in bangladesh 3 who receive their daily news mostly from online news sites. however, no research has been conducted on tackling the increasing number of clickbait titles on these sites and other news websites. for english articles, @xcite built the first large-scale annotated clickbait corpus (webis clickbait corpus 2017) containing 338,517 articles. in the bangla language, the lack of an annotated clickbait-rich dataset is hindering the progress of bangla clickbait detection. we construct the first bangla clickbait corpus, which contains an article's title, content, and other metadata collected from various clickbait-rich websites upon which future researchers can build an effective bangla clickbait detection model. the effectiveness of semi-supervised generative adversarial networks (ss-gans; @xcite have been shown for text classification tasks in @xcite . from our experiments, it is evident that fine-tuning a bangla electra model in this setup improves clickbait detection performance outperforming all other model types. the main contributions of this paper can be summarized as follows: • we create an annotated dataset of 15,056 articles and an unannotated dataset of 65,406 bangla articles rich with clickbait titles. the dataset contains the title, body, domain, article category, publication date, and english translation of title and content. we plan to release both of these datasets upon acceptance of the paper. the origin of clickbait is rooted in tabloids which have been in journalism since the 1980's @xcite . generally, clickbait detection features can be obtained from 3 different origins: clickbait teaser phrase or post text, the attached article that the post text wants the user to click, and metadata for both @xcite . apart from the post text, which is used by most to identify clickbait, the works of @xcite and @xcite also considered the linked article, metadata and used handcrafted features, tf-idf similarity between headline and article content and gradient boosted decision trees (gbdt). @xcite suggested that clickbait detection should be a regression problem instead of a binary classification challenge, as the latter provides a way to measure how much clickbait is in the teaser message. they initiated the webis clickbait challenge 2017, which boosted research activity in clickbait detection giving rise to highly effective and flexible deep learning techniques. for clickbait challenge 2017, @xcite first used self-attentive rnn @xcite to select the important words in the title and created a bigru @xcite network to encode the contextual information. @xcite , on the other hand, incorporated article content into an lstm model @xcite for the clickbait challenge. rony et al. (2017) used continuous skip-gram model (mikolov et al., 2013) to generate the word embedding of clickbait titles. however, indurthi et al. (2020) first investigated the application of transformer regression models in clickbait detection and achieved the first position in the clickbait challenge. besides, hossain et al. (2020) created the first bengali newspaper dataset for bengali fake news detection containing an annotated dataset of ≈ 50k bangla news. to the best of our knowledge, the first attempt to detect clickbait in bangla was made by munna and hossen (2021). they created a dataset on videosharing platforms containing bangla and english video links and used numerical features to detect clickbait links. however, no research has been conducted to tackle clickbaits in written news mediums using the textual features of the article. we present the first clickbait detection dataset in bangla and also provide a comprehensive comparison of various models to detect them. ## human baseline five human annotators who are undergraduate and regular newspaper readers are given 200 news article titles from the test set to annotate. they achieve an inter-annotator agreement of fleiss' kappa @xcite ) score of 0.374, which is fair according to landis and @xcite . compared to our dataset annotators' score, this score is much lower. our annotation process includes investigating english titles first to better form a coherent perception of clickbait titles. by majority voting among the five annotators, we select the final labels and achieve an f1 score of 76.82% and an accuracy of 77.01% on the clickbait class, which serves as the human baseline for bangla clickbait detection shown in ## results and analysis figure 2 shows the roc curve (receiver operating characteristic curve) for all models where the gan-banglabert model achieves the highest auc (area under roc curve) score of 0.8925, which is higher than the banglabert. the high auc score of gan-banglabert suggests that it can distinguish between clickbait and non-clickbait titles more accurately than other models. to investigate whether training in a semisupervised approach improves banglabert's inner representations as stated by @xcite , we plot the average of the last layer hidden representations of gan-banglabert and banglabert using a t-sne projection (van der @xcite in figure 3b . gan-banglabert better separates the clickbait class from the non-clickbait than the banglabert model, proving that training a bert model in a semi-supervised adversarial manner can improve the learned representations of the model and thus improve performance. for creating the unlabelled dataset, we choose clickbait-dense websites from the web to ensure a higher abundance of clickbait titles. to investigate whether this helps performance, we create another unlabelled dataset of the same size from daily prothom alo archive foot_8 , which has a substantially lower clickbait ratio. our model achieves 72.38% f1 score on this second unlabelled set compared to 75.13% f1 score on the original unlabelled set, proving that a higher clickbait ratio in the unlabelled set improves performance on the clickbait class. logistic regression and random forest model on character tf-idf features heavily outperform neural network models like bilstm with attention network and cnn @xcite . these models can effectively identify certain keywords that are very significant in classifying clickbait titles. for instance, a top character feature returned by logistic regression is বলেলন (told), which is a common keyword found in many clickbait titles, e.g., বড় সু খবর িদেয় যা বলেলন দীিঘ (what dighi said about the great news). the poor performance of neural network models can be attributed to bangla pre-trained fasttext and word2vec embeddings, which are trained on the bangla wikipedia dump and are significantly smaller in size than english. training these embeddings on training data and then initializing the neural models with these embeddings may improve performance. all models perform poorly on bait & switch type titles as mentioned in ## conclusion we present the first clickbait detection dataset containing 15,056 labeled new articles and 65,406 unlabelled articles containing article title, content and metadata to enable researchers to use this dataset to build state-of-the-art clickbait detection models. by conducting a comprehensive study on various architectures, we provide a strong baseline for detecting clickbait in bangla articles. we show that training a pre-trained transformer model in a semisupervised approach by incorporating unlabeled data improves performance and inner representation. as simple statistical models perform strongly on clickbait titles, we aim to investigate how these features can be combined with word embeddings to pass into neural networks. we also plan to investigate how features from article content can be utilized to detect clickbait. we wish to publicly release the dataset and code to further progress into bangla clickbait detection.