Matt B ai at S em E val-2023 Task 5: Clickbait spoiler classification via BERT (2023)

Abstract: The Clickbait Spoiling shared task aims at tackling two aspects of spoiling: classifying the spoiler type based on its length and generating the spoiler. This paper focuses on the task of classifying the spoiler type. Better classification of the spoiler type would eventually help in generating a better spoiler for the post. We use BERT-base (cased) to classify the clickbait posts. The model achieves a balanced accuracy of 0.63 as we give only the post content as the input to our model instead of the concatenation of the post title and post content to find out the differences that the post title might be bringing in.

## introduction clickbaits are becoming extremely popular nowadays, and people are falling for them way too easily. since it plays on people's innate curiosity and need for rapid satisfaction, clickbait may be an effective tactic for driving traffic to a website or increasing interaction on social media, hence resulting in revenue. the general conception about clickbaits is that it is often seen as a bad strategy, since it frequently misleads users and gives little value after they click through to the content. this can lead to dissatisfaction, disillusionment, and a lack of faith in the clickbait's source. however, not all clickbait posts are bad and misleading, some people genuinely use clickbaits to gain more engagement, they also have good content. the clickbait spoiling shared task @xcite aims at generating short texts that satisfy the curiosity induced by a clickbait post. spoilers are needed to briefly let the reader know what to expect when the article is clicked on or whether to click on it or not. the task has two subtasks, one in which we have to classify the clickbait spoiler in 3 classes namely "phrase", "passage", and "multi" which are short phrase spoilers, longer passage spoilers, and multiple non-consecutive pieces of text, respectively. the second task is to generate the spoiler itself, which is not discussed here. the main reference paper @xcite ) concluded that improving the spoiler type classification would also improve the generation of spoilers. the dataset contains 5000 english posts which have a manually extracted spoiler for each. in our approach, we have taken the bert-base model and given it the content of the clickbait posts as the input as opposed to the the concatenation of the title and content of the post as done in the original approach. we find that removing the post title has a negative overall impact on the scores. ## background even though clickbaits have been around here for a long time now and topics like clickbait identification are being popular every passing day, the task of spoiling them is fairly new. notable work in this field has been done by the organizers of shared task themselves in which they use multiple approaches to generate spoilers including question answering and passage retrival. @xcite they also found that non-transformer-based methods like naive bayes, svm, or logistic regression perform worse than transformer-based methods like deberta and roberta, which leads us to the conclusion that the task of classification requires much more knowledge and understanding of the language than what can be encoded in these systems. the paper releases its own dataset, a total of 5000 clickbait posts with spoilers. these posts were first collected from social networks where accounts like r/savedyouaclick and @huffpospoilers have already spoiled thousands of clickbait links and articles. each of the 5,000 posts in this dataset consisted of a unique id, the platform from which they were taken, the post's text, the manually extracted title and the main content divided by paragraphs of the linked document, the manually opti-mized spoiler, the character position of the spoiler in the main content, and the type of spoiler. ## system overview we approach the problem of classification of the clickbait posts by using the post's content as the only input. we use bert-base model which is a pretrained model on english language using a masked language modeling (mlm) objective. @xcite we fine-tune this model for our task of classification of the spoiler type. we have used the case-sensitive version of this model, which differentiates between lowercase and uppercase characters. ## experimental setup the data of 5000 posts was split into 3200 for training, 800 for validation and the remaining 1000 for testing. the code to replicate the setup can be found here foot_0 .the post text was then extracted and was given as input to the bert layer, with a dropout of 0.1 which was then passed through a linear layer and then a relu to get final classes prediction. the model is trained using adam optimizer with a learning rate of 1e-6 for 10 epochs. for evaluating the systems, the organizers use balanced accuracy and f1-scores to obtain overall quality for the classification. ## results our system managed to get a balanced accuracy of 0.63 with this approach and f1-score of 0.69, 0.65 and 0.61 for the classification into phrases, passages and multipart, respectively. although we couldn't manage to beat the original papers' scores which they achieved with roberta, compared to their scores for the bert in which they got a balanced accuracy of 0.67. the main reason for this difference apart from the minor experimental setup, is the fact that the post title in the dataset was not used by us. the post content, although has more details than the title, is not sufficient in classification of the type of spoiler to be generated which means that from the title of the clickbait link or post the type of spoiler can be inferred. this can also be manually verified by taking some example annotated spoilers. for example, if the title of a clickbait post has lines like "5 ways to become healthy overnight" or "list of things needed for office work to be done efficiently" it can be inferred that a multi part answer would be the perfect fit for it. ## conclusion in this paper, we compare the performance of the bert-base model on the task of clickbait spoiler classification with two different approaches that differ in the data presented to them. our system uses only the content of the clickbait post and gave a balanced accuracy of 0.63 and other system which concatenated the post title and the post content achieved a balanced accuracy of 0.67. we conclude that the type of spoiler to be generated can be inferred more effectively with the help of the title and it cannot be ignored.