Mr-wallace at S em E val-2023 Task 5: Novel Clickbait Spoiling Algorithm Using Natural Language Processing (2023)

Abstract: This paper presents a model for clickbait spoiling,which aims at generating short texts that satisfy thecuriosity induced by a clickbait post. The modelis split into two tasks: identifying the clickbaittype and spoiling the clickbait. The first task isto classify the spoiler type that the clickbait postwarrants, and the second task is to generate thespoiler for the clickbait post. The model utilizesthe Distilbert-base-uncased model for the first taskand the Bert-base-uncased model for the secondtask. The trained model is optimized through trialand error on different model selections, and hyper-parameters and results are presented in a confusionmatrix. The main reason we utilized Distilbert-base-uncased is that it analyzes words in the con-text of what’s around it. The objective of this modelis to save readers time and spoil the clickbait of dif-ferent articles they may see on different platformslike Twitter and Reddit

## introduction clickbait, in social media, are exaggerated headlines whose main motive is to mislead the reader to "click" on them. they create a nuisance in the online experience by creating a lure towards poor content. online content creators are utilizing more of them to get increased page views and, thereby, more ad revenue without providing the backing content @xcite . clickbait articles usually appeal to a user's emotions and curiosity. these articles are often lacking in quality and information accuracy. clickbait articles have a considerably higher click rate than non-clickbait articles. this has been a rising problem in the 21st century with the growth in internet usage and social media. in semeval-2023, the tasks of clickbait spoiling task is split into two subtasks, as shown in figure 1 , the first task is to identify the clickbait type, and the second task is to actually spoil the clickbait by identifying the sequence(s) of text in the source article that provide the information that the reader is being lured to read about given a particular clickbait post @xcite . for task 1, which is the spoiler type classification, the input is the clickbait post and the linked document, and the task is to classify the spoiler type that the clickbait post warrants (either "phrase," "passage," or "multi"), which can be approached using a supervised text classification approach. for task 2, which is on spoiler generation, the input is the clickbait post and the linked document. the task is to generate the spoiler for the clickbait post, which can be approached using an information extraction or question-answering approach where the question is in the title of the post. more information about the source datasets can be found in figure 2 . we fine-tuned a distilbert @xcite ) model for task 1, which was one of the top ten in terms of precision for classifying the "multi" type of clickbait. the submission was made on the tira platform @xcite . while we were unable to make an official submission for task 2, we developed a bert-based @xcite question-answering-based model for this task and evaluated it on our own held-out validation set. ## background previous work has been done using different question answering like abstractive qa and extractive qa, with abstractive qa models extracting the answer to a given question from within a provided context, and abstraction qa models which generate new text and do not require a labeled answer span @xcite . in this paper, we will discuss and implement an abstractive qa model since this will give the user a more relevant answer which is guaranteed to be in the clickbait article. there have also been applications of passage retrieval models @xcite . previous work has been done to check whether a certain tweet is clickbait or not. one such high-accuracy system utilizes a neural network architecture with bidirectional gated recurrent units (bigru) and a self-attention mechanism to assess clickbait strength @xcite . ## system overview in this section, we provide details about our approach. ## results our system performed in the top ten in terms of precision for the "multi" class classification. figure 7 shows how misclassifications were made by our model. the "multi" classification was the classification in task 1, where the spoiler is a list of multiple items. we were only given the ranking for this specific classification. while the model is training, our script outputs the accuracy rate every 250 steps, so we can see whether the model is overfitting or underfitting. figure 8 is a comparison of distilbert-base-uncased vs. bert-base-uncased and bert-base-ner foot_2 models tuned on this task. bert-based ner (named entity recognition) is used to identify and classify named entities in text using bert (bidirectional encoder representations from transformers) language model. named entities can be people, organizations, locations, etc. from the figure, we can deduce that these types of models aren't the best for this specific task. we use the f1 score for task 2 to improve upon our model. the f1 score is a metric for evaluating the performance of a model on a binary classification task. the harmonic mean of the model's precision and recall is used to calculate the f1 score. precision is calculated as the model's total positive predictions divided by the number of true positive predictions. the recall is calculated by dividing the total number of positive examples in the dataset by the number of true positive predictions the model made. the f1 allowed us to see the overlap between the correct spoiler vs the generator spoiler. the f1 score was a better metric since it gave insight into the overall message/meaning of the spoiler instead of just word-for-word correctness. on the test split of the dataset, the model received an f1 score of 0.85, but generally, the results save users time and give an insight into the answer to what the clickbait/article is about. ## conclusion this paper presents a model for clickbait spoiling, which aims at generating short texts that satisfy the curiosity induced by a clickbait post. the model is split into two tasks: identifying the clickbait type and spoiling the clickbait. the first task is to classify the spoiler type that the clickbait post warrants, and the second task is to generate the spoiler for the clickbait post. the model utilizes the distilbert-base-uncased model for the first task and the bert-base-uncased model for the second task. the trained model is optimized through trial and error on different model selections, and hyperparameters and results are presented in a confusion matrix. from the figure 7 , we were able to deduce that the phrase and passage classification were being misclassified as the other. there was also a low accuracy for the multi-classification, which could be attributed to the lower amount of data for the multi compared to the phrase or multiclassifications. this gave us insight into how to further tune the model. the main reason we utilized distilbert-base-uncased is that it analyzes words in the context of whats around it. the objective of this model is to save readers time and spoil the clickbait of different articles they may see on different platforms like twitter and reddit.