F rancis Wilde at S em E val-2023 Task 5: Clickbait Spoiler Type Identification with Transformers (2023)

Abstract: Clickbait is the text or a thumbnail image that entices the user to click the accompanying link. Clickbaits employ strategies while deliberately hiding the critical elements of the article and revealing partial information in the title, which arouses sufficient curiosity and motivates the user to click the link. In this work, we identify the kind of spoiler given a clickbait title. We formulate this as a text classification problem. We finetune pretrained transformer models on the title of the post and build models for theclickbait-spoiler classification. We achieve a balanced accuracy of 0.70 which is close to the baseline.

## introduction clickbait-spoiling @xcite aims to counteract the manipulative tactics used by clickbait creators and to help people avoid wasting their time on low-quality content. this is achieved by revealing the "suspense" that has been deliberately created in the title by the authors of the post. without clickbait spoiling, the reader has to spend considerable time to satisfy his curiosity by finding the content which was promised in the title. by clickbait-spoiling, the user can easily skim through the key content without wasting valuable time. the clickbait spoiling task at semeval 2023 @xcite consists of two subtasks, spoiler type classification and spoiler generation. both tasks are presented in the english language. we attempted the first task which is spoiler type classification. we use pre-trained transformer models and finetune them on the posttext to train a spoiler-type prediction model. we also experiment with pretrained transformer representations and train classical ml models on these representations for classifying the spoiler type. our final model, a vanilla transformer model finetuned on this dataset, achieves a balanced accuracy metric of 0.7 which is close to the baseline. ## background in the task for clickbait spoiler type classification, the input is a tweet along with its associated metadata. we formulate the problem of spoiler type identification as a text classification task. we hypothesize that a human can easily predict the spoiler type of a clickbait by looking at the post of the clickbait i.e in our case, it is the posttext associated with the tweet. we adopt the method followed by @xcite for detecting the clickbait intensity where the authors use only the posttext field to predict the intensity of the clickbait. we also limit ourselves to using only the posttext field in the dataset. we train models to predict the spoiler type given the posttext of the tweet. we trained models with the combination of posttext and targetparagraphs but they performed worse compared to the posttext alone. hence we settled to using only the posttext for the task. ## transformers transfer learning is a popular approach in natural language processing that involves leveraging knowledge gained from one task to improve performance on another related task. the transformer architecture introduced by @xcite ## system overview we follow the standard way of funetuning transformer models where pretrained transformer models are finetuned. in this method, all the layers of the transformer model and the weights in the classification layer are jointly learned during the finetuning process. we experiment with 4 transformer models i.e bert, roberta, distilbert and distilroberta. we use only the posttext field to do the fine-tuning. we do not do any specific preprocessing. we finetune each model for 3 epochs. we use a learning rate of 2e-5. default values provided by the huggingface library are used for the rest of the configurable parameters. ## experimental setup the organizers of the task have compiled a dataset of 5000 clickbaits. out of this 1000 samples are re-served as test data and their ground truths were kept hidden from the participants. the remaining 4000 samples were released for training the models. out of these 4000 samples, we used 3200 samples for training our models and kept the other 800 samples for development/validation. we use the huggingface transformers library foot_0 for training our models. our metric for evaluation is the accuracy and the f1-score. our training and evaluation was performed on google colab foot_1 . the clickbait evaluation happens on a platform called tira @xcite foot_2 . the participants can either submit a docker image or upload code and models on a virtual machine hosted on tira, which can read the test data and produce the results. the tira platform is used so that the test data is kept confidential and to prevent participants overfitting to the test data. in addition, the participants can also upload the results file on the web and get the results evaluated. ## results ## conclusion the approach followed by us is a very simple method by just using the posttitle. in the future we can experiment with more fields and leverage semi-supervised learning techniques and improve the classification performance. we can explore more transformer models also.