Chick Adams at S em E val-2023 Task 5: Using R o BERT a and D e BERT a to Extract Post and Document-based Features for Clickbait Spoiling (2023)

Abstract: In this manuscript, we describe the participation of the UMUTeam in SemEval-2023 Task 5, namely, Clickbait Spoiling, a shared task on identifying spoiler type (i.e., a phrase or a passage) and generating short texts that satisfy curiosity induced by a clickbait post, i.e. generating spoilers for the clickbait post. Our participation in Task 1 is based on fine-tuning pre-trained models, which consists in taking a pre-trained model and tuning it to fit the spoiler classification task. Our system has obtained excellent results in Task 1: we outperformed all proposed baselines, being within the Top 10 for most measures. Foremost, we reached Top 3 in F1 score in the passage spoiler ranking.

## introduction clickbait is a term used to describe social media posts that aim to attract users' attention and get them to click on those posts or visit a website. this is achieved through catchy titles that often exaggerate or distort the actual content of the post in order to generate traffic or advertising income @xcite . clickbait is considered inappropriate because its resolution is often ordinary or trivial, consisting of little more than a sentence, a short passage or a list of things that could easily have been included in the post. for this reason, the clickbait spoiling shared task @xcite , proposed at semeval 2023, consists of counteracting the negative effects of clickbait providing a spoiler or preview that reveals the main content of a post without requiring the user to click through to the website. the task is divided into two sub-tasks: ## background we made use exclusively of the dataset provided by the organizers, two pre-trained models, post-based features, and document-based features. the dataset consisted of 5000 posts written in english and distributed into three subsets, namely, training, validation, and testing, with 3200, 800, and 1000 posts, respectively. most spoiled clickbait posts comes from twitter (47.5o%) and reddit (36%), whereas the facebook account contributes a bit less (16.5%). our first experiment consisted in examining the number of examples of each type (either "phrase", "passage", "multi") to see if the dataset was balanced or not. the statistics of the dataset concerning the first subtask are shown in ## system overview for solving task 1, we built the system whose architecture is depicted in figure 1 . in a nutshell, our system works as follows. first, the dataset was preprocessed using the techniques pointed out in section 3.1. second, we extracted the post and document features of each item in the dataset. third, we fine-tuned different pre-trained models for the classification of spoilers types. next, different ensemble learning techniques were evaluated to see if the post and document features complemented each other and improved the model performance. finally, the best performing model was chosen to be tested with the test split. the source code is available at github foot_0 ## experimental setup after the preprocessing stage, we have two datasets for task 1, one with post features and one with document features (see section 3.1). both datasets have 3200 items for training and 800 for evaluation. the hyperparameters used for fine-tuning the roberta-large and deberta-v2-xlarge are 10 epochs, 0.01 in weight decay, a batch training of 8, and a learning rate of 1e-5. having two datasets with different features, different ensemble learning strategies have been evaluated on the trained models to see if the two features complement each other and improve the overall performance of the model. in ensemble learning, the output of each trained model with a set of textual features is combined by averaging the prediction (mean) or choosing the prediction with higher probability (max). ## results each model was evaluated using the validation split (see section 2). the result for task 1 is depicted in from ## conclusion in this working notes we describe the participation of the umuteam in the shared task 5 of se-meval 2023, namely, clickbait spoiling. in this shared task, the participants were required to identity spoiler type (i.e., a phrase or a passage) and generating short texts that satisfy curiosity induced by a post clickbait, i.e. generating spoilers for the clickbait post. our system has obtained excellent results in task 1, outperforming all proposed baselines, achieving the top 10 for most measures and a top 3 f1-score in the passage spoiler ranking. as further work, we are planing to measure the correlation between headlines related to clickbaits concerning the financial domain in spanish. for this, we will extend the dataset published in @xcite by incorporating news from financial news sites coming from spanish-speaking countries and will apply the spanish and multilingual variants of roberta and deberta. besides, we intend to extend this dataset with the inclusion of content from satirical media (garcía-díaz and valencia-garcía, 2022) and measure the impact of clickbait in datasets concerning political ideology @xcite . on the other hand, as commented in section 5, the most frequent error in our model is that it confuses passage-type spoilers with multiple-type spoilers. therefore, as future work, we plan to use volumetric and lexical diversity features in the training set to mitigate this error.