Walter Burns at S em E val-2023 Task 5: NLP - CIMAT - Leveraging Model Ensembles for Clickbait Spoiling (2023)

Abstract: This paper describes our participation in the Clickbait challenge at SemEval 2023. In this work, we address the Clickbait classification task using transformers models in an ensemble configuration. We tackle the Spoiler Generation task using a two-level ensemble strategy of models trained for extractive QA, and selecting the best K candidates for multi-part spoilers. In the test partitions, our approaches obtained a classification accuracy of 0.716 for classification and a BLEU-4 score of 0.439 for spoiler generation.

## introduction clickbait is the name given to content whose primary purpose is to lure visitors to a web page. often, this content is designed to appeal to users through sensationalism, aiming to affect their curiosity. once the user takes the bait, very often contains a large text that has little to do with the promised content in the clickbait, and there are only a few sentences that properly "answer" it. in their work, (d. @xcite describes that clickbait texts rely on the cognitive phenomenon known as information gap, consisting of alerting the reader with some relevant information on the text but leaving a gap in the reader's knowledge hence triggering its curiosity and the need to fill this gap. although the purpose of clickbait is to appeal to the reader's curiosity and emotions, this can have side effects as well. as stated in @xcite , clickbaits usually exaggerate headlines to attract the attention of the audience ignoring completely the fact that news should follow such as objectivity, truth and fairness. this can mislead the readers to have a wrong judgment of news events. the clickbait challenge at semeval 2023 @xcite is a shared task whose objective is to develop systems that, given a clickbait and the content of its linked page, can classify the required types of spoiler (task 1) and produce an adequate spoiler (task 2). there are examples of the use of automatic methods for detecting clickbaits @xcite @xcite , in which they aim to develop a system that can identify clickbait content and label it as such before it reaches the user. however, there is less research on systems for automatically "answering" clickbaits. in their work, @xcite proposes "spoiling" the posts by treating the problem as either a passage-retrieval or a question-answering problem. they experimentally found there is a significant difference in favor of the question-answering approach. in the same work, the authors compile the "webis clickbait spoiling corpus 2022", a dataset that consists of 5000 manually spoiled posts, and define three possible categories of spoilers: multi-part, passage, and phrase. in this work, we present our system for this shared task. for task 1, we fine-tune multiple roberta models in a multi-task setting and multiple roberta models in a binary task setting. we propose an attention neural ensemble architecture where all our pre-trained models are automatically weighted with attention mechanisms. for task 2, we approach the problem as extractive questionanswering, we use an ensemble to improve the stability of the predictions given the relatively small dataset. our ensemble approach considers two main aspects: (1) the use of different architectures, (2) averaging different instances of the architecture to improve stability, before extracting the final "answer". when evaluated in a validation partition, this two-level ensemble yields an improvement of around 11% when compared to a deberta single model, which serves as a baseline. ## experimental setup the proposed systems were evaluated using the training and validation partitions provided in the competition: we found these partitions to be adequately split since class proportions were similar between the validation and training sets. we employ the same evaluation metrics used for the shared task. for classification, we evaluate the accuracy, and for spoiler generation, bleu-4. ## ethical issues the creation of an automatic system capable of classifying the spoiler type and generating a spoiler from a query and a web page may help to reduce the amount of time a person spends trying to fill the information gap generated by the clickbait. however, entirely relying on an automatic system may generate another information gap as well by retrieving an incomplete answer, and more importantly, if the system fails to retrieve the correct information, the result can mislead the reader to rely on fake information. to avoid these issues, a more robust automatic spoiler generation system must be designed to retrieve a correct and complete answer independent of the spoiler type and particular text features. ## conclusion this paper described our participation in the clickbait challenge at semeval 2023. our results demonstrate that ensembling techniques yield a performance increase in both classification and spoilergeneration tasks compared to baseline models. for classification, we empirically demonstrate that the combination of different classification models improves accuracy by an important degree compared to just fine-tuning a single model. for the clickbait-spoiling task, we note that the two-level ensemble configuration produces adequate results by addressing the issues of stability and bias of the base models. furthermore, the approach taken on multipart spoilers of taking a set of potential candidates produces better spoilers compared to only taking the best candidate. a dynamic selection of the k parameter depending on the context is left for future work. in conclusion, our approach based on ensembles for both tasks successfully robustly tackles them, resulting in adequate results compared to baseline models.