John Boy Walton at S em E val-2023 Task 5: An Ensemble Approach to Spoiler Classification and Retrieval for Clickbait Spoiling (2023)

Abstract: Clickbait spoiling is a task of generating or retrieving a fairly short text with a purpose to satisfy curiosity of a content consumer without their addressing to the document linked to a clickbait post or headline. In this paper we introduce an ensemble approach to clickbait spoiling task at SemEval-2023. The tasks consists of spoiler classification and retrieval on Webis-Clickbait-22 dataset. We show that such an ensemble solution is quite successful at classification, whereas it might perform poorly at retrieval with no additional features. In conclusion we outline our thoughts on possible directions to improving the approach and shape a set of suggestions to the said features.

## introduction clickbait is a term for a descriptive piece of information (a headline, a snippet, a post in social media etc.) aiming to arouse users' interest by summarizing the contents of a linked document in a deceptive and/or misleading manner. the summary does not suffice to satisfy users' curiosity, whereas the linked document is in a way advertized to do so, which encourages viewers to click the link (and thus cover the arisen curiosity gap). using clickbait is frowned upon as the contents of the linked page does not usually correspond to the expectations of users and in some cases may be inappropriate, offensive or harmful. @xcite address the issue with an idea of clickbait spoiling: 1) predicting the type of a spoiler that would satisfy users' curiosity and thus withhold them from following the link; 2) retrieving the spoiler itself from the document. upon developing a solution @xcite compiled a dataset containing "5,000 "spoilable" clickbait posts" in english collected from twitter, reddit, and facebook, that then were manually spoiled and classified: the webis clickbait spoiling corpus 2022 . semeval-2023 task 5 suggests to introduce a model for clickbait spoiling and bring new insights to the existing approaches, basing on webis-clickbait-22 and (optionally) additional resourses @xcite . our team presents an ensemble solution called websemble (webis-clickbait-22 ensemble) mainly following the steps outlined in @xcite except for an additional (and optional) summarization of the documents before the classification. below we briefly list actions undertaken with our approach upon execution (more in section system overview): note that the ensemble approach mentioned above comes into action only at steps 3.1 and 3.2. for that, groups of either taken "out of the box" or fine-tuned on webis-clickbait-22 dataset transformers @xcite models for text classification and question answering (qa) make separate predictions that are afterwards collapsed into a single ensemble decision. implementations of websemble within semeval-2023 task 5 were submitted to tira @xcite dockerized. source code of the latest version is open and available under https://github.com/cicl-iscl/ semeval23-task5-john-boy-walton . dockerized environment can be pulled from https://hub.docker.com/repository/ docker/maxschmaltz/websemble (tag 0.28.amd). ## related work in our research we mostly oriented ourselves at @xcite . that approach put a significant value on the fact that spoilers can generally be divided into two classes: phrase and passage. and so the authors pay much attention to predicting the label type, depending on which the next steps are forked into two branches, where the spoilers are retrieved differently: phrases are predicted in terms of question answering in squad1.1 format, and passages are obtained with passage retrieval models. liello et al. (2022) present three novel objectives for pre-training transformer-based models, that aim to analyze sentence-and paragraph-level semantics "within and across documents". even though it is not related to clickbait spoiling directly, we reckon it might be quite helpful for improving existing and future clickbait spoiling models, since answer sentence selection models pre-trained with the objectives 1) have a higher performance; 2) are capable of defining information about whether a sentence belongs to a certain paragraph. we believe the distinction between phrases and passages is meaningful and justified enough yet given a slightly heavy weight. we try to show that spoilers might be obtained within a single pipeline agnostic to their types and that the type is more important during postprocessing rather than upon spoiler generation. websemble refers to the following models: summarization: in future we might want to use answer sentence selection models pre-trained with the objectives proposed in @xcite for dealing with mulipart spoilers (more in conclusion). ## system overview as mentioned in introduction, websemble takes the following steps: 1) preprocessing (described in baselines and preprocessing); 2) optional summarization with a single model; 3) spoiler type prediction with an ensemble of text classification models; 4) spoiler retrieval with an ensemble of question answering models; 5) postprocessing. in this section steps 3-5 will be discussed; step 2 will be omitted because the model is used "out of the box". ## experimental setup as we make software submissions, we must include all models into the environment. that leads to a necessity to fine-tune models locally and incorporate them into image only after. for details of finetuning refer to models. required dependencies (versions specified) can be found in project repository under https://github.com/cicl-iscl/ semeval23-task5-john-boy-walton . websemble takes multiple arguments that are described both in repository readme and in appendix. ## conclusion at semeval-2023 task 5 we developed an ensemble-based model aimed to spoil clickbait in four stages (preprocessing excluded): 1) optional summarization; 2) spoiler type prediction; 3) top k spoilers retrieval agnostic to predicted spoiler types; 4) top k preprocessing with respect to spoiler classes. we show that this approach is capable of adequate spoiler type prediction, but in some cases fails to postprocess retrieved spoilers in a way the result could be accepted by a human, which encourages us to try different implementations and conduct further research. we see the following potential courses of further development: