nancy-hicks-gribble at S em E val-2023 Task 5: Classifying and generating clickbait spoilers with R o BERT a (2023)

Abstract: Clickbait spoiling and spoiler type classification in the setting of the SemEval2023 shared task five was used to explore transformer based text classification in comparison to conventional, shallow learned classifying models. Additionally, an initial model for spoiler creation was explored. The task was to classify or create spoilers for clickbait social media posts. The classification task was addressed by comparing different classifiers trained on hand crafted features to pre-trained and fine-tuned RoBERTa transformer models. The spoiler generation task was formulated as a question answering task, using the clickbait posts as questions and the articles as foundation to retrieve the answer from. The results show that even of the shelve transformer models outperform shallow learned models in the classification task. The spoiler generation task is more complex and needs an advanced system.

## introduction clickbait describes the presentation of content on the internet in such a manner, that it encourages the viewer to click on a link 1 . to increase the likelihood of a click, lurid headlines or misleading descriptions are oftentimes used. clickbait spoiling oppositely tries to counteract by precisely summarising the most important information of a given article or document. task five of se-meval2023's discourse and argumentation section revolves around creating clickbait spoilers based on clickbait posts on social media, organized and evaluated through @xcite . further 1 oxford learner's dictionaries information about clickbait spoiling and the underlying data of the challenge is presented in @xcite . the clickbait challenge consisted of two tasks. the first is spoiler type classification and the second is spoiler creation. based on a social media post and the given article the correct spoiler type shall be classified for the first task. for task two, the correct spoiler shall be generated based on the information provided such as the advertised article. three systems based on the roberta @xcite transformer model were submitted for the first task: a multiclass classifier, a system based on three binary classifiers and a multiclass classifier that was further pre-trained. additionally, these systems were compared to different shallow learned classifiers beforehand. as for task 2, a system based on an off the shelve transformer model and spoiler type dependent post processing is proposed for the spoiler creation. the submitted systems are made publicly available on github foot_0 . ## background the dataset for the shared task is provided through the organizers website and contains 3200 rows for training and 800 for validation. 1000 additional spoilers were held back as testing data for the submission. out of 14 existing columns, the most relevant are the text of the post (posttext), the title of the advertised article (targettitle), the articles text (targetparagraphs), the spoiler itself (spoiler) and the type of the spoiler (tag). in general, three spoiler types exist in the dataset. phrase spoilers are a single or a few words, while passage spoilers consist of a whole sentence. multi spoilers have several forms, such as relevant parts of an article's phrases, enumerations or single words. after analyzing the training dataset, one can observe that the classes are not evenly distributed. 1367 phrase, 1274 passage and 559 multi spoilers exist. additionally, the publisher of the articles could be traced back. the articles mainly originate from the huffpost foot_1 , with more than 700 articles, followed by archive.is foot_2 , an internet archive with articles from various sources, with around 500 articles. the spoilers are generally part of the article and oftentimes formulated as a question. therefore it seems sensible to use an extractive question answering method for generating spoilers in task two. ## system overview the systems developed for both sub-tasks can be categorized into three main branches: systems based on shallow learned models and transformer based models for the first task and a transformer based system for question answering for the second task. ## experimental setup a series of experiments were conducted to evaluate the spoiler classification and creation systems. the final systems and configurations were determined for the final submission during these experiments. the weighted f1 score was used as the main metric for the spoiler classification problem because the three classes are distributed unevenly across the datasets. results for the spoiler creation system are reported through the bleu score @xcite . ## conclusion through the use of different approaches, several ideas could be implemented and partly submitted to the semeval2023 challenge. three fine-tuned roberta models were submitted for spoiler classification, with the best performing model showing good predictions for each spoiler type but not exceeding at a specific one. additionally, one of the submitted models has the highest precision in predicting phrase and multi spoilers in the shared task. however, the best performing model did not classify certain spoiler types perfectly but rather classified a large fraction of each correctly. for the spoiler generation, one system could be submitted based on a roberta model. while the results are not satisfactory, first attempts towards type specific spoiler generation could be made. the results suggest that transformer models are currently able to extract and spoil short information out of a text.