Clickbait Spoiling via Question Answering and Passage Retrieval (2022)

Abstract: We introduce and study the task of clickbait spoiling: generating a short text that satisfies the curiosity induced by a clickbait post. Clickbait links to a web page and advertises its contents by arousing curiosity instead of providing an informative summary. Our contributions are approaches to classify the type of spoiler needed (i.e., a phrase or a passage), and to generate appropriate spoilers. A large-scale evaluation and error analysis on a new corpus of 5,000 manually spoiled clickbait posts—the Webis Clickbait Spoiling Corpus 2022—shows that our spoiler type classifier achieves an accuracy of 80%, while the question answering model DeBERTa-large outperforms all others in generating spoilers for both types.

## introduction clickbait is the term used to describe posts in social media that are intended to inappropriately entice their readers to visit a web page. this is achieved through formulations such as sensationalism or cataphors that are believed to create a so-called curiosity gap: "a form of cognitively induced deprivation that arises from the perception of a gap in knowledge or understanding" @xcite . clickbait is perceived as inappropriate since its resolution is usually ordinary or trivial, comprising little more than a phrase, short passage, or a list of things that could just as easily have been included in the post. this observation motivates us to introduce the task of clickbait spoiling: identifying or generating a spoiler for a clickbait post. figure 1 shows four examples of clickbait on twitter, along with spoilers. the first two tweets explicitly or implicitly promise a surprising resolution to spark curiosity, but their spoilers are brief and trivial. the linked page of the first tweet adds almost nothing, and the spoiler of the second is common sense. the third spoiler is a passage from the linked page, and the fourth is a list of things. ## related work following an overview of research on clickbait and its operationalization so far, models of question answering and passage retrieval are examined. ## webis clickbait spoiling corpus 2022 to tackle clickbait spoiling for the first time, we created the webis clickbait spoiling corpus 2022 (webis-clickbait-22), a collection of 5,000 clickbait posts and their associated spoilers. ## evaluation of spoiler type classification in our evaluation, we assume a setup in which a previous clickbait detection would have (perfectly) identified posts as clickbait. to then evaluate the effectiveness of spoiler type classification on such detected clickbait posts, we conduct three experiments: (1) multi-class, (2) one-vs-rest, and (3) onevs-one for the types of phrase and passage spoilers. in all cases, the hyperparameters of the six studied classifiers were optimized based on the validation set of our corpus. for the three featurebased approaches, a chi-square feature selection step selected all post-based features and 70% of the document-based features. the post-based features are weighted 4-times higher than the documentbased features. most hyperparameters of the transformer models were left at their default values, but a grid search was used to find the most effective combination of learning rate (1e-5, 4e-5, 1e-4), warm-up ratio (0.02, 0.06, and 0.1), stack size (8, 16, and 32), number of epochs (1 to 10), and maximum sequence length @xcite @xcite . the substantial improvements of deberta and roberta over the feature-based classifiers in all settings (about 9-10 accuracy points) indicates that classifying the clickbait spoiler type requires more advanced language "understanding" than what is encoded in the basic features that the naïve bayes, svm, or logistic regression classifiers used. ## evaluation of spoiler generation to assess the effectiveness of the question answering and passage retrieval methods for clickbait spoiling, we evaluate both for their respective intended spoiler types, but each also for the respective other spoiler type. multipart spoilers are deferred to future work. we continue to assume that prior clickbait detection (perfectly) identifies clickbait posts as such. our evaluation of the generated spoilers includes quantitative and qualitative assessments (section 6.1). in a pilot study with ten question answering and ten passage retrieval models at their default settings, two models in each category dominate the respective others (section 6.2). the computationally expensive step of hyperparameter optimization is restricted to these four models plus two baselines (section 6.3). then, the effectiveness of spoiling clickbait posts dependent on spoiler type is evaluated (sections 6.4 and 6.5), and compared to an end-to-end clickbait spoiling setup independent of spoiler type (section 6.6). ## conclusion clickbait spoiling is a new task to help social media users who do not want to be manipulated into falling for clickbait links. unlike clickbait detection, which often involves filtering out clickbait posts from users' timelines, clickbait spoiling subverts the curiosity triggered by clickbait, presenting users with the withheld "punchline" in advance. we compile the first large resource for clickbait with associated spoilers. by interpreting clickbait spoiling as either a question answering task or a passage retrieval task, many possible approaches are available to extract from the linked document of a clickbait post the phrase or passage that spoils it. we have explored the effectiveness of a number of state-of-the-art solutions for both tasks in a largescale experiment, including fine-tuning the respective models on our resource to determine their effectiveness for type-specific clickbait spoiling. our experimental setup considers type-specific spoiling on the one hand, but on the other hand it also includes an end-to-end configuration for comparison. overall, our results show that type-agnostic question answering-based spoiling is the most effective yet, but also that spoiler type-specific solutions have the potential to outperform them. in addition to the possibilities explored, there might also be other approaches to clickbait spoiling: for example, paraphrasing technology could be used to directly transform a clickbait post into a version that contains its own spoiler. with respect to multipart spoilers, the use of summarization models could be an interesting direction to select the different parts of the linked document of a clickbait post that make up its multipart spoiler.