Gallagher at S em E val-2023 Task 5: Tackling Clickbait with S eq2 S eq Models (2023)

Abstract: This paper presents the systems and approaches of the Gallagher team for the SemEval-2023 Task 5: Clickbait Spoiling. We propose a method to classify the type of spoiler (phrase, passage, multi) and a question-answering method to generate spoilers that satisfy the curiosity caused by clickbait posts. We experiment with the state-of-the-art Seq2Seq model T5. To identify the spoiler types we used a fine-tuned T5 classifier (Subtask 1). A mixture of T5 and Flan-T5 was used to generate the spoilers for clickbait posts (Subtask 2). Our system officially ranks first in generating phrase type spoilers in Subtask 2, and achieves the highest precision score for passage type spoilers in Subtask 1.

## introduction the goal of semeval-2023 task 5 is to spoil clickbait. clickbait posts are texts that arouse curiosity not by providing informative summaries of articles, but by purposefully teasing and leaving out key information from an article to advertise a web page's content. the aim of this shared task is to generate short texts (spoilers) that satisfy the curiosity induced by clickbait posts and provide more informative summaries of the linked articles. although the first subtask may seem unnecessary, our results suggest that it could be highly helpful for the main purpose of spoiler generation. as clickbait posts are getting more common every day, this task holds more importance than ever. this task is only aimed towards english clickbait posts @xcite . our strategy for this shared task was to approach it as a question-answering problem. our system uses the state-of-the-art sequence-to-sequence (seq2seq) model t5 @xcite . we experiment with different variants of t5, some of which were previously fine-tuned for question-answering tasks. we implement a different model for each spoiler category type due to the contrast between the different types of output sequences. our system achieves competitive results in both subtasks compared to the baseline from the shared task organizers and to the other teams. it has the highest precision for classifying the passage type spoilers in the spoiler classification subtask, and ranks as the top submission for phrase type spoilers in the generation subtask. however, our system struggles with generating longer spoilers such as phrase and multi type spoilers. we submitted our system as a docker image on the tira platform @xcite to increase reproducibility. we release our code foot_0 and the model checkpoints foot_1 . ## system overview we used the t5 model @xcite for both the spoiler classification and spoiler generation tasks. t5 is a state-of-the-art seq2seq model and has shown successful performance for a variety of nlp tasks. it is a text-to-text framework, where the input is fed as a text and a target text is generated for the output. although t5 works for a variety of tasks out-ofthe-box, as it was trained on a large corpus with a mixture of supervised and unsupervised tasks, we did additional training to fine-tune it for the specific needs of the clickbait tasks. we also experimented with variants of t5 and models that were fine-tuned for specific tasks such as question answering. ## experimental setup we used the provided training data only to train our models, and the provided validation split only to evaluate the models. we did not use the validation split to further train our models. no preprocessing is done to the shared task data other than adding the subtask-specific prefixes mentioned in sections 3.1 and 3.2. we used the huggingface transformers library (version 4.6.0) to train our models. we used the adamw optimizer with an initial learning rate of 5 × e -5 and with a weight decay of 0.01 during the training for all models. for the spoiler classification subtask, we trained our models for 10 epochs and with a batch size of 8, and for the spoiler generation subtask, we trained our models for 5 epochs and with a batch size of 4. the maximum token length is 512 for all models except for the experiments with longt5. we use a token length of 1024 with the longt5 experiments. after each epoch, we evaluated the output generated on the validation data. we used the sklearn.metrics.balanced_accuracy_score from the scikit-learn library (version 1.2.1) to evaluate the balanced accuracy score for the spoiler classification subtask and to choose the best-performing model. for the spoiler generation task, we evaluated the output by the bleu-4 score using the evaluate library (version 0.4.0) from huggingface to choose the best-performing spoiler generation model for each spoiler category. we saved the model checkpoints after each epoch. we ran these experiments on lambdalabs cloud instances, with an nvidia a100 gpu. ## conclusion we present a question-answering framework to spoil clickbait posts. we show that the state-ofthe-art seq2seq model t5 and its variants perform well in classifying the spoiler type and in generat-ing the spoilers when fine-tuned on clickbait posts. we found that spoiling clickbait posts is easier for phrase type spoilers and gets progressively more difficult when the spoiler is longer, especially if it requires multi-span extraction like the multi type spoilers. in future work, we plan to improve the performance of the spoiler generation for passage and multi type spoilers. we observed that a more complicated approach is needed for these longer spoiler types rather than a simple extractive questionanswering method.