W ebis @ I mage A rg 2023: Embedding-based Stance and Persuasiveness Classification (2023)

Abstract: This paper reports on the submissions of Webis to the two subtasks of ImageArg 2023. For the subtask of argumentative stance classification, we reached an F1 score of 0.84 using a BERT model for sequence classification. For the subtask of image persuasiveness classification, we reached an F1 score of 0.56 using CLIP embeddings and a neural network model, achieving the best performance for this subtask in the competition. Our analysis reveals that seemingly clear sentences (e.g., “I support gun control”) are still problematic for our otherwise competitive stance classifier and that ignoring the tweet text for image persuasiveness prediction leads to a model that is similarly effective to our top-performing model.

## introduction in recent years, the analysis of the argumentative stance of images and texts has gained significant attention. several shared tasks have been conducted in this area, like the same-side stance classification @xcite on texts, and the image retrieval for arguments @xcite on images. however, especially for images, the task of stance detection is far from being solved @xcite . the imagearg 2023 competition then provided a platform for researchers to explore this task further in the multi-modal context of tweets with images. moreover, the competition featured a second task of predicting whether the image enhanced the persuasiveness of the text. in this paper, we present the work conducted by our team, "feeds," for the imagearg 2023 competition. our efforts led to insightful findings and promising results in both tasks, shedding light on the complexities of combining visual and textual information for argumentative analysis. for subtask a (argumentative stance classification), we employed a bert model (devlin et al., 1 authors contributed equally 2019) with stacked transformer encoders. a separate model was trained for each of the two topics. training encompassed tokenization, batch processing, optimizer, and learning rate optimization for f1 scores on the validation set. our approach achieved an f1 score of 0.84 on the test set. for subtask b (image persuasiveness classification), we employed the clip model @xcite and a linear neural network. we integrated image and text embeddings to have multimodal features fed into the neural network. tests with separate models and combined models for the two tasks were conducted. when removing the text features, we still get similar performance compared to using both features. therefore image features seem more decisive for persuasiveness than the text and the multimodality of this task is hard to leverage. we achieved an f1 score of 0.56 on the test set, which is the highest among all submissions. this paper is structured as follows: section 2 provides a brief overview of related work. in section 3, we detail our methodology and approaches for both subtasks. section 4 presents our results and their implications, while section 5 discusses the obtained results. finally, section 6 concludes the paper, summarizing our contributions and outlining potential directions for future research. ## related work argumentative stance detection is still considered a major problem in nlp. @xcite analyzed several stance classifiers for textual arguments, which achieved an accuracy between 0.50 to 0.77, and identified as challenges an inadequate topic knowledge of classifiers or when arguments only partial agree or disagree. similarly, carnot et al. (2023) identified several challenges for detecting the stance expressed in images when analyzing the submissions to the touché 2022 shared task on image retrieval for argumentation @xcite : bridging the semantic gap for diagrams, ambiguity arising from diverse valuations leading to varied interpretations, the dependence of image understanding on background knowledge, regional relevance, the presence of both stances in one image, irony, and more. all of these also apply here, but maybe to a lesser degree as classifiers were trained for each topic. @xcite dealt with multi-modal analysis in persuasiveness classification. they identified an issue that the image encoder could not capture text like slogans in images. they suggested extracting and using textual features from images. ## task we participated in both imagearg subtasks: subtask a: argumentative stance classification. given a tweet with text and an image, predict if the tweet supports or opposes a topic. subtask b: image persuasiveness classification. given a tweet with text and an image, predict if the image makes the tweet text more persuasive. for both subtasks, the organizers provide a humanannotated dataset of 2k tweets @xcite . 2 submissions are evaluated using f1 score. for illustration, figure 1 shows an example tweet for the gun control topic with associated classes: "support" for subtask a and "yes" for subtask b. 2 the script for downloading the dataset can be found in the shared task's git-repository: https://github.com/ imagearg/imagearg-shared-task ## our approach we employed neural models on text and image embeddings for tackling the tasks. for training, we either trained two separate models for the two topics of the dataset ("gun control" and "abortion") to capture topic-specific characteristics, or trained a combined model on both topics to capture topicindependent features. we then describe data preprocessing (section 4.1), and the models used in subtask a (section 4.2) and b (section 4.3). our code is available online. foot_0 ## results to analyze our approach, we provide both an overview table ( ## conclusion we presented the submissions of team "feeds" to the two subtasks of imagearg 2023 @xcite and results of further analyses we performed after the submission deadline. our approach for argumentative stance classification (subtask a) achieved a commendable f1 score of 0.84, but, as our analysis revealed, it, amongst other issues, struggled with classifying straightforward sentences like "i support gun control" or "i support abortion." additionally, subtask a's model didn't incorporate image data. future work could include images, for example using the visu-albert foot_6 (li et al., 2019) model, enabling classification using both text and images. our approach for image persuasiveness (subtask b) achieved the first position with an f1 score of 0.56. we observed that the model effectively classifies images that do not enhance persuasiveness, but struggles with identifying images that enhance the text's persuasiveness. this highlights the importance of advanced feature engineering to enhance the model's ability to identify nuanced persuasive elements within images. moreover, we found that our classifiers perform nearly as good without considering the text at all. this emphasizes the influential role of clip image embeddings within the model's decision-making process. further investigations are needed for understanding which role, if any, features from the tweet text could play in the classification of this task.