C o W o YTP 1 A tt: A Social Media Comment Dataset on Gender Discourse with Appraisal Theory Annotations (2025)

Abstract: This paper presents the Corpus on Women in YouTube on Performance with Attitude Annotations (CoWoYTP1Att), developed based on Appraisal Theory (Martin & White, 2005). Between September 2020 and May 2021, 14,883 comments were extracted from a YouTube video featuring a compilation of the performance “Un violador en tu camino” (A Rapist in Your Path) by the feminist collective LasTesis, published on the channel of the Costa Rican newspaper La Nación. The extracted comments were manually and automatically classified based on several criteria to determine their relevance to the video. As a result, 5,939 comments were identified as related to the video. These comments were annotated with the three attitude subdomains (affect, judgement, and appreciation) proposed on the Appraisal Theory (Martin & White, 2005), as well as their polarity, target, fragment, and whether the attitude was implicit or explicit. The statistical analysis of the corpus highlights the predominant negative evaluation of individuals present in the comments on this social media platform.

## introduction in december 2019, the costa rican newspaper la nación published a video featuring a compilation of the performance un violador en tu camino (a rapist in your path) by the chilean feminist collective lastesis. in this video, a group of women sang and danced in protest against sexism and the violation of women's rights. although the performance was first presented on november 25th in santiago, chile, women from all over the world later joined to present it in countries such as spain, germany, france, united kingdom, dominican republic, argentina, @xcite . the impact of this performance was so significant that time magazine included lastesis in the list the 100 most influential people of 2020. these performances have sparked a series of reactions that have been widely shared on social media. in this study, only the comment section of the video published on youtube is considered. a total of 14,883 comments were extracted from the video using maxqda, with 5,939 comments automatically identified as related to the video. even though there are multiple videos and manifestations regarding social movements, un violador en tu camino (a rapist in your path) represents the start of a series of performances that were held along 33 countries worldwide. the aim of this paper is to present the corpus on women in youtube on performance with attitude annotations (cowoytp1att) foot_0 , developed based on appraisal theory @xcite . this framework allows for the identification and classification of evaluations of individuals or entities, specifically those associated with feminism, women, or the performance itself. cowoytp1att was annotated with the subdomains of attitude, polarity, and the implicitness of the evaluation. this paper outlines the labels, the annotation process, and the characteristics of the annotated corpus. ## related work appraisal theory has been widely applied to diverse corpora, including diplomatic speeches @xcite , social media @xcite , newspaper commentaries @xcite @xcite , news articles @xcite , fake news (trnavac and põldvere, 2024), and reviews @xcite , demonstrating its versatility in analyzing evaluative language across textual genres. research indicates that the choice of appraisal domains or subdomains is influenced by the textual genre. in diplomatic speeches, judgement with a positive polarity is most common @xcite . in social media, affect dominates @xcite , while newspaper commentaries primarily use judgement with frequent use of negative language to intensify emotions @xcite . in reviews, appreciation is the second most frequent subdomain @xcite . both genuine and fake news also predominantly use judgement @xcite . annotating appraisal presents challenges, including difficulty identifying categories (anisimova and zikánová, 2022), annotator subjectivity, and disagreement among annotators, often resulting in low inter-annotator agreement @xcite @xcite . the need for extensive contextual understanding further complicates the process @xcite @xcite . consequently, some argue that automatic annotation is unreliable @xcite , underscoring the role of linguistically trained human annotators @xcite . to improve annotation consistency and facilitate future research, scholars suggest publishing the datasets @xcite and sharing key annotation decisions and guidelines @xcite . in recent years, research has also focused on automating the identification of appraisal. some studies have employed lexicon-based approaches @xcite and explored methods for identifying appraisal targets @xcite . additionally, large language models (llms) have been used to detect judgement in tweets @xcite and to classify media attitudes towards china in newspaper articles @xcite . furthermore, a recent study by @xcite investigated the use of chatgpt for annotating attitude subdomains in english texts. as demonstrated in the reviewed studies, judgment is the most frequent subdomain across various genres, which we can reasonably expect to be the case in our social media corpus. several studies highlight the inherent difficulty of annotating attitude due to its subjective nature and how this impacts inter-annotator agreement (iaa) results. finally, despite efforts to automate the annotation of attitude, it remains necessary to have annotations reviewed by linguistically trained annotators and to develop high-quality datasets. ## conclusion this paper presents the cowoytp1att corpus, comprising 1,521 spanish-language internet comments on the performance un violador en tu camino (a rapist in your path), annotated using the attitude domain of appraisal theory @xcite . the corpus offers detailed annotations on attitude, polarity, and implicitness. focus on gender roles and evaluations of individuals, providing valuable data for research on discourse (e.g., gender) and pragmatic phenomena (e.g., (im)politeness and speech acts). grounded in appraisal theory @xcite , the dataset distinguishes affect, judgement, and appreciation, yet aligns well with standard sentiment analysis. polarity labels (positive/negative) match traditional sentiment classes, while attitude types add granularity-for instance, differentiating emotions (affect), moral judgments (judgement), and aesthetic values (appreciation). this enables the creation of multi-label or hierarchical sentiment models that go beyond basic polarity. by annotating both explicit and implicit attitudes, the corpus supports complex tasks such as sarcasm and stance detection, where conventional datasets often lack depth. thus, cowoytp1att is a valuable resource for transfer learning, domain adaptation, and building models that capture nuanced sentiment. corpus analysis reveals judgement as the most frequent subdomain, with a predominance of explicit and negative evaluations. these findings indicate that the comments are primarily concerned with negatively evaluating people (judgement), rather than objects or ideas (appreciation). despite this negativity, linguistic strategies for implicit or mitigated evaluations are common. future work includes releasing the corpus in multiple formats, extending annotations, exploring automatic data augmentation, and conducting further analyses to uncover its full research potential.