ROCLING -2021 Shared Task: Dimensional Sentiment Analysis for Educational Texts (2021)

Abstract: This paper presents the ROCLING 2021 shared task on dimensional sentiment analysis for educational texts which seeks to identify a real-value sentiment score of self-evaluation comments written by Chinese students in the both valence and arousal dimensions. Valence represents the degree of pleasant and unpleasant (or positive and negative) feelings, and arousal represents the degree of excitement and calm. Of the 7 teams registered for this shared task for two-dimensional sentiment analysis, 6 submitted results. We expected that this evaluation campaign could produce more advanced dimensional sentiment analysis techniques for the educational domain. All data sets with gold standards and scoring script are made publicly available to researchers.

## introduction the goal of sentiment analysis is to automatically identify affective information within texts. there are two major models to represent affective states: categorical and dimensional approaches @xcite . the categorical approach represents affective states as several discrete classes (e.g., positive, negative, neutral), while the dimensional approach represents affective states as continuous numerical values on multiple dimensions, such as valence-arousal (va) space @xcite , as shown in fig. 1 . the valence represents the degree of pleasant and unpleasant (or positive and negative) feelings, and the arousal represents the degree of excitement and calm. based on this two-dimensional representation, any affective state can be represented as a point in the va coordinate plane by determining the degrees of valence and arousal of given words @xcite @xcite @xcite or texts @xcite @xcite @xcite @xcite . in 2016, we hosted a first dimensional sentiment analysis task for chinese words @xcite . in 2017, we extended this task to include both word-and phrase-level dimensional sentiment analysis @xcite . this year, we explore the sentencelevel dimensional sentiment analysis task on educational texts (students' self-evaluated comments). structured data such as attendance, homework completion and in-class participation have been extensively studied to predict students' learning performance. unstructured data, such as selfevaluation comments written by students, is also a useful data resource because it contains rich emotional information that can help illuminate the emotional states of students @xcite . dimensional sentiment analysis is an effective technique to recognize the valence-arousal ratings from texts, indicating the degree from most negative to most positive for valence, and from most calm to most excited for arousal. this shared task provides an evaluation platform for the development and implementation of advanced techniques for dimensional sentiment analysis in the educational domain. the 33rd conference on computational linguistics and speech processing (rocling 2021) taoyuan, taiwan, october 15-16, 2021. the association for computational linguistics and chinese language processing ## task description in this task, participants are asked to provide a real-valued score from 1 to 9 for both valence and arousal dimensions for each self-evaluation comment. the input format is "sentence_id, sentence", and the output format is "sentence_id, val-lence_rating, arousal_rating". below are the input/output formats of the example sentences. test set: a total of 1,600 sentences were collected from the self-evaluated comments written by university students. each sentence was then annotated with valence-arousal ratings by seven annota- ## evaluation metrics prediction performance is evaluated by examining the difference between machine-predicted ratings and human-annotated ratings, in which valence and arousal are treated independently. the evaluation metrics include mean absolute error (mae) and pearson correction coefficient (r), as shown in the following equations. ## conclusions this study describes an overview of the ro-cling 2021 shared task on dimensional sentiment analysis for educational texts, including task design, data preparation, performance metrics and evaluation results. we hope the data sets collected and annotated for this shared task can facilitate and expedite future development in this research area. therefore, all data sets with gold standard and scoring script are publicly available 4 .