A Dimensional Valence-Arousal-Irony Dataset for C hinese Sentence and Context (2022)

Abstract: Chinese multi-dimensional sentiment detection is a challenging task with a considerable impact on semantic understanding. Past irony datasets are utilized to annotate sentiment type of whole sentences of irony. It does not provide the corresponding intensity of valence and arousal on the sentences and context. However, an ironic statement is defined as a statement whose apparent meaning is the opposite of its actual meaning. This means that in order to understand the actual meaning of a sentence, contextual information is needed. Therefore, the dimensional sentiment intensities of ironic sentences and context are important issues in the natural language processing field. This paper creates the extended NTU irony corpus, which includes valence, arousal and irony intensities on sentence-level; and valence and arousal intensities on context-level, called Chinese Dimensional Valence-Arousal-Irony (CDVAI) dataset. Therefore, this paper analyzes the annotation difference between the human annotators and uses a deep learning model such as BERT to evaluate the prediction performances on CDVAI corpus.

## introduction the popularity of social media has made the exchange of opinions more frequent, and users not only use narratives but also widely use irony, metaphors and other special expressions when commenting on online forums. in the past, the literature has compiled research on irony detection @xcite . although most of the literature lacks a clear and consistent definition of irony, one of the most common features of irony is the inversion of the literal meaning and the semantic turn of the context. in chinese irony, the contrast between positive and negative emotions is often used to indicate the difference between sentences and contexts. this emotional contrast is often used to achieve sarcastic expressions @xcite . for example "great, it's raining, but i didn't bring an umbrella....", the context "it's raining, but i didn't bring an umbrella...." is a negative emotion, keyword "great" is a positive emotion. this emotional contrast enables the expression of irony. in order to improve the performance on the irony recognition task, this study argues that context must be considered to match the characteristics of ironic sentences. as the grammatical structure of the above-mentioned irony suggests, irony emotion detection is quite difficult in natural language processing (nlp). as a result, past research on irony detection is rare, and the work of emotion analysis turned to the study of characteristics of irony language @xcite . with the development of machine learning, some studies have gradually begun to use its methods to predict the degree of irony @xcite . however, due to the limitation of annotation, most of them predict only the degree of irony of the whole sentence @xcite instead of considering the expression differences of the context as mentioned above. in order to improve machine learning of identifying the intensity of irony, scholars have proposed to annotate these structural features or use feature selection to screen important irony spans when studying irony in the english language. @xcite . however, grammatical structures can differ in different languages, and the improvement of irony detection performance cannot be handled in the same way. @xcite proposed the usage of capitalized words as a hint of irony in english. such a method is not suitable for learning features of irony in chinese, as the notion of capitalization does not exist in the chinese language. in conclusion, while these rules have been thoroughly studied in english, their applicability to chinese is an inappropriate approach. although some scholars are studying chinese irony rules @xcite , there are few datasets that are quantified and annotated based on these rules. in conclusion, considering the multi-dimensional valence-arousal-irony (vai) irony sentences and context dataset, it is possible to identify the true meaning of ironic sentences and the emotional state of the sender, which also contributes to the field of chinese nlp. the construction of the existing vai corpus was carried out with the efforts of @xcite . its contribution is to show that the vai indicator has the characteristics of mutual influence. the biggest difference between the corpus established in this study and @xcite is that the context is considered and annotated with va. based on tang's @xcite ) open data on irony sentences, this study proposes to add sentence-level valence, arousal and irony annotations, and context-level valance annotations to the ironic sentences provided by the dataset. this annotation method provides a way to judge the difference in context and semantics in the subsequent analysis of irony sentences. by quantifying emotional indicators, the degree of irony is more clearly understood. this augmented cdvai dataset is the first dataset to do sentiment annotations for irony context. furthermore, this paper proposes a deep learning model based on the work of @xcite , bidirectional encoder representations from transformers (bert) to learn the dimensional vai on the ironic sentences and dimensional va on ironic contexts. the experimental models include (1) using a linear layer with pre-trained bert to predict dimensional vai on sentence and context; (2) sum hidden features of corresponding the context from pre-trained bert (3) concatenate two hidden features of bert from sentence and context, respectively. ## related works metaphor is a feature of irony, which can be expressed as the use of exaggerated keywords with positive emotions to describe things with negative emotions, which also makes the apparent meaning of the sentence opposite to the actual meaning that the speaker expects or wants to convey. it is also frequently used in satirical sentences. since irony is not commonly used in official documents, most researchers turn to social media platforms for data collection and analysis @xcite . due to different research perspectives, the definition of irony is often adjusted. however, researchers have reached a basic consensus in the process of exploration, that is, the basic feature of irony. "irony is an expression in which the true meaning is the opposite of the literal meaning" @xcite proposed an irony identification program (iip) to identify whether a sentence is ironic during the annotation process. using iip, they studied the semantic relations in the grammatical structure of irony according to the context. the above research provides support for the definition of irony in this study. english has few corpora for vai, and most of them are only for va (valence and arousal). in a recent study, @xcite established and annotated the va tags for facebook posts. they used the likert nine-point scale to annotate and found that the two indicators of va had a high correlation. in addition, in the construction of the irony corpus, @xcite proposed the corpus senti-tut to mark the irony and emotional expression of tweets on twitter. their work includes positive and negative emotions, emotional expression, and irony. the corpus considers the concept of valence instead of focusing only on irony. @xcite annotate figurative language such as irony, satire, and metaphor on a 11-point scale at semeval-2015 task 11 and recruit annotators using a crowdsourcing platform. in addition, there are still many foreign languages for the construction of va or i corpus, but literature that while few studies consider and label all three indicators simultaneously (vai), there is a correlation among the three indicators, and the following studies demonstrate the need to do so. the effect of irony on human emotions was found in the study of pfeifer @xcite . regardless of contextual emotion, people who use irony are considered to be in a less negative and less excited state of mind. the study by @xcite . found that stronger irony expressions may have lower valence (more negative) and higher arousal levels, respectively. research on chinese irony corpus xiang et al. (2020) constructed a corpus for irony. the ciron dataset they built contains 8.7k weibo posts. the study only annotated the degree of irony of sentences in the corpus without considering the context. ntu irony corpus @xcite has released that it only provides the ironic sentences and context, but without other sentiment scores such as sentence-level vai. lack of carry out more subtle emotional labeling for the internal grammatical structure, which is impossible to understand clearly on the emotional transitions and semantic changes in the sentences. therefore, the corpus provided in this paper has a greater advantage in understanding the structure of ironic sentences. in the follow-up application in the field of irony, @xcite considered emojis, which are often used on twitter, as a factor when annotating ironic sentences to improve the results. cnn (convolutional neural network) is implemented to pre-train the emoji, and xgboost model is applied for classification. naseem et al. (2020) proposed a t-dice model based on the transformer model to judge post valence, irony and irony classification. it was then connected to bi-lstm (bi-directional long short-term memory) to classify emotions, and its accuracy surpassed the most advanced methods at that time. @xcite found that the effect of bert is better than that of gru in the experimental results of the ciron dataset they built. @xcite improved the bi-gru model based on bert in the chinese sentiment analysis task to achieve the best results. to sum up, in recent years, no matter in sentiment analysis or in irony recognition tasks, lstm and other models that can connect the information of the entire sentence have achieved better results, and based on models with attention mechanisms such as bert or transformers can make the overall model achieves the best results. in summary, this paper will use bert as the basis to identify the vai of the sentence and the va of the context. ## cdvai dataset this paper annotates and extends the ntu irony corpus to a dimensional valence-arousalirony, called cdvai. ntu irony corpus provides ironic sentences and their ironic context. the annotation tasks are the vai intensity of the sentence and the va intensity of the context, respectively. li and huang @xcite analyzed the sentence structure of chinese irony based on the existing corpus, and proposed that context is an important information for judging irony. based on its findings and the sentence structure within the ntu irony corpus, this study defines irony as "irony is an expression in which the true meaning is the opposite of the literal meaning." context is the true meaning of the sentence (usually a negative description), while ironic keywords (usually positive descriptions) are required to make the literal meaning contrary to the context. ## model performance evaluation in order to verify the labeling consistency of this study and the feasibility of irony detection, this study uses a deep learning model to predict indicators. ## conclusion the paper introduced the cdvai dataset which extended from ntu irony corpus. it contains multi-index sentiment annotation and irony context sentiment annotation, which is helpful for developing chinese irony computational methods of detection that allow the model to learn sentimental changes and differences in context. the experimental results showed that the annotation of cdvai dataset provides a learning direction for the bert model as the basic model, so that the model can understand the irony structure. and the third method proposed in this paper, connecting the sentence with the context and then predicting, can effectively improve the effect of the model predicting arousal of context. cdvai dataset does not have enough examples for machine learning experiments, and there is no way to cover a considerable number of fields and complete ironic sentence patterns. nevertheless, the paper is suitable to use as guide data to obtain more samples or as a template for labeling guidelines. furthermore, the proposed cdvai dataset could be combined with other ironic corpora to extend the training sample size so that machine learning algorithms would be improved in the future.