Task and Sentiment Adaptation for Appraisal Tagging (2023)

Abstract: The Appraisal framework in linguistics defines the framework for fine-grained evaluations and opinions and has contributed to sentiment analysis and opinion mining. As developing appraisal-annotated resources requires tagging of several dimensions with distinct semantic taxonomies, it has been primarily conducted manually by human experts through expensive and time-consuming processes. In this paper, we study how to automatically identify and annotate text segments for appraisal. We formulate the problem as a sequence tagging problem and propose novel task and sentiment adapters based on language models for appraisal tagging. Our model, named Adaptive Appraisal (A ˆ2 ), achieves superior performance than baseline adapter-based models and other neural classification models, especially for cross-domain and cross-language settings. Source code for A ˆ2 is available at: https://github.com/ltian678/AA-code.git

## introduction with the development of the web technology, opinion-bearing user generated texts such as reviews, users comments, blogs and microblogs are widespread. sentiment analysis and opinion mining on such texts are prominent nlp tasks that have attracted extensive research studies in the literature @xcite . on the other hand, the appraisal framework @xcite is a systemic functional linguistic theory describing how language is used by writers or speakers to express emotion and opinion. the appraisal framework consists of three subsystems: 1) attitude, which includes personal emotion, judgement and evaluation of entities; 2) engagement, which regards one's own opinions or with respect to others; and 3) graduation, which describes strength of the attitude and engagement expressed. appraisal annotated resources have been used for deeper sentiment and emotion analy-sis @xcite than simple sentiment classification, but building such resources manually requires significant linguistic expertise and is timeconsuming @xcite . automated appraisal tagging would be extremely beneficial to support annotation and analysis efforts by expert linguists. to our best knowledge, the only publicly available appraisal annotated corpus is the simon fraser university opinion and comments corpus (socc) @xcite foot_0 , which is based on news comments. in creating the socc corpus, 663,173 user comments were collected, and expert linguists manually annotated 1,043 comments. to the best of our knowledge, there has been no prior work leveraging machine learning for automatically tagging appraisals. we fill this gap in this paper. we especially target domains with voluminous opinionated texts but zero or very limited appraisal annotation resources, such as blogs and microblogs. our research focuses on sequence tagging for cross-domain and cross-lingual texts with low resources. in the literature, various approaches for sequence tagging tasks have been reported, including transfer learning @xcite , few-shot learning @xcite and multi-task learning @xcite @xcite . however, none of the existing studies on sequence tagging consider zero-shot cross-domain or cross-lingual settings. we propose a model for automatic appraisal tagging. our model a 2 , namely adaptive appraisal, utilises joint task and sentiment adapters based on pre-trained language models for tagging appraisal segments in text sequences. our model leverages the adapter-based transfer learning framework for cross-domain and cross-lingual appraisal tagging. based on the pre-trained language model, we propose the task adapter for appraisal tagging across different domains. for instance, the language that short microblog posts use for expressing appraisals is very different from the language of long blog posts. it is therefore necessary to enable the adaptive ability from one domain to different domains. we further propose the sentiment adapter to capture the sentiment knowledge for appraisal tagging, capitalised on the strong correlation between appraisals and sentiments. rather than employing appraisal annotations for sentiment analysis as in existing studies @xcite , we propose to leverage the rich sentiment analysis resources for automatic appraisal tagging. in this study, we seek to answer the following research questions. to summarise, our contributions are twofold: (1) we propose an adapter-based framework for appraisal tagging; and (2) we propose task and sentiment adapters to further enhance the framework for cross-domain and cross-lingual generalisation ability. ## related work our work is related to sequence tagging -such as named entiy recognition, semantic role labeling, where token sequences in the input text are tagged with class labels. our work is especially related to the task of sequence tagging with low resources for training. in the literature, to address the issue of low resources, sequence taggers based on transfer learning @xcite , few-shot learning @xcite and multi-task learning @xcite @xcite have been reported. our research falls under transfer learning. various transfer learning strategies and techniques have been proposed for nlp tasks addressing the issue of scarce labelled data. early transfer learning algorithms have addressed the target domain data scarcity problem and boosted the model's generalisation ability via learning domainagnostic knowledge for transfer @xcite . modern pre-trained language models (e.g. bert) have achieved the state-ofthe-art performance for a range of natural language processing (nlp) tasks via transfer learning by fine-tuning parameters for different tasks @xcite @xcite @xcite . apart from cross-domain, cross-lingual transfer learning has also been investigated, in particular for partof-speech tagging and dependency parsing @xcite . algorithms have been proposed for transfer learning tasks of low resource languages @xcite . all these transfer learning frameworks require fine-tuning parameters of the full model to achieve knowledge transfer, which limits the capacity for models to adapt to many target domains. adapter-based transfer is a recently proposed parameter-efficient transfer learning mechanism for adapting a pre-trained model to a target task without fine-tuning all parameters. adapter modules was originated from computer vision, to control the convolutions and adapt models to multiple domains @xcite . then, in nlp application, adapters have been widely used for quick adaption in combination with existing large language models to new tasks @xcite and avoiding catastrophic forgetting issues @xcite . @xcite generated adapter parameters from language embeddings for multilingual dependency parsing. @xcite combined the information stored in multiple adapters for most robust transfer learning between monolingual tasks. as adapter modules have been proved effective for efficient transfer learning with large language models, we propose task and sentiment adapters for the cross-domain and cross-lingual appraisal tagging tasks. ## the appraisal framework following the systemic functional linguistics (sfl) theory @xcite , the appraisal framework is a theory describing the linguistic patterns for authors to express emotion and opinion. the appraisal framework consists of three semantic systems including attitude, graduation and engagement. attitude is divided into three sub-systems: affect, judgement and appreciation. affect deals with a person's emotional reactions (e.g. happy, confident), judgement deals with assessing people's behaviour (e.g. powerful, truthful), and appreciation deals with constructing the value of things (e.g. fascinating, exciting). in this work, we focus on the three sub-systems of attitude and their polarity (positive, negative and neutral). figure 1 shows an example for appraisal annotations from the socc dataset @xcite ; each span is labelled with an attitude and its polarity. it should be noted that the linguistic appraisal framework in this study is different from the appraisal theory of emotion @xcite . the appraisal theory of emotion @xcite describes that emotions are the result of the way in which people appraise or evaluate events and situations in terms of their relevance and significance to their goals, needs and values. according to this theory, emotions are generated by the way in which people appraise events and situations, and the specific emotion that is generated depends on the type of appraisal that is made. the functional linguistic appraisal framework @xcite on the other hand, describes that emotions are the result of the way in which people use language to evaluate and interpret events and situations. the specific emotion that is generated depends on the specific linguistic patterns that are used. ## problem statement we frame our appraisal tagging task as a bio sequence tagging problem @xcite , where segments are tagged with the attitude labels of affect, appreciation and judgement, and polarity labels of positive, negative and neutral. let @xmath0, w 2 , ..., w l ) be an input sentence, where w i is the i-th token and l is the sequence length. the objective of the pro-posed model a 2 is to identify a set of attitude tags t @xmath1, i-attitude, o) and a set of polarity tags t @xmath2, i-polarity, o) for each w @xmath3. ## methodology the architecture of adaptive appraisal (a 2 ) model is shown in figure 2 . it comprises two modules: (1) a language model for automatic bio tagging with two target tasks, and (2) a sentiment adapter for learning sentiment specific knowledge and a task adapter for generating task-specific representations. ## datasets for the cross-domain and cross-lingual appraisal tagging task, we conducted experiments on two datasets: the socc and post datasets. the socc dataset comprises of 10,399 opinion news articles and 663,173 comments from the canadian daily newspaper, the glove and mail. in addition to the raw text, the corpus includes specific annotations from multiple perspectives: negation, appraisal, constructiveness and toxicity. among them, we used the appraisal annotations for our experiments. the post dataset comprises of twitter and blog posts annotated with appraisals developed in-house following the same appraisal framework @xcite ; it contains not only english text but also chinese text. in the post dataset, various spans of appraisals were annotated, as evaluation occurs at all levels of languages (words, phrases, clauses or entire sentence). in our experiments, we used the word and phrase level of labels. when there are phrases with more than one appraisal labels overlapping, we opted the longest span label.the data statistics of these two datasets are shown in ## illustration of a prediction error to provide a qualitative analysis for our approach, we showcase an example of an annotated sentence from the socc dataset in a 2 prediction gold annotation ## conclusion we have proposed a 2 , an adapter-based framework for automatically tagging appraisal expressions. we have designed task and sentiment adapters of a small number of additional parameters to improve the capacity of pre-trained language models for quick adaptation for cross-domain and crosslanguage settings.