Prompt-Based Approach for C zech Sentiment Analysis (2023)

Abstract: This paper introduces the first prompt-based methods for aspect-based sentiment analysis and sentiment classification in Czech. We employ the sequence-to-sequence models to solve the aspect-based tasks simultaneously and demonstrate the superiority of our prompt-based approach over traditional fine-tuning. In addition, we conduct zero-shot and few-shot learning experiments for sentiment classification and show that prompting yields significantly better results with limited training examples compared to traditional fine-tuning. We also demonstrate that pre-training on data from the target domain can lead to significant improvements in a zero-shot scenario.

## introduction in recent years, pre-trained bert-like @xcite models based on the transformer @xcite architecture and language modelling significantly improved the performance of various nlp tasks @xcite . the initial approach was to pre-train these models on a large amount of text and then fine-tune them for a specific task. more recently, an approach exploiting the nature of language modelling appeared, called prompting or prompt-based fine-tuning. prompting is a technique that encourages a pre-trained model to make specific predictions by providing a prompt specifying the task to be done @xcite . this new approach became very popular in solving nlp problems in zero-shot or few-shot scenarios, including sentiment analysis @xcite @xcite . most of the current research aimed at languages other than czech, especially english. to the best of our knowledge, no research has focused on any sentiment analysis task in the czech language using prompt-based fine-tuning. to address this lack of research, this paper presents an initial study focusing on two sentiment-related tasks: aspect-based sentiment analysis and sentiment classification in the czech language by applying prompt-based fine-tuning. the sentiment classification (sc), also known as polarity detection, is a classification task where the objective for a given text is to assign one overall sentiment polarity label. usually, the three-class scheme with positive, negative and neutral labels is used, but more labels can be applied @xcite . aspect-based sentiment analysis (absa) is a more detailed task compared to sc, which aims to extract fine-grained information about entities, their aspects and opinions expressed towards them. generally, the goal of absa is to identify the sentiment of each aspect or feature of a product or service. there are multiple definitions and versions of the absa task @xcite @xcite . in this work, we focus on the version of aspect-based sentiment analysis presented in the semeval competitions @xcite , which includes several subtasks. specifically, the tasks are aspect category detection (acd), aspect term extraction (ate), simultaneously detecting (aspect category, aspect term) tuples (acte), and detecting the sentiment polarity (apd) foot_0 of a given aspect term and category (see figure 1 for examples). in addition, we solve the target-aspect-sentiment detection task (tasd) @xcite , which aims to simultaneously detect the aspect category, aspect term and sentiment polarity. this paper presents a novel approach for solving czech sentiment classification and absa tasks using prompt-based fine-tuning. we utilize czech monolingual bert-like models and their language modelling ability to perform prompting for the apd and sc tasks. we use multilingual text-to-text generative models for the remaining absa tasks to generate textual predictions based on prompted input. our approach enables us to solve all these absa tasks at once, and we show that it is superior to the traditional fine-tuning approach for them. we also explore zero-shot and few-shot learning scenarios for apd and sc tasks and show that prompting leads to significantly better results with fewer training examples compared to traditional fine-tuning. additionally, we demonstrate that pretraining on data from a target domain results in great improvements in a zero-shot scenario. our study provides pioneered results for promptbased fine-tuning in czech sentiment. overall, our key contributions are the following: 1) we propose, to the best of our knowledge, the first prompt-based approach for sentiment analysis tasks in czech. 2) we show the superior performance of our prompting approach over traditional fine-tuning for absa tasks. 3) we compare the two approaches and show that prompting achieves better results than traditional fine-tuning in few-shot scenarios. ## related work this section reviews prior works conducted on sentiment analysis in czech. the prompt-based finetuning is a relatively new paradigm in nlp, and to the best of our knowledge, no research has yet explored its application on sentiment analysis in czech. to partly address this research gap, we include prompt-based approaches for analogous sentiment analysis tasks in english. ## data & tasks definition in this section, we describe the aspect-based and sentiment classification datasets. furthermore, we describe in more detail the absa tasks introduced in section 1, on which this paper is focused. ## models & approaches we use pre-trained transformer-based models as backbones for our experiments. we propose a method for solving multiple absa tasks concurrently with sequence-to-sequence models foot_6 , which process text (sequence) as input and produce text (sequence) as output. we employ this approach for the acd, ate, acte and tasd subtasks. to the best of our knowledge, there are no czech monolingual sequence-to-sequence models. therefore, we use the large mt5 @xcite and large mbart @xcite models, which are multilingual versions of the english t5 @xcite and bart @xcite ) models, respectively. we do not use these models for the apd task as they lack prior information about the aspect term and category, which they predict along with the sentiment. the apd task assumes that the model already knows the gold data for the aspect term and category, so we would have to modify the input and output format for the apd task to make a fair comparison. changing the output format would also be required for the sc task. since we focus solely on the czech language, we also wanted to evaluate czech monolingual models. as stated above, there are no monolingual czech sequence-to-sequence models, but only classical czech monolingual bert-like models such as czert @xcite , robeczech @xcite or fernet @xcite . unfortunately, these models are unsuitable for our proposed approach, so we use them only for the apd and sc tasks. these models consist only of the encoder part of the transformer architecture. ## experiments & results in our experiments, we fine-tune the sequenceto-sequence models (mbart, mt5) for the ate, acd, acte, and tasd tasks on the entire absa dataset using both traditional and prompt-based fine-tuning approaches and we report the results as micro f1 scores. the bert-like (encoder-based) models (czert, robeczech and fernet) are finetuned for the apd and sc tasks and report results as accuracy. for these tasks, we further experiment with zero-shot and few-shot scenarios, as well as additional pre-training of the czech models. to ensure the reliability of our results, we perform each experiment five times with different random seed initialization and report the average scores along with a 95% confidence interval. we provide the training details in appendix a.1. ## conclusion in this work, we introduced a sequence-to-sequence method that solves multiple absa tasks simultaneously and can be used with both traditional finetuning and prompting. experiments on the czech dataset show that prompting significantly improves performance. furthermore, we proposed a method for sentiment classification that can also be used with prompting and traditional fine-tuning. we evaluate this method on two czech datasets with three monolingual czech models and demonstrate the effectiveness of prompting for few-shot finetuning, where prompting consistently outperforms the traditional approach. finally, we show that pretraining on the domain data significantly enhances the results, especially in a zero-shot scenario.