Incorporating Syntax and Lexical Knowledge to Multilingual Sentiment Classification on Large Language Models (2024)

Abstract: This paper exploits a sentiment extractor supported by syntactic and lexical resources to enhance multilingual sentiment classification solved through the generative approach, without retraining LLMs. By adding external information of words and phrases that have positive/negative polarities, the multilingual sentiment classification error was reduced by up to 33 points, and the combination of two approaches performed best especially in high-performing pairs of LLMs and languages.

## introduction the generative approach using large language models (llms) performs very well even for nongenerative tasks such as topic classification @xcite and sentiment polarity judgment @xcite . llms have garnered attention, particularly for their capability to generalize languages in addressing those tasks without the need for taskspecific training using ground truth. many tasks can be solved in a zero-shot manner by providing appropriate prompts. however, the language coverage highly depends on llms, and the accuracy can be quite low if the model received insufficient training or tuning on specific languages. retraining llms requires a high cost to cover additional languages, and it tends to cause catastrophic forgetting in languages that have already been covered @xcite . in recent attempts at knowledge editing, difficulties such as knowledge conflicts have been pointed out @xcite , and it is not easy to apply to many languages. thus, we attempt to enhance multilingual sentiment classification based on the generative approach without computationally intensive processes for retraining and tuning llms. as depicted in figure 1 , our approach obtains lexical and syntactic information from an external sentiment extractor and inserts it into the prompt. this approach requires knowledge specific to tasks and languages but has advantages such as stability of outputs and controllability to add arbitrary knowledge to adapt to new languages or domains. the contributions of this paper are threefold: (1) to integrate syntactic and lexical knowledge with prompts for llms, (2) to evaluate sentiment classification using six publicly available llms and parallel data in 15 languages, and (3) to demonstrate the positive impacts by integrating two components without any retraining. ## sentiment classification using generative model this section describes the method to predict sentence polarity using an llm and our proposal to integrate the output of an external sentiment extractor with a prompt. ## conclusion this paper exploited external knowledge for the multilingual sentiment classification task on llms and achieved promising results. it demonstrated that the sentiment extractor can compensate the lack of linguistic knowledge in llms in several languages. additionally, even when incorrect information was provided, the llms were able to handle apparent mistakes, suggesting that the com-bination of the two approaches works best. however, several models did not recognize all the hints, indicating further enhancement of this combination is possible with better prompting techniques.