Exploiting Careful Design of SVM Solution for Aspect-term Sentiment Analysis (2024)

Abstract: Aspect-term sentiment analysis (ATSA) identifies fine-grained sentiments towards specific aspects of the text. While pre-trained language models (PLMs) have set the state-of-the-art (SOTA) for ATSA, they are resource-intensive due to their large model sizes, restricting their wide applications to resource-constrained scenarios. Conversely, conventional machine learning methods, such as Support Vector Machines (SVMs), offer the benefit of less resource requirement but have lower predictive accuracy. This paper introduces an innovative pipeline, termed SVM-ATSA, which bridges the gap between the accuracy of SVM-based methods and the efficiency of PLM-based methods. To improve the feature expression of SVMs and better adapt to the ATSA task, SVM-ATSA decomposes the learning problem into multiple view subproblems, and dynamically selects as well as constructs features with reinforcement learning. The experimental results demonstrate that SVM-ATSA surpasses SOTA PLM-based methods in predictive accuracy while maintaining a faster inference speed and significantly reducing the number of model parameters.

## introduction aspect-term sentiment analysis (atsa) is one of the subtasks in aspect-based sentiment analysis (absa) @xcite , which aims to predict fine-grained sentiment polarities given different aspect terms in a sentence. specifically, given a sentence @xmath0, w 2 , . . . , w n , and let @xmath1, a 2 , . . . , a k be the set of aspect terms extracted from, the goal of this task is to determine the polarity of each aspect term. for instance, consider the sentence "the battery life of this laptop is amazing, but the screen is too dim." the input of the pipeline should be the original sentence and the identified aspect terms "battery life" and "screen". ## our svm-atsa solution in this section, we elaborate on the details of svm-atsa. the overview of svm-atsa is shown in figure 2 , which summarizes the key components of svm-atsa. it decomposes the learning problem train original data sub-problem constructor data balancer sub-problem set feature constructor feature selector sub-problem set for train sub-problem set for test svm training hyperparameter tuner svm model set result set inference train train? yes no problem decomposition of atsa into multiple subproblems based on clustering so that the feature expression of svm can be boosted. besides, svm-atsa ensures the data in each subproblem is balanced to guarantee the model quality. through this subproblem construction, svm-atsa trains a number of svm models in the learning pipeline, instead of restricting to using only one svm model @xcite or simply decomposing the learning problem @xcite . to further enhance the model quality of svm-atsa, we propose techniques including feature construction on the raw features and feature selection to represent each subproblem, and automatically setting hyper-parameter configurations. we power the training process for svm-atsa with zeroth order optimization to calibrate potential flaws for different components in the pipeline. at last, all the svm models are used collectively to make predictions. the whole pipeline of svm-atsa is elaborated in greater detail next. ## related work in this section, we review the work of svm-based methods and plm-based methods for the aspectterm sentiment analysis (atsa) task. ## conclusion we have proposed svm-atsa for the aspect term sentiment analysis (atsa) task. by effectively decomposing the atsa task into multiple subproblems and employing dynamic feature selection through reinforcement learning, svm-atsa successfully reconciles the need for both high predictive accuracy and smaller model size. svm-atsa outperforms the existing sota models based on roberta large and deberta and enjoys substantially fewer parameters. our results show that an svm-based solution can outperform sota language models on the atsa task, when the svmbased learning pipeline is properly designed.