ASAP : A C hinese Review Dataset Towards Aspect Category Sentiment Analysis and Rating Prediction (2021)

Abstract: Sentiment analysis has attracted increasing attention in e-commerce. The sentiment polarities underlying user reviews are of great value for business intelligence. Aspect category sentiment analysis (ACSA) and review rating prediction (RP) are two essential tasks to detect the fine-to-coarse sentiment polarities. ACSA and RP are highly correlated and usually employed jointly in real-world e-commerce scenarios. While most public datasets are constructed for ACSA and RP separately, which may limit the further exploitation of both tasks. To address the problem and advance related researches, we present a large-scale Chinese restaurant review dataset ASAP including 46, 730 genuine reviews from a leading online-to-offline (O2O) e-commerce platform in China. Besides a 5-star scale rating, each review is manually annotated according to its sentiment polarities towards 18 pre-defined aspect categories. We hope the release of the dataset could shed some light on the field of sentiment analysis. Moreover, we propose an intuitive yet effective joint model for ACSA and RP. Experimental results demonstrate that the joint model outperforms state-of-the-art baselines on both tasks.

## introduction with the rapid development of e-commerce, massive user reviews available on e-commerce platforms are becoming valuable resources for both customers and merchants. aspect-based sentiment analysis(absa) on user reviews is a fundamental and challenging task which attracts interests from both academia and industries @xcite @xcite . according to whether the aspect terms are explicitly mentioned in texts, absa can be further classified into aspect term sentiment anal-ysis (atsa) and aspect category sentiment analysis (acsa), we focus on the latter which is more widely used in industries. specifically, given a review "although the fish is delicious, the waiter is horrible!", the acsa task aims to infer the sentiment polarity over aspect category food is positive while the opinion over the aspect category service is negative. the user interfaces of e-commerce platforms are more intelligent than ever before with the help of acsa techniques. for example, figure 1 presents the detail page of a coffee shop on a popular ecommerce platform in china. the upper aspectbased sentiment text-boxes display the aspect categories (e.g., food, sanitation) mentioned frequently in user reviews and the aggregated sentiment polarities on these aspect categories (the orange ones represent positive and the blue ones represent negative). customers can focus on corresponding reviews effectively by clicking the aspect-based sentiment text-boxes they care about (e.g., the orange filled text-box "卫生条件好" (good sanitation)). our user survey based on 7, 824 valid questionnaires demonstrates that 80.08% customers agree that the aspect-based sentiment text-boxes are helpful to their decision-making on restaurant choices. besides, the merchants can keep track of their cuisines and service qualities with the help of the aspect-based sentiment text-boxes. most chinese e-commerce platforms such as taobao foot_0 , dianping foot_1 , and koubei foot_2 deploy the similar user interfaces to improve user experience. users also publish their overall 5-star scale ratings together with reviews. figure 1 displays a sample of 5-star rating to the coffee shop. in comparison to fine-grained aspect sentiment, the overall review rating is usually a coarse-grained synthesis of the opinions on multiple aspects. rating pre-diction(rp) @xcite @xcite ) which aims to predict the "seeing stars" of reviews also has wide applications. for example, to promise the aspect-based sentiment text-boxes accurate, unreliable reviews should be removed before acsa algorithms are performed. given a piece of user review, we can predict a rating for it based on the overall sentiment polarity underlying the text. we assume the predicted rating of the review should be consistent with its groundtruth rating as long as the review is reliable. if the predicted rating and the user rating of a review disagree with each other explicitly, the reliability of the review is doubtful. figure 2 demonstrates an example review of low-reliability. in summary, rp can help merchants to detect unreliable reviews. therefore, both acsa and rp are of great importance for business intelligence in e-commerce, and they are highly correlated and complementary. acsa focuses on predicting its underlying sentiment polarities on different aspect categories, while rp focuses on predicting the user's overall feelings from the review content. we reckon these two tasks are highly correlated and better performance could be achieved by considering them jointly. as far as we know, current public datasets are constructed for acsa and rp separately, which limits further joint explorations of acsa and rp. to address the problem and advance the related researches, this paper presents a large-scale chinese restaurant review dataset for aspect category sentiment analysis and rating prediction, denotes as asap for short. all the reviews in asap are collected from the aforementioned e-commerce platform. there are 46, 730 restaurant reviews attached with 5-star scale ratings. each review is manually annotated according to its sentiment polarities towards 18 fine-grained aspect categories. to the best of our knowledge, asap is the largest chinese large-scale review dataset towards both acsa and rp tasks. we implement several state-of-the-art (sota) baselines for acsa and rp and evaluate their performance on asap. to make a fair comparison, we also perform acsa experiments on a widely used semeval-2014 restaurant review dataset @xcite . since bert @xcite has achieved great success in several natural language understanding tasks including sentiment analysis @xcite @xcite , we propose a joint model that employs the fine-to-coarse semantic capability of bert. our joint model outperforms the competing baselines on both tasks. compared with the prosperity of english resources, high-quality chinese datasets are not rich enough. "chnsenticorp" @xcite , "it168test" @xcite , "weibo" foot_3 , "ctb" @xcite are 4 popular chinese datasets for general sentiment analysis. however, aspect category information is not annotated in these datasets. @xcite presents two chinese absa datasets for consumer electronics (mobile phones and cameras). nevertheless, the two datasets only contain 400 documents (∼ 4000 sentences), in which each sentence only mentions one aspect category at most. bdci foot_4 automobile opinion mining and sentiment analysis dataset @xcite contains 8, 290 user reviews in automobile industry with 10 predefined categories. @xcite summarizes available chinese absa datasets. while most of them are constructed through rule-based or machine learning-based approaches, which inevitably introduce additional noise into the datasets. our asap excels above chinese datasets both on quantity and quality. rating prediction. rating prediction (rp) aims to predict the "seeing stars" of reviews, which represent the overall ratings of reviews. in comparison to fine-grained aspect sentiment, the overall review rating is usually a coarse-grained synthesis of the opinions on multiple aspects. @xcite ; @xcite form this task as a text classification or regression problem. considering the importance of opinions on multiple aspects in reviews, recent years have seen numerous work @xcite @xcite utilizing the information of the aspects to improve the rating prediction performance. this trending also inspires the motivation of asap. most rp datasets are crawled from real-world review websites and created for rp specifically. amazon product review english dataset (mcauley and leskovec, 2013) containing product reviews and metadata from amazon has been widely used for rp @xcite . another popular english dataset comes from yelp dataset challenge 2017 foot_5 , which includes reviews of local businesses in 12 metropolitan areas across 4 countries. openrice foot_6 is a chinese rp dataset composed of 168, 142 reviews. both the english and chinese datasets don't annotate fine-grained aspect category sentiment polarities. ## conclusion this paper presents asap, a large-scale chinese restaurant review dataset towards aspect category sentiment analysis (acsa) and rating prediction (rp). asap consists of 46, 730 restaurant user reviews with star ratings from a leading e-commerce platform in china. each review is manually annotated according to its sentiment polarities on 18 fine-grained aspect categories. besides evaluations of acsa and rp models on asap separately, we also propose a joint model to address acsa and rp synthetically, which outperforms other state-ofthe-art baselines considerably. we hope the release of asap could push forward related researches and applications.