A Hybrid Approach of Opinion Mining and Comparative Linguistic Analysis of Restaurant Reviews (2021)

Abstract: The existing research on sentiment analysis mainly utilized data curated in limited geographical regions and demography (e.g., USA, UK, China) due to commercial interest and availability of review data. Since the user’s attitudes and preferences can be affected by numerous sociocultural factors and demographic characteristics, it is necessary to have annotated review datasets belong to various demography. In this work, we first construct a review dataset BanglaRestaurant that contains over 2300 customer reviews towards a number of Bangladeshi restaurants. Then, we present a hybrid methodology that yields improvement over the best performing lexicon-based and machine learning (ML) based classifier without using any labeled data. Finally, we investigate how the demography (i.e., geography and nativeness in English) of users affect the linguistic characteristics of the reviews by contrasting two datasets, BanglaRestaurant and Yelp . The comparative results demonstrate the efficacy of the proposed hybrid approach. The data analysis reveals that demography plays an influential role in the linguistic aspects of reviews.

## introduction sentiment analysis or opinion mining refers to the process of identifying opinions or sentiments expressed (e.g., positive, negative) in a text document @xcite . the lexicon-based method and machine learning (ml) based method are the two dominant approaches for opinion mining; although, their combinations have been also explored by the researchers. to evaluate the polarity of a piece of text, the lexicon-based methods rely on the sentiment lexicon comprised of opinion-conveying positive or negative terms and a set of rules @xcite . for the lexicon-based methods, the laborious steps of data labeling are not required. the supervised machine learning (ml) based approaches derive the relationship between features of the text segments and the opinions expressed in the writing in a supervised fashion @xcite . therefore, supervised ml classifiers require a significant amount of annotated data for training a predictive model for determining the polarity of a text document. labeling a large amount of text data is not only a challenging but also a tedious and costly process @xcite . researchers also investigated the combinations of both lexicon-based and ml-based approaches to form a hybrid method @xcite @xcite . mining customer opinions towards restaurants has attained popularity in recent years due to its impact on business growth and sustainability. researchers performed a number of studies using the yelp restaurant review and several other datasets. however, existing research on opinion mining in review datasets mainly focused on data curated in some specific demography (e.g., developed countries or countries with large consumer markets) due to commercial interest and abundance of usergenerated review data. as shown in @xcite , the trait and preferences of users can vary across demography. therefore, it is important to generate annotated content for various demography (e.g., geography, user, or language) and analyze them to identify the differences, which can ultimately help better decision-making. nowadays, with the increasing accessibility of the internet and the popularity of social media, opinion data are increasingly becoming available in many other geographies (e.g., bangladesh) and languages. therefore, the main objective of this work is to create a restaurant review dataset from lessexplored demography and introduce a new methodology to improve the performance of sentiment classification. besides, this study aims to explore how the demography affects the linguistic characteristics of reviews. we create a restaurant review dataset, banglarestaurant, which contains more than 2300 customer reviews toward various bangladeshi restaurants. we employ both the lexicon-based and ml-based methods to classify customer's sentiments in the banglarestaurant dataset. to improve the performance of sentiment classification, a hybrid methodology is introduced that leverages a lexicon-based method and an ml-based classifier. we observe an improvement of the f1 score by employing the proposed hybrid approach. we investigate the characteristics of reviews belong to the banglarestaurant dataset written by non-native english speakers and yelp reviews written by english native speakers. the comparative analysis reveals that demography (i.e., nativeness of language and geography) has influences on the various linguistic features of reviews. ## proposed methodology to determine the semantic orientations of the reviews, we employ both the lexicon-based and mlbased methods, as each of them has certain advantages over the other. ## impact of demography in linguistic attributes of review we analyze whether the demography of the reviews has any impact on the linguistic characteristics of the reviews. we consider two datasets, which differ in the following demographic aspects, geography (i.e., the location where reviews were written) and language nativeness of speakers (i.e., whether native or non-native speakers of english wrote reviews). our developed banglarestaurant corpus is curated in bangladesh, where people are nonnative speakers of english (sample reviews shown in figure 2 ). we contrast this dataset with the yelp restaurant reviews, which were written by usabased (mostly) english native speakers (sample reviews shown in figure 3 ). the yelp dataset contains 6860 positive comments and 1676 negative comments in contrast to 1702 positive samples and 613 negative samples in banglarestaurant dataset. to avoid any kind of influence of class distribution and dataset size, we use the same number of reviews from both the banglarestaurant and yelp datasets. for the yelp dataset, we randomly shuffle and then select 1702 positive samples and 613 negative reviews out of 6860 positive and 1676 negative reviews present. we analyze the following characteristics of the reviews in two datasets. for the yelp dataset, the numbers represent the average results of five random selections of 1702 positive comments and 613 negative comments. from ## discussion we observe in the banglarestaurant dataset performance of the lexicon-based approach is close to the ml-based classifiers. the best macro f1 score is obtained from the svm classifier, which is around 89%; the most accurate lexicon-based method lrsentia achieves an accuracy of 86%. this result is expected as supervised ml classifiers usually perform better than lexicon-based methods. from we notice the hybrid approach yields an overall f1 score of 0.915 in the banglarestraurant corpus, an improvement of 6% (0.860) over the best lexicon-based method and 3% (0.892) over the best ml-based classifier. although the increase of the f1 score is not much compared to the best ml classifier, the main advantage of the proposed hybrid approach is that it does not require any annotated data. review data is usually readily available on the web; the primary challenge is to label the data. thus, the proposed hybrid approach can be very effective for addressing the data annotation difficulties. we find the features characteristics of the reviews are distinct in banglarestaurant and yelp datasets, which represent data from different demography. ## summary and conclusions in this work, we introduce a hybrid approach for sentiment classification in a newly created banglarestaurant dataset. the proposed hybrid approach combines the lexicon-based method lrsen-tia with the svm classifier to improve the performance of sentiment classification. the results suggest that lexicon-based methods are mainly effective at classifying reviews that contain strong opinions. however, they struggle to determine sentiment when the polarity is not obvious. hence, it is necessary to incorporate an ml classifier that is robust for complex cases. in addition, we provide a comparative analysis of review data curated in different demography. we investigate various linguistic features of reviews that belong to these two datasets. the first dataset contains (i.e., banglarestaurant) reviews written by non-native english speakers of bangladesh; while, reviews of the other dataset (i.e.,yelp) were written by (mostly) english native speakers located in the usa. we observe differences in the various linguistic characteristics of reviews in these two datasets.