Analyzing the Intensity of Complaints on Social Media (2022)

Abstract: Complaining is a speech act that expresses a negative inconsistency between reality and human’s expectations. While prior studies mostly focus on identifying the existence or the type of complaints, in this work, we present the first study in computational linguistics of measuring the intensity of complaints from text. Analyzing complaints from such perspective is particularly useful, as complaints of certain degrees may cause severe consequences for companies or organizations. We first collect 3,103 posts about complaints in education domain from Weibo, a popular Chinese social media platform. These posts are then annotated with complaints intensity scores using Best-Worst Scaling (BWS) method. We show that complaints intensity can be accurately estimated by computational models with best mean square error achieving 0.11. Furthermore, we conduct a comprehensive linguistic analysis around complaints, including the connections between complaints and sentiment, and a cross-lingual comparison for complaints expressions used by Chinese and English speakers. We finally show that our complaints intensity scores can be incorporated for better estimating the popularity of posts on social media.

## introduction complaining is caused by the gap between reality and people's expectations @xcite . @xcite state that the purpose of complaining is not to confirm that the two parties have reached an agreement but to face-threatening acts. people use complaints to express their concerns or dissatisfaction based on the severity and urgency of situations. researchers from linguistics and psychology have long pointed out that people may shape their complaints to varying degrees @xcite @xcite . @xcite classifies complaints as conflicting speech acts. mild complaints can reach the purpose of venting emotions to promote mental health, but serious complaints can lead to hatred and even bullying behaviors @xcite . in computational linguistics, prior studies primarily focus on building automatic classification models for identifying the existence of complaints @xcite . most recently, @xcite provided a dataset annotated with different severity levels of complaints based on the theory of pragmatics, including four distinct categories "no explicit reproach", "disapproval", "accusation" and "blame". among these studies, we note one missing piece is to measure the intensity of complaints. to illustrate this point, consider two sentences from the newest annotated dataset from @xcite : can i complain to you about the coffee i just received ? and virgin media as usual full of lies lies lies ! ! !. although these two complaints may have the same type "accusation", it is clear that they are different regarding the degree of complaints. as another example, totally not cool. and please reply my dm asap ! ! ! are both classified as "disapproval". however, the latter makes a stronger complaint. analyzing different complaints levels can also be beneficial. companies need to regularly monitor the feedback from users, as certain complaints may significantly impact the reputation of their products. organizations or governments need to monitor people's biggest complaints to understand their urgent needs. in this work, we analyze the intensity of complaints on social media. to the best of our knowledge, it is the first computational linguistics study that tries to automatically capture the complaints intensity from text. we present the first chinese complaints intensity dataset, consisting of 3,103 posts from weibo. we then show that the complaints intensity can be measured from text by building computational models. we further demonstrate the necessity and importance of understanding complaints intensity. this includes a detailed analysis that distinguishes the differences between our complaint intensity scores and original sentiment scores. as a pilot study for complaints in chinese, we also perform a cross-lingual analysis to understand the differences in the complaint expressions used in chinese and english. we have some interesting empirical findings. for example, we observe that english speakers tend to use more ironic expressions than chinese speakers. finally, we show how our annotated corpus can help predict the popularity of posts on social media. ## data in this section, we present the first chinese dataset that is annotated towards the intensity of the complaints reflected from text. ## predicting the intensity of complaints in section 2, we have a dataset annotated with the intensity of complaints from -1 to 1. we now build computational models for predicting the intensity of complaints of a given post. ## complaints as an emotion from in this section, we demonstrate the necessity of building corpus annotated with complaints intensity, by showing the model trained on standard sentiment datasets fails to do well in our complaints intensity prediction task. we also show that analyzing complaints can be a useful complement for sentiment analysis. ## cross-lingual analysis our newly collected complaints intensity dataset is written in chinese, while current existing datasets (preotiuc-pietro et al., 2019; jin and aletras, 2021) contain english tweets. it provides us an opportunity to understand the linguistic differences for complaints made by chinese and english speakers on social media. our cross-lingual analysis is performed in the following way. first, we evenly sample 200 complaint tweets released in jin and aletras (2021) from their defined four categories. similarly, we divide our annotated data with intensity greater than 0 (as complaint posts) into 4 bins for sampling. we ask 5 in-house annotators to mark and then use majority voting to decide if a post makes a direct or indirect complaint, along with the strategy used to make complaints. direct and indirect complaints. according to @xcite , the speech of complaint can be divided into direct and indirect complaints. in social media, direct complaints are addressed to a complainee who is held responsible for the complaint action, and the addressee is fully or partially responsible for this behavior. here is an example for a direct complaint: 学校的管理怎么这么不人性化？ (why is the management of the school so inhumane?) while indirect complaints refer that the recipient is not primarily responsible for perceived complaints and is more about the evaluation of the target or event, such as: 这样做让我们的压力很大,真无语 (this puts us under a lot of pressure, so speechless). we compare the percentages of direct and indirect complaints for two languages based on our annotations. results show that indirect expressions are more likely to be used in chinese posts (80% are indirect complaints). on the contrary, 91% of english tweets make complaints in a direct way. this finding seems to be consistent with the study of @xcite , which demonstrates that chinese people tend to use indirect expressions. strategy. in pragmatics, strategy is an appropriate countermeasure adopted to achieve the purpose of language communication. @xcite conducted a comparative study of english and german complaints from the direct degree and emotional markers. @xcite discussed the choice of chinese and english complaining strategies and proposed an explicit-implicit strategy. implicit complaints are very subtle and even use metaphors to express complaints about the target, which requires more semantic information to capture. in contrast, explicit complaints can be further divided into types of with-redress and withoutredress. with-redress is the strategy of request for repair; without-redress usually contains complaint targets or objects, which can be easily identified by recognizing an obvious complaint word or phrase. ## predicting post popularity finally, we demonstrate that complaint intensity scores from our computational models can help estimate the post popularity on social media. we envision incorporating these scores into existing social media monitoring systems to improve their prediction accuracy. task. predicting the popularity of content on social media has been extensively studied in literature @xcite @xcite . our task is to predict the popularity of a weibo post. specifically, given the popularity prediction p(t i-1 ) at time t i-1 , we wish to predict the popularity p(t i ) at next time step t i . the popularity p(t) is measured by the number of blog posts under the topic at time t. methods. we follow @xcite and consider the following baseline that only uses early prevalence for prediction: we report our post popularity prediction results in ## related work there have been various studies for complaints in linguistics, economics, and public opinion re6 bao et al. (2013) further proposed to use link density and diffusion depth for popularity prediction. above methods were tested on wise 2012 challenge dataset. we tried our best but are not able to have access to this dataset. in our own collected dataset, we do not have link or diffusion information. ## conclusion in this paper, we present the first study of measuring the intensity of complaints from text. we build a corpus of 3,103 chinese weibo posts about complaints, annotated with complaints intensity scores using best-worst scaling method. we then demonstrate that our corpus supports the development of automatic computational models for accurate complaints intensity predictions. furthermore, we study the connections between complaints and sentiment, and perform a cross-lingual comparison for complaints expressions between chinese and english. we finally show that our complaints intensity scores help better estimate the posts popularity on social media.