S ent EMO : A Multilingual Adaptive Platform for Aspect-based Sentiment and Emotion Analysis (2022)

Abstract: In this paper, we present the SentEMO platform, a tool that provides aspect-based sentiment analysis and emotion detection of unstructured text data such as reviews, emails and customer care conversations. Currently, models have been trained for five domains and one general domain and are implemented in a pipeline approach, where the output of one model serves as the input for the next. The results are presented in three dashboards, allowing companies to gain more insights into what stakeholders think of their products and services. The SentEMO platform is available at https://sentemo.ugent.be

## introduction in the sentemo project, we aim to develop a finegrained sentiment analysis and emotion detection system for four languages (dutch, english, french and german). fine-grained sentiment and emotion detection is very interesting for every company or non-profit organization having user data at its disposal. the results of such a system not only provide insights into what the various stakeholders think of specific products or services, but can also be used to analyse sentiment at the company level and thus provide input for employer branding. we aim to meet companies' needs for automation when sentiment analysis is done manually or by using lexicons. with the dashboard, we furthermore want to offer an insightful alternative to black-box sentiment approaches by visualizing results at the aspect level. the aim is to design a fully data-based and adaptable system: companies will be able to improve and fine-tune the output on their own data, and then retrain the system based on that corrected data. thanks to this feedback loop, the system will be continuously customized to company-specific data and the quality will keep on improving. on the one hand, the user interface has an intuitive dashboard that provides a clear representation of the sentiment and emotion detection results, on the other hand, it will also have the functionality to label or correct data and easily retrain the system. in this paper, we present the first prototype of our system, that includes an aspect-based sentiment analysis (absa) and aspect-based emotion analysis (abea) module for dutch. first, we briefly introduce the task of aspect-based sentiment analysis and emotion detection. next, we elaborate on the data we used and the annotation process. in section 4, the experimental set-up and results of the models are discussed. section 5 and 6 cover details of the user interface. finally, we give an outlook of the next steps of the project in section 7. ## aspect-based sentiment and emotion analysis aspect-based sentiment analysis or absa @xcite not only aims at the detection of all sentiment expressions within a given document, but also detects the concepts and aspects (or features) to which they refer. absa is generally decomposed into three subtasks: (1) aspect term extraction, (2) aspect category classification, and (3) aspect polarity classification. we provide more insights into each step in section 4. sometimes it does not suffice to report on a polarity level and it could be useful to know what specific emotions stakeholders experience (e.g. anger, sadness, joy,...) @xcite . especially within customer relation management, it is valuable to detect strong emotions timely to provide an appropriate response. in order to predict emotions on a fine-grained level, we build on the results from the aspect-based sentiment analysis component and provide an additional emotion layer to the predicted positive or negative sentiment. ## data and annotation since the sentemo project is a collaboration with eight belgian companies, we envisaged to collect both in-house data and proprietary user data coming from those project partners. in total, these efforts resulted in data sets covering six different domains: fmcg 2 (non-durable products which are often bought by consumers, e.g. cleaning products, food and self-care products), airline, hotel, product retail, hospital and telecom. regarding the in-house dutch data, 1,000 reviews were each time scraped from bol.com, trustpilot and tripadvisor for the domains fmcg, airline and hotel, respectively. for the other domains, data was received from the project partners. after some basic data cleaning where duplicates and instances written in languages other than dutch were removed, we ended up with data sets consisting of at least 900 instances per domain. in a next step, the data had to be manually enriched or annotated with absa and abea information in order to be able to train and evaluate machine learning systems. annotation consisted of four steps (see figure 1 for an illustration). first, the aspect terms had to be identified in the sentences (e.g. kamer (english: room) in figure 1 ). next, an aspect category corresponding to an entity-attribute pair 3 (e.g. room_style in figure 1 ) was selected. subsequently, the annotator selected the sentiment words (e.g. prachtige (english: beautiful)) and assigned a corresponding sentiment or polarity (positive). we annotated five possible polarities: very positive, positive, neutral, negative and very negative. the sentiments very positive and very negative are only chosen when an intensifier is explicitly present in the text (e.g. very friendly). in a second annotation round, an emotion was added to the aspect term. the annotators could choose from a list of 12 emotions: anger, anticipation, disgust, dissatisfaction, distrust, fear, joy, neutral, sadness, satisfaction, surprise and trust. neutral was only to be used when the sentiment was also tagged as neutral. for the selection of the emotion labels, we based ourselves on plutchik's wheel of emotions @xcite . we started with anger, anticipation, disgust, fear, joy, sadness, surprise and trust and added satisfaction and dissatisfaction for statements with a softer emotion. after testing these emotions on 10 sentences per do-2 fast-moving consumer goods 3 see section 3.1 for more information. main, we also added distrust as a negative opposite for trust. when the writer voiced an opinion about an aspect without explicitly mentioning it, a null annotation was created, which, as illustrated by figure 2 , included the appropriate aspect category (e.g. personnel_friendliness), polarity (e.g. very positive) and emotion (e.g. satisfaction). ## model development once all data were annotated, they were preprocessed and experimental data splits were created in order to experiment with a variety of machine learning algorithms including both feature-based and deep learning approaches. in this section we report on the best approach for each absa and abea sub-task. much work has already been carried out for each task separately, e.g. @xcite for aspect term extraction, @xcite for aspect category classification, @xcite for sentiment classification and padme and kulkarni (2018) for emotion classification. approaches with multi-task learning usually only cover two of the tasks, very often aspect term extraction and sentiment classification (akhtar et al., 2020) or aspect term extraction and aspect category classification @xcite . we opted for a pipeline approach in which we combine a feature-based approach for the first two absa sub-tasks (aspect term extraction and aspect category classification) with a transformer-based architecture for the polarity classification and emotion detection. while we also used transformer-based approaches to tackle the first two sub-tasks, we observed better results using a feature-engineered approach with crf and svm classifiers. note that for each sub-task, results are reported with the gold standard input from the previous task, meaning that potential error percolation from previous steps is not yet taken into account. ## demonstration of the interactive dashboard users can access the sentemo dashboard with their login details via the url sentemo.ugent.be 10 . after logging in, users can upload data to be analysed or look at the analysis of previously uploaded 10 at this moment, a login can only be obtained through one of the members of the sentemo research team data. manage documents gives an overview of the files that have been uploaded. the status indicates whether a file is being processed, is ready or failed. users can drag and drop csv files, which contain the domain in the first column and text in the second column. as soon as the status is set to ready, the results are available in the dashboard. on the analyse texts page, a distinction is made between the results for sentiment and emotion analysis. on the sentiment analysis page, users can see details about the aspect category and polarity classification. the emotion dashboard focuses on emotion classification, but the aspect categories can be used as filters. ## technical implementation the sentemo platform consists of two separate applications: a front-end and a back-end. the front office is a full-stack web application for both users and administrators and is responsible for user management, document management and data visualisation. the back-end, on the other hand, is responsible for text processing and machine learning. both applications are self-contained and hosted on different servers within the same local network. each application can be replicated and/or customised independently as per use case requirements. the data processing workflow is as follows: first, the user uploads a csv file with texts.the csv file is parsed, and the extracted data is stored into a relational database (postgresql foot_5 ). next, a json object with the data is generated and sent to a message queue (rabbitmq foot_6 ). this message queue is read out by the sentemo back-end at predefined intervals. the data is processed by the sentemo bac-kend, and a response with the results is sent as a json object to a second message queue. the sen-temo front office reads this response and stores the data in the relational database. finally, the user is notified that the document has been processed and that data visualisation is now available for the uploaded document. the sentemo front office is built with docker containers foot_7 (as shown by figure 10 ): a custom node.js foot_8 application container, a postgresql relational database container, and a rabbitmq message queue container. this setup is hardware and operating system agnostic, making it easy to deploy on windows, macos, or linux (ubuntu server), regardless of cpu architecture. it can even be run on a raspberry pi 4 model b if needed. a reverse proxy (apache http server) is used to connect the application to the internet. the technology stack of the sentemo consists of a node.js application written in typescript and built with the react 15 framework next.js 16 extended with blitz 17 for session management, security and communication between client-side and server-side. blitz uses a 'zero api' approach that takes care of api calls without a developer needing to explicitly program an api. this approach speeds up the development greatly but requires developers to be aware of where the code needs to be executed, as both client-side and server-side code can live within the same file and will work regardless. prisma 18 is used for object-relational mapping. data visualisation is done with d3 19 to generate interactive svg based charts. tailwind 15 https://reactjs.org/ 16 https://nextjs.org/ 17 https://blitzjs.com/ 18 https://www.prisma.io/ 19 https://d3js.org/ css foot_9 is used as a utility-first css framework and used in conjunction with the bem methodology foot_10 . atomic design foot_11 is used to organise react components. a page is made up using layouts, organisms, molecules and atoms. atoms are the most basic components and organisms are the most complex components, defining major parts of a page. ## future work next steps of the project include adding extra languages to the platform. in the end, models should be available to analyse english, french and german data. for each language, similar data sets will be annotated. the methodologies used are language-independent, as the features used for aspect term extraction and aspect category classification can be applied to other languages. finally, bert-models are available for english, french and german, which allows us to adapt the third and fourth sub-task to these languages as well. on top of that, we want to allow users to indicate what predictions are wrong via an easy-to-use annotation interface, suggest corrections and eventually retrain the models.