Aspect-Based Sentiment Analysis for Open-Ended HR Survey Responses (2024)

Abstract: Understanding preferences, opinions, and sentiment of the workforce is paramount for effective employee lifecycle management. Open-ended survey responses serve as a valuable source of information. This paper proposes a machine learning approach for aspect-based sentiment analysis (ABSA) of Dutch open-ended responses in employee satisfaction surveys. Our approach aims to overcome the inherent noise and variability in these responses, enabling a comprehensive analysis of sentiments that can support employee lifecycle management. Through response clustering we identify six key aspects (salary, schedule, contact, communication, personal attention, agreements), which we validate by domain experts. We compile a dataset of 1,458 Dutch survey responses, revealing label imbalance in aspects and sentiments. We propose few-shot approaches for ABSA based on Dutch BERT models, and compare them against bag-of-words and zero-shot baselines.Our work significantly contributes to the field of ABSA by demonstrating the first successful application of Dutch pre-trained language models to aspect-based sentiment analysis in the domain of human resources (HR).

## introduction understanding employees' preferences and opinions can be of paramount importance in the full employee life cycle, e.g., from recruitment and selection to employee retention, and performance and career management @xcite . in employee satisfaction surveys, open-ended questions may elicit a wide range of aspects. however, conducting large-scale analysis of these responses is challenging because of their user-generated nature. aspect-based sentiment analysis (absa) is the task of identifying and extracting sentiments toward specific aspects from free text, allowing a * work done while on internship at randstad groep nederland more detailed analysis of opinions @xcite , which means they can be a valuable tool for identifying specific areas of (dis)satisfaction. despite the promise of absa, limited research has explored its application beyond english @xcite . moreover, to our knowledge, absa has not been studied in employee satisfaction surveys. this study aims to bridge this research gap by studying absa on dutch open-ended employee satisfaction survey responses. the central research question that we answer in this paper is: rq1 how effective is a bert-based machine learning model in extracting aspect-sentiments from dutch open-ended responses of employee satisfaction surveys? to answer this question, we first aim to answer the following sub-questions: rq1.1 how does the performance of few-shot classification using dutch bert models compare to bag-of-words based baselines in absa? rq1.2 how does the performance of few-shot classification using dutch bert models compare to a zero-shot classification baseline in absa? ## related work this section provides an overview of previous studies on aspect and sentiment classification, absa, and the dutch bert models we employ in this paper: bertje and robbert. ## methodology this section outlines the development and evaluation of the proposed absa model for dutch openended survey responses on employee satisfaction. ## results in this section, we present the results of the experiments. we compare performance of bertje f ewshot and robbert f ewshot to our traditional baselines (sv m and m lp ), and zero-shot bert baselines (bertje zeroshot and robbert zeroshot ). in addition, we apply data augmentation for both (bertje f ewshotda and robbert f ewshotda ). ## discussion this study explores absa in dutch employee satisfaction surveys, using dutch bert-based machine learning models. our findings are in line with findings in prior research @xcite that highlight bert's effectiveness for absa in english. consistent with @xcite , robbert outperforms bertje, indicating the superiority of larger transformer models for sentiment classification, also observed by @xcite . additionally, our study underscores the success of few-shot classification in addressing limited labelled data, consistent with @xcite for aspect classification and @xcite for sentiment classification in english. however, our findings contradict successful application of bert models for zero-shot classification by @xcite . this discrepancy in performance can be attributed to our domain-specific data, in contrast to the diverse dataset used by @xcite . furthermore, this study identified significant label imbalance in aspects and their associated sentiments, as detailed in section 4. to address this, we explored data augmentation using nlpaug @xcite following @xcite . however, this technique improved the macro f1 score for robbert in aspect classification only. ## conclusion analyzing workforce opinions and preferences through aspect-based sentiment analysis has various hr applications. in this paper, we demonstrate the effectiveness of dutch bert models, bertje and robbert, in a few-shot absa experiment using dutch open-ended responses from employee satisfaction surveys. we address three sub-questions to gain insights into the models' performance and potential enhancements for aspectbased sentiment analysis (absa). regarding the first two sub-questions (rq1.1 and rq1.2), few-shot transformer models outperform baseline bow models, significantly improving aspect-sentiment classification measured by macro f1 score. this emphasizes the importance of labeled data for fine-tuning bert models, enhancing performance compared to relying solely on pre-trained knowledge in zero-shot scenarios. it also highlights bow models' superior performance in leveraging individual words compared to zero-shot models struggling with domain transfer. regarding the third sub-research question (rq1.3), this study demonstrates how data augmentation can enhance robbert's aspect classification performance. however, data augmentation does not improve aspect or sentiment classification for bertje or sentiment classification for rob-bert. these findings suggest that the effectiveness of data augmentation varies across models and tasks. in summary, regarding the main research question (rq1) on bert-based models' effectiveness in extracting aspect-sentiments, our findings demonstrate their superiority over traditional bagof-words models and zero-shot classification approaches. to enhance model robustness, future studies should acquire larger and more diverse absa datasets, exposing models to varied openended survey responses for improved generalization to novel data. considering the challenges posed by the limitations of traditional clustering methods, future studies could explore the incorporation of other clustering methods such as fuzzy clustering @xcite or semi-supervised (neural) topic modeling approaches @xcite . moreover, future studies should investigate the disparity in data augmentation success between bertje and robbert. specifically, exploring whether generating augmented sentences using bertje embeddings im-proves bertje model performance, similar to the favorable outcome observed for robbert in this study. understanding such disparities would contribute to a deeper understanding of the relationship between specific models, their embeddings, and the efficiency of data augmentation techniques in absa's broader context.