[image] [image] Ressource protégée - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu

[image] [image]

* Documentation

* FR
  Français (FR)

* Anglais (EN)
  Se connecter

* Portail HAL Inria

* Recherche
  Loading...

## Recherche avancée

Information de documents

Titres

○ Titres

○ Sous-titre

○ Titre de l'ouvrage

○ Titre du volume (Série)

○ Champ de recherche par défaut (multicritères) + texte intégral des PDF

○ Résumé

○ Texte intégral indexé des documents PDF

○ Mots-clés

○ Type de document

○ Sous-type de document

○ Tous les identifiants du document

○ Identifiant HAL du dépôt

○ Langue du document (texte)

○ Pays (Texte)

○ Ville

○ À paraître (true ou false)

Ajouter
# Using NLP to Detect Tradeoffs in Employee Reviews

Justus Schollmeyer

► To cite this version:

Justus Schollmeyer. Using NLP to Detect Tradeoffs in Employee Reviews. 22th International TRIZ Future Conference (TFC), Sep 2022, Warsaw, Poland. pp.209-219, 10.1007/978-3-031-17288-5_19.hal-04449786

HAL Id: hal-04449786

https://inria.hal.science/hal-04449786v1

Submitted on 15 Feb 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
## Auteur

### Auteur (multicritères)

○ Auteur (multicritères)

○ Auteur : Nom complet

○ Auteur : Nom de famille

○ Auteur : Prénom

○ Auteur : Complément de nom, deuxième prénom

○ Auteur : Organisme payeur

○ Auteur : IdHal (chaîne de caractères)

○ Auteur : Fonction

○ Auteur : personID (entier)

○ Auteur : Identifiant de l'organisme payeur

○ Auteur : Identifiant de la structure

○ Directeur de thèse

○ Éditeur

○ Éditeur scientifique

○ Éditeur de la série

## Ajouter

### Structure

#### Structure (multicritères)

○ Structure (multicritères)

○ Structure : Acronyme

○ Structure : Nom

○ Structure : Code

○ Structure : Pays

○ Structure : Type

○ Structure : État dans le référentiel

○ Structure : Identifiant HAL de la structure

○ Structure/équipe de recherche : Pays

○ Structure/regroupement d'équipes : Pays

○ Structure/laboratoire : Pays

○ Structure/regroupement de laboratoires : Pays

○ Structure/institution : Pays

○ Structure/regroupement d'institutions : Pays

## Ajouter
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
Autres

Champ de recherche par défaut (multicritères)

○ <u>Champ de recherche par défaut (multicritères)</u>

○

○ <u>Revues (multicritères)</u>

○ <u>Revue : Éditeur</u>

○ <u>Revue : Titre abrégé</u>

○ <u>Revue : Titre</u>

○ <u>Revue : Identifiant interne</u>

○ <u>Revue : Date de début de publication</u>

○ <u>Revue : ISSN électronique</u>

○ <u>Revue : ISSN</u>

○ <u>Revue : Éditeur</u>

○ <u>Revue : Couleur dans SHERPA/RoMEO</u>

○ <u>Revue : État dans le référentiel</u>

○

○ <u>Colloque (multicritères)</u>

○ <u>Colloque : Titre</u>

○ <u>Colloque : Organisateur</u>

○ <u>Colloque : date de début (Année)</u>

○ <u>Colloque : date de fin (Année)</u>

○ <u>Conférence invitée (oui/non)</u>

○

○ <u>Projets ANR (multicritères)</u>

○ <u>Projet ANR : Acronyme</u>

○ <u>Projet ANR : Acronyme du programme</u>

○ <u>Projet ANR : Code décision (référence)</u>

○ <u>Projet ANR : Nom</u>

○ <u>Projet ANR : Identifiant interne</u>

○ <u>Projet ANR : État dans le référentiel</u>

○

○ <u>Projets européens (multicritères)</u>

○ <u>Projet européen : Acronyme</u>

○ <u>Projet européen : Identifiant de l'appel à projet</u>

○ <u>Projet européen : Référence</u>

○ <u>Projet européen : Nom</u>

○ <u>Projet européen : Date de fin</u>

○ <u>Projet européen : Financement</u>

○ <u>Projet européen : Date de début</u>
# Using NLP to Detect Tradeoffs in Employee Reviews

Justus Schollmeyer¹

¹ University of Bremen, 28359 Bremen, Germany
justusschollmeyer@gmail.com

**Abstract.** This paper presents a methodology to identify industry-specific tradeoffs using natural language processing (NLP) to analyze employee reviews from Glassdoor. The analysis is based on 400,000+ reviews from employees working in the financial sector between 2008 and 2020. For each review, the pros and cons sections are classified in a one-to-many approach in terms of the most prevalent topics in the sector. The most prevalent noun chunks within a representative sample of reviews are used as topics. The classification of reviews is based on the cosine similarity between the sentence embeddings of these topics and the sentence embeddings of the comment sections using a sentence-transformer model. Based on this classification, the count of pro-con topic pairs is tested for statistical significance against a control of randomly generated pairings from the same sample of classifications. The process is repeated 10,000 times and only pairs with a p-value < .05 (after Bonferroni-Holm correction) are considered. If both, a pro-con pair of topics and its opposite pairing are significantly frequent, this combination of topics qualifies as a tradeoff. Depending on whether the same pairing constitutes a significantly frequent pro-con pair, it can be determined whether a solution for the tradeoff exists in the sector (at least within the limitations of this approach). Using 13 topic labels, 2 tradeoffs – *work-life balance* vs. *opportunity* (without solution in the sector) and *work-life balance* vs. *management* (with solution in the sector) – have been identified. In addition to that, 8 more pro-con pairs were identified as significant.

**Keywords:** Tradeoffs, Glassdoor, Employee reviews, Natural Language Processing, Machine Learning.

## 1 Introduction

The goal of this paper is to demonstrate that the free text in employee reviews from online platforms such as Glassdoor [1] or Indeed [2] can be used to identify sector-specific tradeoffs that organizations face from the perspective of their employees. For this purpose, 439,599 Glassdoor reviews written between 2008 and 2020 by employees from the financial industry in the US will be processed using NLP.

The paper starts with a discussion of why tradeoffs matter to innovation (2.1) followed by brief introductions to employee reviews on Glassdoor (2.2) and the use of NLP for their analysis (2.3). Section 3 will introduce the method of using the pros and cons sections to identify significantly frequent pro-con pairs and their opposites.
○ Projet européen : État dans le référentiel

○ Projet européen : Identifiant interne

○

○ Date de production : année

○ Date de mise en ligne : année

○ Date de publication : année

○ Date d'écriture : année

○ Date de modification du dépôt : année

○ Date de dépôt : année

○ Date de publication électronique : année

○

○ Collection HAL (multicritères)

○ Collection HAL : catégorie

○ Collection HAL : Code

○ Collection HAL : Nom

○ Collection HAL : Identifiant interne

○

○ Identifiant interne du contributeur/déposant

○ Nom complet du contributeur/déposant

○

○ Domaines

○ Domaine primaire

○ Domaine racine

○ Sous-domaine niveau 1

○ Sous-domaine niveau 2

○ Sous-domaine niveau 3

○

○ Statut du document

○ Version du document

○ Type de dépôt

○ Type de document

○ ISBN

○ Numéro - référence

○ Identifiant DOI

○

○ Classification

○ Audience

○ Vulgarisation

○ Comité de lecture - texte (oui ou non)

○ Actes de colloque
Section 4 presents the results followed by a brief discussion of the findings and how to improve the approach further (section 5).

# 2 Background

## 2.1 Why tradeoffs matter to innovation

Tradeoffs are problems with conflicting targets. Improving one results in the deterioration of the other and vice versa. Problems that present themselves in this form bear high potential for innovation [3]. This is because to improve the values for both conflicting parameters, it is required to rethink the structure of the system (or some aspects of it).

A simple example: A container to deposit trash is 300m away from someone's house. The trash gets collected in a bin in the kitchen. The larger the bin, the fewer meters must be walked to deposit the trash over a given period. Increasing the bin size comes at the cost of a heavier bin once filled with trash. "Meters walked to the container in a month" can be seen as a negatively sloped function of the filled bin's mass if everything else remains stable (see left side of Fig.1). The function defines the options if only the size of the bin can be changed.

If, however, someone decided to go beyond these artificial limitations, they might come up with a solution that improves both parameters to the better, which can come in two forms: The system is modified such that

(i) the entire function shifts to the better for both parameters, or

(ii) the tradeoff disappears entirely.

Type (i) is a more general version of Schumpeter's definition of innovation: a shift of the production function [4]. Type (ii) is even preferable (at least as far as these parameters are concerned) and lies on the vector of the ideal solution defined in [3] as *function without a system*.

For both types of solutions, creative thinking is required since mental limitations – such as only allowing ourselves to change the size of the bin – must be abandoned. One could e.g., decide to produce less trash. This would shift the function to the better in the sense of (i) (see right side of Fig.1). One could also decide to integrate the walk to the container into one's daily routines, such as going to work, grocery shopping, jogging, etc. The problem becomes obsolete (ii) since no extra time is being spent on walking to the container if it is on the way of other errands. Unsurprisingly, most people apply this strategy to deal with the tradeoff.
○ Référence interne

○ Financement

○ Collaborations

Ajouter

Lancer la recherche
Recherche experte (SolR)

Recherche experte (SolR)

Lancer la recherche

Vers la recherche avancée

• Déposer

Portail INRIA.HAL.SCIENCE
Les publications, logiciels... des scientifiques Inria

• Accueil

• Publications Inria

○ Consulter les derniers dépôts

○ Par équipe de recherche

○ Par auteur

○ Rapports de recherche et techniques

○ Livres blancs

○ Rapports d'activité des équipes

○ Statistiques

• Boite à Outils

○ X2Hal : importer par lot

○ Logiciels : créer le Codemeta.json

○ Créer mon IDHal

○ Créer mon CVHal

○ Haltools : créer ma page de publications

○ Haltools : extraire des données (réservé Inria)

○ Les bonnes pratiques

■ Comment bien s'affilier ?
<figure><img src="image_3.png" /><figcaption>**Fig. 1.** The left side figure shows the meters walked per month as a function of the mass of the bin. The graph assumes that other variables such as type of trash, volume of trash, distance to the container, etc. remain stable. The graph to the right shows how the curve changes if the volume of trash is cut in half.</figcaption></figure>

We cannot know for certain that a tradeoff is not resolvable in principle (because we cannot be certain that all our assumptions are true and exhaustive). However, we can check whether a solution to what looks like a tradeoff already exists. If none exists and we find a type (i) or (ii) solution, we have made a discovery or invention that could result in innovation if implemented at scale [4].

The approach of analyzing employee reviews from the financial sector presented in this paper will show how to distinguish tradeoffs that have a solution in the sector from those that seem to have no solution yet.

## 2.2 Employee reviews on Glassdoor

Glassdoor [1] is a website that allows current and former employees to review their companies by leaving ratings for *career opportunities, compensation & benefits, work-life balance, senior management, and culture & values*. In addition to that, employees can decide whether they approve of the CEO, whether they would recommend the company to a friend, and how they feel about the business outlook for the next six months. Finally, they are asked to write about the pros and the cons and to provide feedback to management. The analysis in this paper makes use of the pros and cons structure in the free text section. For a detailed description of the method see the methods section (section 3) and the code on GitHub [5].

## 2.3 NLP for analyzing Glassdoor reviews

Recent advancements in natural language processing (NLP) have proven useful in *human resource management* allowing for the analysis of qualitative data without the need for laboriously coding text into categories [6]. Both, bag-of-words approaches (e.g., [7]) and approaches based on vector-representations (e.g., [8]) have been used successfully on Glassdoor data. This article relies on both of them: a bag-of-words approach combined with part of speech tagging to identify the most prevalent noun chunks that will serve as categories, and sentence-embeddings to compute the cosine-
■ Quelle version puis-je déposer ?

■ Choisir le bon « type de document »

■ Mettre en valeur ses publications

○ Les tutos IES Inria

■ Lier les notices dans HAL

■ Umbrhal Inria pour l'accessibilité

■ Personnaliser son CVHAL

■ Créer et Personnaliser sa collection

• Documentation

○ Aide en ligne HAL

○ Aide en ligne Haltools

○ Aide en ligne X2hal

○ Tutoriels video HAL

○ Documentation API HAL

○ Ajouter des vignettes

○ Identifiant auteur IdHAL et CV

○ Compléter son dossier ORCID avec ses dépôts HAL

○ Inria et la Science Ouverte

X

X

X

Loading...

X

Le document que vous demandez est en access restreint
jusqu'au 2025-01-01 !

Revenir a la notice

Retourner à l'accueil

Contact

•

Ressources
similarity between the categories and the comments using a sentence-transformer model [9], which maps paragraphs and sentences to a 768 dimensional dense vector space [10].<fnref n="1" />

# 3 Methods

For more details see the implementation of the approach on GitHub [5].

## 3.1 The data

The analysis is based on 439,599 comments from the financial sector featuring 12,868 companies² submitted between April 23rd, 2008, and September 8th, 2020, to Glassdoor.

## 3.2 The labels

To identify the most prevalent categories, 4,000 comments have been sampled from the *pros* and the *cons* sections, respectively. The resulting 8,000 comments were used to identify the 15 most prevalent noun chunks using spaCy [11] (ignoring stop words and removing adjectives by means of spaCy's parts of speech tags). Their number has been further reduced by removing semantic duplicates. Duplicates were determined in terms of the cosine similarity between the categories' semantic vector representations. The vector representations were obtained by means of the *paraphrase-distilroberta-base-v1* transformer model from the Huggingface library [10]. Categories with cosine similarities above .7 were considered as duplicates. To cluster the similarity scores, the similarity matrix is transformed to a distance matrix (i.e., a matrix of dissimilarity scores). Similar topics are identified by means of hierarchical clustering and a dissimilarity threshold of .3, which corresponds to a similarity score of .7. Among such semantic duplicates, the more frequent category was kept unless it contained lemmata appearing in other topics. In the case of the latter, the less frequent category was kept. For example, the topics “work” and “job” are duplicates according to this definition. “Work” is more frequent than “job”. However, “work” also appears in the topic “work life balance”. “Job” is thus kept and “work” discarded. This procedure reduced the 15 categories down to 13.

<footnote n="1">¹ This model is also used to reduce the number of categories by removing semantic duplicates (see section 3).

² The companies comprise organizations from the following industries: Financial Transaction Processing, Investment Banking & Asset Management, Banks & Credit Unions, Lending, Brokerage Services, Financial Analytics & Research, Stock Exchanges, Venture Capital & Private Equity.</footnote>
* Documentation

* FAQ

* API

* OAI-PMH

* AuréHAL

## Informations

* À propos

* Données personnelles

* Mentions légales

* Accessibilité

* Conformité RGAA

## Questions juridiques

* Je publie, quels sont mes droits ?

* Loi pour une République numérique

* Stratégie de non-cession des droits

* SHERPA/RoMEO

## Portails

* Portails institutionnels

* HAL

* HAL SHS

* HAL Thèses

* MédiHAL

## CCSD

* CCSD

* Episciences

* Sciencesconf
## 3.3 Mapping comments to labels

To classify the comments, the same model used for label reduction (see 2.2) is used to generate vector representations [10]: The vector representation of each comment is compared to the vector representation of each label in terms of their cosine similarity. If the similarity score is larger than .35, the label is assigned to the comment.

Three more labels are added to the existing list: *nothing negative to say*, *nothing positive to say*, *none*. Their only role is to identify comments that are to be removed because they do not carry relevant information for the purpose of identifying tradeoffs.

## 3.4 Identifying tradeoffs

The general idea behind the methodology to identify tradeoffs is described in Fig. 2. Each review has a *pros* and a *cons* section. These sections are assigned to category names (i.e., labels) in a one-to-many approach (see 2.2 and 2.3). The procedure results in a list of pro-con pairs (the maximal count of pairs is equal to the number of labels squared; in this case, 169 pairs). Pairs with the same label in the pros and the cons section are discarded (leaving maximally 156 pairs).

Statistical significance of the count of pairs is tested against a control obtained by randomly reshuffling the pairings 10,000 times. A pro-con pair counts as significant, if the Holm-Bonferroni adjusted p-value [12] is below < .05 when compared to the randomly created distribution.<fnref n="3" /> If both, a pro-con pair and its opposite meet the significance test, the combination qualifies as a tradeoff (see Fig. 2).

<footnote n="3">³ The more pairs tested for statistical significance, the higher the probability of obtaining type I errors (false positives). To control for this without unnecessarily increasing the risk of type II errors (false negatives), the Holm-Bonferroni method is used to adjust the family-wise error rates (FWER) for each pair. The p-values are sorted and ranked from lowest to highest starting with rank 0. For each pair that is to be tested, the FWER (in this case .05) is then divided by the number of tests (in this case 156) minus the pair's rank. This results in an array of adjusted FWERs. If an adjusted FWER is smaller than its predecessor, it is set equal to the rate of the previous pair. A pair is significant if its p-value is lower than its adjusted FWER.</footnote>
<figure><img src="image_4.png" /><figcaption>**Fig. 2.** The pros and cons sections of employee reviews are classified in terms of categories (labels) relevant to the sector. Only pairs of labels that are significantly frequent qualify as pro-con pairs. If the opposite of the pro-con pair constitutes another significantly frequent pro-con pair, the combination counts as a tradeoff.</figcaption></figure>

To evaluate whether a sector has a solution for a tradeoff, the same procedure is carried out for the labels within the pros and the cons sections, respectively. The goal is to check whether the *either-or* combination also exists in the form of *both-both*. For example, if significantly many reviewers report *good work-life balance* and *bad management*, and another group reports the opposite, it could still be the case that another group of reviewers reports both *good work-life balance* and *good management*. The latter is called a *pro-pro pair*. Tradeoffs whose labels also appear significantly often as pro-pro pairs count as tradeoffs that have a solution within the sector. In contrast, if a pro-pro combination of the labels constituting a tradeoff is significantly infrequent, it can be assumed to have no obvious solution within the sector – at least not within the constraints of this approach (see Fig. 3).

<figure><img src="image_5.png" /></figure>
**Fig. 3.** Illustration of the decision logic explaining the difference between tradeoffs with a solution in the sector and tradeoffs without such a solution (in the boundaries of this analysis).

# 4 Results

## 4.1 Tradeoffs

Filtering for the most prevalent nouns (noun chunks included) results in the following list of 13 categories used as labels: *company*, *people*, *management*, *employee*, *benefit*, *time*, *job*, *pay*, *opportunity*, *culture*, *hour*, *training*, and *work life balance*.

Running the significance test on the total counts of all pairings reveals two tradeoffs (see Table 1): *management vs. work life balance* (870, *P*<.0001 and 118, *P*<.0001), and *opportunity vs. work life balance* (47, *P*<.0001 and 54, *P*<.0001). The labels *management & work-life balance* appear significantly frequently together in the pros section as a *pro-pro* pair, which suggests that a solution exists within the sector (145, *P*<.0001). In contrast, the pair *opportunity & work life balance* is significantly infrequent (11, *P*<.0001) and thus has no obvious solution in the sector (at least not within the boundaries of this analysis).

<table><caption>**Table 1.** Summary of the tradeoffs.</caption><thead><tr><th>con</th><th>pro</th><th>count</th><th>p-val<br>(adj.)</th><th>cons<br>pairing</th><th>pros<br>pairing</th></tr></thead><tbody><tr><td>management</td><td>work life balance</td><td>870</td><td>0</td><td rowspan="2">insignificant</td><td rowspan="2">frequent</td></tr><tr><td>work life balance</td><td>management</td><td>118</td><td>0</td></tr><tr><td>opportunity</td><td>work life balance</td><td>47</td><td>0</td><td rowspan="2">infrequent</td><td rowspan="2">infrequent</td></tr><tr><td>work life balance</td><td>opportunity</td><td>54</td><td>0</td></tr></tbody></table>

For an illustration of the tradeoff ratios within firms, see Fig. 4.

<figure><img src="image_6.png" /></figure>

<figure><img src="image_7.png" /></figure>
**Fig. 4.** Illustration of the ratios between the two sides of the respective tradeoffs within companies. The y-axis (logarithmic scale) shows the number of companies that fall into the respective ratio bins.

For examples of reviews illustrating the tradeoffs, see table 2.

<table><caption>**Table 2.** Examples of reviews illustrating the tradeoffs.</caption><thead><tr><th>label cons</th><th>label pros</th><th>Cons</th><th>pros</th></tr></thead><tbody><tr><td rowspan="2">work life<br>balance</td><td rowspan="2">opportunity</td><td>Not the best work life<br>balance.</td><td>Opportunity to make a<br>lot of money</td></tr><tr><td>Work/Life balance could<br>be better</td><td>Opportunities for growth<br>and advancement</td></tr><tr><td rowspan="3">opportunity</td><td rowspan="3">work life<br>balance</td><td>opportunity for growth,<br>financial compensation,<br>location</td><td>work-life balance, job<br>security, culture, tuition<br>reimbursement</td></tr><tr><td>Opportunity for<br>advancement and lack of<br>growth</td><td>Family oriented,<br>balanced life, encourage<br>learning</td></tr><tr><td>Long hours and not<br>enough work life<br>balance</td><td>Great people and<br>management</td></tr><tr><td rowspan="2">work life<br>balance</td><td rowspan="2">management</td><td>Work / life balance quite<br>difficult</td><td>Excellent management<br>at all levels</td></tr><tr><td>Sometimes management<br>is unorganized</td><td>The work/Life balance is<br>good, they understand<br>that you have many<br>things going on with<br>your life</td></tr><tr><td>management</td><td>work life<br>balance</td><td>management problems<br>at the middle levels</td><td>culture, benefits,<br>compensation, work life<br>balance</td></tr></tbody></table>

## 4.2 Frequent pro-con pairings

In addition to the 4 pairs constituting the 2 tradeoffs, 8 more pairs have been identified as significantly frequent (see Table 2). None of them constitutes a significantly frequent pro-pro pair except for con: work life balance & pro: culture. Examples can be found in table 4.
<table><caption>**Table 2.** Summary of frequent pro-con pairs.</caption><thead><tr><th>con</th><th>pro</th><th>count</th><th>p-val<br>(adj.)</th><th>cons<br>pairing</th><th>pros<br>pairing</th></tr></thead><tbody><tr><td rowspan="4">hour</td><td>culture</td><td>86</td><td>0</td><td>insignificant</td><td>infrequent</td></tr><tr><td>management</td><td>132</td><td>0</td><td>infrequent</td><td>infrequent</td></tr><tr><td>pay</td><td>177</td><td>0</td><td>infrequent</td><td>infrequent</td></tr><tr><td>training</td><td>94</td><td>0</td><td>infrequent</td><td>insignificant</td></tr><tr><td>management</td><td>benefit</td><td>523</td><td>0.0001</td><td>infrequent</td><td>infrequent</td></tr><tr><td>pay</td><td>work life balance</td><td>280</td><td>0</td><td>infrequent</td><td>infrequent</td></tr><tr><td rowspan="2">work life balance</td><td>culture</td><td>77</td><td>0.0001</td><td>infrequent</td><td>frequent</td></tr><tr><td>training</td><td>85</td><td>0</td><td>infrequent</td><td>insignificant</td></tr></tbody></table>

<table><caption>**Table 4.** Examples of reviews illustrating significantly frequent pro-con pairs.</caption><thead><tr><th>label cons</th><th>label pros</th><th>cons</th><th>pros</th></tr></thead><tbody><tr><td rowspan="3">work life<br>balance</td><td rowspan="2">training</td><td>work/life balance. A lot of<br>information to be<br>responsible for</td><td>Good benefits and<br>good training</td></tr><tr><td>No work life balance and<br>promise big bonuses but do<br>everything they can to cut<br>bonuses and salary<br>increases at year end.</td><td>Gives solid training<br>when starting out.</td></tr><tr><td>culture</td><td>Work life balance not the<br>best. Needs work on<br>diversity of employees.</td><td>Great culture and<br>good people.</td></tr><tr><td rowspan="2">pay</td><td rowspan="2">work life<br>balance</td><td>worklife balance especially<br>for families.</td><td>Great culture and<br>great people.</td></tr><tr><td>pay on the low end</td><td>stress is low and<br>work/life balance is<br>good</td></tr><tr><td rowspan="2">management</td><td rowspan="2">benefit</td><td>pay is well below industry<br>average</td><td>great work-life<br>balance,<br>knowledgeable<br>people, benefits</td></tr><tr><td>Horrible management. HR<br>management worse.</td><td>The benefits are<br>good and not too<br>costly.</td></tr><tr><td rowspan="3">hour</td><td rowspan="3">training</td><td>The management is really<br>horrible.</td><td>The benefits are<br>really great.</td></tr><tr><td>Hours, A lot of turnover</td><td>Training, no other<br>pros to mention</td></tr><tr><td>Long hours and stressful<br>work environment.</td><td>Great training and<br>team environment.</td></tr></tbody></table>
<table><tr><td rowspan="2">pay</td><td>long hours and constant pressure</td><td>pay is good hard to beat</td></tr><tr><td>Long hours on the job</td><td>Good pay for work done</td></tr><tr><td rowspan="2">management</td><td>Hours and ongoing changes..</td><td>Solid management and great location.</td></tr><tr><td>very long hours raise very small</td><td>great work environment management hand off remote work</td></tr><tr><td rowspan="2">culture</td><td>long hours and over time</td><td>like the culture and people</td></tr><tr><td>A lot of hours worked</td><td>Great pay and good culture</td></tr></table>

# 5 Discussion

## 5.1 Tradeoffs

The approach identified two tradeoffs: management vs. work life balance, and opportunity vs. work life balance. A tradeoff between management and work-life balance means that a significant number of people complains about bad management but praises the good work life balance (870, P<.0001), while another group of people complains about poor work-life balance but praises the company for good management (118, P<.0001). That good work-life balance and good management are not incompatible can be seen from the fact that both labels appear significantly frequently together as a pro-pro pair (145, P<.0001). This makes intuitive sense: A manager can be good at managing people and care about their work-life balance if circumstances allow for it. However, more often (870, P<.0001) employees in finance seem to think that they have bad managers. These managers are potentially less organized, which might result in less pressure at work and thus better work-life balance. For example, one employee wrote: *Cons: Sometimes management is unorganized. Pros: The work/Life balance is good, they understand that you have many things going on with your life* (see the examples in Table 2). Less (but significantly) often, the work-life balance is considered bad, but the management is given credit for doing a good job, for example: *Cons: Long hours and not enough work life balance. Pros: Great people and management.*

The second tradeoff between opportunity and work-life balance showed up in fewer reviews but marks a tradeoff with less compatible labels (significantly infrequent pro-pro pairing: 11, P<.0001). The tradeoff suggests that working less comes with fewer opportunities for career development in the financial industry (47, P<.0001), while having less time outside of work – and therefore probably working more – creates
additional opportunities (54, P<.0001). Intuitively, it makes sense and is probably due to time being a scarce resource.

It is striking that work-life balance is involved in both tradeoffs. The concept describes in and of itself a tradeoff evolving around how to allocate one's time. To see whether a solution could be found for *work life balance* vs. *opportunity* in the sense of Fig. 1, and how to explain *work life balance* vs. *management*, more analysis would be required, but this is outside of the scope of this paper. For inspiration regarding the former, we could investigate other sectors or use the generalized version of the *algorithm of inventive problem solving* that was designed for dealing with tradeoffs [13].

## 5.2 Pro-con pairs

The theme underlying the two tradeoffs, i.e., the concern of employees with work-life balance and thus their *time*, is reinforced by the list of significantly frequent pro-con pairs (Table 2). The time-related label, *hours*, appears in the cons section of 4 pairs, and *work life balance* twice in *cons* sections and once in the *pros* section of another pair. 7 out of 8 pairs are thus time related (for a few examples, see Table 4). On the most general level, the explanation seems to be straight forward: employees sell their time to their employer for a salary. If a resource is scarce and different systems exploit the same shared resource, tradeoffs emerge around this resource [14]. While each of the conflicts would deserve further investigation (especially the less obvious ones such as *con: work life balance & pro: culture* (77, P.0001), or *con: management & pro: benefits* (85, P<.0001), it is outside of the scope of this paper.

## 5.3 Possible future improvements on the approach

Four main aspects stand out as potential improvements for the approach. First, the classifier to assign labels to the comments requires further improvement. Currently, it is based on a model that has not been fine-tuned for the task of classifying free text on Glassdoor in terms of the identified labels. Second, it is desirable to use more than 13 labels to identify less obvious combinations. Third, it might be worthwhile to do a comparison between sectors. Fourth, the scope of applicability of the approach could be broadened by making it independent of the pros-cons structure that is immanent to reviews on Glassdoor and Indeed. This might be achievable by adding a layer of sentiment analysis. Currently, the approach piggybacks on the pre-categorization of the reviews themselves.

## 6 Acknowledgment

I am grateful to Glassdoor for making this research possible by providing access to their proprietary data, to Marco Meyer for the discussions of this topic and his advice, to Georg Müller-Christ and his colloquium at the university of Bremen for the feedback and guidance, and to Isabel Estevez for proof-reading the paper.
# 7 References

1. Glassdoor. https://www.glassdoor.com, last accessed 2022/04/29.

2. Indeed. https://www.indeed.com/, last accessed 2022/04/29.

3. Altshuller, G. S. *Creativity as an exact science: the theory of the solution of inventive problems*. (Gordon and Breach Science Publishers, 1984).

4. Schumpeter, J. A. *Business Cycles: A Theoretical, Historical and Statistical Analysis of the Capitalist Process*. (Porcupine Press, 1939).

5. Tradeoff-detection. https://github.com/datamaunz/tradeoff-detection, last accessed 2022/04/29.

6. Guo, F., Gallagher, C. M., Sun, T., Tavoosi, S. & Min, H. Smarter people analytics with organizational text data: Demonstrations using classic and advanced NLP models. *Human Resource Management Journal*, (2021).

7. Luo, N., Zhou, Y. & Shon, J. Employee Satisfaction and Corporate Performance: Mining Employee Reviews on Glassdoor.com. 16 (2016).

8. Smith, D., Choudhury, P., Chen, G. & Agarwal, R. *Weathering the COVID Storm: The Effect of Employee Engagement on Firm Performance During the COVID Pandemic*. https://papers.ssrn.com/abstract=3841779 (2021) doi:10.2139/ssrn.3841779.

9. Reimers, N. & Gurevych, I. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. *arXiv:1908.10084 [cs]* (2019).

10. sentence-transformers/paraphrase-distilroberta-base-v1 · Hugging Face. https://huggingface.co/sentence-transformers/paraphrase-distilroberta-base-v1, last accessed 2022/04/29.

11. spaCy · Industrial-strength Natural Language Processing in Python. https://spacy.io/, last accessed 2022/04/29.

12. Holm, S. A Simple Sequentially Rejective Multiple Test Procedure. *Scandinavian Journal of Statistics* 6, 65–70 (1979).

13. Schollmeyer, J. & Tamuzs, V. Deducing Altshuller's Laws of Evolution of Technical Systems. in vol. AICT-572 55 (Springer International Publishing, 2019). doi: 10.1007/978-3-030-32497-1_6.

14. Schollmeyer, J. & Tamuzs, V. Discovery on Purpose? Toward the Unification of Paradigm Theory and the Theory of Inventive Problem Solving (TRIZ). in *Automated Invention for Smart Industries* (eds. Cavallucci, D., De Guio, R. & Koziołek, S.) 94–109 (Springer International Publishing, 2018). doi:10.1007/978-3-030-02456-7_9.