Estimation of Happiness Changes through Longitudinal Analysis of Employees’ Texts (2024)

Abstract: Measuring happiness as a determinant of wellbeing is increasingly recognized as crucial. While previous studies have utilized free-text descriptions to estimate happiness on a broad scale, limited research has focused on tracking individual fluctuations in happiness over time owing to the challenges associated with longitudinal data collection. This study addresses this issue by obtaining longitudinal data from two workplaces over two and six months respectively. Subsequently, the data is used to construct a happiness estimation model and assess individual happiness levels. Evaluation of the model performance using correlation coefficients shows variability in the correlation values among individuals. Notably, the model performs satisfactorily in estimating 9 of the 11 users’ happiness scores, with a correlation coefficient of 0.4 or higher. To investigate the factors affecting the model performance, we examine the relationship between the model performance and variables such as sentence length, lexical diversity, and personality traits. Correlations are observed between these features and model performance.

heading: 1 Introduction
text: Well-being, happiness, and mental quality of life (referred to as “happiness” herein) are key factors affecting workplace performance (Diener and Biswas-Diener, 2002). Low happiness tends to cause issues such as decreased productivity (Shi et al., 2013) and increased turnover (Hurtz and Donovan, 2000). Hence, monitoring happiness is essential in order to make interventions, such as having them seek consultation at the appropriate time. Consequently, several studies regarding happiness measurement have been conducted. However, these studies primarily utilized questionnairebased survey methods, which do not adequately examine short-term changes in individual happiness in the workplace. Thus, studies suffer from
capturing short-term happiness, because most of the questionnaires, such as the Satisfaction with Life Scale (SWLS) (Diener et al., 1985), are not designed for daily assessment.
This study addresses the challenge of capturing short-term happiness. To achieve this, we leveraged free-text diaries, which can be reported daily and offers a more flexible alternative to traditional questionnaires. Previous studies have demonstrated the feasibility of estimating happiness from text data (Schwartz et al., 2015; Wu et al., 2017; Coşkun and Ozturan, 2018; Jaidka et al., 2020; Kjell et al., 2022; Song and Zhao, 2023). However, these studies primarily focused on estimating overall happiness levels at the mass level, instead of monitoring individual changes in happiness over time.
This study aims to capture longitudinal individual happiness in the workplace (see Figure 1). Thus, we obtained daily self-report texts with happiness scores from employees in their workplaces and then
294
constructed a model based on bidirectional encoder representations from a transformer (BERT) (Devlin et al., 2018) to estimate happiness from the text. To generalize the model, we constructed a model that estimates happiness using only one diary for a specific day.
The contribution of this study is two-fold:
• A new task is proposed that employs happiness estimation model in the workplace (Section 5.1).
• Suggestions are provided for mitigating performance issues using a happiness estimation model (Section 5.2).
The proposed model is considerably simpler than conventional questionnaires, thereby facilitating future psychological research.

heading: 2 Related work
text: We describe studies that solved the task of estimating happiness from text. Previous studies that constructed estimation models can be categorized into two approaches: one that utilizes datasets containing texts annotated by individuals other than the original authors, and another that employs datasets containing texts and happiness levels obtained from the same individuals.
First, we introduce an approach that uses a dataset in which happiness is assigned to a text by a person who is not the author of the text. There are studies that estimate happiness on a large scale from social media posts (Sametoğlu et al., 2023). These studies (Mitchell et al., 2013; Schwartz et al., 2015; Jaidka et al., 2020) use dictionaries such as Linguistic Inquiry and Word Count (LIWC)1 and LabMT (Dodds et al., 2011).
Second, we introduce an approach employing datasets where both the texts and happiness levels are collected from the same individuals (Wu et al., 2017; Kjell et al., 2022; Song and Zhao, 2023). In this case, no specific annotators are involved and no particular criterion is set for happiness. However, the happiness levels of individuals exhibit commonalities, as shown in the texts. These studies used methods such as support vector machines and decision trees. In a study that estimated happiness using BERT with textual responses to lifesatisfaction questions (Kjell et al., 2022), higher performances were obtained compared with other
1https://www.liwc.app
studies. Therefore, in this study, we used BERT as a training method to estimate happiness in the workplace.

heading: 3 Materials

heading: 3.1 Data collection
text: We obtained two types of data: personality traits, which were obtained simultaneously prior to performing a survey; and daily reports, which were obtained throughout the experimental period, i.e., during the survey.
All participants completed a questionnaire at the beginning of the study period. Each item was answered based on a 5-point Likert scale. These scales are customized for investigating cooperativeness and autonomy in the workplace and have been proposed in the papers discussed for each item. We were the only researchers to verify the obtained data.
Cooperativeness The question pertaining to cooperativeness comprised 10 items, such as “I’m concerned about others’ perception of me at the workplace.” These questions refer to the following scale: (Hitokoto and Uchida, 2014). All the questions are listed in Table 4.
Autonomy Each question comprised 10 items, such as “I always try to form my own opinion within the company.” Such questions are based on the scales reported by (Watanabe et al., 2023; Domae et al., 2024). All the questions are listed in Table 5.
Next, we obtained two items: daily report texts and self-reported happiness scores.
Daily report text The daily reports encompass free descriptions in Japanese detailing the participants’ daily lives, with no character limits. Examples of these data are presented in Table 2.
Self-reported happiness score An 11-point scale ranging from 0 (extremely unhappy) to 10 (extremely happy) was used. This study focused on the cognitive aspects of workplace happiness. Questions pertaining to happiness were based on the Cantril ladder (Cantril, 1965), which assesses happiness via comparison to an 11-step ladder.
We used the data of company A, which were used in a previous study to investigate the happiness levels of employees at the company(Ito et al., 2023). Furthermore, we obtained additional data from company B to evaluate the estimation model. The name of each company has been changed to preserve anonymity.
An original web browser-based application was used to obtain the daily reports. The instructions provided to the participants included: “The input timing is typically once or more per working day, thus reflecting the day’s events at the end of the work day.”

heading: 3.2 Data Statistics and Examples
text: Data were obtained from company A, which is a major advertising and marketing company in Japan. In this company, 94 members participated in the input of daily reports over two months (from September 1 to October 31, 2022). Consequently, 1,728 data points were obtained from company A (Table 1). Figure 2 (a) shows the distribution of the data obtained.
Additionally, data were obtained from company B, which is a major Japanese electronics manufacturer. In this company, 11 employees provided daily report texts over six months (from December 19, 2022, to May 18, 2023). Consequently, 652 data points were collected from company B (Table 1). Figure 2 (b) shows the distribution of the data obtained.

heading: 4 Experiments

heading: 4.1 Modeling
text: We constructed a model in which the input was the daily report text and the output was the estimated happiness score. The training data were obtained from company A. In this experiment, we built a bidirectional encoder representation from a transformer (BERT) (Devlin et al., 2018)-based estima-
tion model. Specifically, we constructed a regression model of the AutoModel for Sequence Classification2 using Tohoku University’s pre-trained BERT3. The learning rate was set to 0.000002, the number of epochs to 20, and AdamW (Loshchilov and Hutter, 2017) was used as the optimizer. We did not conduct a detailed validation to select the best model from the existing models because our focus was on whether the task can be solved, not on achieving a high-performance model.

heading: 4.2 Evaluation metrics
text: The model constructed using company A’s data was tested using company B’s data. By testing models with data from a completely different population, we successfully verified the generality of the model under strict conditions. The purpose of this study is not so much to estimate the correct level of happiness as to assess whether changes in happiness are captured. For the evaluation index, we employed the Pearson correlation coefficient, which has been used in previous studies (Song and Zhao, 2023). Subsequently, the correlation coefficient between the estimated happiness score derived from the model (hereafter referred to as “estimated happiness score”) and the self-reported happiness score by employees was computed.

heading: 4.3 User-based Happiness Estimation
text: For each user, we calculated the correlation coefficient between the self-reported and estimated happiness scores. We tested the performance of the model trained on company A’s data in predicting the estimated level of happiness from the texts of individuals in company B. The eleven users in company B are U1 to U11, respectively. In this study, the model was applied to a completely different population.

heading: 5 Results

heading: 5.1 User-based happiness estimation
text: The correlation coefficients of the happiness estimation for each user in company B are shown in Table 3 and the time series is shown in Figure 3. The model performance is defined as the correlation coefficient between the self-reported and estimated happiness scores. In Table 3, we observed that the performance of the model in estimating the
2https://huggingface.co/docs/transformers/ model_doc/auto
3https://github.com/cl-tohoku/bert-japanese
happiness of each user varied. The variations in the correlation coefficients reflect the efficacy of the estimation model in capturing fluctuations in happiness over time.
We discuss the results for the characteristic users. First, the correlation coefficient for U1 in Table 3 could not be calculated because the self-reported happiness score of U1 remained consistently at 8 (Figure 3, U1). For users who answered the same number in the questionnaire but not in the free-response form, different information can be obtained daily. Additionally, incorporating the behaviors of users who responded in such manner into the model may facilitate happiness estimation.
Next, one user indicated a negative correlation coefficient, although not significant (Figure 3, U3). We examined the daily report text of this user and discovered that the user frequently used the expression, “A nice day that...” in the daily report. This suggests that the performance of the estimation model may be affected by differences in the writing style of each user.
As shown in Figure 3, only one user (U5) answered 0 after April. Although the user initially provided diverse texts, after April, his entries were primarily zeros and the content became uniform. Some users could not easily provide accurate estimates after such environmental changes. Although
the occurrence of events can complicate data acquisition, inadequate data may reflect an individual’s happiness level.
Finally, we assessed the happiness estimation for all users. Inter-individual variations in the factors influencing happiness can affect prediction accuracy. Because the training and test datasets used in this study encompass various distinct user profiles, the determinants of happiness may vary considerably. This suggests that incorporating user personality traits and writing habits into the model may improve estimation performance.

heading: 5.2 Analysis
text: The performance of the model depends on the content of the user’s daily report text. Considering user differences as intervening factors may increase the effectiveness of the happiness estimation models. Therefore, we explored the following features for users from U2 to U11, for which correlation coefficients could be computed.
Average length of sentences The model performance may have been influenced by the length of the diary entries. Previous studies showed that the length of sentences differs from author to author (Yule, 1939).
Lexical diversity The model performance may have been influenced by the lexical diversity. Research has demonstrated that lexical diversity differs among individuals (GregoriSignes and Clavel-Arroitia, 2015).
Personality Differences in personality may cause differences in diaries. In this study, the following personality traits were collected during the experiment. This study examines items of cooperativeness and autonomy.
The results are visualized using box and scatter plots (Figure 4). In the scatter plots, the horizontal and vertical axes represent the model performance
and feature, respectively. The box plot partitions the model performance into two sections: aboveand below-median section.
Additionally, we obtained the correlation coefficients and p-values for each of the characteristics. The results showed a trend, although insignificant, which is likely due to the small sample size of 10 participants. Thus, the following discussion is limited to trends in the data, and we will attempt to obtain additional data to validate our analysis.

heading: 5.2.1 Average length of sentences
text: Longer sentences in the diary contain more information. We examined the effect of the average sentence length per user on the model performance (Figure 4 (a)). The results showed that the group with a longer average sentence performed better in estimating happiness. The median values remained consistent across both groups in the box plots.
The results suggest that a longer user diary corresponds to the likelihood of the model performing better. Furthermore, the longer the diary sentences, the more likely they are to contain more information concerning happiness.

heading: 5.2.2 Lexical diversity
text: We examined the relationship between the lexical diversity of each user’s diary and the model performance. Lexical diversity is an evaluation
of whether a user writes different content in each diary. The diversity of each user’s diary was evaluated using the Self-BLEU (Zhu et al., 2018) metric. Self-BLEU uses the BLEU score (Papineni et al., 2002) to evaluate the overall similarity between a group of three or more sentences. A high score indicates that the daily report texts are similar, whereas a low score indicates that the Daily report texts are diverse. In this experiment, we computed BLEU scores using SacreBLEU (Post, 2018).
The results showed that users with low selfBLEU scores performed better (Figure 4 (b)). This finding indicates that greater lexical diversity in the diaries is associated with improved happiness estimation. Moreover, this indicates that individual motivation may affect the model performance. Users who repeatedly inputted the same text from April onward (see Figure 3, U5) had the highest self-BLEU scores and low model performance. Given the diversity of the daily report text before April 1, the self-BLEU score was affected by the events after April. In April, the number of members changed and the number of employees increased. Self-BLEU scores are affected by events that complicate continuous daily reporting.

heading: 5.2.3 Cooperativeness
text: The relationship between cooperativeness and model performance was investigated. For cooperativeness, we used the total score of the answers to the 10 questions.
Four of the five users whose model performances were above the median were relatively highly cooperative within their group (Figure 4 (c)). Cooperative users are more likely to engage with others in the workplace while working. Increased interactions with many people suggest that there is more diversity in work and a greater likelihood of variation in diary content. Additionally, more cooperative users may have been more willing to participate in the experiment. Such users can employ strategies to enhance the content richness. However, users with the lowest cooperativeness were in the above-median model performance group, whereas users with the highest cooperativeness were in the below-median model performance group. Nonetheless, the trend suggests that the model performed well when the users were cooperative.

heading: 5.2.4 Autonomy
text: Next, we investigated the relationship between autonomy and model performance. We used the total score of the answers to the 10 autonomy questions.
U3, located at the bottom left of scatter plot in Figure 4 (d), has the lowest autonomy score and model performance. The lack of autonomy in creating new sentences each time may result in only similar sentences written, thus adversely affecting the model performance.
However, recognizing and accomplishing tasks contribute minimally to their annotation in journals.
As shown in the box plot, the users with the greatest and least autonomy belonged to the group of users exhibiting low performance. This suggests that autonomy does not affect happiness estimation as much as cooperativeness.

heading: 6 Conclusion and future works
text: In this study, we estimated happiness scores from diaries obtained longitudinally over six months using a transformer-based model. The analysis revealed a significant correlation between the estimated and self-reported happiness scores of 9 among 11 participants.
The findings of this study suggest three key avenues for future research and application.
First, happiness can be enhanced via targeted interventions. This study demonstrated the feasibility of estimating happiness levels in real time using free-text descriptions. Furthermore, when a shift in happiness occurs, free-text descriptions can provide insights into the underlying events or experiences.
Second, this study focused on estimating happiness scores from texts, as free texts contain abundant information beyond happiness. A nuanced examination of texts can provide insight into the determinants of happiness, the author’s experiences, and the related emotions. Analyzing the characteristics of texts enables the measurement of happiness and various aspects that contribute to happiness, such as cooperativeness and autonomy.
Third, the efficacy of happiness estimation from texts is affected by individual traits, including writing style and personality. Understanding each user’s diary patterns may enable a preliminary assessment of the happiness estimation performance of the model. This may determine whether another instruction is to be employed to predict the happiness of users with shorter or fewer diary entries more accurately. For example, setting requirements such as the minimum character count for diary entries can enhance the performance. The findings of this study emphasize the importance of considering individual user characteristics when estimating happiness from texts, which can provide a better assessment of happiness.
Limitation
Using this task as a substitute for conventional questionnaires presents five major challenges.
First, cases exist where the relationship between
a specific event and the happiness level is different for each individual. For example, the statement “Today is a holiday.”may elicit happiness in some people but not in others. To understand the relationship between holidays and happiness, a significant amount of information, such as the author’s beliefs, values, and current situation, is required, in addition to sufficient texts.
Second, the model is not generalized. The model used in this study is based on data obtained from experiments conducted by two companies. The participants of this study were from company A, which is an advertising and marketing business company, and company B, which is an electronics manufacturer. Because the vocabulary varies with the company characteristics, more data should be obtained from diverse industries and companies.
Third, the data acquisition was biased. Because both datasets were obtained simultaneously, the respondents were able to describe their happiness level and free texts consistently.
Fourth, this study was conducted using a limited sample size. The findings were validated based on data obtained from 10 participants. To enhance the robustness and applicability of these findings, more data should be obtained from larger cohorts of users.
Fifth, our experiments were restricted to the Japanese language.

heading: Acknowledgments
text: This research was supported by the Japan Science and Technology Agency, Japan Society for the Future, and JPMJMI21J2.

heading: A Appendix
text: A.1 Workplace-based Happiness Estimation To estimate workplace happiness, the happiness level was estimated for each workplace and evaluated by determining the correlation coefficient. The correlation coefficient between the self-reported and estimated happiness scores was 0.49 (Figure 5).

heading: Future Work
text: This research was supported by the Japan Science and Technology Agency, Japan Society for the Future, and JPMJMI21J2