PsyAttention: Psychological Attention Model for Personality Detection
Authors: Baohua Zhang
Yongyi Huang
Wenyao Cui
Zhang Huaping
Jianyun Shang
Venue: EMNLP 2023 Findings (2023)
Keywords: personality detection; BigFive; PsyAttention; psychological features
TL;DR: This paper adapts different psychological models in the proposed PsyAttention  for personality detection, which can effectively encode psychological features.

Abstract: Work on personality detection has tended to incorporate psychological features from different personality models, such as BigFive and MBTI. There are more than 900 psychological features, each of which is helpful for personality detection. However, when used in combination, the application of different calculation standards among these features may result in interference between features calculated using distinct systems, thereby introducing noise and reducing performance. This paper adapts different psychological models in the proposed PsyAttention  for personality detection, which can effectively encode psychological features,  reducing their number by 85%. In experiments on the BigFive and MBTI models,  PysAttention achieved average accuracy of 65.66% and 86.30%, respectively, outperforming state-of-the-art methods,  indicating that it is effective at encoding psychological features.

PsyAttention: Psychological Attention Model for Personality Detection
Baohua Zhang, Yongyi Huang, Wenyao Cui, Huaping Zhang and Jianyun Shang Beijing Institute of Technology, China kevinzhang @ @it.eddu.n
Abstract
John, 1992: Schmitt al., 2008) and MBTI (Boyle. 1995; Hong, 2022; Celli and Lepri, 2018). The Big- Five model divides personality features into five factors(Digman, 1990): Openness (OPN), Consci- entiousness (CON), Extraversion (EXT), Agree- ableness (AGR), and Neuroticism (NEU). These are usually assessed through questionnaires in which people reflect their typical patterns of thinking and behavior. The MBTI model describes personality by 16 types that result from the combi- nation of binary categories in four dimensions' (1) Extraversion (E) vs Introversion (D); (2) Thinking (T) vs Feeling (F); (3) Sensing (S) vs Intuition (N); and (4) Judging (J) vs Perceiving (P).
Work personality detection has tended to incorporate psychological features from differ- ent personality models, such BigFive and MBTI. There are more than 900 psychological features, each of which is helpful for personal- ity detection. However, when used in combi- nation, the application of different calculation standards among these features may result in interference between features calculated using distinct systems, thereby introducing noise and reducing performance. This paper adapts dif- ferent psychological models in the proposed PsyAttention for personality detection, which can effectively encode psychological features, reducing their number by 85%. In experiments on the BigFive and MBTI models, PysAtten- tion achieved average accuracy of 65.66% and 86.30% respectively, outperforming state-of- the-art methods, indicating that it effective at encoding psychological features.
Previous work saw a person's use of language distillation of underlying drives, emotions, and thought patterns (Tausczik and Pennebaker, 2010; Boyd and Pennebaker. 2017). There are many theo- ries and tools for extracting psychological features from texts, such as ARTE (Automatic Readability Tool For English) (Crossley et al., 2019), which has 55 psychological features); SEANCE (Senti- ment Analysis And Cognition Engine) (Crossley et al., 2017), with 271 features; TAACO (Tool For The Automatic Analysis Of Cohesion) (Crossley et al., 2016), with 168 features; and TAALES (Tool For The Automatic Analysis Of Lexical Sophistica- tion (Kyle and Crossley, 2015), with 491 features. These tools use multiple theories to obtain psycho- logical features from text. For example, SEANCE eight theories, TAACO use fifteen theories. However, the application of different calculation standards among these features may result in inter- ference between features calculated using distinct systems, thereby introducing noise and reducing performance. For example, when analyzing senti- ment or emotion features, SEANCE uses more than 10 lexicons. A word's ' meaning may vary accord- ing to the lexicon, which leads to inconsistencies
1 Introduction
Personality detection helps people to manage them- selves and understand themselves, and has been involved in job screening, personal assistants, rec- ommendation systems, specialized health care, counseling psychotherapy, and political forecast- ing (Mehta et al., 2020b). Personality measures originated from psychology (Pervin and John, 1999; Matthews et al., 2003), and psychologists judge their patients' personality features through a series of tests.
The content that users post on social media can give valuable insights into their personali- ties (Christian et al., 2021). Personality detection can identify personality features from this, and of- ten requires combination of psychology, linguis- tics, and computer science.
There are many psychological models of indi- vidual personality, the most famous being the Big- Five (McCrae and Costa Jr, 1989; McCrae and
*Corresponding author
type-indicatorhtml

---

between features, and thus increases noise. In the field of Natural Language Processing (NLP), personality detection aims to identify the personality features of individuals from their use of language. Work can be divided into two categories. One uses psychological features obtained by some statistical methods and uses machine learning tools such as Support Vector Machine (SVM) and K- Nearest Neighbor (KNN) for classification. Others take as text classification task, and directly in- put text to neural network for classification. Work seldom combines psychological features with text embedding by treating them as sequences. How- ever, because psychology and computer science are very different fields, psychologists often de- sign features without considering how they will be introduced into neural network model. NLP researchers ignore relations between those features, and focus on how to extract and encode more fea- tures, ignoring that this will introduce noise to the model, resulting in poor performance.
NRC(Mohammad and Turney, 2013) extract ap- propriate features, which are fed into standard ma- chine learning classífiers such as SVM and Naïve Bayes (NB) (Mehta et al., 2020a) for classification. Abidin et al. (2020) used logistic regression (LR), KNN, and random forest, and found that random forest has the best performance. With the deeper study of psychology. language use has been linked to wide range of psychological correlates (Park et al., 2015; Ireland and Pennebaker, 2010). Pen- nebaker and King (1999) proposed that writing style (c.g., frequency of word use) has correlations with personality. Khan et al. (2019) thought the social behavior of the user and grammatical infor- mation have relation to the user's personality. It was found that combining common-sense knowl- edge with paycho-linguise remarkable improvement in accuracy (Poria et al., 2013). Golbeck et al. (2011) proposed 74 psy- chological features; Nisha et al. (2022) extracted features like age, education level, nationality, first language, country, and religion, and found that XGBoost performed better than SVM and Naïve Bayes.
features resulted in
In this paper, we propose PsyAAttention for per- sonality detection, which can effectively encode psychological features and reduce their number by 85%. The main contributions of this paper are as follows.
Methods based on deep-learning mainly regard personality detection as a classification task. Ma- jumder et al. (2017) proposed to use CNN and Verd2Vec embeddings for personality detection from text; since document is too long, they pro- posed dividing the data into two parts and extract- ing the text features separately using two identical CNN models separately. Yu and Markov (2017) experimented with n-grams (extracted with a CNN) and bidirectional RNNs (forward and backward GRUs). Instead of using the pretrained Word2 model, they trained the word embedding matrix using the skip-gram method to incorporate internet slang, emoticons, and acronyms. Sun et al. (2018) used bidirectional LSTMs (BiLSTMs), concate- nated with a CNN, and combined abstract features with posts to detect a user's personality. Ontoum and Chan (2022) proposed an RCNN model with three CNN layers and one BiLSTM layer. Xue et al. (201 8) proposed an RCNN text encoder with an at- tention mechanism, inputting the text to GRU and performing attention calculation to make full use of the text features, then using GRU encodings input the text in normal and reverse order. Similar to Xue al. (2018), Sun et al. (2018) used an RNN and CNN, divided the text into segments, input them separately to LSTM, and input the merged re-
We propose PsyAttention to adapt different psychological models. Experimental results on two personality detection models demon- strate that the method can improve accuracy;
We demonstrate that incorporating features de- signed by psychologists into neural networks is necessary, as pre-trained models fail to ex- tract all of the psychological features present in the text;
We select some important features in person ality detection task. To the best of our knowl- edge, this is the first method to integrate, ama- lyze, and reduce the number of psychological features for personality detection.
2 Related work
Research on individual personality detection fo- cuses on how to extract more appropriate psycho- logical features from texts and devise classification models. We introduce some methods below.
Methods to obtain personality features from text data generally use text analysis tools such as LIWC(Pennebaker et al., 2001) and
2

---

sults to the CNN extract features. Some work has combined psychological features with deep learn- ing models. Wiechmann et al. (2022) combined human reading behavior features with pretrained models. Kerz et al. (2022) used BiL.TMM to encode 437 proposed features. Neuman et al. (2023) pro- posed an data augmentation methods for modeling human personality.
Tools
Number of Features
Number Features
after Optimization
MBTI
BigFive
SEANCE
271
84
TAACO
168
9
TAALES
491
21
TOTAL
930
102
114
Table I: Number of features used in model
Third and fourth columns: number of features used in MBTI and BigFive Essays datasets, respectively.
However, as the number of psychological fea- tures designed by psychologists increases, the mod- els used to encode them are not improving, and remain a dense layer and BiLSTM. No research has focused on the effective use of psychological features. All models take them as sequence, ig- noring that they are obtained by statistical ap- proach. To better use psychological features, propose Pysattention, along with a feature opti- mization method that can reduce the noise intro- duced by too many features.
lation indicator. We set a threshold, and features with Pearson coefficients above this value are con- sidered more relevant, and should be removed. To reduce the number of psychological features used, the feature with the largest correlation coefficients with other features was chosen to represent each group of correlated features. We leave fewer than 15% of those features, as shown in Table 1, and discussed the Appendix.
3 Model
3.2 Psychological Features Encoder
We introduce PsyAttention, whose model is shown in Figure 1, effectively integrate text and psy- chological features. An attention-base encoder en- codes numerical psychological features, and BERT is fine-tuned with psychological features obtained by tools to extract more personality information.
After obtain and optimize numerical features, they must be encoded and merged with text embed- ding. How to encode these is research challenge. Some researchers encode numerical sequences di- rectly into the network by fully connected neural networks, but this is not effective because numeric sequences are too sparse for neural networks. Kerz et al. (2022) used BiLSTM as the encoder, but the numerical sequences not temporally correlated. All of the psychological features are designed man- ually by psychologists. The 437 features used by Kerz et al. (2022) are discrete values and the or- der between them does not affect the result. other words, feature extraction tools for psycho- logical features, such as TAACO and TAALES, tend to prioritize the extraction of certain categories of features, such as emotional words followed by punctuation usage others, when extracting fea- tures. These tools, which are designed artificially. do not take into account the order of the features, resulting in features that are disordered. Thus, we propose to encode them with a transformer based encoder. Since numerical sequences are not tempo- rally correlated, we remove the position encoding. Taking W (wo, W1, W2,..., ... Wm) as the numeri- cal features of psychology, and Feuender the final output of the encoder, the process can be defined as follows:
3.1 Psychological Feature Optimization
Kerz et al. (2022) proposed the most psychological features in the task of personality detection, and got them by automated text analysis systems such as CoCoGen(Marcus et al., 2016). To detect senti- ment, emotion, and/or effect, they designed: (1) fea- tures of morpho-syntacti complexity; (2) features of lexical richness, diversity, and sophistication; (3) readability features; and (4) lexicon features. These groups of features can be divided into those of ex- pressiveness and emotions contained in the text. To obtain more features, we choose three tools to analyze input texts: (1) SEANCE (Crossley et al., 2017) can obtain the features of sentiment, emotion, and/or effect; (2) TAACO (Crossley et al., 2016) can obtain features of cohesion; and (3) TAALES (Kyle and Crosslcy, 2015) can obtain features of expressive ability. Tablel shows the numbers of features used for each tool.
As mentioned above, too many features can in- troduce noise. Hence they should be used effec- tively. We calculate the Pearson coefficient be- tween each feature and other features as a corre-
H Hmatti alt. = Mutiil_attWW)
(1)
3

---

Objective
aibe
CLS
Nabood
Fine-tune BERT
Psychological Encoder
BERT
Mlndon
Prychological Features Space
Podainga Fudaas
Psychological Features Optimization
TEXTS
SEANCE
TAACO
TAALES
Figure I: Psy PstAttentoon architecture. In Psychological Feature Selection, we select psychological features obtained from multiple tools correlation analysis. In Psychological Encoder, attention-based encoder encodes discrete numeric features. Fine-tuning BERT obtains more information about personality.
HLNorm1
W) (2)
3.4
Personality detection Using PsyAttention model
We design embedding fusion layer to control the information that can be used in the final rep- resentation. We use dense layer calculate the weight, taking FPSY and FBER indicate the embedding of the psychological encoder and BERT, respectively. We can obtain the weights as follow formulas.
Hjfn F FeedForrwardd (.Nomm)))
(3)
(4) LayerNorm, and Forward denote multi-headed attention, the layer of Add Norm, and the Ferw-Forward Network, respectively.
Fencuter =
where Mutil_ate
WPSY = Densel FPSY)
3.3 Fine-tuning BERT
WBERT = Densel (BBERF))
We use BERT (Kenton and Toutanova, 2019) for text feature extraction. The pretrained model has proven its superior performance in most natural language processing tasks. Taking T = (Fo, 3m) the input text, we only use [CLS] as the text embedding. As [CLS] is the first token of the BERT output, the process can be defined H hn Itm BERT ... [0).
Due the fact that the extractable psychological and textual features are determined by the original text, we employ a dynamic weighting scheme to integrate the psychological and textual features. We can obtain the final embedding by following formula.
Then, we employ dense layer to obtain a vector that has the same dimensional as the psychological feature P1 ..
Finalemb =
BeesrFFBERWWeee WBERT (8)
where (- is the concatenation operator.
..... Dense(H) (H)
(5)
After we get the final representation, we input them to fully connected neural network for clas- sification. Taking P as the final result, the process can be defined as
The loss function is shown in formula 6. We fine-tune BERT to make the vector of CL.S more similar to the psychological feature vector.
P (Fiinaleem
(9)
loss = 1 -cssssssmmm (fi f.). (P) (Pr... (Pt.)P)) PP) (6)
It is noteworthy that during the training process. we first conduct training for this step. After fine- tuning BERT, we then fix the current weights and use them to extract textual features.
It is worth noting that the final loss function of our model the cross-entropy, where formula (6) is the loss function for the Fine-tune Bert component only.
4

---

0
C
E
A
N
Ture
1272
1254
1277
1310
1233
66
False
1196
1214
1191
1158
1235
Bigfive
Total
2468
2468
2468
2468
2468
64
Table 2: The statistics of BigFive Essays dataset.
Accuary
MBTI Kaggle
I/E
N/S
T/F
P/J
N or or P
6676
7478
3981
3434
E or or F J Total
1999
1197
4694
5241
8675
8675
8675
8675
Table 3: The statistics of MBTI Kaggle dataset.
num num_attention ltttttattio_layrrr
Figure Results different numbers of attention lay- ers BigFive dataset
4 Experiments
4.1
Dataset
We conducted experiments on two widely used personality benchmark datasets: BigFive Essay datasee(Pennebaker and King, 1999) and MBTI Kaggle dataset (Li et al., 2018). The BigFive Essay dataset consists of 2468 essays written by students and annotated with binary labels of the BigFive personality features, which were obtained through standardized self-reporting questionnaire. The
Mid
62
Accuary
80
average text length is 672 words, and the dataset contains approximately 1.6 million words. The MBTI dataset consists of samples social media interactions from 8675 users, all of whom have in- dicated their MBTI type. The average text length is 1,288 words. The dataset consists of approxi- mately 11 11.2 million words. The detailed statistics are shown in Table 2 and Table 3.
2
10
12
attention layer
Figure The results of different numbers of attention layers on MBTI dataset
To retain the psychological features contained in the text, we perform no pre-processing on the text when extracting psychological features using text analysis tools, and we remove noise such as con- catenation, multilingual punctuation, and multiple spaces before input to the BERT model.
The results are shown in Figures 2 and coders.
3.
Taking BERT-base as the text encoder model, we used four attention head layers, cight psychological feature encoder layers, a learning rate 2e-5, and the Adam optimizer. The maximum text length was 510, the length of psychological features was 108, and other parameters were the same for RERT-base.
4.2 Parameter Settings
We utilized Python 3.8.1, PyTorch 1.13.0, Trans- formers 4.24.0, and scikit-learn 1.1.3 for imple- menting our model. Our training process involved 4 NVIDIA RTX 3090 GPUs. We trained our model with 50 epochs. We have employed split ratio of 7:2:1 for training, validation, and testing respec- tively.
4.3 Baselines and Results
We took the BERT+PSYLING Ensemble model of Kerz et al. (2022) 437 text contours of psycholinguistic features. with BERT as the text encoder. While selected 108 psychological features and designed a psychologi- cal feature encoder based on an attention module, which we think is more effective. We also com-
bascline. who used
We researched the number of psychological en- coders that can obtain the best result, used even numbers from 2 to 12 to test the results, and found that the model obtained the best result with 8 en-
5

---

Bigf'ive lvv Essays
MBTI Kaggle
0
A
N
Avg
VE
N/S
TMF
PIU
Avg
Majumder et al. (2017)
61.1
56.7
58.1
56.7
57.3
58
Kazameini (2020)
62.1
57.8
59.3
56.5
Amirhosseini (2020)
79
86
74.2
65.4
76.1
Psychological+MLPP
60.4
57.3
56,9
57
59.8
58.3
77.6
86.3
72
61.9
74.5
BERT-busc+MMLP
64.6
59.2
60
58.8
60.5
60.6
78.3
86.4
74.4
64.4
75.9
All Meatures+MLPP
61.1
57.4
57.9
58.6
60.5
59.1
78.4
86.6
75.9
64.4
76.3
BERT-large+MI.P
63.4
58.9
59.2
58.3
58.9
59.7
78.8
86.3
76.1
67.2
77.1
Ramezani al. (2022)
56.30
59.18
64.25
60.31
61.14
60.24
Kerz (2022)single)
66.23
60.60
61.61
61.05
61.65
62.28
86.25
84.66
79.65
85.37
BERT
56.67
57.35
57.22
58.91
59.40
57.91
77.74
86.27
60.09
54.67
69.69
BERT+Dense
61.83
56.91
57.95
60.55
57.96
59.04
83.97
84.76
80.95
76.0
81,42
BFRT+BiLLSTM
62.23
58.94
58.42
59.16
59.37
59.62
84.28
88.79
83.95
76.57
83.40
PsyAttention
68.62
64.21
64.43
66.75
64.27
65.66
87.94
91.47
85.24
80.53
86.30
Table 4: Results (classification accuracy) of Personality Detection
pared our model with other models. Ramezani et al. (2022) proposed a Hierarchical Attention Network (HAN) combined with base method predictions to carry out stacking in document-level classification. They combined five basic model predictions with the text features, and input them to dense layer get the final prediction. Mehta al. (2020a) also used psychological features with BERT, and experimented with logistic regression, SVM, mul- tilayer perceptron (MLP) with 50 hidden units, and machine learning models such as those Amirhos- seini (2020), Majumder et al. (2017), and Kaza- meini et al. (2020).
architecture of BERT+BiLSTM is same as that of the of Kerz et al. (2022) (single), the difference is that we use all 930 psychological features while Kerz et al. (2022) (single) only use 437. The ac- curacy of BERT+BiLSTM is lower than that of Kerz et al. (sigl),s which proves that more features will cause more noise.
4.4 Ablation Experiments
We performed ablation experiments, and the sults of which are shown in Table 5. "-psyencoder' indicates that we do not use the Psychological En- coder, '-BERT' indicates that we do not use the Fine-tune BERT, "weight' indicates that we do not use the embedding fusion weight '-BERT indicates that we do not use psychological features in Fine-tune BERT, and '-Features Selection = in- dicates that we use all 930 psychological features. see, the results for -BERT' are reduced the most, indicating that the BERT coding com- ponent is the most important, and the results for "psyencoder' are also reduced, indicating that psy- chological features also have a significant effect on the results. The results of SeealctesSSelection indicate that there is some noise in all features, and it is right to do the feature selection.
We conducted experiments on the BigFive Es- says and Kaggle MBTI Kaggle datasets. The re- sults are shown in Table 4. The PsyAttention listed in Table 4 is our model. Majumder et al. Kazamiinii et al. (2020) and Amirhosseini (2020) represent the results the machine learn- ing models. The others represent the results of the deep learning models. As we can see, the average accuracy of machine learning is lower than that of the deep learning model, which indicated that machine learing models are not as effective as deep learning models. Kerz al. (sigge)) is the model that named in Kerz et al. (2022)'s work. Since we did not em- ploy the stacking approach in our model, we only compared our model with Kerz et al. (sin gle). Both Att-tttnt and Kerz et al. (sin-) gle) deep learning models and psychological features. As can see from Table 4, the accu- racy of PsyAttention is 2% higher than Kerz al. (sigge)) on the BigFive datasets and 0.3% higher on the MBTI dataset, which proves that our model is better than Kerz et al. (siggls The
As
SEE--AATN---LLLIIIS (FT)
4.5 Psychological Features In Pretrained Models
We designed simple experiment to verify whether text embedding includes psychological informa- tion. Consider that almost every model in NLP tasks takes word embedding as input and uses some neural networks extract features, which input to a discriminant model or generative model to get the final representation. If there is psychological information in the text embedding, we can extract
6

---

BigFive Essays
MBTI Kaggle
E
A
N
Avg
I/B
N/S
T/F
P/J
Avg
PsyAttention
68.62
64.21
64.43
66.75
64.27
65.66
87.94
91.47
85.24
80.53
86.30
-psyencoder
61.69
58.13
58.06
60.59
58.19
59.33
84.01
86.94
82.30
77.72
82.74
-BERT
59.92
58.05
$5.94
58.63
56.79
57.87
78.78
83.12
75.98
77.53
78.85
-weight
66.63
62.89
62,20
65.55
62.90
64.03
86.61
88.26
82.19
77.75
83.70
-BERT pey
64.58
61.19
60.54
64.24
62.48
62.63
83.02
86.55
80.59
74.94
81.28
-Features Selection
65,98
62.18
63.32
64.13
60.76
63.27
85,52
91.03
84.35
78.98
84,97
Table Ablation experiment results (classification accuracy)
Methods
MBTI
BigFive
corresponding psychological features. After nor- malization, we plotted the feature distribution of the psychological features contained in both MBTI and BigFive, showed in Figure 4.
BERT
0.575
0.551
BERT & Fine-tune
0.687
0.662
Table 6; Results of cosine similarity
it by some networks. Hence we designed model to extract psychological features from text embed- ding, used BERT to obtain the text embedding. and dense layer to extract psychological features. The objective was to make the vector of CLS more similar to the psychological feature vector. We cal- culated the cosine similarity between the features extracted from the text embedding and those calcu- lated by the psychology tool, and took it the loss function and evaluation metric of the results. The experimental results are shown in Table 6.
We used a BERT-base model. Table 6 shows the results of two encoders, where "BERT" indi- cates the direct use of BERT-base to obtain the text embedding, without fine-tuming on psychologi- cal datasets; BERT & Fine-tune obtained by fine- tuning BERT-base on those two datasets. During training, we fixed the weight of BERT&Fine-tur but did not fix the weight of BERT. As we can see, even if we fine-tune BERT-base, we can only get a cosine similarity of less than 0.7, which indi- cates that there is little psychological information in text embedding, and we need to introduce it in the model.
Figure 4: The common psychological feature distribu- tion used in the MBTI and BigFive Essays datasets, where the x-axis represents the attention score on Big- Five and y-axis represents MBTI, with larger values indicating more important features. We will introduce more in section 6.2.
Table 7 displays the features that scored above 0.6 after separate normalization on the two datasets. "WWL_Sublist_ 9 "_Normed" indicates the number of words in the text that belong to the Academic Word List Sublist 9. "Powends Lasswell" and "Powdoct Lasswell" Lawsweel both represent words related to dominance, respect, money, and power. The former includes words such as "ambition," "am- bitious," "arrangement," "bring," and "challenge," while the latter includes words like "bourgeois." "bourgeoisie," " "capitalism," "civil," " and "collec- tivism."
4.6
The Weights of Psychological Features
After removing a substantial number of features, we sought to investigate the remaining features' impact on the psychological discrimination task. To accomplish this, we introduced attention mechanism calculate the attention scores of the psychological features and explore the importance of different features. Because psychological fea- tures represented by MBTI and BigFive differ, we separately calculated the attention scores for their
It can be observed that the aforementioned features can be categorized into three groups:
7

---

Table 7: The more important psychological features
Features
MBTI
BigFive
Avg
Variable description
Well heing gain: 37 various words
Wibgain Lasswell
0.85
0.81
0.83
related to gain in well being.
Shame GALC
0.88
0.68
0.78
Shame
Sadness GALC
0.69
0.80
0.75
Saduess
GALC
0.75
0.75
0.75
Pride
AWL Sublist Normed
0.72
0.67
0.70
Academic Word List Sublist 9
Power End: 30 words about the
Powends Lasswell
0.70
0.65
0.68
goals of the power process.
Guilt GALC
0.69
0.64
0.66
Guilt
Humility_GAL _GACC
0.60
0.66
0.63
Humility
Power doctrine: 42 words for recognized ideas about power relations and practices.
Powdoct Lasswell
0.65
0.61
0.63
personal emotion-related, personal realization- related, and word usage habit-related. The pri- mary personal emotion-related features are WIb- gais_Lasswell, ShLC. Sadness_GALe Quilt_GALC, and Humility_Gain GALC, which encompass both positive and negative emotions experienced by individuals. The per- sonal realization-relate features mainly consist of Powends Lasswell and Powdoct Lasswell, which are indicative of self-actualization and personal de- velopment. Lastly, the word usage habit feature, AWI._Sublist_9 9__ Normed, relates to the frequency of academic words used by individuals and is be- lieved to reflect their level of rigor. The propor- tion of words in the personal emotion category is over 66%, indicating that emotion a critical characteristic in personal character discrimination. The proportion of words in the self-realization cat- egory is over 20%, reflecting the importance of self-actualization in personal character. The word usage habit feature is believed to be mainly related to whether an individual is rigorous not, ca- sual individuals tend use fewer academic words.
Conclusion
We proposed PysAttentiinn for personality detec- tion, which can effectively encode psychological features, and reduce their number by 85%. We cate- gorized the psychological features contained in the text as either expressiveness or emotions, and ex- tracted the relative features using three text analysis tools. We demonstrated that the pre-trained model contained only a small amount of psychological information, and proposed a method of psycholog- ical feature selection. Unlike models that simply encode a long series of numerical psychological features through BiLSTM, efficiently encode psychological features with a transformer-like en- coder. We also investigate the impact of different psychological features on the results and employe an attention mechanism to identify the more sig- nificant features. In experiments on two classical psychological assessment datasets, the proposed model outperformed state-of-the-ate algorithms in terms of accuracy, demonstrating the effectiveness of PtstAttention.
References
It should be noted that not all emotions are in- cluded in the personal emotion category, such as Loneliness, Envy, and Anxiety. This may be due to the fact that these emotions are often gener- ated during interpersonal interactions, and there- fore may not be as critical for personality detec- tion. Furthermore, it should be noted that emo- tions represent only portion of the content, and other celf-actualizzaioon--rlate characteristies may be even more significant. Personal realization- related features, such as Powends Lasswell Lasseel and Powdoct Lasswel,,, are indicative of an individual's self acffaaitaai and personal development, which are critical components of personal character.
Nur Haziqah Zainal Abidin, Muhammad Akmal Remli, Noorlin Mohd Ali, Danakorn Nincarean Eh Phon, Nooraini Yusoff, Hasyiya Karimah Adli, and Ab- delsalam H Busalim. 2020. Improving intelligent personality prediction using mrers-briges type in- dicator and random forest classifier. International Journal of Advanced Computer Science and Applica- lions, 11(11).
Kazemian- Hassan Amirhosseini, Mohammad Hossein. 2020. Machine learning approach to personality type prediction based on the mnyers-briges type in- dicator@. Multimodal Technologies and Interaction. 4(1):9.
Hassaan
Ryan L. Boyd and James W Pennebaker. 2017. Language-baser personality: new approach to per-
8

---

sonality in digital world. Current opinion in behav- ioral sciences, 18:63-68.
Elma Kerz, Yu Qiao, Sourabh Zanwar, and Daniel Wiechmann. 2022. Pushing on personality detec- tion from verbal behavior: A transformer meets text contours of psycholinguistic features. In Proceedings of the 12th Workshop on Computational Approaches to Subjectivity, Sentiment & Social Media Analysis, pages 182-194.
Gregory J Boyle. 1995. Myers-briggs type indicator (mbti); some psychometric limitations. Australian Psychologist, 30(1):71-74.
Fabio Celli and Bruno Lepri. 2018. Is big five better than mbti? personality computing challenge us- ing twitter data. Computational Linguistics CLiC-it 2018, page 93.
Ali Nawaz Khan, Xiongfei Cao, and Abdul Hameed Pitafi. 2019. Personality traits as predictor of m- payment systems: sem-neural networks approach. Journal of Organizational and End User Computing (JOEUC), 31(4):89-11 110.
Hans Christian, Derwin Suhartono, Andry Chowanda, and Kamal Z Zamli. 2021. Text based personality prediction from multiple social media data sources using pre-trained language model and model averag- ing. Journal of Big Data, 8(1):1220.
Kristopher Kyle and Scott A Crossley. 2015. Automati- cally assessing lexical sophistication: Indices, tools, findings, and application. Tesol Quarterly, 49(4):757- 786.
Scott A Crossley, Kristopher Kyle, and Danielle S Mc- Namara. 2016. The tool for the automatic analysis of text cohesion (taaco): Automatic assessment of local, global, and text cohesion. Behavior research methods, 48(4):1227-122
Charles Li, Monte Hancock, Ben Bowles, Olivia Han- cock, Lesley Perg. Payton Brown, Asher Burrell, Gi- anella Frank, Frankie Stiers, Shana Marshall, et al. 2018. Feature extraction from social media posts for psychometric typing of participants. In International Conference on Augmented Cognition, pages 267-286. Springer.
Scott A Crossley, Kristopher Kyle, and Danielle S Mc- Namara. 2017. Sentiment analysis and social cogni- tion engine (seance); An automatic tool for sentiment, social cognition, and social-order analysis. Behavior research methods, 49(3):803-82
Navonil Majumder, Soujanya Poria, Alexander Gelbukh, and Erik Cambria. 2017. Deep learning-based doe- ument modeling for personality detection from text. IEEE Inielligent Systems, 32(2):74-79.
Scott& A Crossley, Stephen Skalicky, and Mihai Dascalu. 2019. Moving beyond classic readability formulas: New methods and new models. Journal of Research in Reading, 42(3-4):541-56
Ströbel Marcus, Elma Kerz, Daniel Wiechmann, and Stella Neumann. 2016. Conpgen-complexity con- tour generator: Automatic assessment of linguistic complexity using a sliding-window window technique. In Proceedings of the Workshop on Computational Lin- guisties for Linguistic Complexity (CL4LC), pages 23-31.
John M Digman. 1990. Personality structure: Emer- gence of the five-factor model. Annual review of psychology, 41(1):117-44.
Jennifer Golbeck, Cristina Robles, and Karen Turner. 2011. Predicting personality with social media. In CHI' extended abstracts on human factors in com- puting systems, pages 253-262.
Gerald Matthews, Ian Deary, and Martha C White- man. 2003. Personality traits. Cambridge University Press.
Robert R McCrae and Paul T Costa Jr. 1989. More reasons to adopt the five-factor model.
Eun- HounY Hong. 2022. A review of the korean nursing research literature with mbti personality and nursing students. Journal of the Korea Convergence Society, 13(1):225446.
Robert R McCrae and Oliver P John. 1992. An intro- duction to the five-factor model and its applications. Journal of personality, 60(2):175-215.
Molly E Ireland and James W Pennebaker. 2010. Lan- guage style matching in writing: synchrony essays, correspondence, and poetry. Journal of personality and social psychology, (9)3)449
Yash Mehta, Samin Fatehi, Amirmohammad Kazameini, Clemens Stachl, Erik Cambria, and Sauleh Betemadi. 2020a. Bottom-up and top-down: Predicting per- sonality with psycholinguistic and language model features. In 2020 IEEE International Conference on Data Mining (ICDM), pages 1 184-1188. IEEE.
Amirmohammad Kazameini, Samin Fatehi, Yash Mehta, Sauleh Eetemadi, and Erik Cambria. 2020. Per- sonality trait detection using bagged svm over bert word embedding ensembles. arXiv preprint arXiv:2010.01309.
Yash Mchta, Navonil Majumder, Alexander Gelbukh, and Erik Cambria. 2020b. Recent trends in deep learning based personality detection. Artificial Intel- ligence Review, 53(4):2313-233)
Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. Bert: Pre-training of deep bidirec- tional transformers for language understanding. In Proceedings of NAACL-HLT, pages 4171-41888
Saif M Mohammad and Peter D Tumey, 2013. Nrc emo- tion lexicon. National Research Council, Canada, 2:234.
9

---

Yair Neuman, Vladyslav Kozhukhov, and Dan Vi- lenchik. 2023. Data augmentation for modeling hu- man personality: The dexter machine, arXiv preprint arXiv:2301086066
Yla R Tausczik and James W Pennebaker. 2010. The psychological meaning of words: Liwe and comput- erized text analysis methods. Journal of language and social psychology, 2944)244-
Kulsum Akter Nisha, Umme Kulsum, Saifur Rahman, Md Hossain, Partha Chakraborty, Tanupriya Choud- hury, et al. 2022. A comparative analysis of ma- chine learning approaches in personality prediction using mbti. In Computational Intelligence in Pattern Recognition, pages 13-23. Springer.
Daniel Wiechmann, Yu Qiao, Elma Kerz, and Justus Mattem. 2022. Measuring the impact of (psycho-) linguistic and readability features and their spill over effects the prediction of eye movement patterns. arXiv preprint arXiv:2203.08000
Di Xue, Lifa Wu, Zheng Hong, Shize Guo, Liang Gao, Zhiyong Wu, Xiaofeng Zhong. and Jianshan Sun. 2018. Deep carning-hased personality recognition from text posts of online social networks. Applied Intelligence, 48(11))4232 4246.
Sakdipat Ontoum and Jonathan H. Chan. 2022. Per- sonality type based on byiigs type indicator with text posting style by using traditional and deep learning. arXiv preprint 0Xx:22001008771
Jianguo Yu and Konstantin Markov. 2017. Deep learn- ing based personality recognition from facebook sta- tus updates. In 2017 IEEE 8th international confer- ence on awareness science and technology (iCAST), pages 383-387. IEEE
Gregory Park, H Andrew Schwartz, Johannes C Eich- staedt, Margaret L Kern, Michal Kosinski, David J Stillwell, Lyle Ungar, and Martin EP Seligman. 2015. Automatic personality assessment through social media language. Journal of personality and social psychology. 10:96):33.
Limitations
James W Pennebaker. Martha E Francis, and Roger J Booth. 2001. Linguistic inquiry and word count: Mahway: Lawrence Erlhaum Associates, 71(2001):2000
This paper has the following limitations: 1) The BERT encoder cannot completely encode text se- quences with length greater than 512, but almost every sentence in the BigFive dataset is longer than that. We will consider models such as long-former, but this also cannot deal with longer sentences; 2) The research on psychological features is still lim- ited, and there may be some important ones we did not include our model.
Liwc 2001.
James W Pennebaker and Laura A King. 1999. Lin- guistic styles: language use as individual differ- ence. Journal of personality and social psychology. 12(6).1296.
Lawrence Pervin and Oliver P John. 1999, Handbook of personality. Theory and research, 2.
Ethics Statement
Soujanya Poria, Alexandar Gelbukh, Basant Agarwal, Erik Cambria, and Newton Howard. 2013. Com- mon sense knowledge based personality recognition from text. In Mexican International Conference on Artificial Intelligence, pages 484 496. Springer.
All work in this paper adheres to the ACL Ethics Policy.
A Acknowledgements
Majid Ramezani, Mohammad- Reza Feizi-Derakhshi, Mohammad-Ali Balafar, Meysam Asgari- Chenaghlu. Ali-Reza Feizi-Derakhshh, Narjes Hikzad-Khasmale Mehrdad Kenjbar-Khadivi Zoleikha Jahanbakhsh-Nagaadhh Elnaz Zafarani- Moattar. and Taymaz Akan. 2022. Automatic personality prediction: enhanced method using ensemble modeling. Neural Computing and Applications, pages 1-21.
This work was supported by Fundamental Strength- ening Program Technology Field Fund, China
(2022-JCIQ-J1-0930)
Appendix
6.1
The correlation coefficients for all psychological features
David P Schmitt, Anu Realo, Martin Voracek, and Jüri Allik. 2008. Why can't a man be more like woman? differences in big five personality traits across 55 cultures. Journal of personality and social psychol- ogy, 94(1):168.
We introduce the psychological feature optimiza- tion method. We calculated correlation coefficients for all psychological features on both datasets, and plotted the heatmaps shown in Figure 7. The many light-colored blocks in the diagram indicate a large number of highly relevant features. We grouped features with correlation coefficients greater than 0.2, selected those with the highest number of cor- related features from each group as representative,
Xiangguo Sun, Bo Liu, Jiuxin Cao, Junzhou Luo, and Xiaojun Shen. 2018. Who am i? personality detec- tion based on deep learning for texts. In 2018 IEEE international conference on communications (ICC), pages 1-6. IEEE.
10

---

and did not the rest. Table 8 shows some fea- tures with correlation coefficients; after processing feature selection, we only use Joy and Disgust.
We show the features that we used in Table and Table 10. Figure 6 shows the heatmaps cor- relation coefficients for the features we used in two datasets. There are no light-colored blocks, which indicates that these features have low relevance.
6.2 The Attention Score of Features
the BigFive and MBTI Kaggle datasets after summing and nor- malizing the attention scores obtained by inferring all the test data. Figure is the bigger version of Figure 4.
Figure 7 is the importance
The objective of this study is to investigate the relative importance of different features using at- tention scores. By analyzing the attention scores assigned to each feature, aim to identify the most influential characteristics in personality detec- tion.

---

(A)
(b)
Figure 5: Correlation coefficients for all psychological features both datasets. (a) MBTI; (b) BigFive.
(b)
Figure 6: Correlation coefficients for psychological features after selection on both datasets. (a) MBTI; (b): BigFive.
12

---

Feelinglove
Disgust
Fear
Joy
Negative
Positive
Sadness
Surprise
Trust
-0.02
-0.01
0.46
-0.08
0.29
0.01
0.03
0.28
-0,02
0,55
0.08
0.73
0.02
0.63
0.15
0.04
-0.01
0.55
1
0.1
0.65
0.1
0.68
0.19
0.15
0.46
0.08
0.1
1
-0.04
0.77
0.09
0.5
0.71
-0.08
0.73
0.65
-0.04
-0.05
0.73
0.14
-0.04
0.29
0.02
0.1
0.77
-0.05
1
0.04
0.36
0.73
0.01
0.63
0.68
0.09
0.73
0.04
1
0.19
0.05
0.03
0.15
0.19
0.5
0.14
0.36
0.19
0.4
0.28
0.04
0.15
0.71
-0.04
0.73
0.05
0.4
Table 8: Example for feature selection
Arousal nwords
Comnohj
G)
Compassion GALC
Dist GI
Froq, N. 0G
Active_Gi Gl
Sty_Gi Gl
Dim Gi
Contempt GALC
[padi
MoCAeecalemicc ti DP
polurity
Anxiety GALC
Powends Powends_Laswe Lasswell
Disppoomiment GALe
Reloss Lasswell
BNC Siggen Tigram_ Normed_Free Log
Econ, Gi
Lasswell
Disgast
spoken
Sv Gl
Whhpt_ Lasswell
Sky_Gl Gl
Discatitacttoo GALE
Ealloss, Lasswell
WN MMean_Accuuee CW
Enter Lasswell
Skipt Lasswell
Vehicl Gi
Eavy GALC
Isa. I. all
WN Mean Accuracy. FW
Vice
Amusement GALC
Bidgpt_
Guilt,
GuIC GALC
function,
MRC Imageuhility AW
Surelw Lasewell
Relig
Natobj Gl
Humiliy GALC
adjacent adjacent_overiappu sent prp
AW1 Sublist, Normed
Noomms
Hope, GALC
Risc
Card_
Jealousy
Noemed
Time. GI
Causal Gi
Horedom GALE
Joy. Dy_G
levical, lexical densityy pypes
Lypes_CW
Table 9: Feature names used in MBTI dataset
Arcessa mwards
Powpt Lasswell
Rise
Whyp Lasewell
Route
Sublist 5 Normed
Valence
Ansiet_GALC GALC
Cand_Gl
Wingnin Lasswell
Aquatic Gl
AWL Normed
Hostile Gl
Trngain Lasswell
Recnda Lasswell
mwards
Our Gi
AWL Sublist 6 Normed
Bldgp_CC
Disppointmmnt
Compassion,
You_Gl
Passive Gl
Dim Gi
Sadness GALC
Cresempt GALC
Ipadj.
AWI Sublist, 8 Normed
Powtor Lasswell
Powaren_Lasweel
sensilivity
Confensment GALC
Powends Lasswell
Ward Count
Rspoth__
Gl
Doperation GALC
AWI, Sublist Normed
Time G
GALC
Rerelig Lasswell
Disgust_ALL GALC
Enlloss Lasswell
AWL Sublist 9, Normed
Fooe,
Indad]
Reloss, Lasswell
Disatisfaction GALC
COCA News, Rigram, Frequancy Log
Subist Nomoed
Enltor Lasswell
Rsploss Lassell
Lasswell
Envy_ GALC
MRC MRC Mearingfolnes
aall sent
Table 10: Feature names used BigFive dataset
(b)
Figure 7: The attention score for all psychological features on both datasets. (a) MBTI; (b) BigFive.
13

---

muacmen GALC
0.8
amiliyy SALD
StBlRn Normed GALC
0.6
Gere StLL
enssiiiity
Metiot STALI
Streon GALC
mbd
coppoittment
moss Lassuet
GALC
0.4
tatin GALE
inggosched
Linonel
SoAeET JPL Z,MI
www.ttrraaaaaammoommmmmmooo GALE
SnOPr GALE
MLL Su 10nbot__ONNerred Normed
Ccuntic CI
0.2
Sunvit Normed
0.0
0.2
0.4
0.6
0.8
bigfive
Figure 8: The common psychological feature distribution used in the MBTI Kaggle dataset and BigFive Essays dataset, where x-axis represents the attention score on Big and y-axis represents MBTI, with larger values indicating more important features.
14


Reviews:
[{"review_id": "nuSwGWlrGy", "reviewer": "Reviewer_imtg", "summary": "", "questions": "When you report accuracy, adding the chance accuracy (the prevalence of that label) is better.", "limitations": "", "rating": 3, "confidence": 2, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for comments, we will incorporate \"the chance accuracy\" in the next version, as well as the referenced paper you recommended.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper is well-written, the method is explained well, and seems innovative.\nThe improvement is significant.\nThe problem is central in NLP", "reasons_to_reject": "The problem of identifying psychological features was extensively studied, and while improvement is offered by the new method, all in all the result is relatively incremental. ", "questions_to_authors": "When you report accuracy, adding the chance accuracy (the prevalence of that label) is better.", "paper_topic_and_main_contributions": "The paper deals with the problem of psychological traits prediction. The traits are taken from psychological theory (the Big Five model for example). The paper proposes a new approach to fine-tuning BERT, incorporating psychological features extracted using existing pipelines, into the training phase (in the loss function). A test is performed on two large datasets and about 5% improvement in accuracy is obtained. Also, the method proposes a way to perform feature selection and reduce the number of psychological variables that are extracted using existing pipelines (which is in the hundreds).", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "In the related work, you can mention this paper who deals with the problem of modelling personality rather than predicting it.\nhttps://arxiv.org/abs/2301.08606\n"}, "extra_text": {}}, {"review_id": "Oa7i1S8eTh", "reviewer": "Reviewer_y6Lk", "summary": "", "questions": "A. Why addressing  BigFive and MBTI  are stated as models to be extended in Abstract and Introduction  sections while they are used as mere datasets in Experiments. It's better to just state them as datasets throughout the paper.\nB. Why you didn't provide train/validation/test splits and statistics of data as used in the experiments.\nC. Why presenting figure/table names as figure1/table and Figure 1/Table 1, for example. Why not to just use Figure 1 or Table 1?\nD. Why not providing an elaborate explanation on the loss function?\nE.  Where is 'HLayerNorm1' coming from in Equation 3?\nF. Did you use pytorch, Tensorflow/keras, or what package for your model? How many epochs did you train your model? Did you use GPU, what type/name?\n", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments. We will correct the typos, add references to the missing literature, and revise the paper according to your suggestions.\n\nQA. Why addressing BigFive and MBTI are stated as models to be extended in Abstract and Introduction sections while they are used as mere datasets in Experiments. It's better to just state them as datasets throughout the paper.\n\nAnswer: Thank you for comments, we apologize for the confusion, BigFive and MBTI are two different models used in psychology to measure an individual's personality, so we introduce them as models inside Abstract and Introduction. The names of the datasets in the experiment are actually BigFive Essays and Kaggle MBTI , for brevity we use BigFive and MBTI for this, we will revise this in future versions.\n\nQB. Why you didn't provide train/validation/test splits and statistics of data as used in the experiments.\n\nAnswer: We apologize for the oversight in not providing an introduction to our train/validation/test splits. In actuality, we have employed a split ratio of 7:2:1 for training, validation, and testing respectively.  In the section of dataset, we give the statistics about the dataset, we are very sorry that we didn't make a table, we will add the statistics in a future version, thank you for your comments.\n\n\nQC. Why presenting figure/table names as figure1/table and Figure 1/Table 1, for example. Why not to just use Figure 1 or Table 1?\n\nAnswer: Thank you for your suggestion, we will consistently use Figure 1 and Table 1 in future version.\n\n\nQD. Why not providing an elaborate explanation on the loss function?\n\nAnswer: The final loss function of the model is the cross-entropy, and in section 3.3 Equation (6) is our own design of the loss function, Equation (6) is only for the Fine-tuning BERT part of our model.\n\nQE. Where is 'HLayerNorm1' coming from in Equation 3?\n\nAnswer: We apologize for incorrectly entering \"HLNorm1\" as \"HLayerNorm1\", which is actually the result of Equation 2. Thank you for your comments.\n\nQF. Did you use pytorch, Tensorflow/keras, or what package for your model? How many epochs did you train your model? Did you use GPU, what type/name?\n\nAnswer: We utilized Python 3.8.1, PyTorch 1.13.0, Transformers 4.24.0, and scikit-learn 1.1.3 for implementing our model. Our training process involved 4 NVIDIA RTX 3090 GPUs. We trained our model with 50 epochs.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper clearly identifies the problem, proposes a novel way to address it and conducts experiments to support their claims and approach.", "reasons_to_reject": "I only have these concerns from the paper:\n1.  BigFive and MBTI  are stated as models to be extended in Abstract and Introduction  sections while they are used as mere datasets in Experiments. It's better to just state them as datasets throughout the paper unless the authors should provide an extended explanation why they are addressing them like that.\n2. It's imperative to provide train/validation/test splits and statistics of data used in the experiments to aid in understanding the model performance, how the evaluation is being made, and for reproducibility.\n3. Typos and presentation inconsistencies.", "questions_to_authors": "A. Why addressing  BigFive and MBTI  are stated as models to be extended in Abstract and Introduction  sections while they are used as mere datasets in Experiments. It's better to just state them as datasets throughout the paper.\nB. Why you didn't provide train/validation/test splits and statistics of data as used in the experiments.\nC. Why presenting figure/table names as figure1/table and Figure 1/Table 1, for example. Why not to just use Figure 1 or Table 1?\nD. Why not providing an elaborate explanation on the loss function?\nE.  Where is 'HLayerNorm1' coming from in Equation 3?\nF. Did you use pytorch, Tensorflow/keras, or what package for your model? How many epochs did you train your model? Did you use GPU, what type/name?\n", "paper_topic_and_main_contributions": "The paper identifies the use of more psychological features when fused with a deep learning model or used with a mere machine learning model as impeding respective models performance on personality detection, hence, the paper proposes a new model that devises an optimization approach to reduce the superfluous psychological features to a necessary size and feeds this to an attention-based encoder, the output of this is fused with BERT embeddings for personality detection.  ", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "BERT model is not cited in the paper."}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "Typos:       line\nuse-s         084\nemploye    532 \n\nPresentation Improvements:\nOn line 031, the sentence, \"The content that users post on...\" introduces a new idea I suggest to use a transition word/phrase or new paragraph.\nOn line 113, 'selected' should be 'select' since contributions are being proposed.\nBe consistent in language use: On line 164, bidirectional LSTMs and BiLSTM are one thing hence first occurrence of them should be bidirectional LSTMs (BiLSTM) and use BiLSTM throughout the rest of the paper. Same for Natural Language Processing and NLP.  Where is 'HLayerNorm1' coming from in Equation 3?\nOn Table 1, I suggest MBTI and BigFive should be column headers and for 3rd and 4th columns  and \"Number of Features after Optimization\" should be used within table caption.\nOn Figure 1, label other components of model diagram in the middle upper area.\nI suggest to use 'following' than 'follow' like sentence on line 305, \"We can obtain the final embedding by follow formula\".\nI suggest to use 'pre-processing' than 'processing' on line 331.\nLine 343 \"head attention\" should be attention head.\nLine 372, put space between Table and 1 to be Table 1.\nRestructure lines 370-376,  eg. add s in all occurrences of model since the term 'model' is representing plural things. and restructure as stated.\nFor Table 2, replace 'Essays' with 'BigFive'.\nI recommend you should include  a table of ground-truth labels for all datasets. \n"}}, {"review_id": "z5Y7p1DZ19", "reviewer": "Reviewer_QLop", "summary": "", "questions": "Question A: Which attention mechanism was used for the psychological feature encoder?\nQuestion B: How can any attention mechanism work on a sequence of discrete numerical values that do not represent meanings/semantics but just arbitrary values?\nQuestion C: How are the features represented when used as input for the encoder (this perhaps answers Question B too)?\nQuestion D: Can you describe how the datasets have been used for the evaluation (e.g., train/test/validation split, cross-validation (yes/no), distribution of labels in the dataset)?\nQuestion E: As the balance of labels in the datasets not clear, what is the F1 measure of your results?\nQuestion F: Can motivate why it is useful to compare the text and feature embeddings by the cosine similarity?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments. We will correct the typos, add references to the missing literature, and revise the paper according to your suggestions.\nWe will release our code when this paper is accepted. \n\n\nQuestion A: Which attention mechanism was used for the psychological feature encoder?\n\nAnswer A: Self-attention\n\nQuestion B: How can any attention mechanism work on a sequence of discrete numerical values that do not represent meanings/semantics but just arbitrary values?\n\nAnswer B:  We sincerely apologize for the oversight. In our modeling section, we unintentionally omitted a step. Before applying attention mechanisms to discrete numerical features, we utilize a fully connected neural network to generate a feature matrix for these discrete attributes. Then we use transformer’s encoder  to encode it. We deeply appreciate your question, and we will promptly rectify this error in our paper.\n\nQuestion C: How are the features represented when used as input for the encoder (this perhaps answers Question B too)?\n\nAnswer C: We utilize a fully connected neural network to generate a feature matrix for these discrete attributes. These discrete attributes are obtained by three NLP tools such as SÉANCE, TAACO and TAALES.\n\nQuestion D: Can you describe how the datasets have been used for the evaluation (e.g., train/test/validation split, cross-validation (yes/no), distribution of labels in the dataset)?\n\nAnswer D: We have employed a split ratio of 7:2:1 for training, validation, and testing respectively.   For the BigFive Essays dataset,  there are total 2468 items, for the label “O”, there are 1272 items are True and 1196 items are False, for the label “C”, there are 1254 items are True and 1214 items are False, for the label “E”, there are 1277 items are True and 1191 items are False,  for the label “A”, there are 1310 items are True and 1158 items are False,  for the label “N”, there are 1233 items are True and 1235 items are False.  \n\n| BigFive Essays | O     | C     | E     | A     | N     |\n|-------|-------|-------|-------|-------|-------|\n| Ture  | 1272  | 1254  | 1277  | 1310  | 1233  |\n| False | 1196  | 1214  | 1191  | 1158  | 1235  |\n| Total | 2468  | 2468  | 2468  | 2468  | 2468  |\n\nFor Kaggle MBTI dataset there are total 8675 items, for the label “I/E”, there are 6676 items are “I” and 1999 items are “E”, for the label “N/S”, there are 7478 items are “N” and 1197 items are “S”, for the label “T/F”, there are 3981 items are “T” and 4694 items are “F”,  for the label “P/J”, there are 3434 items are “P” and 5241 items are “J”. \n\n| Kaggle MBTI | I/E   | N/S   | T/F   | P/J   |\n|------------------|-------|-------|-------|-------|\n| I or N or T or P | 6676  | 7478  | 3981  | 3434  |\n| E or S or F or J | 1999  | 1197  | 4694  | 5241  |\n| Total            | 8675  | 8675  | 8675  | 8675  |\n\n\nQuestion E: As the balance of labels in the datasets not clear, what is the F1 measure of your results? \nAnswer E:  We are sorry for it, as the baselines did not disclose their F1 scores, we did not report our F1 scores either. \nFor MBTI the F1 values are as follows: “I/E” 0.8697, “N/S”0.8872,  “T/F” 0.8490   and “P/J” 0.7398\n\nQuestion F: Can motivate why it is useful to compare the text and feature embeddings by the cosine similarity?\n\nAnswer F: We argue that the model extracts more psychological features from the text if the embedding of the text is more similar to the embedding of the psychological features, which allows the model to extract more psychological information contained in the text, thereby increasing the accuracy of the model.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper proposes an interesting approach to predict the personality of an author of a text by processing that text.\n- The authors provide a good overview on methods to extract personality-related (linguistic) features from text", "reasons_to_reject": "Missing reproducibility:\n- It is not described or motivated how the attention mechanism on sequences of numerical values works\n- The evaluation experiment is not described (e.g, basic like train/test/validation split ratio or number of folds in case of a cross-validation are missing)\n- The input-representation of the discrete features values to the psychological feature encoder is not described (or at least shown as an example)", "questions_to_authors": "Question A: Which attention mechanism was used for the psychological feature encoder?\nQuestion B: How can any attention mechanism work on a sequence of discrete numerical values that do not represent meanings/semantics but just arbitrary values?\nQuestion C: How are the features represented when used as input for the encoder (this perhaps answers Question B too)?\nQuestion D: Can you describe how the datasets have been used for the evaluation (e.g., train/test/validation split, cross-validation (yes/no), distribution of labels in the dataset)?\nQuestion E: As the balance of labels in the datasets not clear, what is the F1 measure of your results?\nQuestion F: Can motivate why it is useful to compare the text and feature embeddings by the cosine similarity?", "paper_topic_and_main_contributions": "The paper presents the results of an *NLP engineering experiment*. On the one hand it describes PsyAttention as an approach to predict dimensions (e.g., from BigFive or MBTI) of personality of an author of a text using an Psychological Encoder and and a fine-tuned BERT model.\n\nOne contribution is the optimization of the number of features that represent personality in texts. Another contribution is the implementation of a model that consists out of a fine-tuned BERT model and a \"transformer like\" model that uses an attention mechanism to process feature values which are extracted from text.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "- Line 84: use-s ?\n- Line 191: Pys-Attention -> PayAttention\n- Line 262+264: mutli_att -> multi_att ?\n- Line 218+372: Missing space between Table and number (e.g., Table2 -> Table 2)\n- Line 318: kaggle -> Kaggle \n- whole paper: Either always \"Table\" n or always \"table n\" in a sentence (the same for Figure/figure)\n- whole paper: either BigFive, Big Five or big five\n- Figure 4: the caption refers to section 6.2 but the paper has only five sections\n- Figure 5: the font size in the figure is too small\n- Table 2 and 3 should also present the F1 measure\n- Lines 350-369 should be part of Related Work\"\n- Section 4.6: The section does not really fit to the other content of the paper. Better use the space of this section to describe the attention mechanism and the Evaluation in more detail.\n- Line 390-392: The claim that a difference of 0.3% \"proves\" that your model is \"better\" is brave (especially as the evaluation procedure is not described). You should interpret your results more realistic.\n- Missing sections: Discussion of the methods, results and their limitations"}}]

Author rebuttal:

Meta-review: The problem of identifying psychological features was extensively studied, and while improvement is offered by the new method, all in all the result is relatively incremental. During review period, the paper under review has obtained substantial positive feedback, particularly for its high-quality writing and well-articulated methodology. It investigate into a central problem in Natural Language Processing (NLP), offering an innovative approach to predicting an author's personality based on their text. The authors excel in giving a comprehensive overview of methods for extracting personality-related features from text, highlighting the paper's significance in advancing the field.

While the paper is largely commendable, there are areas identified for further refinement, which have minor impacts on its overall quality. For instance, the paper could benefit from greater detail to improve its reproducibility. Including specifics about the attention mechanism on numerical sequences and the input representation for the psychological feature encoder would make the work even more robust.
Reviewer 2's insights, though critical, offer constructive avenues for strengthening the paper. The presentation of BigFive and MBTI models could be clarified by consistently referring to them as datasets, unless a more elaborate explanation is provided. Additionally, the inclusion of details on train/validation/test splits and data statistics could significantly enhance understanding and replicability, while also addressing minor issues like typos and presentation inconsistencies.

Lastly, while it's noted that the research problem has been explored in prior works, the paper still makes a worthwhile, albeit incremental, contribution to the existing literature. The novelty and quality of the paper indicate that with a few refinements, it could serve as an impactful piece in the NLP community. Overall I would like to see this paper into the poster sessions.
Decision: Accept-Findings