# Encoding Text Information with Graph Convolutional Networks for Personality Recognition
## 5 Experiment
### Datasets and Evaluation  
(1) Datasets:  
The experimental data in this paper were two different datasets: myPersonality [38] and essays. The first dataset, myPersonality, included data from 250 Facebook users who had approximately 9917 states and contained a given personality label based on the Big Five personality model. It included users' status information and external information (like release time, network size, etc.). In this paper, we only used the textual information of the users in the dataset. The personality trait distributions of the myPersonality dataset are shown in Table 2.  
The second dataset, essays, included 2468 anonymous essays that were labeled with personality traits for each author. We removed essays containing only the information \"Err: 508\" from the dataset and experimented with the remaining 2467 essays [39]. The personality trait distributions of the essays dataset are shown in Table 3.  
All data went through a preprocessing stage before the experiment. The preprocessing steps included removing URLs, symbols, names, spaces, lowercase letters, stemming, stop words, etc. The data in Bahasa underwent an additional pre-processing process; this was replacement of slang words or non-standard words, which was manually executed. After finishing, they were then translated into English. In the experiment, both the validation and test sets were ten percent of the dataset. The data ratio of the training set, the validation set, and the test set was 8:1:1. We removed words that appeared fewer than five times in the corpus. We found that deleting words that occurred less frequently helped improve performance and saved time. However, when deleting words that occurred fewer than ten times in the corpus, the performance of the model decreased. We think this was because words occurring nearly ten times belonged to a higher frequency, which could affect the performance of personality recognition. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Value** & **OPN** & **CON** & **EXT** & **AGR** & **NEU** \\ \hline Yes & 176 & 130 & 96 & 134 & 99 \\ No & 74 & 120 & 154 & 116 & 151 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Distributions of the myPersonality dataset. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Value** & **OPN** & **CON** & **EXT** & **AGR** & **NEU** \\ \hline Yes & 1271 & 1253 & 1276 & 1310 & 1233 \\ No & 1196 & 1214 & 1191 & 1157 & 1234 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Distributions of the essays dataset. (2) Evaluation:  
Following the previous study [16; 17; 26; 39; 40; 41; 42; 43; 44; 45; 46; 47; 48], which achieved the best performance, we chose the accuracy and F1-score as the primary metrics for evaluating the performance of our models. Specifically, this paper reports the highest accuracy and F1-score of each trait for each dataset with each method. The accuracy rate is our most common evaluation index; generally, the higher the accuracy is, the better the classifier. Precision and recall indicators sometimes conflict, so they need to be considered comprehensively. The most common method is the F-measure (also known as the F-score). The F-score is the harmonic average of precision and recall. The calculation formulas of the F-score and accuracy are as follows:  
\[\text{accuracy }=(\text{TP}+\text{TN})/(\text{TP}+\text{TN}+\text{FP}+\text{ FN}) \tag{12}\]  
\[\text{precision }=\text{TP}/(\text{TP}+\text{FP}) \tag{13}\]  
\[\text{recall }=\text{TP}/(\text{TP}+\text{FN}) \tag{14}\]  
\[F-\text{score }=\frac{\left(\alpha^{2}+1\right)\times\text{{preci}}\,\text{ {sion}}\times\text{recall}}{\alpha^{2}(\text{precision}+\text{recall})} \tag{15}\]  
where \(TP\) indicates the number of positive samples that are correctly predicted. \(TN\) is the number of negative samples that are correctly predicted. \(FP\) represents the number of negative samples that are wrongly predicted. \(FN\) is the number of positive samples that are wrongly predicted. The most common \(F1\) is \(\alpha=1\).