# Personality Emulation Utilizing Large Language Models
## 4 Results
### LLM Validation Results
#### 4.1.1 Non-Normalized LLM Data  
Utilizing no normalization, the average overall accuracy of all LLM models was 42.94%, with a 95% confidence interval range of 39.82% to 46.07%. For this analysis, an LLM score of 0 to 33 was designated low, a score greater than 33 to less than 66 was designated neutral, and a score 66 or greater was designated high. These scores were compared to the true low, neutral, and high trait levels of the emails analyzed to calculate prediction accuracy. The Claude 3 Opus model achieved the highest accuracy of 46.67%, while the Gemma 3 model had the lowest overall accuracy of 39.39%. When comparing individual traits, the LLM models as a whole performed substantially worse on extraversion, scoring only 29.72%, with the next lowest being openness at 43.14%. The accuracy of each model by trait and overall is presented in Table 4.  
As shown previously in Table 3, each LLM model is trained on a specific number of parameters. For open-source models, this count is typically publicly available. For proprietary models such as GPT-4.5 and GPT-4o, however, parameter count is not disclosed. Table 5 provides a comparison of parameter count and LLM model accuracy. As demonstrated, for the models with known parameter counts, there is no clear correlation between parameters and accuracy for detecting personality. The highest-performing models, which are both proprietary models, were GPT-4.5 and Claude 3 Opus. Although the generated emails were meant to represent polar sides of each trait, no LLM scored any trait at a level of zero. Outside of neuroticism, where low meant a more stable individual, only the GPT-4o model provided a score less than 20. The violin plot shown in Figure 8 provides an overview of the distribution of scoring for each trait and model. LLM models have been shown to reflect the cultural values of biases of the language they were trained on, likely leading the LLM models to respond more centrally in their personality scoring [57]. As shown in the plot, models were less likely to provide lower ratings, especially for conscientiousness and agreeableness. For extraversion, the distribution of scoring is more uniform; however, the majority of scores land between 20 and 80. For neuroticism, a lower score meant the response portrayed a more emotionally stable individual. Overall, models rarely provided scores at either extreme, especially for lower scores. This suggests that either the responses did not emulate extremes of any trait, or that the models exhibited a bias to score traits more favorably. The high and low scores by trait for each model are provided in Table 6. Additionally, confusion matrices created from the combined scoring of all LLMs are provided in Figure 9. \begin{table}
\begin{tabular}{l c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**Consient.**} & \multicolumn{2}{c}{**Agreeab.**} & \multicolumn{2}{c}{**Neurotic.**} & \multicolumn{2}{c}{**Openness**} & \multicolumn{2}{c}{**Extrav.**} \\  & **Min** & **Max** & **Min** & **Max** & **Min** & **Max** & **Min** & **Max** & **Min** & **Max** \\ \hline DeepSeek R1 & 40 & 95 & 45 & 95 & 10 & 75 & 20 & 90 & 20 & 85 \\ Gemma 3-27b & 25 & 92 & 30 & 95 & 15 & 85 & 30 & 85 & 20 & 75 \\ GPT-4.5 & 45 & 95 & 40 & 95 & 12 & 85 & 25 & 95 & 20 & 85 \\ GPT-4o & 45 & 90 & 40 & 95 & 10 & 75 & 20 & 95 & 10 & 80 \\ Claude Opus & 60 & 90 & 30 & 95 & 20 & 75 & 20 & 90 & 20 & 80 \\ Hermes 3-405b & 50 & 90 & 30 & 95 & 20 & 80 & 25 & 95 & 20 & 80 \\ Llama 4-Maverick & 35 & 85 & 40 & 90 & 25 & 70 & 30 & 90 & 35 & 90 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Minimum and maximum trait scores for each model across the Big Five traits (non-normalized).