# Personality Emulation Utilizing Large Language Models
## 4 Results
### LLM Validation Results
#### 4.1.2 Normalization with Standard Deviation  
To address the models' tendency to not provide ratings at either extreme, a few normalization methods were tested. The first attempted normalization method was to adjust the score classification range using the standard deviation by individual trait for each model. The formula used to create the classification range is provided in Equation (1). For this formula \(\mu\) represents the mean score for each trait, N represents an arbitrary value, and \(\sigma\) represents the standard deviation. Therefore, the range can be shifted utilizing some value of N. Two different approaches were used: a static value of N applied uniformly across all traits and models, and another with a dynamic value of N calculated for each trait so that one-third of scores are equally classified as low, neutral, and high.  
\[\text{Classification}(x)=\begin{cases}\text{Low}&\text{if }x<\mu-N\sigma\\ \text{High}&\text{if }x>\mu+N\sigma\\ \text{Neutral}&\text{otherwise}\end{cases} \tag{1}\]  
For static values of N, values of 0.25, 0.5, 0.75, and 1 were used. A lower N value correlates to a smaller neutral range, leading to more scores being classified as either high or low. As a result, since the majority of responses consisted of traits that were either high or low (apart from the three emails that were all neutral traits), a smaller value of N results in a better accuracy for each trait. Utilizing a value of N = 1 results in the lowest accuracy values. Using a dynamic value of N calculated for each model and trait results in an accuracy similar to the value of N = 0.25. With a dynamic value for N, the average accuracy across all models was 60.84% with a 95% confidence interval of 57.9% to 60.84%. The value of N was calculated by for each trait, utilizing the mean and standard deviation to determine what value would result in one-third of values being classified as low, neutral, and high. A comparison of the average accuracy across all models for all five traits is provided in Figure 10.  
More granular analysis of the dynamic N scenario allows for the determination of how the LLM models mispredicted various traits. Figure 11 provides an overview of the proportion of predictions by the LLM model that were completely correct, near misses, or complete errors. The green highlighted column represents the proportion of cases  
Figure 10: Accuracy for different values of N.  
that were predicted correctly, the yellow columns represent predictions one label off, and the red cases represent complete misclassifications. Under dynamic standard deviation normalization, roughly all models represented around one-third of correct high and low responses. This corresponds to the normalization tactic of placing one-third of the scores in each classification category. Compared to other traits, models were less likely to make near-miss errors when predicting neuroticism, reflected by the low proportion of predictions in the yellow \"Near Miss\" column. The proportion of complete error predictions, represented in the red column, varied largely across models and traits.  
Figure 11: Proportion of correct, near-miss, and complete error predictions by model and trait. The green columns match predictions that were completely correct by LLM models (high predicted high, low predicted low, etc.). The yellow highlighted columns represent the near-miss cases, and the red highlighted columns represent the complete error predictions.