# Detecting Personally Identifiable Information Through Natural Language Processing: A Step Forward
## 4 Results
### Experimental Results Obtained from Model Testing  
During the testing phase, we assessed the performance of the models using not just test accuracy but also other well-known metrics like precision, recall, and F1-score. The experimental results are summarized in Table 5. An analysis of the data in the table reveals that the model with the highest performance across all metrics is the one trained using the training-validation-test technique (this appears in the first row of Table 5). This model achieved the following results:  
* Accuracy: 99.558%;
* Precision: 99.564%;
* Recall: 99.558%;
* F1 score: 99.559%.  
These results are very strong, indicating that the model is performing exceptionally well in all aspects of classification.  
On the other hand, the model with the worst performance, based on the same metrics, is the 3-fold model. This model showed the following results:  
* Accuracy: 80.845%;
* Precision: 89.393%;
* Recall: 80.845%;
* F1 score: 82.219%.  
As we can see, the 3-fold model is significantly less effective than the training-validation-test model, with noticeably lower scores in all the metrics.  
To further illustrate these differences, we can refer to the four images in Figure 8. These images show the average values of accuracy, precision, recall, and F1-score for the models. From top to bottom, in a clockwise direction, we can observe the following trend:  
* The model trained with the training-validation-test technique provides the most reliable results overall, with the stratified k-fold being a better alternative than the regular k-fold for ensuring balanced performance in classifying PII. The training-validation-test leads to more accurate and stable classification outcomes.  
\begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Model** & **Accuracy** & **Precision** & **Recall** & **F1-Score** \\ \hline Training-Validation-Test & 99.558\% & 99.564\% & 99.558\% & 99.559\% \\ \hline K-fold 1 & 99.240\% & 99.262\% & 99.240\% & 99.244\% \\ \hline K-fold 2 & 98.904\% & 98.945\% & 98.904\% & 98.912\% \\ \hline K-fold 3 & 80.845\% & 89.393\% & 80.845\% & 82.219\% \\ \hline K-fold 4 & 97.968\% & 98.106\% & 97.968\% & 97.993\% \\ \hline K-fold 5 & 99.346\% & 99.361\% & 99.346\% & 99.349\% \\ \hline Stratified k-fold 1 & 98.480\% & 98.496\% & 98.480\% & 98.466\% \\ \hline Stratified k-fold 2 & 98.657\% & 98.723\% & 98.657\% & 98.669\% \\ \hline Stratified k-fold 3 & 95.953\% & 96.542\% & 95.953\% & 96.057\% \\ \hline Stratified k-fold 4 & 99.454\% & 99.512\% & 99.387\% & 99.485\% \\ \hline Stratified k-fold 5 & 99.417\% & 99.431\% & 99.417\% & 99.419\% \\ \hline \hline \end{tabular}
\end{table}
Table 5: Experimental results obtained during the testing phase.  
* The models trained with the stratified k-fold technique tend to have higher performance across all the metrics compared to the models trained with the standard k-fold method. This suggests that ensuring a balanced distribution of classes within the folds helps improve the model's performance in detecting PII.
* The stratified k-fold technique makes sure that each fold has a similar proportion of each class, which is especially important when dealing with unbalanced data. In contrast, with regular k-fold cross-validation, the data might not be as well balanced across the different folds, which could lead to poorer generalization and performance.