# Detecting Personally Identifiable Information Through Natural Language Processing: A Step Forward
## 3 Materials and Methods
### Model Testing  
After training was completed, the models underwent rigorous testing using a separate dataset, distinct from the training set. The goal of this test was to assess the system's performance on data that it had not encountered before. Various metrics, such as precision, recall, and F1-score, were used to evaluate the model's effectiveness. During this phase, the \begin{table}
\begin{tabular}{l l l} \hline
1. & Name [SS] & 28. & Family member names [SS] \\
2. & Email [VS] & 29. & Place of birth [SS] \\
3. & Phone number [VS] & 30. & Citizenship [SS] \\
4. & Credit card number [VS] & 31. & Gender [SS] \\
5. & Passport number [VS] & 32. & Sexual orientation [SS] \\
6. & Social security number [VS] & 33. & Race/ethnicity [SS] \\
7. & Date of birth [SS] & 34. & Nationality [SS] \\
8. & Address [SS] & 35. & Religious beliefs [SS] \\
9. & Zip code [SS] & 36. & Political affiliation [SS] \\
10. & IP address [VS] & 37. & Professional license number [SS] \\
11. & Bank account number [VS] & 38. & Education history [SS] \\
12. & Routing number [SS] & 39. & Employment history [SS] \\
13. & Driver’s license number [SS] & 40. & Income level [LS] \\
14. & License plate number [SS] & 41. & Financial status [LS] \\
15. & Vehicle identification number [LS] & 42. & Social media handles [LS] \\
16. & Bank account details [VS] & 43. & Account numbers [VS] \\
17. & Medical record number [VS] & 44. & Transaction history [VS] \\
18. & Health insurance number [VS] & 45. & Digital signature [VS] \\
19. & Employee number [SS] & 46. & Purchase history [SS] \\
20. & Student ID [SS] & 47. & Subscription information [SS] \\
21. & Government ID [SVS] & 48. & Health information (e.g., conditions, treatments) [VS] \\
22. & Fingerprint [VS] & 49. & Emergency contact information [LS] \\
23. & Voiceprint [VS] & 49. & Insurance policy number [VS] \\
24. & Face image [VS] & 50. & Academic records [SS] \\
25. & Geolocation [VS] & 51. & Academic records [SS] \\
26. & Biometric data [VS] & 52. & Tax identification number [LS] \\
27. & Mother’s maiden name [SS] & 53. & License key or serial number [SS] \\  & & 54. & Location data (GPS) [VS] \\ \hline \end{tabular}
\end{table} 
Table 2: PII classes present in the pii-masking-200k dataset. Figure 4: Composition of the (re-)training dataset. impact of data variability on the model's behavior was also analyzed, and response times were considered to ensure the system could operate in real-time or near-real-time scenarios. We put effort into creating a testing dataset, separate from the training dataset, that is truly representative. To do this, we collected data from the following four publicly available and well-documented sources:  
* \"pii-masking-43k\" [41]: This dataset, released by Ai4 Privacy, is similar to the larger pii-masking-200k dataset. We chose it because it is known for its reliable data. It has been used to optimize the Distilled BERT model, achieving impressive results: precision (99.86%), recall (99.89%), and accuracy (99.45%). We included it as the core part of the testing dataset since it provides different types of data compared to the larger pii-masking-200k dataset. In past research, the pii-masking-43k dataset has been used to fine-tune various models, including RoBERTa and GPT-2. The dataset provides pre-trained models, which serve as a starting point to build effective PII detection tools. * \"Dialog Dataset\" [42]: This dataset contains chatbot conversations without any PII. We used it to add variety to our test data, selecting 3726 records to make sure the model could handle different formats of conversational data. * \"Movie Review\" [43]: Known as the \"IMDB Dataset of 50K Movie Reviews,\" this is a popular dataset for text classification. It is often used for sentiment analysis of movie reviews, with comments labeled as \"positive\" or \"neutral.\" We checked that no PII was included, and we selected 10,000 records to make sure the model handles sentiment detection correctly while avoiding privacy issues.