## 4 Experimental Results
### Performance Results of the KPF-BERT and KR-ELECTRA Models  
The experimental results for each model are summarized in Table 11. For a batch size of 24, the KPF-BERT [33] model trained on dialog data achieved the highest performance of 93.4 percent. When examined individually, KPF-BERT showed a PNE detection rate 0.9 percent higher when trained at the conversational level than at the sentence level. KR-ELECTRA [34] showed a detection rate 9.9 percent higher when trained at the sentence level than at the conversational level. This is because KPF-BERT is trained on articles longer than 512 subwords. We trained KPF-BERT to process documents independently by providing a stride to address this issue. Consequently, it performed better in conversational training when we set the maximum sequence length to 512. Owing to its maximum sequence length of 128, KR-ELECTRA is a pretrained model that performs better when trained on shorter sentence units.  
Therefore, performance depends on the maximum sequence length suitable for each model. KPF-BERT, which can handle many tokens when trained on the dialog level,  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multicolumn{1}{c}{**Model**} & \multicolumn{1}{c}{**Batch Size**} & \multicolumn{1}{c}{**Max Seq Length**} & \multicolumn{1}{c}{**Precision**} & \multicolumn{1}{c}{**Recall**} & \multicolumn{1}{c}{**F1-Score**} \\ \hline \multirow{3}{*}{KPF-BERT} & Sentence unit & 24 & 128 & 0.916 & 0.934 & 0.925 \\  & Dialog unit & 24 & 256 & 0.923 & 0.945 & 0.934 \\  & Dialog unit & 4 & 512 & 0.932 & 0.955 & 0.943 \\ \hline \multirow{3}{*}{KR-ELECTRA} & Sentence unit & 24 & 128 & 0.919 & 0.944 & 0.931 \\  & Dialog unit & 24 & 256 & 0.749 & 0.936 & 0.832 \\ \cline{1-1}  & Dialog unit & 4 & 512 & 0.757 & 0.944 & 0.834 \\ \hline \hline \end{tabular}
\end{table}
Table 11: Experiment results of models.  
Figure 2: Example of personal information detection in dialog texts.  
demonstrated the best performance. Therefore, identifying the context of the dialog can be considered helpful in detecting PNEs.  
After confirming the excellent performance of KPF-BERT in dialog-level training, we tuned the hyperparameters to further enhance its performance. After experimenting with batch sizes of 4 and maximum sequence lengths of 512 for both models, KPF-BERT achieved an improved F1-score of 94.3 percent, 0.9 percentage points higher than the previous best performance of 93.4 percent.  
The experimental results for the personal information category of KPF-BERT are summarized in Table 12. According to the results, KPF-BERT showed the highest performance with an F1-score of 0.943. In identifying 33 specific items of personal information, numerical data with patterns or restricted forms of personal information showed a high identification performance of over 90 percent. Specific unique identification information, such as passport and driver's license numbers, blood type, mobile phone number, general phone/fax number, email address, URL, and IP address, demonstrated a 100 percent detection rate, with an F1-score of 1.0. Sixteen items, including date of birth, height, weight, and some additional items, showed a recall of 1.0, indicating a consistent pattern (e.g., passport number: M123A4567, mobile phone number: 010-1234-5678) among these items. However, we observed a few false detections, such as mistaking a date for the date of birth or incorrectly identifying simple criteria unrelated to personal information, such as height or weight. Nicknames, clubs/societies, and places are items with diverse forms and uses, which complicates predictions. Job title/position, although simpler in form than the previous three items, showed relatively low detection rates because many cases that do not correspond to job titles/positions, such as occupations, titles, and relationships, were included.