# Deep Learning Framework for Advanced De-Identification of Protected Health Information
## 4 Results and Discussion
### CRF Model
\begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Categories** & **Precision (P\%)** & **Recall (R\%)** & **F1-Score (F1\%)** & **Support** \\ \hline B-AGE & 0.56 & 0.48 & 0.52 & 764 \\ B-City & 0.76 & 0.51 & 0.61 & 260 \\ B-Country & 0.68 & 0.11 & 0.19 & 117 \\ B-Date & 0.99 & 0.98 & 0.98 & 4980 \\ B-Device & 0 & 0 & 0 & 8 \\ B-Doctor & 0.65 & 0.57 & 0.60 & 1912 \\ B-Email & 0 & 0 & 0 & 1 \\ B-FAX & 0 & 0 & 0 & 2 \\ B-Hospital & 0.87 & 0.59 & 0.71 & 875 \\ B-ID\_Num, & 0.98 & 0.76 & 0.86 & 195 \\ B-Location & 0 & 0 & 0 & 13 \\ Other & & & & \\ B-Medical & 0.97 & 0.96 & 0.96 & 422 \\ Record & & & & \\ B- & & & & \\ Organization & 0.50 & 0.07 & 0.13 & 82 \\ B-Patient & 0.53 & 0.25 & 0.34 & 879 \\ B-Phone & 0.96 & 0.93 & 0.94 & 215 \\ B-Profession & 0.59 & 0.21 & 0.31 & 179 \\ B-State & 0.91 & 0.76 & 0.83 & 190 \\ B-Street & 0.98 & 0.92 & 0.95 & 136 \\ B-Username & 0.99 & 0.90 & 0.94 & 92 \\ B-ZIP & 0.99 & 0.96 & 0.97 & 140 \\ I-Age & 0 & 0 & 0 & 1 \\ I-City & 0.83 & 0.36 & 0.50 & 81 \\ I-Country & 0 & 0 & 0 & 13 \\ I-Date & 0.95 & 0.88 & 0.91 & 509 \\ I-Device & 0 & 0 & 0 & 2 \\ I-Doctor & 0.73 & 0.74 & 0.74 & 1350 \\ I-FAX & 0 & 0 & 0 & 1 \\ I-Health Plan & 0 & 0 & 0 & 0 \\ I-Hospital & 0.93 & 0.78 & 0.85 & 700 \\ I-ID\_Num & 1.00 & 0.57 & 0.73 & 14 \\ I-Location & & 0 & 0 & 7 \\ I-Medical & & & & \\ Record & 0 & 0 & 0 & 22 \\ I-Organization & 0.44 & 0.18 & 0.26 & 61 \\ I-Patient & 0.54 & 0.30 & 0.38 & 479 \\ I-Phone & 0.96 & 0.94 & 0.95 & 48 \\ I-Profession & 0.77 & 0.38 & 0.51 & 156 \\ I-State & 0 & 0 & 0 & 15 \\ I-Street & 0.91 & 0.92 & 0.92 & 280 \\ Other “O” & 0.99 & 1.00 & 0.99 & 301,937 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Evaluation findings for each of the 23 PHIAA categories using the CRF model. When analyzing the results according to B-tags and I-tags as presented in Table 7 and Figure 5, the B-tags indicate the beginning of entities; hence, it is critical to precisely define these boundary errors in determining the beginning of an entity, as they may result in a series of I-tag mistakes, which reduces its overall effectiveness. The I-tags often consist of fragments of longer or more complex items. Misclassifications are more common when an entity is extended (as shown by I-tags), especially when the model has difficulty recognizing contextual details. Moreover, several I-tags show noticeably lower frequencies of occurrence (that is, fewer instances in the dataset). As an illustration, categories like I-AGE and I-COUNTRY show shallow support (with only 1 or 13 instances), which reduces the model's performance. However, compared to the extension of the entity with I-tags, the identification of the initiation of an entity with B-tags typically performs better since it is frequently less complicated and dependent on contextual clues. Furthermore, the dataset's restricted frequency of I-tag categories might exacerbate I-tags' poor results compared to B-tags. Figure 4: Results of metrics for categories with over 1000 records. Figure 3: Support in each category without other in CRF model. The model's performance, however, shows a strong bias towards high-frequency labels, particularly the \"Other\" (O) label, which constitutes 95% of the dataset records as shown in Table 5 and Figure 3. This label achieves high precision and an F1-score of 99%, indicating that the model is highly accurate for the \"Other\" class. Similarly, categories with higher support counts, such as B-DATE, B-DOCTOR, and I-DOCTOR, also display strong metrics as presented in Figure 4. However, the model struggles with lower-frequency categories, particularly those labeled with I-tags, which represent internal segments of entities and often require more nuanced contextual understanding. The infrequent occurrence of certain I-tags, such as I-AGE and I-COUNTRY with only 1 and 13 instances, respectively, hinders the model's ability to generalize effectively for these classes.