# Deep Learning Framework for Advanced De-Identification of Protected Health Information
## 3 Methodology
### Bi-LSTM-CRF Model  
The idea behind the sequence labeling challenge is to convert an input sequence into a corresponding, identically sized output sequence. As a result, constructing a Recurrent  
\begin{table}
\begin{tabular}{c l c c} \hline \hline \# & **PHI Categories** & **Number of B-Label** & **Number of I-Label** \\ \hline
1 & Date & 12,437 & 1362 \\
2 & Patient & 2184 & 1184 \\
3 & Age & 1993 & 10 \\
4 & Profession & 411 & 346 \\
5 & Medical Record & 1028 & 47 \\
6 & Hospital & 2305 & 1819 \\
7 & Doctor & 4782 & 3466 \\
8 & Street & 350 & 713 \\
9 & City & 651 & 170 \\
10 & State & 502 & 18 \\
11 & ZIP & 350 & 0 \\
12 & Phone & 523 & 100 \\
13 & Id\_Num & 455 & 30 \\
14 & Username & 356 & 0 \\
15 & Organization & 205 & 173 \\
16 & FAX & 10 & 2 \\
17 & Country & 183 & 21 \\
18 & BIOID & 1 & 0 \\
19 & Location-Other & 17 & 15 \\
20 & Device & 15 & 2 \\
21 & EMAIL & 5 & 0 \\
22 & Health Plan & 1 & 1 \\
23 & URL & 2 & 4 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Number of B-labels and I-labels for PHI categories.  
Neural Network with a sufficient number of hidden layers has produced outstanding outcomes for sequence-to-sequence conversions. Here, recurrent connections serve as a kind of memory that allows the network's internal state to hold on to insights from the previous inputs. The label applied to the input data is represented by the final output, which is established by taking previous inputs into account as well. There have been several versions of Recurrent Neural Nets developed, but the LSTM architecture stands out since it solves the problem of disappearing gradients effectively. Therefore, the LSTM network is frequently preferred for labeling words inside possibly long sentences. However, since the whole phrase is known ahead of time, the label assignment of a given word can be carried out more precisely if it takes into account the words that come before and after it. It is more efficient to represent these dependencies by using the Bidirectional LSTM [32]. To analyze data from both directions of the input sequence, the model's network design (as illustrated in Figure 2) includes forward and backward LSTM units. The model is more capable of capturing word dependencies, regardless of where they are in the sentence, due to this bidirectional arrangement. The model is able to comprehend each word's meaning and grammatical structure better as a result of the concatenation of the PoS vectors and the embedding layer output. As each word's label can be predicted individually by the Bi-LSTM, the relationships between labels in the output sequence cannot be captured by the model. One word that is labeled as the beginning of an entity (B) in PoS tagging, for instance, increases the possibility that the following word will be marked as the inner (I) of the same entity. CRF comes in. To guarantee that the predicted labels adhere to correct sequential patterns, the CRF layer is layered on top of the Bi-LSTM. The possibility of label transitions (e.g., from B to I, or from I to O) is taken into consideration to effectively model the relations between the expected labels.