# Biomedical Flat and Nested Named Entity Recognition: Methods, Challenges, and Advances
## 5 Analysis of Entity Types and Span Representation
### Contribution of Re-Context and Decoding Layers
#### 5.3.2 Comparison of Decoder Layers  
In this section, we trained and evaluated four decoding layers frequently used in NER models: _softmax_, CRF, RNN, and span labeling. _Softmax_, CRF, and RNN are employed for the sequence labeling approach, which is common for flat NER tasks. On the other hand, the span labeling layer is not as commonly used for flat NER. Nonetheless, we included it in the experiment to observe the adaptability of the span labeling approach to flat NER.  
_Softmax_ is favored for its computational efficiency and straightforward implementation. It is used to assign probabilities to each class for individual tokens independently. On the other hand, CRF and RNN capture the dependencies between tokens. CRF learns transition probabilities between adjacent tokens, capturing that certain tags are likelier to follow others. RNN maintains information about previous tokens when predicting the tag of the current token, capturing sequential dependencies. Both CRF and RNN require higher computational costs than _softmax_. On the other hand, the span labeling layer was adapted for flat NER tasks in this experiment. The model with a span labeling layer enumerated candidate spans of up to 8 words, generating span representations using a concatenation strategy. The model then classified these candidate spans based on their representations. To ensure a flat structure, nested entities were consolidated by prioritizing the later and longer spans that are classified as entities.  
According to Table 6, the RNN and CRF exhibited high overall performance. Specifically, RNN performed best on the NCBI disease and BC5 CDR chemical datasets, and CRF secured the second-best performance across all datasets. This indicates that dependency information between tokens is valuable in the NER model. Notably, the span labeling approach showed low performance on the NCBI disease and BC5 CDR chemical datasets, achieving high performance only on BC2 GM. There is even a +0.86% difference from _softmax_. These results are intriguing, considering the simplicity of the post-process of handling nested entities. However, it should be noted that the span labeling layer incurs a higher computational cost than CRF and RNN due to the generation and classification of numerous candidate spans. Therefore, CRF and RNN can be partial and effective choices when considering both computational cost and performance.  
\begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline \multirow{2}{*}{
\begin{tabular}{c} **Re-Context** \\ **Layer** \\ \end{tabular} } & \multicolumn{3}{c}{**NCBI Disease**} & \multicolumn{3}{c}{**BC2 GM**} & \multicolumn{3}{c}{**BC5 CDR Chemical**} \\  & **P** & **R** & **F** & **P** & **R** & **F** & **P** & **R** & **F** \\ \hline Base model & 86.14 & 89.10 & 87.60 & 82.85 & 84.08 & 83.46 & 92.36 & 93.46 & 92.90 \\ \hline \(1\)dCNN & **86.93** & 89.08 & 87.99 & 82.98 & 84.14 & 83.55 & 92.67 & **93.89** & 93.28 \\ \hline \(2\)dCNN & 86.70 & 88.94 & 87.80 & 83.19 & 84.44 & 83.81 & 92.42 & 93.29 & 92.86 \\ \hline BiLSTM & 86.91 & 89.19 & **88.03** & 83.05 & 84.17 & 83.60 & 92.72 & 93.38 & 93.05 \\ \hline \(1\)dCNN-attn & 86.73 & **89.31** & 88.00 & 83.02 & **84.64** & 83.82 & **92.99** & 93.28 & 93.13 \\ \hline \(2\)dCNN-attn & 86.33 & 88.96 & 87.63 & 83.02 & 84.30 & 83.65 & 92.84 & 93.75 & **93.29** \\ \hline BiLSTM-attn & 86.52 & 89.21 & 87.84 & **83.43** & 84.37 & **83.90** & 92.41 & 93.28 & 92.84 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Comparison of the flat NER performance according to the re-contextualization layer.