# Biomedical Flat and Nested Named Entity Recognition: Methods, Challenges, and Advances
## 5 Analysis of Entity Types and Span Representation
### Contribution of Re-Context and Decoding Layers
#### 5.3.1 Comparison of Re-Context Layers  
Re-context layers have been employed to obtain additional contextual information; however, to the best of our knowledge, their effectiveness has not been comprehensively evaluated since the advent of PLMs. This section evaluates the effects of six re-context layers on NER tasks: Bi-LSTM, 1-dimension CNN (1 dCNN), 2-dimension CNN (2 dCNN), BiLSTM-attention (BiLSTM-attn), 1 dCNN-attn, and 2 dCNN-attn. Bi-LSTM, 1 dCNN, and attention mechanisms have traditionally been applied to text data, whereas 2 dCNN has primarily been employed in image-related tasks due to its ability to capture local patterns and features. However, since [PERSON] et al. suggested that the localized perspective of 2 dCNN can enhance the performance of NER tasks [75], our NER experiments included it as a re-context layer.  
In our experimental setup, the input dimensions were in the shape of (sentence length, embedding size). The 1 dCNN layer had a kernel size of 5, a padding size of 2, and an output channel size double that of the input size. The 2 dCNN layer was designed with a kernel size of (5, 5), a stride size of (1, 2), and circular padding of (2, 256). The settings for the 2 dCNN layer were referenced in a previous study [75]. The decoding layer utilized a _softmax_ layer for sequence labeling, following the IOB tagging format.  
\begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline
**Span Repr.** & \multicolumn{2}{c}{**GENIA**} & \multicolumn{3}{c}{**BB19**} & \multicolumn{3}{c}{**SciERC**} \\
**Strategy** & **P** & **R** & **F** & **P** & **R** & **F** & **P** & **R** & **F** \\ \hline Concatenation & 78.07 & 76.96 & 77.47 & 78.31 & 72.63 & 75.35 & 65.78 & 67.91 & 66.78 \\ Max-pooling & 78.11 & 76.73 & 77.40 & 79.52 & 73.80 & 76.55 & **67.57** & 68.09 & 67.81 \\ Mean-pooling & **78.84** & 76.37 & 77.58 & **79.92** & **74.89** & **77.31** & 67.52 & 68.48 & 67.98 \\ PL-Marker & 78.34 & **78.01** & **78.16** & 79.17 & 73.25 & 76.09 & 67.53 & **70.27** & **68.86** \\ \hline \hline \end{tabular}  
* **Bold** indicates the best scores, and underline indicates second-best scores.  
\end{table}
Table 4: Comparison of span classification performance according to span representation strategies.  
Table 5 compares the performance of six models with re-context layers to the base model, which is a model without any re-context layer. The majority of models with the re-context layer generally outperformed the base model (except for 2 dCNN and BiLSTM-attn on BC5 CDR chemical); however, the differences from the base model were not significant. Specifically, the differences from the best model were \(-0.43\) in NCBI disease, \(-0.44\) in BC2 GM, and \(-0.39\) in BC5 CDR chemical. We also observed that the performance differences between re-context layers were not substantial. These results suggest that the PLM already contains sufficient contextual information for NER tasks, so the additional re-context layers may offer only marginal improvements. Therefore, the focus should be on optimizing other components of the NER to achieve better performance.