# Named Entity Recognition Networks Based on Syntactically Constrained Attention
## 4 Materials and Methods
### Results  
The experimental results were compared with the following model as a baseline. It is worth noting that since two strategies were designed, in order to obtain syntactically constrained attention \(A_{SC}^{i}\), only the model with the better overall results, based on the hard-pruning strategy, is compared here with the baselines. The baseline models we chose include the following: (1) [PERSON] et al. (2016) regarded nested NER as a sequence-to-sequence problem. (2) [PERSON] et al. (2017) proposed a generic method for information extraction using dynamically constructed span graphs with shared span representations. (3) [PERSON] et al. (2017) transformed the NER task into an MRC task by formalizing entity extraction as a question that extracts the answer span of the question. Since different queries are proposed for different entity types, this paradigm is naturally able to alleviate the entity nesting problem. (4) [PERSON] et al. (2018) used the idea of parsing graphs' dependency tree relttionships, then provided a global input via the biaffine model, and, finally, scored the start and end tokens in the sentence separately. (5) [PERSON] et al. (2019) injected the embedding of semantic-type words into the entity embedding, in order to reduce the differences in context commonalities, and then combined them with the existing entity embedding through linear aggregation to obtain good results. (6) [PERSON] et al. (2020) described the NER task as an entity generation sequence task, without specifically designing the markup schema or using the enumeration span approach. (7) [PERSON] et al. (2021) first traversed all possible text spans to identify entity fragments, and then converted all entity fragments into a relational classification task, in order to determine as to whether entity fragments were overlapping or inherited. (8) [PERSON] et al. (2021) \begin{table}
\begin{tabular}{c c c c} \hline \hline  & \multicolumn{3}{c}{**Sentences**} \\ \cline{2-4}  & \# **Train** & \# **Dev** & \# **Test** \\ \hline
**ACE2004** & 6200 & 745 & 812 \\
**ACE2005** & 7294 & 971 & 1057 \\
**GENIA** & 18,546 & 15,023 & 1669 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Dataset Statistics.“#” denotes the amount. \begin{table}
\begin{tabular}{c c} \hline \hline
**Hyper-Parameter** & **Value** \\ \hline Weight Initialization & BERT \\ Batch-size & 16 \\ Learning rate & 2.00 \(\times\) 10\({}^{-5}\) \\ Warmup Proportion & 0.1 \\ Optimizer & AdamW \\ epoch & 20 \\ Dropout rate & 0.2–0.4 \\ Max Sequence Length & 300 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Hyperparameters of the model, where the ACE2004 and ACE2005 datasets use BERT-base-uncased and GENIA uses BioBERT-base-cased-v1.2. designed a representation method based on a pointer network to parse nested named entities. Table 4 presents the performance of SCAN-Net against baseline approaches on the ACE04, ACE05, and GENIA datasets, demonstrating the best F1 scores achieved by our model can be compared to current advanced nested NER models. Notably, SCAN-Net outperforms [PERSON] et al. [1] by 0.28%, 0.79%, and 1.54% on the ACE04, ACE05, and GENIA datasets, respectively. This improvement is reasonable, since the syntactic constraint focus of SCAN-Net helps the model reduce the interference of redundant information in sentences, allowing it to pay more attention to important information in the context. This represents a significant improvement over models based on machine reading comprehension frameworks. Moreover, there is a substantial improvement of 1.54% observed for performances on the GENIA dataset. We speculate that since BioBERT is a biomedical text pretraining model that is based on BERT, it uses a large corpus of biomedical literature during the pre-training process, introducing knowledge embedded in the biomedical domain. Therefore, BioBERT performs exceptionally well on the biomedical named entity recognition dataset GENIA.