## 4 Experiment and Result
### Experimental Setup  
This study conducted extensive experiments on two datasets, converting text into the input format required by the BERT model, using the BERT pre-trained model as the basis, and then connecting a recurrent neural network (RNN) and an attention mechanism to capture deeper contextual information in the sequence. New memory units (GLMU and RMN) are introduced to enhance the ability to capture long-distance dependencies. Fully connected layers are used for feature integration and classification. This study uses a cross-validation method to ensure the reliability of the results. Each dataset is divided into five subsets, four of which are used for model training and one for validation. This process is repeated five times for each dataset, rotating the validation set and training set to cover all the data. This method improves the stability and reliability of the experimental results. In order to evaluate the effectiveness of the proposed model, we conducted comparative experiments with several pre-trained models as follows:  
1. BERT-base [23]: BERT is a pre-trained language model developed by the Google AI Language team based on Transformer architecture. It utilizes a bidirectional encoder to understand contextual relationships in text. BERT-base consists of 12 Transformer layers, each with 768 hidden units and 12 self-attention heads. 2. BERT-wurm [24]: BERT-wurm, developed by the Xunfei Joint Laboratory of HIT University, is a variant of BERT that employs the whole-word masking strategy during pre-training. This approach masks entire words rather than just parts of words. 3. MacBERT-base [25]: MacBERT, an improved version of BERT, incorporates several enhancements including a revised masking strategy and a denoising task to bolster the model's robustness and generalization. MacBERT-base uses \"word-level masking\" and \"sentence reconstruction\" strategies to better learn contextual semantics during pre-training. 4. RoBERTa [26]: RoBERTa, developed by Facebook AI, is an enhanced version of BERT that optimizes the pre-training process. Improvements include utilizing a larger training dataset, extending training duration, eliminating the next-sentence prediction (NSP) task, and dynamically varying the masking pattern. 5. BERT-wurm-ext [27]: BERT-wurm-ext, developed by the Xunfei Joint Laboratory of HITC, is an extended version of BERT-wurm. It enhances BERT-wurm by utilizing a larger Chinese corpus for pre-training while continuing to apply the whole-word masking strategy. We validate the effectiveness of our proposed Chinese medical named entity recognition model, which integrates context-dependent perception and novel memory units, by comparing it with existing methods. This comparison is conducted using the MCSCSet and CMeeE datasets. The experimental results are presented in Tables 4 and 5. Our model, BERT-GLMU+RMN-FC-CRF, achieves optimal precision, recall, and F1 scores on the MCSCSet dataset under identical parameter settings, demonstrating its superior performance in named entity recognition. The experimental results show that BERT-GLMU+RMN-FC-CRF performs exceptionally well on both datasets, achieving F1 scores of 91.53 and 64.92, significantly outperforming other models such as MacBERT-large, BERT-base-CRF, RoBERTa-CRF, BERT-Biaffine, Span-level NER, and PSA. This demonstrates that our proposed model offers substantial advantages in processing medical texts, particularly in recognition accuracy and recall. This advantage is attributed to innovations in the model structure, such as the integration of RNNs with a multi-head attention mechanism and the introduction of novel memory units and fully connected layers. The multi-head attention mechanism allows the model to better discern the importance of each entity and the complex relationships between them, which is especially effective in handling multi-dimensional entity relationships and rich semantic hierarchies in medical texts.