# GoalBERT: A Lightweight Named-Entity Recognition Model Based on Multiple Fusion
###### Abstract  
Named-Entity Recognition (NER) as a core task in Natural Language Processing (NLP) aims to automatically identify and classify specific types of entities from unstructured text. In recent years, the introduction of Transformer architecture and its derivative BERT model has pushed the performance of NER to unprecedented heights. However, these models often have high requirements for computational power and memory resources, making it difficult to train and deploy them on small computing platforms. Although ALBERT as a lightweight model uses parameter sharing and matrix decomposition strategies to reduce memory consumption to some extent consumption, it does not effectively reduce the computational load of the model. Additionally, due to its internal sharing mechanism, the model's understanding ability of text is reduced leading to poor performance in named-entity recognition tasks. To address these challenges, this manuscript proposes an efficient lightweight model called GoalBERT. The model adopts multiple fusion technologies by integrating a lightweight and efficient BiGRU that excels at handling context into part of the Transformer's self-attention layers. This reduces the high computational demand caused by stacking multiple self-attention layers while enhancing the model's ability to process context information. To solve the problem of gradient disappearance and explosion during training, residual connections are added between core layers for more stable training and steady performance improvement. Experimental results show that GoalBERT demonstrates recognition accuracy comparable to standard models with accuracy surpassing ALBERT by 10% in multi-entity type scenarios. Furthermore, compared to standard models, GoalBERT reduces memory requirements by 200% and improves training speed by nearly 230%. Experimental results indicate that GoalBERT is a high-quality lightweight model. named entity recognition; natural language processing; lightweight model; multiple fusion technologies +
Footnote †: journal: applied sciences  
With the increasing length of the text, the ability to understand context becomes more important. Using LSTM and GRU networks effectively enhances contextual understanding. However, as a single-directional recurrent neural network, it cannot comprehend the semantic impact of subsequent text on preceding text. Therefore, improved models such as Bi-LSTM and Bi-GRU bidirectional RNN have been developed [8]. These models not only enhance contextual dependencies but also consider the influence of subsequent text on preceding relevant entity statements, thereby improving overall accuracy. However, the training speed of RNN models is affected by their temporal nature. When trained without restrictions on extremely long texts, errors at each time step are updated through backpropagation, which gradually multiplies weights by chain derivatives leading to issues like vanishing or exploding gradients [9]. Convolutional Neural Networks (CNNs) offer a solution to these problems by leveraging the weight-sharing advantage of convolutional kernels, which reduces computational complexity during data transmission. By allowing multiple convolutional kernels to operate in parallel, CNNs significantly accelerate the training and fitting process [10]. Further analysis suggests that since NER is inherently a classification task, an independent inference module can be added after text feature extraction to present multi-classification results probabilistically. However, if not properly constrained, this approach can still produce incorrect outputs, neglecting the dependencies between labels [11]. Therefore, incorporating Conditional Random Fields (CRF) to learn cross-label dependencies and enhance the validity of model output is a crucial step.