# GoalBERT: A Lightweight Named-Entity Recognition Model Based on Multiple Fusion
## 3 Methods
However, employing ALBERT's strategy does not actually lower the computational resources needed, as the computational load of each Transformer layer mainly depends on the self-attention mechanism and the feed-forward network, both of which are unrelated to the number of parameters. Therefore, the computational complexity of each layer remains unchanged. Furthermore, regarding training time, the model's overall training process is like that of BERT, with the computational loads for forward and backward propagation largely the same as those of the standard BERT model. Simply reducing the number of parameters does not substantially decrease training time. Additionally, the parameter-sharing mechanism limits the model's ability to learn from more diverse textual content simultaneously, resulting in a significant reduction in accuracy for entity recognition tasks on long texts. In summary, the parameter-sharing and matrix decomposition strategies mainly reduce the model's memory requirements, allowing deployment on devices with limited memory capacity. However, they do not significantly reduce the actual computational load or overall training time compared to the original BERT, and the model's performance sharply declines when processing long texts, making it less suitable for high-quality NER tasks. To thoroughly address the issues of ALBERT's model strategy not genuinely reducing computational load and not significantly decreasing training time, we focused on the internal structure of the model. After analysis, it became clear that most of the computational load is occupied by the Transformer modules within the model, which contain numerous self-attention layers. These layers, requiring extensive computational power and memory support, are a major source of the burden. Therefore, we adopted a multi-fusion approach by replacing some of the self-attention mechanisms with BiGRU layers while maintaining the total number of layers in the model. This approach reduces the overall computational load required by the original Transformer modules. On the other hand, the excellent gating mechanism of BiGRU effectively handles long-range dependencies, thereby enhancing the model's ability to process long texts. This technique can simultaneously utilize both past and future information to capture the global context. The structure of the GRU consists of an update gate and a reset gate: the update gate determines how much of the previous time step's information should be retained in the current hidden state and how much new information should be accepted, while the reset gate decides how much of the previous time step's information is retained. Figure 2: GoalBERT model One-hot process. time step's information should be forgotten when calculating the candidate hidden state at the current time step. The GRU's update process is shown in Equations (1)-(4). \[z_{t} =\sigma(W_{z}\cdot[h_{t-1},x_{t}]) \tag{1}\] \[r_{t} =\sigma(W_{r}\cdot[h_{t-1},x_{t}])\] (2) \[\widehat{h_{t}} =\tanh(W_{h}\cdot[r_{h}.h_{t-1},x_{t}])\] (3) \[h_{t} =(1-z_{t})\cdot h_{t-1}+z_{t}.\widetilde{h_{t}} \tag{4}\]  
Among them, \(z_{t}\) is the update gate, \(r_{t}\) is the reset gate, \(\widetilde{h_{t}}\) is the candidate state, and \(h_{t}\) is the current state. We employ a multi-integration approach to combine BiGRU with the Transformer, fully leveraging the global information capture capability of the Transformer model and the local pattern capture ability of BiGRU. This enhances the model's feature representation and generalization capabilities. Compared to the traditional method of integrating BiGRU at the Transformer output, our approach significantly improves the model's expressive power. By combining BiGRU with the self-attention mechanism of the Transformer, the model's understanding and expression of complex semantics are further enhanced. In contrast, the traditional integration method only allows a limited amount of data to be processed for feature extraction by BiGRU, thereby failing to fully utilize the complementary strengths of both components.