## 3 Proposed Method
### Encoding Layer  
In the encoding layer, we utilize the conventional BiLSTM to understand the fused representation obtained from the representation layer. BiLSTM stands out as an effective variant of RNN, capable of retaining information from both preceding and subsequent neural nodes, allowing for it to capture contextual information and long-distance dependencies within sequences [20].  
The internal structure of the LSTM cell is shown in Figure 4. Each neural node of LSTM consists of an input gate, a forgetting gate, and an output gate. Utilizing these gate control units makes it feasible to determine whether to preserve or discard node-related information at each stage, thereby enabling the detection of long-distance dependencies. Specifically, the input gate is responsible for determining whether to retain the current node's input information, while the forgetting gate determines whether to retain information from the previous neural node's hidden layer. Lastly, the output gate decides whether to pass on the current node's output to the subsequent node.  
Assuming that the output of the hidden layer of the last node is \(h_{t-1}\), and the input of the current node is \(x_{t}\), the forgetting gate \(f_{t}\), the input gate \(i_{t}\), and the output gate \(o_{t}\) can be calculated as Equations (7)-(9):  
\[f_{t}=\sigma(W_{xf}x_{t}+W_{hf}h_{t-1}+b_{f}) \tag{7}\]  
\[i_{t}=\sigma(W_{xi}x_{t}+W_{hi}h_{t-1}+b_{i}) \tag{8}\]  
\[o_{t}=\sigma(W_{xo}x_{t}+W_{ho}h_{t-1}+b_{o}) \tag{9}\]  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Masking Strategy** & **Result** \\ \hline Original Text & The patient was diagnosed with rectal cancer disease \\ BERT & The patient was diagnosed with rectal [MASK]@ \\ RoBERTa-wwm & The patient was diagnosed with rectal [MASK]@ \\ The patient was diagnosed with [MASK] [MASK]@ \\ \hline \hline \end{tabular}
\end{table}
Table 1: An Example of whole word masking mechanism of RoBERTa-wwm.  
where \(\sigma\) is the sigmoid activation function; \(W_{xf}\), \(W_{hf}\), \(W_{xi}\), \(W_{hi}\), \(W_{xo}\), and \(W_{ho}\) are trainable weights, and \(b_{f}\), \(b_{i}\), and \(b_{o}\) are biases.  
With these calculated gates, the memory cell corresponding to the current node can be calculated as Equation (10):  
\[c_{t}=f_{t}c_{t-1}+i_{t}tanh(W_{xc}x_{t}+W_{hc}h_{t-1}+b_{c}) \tag{10}\]  
where \(tanh\) is the activation function; \(W_{xc}\) and \(W_{hc}\) are trainable weights, and \(b_{c}\) is a bias.  
Then, we can obtain the output of the hidden layer of the current node with the output gate \(o_{t}\), which can be calculated as Equation (11):  
\[h_{t}=o_{t}tanh(c_{t}) \tag{11}\]  
The described process outlines the computation of LSTM. BiLSTM comprises both a forward LSTM and a backward LSTM, allowing for the simultaneous capture of information from both the preceding and following moments. It can be calculated as Equations (12)-(14):  
\[\overset{\rightarrow}{h_{t}}=LSTM(x_{t}) \tag{12}\]  
\[\overset{\leftarrow}{h_{t}}=LSTM(x_{t}) \tag{13}\]  
\[h_{t}=\begin{bmatrix}\overset{\rightarrow}{h_{t}},\overset{\leftarrow}{h_{ t}}\\ \end{bmatrix} \tag{14}\]  
where \(\overset{\rightarrow}{h_{t}}\) and \(\overset{\leftarrow}{h_{t}}\) denote the outputs in both directions.