## 3 Proposed Method
### Mask-Gating Attention Layer  
The mask-gating attention [36] mechanism is employed to extract global information from the transformed embedding vectors and dynamically emphasize informative elements. This mechanism provides two key advantages for the model. First, the mask-gating attention mechanism introduces element-wise product operations between the mask and the hidden layer or feature embedding layer. This multiplicative operation efficiently captures complex feature interactions in a unified fashion. By applying the element-wise product, the model can amplify or suppress specific elements based on their importance, allowing for a better representation of intricate feature interactions. This enhances the model's ability to capture and leverage complex relationships between different elements, ultimately improving its overall performance. Second, the mask-gating attention mechanism enables fine-grained bit-wise attention driven by the input codewords. By utilizing this attention mechanism, the model can reduce the impact of noise present in the embeddingand feedforward layers. The attention mechanism allows the model to selectively focus on relevant bits or elements in the input codewords, effectively filtering out noisy or irrelevant information. This fine-grained attention helps enhance the model's discriminative power, as it can prioritize and attend to the most informative elements while disregarding noise or irrelevant details. The structure of the mask-gating attention mechanism is shown in Figure 2.  
The mask-gating attention mechanism consists of two steps. The first step obtains the codeword-element-guided mask \(V_{mask}\) based on two fully connected (FC) layers with identity functions. This mask is generated based on the global contextual information of the input instance and is applied during training to modulate the optimization of parameters in a data-dependent manner. The first FC layer is known as the \"aggregation layer,\" and it is a wider layer than the second FC layer to better capture global contextual knowledge in the transformed codeword vector. The first FC layer has the training weight parameters \(W_{g,1}\), where \(g\) represents the \(g\)-th FC layer. The second FC layer, known as the \"projection layer,\" reduces the dimensionality to the same size as the transformed embedding vector \(\mathbf{V}\), with the parameters \(\mathbf{W_{g,2}}\) as follows:  
\[\mathbf{V}_{mask}=\mathbf{W_{g,2}}(\text{RELU}(\mathbf{W_{g,1}}\mathbf{V}+\mathbf{\gamma_{g,1}}))+ \mathbf{\beta_{g,2}} \tag{9}\]  
where \(\text{RELU}(\cdot)\) denotes the Rectified Linear Unit activation function, the dimension of \(\mathbf{W_{g,1}}\) is (\(B\), \(q\), \(k\)f), and the dimension of \(\mathbf{W_{g,2}}\) is (\(B\), \(u\), \(q\)). \(q\) and \(u\) denote the number of neurons in the aggregation layer and the projection layer. \(\mathbf{\gamma_{g,1}}\) and \(\mathbf{\beta_{g,2}}\) denote the biases of the two FC layers. The second step is to calculate the output of the mask-gating attention layer based on the codeword-element-guided mask and the transformed embedding vector \(\mathbf{V}\). In this work, element-wise products are utilized to incorporate the global contextual information gathered by the codeword-element-guided mask with the embedding layer, as follows:  
\[\mathbf{V_{out}}=\mathbf{V_{mask}}\odot\mathbf{V} \tag{10}\]  
where \(\mathbf{V_{out}}\) denotes the output of the mask-gating attention layer and \(\odot\) denotes the element-wise product of two vectors. The definition of the element-wise product is as follows:  
\[\mathbf{V_{a}}\odot\mathbf{V_{b}}=[V_{a,1}\cdot V_{b,1},V_{a,2}\cdot V_{b,2},\ldots,V_ {a,e}\cdot V_{b,e}] \tag{11}\]  
where \(e\) denotes the size of the vectors \(\mathbf{V_{a}}\) and \(\mathbf{V_{b}}\) (\(1\leq a,b\leq k\)). Figure 2: Structure of the mask-gating attention mechanism.