## 3 Methodology
### Sentiment Lexicon-Enhanced Dynamic Modulation Layer
#### 3.3.3 Proposed Activation Function  
This section provides a detailed explanation of the proposed activation function \(\mathbf{u}_{i}=\sigma(\mathbf{e}_{i})\) with sentiment-oriented dynamic weights. It is the core component of the dynamic modulation layer, aimed to enhance the sensitivity and responsiveness of models based on Hard-prompt. Considering the characteristic of the uniform distribution of \(k_{i}\in\ [-1,1]\) and after several rounds of verification, \(\tanh(x)\) is chosen as the primary activation function. \(\tanh(x)\) has good gradient properties, is smooth and differentiable within its domain, and can provide continuous gradient information, which is conducive to the stable training and optimization of the model. Compared with asymmetric activation functions (such as \(\text{ReLU}\in[0,\infty)\)), it may not effectively handle negative sentiment tendencies. Compared with symmetric functions (e.g., \(\text{sigmoid}(x)\)), \(\tanh(x)\) has a more significant gradient in the area close to 0, which helps alleviate the problem of vanishing gradients. Let \(x=\mathbf{e}_{i}\). The mathematical expression of the activation function is  
\[\sigma(x)=\text{Scale}\cdot W_{\text{norm}}(x)\cdot\tanh(x) \tag{12}\]  
Scale is a learnable scaling factor, initialized to 1 and optimized through backpropagation during the model training process. \(W_{\text{norm}}(x)\) represents the normalization of the weight function, ensuring that the output is within a controlled range. \(W_{\text{min}}\) and \(W_{\text{min}}\), respectively, denote the maximum and minimum values of the weight function \(W(x)\). This step is crucial for maintaining the stability of the model, especially when the range of input values is large, and it is defined as follows:  
\[W_{\text{norm}}(x)=\frac{W(x)-W_{\text{min}}}{W_{\text{max}}-W_{\text{min}}} \tag{13}\]  
The \(W(x)\) represents the original weight function, defined as  
\[W(x)=e^{a|x-x^{3}|+b|x+x^{3}|} \tag{14}\]  
The design of \(|x\pm x^{3}|\) is to provide a nonlinear transformation based on a continuous function so that \(f(\mathbf{e}_{i})\) has different responses to inputs of different sentiment polarities. This is considered an adaptive mechanism that enhances the model's sensitivity to specific sentiment polarities by adjusting the input values nonlinearly. Specifically, for large input values, the cubic root transformation smooths the effect, preventing the vanishing gradient problem. Conversely, for small input values, the transformation amplifies the impact of the input, allowing the model to more effectively process weak sentiment signals. Parameters \(a\) and \(b\) are tuned through _Bayesian Optimization_ to ensure the model can adapt to various sentiment polarities under different input conditions \(\mathbf{e}_{i}\). These parameters control the sensitivity of the activation function to input values, dynamically balancing the response of the model to both weak and strong sentiments, thus enhancing the robustness and adaptability. Figure 3 illustrates the sentiment score distributions of two real app review cases (Review 1 labeled as negative, Review 2 labeled as positive), demonstrating the effect of the lexicon-enhanced dynamic modulation layer in refining token-level sentiment representations through supervised learning. In each sub-figure, the top row (labeled \"Lexicon Score\") displays the dimensionality-reduced initial sentiment scores derived from the external lexicon. The bottom row (labeled \"Modulation\") shows the output after dynamic activation modulation. In Review 1 (negative), highly negative tokens are retained with strong sentiment values, such as \"not\" (\(-1.00\rightarrow-0.76\)), while misleading positive tokens, such as \"like\" (\(0.70\rightarrow\)\(0.36\)) and \"friendly\" (\(0.55\rightarrow\)\(0.21\)), are downweighted.