## 4 Experiments
### Ablation Study
#### 4.5.3 Comparison of Additive Attention Mechanisms  
In this section, this article will demonstrate the impact of the additive attention mechanism [11], scaled dot-product attention mechanism [27], local attention mechanism [28], adaptive attention mechanism [29], and self-attention mechanism [30] on optimizing news  
Figure 2: Comparison of residual structure and attention layers.  
data. Based on the experimental results presented here, it is possible to determine which mechanism yields the best performance. Table 4 shows the results of each method after five epochs.  
The additive attention mechanism calculates attention weights by computing the similarity between a query vector and a key vector and applying the attention weights to a value vector. Its computation process involves two steps: first, using a neural network or linear transformation to map the query vector and key vector to the same dimension space, and second, computing the similarity between the mapped query vector and key vector, usually using the dot product or other similarity measurement methods. The additive attention mechanism is suitable for short sequences and small-scale attention processing. From the line chart, it can be seen that the performance of the additive attention mechanism gradually improves, which is highly suitable for short-sequence processing tasks such as news headline processing. For scaled dot-product attention, attention weights are calculated by computing the dot product between a query vector and a key vector, scaling it, and then normalizing the scores into attention weights using the softmax function. The self-attention mechanism establishes associations between different positions in the input sequence. It calculates attention weights by mapping queries, keys, and values into different linear transformations, and applies the attention weights to values to generate the final representation. The local attention mechanism is based on distance and is commonly used to handle longer input sequences. It considers the distance between each position in the input sequence and the target position. It calculates attention weights by normalizing the distance, such that positions closer to the target position receive higher attention weights. The local attention mechanism is suitable for tasks with clear positional relationships in the sequence, such as machine translation and speech recognition tasks. Adaptive attention can flexibly control attention by learning weight parameters based on the requirements of the task to adaptively weigh the input features.  
Comparing these mechanisms, it is evident from Figure 3 that the additive attention mechanism yields the best results in our model. Therefore, we consider the additive attention mechanism to be well-suited for the characteristics of short sequences and small-scale attention processing in this study. In comparison to other methods, it is particularly suitable for handling news titles.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Ways**} & \multicolumn{4}{c}{**Evaluation Indicator**} \\ \cline{2-5}  & **AUC** & **MRR** & **nDCG@5** & **nDCG@10** \\ \hline Self-attention mechanism & 65.05 & 30.79 & 33.73 & 40.21 \\ Scaled dot-product attention mechanism & 65.33 & 31.27 & 34.19 & 40.62 \\ Local attention mechanism & 65.54 & 30.75 & 33.65 & 40.29 \\ Adaptive attention mechanism & 65.67 & 31.39 & 34.42 & 40.83 \\ Additive attention mechanism & **66.65** & **32.05** & **35.32** & **41.58** \\ \hline \hline \end{tabular}
\end{table}
Table 4: Results of various optimization mechanisms.