# Weight Adjustment Framework for Self-Attention Sequential Recommendation
## 4 Experiments
### Hyperparameter Study  
The number of layers and heads in self-attention significantly impacts the model. We analyzed the effect of parameter settings on WAF-SR using the Toys dataset. As shown in Table 4, we fixed the number of heads at two, and as the number of layers increased to the third layer, the gains began to diminish, possibly due to overfitting leading to a decline in model performance. Due to limited experimental equipment resources, we did not stack more layers. We visualized the impact of the attention layer number, as shown in Figure 2. With the number of layers fixed at two, as the number of heads increased (starting from two heads, with performance data identical to when the number of layers was two), the overall performance of the model did not improve. Too many heads might introduce too many parameters, increasing the model's complexity and leading to overfitting or a decrease in learning efficiency. Therefore, when designing models based on self-attention, it is crucial to choose an appropriate number of layers and heads to ensure that the model can maintain sufficient expressive power while avoiding unnecessary computational resource wastage and performance degradation. Through further experimental analysis, we recommend carefully balancing different combinations of layers and heads when applying the self-attention mechanism to find the optimal configuration balance. This balance helps improve model performance while controlling model complexity and computational costs.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Settings**} & \multicolumn{2}{c}{**Recall**} & \multicolumn{2}{c}{**NDCG**} \\ \cline{2-5}  & @**10** & @**20** & @**10** & @**20** \\ \hline
1 layer & 0.0846 & 0.1205 & 0.0376 & 0.0466 \\
2 layers & 0.0849 & 0.1219 & 0.0382 & 0.0474 \\
3 layers & 0.0845 & 0.1223 & 0.0375 & 0.0470 \\ \hline
4 heads & 0.0834 & 0.1229 & 0.0371 & 0.0471 \\
8 heads & 0.0839 & 0.1225 & 0.0376 & 0.0473 \\ \hline \hline \end{tabular}
\end{table}
Table 4: The impact of different numbers of layers and heads in the attention layer of the Toys dataset.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**Recall**} & \multicolumn{2}{c}{**NDCG**} \\ \cline{2-5}  & @**10** & @**20** & @**10** & @**20** \\ \hline WAF-SR & 0.0844 & 0.1258 & 0.0389 & 0.0493 \\ w/o logical & 0.0782 & 0.1176 & 0.0359 & 0.0459 \\ representation & w/o filter layer & 0.0807 & 0.1172 & 0.0363 & 0.0455 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Ablation study on Beauty dataset.  
We analyzed the impact of different values of the hyperparameter \(\delta\) in Equation (28) on the Beauty dataset, and the experimental results are shown in Figure 3. On the Recall@20 metric, the model achieves a superior performance when the hyperparameter \(\delta\) is set to either 0.03 or 0.07. Furthermore, the model obtains an outstanding performance on the NDCG@20 metric at a \(\delta\) value of 0.07. By appropriately adjusting the value of \(\delta\), we can effectively enhance the model's performance in terms of accuracy and personalized recommendations. Considering the performance of the above two indicators, we ultimately set the value of \(\delta\) to 0.07 for the Beauty dataset and to 0.03 for both the Toys and Yelp datasets.