# Weight Adjustment Framework for Self-Attention Sequential Recommendation
## 2 Related Work  
In the early stages of recommender systems, [PERSON] et al. [21] proposed a universal optimization criterion for personalized ranking called BPR, which played a revolutionary role in the research of recommender systems. BPR aims to directly optimize the quality of rankings, rather than the traditional accuracy of rating prediction. Through this method, BPR can more effectively handle implicit feedback data, such as clicks or purchases. With the advancement of deep learning technology, an increasing number of people are applying deep learning techniques to the field of sequential recommendation. Researchers have started to use complex network models such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), and graph neural networks (GNNs). GRU4 Rec [22] was the first to utilize RNNs to model the sequence of user behaviors within a session to predict the user's next possible action. The session-modeling part employs RNNs to capture the sequence of user behaviors within a session, generating a fixed-length vector representation. The prediction part takes this vector representation as input and uses a feedforward neural network to predict the next possible action. [PERSON] [23] models user interests by embedding items in a sequence into a continuous vector space and then applying CNNs to the embedded vectors to extract significant features within the item sequence. [PERSON] et al. [24] proposed a graph-neural-network-based session recommendation model, SR-GNN, aimed at exploring complex transition relationships between items and generating accurate item-embedding vectors. SR-GNN constructs directed graphs from sequences and captures transitions between items through this structure, thereby producing precise item-embedding vectors. Recently, Transformer-based methods for sequential recommendation have emerged in the mainstream, due to their efficient parallel computation capabilities and their significant potential in capturing the sequential behaviors of users. SASRec [13] replaces RNNs with the self-attention mechanism to better capture long-term dependencies and global relationships, efficiently handling long sequences. Although SASRec effectively encodes user historical interactions into hidden vectors for recommendation using a self-attention mechanism, the inherent limitations of its unidirectional model impact the completeness of the hidden representations in the user-behavior sequence. To address this issue, BERT4 Rec [14] employs a deep bidirectional self-attention mechanism to more comprehensively model the user-behavior sequence. This allows each historical item in the sequence to consider the contextual information from both its left and right sides for more accurate recommendations. Similar to the original Transformer model, SASRec is fundamentally a non-personalized model that does not include personalized embeddings for individual users. SSE-PT [25] overcomes the non-personalization limitations of the SASRec model by introducing stochastic shared embedding (SSE) regularization. SASRec and BERT4 Rec, which are recommendation systems based on self-attention networks (SANs), exhibit three main limitations. Firstly, traditional sequential recommendation systems must handle a large number of items, but due to the long-tail distribution of items, many have sparse interaction data. This leads to the insufficient training of em beddings for these infrequently interacted-with items, thereby affecting the accuracy of the corresponding attention weights. Secondly, in the SAN model, the item embeddings and position embeddings are directly added together. This method may introduce noise and limit the model's ability to capture patterns in user-behavior sequences. Thirdly, the SAN model requires direct interactions between all historical items of a user (termed item-to-item interactions), which results in the computational requirements in terms of time and space growing quadratically as the length of the historical sequence increases.