# Weight Adjustment Framework for Self-Attention Sequential Recommendation
## 3 Methodology
### Logical Representation Layer  
Previous sequential recommendation models such as Locker (2016), BERT4 Rec (2017), and SASRec (2017) generally utilized traditional embedding representations. Traditional embedding representations often fail to fully capture the dynamic changes and uncertainties in user tastes, which may lead to suboptimal performance in recommendation systems in understanding both long-term and short-term user preferences. To address this issue, we have introduced a logical representation method to model the complexity of user behavior and preferences more accurately. In our model, the logical representation plays a crucial role; it uses a transformation matrix to map item IDs to a logical space that is better suited for representing and handling the uncertainties of user preferences. Then, multiple independent beta distributions are used to represent each item. The beta distribution is a continuous probability distribution defined within the range [0, 1], which simulates the uncertainty or variability of user preferences for items. The beta distribution is defined as follows:  
\[p_{[(\alpha,\beta)]}(x)= \frac{1}{B(\alpha,\beta)}x^{\alpha-1}(1-x)^{\beta-1} \tag{19}\]  
where \(x\in[0,1]\), and the shape parameters \(\alpha,\beta\in[0,\infty]\) determine the shape of the beta distribution, including its peak position, degree of skewness, and tail thickness. In other words, the shape parameter affects the shape and properties of the distribution, which, in turn, can be used to model uncertainty in user preferences for items. \(B(\alpha,\beta)\) is the beta function. The embedding matrix of input item ID, through two transition matrices, \(W_{\alpha}\) and \(W_{\beta}\in R^{d\times d}\), transforms the item-embedding matrix \(e^{u}\) into two shape matrices, \(\alpha\) and \(\beta\in R^{m\times d}\), where m is the length of the user-interaction sequence:  
\[\begin{cases}\alpha=e^{u}\text{W}_{\alpha}\\ \beta=e^{u}\text{W}_{\beta}\end{cases} \tag{20}\]Define a multi-dimensional vector \(v_{i}=[(\alpha_{i},\beta_{i})]=[(\alpha_{i,1},\beta_{i,1}),(\alpha_{i,2},\beta_{i,2 }),\cdots,(\alpha_{i,d},\beta_{i,d})]\), each dimension is described by a pair of shape parameters of the beta distribution to model the uncertainty in that dimension. Its probability density function \(p_{v_{i}}(x)\) can be expressed as follows:  
\[p(v_{i})=\left[p_{[(\alpha_{i,1},\beta_{i,1})]}(x),p_{[(\alpha_{i,2},\beta_{i,2 })]}(x),\cdots,p_{[(\alpha_{i},\beta_{i,d})]}(x)\right] \tag{21}\]  
This is a multidimensional expression, and each dimension has a corresponding beta distribution. Given a user behavior \(\mathcal{V}_{n}=\{v_{1},v_{2},\cdots,v_{m}\}\), the corresponding embedding can be expressed as \(v_{1}=[(\alpha_{1},\beta_{1})],v_{2}=[(\alpha_{2},\beta_{2})],\cdots,v_{m}=[( \alpha_{m},\beta_{m})]\). Define the output \(\overline{v}=\left[(\overline{\alpha},\overline{\beta})\right]=\mathcal{C}( \{v_{1},v_{2},\cdots,v_{m}\})\), where \(\mathcal{C}\) is a probability weighting operation, which can be expressed as follows:  
\[\overline{v}=\left[\left(\sum_{i=1}^{m}\ w_{i}\odot\alpha_{i},\sum_{i=1}^{m} \ w_{i}\odot\beta_{i}\right)\right] \tag{22}\]  
where \(\Sigma\) and \(\odot\) represent the element-wise summation and product, respectively. \(w_{i}\in R^{d}\) is a weight vector, and, in the j-th dimension, \(w_{i,j}\) satisfies \(\Sigma_{i=1}^{m}w_{i,j}=1\). We use an attention mechanism to learn the importance of different items:  
\[w_{i}=\frac{exp(MLP(\alpha_{i}\oplus\beta_{i}))}{\sum_{j}\ exp\big{(}MLP( \alpha_{j}\oplus\beta_{j})\big{)}} \tag{23}\]  
where \(\oplus\) represents the splicing operation and \(MLP\) is a multi-layer perceptron that takes the connection of \(\alpha\) and \(\beta\) as its input.