# Multi-Feature Behavior Relationship for Multi-Behavior Recommendation
## 4 MFBR Model
### Feature Encoder Layer  
**Sequence Encoder**. To fully mine the multi-behavior interaction information of the user, it is necessary to efficiently construct the dependencies within each behavior. The task of the _Sequence Encoder_ is to generate a behavior representation vector \(u^{s,k}\) for each sequence of behavior \(k\). In [12], the authors show that low-rank factorized self-attention networks are able to model sequential relationships between items more accurately than self-attention networks. Therefore, we leverage this network to build a more precise representation of user interests. Low-rank factorized self-attention networks work by projecting the things that a user interacts with onto a low minority of latent interests. Specifically, for each behavior sequence \(S_{u}\) of the user \(u\) we can create an item embedding matrix \(E^{k}\in R^{n\times d}\) as the input of the self-attention network, where \(d\) is the initial vector dimension. We convert \(E^{k}\) to three matrices \(E_{Q}=E^{k}W_{Q},E_{K}=E^{k}W_{K}\) and \(E_{V}=E^{k}W_{V}\) by linear projection, where \(E_{Q},E_{K}\) and  
Figure 2: Model architecture of MFBR. \(E_{V}\in R^{n\times d}\), \(W_{Q},W_{K}\), and \(W_{V}\in R^{d\times d}\). Then, the matrices \(E_{K}\) and \(E_{V}\) are mapped to the latent interest matrices \(\hat{E}_{K}\) and \(\hat{E}_{V}\) by a function \(f\), which is calculated as follows:  
\[ \begin{split}\hat{E}_{K}&=f(E_{K})=(soft\max(E_{K} \cdot M_{1}^{T}))^{T}\cdot E_{K},\\ \hat{E}_{V}&=f(E_{V})=(soft\max(E_{V}\cdot M_{2}^{T}) )^{T}\cdot E_{V}\end{split}  \tag{1}\]  
where \(E_{K}E_{V}\in R^{d^{\prime}\times d}\) are the latent interest matrix, \(M_{1}M_{2}\in R^{d^{\prime}\times d}\) is the learnable parameter matrix, and \(d^{\prime}\) is the dimension of the mapped vector. Considering that the self-attention mechanism cannot perceive temporal information, we employ decoupled positional coding to model the sequential relationship between things in order to capture the relationships between them, which can explicitly specify the sequential relationship [12]. Specifically, given a positional embedding vector \(P^{k}\in R^{n\times d}\), \(P_{Q}=P^{k}U_{Q}\) and \(P_{K}=P^{k}U_{K}\) are obtained by linear projection, where \(U_{Q},U_{K}\in R^{d\times d}\). Subsequently, the attention representation is constructed as follows:  
\[ \begin{split}\tilde{S}^{k}&=\tilde{A}_{item}\cdot f (E^{k}W_{V})+\tilde{A}_{pos}\cdot E^{k}W_{V}\\ \tilde{A}_{item}&=\text{softmax}(\frac{E_{Q}\cdot f (E_{K})^{T}}{\sqrt{d}})\\ \tilde{A}_{pos}&=\text{softmax}(\frac{P_{Q}\cdot(P_{ K})^{T}}{\sqrt{d}})\end{split}  \tag{2}\]  
where \(\tilde{A}_{item}\), \(\tilde{A}_{pos}\) is the attention matrix. The method models item sequences independently from position sequences and can explicitly consider the sequential relationships for the purpose of decoupling position encoding. To enhance the expressiveness of attention embedding vectors, we stack multiple layers of networks. In addition, we utilize a feed-forward network at each attention layer to provide nonlinearity to the model. The sequence representation of the final obtained specific behavior is denoted as \(u^{s,k}\). **Graph Encoder. ** The user-item interactions can constitute a user-item heterogeneous graph, which contains information about the higher-order user-item behavioral interactions that in turn can reflect the overall preferences of the user. To characterize the user's preferences, the graph encoder executes embedding propagation through the multi-behavior interaction graph \(G=(U,V,E)\), encoding the higher-order relationships of user-item interactions. Specifically, a low-dimensional representation of the multi-behavior interaction vector \(X\) is constructed by pre-training initialization to obtain the user and item initial representation vectors \(h_{i}^{0}\) and \(h_{j}^{0}\)[21].