# A Comprehensive Survey of Recommender Systems Based on Deep Learning
## 3 Overview of the Recommender Systems
### Sequential Recommendation
#### 3.2.2 Long- and Short-Term Sequence Recommendation
The self-attention mechanism of this model was integrated within the recommendation domain, amalgamating both fundamental user details and context information. SDM [36] takes a nuanced approach by segregating user interests. It employs LSTM, complemented by multi-head attention mechanisms, to home in on immediate user inclinations. The culmination of this model witnessed the amalgamation of both fleeting and lasting user interests, encapsulating the user's essence in a vector representation. Compared to previous models, this approach employs gating mechanisms to act as weights, ultimately outputting a user behavior vector, as shown in Equations (7) and (8). \[G_{u}^{t}=sigmoid\ (W\,^{1}e_{u}+W\,^{2}s_{t}^{u}+W\,^{3}p^{u}+b) \tag{7}\]\[o_{t}^{u}\ =\ (1-\ G_{t}^{u})\odot\ p^{u}+G_{t}^{u}\odot\ s_{t}^{i} \tag{8}\]  
Here, \(\mathrm{e_{u}}\) is the user embedding, while \(\mathrm{s_{u}}\) and \(\mathrm{q_{u}}\) are data derived from user embeddings processed via different attention networks. This facilitates a more effective data processing strategy. Unlike prior works that simply used parameters as weights, this current approach enables short-term behaviors to be better integrated. Finally, the Trans2D model [84] showcased a pioneering expandable attention mechanism. This module adeptly discerns user predilections for specific item attributes, which are gleaned from their behavioral sequence data, incrementally elevating the efficacy of the recommendation process. Compared to recommendation algorithms based on RNNs, those relying on convolutional neural networks (CNNs) present greater challenges. [PERSON] et al. [85] devised a 3 DCNN model for high-dimensional modelling, demonstrating its efficacy when the given user sequences are associated with intricate element features. [PERSON] et al. [86] encoded interaction sequences into three-dimensional vectors, employing 2D convolutional filters to learn local characteristics. [PERSON] and [PERSON] et al. [87] proposed the Caser model, building upon the foundations of prior convolutional techniques. They employed both horizontal and vertical convolutional filters to capture continuous patterns in pointwise, conjoint, and skip behaviors, achieving promising results. The vertical convolution operation is shown in Equations (9) and (10). \[c_{i}^{k}=\Phi_{c}(E_{Li+h-1}\odot F^{k}) \tag{9}\]  
\[\widehat{c}_{i}^{k}=\sum_{l=1}^{L}\widehat{F}_{l}^{k}\cdot E_{l} \tag{10}\]  
where E serves as a matrix formed by stacking embeddings to facilitate the convolution operations. \(\mathrm{F^{k}}\) slides over E from top to bottom during the horizontal convolution process, capturing the user's behavioral patterns. Behavioral patterns are how users interact with different items at different points in time. The vertical convolution operates in the embedding space, focusing on how to group or categorize different items based on their embedding representations. By combining these two types of convolutions, the model is capable of capturing user behavior patterns in two distinct dimensions. It also adapts the embedding representations according to the dynamics of the user behaviors. However, this approach separately learns embeddings for users and items, failing to capture inherent interaction information and overlooking contextual details. GRU4 Rec [37] integrates an RNN into session-based recommender systems, treating the interactions within a session as the historical context for sequence modelling. The highlight of this work is its rational optimization of sequence recommendation based on an RNN. With continued research advancements, a multitude of improved models have emerged. These models aim to delve deeper into item information by optimizing their model architectures and training methodologies, thereby enhancing the produced recommendation outcomes. One such enhanced RNN model evolved from GRU4 Rec employs data augmentation techniques to bolster the stability of the model training process [88].