# A Comprehensive Survey of Recommender Systems Based on Deep Learning
## 3 Overview of the Recommender Systems
### Sequential Recommendation
#### 3.2.1 Standard Sequence Recommendation  
Standard sequence recommendation refers to recommendation algorithms that extract user representations using a single behavior sequence [64]. Traditional sequence recommendation algorithms incorporate various prevalent sequence modelling techniques, such as pooling, RNNs, CNNs, memory networks, and attention mechanisms. This approach has been employed in Google's recommendation model. Due to its simplicity and efficacy, it stands as one of the most prevalent techniques for leveraging sequence features. Neural networks have also found extensive applications in sequence recommendation tasks.  
In the initial stages of sequence-based recommendation models, the primary methods employed are Markov chains [33] and feature-based matrix factorization [65; 66]. Compared to deep learning techniques, Markov chains have limitations in terms of handling complex data. Researchers employ personalized Markov chains to model individual users, leading to enhanced efficiency. Some people have pointed out that the Markov chain approach underperforms when modelling joint-level sequence patterns and fails to capture the effects of skip behaviors between users and items on recommendation outcomes. While factorization-based methods might model sequences by summing up sequence item vectors, they neglect the influence of order within the user interaction sequences on the recommendation results. Traditional sequence recommendation methods primarily utilize Markov chains and their derivatives for predicting a user's next action. They operate under the assumption that a user's subsequent action is solely dependent on their recent few actions.  
Compared to traditional recommendation models, deep learning models have demonstrated superior capabilities with respect to capturing the evolution trends of user sequences. The evolution of deep learning in sequential recommendation can be delineated via a series of transformative advancements. Recurrent neural networks (RNNs) have demonstrated their potent capabilities [67] and have been extensively utilized in sequence-based recommendation. Sequence recommendation models based on RNNs can be represented as Equation (5).  
\[h_{t}=\sigma\ (\ W_{hh}\ h_{t-1}+W_{hx}x_{t}+b_{h}\ ) \tag{5}\]  
where \(x_{t}\) represents the input and \(h_{t}\) denotes the hidden state. Finally, a different activation function for output is selected. Some people introduced CNNs to sequence recommendation tasks [68]. They underlined the fact that the existing Markov chain models can only model point-level sequential patterns and fall short when representing set-level patterns. They employ convolutional layers to extract local features from the user's historical behavior sequence, capturing these features from different embedding perspectives (vertical and horizontal). Different convolutional kernels are utilized to extract features from the input sequence. These features are subsequently used to generate the final recommendation list. An attention mechanism offers a notable solution to this gap.  
Based on this foundation, SASRec [34] introduced a sequence recommendation method founded on self-attention to improve its effect. Their approach employs a multihead self-attention mechanism and positional encoding to handle the input user behavior sequence. However, for an extensive sequence, earlier interactions might be overlooked. Consequently, researchers have begun to investigate recommendation models that integrate both long- and short-term interactions.