# A Comprehensive Survey of Recommender Systems Based on Deep Learning
## 3 Overview of the Recommender Systems
### Sequential Recommendation
#### 3.2.2 Long- and Short-Term Sequence Recommendation  
The evolution of long- and short-term sequence recommendation reflects the progression of recommender systems from simple collaborative filtering methods to sophisticated deep learning models. With continual advancements in technology, models are becoming increasingly adept at distinguishing and integrating two user preferences, thereby facilitating more accurate and personalized recommendations. Long-term preferences reflect their relatively stable interests, while their recent preferences indicate their recent interest changes [69]. Although user preferences change dynamically over time [70], some long-term stable preferences that influence user behaviors remain. The existing approaches tend to combine two preferences [71]. Taking into account that interactions between two behaviors may exert diverse effects on the user's present interests [72]. Compared to recent interaction sequences, long-term interactions tend to evolve more slowly [73], thereby reducing the demand for real-time responsiveness [74]. [PERSON] [75] categorizes user behaviors into two behaviors, employing a multilayer attention network for modelling purposes. By coupling users' two preferences, an optimal user representation is generated, enhancing the output recommendation results. This hybrid representation is calculated as shown in Equation (6). \[u_{t}^{hybrid}=\beta_{0}u_{t-1}^{long}+\sum_{j\in S_{t}^{u}}\beta_{j}v_{j} \tag{6}\]  
where \(\mathrm{v_{j}}\) represents the rating embedding of an item, which is indicative of the short-term embedding, while \(\mathrm{u_{t-1}}\) denotes the long-term embedding. \(\beta\) denotes varying attention scores. This formulation captures the dynamic nature and differentiates between the contributions of items towards predicting the next. RNNs encounter challenges like gradient vanishing and gradient explosion when dealing with lengthy sequences, diminishing their proficiency in handling extended sequences. To tackle this problem, long short-term memory networks (LSTMs) [76] were initially introduced, introducing the fundamental concept of gating mechanisms that effectively addressed the issue of vanishing gradients. Recommender systems often use LSTM or gate recurrent units (GRUs) for user modelling. LSTM is an enhanced version of an RNN [35]. It has been demonstrated that LSTM outperforms conventional RNNs in sequence recommendation tasks. A GRU, another variant of LSTM, is computationally simpler yet equally effective. Recognizing that the two-term impacts of preferences might differ, attention mechanisms have also been employed. The attention mechanisms used in deep learning stem from the idea that humans are drawn to the significant parts of a target. Their origin can be traced back to the research performed by [PERSON] et al. [77]. They used attention mechanisms to model the importance of the output using different parts of the input sentence. Building upon this foundation, vanilla attention was introduced to function as an RNN decoder, and it has been extensively used in sequence recommendation [78]. However, the self-attention mechanism (originating from the transformer in Google's 2017 neural machine translation work [79]) has also been deployed in sequence recommendation. Contrary to vanilla attention, self-attention does not involve an RNN structure, but it has achieved better performance in recommender systems than RNN-based models [80]. Building upon this idea, a GRU [81] offered streamlined versions of the LSTM gating processes. Venturing into the realm of natural language processing, the transformer model [82] gained widespread acclaim. It established unparalleled standards in terms of discerning the dependencies between sequences, which paved the way for swift parallel computations and accelerated sequence information extraction [83]. This foundation was ingeniously adopted in Alibaba's DIN model.