# Towards Cognitive Recommender Systems
## 2 Background and Related Work
### Sequential Recommenders
#### 2.2.2 Model-Based Approaches  
From the early 2010s, researchers start to develop model-based approaches. These approaches are mainly performed based on statistics and machine learning techniques [5]. Different from the model-free sequential recommenders, model-based approaches can deal with an ordered and complex sequence of items. Current methods in this category can be divided into three main classes--_Markov Chain-based_ models, _factorization-based_ models, and _neural network-based models_. **Markov Chain-Based Models**. Markov Chains model has been known as a straightforward technique to model the transitions over user-item interactions in a sequence. Based on the specific technique used, Markov chain-based RSs can be divided into basic Markov Chain-based approaches and latent Markov embedding-based approaches. The former refers to the situation where the transition probability of every state is computed only based on the previous state. Following that, auxiliary information, for example, context features, can be used to enrich the Markov Chain model and improve the recommendation accuracy [43]. Moreover, in one work [44], the authors combined a mixed order Markov Chains with the Web content information to have a more accurate Web-recommendation. They formed a hybrid web-recommendation system in which they used Markov-based methods to model users' Web browsing behavior and combined it with content-based filtering models to extract contents from the links that users have visited online. While the basic Markov Chain-based approaches can only model the simple and basic dependencies between the states, the latent Markov embedding-based approaches are able to capture the complex relationships between the states as the transition probabilities are calculated based on the Euclidean distance of embedded Markov Chains [45]. **Factorization-Based Models**. Matrix factorization or tensor factorization is the most widely used techniques in recommendation systems, which map users and items into latent spaces [46]. In sequential RSs, the matrix or tensor is constituted of interactions rather than the ratings [3]. For instance, FPMC is built based on the combination of matrix factorization and Markov chains to predict the next action [47]. **Neural Network-Based Models**. These classes of approaches have attracted researchers due to the great capability of neural networks in modelling complex relations between items and users. Basically, with respect to the number of incorporated layers, neural network-based models can be divided into _Shallow network_ and _Deep network_. The shallow network transfers items to a low-dimensional space inspired by the great success of some embedding technique in Natural Language Processing (NLP) such as word embedding [48]. In recent years, we have witnessed the tremendous success of the deep learning in solving many complex tasks in a wide range of applications such as speech and face recognition, NLP tasks and recommendation problem [49]. Typically, deep neural network-based approaches can be exploited either through basic deep neural networks or some advanced models. Recurrent Neural Networks (RNNs) [50] Convolution Neural Networks (CNNs) [51] and Graph Neural Networks (GNNs) [52] are the dominant approaches in basic deep neural networks. Although the mentioned approaches can model complex users and items relations, they may face some challenges. CNN-based approaches may lose some information due to the max-pooling operation and they may be limited in capturing long-term dependencies due to the restriction on sizes of the filters. While RNN-based models assume that there is a strict order between items in a sequence and this may generate false dependencies. To deal with the problem of basic neural networks, some advanced models are utilized, such as--(a) attention mechanism: which is a well-known technique in SRs due to its great capability to focus on selective parts [PERSON] et al.