## 2 Related work  
Multiple studies related to recommender systems for the IoT are found in the literature. Some of these works exploit the traditional recommender system approaches: collaborative filtering, content-based and hybrid-based. The collaborative filtering approach makes recommendations on items for a particular user based on the ratings of previous user [14]. The authors in [15] proposed a unified collaborative filtering model based on a probabilistic matrix factorization recommender system. It utilized three kinds of relations to extract the latent factors among these relations to construct accurate recommendations. Although the collaborative filtering approach has been adapted in numerous studies [16; 17; 18; 19], there are potential shortcomings that make it inefficient for RSIoT particularly, in terms of large amount of data, cold start problem, and data sparsity. In content-based, instead of relying on ratings, it recommends items that are similar to the items previously targeted by the user [20]. The authors in [21] adopt a content-based solution for the recommender engine in their AGILE project, which aims to improve the health conditions of users. The CB provides a number of features compared with CF, such as creating a profile for each user that depends on the history of his rating, deciding recommended items based on the extracted features of each item, and dealing with cold start problem; nonetheless, it has a limitation where it builds its recommendations based on items and their features only without considering any additional features that can tackle the RSIoT issues. The hybrid-based approach combines two or more approaches to build an RS, such as combining the collaborative and content-based methods where the limitations of each can be addressed [22]. The authors in [23; 24] built their recommender system engine using a hybrid recommendation algorithm. Combining two approaches, such as content-based with collaborative, may address their limitations but fails to tackle RSIoT problems: the dynamicity in human activity patterns, and updating the system automatically without needing feedback from users. Figure 1: Motivation scenario. Consequently, other studies started to shift for artificial intelligence techniques that are able to present data in a good way and to deal with complex interaction pattern. For example, machine learning was adopted to create a home automation framework [25]. It can be used as recommender system to assist elderly people so, they can live independently and safely. Here, the home appliances, like the TV, lights, etc. will be controlled using the user voice. In addition, If someone needs to monitor the elderly people, they can remotely control the smart devices with minimal cost and small effort. Deep learning algorithms also have been adapted for building a recommender system platform [26]. It helps doctors to determine the rehabilitation nutrition plan for cancer patients. However, machine learning and deep learning algorithms learn from offline data, which makes a recommender system for the IoT neither able to tackle any change of the human pattern nor be updated automatically. The reinforcement learning (RL) approach has been adapted in the implementation of recommender systems for IoT environments. RL makes it possible for the learning of the dynamics of a given environment and offers an architecture, which maximizes the long-term reward particularly for continuous record updates. In the work presented by [PERSON] et al. [27; 28], an inverse RL was adopted to model user behaviours while [PERSON] et al. [6] designed a recommender system for monitoring the daily indoor activities for people with mild cognitive impairment using RL. Most RL algorithms that deal with the modelling of dynamic environmental factors usually focus on matching each state for an action using different sequence of policy implementation. By considering future rewards, current actions are observed on how they influence next action.