## 5 Evaluation  
First, the evaluation of the effectiveness of the CB approach in recommending the correct item to a user in case the user's current activity needs a prompt is carried out. All the extracted features are utilized by the system as contexts to make a recommendation of the correct item. One publicly available CB python package is selected for our experiments. The package offers two types of models: full batch and streaming models. Due to sample limitations of the datasets, the streaming models, namely SGDClassifier (SGD) and LinearRegression (OLS), are focused on. Both models are sensitive to hyper-parameters, such as beta_prior or smoothing. Nevertheless, SGDClassifier offers stochastic matrices, while LinearRegression (OLS) has matrices, which are closed to the solution, and it updates them incrementally. Details about the parameters are given in Table 2. Figure 5 shows a set of policies used for each model. For the PUCK Figure 5a and the ARAS (house A) (Figure 5b) datasets, the SGD model--particularly the Softmax Explorer policy--is more robust, and it provides a better cumulative mean reward of item recommendations. On the other hand ARAS (house B) Figure 5c and ADL Figure 5d are more likely to provide good results by LinUCB policy of OSL model. This plot confirms that both models lead to a promising results based on each dataset from several policies being used. Table 3 shows the cumulative mean reward of our system by considering the three datasets. The Reward Delay Period \(T_{r}\), as earlier explained, helps in the determination of the suitable time for an agent to receive the reward as a feedback of the recommended item. Tuning \(T_{r}\) is important as decreasing it could indicate that the recommended item is not used, while increasing \(T_{r}\) could confuse the agent--specifically when the user starts to use other items before receiving the feedback about the recommended one. The results in Figures 6-8 show how \(T_{r}\) can affect the performance. The reward delay period with 5 s (see Figure 6) provides higher performance with the three datasets compared with the reward delay period of 10 and 15 s as shown in Figures 7 and 8, respectively. Policies are affected by the reward delay period; for instance, softmax explorer of the SGD model has a good performance with 5 s among the datasets but this performance starts to be reduced when the reward delay period is increased to 10 and 15 s. Table 4 summarises all cumulative mean rewards of our system among three datasets using different reward delay periods. There are two reasons that make the reward delay period of our system vary from dataset to another or from user to user in real time system. The first reason is that each activity has specific items and each of these items existing in different place. For example, preparing a cup of coffee activity includes the following items: a cup, sugar, milk and a coffee machine. Thus, if the agent recommends milk and the fridge is too far from the user, it will take some time to receive the correct response, while if the agent recommends a cup and the user is standing near the cupboard, it will take less time to receive the response. The other reason is that users have different behaviours in their response; some of them respond immediately after they receive the recommended item, and others may take a little bit longer. However, our system is targeted to Alzheimer's patient where it should expect a reward delay period with a long time compared with a healthy person. Here, we treat \(T_{r}\) as a hyperparameter that can be adjusted based on each item; we will leave this to our future work. In addition, it is observed that the system achieved the desired result of not requiring any feedback from the user to receive the reward. Consequently, it is calculated automatically after the Reward Delay Period. This feature is focused on because our system deals Alzheimer's patients who experience difficulty in holding a smartphone and confirm their response for recommendations.