## 4 Dataset
### PUCK Dataset  
The PUCK dataset is a public dataset published in 2011. The PUCK dataset collected from a Kyoto smart home testbed located in Washington State University in two-story apartments with one living room, one dining area and one kitchen on the first floor as well as one bathroom and three bedrooms on the second floor.  
It combines three types of sensory data: (1) environmental sensors, including motion sensors on ceilings, door, sensors on room entrances, kitchen cabinet doors, microwave, and refrigerator doors, temperature sensors in rooms, power meter, burner sensor, water usage sensors, and telephone usage sensors, (2) items sensors for usage monitoring, and (3) two wearable sensors. Eight complex activities are defined: Sweep and Dust, DVD Selection and Operation, Prepare Meal, Fill Medication Dispenser, Water Plants, Outfit Selection, Write Birthday Card, and Converse on Phone. Activities are divided into ordered steps, which can help detect whether the activity is completed correctly.  
Figure 4: (1) The agent recommends a coffee machine to [PERSON], whereas she uses milk instead; then the system waits for 15 s; (2) The feedback is received by the system as a reward, and it is calculated accordingly as the coffee machine is the wrong item; and (3) two kinds of updating for the system: a minor update is after receiving the reward and a major update after certain period around 24 h.  
Features Engineering  
The PUCK dataset has four fields (date, time, sensor ID and sensor value). To adapt the PUCK dataset for our system, the following steps were taken to process the PUCK dataset by extracting the required features:  
1. Combining the environmental data sensors (motion, items, power meter, burner, water usage, door etc.) with the wearable sensors for each participant by matching the time step among them.
2. Labelling complex activities for the whole dataset.
3. Extracting the start and the end of each activity as a session to define when the user needs a prompt.
4. Selecting only the common sensors among all participants where the total measurement counts and participants each greater than 25 th percentiles.
5. Dividing the sensors into four groups to be processed: movement sensors, motion sensors, count sensors and continuous values sensors and process each group as follows: 1. In movement sensors group, each measurement includes six values (X, Y, Z, Yaw, Roll and Pitch). We extracted the following features: Mean (X, Y, Z, YY, RR and PP), STD (X, Y, Z, YY, RR and PP) Correlations (X//Y//Z) and (Yaw//Roll//Pitch), which leads to 36 features in total. 2. For motions sensors group, if at least one trigger in a group is counted as trigger for the group, count and then compute the fraction counts across the groups. Based on the PUCK dataset, we have 11 groups (features) altogether. 3. Count sensors, which have on, off measurements, such as (door, item, shake and medicine container sensor), we count and compute the fraction counts of each session (20 features). 4. For the last group, we calculate the average for continuous value sensors, such as electricity and temperature (three features).
6. After extracting the all features (70 features), we apply the previous groups process for all the participant sessions.  
Two methods are taken to overcome the item usage imbalance problem (i.e., only a small number of items are frequently used): first, the outliers from the items are dropped. This method is simple but very effective in improving the performance and secondly, the sampling order random points in activity sessions to increase the prompt points, although this does not help balance the item usages, such as dropping the outliers.