# Personalization of Robot Behavior Using Approach Based on Model Predictive Control
## 3 Proposed Solution
As a result, the action selection is greedy, as only immediate reaction to the action is predicted (without delayed effects), and the action providing the lowest cost in the next step is chosen. However, the algorithm is general and allows us to extend the time horizon, which would allow the robot to find more sophisticated action  
Figure 2: Decision module architecture based on MPC approach. plans. Also, the cost function can be more complex and include energy consumption by a given action, duration of the action, etc. To provide adaptability of robot behavior, this architecture is extended as follows. The state after the best-action execution is observed using sensors, and the collected data are used as _User feedback_. These results are evaluated using the _reward function_. They are used to improve and reevaluate the _User model_. The complete decision process, including adaptation, is presented in Algorithm 1. _Main_loop_ procedure is the primary function in the decision module that initiates and calls other functions. Here, we present a simplified and abstract version of it. In reality, our framework operates in parallel, is partially distributed, and follows an event-driven paradigm, making it challenging to represent as a single function. The _user_model_ parameter is an object representing the _User model_, while _list_of_interactions_ is a list of planned user-robot interactions (the subsequent stages of the conversation between the robot and the user in our case). For more details, please refer to [11]. The _list_of_robot_actions_ contains all possible robot actions. _adaptive_decision_ function uses _User model_ to predict user reactions to all possible robot actions in a given state and selects the best action for execution. _get_best_action_ function chooses the optimal action from the robot action list based on the user reactions predicted. The _apply_user_feedback_ function calculates and returns a reward function based on the sensor data and the performed action, using Equation (1). Depending on the model used, re-evaluation can be executed incrementally or in all-at-once batch mode. Incremental algorithms, like reinforcement learning, can learn continuously, using new data only, and do not require retraining to update the model. Algorithms without such capabilities, such as Random Forest, require training on the whole training data (base dataset combined with data from last experiments with the user)to improve the model each time new data appear. ```
procedureMain_loop(user_model,list_of_interactions,list_of_robot_actions) for each interaction \(\in\) list_of_interactions do  Robot_execute_interaction(interaction)  sensor_data \(\leftarrow\) get_sensor_data() robot_action \(\leftarrow\) ADAPTIVE_DECISION(user_model,sensor_data,list_of_robot_actions) Robot_execute_action(robot_action) reward \(\leftarrow\) APPLY_USER_FEEDBACK(user_model,robot_action,sensor_data) function adaptive_Decision(user_model,sensor_data,list_of_robot_actions) user_reactions \(\leftarrow\) for each robot_action \(\in\) list_of_robot_actions do  reaction \(\leftarrow\) user_model_predict(sensor_data,robot_action) user_reactions.add(reaction) best_action \(\leftarrow\) GET_BEST_ACTION(list_of_robot_actions,user_reactions) return best_action functionget_best_action(list_of_robot_actions,user_reactions) best_action_index \(\leftarrow\) argmax(user_reactions) return list_of_robot_actions[best_action_index] functionapply_User_FEEDBACK(user_model,performed_action,sensor_data) reward \(\leftarrow\) get_reward(performed_action,sensor_data) user_model.update(performed_action,sensor_data,reward) return reward
```  
**Algorithm 1**_Adaptive decision module_ algorithm  
In this approach, the personalization of robot behavior is achieved by developing an appropriate model. There are two ways to accomplish this: by enhancing the model with additional training data to improve its performance and by refining the reward function.