# Personalization of Robot Behavior Using Approach Based on Model Predictive Control
## 3 Proposed Solution
By collecting more real data from the user and retraining the user model, the model becomes better adapted to the user's behavior. The reward function, which initially represents a theoretical baseline user, can also be adjusted to reflect the preferences of the specific user as more data is collected, improving personalization. Such an adjustment would require manually changing the values of the emotions and gaze factors or using some learning algorithm; see Equation (1). The reward function \(Q(A)\) assigns a real value to the change in the user's emotional state and the user's gaze pattern after the robot's action. It corresponds to the MPC goal function. The adaptive decision module selects an action with the highest predicted value, so the reward function should return a value that represents the quality of the action in the current situation. Its definition depends on the application and the available sensors. In our research, it is calculated as a linear composition of recognized user emotions and parameters describing gaze contact between the user and the robot:  
\[Q_{e}(A)=\sum_{x\in X}((e_{x}^{\prime}-e_{x})*F_{x}),X=\{a,d,f,h,s, su,n\} \tag{1}\] \[Q_{g}(A)=\sum_{n\in N}((g_{n}^{\prime}-g_{n})*F_{n}),N=\{c,d,u,l, r,di\}\] \[Q(A)=Q_{e}(A)+Q_{g}(A).\]  
These functions allow the analysis of the change in the observed behavior of a user after the execution of an action \(A\). \(Q_{e}(A)\) represents emotion quality. It is a sum of the differences between the emotions after (\(e_{x}^{\prime}\)) and before (\(e_{x}\)) execution of the action multiplied by a factor \(F_{x}\). The factor values \(F_{x}\) were manually chosen to closely align with a rule-based expert system that represents the user in the simulated environment. Specifically, they were selected to give a high-quality score for actions preferred by a user simulated by the expert system and a low score for actions that differ significantly from the user's preferences. The most distinct examples of differing actions would be opposites, such as increasing volume versus decreasing volume. Emotion \(x\) is one of the following emotions: angry (a), disgust (d), fear (f), happy (h), sad (s), surprise (su), and neutral (n). The values for \(F_{x}\) are \(\{-6.75\), \(-5.5\), \(-10.0\), \(5.5\), \(-2.5\), \(1.5\), \(4.0\)}, respectively. They are designed to penalize negative changes and encourage positive or neutral ones. Similarly, we calculate \(Q_{g}(A)\) representing the quality of the human gaze. The following gaze directions were used in the calculations: contact with the robot (c), down (d), up (u), left (l), right (r), and diagonal (d). The values of \(F_{n}\) are also manually chosen and are equal to \(\{4.0\), \(-1.0\), \(-1.75\), \(0.5\), \(-3.625\), \(-0.25\)}, respectively, to promote engagement understood as gaze contact. The reward function \(Q(A)\) is a sum of emotion and gaze quality. Figure 3: Flowchart presenting adaptive decision module computations. One might ask why create a new uncertain function instead of using an already existing function for data generation. The biggest advantage of such an approach is the ease of interpretation of how such a function works, the low computational complexity, and the simple application for future, real-time experiments.