## 5 Robot Decision-Making and Task Planning
### POMDP Decision-Making  
For the problem formulation, we relied on the generic POMDP design theory [37], however, tailored to the explicit medication assistive scenario, where the problem domain comprises the environment, the human and the robot. Towards this direction, the discrete POMDP is designed as a tuple \(P=\{S,A,\Omega,R,T,b_{0}\}\), where \(S=\{s_{1},s_{2},...,s_{n}\}\) denotes the **States** space that determines the condition of the environment, the human and the robot at each time \(t\). \(A=\{a_{1},a_{2},...,a_{n}\}\) denotes the **Actions** space that encloses all the actions that the robot is able to perform so as to interact with the human and the environment. \(\Omega=\{\omega_{1},\omega_{2},...,\omega_{n}\}\) denotes the **Observations** space that comprises the robot perception input from the human and the environment, yet under the assumption that an observation \(\omega\) partially describes the state of the previous entities. \(R=(A,S)\) comprises a **Reward** function that determines the restrictions imposed by penalizing or endorsing specific robotic actions (\(A\)) during the interaction with the human and the environment (\(S\)). The solution of the POMDP model after its definition is produced using the existing solvers [38]. The outcome is an action selection policy \(\pi\) that maximizes the sum of the expected future reward up to a specific time. This policy comprises a mapping from the current state belief probability to the action space \(A\). Given the computed policy, the robot can select an optimal action by computing its belief state based on the following update rule:  
\[b^{\prime}(s^{\prime})=\frac{O(s^{\prime},a,\omega)\sum_{s\in S}T(s,a,s^{ \prime})b(s)}{P(\omega|a,b)} \tag{10}\]  
where \(b^{\prime}\) is the updated belief, \(b\) is the given belief at the previous time step and \((a,\omega)\) is the latest combination of robot action and observation. It is apparent that the computation of optimal policies is characterized by an exponential computational growth. A single step of value iteration to compute the next selected action is on the order of \(|C_{t}|=O(|A||C_{t-1}||^{|\Omega|})\), where \(|C_{t-1}|\) corresponds to the number of components required to represent the next selected action at iteration \(t-1\), while the computational burden is estimated by taking into consideration the number of iterations in each step for the \(O(|S|^{2}A||C_{t-1}||^{|\Omega|})\). In robot assistance during medication adherence, the number of states and actions grows drastically by considering an abundance of environment, human and robot states, and many robotic actions that need to be determined. Thus, an abstraction of the state and action space given the awareness of the robot for the user has been applied, following our previous work [39]. Specifically, since the state space is partially observable, it can only be conceptually grouped by defining scalable blocks of states that correspond to distinct levels of robot alert (LoRA), \(\overline{S}=\{S_{H},S_{M},S_{L}\}\). Herein, the state space is conceptually partitioned in three levels of robot alerts, namely _High_, _Medium_ and _Low_. The states that may belong to the \(S_{H}\) correspond to phases in the assistive task for which the human requires drastic assistance from the robot. The \(S_{M}\) defines the group of states within the task, in which the robot has already been engaged in an assistive task and the levels of awareness about the human have been moderated. Finally, the \(S_{L}\) outlines these states where the assistive scenario has been resolved, the required intervention is diminished and the robot is complacent about the status of the human. The conceptual partitioning of the state space indirectly defines groups of robotic actions, the context of which is related to the type of robot intervention required for the scenario propagation, given the current robot awareness about the human.