## 4 Methodology  
### STPA-Related POMDP Formulation  
In our approach, we interpret the STPA outcomes and link them with specific ecosystem states and actions, either CAs or UCAs. For the proposed problem formulation, we read the generic POMDP design theory [34] in a safety-related manner, considering the explicit constraint and unconstrained dynamics where the problem domain comprises in our case study the environment, the operator, the robot and the peripheral monitoring components. A discrete POMDP is designed as a tuple \(P=\{S,A,\Omega,R\ O,T,b_{0}\}\) where:  
* \(S=\{s_{1},s_{2},...,s_{n}\}\) denotes the **States** space that determines the condition of the environment, the operator, the robot and the peripheral monitoring components at each time \(t\). * \(A=\{a_{1},a_{2},...,a_{n}\}\) denotes the **Actions** space that comprises all the system actions (including nominal CAs and UCAs) that the ecosystem agents (i.e., robot and operator) are able to perform so as to complete the collaborative task. * \(\Omega=\{\omega_{1},\omega_{2},...,\omega_{n}\}\) denotes the **Observations** space that comprises the outcomes of each above mentioned action, but under the assumption that an observation \(\omega\) partially describes the state of the previous entities. * \(R=(A,S)\) comprises a **Reward** function that determines the restrictions imposed by penalizing or favoring specific selected actions (\(A\)) during the HRC (\(S\)). The measurement of the probability distribution of the initial state (\(b_{0}\)), the states transitions (\(T\)) and the observations (\(O\)), constitute the fundamental part during the design of POMDPs. This is a challenging task, and the initialization of the probability values over these parameters is typically applied empirically or by assuming a distribution function based on the nature of the problem. * The probability distribution of the initial state comprises the likelihood about the environment, the robot, the operator and the subordinate components to be in specific state \(s\) at the time \(t=0\) such as: \[b_{0}(s)=P(s_{0}=s)\] (1)
* The probability distribution of the _state transition_ comprises the probability of propagating to state \(s^{\prime}\) given that the domain is in state \(s\) and the system selects an action \(a\) and its respective expression is provided as: \[T(s,a,s^{\prime})=P(s_{t}=s^{\prime}|s_{t-1}=s,a_{t-1}=a)\] (2)
* The probability distribution of the _observations_ comprises the uncertainty of the model to receive a specific observation \(\omega\), e.g., feedback, considering that the ecosystem is in state \(s\) and the prompting model has selected to perform the action \(a\), also expressed as: \[O(s,a,\omega)=P(\omega_{t}=\omega|s_{t-1}=s,a_{t-1}=a)\] (3)* The probability distribution about the _current state_ of the ecosystem to be in \(s\), being partially observable through observation \(\omega\). Since it is not possible to define the current state with complete certainty, a belief distribution is maintained to express the history of the selected actions and state transitions of the domain such as at time \(t\), the operator, the robot, the environment and the subordinate monitoring components are at state \(s\) considering the sequence of past combination of actions and observations as follows: \[b_{t}(s)=P(s_{t}=s|\omega_{t},a_{t-1},\omega_{t-1},...,a_{0},b_{0})\] (4)  
The explicit definition of the aforementioned probabilities indicate the design of a well-formed POMDP model, the solution of which can be achieved through the existing solvers [35]. The outcome of this solution is an action selection policy \(\pi\) that maximizes the sum of the expected future reward up to a specific time. This policy comprises a mapping from the current state belief probability to the action space \(A\).