# A Stackelberg Trust-Based Human-Robot Collaboration Framework for Warehouse Picking
## 4 Stackelberg Trust-Based Human-Robot Collaboration Framework for Warehouse Picking
### Trust Prediction Based on the POSG
#### 4.2.1 The POSG Model  
**Definition 1**. : _An HRC with trust for warehouse picking is defined as a POSG \(\mathcal{M}_{POSG}=\left\{I,S,A^{i},O,obs,T,R,b_{0}\right\}\), where_  
* \(I=\left\{robot(r),human(h)\right\}\)_: a set of players (human and robot);_
* \(S\) _is a finite set of states;_
* \(A^{i}\) _is a finite set of actions of player_ \(i\in I\)_;_
* \(O\) _is a finite set of observations;_
* \(obs\) _is an observation function;_
* \(T\) _is a transition function;_
* \(R(a^{\prime}a^{h}s^{\prime}s^{h})\) _is the reward function when actions_ \((a^{\prime},a^{h})\) _have been jointly played in the joint state_ \((s^{\prime},s^{h})\)_;_
* \(b_{0}\) _is the initial belief over states._  
The game flow is shown in Figure 4, where humans and robots are participants. In this figure, circles represent states, squares represent actions, and diamonds represent reward functions. At time \(t\), the actions \((a_{t}^{r},a_{t}^{h})\) represent robot and human actions for task execution, which is observable for each player. The state at time \(t\) is \(s_{t}=\left\{\tau_{t},w_{t}^{i},f_{t},\hat{A}_{t}\right\}\), which includes the following: \(\tau_{t}\) means task type and is randomly distributed with probability \(P_{\tau}\), which influences the robot's success rate, and both the human and robot can identify task types; \(w_{t}^{i}\) means the work states of humans \(w_{t}^{h}\) and robots \(w_{t}^{r}\), observable by each other, which depend only on their own actions and are determined by the transition function \(T\big{(}w_{t}^{i}|a_{t}^{i}\big{)}\); \(f_{t}\) means the task outcome (success/failure), influenced by the task type and robot work state and is determined by \(T(f_{t}|w_{t}^{r}\tau_{t})\); \(\hat{A}_{t}\) means the human trust state, which is an internal trust level represented as a partially observable (gray circles in Figure 4) state accessible to the robot through the observation function (red dashed line in Figure 4). The solution for partially observable states involves establishing observations \(o_{t}\in O\) and updating them using the observation function \(obs(o_{t+1}|o_{t}w_{t}^{r}w_{t}^{h})\). The HRC system's real-time reward \(R(a_{t}^{r}a_{t}^{h}w_{t}^{r}w_{t}^{h})\) depends on the joint human-robot state and actions. The initial belief \(b_{0}\), representing the probability of distribution over states, is the shared knowledge between humans and robots. Some of the symbol interpretations in this framework are given in Appendix A.  
Figure 4: POSG for HRC with trust. **Definition 2**. : _State Transitions As shown in Figure 5, the state transition diagram at time \(t\) is depicted, and the explanation of states is given in Appendix B. Nodes with a dotted matrix background represent the \"Task Start\" state and indicate the task type. Nodes with a vertical line background represent the robot's work state, while nodes with a grid background represent the human's work state. At the beginning of a task, the robot first decides whether to execute the task. After observing the robot's choice, the human makes a corresponding decision. Once the human executes their decision, they transition to the corresponding work state based on the observed outcome of the robot's task execution._  
**Definition 3**. : _Trust In this study, trust is defined as a hidden state of the human, which is not directly observable by the robot. The human's trust level changes based on the robot's actions for different tasks and the human's corresponding responses. For example, if the robot opts not to execute a task, the human's trust in the robot decreases. Trust is also task-dependent; for example, certain task types may exacerbate or mitigate trust changes. When the human decides to check the robot's task execution, their trust level may change positively or negatively, depending on the observed outcomes._  
The evolution of human trust is modeled as a linear Gaussian system, a method widely used in related studies [18; 20; 36]. The trust transition is linked to the work states of both the human and the robot:  
\[