## 4 Proposed System for Patient-Robot Co-Navigation
### Shared Control in Crowded Environments Using Reinforcement Learning  
#### 4.2.1 Shared Control Considering Only Dynamic Obstacles (Model V1)  
Shared control between the patient and the robot is crucial for two reasons: (i) the robot needs to adjust its speed according to the patient's gait velocity to maintain interaction with the patient, and (ii) the robot needs to navigate the patient safely through the crowded environment. In the proposed work, RL is used to achieve shared control between the robot and the human. RL methods are already used to enable robots to navigate, as mentioned in Section 2.2; however, the robots do not have physical interactions with humans. In the presented work, a deep RL architecture is proposed, which extends the crowd navigation capabilities of the network architecture proposed by [PERSON] et al. [43], where st-graph is used, and each factor of st-graph observes node and edge features at each time step. Figure 4 shows our network architecture and consists of four recurrent neural networks (RNNs), where each is a factor graph representation of the st-graph; (i) crowd spatial edge \(RNN_{c}\), (ii) patient temporal edge \(RNN_{p}\), (iii) robot temporal edge \(RNN_{r}\), and (iv) node \(RNN_{n}\). Figure 3: (a) Occupancy grid from LIDAR data; (b) leg detection from our prior work (right) [15]. The \(RNN_{c}\) grasps the spatial interactions between each detected human in the crowd and the robot. The spatial interactions at a particular time (t) can be represented as a list of distances from each human to the robot, which can be represented as \(\bigcup_{i=1}^{t}(p_{ci,x}-p_{rx},p_{ci,y}-p_{ry})\), where \((p_{cn,x},p_{cn,y})\) are the 2D coordinates of the \(n\)th human with respect to the robot's base coordinate system \((p_{r,x},p_{r,y})\). The \(RNN_{r}\) captures the relationship between the temporal dependency of the robot's velocity and the spatial crowd. To learn the temporal dependency, the temporal features of the robot, which are the corresponding location and velocities of the robot in the x and y direction, denoted by \((p_{r,x},p_{r,y},v_{r,x},v_{r,y})\), are used as input so that the RNN can learn to adjust its trajectory from the initial starting point to the goal. The goal coordinates are denoted by \((g_{r,x},g_{ry})\); the radius and heading direction of the robot are denoted by \(\rho_{robot}\) and \(\theta\), respectively. The \(\rho_{robot}\) in our case is set to 1.21 m, which is the radius of the Clearpath Ridgeback robot, 0.96 m + 0.25 m, which is the boundary of the robot. The \(RNN_{p}\) captures the temporal dynamics between patient gait velocity and the robot. In order to implement the shared control between the patient and the robot, this RNN takes the patient's gait velocity in x and y directions, which can be represented as \((v_{p,x},v_{p,y})\). The outputs of \(RNN_{r}\), \(RNN_{c}\), and \(RNN_{p}\) are required to be combined in such a way that the importance of each RNN output at that particular instance is known. For example, if the patient is close enough to the robot but very close to another human, at that instance, the highest priority of the robot should be to move away from the human. Similarly, if the humans are far from the robot but the robot is moving away from the patient at that instance, the robot should decrease its velocity so that the patient will not lose contact with the robot. An attention mechanism is used to teach the model to recognize these traits from the inputs and adjust accordingly to the robot's velocity. The attention mechanism proposed by [PERSON] et al. [51] is used in this framework. For humans in the environment, let the output of the \(RNN_{c}\) at a given time \(t\) be \([s^{t}_{ci},s^{t}_{c_{2}},s^{t}_{c_{3}},\ldots,s^{t}_{c_{n}}]\), where \(c_{i}\) denotes the \(i\)th human. Similarly, let the output of the \(RNN_{p}\) be \(s^{t}_{p}\) at a given time \(t\).