## 4 Development of RHA-LLM Communication for Patient Care
### RHA Software Prototype with ROS and ChatGPT4
#### 4.2.1 Preparation of Contextual Information  
Contextual information for the RHA operations is created according to the patient room Gazebo simulation. The contextual information includes robot task specifications, patient room information, robot profile, and an interaction guideline. Figure 2: Software architecture for prototype software. * The patient room description (Appendix A) provides a detailed spatial layout and configuration of the room, including precise locations of objects and people, which are crucial for task execution. The JSON description includes data on the patient's position, the location of objects like a water cup and bed, and predetermined locations for patient monitoring and robot interaction. The layout and object locations are aligned with the Gazebo simulation, maintaining consistency between the knowledge base and the simulated environment. While these descriptions were manually coded according to the Gazebo simulation, a future study should focus on implementing real-time updates to this file, ensuring that the locations and statuses of objects and patients are continuously updated. * The task specifications (Appendix B) provide detailed instructions on the actions the RHA should take in specific situations, such as fetching resources or performing routine assessments. Each task specification includes a general description of the task, defines the role of the RHA, offers guidelines for understanding when the task should be performed, and lists the required actions. For example, the \"fetch resource\" task is triggered when a nurse or patient requests an item (e.g., medical supplies, water). The process involves four sequential actions: navigating to the item's location, picking up the item, navigating to the designated delivery point, and handing over the item. Each action is clearly defined with the corresponding skill and input requirements. As noted in the specification, the actions for the \"fetch resource\" task must be carried out sequentially. Without such notes, multiple actions could be triggered simultaneously. Depending on the task, certain actions can be executed in both sequential and parallel modes. * The interaction guidelines (Appendix C) provide a structured protocol for communication between the RHA and the LLM. When the RHA or a person sends a prompt to the LLM, the LLM processes the request and returns a response formatted in JSON for the execution of robot skills. Responses from the LLM are categorized into two main types. Type 1 responses are action-triggering responses. When the RHA needs to initiate an action, the LLM generates a Type 1 response in a strictly formatted JSON that specifies the required function (e.g., \"pickObject\", \"navigateToLocation\") along with its corresponding input arguments. Type 2 responses are status-checking responses. Type 2 responses are generated when the robot sends prompts updating the LLM about its current status or task progress. For example, when the RHA starts navigating to a location, it sends a prompt like \"I am starting to navigate to the patient's bedside\". The LLM then generates a Type 2 response, indicating the robot is \"busy\" or occupied with the current task. This ensures that no new actions are assigned to the robot until it completes the ongoing task. All the responses have descriptions providing additional context or feedback to the human supervisor based on this status update. * The robot profile (Appendix D) informs the LLM about the RHA's identity, type, and the list of skills it can perform. This profile outlines essential information such as the robot's ID, \"rha_unmcr_1\", identifying it as a specific RHA unit in the healthcare environment. The profile also specifies task specifications that the RHA should follow, allowing clinicians to apply different task instructions based on the specific needs and dynamics of the patient room. A list of available skills is also provided, enabling the LLM to understand the capabilities the RHA can execute.