## 4 Development of RHA-LLM Communication for Patient Care  
### Framework Overview  
Figure 1 illustrates the proposed framework operating in two phases: (1) preparation of RHA knowledge base and (2) RHA-Clinician-LLM dialogue for task execution. In the knowledge base preparation phase, contextual information is provided to the LLM in three forms. First, structured knowledge includes details about the patient room configuration, patient information, patient vital signs, bed occupancy status, clinician information, and medication schedules, etc., that are typically generated by various software programs. These data can be transformed into machine-readable formats (e.g., JSON or XML) via APIs or direct outputs, making them comprehensible to modern LLMs. Second, unstructured knowledge is provided by healthcare professionals in natural language and interpreted by the LLM's language processing capabilities. This may include patient-specific care instructions, updates on conditions, or guidelines for emergency scenarios. Third, robot-related knowledge involves both structured and unstructured information about the RHA's capabilities and operational parameters. Structured information includes robot properties, skills, and actuation limits, while unstructured data, like task specifications and RHA-human-LLM interaction guidelines, define roles and responsibilities for humans and robots.  
In the task execution phase, the LLM uses the contextual information stored in the knowledge base, combined with its built-in common-sense reasoning, to interpret scenarios and guide RHA operations through an ongoing dialogue. Task specifications and RHA-human-LLM interaction guidelines inform the LLM's decision-making for the robot. Task specifications define the goal, sequence of actions, and required skills for each action, while the RHA-human-LLM interaction guidelines clarify the roles of the LLM, robot,  
Figure 1: Overview of LLM-Based Task Planning for Robotic Health Attendant (RHA).  
clinicians, and patients, as well as response formats. A command from the user can trigger an individual action or multiple actions according to a task specification. For example, a command like \"I am a nurse. Go to the designated waiting area\". prompts the LLM to retrieve the \"navigateToLocation\" skill from the robot-related knowledge. The waiting area location is stored in the patient room configuration within the knowledge base, allowing the LLM to provide the waiting area coordinates as the input argument for the RHA to execute. For a more complex command such as \"I am a patient. Bring me water.\", the LLM refers to the \"fetch resource\" task specification. The location of the water cup is also stored in the patient room configuration, enabling the LLM to guide the RHA through sequential execution of four skills: \"navigateToItemLocation\" (with the water cup's location as input), \"pickObject\", \"navigateToLocation\" (with the patient's location as input), and \"releaseObject\", as described in the task specification. During skill execution, the RHA sends real-time progress updates to the LLM, such as \"I have completed navigating to the water cup. What is my next action?\" The LLM then responds in two formats: natural language (e.g., \"Your next action is to pick up the water cup\") for human users and JSON for the robot to execute the \"pickObject\" skill. This framework allows for easy implementation without the need for extensive hardcoding, as the LLM, with its built-in common sense and access to the knowledge base, autonomously selects the appropriate functions, identifies matching input arguments, and organizes the execution of multiple skills in sequence or parallel.