# Designing Social Robots with LLMs for Engaging Human Interaction
## 3 System Design
### Memory, Persona, and Dialogue Continuity
#### 3.2.3 Latency Reduction and Social Cues  
To support fluid and socially intuitive interactions, the system integrates several real-time mechanisms that minimise latency and enhance the robot's embodied expresssiveness. These components are designed to improve not only the speed and clarity of communication, but also the perceived responsiveness and social presence of the robot.  
The speech output pipeline is optimised to reduce delay by streaming the LLM-generated response incrementally as it is being produced. Once transcribed user speech has been sent to the LLM, the generated text is streamed in real time to an external neural TTS engine, which begins synthesising audio without waiting for the full response. This audio stream is then transmitted to the robot via a secure SSH tunnel and played back through the robot's internal speaker. This architecture avoids buffering artifacts and significantly reduces the perceived latency between user utterance and robot response, contributing to the impression of immediacy and attentiveness.  
Turn-taking is managed through a VAD mechanism combined with a user-calibrated silence threshold. When a pause of approximately 800 ms is detected, the system temporarily considers the turn to be complete and sends the input to the LLM to generate a response [41]. This threshold is grounded in empirical studies of human dialogue, where pauses of around 600-800 ms typically signal turn completion and provide sufficient time for planning a response. By anticipating the end of a turn before the full threshold is reached, the system ensures that the response is ready promptly, supporting a natural conversational rhythm.  
However, if voice activity resumes within the threshold window--indicating that the user was merely pausing mid-thought--the pending response is discarded and the system resumes listening. The silence threshold is adjustable and was calibrated prior to each session based on the user's speech rhythm, particularly to accommodate elderly participants who might exhibit longer or more reflective pauses. This mechanism balances the need for responsive interaction with sensitivity to individual pacing, enabling more respectful and natural dialogue, especially in reflective conversations.  
Simultaneously, the robot's nonverbal behaviour is dynamically adapted to the conversational context. [PERSON] performs contextually appropriate movements, including face tracking to simulate eye contact and head gestures such as nodding or tilting. These movements are synchronised with speech output, reinforcing verbal content and enhancing perceptions of engagement. Turn-taking cues are further supported by changes in [PERSON]'s eye LEDs, which signal whether the robot is listening, processing, or speaking. These visual cues help to regulate the interaction flow and provide users with a clearer sense of shared conversational control.  
To improve audio input quality, particularly in non-controlled home environments, the system employs an external USB microphone array positioned near the user. This setup reduces interference from the robot's internal cooling fans and enhances speech recognition robustness, especially for softly spoken or accented speech. The improved audio capture not only increases transcription accuracy, but also enhances the reliability of voice activity detection, thereby reducing false turn completions or missed user input.