# Can Gestural Filler Reduce User-Perceived Latency in Conversation with Digital Humans?
## 1 Introduction
The computationally intensive procedures, e.g., NLP and speech/gesture generations, are processed on the server side and then sent back to the client. Through these processes, the inevitable latency occurs between the user's question and the answer of the digital human [32]. In addition, language models such as GPT3 [33] have limitations in that they can generate ethically inappropriate responses in the given circumstances or answers irrelevant to the user's questions. Therefore, if necessary, a real person has to intervene to provide an appropriate answer to the user's question. Of course, an inevitable time delay occurs in that process of intervention. The time delay during the conversation with a digital human can reduce the usability of the system and users' satisfaction [34]. While we are unaware of psychophysical experiments investigating the detailed latency requirements in conversation with digital humans, the two-second rule was proposed in man-computer conversational transactions [35], as well as in human-robot interaction; the satisfaction and naturalness decreased after 1 s [36], which is in line with a similar study reporting 0.9 s [37]. Such a short latency allowance seems complicated to achieve considering the current limitations on inference times of TTS, gesture generation, and conversation generation models, even in the case where a computationally powerful server processes all the models and streams the rendered digital humans through a 5G network [38]. Previous studies [39; 40; 41] attempted to mitigate user-perceived latency using various conversational fillers, such as \"Uhm\" or \"Wait a minute\". However, when it comes to everyday face-to-face communication, people rely not only on verbal cues, but also on nonverbal cues, such as gestures and facial expressions [42; 43; 44]; gestures, in particular, have been overlooked. This study, thus, examined whether the gestural filler for digital humans can mitigate user-perceived latency in conversation with a digital human. We compared the effect of conversational fillers and gestural fillers on the user's perception of two conversation types: informative and casual conversation. Considering that speech-gesture match can affect the user's perception [45; 46], we also explored the effect of whether the conversation types and gesture fillers matched or not. In short, this paper addresses the following research questions:  
* Can gestures of digital humans, i.e., gestural fillers, reduce perceived latency in conversation?RQ2: Does the gesture to reduce perceived latency vary by conversation type? RQ3: How do gestural and conversational fillers affect users' perception of digital humans when delays occur during the conversation? The rest of the paper is organized as follows: in Section 2, we summarize previous research on ECAs, the effects of latency in conversation with ECAs, as well as fillers--techniques devised to reduce perceived latency--and the importance of non-verbal cues; in Section 3, we describe the experimental setup and the conversational digital human system we used for the following two studies; in Section 4, we detail the first experiment performed to investigate the correlation between gestural fillers and conversation types; then, in Section 5, we report the second experiment exploring the effects of gestural fillers compared to the conversational fillers; finally, Section 6 concludes the paper and discusses future research directions.