Conversational Grounding in Multimodal Dialog Systems (2023)

Abstract: The process of “conversational grounding” is an interactive process that has been studied extensively in cognitive science, whereby participants in a conversation check to make sure their interlocutors understand what is being referred to. This interactive process uses multiple modes of communication to establish the information between the participants. This could include information provided through eye-gaze, head movements, intonation in speech, along with the content of the speech. While the process is essential to successful communication between humans and between humans and machines, work needs to be done on testing and building the capabilities of the current dialogue system in managing conversational grounding, especially in multimodal medium of communication. Recent work such as Benotti and Blackburn have shown the importance of conversational grounding in dialog systems and how current systems fail in them. This is essential for the advancement of Embodied Conversational Agents and Social Robots. Thus my PhD project aims to test, understand and improve the functioning of current dialog models with respect to Conversational Grounding.

## spoken dialogue system (sds) research it seems clear that spoken dialogue systems(sds) will start incorporating visual elements as situated dialogs become more prominent with the rise of use cases such as embodied conversational agents and social robots, in the coming years. i also believe that these agents will begin to serve as personal assistants, capable of helping users in learning new skills and also managing their daily schedules. thus, research in the field of extracting accurate information from users over time and using it effectively would be very important as well. hence, further work on grounding, including clarifying user communicative intentions through multimodal information, and incorporating world knowledge effectively, will be essential for fulfilling the potential of spoken dialog systems.