Minimal Yet Big Impact: How AI Agent Back-channeling Enhances Conversational Engagement through Conversation Persistence and Context Richness (2024)

Abstract: The increasing use of AI agents in conversational services, such as counseling, highlights the importance of back-channeling (BC) as an active listening strategy to enhance conversational engagement. BC improves conversational engagement by providing timely acknowledgments and encouraging the speaker to talk. This study investigates the effect of BC provided by an AI agent on conversational engagement, offering insights for future AI conversational service design. We conducted an experiment with 55 participants, divided into Todak_BC and Todak_NoBC groups based on the presence or absence of the BC feature in Todak, a conversational agent. Each participant engaged in nine sessions with predetermined subjects and questions. We collected and analyzed approximately 6 hours and 30 minutes of conversation logs to evaluate conversational engagement using both quantitative (conversation persistence, including conversation duration and number of utterances) and qualitative metrics (context richness, including self-disclosure and topic diversity). The findings reveal significantly higher conversational engagement in the Todak_BC group compared to the Todak_NoBC group across all metrics (p<0.05). Additionally, the impact of BC varies across sessions, suggesting that conversation characteristics such as question type and topic sensitivity can influence BC effectiveness.

## introduction the increasing demand for ai-driven counseling and conversational services @xcite is driven by their benefits of anonymity, accessibility, and lack of biases @xcite @xcite , highlighting the importance of back-channeling (bc) as a crucial element of active listening strategies @xcite to enhance client conversational engagement. counseling experts emphasize the importance of active client participation for effective counseling @xcite . active conversational engagement involves clients actively participating and communicating their feelings and thoughts. bc includes verbal and non-verbal cues like nodding or saying "uh-huh" in response to the speaker @xcite , which express interest and enhance the speaker's conversational engagement @xcite . a comprehensive understanding of the impact of bc presented by ai agents on conversational engagement is crucial for developing ai systems, especially for applications requiring deep interactions, such as counseling. despite its importance, limited research has directly examined bc's impact on engagement. most existing studies focus on user perceptions of bc configurations @xcite @xcite or are conducted in non-conversational contexts @xcite . while some research explores the relationship between bc and engagement @xcite , it primarily addresses emotional aspects of user utterances without demonstrating bc's role in enhancing overall engagement, such as conversation persistence. moreover, few studies analyze actual ai-human conversation logs, which is crucial for understanding bc's effectiveness in real-world ai services. to address these gaps in the literature, our study aims to provide a more comprehensive understanding of how bc influences conversational engagement. specifically, we analyze conversations between users and ai agents with bc features, focusing on both quantitative and qualitative aspects. our research seeks to answer the following question: rq: how does the presence of back-channeling (bc) features in ai agents influence conversational engagement, both quantitatively and qualitatively? to answer this question, we conducted a user experiment with todak, an ai-based conversational system featuring bc. 55 participants were divided into two groups based on the presence or absence of bc, and conversation logs were collected across nine sessions, totaling approximately 6 hours and 30 minutes. we analyzed these logs using four metrics. the results showed that users interacting with the ai agent expressing bc had significantly higher conversational engagement scores than those interacting without bc (@xmath0.05), demonstrating the effectiveness of bc expressed by the ai agent. additional analysis indicated that the effectiveness of bc varies with conversation characteristics. the contributions of our research are as follows: first, we propose bc as a method to enhance conversational engagement in ai interactions and validate its effectiveness. second, we assessed conversational engagement using quantitative and qualitative metrics from actual conversation logs, a method overlooked in previous studies. our findings confirm that the utility of bc varies with conversation characteristics, providing valuable guidance for the future design of ai agent conversations. ## back-channeling and conversational engagement bc signals from a listener in human-to-human conversation play a crucial role as an attentive listening strategy in eliciting the speaker's speech and persisting the conversation @xcite , thus contributing to conversational engagement @xcite @xcite . conversation persistence measures how long the conversation lasts and how much a speaker talks, serving as a quantitative indicator of engagement. to investigate whether bc in ai agents contributes to conversation persistence, we designed an ai agent called todak and conducted a user experiment. partici-pants were divided into two groups: todak_bc and todak_nobc, and each participant engaged in conversational sessions. the first hypothesis is: hypothesis 1: conversation persistence in human-to-human conversation, bc also enhances context richness by promoting intimacy and trust, leading to increased self-disclosure and deeper conversations @xcite @xcite . additionally, bc aids in narrative formation @xcite , increasing topic diversity and making discourse richer. conversation context richness assesses how deep and varied a speaker's conversation is, serving as a qualitative indicator of engagement. the second hypothesis is: ## conversational engagement analysis based on session characteristics based on hypothesis testing insights, we conducted further analysis to explore factors influencing bc effectiveness. we categorized the nine sessions into three groups: closed-ended questions with low or medium subject sensitivity (cq_ls: sessions 2, 4, 5), open-ended questions with low or medium subject sensitivity (oq_ls: sessions 1, 3, 6), and open-ended questions with high subject sensitivity (oq_hs: sessions 7, 8, 9). comparing the four conversational engagement metrics between to-dak_bc and todak_nobc within these categories revealed factors affecting bc effectiveness. results are shown in figure 4 . ## conclusion we investigated the effectiveness of bc in improving conversational engagement between humans and ai agents. our user experiment results demonstrated that bc can indeed influence human conversational engagement, contributing to both quantitative and qualitative improvements in conversations. this highlights the importance of considering bc when designing conversational strategies for future ai agent services.