Data Augmentation Integrating Dialogue Flow and Style to Adapt Spoken Dialogue Systems to Low-Resource User Groups (2024)

Abstract: This study addresses the interaction challenges encountered by spoken dialogue systems (SDSs) when engaging with users who exhibit distinct conversational behaviors, particularly minors, in scenarios where data are scarce. We propose a novel data augmentation framework to enhance SDS performance for user groups with limited resources. Our approach leverages a large language model (LLM) to extract speaker styles and a pre-trained language model (PLM) to simulate dialogue act history. This method generates enriched and personalized dialogue data, facilitating improved interactions with unique user demographics. Extensive experiments validate the efficacy of our methodology, highlighting its potential to foster the development of more adaptive and inclusive dialogue systems.

## introduction as an innovative technology at the forefront of artificial intelligence and speech processing, spoken dialogue systems (sdss) have attracted significant interest from both academia and industry @xcite @xcite . despite the powerful capabilities of large language models (llms), traditional sds remain a focal point of research due to their superior control and interpretability @xcite . these systems are predominantly trained using data from human-to-human interactions, which highlight varying speaking styles, such as clarity of intentions, as depicted in figure 1 . this variability necessitates that human speakers adjust their dialogue strategies when engaging with different users. for instance, compared to adults, minors often exhibit less clarity in their intentions and give ambiguous responses, requiring more confirmatory language or additional inquiries to better adapt to the unique speaking styles of younger users. this adaptive approach is crucial for enhanc-👦 figure 1 : real human-to-human conversations. speakers adopt various dialogue acts when interacting with users employing diverse speaking styles. however, adapting sdss to these distinctive speaking styles typically requires a wealth of annotated dialogue data, which can be challenging to obtain in abundance owing to the minority status of users employing unique conversational strategies or behaviors. to enhance the performance of dialogue systems when engaging with data-scarce user groups exhibiting distinct speaking styles, targeted data augmentation is imperative, enabling the system to better cater to their needs. this study introduces a tailored data augmentation framework designed specifically for lowresource user groups exhibiting distinctive conversational behaviors. recognizing the unique conversational behaviors and challenges associated with minors and the inherent difficulty in obtaining their data @xcite , our study conducts experiments utilizing dialogue data from minors to facilitate targeted data augmentation for this demographic. as depicted in figure 1 , the unique speaking style of users directly influences the speaker's dialogue acts (das) and indirectly shape response content. therefore, our data augmentation framework focuses on the speaking styles of users and the trajectory of das. specifically, we utilized a llm to extract the speaking styles of such users and speakers interacting with them. we then fine-tuned a pre-trained language model (plm) using all available data in a low-resource setting to create varied histories of das for speakers interacting with these user groups. the resulting speaker styles and da histories were input into the llm to produce customized training dialogue data for these users. the primary goal is to enhance the model's ability to predict das when interacting with low-resource groups with unique speaking styles, as controlling the content of generated responses through das is deemed effective @xcite . this study's contributions are outlined below. ## related work the scarcity of annotated data and the challenge of data imbalance are persistent issues in various artificial intelligence domains @xcite @xcite @xcite . to address these effectively, data augmentation techniques have been employed, as demonstrated in prior research across different tasks @xcite . for instance, schick and schütze ( 2021 ) generated text similarity datasets from scratch by instructing a large plm. similarly, @xcite and @xcite enhanced data by manipulating individual utterances within dialogues-such as adding, deleting, changing their order, or regenerating them-while preserving the original meaning, which improved model performance in dialogue summarization tasks. while the abovementioned methods focus on generating individual sentences, our study aims to create coherent dialogues comprising multiple sentences tailored for specific target groups. @xcite utilized gpt-2 (radford et al., 2019) to develop user and agent bots, generating comprehensive task-oriented dialogues through bot interactions, demonstrating notable enhancements in low-resource scenarios with datasets multiwoz @xcite and per-sonachat @xcite . recently, with the advanced text generation capabilities of llms, researchers have started using llms for data augmentation @xcite @xcite . for instance, @xcite guided llms to generate a broad spectrum of social dialogues using social commonsense knowledge from a knowledge graph. @xcite generated domain-specific, task-oriented dialogues by extracting dialogue paths from out-of-domain conversations. the concept of dialogue paths in their work aligns with the concept of da history in our research. however, the key distinction is that while they extract da paths from existing data, we generate tailored da histories based on existing data, specifically optimized for target user groups. ## experiment to evaluate the effectiveness of the proposed data augmentation framework, we conducted experiments using data from minors who employed unique conversational styles and strategies in actual dialogues within the dataset. these experiments were carried out in a low-resource setting across multiple splits, each utilizing different subsets of data from minors. we trained multiple da prediction models on datasets of varying sizes, including models trained with augmented data added to the existing datasets. ## results and analysis additionally, since zero-shot does not use minor's dialogues, the training data remains consistent across the four different splits. the variation in zero-shot's performance across the splits further underscores the differences in the model's adaptability to different minors, with the third split proving most challenging. across the four splits, the performance of our proposed data augmentation framework, low-resource + aug (ours), almost all surpassed that of low-resource on both t5 and gpt-neox in terms of mean exact and partial match rates. this demonstrates that even in a low-resource setting, our method successfully captures the characteristics of minor speakers and generates dialogue flows that align with minor speaking behaviors, thereby guiding the generation of training dialogues. however, even though we augmented the data to match the quantity of the full-resource in each split, full-resource typically showed superior performance. a possible explanation is the lack of quality control, which meant that subpar data was not filtered out, leading to poorer adaptation compared to full-resource, which used data exclusively from real human conversations. additionally, the "travel agency task dialogue corpus," derived from video calls and manually transcribed, may contain colloquial filler words and other informal elements in its complete utterances. in contrast, chatgpt-generated dialogues tend to be more structured and fluid. this stylistic difference could also contribute to the observed performance disparity between low-resource + aug (ours) and full-resource. ## conclusion we introduced a data augmentation method designed to enhance the performance of the da prediction model for users with limited data and unique conversational styles. our experiments confirmed the reliability of the proposed method and the effectiveness of its components. while this study did not exhaustively explore the full potential for improvement of the proposed method, we plan to further evaluate this aspect in our future work.