Voice and Choice: Investigating the Role of Prosodic Variation in Request Compliance and Perceived Politeness Using Conversational TTS (2024)

Abstract: As conversational Text-to-Speech (TTS) technologies become increasingly realistic and expressive, understanding the impact of prosodic variation on speech perception and social dynamics is crucial for enhancing conversational systems. This study explores the influence of prosodic features on listener responses to indirect requests using a specifically designed conversational TTS engine capable of controlling prosody, and generating speech across three different speaker profiles: female, male, and gender-ambiguous. We conducted two experiments to analyse how naturalistic variations in speech rate and vocal energy (projection) impact the likelihood of request compliance and perceived politeness. In the first experiment, we examined how prosodic modifications affect the perception of politeness in permission- and service requests. In the second experiment participants compared pairs of spoken requests, each rendered with different prosodic features, and chose which they were more likely to grant. Results indicate that both faster speech rates and higher projection increased the willingness to comply, though the extent of this influence varied by speaker gender. Higher projection in service request increases the chance of being granted more than in permission requests. Politeness has a demonstrated positive impact on the likelihood of requests being granted, this effect is stronger for the male voice compared to female and gender-ambiguous voices.

## introduction the importance of pragmatics in the development of conversational technologies is becoming increasingly critical @xcite . as text-to-speech (tts) systems achieve greater realism in speech generation, a significant gap persists in understanding the pragmatic effects these technologies have within interactions. by modeling prosodic features based on empirical research, conversational tts can be made more engaging and effective in a variety of interactive contexts. the necessity for human-oriented pragmatics in these systems is particularly evident in scenarios requiring compliance to requests. moreover, understanding the subtleties of how politeness is conveyed in language and speech is crucial for comprehending the factors influencing request compliance, especially in conversational systems that utilise tts. people use a variety of spoken strategies to manage the potential threats that communication can pose to their own and others' self-esteem and autonomy @xcite . indirect requests are employed to mitigate face-threatening acts, demonstrate respect for the listener's autonomy, and maintain the fundamentally cooperative and prosocial nature of human communicative behaviour and respect between interlocutors @xcite . they are a fundamental aspect of polite discourse, reflecting the speaker's sensitivity to social dynamics and the listener's ability to interpret and respond to nuanced communicative cues @xcite . initiating actions, such as requests, can be seen as a basic form of social coercion @xcite . requests have a bearing on the sequential organisation of the ensuing talk, while also restricting the agency of the requestee, and even threatening their autonomy @xcite . the linguistic structure of requests influences politeness and compliance. for example, the choice between using an imperative form, which might seem direct and blunt, and opting for a more conditional or interrogative form can alter the level of imposition perceived by the interlocutor. @xcite emphasise the strategic use of politeness markers like 'please' in everyday requests, demonstrating how these markers are employed to manage face-threats in ill-fitted interactional contexts, particularly when requests could be seen as intrusive or when they encounter resistance from the requestee. research on modal constructions in requests reveals further nuances in how these requests are framed and understood in different contexts @xcite . modal verbs like 'could' or 'might' introduce a level of uncertainty or optionality into the request, thereby softening it and enhancing its politeness. @xcite highlights the importance of the "infrastructure" that underpins requests, which includes the social and interactional contexts influencing how requests are made and received. understanding the interplay between prosodic features and sociolinguistic norms is essential for designing effective conversational agents. in our study, we develop and employ a prosodycontrollable gender-ambiguous tts system as a research tool to conduct controlled experiments assessing the role of prosodic variation in request compliance. this approach allows us to isolate the impact of vocal traits from gender biases without relying on the ability of voice actors to consistently reproduce prosodic variations. the key contributions of this research are the following: we pioneer the use of a gender-ambiguous neural tts built on spontaneous speech in perceptual studies, which allows for an unprecedented exploration of how gender perception influences listener responses to prosodic variations. moreover, this study provides empirical evidence on how natural variations in speech rate and vocal energy influence the listener's perception of politeness, and their responsiveness to indirect requests. our findings illustrate that the impact of prosodic variations can differ based on the speaker's gender profile, contributing to a more tailored approach in the design of tts systems to accommodate diverse user interactions. ## overview of the method in this study, we develop and utilise a multi-speaker tts model built upon two corpora of spontaneous speech, to investigate the impact of prosodic variations on listener responses to indirect requests. tts model development: a spontaneous conversational tts system is engineered to include gender-ambiguous voice capabilities, using a mod-ified tacotron 2 architecture @xcite that allows for the dynamic control of prosodic features at the utterance level. this setup enables exploring how different prosodic renderings affect listener perceptions. stimuli design and synthesis: stimuli for the experiments were designed to directly address our research questions regarding the social dynamics of request-making in conversation. using an interactive interface, we synthesised these stimuli, ensuring each varied systematically in prosody according to predefined settings. this approach allowed precise control over the acoustic and prosodic variables of interest. verification of stimuli: before deployment in experiments, all stimuli are tested for naturalness, gender ambiguity, and the presence of significant acoustic-prosodic differences across conditions, using objective measures. these verifications support the reliability and validity of the stimuli used in the subsequent online listening tests. ## discussion one of the limitations in our study is that our experiments utilised only one voice per gender. to enhance the generalisability of our findings, future work will explore the use of voice conversion technologies to create a wider variety of stimuli across different gender profiles. additionally, while this study primarily focused on prosodic features such as pitch, energy and speech rate, there are numerous other features in request articulation that warrant exploration. these include voice quality, placement of emphasis, the strategic use of pauses, and utterance-final intonation which can influence the perception of requests. moreover, as levinson (2024) points out, utterances are unlikely to be action-determinate by virtue of their form alone. experiment 2 addresses this to an extent by presenting listeners with an imaginary scenario, but it is important to acknowledge the inherent limitations of controlled listening experiments in simulating the complex dynamics of wider social contexts. consequently, the findings from this experiment should be further evaluated in more realistic, interactive scenarios where deeper contextual embeddings can be implemented. reflecting the rate of technological advancements, we expect to see an increasing demand for personalised, conversational tts to represent and display the identity of individuals in real and virtual environments. one group, individuals with disabilities (e.g., cerebral palsy, autism, adult-onset disorders) who rely on computer-based augmentative communication technologies, already use tts to engage in real-time spoken conversations. the lack of pragmatically appropriate and effective tts to accomplish various conversational tasks, including indirect requests, is a common critique of commercial acts. our findings specifically show that adjustments to prosodic features such as speech rate and vocal effort significantly impact the perceived politeness of requests, and also affect compliance rates. this is particularly important for the design of tts systems in acts, where effectively conveying requests in a polite manner is essential for users with communication challenges. our hope is that incorporating these insights, developers can better equip conversational systems to meet the varied communication demands of individuals, ensuring more respectful and successful interactions across both real and virtual settings. ## conclusions this study has demonstrated that the perception of politeness significantly enhances the likelihood of requests being granted. the effectiveness of changing politeness through prosody is stronger for the male voice compared to female and genderambiguous voices. additionally, higher vocal effort in action requests significantly increases the chances of compliance, more so than in permission requests. this highlights the significant role that prosodic manipulation of tts can play in enhancing the effectiveness of communicative acts within spoken dialogue systems to accommodate diverse user interactions more effectively.