Prosody: Models, Methods, and Applications (2021)

Abstract: Prosody is essential in human interaction, enabling people to show interest, establish rapport, efficiently convey nuances of attitude or intent, and so on. Some applications that exploit prosodic knowledge have recently shown superhuman performance, and in many respects our ability to effectively model prosody is rapidly advancing. This tutorial will overview the computational modeling of prosody, including recent advances and diverse actual and potential applications.

## motivation prosody is essential in human interaction, enabling people to show interest, establish rapport, efficiently convey nuances of attitude or intent, and so on. this tutorial will overview the computational modeling of prosody, including recent advances and diverse actual and potential applications. we define prosody broadly, as the aspects of spoken utterances that are not governed by segmental contrasts. some applications that exploit prosodic knowledge have recently shown superhuman performance, and our ability to effectively model prosody is rapidly advancing. yet prosody remains challenging to work with because it operates close to the limits of conscious introspection, and because most spoken utterances involve multiple prosodic dimensions simultaneously serving multiple communicative functions. intuitions about prosody are often a weak guide for applied work, but a little bit of basic knowledge can go a long way. throughout, diversity will be a recurring theme, in terms of the different ways in which prosody serves different kinds of functions, in terms of differences in prosodic behaviors across genres, in terms of prosody in typologically-different languages, and in terms of diverse applications. ## target audiences we envisage three main audiences. 1. many students of computational linguistics have little exposure to prosody, and what they do learn is usually 10 to 20 years out of date. there are great opportunities in industry for speech scientists and engineers (as distinct from language scientists and engineers in general) with unmet needs in the tech giants, in traditional industries, and in start ups. the rise of conversational agents has greatly increased student interest in speech, and we hope that our tutorial will help satisfy their curiosity and open doors for students who might not otherwise even be considered for positions in this field. while today most aspects of speech processing are handled by algorithms which are also used for other computational linguistics purposes, prosody, as a phenomenon entirely unique to the spoken language, has different properties and different functions from the rest of language, and is thus possibly the most important aspect of speech for students to learn about. but almost anyone in computational linguistics may benefit from this tutorial, as prosody is a topic of wide cross-cutting relevance, including to grammar, discourse, pragmatics, nonverbal communication, and language learning. considering the roles and nature of prosody may provide insight and new ways to look at both classic problems and emerging applications, such as those involving multimodalilty, hard realtime performance, and perceptions of systems as humanlike agents. this tutorial will be at an introductory level, assuming no previous knowledge of prosody. we expect that most participants will be familiar with basic issues in modeling language and in standard methods for learning from data, but no specific knowledge will be assumed. familiarity with basic phonetics and phonology would be helpful, but is again not assumed. ## resources available at http://www.cs.utep.edu/nigel/introto-prosody/ .