Prosodic Analysis and Modelling of Conversational Elements for Speech Synthesis.

Currently, speech synthesisers sound natural when reading text aloud. However, there is an increasing interest in more expressive speech synthesisers for many new applications. Giving the synthesiser the ability to produce speech in a variety of styles is a way to gain this expressiveness. Speech in conversations is the most widely used type of speech. Therefore, more than other aspects, Text-to-Speech systems do need to deal with elements that differentiate between reading and conversation. This thesis focuses on describing the characteristic elements of conversational speech and proposing models to synthesise them. Conversational speech differs significantly from reading aloud due to the inclusion of a variety of prosodic resources that affect the rhythm of the utterances. These resources are, in fact, strategies to achieve communicative tasks between the speaker and the addressee. One type of strategy is disfluency. They are very frequent in normal speech, and in contrast to what one might expect, they carry information and are useful for human communication. Other characteristic elements of conversational speech are closed defined phrases that are repeated through speech and that carry affective rather than propositional information. I present an analysis of three different corpora. One is very similar to the most widely used corpora in the literature for the study of disfluencies. It consists of a set of conversations. Another one was recorded for machine translation and consists of a set of speeches. The third one was recorded from ill-formed sentences. These are sentences generated by automatic systems with incorrect syntactic structures. The corpus with the highest number of disfluencies is the conversational one. This supports the claim that disfluencies are not related to problems in speaking or planning the discourse, but are inherent to conversations. Disfluencies are disruptions of the speech flow, and they can be seen as discontinuities. It is reasonable to assume that the prosody will be different from that expected for the same elements within a fluent sentence only around such discontinuities. Speech is fluent between discontinuities; thus, we can define regions within which speech is fluent and name them fluency regions. Following the previous hypothesis, considering that the elements in a disfluency are fluent within the correct context, it implies there exists a sentence where we could insert an element of a disfluency and the sentence is still perceived as fluent. We refer to this sentence as the underlying fluent sentence. The use of this assumption is supported by studying the impact of disfluencies in a sentence on the speech rate and pitch. Findings show that the inclusion of disfluencies only generates local modifications around fluency boundaries with respect to the underlying fluent sentences. The prosodic parameters that better describe fluency boundaries have been identified for each type of conversational elements: hesitation, repetitions, filled pauses and wrappers. Regression models are proposed to predict the prosody of these elements. The approach proposed in this thesis, which relies on the underlying fluent sentence assumption, can take advantage of prosodic models trained on fluent speech. Thus, there is no need to re-train the prosodic models. In unit selection speech synthesis, the creation of the inventory is a very important step for the final quality of the system. Some new units have to be recorded for one of the conversational elements studied. I discuss the corpus design for the creation of inventories for unit-selection-based conversational speech synthesis. Then, I also present the conversational module implemented in the Ogmios speech synthesis system. The system is now able to generate conversational elements by inserting SSML (Speech Synthesis Markup Language) marks in the input text. A perceptual test was carried out by 28 participants. I showed that the overall listening effort of the system corresponds to a MOS score of four to five for the baseline system, as well as for the proposed systems. For naturalness, the system reached a median value of four for fluent sentences, and this value is maintained after the insertion of conversational elements. In the case of hesitations, naturalness was slightly degraded, and for repetitions, it clearly improved. From these results, we claim that our synthesiser is now able to generate conversational speech at a certain level without decreasing the naturalness of the system, and even increasing it in some cases.