A Tone-based Hierarchical Structure of C hinese Prosody (2024)

Abstract: “In Chinese speech engineering, many projects use a conventional, syllable-based prosodic hierarchyas an underlying framework to process natural or synthesized speech. However, Chinese as a tonelanguage has its own way of expressing prosody, that is, through tonal interaction, especially tonesandhi. By utilizing the capacity of tone as a dual unit of pitch and timing, the present study proposesa tone-based, three-layer-four-level structure for Chinese prosody. The three layers are tone, toneprosody, and intonation, respectively composed of one level of pitch units, two levels of toneprosody units (basic and derived), and one level of intonation units. These four levels of units areused to replace syllable, prosodic word, phonological phrase, and intonational phrase in aconventional hierarchy. Tone prosody units are established based on sizes or types of tone sandhidomains, so when applied to the same clause uttered in Mandarin and Shanghai Wu Chinese, theyare timed differently and branched toward different directions at different levels, hence capable ofcapturing rhythmic and melodic patterns of the two distinctive types of Chinese. Overall, given itstheory-friendly design, the proposed structure may be used as a unifying framework in Chinesespeech engineering.”

## introduction in speech engineering, it is notoriously difficult to improve the naturalness of synthesized speech due to multiple linguistic and non-linguistic factors involved. it is especially true for chinese speech synthesis, because it is hard to find a unifying prosodic framework optimized for the whole chinese language family, both standard mandarin chinese and other chinese dialects. at present, most models assume a syllablebased prosodic hierarchy developed for non-tonal languages. such a hierarchy is conventionally divided into eight layers from bottom to top: mora, syllable, foot, prosodic word, clitic, phonological phrase, intonational phrase, and utterance @xcite . this hierarchy is built around stress foot, hence readily applicable to stress languages like english. for chinese, according to @xcite , a simplified version of this conventional hierarchy is generally assumed, and it has three basic levels, prosodic word (pw), prosodic phrase (pp), and intonational phrase (ip). however, the applicability of such a simplified hierarchy to chinese is still questionable, because chinese has tone rather than stress on its syllables and the status of tone is not reflected in this hierarchy. for example, @xcite found that mandarin chinese is more syllable-timed and also more melodious than french, and french in turn more syllable-timed and also more melodious than english. in addition, english declarative sentences show an overall falling melody whereas french ones an overall rising melody. @xcite conducted a further study of four chinese dialects, northern (including mandarin), cantonese, wu, and min. her results show that the four dialects all tend to be syllable-timed and melodious, but to different degrees: cantonese and northern dialects are more syllable-timed and also more melodious than wu and min. she attributed these differences to their distinctive tone inventories and tonal behaviours. last, many experimental studies find that different phonetic cues are used to signal boundaries of different prosodic units @xcite . in mandarin, an intonational phrase is cued by pitch height decline towards the end, followed by a long pause; and a phonological phrase by final lengthening. prosodic words, on the other hand, do not have clear boundary cues, but they can be cued by tone sandhi. ## the proposed prosodic structure the underlying assumption of the proposed structure is that the pitch-timing duality of tone is capable of directly or indirectly reflecting chinese melodic and rhythmic patterns. in other words, tone alone is sufficient to characterize chinese prosody. therefore, the syllable layer in the conventional prosodic hierarchy can be replaced by the tone layer. also, the syllable-based stress foot can be replaced by a tonebased unit, called a tone prosody unit (tpu) in the present study. a tone sandhi domain is a typical tpu. since tone sandhi is closely related to morpho-syntactic conditions, a sandhi-domain-based tpu helps mitigate the need to distinguish words from phrases. tpus can be divided into two types and placed on two levels. there is good reason to differentiate them in terms of levels, which will be explained soon. now the tone-based prosodic structure can be illustrated below: in figure 1 , the tone-based structure has three core layers: t, tp, and i. they are composed of different prosodic units. pu refers to different pitch height components, usually represented by pitch levels, high, mid, and low (h/m/l), or pitch values from 5 to 1 on a five-level scale. for example, the mandarin fourth tone (t4), a falling tone, is usually represented as hl or 51, indicating that it contains two pus, h and l. note that if a bu contains two tones, a tonal interaction process (including tone sandhi) will be evoked to reduce the pu number due to the timing pressure, but the pu in the head position will be retained due to the need to create prosodic prominence @xcite . take the mandarin third tone (t3) sandhi for example. if a bu contains a t3.t3 sequence (llh.llh), it will undergo the t3 sandhi, turning t3.t3 into a sequence similar c c l 2 0 2 4 to t2.t3 (lh.llh). in connected speech, the resulting pitch sequence lh.llh is often reduced to lhl, where the central h in the head position is retained. 2 next, the structure has bu and du in the same tp layer but on different levels. they are in the same layer because they both can be domains of tone sandhi and share some structural features; for example, they both can be recursively built by adjoining another tone, a process similar to cliticization. their division, on the other hand, is for the purpose of accommodating tone sandhi domains of different sizes or types. later, examples will be given to show their differences. generally, they encompass the conventionally defined prosodic word, clitic, and phonological phrase, but there is no strict one-on-one relationship between bu and prosodic word as well as between du and phonological phrase. last, the upper level iu is equivalent to the conventionally defined intonational phrase. overall, the proposed structure is aimed to capture chinese the t3 sandhi applies not only in bus as shown above but also across adjacent units as shown below: ## application of the proposed structure to an utterance this section shows how the tone-based structure is further applied to a clause, "once, the north wind and the sun were disputing which was stronger", uttered by the same female speaker respectively in mandarin and shanghai wu. first, the pitch contour of the mandarin utterance is displayed below: in figure 3 , most units have a size of two or three tones. also, the first few units are similarly timed, as evidenced by their time intervals around 0.5s. note that the three-tone unit you.yi.hui ('once') has a little longer interval than the following two three-tone units, and the two-tone unit zheng.lun ('dispute') has a little shorter interval than the preceding three-tone units. this pattern suggests that both units are on the du level by themselves, so the final lengthening effect applies to them, making the former longer than and the latter close in length to regular three-tone units. in terms of melody, the whole contour lowers at the end, as evidenced by a large pitch height difference between the two high-falling t4s, one on tai in the word tai.yang ('sun') and the other on the final word da ('big'). overall, the contour is relatively melodious. now the underlying tone-based prosodic structure can be drawn for the mandarin utterance as follows: t1.t2)]}{[(t3.t1.t1)(t4.t2.t4)][(t1.t4)][(t2.t0)(t3.t4)][(t4)]} {you.yi.hui}{bei.feng.gen-tai.yang.zai-zheng.lun-shui.de-ben.shi-da} 'once, the north wind and the sun were disputing which was stronger' in figure 4 , t1(55), t2(35), t3(214), and t4(51) are full tones and t0 is the light tone. note that full tones can reduce to t0, a process called neutralization. for example, the high-rising t2 on yang in the word 5 @xcite proposed two similar units, lexical and phrasal mrus (minimal rhythmic units), to account for different tone sandhi domains. however, he did not discuss their status in a prosodic hierarchy, let alone their linking relationship. ## a comparison with other structures hirst (2005) adopted a simple syllable-based four-level structure to capture the melodic and rhythmic differences between english and french. for english, the four levels of units from the bottom up are syllable (syl.), rhythm unit (ru), tonal unit (tu), and intonation unit (iu). an ru can cover a trochaic stress foot and monosyllables outside the foot, delimited by word boundaries. a tu can cover rus, delimited either by a foot or by an iu boundary. this structure allows a nuclear accent to occur non-phrase-finally, reflecting the left-heavy prominence of english ([h l]). for french, the four levels of units are syl., tu, ru, and iu, with ru and tu reversed. tu being placed under ru indicates that no nuclear accent occurs phrase-initially, reflecting the right-heavy prominence of french ([l h]). accordingly, english has a falling melody while french a rising one, and english has a stress-timed rhythm while french a syllable-timed one. the advantage of this structure is its ability to integrate prosodic functions into a hierarchy without immediate morpho-syntactic concerns. however, this simple structure is not applicable to chinese, because chinese syllables are mostly toned, and the prosodic and morpho-syntactic interface is mainly made visible through tone sandhi rather than stress foot. also, tu and ru, even if applicable to chinese, can be consolidated into one given the pitch-timing duality of tone, and the tpu is just such one. in addition, the tone-based structure is applicable to various chinese dialects and able to provide a unified interface between prosody and morpho-syntax. this is because the prosodic features of different chinese dialects can be reflected by the nature, head position, and branching direction of their tpus. as long as tone sandhi patterns are identified, two levels of tpu (bu/du) can be formed regardless of whether they are associated with words, phrases, or clauses. now english, french, and chinese are compared below in terms of their four-level structures and prosodic patterns. a simplified conventional structure is also included. prosodic structure english french chinese conventional level 4 level 3 level 2 level 1 iu ↑ tu ↑ ru ↑ syl. iu ↑ ru ↑ tu ↑ syl. iu ↑ du ↑ bu ↑ pu ip ↑ pp ↑ pw ↑ syl. melodic pattern [h l] [l h] [h l]/[l h] not specified rhythmic pattern stress-timing syllable-timing tpu-timing syllable-timing the last two rows in figure 7 suggests that a syllable-based structure, however simple, is neither fully nor directly able to reveal chinese melodic and rhythmic patterns as the tone-based one does. this is because chinese can have the melodic pattern either falling or rising (corresponding to left-or right-heavy) at c c l 2 0 2 4 different levels (hence [h l]/[l h]). also, tpu-timing is a more appropriate term used to describe chinese rhythm than stress-or syllable-timing, as it is entirely pu-based, and the pu number helps determine the tone and subsequent tpu length. ## implications of the proposed structure for prosody modeling the above section shows that the proposed tp layer is related to different sizes or types of tone sandhi domains but does not directly refer to their morpho-syntactic status. in other words, tpu-based prosody processing does not need to rely heavily on grammatical markings, hence compatible with various theorydependent models. however, any new model is subject to empirical validation, and the proposed one is no exception. in order to gain some insight for the future validation work, two prosody modeling studies are discussed below, because they are related to implications of the proposed structure. first, hirst (2015) developed a modeling tool, prozed, based on the four-level english structure shown in ability to maintain a good text and speech alignment, especially when functional words and verbal phrases are involved, both prone to phonology-syntax mismatch. when a mismatch occurs, the se can be adjusted accordingly in size, since they are not confined to grammatical structures. the present study considers an se in essence a bu or bu', so it is expected that a bu-based system works as well. nonetheless, the proposed structure is yet to be applied in future modeling projects and validated with favorable results. ## conclusion the present study illustrates the tone-based prosodic structure and its application in two types of chinese. since this structure can accommodate both prosodically and morpho-syntatically based tone sandhi domains, there is no need to consider whether or not syllables are stressed: prosodic units are headed on each level and timed according to size. unlike the above mentioned syllable-based structures, the proposed one not only helps capture melodic and rhythmic patterns of different chinese dialects but also provides a theory-friendly framework for chinese speech engineering.