SPEECH SYNTHESIS APPARATUS

Abstract: The invention provides a speech synethesising appar­ atus in which a character string translator 1 converts each word in an encoded input character string into a phoneme string corresponding to the characters in the word using a word memory 2. Next, a variable phoneme detector 3 detects those phonemes in the string the values of whose prosodic parameters may be modified due to the existence of an influencing phoneme at another location in the phoneme string, by comparing each phoneme with phonemes stored in variable phoneme memory 4. If a variable phoneme is detected in the phoneme string, a search is made by an influencing phoneme detector 5 for influencing phonemes at the location in the phoneme string indicated by the variable phoneme detector. The variable phoneme memory stored, along with each variable phoneme, the predetermined relative location at which an influencing phoneme may be found. If an influencing phoneme is detected at the appropri­ ate location relative to the variable phoneme the influencing phoneme detector will output data representative of a modification in the value of a selected parameter (duration, pitch or power) of the variable phoneme. The phoneme string is then delivered to a parameter value determining unit 7, where standard and modified data are combined. Finally, the phoneme string, parameter values, and modifica­ tion data are supplied a parametric synthesizer 7, which assembles them into synthetic speech.

Description: SPEECH SYNTHESIS APPARATUS BACSC;ROUND OF THE INVENTION One of the known methods for transforming character strings into synthetic speech is nsyntbesis by rule." In this method, a character string is first transformed into a sequence of phonemes. Next, the prosodic parameters (the duration, pitch and power) of each phoneme are determined, and speech segments having those parameters are selected from a library of the spectral envelopes of such speech segments. Finally, the phoneme sequence and the parameters are provided to a well-known speech synthesizer which assembles the segments into synthetic speech, adjusting the parameters and connecting them into ntoreobless natural speech. Although it is possible to synthesize speech using standard values of each parameter for each phoneme, it is also possible to vary the duration of those phonemes which consist of a consonant-vowel (CV) combination. When one of these variable phonemes is encountered in a phoneme string, its duration may be modified (changed from the standard value) by considering the influence of the phoneme Immediately before or after the variable phoneme in the phoneme string. However, even when the duration of CV phonemes is modified in accordance with the influence of influencing phonemes Immediately adjacent the variable phoneme, the synthetic speech produced is rather unnatural and unclear. SUMMARY OF THE INVENTION One object of this invention is to produce synthetic speech of such clarity and high quality that it is very nearly natural speech. Another object of the invent ion is to produce synthetic speech in which the prosodic parameters of variable phonemes are mollified as a function of the existence of influencing phonemes at locations in the phoneme string other than the two locatlons immediately adjacent the variable phoneme (as a funtlon of a non-influencing phoneme in a non-adjacent mora). The invention is based on the careful observation of natural speech by the present inventors. They found, for example, that the duration of a variable phoneme consisting of a double consonant is influenced more by the kind of phoneme which exists in the phoneme string in a location one phoneme removed from the double consonant than by the kind of phoneme which exists immediately adjacent the double consonant tn the phoneme string. They observed that, in Japanese speech, a double consonant lasts several milliseconds longer when a "prolonged sounds or a syllable nasal tN" Is at a location In the phoneme string two morae subsequent to the double consonant, The Invention comprises five main elements, A character string translator accepts encoded character strlngs from a device, such as a keyboard, which is capable of inputting character strings electronlcally. Using a word memory, the character string translator converts each word in the Input character string into a phoneme string corresponding to the characters in the word. Next, a variable phoneme detector detects those phonemes in the string the values of whose prosodic parameters may be modified due to the existence of an influencing phoneme at a location in the phoneme string which is at least one mora removed from the location of the variable phoneme. If variable phonemes are detected in the phoneme string, a search is made by an influencing phoneme detector for influencing phonemes at the location in the phoneme string Indicated by the variable phoneme detector. The variable phoneme detector is associated with a variable phoneme memory which stores, along with each variable phoneme, the predetermined location at which an influencing phoneme will influence the value of each parameter of the variable phoneme. If an influ- encing phoneme is detected at the appropriate location In relation to the variable phoneme the influencing phoneme detector will output data representative of a modification in the value of a selected parameter (duration, pitch or power) of the variable phoneme. The phoneme string is then delivered to a parameter value determining unlt, which stores standard values of the parameters for all phonemes. Standard values may be modified in response to the modification data supplied by the influencing phoneme detector. FlraUy, the phoneme string, parameter values, and modification data are supplied to a wellknown parametric synthesizer, which assembles them Into synthetic speech. Of course, since this invention is an electronic apparatus, it does not perform opcratlons on "characters" or "phonemes" but rather on electrical codes representing characters and phonemes. This fact will be silently recognized throughout the specification. BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 is a block diagram of one embodiment of the improved speech synthesis apparatus. Figure 2 is a schematic diagram showing in more detail the character string translator of Figure 1. Figure 3 is a schematic diagram showing in more detail the variable phoneme detector of Figure 1. Figure 4 is a schematic diagram showing in more detail the influencing phoneme detector of Figure 1. Figure 5 ifs a schematic diagram showing in more detall the parameter value determining unit of Figure 1. Flgure 6 is a block diagram of another embodiment of the improved speech synthesis apparatus. DETAILED DESC:RIPTION OF THE PREFERRED EMBODIMENT Fig. 1 is a block diagram of the improved speech synthesls apparatus. An encoded character string is Input from a devlce (not shown) such as a keyboard having character l;c,vs, a memory device which stores character strings (such as used in a word processor), or a communication devlce receiving character strings through communication lines. Character string translator 1 translates the character codes making up the character string into phoneme codes representing a phoneme string, using word memory 2. The phoneme string is supplied to variable phoneme detector 3, which detects variable phonemes. In a Japanese phoneme string, these variable phonemes include the double consonant (hereinafter indicated by "B"), syllabic nasal "not, and prolonged sound (hereinafter indicated by "L"). Such variable phonemes are stored In variable phoneme memory 4 which is used by detector 3 in the detectlon of variable phonemes. As is described later, it a variable phoneme is detected, it is associated with at.ribution data from which parameter value modifications may be determlned. The output of detector 3 includes Information regarding the predetermined location at which an innu encing phoneme must be found in order to influence the value of a parameter (the duration, in the preferred embodiment) of the detected variable phoneme. Influencing phoneme detector 5 determines whether there exists an influencing phoneme, such as a double consonant, syllabic nasal "N" or prolonged sound at the location Indicated by detector 3. Parameter value determining unit 7 stores the normal values of parameters such as the duration of the phoneme, its pitch or its power. If both the variable phoneme and a corresponding influencing phoneme are detected unit 7 outputs a modified value of the glven parameter to parametric synthesizer 8, which may be any well-known synthesizer, for example formant-type, Parcoretype, and Cepstrum-type synthesizer. Fig. 2 shows In more detail the character string translator 1 of Fig. 1. A character string (which normally constltutes one word) enters input register 101 under control of input control circuit 1D2, Read control circuit 103 supplies an initial address to word memory 2, and a word Is read out into register 104. Word memory 2 stores a plurality of words (In segment 104A) together with the corresponding phoneme 6trings into which the words are translated tin segment 104B). The word from segment 104A is supplied to comparator 100 and compared with the content of input register 101. If the input chaw acter string in input register 101 is not the same as the character string from segment 104A, comparator 105 produces a signal on line 106. Responding to the signal on line 106, read control circuit 103 increments its internal counter (not shown) and provides the incremented address to word memory 2, so that the next word In memory 2 (together with the corresponding phoneme strlng) i8 read out into register 104. These operations are repeated untll comparator 105 detects identity between the input character string and the character string retrieved from memory 2. When identity is detected between the input character string in input register 101 and the character string in segment 1D4A of register 104, the comparator produces a signal on line 107, causing the phoneme string In segment 104B to be written into phoneme string memory 108. At the same time, input control circuit 102 causes the nsxt character string to enter input register 101, and read control circuit 103 supplies the initial address to word memory 2 in preparation for the next translation operation. Pig. 3 shows In more detail the variable phoneme detector 3 of Fig. 1. Input control circuit 301 accesses phoneme string memory 108 and successively transfers phonemes into buffer 302. Most Japanese speech consists of four morae, as follows (a timoran is a basic unit of time in speech; it may contain one or more phonemes): (a) a consonant-vowel combination ("CV") (b) an independent vowel, or a prolonged sound (symbolized as nL") (c) a double consonant (symbolized as "B") (d) a syllabic nasal "N" Read control circuit 303 supplies an initial address to variable phoneme memory 4, whIch stores variable phonemes in association with the predetermined location in the phoneme string at which an influencing phoneme will influence the value of the duration of each variable phoneme. Variable phoneme data from memory 4 is written into register 304, which consists of a phoneme segment 304A and a relative location segment 304B. The relative location data in relative location segment 304B consists of at least one location in the phoneme string, relative ro the variable phoneme, where an influencing phoneme may be located to influence the duration of the variable phoneme. For example, if the variable phoneme is a double consonant, "Q", its relative location data may be "+2n, which means that the duration of the double consonant is influenced by an influencing phoneme located two morae later in the phoneme string. Comparator 305 compares the phonemes in phoneme buffer 302 and the phoneme segment 304A of register 304. If they are not identical, comparator 305 produces a signal on line 306. Responding to the signal on line 306, read control circuit 303 increments its internal counter (not shown) and provides the incremented address to variable phoneme memory 4, so that the next variable phoneme in memory 4 Is read out into register 3D4, These operations are repeated until comparator 305 detects identity between the phoneme in buffer 302 and the phoneme retrieved from memory 4 as stored In segment 304A, or unttl all variable phonemes in memory 4 have been retrieved. When identity is detected between the phoneme In buffer 302 and the phoneme in segment 304A, comparator 305 produces a signal on line 307, causing the phoneme from buffer 302 and the relative location data in segment 304B (through selector 308) to be written into memory 309. The selector 308 normally selects the relative location data from segment 304B; however, if identity Is never detected by comparator 305, read control circuit 303 outputs an END signal on line 310, causlng selector 308 to select "O" as the relative location data, Indicating that the phonemes are not variable phonemes. Fig. 4 shows In more detail the influencing phoneme detector 5 of Fig. 1. Read control circuit 501 controls counter 502, whose content indicates an address memory 309. The present address in counter 502 Is supplied to memory 309 through adder 503 which normally adds "O" to the present address. The data read out from memory 309 are provlded to register 505 through selector 504 which normally selects register 506. The data consist of phonemes (in segment 505A) and relative locations.(in segment 505B). The relative location information in segment 505B Is supplied to zero detector which produces a signal on line 507 when the relative location infor- mation in segmenet S05B is "O" or on line 508 when the relative loca tion information is not tO". in the latter case, the signal on the line 508 Is provided to read control circuit 501 to prevent the Incrementing of counter 502, to the sector 504 to select register 506, and to read control circuit 510 to supply an initial address to parameter value variatlon memory B, That is, when the relative location Information Is not "on, the relative location Is added to the present address in counter 502 by adder 503, and the sum Is supplied as an address to memory 309 so that the corresponding data is read from memory 309 and stored in register 506 via selector 504. Parameter value variation memory 6 is accessed by read control circuit 510 and supplies to register 512 data consisting of a phoneme, In segment 512A (an influencing phoneme) and, In segment 512B, attri- button data from which the change in duration of the variable phoneme may be determined. Comparator 513 compares the phoneme in segment 512A with that In segment 506A, producing a signal on line 514 it it detects identity and on line 515 otherwise. Responding to the signal on line 514, the phoneme In segment 5D5A and the attribution data in segment 512B are written into a checked data memory 520. On the other hand, the signal on line 515 is provided to read control 510 to increment an inner counter (not shown) and output the next address to memory 511. The comparator compares the phoneme in rcgister SOf with successively retrieved phonemes in register 512. If none of the phonemes In memory 511 ls the same as with the phoneme In register 506, read control circuit 510 outputs an END signal to selector 516. Selector 516 usually selects the attrlbutlon data segment of memory 512 as input data but instead selects "O" upon receipt of an END signal. Therefore, the phoneme in segment SOSA is written Into checked data memory 520 with attribution data equal to "O", When zero detector 50B detects a tO", it outputs a signal on line 507. In this case, the phoneme in segment 505A is not a variable phoneme, so It is written into the checked data memory 520 with at tributi on data equal to t10". Fig. 5 shows in more detail the parameter value determining unit 7 of Fig. 1. Data consisting of phonemes in combination with attribution data are successively read from checked data memory 520 into register 701. Parameter value memory 702 stores standard values of the parameters for every phoneme. (It Is also possible, instead of storing all the values, to use a parameter calculator according to a phoneme code.) A phoneme in segment 701A of register 701 is supplied to parameter value memory 702 as an address; memory 702 then outputs the corresponding parameter value to adder 703. Modifying data memory 704 stores parameter value modifications and outputs them when addressed by attribution data from segment 701B. The standard parameter values frm memory 702 and the modification data from memory 704 are added by adder ?03 and the sums supplied along with the phoneme string to parametric synthesizer 8 of Fig. 1 for assembly Into synthetic speech. A pitch modifying circuit 710 and a power modifying circuit 720 may be provided for rrodifying pitch or power data in a slmilar manner. TABLE 1 Duration of Character Phoneme (tQ) of "Q" Example String String Phoneme A. Kessen KE/Q/SE/N tQ = tm + ta B. Kesseki KE/Q/SE/KI tQ = tm C. Ikkan I/q/KA/N tQ = tm + ta D. Ikkatsu I/Q/KA/TSU tQ = tm Table 1 shows how the duration of a phoneme ls modified. In example A, when the character string "kessen" (a Japanese word meaning "decisive battle") is entered, character string translator 1 translates it Into the phoneme string KE/QISEIN, where "1" indicates divisions between morae. Variable phoneme detector 3 detects a variable phoneme "Q" in the second location; consequently, influencing phoneme detector 5 should search for an influencing phoneme. In this example, there is an influencing phoneme "N" in the fourth location In the phoneme string, so parameter value determining unit 7 adds a modifying value ta to the standard double consonant duration tm, which means that the duration of phoneme "Q" is given by tQ = tm + ta. If character string t'kesseki" (meaning "absent") Is entered (example B), the translated phoneme string is KE/QISE/KI. Varlable phoneme detector 3 again detects a variable phoneme "Q" In the second location, but influencing phoneme detector 5 does not detect an influencing phoneme. Therefore, parameter value determining unit 7 determines the duration of the phoneme "Q" as tQ = tm, the standard double consonant duration. In example C, "Ikkan" (meaning "consistently") is translated into the phoneme string I1QIKA/N including variable phoneme "N" In the fourth location. The duration tQ of the second phoneme is therefore given by tQ = tm + tao The character string of example D, ''Ikkatsu" (meanlng "togethern), is translated into the phoneme string I/Q/KA/TSU, which Includes variable phoneme "Q" in the second location but no influencing phoneme, so that the duration of the second phoneme is tQ = tm. The standard duration $ m of a double consonant in Japanese Is 170 ms, and additional duration ta is, for example, 50 ms. According to this embodiment, clear and naturalized speech can be synthesized by considering the influence of a non-adjacent phoneme on the duration of a variable phoneme. Fig. 6 Is a block diagram of another embodiment of this inven tion. Independent vowel detector 11, neighboring vowel detector 12, and prolonged sound transforming unlt 13 are insertea between char- acter string translator 1 and variable phoneme detector 3 of Fig. 1. The remaining elements of Fig. 6 are the same as in Fig. 1, so their descriptions are omitted here. Most Japanese words consist of four morae; and of these, words havlng a syllabic nasal "Nn the fourth location appear most frequently. The present Inventors have observed that the duration of the syllabic nasal "N" In the fourth location is related to the existence of a syllable nasal "N" or prolonged sound (L) in the second location. Independent vowel detector 11, including a comparator (not shown), detects whether the phoneme string includes independent vowels 0, U, or I. If the phoneme string includes such vowels, neighboring vowel detector 12, also including a comparator, detects the identity of the phoneme immediately preceding the detected vowel. Then prolonged sound transforming unit 1X, including a code converter, transforms the detected independent vowel into the prolonged sound Ii If and only if the combination of the detected independent vowel and the immedlately preceding phoneme fall into one ot the following later gorses: (1) The detected independent vowel is either U or O and the preceding phoneme is elther U or 0. (2) The detected independent vowel is I and the preceding phoneme Is either I or E. Table 2 shows examples of these cases. TABLE 2 Original Translated Result of Character Phoneme Vowel Example String String Transformation E. Kuudoo KUUDOO KULDOL F. Ookl OOKII OLKIt G. Keikel KEIKEl KELKEL The reason for transformation of vowels into prolonged sounds Is that the above independent vowels become prolonged sounds of the immediately preceding vowels in natural Japanese speech. Moreover, it is helpful, In determining the parameter values of the syllabic nasal or prolonged sound, to consider the influence of a precedlng phoneme in the phoneme string. TABLE 3 Duration of Character Phoneme Fourth Example String String Phoneme H. Dangan DA/N/GA/N t4 = t2 1. Keisan KE/I/SA/N t4 = t2 J. Tousel TO/U/SE/I tq = t2 K. lintel Nl/N/TE/I t4 = t2 Table 3 shows examples of the determination of phoneme duration. In example H, the character string "dangan" (meening "bullet") includes the syllabic nasal sound "N" In the fourth location. The duration of this sound is influenced by the syllable nasal sound N in the second location, so the duration of the former is the same as the latter (t4 = t2). These examples indicate that the duration of the syllabic nasal nN" or the prolonged sound L is the same as the duration of the second preceding mora If the phoneme is either at that location the syllabic nasal "N" or the prolonged sound (L). Although illustrative embodiments of the present invention have been described with reference to the accompanying drawings, it Is to be understood that the invention is not limited to those precise embodlments and that various changes and modifications may be effected therein by one skilled in the art without departing from the scope of the present invention. Specifically, while the preferred embodiment has been illustrative as synthesizing Japanese speech, it Is equally applicable to speech of any other language.

Claims: CLAIMS: 1. An apparatus for creating synthetic speech from a character string, comprising: translating means for translating the character string into a phoneme string, parameter value determining means for storing a value of a selected parameter of each phoneme in the phoneme string, and parametric synthesizer means for assembling the phoneme string into synthetic speech using the stored values, characterised by variable phoneme detecting means coupled to said translating means for detecting, in the phoneme string, a variable phoneme, the value of the selected parameter of which is a function of the identity of an influencing phoneme in the phoneme string; and influencing phoneme detecting means coupled to said variable phoneme detecting means for determining the identity of the influencing phoneme, said parameter value determining means being responsive to the result of the determination for modifying the stored value of the selected parameter. 2. An apparatus as claimed in claim -1 wherein the selected parameter is duration. 3. An apparatus as claimed in claim 1 or 2 wherein the predetermined location is at least one mora removed from the variable phoneme in the phoneme string. 4. An apparatus as claimed in claim 1, 2 or 3 wherein the translating means has an associated memory for storing a phoneme string corresponding to each of a plurality of words, the translating means having comparing means for outputting the corresponding phoneme string when a stored word matches a portion of the character string. 5. An apparatus as claimed in one of claims 1 to 4 further comprising memory means associated with said variable phoneme detecting means for storing, in association with a variable phoneme, the relative location at which an influencing phoneme will influence the value of the parameter of the stored variable phoneme, the variable phoneme detecting means including comparing means for outputting the associated predetermined location when a stored variable phoneme is detected in the phoneme string. 6. An apparatus as claimed in any preceding claim further comprising memory means associated with the influencing phoneme detecting means for storing, in association with the identity of an influencing phoneme, data representative of a modification in the stored value of the selected parameter, the influencing phoneme detecting means including comparing means for outputting the associated data when the stored influencing phoneme is in the predetermined location. 7. An apparatus as claimed in any of claims 1 to 6 wherein the variable phoneme is a syllabic nasal "N". 8. An apparatus as claimed in any of claims 1 to 6 wherein the variable phoneme is a double consonant. 9. An apparatus for creating synthetic speech from a string of character codes, comprising: code translating means for translating the string of character codes into a string of phoneme codes, and speech synthesizer means for assembling the string of phoneme codes into synthetic speech, characterised by detecting means for detecting, in said string of phoneme codes, a pair of phoneme codes in non-adjacent morae, and modifying means coupled to said detecting means for modifying a selected parameter of a phoneme represented by one of said phoneme codes in non-adjacent morae. 10. A method of creating synthetic speech for a character String, comprising the steps of: (a) translating the character string into a phoneme string; (b) searching in the phoneme string for any variable phoneme which has a prosodic parameter which is alterable by the presence in the phoneme string of an influencing phoneme in non-adjacent morale; (c) searching in the phoneme string for influencing phonemes for any variable phonemes found; (d) determining the influence on any variable phoneme of any influencing phoneme located, and (e) assembling into synthetic speech the phoneme string with individual phonemes as modified by any determined influences.

Inventors: NOMURA, NORIMASA, WATANABE, SADAKAZU
Applicants: KABUSHIKI KAISHA TOSHIBA
Application year: 1984
Main classification: G10L5
IPC codes: G10L13