Abstract:
A method is presented for forming the excitation signal for a glottal pulse model based parametric speech synthesis system. In one embodiment, fundamental frequency values are used to form the excitation signal. The excitation is modeled using a voice source pulse selected from a database of a given speaker. The voice source signal is segmented into glottal segments, which are used in vector representation to identify the glottal pulse used for formation of the excitation signal. Use of a novel distance metric and preserving the original signals extracted from the speakers voice samples helps capture low frequency information of the excitation signal. In addition, segment edge artifacts are removed by applying a unique segment joining method to improve the quality of synthetic speech while creating a true representation of the voice quality of a speaker.

Description:
BACKGROUND 
       [0001]    The present invention generally relates to telecommunications systems and methods, as well as speech synthesis. More particularly, the present invention pertains to the formation of the excitation signal in a Hidden Markov Model based statistical parametric speech synthesis system. 
       SUMMARY 
       [0002]    A method is presented for forming the excitation signal for a glottal pulse model based parametric speech synthesis system. In one embodiment, fundamental frequency values are used to form the excitation signal. The excitation is modeled using a voice source pulse selected from a database of a given speaker. The voice source signal is segmented into glottal segments, which are used in vector representation to identify the glottal pulse used for formation of the excitation signal. Use of a novel distance metric and preserving the original signals extracted from the speakers voice samples helps capture low frequency information of the excitation signal. In addition, segment edge artifacts are removed by applying a unique segment joining method to improve the quality of synthetic speech while creating a true representation of the voice quality of a speaker. 
         [0003]    In one embodiment, a method is presented to create a glottal pulse database from a speech signal, comprising the steps of: performing pre-filtering on the speech signal to obtain a pre-filtered signal; analyzing the pre-filtered signal to obtain inverse filtering parameters; performing inverse filtering of the speech signal using the inverse filtering parameters; computing an integrated linear prediction residual signal using the inversely filtered speech signal; identifying glottal segment boundaries in the speech signal; segmenting the integrated linear prediction residual signal into glottal pulses using the identified glottal segment boundaries from the speech signal; performing normalization of the glottal pulses; and forming the glottal pulse database by collecting all normalized glottal pulses obtained for the speech signal. 
         [0004]    In another embodiment, a method is presented to form parametric models, comprising the steps of: computing a glottal pulse distance metric between a number of glottal pulses; clustering the glottal pulse database into a number of clusters to determine centroid glottal pulses; forming a corresponding vector database by associating a vector with each glottal pulse in the glottal pulse database, wherein the centroid glottal pulses and the distance metric is defined mathematically to determine association; determining Eigenvectors of the vector database; and forming parametric models by associating a glottal pulse from the glottal pulse database to each determined Eigenvector. 
         [0005]    In yet another embodiment, a method is presented to synthesize speech using input text, comprising the steps of: a) converting the input text into context dependent phone labels; b) processing the phone labels created in step (a) using trained parametric models to predict fundamental frequency values, duration of the speech synthesized, and spectral features of the phone labels; c) creating an excitation signal using an Eigen glottal pulse and said predicted one or more of: fundamental frequency values, spectral features of phone labels, and duration of the speech synthesized; and d) combining the excitation signal with the spectral features of the phone labels using a filter to create synthetic speech output. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0006]      FIG. 1  is a diagram illustrating an embodiment of an Hidden Markov Model based Text to Speech system. 
           [0007]      FIG. 2  is a diagram illustrating an embodiment of a signal. 
           [0008]      FIG. 3  is a diagram illustrating an embodiment of excitation signal creation. 
           [0009]      FIG. 4  is a diagram illustrating an embodiment of excitation signal creation. 
           [0010]      FIG. 5  is a diagram illustrating an embodiment of overlap boundaries. 
           [0011]      FIG. 6  is a diagram illustrating an embodiment of excitation signal creation. 
           [0012]      FIG. 7  is a diagram illustrating an embodiment of glottal pulse identification. 
           [0013]      FIG. 8  is a diagram illustrating an embodiment of glottal pulse database creation. 
       
    
    
     DETAILED DESCRIPTION 
       [0014]    For the purposes of promoting an understanding of the principles of the invention, reference will now be made to the embodiment illustrated in the drawings and specific language will be used to describe the same. It will nevertheless be understood that no limitation of the scope of the invention is thereby intended. Any alterations and further modifications in the described embodiments, and any further applications of the principles of the invention as described herein are contemplated as would normally occur to one skilled in the art to which the invention relates. 
         [0015]    Excitation is generally assumed to be a quasi-periodic sequence of impulses for voiced regions. Each sequence is separated from the previous sequence by some duration, such as T 0 =1/F 0 , where T 0  represents pitch period and F 0  represents fundamental frequency. The excitation, in unvoiced regions, is modeled as white noise. In voiced regions, the excitation is not actually impulse sequences. The excitation is instead a sequence of voice source pulses which occur due to vibration of the vocal folds. The pulses&#39; shapes may vary depending on various factors such as the speaker, the mood of the speaker, the linguistic context, emotions, etc. 
         [0016]    Source pulses have been treated mathematically as vectors by length normalization (through resampling) and impulse alignment, as described in European Patent EP 2242045 (granted Jun. 27, 2012, inventors Thomas Drugman, et al.) The final length of normalized source pulse signal is resampled to meet the target pitch. The source pulse is not chosen from a database, but obtained over a series of calculations which compromise the pulse characteristics in the frequency domain. In addition, the approximate excitation signal used for creating a pulse database does not capture low frequency source content as there is no pre-filtering done while determining the Linear Prediction (LP) coefficients, which are used for inverse filtering. 
         [0017]    In statistical parametric speech synthesis, speech unit signals are represented by a set of parameters which can be used to synthesize speech. The parameters may be learned by statistical models, such as HMMs, for example. In an embodiment, speech may be represented as a source-filter model, wherein source/excitation is a signal which when passed through an appropriate filter produces a given sound.  FIG. 1  is a diagram illustrating an embodiment of a Hidden Markov Model (HMM) based Text to Speech (TTS) system. An embodiment of an exemplary system may contain two phases, for example, the training phase and the synthesis phase. 
         [0018]    The Speech Database  105  may contain an amount of speech data for use in speech synthesis. During the training phase, a speech signal  106  is converted into parameters. The parameters may be comprised of excitation parameters and spectral parameters. Excitation Parameter Extraction  110  and Spectral Parameter Extraction  115  occurs from the speech signal  106  which travels from the Speech Database  105 . A Hidden Markov Model  120  may be trained using these extracted parameters and the Labels  107  from the Speech Database  105 . Any number of HMM models may result from the training and these context dependent HMMs are stored in a database  125 . 
         [0019]    The synthesis phase begins as the context dependent HMMs  125  are used to generate parameters  140 . The parameter generation  140  may utilize input from a corpus of text  130  from which speech is to be synthesized from. The text  130  may undergo analysis  135  and the extracted labels  136  are used in the generation of parameters  140 . In one embodiment, excitation and spectral parameters may be generated in  140 . 
         [0020]    The excitation parameters may be used to generate the excitation signal  145 , which is input, along with the spectral parameters, into a synthesis filter  150 . Filter parameters are generally Mel frequency cepstral coefficients (MFCC) and are often modeled by a statistical time series by using HMMs. The predicted values of the filter and the fundamental frequency as time series values may be used to synthesize the filter by creating an excitation signal from the fundamental frequency values and the MFCC values used to form the filter. 
         [0021]    Synthesized speech  155  is produced when the excitation signal passes through the filter. The formation of the excitation signal  145  is integral to the quality of the output, or synthesized, speech  155 . Low frequency information of the excitation is not captured. It will thus be appreciated that an approach is needed to capture the low frequency source content of the excitation signal and to improve the quality of synthetic speech. 
         [0022]      FIG. 2  is a graphical illustration of an embodiment of the signal regions of a speech segment, indicated generally at  200 . The signal has been broken down into segments based on fundamental frequency values for categories such as voiced, unvoiced, and pause segments. The vertical axis  205  illustrates fundamental frequency in Hertz (Hz) while the horizontal axis  210  represents the passage of milliseconds (ms). The time series, F 0 ,  215  represents the fundamental frequency. The voiced region,  220  can be seen as a series of peaks and may be referred to as a non-zero segment. The non-zero segments  220  may be concatenated to form an excitation signal for the entire speech, as described in further detail below. The unvoiced region  225  is seen as having no peaks in the graphical illustration  200  and may be referred to as zero segments. The zero segments may represent a pause or an unvoiced segment given by the phone labels. 
         [0023]      FIG. 3  is a diagram illustrating an embodiment of excitation signal creation indicated generally at  300 .  FIG. 3  illustrates the creation of the excitation signal for both unvoiced and pause segments. The fundamental frequency time series values, represented as F 0 , represent signal regions  305  that are broken down into voiced, unvoiced, and pause segments based on the F 0  values. 
         [0024]    An excitation signal  320  is created for unvoiced and pause segments. Where pauses occur, zeros (0) are placed in the excitation signal. In unvoiced regions, white noise of appropriate energy (in one embodiment, this may be determined empirically by listening tests) is used as the excitation signal. 
         [0025]    The signal regions,  305 , along with the Glottal Pulse  310  are used for excitation generation  315  and subsequent generation of the excitation signal  320 . The Glottal Pulse  310  comprises an Eigen glottal pulse that has been identified from the glottal pulse database, the creation of which is described in further detail in  FIG. 8  below. 
         [0026]      FIG. 4  is a diagram illustrating an embodiment of excitation signal creation for a voiced segment, indicated generally at  400 . It is assumed that a Eigen glottal pulse has been identified from the glottal pulse database (described in further detail in  FIG. 7  below). The signal region  405  comprises F 0  values, which may be predicted by models, from the voiced segment. The lengths of the F 0  segments, which may be represented by N f , are used to determine the length of the excitation signal using the mathematical equation: 
         [0000]        F   0 ( n )= f   s   *N   f *5/1000. 
         [0027]    Where f s  represents the sampling frequency of the signal. In a non-limiting example, the value of 5/1000 represents the interval of 5 ms durations that the F 0  values are determined for. It should be noted that any interval of a designated duration of a unit time may be used. Another array, designated as F′ 0 (n), is obtained by linearly interpolating the F 0  array. 
         [0028]    From the F 0  values, glottal boundaries are created,  410 , which mark the pitch boundaries of the excitation signal of the voiced segments in the signal region  405 . The pitch period array may be computed using the following mathematical equation: 
         [0000]    
       
         
           
             
               
                 T 
                 0 
               
                
               
                 ( 
                 n 
                 ) 
               
             
             = 
             
               
                 f 
                 s 
               
               
                 
                   F 
                   0 
                   ′ 
                 
                  
                 
                   ( 
                   n 
                   ) 
                 
               
             
           
         
       
     
         [0029]    Pitch boundaries may then be computed using the determined pitch period array as follows: 
         [0000]        P   0 ( i )=Σ j=0   i    T   0 ( P   0 ( i− 1)
 
         [0030]    Where P 0 (0)=1, i=1, 2, 3, . . . K, and where P(K+1) just crosses length of the array T 0 (n). 
         [0031]    The glottal pulse  415  is used along with the identified glottal boundaries  410  in the overlap adding  420  of a glottal pulse beginning at each glottal boundary. The excitation signal  425  is then created through the process of “stitching”, or segment joining, to avoid boundary effects which are further described in  FIGS. 5 and 6 . 
         [0032]      FIG. 5  is a diagram illustrating an embodiment of overlap boundaries, indicated generally at  500 . The illustration  500  represents a series of glottal pulses  515  and overlapping glottal pulses  520  in the segment. The vertical axis  505  represents the amplitude of excitation. The horizontal axis  510  may represent the frame number. 
         [0033]      FIG. 6  is a diagram illustrating an embodiment of excitation signal creation for a voiced segment, indicated generally at  600 . “Stitching” may be used to form the final excitation signal of voiced segments (from  FIG. 4 ), which is ideally devoid of boundary effects. In an embodiment, any number of different excitation signals may have been formed through the overlap add method illustrated in  FIG. 4  and in the diagram  500  ( FIG. 5 ). The different excitation signals may have a constantly increasing amount of shifts in glottal boundaries  605  and an equal amount of circular left shift  630  for the glottal pulse signal. In one embodiment, if the glottal pulse signal  615  is of a length less than the corresponding pitch period, then the glottal pulse may be zero extended  625  to the length of the pitch period before circular left shifting  630  is performed. Different arrays of pitch boundaries (represented as P m (i), m=1, 2, . . . M−1) are formed with each of the same length as P 0 . The arrays are computed using the following mathematical equation: 
         [0000]        P   m ( i )= P   0 ( i )+ m*w    
         [0034]    Where w is generally taken as 1 msec or, in terms of samples, 
         [0000]    
       
         
           
             
               
                 f 
                 s 
               
               1000 
             
             . 
           
         
       
     
         [0000]    For a sampling frequency of f s =16,000, w=16, for example. The highest pitch period present in the given voice segment is represented as m*w. Glottal pulses are created and associated with each pitch boundary array P m . The glottal pulses  620  may be obtained from the glottal pulse signal of some length N by first zero extending it to the pitch period and then circularly left shifting it by m*w samples. 
         [0035]    For each set of frame boundaries, an excitation signal  635  is formed by initializing the glottal pulses to zero (0). Overlap add  610  is used to add the glottal pulse  620  to the first N samples of the excitation, starting from each pitch boundary value of the array P m (i), i=1, 2, . . . K. The formed signal is as a single stitched excitation, corresponding to the shift, m. 
         [0036]    In an embodiment, the arithmetic mean of all of the single stitched excitation signals is then computed  640 , which represents the final excitation signal for the voiced segment  645 . 
         [0037]      FIG. 7  is a diagram illustrating an embodiment of glottal pulse identification, indicated generally at  700 . In an embodiment, any two given glottal pulses may be used to compute the distance metric/dissimilarity between them. These are taken from the glottal pulse database  840  created in process  800  (further described in  FIG. 8  below). The computation may be performed by decomposing the two given glottal pulses x i , y i  into sub-band components x i   (1) , x i   (2) , x i   (3)  and y i   (1) , y i   (2) , y i   (3) . The given glottal pulse may be transformed into the frequency domain by using a method such as Discrete Cosine Transform (DCT), for example. The frequency band may be split into a number of bands, which are demodulated and converted into time domain. In this example, three bands are used for illustrative purposes. 
         [0038]    The sub-band distance metric is then computed between corresponding sub-band components of each glottal pulses, denoted as d s (x i   (1) , y i   (1) ). The sub-band metric, which may be represented as d s (f, g), where d s  represents the distance between the two sub-band components f and g, may be computed as described in the following paragraphs. 
         [0039]    The normalized circular cross correlation function between f and g is computed. In one embodiment, this may be denoted as R f, g (n)=f★g, where ‘★’ denotes normalized circular cross correlation operation between two signals. The period for circular cross correlation is taken to be the highest of lengths of the two signals f and g. The shorter signal is zero extended. The Discrete Hilbert Transform of normalized circular cross correlation is computed and denoted as R f, g   h (n). Using the normalized circular cross correlation and the Discrete Hilbert Transform of the normalized circular cross correlation, the signal may be determined as: 
         [0000]        H   f, g ( n )=√{square root over ( R   f, g ( n ) 2   +R   f, g   h ( n ) 2 )}{square root over ( R   f, g ( n ) 2   +R   f, g   h ( n ) 2 )}.
 
         [0040]    The cosine of the angle between the two signals f and g may be determined using the mathematical equation: 
         [0000]      cos θ( f, g )=maximum value of the signal  H   f, g ( n ) over all  n.  
 
         [0041]    The sub-band metric, d s (f, g), between the two sub-band components f and g may be determined as: 
         [0000]        d   s ( f, g )=√{square root over (2(1−cos θ( f, g ))}.
 
         [0042]    The distance metric between the glottal pulses is finally determined mathematically as: 
         [0000]    
       
         
           
             
               d 
                
               
                 ( 
                 
                   
                     x 
                     i 
                   
                   , 
                   
                     y 
                     i 
                   
                 
                 ) 
               
             
             = 
             
               
                 
                   
                     d 
                     s 
                     2 
                   
                    
                   
                     ( 
                     
                       
                         x 
                         i 
                         
                           ( 
                           1 
                           ) 
                         
                       
                       , 
                       
                         y 
                         i 
                         
                           ( 
                           1 
                           ) 
                         
                       
                     
                     ) 
                   
                 
                 + 
                 
                   
                     d 
                     s 
                     2 
                   
                    
                   
                     ( 
                     
                       
                         x 
                         i 
                         
                           ( 
                           2 
                           ) 
                         
                       
                       , 
                       
                         y 
                         i 
                         
                           ( 
                           2 
                           ) 
                         
                       
                     
                     ) 
                   
                 
                 + 
                 
                   
                     d 
                     s 
                     2 
                   
                    
                   
                     ( 
                     
                       
                         x 
                         i 
                         
                           ( 
                           3 
                           ) 
                         
                       
                       , 
                       
                         y 
                         i 
                         
                           ( 
                           3 
                           ) 
                         
                       
                     
                     ) 
                   
                 
               
             
           
         
       
     
         [0043]    The glottal pulse database  840  may be clustered into a number of clusters, for example  256  (or M), using a modified k-means algorithm  705 . Instead of using the Euclidean distance metric, the distance metric defined above is used. The centroids of a cluster are then updated with that element of the cluster whose sum of squares of distances from all other elements of that cluster is minimum such that: 
         [0000]        D   m   =Σ   i=1   N   d   2 ( g   i , g m ) is minimum for  m=c,  the cluster centroid. 
         [0044]    In an embodiment, the clustering iterations are terminated when there is no shift in any of the centroids of the k clusters. 
         [0045]    A vector, a set of N real numbers, for example  256 , is associated with every glottal pulse  710  in the glottal pulse database  840  to form a corresponding vector database  715 . In one embodiment, the associating is performed for a given glottal pulse x i , a vector V i =[ψ 1 (x i ), ψ 2 (x i ), ψ 3 (x i ), . . . ψ j (x i )], where ψ j (x i )=d 2 (x i , c j )−d 2 (x i , x 0 )−d 2 (c j , x 0 ) and, x 0  is a fixed glottal pulse picked from the database and d 2 (x i , c j ) represents the square of the distance metric defined above between two glottal pulses x i  and c j  and assuming that c 1 , c 2 , . . . c i , . . . c 256  are the centroid glottal pulses determined by clustering. 
         [0046]    Thus, the vector associated with the given glottal pulse x i  may be computed with the mathematical equation: 
         [0000]        V   i =[ψ 1 ( x   i ), ψ 2 ( x   i ), ψ 3 ( x   i ), . . . ψ j ( x   i ), . . . ψ 256 ( x   i )]
 
         [0047]    In step  720 , Principal Component Analysis (PCA) is performed to compute Eigenvectors of the vector database  715 . In one embodiment, any one Eigenvector may be chosen  725 . The closest matching vector  730  to the chosen Eigenvector from the vector database  715  is then determined in the sense of Euclidean distance. The glottal pulse from the pulse database  840  which corresponds to the closest matching vector  730  is regarded as the resulting Eigen glottal pulse  735  associated with an Eigenvector. 
         [0048]      FIG. 8  is a diagram illustrating an embodiment of glottal pulse database creation indicated generally at  800 . A speech signal,  805 , undergoes pre-filtering, such as pre-emphasis  810 . Linear Prediction (LP) Analysis,  815 , is performed using the pre-filtered signal to obtain the LP coefficients. Thus, low frequency information of the excitation may be captured. Once the coefficients are determined, they are used to inverse filter,  820 , the original speech signal,  805 , which is not pre-filtered, to compute the Integrated Linear Prediction Residual (ILPR) signal  825 . The ILPR signal  825  may be used as an approximation to the excitation signal, or voice source signal. The ILPR signal  825  is segmented  835  into glottal pulses using the glottal segment/cycle boundaries that have been determined from the speech signal  805 . The segmentation  835  may be performed using the Zero Frequency Filtering Technique (ZFF) technique. The resulting glottal pulses may then be energy normalized. All of the glottal pulses for the entire speech training data are combined in order to form the glottal pulse database  840 . 
         [0049]    While the invention has been illustrated and described in detail in the drawings and foregoing description, the same is to be considered as illustrative and not restrictive in character, it being understood that only the preferred embodiment has been shown and described and that all equivalents, changes, and modifications that come within the spirit of the invention as described herein and/or by the following claims are desired to be protected. 
         [0050]    Hence, the proper scope of the present invention should be determined only by the broadest interpretation of the appended claims so as to encompass all such modifications as well as all relationships equivalent to those illustrated in the drawings and described in the specification.