Abstract:
A method of morphing speech from an original speaker into the speech of a second, target speaker with decomposing either speech into source and filter, and without the need to determine the formant positions by warping spectral envelops.

Description:
CROSS REFERENCE TO RELATED APPLICATION 
       [0001]    This invention claims priority to Provisional Patent Application No. 61/557,756 titled Method for First Order Morphing. 
     
    
     STATEMENT OF FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT 
       [0002]    Not Applicable 
       NAMES OF PARTIES TO A JOINT RESEARCH AGREEMENT 
       [0003]    Not Applicable 
       INCORPORATION-BY-REFERENCE OF MATERIAL SUBMITTED ON COMPACT DISC 
       [0004]    Not Applicable 
       BACKGROUND OF THE INVENTION 
       [0005]    1. Field of the Invention 
         [0006]    This invention relates the field of voice morphing. 
         [0007]    2. Description of the Related Art 
         [0008]    Voice morphing is the science of transforming a first person&#39;s voice into a second persona&#39; voice, or a reasonably acceptable approximation. In order to have the first or original speakers speech “sound like” the second or target speakers speech, it is important to mimic the pitch of the second speaker, and to have the spectral energy peaks of the first speaker approximately in the same place that these peaks appear in the spectrum of the second speaker. It is useful to think of speech as a “source”, whether pitch or noise, and a “filter”, typically made up of the resonances associated with the throat, mouth, and noise in a person. (There are alternate definitions of a filter, like those used by a parrot, or electrical filters, often described with poles, or resonances and bandwidths). In general if there is close approximation of the general pitch values and the resonance positions in the spectrum to those of a particular person, then the speech “sounds like” that person. A third variable, speaking rate, also affects how a person sounds. 
         [0009]    Since the early days of speech coders based on LPC (Linear Predictive Coding), speech has been manipulated by changing the pitch of the signal, the “formants” of the signal, or both, made to sound like another speaker. 
         [0010]    All of the modern systems of voice morphing require decomposition of the speech signal into a pitch or “source”, and a spectrum or “filter” portion. This signal processing algorithm is well known to one skilled in the art of speech or voice morphing. 
         [0011]    There are three inter-dependent issues that must be solved before building a voice morphing system. Firstly, it is important to develop a mathematical model to represent the speech signal so that the synthetic speech can be regenerated and prosody, i.e. rhythm, stress, etc. of speech, can be manipulated without artifacts. Secondly, the various acoustic cues which enable humans to identify speakers must be identified and extracted. Thirdly, the type of conversion function and the method of training and applying the conversion function must be decided. 
         [0012]    This decomposition process is error prone, computationally difficult, and the reconstructions are generally only rough approximates of the speech of a particular person. 
         [0013]    Creating an efficient and effective transformation between a first speaker and a second target speaker can be done by measuring the average pitch of each speaker, measuring the “formant positions” of speech of each speaker, and then transforming the speech of the first speaker to match both the average pitch and formant positions of the second speaker 
         [0014]      FIG. 1  is a high level flow diagram of a traditional voice morphing system. 
         [0015]    Referring to  FIG. 1 , At Step  110 , the invention obtains the speech from a first speaker. Similarly, at Step  120 , the invention obtains the speech from a second speaker. The pitch and formants of the first speaker are measured at step  130 , and the formants of the second speaker are measured at step  140 . At step  150  the formants and pitch of the first speaker are transformed to match the formants and pitch of the second speaker 
         [0016]    There are two equivalent processes to accomplish this task, described in  FIGS. 2 and 3 . 
         [0017]    The morphing algorithm requires two parameters for each speaker: the average pitch of each speaker and the formant position warping function to move formants from the first speaker to the second speaker. This can be one of many forms: The average change in the formant frequency to best match each speaker&#39;s formants, the cumulative distribution of each formant for the speech of each talker, or the cumulative distribution of the first three (or four) formants of each speaker over some corpus of speech. 
         [0018]    Note that this process does not describe mimicking the accent of either speaker, nor does it affect other process (like word choice, unusual emphasis, idiosyncratic pronunciations, and others) that can affect the identity of a speaker. We are rather creating a framework onto which these more subtle transformations can be later applied, if required or desired. 
         [0019]    This patent describes a non-decompositional computationally efficient method to implement voice morphing. 
       BRIEF SUMMARY OF THE INVENTION 
       [0020]    The invention herein describes relates to an exemplary method of morphing the speech of one person into the speech of another, i.e. to make one person sound like another. The traditional means include finding the pitch and formants of each speak and performing a match. In this invention, the difficult task of locating formants is avoided. Rather, the spectral envelops are matched and the spectral envelop of the first speakers voice is warped to be statistically similar to the spectral envelop of the second speakers voice. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0021]      FIG. 1  illustrates a high level flow diagram of the state of the art in voice morphing. 
           [0022]      FIG. 2  illustrates a more detailed flow diagram of the state of the art in voice morphing. 
           [0023]      FIG. 3  illustrates changing the pitch of a first speakers voice to match the pitch of a second speakers voice 
           [0024]      FIG. 4   i  illustrates a flow diagram showing matching the formants of a first speaker&#39;s voice to a second speakers voice 
           [0025]      FIG. 5  illustrates a flow diagram of the invention 
           [0026]      FIG. 6  illustrates a spectral representation of the invention 
       
    
    
     DETAILED DESCRIPTION OF THE EMBODIMENTS 
       [0027]    We describe the simplest implementation of voice warping here, and discuss the more sophisticated forms later. 
         [0028]      FIG. 2  illustrates a high level flow diagram of a preferred embodiment of voice warping. At Step  210 , the invention obtains speech from a first and second voice. At Step  220  the pitch of each speaker is measured. Pitch is measured in those voice portions of the speech. The measurement may be done in any number of ways well known to someone skilled in the art. Autocorrelation based pitch measurement, time domain signal matching, cepstral based pitch frequency analysis, combination methods, physical pitch measurements using optical or acoustic signals. However the pitch is measured, the pitch measurements associated with some corpus of each speaker are averaged to create some value. 
         [0029]    The second speaker&#39;s pitch is adjusted to match the first speaker pitch at step  230 . At Step  240  the invention determines how much the second speaker&#39;s formants much be moved to match the formants of the first speaker. The formants of the second speaker&#39;s speech are moved frame by frame to match the function of the first speaker&#39;s formants at Step  250 . At Step  260 , the signal is reconstructed frame by frame. The entire signal is reconstructed at step  270 . 
         [0030]      FIG. 3 , illustrates a flow diagram of matching the pitch of the first speaker to the pitch of the second speaker. The pitch of the first speaker is adjusted to match the pitch of the second speaker using a band-limited resampling algorithm, but without knowing the time value of the pitch at each time. At Step  310 , the invention obtains the speech from a first and second speaker. Each speaker&#39;s speech is sampled at step  320 . At Step  330 , the invention determines the pitch differential between the first speaker&#39;s speech and the second speaker&#39;s speech. The resampling frequency is adjusted so that the average pitch of the first speaker when computed on the resampled signal, but assuming that the sampling rate is that of the second signal from the second speaker, now matches the average pitch of the second speaker.  FIG. 4  illustrates a flow diagram to make the formant locations match. At Step  410  the invention computes the average formant value for the first and second speaker. At Step  420 , the invention computes the amount that the first speakers formants must be moved to match the second speakers formants. This differential is merely the ration of the average values the second speaker&#39;s formant divided by the average value of the first speaker&#39;s formants. At Step  430 , the invention moves the formants.  FIG. 5  illustrates how the formants in the first speakers speech are moved. At Step  510 , the invention windows the speech. At Step  520 , the invention computes the log magnitude spectrum, remembering the phrase at each frequency, at Step  530 , the invention computes the log magnitude cepstrum at each frequency, remembering the phase. At Step  540 , the spectral envelop in frequency space is moved. For each frequency w, we know A(w). For each frequency w, we can find A′(w) (which would have been seen if the first speaker was actually the second speaker) by
       1. find w′=w*the ration of the speakers formants.   2. B(w′)=A′(w)
 
Having computed A′ at each point w, we can compute a gain(w)=A(w)−A′(w).
       
 
         [0033]    At Step  550 , the invention adjusts the spectrum for this frame by the gain at each frequency. This moves the formants (or any other spectral feature) by the ration of the speaker&#39;s formants. At Step  560 , the invention reconstructs the frame of signal by reinserting the phase at each frequency and doing an inverse transform. This can be done in either the log cepstral domain or in the power domain using an appropriate arithmetic operation. At Step  560 , the inventions reconstruct the entire signal using overlap-and-add reconstruction, as is normal in zero-phase filtering operations. 
         [0034]    The remaining detail is the computation of the envelope of a log spectrum of a frame. An example of this computation may be understood by examining  FIG. 6  a as follows: 
         [0035]    In  FIG. 6 , Log Spectrum  610  is the log magnitude spectrum of a frame of speech. The cepstrally smoothed average is line  620 , computed by: 1) Taking the Fourier transform of the Log Spectrum  610 ; Setting all but the 16 lowest frequency cepstral components to zero; taking an inverse Fourier transform of the cepstrum. The number of non-zero cepstral parameters may be chosen but is generally in the range 10 to 30. 
         [0036]    This “cepstrally smoothed” value is used in many other algorithms to represent the spectrum, but it is not what a person hears. Rather, the person hears the energy at the peaks of the spectrum, which we refer to as the “envelope” of the spectrum. The envelop is computed as follows: 1) Compute an auxiliary spectrum consisting of, at each frequency, the maximum of the spectrum and the “cepstrally smoothed” spectrum; Cepstrally smooth that auxiliary spectrum as we did above. 
         [0037]    Finally, compute the envelope as, at each frequency, the value of the smoothed log spectrum plus the difference of the smoothed auxiliary spectrum and the smoothed log spectrum times a constant (empirically determined as 4, but may be between 3 and 4). 
         [0038]    Following this algorithm, it is possible to move pitch and formants independently, simultaneously, and efficiently, changing speaker A to mimic speaker B. However, the pitch change described here changes the length of the speech signal by a proportion that is the proportion of pitch change. This may be ignored, or it may be corrected by using some standard procedures, all of which are well known to someone of ordinary skills in the art.