Abstract:
Speaker normalization is carried out based on biometric information available about a speaker, such as his height, or a dimension of a bodily member or article of clothing. The chosen biometric parameter correlates with the vocal tract length. Speech can be normalized based on the biometric parameter, which thus indirectly normalizes the speech based on the vocal tract length of the speaker. The inventive normalization can be used in model formation, or in actual speech recognition usage, or both. Substantial improvements in accuracy have been noted at little cost. The preferred biometric parameter is height, and the preferred form of scaling is linear scaling with the scale factor proportional to the height of the speaker.

Description:
BACKGROUND OF THE INVENTION  
         [0001]    1. Field of the Invention  
           [0002]    The present invention relates to speech recognition, and more particularly relates to an apparatus and method for speaker normalization based on biometrics.  
           [0003]    2. Brief Description of the Prior Art  
           [0004]    It has been well established in the field of automatic speech recognition that normalizing waveforms to account for the vocal tract differences among speakers yields more accurate results than can be obtained in systems which do not include such normalization. If an open vocal tract model is assumed, such as would be appropriate for an open vowel (for example /UH/), a uniform tube model provides a good approximation to the vocal tract, as discussed by Lawrence W. Rabiner and Ronald W. Schafer in the text  Digital Processing of Speech Signals  published by Prentice-Hall, Inc., Englewood Cliffs, N.J. 07632 in 1978.  
           [0005]    In the uniform tube model, when one scales the tube by a factor 1/k, this results in a scaling of all of the resonances of the tube by k and, therefore, a linear scaling of the frequency axis is appropriate. In practice, linear scaling has been shown to be effective in normalizing for differences in vocal tract length. In implementation, once a form of frequency scaling (for example, linear scaling, f*=kf) has been chosen, the remaining question is how to determine a scale factor k i  for each speaker i.  
           [0006]    It has been known in the prior art to derive an estimated scale factor based on formant positions, as set forth in the paper “A Parametric Approach to Vocal Tract Length Normalization” by Ellen Eide and Herbert Gish as published by the IEEE in the Proceedings of the ICASSP of 1996, at pages 346-48.  
           [0007]    Other results have been published for general speech corpora based on exhaustive search, for example, refer to “Speaker Normalization Using Efficient Frequency Warping Procedures” by Li Lee and Richard C. Rose, as published at pages 353-56 of the aforementioned 1996 ICASSP Proceedings, and “Speaker Normalization on Conversational Telephone Speech” by Steven Wegmann et al., as published at pages 339-41 of the aforementioned proceedings.  
           [0008]    One case of interest is the situation where a database is available which contains biometric information in the form of a biometric parameter (such as speaker height) which would permit the normalization factor for each speaker to be computed by taking the ratio of the value of the speaker&#39;s biometric parameters to some measure of an average value of the biometric parameter, such as the average across all speakers in the training database.  
           [0009]    In view of the foregoing, there is a need in the prior art for a speaker normalization apparatus and method which are based on biometrics pertaining to the speaker.  
         SUMMARY OF THE INVENTION  
         [0010]    The present invention, which addresses the needs identified in the prior art, provides a method of speaker normalization. The method includes the steps of receiving a first biometric parameter, calculating a first frequency scaling factor based on the first biometric parameter, and extracting acoustic features from speech of a user in accordance with the first frequency scaling factor. The first biometric parameter is correlated to vocal tract length of a given user of a speech recognition system.  
           [0011]    The present invention further provides an apparatus for speaker normalization, which includes a biometric parameter module, a calculation module, and an acoustic feature extractor. The biometric parameter module receives the first biometric parameter which is correlated to the vocal tract length of the user. The calculation module calculates the first frequency scaling factor based on the first biometric parameter. The acoustic feature extractor extracts acoustic features from speech of the user in accordance with the first frequency scaling factor.  
           [0012]    The present invention can be implemented in hardware, software, or a combination of hardware and software, and accordingly also encompasses a program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for speaker normalization as set forth herein.  
           [0013]    Accordingly, it will be appreciated that the method and apparatus of the present invention provide an improvement over prior-art approaches, inasmuch as an appropriate scaling factor can be readily determined, so as to improve the accuracy of an associated speech recognition system, based on biometric data pertaining to users of the system, which may be, for example, pre-stored, or which may be ascertained during an interaction with the speaker. 
       
    
    
       [0014]    These and other features and advantages of the present invention will be appreciated by reading the following specification, taking in conjunction with the accompanying drawings, and the scope of the invention will be set forth in the appended claims.  
       BRIEF DESCRIPTION OF THE DRAWINGS  
       [0015]    [0015]FIG. 1 is a flow chart illustrating one form of a method for speaker normalization based on biometrics in accordance with the present invention;  
         [0016]    [0016]FIG. 2 is a flow chart showing one method in which acoustic features may be extracted in accordance with the present invention; and  
         [0017]    [0017]FIG. 3 is a schematic diagram of an apparatus for speaker normalization based on biometrics in accordance with the present invention. 
     
    
     DETAILED DESCRIPTION OF THE INVENTION  
       [0018]    Reference should now be had to the flow chart of FIG. 1. A method of speaker normalization, in accordance with the present invention, includes the step of receiving a first biometric parameter, as suggested at block  104 . The first biometric parameter can be designated as γ i . The first biometric parameter can be correlated to the vocal tract length of an i th  user of a speech recognition system. As suggested at block  106 , a first frequency scaling factor, which may be designated k i , can be calculated based on the first biometric parameter. Further, as suggested at block  108 , acoustic features can be extracted from the speech of the i th  user in accordance with the first frequency scaling factor.  
         [0019]    The first biometric parameter can be, for example, height, a spatial dimension of a preselected bodily member (such as an arm or a leg), or the size of an article of clothing which correlates with the spatial dimension. For example, the spatial dimension can include length, width, girth and the like, and the size could be sleeve length, inseam, shoe size and the like. At present, it is believed that height is the preferred biometric parameter to employ, but is to be emphasized that all other suitable biometric parameters are within the scope of the present invention.  
         [0020]    In one aspect of the invention, after starting the process, as suggested by block  100 , interaction with the user can be carried out in accordance with block  102 . In certain embodiments of the invention, the interaction with the user could include querying the user for a value indicative of the first biometric parameter. In this case, the step of receiving the first biometric parameter could include receiving the parameter from the user in response to the querying. The value indicative of the first biometric parameter should be broadly understood to include an actual value of the parameter, such as height, or a numerical size of an article of clothing, but could also include a more approximate estimate of the user&#39;s measurements, such as “tall,” “average” or “short”; “small,” “medium” or “large”; and the like.  
         [0021]    In other aspects of the invention, the first biometric parameter can be sensed by automated observation of the user; by way of example, the user&#39;s height could be determined by a system of electric eyes or other sensors.  
         [0022]    In addition to the aforementioned interaction with the user in the form of querying the user or automatically sensing the biometric parameter, other forms of interaction with the user can be carried out. For example, the user can be identified in any appropriate fashion and the step of receiving the first biometric parameter can be carried out by extracting the first biometric parameter from a database of users based on the identify of the user. The identification can be preformed in any suitable manner, for example, by determining the user&#39;s name, identification number, etc. In one embodiment of the invention, the identification can be performed using speaker identification, through any of a number of speaker identification routines which are well known in the art.  
         [0023]    The step of extracting the acoustic features is preferably carried out by frequency axis frequency scaling in accordance with the first frequency scaling factor k i . Although all forms of frequency scaling are contemplated as being within the scope of the present invention, it is presently believed that the preferred form of frequency scaling is linear frequency scaling, in accordance with the equation: 
           f*=k   i   f   (1) 
         [0024]    wherein:  
         [0025]    f* is a scaled frequency, and  
         [0026]    f is an original frequency.  
         [0027]    The first frequency scaling factor can advantageously be determined in accordance with the equation: 
           k   i =γ i /γ av   (2) 
         [0028]    wherein γ av  is an average value of the first biometric parameter for a representative population of users. For example, it could be the average height over a representative population of users.  
         [0029]    Thus, it will be appreciated that the first barometric parameter γ i , can be the height, h i , of the user, and that the first frequency scaling factor, in this case, can be determined in accordance with  
           k   i   =h   i   /h   av ,  (3) 
         [0030]    where  h   av  is an average height for the representative population of the users. The frequency scaling is preferably achieved by digital signal processing of a signal representative of the speech of the user; however, other suitable techniques of frequency scaling are contemplated as being within the scope of the present invention.  
         [0031]    In one application of the present invention, the user can be a training speaker, and the step of extracting the acoustic features can further include the sub-step of formatting the acoustic features for subsequent acoustic model building. Construction of acoustic models is well known in the art.  
         [0032]    In another aspect of the present invention, the user can be a “testing” speaker, or an actual user of a system, in which case the step of extracting the acoustic features can include the sub-step of formatting the acoustic features for subsequent speech recognition. Thus, it will be appreciated that the “stop” block  112  in FIG. 1 merely represents the optional end of the normalization process, but the extracted acoustic features can be utilized for model building, speech recognition, or any other desired purpose following the normalization.  
         [0033]    Attention should now be given to FIG. 2, which presents a flow chart illustrating one optional manner in which the extraction of acoustic features can be accomplished in block  108 . For convenience, sub-steps depicted in FIG. 2 have received a four digit reference character commencing with the digits  108  and having digits  1 - 6  appended thereto, that is, numbers  1081 - 1086  are employed. As shown at block  1081 , a signal representative of the speech of the user can be transformed from a time domain to a frequency domain, in order to obtain a resulting frequency spectrum having a frequency axis. Any suitable type of transform can be employed, but at present, a fast Fourier transform (FFT) is believed to be preferable. In block  1082 , the log-magnitude of the resultant frequency spectrum can be calculated. Then, in block  1083 , the frequency axis can be scaled according to the first frequency scaling factor k i .  
         [0034]    In block  1084 , the frequency axis can be partitioned into bins, while in block  1085  the spectral components falling into each of the bins can be summed so as to obtain a binned spectrum. Then, in block  1086 , the binned spectrum can be inverse-transformed to obtain time-domain acoustic features for each of a series of time windows.  
         [0035]    The scaling sub-step can include any suitable form of scaling, but preferably includes linear scaling in accordance with equation 1 above.  
         [0036]    Reference should now again be had to FIG. 1. As suggested at block  110 , the speaker normalization process can be repeated as needed for other users of the speech recognition system. Thus, for example, a second biometric parameter, designated as ζ j , which is correlated to the vocal tract length of a j th  user of the speech recognition system, can be received. Further, a second frequency scaling factor k j  can be calculated based on the second biometric parameter, and acoustic features from the speech of the j th  user can be extracted in accordance with the second frequency scaling factor.  
         [0037]    In such case, before any of the biometric parameters are received at block  104 , interaction can be carried out with the user in accordance with block  102 , such that the i th  user can be queried regarding which of N biometric parameters the i th  user wishes to employ as the first biometric parameter, while the j th  user can be queried regarding which of the N biometric parameters the j th  user wishes is to employ as the second biometric parameter. It should be appreciated that such interaction would be carried out when N is greater than or equal to 2. It should further be understood that the first and second biometric parameters could be the same biometric parameter, or different users could employ different biometric parameters such that the first and second biometric parameters were different biometric parameters.  
         [0038]    In view of the foregoing, it will be appreciated that, in a method in according to the present invention, scaling of the frequency axis for each speaker can preferably be performed in accordance with Equation 1 so as to account for the vocal tract length of the i th  speaker. The present invention seeks to estimate the scaling factor k i  for the i th  speaker. Such scaling factor is preferably determined in accordance with Equation 2, and most preferably is determined in accordance with Equation 3, where, as noted, h av  can be taken as the average height across speakers in the training database. The normalization computation permits enhanced accuracy in a speech recognition system with very little cost.  
         [0039]    As noted, while speaker height is the preferred biometric parameter, any biometric data which is correlated with the vocal tract length can be employed. Further, and again as noted, the normalization can be performed using any appropriate scaling function of the frequency axis, not merely the linear function of the preferred embodiment.  
         [0040]    As discussed above, scaling can be employed on training speakers, on testing speakers (i.e., system users), or on both. If the normalization is performed during both training and test (i.e. use), any appropriate value can be used for γ av  or h av . However, when the normalization is done only during test, i.e. use, the value of γ av  should be the average value of the appropriate biometric parameter for the representative population of users, and the value h av  should be an estimate of the average speaker height.  
         [0041]    It will be appreciated by those of skill in the art that the method and apparatus of the present invention are intended for use with speaker-independent speech recognition. When the normalization method and apparatus of the present invention are used during training, the speakers who are donating the data can be asked to provide appropriate biometric data. The frequency axis for each of the donating speakers can then be scaled according to the corresponding biometric parameter, and the acoustic model can subsequently be built. The resultant model can be used in testing (i.e. actual use), both in the case where scaling is employed on the actual user, and the case where it is not. In experiments, it has been found that the effects are substantially additive. That is to say, normalizing the training speakers but not the test (use) speakers gives a gain which can be designated as G. Normalizing the test (use) speakers but not the training speakers gives a gain which can be designated as K. Normalizing both groups of speakers gives a gain in recognition accuracy which is close to the value of G+K. In testing the present invention, a value of h av =68 inches was employed, as this is the average height of American adults.  
         [0042]    It will be appreciated that in an ordinary speech recognition system, without normalization, acoustic features are simply extracted from the user&#39;s speech and are subsequently decoded to determine a word hypothesis. In the present invention, an appropriate biometric parameter is determined for the user, a scale factor is calculated, and feature extraction occurs including frequency scaling based on the calculated scale factor, all as previously discussed and illustrated with respect to FIG. 1.  
         [0043]    Referring again to FIG. 2, it will be appreciated that, except for the step of scaling the frequency axis in block  1083 , the depicted sub-steps are similar to those used for feature extraction without frequency scaling.  
         [0044]    As noted, the transformation of the speech signal from the time domain to the frequency domain is preferably carried out by computing fast Fourier transforms. In such case, block  1086  would involve the computation of appropriate inverse fast Fourier transforms. It should be appreciated that decoding of the acoustic features produced by the present invention can proceed just as in a system where no normalization is performed, however, the features which are employed are those which result from the above-described scaling of the frequency axis.  
         [0045]    Attention should now be given to FIG. 3 which depicts an apparatus for speaker normalization in accordance with the present invention. The apparatus is designated generally as  118 . The apparatus includes a biometric parameter module  120  which receives the first biometric parameter, γ i , as discussed above. The apparatus further includes a calculation module  122  which calculates the first frequency scaling factor, k i , as discussed above. Further, the apparatus includes an acoustic feature extractor  124  which extracts acoustic features from speech of the i th  user in accordance with the first frequency scaling factor. As set forth above, height is the preferred choice for the first biometric parameter, but any of the other parameters discussed above can be employed.  
         [0046]    In some embodiments of apparatus  118 , a user interface module  126  can be provided in order to interface with a user  132 . For example, the user interface module  126  could query the user  132  for a value indicative of the first biometric parameter, such that the biometric parameter module  120  could receive the first biometric parameter from the user  132 , via the user interface module  126 , in response to the querying.  
         [0047]    In another aspect of the invention, user interface module  126  could sense the first biometric parameter by automated observation of the user, as discussed above with respect to the method of the present invention.  
         [0048]    In other aspects of the present invention, the apparatus  118  can optionally include a database of users  128  which contains biometric parameters of given users stored with identities of the given users. “Identities” in this context should be broadly understood to refer to any appropriate data which permits the matching of the stored biometric parameters with the users. In the case when database  128  is employed, the user interface module  126  can be configured to perform identification on the user  132  so as to determine the identity of the user  132 , and the user interface module  126  can provide appropriate control to extract the first biometric parameter from the database of users  128  based on the identity of the user  132 .  
         [0049]    In still another aspect of the invention, the apparatus  118  can optionally include a speaker identification module  130  which is coupled to the user interface module  126 . The interface module  126  can communicate with the speaker identification module  130  so as to perform identification of the user  132  via speaker identification. As noted above, any number of well-known routines are available for speaker identification.  
         [0050]    The acoustic feature extractor  124  can perform frequency axis frequency scaling in accordance with the first frequency scaling factor. Such scaling preferably includes linear frequency scaling in accordance with Equation 1 above. The first frequency scaling factor can be determined in accordance with Equation 2 above, and preferably in accordance with Equation 3 above. The feature extractor  124  can preferably perform the frequency scaling via digital signal processing of a signal representative of the speech of the user.  
         [0051]    The apparatus  118  can be employed for both training speakers and testing speakers (i.e., actual users). When employed with training speakers, the acoustic feature extractor  124  can format the acoustic features for subsequent acoustic model building, as discussed above. When the user  132  is a testing speaker (i.e., actual user of a speech recognition system), the acoustic feature extractor  124  can format the acoustic features for subsequent speech recognition, as discussed above.  
         [0052]    The acoustic feature extractor  124  can be appropriately configured to perform the method steps depicted in FIG. 2 and discussed above.  
         [0053]    It will be appreciated that the biometric parameter module  120  can be configured to receive a second biometric parameter, ζ j , correlated to vocal tract length of a j th  user of the speech recognition system, while the calculation module  122  can be configured to calculate a second frequency scaling factor, k j , based on the second biometric parameter. Further, the acoustic feature extractor  124  can be configured to extract acoustic features from speech of the j th  user in accordance with the second frequency scaling factor. In this case, the user interface module  126  can be configured to query the i th  and j th  users, as discussed above with regard to the method, as to which of N biometric parameters each wishes to employ as the corresponding biometric parameter. Again, the first and second biometric parameters can be the same biometric parameter or can be different biometric parameters.  
         [0054]    The methods of the present invention can be carried out using any suitable apparatus, and the apparatus of the present invention can be implemented in any of a number of ways. At present, it is believed preferable that the present invention should be implemented in software by programing a suitable general purpose computer, such as a personal computer, an enterprise server, or the like. In one preferred embodiment of the invention, the modules  120 ,  122 ,  130  can be implemented as software programs, as can the acoustic feature extractor  124 . The user interface module  126  can be implemented as an appropriate software program which interfaces with appropriate hardware devices in order to communicate with user  132 . For example, module  126  could interface with a microphone and speaker, a keyboard, a mouse, a display, a track ball, a joy stick, or any other suitable device. Further, user interface module  126  could interface with appropriate sensors, such as electric eyes (e.g., a plurality of photodiode and beam arrangements) or the like designed to sense the height of user  132 , or sensors designed to sense any other detectable biometric parameter. The database  128  can reside on a suitable mass storage device such as a hard disk, floppy disk, CD-ROM, tape or the like, and can be loaded into RAM as desired. Any suitable form of computer storage can be employed. In addition to the preferred implementation on a digital computer, as appropriate, portions of the apparatus can be implemented using analog computation devices, or employing various types of hardware, such as, for example, application specific integrated circuits (ASICs), circuits formed using discrete components, or hybrid circuits. In the preferred digital computer-implemented embodiment, any appropriate type of processor can be employed. As noted, this could be an enterprise server or personal computer. Further, any other type of processor-based system could be employed, and still further, processing could be carried out in a distributed fashion by more than one processor.  
         [0055]    The interconnections among the various components  120 ,  122 ,  124 ,  126 ,  128 ,  130  are illustrative in nature, and other appropriate interconnections are within the scope of the present invention. Further, it should be appreciated that the various elements can be implemented as separate software modules, or as part of one large code.  
         [0056]    The user interface module  126  can provide control to extract the first biometric parameter from the database  128 . Such control can be provided directly to database  128 , through biometric parameter module  120 , or in any other desired fashion.  
         [0057]    In view of the foregoing, it will be appreciated that the present invention contemplates a program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for speaker normalization in accordance with the present invention. The program storage device may include a program of instructions to carry out any or all of the method steps set forth herein, or any appropriate subset thereof  
         [0058]    While there have been described what are presently believed to be the preferred embodiments of the invention, those skilled in the art will realize that various changes and modifications can be made to the invention without departing from the spirit of the invention, and it is intended to claim all such changes and modifications as fall within the scope of the invention.