Source: http://www.google.com/patents/US20030009333?dq=6377161
Timestamp: 2017-11-18 10:13:53
Document Index: 786929056

Matched Legal Cases: ['art 2', 'application no. 08', 'Application no. 60', 'application no. 08', 'application no. 60', 'application no. 08', 'application no. 60']

Patent US20030009333 - Voice print system and method - Google Patents
The voice print system of the present invention is a subword-based, text-dependent automatic speaker verification system that embodies the capability of user-selectable passwords with no constraints on the choice of vocabulary words or the language. Automatic blind speech segmentation allows speech to...http://www.google.com/patents/US20030009333?utm_source=gb-gplus-sharePatent US20030009333 - Voice print system and method
Publication number US20030009333 A1
Also published as EP0954854A1, EP0954854A4, US6539352, US6760701, WO1998022936A1
Publication number 042832, 10042832, US 2003/0009333 A1, US 2003/009333 A1, US 20030009333 A1, US 20030009333A1, US 2003009333 A1, US 2003009333A1, US-A1-20030009333, US-A1-2003009333, US2003/0009333A1, US2003/009333A1, US20030009333 A1, US20030009333A1, US2003009333 A1, US2003009333A1
Inventors Manish Sharma, Xiaoyu Zhang, Richard Mammone
Referenced by (84), Classifications (23), Legal Events (5)
Voice print system and method
US 20030009333 A1
at least one classifier, connected to the segmenting means, wherein the classifier models the pluraility of subwords and outputs one or more classifier scores.
11. An automatic speaker verification method, comprising the steps of:
segmenting the enrollment speech, wherein the enrollment speech is segmented into a plurality of subwords; and
modelling the pluraility of subwords using one or more classifier models resulting in an output of one of more classifier scores.
12. The automatic speaker verification method of claim 11, further comprising the steps of:
14. The automatic speaker verification method of claim 11, wherein the step of modeling further comprises the steps of:
15. The automatic speaker verification method of claim 11, further comprising the steps of:
19. An automatic speaker verification method, comprising the steps of:
storing an estimate of the enrollment channel, the estimate being a filter representing characteristics of the enrollment channel;
inverse filtering the test speech to create filtered test speech;
recalling the estimate of the enrollment channel
filtering the filtered test speech through the recalled estimate of the enrollment channel to create enrollment filtered test speech; and
determining whether the enrollment filtered test speech comes from the same person as the enrollment speech.
22. An automatic speaker verification method, comprising the steps of:
25. An automatic speaker verification method, including the steps of:
30. An automatic speaker verification method, wherein the results of prior verifications are stored, including the steps of:
fusing the results of each classifier score using a fusion constant and weighing function to generate a final score; and
comparing final score to a threshold value to determine whether the test speech and enrollment speech are from the known individual.
31. The automatic speaker verification method of claim 30, further comprising the step of:
32. The automatic speaker verification method of claim 30, further comprising the steps of:
33. The automatic speaker verification method of claim 30, further comprising the steps of:
34. An automatic speaker verification method, comprising the steps of:
35. The automatic speaker verification method of claim 34, wherein the known speech is obtained over an enrollment channel, wherein the step of processing further comprises the step of filtering the test speech through a filter having characteristics of the enrollment channel, and wherein the step of generating subwords further comprises the step of spotting one or more key words/key phrases in the processed test speech.
36. The automatic speaker verification method of claim 34, further comprising the steps of:
Feature extraction is then performed to extract features of the user's voice, such as pitch, spectral frequencies, intonations, etc. . . Feature extraction may also focus, or capture, desired segments of the voice sample and reject other unwanted segments. The feature extraction process generates a number of vectors relating to features of the voice segment. Using the feature vectors, a key word/key phrase reference template may be generated and stored in a voice print database. The reference template is used during testing to locate the spoken password from extraneous speech or noise.
[0050]FIG. 1A is a diagram of a enrollment component of the present invention.
[0051]FIG. 1B shows pseudo-code for creating a filter to perform the channel estimation shown in FIG. 1A.
[0052]FIG. 1C shows pseudo-code for inverting the filter of FIG. 1B.
[0053]FIG. 2 is a diagram of a testing component of the present invention.
[0054]FIGS. 3A and 3B are flow diagrams of a channel adaptation module, shown in FIG. 2, of the present invention.
[0055]FIG. 4 is a flow diagram of a key word/key phrase spotter, shown in FIG. 2, of the present invention.
[0056]FIG. 5 is a diagram of an utterance representation in the feature vector domain.
[0057]FIG. 6 is a diagram of dynamic time warping used to obtain a reference template in the key word/key phrase spotter of FIG. 4.
[0058]FIG. 7 is a diagram of dynamic time warping distortion, used to match the reference template of FIG. 6 to test speech.
[0059]FIG. 8 is a flow diagram of a fusion adaptation module shown in FIG. 2.
[0060]FIG. 9 is a flow diagram of a threshold adaptation module shown in FIG. 2.
[0061]FIG. 10 is a diagram of a model adaptation module used in the system of FIGS. 1 and 2.
[0062]FIG. 11 is a diagram of a bootstrapping component, used to generate antispeaker data in the system of FIG. 1A.
1. Enrollment Component—Detailed Description.
[0066]FIG. 1A shows the enrollment component 10. As shown, the first step 20 is to obtain enrollment speech (the password) and to obtain 26 an index , such as the user's name or credit card number. The enrollment speech may be obtained via a receiver, telephone or other sources, and be received from any transmission media, digital or analog, including terrestrial links, land lines, satellite, microwave, etc... More than one sample of enrollment speech should be supplied, each of which is used to generate multiple data sets. Preferably, four enrollment samples are supplied and processed.
F −1 log(|Ŝ(ω)|)=F −1 log(|S(ω)+F −1 log(|X(ω)|)
The subword data from the speaker being trained is labeled as enrollment speaker data. Because there is a no linguistic labelling information in the antispeaker database 110, the entire database 110 is searched for the closet subword data from other speakers. This data is labeled the anti-speaker data. The mean vector and covariance matrix of the subword segments obtained from subword generation are used to find the “close” subwords. An anti-speaker module 120 searches the antispeaker database 110 to find the “close” subwords of antispeaker data, which are used in the NTN model 20 . Preferably, 20 “close” subwords are identified. The anti-speaker data in the antispeaker database 110 is either manually created, or created using a “bootstrapping” component, described below with reference to Figure Because a “leave-one-out” system 100 is employed with multiple (N) samples, the classifier models 80, 90 are trained by comparing antispeaker data with N-1 samples of enrollment speech. Both modules 80, 90 can determine a score for each spectral feature vector of a subword segment. The individual scores of the NTN 80 and GMM 90 modules can be combined, or “fused” by a classifier fusion module 130 to obtain a composite score for the subword. Since these two modeling approaches tend to have errors that are uncorrelated, it has been found that performance improvements can be obtained by fusing the model outputs 130. In the preferred embodiment, the results of the neural tree network 80 and the Gaussian mixture model 90 are fused 130 using a linear opinion pool, as described below. However, other ways of combining the data can be used with the present invention including a log opinion pool or a “voting” mechanism, wherein hard decisions from both the NTN and GMM are considered in the voting process.
With continued reference to FIG. 1A, one of modules used to model the subword segments of the user password is an NTN module 80. The NTN is a hierarchical classifier that uses a tree architecture to implement a sequential linear decision strategy. Specifically, the training data for a NTN consists of data from a target speaker, labeled as one, along with data from other speakers (antispeaker data) that are labeled as zero. The NTN learns to distinguish regions of feature space that belong to the target speaker from those that are more likely to belong to an impostor. These regions of feature space correspond to leaves in the NTN that contain probabilities. These probabilities represent the likelihood of the target speaker having generated data that falls within that region of feature space. In the preferred embodiment, NTN modeling 220 is performed using the following forward pruning criteria: (a) maximum depth of four, (b) pruned nodes containing less than 10% of data vectors at the root. The NTN scores for individual feature vectors are accumulated across subwords by an NTN scoring algorithm 145. The functioning of NTN networks with respect to speaker recognition is disclosed in K. R. Farrell, R. J. Mammone, and K. T. Assaleh, “Speaker Recognition using Neural Networks and Conventional Classifiers”, IEEE Trans. Speech and Audio Processing, 2(1), part 2 (1994), and U.S. patent application 08/159,397, filed Nov. 29, 1993, entitled “Rapidly Trainable Neural Tree Network”, U.S. patent application Ser. No. 08/479,012 entitled “Speaker Verification System,” U.S. patent application no. 08/827,562 entitled “Blind Clustering of Data With Application to Speech Processing Systems”, filed on Apr. 1, 1997, and its corresponding U.S. Provisional Application no. 60/014,537 entitled “Blind Speech Segmentation”, filed on Apr. 2, 1996, each of which is incorporated herein by reference in its entirety.
As discussed previously, a Gaussian mixture model (GMM) 90 is also used to model each of the subwords. In the GMM, a region of feature space for a target speaker is represented by a set of multivariate Gaussian distributions. In the preferred embodiment, the mean vector and covariance matrix of the subword segments are obtained as a by-product of subword generation using automatic blind speech segmentation and are saved as part of the GMM module, as described in U.S. patent application no. 08/827,562 entitled “Blind Clustering of Data With Application to Speech Processing Systems”, filed on Apr. 1, 1997, and its corresponding U.S. provisional application no. 60/014,537 entitled “Blind Speech Segmentation”, filed on Apr. 2, 1996, both of which are herein incorporated by reference. The GMM probability distribution function is expressed as: p  ( x ) = ∑ i = 1 G   P  ( ω i )  p  ( x / μ i , O 2 ) .
Each of the G mixture components is defined by a mixture weight P(ωi) and multi-dimensional normal distribution function p(x/μi, σι2), where μi is the mean vector and σιis the covarience matrix. In the preferred embodiment, the normal distribution is constrained to have a diagonal covariance matrix defined by the vector σι. The PDF is used to produce the sub-word GMM score.
The data fusion function for n classifiers, S(x), is governed by the following linear opinion pool equation: S  ( α ) = ∑ i = 1 n   α i  s i
In this equation S(α) is the probability of the combined system, αi are weights, and si(α) is the probability output by the ith classifier, and n is the number of classifiers; αi is between zero and one and the sum of all α's is equal to one. If two classifiers are used (n=2), si is the score of the first classifier and s2 is the score of the second classifier. In this instance the equation becomes:
The variable α is set as a constant (although it may be dynamically adapted as discussed below), and functions to provide more influence on one classifier method as opposed to the other. For example, if the NTN method 80 was found to be more accurate, the first classifier s1 would be more important, and a would be made greater than 0.5, or its previous value. Preferably, α is only incremented or decremented by a small amount, ε.
[0110]FIG. 2 shows a general outline of the testing component 150, which has many features similar to those described with respect to the enrollment component 10 of FIG. 1A. The testing component 150 is used to determine whether test speech received from a user sufficiently matches identified stored speech characteristics so as to validate that the user is in fact the person whose speech was stored.
The total distance function D is of the form: D = ∑ n = 1 N   d  ( R  ( n ) , T  ( w  ( n ) ) ) .
The reference utterance 350 could be chosen in a number of ways. The preferred method is to select the enrollment utterance with the minimum duration.
selection=arg min {Li} i=1, 2, 3, 4 . . . where Li is the utterance length i of utterances i=1, 2,3,4.
selection=arg min/1 |Lavg −L i |i=1,2,3,4
The utterance selected is the one with minimum di.
Selection=arg min/1 di
After selecting 300 a reference utterance 350, all the remaining utterances are “warped” 310 onto the reference utterance 350 using the DTW algorithm. FIG. 6 shows four utterances 370, 380, 390 and 400 of different lengths (L1, L3, L4) “warped” onto the reference utterance 350, which produces four “warped” utterances 410, 420, 430 and 440 of length Lref. The four warped utterances 410, 420, 430 and 440 are averaged 320 to form a reference template 450 of length Lref.
After subword generation 210 is performed, scoring 240, 250 using the techniques previously described with respect to FIG. 2 (i.e. multiple classifiers such as GMM 230 and NTN 220) is performed on the subwords. Scoring using the NTN and GM classifiers 220, 230 is disclosed in U.S. patent application Ser. No. _________, entitled “Model Adaption System And Method For Speaker Verification,” filed on Nov. 3, 1997 by Kevin Farrell and William Mistretta, U.S. patent application no. 08/827,562 entitled “Blind Clustering of Data With Application to Speech Processing Systems”, filed on Apr. 1, 1997, and its corresponding U.S. provisional application no. 60/014,537 entitled “Blind Speech Segmentation”, filed on Apr. 2, 1996, each of which is herein incorporated by reference in its entirety.
As shown in FIG. 2, a fusion adaptation module 290 is connected to the classifier fusion module 280. The fusion adaptation module 290 changes the constant, a, in the linear pool data fusion function described previously with respect to FIG. 2, which is: S  ( α ) = ∑ i = 1 n   α i  s i
S=αs1+(1−α)s2
Model adaptation 540 may also occur as described in copending Provisional Application Serial No. ______, entitled “Model Adaption System And Method For Speaker Verification,” filed on Nov. 3, 1997 by Kevin Farrell and William Mistretta.
[0172]FIG. 11 shows a bootstrapping component 700. The bootstrapping component 700 first obtains antispeaker speech 710, and then preprocess the speech 720 as previously described with respect to FIG. 1A. The antispeaker speech may be phrases from any number of speakers who will not be registered in the database as users. Next, the antispeaker speech is inverse-channel filtered 730 to remove the effects of the antispeaker channel as described with respect to FIGS. 1 and 2. As shown in FIG. 11, the processed and filtered antispeaker speech then undergoes feature extraction 770. The feature extraction may occur as previously described with respect to FIG. 1A. Next, the antispeaker speech undergoes sub-word generation 750, using the techniques previously described with respect to FIG. 1A. The preferable method of sub-word generation is automatic blind speech segmentation, discussed previously with respect to FIG. 1A. The sub-words are then registered as antispeaker data 760 in the database.
US7970611 May 2, 2006 Jun 28, 2011 Voice.Trust Ag Speaker authentication in digital communication networks
US8370925 Jul 29, 2008 Feb 5, 2013 International Business Machines Corporation User policy manageable strength-based password aging
US8775178 * Oct 27, 2009 Jul 8, 2014 International Business Machines Corporation Updating a voice template
US9646605 * Jan 22, 2013 May 9, 2017 Interactive Intelligence Group, Inc. False alarm reduction in speech recognition systems using contextual information
US9672815 * Jul 20, 2012 Jun 6, 2017 Interactive Intelligence Group, Inc. Method and system for real-time keyword spotting for speech analytics
US9711148 * Jul 18, 2013 Jul 18, 2017 Google Inc. Dual model speaker identification
US20090171660 * Dec 18, 2008 Jul 2, 2009 Kabushiki Kaisha Toshiba Method and apparatus for verification of speaker authentification and system for speaker authentication
US20100106501 * Oct 27, 2009 Apr 29, 2010 International Business Machines Corporation Updating a Voice Template
US20140025379 * Jul 20, 2012 Jan 23, 2014 Interactive Intelligence, Inc. Method and System for Real-Time Keyword Spotting for Speech Analytics
US20140207457 * Jan 22, 2013 Jul 24, 2014 Interactive Intelligence, Inc. False alarm reduction in speech recognition systems using contextual information
US20150039304 * Aug 1, 2014 Feb 5, 2015 Verint Systems Ltd. Voice Activity Detection Using A Soft Decision Mechanism
EP1843325A1 * Apr 3, 2006 Oct 10, 2007 Voice.Trust Ag Speaker authentication in digital communication networks
WO2017008075A1 * Jul 11, 2016 Jan 12, 2017 Board Of Regents, The University Of Texas System Systems and methods for human speech training
U.S. Classification 704/246, 704/E17.006, 704/E15.011, 704/E15.005, 704/E17.009
International Classification G10L15/07, G10L15/04, G10L17/10, G10L17/04, G10L15/18, G10L15/16, G10L15/10
Cooperative Classification G10L15/04, G10L15/16, G10L15/1815, G10L17/10, G10L17/04, G10L15/07, G10L15/10
European Classification G10L17/10, G10L17/04, G10L15/07, G10L15/04