Abstract:
A language-independent speaker-recognition system based on parallel cumulative differences in dynamic realization of phonetic features ( i.e. , pronunciation) between speakers rather than spectral differences in voice quality. The system exploits phonetic information from many phone recognizers to perform text independent speaker recognition. A digitized speech signal from a speaker is converted to a sequence of phones by each phone recognizer. Each phone sequence is then modified based on the energy in the signal. The modified phone sequences are tokenized to produce phone n-grams that are compared against a speaker and a background model for each phone recognizer to produce log-likelihood ratio scores. The log-likelihood ratio scores from each phone recognizer are fused to produce a final recognition score for each speaker model. The recognition score for each speaker model is then evaluated to determine which of the modeled speakers, if any, produced the digitized speech signal.

Description:
BACKGROUND OF INVENTION 
     1. Field of the Invention 
     This invention relates generally to speaker recognition and more particularly to text-independent speaker identification over a large population. 
     2. Description of the Related Art 
     Speaker recognition technology has many uses including data mining and metadata for automated court, meeting, and lecture transcriptions. Speaker recognition technology can be used in real-time in concert with speech recognition to enhance broadcast closed-captioned features by providing the identity of the speaker along with the actual words spoken. 
     Traditional speech recognition and speaker verification use similar analysis tools to achieve their goals. An input utterance is first processed to determine its essential characteristics. Typically, input utterances are converted to cepstral coefficients. A cepstrum is an inverse Fourier transform of the log power spectrum. The cepstral coefficients for training utterances are saved in a training phase. This training set may consist only of short utterances of passwords for speaker verification systems to extensive vocabulary recitals for speech recognition. In speech recognition, an input utterance is compared to models based on saved training data to determine which utterance is most similar. In a generalized speech recognition system the saved training information must be a generalized representation of many people&#39;s way of forming an utterance while in speaker verification the training information represents the individual characteristics of the speaker and the verification system tries to determine if an authorized person&#39;s input utterance is sufficiently close to the training data and can be distinguished from an impostor&#39;s utterances. As a result, the training in a speaker verification system emphasizes individual characteristics, while in a speech recognition system the characteristics are generalized over many individual speakers. 
     Speaker recognition systems determine the identity of a conversational speaker. They typically do not have the advantage of being able to compare a short spoken password to the same short training data. In many ways speaker recognition systems share many of the attributes of speech recognition systems: the input samples and training data are not limited to simple verification phrases, there is little a priori knowledge of the input, the pace of speech is uncontrolled, and environmental effects, including the characteristics of the transmission device and media (telephone circuit, radio or broadcast channel), are unknown. 
     For example, one technique for speaker recognition would be to use speech recognition to capture the exact sequence of phones, examine the acoustic phonetic details of different speakers producing the same sounds and sequences of sounds, and compare these details across speakers or score them for each speaker against a model. 
     As an extreme example, given speakers A, B, and C, where speaker A lisps and speaker B stutters; then given perfect recognition of a large enough sample of speech by all three, the acoustic scores of the [s] and [sh] sounds might distinguish A from B and C, and either the acoustic scores or the Hidden Markov Model (HMM) path traversed by the initial stop consonants, for example, might distinguish B from C and A. 
     A problem with this approach is that speech recognizers are usually optimized for the recognition of words, not of phones; use word n-gram statistics to guide their decisions; and train their acoustic processing, model topologies, and time alignment to ignore speaker differences. 
     These very difficulties in speech recognition systems are in essence the differences in speakers—the differences that can be used for speaker recognition. Most practical methods of speaker recognition, and especially those with very limited training, are based on differences in broadly defined voice quality, rather than on these speaker differences. Individual speakers use different inventories of phones, or speech sounds. These cumulative differences in a speaker&#39;s pronunciation, represented by phones, can be exploited to recognize a speaker. Previous methods have used sequences of phonotactic constrained phonemes to extract and cluster acoustic features to recognize speakers or languages. There is a distinction between phonemes, defined by a language, as given by the dictionary, and phones, the actual pronunciation, as given by the acoustics. This invention exploits cumulative phone differences to identify the speaker. The dynamics of pronunciation contribute to human recognition of speakers, however, exploiting such information automatically is difficult because, in principle, comparisons must be made between different speakers saying essentially the same things. 
     What is needed is a tool that will consistently recognize and classify as many phonetic states as possible, regardless of their linguistic roles ( i.e., what words are being spoken), using sufficiently sensitive acoustic measurements, so that comparisons can be made among different speakers&#39; realizations of the same speech gestures. 
     SUMMARY OF INVENTION 
     In consideration of the problems detailed above and the limitations enumerated in the partial solutions thereto, an object of the present invention is to provide an improved speaker recognition system using the phonetics of pronunciation. 
     Another object of the instant invention is to provide a speaker recognition system that operates independently of the text spoken, sex of the speaker, language used by the speaker, and audio characteristics of the input channel. 
     Another object of the instant invention is to provide a speaker recognition system that is not wholly dependent on acoustic measurements but relies on higher level speaker information. 
     Still another object of the instant invention is to provide a speaker recognition system that uses parallel streams of highly uncorrelated multi-lingual features. 
     Still another object of the instant invention is to provide a speaker recognition system that can exploit the long-term speaker characteristics associated with large amounts of training data. 
     Yet another object of the instant invention is to provide a speaker recognition system that makes use of n-gram modeling. 
     In order to attain the objectives described above, according to an aspect of the present invention, there is provided a method of and device for phone-based speaker recognition whereby speakers may be recognized and distinguished based on higher level speaker information and the cumulative differences of phonetic features of their voices. 
     The instant invention is a speaker-recognition system based only on phonetic sequences, instead of the traditional acoustic feature vectors. Although the phones are detected using acoustic feature vectors, the speaker recognition is performed strictly from the phonetic sequence created by the phone recognizer(s). Speaker recognition is performed using the outputs of one or more recognizers each trained on a different linguistic characteristic. Recognition of the same speech sample by these recognizers constitutes different views of the phonetic states and state sequences uttered by the speaker. 
     The instant system performs phonetic speaker recognition in four steps. First, a phone recognizer processes test speech utterance to produce phone sequences. Then a test speaker model is generated using phone n-gram counts. Next, the test speaker model is compared to the hypothesized speaker models and a background model. Finally, the scores from the hypothesized speaker models and background model are combined to form a single recognition score. 
     In some embodiments multiple-language phone sequences are accommodated by incorporating phone recognizers trained on several languages resulting in a matrix of hypothesized speaker models and background models. 
    
    
     The aforementioned features, objects, and advantages of this device and method over the prior art will become apparent to those skilled in the art from the following detailed description and accompanying drawings. 
     BRIEF DESCRIPTION OF DRAWINGS 
     Our invention can best be understood when reading the following specification with reference to the accompanying drawings, which are incorporated in and form a part of the specification, illustrate alternate embodiments of the present invention, and together with the description, serve to explain the principles of the invention. In the drawings: 
     FIG. 1 is a schematic diagram showing one embodiment of the phone-based speaker recognition system; and 
     FIG. 2 is a schematic diagram of a recognition scorer in a device for phone-based speaker recognition. 
    
    
     DETAILED DESCRIPTION 
     FIG. 1 is a schematic block diagram of an exemplary phone-based speaker recognition system  100  (system  100 ). In this exemplar embodiment of the instant invention a digitized voice signal of the speaker to be identified is received at the voice input  110  and forwarded in parallel to one or more phone recognizers  120 . 
     Each phone recognizer  120  converts the input digitized voice signal into a time ordered stream of phones representing the sounds produced by the speaker in the digitized voice signal. The phone recognizers  120  operate without phonotactic constraints, e.g., they do not impose any constraints on the order of the sounds. Each of the phone recognizers  120  is designed for one or more different linguistic characteristics, such as the language, sex, dialect, age, education, or social standing of the speaker. In the preferred embodiment the phone recognizers  120  are designed for different languages. 
     In some simple embodiments, where cost and complexity are factors, a system  100  with only one phone recognizer  120  may be employed. Such a system generally will not achieve the accuracy of a multiple phone recognizer  120  system  100  but may work sufficiently well where there is little variation in linguistic characteristics among the speakers to be identified. 
     Various phone recognizers  120  are well known in the art and any meeting the functionality described above may be employed in the instant system  100 . In the preferred embodiment the phone recognizers  120  used by the Parallel Phone Recognition with Language Modeling (PPRLM) language identification system are used for creating the time ordered stream of phones. These phone recognizers  120  calculate twelve cepstral (c′ 1 -C′ 12 ) and thirteen delta-cepstral (c′ 0 -c′ 12 ) features on 20 ms frames with 10 ms updates. The cepstral and delta-cepstra are sent as two independent streams to fully connected, three-state, null-grammar Hidden Markov Models (HMM). They use a Viterbi HMM decoder implemented with a modified version of the Cambridge University HMM Toolkit. The output probability densities for each observation stream (cepstra and delta-cepstra) in each state are modeled as six univariate Gaussian densities. 
     In the preferred embodiment the digitized voice signal of the speaker to be identified on voice input is also forwarded in parallel to a speech activity detector  125  where the presence of speech is detected. The speech activity detector  125  signals the phone recognizers to change any corresponding phones in the time ordered stream of phones to silence tags when no speech is identified throughout the phone&#39;s duration. This eliminates erroneous phones caused by spurious input signal noise. 
     In the preferred embodiment a start tag is inserted at the beginning and a stop tag is inserted at the end of each time ordered stream of phones. Additionally, each silence phone and each consecutive stream of silence phones in the time ordered stream of phones are replaced with a stop and start tag pair thus creating a modified time ordered stream of phones. 
     Each time ordered stream of phones, or in the preferred embodiment, modified time ordered stream of phones, hereinafter “phone stream,” is transmitted to a corresponding tokenizer  130 . Each tokenizer  130  receives the phone stream and creates a set containing all phone n-grams in the phone stream and the number of times each phone n-gram occurred. The number of phones used in the n-gram pattern is defined by n where n is a positive integer, and patterns with more than one value of n can be collected. In the simplest embodiment, where n=1, assaying the phone uni-grams is the only task of the tokenizer  130 . In the preferred embodiment, where n=3, the tokenizer  130  must assemble and assay existing phone tri-grams. 
     Each tokenizer  130  output is in turn received by a corresponding recognition scorer  140 . As shown in FIG. 2 the phone n-gram set is transmitted to one or more speaker model scorers  150  and the background model scorer  160  in the corresponding recognition scorer  140 . 
     Each speaker model scorer  150  computes a speaker log-likelihood score using a speaker model  155  for each phone n-gram in the phone stream. This score is an estimate of the possibility that the phone n-gram p was produced by the speaker represented by the speaker model  155 . In the preferred embodiment, the speaker log-likelihood score is calculated using              S   i          (   p   )       =     log        [         H   i          (   p   )         N     H   i         ]         ,                          
     where S i (p) is the speaker log-likelihood score for phone n-gram p for speaker i, H i (p) is the number of occurrences of phone n-gram p for speaker i as found in its speaker model  155 , and N  Hi  is the total number of phone n-grams for speaker i. 
     The speaker model  155  contains the number of occurrences of each phone n-gram that occurred in the speaker training speech set collected from a particular speaker. The speaker model  155  is created by passing the speaker training speech set through the phone recognizer  120 , speech activity detector  125 , and tokenizer  130 . The phone n-grams created by the tokenizer  130  are counted and stored in the speaker model  155 . Speaker recognition performance can be increased by ignoring infrequent and too frequent phone n-grams in the speaker model  155 . 
     Each background model scorer  160  computes a background log-likelihood score using a background model  165  for each phone n-gram in the phone stream. This score is an estimate of the possibility that the phone n-gram p was produced by a set of background speakers represented by the background model  165 . In the preferred embodiment, the log-likelihood score is calculated using            B        (   p   )       =     log        [       U        (   p   )         N   U       ]         ,                          
     where, B(p) is the log-likelihood score for phone n-gram p for the background model, U(p) is the number of occurrences of phone n-gram p for a set of background speakers as found in background model  165 , and N U  is the total number of phone n-grams for the background model. 
     The background model  165  contains the number of occurrences of each phone n-gram that occurred in the background training speech set collected from many speakers, excluding all speakers to be recognized. The background model  165  is created by passing the background training speech set through the phone recognizer  120 , speech activity detector  125 , and tokenizer  130 . The phone n-grams created by the tokenizer  130  are counted and stored in the background model  165 . Speaker recognition performance can be increased by ignoring infrequent and too frequent phone n-grams in the background model  165 . 
     The ratio scorer  170  combines the speaker log-likelihood score from the speaker model scorer  150  and the background log-likelihood score from the background model scorer  160  to produce a log-likelihood ratio. In the preferred embodiment, the log likelihood ratio is calculated using            η   i     =         ∑   P                     (       w        (   p   )            [         S   i          (   p   )       -     B        (   p   )         ]       )           ∑   P          w        (   p   )             ,                          
     where S i  (p) is the log-likelihood score for phone n-gram p for speaker i, B(p) is the log-likelihood score for phone n-gram p for the background model, and w(p) is a weighting function. In the preferred embodiment, the weighting function is calculating using w(p)=cp) 1−d , where, c(p) is the number of phone n-grams occurring in the digitized voice signal of the speaker to be identified, and d is a discounting factor ranging from 0.0 to 1.0. When d=1 all phone n-grams occurring in the phone stream are weighted equally. When d=0, each phone n-gram is weighted by the number of times it occurs in the phone stream. In the preferred embodiment, d=1. 
     The recognition scorers  140  produces a set of k log-likelihood ratio scores, one for each speaker model  155  and corresponding to its phone recognizer. These scores are passed to the fusion scorer  190  where the specific speaker log-likelihood ratio scores from each precognition scorers  140  are combined to form a single speaker score. In the simplest embodiment, the log-likelihood ratio score produced from a single phone recognizer  120 , tokenizer  130 , and recognition scorer  140  is the output of the fusion scorer  190 . In the preferred embodiment, log-likelihood ratio scores created from multiple phone recognizers  120 , tokenizers  130 , and recognition scorers  140  are combined to form a final recognition score in fusion scorer  190 . A simple fusion sums each equally-weighted log-likelihood ratio score. In the preferred embodiment, the fusion is a non-linear combination utilizing a Gaussian mixture model with k-means clustering. 
     Each score output from fusion scorer  190  is passed to speaker selector  200 . The speaker selector  200  evaluates the k scores from fusion scorers  190  to either select a single speaker or reject all modeled speakers. In the preferred embodiment, the largest fusion score greater than a predefined threshold determines the speaker selected. I f the largest fusion score does not exceed the threshold then all possible modeled speakers are rejected. 
     Although various preferred embodiments of the present invention have been described herein in detail to provide for complete and clear disclosure, it will be appreciated by those skilled in the art, that variations may be made thereto without departing from the spirit of the invention or the scope of the appended claims.