THRESHOLD SETTING AND TRAINING OF A SPEAKER VERIFICATION SYSTEM

Abstract: Speaker verification system in comprising model building means which build n different speaker models, each based on n-1 tokens, for each model one independent token being available for estimating the distance between the model and an utterance that has not be used to build the model. Estimation means estimate the central tendency of the distance between the speaker's model and newly prodced utterances of the same speaker, and also its variance. The central tendency of distances between the model and utterances is optimised, based on linear interpolation: Th(new) = b * CTi + (1 -b) * CTt, where Th(new) is the optimal threshold, CTi is the central tendency obtained from pre-recorded impostor speech, CTt is the central tendency estimated from the enrolment speech of the new customer, and b is the interpolation parameter, that is optimised using additional pre-recorded utterances not used in estimating CTi.

Description: THRESHOLD SETTING AND TRAINING OF A SPEAKER VERIFICATION SYSTEM BACKGROUND OF THE INVENTION The invention referres to a speaker verification system. Speaker verification (SV) systems are systems in which models of each customer must be built during an enrolment process, accept/reject thresholds must be established during the same enrolment process and speech of customers who claim a certain identity must be compared to the claimed speaker's model, to determine whether the identity claim is likely to be true This patent application addresses the second sub-process mentioned above, i.e., the estimation of the accept/reject threshold during enrolment. Speech is a behavioural biometric measure. As all other behaviour, speech behaviour is variable. Therefore, it is not possible to build exact models of a person's speech behaviour. Rather, models must always consist of some combination of central tendencies and the attendant variance around the central tendency value of all parameters with which the speech is characterised. By consequence, the process of verifying a claimed identity is always statistical in nature: one must test what the likelihood is that the newly observed speech pattern is indeed produced by the person who has enrolled the model (i.e., the person whose identity is claimed by the speaker) . Speaker verification systems may use a wide range of parameters to characterise the speech, including spectral coefficients, Mel Frequency coeffeicients, Cepstral coefficients, Mel Cepstral coefficients. Pitch, Loudness, etc. All these different parameter representations are used in essentially the same process during model enrolment: for all individual coefficients central tendencies and variances must be estimated. This patent application -2- applies to all speaker verification systems that build models consisting of explicit or implicit sets of central tendencies and variances of all speech coefficients. It is generally known in the field that ideally the estimation of the model parameters would require speech that is recorded in many different sessions. This is so because only multiple sessions can produce data about the behavioural variations in the client (e.g. due to physiological and psychological status) and variations in the handset microphone, acoustic background noise, etc. However, for Human Factors reasons most operational speaker verification systems must limit the number of enrolment sessions to one or two. This is obviously not enough to capture all variation. Therefore, in general the estimates of the model parameters are not fully reliable. In addition to the parameters of the speaker model, another pair of statistical distributions must be estimated during the enrolment process, viz. the distribution of the distances to the speaker model of new utterances of the same speaker, and the distribution the distances of suitable utterances produced by impostor speakers to this speaker's model. This pair of distributions is needed to enable the system to determine whether a new utterance is more likely to have been produced by the speaker who has enrolled the model or by an impostor speaker. In estimating the distribution of the distances of impostor speaker utterances to the newly enrolled speaker's model it may be possible to use speech of many speakers that has been recorded well before the start of the enrolment session. Therefore, it may be possible to estimate the distance distribution for impostor speakers relatively easily and reliably (provided, of course, that the model parameters are reliable themselves) . Estimating the distribution of the distances of the newly enrolled customer to her or his own model is much more difficult, because usually no independent speech utterances are available for that purpose: to keep the enrolment procedure as short and easy as possible, usually all speech produced during the enrolment session must be used to estimate the model parameters. Consequently, no data are left that would allow an independent estimate of the distance between the model and newly produced utterances of the same speaker. Moreover, all speech that is available for the estimation of the true customer's distance to his/her own model is recorded under the same conditions as the speech used to build the models. Therefore, one must expect that the estimates of the distance of new utterances the true customer's utterances to his/her models will have a strong bias towards small values. Almost all scientific research on speaker verification has been based on pre-recorded corpora of utterances of true and impostor customers. Some utterances are set apart for building the models; the remaining speech is used for testing. In these tests the distance of all test utterances to a speaker model are computed, both for the test utterances of the the true customers and for the test utterances of the impostor speakers. This process yields two distributions, viz. one distribution of the distances of the true customer to her/his model, and another distribution of the distances of the impostor speakers' utterances to the same model. Based on these distributions it is then possible to determine the distance threshold that minimises the number or the proportion of false accept or false reject decisions. Here, a false accept decision means that the distance to the model of an impostor utterance is so small, that it falls well within the distribution of the distances of the true customer to her/his own model, and must therefore be accepted as if it was indeed produced by the true customer. False reject means that the distance between an utterance of the true customer and her/his own model happened to be so large that it falls well within the distribution of impostor utterances, and therefore must be considered as an utterance produced by a speaker different from the true customer. In this process the continuum of the distances between a model and a new utterances is divided into two halves: distances that are smaller than some threshold lead to the acceptance of the utterance as produced by the speaker who enrolled the model, while distances larger than this threshold will be rejected, because they are considered as produced by some other speaker. The process sketched in this paragraph is known as 'a posteriori threshold optimisation', because it uses all data in the corpus, thereby biassing the threshold setting. In real world applications of speaker verification systems 'a posteriori threshold optimisation' is obviously impossible, because accept/reject decisions must be made immediately for each new utterance, whether it has been produced by a true customer or an impostor. Thus, in real world applications of speaker verification systems the • accept/reject threshold must be estimated 'a priori', using only true customer speech recorded during the enrolment session. This obviously requires a solution for the problem described above, viz. that the estimates that can be obtained from speech recorded in the enrolment session are by necessity biassed. This patent application describes a number of ways in which the bias can be minimised, so that maximally reliable estimates of the accept/reject threshold can be obtained. SUMMARY OF THE INVENTION In this patent application two classes of solutions to the problem of a priori threshold estimation are described. -5- Moreover, the two classes can be combined, so as to obtain even better results. Both classes of techniques address the issue of improving the estimates of the distance between the newly built model and utterances of the true customer. 5 The estimation of the distribution of distances to impostor utterances is not addressed. Leaving One Out The bias in the estimates of the distance between new utterances of the true customer and the newly built model to can be reduced by a new application of the 'leaving one out' technique. 'Leaving one out' as a general experimental procedure has been used previously, e.g. in testing speaker dependent automatic speech recognizers for isolated words on the basis of pre-recorded speech corpora. Here, the i 5 technique is used differently, omong others to circumvent the lack of a suitable pre-recorded corpus of utterrances. During a typical enrolment session for a speaker verification system a new speaker is requested to produce n o tokens of some set of phrases, where typically n > 2. We have developed a procedure in which n different speaker models are built, each based on n-1 tokens. For each model one independent token is then available to estimate the distance between the model and an 5 utterance that has not been used to build the model. This procedure allows us to estimate not only the central tendency of the distance between the new speaker's model and newly produced utterances of this same speaker, but also its variance. Obviously, since all utterances are 0 recorded under similar conditions, both central tendency and variance are still biassed towards too small values. Therefore, it is advantageous to combine this technique with one or more versions of the distance interpolation technique described in the next subsection. -6- Distance Interpolation This subsection describes two different approaches to the estimation of the accept/reject threshold from enrolment speech combined with pre-recorded impostor speech. In the first method, only the central tendency of the distances between the new model and true customer utterances or impostor utterances is optimised. The variances of the two distributions are not used. No assumptions are needed about the statistical nature of the distributions. Optimisation is based on linear interpolation: Th(new) = b * CTi + (1 -b) * CTt, where Th (new) is the optimal threshold, CTi is the central tendency obtained from pre-recorded impostor speech, and CTt is the central tendency estimated from the enrolment speech of the new customer, b is the interpolation parameter, that is optimised using additional pre-recorded impostor utterances that were not used in estimating Cti. In the second method it is explicitly assumed that the distance distributions of both true customers and impostors approach the Gaussian distribution. To improve the estimates, enrolment speech and pre-recorded impostor utterances are segmented into a large number of theoretically independent parts, for each of which the distance to the newly enrolled model is computed. The Central Tendencies of the distance distributions are then corrected to remove the bias caused by the fact that the enrolment speech has been used both for building the model and for computing the distances to the model. The optimal correction parameter h is optimised using additional pre-recorded impostor speech. Both methods set forth in this sub-section can be applied to optimise the correction factors either speaker independently, where a single optimal value is computed that applies to all customers, or speaker dependently, where an optimal correction factor is estimated for each newly enrolled customer. More detailed descriptions of the methods described above can be found in two papers attached to this patent application (Attachments I and II) . In these papers it is shown that the newly introduced methods improve the operational performance of speaker verification systems. Finally, an implementation of a speaker verification system according to the present invention is visualized in figure 1, while the outstanding capabilities of the invention in illustrated in Attachment III. Figure 1 schematically shows a system according to the invention, comprising a number of co-operating, hardware or software based modules, viz. a receiving module 1, a model building module 3, a threshold module 4, an estimating module 5, a combining module 6 and an optimisation module 7. Said modules are interconnected by a (software or hardware) bus 8 and controlled by a "control & output" module 9, which also ouputs an "accept/reject" signal resulting from the processes executed in the various modules of the shown speaker verification system. The receiving module 1 receives utterances of a speaker 2 during an enrolment process, during which speaker 2 produces n tokens of some set of phrases. Model building module 3 builds one or more models consisting of explicit or implicit sets of central tendencies and variances of the speech coefficients of the utterances received via receiving module 1. Threshold module 4 establishes accept/reject thresholds during said enrolment process, while estimating module 5. Model building module 3 builds n different speaker models, each based on n-1 tokens, for each model one independent token being available for estimating, by the estimating module 5, the distance between the model and an utterance that has not been used to build the model. The estimation module 5 estimates the central tendency of the distance between the speaker's model and newly produced utterances of the same speaker, and also its variance. The estimation of the accept/reject threshold from enrolment speech is combined, by combining module 6, with pre-recorded impostor speech, whereby the central tendency of the distances between the model and utterances is optimised by optimising module 7. Optimisation, in the optimisation module 7, is executed by linear interpolation: Th(new) - b * CTi + (1 -b) * CTt, where Th(new) is the optimal threshold, CTi is the central tendency obtained from pre-recorded impostor speech, CTt is the central tendency estimated from the enrolment speech of the new customer, and b is the interpolation parameter, that is optimised using additional pre-recorded utterances not used in estimating Cti. Enrolment speech and pre-recorded impostor utterances are segmented, in said optimising module 7, into a large number of theoretically independent parts, for each of which the distance to the newly enrolled model is computed. The central tendencies of the distance distributions are corrected, in the optimising module 7, removing a bias caused by the fact that the enrolment speech has been used both for building the model and for computing the distances to the model. In the optimising module 7 a single optimal value is computed that applies to all speakers. Preferrably, in the optimising module 7 an optimal correction factor is estimated for each newly enrolled speaker. ATTACHMENT 1 - 9 TECHNIQUES FOR A PRIORI DECISION THRESHOLD ESTIMATION IN SPEAKER VERIFICATION J Lindberg'. J Koolwaaif, H.-P. Hutter 3 , D Genoud 4 . J.-B. Pierrot 5 , M. Blomberg 1 , F Bimbo f 6 lindberg@speech.kth.se koolwaaij @let.kun.nl hans-peter.hutter@ubs.com genoud @ιdιap.ch pierrot@sig.enst.tr mats@speech.kth.se bimbot@sig.enst.fr http.7/www PTT-Telecom.nl/cave tliat false acceptance and false rejection rates become equal. ABSTRACT The EER gives a good estimate of die modeling module ot A key pio lem for field applications in speaker verification is the SV system. The EER does, however, not give much the issue ot a priori threshold setting In the context of the intormation about the performance to expect in a field CAVF pro|ect several methods for estimating speaker- application. In such a case the decision threshold(s) must be independent and speaker-dependent decision thiesholds weie estimated a priori during the enrollment phase Bayesian compaied Relevant parameters are estimated fiom theory indicate that the decision threshold; s) could be development data only. 1 e without resorting to additional predicted for the false acceptance and false lejection costs client data The various approaches were tested on the Dutch The mismatch between the speaker and (non-speaker) SESP database model(s) and the real data distributions requires adjustments of the threshold(s) for efficient decisions to be made RESUME Part of the results in this paper is also reported in [8] In this Un des pioblemes importants en verification du locuteur paper a new method (SD-4) has been added and this method poi (e sui 1 estimation du seuil de decision Dans le cadre du is compared to the results reported in [8] pioiet euiopeen CAVE, plusieurs methodes d estimation en mode dependant et independent du locuteur sont companies 2. THEORETICAL BACKGROUND I es paiameties servant a revaluation de ce seuil sont estimes umquemenl gi ce au donnees de developpemenl et sans ajout 2.1 Notations de donnees supplementaires pour les clients Les diffeientes Let X denote a speaker, and X his probabilistic model Let X appioches sont testees sur la base de donnees en langue denote the non-speaker model for speaker X l e the model ol neei landaise SFSP the rest of the population Let Y be a speech utterance claimed as being from speaker X 1. INTRODUCTION The CAV1- pi iecl (CAller VF.πfication in Banking and If we denote as X (resp XI the acceptance desp leiection) Telecommunications) was a 2-year pro|cc( that ended m decision of the system, and p λ (resp p ) the a piion decembei 1997 It was supported by the Language probability of the claimed speaker to be iiesp not to be) Fngineeπng Sector ot the Telematics Applications speaker X . the total cost function of the system is p | i'logianimo ol the European Union, and loi the Swiss paitneis b\ the Olfice Federal de TEducation et de la Science c=c +Q (x\x) i P[ χ \ x ) ( Bundesami lui Bildung und Wissenschall) The partners wr p - ■ p(x\x)- ( 1 ) weie Dutcli l'TT Telecom. KUN. KTH. ENST. UBILAB where P(XIX) and P(XIX) denote respeclivelv the piobabilits II MAP VOCAI IS. TELIA and Swiss Telecom PTT In the lealm ol the pιo|ect, 2 telephone-based system which used of a talse acceptance and of a false re j ection while C ( | N| and Spcakci Veiihcation (SV) were developed and assessed C (\ ι \ ) represent the corresponding costs (assuming a null cost Woi k Package 4 (WP4) in this project locused on the for a true acceptance and a true rejection) l seaich and development aspects The SV system used m the expeiimenls leporled here is die cave-WP4 generic SV 2.2 PDF Ratio and Bayesian Threshold svsiem | 7] based on the HTK sottwaie plalloim [2| II we now denote by Pχ and Pχ the Probability Densitv 1 aboialois evaluations of SV systems usually base then Functions (PDFs) of the speaker and ol the non-speakei assessments on the Equal Error Rate (EFR) The FFR is obtained h\ a postei ion setting the decision thιeshold(s) so KTH Dept of Speech. Music and Hearing. SE- 10044 Stockholm. SWEDEN-EU : KUN Dept ot I anguage & Speech. Frasmusplcin 1. NL 652 HT Nijmegen. THE NETHERLANDS EU ' Ubilab Union Bank of Switzerland. BalinholsUasse 45 CH-8021. Zurich. SWITZERLAND 4 IDIAP Rue du Simplon 4. Case Postalc 592 CH-1920 Maitign . SWITZERLAND ' FNST Dept Signal. CNRS - URA 820 46 Rue Barrault 75634 Pans cedex 13 FRANCE EU ' Also with IRISA / CNRS & INRIA. FRANCF -FU 10 - isiiibutions the minimisation of C in equation ( 1 ) is If n is large enough, the utterance log-likelihood latio can be obtained bv implementing the PDF Ratio (PR) test [4] assumed to follow a Gaussian distribution Tins distribution accept is different depending on whether the speech utterance Y was i P Λ Y ) > pronounced by speaker X or by an impostor X log LRJY\X ) → G(M X ; S X ) reject (8) wlieie R is the Bavesian threshold log LR χ (γ\x)→ G{M ,S χ ) and similarly C (*\x) Px R = (3) log LR χ ' {γ\x)→ G(m x ;s x ) C [ \ x) Px (9) log LR' (y\x)→ G{m^;s x ) 2.3 Half total Error rate with the obvious relations Fquation (3) shows that the optimal threshold only depends M , ■ nn -, nm on the false acceptance / false rejection cost ratio and the ( 10) ■ ns y impostoi / client a pi ion probability ratio In the paiticular S x = ns x case ot equal costs of 0 5 and when clients and impostors are As opposed to the utterance log-likelihood l atio the liame assumed a priori equiprobable. the choice of Θ=l as a based log-likelihood ratio does not geneiallv lollow a (.lecision thieshold should then lead to a minimum of the Half Gaussian distribution. But, if we denote as u^ and σ (resp total FI IOI Rate U, and σ ) the mean and variance of the distiibution ot the rrame-based client (resp impostor) log-likelihood i.uio HTER = - j (χ\χ ) '+ P X X (4) 2 log lrχ(v, IX) (resp log /rχfv, IX)), and it we assume that the trame-based scores are statistically independent w have 2.4 Likelihood Ratio and Threshold Adjustment (according to the Limit Central Theorem) In piactice however, the PR in equation (2) is calculated m χ = μ x m χ = μ χ orn likelihood lunctions. i e estimations of the PDFs, which ( 1 1 ) do not match the exact speaker and non-speaker distributions :/Jn ~ xlJ" As consequence, it is usually necessaiy to adjust the Under the assumption that the client and impostoi log-I R thieshold ot the PR test accordingly, in order to correct for follow Gaussian distπbution. the optimal decision thieshold die impiopei In between the model and the data [5] Thus the can be obtained as PR lesi becomes a I ikelihood Ratio (LR) test Λ . . accept G(M x ,S x \ή θ v (Λ)= arg, ( 12) LR (Y) - p λ γ) > GiM f . Sx Xt) ' w θχ (Λ) (5) < reject and similarly tor log-LR' wheie and P / denote the respective model likelihood In piactice it is feasible to obtain reasonable estimates ol NL and S fiom scores yielded by a population ol p\cιulo- lunctions loi the speaker and Ihe non-speakei. and Θ X (R) is a lmpostors Conversely, in reai app calions M and χ have speakei ( nd cost-) dependent threshold to be estimated from the enrollment data ihemseKes and aie 2.5 Gaussian log-LR model theiefore strongly biased, especially m the case w hen \eι \ tew enrollment data are available In most cases the logarithm of LR ( Y) is obtained as the sum ol the logjiithm ot the frame-based likelihood latio scores 3. SPEAKER-INDEPENDENT (SI) //vn I THRESHOLD A classical method tor adjusting the threshold in equation (5) i g /j? / (y)= ^i g /ι,(\ I ) (6) consists in estimating a speaker-independent ihieshoid so as to optimise the cost function ot equation ( 11 In pi actice tins optimisation is earned out on a development data set w eie \ denotes the ; th tiame in utteiance Y ol total length composed ot enrollment and test data loi a population ol ;/ In some variants, the average log-LR is used instead of the speakers which is distinct from (but repiesentatn e ol " ) the loa-l R actual client population In our experiments we ha\e tested the SI method both with unnorma sed and noimalised I R lo /./?' (/)= -log Λ ,(Y) (7) We denote these two approaches as SI and SI N ιespeαiNel\ We w ill lelei to these two quantities as unnoimalised and The SI and SI-N methods do not make an\ patticulai noimalised I R iespectivelv assumption as regards the shape of the log I R disti ibution However, the fact that the threshold is speakei independent - 11 - lelies on the hypothesis that the mismatch between the likelihood function and the actual client PDF translates into a Θ X (R)-- arg, Gfø t) (16) client-independent shift between the log-PR and the log-LR G ( . This is obviously a very simplistic hypothesis as part of the model mismatch is certainly variable across speakers 4.4 Method SD-4 The fourth SD method can be viewed as a speaker dependent 4. SPEAKER-DEPENDENT (SD) adjustment of an estimated SI threshold Θ (R) is obtained as THRESHOLD a linear combination of the Si-threshold and estimates of M Conveisely the estimation of a speaker-dependent threshold and M, accounting tor the vartabUity in modeling accuracy can be hindeied by the lack of proper data for estimating that Θ x (R Θ SI + γfM j ■ M ( 17) thieshold Indeed in the context of practical applications eniollment matenal is so limited that it is not reasonable to where M x is obtained from the enrolment set ot a leseive some of it for threshold setting The speaker- development population, whereas M x is the (biased) estimate dependent thieshold must be derived from the same client of M x Parameter γ is optimised on the development data as those used for training the client model (and from population some pseudo-impostor data) In the next sections we present 4 methods for speaker- 5. DATABASE dependent TS Methods SD-1. SD-2 and SD 4 weie tested with the unorma sed log-LR whereas SD-3 was used with All our experiments on TS were earned out on the realistic noima sed scoies (log-LR') telephone speech database SESP [1], collected by KPN Research It contains telephone utterances trom 21 male and 4.1 Method SD-1 20 female speakers calling with different handsets (including some calls from mobile phones) from a wide vanety ot SD 1 consists ot estimating Θ (R) as a neai combination ot places Duπng each call, the speaker was asked to utter a the log I R mean M . and variance S , following an number of items, including a speaker-dependent sequence ol appioach similai to the one proposed by Furui [6] 14 digits (twice) and a few other sequences ot 14 digits conesponding to other speakers Θ X (R)= M X + OS X ( 13) Each session contains, therefore. 2 utterances of the client wheie M, and S are obtained from pseudo-impostor data card number For the experiments desenbed in this paper we wheieas is optimised on a development population used 2 enrollment sessions with a low level ot background noise, corresponding to 2 calls placed trom 2 diilcrenl 4.2 Method SD-2 handsets Two other calls were reserved as extended The second method lelies on an estimation of Θ (R) using enrollment matenal The rest of the calls weie used as test also the client scoie obtained with the eniollment data In this matenal method (-) (R) is obtained as a linear combination of In our experiment on TS. we have split the SFSP data into 2 estimates ol M and M. sub-populations which we denote SESP a and SFSP b SESP a contains 11 male and 10 female speakeis while Θ γ (/?) = βΛ^ + (l - β)M , ( 14) SESP-b contains 10 male and 10 female speakers Fach data set is composed of approximateh 800 genuine d ials and 250 wheie M x is obtained fiom pseudo-impostoi data wheieas impostor attempts trom odier clients (out ol which about 75 f M . is the (biased) estimate ol M Paiameter β is optimised are same-sex attempts) We use SESP b as pseudo-nnposiois on a development population and development data for SESP-a and vice veisa Acoustic lcatures are 16 LPC cepstral coetficienls with log 4.3 Method SD-3 energy together with their first and second derivatives This method is explicitely based on the Gaussian model of Cepstral mean subtraction is applied Our tests weie carried utteiance log I R distribution, as exposed in | 5| The method out using Left-Right HMM digit models with 2 dillerent uses the Gaussian model introduced in subsection 2 5 topologies p=2 states per phoneme q=3 Gaussian densities Fshmales tj χ and σ of μ^ and σ x aie initially obtained fiom per state, and p=3 states per phoneme q=2 Gaussian densities the client eniollment data, whereas μ and 0 aie estimated per state In these experiments, both the client and woi ld horn the pseudo impostor population Then a speaker model, have the same topology These conliguiations weie independent correction It is applied to (L^ only chosen as they were those that worked best in teims ol Fqual Error Rate, in previous expenments on SESP 1 1 1 = μ -Λ ( 15) In all our experiments we aim at optimising the HTFR wheie h is optimised on a development population Then, defined m equation (4) eslimates ot n s N . m x and s are obtained fiom fl^. σ x , μfc and σ x as in equation (1 1 ) Finally Θ (R) is obtained as in quation ( 12) - 12 Table 1 Fqual Error Rates and comparative results foi seveial a priori tlireshold setting metliods, on the SFSP a and SESP-b databases Future work will consolidate these results bv extending the 6. RESULTS amount ot cxpenments and the size ot the database and bv mpichensive lesults are reported in Table 1 We provide testing the mem ot the various methods loi Thieshold Selling sepaialc peiloimances tor SESP-a and SESP b We fust give methods lor other cost functions than the HTF Fqual Fπoi Rates tor both unnormalised and normalised likelihood scoies Then we give the performance with the 8. REFERENCES lι\ed ihieshoid followed by those obtained with the vanous [ 1 ] Bimbot F . Huttcr H P Jaboulel C oolwaaq I speakei d pend nt TS methods presented above Lindberg J and Pierrot J -B Speakei Veiilicalion in the Telephone Network Research activities in (he CAVF 7. COMMENTS AND CONCLUSIONS Project Pioc EUROSPEECH ( >7 Rhodes Gieece On oui lask noimalisation by the utterance length seems to 1997 Vol 2. pp971 -974 have little eltect But SESP utterances all have quite similar | 21 Young S . Jansen J Odell J Oilason I) Woodland P lengths Theiefoie the real impact of normalisation can not The HTK BOOK HTK 2 0 Manual 1 95 b studied accui ately |3] Duda R O Hart P E "Pattern Classification and Scene I ooscK speaking the HTER is about 3 to 5 times largei than Analysis . John Wiley & Sons 1973 FFR Tins sticsses once more the tact that the FhR tiguie is a |4| Schait L L . "Statistical Signal Pioccssiin; Detection \ cι\ optimistic evaluation of the actual perlormanee ol a SV Estimation and Time Analysis Addison Weslev svsteni Publishing Company. 1991 [5] Bimbot F , Genoud D , "Likelihood i.ui adiustincnl loi All methods yield similar results, except method SD 2 and the compensation of model mismatch in speakei SD 4 w hich seem to perform consistently better This mav verification . Proc EUROSPEECH ( )7 Rhodes Gieece come liom the lact that these methods only use the means ot 1997 Vol 2. ppl387 1390 the log I R distiibutions. winch arc probably estimated more [6] Furui S "Cepstral Analysis Technique loi Automatic leliahh than the vanances. given the small amount ol data Speaker Verification ' IEEF Trans on ASSP vol 29 no ind the stioim bias in the client estimates 2 pp 254-272. 1981 It must also be noted that the SI methods do not pettoim [7] Jaboulet C Koolwaay J Lindbem I Pienot I B especially woise than the SD methods which tends to show Bimbot F The cave-WP4 gcneiic speakei vei ilicalion that a lai ge part of the model mismatch can be accounted for system Pioc RLA2C Avignon France 1998 In ,ι speakei independent shift of the Bayesian threshold |8| Pierrot J B Lindberg J Koolwaan I Hullei H P Quite impoi tant differences are obseived between Genoud D Blomberg M Bimbot F A eompai ison ol pei loimances obtained on SESP-a and SESP b which a priori threshold setting proceduics loi speakei lllustiales the lelatively large confidence inteival that must be ventication m the cave project I'HX IL 1 S S/' Seattle l iken i lo account when interpreting these results USA 1998 - 13 - ATTACHMENT 2 A COMPARISON OF A PRIORI THRESHOLD SETTING PROCEDURES FOR SPEAKER VERIFICATION IN THE CAVE PROJECT J.-B. Pierrot 1 , J. Lindberg 2 , J. Koolwaaij 3 , H.-P. Hutter 4 , D. Genoud 5 . M. Blomberg 2 , F. Bimbot 1 . (1) ENST / CNR.S (2) KTH (3) KUN (4) Ubilab-UBS (5) IDIAP pierrotβsig. enst.fr lindbergQspeech.kth.se koolwaaij91et.kun.nl hans-peter.hutter8ubs.com genoudβidiap.ch matsflspeech.kth.se bimbotβsig. enst.fr XS http : // BW . PTT-Telecom.nl/cave ABSTRACT Laboratory evaluations of SV systems are usually based on the Equal Error Rate (EER), obtained by The issue of a priori threshold setting in speaker a posteriori setting the decision threshold(s) so as to verification is a key problem for field applications. In equalise the false rejection and acceptance rates. InI lie context of the CAVE project, we compared sevdeed, the EER gives a good idea of the quality of the eral methods for estimating speaker-independent and modeling module in a SV system. However, in the speaker-dependent decision thresholds. Relevant pacontext of field applications, a specific procedure must rameters are estimated from development data only, i.e be implemented in order to set the decision threshwithout resorting to additional client data. The various old a priori, namely during the enrollmen procedure. approaches are tested on the Dutch SESP database. Whereas Bayesian theory indicates that he- decision threshold could be readily predicted for the false re¬ 1. INTRODUCTION jection and false acceptance costs, the mismatch between the speaker (and non-speaker) model(s) and the The CΛYΕ project (CAller VErification in Banking real data distributions requires the adjiistciiient of the and Telecommunications) is a 2-year project, supported threshold for an efficient decision. by I he Language Engineering Sector of the TelematThis paper reports on a series of comparative experics Applications Programme of the European Union, iments on a priori Threshold Setting (TS) carried out and for the Swiss partners by the Office Federal de by WP4. We first recall the main theoretical aspects lΕducal.ion et de la Science (Buiidesamt fiir Bildung involved in TS. Then, we express several TS procedures mid Wisseiischaft). The partners are Dutch PTT Tele- under a common formalism. Finally, we compare t eir cm. KI ' .X. KTH. ENST. UBILΛB, IDIAP. VOCΛLIS. efficiency on a task of speaker verification on a realistic TELIA and Swiss Telecom PTT. It started on Decemtelephone speech database (the SESP database). ber I" ' '. 1995. The technical objectives of the CAVE project are to design, implement and assess 2 telephone- 2. THEORETICAL ASPECTS based systems which use Speaker Verification (SV) technology. Work Package 4 (WP4) of this project focuses 2.1. Notations on the research and development aspects. The speaker veiilicalion system used in the experiments reported Let Λ' denote a speaker, and X his probabilistic model. ln-re is the (ieneric CAVE-WP4 SV system [1]. based Let Λ' denote the non-speaker model for speaker Λfc i.e on I lie HTK software platform [2]. the model of the rest of the population. Let V be a speech utterance claimed as being from speaker Λfc I EN'S T - Dept Signal. CNRS - URA 820. 16 Rue Baπault. 7.-,<;: > ,l Paris cedex 1:3, FRANCE-EU If e denote as Λ ' (resp. X) the acceptance (resp. 'Kill. Department of Speech, Music and Hearing. Drottning rejection) decision of the system, and \ (resp. /» v ) Kiislinas Vag 31. S-100 4 4 Stockholm. SWEDEN-EU the a priori probability of the claimed speaker t <> he Kt'N. Depl of Language & Speech, Erasnmsplein 1. NL-6525 I IT Nijmegcn. THE NETHERLANDS-EU (resp. not. to be) speaker Λ " . the total cost function of I I l)ilab. t uion Bank of Switzerland. Balmliofstrasse 15. CH- the system is [3] : Nii.'l. Zurich. SWITZERLAND IDIΛ . Rue du Simplon 4. Case Poslale .ϊ!>_. CII-H120 ar- linnv. SWITZERLAND C = ( (λ ' |λ-) μχ.P(X\X) + C l (λ i .'|.λ y ' t .μχ.l > (.\\.\) I I ) 14 where P(Λ * |Λ " ) and P(X\X) denote respectively the 2.5. Gaussian log-LR model probability of a false acceptance and of a false rejection, In most cases, the logarithm of LRχ(Y) is obtained as while <" (λ -ι^- 1 and C j .,... represent the corresponding the sum of the logarithm of the frame-based likelihood costs (assuming a null cost for a true acceptance and a ratio scores lrχ (y,) : true rejection). 2.2. PDF Ratio and Bayesian Thieshold log LRχ(Y) = ∑ log lr x (y,) (6) If we now denote by P,χ and P x the Probability Density Functions (PDFs) of the speaker and of the non-speaker where y, denotes the i th frame in utterance V, of total distributions, the minimisation of C in equation (1) is length n. In some variants, the average log-LR is used obtamefl b\ implementing the PDF Ratio (PR) test [4] instead of the LR : accept log LR'χ{Y) = - log LRχ-() (7) Px (Y > PRv R (2) χ < reject We will refer to this two quantities a unnornialised and normalised LR, respectively. where R is the Bayesian threshold The frame-based log-likelihood ratio lot/ — lilt n is large enough, log LRx (Y) can be assumed C, (X\X) Px R (3) to follow a Gaussian distribution. This distribution is C ( x\x) Px different depending on whether the speech utterance Y was pronounced by speaker Λ ' or by an impostor X 2.3. Half Total Error Rate log LRx (Y\X) Λs can be yen from equation (3), the optimal threshold log LRx (Y\X) (8) (Mχ- Sx should only depend on the false acceptance / rejection co i ratio and the impostor / client a priori probability and log Lfifcv(V|Λ ' ) g(m \ . ) ratio In the particular case when the costs |λ -ι λ) and log Lβ'.vO ' lA ' ) — t (,„ . , ) (») (' I A . I . A . , ) are equal to 0.5. and when clients and impostors are assumed a priori equiprobable, the choice of θ = 1 The frame-based log-likelihood ratio does not generally as a decision threshold should then lead to a minimum follow a Gaussian distribution. We will fit-note as of I he Half Total Error Rate ■ currently tn progress and if the frame-based log-likelihood rat it) values are supposed statistically independent, log I.Rγ(Y) 1IT R = ^[P(X\X)+ P(X\X) (4) can be assumed to follow one of two conditional Gaussian distribution Q {μχ\ <χ) oτ Q {μ λ rr ). depending 2.4. Likelihood R atio and Threshold Adjustment on whether Y was uttered by the client speaker X or by an impostor (Λ). According to the Limit ('(Mitral The¬ In piadife. however, the PR in equation (2) is calcuorem, μx = 777 Λ" and μ x = riχ. where in \ and in lated liom likelihood functions, i.e estimations of the are the means of the client (resp impostor) frame- 1'DI ' s. winch do not match the exact speaker and non- based log-likelihood ratio values, while \ = AΛ /V " spe.ikei distributions. As a consequence, it is usually and σ x — t>χ/ \ / , with Λ\ ami A \ denoting the stannecessaiv io d|tis! the threshold of the R lest accorddard deviations of the frame-based distiibution Sole ingly, in order to correct for the improper fit between that, in general, the frame-based log-likelihood values the model and the data [5]. do not follow themselves a Gaussian distribution I bus ihe PR test becomes an LR (Likelihood Ra¬ In the rest of the paper, we will indicate bv a stai tio) If t the estimates of μx, σx. m \ and Λ obtained from the accept enrollment data themselves. ι e the same data as those P χ (Y > used to estimate the parameters of the client model .V UtviY) = < ©λ-(β) P (Y rejcrt 3. SPEAKER-INDEPENDENT (SI) THRESHOLD where l ami Pγ denotes the respective model likelihood functions for the speaker and the non-speaker, A classical method for adjusting the threshold θ \ (/ι) and ( -) v ( R) is a speaker- (and cost-) dependent thresh- in equation (5) consists m estimating a speaker-independent threshold so as to optimise the tost lunclioii ol 15 equation ( 1 ). In practice, this optimisation is carried speaker was asked to utter a number of items, includout on a development data set, composed of enrolling a speaker-dependent sequence of 14 digits (twice) ment and test data for a population of speakers which and a few other sequence of 14 digits, corresponding to is distinct from (but representative of) the actual client other speakers. population. We will refer to this method as method SI. Each session contains therefore 2 utterances of the However, the fact that the threshold is speaker- client card number. For the experiments described independent relies on the hypothesis that the mismatch in this paper we used 2 enrollment sessions with a between the likelihood function and the actual client low level of background noise, corresponding to 2 calls PDF translates into a client-independent shift between placed from 2 different handsets. Two other calls were t he log-PR and the log-LR. This is obviously a too simreserved as extended enrollment material. The rest of plistic hypothesis as the model mismatch is certainly the data was used as test material. variable across speakers. The SESP data are very realistic in many aspects. SI : without normalisation vs SI-N : with normaliHowever, the obvious factor that make them signifisation. cantly different from those that could be expected from a field test data collection, is the lack of intentional impostor attempts. 4. SPEAKER-DEPENDENT ( SD ) In our experiment on TS, we have split the SESP THRESHOLD data into 2 sub-populations which we denote SESP- a and SESP-b. SESP-a contains 10 male and 10 fe¬ Conversely, the estimation of a speaker-dependent threshmale speakers while SESP-b contains 11 male and 10 old accounting for the variability in modeling accuracy female speakers. Each data set is composed of approxcan be hindered by the lack of proper data for estimatimately 800 genuine trials and 250 impostor attempts i ng t hat threshold. Indeed, in the context of practical from other clients (out of which about 75 % are same- a pplications, it is not feasible to reserve any enrollment sex attempts). We use SESP-b as development data mat erial for t hreshold setting. The speaker-dependent for SESP-a and vice-versa t hreshold must be derived from t he enrollment data I lieinselves. Do wc agree on the last paragraph, or did I misun- 6. RESULTS (l( i stood Kl ' N 's method ? If in fact we had all scores corresponding to log-ltkt liliood ratios normalised by the number of distinc tly assigned 4.1. Method SD1 frames, we could focus on this, and I belie ι>( this is what gives the best scores, isn't it the rase with you St < ond mi I od from KCN / identical to Furi s / to be KTH Ϋ The paper ivould gain a lot in sniiplicilt/ and i <>nι it li consistency. KUN, do you anticipate being abb to produce corresponding results within a rra.sonabli ddti y 4.2. Method SD2 (say lunch time Wednesday) ? And what about you Jran-benoit ? I am still esxpecting fresh results fi om I h u d m i Ihod from KUX / best perfor ming you. with m ean correction. Where arc t ey ^ 4.3. Method SD3 7. CONCLUSIONS lii bot/Cdioud (Eurospeech 97) tested by Pierrot / to • Normalisation has little effect (on TH ES E dat a ) b< ( ompl( l(d • HTER is about 3 to 5 times larger t han E ER 5. DATABASE • All methods perform similarily except SD which seems consistenltv better. A l l ou r experiments on TS were carried on the realistic t elephone speech database SESP. S ES P was collected by KPN Research. It contains 8. REFERENCES t elephone ut terances from 21 male and 20 female speakers calling wit h different handsets ( including some calls [1] F. BIMBOT, H.-P. H I TT E R . C from mobile phones) from a wide variety of places (such JABOULET. J . KOOLWAA 1.I . J . LI N Das rest au rants, public phones and airport departure BERG , J .-B. PIERROT ■ Spt akt i nfi- lounges) . A ll t he recordings were made between March cation in the telephone net work ■ i t st arch a nd M ay 199 1. A substant ial proportion of t he calls activities in the CA VE proje ct. Proc Eu- w as made from foreign countries. During each call, the rospeech ' 97. pp. 971-97 1 1997. - 16 | TS method eval. set dev. set P = 2, q - 3 P = 3. ? = 2 | a posteriori sp.-dep. thresholds) EER EER 1 EER SESP-a 0.57 0.99 SESP-b _ 0.46 0.63 1 EER-N SESP-a 0.57 0.99 SESP-b _ 0.26 0.89 | a priori FR FA HTER FR FA HTER 1 θ = 1 SESP-a - 12.11 0.25 6.18 13.25 0.25 6.75 1 SESP-b - 8.21 0.00 4.10 9.76 0.00 4.88 1 SI SESP-a SESP-b 0.86 4.60 2.73 1.85 4.01 2.93 SESP-a 2 22 2. 72 2.47 2.59 2.41 2.50 SESP-b SESP-a 1.72 1.73 1.72 1.47 1.61 1.54 1 SESP-b 0.26 2.26 1.26 0.97 1.91 1-44 SI-N SESP-a SESP-b 1.63 4.95 3.29 2.73 2.15 2.44 SESP-a 3.06 1.62 2.3J, 3.06 1.44 2.25 SESP-b SESP-a 2.25 1.96 2.11 2.12 1.61 1.87 [ SESP-b 0.42 2.66 1.54 1.59 1.61 1.60 1 S D- l SESP-a SESP-b 4.08 2.26 3.17 3.25 3.59 3. 12 SESP-a 3.35 2.26 2.80 3. 72 2.43 3.07 SESP-b SESP-a 1.05 3.69 2.37 1.44 2.98 2.21 1 SESP-b 1.17 3.00 2.09 0. 79 3.3.', 2.07 I SD-2 SESP-a SESP-b 2.83 1.82 2.:V2 • 2.12 2. 2 2.62 SESP-a 2.82 1.82 2.32 2.61 2.52 2.51 SESP-b SESP-a 1.28 1.12 1.20 1.02 1.80 1 .4 1 1 ESP-b 1.28 1.12 1.20 1.28 1.52 1.40 I S D-3 SES P-a SESP-b 4.86 1.66 3.26 2.80 1.89 2.35 ESP- 0.97 2.20 1.58 1.08 1.92 1.50 SESP-b SESP-a 1.65 2.44 2.05 1.76 3.11 2.43 1 SESP-b 0.57 2. 71 1.64 0.57 3.25 1.62 I - 17 - [2] S. YOUNG, J. JANSEN, J. ODELL, D. OLLASON, P. WOODLAND : The HTK BOOK, HTK 2.0 Manual. 1995. [3] R.O. DUDA, P.E. HART : Pattern Classification and Scene Analysis. John Wiley & Sons, 1973. [4] L.L. SCHARF : Statistical Signal Processing. Detection, Estimation and Tune Series Analysis. Addison- Wesley Publishing Company, 1991. [5] F. BIMBOT, D. GENOUD : Likelihood ratio adjustment for the compensation of model mismatch in speaker verification, Proc. Eurospeech '97, pp. 1387-1390

Claims: - 18 - CLAIMS 1. Speaker verification system, comprising receiving means (1) for receiving utterances of speakers during an enrolment process, during which a speaker (2) produces n tokens of some set of phrases, model building means (3) for building one or more models consisting of explicit or implicit sets of central tendencies and variances of all speech coefficients of utterances received via said receiving means, threshold means (4) for establishing accept/reject thresholds during said enrolment process, and estimating means (5), characterized in that said model building means (3) build n different speaker models, each based on n-1 tokens, for each model one independent token being available for estimating, by said estimating means «5 (5) , the distance between the model and an utterance that has not been used to build the model. 2. Speaker verification system according to claim 1, characterized in that said estimation means (5) estimate the central tendency of the distance between the speaker's o model and newly produced utterances of the same speaker, and also its variance. 3. Speaker verification system according to claim 1, in which the estimation of the accept/reject threshold from enrolment speech is combined, by combining means (6), with 5 pre-recorded impostor speech, characterized in that the central tendency of the distances between the model and utterances is optimised by optimising means (7) . 4. Speaker verification system according to claim 3, characterized in that said optimisation, in said o optimisation means (7) , is executed by linear interpolation: Th(new) = b * CTi + (1 -b) * CTt, where Th(new) is the optimal threshold, CTi is the central tendency obtained from pre-recorded impostor speech, CTt is the central tendency estimated from the enrolment speech of - 19 - the new customer, and b is the interpolation parameter, that is optimised using additional pre-recorded utterances not used in estimating Cti. 5. Speaker verification system according to claim 3, 5 characterized in that enrolment speech and pre-recorded impostor utterances are segmented, in said optimising means (7), into a large number of theoretically independent parts, for each of which the distance to the newly enrolled model is computed. no 6. Speaker verification system according to claim 5, characterized in that the central tendencies of the distance distributions are corrected, in said optimising means (7) , removing a bias caused by the fact that the enrolment speech has been used both for building the model - 5 and for computing the distances to the model. 7. Speaker verification system according to claim 3, characterized in that, in said optimising means (7), a single optimal value is computed that applies to all speakers. o 8. Speaker verification system according to claim 3, characterized in that, in said optimising means (7) , an optimal correction factor is estimated for each newly enrolled speaker.

Inventors: BOVES LODEWIJK WILLEM JOHAN, BOVES, LODEWIJK, WILLEM, JOHAN
Applicants: BOVES LODEWIJK WILLEM JOHAN, KONINKL KPN NV, BOVES, LODEWIJK, WILLEM, JOHAN, KONINKLIJKE KPN N.V.
Application year: 1999
Main classification: G10L5
IPC codes: G10L17