Professor B: Yeah . OK . So I guess that what may be a reasonable is if I uh first make a report on what 's happening in Aurora in general , at least what from my perspective .
PhD E: Yeah . That would be great .
Professor B: And and uh so , I I think that Carmen and Stephane reported on uh Amsterdam meeting ,
PhD D: Uh o
Professor B: which was kind of interesting because it was for the first time we realized we are not friends really , but we are competitors . Cuz until then it was sort of like everything was like wonderful and PhD E: Yeah . It seemed like there were still some issues ,
Professor B: Yeah .
PhD E: right ? that they were trying to decide ?
Professor B: There is a plenty of there 're plenty of issues .
PhD E: Like the voice activity detector ,
Professor B: Well and what happened was that they realized that if two leading proposals , which was French Telecom Alcatel , and us both had uh voice activity detector . And I said " well big surprise , I mean we could have told you that n n n four months ago , except we didn't because nobody else was bringing it up " .
PhD E: Right .
Professor B: Obviously French Telecom didn't volunteer this information either , cuz we were working on mainly on voice activity detector for past uh several months
PhD E: Right .
Professor B: because that 's buying us the most uh thing . And everybody said " Well but this is not fair . We didn't know that . " And of course uh the it 's not working on features really . And be I agreed .
PhD E: Right .
Professor B: I said " well yeah , you are absolutely right , I mean if I wish that you provided better end point at speech because uh or at least that if we could modify the recognizer , uh to account for these long silences , because otherwise uh that that th that wasn't a correct thing . " And so then ev ev everybody else says " well we should we need to do a new eval evaluation without voice activity detector , or we have to do something about it " .
PhD E: Right .
Professor B: And in principle I uh I we agreed .
PhD E: Mm - hmm .
Professor B: We said uh " yeah " . Because uh but in that case , uh we would like to change the uh the algorithm because uh if we are working on different data , we probably will use a different set of tricks .
PhD E: Right .
Professor B: But unfortunately nobody ever officially can somehow acknowledge that this can be done , because French Telecom was saying " no , no , no , now everybody has access to our code , so everybody is going to copy what we did . " Yeah well our argument was everybody ha has access to our code , and everybody always had access to our code . We never uh uh denied that . We thought that people are honest , that if you copy something and if it is protected protected by patent then you negotiate , or something ,
PhD E: Yeah . Right .
Professor B: right ? I mean , if you find our technique useful , we are very happy .
PhD E: Right .
Professor B: But And French Telecom was saying " no , no , no ,
PhD E: Mm - hmm .
Professor B: there is a lot of little tricks which uh sort of uh cannot be protected and you guys will take them , " which probably is also true . I mean , you know , it might be that people will take uh uh th the algorithms apart and use the blocks from that . But I somehow think that it wouldn't be so bad , as long as people are happy abou uh uh uh honest about it .
PhD E: Yeah .
Professor B: And I think they have to be honest in the long run , because winning proposal again uh what will be available th is will be a code . So the uh the people can go to code and say " well listen this is what you stole from me "
PhD E: Mm - hmm .
Professor B: you know ?
PhD E: Right .
Professor B: " so let 's deal with that " .
PhD E: Right .
Professor B: So I don't see the problem . The biggest problem of course is that f that Alcatel French Telecom cl claims " well we fulfilled the conditions . We are the best . Uh . We are the standard . " And e and other people don't feel that , because they so they now decided that that is the whole thing will be done on well - endpointed data , essentially that somebody will endpoint the data based on clean speech , because most of this the SpeechDat - Car has the also close speaking mike and endpoints will be provided .
PhD E: Mm - hmm . Ah .
Professor B: And uh we will run again still not clear if we are going to run the if we are allowed to run uh uh new algorithms , but I assume so . Because uh we would fight for that , really . uh but since uh u u n u at least our experience is that only endpointing a a mel cepstrum gets uh gets you twenty - one percent improvement overall and twenty - seven improvement on SpeechDat - Car
PhD E: Hmm .
Professor B: then obvious the database uh I mean the the the uh the baseline will go up . And nobody can then achieve fifty percent improvement .
PhD E: Right .
Professor B: So they agreed that uh there will be a twenty - five percent improvement required on on uh h u m bad mis badly mismatched PhD E: But wait a minute , I thought the endpointing really only helped in the noisy cases .
Professor B: It uh PhD E: Oh , but you still have that with the MFCC .
Professor B: Y yeah .
PhD E: OK .
Professor B: Yeah but you have the same prob I mean MFCC basically has an enormous number of uh insertions .
PhD E: Yeah . Right . Yeah . Yeah . Yeah .
Professor B: And so , so now they want to say " we we will require fifty percent improvement only for well matched condition , and only twenty - five percent for the serial cases . "
PhD E: Hmm .
Professor B: And uh and they almost agreed on that except that it wasn't a hundred percent agreed . And so last time uh during the meeting , I just uh brought up the issue , I said " well you know uh quite frankly I 'm surprised how lightly you are making these decisions because this is a major decision . For two years we are fighting for fifty percent improvement and suddenly you are saying " oh no we we will do something less " , but maybe we should discuss that . And everybody said " oh we discussed that and you were not a mee there " and I said " well a lot of other people were not there because not everybody participates at these teleconferencing c things . " Then they said " oh no no no because uh everybody is invited . " However , there is only ten or fifteen lines , so people can't even con you know participate . So eh they agreed , and so they said " OK , we will discuss that . " Immediately Nokia uh raised the question and they said " oh yeah we agree this is not good to to uh dissolve the uh uh the uh the criterion . "
PhD E: Mm - hmm .
Professor B: So now officially , Nokia is uh uh complaining and said they they are looking for support , uh I think QualComm is uh saying , too " we shouldn't abandon the fifty percent yet . We should at least try once again , one more round . "
PhD E: Mm - hmm .
Professor B: So this is where we are .
PhD E: Mm - hmm .
Professor B: I hope that I hope that this is going to be a adopted .
PhD E: Hmm .
Professor B: Next Wednesday we are going to have uh another uh teleconferencing call , so we 'll see what uh where it goes .
PhD E: So what about the issue of um the weights on the for the different systems , the well - matched , and medium - mismatched and Professor B: Yeah , that 's what that 's a g very good uh point , because David says " well you know we ca we can manipulate this number by choosing the right weights anyways . " So while you are right but uh you know but
PhD E: Mm - hmm .
Professor B: Uh yeah , if of course if you put a zero uh weight zero on a mismatched condition , or highly mismatched then then you are done .
PhD E: Mm - hmm .
Professor B: But weights were also deter already decided uh half a year ago . So PhD E: And they 're the staying the same ?
Professor B: Well , of course people will not like it . Now What is happening now is that I th I think that people try to match the criterion to solution .
PhD E: Mm - hmm .
Professor B: They have solution . Now they want to make sure their criterion is PhD E: Right .
Professor B: And I think that this is not the right way .
PhD E: Yeah .
Professor B: Uh it may be that that Eventually it may ha may ha it may have to happen . But it 's should happen at a point where everybody feels comfortable that we did all what we could .
PhD E: Mm - hmm .
Professor B: And I don't think we did .
PhD E: Mm - hmm .
Professor B: Basically , I think that that this test was a little bit bogus because of the data and uh essentially there were these arbitrary decisions made , and and everything . So , so so this is so this is where it is . So what we are doing at OGI now is uh uh uh working basically on our parts which we I think a little bit neglected , like noise separation . Uh so we are looking in ways is in uh which uh with which we can provide better initial estimate of the mel spectrum basically , which would be a l uh , f more robust to noise , and so far not much uh success .
PhD E: Hmm .
Professor B: We tried uh things which uh a long time ago Bill Byrne suggested , instead of using Fourier spectrum , from Fourier transform , use the spectrum from LPC model . Their argument there was the LPC model fits the peaks of the spectrum , so it may be m naturally more robust in noise . And I thought " well , that makes sense , " but so far we can't get much much out of it .
PhD E: Hmm .
Professor B: uh we may try some standard techniques like spectral subtraction and PhD E: You haven't tried that yet ?
Professor B: not not not much . Or even I was thinking about uh looking back into these totally ad - hoc techniques
PhD E: Hmm .
Professor B: like for instance uh Dennis Klatt was suggesting uh the one way to uh deal with noisy speech is to add noise to everything .
PhD E: Hmm !
Professor B: So . {comment} I mean , uh uh add moderate amount of noise to all data .
PhD E: Oh !
Professor B: So that makes uh th any additive noise less addi less a a effective ,
PhD E: I see .
Professor B: right ? Because you already uh had the noise uh in a PhD E: Right .
Professor B: And it was working at the time . It was kind of like one of these things , you know , but if you think about it , it 's actually pretty ingenious . So well , you know , just take a take a spectrum and and and add of the constant , C , to every every value .
PhD E: Well you 're you 're basically y Yeah . So you 're making all your training data more uniform .
Professor B: Exactly . And if if then if this data becomes noisy , it b it becomes eff effectively becomes less noisy basically .
PhD E: Hmm .
Professor B: But of course you cannot add too much noise because then you 'll s then you 're clean recognition goes down , but I mean it 's yet to be seen how much , it 's a very simple technique .
PhD E: Mm - hmm .
Professor B: Yes indeed it 's a very simple technique , you just take your spectrum and and use whatever is coming from FFT , add constant ,
PhD E: Hmm .
Professor B: you know ? on onto power spectrum . That that Or the other thing is of course if you have a spectrum , what you can s start doing , you can leave start leaving out the p the parts which are uh uh low in energy and then perhaps uh one could try to find a a all - pole model to such a spectrum . Because a all - pole model will still try to to to put the the continuation basically of the of the model into these parts where the issue set to zero . That 's what we want to try . I have a visitor from Brno . He 's a kind of like young faculty . pretty hard - working so he so he 's so he 's looking into that .
PhD E: Hmm .
Professor B: And then most of the effort is uh now also aimed at this e e TRAP recognition . This uh this is this recognition from temporal patterns .
PhD E: Hmm ! What is that ?
Professor B: Ah , you don't know about TRAPS !
Grad A: Hmm .
PhD E: The TRAPS sound familiar , I but I don't Professor B: Yeah I mean tha This is familiar like sort of because we gave you the name , but , what it is , is that normally what you do is that you recognize uh speech based on a shortened spectrum .
PhD E: Mm - hmm . Mm - hmm .
Professor B: Essentially L P - LPC , mel cepstrum , uh , everything starts with a spectral slice . Uh so if you s So , given the spectrogram you essentially are sliding sliding the spectrogram along the uh f frequency axis
PhD E: Mm - hmm .
Professor B: and you keep shifting this thing , and you have a spectrogram .
PhD E: Mm - hmm .
Professor B: So you can say " well you can also take the time trajectory of the energy at a given frequency " , and what you get is then , that you get a p vector .
PhD E: Mm - hmm .
Professor B: And this vector can be a a s assigned to s some phoneme . Namely you can say i it I will I will say that this vector will eh will will describe the phoneme which is in the center of the vector . And you can try to classify based on that .
PhD E: Hmm .
Professor B: And you so you classi so it 's a very different vector , very different properties , we don't know much about it , but the truth is PhD E: Hmm . But you have many of those vectors per phoneme ,
Professor B: Well , so you get many decisions .
PhD E: right ? Uh - huh .
Professor B: And then you can start dec thinking about how to combine these decisions . Exactly , that 's what yeah , that 's what it is .
PhD E: Hmm . Hmm .
Professor B: Because if you run this uh recognition , you get you still get about twenty percent error uh twenty percent correct . You know ,
PhD E: Hmm .
Professor B: on on like for the frame by frame basis , so uh uh so it 's much better than chance .
PhD E: How wide are the uh frequency bands ?
Professor B: That 's another thing . Well c currently we start I mean we start always with critical band spectrum . For various reasons . But uh the latest uh observation uh is that you you you are you can get quite a big advantage of using two critical bands at the same time .
Grad A: Are they adjacent , or are they s
Professor B: Adjacent , adjacent .
Grad A: OK .
Professor B: And the reasons there are some reasons for that . Because there are some reasons I can I could talk about , will have to tell you about things like masking experiments which uh uh uh uh yield critical bands , and also experiments with release of masking , which actually tell you that something is happening across critical bands , across bands . And PhD E: Well how do you how do you uh convert this uh energy over time in a particular frequency band into a vector of numbers ?
Professor B: It 's uh uh uh I mean time T - zero is one number , time t
PhD E: Yeah but what 's the number ? Is it just the Professor B: It 's a spectral energy , logarithmic spectral energy ,
PhD E: it 's just the amount of energy in that band from f in that time interval .
Professor B: yeah . Yes , yes . Yes , yes .
PhD E: OK .
Professor B: And that 's what that 's what I 'm saying then , so this is a this is a starting vector . It 's just like shortened f spectrum , or something . But now we are trying to understand what this vector actually represents ,
PhD E: Mm - hmm .
Professor B: for instance a question is like " how correlated are the elements of this vector ? " Turns out they are quite correlated , because I mean , especially the neighboring ones , right ? They they represent the same almost the same configuration of the vocal tract .
PhD E: Yeah . Yeah . Mm - hmm .
Professor B: So there 's a very high correlation . So the classifiers which use the diagonal covariance matrix don't like it . So we 're thinking about de - correlating them .
PhD E: Hmm .
Professor B: Then the question is uh " can you describe elements of this vector by Gaussian distributions " , or to what extent ? Because uh And and and so on and so on . So we are learning quite a lot about that . And then another issue is how many vectors we should be using ,
PhD E: Hmm .
Professor B: I mean the so the minimum is one .
PhD E: Mm - hmm .
Professor B: But I mean is the is the critical band the right uh uh dimension ? So we somehow made arbitrary decision , " yes " . Then but then now we are thinking a lot how to uh how to use at least the neighboring band because that seems to be happening This I somehow start to believe that 's what 's happening in recognition . Cuz a lot of experiments point to the fact that people can split the signal into critical bands , but then oh uh uh so you can you are quite capable of processing a signal in uh uh independently in individual critical bands . That 's what masking experiments tell you . But at the same time you most likely pay attention to at least neighboring bands when you are making any decisions , you compare what 's happening in in this band to what 's happening to the band to to to the to the neighboring bands . And that 's how you make uh decisions . That 's why the articulatory events , which uh F F Fletcher talks about , they are about two critical bands . You need at least two , basically . You need some relative , relative relation .
Grad A: Hmm .
Professor B: Absolute number doesn't tell you the right thing .
PhD E: Hmm .
Professor B: You need to you need to compare it to something else , what 's happening but it 's what 's happening in the in the close neighborhood . So if you are making decision what 's happening at one kilohertz , you want to know what 's happening at nine hundred hertz and it and maybe at eleven hundred hertz , but you don't much care what 's happening at three kilohertz .
PhD E: So it 's really w It 's sort of like saying that what 's happening at one kilohertz depends on what 's happening around it . It 's sort of relative to it .
Professor B: To some extent , it that is also true . Yeah . But it 's but for but for instance , th uh uh what what uh humans are very much capable of doing is that if th if they are exactly the same thing happening in two neighboring critical bands , recognition can discard it .
PhD E: Mm - hmm .
Professor B: Is what 's happening PhD E: Hmm .
Grad A: Hey !
Professor B: Hey ! OK , we need us another another voice here .
PhD E: Hey Stephane .
Professor B: Yeah , I think so . Yeah ?
PhD E: Yep . Sure . Go ahead .
Professor B: And so so so for instance if you d if you a if you add the noise that normally masks masks the uh the the signal right ?
PhD E: Mm - hmm .
Professor B: and you can show that in that if the if you add the noise outside the critical band , that doesn't affect the the decisions you 're making about a signal within a critical band .
PhD E: Hmm .
Professor B: Unless this noise is modulated . If the noise is modulated , with the same modulation frequency as the noise in a critical band , the amount of masking is less . The moment you moment you provide the noise in n neighboring critical bands .
PhD E: Mmm .
Professor B: So the s m masking curve , normally it looks like sort of I start from from here , so you {comment} you have uh no noise then you you you are expanding the critical band , so the amount of maching is increasing . And when you e hit a certain point , which is a critical band , then the amount of masking is the same .
PhD E: Mmm .
Professor B: So that 's the famous experiment of Fletcher , a long time ago . Like that 's where people started thinking " wow this is interesting ! " So .
PhD E: Yeah .
Professor B: But , if you if you if you modulate the noise , the masking goes up and the moment you start hitting the another critical band , the masking goes down . So essentially essentially that 's a very clear indication that that that cognition can take uh uh into consideration what 's happening in the neighboring bands . But if you go too far in a in a if you if the noise is very broad , you are not increasing much more , so so if you if you are far away from the signal uh from the signal f uh the frequency at which the signal is , then the m even the when the noise is co - modulated it it 's not helping you much .
PhD E: Mm - hmm . Yeah . Mm - hmm .
Grad A: Hmm .
Professor B: So . So things like this we are kind of playing with with with the hope that perhaps we could eventually u use this in a in a real recognizer .
Grad A: Mm - hmm .
Professor B: Like uh partially of course we promised to do this under the the the Aurora uh program .
PhD E: But you probably won't have anything before the next time we have to evaluate ,
Professor B: Probably not .
PhD E: right ?
Professor B: Well , maybe , most likely we will not have anything which c would comply with the rules .
PhD E: Yeah . Ah .
Professor B: like because uh uh
PhD E: Latency and things .
Professor B: latency currently chops the require uh significant uh latency amount of processing ,
PhD E: Mm - hmm .
Professor B: because uh we don't know any better , yet , than to use the neural net classifiers , uh and uh and uh TRAPS .
PhD E: Yeah .
Grad A: Mm - hmm .
Professor B: Though the the work which uh everybody is looking at now aims at s trying to find out what to do with these vectors , so that a g simple Gaussian classifier would be happier with it .
PhD C: Hmm .
PhD E: Mm - hmm .
Professor B: or to what extent a Gaussian classifier should be unhappy uh that , and how to Gaussian - ize the vectors , and PhD E: Hmm .