B: Is it starting now ?
E: Yep .
B: So what what from what B: Whatever we say from now on , it can be held against us ,
A: Hello ?
B: right ?
E: That 's right .
B: and uh
A: It 's your right to remain silent .
B: Yeah .
B: So I I the the problem is that I actually don't know how th these held meetings are held ,
B: if they are very informal and sort of just people are say what 's going on
E: Yeah .
B: and
E: Yeah , that 's usually what we do .
E: We just sorta go around and people say what 's going on , what 's the latest uh B: OK .
B: Yeah .
B: OK .
B: So I guess that what may be a reasonable is if I uh first make a report on what 's happening in Aurora in general , at least what from my perspective .
E: Yeah .
E: That would be great .
D: Uh o
B: And and uh so , I I think that Carmen and Stephane reported on uh Amsterdam meeting ,
B: which was kind of interesting
B: because it was for the first time we realized we are not friends really , but we are competitors .
B: Cuz until then it was sort of like everything was like wonderful
B: and E: Yeah .
E: It seemed like there were still some issues ,
B: Yeah .
E: right ?
E: that they were trying to decide ?
B: There is a plenty of there 're plenty of issues .
E: Like the voice activity detector ,
B: Well and what happened was that they realized that if two leading proposals , which was French Telecom Alcatel , and us both had uh voice activity detector .
E: Right .
B: And I said " well big surprise , I mean we could have told you that n n n four months ago , except we didn't because nobody else was bringing it up " .
B: Obviously French Telecom didn't volunteer this information either ,
E: Right .
B: cuz we were working on mainly on voice activity detector for past uh several months
B: because that 's buying us the most uh thing .
B: And everybody said " Well but this is not fair . We didn't know that . "
B: And of course uh the it 's not working on features really .
E: Right .
B: And be I agreed .
B: I said " well yeah ,
B: you are absolutely right ,
B: I mean if I wish that you provided better end point at speech because uh B: or at least that if we could modify the recognizer , uh to account for these long silences ,
B: because otherwise uh that that th that wasn't a correct thing . "
B: And so then ev ev everybody else says " well we should we need to do a new eval evaluation without voice activity detector ,
B: or we have to do something about it " .
E: Right .
B: And in principle I uh I we agreed .
E: Mm - hmm .
B: We said uh " yeah " .
B: Because uh B: but in that case , uh we would like to change the uh the algorithm
B: because uh if we are working on different data , we probably will use a different set of tricks .
E: Right .
B: But unfortunately nobody ever officially can somehow acknowledge that this can be done ,
B: because French Telecom was saying " no , no , no , now everybody has access to our code ,
B: so everybody is going to copy what we did . "
B: Yeah
B: well our argument was everybody ha has access to our code , and everybody always had access to our code .
B: We never uh uh denied that .
B: We thought that people are honest , that if you copy something and if it is protected protected by patent then you negotiate , or something ,
E: Yeah .
E: Right .
B: right ?
B: I mean , if you find our technique useful , we are very happy .
E: Right .
E: Mm - hmm .
B: But And French Telecom was saying " no , no , no ,
B: there is a lot of little tricks which uh sort of uh cannot be protected and you guys will take them , " which probably is also true .
B: I mean , you know , it might be that people will take uh uh th the algorithms apart and use the blocks from that .
B: But I somehow think that it wouldn't be so bad , as long as people are happy abou uh uh uh honest about it .
E: Yeah .
B: And I think they have to be honest in the long run , because winning proposal again uh what will be available th is will be a code .
E: Mm - hmm .
B: So the uh the people can go to code and say " well listen this is what you stole from me "
E: Right .
B: you know ?
E: Right .
B: " so let 's deal with that " .
B: So I don't see the problem .
B: The biggest problem of course is that f that Alcatel French Telecom cl claims " well we fulfilled the conditions .
B: We are the best .
B: Uh . We are the standard . "
B: And e and other people don't feel that ,
B: because they so they now decided that that is the whole thing will be done on well - endpointed data ,
B: essentially that somebody will endpoint the data based on clean speech ,
B: because most of this the SpeechDat - Car has the also close speaking mike and endpoints will be provided .
E: Mm - hmm .
E: Ah .
B: And uh we will run again B: still not clear if we are going to run the if we are allowed to run uh uh new algorithms ,
B: but I assume so .
B: Because uh we would fight for that , really .
B: uh but since uh u u n u B: at least our experience is that only endpointing a a mel cepstrum gets uh gets you twenty - one percent improvement overall and twenty - seven improvement on SpeechDat - Car
E: Hmm .
B: then obvious the database uh I mean the the the uh the baseline will go up .
E: Right .
B: And nobody can then achieve fifty percent improvement .
B: So they agreed that uh there will be a twenty - five percent improvement required on on uh h u m bad mis badly mismatched E: But wait a minute ,
E: I thought the endpointing really only helped in the noisy cases .
B: It uh E: Oh , but you still have that with the MFCC .
E: OK .
B: Y yeah .
E: Yeah .
B: Yeah but you have the same prob I mean MFCC basically has an enormous number of uh insertions .
E: Right .
E: Yeah .
E: Yeah .
E: Yeah .
B: And so , so now they want to say " we we will require fifty percent improvement only for well matched condition , and only twenty - five percent for the serial cases . "
E: Hmm .
B: And uh and they almost agreed on that except that it wasn't a hundred percent agreed .
B: And so last time uh during the meeting , I just uh brought up the issue ,
B: I said " well you know uh quite frankly I 'm surprised how lightly you are making these decisions
B: because this is a major decision .
B: For two years we are fighting for fifty percent improvement
B: and suddenly you are saying " oh no we we will do something less " ,
B: but maybe we should discuss that .
B: And everybody said " oh we discussed that and you were not a mee there "
B: and I said " well a lot of other people were not there because not everybody participates at these teleconferencing c things . "
B: Then they said " oh no no no because uh everybody is invited . "
B: However , there is only ten or fifteen lines , so people can't even con you know participate .
B: So eh they agreed , and so they said " OK , we will discuss that . "
B: Immediately Nokia uh raised the question
B: and they said " oh yeah we agree this is not good to to uh dissolve the uh uh the uh the criterion . "
E: Mm - hmm .
B: So now officially , Nokia is uh uh complaining and said they they are looking for support ,
B: uh I think QualComm is uh saying , too " we shouldn't abandon the fifty percent yet .
B: We should at least try once again , one more round . "
E: Mm - hmm .
E: Mm - hmm .
B: So this is where we are .
E: Hmm .
B: I hope that I hope that this is going to be a adopted .
B: Next Wednesday we are going to have uh another uh teleconferencing call , so we 'll see what uh where it goes .
E: So what about the issue of um the weights on the for the different systems ,
E: the well - matched , and medium - mismatched and B: Yeah ,
B: that 's what that 's a g very good uh point ,
B: because David says " well you know we ca we can manipulate this number by choosing the right weights anyways . "
E: Mm - hmm .
B: So while you are right but uh you know but
B: Uh yeah ,
B: if of course if you put a zero uh weight zero on a mismatched condition , or highly mismatched then then you are done .
E: Mm - hmm .
B: But weights were also deter already decided uh half a year ago .
E: And they 're the staying the same ?
B: So B: Well , of course people will not like it .
E: Mm - hmm .
B: Now What is happening now is that I th I think that people try to match the criterion to solution .
B: They have solution .
B: Now they want to make sure their criterion is E: Right .
B: And I think that this is not the right way .
E: Yeah .
B: Uh it may be that that B: Eventually it may ha may ha it may have to happen .
E: Mm - hmm .
B: But it 's should happen at a point where everybody feels comfortable that we did all what we could .
E: Mm - hmm .
B: And I don't think we did .
B: Basically , I think that that this test was a little bit bogus because of the data
B: and uh essentially there were these arbitrary decisions made , and and everything .
B: So , so so this is so this is where it is .
B: So what we are doing at OGI now is uh uh uh working basically on our parts which we I think a little bit neglected ,
B: like noise separation .
B: Uh so we are looking in ways is in uh which uh with which we can provide better initial estimate of the mel spectrum basically ,
B: which would be a l uh , f more robust to noise ,
B: and so far not much uh success .
E: Hmm .
B: We tried uh things which uh a long time ago Bill Byrne suggested ,
B: instead of using Fourier spectrum , from Fourier transform , use the spectrum from LPC model .
B: Their argument there was the LPC model fits the peaks of the spectrum , so it may be m naturally more robust in noise .
B: And I thought " well , that makes sense , " but so far we can't get much much out of it .
E: Hmm .
B: uh we may try some standard techniques like spectral subtraction and E: You haven't tried that yet ?
B: not not not much .
E: Hmm .
B: Or even I was thinking about uh looking back into these totally ad - hoc techniques
B: like for instance uh Dennis Klatt was suggesting uh the one way to uh deal with noisy speech is to add noise to everything .
E: Hmm !
B: So . <comment> I mean , uh uh add moderate amount of noise to all data .
E: Oh !
E: I see .
B: So that makes uh th any additive noise less addi less a a effective ,
B: right ?
E: Right .
B: Because you already uh had the noise uh in a B: And it was working at the time .
B: It was kind of like one of these things , you know , but
B: if you think about it , it 's actually pretty ingenious .
B: So well , you know , just take a take a spectrum and and and add of the constant , C , to every every value .
E: Well you 're you 're basically y
E: Yeah .
E: So you 're making all your training data more uniform .
B: Exactly .
B: And if if then if this data becomes noisy , it b it becomes eff effectively becomes less noisy basically .
E: Hmm .
B: But of course you cannot add too much noise because then you 'll s then you 're clean recognition goes down ,
B: but I mean it 's yet to be seen how much ,
B: it 's a very simple technique .
E: Mm - hmm .
B: Yes indeed it 's a very simple technique ,
B: you just take your spectrum and and use whatever is coming from FFT , add constant ,
E: Hmm .
B: you know ?
B: on onto power spectrum .
B: That that B: Or the other thing is of course if you have a spectrum , what you can s start doing , you can leave start leaving out the p the parts which are uh uh low in energy
B: and then perhaps uh one could try to find a a all - pole model to such a spectrum .
B: Because a all - pole model will still try to to to put the the continuation basically of the of the model into these parts where the issue set to zero .
B: That 's what we want to try .
B: I have a visitor from Brno .
B: He 's a kind of like young faculty .
B: pretty hard - working so he so he 's so he 's looking into that .
E: Hmm .
B: And then most of the effort is uh now also aimed at this e e TRAP recognition .
B: This uh this is this recognition from temporal patterns .
E: Hmm !
E: What is that ?
B: Ah , you don't know about TRAPS !
A: Hmm .
E: The TRAPS sound familiar , I but I don't B: Yeah
B: I mean tha
B: This is familiar like sort of because we gave you the name ,
B: but , what it is , is that normally what you do is that you recognize uh speech based on a shortened spectrum .
E: Mm - hmm .
E: Mm - hmm .
B: Essentially L P - LPC , mel cepstrum , uh , everything starts with a spectral slice .
B: Uh so if you s So , given the spectrogram you essentially are sliding sliding the spectrogram along the uh f frequency axis
E: Mm - hmm .
B: and you keep shifting this thing , and you have a spectrogram .
E: Mm - hmm .
B: So you can say " well you can also take the time trajectory of the energy at a given frequency " ,
E: Mm - hmm .
B: and what you get is then , that you get a p vector .
B: And this vector can be a a s assigned to s some phoneme .
B: Namely you can say i it I will I will say that this vector will eh will will describe the phoneme which is in the center of the vector .
B: And you can try to classify based on that .
E: Hmm .
B: And you so you classi
B: so it 's a very different vector , very different properties ,
B: we don't know much about it ,
E: Hmm .
B: but the truth is E: But you have many of those vectors per phoneme ,
B: Well , so you get many decisions .
E: right ?
E: Uh - huh .
B: And then you can start dec thinking about how to combine these decisions .
E: Hmm .
B: Exactly , that 's what yeah , that 's what it is .
E: Hmm .
B: Because if you run this uh recognition , you get you still get about twenty percent error B: uh twenty percent correct .
E: Hmm .
B: You know ,
B: on on like for the frame by frame basis ,
B: so uh uh so it 's much better than chance .
E: How wide are the uh frequency bands ?
B: That 's another thing .
B: Well c currently we start I mean we start always with critical band spectrum .
B: For various reasons .
B: But uh the latest uh observation uh is that you you you are you can get quite a big advantage of using two critical bands at the same time .
A: Are they adjacent ,
A: or are they s
B: Adjacent , adjacent .
A: OK .
B: And the reasons there are some reasons for that .
B: Because there are some reasons I can I could talk about , will have to tell you about things like masking experiments which uh uh uh uh yield critical bands ,
B: and also experiments with release of masking , which actually tell you that something is happening across critical bands , across bands .
B: And E: Well how do you how do you uh convert this uh energy over time in a particular frequency band into a vector of numbers ?
B: It 's uh uh uh I mean time T - zero is one number , time t
E: Yeah but what 's the number ?
E: Is it just the B: It 's a spectral energy , logarithmic spectral energy ,
E: it 's just the amount of energy in that band from f in that time interval .
B: yeah .
B: Yes , yes .
B: Yes , yes .
E: OK .
B: And that 's what that 's what I 'm saying then ,
B: so this is a this is a starting vector .
B: It 's just like shortened f spectrum , or something .
E: Mm - hmm .
B: But now we are trying to understand what this vector actually represents ,
B: for instance a question is like " how correlated are the elements of this vector ? "
B: Turns out they are quite correlated , because I mean , especially the neighboring ones ,
E: Yeah .
B: right ?
B: They they represent the same almost the same configuration of the vocal tract .
E: Yeah .
E: Mm - hmm .
B: So there 's a very high correlation .
B: So the classifiers which use the diagonal covariance matrix don't like it .
B: So we 're thinking about de - correlating them .
E: Hmm .
B: Then the question is uh " can you describe elements of this vector by Gaussian distributions " , or to what extent ?
B: Because uh B: And and and so on and so on .
B: So we are learning quite a lot about that .
E: Hmm .
B: And then another issue is how many vectors we should be using ,
B: I mean the so the minimum is one .
E: Mm - hmm .
B: But I mean is the is the critical band the right uh uh dimension ?
B: So we somehow made arbitrary decision , " yes " .
B: Then but then now we are thinking a lot how to uh how to use at least the neighboring band because that seems to be happening B: This I somehow start to believe that 's what 's happening in recognition .
B: Cuz a lot of experiments point to the fact that people can split the signal into critical bands ,
B: but then oh uh uh
B: so you can you are quite capable of processing a signal in uh uh independently in individual critical bands .
B: That 's what masking experiments tell you .
B: But at the same time you most likely pay attention to at least neighboring bands
B: when you are making any decisions , you compare what 's happening in in this band to what 's happening to the band to to to the to the neighboring bands .
B: And that 's how you make uh decisions .
B: That 's why the articulatory events , which uh F F Fletcher talks about , they are about two critical bands .
B: You need at least two , basically .
B: You need some relative , relative relation .
A: Hmm .
E: Hmm .
B: Absolute number doesn't tell you the right thing .
B: You need to you need to compare it to something else , what 's happening
B: but it 's what 's happening in the in the close neighborhood .
B: So if you are making decision what 's happening at one kilohertz , you want to know what 's happening at nine hundred hertz and it and maybe at eleven hundred hertz ,
B: but you don't much care what 's happening at three kilohertz .
E: So it 's really w
E: It 's sort of like saying that what 's happening at one kilohertz depends on what 's happening around it .
E: It 's sort of relative to it .
B: To some extent , it that is also true .
E: Mm - hmm .
B: Yeah .
B: But it 's but for but for instance , th uh uh what what uh humans are very much capable of doing is that if th if they are exactly the same thing happening in two neighboring critical bands , recognition can discard it .
E: Hmm .
B: Is what 's happening B: Hey !
A: Hey !
B: OK , we need us another another voice here .
E: Hey Stephane .
B: Yeah ,
B: I think so .
E: Yep .
B: Yeah ?
E: Sure .
E: Go ahead .
B: And so so so for instance if you d if you a if you add the noise that normally masks masks the uh the the signal
E: Mm - hmm .
B: right ?
B: and you can show that in that if the if you add the noise outside the critical band , that doesn't affect the the decisions you 're making about a signal within a critical band .
E: Hmm .
B: Unless this noise is modulated .
B: If the noise is modulated , with the same modulation frequency as the noise in a critical band , the amount of masking is less .
E: Mmm .
B: The moment you moment you provide the noise in n neighboring critical bands .
B: So the s m masking curve , normally it looks like sort of B: I start from from here ,
B: so you <comment> you have uh no noise then you you you are expanding the critical band , so the amount of maching is increasing .
B: And when you e hit a certain point , which is a critical band , then the amount of masking is the same .
E: Mmm .
B: So that 's the famous experiment of Fletcher , a long time ago .
B: Like that 's where people started thinking " wow this is interesting ! "
E: Yeah .
B: So .
B: But , if you if you if you modulate the noise , the masking goes up and the moment you start hitting the another critical band , the masking goes down .
B: So essentially essentially that 's a very clear indication that that that cognition can take uh uh into consideration what 's happening in the neighboring bands .
B: But if you go too far in a in a if you if the noise is very broad , you are not increasing much more ,
B: so so if you if you are far away from the signal uh from the signal f uh the frequency at which the signal is , then the m even the when the noise is co - modulated it it 's not helping you much .
E: Mm - hmm .
E: Yeah .
E: Mm - hmm .
B: So .
A: Hmm .
B: So things like this we are kind of playing with with with the hope that perhaps we could eventually u use this in a in a real recognizer .
A: Mm - hmm .
B: Like uh partially of course we promised to do this under the the the Aurora uh program .
E: But you probably won't have anything before the next time we have to evaluate ,
E: right ?
B: Probably not .
E: Yeah .
B: Well , maybe , most likely we will not have anything which c would comply with the rules .
E: Ah .
B: like because uh uh
E: Latency and things .
B: latency currently chops the require uh significant uh latency amount of processing ,
E: Mm - hmm .
B: because uh we don't know any better , yet , than to use the neural net classifiers , uh and uh and uh TRAPS .