What are we talking about today ? phd e: well , first there are perhaps these  Meeting Recorder digits that we tested . professor a: And for one thing that that sure shows the difference between having a lot of  training data or not , phd e: Of data ? Yeah . professor a: the  The best kind of number we have on the English  on near microphone only is is  three or four percent . professor a: And  it 's significantly better than that , using fairly simple front - ends on on the   , with the SRI system . professor a: So I th I think that the  But that 's that 's using  a a pretty huge amount of data , mostly not digits , of course , but but then again Well , yeah . In fact , mostly not digits for the actual training the H M Ms whereas  in this case we 're just using digits for training the H M phd e: Yeah . professor a: Did anybody mention about whether the the SRI system is a is is doing the digits  the wor as a word model or as  a sub s sub - phone states ? phd e: I guess it 's it 's  allophone models , professor a: Yeah . There is one difference Well , the SRI system the result for the SRI system that are represented here are with adaptation . So there is It 's their complete system and including on - line  unsupervised adaptation . phd e: And if you don't use adaptation , the error rate is around fifty percent worse , I think , if I remember . professor a: It 's tha it 's that much , huh ? phd e: Nnn . professor a: But but  what what I think I 'd be interested to do given that , is that we we should  take I guess that somebody 's gonna do this , right ? is to take some of these tandem things and feed it into the SRI system , right ? phd e: Yeah . Our back - end is is fairly simple but until now , well , the attempts to improve it or have fail Ah , well ,   what Chuck tried to to to do professor a: Yeah , but he 's doing it with the same data , right ?  so to So there 's there 's there 's two things being affected . One is that that , you know , there 's something simple that 's wrong with the back - end . professor a:  I I don't know if he got to the point of playing with the  number of Gaussians yet phd e:   So , yeah , we could retrain some of these tandem on on huge professor a: Well , you could do that , but I 'm saying even with it not with that part not retrained , just just using having the H M Ms much better H M phd e: Ah , yeah . But what would be interesting to see also is what what perhaps it 's not related , the amount of data but the  recording conditions . Because it 's probably not a problem of noise , because our features are supposed to be robust to noise . phd e: It 's not a problem of channel , because there is  normalization with respect to the channel . What what is the problem that you 're trying to explain ? phd e: The the fact that the result with the tandem and Aurora system are  so much worse . professor a: I  but I 'm I 'm almost certain that it it  , that it has to do with the  amount of training data . professor a: But but having a huge If if you look at what commercial places do , they use a huge amount of data .  , ordinarily you would say " well , given that you have enough occurrences of the digits , you can just train with digits rather than with , you know " phd e:   professor a: But the thing is , if you have a huge in other words , do word models But if you have a huge amount of data then you 're going to have many occurrences of similar  allophones . professor a: So it 's  I I think it has to be that , because , as you say , this is , you know , this is near - microphone , phd e:   Now , some of it could be the fact that  let 's see , in the in these multi - train things did we include noisy data in the training ? phd e: Yeah . Well , actually we see that the clean train for the Aurora proposals are are better than the multi - train , professor a: It is if Yeah . phd e: Well , o I guess what I meant is that well , let 's say if we if we add enough data to train on the  on the Meeting Recorder digits , I guess we could have better results than this . What I meant is that perhaps we can learn something  from this , what 's what 's wrong  what what is different between TI - digits and these digits and professor a: What kind of numbers are we getting on TI - digits ? phd e: It 's point eight percent , so . professor a: So in the actual TI - digits database we 're getting point eight percent , phd e: Yeah . professor a: and here we 're getting three or four three , let 's see , three for this ? phd e:   Sure , but  ,  point eight percent is something like double  or triple what people have gotten who 've worked very hard at doing that . professor a: And and also , as you point out , there 's adaptation in these numbers also . So if you , you know , put the ad adap take the adaptation off , then it for the English - Near you get something like two percent . professor a: It 's , you know , we used a simple HTK system with a modest amount of data . And this is a a , you know , modern  system  has has a lot of nice points to it . professor a: But to me it just it just meant a practical point that  if we want to publish results on digits that that people pay attention to we probably should  Cuz we 've had the problem before that you get show some nice improvement on something that 's that 's  ,  it seems like too large a number , and   people don't necessarily take it so seriously . So the three point four percent for this  is is  So why is it It 's an interesting question though , still . Why is why is it three point four percent for the d the digits recorded in this environment as opposed to the  point eight percent for for for the original TI - digits database ?  . So ignore ignoring the the the SRI system for a moment , phd e: I I I don't I   professor a: just looking at the TI - di the  tandem system , if we 're getting point eight percent , which , yes , it 's high . It 's , you know , it it 's not awfully high , phd e:   professor a: Right ?  , there 's even though it 's close - miked there 's still there really is background noise . And  I suspect when the TI - digits were recorded if somebody fumbled or said something wrong or something that they probably made them take it over . professor a: It was not  there was no attempt to have it be realistic in any in any sense at all . TI - digit is it 's very , very clean and it 's like studio recording professor a:   But professor a: It 's I think it 's it 's the indication it 's harder .  , they 're much much better , but still you 're getting something like one point three percent for  things that are same data as in T TI - digits the same same text . And  , I 'm sure the same same system would would get , you know , point point three or point four or something on the actual TI - digits . professor a: Which I find sort of interesting cause I think this is closer to   it 's still read . But I still think it 's much closer to to what what people actually face ,  when they 're they 're dealing with people saying digits over the telephone . I don't think   , I 'm sure they wouldn't release the numbers , but I don't think that  the  the the companies that that do telephone speech get anything like point four percent on their digits . I 'm I 'm I 'm sure they get  , for one thing people do phone up who don't have   Middle America accents and it 's a we we it 's it 's it 's US . professor a: Did we end up giving up on on , any Eurospeech submissions , phd e: But professor a: or ? I know Thilo and Dan Ellis are are submitting something , but  . I I guess e the only thing with these the Meeting Recorder and , well , So , I think , yeah I think we basically gave up . Now , actually for the for the Aur -  phd e: But professor a: we do have stuff for Aurora , right ? Because because we have ano an extra month or something . professor a: and we have We don't we don't have to flood it with papers . Perhaps the point is that we 've been working on is , yeah , we have put the  the good VAD in the system and it really makes a huge difference . I think , yeah , this is perhaps one of the reason why our system was not not the best , because with the new VAD , it 's very the results are similar to the France Telecom results and perhaps even better sometimes . The problem is that it 's very big and we still have to think how to where to put it and  , professor a:   phd e: because it it well , this VAD  either some delay and we if we put it on the server side , it doesn't work , because on the server side features you already have LDA applied from the f from the terminal side and so you accumulate the delay so the VAD should be before the LDA which means perhaps on the terminal side and then smaller and professor a: So wha where did this good VAD come from ? phd e: So . So it 's the network trained it 's the network with the huge amounts on hidden of hidden units , and  nine input frames compared to the VAD that was in the proposal which has a very small amount of hidden units and fewer inputs . Yeah , but they had to get rid of it because of the space , didn't they ? phd e: Yeah . But the abso assumption is that we will be able to make a VAD that 's small and that works fine .  ,  i if if there 's a if if I I don't know what the thinking was amongst the the the the ETSI folk but  if everybody agreed sure let 's use this VAD and take that out of there phd e:   They just want , apparently they don't want to fix the VAD because they think there is some interaction between feature extraction and and VAD or frame dropping But they still want to just to give some  requirement for this VAD because it 's it will not be part of they don't want it to be part of the standard . But I was thinking that that  s " Sure , there may be some interaction , phd e: Nnn . professor a: but I don't think we need to be stuck on using our or OGI 's VAD .  I designed a new a new filter because when I designed other filters with shorter delay from the LDA filters , there was one filter with fif sixty millisecond delay and the other with ten milliseconds professor a: Right . phd e: and  Hynek suggested that both could have sixty - five sixty - s I think it 's sixty - five . Both should have sixty - five because professor a: You didn't gain anything , right ? phd e: Yeah . phd e: Yeah , and then we 've started to work with this of  voiced - unvoiced stuff . phd e: And next week I think we will perhaps try to have  a new system with   MSG stream also see what what happens . phd d: No , I w I begin to play with Matlab and to found some parameter robust for voiced - unvoiced decision . And we they we found that maybe w is a classical parameter , the sq the variance between the  FFT of the signal and the small spectrum of time we after the  mel filter bank . So , basically we wa want to look at something like the ex the ex excitation signal and professor a: Right . phd d: The the mix of the two , noise and unnoise , and the signal is this . professor a: Well , I 'm s  There 's None of these axes are labeled , so I don't know what this What 's this axis ? phd d:  this is  this axis is nnn , " frame " . professor a: And what 's th what this ? phd d: this is  energy , log - energy of the spectrum . Of the this is the variance , the difference between the spectrum of the signal and FFT of each frame of the signal and this mouth spectrum of time after the f may fit for the two , professor a: For this one . phd d: Pity , but I don't have two different professor a: And presumably when there 's a a phd e: So this should the the the t voiced portions . This is trying to obtain with LPC model the spectrum but using Matlab without going factor and s professor a: No pre - emphasis ? Yeah . So now I wonder  , do you want to I know you want to get at something orthogonal from what you get with the smooth spectrum  . But if you were to really try and get a voiced - unvoiced , do you do you want to totally ignore that ?  , do you do you  , clearly a a very big very big cues for voiced - unvoiced come from  spectral slope and so on , right ? phd e:   phd d: if s @ @ val value is indicative that is a voice frame and it 's low values professor a: Yeah . Well , you probably want  , certainly if you want to do good voiced - unvoiced detection , you need a few features . But , you know , people look at at slope and  first auto - correlation coefficient , divided by power . professor a: Or or   there 's  I guess we prob probably don't have enough computation to do a simple pitch detector or something ?  with a pitch detector you could have a have a an estimate of of what the phd e: Mmm . Or maybe you could you just do it going through the P FFT 's figuring out some  probable  harmonic structure . phd d: you have read up and you have a paper , the paper that you s give me yesterday . But Yeah , but it 's not it 's , yeah , it 's it 's another problem . phd e: What 's this again ? Is it the mel - filters ? phd d: Yeah like this . phd e: and what we clearly see is that in some cases , and it clearly appears here , and the the harmonics are resolved by the f Well , there are still appear after mel - filtering , professor a:   phd e: and it happens for high pitched voice because the width of the lower frequency mel - filters is sometimes even smaller than the pitch . phd e: And so what happens is that this  , add additional variability to this envelope and  professor a: Yeah . phd e: so we were thinking to modify the mel - spectrum to have something that that 's smoother on low frequencies . What I was talking about was just , starting with the FFT you could you could  do a very rough thing to estimate estimate  pitch . professor a: And   , given you know , given that ,  you could   come up with some kind of estimate of how much of the low frequency energy was was explained by by   those harmonics . And and so if you if you just you know subtracted off  your guess of the harmonics then something like this would end up with quite a bit lower energy in the first fifteen hundred hertz or so and and our first kilohertz , even . professor a: And  if was  noisy , the proportion that it would go down would be if it was if it was unvoiced or something . What 's up with you ? grad b:  our t I went to talk with  Mike Jordan this this week professor a:   grad b:  and  shared with him the ideas about  extending the Larry Saul work and  I asked him some questions about factorial H M so like later down the line when we 've come up with these these feature detectors , how do we how do we  you know ,  model the time series that that happens  and and we talked a little bit about factorial H M Ms and how  when you 're doing inference or w when you 're doing recognition , there 's like simple Viterbi stuff that you can do for for these H M and the  the great advantages that  a lot of times the factorial H M Ms don't  don't over - alert the problem there they have a limited number of parameters and they focus directly on on  the sub - problems at hand so you can imagine  five or so parallel  features  transitioning independently and then at the end you you  couple these factorial H M Ms with  with  undirected links  based on based on some more data . grad b: So he he seemed he seemed like really interested in in  in this and said said this is this is something very do - able and can learn a lot and  yeah , I 've just been continue reading  about certain things . grad b:  thinking of maybe using   m modulation spectrum stuff to  as features  also in the in the sub - bands professor a:   grad b: because it seems like the modulation  spectrum tells you a lot about the intelligibility of of certain  words and stuff So ,  . And  so I 've been looking at Avendano 's work and   I 'll try to write up in my next stat status report a nice description of what he 's doing , but it 's it 's an approach to deal with reverberation or that the aspect of his work that I 'm interested in the idea is that  normally an analysis frames are  too short to encompass reverberation effects  in full . You miss most of the reverberation tail in a ten millisecond window and so you you 'd like it to be that  the reverberation responses  simply convolved  in , but it 's not really with these ten millisecond frames cuz you j But if you take , say , a two millisecond  window I 'm sorry a two second window then in a room like this , most of the reverberation response is included in the window and the then it  then things are l more linear . It is it is more like the reverberation response is simply c convolved and  and you can use channel normalization techniques like  in his thesis he 's assuming that the reverberation response is fixed . He just does  mean subtraction , which is like removing the DC component of the modulation spectrum and that 's supposed to d  deal  deal pretty well with the  reverberation and  the neat thing is you can't take these two second frames and feed them to a speech recognizer  so he does this  method training trading the  the spectral resolution for time resolution and  come ca  synthesizes a new representation which is with say ten second frames but a lower s  frequency resolution . I guess it 's these are called " time frequency representations " and h he 's making the the time sh  finer grained and the frequency resolution  less fine grained . grad c: s so I 'm I guess my first stab actually in continuing his work is to  re - implement this this thing which  changes the time and frequency resolutions cuz he doesn't have code for me . grad c: Oh , and  , another f first step is  , so the the way I want to extend his work is make it able to deal with a time varying reverberation response  and  we don't really know how fast the  the reverberation response is varying the Meeting Recorder data  so  we we have this  block least squares  imp echo canceller implementation and  I want to try finding the the response , say , between a near mike and the table mike for someone using the echo canceller and looking at the echo canceller taps and then see how fast that varies from block to block . grad c: That should give an idea of how fast the reverberation response is changing . S so  y you do I think you read some of the the zeros as O 's and some as zeros . grad c: Is there a particular way we 're supposed to read them ? phd e: There are only zeros here . " O " " O " " O " " O " " O " " O " and " zero " are two ways that we say that digit . professor a: So it 's grad b: Ha ! phd e: But professor a: so it 's i phd e: Perhaps in the sheets there should be another sign for the if we want to the the guy to say " O " or professor a: No . professor a:  in digit recognition we 've done before , you have you have two pronunciations for that value , " O " and " zero " . phd e: But it 's perhaps more difficult for the people to prepare the database then , if because here you only have zeros professor a: No , they just write phd e: and and people pronounce " O " or zero professor a: they they write down OH . phd e: Yeah but if the sh the sheet was prepared with a different sign for the " O " .  , you 'd have to tell them " OK when we write this , say it tha " , phd e: OK . professor a: you know , and you just They just want people to read the digits as you ordinarily would phd e:   Is this a change from the last batch of of  forms ? Because in the last batch it was spelled out which one you should read . It was it was spelled out , and they decided they wanted to get at more the way people would really say things . professor a: That 's also why they 're they 're bunched together in these different groups . Actually , let me just s since since you brought it up , I was just it was hard not to be self - conscious about that when it after we since we just discussed it . But I realized that that  when I 'm talking on the phone , certainly , and and saying these numbers , I almost always say zero . So that that that 's the habit I 'm in , but some people say " O " and grad b: Yeah I normally say " O " cuz it 's easier to say