phd a: And what was that , Morgan ? What project ? professor b: the , TORRENT chip . professor b: we went through it Jim and I went through old emails at one point and and for two years there was this thing saying , yeah , we 're we 're two months away from being done . phd a: So , should we just do the same kind of deal where we go around and do , status report kind of things ? OK . phd a: Any objection ? Do y OK , M professor b: All in favor phd a: Do you want to start , Morgan ? Do you have anything , or ? professor b: I don't do anything . I No , I I 'm involved in discussions with with people about what they 're doing , but I think they 're since they 're here , they can talk about it themselves . Why don't you go ahead , Barry ? grad f: you 're gonna talk about Aurora stuff , per se ? phd a: OK . Well , this past week I 've just been , getting down and dirty into writing my my proposal . I just finished a section on ,  on talking about these intermediate categories that I want to classify , as a as a middle step . And , I hope to hope to get this ,  a full rough draft done by , Monday so I can give it to Morgan . phd a: So , is the idea you 're going to do this paper and then you pass it out to everybody ahead of time and ? grad f: Right , right . So , y you write up a proposal , and give it to people ahead of time , and you have a short presentation . phd a: Have you d ? I was just gonna ask , do you want to say any a little bit about it , grad f: Y s phd a: or ? Mmm .  , a little bit about ? phd a: Wh - what you 're what you 're gonna You said you were talking about the , particular features that you were looking at , grad f: Oh , the the phd a: or grad f: Right . Well , I was , I think one of the perplexing problems is , for a while I was thinking that I had to come up with a complete set of intermediate features in intermediate categories to to classify right away .  , build a system that , classifies these ,  these feat  , these intermediate categories using , multi - band techniques . Look at then I would look at the errors produced in the phoneme recognition and say , OK , well , I could probably reduce the errors if I included this extra feature or this extra intermediate category . phd a: So you 're gonna use TIMIT ? grad f: for that for that part of the the process , yeah , I would use TIMIT . grad f: that 's that 's ,  that 's just the ph the phone recognition task . So , I would mov I would then shift the focus to , something like Schw - Switchboard , where I 'd I would be able to ,  to model , intermediate categories that span across phonemes , phd a:   grad f: not just within the phonemes , themselves , and then do the same process there , on on a large vocabulary task like Switchboard .  , and for that for that part I would I 'd use the SRI recognizer since it 's already set up for for Switchboard . phd a: It 's about a month from now ? grad f: It 's a it 's a month and and a week . phd a: So , you want to go next , Dave ? And we 'll do grad e: Oh . And , we we got an improvement , in word error rate , training on the TI - digits data set and testing on Meeting Recorder digits of , six percent to four point five percent , on the n on the far - mike data using PZM F , but , the near - mike performance worsened , from one point two percent to two point four percent . And , wh why would that be , considering that we actually got an improvement in near - mike performance using HTK ? And so , with some input from , Andreas , I have a theory in two parts .  , first of all HTK sorry , SR - the SRI system is doing channel adaptation , and so HTK wasn't .  , so this ,  This mean subtraction approach will do a kind of channel normalization and so that might have given the HTK use of it a boost that wouldn't have been applied in the SRI case . So those finer - grained acoustic models could be more sensitive to the artifacts in the re - synthesized audio . And me and Barry were listening to the re - synthesized audio and sometimes it seems like you get of a bit of an echo of speech in the background . And so that seems like it could be difficult for training , cuz you could have different phones lined up with a different foreground phone , depending on the timing of the echo . So , I 'm gonna try training on a larger data set , and then , eh , the system will have seen more examples o of these artifacts and hopefully will be more robust to them . professor b: I had another thought just now , which is , remember we were talking before about we were talking in our meeting about , this stuff that some of the other stuff that Avendano did , where they were , getting rid of low - energy sections ?  , if you if you did a high - pass filtering , as Hirsch did in late eighties to reduce some of the effects of reverberation , Avendano and Hermansky were arguing that , perhaps one of the reasons for that working was ma may not have even been the filtering so much but the fact that when you filter a an all - positive power spectrum you get some negative values , and you gotta figure out what to do with them if you 're gonna continue treating this as a power spectrum . So if you imagine a a waveform that 's all positive , which is the time trajectory of energy , and , shifting it downwards , and then getting rid of the negative parts , that 's essentially throwing away the low - energy things . And it 's the low - energy parts of the speech where the reverberation is most audible . You know , you have the reverberation from higher - energy things showing up in So in this case you have some artificially imposed reverberation - like thing .  , you 're getting rid of some of the other effects of reverberation , but because you have these non - causal windows , you 're getting these funny things coming in , at n And , what if you did ?  , there 's nothing to say that the the processing for this re - synthesis has to be restricted to trying to get it back to the original , according to some equation . professor b: And one of the things you could do is , you could do some sort of VAD - like thing grad e:   professor b: and you actually could take very low - energy sections and set them to some some , very low or or near zero value .  , I 'm just saying if in fact it turns out that that these echoes that you 're hearing are ,  grad e:  - huh . professor b: or pre - echoes , whichever they are are are , part of what 's causing the problem , you actually could get rid of them . professor b: so that if you made a mistake you were more likely to keep in an echo than to throw out speech . phd g: what is the reverberation time like there ? grad e: In thi in this room ?  phd g: On , the the one what the s in the speech that you are you are using like ? grad e: Y Yeah . professor b: so it 's these are just microphone this micro close microphone and a distant microphone , he 's doing these different tests on . I think it 's I would guess , point seven , point eight seconds f  , R T grad f:  ! professor b: something like that ? But it 's you know , it 's this room . But the other thing is , he 's putting in w I was using the word " reverberation " in two ways . He 's also putting in , a he 's taking out some reverberation , but he 's putting in something , because he has averages over multiple windows stretching out to twelve seconds , which are then being subtracted from the speech . And since , you know , what you subtract , sometimes you 'll be you 'll be subtracting from some larger number and sometimes you won't . professor b: So you can end up with some components in it that are affected by things that are seconds away .  , and if it 's a low energy compo portion , you might actually hear some funny things . grad e: O o one thing , I noticed is that , the mean subtraction seems to make the PZM signals louder after they 've been re - synthesized . So I was wondering , is it possible that one reason it helped with the Aurora baseline system is just as a kind of gain control ? Cuz some of the PZM signals sound pretty quiet if you don't amplify them . I don't see why why your signal is louder after processing , because yo grad e: Yeah . professor b: I don't think just multiplying the signal by two would have any effect .  , I think if you really have louder signals , what you mean is that you have better signal - to - noise ratio . phd c: Well , well professor b: So if what you 're doing is improving the signal - to - noise ratio , then it would be better . professor b: But just it being bigger if with the same signal - to - noise ratio grad e: It w i i it wouldn't affect things . phd c: Well , the system is use the absolute energy , so it 's a little bit dependent on on the signal level . professor b: So if the if the if you change in both training and test , the absolute level by a factor of two , it will n have no effect . phd a: Did you add this data to the training set , for the Aurora ? Or you just tested on this ? grad e:   . Did I w what ? phd a: Well , Morgan was just saying that , as long as you do it in both training and testing , it shouldn't have any effect . But I didn't I 'm not sure if it made the clean ti TI - digits any louder . If it 's if it 's like , if it 's trying to find a a reverberation filter , it could be that this reverberation filter is making things quieter .  , there 's there 's nothing inherent about removing if you 're really removing , grad e: Nuh - huh . phd a: I wonder if there could be something like ,  for s for the PZM data , phd c: Eh phd a: you know , if occasionally , somebody hits the table or something , you could get a spike . I 'm just wondering if there 's something about the ,  you know , doing the mean normalization where , it it could cause you to have better signal - to - noise ratio . It it i maybe i If ,  Subtracting the the mean log spectrum is is is like dividing by the spectrum . So , depending what you divide by , if your if s your estimate is off and sometimes you 're you 're you 're getting a small number , you could make it bigger . professor b: So , it 's it 's just a a question of there 's It it could be that there 's some normalization that 's missing , or something to make it grad e:   professor b: y you 'd think it shouldn't be larger , but maybe in practice it is . So , you trained it on TI - digits ? But except this , it 's exactly the same system as the one that was tested before and that was trained on Macrophone . Right ? So on TI - digits it gives you one point two percent error rate and on Macrophone it 's still O point eight . grad e: If you 're talking about the Macrophone results that Andreas had about , a week and a half ago , I think it 's the same system . So you use VTL -  , vocal tract length normalization and , like MLLR transformations also , grad e:   phd c: and professor b: I 'm sorry , was his point eight percent , er , a a result on testing on Macrophone or or training ? phd c: all that stuff . grad e: That 's phd c: It was training on Macrophone and testing yeah , on on meeting digits . I I 've just been text testing the new Aurora front - end with well , Aurora system actually so front - end and HTK , acoustic models on the meeting digits and it 's a little bit better than the previous system . professor b: So , what w ? phd c: And phd g: With the with the HTK back - end ? What we have for Aurora ? phd c: Yeah . phd g: I know in the meeting , like phd c: On the meeting we have two point seven . phd c: we have the new LDA filters , and I think , maybe I didn't look , but one thing that makes a difference is this DC offset compensation .  , eh Do y did you have a look at at the meet  , meeting digits , if they have a DC component , or ? grad e: I I didn't .  , any all of the mikes have the DC removal some capacitor sitting right in that bias it . professor b: typi you know , unless Actually , there are instrumentation mikes that that do pass go down to DC . And you can get , I think it was I think it was in the Wall Street Journal data that that I can't remember , one of the DARPA things . professor b: we didn't we didn't know about for a while , while we were messing with it . And , the interesting thing that I tried was , Adam and Morgan had this idea , since my original attempts to , take the mean of the phase spectra over time and normalize using that , by subtracting that off , didn't work .  , so , well , that we thought that might be due to , problems with , the arithmetic of phases . They they add in this modulo two pi way and , there 's reason to believe that that approach of taking the mean of the phase spectrum wasn't really mathematically correct . So , what I did instead is I took the mean of the FFT spectrum without taking the log or anything , and then I took the phase of that , and I subtracted that phase off to normalize . professor b: You see , all he has to do is go back and reverse what he did before , and he 's really got something . phd a: Well , could you take what was left over and then subtract that ? professor b: Ex - exactly . phd g: Oh , it 's professor b: Just listen very carefully to what I say and do the opposite . I 'm more interested in trying to figure out what 's still the difference between the SRI system and the Aurora system . So , I think I will maybe train , like , gender - dependent models , because this is also one big difference between the two systems .  , the other differences were the fact that maybe the acoustic models of the SRI are more SRI system are more complex . But , Chuck , you did some experiments with this and phd a: It didn't seem to help in the HTK system . professor b: Well , it sounds like they also have he he 's saying they have all these , different kinds of adaptation . phd a: Like they do ,  I 'm not sure how they would do it when they 're working with the digits , phd c: The vocal tr phd a: but , like , in the Switchboard data , there 's ,  conversation - side normalization for the non - C - zero components , phd c: Yeah . phd c: So , it might be it might be better with it might be worse if the channel is constant , phd a: Yeah . phd g: And the acoustic models are like - k triphone models or or is it the whole word ? phd c: SRI it 's it 's tr grad f: SRI . So there 's there 's these kind of , pooled models and and they can go out to all sorts of dependencies . professor b: They have tied states and I think I I I don't real I 'm talk I 'm just guessing here . And maybe see with Andreas if Well , I I don't know how much it helps , what 's the model . phd a: So so the n stuff on the numbers you got , the two point seven , is that using the same training data that the SRI system used and got one point two ? phd c: That 's right . grad e: You know , the the Aurora baseline is set up with these ,  this version of the clean training set that 's been filtered with this G - seven - one - two filter , and , to train the SRI system on digits S - Andreas used the original TI - digits , under U doctor - speech data TI - digits , which don't have this filter . professor b: So is that ?  , are are these results comparable ? So you you were getting with the , Aurora baseline something like two point four percent on clean TI - digits , when , training the SRI system with clean TR digits TI - digits . And , so , is your two point seven comparable , where you 're , using , the submitted system ? phd c: Yeah . grad e: W w it was one one point two phd c: Ye grad e: with the SRI system , professor b: I 'm sorry . professor b: OK , so the comparable number then ,  for what you were talking about then , since it was HTK , would be the  , two point f phd c: It was four point something . Right ? The HTK system with , b grad e: D d professor b: Oh , right , right , right , right . phd c: MFCC features grad e: Do you mean the b ? The baseline Aurora - two system , trained on TI - digits , tested on Meeting Recorder near , I think we saw in it today , and it was about six point six percent . And another thing I I maybe would like to do is to just test the SRI system that 's trained on Macrophone test it on , the noisy TI - digits , professor b: Yeah . But I wonder if it 's just because maybe Macrophone is acoustically closer to the meeting digits than than TI - digit is , which is TI - digits are very clean recorded digits professor b:   phd c: and phd a: You know , it would also be interesting to see ,  to do the regular Aurora test , phd c: f s phd a: but use the SRI system instead of HTK . phd c: So we don't professor b: You 'd have to train the SRI system with with all the different languages . phd a: So , like , comple professor b: It 'd be a lot of work . phd a: I guess the work would be into getting the the files in the right formats , or something . phd a: Because when you train up the Aurora system , you 're ,  you 're also training on all the data . professor b: That 's true , but I think that also when we 've had these meetings week after week , oftentimes people have not done the full arrange of things phd a:   professor b: because on on whatever it is they 're trying , because it 's a lot of work , even just with the HTK . professor b: So , it 's it 's a good idea , but it seems like it makes sense to do some pruning phd a:   professor b: first with a a test or two that makes sense for you , phd a: Yeah . But , just testing on TI - digits would already give us some information about what 's going on .  , the next thing is this this VAD problem that ,  So , I 'm just talking about the the curves that I I sent I sent you so , whi that shows that when the SNR decrease , the current VAD approach doesn't drop much frames for some particular noises , which might be then noises that are closer to speech , acoustically . I They were supp Supposedly , in the next evaluation , they 're going to be supplying us with boundaries . First of all , the boundaries might be ,  like we would have t two hundred milliseconds or before and after speech . professor b: Do we ?  , is there some reason that we think that 's the case ? phd c: And No . professor b: But maybe we 'll get some insight on that when when , the gang gets back from Crete . professor b: And then the thing is if if they really are going to have some means of giving us fairly tight , boundaries , then that won't be so much the issue . phd g: Because w we were wondering whether that VAD is going to be , like , a realistic one or is it going to be some manual segmentation . And then , like , if if that VAD is going to be a realistic one , then we can actually use their markers to shift the point around , the way we want professor b:   phd g: to find a  , rather than keeping the twenty frames , we can actually move the marker to a point which we find more suitable for us . phd g: But if that is going to be something like a manual , segmenter , then we can't use that information anymore , phd c:   phd g: because that 's not going to be the one that is used in the final evaluation . There 's an  , I think it 's still for even for the evaluation , it might still be interesting to work on this because the boundaries apparently that they would provide is just , starting of speech and end of speech  , at the utterance level . phd c: So phd g: with some pauses in the center , provided they meet that whatever the hang - over time which they are talking . professor b: So if you could get at some of that ,  phd c: So professor b: although that 'd be hard . It might be useful for , like , noise estimation , and a lot of other things that we want to work on . So I did I just started to test putting together two VAD which was was not much work actually .  , I im re - implemented a VAD that 's very close to the , energy - based VAD that , the other Aurora guys use . phd c: and , detect detecting the first group of four frames that have a energy that 's above this threshold , and , from this point , tagging the frames there as speech . And it really removes it , still o on the noises where our MLP VAD doesn't work a lot . phd c: professor b: Cuz I would have thought that having some kind of spectral information , phd c: and professor b: you know , in the old days people would use energy and zero crossings , for instance  , would give you some better performance . Right ? Cuz you might have low - energy fricatives or or ,  stop consonants , or something like that . So , your point is will be to u use whatever professor b: Oh , that if you d if you use purely energy and don't look at anything spectral , then you don't have a good way of distinguishing between low - energy speech components and nonspeech . professor b: just as a gross generalization , most nonsp many nonspeech noises have a low - pass kind of characteristic , some sort of slope . And and most , low - energy speech components that are unvoiced have a a high - pass kind of characteristic phd c:   professor b: you know , at the beginning of a of a of an S sound for instance , just starting in , it might be pretty low - energy , phd c:   Whereas , a a lot of rumble , and background noises , and so forth will be predominantly low - frequency .  , you know , by itself it 's not enough to tell you , but it plus energy is sort of phd c: Yeah . professor b: if you look up in Rabiner and Schafer from like twenty - five years ago or something , that 's sort of what they were using then . It it might be that what I did is so , removes like low ,  low - energy , speech frames . Because the way I do it is I just I just combine the two decisions so , the one from the MLP and the one from the energy - based with the with the and operator . So if the energy - based dropped dropped low - energy speech , mmm , they they are they are lost . phd c: But s still , the way it 's done right now it it helps on on the noises where it seems to help on the noises where our VAD was not very good . But but , I guess what you 're saying is that the the MLP - based one has the spectral information . professor b: Well , you can imagine phd c: The way I use a an a " AND " operator is So , it I ,  professor b: Is ? phd c: The frames that are dropped by the energy - based system are are , dropped , even if the , MLP decides to keep them . phd a: No professor b: but  , I guess in principle what you 'd want to do is have a  , a probability estimated by each one and and put them together . phd a: Something that that I 've used in the past is ,  when just looking at the energy , is to look at the derivative . phd a: But , I 'm I 'm trying to remember if that requires that you keep some amount of speech in a buffer . phd c: Well , actually if I don't maybe don't want to work too much of on it right now . I just wanted to to see if it 's what I observed was the re was caused by this this VAD problem .  , which I 've just started yesterday to launch a bunch of , twenty - five experiments , with different , values for the parameters that are used . So , it 's the Makhoul - type spectral subtraction which use an over - estimation factor . So , we substr I subtract more , noise than the noise spectra that is estimated on the noise portion of the s  , the utterances . And after subtraction , I also add a constant noise , and I also try different , noise , values and we 'll see what happen . But st still when we look at the ,  Well , it depends on the parameters that you use , but for moderate over - estimation factors and moderate noise level that you add , you st have a lot of musical noise . On the other hand , when you subtract more and when you add more noise , you get rid of this musical noise but maybe you distort a lot of speech . So the next thing , maybe I what I will try to to do is just to try to smooth mmm , the ,  to smooth the d the result of the subtraction , to get rid of the musical noise , using some kind of filter , or phd g: Can smooth the SNR estimate , also . So , to get something that 's would be closer to what you tried to do with Wiener filtering . phd c: It phd g: And it 's phd c: Maybe you can phd g: go ahead . And there are there were some bugs in the program , so I was p initially trying to clear them up . Because one of the bug was I was assuming that always the VAD  , the initial frames were silence . So the it wasn't estimating the noise initially , and then it never estimated , because I assumed that it was always silence . phd c: So , in some cases s there are also phd g: SpeechDat - Car Italian . And , so once it was cleared , I ran a few experiments with different ways of smoothing the estimated clean speech and how t estimated the noise and , eh , smoothing the SNR also . And so the the trend seems to be like , smoothing the current estimate of the clean speech for deriving the SNR , which is like deriving the Wiener filter , seems to be helping . So we 'll have , like , a few results where the estimating the the More smoothing is helping . And , so I 'm I 'm trying a few more experiments with different time constants for smoothing the noise spectrum , and smoothing the clean speech , and smoothing SNR . So , one is fixed in the line , like Smoothing the clean speech is is helping , so I 'm not going to change it that much . But , the way I 'm estimating the noise and the way I 'm estimating the SNR , I 'm just trying trying a little bit . So , that h And the other thing is , like , putting a floor on the , SNR , because that if some In some cases the clean speech is , like when it 's estimated , it goes to very low values , so the SNR is , like , very low . So , I 'm thinking of , like , putting a floor also for the SNR so that it doesn't vary a lot in the low - energy regions . The results are , like So far I 've been testing only with the baseline , which is which doesn't have any LDA filtering and on - line normalization . So it 's just VAD , plus the Wiener filter , plus the baseline system , which is , just the spectral  , the mel sp mel , frequency coefficients . And the other thing that I tried was but I just took of those , Carlos filters , which Hynek had , to see whether it really h helps or not . And it 's it seems to be like it 's not hurting a lot by just blindly picking up one filter which is nothing but a four hertz a band - pass m m filter on the cubic root of the power spectrum . So there must be something that I can that can be done with that type of noise compensation also , which I guess I would ask Carlos about that .  , how how he derived those filters and and where d if he has any filters which are derived on OGI stories , added with some type of noise which what we are using currently , or something like that . So maybe I 'll professor b: This is cubic root of power spectra ? phd g: Yeah . professor b: So , if you have this band - pass filter , you probably get n you get negative values . phd g: So it has , like the spectrogram has , like  , it actually , enhances the onset and offset of  , the the begin and the end of the speech . So it 's there seems to be , like , deep valleys in the begin and the end of , like , high - energy regions , professor b:   phd g: So , those are the regions where there are , like when I look at the spectrogram , there are those deep valleys on the begin and the end of the speech . There are a few very not a lot of because the filter doesn't have a really a deep negative portion , so that it 's not really creating a lot of negative values in the cubic root . So , I 'll I 'll s may continue with that for some w I 'll I 'll Maybe I 'll ask Carlos a little more about how to play with those filters , and but while making this Wiener filter better . professor b: last week you were also talking about building up the subspace stuff ? phd g: Yeah . I I I would actually m m didn't get enough time to work on the subspace last week . phd a: How about you , Carmen ? phd d: Well , I am still working with , eh , VTS . And , one of the things that last week , eh , say here is that maybe the problem was with the diff because the signal have different level of energy . professor b:  ? phd d: And , maybe , talking with Stephane and with Sunil , we decide that maybe it was interesting to to apply on - line normalization before applying VTS . But then we decided that that 's it doesn't work absolutely , because we modified also the noise . I don't hav I don't this is I didn't do the experiment yet to apply VTS in cepstral domain . professor b: The other thing is So so , in i i and Not and C - zero would be a different So you could do a different normalization for C - zero than for other things anyway .  , the other thing I was gonna suggest is that you could have two kinds of normalization with with , different time constants . So , you could do some normalization s  , before the VTS , and then do some other normalization after . phd d: Well , we s decide to m to to obtain the new expression if we work in the cepstral domain . It 's k it 's k It 's quite a lot It 's a lot of work . phd d: And I want to know if if we have some feeling that the result I I would like to know if I don't have any feeling if this will work better than apply VTS aft in cepstral domain will work better than apply in m mel in filter bank domain . Well , you 're I think you 're the first one here to work with VTS , so , maybe we could call someone else up who has , ask them their opinion . phd c: Actually , the VTS that you tested before was in the log domain and so the codebook is e e kind of dependent on the level of the speech signal . phd d: Yeah ? phd c: And So I expect it If if you have something that 's independent of this , I expect it to it to , be a better model of speech . And then you have one number which is very dependent on the level cuz it is the level , phd d:   phd d: Ye phd c: Because it 's like first doing general normalization phd d: Yea phd c: and then noise removal , which is phd d: Yeah . We I was thinking to to to estimate the noise with the first frames and then apply the VAD , professor b:   phd d: We we see Well , I am thinking about that and working about that , professor b: Yeah .  , one of the things we 've talked about maybe it might be star time to start thinking about pretty soon , is as we look at the pros and cons of these different methods , how do they fit in with one another ? Because we 've talked about potentially doing some combination of a couple of them . Maybe maybe pretty soon we 'll have some sense of what their characteristics are , phd d:  -