PhD E: What 's new ?
Professor B: Well , I mean , I think maybe the thing to me might be I me I 'm sure you 've just been working on on , uh , details of that since the meeting , right ? And so PhD A: Mmm , since the meeting , well , I I 've been I 've been train training a new VAD and a new feature net .
Professor B: That was that was Tuesday . OK .
PhD A: So they should be ready . Um .
Professor B: But I guess maybe the thing since you weren't yo you guys weren't at that that meeting , might be just just to , um , sort of recap , uh , the the conclusions of the meeting .
PhD A: Mm - hmm .
PhD E: Oh , great .
Professor B: So .
PhD E: You 're talking about the meeting with Hynek ?
Professor B: Yeah . Cuz that was sort of , uh we we 'd sort of been working up to that , that that , uh , he would come here this week and and we would sort of PhD E: Uh - huh .
Professor B: Since he 's going out of town like now , and I 'm going out town in a couple weeks , uh , and time is marching , sort of , given all the mu many wonderful things we could be working on , what what will we actually focus on ?
PhD E: Mm - hmm .
Professor B: And , uh and what do we freeze ? And , you know , what do we ? So , um . I mean , this software that these guys created was certainly a a key part . So then there 's something central and there aren't at least a bunch of different versions going off in in ways that differ trivially . Uh , um , and , um ,
PhD E: Yeah . That 's that 's nice .
Professor B: and then within that , I guess the idea was to freeze a certain set of options for now , to run it , uh , a particular way , and decide on what things are gonna be experimented with , as opposed to just experimenting with everything . So keep a certain set of things constant . So , um . Uh , maybe describe roughly what what we are keeping constant for now , or ?
PhD A: Yeah . Well . So we 've been working like six weeks on on the noise compensation and we end up with something that seems reasonable . Um .
PhD E: Are you gonna use which of the two techniques ?
PhD A: So finally it 's it 's , um , Wiener filtering on FFT bins . And it uses , uh , two steps , smoothing of the transfer function , the first step , that 's along time , which use recursion . And after this step there is a further smoothing along frequency , which use a sliding window of twenty FFT bins . Mmm . And , uh PhD E: So this is on the uh , before any mel scaling has been done ?
PhD A: Yeah , yeah .
PhD E: This is PhD A: It was Professor B: This this smoothing is done on the estimate , um , of what you 're going to subtract ? Or on the thing that has already had something subtracted ?
PhD A: Yeah . Uh , it 's on the transfer function . So Professor B: Oh , it 's on the transfer function for the Wiener filter .
PhD A: Yeah .
Professor B: Yeah , OK .
PhD A: Yeah , so basically we tried different configuration within this idea . We tried u u applying this on mel bands , having spectral subtraction instead of wiener filtering . Um . Well , finally we end up with this configuration that works , uh , quite well . So we are going to fix this for the moment and work on the other aspects of the whole system .
PhD E: Mm - hmm .
PhD A: So Professor B: Actually , let me int eh , Dave isn't here to talk about it , but let me just interject . This module , in principle , i I mean , you would know whether it 's true in fact , is somewhat independent from the rest of it . I mean , because you you re - synthesize speech , right ?
PhD A: Mm - hmm .
Professor B: So , um . Uh , well you don't I guess you don't re - synthesize speech , but you could PhD A: We we do not fo
Professor B: Uh , but you could .
PhD A: Well well , we do , but we don't don't re - synthesize . In in the program we don't re - synthesize and then re - analyze once again . We just use the clean FFT bins .
Professor B: But you have a re - synthesized thing that you that 's an an option here .
PhD A: This is an option that then you can Yeah .
Professor B: Yeah , I gu I guess my point is that , um , i in some of the work he 's doing in reverberation , one of the things that we 're finding is that , uh , it 's it 's for the for an artificial situation , we can just deal with the reverberation and his techniques work really well . But for the real situation uh , problem is , is that you don't just have reverberation , you have reverberation in noise . And if you don't include that in the model , it doesn't work very well . So in fact it might be a very nice thing to do , to just take the noise removal part of it and put that in front of what he 's looking at . And , uh , generate new files or whatever , and and , uh , uh and then do the reverberation part .
PhD A: Mm - hmm .
Professor B: So it 's PhD D: Mmm .
Professor B: Anyway .
PhD E: So Dave hasn't tried that yet ?
Professor B: No , no . He 's I mean , e
PhD E: I guess he 's busy with Professor B: Yeah , prelims , right .
Grad C: Pre - prelim hell .
Professor B: Yeah .
PhD E: Yeah .
Professor B: So .
PhD E: Yeah .
Professor B: Uh , but but , you know , that 'll uh , it 's clear that we , uh we are not with the real case that we 're looking at , we can't just look at reverberation in isolation because the interaction between that and noise is is considerable . And that 's I mean , in the past we 've looked at , uh , and this is hard enough , the interaction between channel effects and and , uh and additive noise , uh , so convolutional effects and and additive effects . And that 's hard enough . I mean , I don't think we really I mean , we 're trying to deal with that . In a sense that 's what we 're trying to deal with in this Aurora task . And we have , uh , the , uh , uh , LDA stuff that in principle is doing something about convolutional effects . And we have the noise suppression that 's doing something about noise . Uh , even that 's hard enough . And and the on - line normalization as well , in that s category . i i There 's all these interactions between these two and that 's part of why these guys had to work so hard on on juggling everything around . But now when you throw in the reverberation , it 's even worse , because not only do you have these effects , but you also have some long time effects . And , um , so Dave has something which , uh , is doing some nice things under some conditions with with long time effects but when it 's when there 's noise there too , it 's it 's it 's pretty hard . So we have to start Since any almost any real situation is gonna have uh , where you have the microphone distant , is going to have both things , we we actually have to think about both at the same time .
PhD E: Hmm .
Professor B: So , um So there 's this noise suppression thing , which is sort of worked out and then , uh , maybe you should just continue telling what what else is in the the form we have .
PhD A: Yeah , well , the , um , the other parts of the system are the the blocks that were already present before and that we did not modify a lot .
Professor B: So that 's again , that that 's the Wiener filtering , followed by , uh uh , that 's done at the FFT level . Then PhD A: Yeah , th then the mel filter bank ,
Professor B: Mm - hmm .
PhD A: then the log operation ,
Professor B: Mm - hmm .
PhD A: Mmm .
Professor B: The the the filtering is done in the frequency domain ?
PhD A: Yeah .
Professor B: Yeah , OK . And then the mel and then the log , and then the
PhD A: Then the LDA filter ,
Professor B: LDA filter .
PhD A: mmm , then the downsampling ,
Professor B: And then uh downsample ,
PhD A: DCT ,
Professor B: DCT ,
PhD A: then , um , on - line normalization ,
Professor B: on - line norm ,
PhD A: followed by upsampling . Then finally , we compute delta and we put the neural network also .
Professor B: Right , and then in parallel with an a neural net . And then following neural net , some probably some orthogonalization .
PhD A: Yeah .
Professor B: Uh Um .
PhD A: And finally frame dropping , which um , would be a neural network also , used for estimated silence probabilities . And the input of this neural network would be somewhere between log mel bands or one of the earlier stages of the processing .
Professor B: Mm - hmm . So that 's sort of most of this stuff is yeah , is operating parallel with this other stuff .
PhD A: Mm - hmm .
Professor B: Yeah . So the things that we , um , uh , I guess we sort of uh , There 's there 's some , uh , neat ideas for V A So , I mean , in I think there 's sort of like There 's a bunch of tuning things to improve stuff . There 's questions about various places where there 's an exponent , if it 's the right exponent , or ways that we 're estimating noise , that we can improve estimating noise . And there 's gonna be a host of those . But structurally it seemed like the things the main things that that we brought up that , uh , are are gonna need to get worked on seriously are , uh , uh , a  a significantly better VAD , uh , putting the neural net on , um , which , you know , we haven't been doing anything with , the , uh , neural net at the end there , and , uh , the , uh , opening up the second front . Uh .
PhD E: The other half of the channel ?
Professor B: Yeah , yeah , I mean , cuz we we have we have , uh , uh , half the the , uh , data rate that they allow .
PhD E: That what you mean ?
Professor B: And , uh , so the initial thing which came from , uh , the meeting that we had down south was , uh , that , um , we 'll initially just put in a mel spectrum as the second one . It 's , you know , cheap , easy . Uh . There 's a question about exactly how we do it . We probably will go to something better later , but the initial thing is that cepstra and spectra behave differently ,
PhD E: Mm - hmm .
Professor B: so . Um , {comment} I think Tony Robinson used to do I was saying this before . I think he used to do mel , uh , spectra and mel cepstra . He used them as alternate features . Put them together .
PhD E: Hmm .
Professor B: Uh .
PhD E: So if you took the system the way it is now , the way it 's fro you 're gonna freeze it , and it ran it on the last evaluation , where it would it be ?
PhD A: Mm - hmm . It , uh ,
PhD E: In terms of ranking ?
PhD A: Ri - right now it 's second .
PhD D: Second .
PhD A: Um .
PhD E: Mm - hmm .
Professor B: Although you you know , you haven't tested it actually on the German and Danish , have you ?
PhD A: No , we didn't . No , um .
Professor B: Yeah .
PhD E: So on the ones that you did test it on it would have been second ?
Professor B: Yeah . Would it I mean But When you 're saying second , you 're comparing to the numbers that the , uh that the best system before got on , uh also without German and Danish ?
PhD A: Yeah , yeah .
Professor B: Yeah , OK .
PhD D: And th the ranking actually didn't change after the German and Danish . So , yeah .
Professor B: Well ranking didn't before , but I 'm just asking where this is to where theirs was without the German and Danish ,
PhD A: Yeah .
PhD D: Yeah .
PhD A: Mmm .
PhD D: Yeah .
Professor B: right ?
PhD D: Yeah , yeah .
Professor B: So .
PhD E: Where where where were we actually on the last test ?
Professor B: Oh , we were also esp essentially second , although there were there were I mean , we had a couple systems and they had a couple systems . And so , I guess by that we were third , but I mean , there were two systems that were pretty close , that came from the same place .
PhD E: Uh - huh . I see . OK .
Professor B: Uh , so institutionally we were  we were second , with , uh , the third third system .
PhD E: We 're so this second that you 're saying now is system - wide second ?
Professor B: See Uh , no I think it 's also institutional , isn't it ?
PhD E: Still institutionally second ?
Professor B: Right ? I mean , I think both of their systems probably PhD A: Uh , we are between their two systems . So
Professor B: Oh , are we ?
PhD A: I It is a triumph .
PhD D: Yeah .
Professor B: Is it ?
PhD D: Their their first system is fifty - four point something . And , uh , we are fifty - three point something .
PhD A: But everything is within the range of one one percent .
PhD D: And their second system is also fifty - three point something . Yeah , one percent .
Professor B: Yeah , so so basically they 're all they 're all pretty close .
PhD E: Oh , wow !
PhD A: So .
PhD E: That 's very close .
PhD D: Yeah .
PhD E: Yeah .