A: Yeah .
D: Mm - hmm .
C: yeah .
D: And , yeah ,
D: some people also if it 's a negative value they , uh , re - compute it using inter interpolation from the edges and bins .
B: For frames , frequency bins .
C: Yeah .
D: Well , there are different things that you can do .
A: Oh .
C: People can also , uh , reflect it back up and essentially do a full wave rectification instead of a instead of half wave .
A: Oh .
C: But it was just a thought that that it might be something to try .
D: Mm - hmm .
D: Mm - hmm .
D: Yep .
D: Well , actually I tried , something else based on this , um , is to to put some smoothing ,
D: um , because it seems to to help or it seems to help the Wiener filtering
C: Mm - hmm .
D: and ,
D: mmm D: So what I did is , uh , some kind of nonlinear smoothing .
D: Actually I have a recursion that computes D: Yeah , let me go back a little bit .
D: Actually , when you do spectral subtraction you can , uh , find this this equivalent in the s in the spectral domain .
D: You can uh compute ,
D: y you can say that d your spectral subtraction is a filter ,
D: um , and the gain of this filter is the , um , signal energy minus what you subtract , divided by the signal energy .
D: And this is a gain that varies over time , and , you know , of course , uh , depending on the s on the noise spectrum and on the speech spectrum .
D: And D: what happen actually is that during low SNR values , the gain is close to zero
D: but it varies a lot .
D: Mmm ,
D: and this this is the cause of musical noise and all these the <comment> the fact you we go below zero one frame and then you can have an energy that 's above zero .
C: Mm - hmm .
D: And D: Mmm .
D: So the smoothing is I did a smoothing actually on this gain , uh , trajectory .
D: But it 's the smoothing is nonlinear in the sense that I tried to not smooth if the gain is high ,
D: because in this case we know that , uh , the estimate of the gain is correct
D: because we we are not close to to to zero ,
D: um , and to do more smoothing if the gain is low .
D: Mmm .
D: Um .
D: Yeah .
D: So ,
D: well ,
D: basically that 's this idea ,
D: and it seems to give pretty good results ,
D: uh , although I 've just just tested on Italian and Finnish .
D: And on Italian it seems my result seems to be a little bit better than the Wiener filtering ,
B: Mm - hmm .
D: right ?
B: Yeah , the one you showed yesterday .
B: Right ?
C: Yeah .
D: Uh , I don't know if you have these improvement the detailed improvements for Italian , Finnish , and Spanish there
B: Fff .
D: or you have just have your own .
B: No , I don't have , for each ,
B: I I just just have the final number here .
D: Mm - hmm .
C: So these numbers he was giving before with the four point three , and the ten point one , and so forth , those were Italian , right ?
B: Yeah , yeah , yeah .
C: Yeah .
D: Uh B: So so , no ,
B: I actually didn't give you the number which is the final one ,
D: uh , no ,
D: we 've B: which is , after two stages of Wiener filtering .
B: I mean , that was I just well , like the overall improvement is like fifty - six point five .
C: Right .
B: So ,
D: Mm - hmm .
B: I mean , his number is still better than what I got in the two stages of Wiener filtering .
D: Yeah .
C: Right .
D: On Italian .
D: But on Finnish it 's a little bit worse , apparently .
B: Mm - hmm .
D: Um C: But do you have numbers in terms of word error rates on on Italian ? So just so you have some sense of reference ?
D: Yeah .
D: Uh , so , it 's , uh , three point , uh , eight .
C: Uh - huh .
D: Am I right ?
B: Oh , OK .
B: Yeah , right ,
B: OK .
D: And then , uh , d uh , nine point , uh , one .
C: Mm - hmm .
D: And finally , uh , sixteen point five .
C: And this is , um , spectral subtraction plus what ?
D: Plus plus nonlinear smoothing .
D: Well , it 's the system it 's exactly the sys the same system as Sunil tried ,
C: On - line normalization and LDA ?
D: but C: Yeah .
D: Yeah .
C: Yeah .
D: But instead of double stage Wiener filtering , it 's it 's this smoothed spectral subtraction .
D: Um , yeah .
C: Right .
A: What is it the , um , France Telecom system uses
A: for Do they use spectral subtraction , or Wiener filtering ,
A: or ?
B: They use spectral subtraction ,
B: right .
D: For what ?
B: French Telecom .
D: It it 's Wiener filtering ,
B: Oh , it 's it 's Wiener filtering .
D: am I right ?
B: Sorry .
D: Well , it 's some kind of Wiener filtering A: Oh .
B: Yeah , filtering .
B: Yeah , it 's not exactly Wiener filtering
B: but some variant of Wiener filtering .
D: Yeah .
A: I see .
C: Yeah ,
B: Yeah .
C: plus , uh , I guess they have some sort of cepstral normalization , as well .
B: s
B: They have like D: Mm - hmm .
B: yeah ,
B: th the just noise compensation technique is a variant of Wiener filtering ,
B: plus they do some some smoothing techniques on the final filter .
B: The th they actually do the filtering in the time domain .
A: Mmm .
D: Yeah .
B: So they would take this HF squared back , taking inverse Fourier transform .
A: Hmm .
B: And they convolve the time domain signal with that .
A: Oh , I see .
B: And they do some smoothing on that final filter , impulse response .
A: Hmm .
D: But they also have two two different smoothing @ @ .
D: One in the time domain
B: I mean , I 'm I 'm @ @ .
D: and one in the frequency domain by just taking the first , um , coefficients of the impulse response .
B: But .
D: So , basically it 's similar .
D: I mean , what you did , it 's similar
B: It 's similar in the smoothing
D: because you have also two two kind of smoothing .
B: and D: One in the time domain ,
B: Yeah .
D: and one in the frequency domain ,
B: Yeah .
B: The frequency domain .
D: yeah .
A: Does the smoothing in the time domain help D: Um A: Well , do you get this musical noise stuff with Wiener filtering
A: or is that only with , uh , spectral subtraction ?
B: No , you get it with Wiener filtering also .
D: Yeah .
A: Does the smoothing in the time domain help with that ?
A: Or some other smoothing ?
B: Oh , no ,
B: you still end up with zeros in the s spectrum .
D: Yeah .
B: Sometimes .
C: I mean , it 's not clear that these musical noises hurt us in recognition .
A: Hmm .
C: We don't know if they do .
A: Hmm .
B: Yeah .
C: I mean , they they sound bad .
C: But we 're not listening to it , usually .
A: Mm - hmm .
B: Yeah , I know .
D: Mm - hmm .
A: Hmm .
D: Uh , actually the the smoothing that I did do here reduced the musical noise .
D: Well , it B: Mm - hmm .
D: Mmm .
B: Yeah ,
B: yeah ,
B: the D: Well , I cannot you cannot hear beca
D: well , actually what I d did not say is that this is not in the FFT bins .
D: This is in the mel frequency bands .
D: Um D: So ,
D: it could be seen as a f a a smoothing in the frequency domain because I used , in ad mel bands in addition and then the other phase of smoothing in the time domain .
D: Mmm .
D: But , when you look at the spectrogram , if you don't have an any smoothing , you clearly see , like in silence portions , and at the beginning and end of speech , you see spots of high energy randomly distributed over the the spectrogram .
A: Mm - hmm .
A: Mm - hmm .
D: Um A: That 's the musical noise ?
D: Which is musical noise ,
A: Mm - hmm .
D: yeah ,
D: if if it If you listen to it D: uh , if you do this in the FFT bins , then you have spots of energy randomly distributing .
D: And if you f if you re - synthesize these spot sounds as , like , sounds ,
A: Mm - hmm .
D: uh D: And C: Well , none of these systems , by the way , have I mean , y you both are are working with , um , our system that does not have the neural net ,
C: right ?
D: Mm - hmm .
B: Yep .
B: Yeah .
C: OK .
D: Yeah .
C: So one would hope , presumably , that the neural net part of it would would improve things further as as they did before .
D: Yeah .
D: Um D: Yeah ,
D: although if if we , um , look at the result from the proposals , <comment> one of the reason , uh , the n system with the neural net was , um , more than well , around five percent better , is that it was much better on highly mismatched condition .
D: I 'm thinking , for instance , on the TI - digits trained on clean speech and tested on noisy speech .
C: Mm - hmm .
D: Uh , for this case , the system with the neural net was much better .
C: Mm - hmm .
D: But not much on the in the other cases .
C: Yeah .
D: And
D: if we have no , uh , spectral subtraction or Wiener filtering , um , i the system is Uh , we thought the neural neural network is much better than before ,
D: even in these cases of high mismatch .
D: So , maybe the neural net will help less
D: but , um C: Maybe .
A: Could you train a neural net to do spectral subtraction ?
C: Yeah ,
C: it could do a nonlinear spectral subtraction
D: Mm - hmm .
C: but I don't know if it C: I mean , you have to figure out what your targets are .
A: Yeah ,
A: I was thinking if you had a clean version of the signal and and a noisy version , and your targets were the M F - uh , you know , whatever , frequency bins D: Mm - hmm .
C: Right .
D: Mm - hmm .
C: Yeah , well , that 's not so much spectral subtraction then ,
D: Mm - hmm .
C: but but but it 's but at any rate , yeah , people ,
C: uh A: People do that ?
C: y yeah ,
C: in fact , we had visitors here who did that
C: I think when you were here ba way back when .
D: Mm - hmm .
A: Hmm .
C: Uh ,
C: people d done lots of experimentation over the years with training neural nets .
C: And it 's not a bad thing to do .
C: It 's another approach .
A: Hmm .
D: Mm - hmm .
C: M I mean , it 's it , um C: The objection everyone always raises , which has some truth to it is that , um , it 's good for mapping from a particular noise to clean
C: but then you get a different noise .
A: Mm - hmm .
C: And the experiments we saw that visitors did here showed that it there was at least some , um , <comment> gentleness to the degradation when you switched to different noises .
C: It did seem to help .
C: So that you 're right ,
C: that 's another another way to go .
A: How did it compare on I mean , for for good cases where it it uh , stuff that it was trained on ?
A: Did it do pretty well ?
C: Oh , yeah ,
C: it did very well .
A: Mmm .
C: Yeah .
A: Mmm .
D: Mm - hmm .
C: Um ,
C: but to some extent that 's kind of what we 're doing .
C: I mean , we 're not doing exactly that ,
C: we 're not trying to generate good examples
C: but by trying to do the best classifier you possibly can , for these little phonetic categories ,
A: Mm - hmm .
A: You could say it 's sort of built in .
C: It 's C: Yeah ,
C: it 's kind of built into that .
A: Hmm .
C: And and that 's why we have found that it it does help .
A: Mm - hmm .
C: Um C: so ,
C: um ,
C: yeah , I mean , we 'll just have to try it .
C: But I I would I would I would imagine that it will help some .
C: I mean , it we 'll just have to see whether it helps more or less the same ,
C: but I would imagine it would help some .
D: Mm - hmm .
C: So in any event , all of this I was just confirming that all of this was with a simpler system .
D: Yeah ,
D: yeah .
C: OK ?
D: Um ,
D: Yeah , so this is th the , um D: Well , actually , this was kind of the first try with this spectral subtraction plus smoothing ,
C: Mm - hmm .
D: and I was kind of excited by the result .
C: Mm - hmm .
D: Um , then I started to optimize the different parameters .
D: And ,
D: uh , the first thing I tried to optimize is the , um , time constant of the smoothing .
D: And it seems that the one that I chose for the first experiment was the optimal one ,
D: so uh ,
C: It 's amazing how often that happens .
D: Um ,
D: so this is the first thing .
D: Um D: Yeah , another thing that I it 's important to mention is , um , that this has a this has some additional latency .
D: Um .
D: Because when I do the smoothing , uh , it 's a recursion that estimated the means , so of the g of the gain curve .
D: And
D: this is a filter that has some latency .
D: And I noticed that it 's better if we take into account this latency .
D: So , instead o of using the current estimated mean to , uh , subtract the current frame , it 's better to use an estimate that 's some somewhere in the future .
D: Um A: And that 's what causes the latency ?
D: Yeah .
C: Mm - hmm .
A: OK .
B: You mean , the m the mean is computed o based on some frames in the future also ?
B: Or or no ?
D: It 's the recursion ,
D: so it 's it 's the center recursion ,
D: right ?
B: Mm - hmm .
D: Um D: and the latency of this recursion is around fifty milliseconds .
C: One five ?
C: One five ?
C: Five zero ?
D: Five zero ,
D: yeah .
C: Five zero .
D: Um ,
C: Yeah .
D: mmm .
B: I 'm sorry ,
B: why why is that delay coming ?
B: Like , you estimate the mean ?
D: Yeah ,
D: the mean estimation has some delay ,
D: right ?
D: I mean , the the filter that that estimates the mean has a time constant .
B: Oh , yeah .
B: It isn't B: OK ,
B: so it 's like it looks into the future also .
D: Yeah .
B: OK .
C: What if you just look into the past ?
D: It 's , uh , not as good .
D: It 's not bad .
C: How m by how much ?
D: Um , it helps a lot over the ba the baseline
D: but , mmm C: By how much ?
D: it D: It 's around three percent , um , relative .
C: Worse .
D: Yeah .
D: Yeah .
D: Um ,
C: Hmm .
D: mmm D: So , uh C: It 's depending on how all this stuff comes out
C: we may or may not be able to add any latency .
D: Yeah , but D: Yeah .
D: So , yeah , it depends .
D: Uh , y actually , it 's it 's l it 's three percent .
D: Right .
D: Mmm .
D: Yeah ,
D: b but I don't think we have to worry too much on that right now while you kno .
D: Mm - hmm .
C: Um ,
D: So C: s Yeah , I mean , I think the only thing is that C: I would worry about it a little .
D: Mm - hmm .
C: Because if we completely ignore latency , and then we discover that we really have to do something about it , we 're going to be find ourselves in a bind .
D: Mm - hmm .
C: So , um ,
C: you know , maybe you could make it twenty - five .
D: Yeah .
C: You know what I mean ?
D: Oh yes .
C: Yeah , just , you know , just be be a little conservative
C: because we may end up with this crunch where all of a sudden we have to cut the latency in half or something .
D: s
D: Mm - hmm .
D: Yeah .
C: OK .
D: Um .
D: So ,
D: yeah , there are other things in the , um , algorithm that I didn't , uh , @ @ a lot yet ,
A: Oh !
D: which A: Sorry .
A: A quick question just about the latency thing .
A: If if there 's another part of the system that causes a latency of a hundred milliseconds , is this an additive thing ?
A: Or c or is yours hidden in that ?
D: Mm - hmm .
D: No ,
A: Uh D: it 's it 's added .
A: It 's additive .
D: Mm - hmm .
A: OK .
B: We can B: OK .
B: We can do something in parallel also , in some like some cases like , if you wanted to do voice activity detection .
A: Uh - huh .
B: And we can do that in parallel with some other filtering you can do .
D: Mmm .
B: So you can make a decision on that voice activity detection
B: and then you decide whether you want to filter or not .
D: Yeah .
B: But by then you already have the sufficient samples to do the filtering .
A: Mm - hmm .
B: So B: So , sometimes you can do it anyway .
A: I mean , couldn't ,
A: uh A: I A: Couldn't you just also I mean , i if you know that the l the largest latency in the system is two hundred milliseconds , don't you couldn't you just buffer up that number of frames
A: and then everything uses that buffer ?
B: Yeah .
A: And that way it 's not additive ?
C: Well , in fact , everything is sent over in buffers cuz of C: isn't it the TCP buffer some ?
B: You mean , the the data , the super frame or something ?
D: Mm - hmm .
C: Yeah ,
C: yeah .
D: Yeah .
B: Yeah , but that has a variable latency
B: because the last frame doesn't have any latency
D: Mm - hmm .
B: and first frame has a twenty framed latency .
B: So you can't r rely on that latency all the time .
C: Yeah .
B: Because D: Yeah .
B: I mean the transmission over over the air interface is like a buffer .
B: Twenty frame A: Yeah .
B: twenty four frames .
A: Yeah .
B: So B: But the only thing is that the first frame in that twenty - four frame buffer has a twenty - four frame latency .
B: And the last frame doesn't have any latency .
A: Mm - hmm .
B: Because it just goes as B: Yeah .
A: Yeah , I wasn't thinking of that one in particular
A: but more of , you know , if if there is some part of your system that has to buffer twenty frames , uh , can't the other parts of the system draw out of that buffer and therefore not add to the latency ?
C: Yeah .
C: Yeah .
C: And and that 's sort of one of the C: all of that sort of stuff is things that they 're debating in their standards committee .
A: Oh !
A: Hmm .
D: Mm - hmm .
D: Yeah .
D: So ,
D: um ,
D: there is uh , <comment> these parameters that I still have to to look at .
D: Like , I played a little bit with this overestimation factor ,
D: uh ,
D: but I still have to to look more at this ,
D: um ,
D: at the level of noise I add after . Uh , I know that adding noise helped , um , the system just using spectral subtraction without smoothing ,
D: but I don't know right now if it 's still important or not , and if the level I choose before is still the right one .