PhD A: Cool . Do you wanna go , Stephane ?
PhD D: Uh , yeah . So , I 've been , uh , working still on the spectral subtraction . Um , So to r to remind you  a little bit of of what I did before , is just to apply some spectral subtraction with an overestimation factor also to get , um , an estimate of the noise , uh , spectrum , and subtract this estimation of the noise spectrum from the , uh , signal spectrum , {comment} but subtracting more when the SNR is is , uh , low , which is a technique that it 's often used .
PhD A: " Subtracting more " , meaning ?
PhD D: So you overestimate the noise spectrum . You multiply the noise spectrum by a factor , uh , which depends on the SNR .
PhD A: Oh , OK . I see .
PhD D: So , above twenty DB , it 's one , so you just subtract the noise .
PhD A: Mm - hmm .
PhD D: And then it 's b Generally Well , I use , actually , a linear , uh , function of the SNR ,
PhD A: Mm - hmm .
PhD D: which is bounded to , like , two or three , {comment} when the SNR is below zero DB .
PhD A: Mm - hmm . Mm - hmm .
PhD D: Um , doing just this , uh , either on the FFT bins or on the mel bands , um , t doesn't yield any improvement
Professor C: Oh ! Um , uh , what are you doing with negative , uh , powers ?
PhD D: o Yeah . So there is also a threshold , of course , because after subtraction you can have negative energies ,
PhD A: Mm - hmm .
PhD D: and So what I I just do is to put , uh to to add to put the threshold first and then to add a small amount of noise , which right now is speech - shaped . Um PhD A: Speech - shaped ?
PhD D: Yeah , so it 's a it has the overall overall energy , uh pow it has the overall power spectrum of speech . So with a bump around one kilohertz .
PhD A: So when y when you talk about there being something less than zero after subtracting the noise , is that at a particular frequency bin ?
PhD D: i Uh - huh . Yeah .
PhD A: OK .
PhD D: There can be frequency bins with negative values .
PhD A: And so when you say you 're adding something that has the overall shape of speech , is that in a in a particular frequency bin ? Or you 're adding something across all the frequencies when you get these negatives ?
PhD D: For each frequencies I a I 'm adding some , uh , noise , but the a the amount of the amount of noise I add is not the same for all the frequency bins .
PhD A: Ah ! OK . I gotcha . Right .
PhD D: Uh . Right now I don't think if it makes sense to add something that 's speech - shaped , because then you have silence portion that have some spectra similar to the sp the overall speech spectra .
PhD A: Mm - hmm .
PhD D: But Yeah . So this is something I can still work on ,
PhD A: So what does that mean ?
PhD D: but Hmm .
PhD A: I 'm trying to understand what it means when you do the spectral subtraction and you get a negative . It means that at that particular frequency range you subtracted more energy than there was actually PhD D: That means that Mm - hmm . Yeah . So so yeah , you have an an estimation of the noise spectrum , but sometimes , of course , it 's as the noise is not perfectly stationary , sometimes this estimation can be , uh , too small , so you don't subtract enough . But sometimes it can be too large also . If if the noise , uh , energy in this particular frequency band drops for some reason .
PhD A: Mm - hmm . Mm - hmm .
PhD D: Mmm .
PhD A: So in in an ideal word i world {comment} if the noise were always the same , then , when you subtracted it the worst that i you would get would be a zero . I mean , the lowest you would get would be a zero , cuz i if there was no other energy there you 're just subtracting exactly the noise .
Professor C: Right .
PhD D: Mm - hmm ,
Professor C: Yep , there 's all there 's all sorts of , uh , deviations from the ideal here .
PhD D: yeah .
Professor C: I mean , for instance , you 're you 're talking about the signal and noise , um , at a particular point . And even if something is sort of stationary in ster terms of statistics , there 's no guarantee that any particular instantiation or piece of it is exactly a particular number or bounded by a particular range .
PhD D: Mm - hmm .
Professor C: So , you 're figuring out from some chunk of of of the signal what you think the noise is . Then you 're subtracting that from another chunk ,
PhD A: Mm - hmm .
Professor C: and there 's absolutely no reason to think that you 'd know that it wouldn't , uh , be negative in some places .
PhD D: Mm - hmm . Hmm .
Professor C: Uh , on the other hand that just means that in some sense you 've made a mistake because you certainly have stra subtracted a bigger number than is due to the noise .
PhD A: Mm - hmm .
Professor C: Um Also , we speak the whole where all this stuff comes from is from an assumption that signal and noise are uncorrelated . And that certainly makes sense in s in in a statistical interpretation , that , you know , over , um , all possible realizations that they 're uncorrelated
PhD A: Mm - hmm .
Professor C: or assuming , uh , ergodicity that i that i um , across time , uh , it 's uncorrelated . But if you just look at a quarter second , uh , and you cross - multiply the two things , uh , you could very well , uh , end up with something that sums to something that 's not zero . So in fact , the two signals could have some relation to one another . And so there 's all sorts of deviations from ideal in this . And and given all that , you could definitely end up with something that 's negative . But if down the road you 're making use of something as if it is a power spectrum , um , then it can be bad to have something negative . Now , the other thing I wonder about actually is , what if you left it negative ? What happens ?
PhD B: Is that the log ?
Professor C: I mean , because Um , are you taking the log before you add them up to the mel ?
PhD B: After that . No , after .
Professor C: Right . So the thing is , I wonder how if you put your thresholds after that , I wonder how often you would end up with , uh with negative values .
PhD B: But you will But you end up reducing some neighboring frequency bins @ @ in the average , right ? When you add the negative to the positive value which is the true estimate .
Professor C: Yeah . But nonetheless , uh , you know , these are it 's another f kind of smoothing , right ? that you 're doing .
PhD B: Yeah .
Professor C: Right . So , you 've done your best shot at figuring out what the noise should be , and now i then you 've subtracted it off . And then after that , instead of instead of , uh , uh , leaving it as is and adding things adding up some neighbors , you artificially push it up .
PhD B: Hmm .
Professor C: Which is , you know , it 's there 's no particular reason that that 's the right thing to do either , right ?
PhD B: Yeah , yeah .
Professor C: So , um , uh , i in fact , what you 'd be doing is saying , " well , we 're d we 're we 're going to definitely diminish the effect of this frequency in this little frequency bin in the in the overall mel summation " . It 's just a thought . I d I don't know if it would be PhD A: Sort of the opposite of that would be if if you find out you 're going to get a negative number , you don't do the subtraction for that bin .
PhD B: Yeah . Uh - huh . That is true .
Professor C: Nnn , yeah ,
PhD D: Mm - hmm .
Professor C: although PhD A: That would be almost the opposite , right ? Instead of leaving it negative , you don't do it . If your if your subtraction 's going to result in a negative number , you you don't do subtraction in that .
Professor C: Yeah , but that means that in a situation where you thought that that the bin was almost entirely noise , you left it .
PhD A: Yeah . Yeah , I 'm just saying that 's like the opposite .
PhD B: We just Professor C: Uh .
PhD B: Yeah .
Professor C: Yeah .
PhD A: Yeah .
Professor C: Well , yeah that 's that 's the opposite ,
PhD D: Mm - hmm .
Professor C: yeah .
PhD D: And , yeah , some people also if it 's a negative value they , uh , re - compute it using inter interpolation from the edges and bins .
PhD B: For frames , frequency bins .
Professor C: Yeah .
PhD D: Well , there are different things that you can do .
PhD A: Oh .
Professor C: People can also , uh , reflect it back up and essentially do a full wave rectification instead of a instead of half wave .
PhD A: Oh .
Professor C: But it was just a thought that that it might be something to try .
PhD D: Mm - hmm . Mm - hmm . Yep . Well , actually I tried , something else based on this , um , is to to put some smoothing , um , because it seems to to help or it seems to help the Wiener filtering
Professor C: Mm - hmm .
PhD D: and , mmm So what I did is , uh , some kind of nonlinear smoothing . Actually I have a recursion that computes Yeah , let me go back a little bit . Actually , when you do spectral subtraction you can , uh , find this this equivalent in the s in the spectral domain . You can uh compute , y you can say that d your spectral subtraction is a filter , um , and the gain of this filter is the , um , signal energy minus what you subtract , divided by the signal energy . And this is a gain that varies over time , and , you know , of course , uh , depending on the s on the noise spectrum and on the speech spectrum . And what happen actually is that during low SNR values , the gain is close to zero but it varies a lot . Mmm , and this this is the cause of musical noise and all these the {comment} the fact you we go below zero one frame and then you can have an energy that 's above zero .
Professor C: Mm - hmm .
PhD D: And Mmm . So the smoothing is I did a smoothing actually on this gain , uh , trajectory . But it 's the smoothing is nonlinear in the sense that I tried to not smooth if the gain is high , because in this case we know that , uh , the estimate of the gain is correct because we we are not close to to to zero , um , and to do more smoothing if the gain is low . Mmm . Um . Yeah . So , well , basically that 's this idea , and it seems to give pretty good results , uh , although I 've just just tested on Italian and Finnish . And on Italian it seems my result seems to be a little bit better than the Wiener filtering ,
PhD B: Mm - hmm . Yeah , the one you showed yesterday .
PhD D: right ?
PhD B: Right ?
Professor C: Yeah .
PhD D: Uh , I don't know if you have these improvement the detailed improvements for Italian , Finnish , and Spanish there
PhD B: Fff . No , I don't have , for each ,
PhD D: or you have just have your own .
PhD B: I I just just have the final number here .
PhD D: Mm - hmm .
Professor C: So these numbers he was giving before with the four point three , and the ten point one , and so forth , those were Italian , right ?
PhD B: Yeah , yeah , yeah . So so , no ,
Professor C: Yeah .
PhD D: Uh PhD B: I actually didn't give you the number which is the final one ,
PhD D: uh , no , we 've PhD B: which is , after two stages of Wiener filtering . I mean , that was I just well , like the overall improvement is like fifty - six point five . So ,
Professor C: Right .
PhD D: Mm - hmm .
PhD B: I mean , his number is still better than what I got in the two stages of Wiener filtering .
PhD D: Yeah .
Professor C: Right .
PhD D: On Italian . But on Finnish it 's a little bit worse , apparently .
PhD B: Mm - hmm .
PhD D: Um Professor C: But do you have numbers in terms of word error rates on on Italian ? So just so you have some sense of reference ?
PhD D: Yeah . Uh , so , it 's , uh , three point , uh , eight .
Professor C: Uh - huh .
PhD D: Am I right ?
PhD B: Oh , OK . Yeah , right , OK .
PhD D: And then , uh , d uh , nine point , uh , one .
Professor C: Mm - hmm .
PhD D: And finally , uh , sixteen point five .
Professor C: And this is , um , spectral subtraction plus what ?
PhD D: Plus plus nonlinear smoothing . Well , it 's the system it 's exactly the sys the same system as Sunil tried ,
Professor C: On - line normalization and LDA ?
PhD D: but Professor C: Yeah . Yeah .
PhD D: Yeah . But instead of double stage Wiener filtering , it 's it 's this smoothed spectral subtraction . Um , yeah .
PhD A: What is it the , um , France Telecom system uses
Professor C: Right .
PhD A: for Do they use spectral subtraction , or Wiener filtering , or ?
PhD B: They use spectral subtraction , right .
PhD D: For what ?
PhD B: French Telecom .
PhD D: It it 's Wiener filtering ,
PhD B: Oh , it 's it 's Wiener filtering .
PhD D: am I right ?
PhD A: Oh .
PhD B: Sorry .
PhD D: Well , it 's some kind of Wiener filtering PhD B: Yeah , filtering . Yeah , it 's not exactly Wiener filtering but some variant of Wiener filtering .
PhD D: Yeah .
PhD A: I see .
PhD B: Yeah .
Professor C: Yeah , plus , uh , I guess they have some sort of cepstral normalization , as well .
PhD B: s They have like yeah , th the just noise compensation technique is a variant of Wiener filtering ,
PhD D: Mm - hmm .
PhD B: plus they do some some smoothing techniques on the final filter . The th they actually do the filtering in the time domain .
PhD A: Mmm .
PhD D: Yeah .
PhD A: Hmm .
PhD B: So they would take this HF squared back , taking inverse Fourier transform . And they convolve the time domain signal with that .
PhD A: Oh , I see .
PhD B: And they do some smoothing on that final filter , impulse response .
PhD A: Hmm .
PhD D: But they also have two two different smoothing @ @ .
PhD B: I mean , I 'm I 'm @ @ .
PhD D: One in the time domain and one in the frequency domain by just taking the first , um , coefficients of the impulse response .
PhD B: But .
PhD D: So , basically it 's similar . I mean , what you did , it 's similar
PhD B: It 's similar in the smoothing and PhD D: because you have also two two kind of smoothing .
PhD B: Yeah .
PhD D: One in the time domain , and one in the frequency domain ,
PhD B: Yeah . The frequency domain .
PhD D: yeah .
PhD A: Does the smoothing in the time domain help PhD D: Um PhD A: Well , do you get this musical noise stuff with Wiener filtering or is that only with , uh , spectral subtraction ?
PhD B: No , you get it with Wiener filtering also .
PhD D: Yeah .
PhD A: Does the smoothing in the time domain help with that ? Or some other smoothing ?
PhD B: Oh , no , you still end up with zeros in the s spectrum . Sometimes .
PhD D: Yeah .
PhD A: Hmm .
Professor C: I mean , it 's not clear that these musical noises hurt us in recognition .
PhD A: Hmm .
Professor C: We don't know if they do .
PhD B: Yeah .
Professor C: I mean , they they sound bad .
PhD A: Mm - hmm .
PhD B: Yeah , I know .
Professor C: But we 're not listening to it , usually .
PhD D: Mm - hmm .
PhD A: Hmm .
PhD D: Uh , actually the the smoothing that I did do here reduced the musical noise . Well , it PhD B: Mm - hmm . Yeah , yeah ,
PhD D: Mmm .
PhD B: the PhD D: Well , I cannot you cannot hear beca well , actually what I d did not say is that this is not in the FFT bins . This is in the mel frequency bands . Um So , it could be seen as a f a a smoothing in the frequency domain because I used , in ad mel bands in addition and then the other phase of smoothing in the time domain . Mmm . But , when you look at the spectrogram , if you don't have an any smoothing , you clearly see , like in silence portions , and at the beginning and end of speech , you see spots of high energy randomly distributed over the the spectrogram .
PhD A: Mm - hmm . Mm - hmm .
PhD D: Um PhD A: That 's the musical noise ?
PhD D: Which is musical noise ,
PhD A: Mm - hmm .
PhD D: yeah , if if it If you listen to it uh , if you do this in the FFT bins , then you have spots of energy randomly distributing . And if you f if you re - synthesize these spot sounds as , like , sounds ,
PhD A: Mm - hmm .
PhD D: uh Professor C: Well , none of these systems , by the way , have I mean , y you both are are working with , um , our system that does not have the neural net ,
PhD D: And PhD B: Yep .
Professor C: right ?
PhD B: Yeah .
PhD D: Mm - hmm .
Professor C: OK . So one would hope , presumably , that the neural net part of it would would improve things further as as they did before .
PhD D: Yeah . Yeah . Um Yeah , although if if we , um , look at the result from the proposals , {comment} one of the reason , uh , the n system with the neural net was , um , more than well , around five percent better , is that it was much better on highly mismatched condition . I 'm thinking , for instance , on the TI - digits trained on clean speech and tested on noisy speech .
Professor C: Mm - hmm .
PhD D: Uh , for this case , the system with the neural net was much better .
Professor C: Mm - hmm .
PhD D: But not much on the in the other cases .
Professor C: Yeah .
PhD D: And if we have no , uh , spectral subtraction or Wiener filtering , um , i the system is Uh , we thought the neural neural network is much better than before , even in these cases of high mismatch . So , maybe the neural net will help less but , um Professor C: Maybe .
PhD A: Could you train a neural net to do spectral subtraction ?
Professor C: Yeah , it could do a nonlinear spectral subtraction
PhD D: Mm - hmm .
Professor C: but I don't know if it I mean , you have to figure out what your targets are .
PhD A: Yeah , I was thinking if you had a clean version of the signal and and a noisy version , and your targets were the M F - uh , you know , whatever , frequency bins PhD D: Mm - hmm .
Professor C: Right .
PhD D: Mm - hmm .
Professor C: Yeah , well , that 's not so much spectral subtraction then ,
PhD D: Mm - hmm .
Professor C: but but but it 's but at any rate , yeah , people , uh PhD A: People do that ?
Professor C: y yeah , in fact , we had visitors here who did that I think when you were here ba way back when .
PhD D: Mm - hmm .
PhD A: Hmm .
Professor C: Uh , people d done lots of experimentation over the years with training neural nets . And it 's not a bad thing to do . It 's another approach .
PhD A: Hmm .
Professor C: M I mean , it 's it , um PhD D: Mm - hmm .
Professor C: The objection everyone always raises , which has some truth to it is that , um , it 's good for mapping from a particular noise to clean but then you get a different noise .
PhD A: Mm - hmm .
Professor C: And the experiments we saw that visitors did here showed that it there was at least some , um , {comment} gentleness to the degradation when you switched to different noises . It did seem to help . So that you 're right , that 's another another way to go .
PhD A: How did it compare on I mean , for for good cases where it it uh , stuff that it was trained on ? Did it do pretty well ?
Professor C: Oh , yeah , it did very well .
PhD A: Mmm .
Professor C: Yeah .
PhD A: Mmm .
Professor C: Um ,
PhD D: Mm - hmm .
Professor C: but to some extent that 's kind of what we 're doing . I mean , we 're not doing exactly that , we 're not trying to generate good examples but by trying to do the best classifier you possibly can , for these little phonetic categories ,
PhD A: Mm - hmm . You could say it 's sort of built in .
Professor C: It 's Yeah , it 's kind of built into that .
PhD A: Hmm .
Professor C: And and that 's why we have found that it it does help .
PhD A: Mm - hmm .
Professor C: Um so , um , yeah , I mean , we 'll just have to try it . But I I would I would I would imagine that it will help some . I mean , it we 'll just have to see whether it helps more or less the same , but I would imagine it would help some .
PhD D: Mm - hmm .
Professor C: So in any event , all of this I was just confirming that all of this was with a simpler system .
PhD D: Yeah ,
Professor C: OK ?
PhD D: yeah . Um , Yeah , so this is th the , um Well , actually , this was kind of the first try with this spectral subtraction plus smoothing ,
Professor C: Mm - hmm .
PhD D: and I was kind of excited by the result .
Professor C: Mm - hmm .
PhD D: Um , then I started to optimize the different parameters . And , uh , the first thing I tried to optimize is the , um , time constant of the smoothing . And it seems that the one that I chose for the first experiment was the optimal one , so uh ,
Professor C: It 's amazing how often that happens .