PhD A: So , uh , you want to go next , Dave ? And we 'll do Grad E: Oh . OK , sure . So , um , last week I finally got results from the SRI system about this mean subtraction approach . And , um , we we got an improvement , uh , in word error rate , training on the TI - digits data set and testing on Meeting Recorder digits of , um , six percent to four point five percent , um , on the n on the far - mike data using PZM F , but , um , the near - mike performance worsened , um , from one point two percent to two point four percent . And , um , wh why would that be , um , considering that we actually got an improvement in near - mike performance using HTK ? And so , uh , with some input from , uh , Andreas , I have a theory in two parts . Um , first of all HTK sorry , SR - the SRI system is doing channel adaptation , and so HTK wasn't . Um , so this , um This mean subtraction approach will do a kind of channel normalization and so that might have given the HTK use of it a boost that wouldn't have been applied in the SRI case . And also , um , the Andreas pointed out the SRI system is using more parameters . It 's got finer - grained acoustic models . So those finer - grained acoustic models could be more sensitive to the artifacts in the re - synthesized audio . Um . And me and Barry were listening to the re - synthesized audio and sometimes it seems like you get of a bit of an echo of speech in the background . And so that seems like it could be difficult for training , cuz you could have different phones lined up with a different foreground phone , um , depending on the timing of the echo . So , um , I 'm gonna try training on a larger data set , and then , eh , the system will have seen more examples o of these artifacts and hopefully will be more robust to them . So I 'm planning to use the Macrophone set of , um , read speech , and , um Hmm .
Professor B: I had another thought just now , which is , uh , remember we were talking before about we were talking in our meeting about , uh , this stuff that some of the other stuff that Avendano did , where they were , um , getting rid of low - energy sections ? Um , uh , if you if you did a high - pass filtering , as Hirsch did in late eighties to reduce some of the effects of reverberation , uh , uh , Avendano and Hermansky were arguing that , uh , perhaps one of the reasons for that working was ma may not have even been the filtering so much but the fact that when you filter a an all - positive power spectrum you get some negative values , and you gotta figure out what to do with them if you 're gonna continue treating this as a power spectrum . So , what what Hirsch did was , uh , set them to zero set the negative values to zero . So if you imagine a a waveform that 's all positive , which is the time trajectory of energy , um , and , uh , shifting it downwards , and then getting rid of the negative parts , that 's essentially throwing away the low - energy things . And it 's the low - energy parts of the speech where the reverberation is most audible . You know , you have the reverberation from higher - energy things showing up in So in this case you have some artificially imposed reverberation - like thing . I mean , you 're getting rid of some of the other effects of reverberation , but because you have these non - causal windows , you 're getting these funny things coming in , uh , at n And , um , what if you did ? I mean , there 's nothing to say that the the processing for this re - synthesis has to be restricted to trying to get it back to the original , according to some equation . I mean , you also could , uh , just try to make it nicer .
Grad E: Uh - huh .
Professor B: And one of the things you could do is , you could do some sort of VAD - like thing
Grad E: Mm - hmm .
Professor B: and you actually could take very low - energy sections and set them to some some , uh , very low or or near zero value . I mean , uh , I 'm just saying if in fact it turns out that that these echoes that you 're hearing are , uh Grad E: Uh - huh .
Professor B: or pre - echoes , whichever they are are are , uh , part of what 's causing the problem , you actually could get rid of them .
Grad E: Uh - huh .
Professor B: Be pretty simple . I mean , you do it in a pretty conservative way
Grad E: OK .
Professor B: so that if you made a mistake you were more likely to keep in an echo than to throw out speech .
Grad E: Hmm .
PhD G: Um , what is the reverberation time like there ?
Grad E: In thi in this room ? Uh PhD G: On , uh , the the one what the s in the speech that you are you are using like ?
Grad E: Y Yeah . I I I I don't know .
Professor B: So , it 's this room .
PhD G: It 's , uh Professor B: It 's it 's this room .
PhD G: Oh , this room ?
Professor B: So PhD G: OK .
Professor B: so it 's these are just microphone this micro close microphone and a distant microphone , he 's doing these different tests on .
Grad F: Oh .
Professor B: Uh , we should do a measurement in here . I g think we never have . I think it 's I would guess , uh , point seven , point eight seconds f uh , R T
Grad F: Hmm !
Professor B: something like that ? But it 's you know , it 's this room .
PhD G: Mm - hmm .
Professor B: So .
PhD G: OK . Mm - hmm .
Professor B: Uh . But the other thing is , he 's putting in w I was using the word " reverberation " in two ways . He 's also putting in , uh , a he 's taking out some reverberation , but he 's putting in something , because he has averages over multiple windows stretching out to twelve seconds , which are then being subtracted from the speech . And since , you know , what you subtract , sometimes you 'll be you 'll be subtracting from some larger number and sometimes you won't . And PhD G: Mm - hmm . Mm - hmm .
Professor B: So you can end up with some components in it that are affected by things that are seconds away . Uh , and if it 's a low energy compo portion , you might actually hear some funny things .
PhD G: Yeah .
Grad E: O o one thing , um , I noticed is that , um , the mean subtraction seems to make the PZM signals louder after they 've been re - synthesized . So I was wondering , is it possible that one reason it helped with the Aurora baseline system is just as a kind of gain control ? Cuz some of the PZM signals sound pretty quiet if you don't amplify them .
PhD C: Mm - hmm . I don't see why why your signal is louder after processing , because yo
Grad E: Yeah . I don't know why - y , uh , either .
PhD C: Yeah .
Professor B: I don't think just multiplying the signal by two would have any effect .
PhD C: Mm - hmm .
Grad E: Oh , OK .
Professor B: Yeah . I mean , I think if you really have louder signals , what you mean is that you have better signal - to - noise ratio .
PhD C: Well , well Professor B: So if what you 're doing is improving the signal - to - noise ratio , then it would be better .
PhD C: Mm - hmm .
Professor B: But just it being bigger if with the same signal - to - noise ratio Grad E: It w i i it wouldn't affect things .
Professor B: No .
PhD C: Yeah .
Grad E: OK .
PhD C: Well , the system is use the absolute energy , so it 's a little bit dependent on on the signal level . But , not so much , I guess .
Professor B: Well , yeah . But it 's trained and tested on the same thing .
PhD C: Mmm .
Professor B: So if the if the if you change in both training and test , the absolute level by a factor of two , it will n have no effect .
PhD C: Mm - hmm . Yeah .
PhD A: Did you add this data to the training set , for the Aurora ? Or you just tested on this ?
Grad E: Uh Um . Did I w what ?
PhD A: Well , Morgan was just saying that , uh , as long as you do it in both training and testing , it shouldn't have any effect .
Grad E: Sorry ? Yeah .
PhD A: But I I was sort of under the impression that you just tested with this data .
Grad E: I I b
PhD A: You didn't train it also .
Grad E: I Right . I trained on clean TI - digits . I I did the mean subtraction on clean TI - digits . But I didn't I 'm not sure if it made the clean ti TI - digits any louder .
Professor B: Oh , I see .
Grad E: I only remember noticing it made the , um , PZM signal louder .
Professor B: OK . Well , I don't understand then . Yeah .
Grad E: Huh . I don't know . If it 's if it 's like , if it 's trying to find a a reverberation filter , it could be that this reverberation filter is making things quieter . And then if you take it out that taking it out makes things louder . I mean .
Professor B: Uh , no . I mean , uh , there 's there 's nothing inherent about removing if you 're really removing ,
Grad E: Nuh - huh .
Professor B: uh , r uh , then I don't see how that would make it louder .
Grad E: The mean . OK . Yeah , I see .
Professor B: So it might be just some Grad E: Yeah . OK . So I should maybe listen to that stuff again .
Professor B: Yeah . It might just be some artifact of the processing that that , uh , if you 're Uh , yeah . I don't know .
Grad E: Oh . OK .
PhD A: I wonder if there could be something like , uh for s for the PZM data ,
PhD C: Eh
PhD A: uh , you know , if occasionally , uh , somebody hits the table or something , you could get a spike . Uh . I 'm just wondering if there 's something about the , um you know , doing the mean normalization where , uh , it it could cause you to have better signal - to - noise ratio . Um .
Professor B: Well , you know , there is this . Wait a minute . It it i maybe i If , um Subtracting the the mean log spectrum is is is like dividing by the spectrum . So , depending what you divide by , if your if s your estimate is off and sometimes you 're you 're you 're getting a small number , you could make it bigger .
PhD A: Mm - hmm .
Grad E: Mm - hmm .
Professor B: So , it 's it 's just a a question of there 's It it could be that there 's some normalization that 's missing , or something to make it Grad E: Mm - hmm .
Professor B: Uh , y you 'd think it shouldn't be larger , but maybe in practice it is . That 's something to think about .
Grad E: Hmm .
Professor B: I don't know .
PhD C: I had a question about the system the SRI system . So , you trained it on TI - digits ? But except this , it 's exactly the same system as the one that was tested before and that was trained on Macrophone . Right ? So on TI - digits it gives you one point two percent error rate and on Macrophone it 's still O point eight . Uh , but is it exactly the same system ?
Grad E: Uh . I think so .
PhD C: Hmm .
Grad E: If you 're talking about the Macrophone results that Andreas had about , um , a week and a half ago , I think it 's the same system .
PhD C: Mm - hmm . So you use VTL - uh , vocal tract length normalization and , um , like MLLR transformations also ,
Grad E: Mm - hmm .
PhD C: and Professor B: I 'm sorry , was his point eight percent , er , a a result on testing on Macrophone or or training ?
PhD C: all that stuff .
Grad E: That 's PhD C: It was training on Macrophone and testing yeah , on on meeting digits .
Professor B: Oh . So that was done already . So we were Uh , and it 's point eight ? OK .
PhD C: Mm - hmm .
Professor B: OK .
PhD C: Yeah . I I 've just been text {comment} testing the new Aurora front - end with well , Aurora system actually so front - end and HTK , um , acoustic models on the meeting digits and it 's a little bit better than the previous system . We have I have two point seven percent error rate . And before with the system that was proposed , it 's what ? It was three point nine . So .
Professor B: Oh , that 's a lot better .
PhD C: We are getting better .
Professor B: So , what w ?
PhD C: And PhD G: With the with the HTK back - end ? What we have for Aurora ?
PhD C: Yeah . Two point seven .
PhD G: I know in the meeting , like PhD C: On the meeting we have two point seven .
PhD G: Right . Oh .
Grad F: That 's with the new IIR filters ?
PhD C: Uh . Yeah , yeah . So , yeah ,
Grad F: OK .
PhD C: we have the new LDA filters , and I think , maybe I didn't look , but one thing that makes a difference is this DC offset compensation . Uh , eh Do y did you have a look at at the meet uh , meeting digits , if they have a DC component , or ?
Grad E: I I didn't . No .
PhD C: Oh .
Professor B: Hmm .
PhD G: No . The DC component could be negligible . I mean , if you are recording it through a mike . I mean , any all of the mikes have the DC removal some capacitor sitting right in that bias it .
Professor B: Yeah . But this uh , uh , uh , no . Because , uh , there 's a sample and hold in the A - toD. And these period these typically do have a DC offset .
PhD G: Oh , OK .
Professor B: And and they can be surprisingly large . It depends on the electronics .
PhD G: Oh , so it is the digital OK . It 's the A - toD that introduces the DC in .
Professor B: Yeah . The microphone isn't gonna pass any DC .
PhD G: Yeah . Yeah . Yeah .
Professor B: But but ,
PhD G: OK .
Professor B: typi you know , unless Actually , there are instrumentation mikes that that do pass go down to DC . But but ,
PhD G: Mm - hmm .
Professor B: uh , no , it 's the electronics . And they and PhD G: Mm - hmm .
Professor B: then there 's amplification afterwards . And you can get , I think it was I think it was in the Wall Street Journal data that that I can't remember , one of the DARPA things . There was this big DC - DC offset
PhD A: Mm - hmm .
Professor B: we didn't we didn't know about for a while , while we were messing with it . And we were getting these terrible results . And then we were talking to somebody and they said , " Oh , yeah . Didn't you know ? Everybody knows that . There 's all this DC offset in th " So , yes . You can have DC offset in the data .
PhD G: Oh , OK .
Professor B: Yeah .
PhD G: OK .
PhD A: So was that was that everything , Dave ?
Grad E: Oh . And I also , um , did some experiments about normalizing the phase . Um . So I c I came up with a web page that people can take a look at . And , um , the interesting thing that I tried was , um , Adam and Morgan had this idea , um , since my original attempts to , um , take the mean of the phase spectra over time and normalize using that , by subtracting that off , didn't work . Um , so , well , that we thought that might be due to , um , problems with , um , the arithmetic of phases . They they add in this modulo two pi way and , um , there 's reason to believe that that approach of taking the mean of the phase spectrum wasn't really mathematically correct . So , what I did instead is I took the mean of the FFT spectrum without taking the log or anything , and then I took the phase of that , and I subtracted that phase off to normalize . But that , um , didn't work either .
Professor B: See , we have a different interpretation of this . He says it doesn't work . I said , I think it works magnificently , but just not for the task we intended . Uh , it gets rid of the speech .
PhD A: What does it leave ?
Grad F: Uh , gets rid of the speech .
Professor B: Uh , it leaves you know , it leaves the junk . I mean , I I think it 's it 's tremendous .
Grad F: Oh , wow .
Professor B: You see , all he has to do is go back and reverse what he did before , and he 's really got something .
PhD A: Well , could you take what was left over and then subtract that ?
Professor B: Ex - exactly . Yeah , you got it .
Grad F: Yeah .
PhD G: Yeah .
Professor B: So , it 's it 's a general rule .
PhD G: Oh , it 's Professor B: Just listen very carefully to what I say and do the opposite . Including what I just said .
Grad E: And , yeah , that 's everything .