PhD C: Mm - hmm .
Professor F: And at that point , you know , we were pretty new to the data and we 'd never really , like , looked at it on a screen and then when we just put it on the screen {comment} and wroop !
PhD C: Mm - hmm .
Professor F: there 's this big DC offset . So , um , in PLP
PhD G: There was a like a hum or some or when they recorded it ?
Professor F: No . It 's just , it it 's it 's not at all uncommon for for recorded electronics to have different , um , DC offsets .
PhD G: Or just ? Huh .
Professor F: It 's it 's , you know , no big deal . It 's you know , you could have ten , twenty , maybe thirty millivolts , whatever , and it 's consistently in there . The thing is , most people 's front - ends have pre - emphasis with it , with zero at zero frequency , so that it 's irrelevant . Uh , but with P L P , we didn't actually have that . We had we had the equivalent of pre - emphasis in a a , uh , Fletcher - Munson style weighting that occurs in the middle of P L but it doesn't actually have a zero at zero frequency ,
PhD G: Hmm .
Professor F: like , eh , uh , typical simple fr pre - emphasis does . We had something more fancy . It was later on it didn't have that . So at that point I reali " oh sh we better have a have a high - pass filter " just , you know  just take care of the problem . So I put in a high - pass filter at , uh , I think ninety ninety hertz or so uh , for a sixteen kilohertz sampling rate . And I never put anything in to adjust it for different different sampling rates . And so well , so , you know , the code doesn't know anything about that and so this is all at eight kilohertz and so it was at forty - five hertz instead of at  instead of at ninety .
PhD C: Hmm .
Professor F: So , um , I don't know if Dan fixed it or or , uh , what he PhD C: Well , he made it a parameter .
Professor F: He made it a parameter . So . Yeah , I guess if he did it right , he did fix it and then and then it 's taking care of sampling rate , which is great .
PhD D: What what is the parameter ?
Professor F: He had a PhD D: Is it , uh , just the f lower cut - off that you want ?
PhD C: It 's called , uh , H - HPF .
Professor F: H Yeah . Does HPF on on his feat feature .
PhD C: u And but HPF , you know , when you put a number after it , uses that as the hertz value of the cut - off .
PhD D: Mm - hmm . Oh , OK .
Professor F: Yeah .
PhD C: So .
Professor F: I mean , frankly , we never did that with the RASTA filter either ,
PhD C: Mm - hmm .
Professor F: so the RASTA filter is actually doing a different thing in the modulation spectral domain depending on what sampling rate you 're doing , which is another old old bug of mine .
PhD C: Mm - hmm .
Professor F: But , um Um . So that that was the problem there was th we we we had always intended to cut off below a hundred hertz
PhD C: Mm - hmm .
Professor F: and it just wasn't doing it , so now it is . So , that hep that helped us by , like , eight tenths of a percent . It still wasn't a big deal .
PhD C: OK . Well , but , um Well , uh , again , after completing the current experiments , we 'll  we can add up all the uh differences
Professor F: Oh , yeah .
PhD C: and  and an
Professor F: But but , I guess my my point was that that , um , the hybrid system thing that we did was , uh , primitive in many ways .
PhD C: Y Right .
Professor F: And I think I agree with you that if we fixed lots of different things and they would all add up , we would probably have a a a competitive system . But I think not that much of it is due to the front - end per se . I think maybe a couple percent of it is , as far as I can see from this .
PhD C: Mm - hmm .
Professor F: Uh , unless you call well , if you call VTL the front - en front - end , that 's , uh , a little more . But that 's sort of more both , kind of .
PhD D: One experiment we should we 'll probably need to do though when um , at some point , is , since we 're using that same the net that was trained on PLP without all these things in it , for the tandem system , we may wanna go back and retrain ,
Professor F: Right ? But .
PhD C: Well , that 's what I meant , in fact . Yeah .
PhD D: yeah , yeah , for the tandem . You know , so we can see if it what effect it has on the tandem processing .
PhD C: So so , the thing is is do we expect ?
Professor F: Mm - hmm .
PhD C: eh At this point I 'm as I mean , you know e I 'm wondering is it Can we expect , uh , a tandem system to do better than a properly trained you know , a Gaussian system trained directly on the features with , you know , the right ch choice of parameters ?
Professor F: Well , that 's what we 're seeing in other areas . Yes . Right ? So , it 's so , um , um PhD D: So , we But but we may not . I mean , if it doesn't perform as well , we may not know why . Right ? Cuz we need to do the exact experiment .
PhD C: Right .
Professor F: I mean , the reason to think it should is because you 're putting in the same information and you 're transforming it to be more discriminative . So . Um . Now the thing is , in some databases I wouldn't expect it to necessarily give you much and and part of what I view as the real power of it is that it gives you a transformational capability for taking all sorts of different wild things that we do , not just th the standard front - end , but other things , like with multiple streams and so forth ,
PhD C: Mm - hmm .
Professor F: and allows you to feed them to the other system with this through this funnel . Um , so I think I think that 's the real power of it . I wouldn't expect huge in huge improvements . Um , but it should at least be roughly the same and maybe a little better .
PhD C: Mm - hmm .
Professor F: If it 's , you know , like way way worse then , you know PhD C: Right .
PhD D: So , Morgan , an another thing that Andreas and I were talking about was , so @ @ {comment} in the first experiment that he did we just took the whole fifty - six , uh , outputs and that 's , um , basically compared to a thirty - nine input feature vector from either MFCC or PLP .
Professor F: Mm - hmm . Mm - hmm .
PhD D: But one thing we could do is Professor F: Let let me let me just ask you something . When you say take the fifty - six outputs , these are the pre final nonlinearity outputs
PhD D: Yeah . Through the regular tandem outputs .
Professor F: and they 're and through the KLT .
PhD D: Through the KLT . All that kinda stuff .
Professor F: OK . And so so then you u Do you use all fifty - six of the KLT
PhD D: That 's what we did .
Professor F: or ?
PhD D: Right ? So one thing we were wondering is , if we did principal components and , say , took out just thirteen , and then did deltas and double - deltas on that Professor F: OK . Yes .
PhD D: so we treated the th first thirteen as though they were standard features .
Professor F: Yes . Yeah .
PhD D: I mean , did Dan do experiments like that to ?
Professor F: Uh . Talk with Stephane . He did some things like that . It was either him or Carmen . I forget .
PhD C: Mm - hmm .
PhD D: Mmm .
Professor F: I mean these were all different databases and different you know , in HTK and all that ,
PhD D: Yeah .
Professor F: so i it it may not apply . But my recollection of it was that it didn't make it better but it didn't make it worse .
PhD D: Hmm .
Professor F: But , again , given all these differences , maybe it 's more important in your case that you not take a lot of these low - variance , uh , components .
PhD D: Cuz in a sense , the net 's already got quite a bit of context in those features ,
Professor F: Yeah .
PhD D: so if we did deltas and double - deltas on top of those , we 're getting sort of even more .
Professor F: Which could be good or not .
PhD D: Yeah .
Professor F: Yeah . Yeah . Worth trying .
PhD C: But there the main point is that , um , you know , it took us a while but we have the procedure for coupling the two systems debugged now and I mean , there 's still conceivably some bug somewhere in the way we 're feeding the tandem features uh , either generating them or feeding them to this to the SRI system ,
PhD D: Mm - hmm . Yeah .
PhD C: but it 's Professor F: There might be , cuz that 's a pretty big difference .
PhD C: Yeah . And I 'm wondering how we can how we can debug that .
Professor F: But
PhD D: Yeah .
PhD C: I mean how Um .
Professor F: Hmm .
PhD C: I 'm actually f quite sure that the feeding the features into the system and training it up ,
Professor F: What if ?
PhD C: that that I think that 's this that 's essentially the same as we use with the ce with the P L P fe features . And that 's obviously working great . So . I um .
PhD D: Yeah . There could be a bug in in the somewhere before that .
PhD C: There we could the another degree of freedom is how do you generate the K L T transform ?
PhD D: Mm - hmm .
PhD C: Right ? We to
Professor F: That 's PhD D: Right .
Professor F: well , and another one is the normalization of the inputs to the net .
PhD C: Yeah .
Professor F: These nets are trained with particular normalization and when that gets screwed up it it can really hurt it .
PhD D: I 'm doing what Eric E Eric coached me through then that part of it , so I 'm pretty confident in that .
Professor F: OK .
PhD D: I mean , the only slight difference is that I use normalization values that , um , Andreas calculated from the original {comment} PLP ,
PhD C: Right .
PhD D: which is right .
PhD C: Right .
PhD D: N Yeah . So , I u I do Oh , we actually don't do that normalization for the PLP , do we ? For the st just the straight PLP features ?
PhD C: No . The the SRI system does it .
PhD D: S R I system does that . Right .
PhD C: Yeah .
Professor F: Right . Well , you might e e
PhD C: So , there 's there is there is room for bugs that we might not have discovered ,
PhD D: So that 's that 's another Yeah .
Professor F: Yeah .
PhD D: Mm - hmm .
Professor F: Yeah . I I would actually double check with Stephane at this point ,
PhD C: but Professor F: cuz he 's probably the one here I mean , he and Dan are the ones who are at this point most experienced with the tandem
PhD D: Mm - hmm .
Professor F: thing and there may there may be some little bit here and there that is not not being handled right .
PhD D: Yeah . It 's hard with features , cuz you don't know what they should look like . I mean , you can't just , like , print the the values out in ASCII and , you know , look at them , see if they 're Professor F: Not unless you had a lot of time
PhD G: Well Professor F: and PhD G: eh , and also they 're not I mean , as I understand it , you you don't have a way to optimize the features for the final word error . Right ?
PhD C: Right .
PhD G: I mean , these are just discriminative , but they 're not , um , optimized for the final PhD C: They 're optimized for phone discrimination , not for PhD G: Right . So it there 's always this question of whether you might do better with those features if there was a way to train it for the word error metric that you 're actually that you 're actually Professor F: That 's right . Well , the other Yeah , th the PhD C: Mm - Mmm .
Professor F: Well , you actually are . But but it but in an indirect way .
PhD G: Well , right . It 's indirect , so you don't know Professor F: So wha w what an and you may not be in this case , come to think of it , because , uh , you 're just taking something that 's trained up elsewhere . So , what what you what you do in the full procedure is you , um , uh , have an embedded training . So in fact you the the net is trained on , uh , uh , a , uh , Viterbi alignment of the training data that comes from your full system . And so that 's where the feedback comes all around , so that it is actually discriminant . You can prove that it 's it 's a , uh If you believe in the Viterbi assumption that , uh , getting the best path , uh , is almost equivalent to getting the best , uh , total probability , um , then you actually do improve that by , uh by training up on local local , uh local frames . But , um , we aren't actually doing that here , because we did we did that for a hybrid system , and now we 're plugging it into another system and so it isn't  i i i it wouldn't quite apply here .
PhD C: Do y
PhD D: So another huge experiment we could do would be to take the tandem features , uh , do SRI forced alignments using those features , and then re - do the net with those .
Professor F: Mm - hmm .
PhD G: Mmm , uh Exactly . Exactly .
Professor F: Yeah .
PhD G: So that you can optimize it for the word error .
PhD C: But Professor F: Yeah . Another thing is since you 're not using the net for recognition per se but just for this transformation , it 's probably bigger than it needs to be .
PhD G: Yeah .
Professor F: So that would save a lot of time .
PhD C: And there 's a mismatch in the phone sets . So , you 're using a l a long a larger phone set than what PhD D: Mmm .
Professor F: Yeah . Actually all those things could could could could , uh could affect it as well .
PhD D: Yeah . Yeah .
Professor F: The other thing , uh , just to mention that Stephane this was an innovation of Stephane 's , which was a pretty neat one , uh , and might particularly apply here , given all these things we 're mentioning . Um , Stephane 's idea was that , um , discriminant , uh , approaches are great . Even the local ones , given , you know , these potential outer loops which , you know , you can convince yourself turn into the global ones . Um , however , there 's times when it is not good . Uh , when something about the test set is different enough from the training set that that , uh , the discrimination that you 're learning is is is not a good one .
PhD C: Mm - hmm .
Professor F: So , uh , his idea was to take as the input feature vector to the , uh , Gaussian mixture system , uh , a concatenation of the neural net outputs and the regular features .
PhD C: Oh , we already talked about that .
PhD G: Yeah . That Professor F: Yeah .
PhD C: El
PhD D: Mm - hmm .
PhD G: Didn't you did you do that already
PhD C: Yeah . No , but we we when when we when I first started corresponding with Dan about how to go about this , I think that was one of the things that we definitely went there .
PhD G: or ? Oh . That makes a lot of sense . Huh .
Professor F: Yeah . Yeah . I mean , I 'm sure that Stephane wasn't the first to think of it ,
PhD C: Yeah .
Professor F: but actually Stephane did it
PhD C: Uh - huh . And i does it help ?
Professor F: and  and and it helped a lot .
PhD C: Oh , OK .
Professor F: Yeah . So that 's that that 's our current best best system in the , uh  uh , in the Aurora thing .
PhD C: Oh . OK .
PhD G: Yeah . That makes sense .
PhD C: And do you do a KLT transform on the con on the combined feature vector ?
Professor F: Yeah .
PhD G: As you should never do worse .
Professor F: I I , uh , missed what you said .
PhD C: Do you d you do a KLT transform on the combined feature vector ?
Professor F: Yeah .
PhD C: OK .
Professor F: Well , actually , I , uh you should check with him , because he tried several different combinations .
PhD C: Because you end up with this huge feature vector , so that might be a problem , a unless you do some form of dimensionality reduction .
Professor F: Yeah . I , uh , th what I don't remember is which came out best . So he did one where he put o put e the whole thing into one KLT , and another one , since the the PLP things are already orthogonalized , he left them alone and and just did a KLT on the on the on the net outputs
PhD C: Mm - hmm . Mmm .
Professor F: and then concatenated that . And I don't remember which was better .
PhD D: Did he did he try to ? So he always ended up with a feature vector that was twice as long as either one of the ?
Professor F: No . I don't know , i I I don't know . You have to check with him .
PhD D: Yeah .
PhD C: OK . Actually , I have to run .
Professor F: I 'm into big ideas these days .
PhD G: Yeah .
PhD C: Uh .
PhD G: We need to close up cuz I need to save the data and , um , get a call .
Professor F: Not to mention the fact that we 're missing snacks . Yeah .
PhD G: Right .
Professor F: Uh
PhD G: Did people wanna do the digits
Professor F: Um .
PhD G: or , um , do them together ?
Professor F: I I g I think , given that we 're in a hurry for snacks , maybe we should do them together .
PhD G: I don't know . Should we just ? OK . I mean , are we trying to do them {nonvocalsound} in synchrony ? That might be fun .
Professor F: Well , it 's  it 's it 's not You know , it 's not gonna work out
PhD G: Adam 's not here , so he 's not here to tell me no .
Professor F: but we could we could just , uh , uh , see if we find a rhythm , you know , what PhD G: Sure .
Professor F: Uh , O 's or zeroes , we wanna agree on that ?
PhD G: Maybe just whatever people would naturally do ? I don't know .
Professor F: Oh , but if we were a singing group , we would wanna decide . Right ?
PhD G: Be harmony . Yeah . {comment} Yeah .
Postdoc A: Mine 's identical to yours .
Professor F: We might wa
Postdoc A: Is that correct ?
PhD G: Sorry . So I set up and we didn't have enough digit forms
Postdoc A: Oh . I see .
Professor F: So these are excellent .
PhD G: so I xeroxed the same one seven times .
Postdoc A: Oh . I see .
Professor F: Why don't we do zer i Anyone have a problem with saying zero ? Is zero OK ?
PhD G: No .
Postdoc A: Yeah .
Professor F: OK . One and a two and three .
PhD G: e
Professor F: Once more with feeling .
PhD G: And th
Professor F: No , just k just kidding . Oh , yeah . It was .