Professor E: Um . So other than reading digits , what 's our agenda ?
PhD F: I don't really have , uh , anything new . Been working on Meeting Recorder stuff . So .
Professor E: OK . Um . Do you think that would be the case for next week also ? Or is is , uh ? What 's your projection on ?
PhD F: Um .
Professor E: Cuz the one thing the one thing that seems to me we really should try , if you hadn't tried it before , because it hadn't occurred to me it was sort of an obvious thing is , um , adjusting the , uh , sca the scaling and , uh , insertion penalty sorta stuff .
PhD F: I did play with that , actually , a little bit . Um . What happens is , uh , when you get to the noisy stuff , you start getting lots of insertions .
Professor E: Right .
PhD F: And , um , so I 've tried playing around a little bit with , um , the insertion penalties and things like that .
Professor E: Yeah .
PhD F: Um . I mean , it it didn't make a whole lot of difference . Like for the well - matched case , it seemed like it was pretty good . Um . I could do more playing with that , though . And , uh Professor E: But you were looking at mel cepstrum .
PhD F: and see . Yes .
Professor E: Right .
PhD F: Oh , you 're talking about for th for our features .
Professor E: Right . So , I mean , i it it 's not the direction that you were working with that we were saying what 's the uh , what 's the best you can do with with mel cepstrum . But , they raised a very valid point ,
PhD F: Mmm .
Professor E: which , I guess So , to first order I mean , you have other things you were gonna do , but to first order , I would say that the conclusion is that if you , um , do , uh , some monkeying around with , uh , the exact HTK training and @ @ {comment} with , uh , you know , how many states and so forth , that it it doesn't particularly improve the performance . In other words , that even though it sounds pretty dumb , just applying the same number of states to everything , more or less , no matter what language , isn't so bad . Right ? And I guess you hadn't gotten to all the experiments you wanted to do with number of Gaussians ,
PhD F: Right .
Professor E: but , um , let 's just If we had to if we had to draw a conclusion on the information we have so far , we 'd say something like that . Right ?
PhD F: Mm - hmm .
Professor E: Uh , so the next question to ask , which is I think the one that that that Andreas was dre addressing himself to in the lunch meeting , is , um , we 're not supposed to adjust the back - end , but anybody using the system would .
PhD F: Yeah .
Professor E: So , if you were just adjusting the back - end , how much better would you do , uh , in noise ? Uh , because the language scaling and insertion penalties and so forth are probably set to be about right for mel cepstrum .
PhD F: Mm - hmm .
Professor E: But , um , they 're probably not at all set right for these things , particularly these things that look over , uh , larger time windows , in one way or another with with LDA and KLT and neural nets and all these things . In the fa past we 've always found that we had to increase the insertion penalty to to correspond to such things . So , I think that 's , uh , @ @ {comment} that 's kind of a first - order thing that that we should try .
PhD F: So for th so the experiment is to , um , run our front - end like normal , with the default , uh , insertion penalties and so forth , and then tweak that a little bit and see how much of a difference it makes
Professor E: So by " our front - end " I mean take , you know , the Aurora - two s take some version that Stephane has that is , you know , our current best version of something .
PhD F: if we were Mm - hmm .
Professor E: Um . I mean , y don't wanna do this over a hundred different things that they 've tried but , you know , for some version that you say is a good one . You know ? Um . How how much , uh , does it improve if you actually adjust that ?
PhD F: OK .
Professor E: But it is interesting . You say you you have for the noisy How about for the for the mismatched or or or or the or the medium mismatched conditions ? Have you ? When you adjusted those numbers for mel cepstrum , did it ?
PhD F: Uh , I I don't remember off the top of my head . Um . Yeah . I didn't even write them down . I I I don't remember . I would need to Well , I did write down , um So , when I was doing I just wrote down some numbers for the well - matched case .
Professor E: Yeah .
PhD F: Um . Looking at the I wrote down what the deletions , substitutions , and insertions were , uh , for different numbers of states per phone .
Professor E: Yeah .
PhD F: Um , but , uh , that that 's all I wrote down .
Professor E: OK .
PhD F: So . I I would Yeah . I would need to do that .
Professor E: OK . So PhD F: I can do that for next week .
Professor E: Yeah . And , um Yeah . Also , eh , eh , sometimes if you run behind on some of these things , maybe we can get someone else to do it and you can supervise or something . But but I think it would be it 'd be good to know that .
PhD F: OK . I just need to get , um , front - end , uh , stuff from you
PhD B: Hmm .
PhD F: or you point me to some files that you 've already calculated .
PhD B: Yeah . Alright .
Professor E: OK . Uh .
PhD F: I probably will have time to do that and time to play a little bit with the silence model .
Professor E: Mm - hmm .
PhD F: So maybe I can have that for next week when Hynek 's here .
Professor E: Yeah .
PhD B: Mm - hmm .
Professor E: Yeah . Cuz , I mean , the the other That , in fact , might have been part of what , uh , the difference was at least part of it that that we were seeing . Remember we were seeing the SRI system was so much better than the tandem system .
PhD F: Hmm .
Professor E: Part of it might just be that the SRI system , they they they always adjust these things to be sort of optimized ,
PhD F: Is there ?
Professor E: and PhD F: I wonder if there 's anything that we could do to the front - end that would affect the insertion Professor E: Yes . I think you can .
PhD F: What could you do ?
Professor E: Well , um uh , part of what 's going on , um , is the , uh , the range of values . So , if you have something that has a much smaller range or a much larger range , and taking the appropriate root .
PhD F: Oh . Mm - hmm .
Professor E: You know ? If something is kind of like the equivalent of a bunch of probabilities multiplied together , you can take a root of some sort . If it 's like seven probabilities together , you can take the seventh root of it or something , or if it 's in the log domain , divide it by seven .
PhD F: Mm - hmm .
Professor E: But but , um , that has a similar effect because it changes the scale of the numbers of the differences between different candidates from the acoustic model
PhD F: Oh , right .
Professor E: as opposed to what 's coming from the language model .
PhD F: So that w Right . So , in effect , that 's changing the value of your insertion penalty .
Professor E: Yeah . I mean , it 's more directly like the the language scaling or the , uh the model scaling or acoustic scaling ,
PhD F: That 's interesting .
Professor E: but you know that those things have kind of a similar effect to the insertion penalty
PhD F: Mm - hmm .
Professor E: anyway . They 're a slightly different way of of handling it .
PhD F: Right .
Professor E: So , um PhD F: So if we know what the insertion penalty is , then we can get an idea about what range our number should be in ,
Professor E: I think so .
PhD F: so that they match with that .
Professor E: Yeah . Yeah . So that 's why I think that 's another reason other than curiosity as to why i it would in fact be kinda neat to find out if we 're way off . I mean , the other thing is , are aren't we seeing ? Y y
PhD F: Mm - hmm .
Professor E: I 'm sure you 've already looked at this bu in these noisy cases , are ? We are seeing lots of insertions . Right ? The insertion number is quite high ?
PhD B: Yeah .
Professor E: I know the VAD takes pre care of part of that ,
PhD F: Yeah .
PhD B: Yeah .
Professor E: but PhD F: I 've seen that with the mel cepstrum . I don't I don't know about the Aurora front - end , but PhD B: I think it 's much more balanced with , uh when the front - end is more robust . Yeah . I could look at it at this . Yeah . Mm - hmm .
Professor E: Yeah . Wha - what 's a typical number ?
PhD B: I don't I don't know .
Professor E: Do we ? Oh , you oh , you don't know .
PhD B: I don't have this in Professor E: OK . I 'm sure it 's more balanced ,
PhD B: Mm - hmm .
Professor E: but it it it wouldn't surprise me if there 's still PhD B: Mm - hmm .
Professor E: I mean , in in the the the old systems we used to do , I I uh , I remember numbers kind of like insertions being half the number of deletions , as being and both numbers being tend to be on the small side comparing to to , uh , substitutions .
PhD B: Mm - hmm .
PhD F: Well , this the whole problem with insertions was what I think , um , we talked about when the guy from OGI came down that one time and and that was when people were saying , well we should have a , uh , uh , voice activity detector Professor E: Right .
PhD F: that , because all that stuff {comment} that we 're getting thr the silence that 's getting through is causing insertions . So .
PhD B: Mmm .
Professor E: Right .
PhD F: I 'll bet you there 's still a lot of insertions .
PhD B: Mm - hmm .
Professor E: Yeah . And it may be less of a critical thing . I mean , the fact that some get by may be less of a critical thing if you , uh , get things in the right range .
PhD F: Mm - hmm .
Professor E: So , I mean , the insertions is is a symptom . It 's a symptom that there 's something , uh , wrong with the range .
PhD F: Right .
Professor E: But there 's uh , your your your substitutions tend to go up as well . So , uh , I I I think that ,
PhD F: Mm - hmm .
Professor E: uh , the most obvious thing is just the insertions , @ @ . But Uh um . If you 're operating in the wrong range I mean , that 's why just in general , if you change what these these penalties and scaling factors are , you reach some point that 's a that 's a minimum . So . Um . Um . We do have to do well over a range of different conditions , some of which are noisier than others . Um . But , um , I think we may get a better handle on that if we if we see Um , I mean we ca it 's if we actually could pick a a a more stable value for the range of these features , it , um , uh , could Uh Even though it 's it 's it 's true that in a real situation you can in fact adjust the these these scaling factors in the back - end , and it 's ar artificial here that we 're not adjusting those , you certainly don't wanna be adjusting those all the time . And if you have a nice front - end that 's in roughly the right range PhD F: Hmm .
Professor E: I remember after we got our stuff more or less together in the previous systems we built , that we tended to set those scaling factors at kind of a standard level , and we would rarely adjust them again , even though you could get a PhD F: Mm - hmm .
Professor E: for an evaluation you can get an extra point or something if you tweaked it a little bit . But , once we knew what rou roughly the right operating range was , it was pretty stable , and Uh , we might just not even be in the right operating range .
PhD F: So , would the ? Uh , would a good idea be to try to map it into the same range that you get in the well - matched case ? So , if we computed what the range was in well - matched , and then when we get our noisy conditions out we try to make it have the same range as ?
Professor E: No . You don't wanna change it for different conditions . No . No . I I I What what I 'm saying PhD F: Oh , I wasn't suggesting change it for different conditions . I was just saying that when we pick a range , we we wanna pick a range that we map our numbers into Professor E: Yeah .
PhD F: we should probably pick it based on the range that we get in the well - matched case . Otherwise , I mean , what range are we gonna choose to to map everything into ?
Professor E: Well . It depends how much we wanna do gamesmanship and how much we wanna do I mean , i if he it to me , actually , even if you wanna be play on the gamesmanship side , it can be kinda tricky . So , I mean , what you would do is set the set the scaling factors , uh , so that you got the best number for this point four five times the  you know , and so on .
PhD F: Mm - hmm .
Professor E: But they might change that those weightings .
PhD F: Yeah .
Professor E: Um . So Uh I just sorta think we need to explore the space . Just take a look at it a little bit .
PhD F: Mm - hmm .
Professor E: And we we we may just find that that we 're way off .
PhD F: OK . Mm - hmm .
Professor E: Maybe we 're not . You know ? As for these other things , it may turn out that , uh , it 's kind of reasonable . But then I mean , Andreas gave a very reasonable response , and he 's probably not gonna be the only one who 's gonna say this in the future of , you know , people people within this tight - knit community who are doing this evaluation are accepting , uh , more or less , that these are the rules . But , people outside of it who look in at the broader picture are certainly gonna say " Well , wait a minute . You 're doing all this standing on your head , uh , on the front - end ,
PhD F: Yeah .
Professor E: when all you could do is just adjust this in the back - end with one s one knob . "
PhD F: Mm - hmm .
Professor E: And so we have to at least , I think , determine that that 's not true , which would be OK , or determine that it is true , in which case we want to adjust that and then continue with with what we 're doing . And as you say as you point out finding ways to then compensate for that in the front - end also then becomes a priority for this particular test ,
PhD F: Right .
Professor E: and saying you don't have to do that .
PhD F: Mm - hmm .