E: Have you ?
E: When you adjusted those numbers for mel cepstrum , did it ?
F: Uh , I I don't remember off the top of my head .
F: Um .
F: Yeah . I didn't even write them down .
F: I I I don't remember .
F: I would need to F: Well , I did write down ,
F: um F: So , when I was doing I just wrote down some numbers for the well - matched case .
E: Yeah .
F: Um .
F: Looking at the I wrote down what the deletions , substitutions , and insertions were ,
F: uh ,
F: for different numbers of states per phone .
E: Yeah .
F: Um , but , uh , that that 's all I wrote down .
E: OK .
F: So .
F: I I would F: Yeah .
F: I would need to do that .
E: OK .
E: So F: I can do that for next week .
E: Yeah .
E: And , um E: Yeah .
E: Also , eh , eh , sometimes if you run behind on some of these things , maybe we can get someone else to do it
E: and you can supervise or something .
E: But E: but I think it would be it 'd be good to know that .
F: OK .
F: I just need to get , um , front - end , uh , stuff from you
B: Hmm .
F: or you point me to some files that you 've already calculated .
B: Yeah .
B: Alright .
E: OK .
E: Uh .
F: I probably will have time to do that and time to play a little bit with the silence model .
E: Mm - hmm .
F: So maybe I can have that for next week when Hynek 's here .
E: Yeah .
B: Mm - hmm .
E: Yeah .
E: Cuz , I mean , the the other E: That , in fact , might have been part of what , uh , the difference was E: at least part of it that that we were seeing .
E: Remember we were seeing the SRI system was so much better than the tandem system .
F: Hmm .
E: Part of it might just be that the SRI system , they they they always adjust these things to be sort of optimized ,
F: Is there ?
E: and F: I wonder if there 's anything that we could do to the front - end that would affect the insertion E: Yes .
E: I think you can .
F: What could you do ?
E: Well , um E: uh ,
E: part of what 's going on , um , is the , uh , the range of values .
E: So , if you have something that has a much smaller range or a much larger range , and taking the appropriate root .
F: Oh .
F: Mm - hmm .
E: You know ?
E: If something is kind of like the equivalent of a bunch of probabilities multiplied together , you can take a root of some sort .
E: If it 's like seven probabilities together , you can take the seventh root of it or something ,
E: or if it 's in the log domain , divide it by seven .
F: Mm - hmm .
E: But E: but ,
E: um ,
E: that has a similar effect
E: because it changes the scale of the numbers of the differences between different candidates from the acoustic model
F: Oh ,
F: right .
E: as opposed to what 's coming from the language model .
F: So that w
F: Right .
F: So , in effect , that 's changing the value of your insertion penalty .
E: Yeah .
E: I mean , it 's more directly like the the language scaling or the , uh the model scaling or acoustic scaling ,
F: That 's interesting .
E: but you know that those things have kind of a similar effect to the insertion penalty
F: Mm - hmm .
E: anyway . They 're a slightly different way of of handling it .
F: Right .
E: So , um F: So if we know what the insertion penalty is , then we can get an idea about what range our number should be in ,
F: so that they match with that .
E: I think so .
E: Yeah .
E: Yeah .
E: So that 's why I think
E: that 's another reason other than curiosity as to why i it would in fact be kinda neat to find out if we 're way off .
F: Mm - hmm .
E: I mean , the other thing is , are aren't we seeing ? Y y
E: I 'm sure you 've already looked at this
E: bu in these noisy cases , are ? We are seeing lots of insertions .
E: Right ?
E: The insertion number is quite high ?
E: I know the VAD takes pre care of part of that ,
B: Yeah .
F: Yeah .
E: but F: I 've seen that with the mel cepstrum .
B: Yeah .
F: I don't I don't know about the Aurora front - end ,
F: but B: I think it 's much more balanced with , uh when the front - end is more robust .
B: Yeah .
B: I could look at it at this .
B: Yeah .
E: Yeah .
E: Wha - what 's a typical number ?
B: Mm - hmm .
B: I don't I don't know .
E: Do we ?
E: Oh , you oh , you don't know .
E: OK .
B: I don't have this in E: I 'm sure it 's more balanced ,
E: but it it it wouldn't surprise me if there 's still B: Mm - hmm .
E: I mean , in in the the the old systems we used to do , I I uh , I remember numbers kind of like insertions being half the number of deletions , as being and both numbers being tend to be on the small side comparing to to , uh , substitutions .
B: Mm - hmm .
B: Mm - hmm .
F: Well , this F: the whole problem with insertions was what I think , um , we talked about when the guy from OGI came down that one time
F: and F: and that was when people were saying , well we should have a , uh , uh , voice activity detector E: Right .
F: that , because all that stuff <comment> that we 're getting thr the silence that 's getting through is causing insertions .
F: So .
E: Right .
B: Mmm .
F: I 'll bet you there 's still a lot of insertions .
B: Mm - hmm .
E: Yeah .
E: And it may be less of a critical thing .
E: I mean , the fact that some get by may be less of a critical thing if you , uh , get things in the right range .
F: Mm - hmm .
E: So , I mean , the insertions is is a symptom .
E: It 's a symptom that there 's something , uh , wrong with the range .
F: Right .
E: But there 's uh , your your your substitutions tend to go up as well .
E: So , uh , I I I think that ,
F: Mm - hmm .
E: uh , the most obvious thing is just the insertions , @ @ .
E: But E: Uh E: um .
E: If you 're operating in the wrong range I mean , that 's why just in general , if you change what these these penalties and scaling factors are , you reach some point that 's a that 's a minimum .
E: So .
E: Um .
E: Um .
E: We do have to do well over a range of different conditions ,
E: some of which are noisier than others .
E: Um .
E: But , um , I think we may get a better handle on that if we if we see E: Um , I mean we ca
E: it 's if we actually could pick a a a more stable value for the range of these features , it , um , uh , could E: Uh E: Even though it 's it 's it 's true that in a real situation you can in fact adjust the these these scaling factors in the back - end , and it 's ar artificial here that we 're not adjusting those , you certainly don't wanna be adjusting those all the time .
F: Hmm .
E: And if you have a nice front - end that 's in roughly the right range E: I remember after we got our stuff more or less together in the previous systems we built , that we tended to set those scaling factors at kind of a standard level ,
E: and we would rarely adjust them again ,
E: even though you could get a F: Mm - hmm .
E: for an evaluation you can get an extra point or something if you tweaked it a little bit .
E: But ,
E: once we knew what rou roughly the right operating range was , it was pretty stable ,
E: and Uh , we might just not even be in the right operating range .
F: So , would the ?
F: Uh , would a good idea be to try to map it into the same range that you get in the well - matched case ?
F: So , if we computed what the range was in well - matched , and then when we get our noisy conditions out we try to make it have the same range as ?
E: No . You don't wanna change it for different conditions .
E: No .
E: No .
E: I I I What what I 'm saying F: Oh , I wasn't suggesting change it for different conditions .
F: I was just saying that when we pick a range , we we wanna pick a range that we map our numbers into F: we should probably pick it based on the range that we get in the well - matched case .
E: Yeah .
F: Otherwise , I mean , what range are we gonna choose
F: to to map everything into ?
E: Well . It depends how much we wanna do gamesmanship and how much we wanna do E: I mean , i if he
E: it to me , actually , even if you wanna be play on the gamesmanship side , it can be kinda tricky .
E: So , I mean , what you would do is set the set the scaling factors , uh , so that you got the best number for this point four five times the  you know , and so on .
F: Mm - hmm .
E: But they might change that E: those weightings .
F: Yeah .
E: Um .
E: So E: Uh E: I just sorta think we need to explore the space .
F: Mm - hmm .
E: Just take a look at it a little bit .
F: OK .
E: And we we we may just find that that we 're way off .
F: Mm - hmm .
E: Maybe we 're not .
E: You know ?
E: As for these other things , it may turn out that , uh , it 's kind of reasonable .
E: But then E: I mean , Andreas gave a very reasonable response ,
E: and he 's probably not gonna be the only one who 's gonna say this in the future E: of , you know ,
E: people people within this tight - knit community who are doing this evaluation are accepting , uh , more or less , that these are the rules .
F: Yeah .
E: But , people outside of it who look in at the broader picture are certainly gonna say " Well , wait a minute . You 're doing all this standing on your head , uh , on the front - end ,
E: when all you could do is just adjust this in the back - end with one s one knob . "
F: Mm - hmm .
E: And
E: so we have to at least , I think , determine that that 's not true ,
E: which would be OK ,
E: or determine that it is true ,
E: in which case we want to adjust that
E: and then continue with with what we 're doing .
F: Right .
E: And as you say as you point out finding ways to then compensate for that in the front - end also then becomes a priority for this particular test ,
E: and saying you don't have to do that .
F: Mm - hmm .
E: So .
E: OK .
E: So , uh E: What 's new with you ?
B: Uh .
B: So there 's nothing new .
E: Uh , what 's old with you that 's developed ?
B: Um .
B: I 'm sorry ?
E: You E: OK .
E: What 's old with you that has developed over the last week or two ?
B: Mmm .
B: Well , so we 've been mainly working on the report
B: and and B: Yeah .
F: Mainly working on what ?
B: On the report of the work that was already done .
F: Oh .
B: Um .
F: How about that ?
B: Mm - hmm .
B: That 's all .
F: Any - anything new on the thing that , uh , you were working on with the , uh ?
C: I don't have results yet .
F: No results ?
F: Yeah .
E: What was that ?
F: The the ,
F: uh ,
A: Voicing thing .
F: voicing detector .
E: I mean , what what 's what 's going on now ?
E: What are you doing ?
C: Uh , to try to found , nnn , robust feature for detect between voice and unvoice .
C: And we w we try to use the variance of the es difference between the FFT spectrum and mel filter bank spectrum .
E: Yeah .
C: Uh , also the another parameter is relates with the auto - correlation function .
E: Uh - huh .
C: R - ze energy and the variance a also of the auto - correlation function .
E: Uh - huh .
E: So , that 's E: Yeah . That 's what you were describing , I guess , a week or two ago .
E: So .
C: Yeah .
C: But we don't have res we don't have result of the AURO for Aurora yet .
E: Mm - hmm .
C: We need to train the neural network
C: and E: So you 're training neural networks now ?
C: No ,
C: not yet .
E: So , what wha wh wha what what 's going on ?
C: Well , we work in the report , too ,
C: because we have a lot of result ,
E: Uh - huh .
C: they are very dispersed ,
C: and was necessary to to look in all the directory to to to give some more structure .
E: So . B So B: Yea
E: Yeah .
E: I if I can summarize , basically what 's going on is that you 're going over a lot of material that you have generated in furious fashion ,
E: f generating many results and doing many experiments
E: and trying to pull it together into some coherent form to be able to see wha see what happens .
C: Hm - hmm .
B: Uh ,
B: y yeah . Basically we we 've stopped , uh , experimenting ,
E: Yes ?
B: I mean . We 're just writing some kind of technical report .
B: And F: Is this a report that 's for Aurora ?
F: Or is it just like a tech report for ICSI ,
C: No .
F: or ?
B: Yeah .
C: For ICSI .
F: Ah .
F: I see .
B: Yeah .
C: Just summary of the experiment and the conclusion
C: and
C: something like that .
E: Yeah .
B: Mm - hmm .
E: OK .
E: So , my suggestion , though , is that you you not necessarily finish that .
E: But that you put it all together so that it 's you 've got you 've got a clearer structure to it .
E: You know what things are ,
E: you have things documented ,
E: you 've looked things up that you needed to look up .
E: So that , you know so that such a thing can be written .
B: Mm - hmm .
E: And , um E: When when when do you leave again ?
C: Uh , in July .
C: First of July .
E: First of July ?
E: OK .
E: And that you figure on actually finishing it in in June .
E: Because , you know , you 're gonna have another bunch of results to fit in there anyway .
B: Mm - hmm .
C: Mm - hmm .
E: And right now it 's kind of important that we actually go forward with experiments .
C: It 's not .
E: So so , I I think it 's good to pause , and to gather everything together and make sure it 's in good shape ,
E: so that other people can get access to it
E: and so that it can go into a report in June .
E: But I think to to really work on on fine - tuning the report n at this point is is probably bad timing , I I think .
B: Mm - hmm .
B: Yeah .
B: Well , we didn't B: we just planned to work on it one week on this report ,
B: not no more , anyway .
B: Um .
E: But you ma you may really wanna add other things later anyway
B: Yeah .
B: Mm - hmm .
E: because you B: Mmm .
E: There 's more to go ?
B: Yeah .
B: Well ,
B: so I don't know . There are small things that we started to to do .
B: But F: Are you discovering anything , uh , that makes you scratch your head as you write this report ,
F: like why did we do that ,
F: or why didn't we do this ,
B: Uh .
F: or ?
B: Yeah .
B: Yeah .
B: And B: Actually , there were some tables that were also with partial results .
B: We just noticed that ,
B: wh while gathering the result that for some conditions we didn't have everything .
F: Mmm .
B: But anyway .
B: Um .
B: Yeah , yeah .
B: We have , yeah , extracted actually the noises from the SpeechDat - Car .
B: And so ,
B: we can train neural network with speech and these noises .
B: Um .
B: It 's difficult to say what it will give ,
B: because when we look at the Aurora the TI - digits experiments , um , they have these three conditions that have different noises ,
B: and apparently this system perform as well on the seen noises on the unseen noises and on the seen noises .
B: But ,
B: I think this is something we have to try anyway .
B: So B: adding the noises from from the SpeechDat - Car .
B: Um .
E: That 's E: that 's , uh E: that 's permitted ?
B: Uh .
B: Well ,
B: OGI does did that .
B: Um .
B: At some point they did that for for the voice activity detector .
C: Uh , for a v VAD .
B: Right ?
B: Um .
F: Could you say it again ?
F: What what exactly did they do ?
B: They used some parts of the , um , Italian database to train the voice activity detector , I think .
B: It E: Yeah . I guess the thing is E: Yeah .
E: I guess that 's a matter of interpretation .
E: The rules as I understand it , is that in principle the Italian and the Spanish and the English E: no ,
E: Italian and the Finnish and the English ? were development data
B: Yeah .
B: And Spanish , yeah .
E: on which you could adjust things .
E: And the and the German and Danish were the evaluation data .
B: Mm - hmm .
E: And then when they finally actually evaluated things they used everything .
B: Yeah .
B: That 's right .
E: So B: Uh E: Uh ,
E: and it is true that the performance , uh , on the German was E: I mean , even though the improvement wasn't so good , the pre the raw performance was really pretty good .
B: Mm - hmm .
E: So E: And , uh ,
E: it it doesn't appear that there 's strong evidence that even though things were somewhat tuned on those three or four languages , that that going to a different language really hurt you .
E: And the noises were not exactly the same .
E: Right ?
E: Because it was taken from a different ,
E: uh E: I mean they were different drives .
B: Different cars .
E: I mean , it was it was actual different cars and so on .
B: Yeah .
B: Yeah .
E: So .
E: Um , it 's somewhat tuned .
E: It 's tuned more than , you know , a a a a B: Mm - hmm .
E: You 'd really like to have something that needed no particular noise at all ,
E: maybe just some white noise or something like that a at most .
B: Mm - hmm .
E: But that 's not really what this contest is .
E: So .
E: Um , I guess it 's OK .
B: Mm - hmm .
F: I think it 's E: That 's something I 'd like to understand before we actually use something from it ,
E: because it would