professor e: I was saying Hynek 'll be here next week , Wednesday through Friday  , through Saturday , and , I won't be here Thursday and Friday . But my suggestion is that , at least for this meeting , people should go ahead , cuz Hynek will be here , and , you know , we don't have any Czech accent yet , as far as I know , so There we go . So other than reading digits , what 's our agenda ? phd f: I don't really have , anything new . Do you think that would be the case for next week also ? Or is is ,  ? What 's your projection on ? phd f:  . professor e: Cuz the one thing the one thing that seems to me we really should try , if you hadn't tried it before , because it hadn't occurred to me it was sort of an obvious thing is , adjusting the , sca the scaling and , insertion penalty sorta stuff . phd f: And , so I 've tried playing around a little bit with , the insertion penalties and things like that . So , i it it 's not the direction that you were working with that we were saying what 's the  , what 's the best you can do with with mel cepstrum . professor e: which , I guess So , to first order  , you have other things you were gonna do , but to first order , I would say that the conclusion is that if you , do , some monkeying around with , the exact HTK training and @ @ with , you know , how many states and so forth , that it it doesn't particularly improve the performance . In other words , that even though it sounds pretty dumb , just applying the same number of states to everything , more or less , no matter what language , isn't so bad . Right ? And I guess you hadn't gotten to all the experiments you wanted to do with number of Gaussians , phd f: Right . professor e: but , let 's just If we had to if we had to draw a conclusion on the information we have so far , we 'd say something like that . professor e: so the next question to ask , which is I think the one that that that Andreas was dre addressing himself to in the lunch meeting , is , we 're not supposed to adjust the back - end , but anybody using the system would . professor e: So , if you were just adjusting the back - end , how much better would you do , in noise ?  , because the language scaling and insertion penalties and so forth are probably set to be about right for mel cepstrum . professor e: But , they 're probably not at all set right for these things , particularly these things that look over , larger time windows , in one way or another with with LDA and KLT and neural nets and all these things . In the fa past we 've always found that we had to increase the insertion penalty to to correspond to such things . So , I think that 's , @ @ that 's kind of a first - order thing that that we should try . phd f: So for th so the experiment is to , run our front - end like normal , with the default , insertion penalties and so forth , and then tweak that a little bit and see how much of a difference it makes professor e: So by " our front - end "  take , you know , the Aurora - two s take some version that Stephane has that is , you know , our current best version of something .  , y don't wanna do this over a hundred different things that they 've tried but , you know , for some version that you say is a good one . You say you you have for the noisy How about for the for the mismatched or or or or the or the medium mismatched conditions ? Have you ? When you adjusted those numbers for mel cepstrum , did it ? phd f: I I don't remember off the top of my head . I would need to Well , I did write down ,  So , when I was doing I just wrote down some numbers for the well - matched case . Looking at the I wrote down what the deletions , substitutions , and insertions were , for different numbers of states per phone . Also , eh , eh , sometimes if you run behind on some of these things , maybe we can get someone else to do it and you can supervise or something . phd f: I probably will have time to do that and time to play a little bit with the silence model . Cuz , the the other That , in fact , might have been part of what , the difference was at least part of it that that we were seeing . professor e: Part of it might just be that the SRI system , they they they always adjust these things to be sort of optimized , phd f: Is there ? professor e: and phd f: I wonder if there 's anything that we could do to the front - end that would affect the insertion professor e: Yes . phd f: What could you do ? professor e: Well , part of what 's going on , is the , the range of values . So , if you have something that has a much smaller range or a much larger range , and taking the appropriate root . professor e: You know ? If something is kind of like the equivalent of a bunch of probabilities multiplied together , you can take a root of some sort . If it 's like seven probabilities together , you can take the seventh root of it or something , or if it 's in the log domain , divide it by seven . professor e: But but , that has a similar effect because it changes the scale of the numbers of the differences between different candidates from the acoustic model phd f: Oh , right .  , it 's more directly like the the language scaling or the ,  the model scaling or acoustic scaling , phd f: That 's interesting . professor e: but you know that those things have kind of a similar effect to the insertion penalty phd f:   professor e: So ,  phd f: So if we know what the insertion penalty is , then we can get an idea about what range our number should be in , professor e: I think so . So that 's why I think that 's another reason other than curiosity as to why i it would in fact be kinda neat to find out if we 're way off . professor e: I 'm sure you 've already looked at this bu in these noisy cases , are ? We are seeing lots of insertions . professor e: I know the VAD takes pre care of part of that , phd f: Yeah . I don't I don't know about the Aurora front - end , but phd b: I think it 's much more balanced with ,  when the front - end is more robust . Wha - what 's a typical number ? phd b: I don't I don't know . professor e: but it it it wouldn't surprise me if there 's still phd b:   professor e: in in the the the old systems we used to do , I I  , I remember numbers kind of like insertions being half the number of deletions , as being and both numbers being tend to be on the small side comparing to to , substitutions . phd f: Well , this the whole problem with insertions was what I think , we talked about when the guy from OGI came down that one time and and that was when people were saying , well we should have a , voice activity detector professor e: Right . phd f: that , because all that stuff that we 're getting thr the silence that 's getting through is causing insertions .  , the fact that some get by may be less of a critical thing if you , get things in the right range . If you 're operating in the wrong range  , that 's why just in general , if you change what these these penalties and scaling factors are , you reach some point that 's a that 's a minimum . We do have to do well over a range of different conditions , some of which are noisier than others . But , I think we may get a better handle on that if we if we see  ,  we ca it 's if we actually could pick a a a more stable value for the range of these features , it , could  Even though it 's it 's it 's true that in a real situation you can in fact adjust the these these scaling factors in the back - end , and it 's ar artificial here that we 're not adjusting those , you certainly don't wanna be adjusting those all the time . And if you have a nice front - end that 's in roughly the right range phd f:  . professor e: I remember after we got our stuff more or less together in the previous systems we built , that we tended to set those scaling factors at kind of a standard level , and we would rarely adjust them again , even though you could get a phd f:   professor e: for an evaluation you can get an extra point or something if you tweaked it a little bit . But , once we knew what rou roughly the right operating range was , it was pretty stable , and  , we might just not even be in the right operating range . phd f: So , would the ?  , would a good idea be to try to map it into the same range that you get in the well - matched case ? So , if we computed what the range was in well - matched , and then when we get our noisy conditions out we try to make it have the same range as ? professor e: No . I I I What what I 'm saying phd f: Oh , I wasn't suggesting change it for different conditions . I was just saying that when we pick a range , we we wanna pick a range that we map our numbers into professor e: Yeah . phd f: we should probably pick it based on the range that we get in the well - matched case . It depends how much we wanna do gamesmanship and how much we wanna do  , i if he it to me , actually , even if you wanna be play on the gamesmanship side , it can be kinda tricky . So , what you would do is set the set the scaling factors , so that you got the best number for this point four five times the you know , and so on . You know ? As for these other things , it may turn out that , it 's kind of reasonable . But then  , Andreas gave a very reasonable response , and he 's probably not gonna be the only one who 's gonna say this in the future of , you know , people people within this tight - knit community who are doing this evaluation are accepting , more or less , that these are the rules . But , people outside of it who look in at the broader picture are certainly gonna say " Well , wait a minute . You 're doing all this standing on your head , on the front - end , phd f: Yeah . professor e: when all you could do is just adjust this in the back - end with one s one knob . professor e: And so we have to at least , I think , determine that that 's not true , which would be OK , or determine that it is true , in which case we want to adjust that and then continue with with what we 're doing . And as you say as you point out finding ways to then compensate for that in the front - end also then becomes a priority for this particular test , phd f: Right . professor e: what 's old with you that 's developed ? phd b: I 'm sorry ? professor e: You OK . What 's old with you that has developed over the last week or two ? phd b: Mmm . phd f: Mainly working on what ? phd b: On the report of the work that was already done . phd f: How about that ? Any - anything new on the thing that , you were working on with the ,  ? phd c: I don't have results yet . professor e: What was that ? phd f: The the , grad a: Voicing thing . professor e: what what 's what 's going on now ? What are you doing ? phd c: to try to found , nnn , robust feature for detect between voice and unvoice . And we w we try to use the variance of the es difference between the FFT spectrum and mel filter bank spectrum . But we don't have res we don't have result of the AURO for Aurora yet . phd c: and professor e: So you 're training neural networks now ? phd c: No , not yet . professor e: So , what wha wh wha what what 's going on ? phd c: Well , we work in the report , too , because we have a lot of result , professor e:  - huh . phd c: they are very dispersed , and was necessary to to look in all the directory to to to give some more structure . I if I can summarize , basically what 's going on is that you 're going over a lot of material that you have generated in furious fashion , f generating many results and doing many experiments and trying to pull it together into some coherent form to be able to see wha see what happens . And phd f: Is this a report that 's for Aurora ? Or is it just like a tech report for ICSI , phd c: No . But that you put it all together so that it 's you 've got you 've got a clearer structure to it . You know what things are , you have things documented , you 've looked things up that you needed to look up . professor e: So so , I I think it 's good to pause , and to gather everything together and make sure it 's in good shape , so that other people can get access to it and so that it can go into a report in June . But I think to to really work on on fine - tuning the report n at this point is is probably bad timing , I I think . Well , we didn't we just planned to work on it one week on this report , not no more , anyway . But phd f: Are you discovering anything , that makes you scratch your head as you write this report , like why did we do that , or why didn't we do this , phd b:  . We just noticed that , wh while gathering the result that for some conditions we didn't have everything . It 's difficult to say what it will give , because when we look at the Aurora the TI - digits experiments , they have these three conditions that have different noises , and apparently this system perform as well on the seen noises on the unseen noises and on the seen noises . phd f: Could you say it again ? What what exactly did they do ? phd b: They used some parts of the , Italian database to train the voice activity detector , I think . The rules as I understand it , is that in principle the Italian and the Spanish and the English no , Italian and the Finnish and the English ? were development data phd b: Yeah .  professor e: So  , and it is true that the performance , on the German was  , even though the improvement wasn't so good , the pre the raw performance was really pretty good . professor e: So And , it it doesn't appear that there 's strong evidence that even though things were somewhat tuned on those three or four languages , that that going to a different language really hurt you . professor e: You 'd really like to have something that needed no particular noise at all , maybe just some white noise or something like that a at most . professor e: That 's something I 'd like to understand before we actually use something from it , phd f: I think it 's professor e: because it would phd f: it 's probably something that , mmm , the you know , the , experiment designers didn't really think about , because I think most people aren't doing trained systems , or , you know , systems that are like ours , where you actually use the data to build models . professor e: except that , that 's what we used in Aurora one , and then they designed the things for Aurora - two knowing that we were doing that . phd f: And they didn't forbid us right ? to build models on the data ? professor e: No . But , I think I think that it it it probably would be the case that if , say , we trained on Italian , data and then , we tested on Danish data and it did terribly , that that it would look bad . You know , maybe there 's parameters that other people have used you know , th that they have tuned in some way for other things . So it 's it 's ,  We should we should Maybe that 's maybe a topic Especially if you talk with him when I 'm not here , that 's a topic you should discuss with Hynek phd b:   phd f: Do we know anything about the speakers for each of the , training utterances ? phd b: What do you mean ? We we phd f: Do you have speaker information ? professor e: Social security number phd f: That would be good . professor e: What kind of information do you mean ? phd f: Well , I was thinking about things like , you know , gender ,  you know , gender - specific nets and , vocal tract length normalization . I d I don't I didn't know what information we have about the speakers that we could try to take advantage of .  , again , i if you had the whole system you were optimizing , that would be easy to see . But if you 're supposedly just using a fixed back - end and you 're just coming up with a feature vector , w w I 'm not sure  , having the two nets Suppose you detected that it was male , it was female you come up with different phd f: Well , you could put them both in as separate streams or something . phd b: Do you have something simple in mind for  , vocal tract length normalization ? phd f:  no . And so I I  , you could maybe use the ideas a similar idea to what they do in vocal tract length normalization . You know , you have some sort of a , general speech model , you know , maybe just a mixture of Gaussians that you evaluate every utterance against , and then you see where each , you know , utterance like , the likelihood of each utterance . You divide the the range of the likelihoods up into discrete bins and then each bin 's got some knob  , setting .  , that  really doesn't sound like a real - time thing with less than two hundred milliseconds , latency that and where you 're not adjusting the statistical engine at all . You know , you can only Right ? phd f: Oh , professor e: Each frame comes in and it 's gotta go out the other end . professor e: But as far as ,  Like I thought BBN did a thing with , vocal tract normalization a ways back . With with , l trying to identify third formant average third formant using that as an indicator of phd f: I don't know . You know , third formant I if you imagine that to first order what happens with , changing vocal tract is that , the formants get moved out by some proportion phd f:   professor e: So , if you had a first formant that was one hundred hertz before , if the fifty if the vocal tract is fifty percent shorter , then it would be out at seven fifty hertz , and so on . Whereas the third formant which might have started off at twenty - five hundred hertz , you know , might be out to thirty - seven fifty , you know so it 's at So , although , you frequently get less distinct higher formants , it 's still third formant 's kind of a reasonable compromise , and phd f:   professor e: So , I think , eh , if I recall correctly , they did something like that . professor e: You know ? That 's more like looking at third formant over over a turn or something like that , phd b:   But on the other hand , male female is a is a is a much simpler categorization than figuring out a a factor to , squish or expand the the spectrum . Y you could imagine that  , just like we 're saying voiced - unvoiced is good to know  , male female is good to know also . professor e: But , you 'd have to figure out a way to to to , incorporate it on the fly .  , I guess , as you say , one thing you could do is simply , have the the male and female output vectors you know , tr nets trained only on males and n trained only on females or or , you know . I don't know if that would really help , because you already have males and females and it 's  -  putting into one net . So is it ? phd f: Is it balanced , in terms of gender the data ? phd b: Mmm . There is something perhaps , I could spend some days to look at this thing , cuz it seems that when we train networks on let 's say , on TIMIT with MSG features , they they look as good as networks trained on PLP . But , when they are used on on the SpeechDat - Car data , it 's not the case oh , well . The MSG features are much worse , and so maybe they 're , less more sensitive to different recording conditions , or Shou professor e: Shouldn't be . What what 's the ,  ? Do you kno recall if the insertions were were higher with MSG ? phd b: I don't know . professor e: so ,  MSG is very , very dif Eh , PLP is very much like mel cepstrum . professor e: So , if it 's very different , then this is the sort of thing  I 'm really glad Andreas brought this point up . And even though we 're not allowed to do that , again we maybe could reflect that back to our use of the features . professor e: So if it if in fact ,  The problem might be that the range of the MSG features is quite different than the range of the PLP or mel cepstrum . But , it 's d it 's after Well , it 's tandem features , so Mmm . We we have estimation of post posteriors with PLP and with MSG as input , professor e: Yeah . professor e: But i it it it it doesn't necessarily You know , they could be ,  Do - doesn't tell you what the variance of the things is . professor e: Right ? Cuz if you 're taking the log of these things , it could be ,  Knowing what the sum of the probabilities are , doesn't tell you what the sum of the logs are . So we should look at the likelihood , or or what ? Or well , at the log , perhaps , and professor e: Yeah . professor e: Or what you know , what you 're  the thing you 're actually looking at . But professor e: What do they look like ? phd f: No And so th the ,  for the tandem system , the values that come out of the net don't go through the sigmoid . Whatever they are at that point , are they something for which taking a square root or cube root or fourth root or something like that is is gonna be a good or a bad thing ? So . professor e: and that 's something that nothing nothing else after that is gonna  , things are gonna scale it  , you know , subtract things from it , scale it from it , but nothing will have that same effect . Cuz if if the log probs that are coming out of the MSG are really big , the standard insertion penalty is gonna have very little effect professor e: Well , the Right . It 's something that , and then it 's going through this transformation that 's probably pretty close to It 's , eh , whatever the KLT is doing . professor e: But still it 's it 's not gonna probably radically change the scale of things . It may be entirely off and and it may be at the very least it may be quite different for MSG than it is for mel cepstrum or PLP . So that would be So the first thing I 'd look at without adjusting anything would just be to go back to the experiment and look at the , substitutions , insertions , and deletions . And if the if the ,  i if there 's a fairly large effect of the difference , say , the r ratio between insertions and deletions for the two cases then that would be , an indicator that it might might be in that direction . But , professor e: Anything else ? phd b: my my point was more that it it works sometimes and but sometimes it doesn't work . phd b: And it works on TI - digits and on SpeechDat - Car it doesn't work , and professor e: Yeah . professor e: and And , sometimes , you know , there 's enough evidence for something to work and then it 's harder , it breaks . professor e: so it 's But it but , i it it could be that when you say it works maybe we could be doing much better , even in TI - digits . Well , there is also the spectral subtraction , which ,  I think maybe we should , try to integrate it in in our system . phd b: But , professor e: O phd b: I think that would involve to to mmm use a big a al already a big bunch of the system of Ericsson . Because he has spectral subtraction , then it 's followed by , other kind of processing that 's are dependent on the  , if it 's speech or noi or silence . phd b: And there is this kind of spectral flattening after if it 's silence , and and s I I think it 's important , to reduce this musical noise and this this increase of variance during silence portions . This was in this would involve to take almost everything from from the this proposal and and then just add some kind of on - line normalization in in the neural network . Well , I took a lot of time just getting my taxes out of the way multi - national taxes . So , I 'm I 'm starting to write code now for my work but I don't have any results yet .  , i it would be good for me to talk to Hynek , I think , when he 's here . grad d: Do you know what his schedule will be like ? professor e: he 'll be around for three days . I 'll ,  You know , he 's he 'll he 'll be talking with everybody in this room So . phd f: But you said you won't you won't be here next Thursday ? professor e: Not Thursday and Friday . That 's just that 's that 's one of the big advantages of not making much money is the taxes are easier . professor e: Have to do So you you have to do two returns ? grad d: Mmm . For tw That 's right , ju phd f: But not for this next year ? professor e: Two thousand . grad d: I 'll I 'll still have a bit of Canadian income but it 'll be less complicated because I will not be a considered a resident of Canada anymore , so I won't have to declare my American income on my Canadian return . I just , continuing looking at , ph  , phonetic events , and , this Tuesday gonna be , meeting with John Ohala with Chuck to talk some more about these , ph  , phonetic events . No  , why don't you say something about what it is ? grad a: Oh , you oh , you want you want details . I thought we 'd , you know grad a: I was hoping I could wave my hands . So , once wa I I was thinking getting getting us a set of acoustic events to  , to be able to distinguish between , phones and words and stuff . And  , once we we would figure out a set of these events that can be , you know , hand - labeled or or derived , from h the hand - labeled phone targets .  , we could take these events and , do some cheating experiments , where we feed , these events into an SRI system , eh , and evaluate its performance on a Switchboard task . grad d: Hey , Barry ? Can you give an example of an event ? grad a: Yeah . professor e: Whose paper is it ? grad a: this is a paper by Hubener and Cardson Benson Bernds - Berndsen . phd f: There 's ,  in my mind , anyways , there 's a difference between , acoustic features and acoustic events . And I think of acoustic features as being , things that linguists talk about , like ,  professor e: So , stuff that 's not based on data . Versus an acoustic event , which is just some something in the acoustic signal that is fairly easy to measure . professor e: when we did the SPAM work  , there we had we had this notion of an , auditory @ @ auditory event .  , there 's certainly a bunch of a bunch of places where you know that neurons are gonna fire because something novel has happened . But there 's certainly other things beyond what we talked about there that aren't just sort of rapid changes , but phd f: It 's kinda like the difference between top - down and bottom - up . You know , you look at the phone and you say this phone is supposed to be you know , have this feature , this feature , and this feature . phd f: What ? And then that you know , that may map to this phone sometimes , and sometimes it may not . And ,  and then from that point on , I would , s design robust event detectors , in a similar , wa spirit that Saul has done w  , with his graphical models , and this this probabilistic AND - OR model that he uses .  , eh , try to extend it to ,  to account for other other phenomena like , CMR co - modulation release . And ,  and maybe also investigate ways to to modify the structure of these models , in a data - driven way , similar to the way that , Jeff Jeff , Bilmes did his work .  , and while I 'm I 'm doing these , event detectors , you know , I can ma mea measure my progress by comparing , the error rates in clean and noisy conditions to something like , neural nets .  , and So so , once we have these these , event detectors , we could put them together and and feed the outputs of the event detectors into into the SRI ,   system , and ,  and test it on on Switchboard or , maybe even Aurora stuff . professor e: By the way , there 's , a couple people who are gonna be here I forget if I already told you this , but , a couple people who are gonna be here for six months . professor e: there 's a Professor Kollmeier , from Germany who 's , quite big in the , hearing - aid signal - processing area and , Michael Kleinschmidt , who 's worked with him , who also looks at auditory properties inspired by various , brain function things . professor e: So , I think they 'll be interesting to talk to , in this sort of issue as these detectors are are , developing . professor e: So , he looks at interesting interesting things in in the different ways of looking at spectra in order to to get various speech properties out . And like I say , I I encourage you to go ahead and meet , next week with , Hynek