grad e: Hello ? professor a: So everybody everybody 's on ? phd d: Today 's professor a: Yeah . professor a: eh e and  I guess Chuck you weren't there either , so the  phd b: I was there . professor a: Oh you were there ? phd b: With Hynek ? professor a: Yeah . professor a: What was the w what was the downsampling problem again ? phd c: So we had professor a: I forget . phd c: So the fact that there there is no  low - pass filtering before the downsampling . phd c: There is because there is LDA filtering but that 's perhaps not  the best w m professor a: Depends what it 's frequency characteristic is , yeah . So we discussed about this , about the  professor a: Was there any conclusion about that ? phd c:  " try it " . So again this is th this is the downsampling  of the  the feature vector stream phd c:  . professor a: and  Yeah I guess the the  LDA filters they were doing do have   let 's see , so the the the feature vectors are calculated every ten milliseconds so  the question is how far down they are at fifty fifty hertz . Does anybody know what the frequency characteristic is ? phd c: We don't have yet professor a: Oh OK . And the third point  was the  , yeah , the on - line normalization where , well , the recursion f recursion for the mean estimation is a filter with some kind of delay professor a: Yeah . For this , the conclusion of Hynek was , well , " we can try it but " professor a:  - huh . professor a: Try try what ? phd c: So try to   take into account the delay of the recursion for the mean estimation . And so while discussing about these these LDA filters , some i issues appeared , like well , the fact that if we look at the frequency response of these filters it 's  , well , we don't know really what 's the important part in the frequency response and there is the fact that in the very low frequency , these filters don't don't really remove a lot .  and that 's probably a reason why , yeah , on - line normalization helps because it it , professor a: Right . Yeah , but perhaps everything could should be could be in the filter ,  the the mean normalization and Yeah . phd c: And this was this LDA  tuning perhaps and Hynek proposed again to his  TRAPS , so . phd c: Yeah , professor a:  I g I guess the key thing for me is is figuring out how to better coordinate between the two sides phd c:  . professor a:  I was talking with Hynek about it later and the the sort of had the sense sort of that that neither group of people wanted to to bother the other group too much . And and I don't think anybody is , you know , closed in in their thinking or are unwilling to talk about things but I think that you were sort of waiting for them to tell you that they had something for you and and that and expected that they would do certain things and they were sor they didn't wanna bother you phd c:   professor a: and they were sort of waiting for you and and and  we ended up with this thing where they they were filling up all of the possible latency themselves , and they just had hadn't thought of that .  it 's true that maybe maybe no one really thought about that that this latency thing would be such a a strict issue phd c: Yeah . Well professor a: in in  the other phd c: Yeah I don't know what happened really , but professor a: Yeah . Because , well , we discussed about that about this problem and they told us " well , we will do all that 's possible to have enough space for a network " but then , yeah , perhaps they were too short with the time and professor a: Then they couldn't . Well maybe we should just   you 're you 're bus other than that you folks are busy doing all the all the things that you 're trying that we talked about before right ? And this machines are busy and you 're busy phd c: Yeah . professor a: Let 's let 's , I think that as as we said before that one of the things that we 're imagining is that  there there will be  in the system we end up with there 'll be something to explicitly   do something about noise phd c:   professor a: in addition to the  other things that we 're talking about and that 's probably the best thing to do . And there was that one email that said that it sounded like   things looked very promising up there in terms of  I think they were using Ericsson 's approach or something and in addition to They 're doing some noise removal thing , right ? phd c: Yeah , yeah . phd c: And phd d: Yeah , I modified it well , modifying I studied Barry 's sim code , more or less . and we have some the feature for Italian database and we will try with this feature with the filter to find the result . phd d: and maybe try another type of spectral subtraction , I don't professor a: When you say you don't have a result yet you mean it 's it 's just that it 's in process or that you it finished and it didn't get a good result ? phd d: No . No , no n we have n we have do the experiment only have the feature the feature but the experiment have phd c: Yeah . So  I suggest actually now we we we sorta move on and and hear what 's what 's what 's happening in in other areas like what 's what 's happening with your investigations about echos and so on . grad f: Oh  Well  I haven't started writing the test yet , I 'm meeting with Adam today professor a:   grad f:  and he 's going t show me the scripts he has for  running recognition on mee Meeting Recorder digits . grad f:  I also  haven't got the code yet , I haven't asked Hynek for for the for his code yet . Cuz I looked at  Avendano 's thesis and I don't really understand what he 's doing yet but it it it sounded like  the channel normalization part  of his thesis  was done in a a bit of I don't know what the word is , a a bit of a rough way  it sounded like he  he he it it wasn't really fleshed out and maybe he did something that was interesting for the test situation but I I 'm not sure if it 's what I 'd wanna use so I have to I have to read it more , I don't really understand what he 's doing yet . Yeah I haven't read it in a while so I 'm not gonna be too much help unless I read it again , phd d: It 's my phd c: Oh yeah ? phd d: I know this is mine here . The  so you , and then you 're also gonna be doing this echo cancelling between the the close mounted and the and the the the what we 're calling a cheating experiment  of sorts between the distant grad f:  I I 'm ho Right . grad f: I I think he 's at least planning to do it for the cl close - mike cross - talk and so maybe I can just take whatever setup he has and use it . Yeah actually  he should  I wonder who else is I think maybe it 's Dan Ellis is going to be doing  a different cancellation . One of the things that people working in the meeting task wanna get at is they would like to have cleaner close - miked recordings . So  this is especially true for the lapel but even for the close close - miked  cases  we 'd like to be able to have  other sounds from other people and so forth removed from So when someone isn't speaking you 'd like the part where they 're not speaking to actually be So what they 're talking about doing is using ec  echo cancellation - like techniques . It 's not really echo but  just   taking the input from other mikes and using   a  an adaptive filtering approach to remove the effect of that  other speech .  what was it , there was there was some some some point where eh  Eric or somebody was was speaking and he had lots of silence in his channel and I was saying something to somebody else  which was in the background and it was not it was recognizing my words , which were the background speech on the close close mike . phd b: Yeah that was actually my I was wearing the I was wearing the lapel and you were sitting next to me , professor a: Oh you it was you I was Yeah . phd b: and I only said one thing but you were talking and it was picking up all your words . So I think I think Dan Ellis or somebody who was working with him was going to  work on that . And  I don't know if we 've talked lately about the the plans you 're developing that we talked about this morning  I don't remember if we talked about that last week or not , but maybe just a quick reprise of of what we were saying this morning . So continuing to  extend phd b: What about the stuff that  Mirjam has been doing ? And and S Shawn , yeah . professor a: But that 's   all that 's is a a certainly relevant  study and , you know , what are the features that they 're finding . professor a:  what are the variables , what we 're calling this one , what are the variables that they 're found finding useful phd c:  . professor a:  for phd b: And their their targets are based on canonical mappings of phones to acoustic f features . And that 's certainly one thing to do and we 're gonna try and do something more f more fine than that but   so  So I guess you know what , I was trying to remember some of the things we were saying , do you ha still have that ? Yeah . professor a: There 's those that  yeah , some of some of the issues we were talking about was in j just getting a good handle on on  what " good features " are and phd b: What does what did  Larry Saul use for it was the sonorant  detector , right ? How did he H how did he do that ? Wh - what was his detector ?   professor a: And the other thing you were talking about is is is where we get the targets from . So  , there 's these issues of what are the what are the variables that you use and do you combine them using the soft " AND - OR " or you do something , you know , more complicated  and then the other thing was so where do you get the targets from ? The initial thing is just the obvious that we 're discussing is starting up with phone labels from somewhere and then  doing the transformation . But then the other thing is to do something better and eh w why don't you tell us again about this this database ? This is the phd b:  ! professor a: And then tell them to talk naturally ? Yeah , yeah . professor a: Maybe you could go to these parlors and and you could , you know you know have have , you know , reduced rates if you if you can do the measurements . You could what you could do is you could sell little rings and stuff with embedded you know , transmitters in them and things professor a: Yeah . phd b:  ! There 's a bunch of data that l around , that people have done studies like that w way way back right ?  I can't remember where  Wisconsin or someplace that used to have a big database of Yeah . I remember there was this guy at A T - andT , Randolph ? or r What was his name ? Do you remember that guy ?  , researcher at A T - andT a while back that was studying , trying to do speech recognition from these kinds of features . professor a: Do you mean eh but you  Mar phd c: Well he was the guy the guy that was using professor a: you mean when was was Mark Randolph there , or ? phd b: Mark Randolph . phd c: Is it the guy that was using the pattern of pressure on the tongue or ? phd b: I can't remember exactly what he was using , now . But I know I just remember it had to do with you know  positional parameters phd c: What Yeah . So the only the only  hesitation I had about it since ,  I haven't see the data is it sounds like it 's it 's continuous variables and a bunch of them . professor a: I don't know how complicated it is to go from there What you really want are these binary labels , and just a few of them . And maybe there 's a trivial mapping if you wanna do it and it 's e but it I I I worry a little bit that this is a research project in itself , whereas  if you did something instead that like  having some manual annotation by  you know , linguistics students , this would there 'd be a limited s set of things that you could do a as per our discussions with with John before phd b:   professor a: but the things that you could do , like nasality and voicing and a couple other things you probably could do reasonably well .  the other thing you could do is boot trying to to  get those binary variables and take the continuous variables from  the   the data itself there , but I I 'm not sure phd b: Could you cluster the just do some kind of clustering ? professor a: Guess you could , yeah . So anyway that 's that 's  that 's another whole direction that cou could be looked at .  in general it 's gonna be for new data that you look at , it 's gonna be hidden variable because we 're not gonna get everybody sitting in these meetings to wear the pellets and  . phd b: So you 're talking about using that data to get  instead of using canonical mappings of phones . phd b: So you 'd use that data to give you sort of what the the true mappings are for each phone ? grad e:   So wh yeah , where this fits into the rest in in my mind , I guess , is that  we 're looking at different ways that we can combine  different kinds of of rep front - end representations  in order to get robustness under difficult or even , you know , typical conditions . And part of it , this robustness , seems to come from  multi - stream or multi - band sorts of things and Saul seems to have a reasonable way of looking at it , at least for one one  articulatory feature . The question is is can we learn from that to change some of the other methods we have , since  , one of the things that 's nice about what he had I thought was that that it it  the decision about how strongly to train the different pieces is based on  a a reasonable criterion with hidden variables rather than  just assuming that you should train e e every detector  with equal strength towards  it being this phone or that phone . It 's a soft " AND " , I guess but in in principle you you wanna get a strong concurrence of all the different things that indicate something and then he " OR 's " across the different soft " OR 's " across the different  multi - band channels . And  the weight yeah , the target for the training of the " AND " " AND ' ed " things is something that 's kept  as a hidden variable , and is learned with EM . Whereas what we were doing is is  taking the phone target and then just back propagating from that phd b: So he doesn't have professor a: which means that it 's it 's  i It could be for instance that for a particular point in the data you don't want to   train a particular band train the detectors for a particular band . You you wanna ignore that band , cuz that 's a Ban - band is a noisy noisy measure . professor a: And we don't We 're we 're still gonna try to train it up . In our scheme we 're gonna try to train it up to do as well well as it can at predicting . phd b: So he doesn't have to have truth marks or Ho grad e: F right , and  he doesn't have to have hard labels . professor a: Well at the at the tail end , yeah , he has to know what 's where it 's sonorant . But he 's but what he 's - but what he 's not training up  what he doesn't depend on as truth is grad e: Right . professor a:  I guess one way of describing would be if if a sound is sonorant is it sonorant in this band ? Is it sonorant in that band ? grad e: Right . professor a: Is it sonorant in that band ? i It 's hard to even answer that what you really mean is that the whole sound is sonorant . professor a: then it comes down to , you know , to what extent should you make use of information from particular band towards making your decision . And   we 're making in a sense sort of this hard decision that you should you should use everything  with with  equal strength . professor a: And  because in the ideal case we would be going for posterior probabilities , if we had  enough data to really get posterior probabilities and if the if we also had enough data so that it was representative of the test data then we would in fact be doing the right thing to train everything as hard as we can . But  this is something that 's more built up along an idea of robustness from from the beginning and so you don't necessarily want to train everything up towards the phd b: So where did he get his  his tar his  high - level targets about what 's sonorant and what 's not ? grad e: From  canonical mappings  at first phd b: OK . grad e: and then it 's unclear  eh phd b: Using TIMIT ? or using grad e: using TIMIT phd b:  - huh .  we ha we have a kind of iterative training because we do this embedded Viterbi ,  so there is some something that 's suggested , based on the data but it 's it 's not I think it s doesn't seem like it 's quite the same , cuz of this cuz then whatever that alignment is , it 's that for all all bands . professor a: Well no , that 's not quite right , we did actually do them separate tried to do them separately so that would be a little more like what he did . But it 's still not quite the same because then it 's it 's  setting targets based on where you would say the sound begins in a particular band . Might be closer I guess if we did a soft soft target   embedded neural net training like we 've done a few times  f the forward  do the forward calculations to get the gammas and train on those .  what 's next ? phd b: I could say a little bit about w stuff I 've been playing with . You 're playing ? phd b: I  Huh ? professor a: You 're playing ? phd b: Yes , I 'm playing .  so I wanted to do this experiment to see   what happens if we try to  improve the performance of the back - end recognizer for the Aurora task and see how that affects things . And so I had this  I think I sent around last week a this plan I had for an experiment , this matrix where I would take the  the original  the original system . So there 's the original system trained on the mel cepstral features and then com and then  optimize the b HTK system and run that again . So look at the difference there and then  do the same thing for the ICSI - OGI front - end . professor a: What which test set was this ? phd b: This is that I looked at ? professor a:   phd b: So as far as I 've gotten is I 've  been able to go through from beginning to end the  full HTK system for the Italian data and got the same results that  that  Stephane had . So  I started looking to and now I 'm I 'm sort of lookin at the point where I wanna know what should I change in the HTK back - end in order to try to  to improve it . One of the first things I thought of was the fact that they use the same number of states for all of the models professor a:   phd b: and so I went on - line and I  found a pronunciation dictionary for Italian digits professor a:   phd b: and just looked at , you know , the number of phones in each one of the digits .  you know , sort of the canonical way of setting up a an  system is that you use  three states per phone and  so then the the total number of states for a word would just be , you know , the number of phones times three . And so when I did that for the Italian digits , I got a number of states , ranging on the low end from nine to the high end , eighteen . Now you have to really add two to that because in HTK there 's an initial null and a final null so when they use  models that have eighteen states , there 're really sixteen states . And so  their guess of eighteen states seems to be pretty well matched to the two longest words of the Italian digits , the four and five which  , according to my , you know , sort of off the cuff calculation , should have eighteen states each . So my guess  And then if you I I printed out a confusion matrix   for the well - matched case , and it turns out that the longest words are actually the ones that do the best . So my guess about what 's happening is that you know , if you assume a fixed the same amount of training data for each of these digits and a fixed length model for all of them but the actual words for some of them are half as long you really  have , you know , half as much training data for those models . Because if you have a long word and you 're training it to eighteen states ,  you 've got you know , you 've got the same number of Gaussians , you 've gotta train in each case , professor a:   phd b: but for the shorter words , you know , the total number of frames is actually half as many . phd b: So it could be that , you know , for the short words there 's because you have so many states , you just don't have enough data to train all those Gaussians . So  I 'm going to try to  create more word - specific   prototype H M Ms to start training from . professor a: Yeah , it 's not at all uncommon you do worse on long word on short words than long words anyway just because you 're accumulating more evidence for the for the longer word , phd b:   phd b: Yeah so I 'll I 'll , the next experiment I 'm gonna try is to just  you know create  models that seem to be more w matched to my guess about how long they should be . phd b: And as part of that  I wanted to see sort of how the  how these models were coming out , you know , what w when we train up  th you know , the model for " one " , which wants to have nine states , you know , what is the  what do the transition probabilities look like in the self - loops , look like in in those models ? And so I talked to Andreas and he explained to me how you can calculate the expected duration of an  just by looking at the transition matrix professor a:   phd b: and so I wrote a little Matlab script that calculates that and so I 'm gonna sort of print those out for each of the words to see what 's happening , you know , how these models are training up , professor a:   I d I did quickly , I did the silence model and and  that 's coming out with about one point two seconds as its average duration and the silence model 's the one that 's used at the beginning and the end of each of the string of digits . And so the S P model , which is what they put in between digits , I I haven't calculated that for that one yet , but  . So they basically their their model for a whole digit string is silence digit , SP , digit , SP blah - blah - blah and then silence at the end . professor a: Are the SP 's optional ?  skip them ? phd b: I have to look at that , but I 'm not sure that they are . Now the one thing about the S P model is really it only has a single s emitting state to it . phd b: So if it 's not optional , you know , it 's it 's not gonna hurt a whole lot professor a: I see . phd b: and it 's tied to the center state of the silence model so it 's not its own  It doesn't require its own training data , professor a:   phd b: So it , it 's pretty good the way that they have it set up , but  i So I wanna play with that a little bit more . I 'm curious about looking at , you know how these models have trained and looking at the expected durations of the models and I wanna compare that in the the well - matched case f to the unmatched case , and see if you can get an idea of just from looking at the durations of these models , you know , what what 's happening . professor a: Yeah , I think that  , as much as you can , it 's good to d sort of not do anything really tricky . professor a: Not do anything that 's really finely tuned , but just sort of eh you know you t you i z phd b: Yeah . professor a: The premise is kind of you have a a good person look at this for a few weeks and what do you come up with ? phd b:   professor a: And  phd b: And Hynek , when I wa told him about this , he had an interesting point , and that was th  the the final models that they end up training up have I think probably something on the order of six Gaussians per state . And Hynek was saying that well , probably in a real application , you wouldn't have enough compute to handle models that are very big or complicated . But you know , it depends on what the actual application is and it 's really hard to know what your limits are in terms of how many Gaussians you can have . professor a: I I I what I thought you were gonna say i but which I was thinking was  where did six come from ? Probably came from the same place eighteen came from . professor a:  that 's another parameter , right ? that that maybe , you know ,  you really want three or nine or phd b: Yeah , yeah . Well one thing  , if I if if I start  reducing the number of states for some of these shorter models that 's gonna reduce the total number of Gaussians . professor a: how much better can you make it ? And  since they 're only simple things there 's nothing that you 're gonna do that is going to blow up the amount of computation phd b:   professor a: if you found that nine was better than six that would be O K , I think , actually . I really wasn't even gonna play with that part of the system yet , professor a:  -  , OK . phd b: I was just gonna change the the t professor a: Yeah , just work with the models , yeah . So  what 's  I guess your plan for You you you guys ' plan for the next next week is just continue on these these same things we 've been talking about for Aurora and phd c: Yeah , I guess we can try to have some kind of new baseline for next week perhaps . And then do other things , play with the spectral subtraction , and retry the MSG and things like that . I think that after all of this   confusion settles down in another some point a little later next year there will be some sort of standard and it 'll get out there and hopefully it 'll have some effect from something that that has  been done by our group of people but  e even if it doesn't there 's there 's go there 'll be standards after that . phd b: Does anybody know how to  run Matlab sort of in batch mode like you c send it s a bunch of commands to run and it gives you the output . Is it possible to do that ? grad e: I I think  Mike tried it phd b: Yeah ? grad e: and he says it 's impossible so he went to Octave . phd c: What is Octave so ? It 's a free software ? grad e: What 's that ?  , Octave ? phd c: Yeah . phd c: And it does the same syntax and everything eh like Matlab , or ? grad e:  i it 's a little behind , it 's the same syntax but it 's a little behind in that Matlab went to these like  you can have cells and you can you can  implement object - oriented type things with Matlab .  Octave doesn't do that yet , so I think you , Octave is kinda like Matlab  four point something or . phd b: If it 'll do like a lot of the basic matrix and vector stuff grad e: The basic stuff , right