Professor A: Oh . You 're playing ?
PhD B: I um Huh ?
Professor A: You 're playing ?
PhD B: Yes , I 'm playing . Um so I wanted to do this experiment to see um uh what happens if we try to uh improve the performance of the back - end recognizer for the Aurora task and see how that affects things . And so I had this um I think I sent around last week a  this plan I had for an experiment , this matrix where I would take the um the original um the original system . So there 's the original system trained on the mel cepstral features and then com and then uh optimize the b HTK system and run that again . So look at the difference there and then uh do the same thing for the ICSI - OGI front - end .
Professor A: What which test set was this ?
PhD B: This is that I looked at ?
Professor A: Mm - hmm .
PhD B: Uh I 'm looking at the Italian right now .
Professor A: Mm - hmm .
PhD B: So as far as I 've gotten is I 've uh been able to go through from beginning to end the um full HTK system for the Italian data and got the same results that um that uh Stephane had . So um I started looking to and now I 'm I 'm sort of lookin at the point where I wanna know what should I change in the HTK back - end in order to try to uh to improve it . So . One of the first things I thought of was the fact that they use the same number of states for all of the models
Professor A: Mm - hmm .
PhD B: and so I went on - line and I uh found a pronunciation dictionary for Italian digits
Professor A: Mm - hmm .
PhD B: and just looked at , you know , the number of phones in each one of the digits . Um you know , sort of the canonical way of setting up a an HMM system is that you use um three states per phone and um so then the the total number of states for a word would just be , you know , the number of phones times three . And so when I did that for the Italian digits , I got a number of states , ranging on the low end from nine to the high end , eighteen . Um . Now you have to really add two to that because in HTK there 's an initial null and a final null so when they use uh models that have eighteen states , there 're really sixteen states . They 've got those initial and final null states . And so um  their guess of eighteen states seems to be pretty well matched to the two longest words of the Italian digits , the four and five which um , according to my , you know , sort of off the cuff calculation , should have eighteen states each .
Professor A: Mm - hmm .
PhD B: And so they had sixteen . So that 's pretty close . Um  but for the most of the words are sh much shorter . So the majority of them wanna have nine states . And so theirs are s sort of twice as long . So my guess uh And then if you I I printed out a confusion matrix um uh for the well - matched case , and it turns out that the longest words are actually the ones that do the best . So my guess about what 's happening is that you know , if you assume a fixed the same amount of training data for each of these digits and a fixed length model for all of them but the actual words for some of them are half as long you really um have , you know , half as much training data for those models . Because if you have a long word and you 're training it to eighteen states ,  uh you 've got you know , you 've got the same number of Gaussians , you 've gotta train in each case ,
Professor A: Mm - hmm .
PhD B: but for the shorter words , you know , the total number of frames is actually half as many .
Professor A: Mm - hmm .
PhD B: So it could be that , you know , for the short words there 's because you have so many states , you just don't have enough data to train all those Gaussians . So um I 'm going to try to um create more word - specific um uh prototype H M Ms to start training from .
Professor A: Yeah , I mean , it 's not at all uncommon you do worse on long word on short words than long words anyway just because you 're accumulating more evidence for the for the longer word ,
PhD B: Mm - hmm .
Professor A: but .
PhD B: Yeah so I 'll I 'll , the next experiment I 'm gonna try is to just um you know create uh models that seem to be more w matched to my guess about how long they should be .
Professor A: Mm - hmm .
PhD B: And as part of that um I wanted to see sort of how the um how these models were coming out , you know , what w when we train up uh th you know , the model for " one " , which wants to have nine states , you know , what is the uh what do the transition probabilities look like in the self - loops , {comment} look like in in those models ? And so I talked to Andreas and he explained to me how you can calculate the expected duration of an HMM just by looking at the transition matrix
Professor A: Mm - hmm .
PhD B: and so I wrote a little Matlab script that calculates that and so I 'm gonna sort of print those out for each of the words to see what 's happening , you know , how these models are training up ,
Professor A: Mm - hmm . Mm - hmm .
PhD B: you know , the long ones versus the short ones . I d I did quickly , I did the silence model and  and um that 's coming out with about one point two seconds as its average duration and the silence model 's the one that 's used at the beginning and the end of each of the string of digits .
Professor A: Wow . Lots of silence .
PhD B: Yeah , yeah . And so the S P model , which is what they put in between digits , I I haven't calculated that for that one yet , but um . So they basically their  their model for a whole digit string is silence digit , SP , digit , SP blah - blah - blah and then silence at the end . And so .
Professor A: Are the SP 's optional ? I mean skip them ?
PhD B: I have to look at that , but I 'm not sure that they are . Now the one thing about the S P model is really it only has a single s emitting state to it .
Professor A: Mm - hmm .
PhD B: So if it 's not optional , you know , it 's it 's not gonna hurt a whole lot
Professor A: I see .
PhD B: and it 's tied to the center state of the silence model so it 's not its own um It doesn't require its own training data ,
Professor A: Mm - hmm .
PhD B: it just shares that state .
Professor A: Mm - hmm .
PhD B: So it , I mean , it 's pretty good the way that they have it set up , but um i So I wanna play with that a little bit more . I 'm curious about looking at , you know how these models have trained and looking at the expected durations of the models and I wanna compare that in the the well - matched case f to the unmatched case , and see if you can get an idea of just from looking at the durations of these models , you know , what what 's happening .
Professor A: Yeah , I mean , I think that uh , as much as you can , it 's good to d sort of not do anything really tricky .
PhD B: Mm - hmm .
Professor A: Not do anything that 's really finely tuned , but just sort of eh you know you t you i z
PhD B: Yeah .
Professor A: The premise is kind of you have a a good person look at this for a few weeks and what do you come up with ?
PhD B: Mm - hmm . Mm - hmm .
Professor A: And uh
PhD B: And Hynek , when I wa told him about this , he had an interesting point , and that was th um the the final models that they end up training up have I think probably something on the order of six Gaussians per state . So they 're fairly , you know , hefty models . And Hynek was saying that well , probably in a real application , you wouldn't have enough compute to handle models that are very big or complicated . So in fact what we may want are simpler models .
Professor A: Could be .
PhD B: And compare how they perform to that . But you know , it depends on what the actual application is and it 's really hard to know what your limits are in terms of how many Gaussians you can have .
Professor A: Right . And that , I mean , at the moment that 's not the limitation , so .
PhD B: Mm - hmm .
Professor A: I mean , I I I what I thought you were gonna say i but which I was thinking was um where did six come from ? Probably came from the same place eighteen came from . You know , so .
PhD B: Yeah . Right .
Professor A: Uh that 's another parameter , right ? that that maybe , you know , uh you really want three or nine or PhD B: Yeah , yeah . Well one thing I mean , if I if if I start um reducing the number of states for some of these shorter models that 's gonna reduce the total number of Gaussians .
Professor A: Right .
PhD B: So in a sense it 'll be a simpler system .
Professor A: Yeah . Yeah . But I think right now again the idea is doing just very simple things
PhD B: Yeah .
Professor A: how much better can you make it ? And um since they 're only simple things there 's nothing that you 're gonna do that is going to blow up the amount of computation
PhD B: Mm - hmm .
Professor A: um so
PhD B: Right . Right .
Professor A: if you found that nine was better than six that would be O K , I think , actually .
PhD B: Mm - hmm .
Professor A: Doesn't have to go down .
PhD B: Yeah . I really wasn't even gonna play with that part of the system yet ,
Professor A: Mm - hmm , OK .
PhD B: I was just gonna change the the t
Professor A: Yeah , just work with the models , yeah .
PhD B: yeah , just look at the length of the models and just see what happens .
Professor A: Yeah .
PhD B: So .

Summary: PhD B was conducting an experiment to improve the backend performance of the Aurora recognizer. The goal was recognizing the HTK system by challenging the canonical paradigm.