PhD I: SRI recognition ? Oh .
Grad F: That wasn't me .
Professor D: Uh .
PhD I: Um . well ,
Professor D: Who 's that ?
PhD I: we have lots of them . Uh , I dunno . Chuck , do you have any any updates ?
PhD B: N I 'm successfully , uh , increasing the error rate . Uh Grad F: That 's good .
Grad H: Mmm .
PhD I: Oh .
PhD G: Lift the Herve approach .
PhD B: Yeah . So , I mean I 'm just playing with , um , the number of Gaussians that we use in the the recognizer , and PhD I: Well , you have to sa you have to tell people that you 're you 're doing you 're trying the tandem features .
PhD B: Yes , I 'm using tandem features .
Grad F: Oh you are ?
PhD B: And Grad F: Cool .
PhD I: A and I 'm still tinkering with the PLP features .
Grad F :
Professor D: Yeah , I got confused by the results . It sai because uh , the meeting before , you said " OK , we got it down to where they 're they 're within a tenth of a percent " .
PhD B: That was on males .
PhD I: Right . That was that was before I tried it on the females .
Professor D: Oh .
PhD I: See , women are nothi are , trouble .
Professor D: It 's the women are the problem . OK .
PhD I: Right ? As we all know . So .
PhD G: Well , let 's just say that men are simple .
PhD I: So {comment} so , when So I I had I ha
Grad F: That was a quick response .
PhD I: So , we had reached the point where PhD G: I 'm well rehearsed .
Professor D: Yeah .
PhD I: we had reached the point where , {comment} um , on the male portion of the development set , the , um or one of the development sets , I should say  the , um the male error rate with , uh , ICSI PLP features was pretty much identical with , uh , SRI features . which are MFCC . So , um , then I thought , " Oh , great . I 'll j I 'll just let 's make sure everything works on the females . " And the error rate you know , there was a three percent difference .
Professor D: Oh . Uh - huh .
PhD I: So ,
PhD G: Is there less training data ?
PhD I: uh PhD G: I mean , we don
PhD I: No , actually there 's more training data .
PhD G: This is on just digits ?
Professor D: No .
PhD I: No , no .
Grad F: No .
PhD B: Hub - five .
Grad F: It 's , uh , Swi
PhD G: Oh , sorry . OK . This is on PhD I: This is Hub - five .
PhD G: Oh , OK .
Grad F: Hub - five . Yeah .
PhD I: Yeah . Um , and the test data is CallHome and Switchboard . So , uh so then um Oh , and plus the the vocal tract length normalization didn't actually made things worse . So something 's really seriously wrong . So Um Professor D: Aha ! OK .
PhD I: So So Professor D: So but you see , now , between between the males and the females , there 's certainly a much bigger difference in the scaling range , than there is , say , just within the males . And what you were using before was scaling factors that were just from the the m the SRI front - end . And that worked that worked fine .
PhD I: That 's true . Yeah .
Professor D: Uh , but now you 're looking over a larger range and it may not be so fine .
PhD I: Well , um So I just d so the one thing that I then tried was to put in the low - pass filter , which we have in the So , most most Hub - five systems actually band - limit the uh , at about , uh , thirty - seven hundred , um , hertz .
Professor D: Uh - huh .
PhD I: Although , you know , normally , I mean , the channel goes to four four thousand . Right ? So , um And that actually helped , uh uh , a little bit .
Professor D: Uh - huh .
PhD I: Um and it didn't hurt on the males either . So , um And I 'm now , uh , trying the Oh , and suddenly , also the v the vocal tract length normalization only in the test se on the test data . So , you can do vocal tract length normalization on the test data only or on both the training and the test .
Professor D: Yeah .
PhD I: And you expect it to help a little bit if you do it only on the test , and s more if you do it on both training and test .
Professor D: Yeah .
PhD I: And so the It now helps , if you do it only on the test , and I 'm currently retraining another set of models where it 's both in the training and the test , and then we 'll we 'll have , hopefully , even better results . So But there 's It looks like there will still be some difference , maybe between one and two percent , um , for the females .
Professor D: Huh .
PhD I: And so , um , you know , I 'm open to suggestions .
Grad F: Mm - hmm .
PhD I: And it is true that the , uh that the  you know , we are using the But it can't be just the VTL ,
Professor D: Uh - huh .
PhD I: because if you don't do VTL in both systems , uh , you know , the the females are considerably worse in the with the PLP features .
Professor D: No no . I I remember that .
Grad F: It 's much worse . Yeah .
PhD I: So there must be some something else going on .
PhD G: Well , what 's the standard ? Yeah , so I thought the performance was actually a little better on females than males .
Grad F: That 's what I thought , too .
PhD I: Um , that ye {comment} overall , yes , but on this particular development test set , they 're actually a little worse . But that 's beside the point . We 're looking at the discrepancy between the SRI system and the SRI system when trained with ICSI features .
PhD G: Right . I 'm just wondering if that if if you have any indication of your standard features ,
Grad F: What 's Are the freq ?
PhD G: you know , if that 's also different or in the same direction or not .
Professor D: You 're This is lemme ask a q more basic que
PhD G: Cuz Professor D: I mean , is this , uh uh , iterative , Baum - Welch training ?
PhD I: Mm - hmm .
Professor D: Or is it Viterbi training ? Or ?
PhD I: It 's Baum - Welch training .
Professor D: Baum - Welch training . And how do you determine when to to stop iterating ?
PhD I: Um Well , actually , we we just basically do a s a fixed number of iterations .
Grad F: Hmm .
PhD I: Uh , in this case four . Um , which Eh , we used to do only three , and then we found out we can squeeze And it was basically , we 're s we 're keeping it on the safe side . But you 're d Right . It might be that one more iteration would would help , but it 's sort of
Professor D: Or maybe or maybe you 're doing one too many .
PhD I: you know .
Professor D: I mean it 's it 's PhD I: No , but with Baum - Welch , there shouldn't be an over - fitting issue , really .
Professor D: Uh . {comment} Well , there can be . Sure .
Grad F: Well , you can try each one on a cross - validation set ,
PhD I: Um .
Professor D: It d if you if you remember some years ago Bill Byrne did a thing where he was he was looking at that ,
Grad F: can't you ?
Professor D: and he showed that you could get it .
PhD I: Yeah .
Professor D: So . But {comment} but  but , um PhD I: Well , yeah . We can Well , that 's that 's the easy one to check ,
Professor D: Yeah .
PhD I: because we save all the intermediate models
Grad F: Do you ?
PhD I: and we can Professor D: And in each case , ho
Grad F: What ?
Professor D: um , I 'm sorry in each case how do you determine , you know , the the usual fudge factors ? The , uh  the , uh , language , uh , scaling , acoustic scaling , uh , uh PhD I: Um I uh {comment} I 'm actually re - optimizing them . Although that hasn't shown to make a big difference .
Professor D: OK . And the pru the question he was asking at one point about pruning , uh Remember that one ?
PhD I: Pruning ?
Professor D: Well , he was he 's it looked like the probabil at one point he was looking at the probabilities he was getting out at the likelihoods he was getting out of PLP versus mel cepstrum , and they looked pretty different ,
PhD I: Pruning in the ?
PhD B: Yeah , the likelihoods were lower for the PLP .
Professor D: as I recall .
PhD G: Oh .
Professor D: And so , uh , there 's the question PhD I: I you mean did you see this in the SRI system ?
PhD B: Mm - hmm . Was just looking through the log files ,
PhD I: Um . Well , the likelihoods are PhD B: and PhD I: You can't directly compare them , because , for every set of models you compute a new normalization . And so these log probabilities , they aren't directly comparable
PhD B: Oh .
PhD I: because you have a different normalization constants for each model you train .
PhD B: Hmm .
Professor D: But , still it 's a question PhD I: So Professor D: if you have some threshold somewhere in terms of beam search or something ,
PhD B: Well , yeah . That 's what I was wondering .
Professor D: or ?
PhD I: W yeah . I mean Uh PhD B: I mean , if you have one threshold that works well because the range of your likelihoods is in this area PhD I: We prune very conservatively . I mean , as we saw with the meeting data , um we could probably tighten the pruning without really So we we basically we have a very open beam .
Professor D: But , you 're only talking about a percent or two .
PhD B: Yeah .
Professor D: Right ? Here we 're - we 're saying that we there gee , there 's this b eh , there 's this difference here . And it See cuz , i i {comment} there could be lots of things . Right ? But but but but , um , let 's suppose just for a second that , uh , we 've sort of taken out a lot of the the major differences , uh , between the two .
PhD I: Right . Course . Mm - hmm . Right .
Professor D: I mean , we 're already sort of using the mel scale and we 're using the same style filter integration , and and , well , we 're making sure that low and high PhD I: Actually , there is the difference in that . So , for the PLP features we use the triangular filter shapes . And for the in the SRI front - end we use the trapezoidal one .
Grad F: And what 's the top frequency of each ?
PhD I: Well , now it 's the same . It 's thirty thirty to seven hundred and sixty hertz .
Grad F: Yeah . Exp - one 's triangular , one 's trapezoidal . So PhD I: No , no . But Professor D: Before we i i th with straight PLP , it 's trapezoidal also .
PhD I: Well But Professor D: But then we had a slight difference in the in the scale . Uh , so .
PhD I: Since currently the Feacalc program doesn't allow me to change the filter shape independently of the scale .
Grad F: Uh - huh .
PhD I: And , I did the experiment on the SRI front - end where I tried the y where the standard used to be to use trapezoidal filters . You can actually continuously vary it between the two . And so I wen I swi I tried the trap eh , triangular ones . And it did slightly worse , but it 's really a small difference .
Grad F: Hmm .
Professor D: Coup - Couple tenths of a percent or something .
PhD I: So Grad F: OK .
Professor D: Right .
Grad F: So it 's not just losing some frequency range .
PhD I: Yeah , exactly . So , it 's not I don't think the filter shape by itself will make a huge {comment} difference .
Professor D: Yeah . Right . So the oth the other thing that Grad F: Yeah .
Professor D: So , f i We 've always viewed it , anyway , as the major difference between the two , is actually in the smoothing , that the that the , um , PLP , and and the reason PLP has been advantageous in , uh , slightly noisy situations is because , PLP does the smoothing at the end by an auto - regressive model ,
PhD I: Mm - hmm . Mm - hmm .
Professor D: and mel cepstrum does it by just computing the lower cepstral coefficients .
PhD I: Mm - hmm .
Professor D: Um . So , um Mm - hmm .
PhD I: OK . So one thing I haven't done yet is to actually do all of this with a much larger with our full training set . So right now , we 're using a I don't know , forty ? I i it 's it 's eh {comment} it 's a f training set that 's about , um , you know , by a factor of four smaller than what we use when we train the full system . So , some of these smoothing issues are over - fitting for that matter .
Professor D: Mm - hmm .
PhD I: And the Baum - Welch should be much less of a factor , if you go full whole hog .
Professor D: Could be . Yeah .
PhD I: And so , w so , just um so the strategy is to first sort of treat things with fast turn - around on a smaller training set and then , when you 've sort of , narrowed it down , you try it on a larger training set .
Professor D: Yeah .
PhD I: And so , we haven't done that yet .
Professor D: Now the other que related question , though , is is , uh , what 's the boot models for these things ?
PhD I: Th - th the boot models are trained from scratch . So we compute , um So , we start with a , um , alil alignment that we computed with the b sort of the best system we have . And and then we train from scratch . So we com we do a , you know , w um  We collect the uh , the observations from those alignments under each of the feature sets that that we train . And then , from there we do , um There 's a lot of , actually  The way it works , you first train a phonetically - tied mixture model . Um . You do a total of First you do a context - independent PTM model . Then you switch to a context You do two iterations of that . Then you do two iterations of of of context - dependent phonetically - tied mixtures . And then from that you you do the you you go to a state - clustered model ,
Professor D: Yeah .
PhD I: and you do four iterations of that . So there 's a lot of iterations overall between your original boot models and the final models . I don't think that Hmm . We have never seen big differences . Once I thought " oh , I can Now I have these much better models . I 'll re - generate my initial alignments . Then I 'll get much better models at the end . " Made no difference whatsoever . It 's I think it 's eh , i
Professor D: Right . Well , mis for making things better .
PhD I: the boot models are recur
Professor D: Yeah . But , this for making things worse . This it migh Th - the thought is is is possible another possible partial cause is if the boot models used a comple used a different feature set , that PhD I: Mm - hmm . Mm - hmm . But there are no boot models , in fact . You you 're not booting from initial models . You 're booting from initial alignments .
Professor D: Which you got from a different feature set .
PhD I: That 's correct .
Professor D: So , those features look at the data differently , actually .
PhD I: Yeah , but Professor D: I mean , you know , they they will find boundaries a little differently , though You know , all th all that sort of thing is actually slightly different . I 'd expect it to be a minor effect ,
PhD I: But but but , what I 'm what I 'm saying is Professor D: but PhD I: So , we e w f w For a long time we had used boot alignments that had been trained with a  with the same front - end but with acoustic models that were , like , fifteen percent worse than what we use now .
Professor D: Mm - hmm .
PhD I: And with a dict different dictionary with a considerably different dictionary , which was much less detailed and much less well - suited .
Professor D: Mm - hmm . Yeah .
PhD I: And so , then we switched to new boot alignments , which which now had the benefit of all these improvements that we 've made over two years in the system .
Professor D: Right .
PhD I: And , the result in the end was no different .
Professor D: Right .
PhD I: So , what I 'm saying is , the exact nature of these boot alignments is probably not a big factor in the quality of the final models .
Professor D: Yeah , maybe not . But it it I st still see it as I mean , there 's there 's a history to this , too ,
PhD I: Yeah .
Professor D: but I uh , I don't wanna go into ,
PhD I: Mm - hmm .
Professor D: but but I I I th I think it could be the things that it the data is being viewed in a certain way , uh , that a beginning is here rather than there and so forth ,
PhD I: Yeah . Right .
Professor D: because the actual signal - processing you 're doing is slightly different .
PhD I: Right .
Professor D: But , it 's it 's that 's probably not it .
PhD I: Yeah . Anyway , I I I should really reserve , uh , any conclusions until we 've done it on the large training set , um , and until we 've seen the results with the with the VTL in training .
Professor D: Yeah . At some point you also might wanna take the same thing and try it on , uh , some Broadcast News data or something else that actually has has some noisy  noisy components , so we can see if any conclusions we come to holds across different data .
PhD I: So . Yeah . Right .
Professor D: Uh PhD I: And , uh , with this , I have to leave .
Professor D: OK .
Grad H: Hmm !
Professor D: So , is there something quick about Absinthe that you ?
PhD I: With this said .
Grad F: Uh . Just what we were talking about before , which is that I ported a Blass library to Absinthe , and then got got it working with fast - forward , and got  a speedup roughly proportional to the number of processors times the clock cycle .
PhD I: Oh .
Grad F: So , that 's pretty good .
PhD I: Oh ! Cool .
Grad F: Um , I 'm in the process of doing it for Quicknet , but there 's something going wrong and it 's about half the speed that I was estimating it should be , and I 'm not sure why .
PhD I: Mm - hmm .
Grad F: But I 'll keep working on it . But the what it means is that it 's likely that for net training and forward passes , we 'll Absinthe will be a good machine . Especially if we get a few more processors and upgrade the processors .
PhD I: A few more processors ? How many are you shooting for ?
Grad F: There 're five now . It can hold eight .
PhD I: Oh , OK .
Professor D: Yeah , we 'll just go buy them , I guess .
Grad F: And it 's also five - fifty megahertz and you can get a gigahertz .
PhD I: Yeah .
Grad F: So .
PhD I: Can you mix t uh , processors of different speed ?
Grad F: I don't think so . I think we 'd have to do all PhD I: OK .
Professor D: Probably just throw away the old ones , and Grad F: Yep .
Professor D: Thank you for the box ,
PhD I: Oh , OK .
Professor D: and  I 'll just go buy their process .
Grad H: Hmm !
PhD I: Maybe we can stick them in another system . I dunno .
Grad F: We 'd have to get a almost certainly have to get a , uh , Netfinity server .
PhD I: I see .
Grad F: They 're pretty pretty specialized .
Professor D: Yeah . OK .
PhD I: OK .