Professor A: We 're going ? OK . Sh - Close your door on door on the way out ?
Grad B: OK . Thanks .
Professor A: Thanks .
Grad B: Oh .
Professor A: Yeah . Probably wanna get this other door , too . OK . So . Um .  What are we talking about today ?
PhD E: Uh , well , first there are perhaps these uh Meeting Recorder digits that we tested .
Professor A: Oh , yeah . That was kind of uh interesting .
PhD E: So .
Professor A: The both the uh  the SRI System and the oth
PhD E: Um .
Professor A: And for one thing that that sure shows the difference between having a lot of uh training data or not ,
PhD E: Of data ? Yeah .
Professor A: uh , the uh  The best kind of number we have on the English uh on near microphone only is is uh three or four percent .
PhD E: Mm - hmm .
Professor A: And uh it 's significantly better than that , using fairly simple front - ends on  on the uh  uh , with the SRI system .
PhD E: Mm - hmm .
Professor A: So I th I think that the uh But that 's that 's using uh a a pretty huge amount of data , mostly not digits , of course , but but then again Well , yeah . In fact , mostly not digits for the actual training the H M Ms whereas uh in this case we 're just using digits for training the H M
PhD E: Yeah . Right .
Professor A: Did anybody mention about whether the the SRI system is a  is is doing the digits um the wor as a word model or as uh a sub s sub - phone states ?
PhD E: I guess it 's it 's uh allophone models ,
Professor A: Yeah . Probably .
PhD E: so , well Professor A: Huh ?
PhD E: Yeah . I think so , because it 's their very d huge , their huge system .
Professor A: Yeah .
PhD E: And . But . So . There is one difference Well , the SRI system the result for the SRI system that are represented here are with adaptation . So there is It 's their complete system and including on - line uh unsupervised adaptation .
Professor A: That 's true .
PhD E: And if you don't use adaptation , the error rate is around fifty percent worse , I think , if I remember .
Professor A: OK .
PhD E: Yeah .
Professor A: It 's tha it 's that much , huh ?
PhD E: Nnn . It 's Yeah . It 's quite significant .
Professor A: Oh . OK .
PhD E: Yeah .
Professor A: Still .
PhD E: Mm - hmm .
Professor A: But but uh what what I think I 'd be interested to do given that , is that we we should uh take I guess that somebody 's gonna do this , right ? is to take some of these tandem things and feed it into the SRI system , right ?
PhD E: Yeah .
Professor A: Yeah .
PhD E: We can do something like that .
Professor A: Yeah . Because PhD E: Yeah . But But I guess the main point is the data because uh I am not sure . Our back - end is is fairly simple but until now , well , the attempts to improve it or have fail Ah , well , I mean uh what Chuck tried to to to do
Professor A: Yeah , but he 's doing it with the same data , right ? I mean so to  So there 's there 's there 's two things being affected .
PhD E: Yeah . So it 's Yeah .
Professor A: I mean . One is that that , you know , there 's something simple that 's wrong with the back - end . We 've been playing a number of states
PhD E: Mm - hmm .
Professor A: uh I I don't know if he got to the point of playing with the uh number of Gaussians yet
PhD E: Mm - hmm .
Professor A: but but uh , uh , you know . But , yeah , so far he hadn't gotten any big improvement ,
PhD E: Mm - hmm .
Professor A: but that 's all with the same amount of data which is pretty small .
PhD E: Yeah .
Professor A: And um .
PhD E: Mmm . So , yeah , we could retrain some of these tandem on on huge Professor A: Well , you could do that , but I 'm saying even with it not with that part not retrained , just just using having the H M Ms much better H M
PhD E: Ah , yeah . Just f for the HMM models .
Professor A: Yeah .
PhD E: Yeah . Mm - hmm . Mm - hmm .
Professor A: Um . But just train those H M Ms using different features , the features coming from our Aurora stuff .
PhD E: Yeah .
Professor A: So .
PhD E: Yeah . But what would be interesting to see also is what what perhaps it 's not related , the amount of data but the um recording conditions . I don't know . Because it 's probably not a problem of noise , because our features are supposed to be robust to noise .
Professor A: Well , yeah .
PhD E: It 's not a problem of channel , because there is um  normalization with respect to the channel . So Professor A: I I I 'm sorry . What what is the problem that you 're trying to explain ?
PhD E: The the fact that the result with the tandem and Aurora system are uh so much worse .
Professor A: That the Oh . So much worse ? Oh .
PhD E: Yeah .
Professor A: I uh but I 'm I 'm almost certain that it it  I mean , that it has to do with the um amount of training data .
PhD E: It Professor A: It it 's it 's orders of magnitude off .
PhD E: Yeah but Yeah . Yeah but we train only on digits and it 's it 's a digit task , so . Well .
Professor A: But but having a huge If  if you look at what commercial places do , they use a huge amount of data .
PhD E: It Mm - hmm .
Professor A: This is a modest amount of data .
PhD E: Alright . Yeah .
Professor A: So . I mean , ordinarily you would say " well , given that you have enough occurrences of the digits , you can just train with digits rather than with , you know " PhD E: Mm - hmm . Mm - hmm .
Professor A: But the thing is , if you have a huge in other words , do word models But if you have a huge amount of data then you 're going to have many occurrences of similar uh allophones .
PhD E: Right . Mmm .
Professor A: And that 's just a huge amount of training for it .
PhD E: Yeah .
Professor A: So it 's um  I I think it has to be that , because , as you say , this is , you know , this is near - microphone ,
PhD E: Mm - hmm .
Professor A: it 's really pretty clean data .
PhD E: Mm - hmm .
Professor A: Um . Now , some of it could be the fact that uh let 's see , in the in these multi - train things did we include noisy data in the training ?
PhD E: Yeah .
Professor A: I mean , that could be hurting us actually , for the clean case .
PhD E: Yeah . Well , actually we see that the clean train for the Aurora proposals are are better than the multi - train ,
Professor A: It is if Yeah .
PhD E: yeah .
Professor A: Yeah . Cuz this is clean data , and so that 's not too surprising .
PhD E: Mm - hmm .
Professor A: But um . Uh . So .
PhD E: Well , o I guess what I meant is that well , let 's say if we if we add enough data to train on the um on the Meeting Recorder digits , I guess we could have better results than this .
Professor A: Uh - huh . Mm - hmm .
PhD E: And . What I meant is that perhaps we can learn something uh from this , what 's what 's wrong uh what what is different between TI - digits and these digits and Professor A: What kind of numbers are we getting on TI - digits ?
PhD E: It 's point eight percent , so .
Professor A: Oh . I see .
PhD E: Four - Fourier .
Professor A: So in the actual TI - digits database we 're getting point eight percent ,
PhD E: Yeah . Yeah .
Professor A: and here we 're getting three or four three , let 's see , three for this ?
PhD E: Mm - hmm .
Professor A: Yeah . Sure , but I mean , um point eight percent is something like double uh or triple what people have gotten who 've worked very hard at doing that .
PhD E: Mm - hmm .
Professor A: And and also , as you point out , there 's adaptation in these numbers also . So if you , you know , put the ad adap take the adaptation off , then it for the English - Near you get something like two percent .
PhD E: Mmm .
Professor A: And here you had , you know , something like three point four . And I could easily see that difference coming from this huge amount of data that it was trained on .
PhD E: Mm - hmm .
Professor A: So it 's PhD E: Mm - hmm .
Professor A: You know , I don't think there 's anything magical here .
PhD E: Yeah .
Professor A: It 's , you know , we used a simple HTK system with a modest amount of data . And this is a a , you know , modern uh system uh has has a lot of nice points to it .
PhD E: Yeah . Mm - hmm .
Professor A: Um . So . I mean , the HTK is an older HTK , even . So . Yeah it it 's not that surprising .
PhD E: Mm - hmm .
Professor A: But to me it just it just meant a practical point that um if we want to publish results on digits that that people pay attention to we probably should uh Cuz we 've had the problem before that you get show some nice improvement on something that 's that 's uh , uh it seems like too large a number , and uh uh people don't necessarily take it so seriously .
PhD E: Mm - hmm .
Professor A: Um . Yeah . Yeah . So the three point four percent for this uh is is uh So why is it It 's an interesting question though , still . Why is why is it three point four percent for the d the digits recorded in this environment as opposed to the uh point eight percent for for for the original TI - digits database ? Um .
PhD E: Yeah . th that 's th that 's my point
Professor A: Given given the same Yeah . So ignore ignoring the the the SRI system for a moment ,
PhD E: I I I don't I Mm - hmm .
Professor A: just looking at the TI - di the uh tandem system , if we 're getting point eight percent , which , yes , it 's high . It 's , you know , it it 's not awfully high ,
PhD E: Mm - hmm .
Professor A: but it 's , you know it 's it 's high . Um . Why is it uh four times as high , or more ?
PhD E: Yeah , I guess .
Professor A: Right ? I mean , there 's  even though it 's close - miked there 's still there really is background noise .
PhD E: Mm - hmm .
Professor A: Um . And uh I suspect when the TI - digits were recorded if somebody fumbled or said something wrong or something that they probably made them take it over .
PhD E: Mm - hmm .
Professor A: It was not I mean there was no attempt to have it be realistic in any in any sense at all .
PhD E: Well . Yeah . And acoustically , it 's q it 's I listened . It 's quite different . TI - digit is it 's very , very clean and it 's like studio recording
Professor A: Mm - hmm .
PhD E: whereas these Meeting Recorder digits sometimes you have breath noise and Mmm .
Professor A: Right . Yeah . So I think they were PhD E: It 's {nonvocalsound} not controlled at all , I mean .
Professor A: Bless you .
Grad B: Thanks .
Professor A: I Yeah . I think it 's it 's So . Yes .
PhD E: Mm - hmm . But
Professor A: It 's I think it 's it 's the indication it 's harder .
PhD E: Yeah .
Professor A: Uh . Yeah and again , you know , i that 's true either way . I mean so take a look at the uh  um , the SRI results . I mean , they 're much much better , but still you 're getting something like one point three percent for uh things that are same data as in T TI - digits the same same text .
PhD E: Mm - hmm .
Professor A: Uh . And uh , I 'm sure the same same system would would get , you know , point point three or point four or something on the actual TI - digits . So this I think , on both systems the these digits are showing up as harder .
PhD E: Mmm .
Professor A: Um .
PhD E: Mm - hmm .
Professor A: Which I find sort of interesting cause I think this is closer to uh I mean it 's still read . But I still think it 's much closer to to what what people actually face , um when they 're they 're dealing with people saying digits over the telephone . I mean . I don't think uh I mean , I 'm sure they wouldn't release the numbers , but I don't think that uh the uh the the companies that that do telephone speech get anything like point four percent on their digits . I 'm I 'm I 'm sure they get Uh , I mean , for one thing people do phone up who don't have uh uh Middle America accents and it 's a we we it 's it 's it 's US .
PhD E: Mm - hmm .

Summary: The team was concerned that there was something wrong with the back-end of the model. The professor was insistent that the problem was probably with the lack of data. The team also speculated that their data was more challenging.