PhD E: What about the , um uh , the new part of the evaluation , the , uh , Wall Street Journal part ?
Professor B: Right . Right . Um . Have you ever ? Very good question . Have you ever worked with the Mississippi State h uh , software ?
PhD A: Sorry .
PhD E: No . Not yet .
Professor B: Oh . Well you you may be called upon to help , uh , uh , on account of , uh , all the work in this stuff here has been , uh , with small vocabulary .
PhD E: OK . Mm - hmm . So what how is the , uh , interaction supposed to happen ? Uh , I remember the last time we talked about this , it was sort of up in the air whether they were going to be taking , uh , people 's features and then running them or they were gonna give the system out or PhD D: Yeah . Yeah .
PhD E: Oh , so they 're gonna just deliver a system basically .
PhD D: Yeah , yeah .
Professor B: Do we already have it ?
PhD D: Yeah , th I I guess it 's almost ready .
PhD E: Uh - huh .
PhD D: So That 's what So they have released their , uh , document , describing the system .
Professor B: Maybe you could , uh , point it at Chuck ,
PhD E: I see .
Professor B: because , I mean PhD D: Sure .
PhD E: So we 'll have to grab this over CVS or something ?
PhD D: It - no , it 's just downloadable from their from their web site .
PhD E: Is that how they do it ? OK .
Professor B: Cuz one of the things that might be helpful , if you 've if you 've got time in all of this is , is if if these guys are really focusing on improving , uh , all the digit stuff , uh , maybe and you got the front - end from them , maybe you could do the runs for the PhD E: OK . Mm - hmm .
Professor B: and and , you know , iron out hassles that that you have to , uh , tweak Joe about or whatever ,
PhD E: Sure .
Professor B: because you 're more experienced with running the large vocabulary stuff .
PhD E: OK .
Professor B: S
PhD D: So I 'll point you to the web site and the mails corresponding . So I
PhD E: And it but it 's not ready yet , the system ?
PhD D: Uh , I I think they are still , uh , tuning something on that . So they 're like , d they 're varying different parameters like the insertion penalty and other stuff , and then seeing what 's the performance .
PhD E: Are those going to be parameters that are frozen , nobody can change ? Or ?
PhD D: Uh , w I guess there is , uh , time during which people are gonna make suggestions .
PhD E: Oh , but everybody 's gonna have to use the same values .
PhD D: After that .
PhD E: Oh ! Interesting .
PhD D: Yeah , I guess .
PhD E: OK .
PhD D: So these sugges these this , uh , period during which people are gonna make suggestions is to know whether it is actually biased towards any set of features or Professor B: Yeah , so I th th certainly the thing that I would want to know about is whether we get really hurt , uh , on in insertion penalty , language model , scaling , sorts of things .
PhD E: Using our features .
Professor B: Yeah , yeah .
PhD E: Yeah .
Professor B: Uh , in which case , um , H Hari or Hynek will need to , you know , push the case more about about this .
PhD E: Mm - hmm .
Professor B: Um .
PhD E: And we may be able to revisit this idea about , you know , somehow modifying our features to work with Professor B: Yes . In this case , that 's right .
PhD E: Yeah .
Professor B: That 's right . Um , some of that may be , uh , a last minute rush thing because if the if our features are changing Uh .
PhD E: Yeah .
Professor B: Uh . But , um . Yeah , the other thing is that even though it 's months away , uh , it 's starting to seem to me now like November fifteenth is right around the corner . And , um , if they haven't decided things like this , like what the parameters are gonna be for this , uh , when " deciding " is not just somebody deciding . I mean , in fact there should be some understanding behind the , uh , deciding , which means some experiments and and so forth . It it it seems pretty tight to me .
PhD E: So wha what 's the significance of November fifteenth ?
Professor B: That 's when the evaluation is .
PhD E: OK .
Professor B: Yeah . So , yeah , so after But , you know , they may even decide in the end to push it off . It wouldn't , you know , entirely surprise me . But , uh , due to other reasons , like some people are going away , I 'm I 'm hoping it 's not pushed off for a l a long while . That would be , uh put us in an awkward position . But Anyway .
PhD E: OK .
Professor B: Great . Yeah , I think that 'll be helpful . There 's there 's not anybody OGI currently who 's who 's , uh , working with this and and
PhD E: Is is this part of the evaluation just a small part , or ho how important is this to the overall ?
Professor B: I I think it 's it 's , um it depends how badly you do . I mean , I think that it it is Uh .
PhD D: b
PhD E: This is one of those things that will be debated afterwards ?
Professor B: Yeah . Well , I mean , it 's it 's Conceptually , it my impression , again , you guys correct me if I 'm wrong , but my impression is that , um , they want it as a double check . That you haven't come across you haven't invented features which are actually gonna do badly for a a significantly different task , particularly one with larger vocabulary . And , um , but it 's not the main emphasis .
PhD E: Mmm .
Professor B: I mean , the truth is , most of the applications they 're looking at are pretty small vocabulary .
PhD E: Mmm .
Professor B: So it 's it 's a double check . So they 'll probably assign it some sort of low weight .
PhD E: Seems to me that if it 's a double check , they should give you a one or a zero . Y you passed the threshold or you didn't pass the threshold , and they shouldn't even care about what the score is .
Professor B: Yeah . But , I mean , we 'll we 'll we 'll see what they come up with . Uh , but in in the current thing , for instance , where you have this well - matched , moderately - matched , and and mis highly - mismatched , uh , the emphasis is somewhat on the on the well - matched , but it 's only a a marginal ,
PhD E: Yeah .
Professor B: right ? It 's like forty , thirty - five , twenty - five , or something like that . So you still if you were way , way off on the highly - mismatched , it would have a big effect .
PhD D: Yeah .
PhD E: Mm - hmm .
Professor B: And , um , it wouldn't surprise me if they did something like that with this . So again , if you 're if you get If it doesn't help you much , uh , for noisy versions of this of large vocabulary data , then , uh , you know , it may not hurt you that much .
PhD E: Oh .
Professor B: But if it if you don't if it doesn't help you much at all , um , or to put it another way , if it helps some people a lot more than it helps other people , uh , if their strategies do , then PhD E: Mm - hmm . So is this , uh ? Uh , Guenter was putting a bunch of Wall Street Journal data on our disks .
Professor B: That 's it .
PhD E: So that 's the data that we 'll be running on ?
Professor B: Yeah .
PhD E: I see . OK .
Professor B: Yeah . So we have the data , just not the recognizer . OK .
PhD E: So this test may take quite a while to run , then . May - judging by the amount of data that he was putting on .
Professor B: Uh , well there 's training and test , right ?
PhD E: I I guess , I 'm not sure .
Professor B: No , I mean , if it 's like the other things , there 's there 's data for training the H M Ms and and data for testing it .
PhD E: I just Professor B: So I wouldn't So it it 's PhD E: OK . So there 's Professor B: So , training the recognizer , but , um Um . But I think it 's trained on clean and Is it trained on clean and and test on ?
PhD D: The Wall Street ?
Professor B: Yeah .
PhD A: Apparently , no . It 's training on a range between ten and twenty DB , I think , and testing between five and fifteen .
PhD D: Mm - hmm . Yeah .
PhD A: That 's what I got on Professor B: OK .
PhD D: It 's , uh It 's like a medium medium - mismatch condition , sort of .
PhD A: Yeah ,
Professor B: I see .
PhD A: and So the noise is There is a range of different noises also um which are selected randomly and added randomly , uh , to the files . And there are noises that are different from the noises used on TI - digits .
Professor B: Yeah . Yeah . I mean , I wouldn't imagine that the amount of testing data was that huge . They probably put training uh , almost certain they put training data there too . Maybe not . So . That 's that . Anybody have anything else ?
PhD E: Uh , one one last question on that . When did they estimate that they would have that system available for download ?
PhD D: Um , I guess I guess one some preliminary version is already there .
PhD E: Oh , so there 's w something you can download to just learn ?
PhD D: Yeah , it 's already there . Yeah .
PhD E: OK ,
PhD D: But they 're actually parallel - y doing some modifications also , I think .
PhD E: good .
PhD D: So I guess the f final system will be frozen by middle of , like , one more week maybe .
PhD E: OK .
Professor B: Oh , well that 's pretty soon .
PhD D: Yeah , that 's just one more .
Grad C: Is this their , um , SVM recognizer ?
PhD D: No , it 's just a straightforward HMM .
Professor B: You know , their their They have a lot of options in their recognizer and and the SVM is one of the things they 've done with it , but it 's not their more standard thing .
Grad C: Oh , OK . Uh - huh .
Professor B: For the most part it 's it 's Gaussian mixtures .
Grad C: Oh , OK . Oh , OK .
Professor B: Yeah .
PhD D: It 's just a HMM , Gaussian mixture model .
Grad C: Gaussian mixture HMM .
Professor B: Yeah .
Grad C: OK .
Professor B: Yeah , the SVM thing was an HMM also . It was just a it it it was like a hybrid , like Grad C: Mm - hmm .
PhD D: Yeah , this is a g yeah , this i
Professor B: what ?
PhD D: yeah .
Professor B: Yeah .
PhD E: So , just so that I understand , they 're providing scripts and everything so that basically , uh , you you push a button and it does training , and then it does test , and everything ? Is that the idea ?
PhD D: I I I think yeah , I I guess something like that . It 's like  as painless as possible ,
PhD E: Mm - hmm .
PhD D: is what Do they provide all the scripts , everything , and then Just ,
PhD E: I see . Hmm . Somehow yo there 's hooks to put your features in and PhD D: ju Yeah , I th I think .
PhD E: Hmm .
Professor B: Hmm . Yeah , um . In fact , I mean , if you look into it a little bit , it might be reasonable You know Joe , right ? Yeah .
PhD E: Mm - hmm .
Professor B: Just to sort of ask him about the issue of , um , different features having different kinds of , uh , scaling characteristics and so on . So that , you know , w w possibly having entirely different optimal values for for the usual twiddle factors and what 's what 's the plan about that ?
PhD E: OK .
PhD D: So sh shall we , like , add Chuck also to the mailing lists ? It may be better , I mean , in that case if he 's going to Professor B: Yeah .
PhD D: Because there 's a mailing list for this .
Professor B: Is that OK ?
PhD E: Yeah , that 'd be great .
PhD D: Yeah , I guess maybe Hari or Hynek , one of them , has to send a mail to Joe . Or maybe if you PhD E: I I could send him an email .
PhD D: Well , yeah , to add or maybe wh
PhD E: I I know him really well .
PhD D: Yeah , so that 's just fine .
PhD E: I I was just talking with him on email the other day actually .
PhD D: So Professor B: Uh , yeah , and just , um , se maybe see .
PhD D: So PhD E: About other things , but .
Professor B: Do you have Hari 's , uh ?
PhD E: I have Hari 's Professor B: Yeah , so maybe just CC Hari and say that you 've just been asked to handle the large vocabulary part here , and , uh , you know ,
PhD E: OK . Would it be better if I asked Hari to ask Joe ?
Professor B: Uh . Why don't you just ask Joe but CC Hari , and then in the note say , " Hari , hopefully this is OK with you " .
PhD E: OK .
Professor B: And then if Joe feels like he needs a confirmation , Hari can answer it .
PhD E: OK .
PhD D: Yeah .
Professor B: That way you can get started asking {comment} Joe quickly while he 's while he 's maybe still , you know , putting in nails and screws and  Yeah .
PhD D: And there is an , uh , archive of all the mails that has been gon that has gone , uh , between these people among these people . So just you can see all this mails in the ISIP web site PhD E: OK .
PhD D: Mississippi web site .
PhD E: OK . Is that a password controlled ?
PhD D: Yeah , it 's password protected .
PhD E: OK .
PhD D: So , like like , it 's , like Professor B: Have you thought about how long would be uh , most useful for you to go up to OGI ?
PhD A: I don't know , uh . We can  For September , we can set up a work schedule and we can maybe work independently . And then at some point it maybe be better to work together again .
Professor B: Oh , so you 're you 're imagining more that you would come back here first for a while and then and then go up there ?
PhD A: I Professor B: I mean , it 's to you .
PhD A: Maybe , yeah .
Professor B: I ju you guys are Well , y anyway , you don't have to decide this second but thi think about it about what what you would think would be the the best way to work it . I 'll
PhD A: But , uh Huh . Mm - hmm .
Professor B: support it either way , so .
PhD A: Mm - hmm Right .