C: So , uh B: Alright .
F: Um , so I wanted to discuss digits briefly , but that won't take too long .
C: Oh good .
C: Right .
C: OK , agenda items ,
C: Uh , we have digits ,
C: What else we got ?
A: New version of the presegmentation .
C: New version of presegmentation .
B: Um , do we wanna say something about the , an update of the , uh , transcript ?
G: Yeah , why don't you summarize the C: Update on transcripts .
G: And I guess that includes some the filtering for the , the ASI refs , too .
B: Mmm .
C: Filtering for what ?
G: For the references that we need to go from the the fancy transcripts to the sort of <nonvocalsound> brain - dead .
B: It 'll it 'll be basically it 'll be a re - cap of a meeting that we had jointly this morning .
C: Uh - huh .
G: With Don , as well .
B: Mm - hmm .
C: Got it .
C: Anything else more pressing than those things ?
C: So So , why don't we just do those .
C: You said yours was brief , so F: OK .
F: OK well , the , w uh as you can see from the numbers on the digits we 're almost done .
F: The digits goes up to about four thousand .
F: Um , and so , uh , we probably will be done with the TI - digits in , um , another couple weeks . um , depending on how many we read each time .
F: So there were a bunch that we skipped .
F: You know , someone fills out the form and then they 're not at the meeting and so it 's blank .
F: Um , but those are almost all filled in as well .
F: And so , once we 're it 's done it would be very nice to train up a recognizer and actually start working with this data .
D: So we 'll have a corpus that 's the size of TI - digits ?
F: And so F: One particular test set of TI - digits .
D: Test set , OK .
F: So , I I extracted , Ther - there was a file sitting around which people have used here as a test set .
F: It had been randomized and so on
F: and that 's just what I used to generate the order . of these particular ones .
D: Oh ! Great .
D: Great .
F: Um C: So , I 'm impressed by what we could do , Is take the standard training set for TI - digits , train up with whatever , you know , great features we think we have , uh for instance , and then test on uh this test set .
C: And presumably uh it should do reasonably well on that ,
C: and then , presumably , we should go to the distant mike , and it should do poorly .
D: Yeah .
F: Right .
C: And then we should get really smart over the next year or two , and it that should get better .
F: And inc increase it by one or two percent , yeah .
C: Yeah , Yeah .
F: Um , but , in order to do that we need to extract out the actual digits .
C: Right .
F: Um , so that the reason it 's not just a transcript is that there 're false starts , and misreads , and miscues and things like that .
F: And so I have a set of scripts and X Waves where you just select the portion , hit R ,
F: um , it tells you what the next one should be , and you just look for that .
F: You know , so it it 'll put on the screen , " The next set is six nine , nine two two " .
F: And you find that , and , hit the key and it records it in a file in a particular format .
C: So is this F: And so the the question is , should we have the transcribers do that or should we just do it ?
F: Well , some of us .
F: I 've been do I 've done , eight meetings , something like that , just by hand .
F: Just myself , rather .
F: So it will not take long .
F: Um C: Uh , what what do you think ?
B: My feeling is that
B: we discussed this right before coffee
B: and I think it 's a it 's a fine idea partly because , um , it 's not un unrelated to their present skill set ,
B: but it will add , for them , an extra dimension , it might be an interesting break for them .
B: And also it is contributing to the , uh , c composition of the transcript
B: cuz we can incorporate those numbers directly and it 'll be a more complete transcript .
B: So I 'm I think it 's fine , that part .
F: There is there is C: So you think it 's fine to have the transcribers do it ?
B: Mm - hmm .
C: Yeah , OK .
F: There 's one other small bit , which is just entering the information which at s which is at the top of this form , onto the computer , to go along with the where the digits are recorded automatically .
D: Good .
C: Yeah .
F: And so it 's just , you know , typing in name , times time , date , and so on .
F: Um , which again either they can do , but it is , you know , firing up an editor , or , again , I can do .
F: Or someone else can do .
B: And , that , you know , I 'm not , that that one I 'm not so sure if it 's into the the , things that , I , wanted to use the hours for ,
B: because the , the time that they 'd be spending doing that they wouldn't be able to be putting more words on .
C: Mmm .
B: But that 's really your choice , it 's your D: So are these two separate tasks that can happen ?
D: Or do they have to happen at the same time before F: No they don't have this you have to enter the data before , you do the second task , but they don't have to happen at the same time .
F: So it 's it 's just I have a file whi which has this information on it , and then when you start using my scripts , for extracting the times , it adds the times at the bottom of the file .
D: OK .
F: And so , um , I mean , it 's easy to create the files and leave them blank , and so actually we could do it in either order .
D: Oh , OK .
F: Um , it 's it 's sort of nice to have the same person do it just as a double - check , to make sure you 're entering for the right person .
F: But , either way .
C: Yeah .
C: Yeah just by way of uh , uh , a uh , order of magnitude , uh , um , we 've been working with this Aurora , uh data set .
C: And , uh , the best score , on the , nicest part of the data , that is , where you 've got training and test set that are basically the same kinds of noise and so forth , uh , is about ,
C: uh C: I think the best score was something like five percent , uh , error , per digit .
C: So , that F: Per digit .
A: Per digit .
C: You 're right .
C: So if you were doing ten digit , uh , recognition , you would really be in trouble .
D: Mm - hmm .
C: So So the The point there , and this is uh car noise uh , uh things , but but real real situation ,
C: well , " real " ,
C: Um , the uh there 's one microphone that 's close , that they have as as this sort of thing , close versus distant .
C: Uh but in a car ,
C: instead of instead of having a projector noise it 's it 's car noise .
C: Uh but it wasn't artificially added to get some some artificial signal - to - noise ratio .
C: It was just people driving around in a car .
C: So , that 's that 's an indication ,
C: uh that was with , many sites competing , and this was the very best score and so forth ,
C: so . More typical numbers like
D: Although the models weren't , that good ,
D: right ?
D: I mean , the models are pretty crappy ?
C: You 're right .
C: I think that we could have done better on the models ,
C: but the thing is that we got this this is the kind of typical number , for all of the , uh , uh , things in this task , all of the , um , languages .
C: And so I I think we 'd probably the models would be better in some than in others .
D: Hmm .
C: Um , so , uh . Anyway , just an indication once you get into this kind of realm even if you 're looking at connected digits it can be pretty hard .
B: Hmm .
B: It 's gonna be fun to see how we , compare at this .
C: Yeah .
D: How did we do on the TI - digits ?
B: Very exciting . s @ @ .
F: Well the prosodics are so much different
F: s it 's gonna be , strange .
F: I mean the prosodics are not the same as TI - digits , for example .
C: Yeah .
F: So I 'm I 'm not sure how much of effect that will have .
G: What do you mean , the prosodics ?
D: H how do F: Um , just what we were talking about with grouping .
F: That with these , the grouping , there 's no grouping at all , and so it 's just the only sort of discontinuity you have is at the beginning and the end .
G: So what are they doing in Aurora ,
G: are they reading actual phone numbers ,
F: Aurora I don't know .
F: I don't know what they do in Aurora .
G: or , a a digit at a time , or ?
C: Uh , I 'm not sure how G: Cuz it 's C: no , no I mean it 's connected it 's connected , uh , digits ,
G: Connected .
C: yeah .
G: So there 's also the not just the prosody but the cross the cross - word modeling is probably quite different .
C: But .
F: But Right .
F: But in TI - digits , they 're reading things like zip codes and phone numbers and things like that ,
D: H How
G: Right .
F: so it 's gonna be different .
D: do we do on TI - digits ?
F: I don't remember .
F: I mean , very good ,
F: right ?
C: Yeah , I mean we were in the .
F: One and a half percent , two percent , something like that ?
C: Uh , I th no I think we got under a percent ,
F: Oh really ?
C: but it was but it 's but I mean . The very best system that I saw in the literature was a point two five percent or something that somebody had at at Bell Labs , or . Uh , but . But , uh , sort of pulling out all the stops .
F: OK .
B: s @ @ . It s strikes me that there are more each of them is more informative because it 's so , random ,
F: Alright .
D: Hmm .
C: But I think a lot of systems sort of get half a percent , or three - quarters a percent ,
F: Right .
C: and we 're we 're in there somewhere .
F: But that I mean it 's really it 's it 's close - talking mikes , no noise , clean signal , just digits , I mean , every everything is good .
C: Yeah .
G: It 's the beginning of time in speech recognition .
F: Yes , exactly .
C: Yeah .
F: And we 've only recently got it to anywhere near human .
G: It 's like the , single cell , you know , it 's the beginning of life ,
D: Pre - prehistory .
G: yeah .
F: And it 's still like an order of magnitude worse than what humans do .
G: Right .
C: Yeah .