E: Mm - hmm .
F: but but uh D: Right .
E: Yeah , I also would like to argue for that
E: cuz it it seems to me that , um , there 's a real strength in having the same test replicated in a whole bunch of times
E: and adding to that basic test bank .
D: Right .
E: Hmm ?
E: Cuz then you have , you know , more and more , u chances to get away from random errors .
E: And I think , um , the other thing too is that right now we have sort of a stratified sample with reference to dialect groups ,
E: and it might be there might be an argument to be made for having uh f for replicating all of the digits that we 've done ,
E: which were done by non - native speakers
E: so that we have a core that totally replicates the original data set ,
E: which is totally American speakers ,
E: and then we have these stratified additional language groups overlapping certain aspects of the database .
D: Right .
D: I think that uh trying to duplicate , spending too much effort trying to duplicate the existing TI - digits probably isn't too worthwhile
D: because the recording situation is so different .
F: Yeah .
D: It 's gonna be very hard to be comparable .
E: Except that if you have the stimuli comparable , then it says something about the the contribution of setting
F: No it 's it 's not the same .
F: A little bit ,
E: and F: but the other differences are so major .
E: OK .
F: They 're such major sources of variance that it 's it 's it 's uh D: Yeah I mean read versus not .
E: What 's an example of a of m some of the other differences ?
E: Any other a difference ?
F: Well i i individual human glottis is going to be different for each one ,
E: OK .
F: you know , it 's just There 's so many things .
E: OK .
F: it 's it and and enunciation .
D: Well , and not just that ,
D: I mean the uh the corpus itself .
D: I mean , we 're collecting it in a read digit in a particular list ,
D: and I 'm sure that they 're doing more specific stuff .
D: I mean if I remember correctly it was like postman reading zipcodes and things like that .
F: TI - digits was ?
F: I thought I thought it was read .
D: I thought so .
D: Was it read ?
F: Yeah , I think the reading zipcode stuff you 're thinking of would be OGI .
D: Oh , I may well be .
F: Yeah , no TI - digits was read in th in read in the studio I believe .
D: I haven't ever listened to TI - digits .
F: Yeah .
D: So I don't really know how it compares .
F: Yeah .
F: But it but D: But but regardless it 's gonna it 's hard to compare cross - corpus .
F: It - it 's different people is the is the core thing .
D: So .
F: And they 're different circumstances with different recording environment and so forth ,
E: OK , fine .
F: so it 's it 's it 's really pretty different .
F: But I think the idea of using a set thing was just to give you some sort of framework ,
F: so that even though you couldn't do exact comparisons , it wouldn't be s valid scientifically at least it 'd give you some kind of uh frame of reference .
F: Uh , you know it 's not E: OK .
B: Hey Liz , What what do the groupings represent ?
B: You said there 's like ten different groupings ?
G: Right , just groupings in terms of number of groups in a line ,
G: and number of digits in a group ,
G: and the pattern of groupings .
B: Mm - hmm .
B: Are the patterns like are they based on anything or
G: Um , I I just roughly looked at what kinds of digit strings are out there ,
G: and they 're usually grouped into either two , three , or four , four digits at a time .
B: Oh .
G: And they can have ,
G: I mean , actually , things are getting longer and longer .
G: In the old days you probably only had three sequences , and telephone numbers were less , and so forth .
G: So , there 's between , um G: Well if you look at it , there are between like three and five groups ,
G: and each one has between two and four groupings
G: and G: I purposely didn't want them to look like they were in any kind of pattern .
B: Mmm .
G: So
D: And which group appears is picked randomly , and what the numbers are are picked randomly .
B: Mm - hmm .
G: Right .
D: So unlike the previous one , which I d simply replicated TI - digits , this is generated randomly .
B: Mmm , oh , OK .
A: Oh OK .
G: But I think it 'd be great i to be able to compare digits ,
G: whether it 's these digits or TI - digits , to speakers , um , and compare that to their spontaneous speech ,
G: and then we do need you know a fair amount of of digit data
G: because you might be wearing a different microphone
G: and , I mean so it 's it 's nice to have the digits you know , replicated many times .
D: Mm - hmm .
G: Especially for speakers that don't talk a lot .
G: So um ,
D: Yeah .
G: for adaptation .
G: No , I 'm serious ,
G: so we have a problem with acoustic adaptation ,
F: Yeah .
D: Yeah all we have for some people is digits .
A: Yeah .
G: and we 're not using the digit data now ,
G: but you know D: Oh , you 're not .
G: Not for adaptation , nope .
G: v W we 're not we were running adaptation only on the data that we ran recognition on
G: and I 'd As soon as someone started to read transcript number , that 's read speech
G: and I thought " well , we 're gonna do better on that ,
D: Oh I see .
G: that 's not fair to use " .
D: Oh yeah that 's true , absolutely .
G: But , it might be fair to use the data for adaptation ,
A: OK .
G: so .
G: So those speakers who are very quiet , <comment> shy D: That would be interesting to see whether that helps .
G: r Right D: Do you think that would help adapting on B: Like Adam ?
D: Yeah .
F: Yeah .
D: Yeah , I have a real problem with that .
G: Well , it sh I mean it 's the same micropho
G: see the nice thing is we have that in the in the same meeting ,
D: Right .
F: Yeah .
G: and so you don't get D: Same same acoustics ,
D: same microphone ,
D: same channel .
A: Yeah .
G: Right , and so I still like the idea of having some kind of digit data .
D: OK .
D: Good .
F: Yeah I mean , for the for the um acoustic research ,
F: for the signal - processing , farfield stuff , I see it as as as the place that we start .
F: But , th I mean , it 'd be nice to have twenty hours of digits data ,
F: but but uh the truth is I 'm hoping that we we through the the stuff that that you guys have been doing as you continue that , we get , uh , the best we can do on the spontaneous stuff uh , uh nearfield ,
F: and then um , we do a lot of the testing of the algorithms on the digits for the farfield ,
F: and at some point when we feel it 's mature and we understand what 's going on with it then we we have to move on to the spontaneous data with the farfield .
F: So .
G: The only thing that we don't have ,
E: Great .
G: I know this sounds weird ,
G: and maybe it 's completely stupid ,
G: but we don't have any overlapping digits .
D: Yeah , we talked about that a couple times .
G: An - yea I know it 's weird ,
G: but um A: Overlapping digits !
D: The the problem I see with trying to do overlapping digits is the cognitive load .
G: Alright everybody 's laughing .
G: OK .
C: Dueling digits .
D: No it 's it 's not stupid ,
D: it 's just I mean , try to do it .
G: I 'm just talkin for the stuff that like Dan Ellis is gonna try ,
D: I mean , here , let 's try it .
G: you know , cross - talk cancellation .
F: Let 's try it .
D: You read the last line , I 'll read the first line .
G: OK .
A: Oh !
G: Wait oh it these are all the same forms .
F: Sixty - one .
G: OK <comment> So but D: So so you read the last line , I 'll read the first line .
G: So you plu you plug your ears .
F: No , I 'll p
D: Oh I guess if you plug you 're ears you could do it ,
D: but then you don't get the the same effects .
A: Yeah .
G: Well , what I mean is actually no not the overlaps that are well - governed linguistically ,
G: but the actual fact that there is speech coming from two people
G: and the beam - forming stuf
D: Yeah .
G: all the acoustic stuff that like Dan Ellis and and company want to do .
D: Oh I see .
G: Digits are nice and well behaved ,
G: I mean
G: Anyway , it 's just a thought .
D: I guess we could try .
D: We could try doing some .
G: It it would go faster .
B: Parallel .
G: It would take one around <comment> amount of ti
B: It 's the P - make of digit reading .
D: Well Well OK . Well let 's try it .
G: That 's right .
G: I I mea I 'm I was sort of serious ,
G: but I really , I mean , I 'm I don't feel strongly enough that it 's a good idea ,
F: See , y
D: You do the last line , I 'll do the first line .
G: so .
F: OK .
D: O . <comment> That 's not bad .
F: No , I can do it .
G: A and that prosody was great , by the way .
B: I couldn't understand a single thing you guys were saying .
E: I think it was numbers ,
E: but I 'm not sure .
G: It it sort of sounded like a duet , or something .
A: Yeah .
B: Performance art .
F: Alright , let 's try three at once you you pick one in the middle .
A: The Aurora theater .
G: OK .
F: Go .
G: I 'm sorry .
G: I 'm mean I think it 's doable ,
D: The poor transcribers
G: I 'm just D: they 're gonna hate us .
G: So , we we could have a round like where you do two at a time ,
G: and then the next person picks up when the first guy 's done , or something .
A: So pairwise .
F: Oh like a round ,
F: yeah , like in a a G: Like a ,
A: Yeah , just pairwise ,
F: yeah .
G: what do you call it ?
G: Li - a r like D: A round .
A: or
F: Row , row , row your boat .
C: Round .
A: yeah .
D: Mm - hmm .
F: Yeah .
G: yeah , like that .
F: OK .
B: It 's gonna require some coordination .
G: Then it would go like h twice as fast , or a third as fast .
G: Anyway , it 's just a thought .
E: You have to have a similar pace .
F: Yeah .
G: I 'm actually sort of serious if it would help people do that kind o
G: but the people who wanna work on it we should talk to them .
F: I don't think we 're gonna collect vast amounts of data that way ,
G: So .
D: Mmm .
F: but I think having a little bit might at least be fun for somebody like Dan to play around with ,
G: OK .
D: I think maybe if we wanted to do that we would do it as a separate session ,
F: yeah .
G: Yeah .
D: something like that
D: rather than doing it during a real meeting
D: and you know , do two people at a time
D: then three people at a time and things like that .
D: So .
G: Can try it out .
G: If we have nothing if we have no agenda we could do it some week .
D: See see what Dan thinks .
D: Yeah , right .
F: Yeah , yeah .
G: OK .
F: Spend the whole time reading digits with different qu quantities .
E: c c Can I can I have an another another question w about this ?
D: I thought this was gonna be fast .
D: Oh well .
E: So , um , there are these digits ,
E: which are detached digits ,
E: but there are other words that contain the same general phon phoneme sequences .
E: Like " wonderful " has " one " in it
E: and and Victor Borge had a had a piece on this where he inflated the digits .
E: Well , I wonder if there 's , um , an if there would be a value in having digits that are in essence embedded in real words to compare in terms of like the articulation of " one " in " wonderful " versus " one " as a digit being read .
F: That 's " two " bad .
F: Yeah .
G: I 'm all " four " it .
E: There you go .
D: Not after I " eight " though .
F: Uh , they don't all work as well , do they ?
F: Hmm .
F: What does nine work in ?
D: Uh .
F: Uh ,
C: Nein !
C: You scream it .
D: Nein !
F: Oh . In German ,
D: You have to be German ,
B: It 's great for the Germans .
F: yeah .
A: That 's German , yeah .
G: Oh , oh !
D: yeah .
E: Nein .
F: That 's right !
A: Yeah .
D: Oh !
C: It only sounds w good when you scream it , though .
C: So .
F: I think everybody 's a little punchy here today .
E: Well , I mean , I just wanted to offer that as a possible task
F: Yes .
E: because , you know , if we were to each read his embedded numbers words in sent in sentences
E: cuz it 's like an entire sketch he does
E: and I wouldn't take the inflated version .
E: So he talks about the woman being " two - derful " ,
E: and E: and a But , you know , if it were to be deflated , just the normal word , it would be like a little story that we could read .
F: Mm - hmm .
E: I don't know if it would be useful for comparison ,
E: but it 's embedded numbers .
F: Well I don't know .
D: I think for something like that we 'd be better off doing like uh TIMIT .
F: Well I think the question is what the research is ,
F: so I mean , I presume that the reason that you wanted to have these digits this way is because you wanted to actually do some research looking at the prosodic form here .
D: Hmm .
G: Right , yeah .
F: Yeah OK .
F: So if somebody wanted to do that , if they wanted to look at the the the difference of the uh phones in the digits in the context of a word versus uh the digits a a non - digit word versus in digit word , uh that would be a good thing to do ,
F: but I think someone would have to express interest in that .
E: I see .
F: I think , to E: OK .
F: I mean if you were interested in it then we could do it , for instance .
E: OK , thank you .
E: Huh .
D: OK , are we done with digits ?
D: Um ,
D: We have ASR results from Liz ,
D: transcript status from Jane ,
D: and disk space and storage formats from Don .
D: Does do we have any prefer preference on which way we wanna we wanna go ?
G: Well I was actually gonna skip the ASR results part , in favor of getting the transcription stuff talked about
D: Mm - hmm .
G: since I think that 's more important to moving forward ,
G: but I mean Morgan has this paper copy
G: and if people have questions ,
G: um , it 's pretty preliminary in terms of ASR results
G: because we didn't do anything fancy ,
G: but I think e just having the results there ,
G: and pointing out some main conclusions
G: like it 's not the speaking style that differs ,
G: it 's the fact that there 's overlap that causes recognition errors .
G: And then , the fact that it 's almost all insertion errors ,
G: which you would expect
G: but you might also think that in the overlapped regions you would get substitutions and so forth ,
G: um , leads us to believe that doing a better segmentation , like your channel - based segmentation , or some kind of uh , echo cancellation to get basically back down to the individual speaker utterances would be probably all that we would need to be able to do good recognition on the on the close - talking mikes .
D: Um , why don't you , if you have a hard copy , why don't you email it to the list .
A: So these G: So , that 's about the summary G: But this is Morgan has this paper .
A: Yeah , yeah .
G: I mean he he F: Yeah , so it 's the same thing ?
D: Oh it 's in the paper .
G: it it 's that paper .
F: It 's the same thing I mailed to every everybody that w where it was ,
D: OK .
G: Yeah , yeah .
G: So , we basically , um , did a lot of work on that
D: OK then , it 's already been mailed .
F: yeah .
G: and it 's G: Let 's see , th
G: I guess the other neat thing is it shows for sure w that the lapel , you know within speaker is bad .
D: Horrible ?
G: And it 's bad because it picks up the overlapping speech .
A: So , your your ASR results were run on the channels synchronized ,
G: Yes , cuz that 's all that w had been transcribed at the time ,
A: OK .
A: OK .
A: OK .
G: um
G: but as we I mean I wanted to here more about the transcription .
G: If we can get the channel asynchronous or the A: Yeah .
G: the closer t
G: that would be very interesting for us
G: because we B: So if F: Yeah , that 's that 's why I only used the part from use
A: Yeah .
F: which we had uh about uh about the alt over all the channels
A: Yeah .
G: Right .
G: That 's A: Yeah sure .
A: Yeah .
F: or mixed channel
F: rather mixed signal .
A: Yeah .
B: So if there was a segment of speech this long
G: cuz A: Yeah .
D: And someone said " oh " in the front in the middle .
B: and oh and someone said " oh , " the whole thing was passed to the recognizer ?
A: There were several speakers in it ,
A: yeah .
G: That 's right . In fact I I pulled out a couple classic examples in case you wanna u use them in your talk of
C: Mm - hmm .
B: That 's why there 's so many insertion errors ?
G: Chuck on the lapel ,
G: so Chuck wore the lapel three out of four times .
C: Mmm .
D: I noticed that Chuck was wearing the lapel a lot .
G: Um , yeah ,
B: Early on , yeah .
G: and I wore the lapel once ,
G: and for me the lapel was OK .
G: I mean I still G: and I don't know why .
G: I 'm G: But um ,
G: for you it was D: Probably how you wear it wore it I would guess .
G: Or who was next to me or something like that .
C: Yeah , where you were sitting probably affected it .
A: Yeah .
G: Right , but when Chuck wore the lapel and Morgan was talking there 're a couple really long utterances
G: where Chuck is saying a few things inside ,
G: and it 's picking up all of Morgan 's words pretty well
G: and so the rec you know , there 're error rates
G: because of insertion G: Insertions aren't bounded ,
G: so with a one - word utterance and ten insertions you know you got huge error rate .
D: Uh - huh .
A: Yeah .
G: And that 's that 's where the problems come in .
G: So I this is sort of what we expected ,
G: but it 's nice to be able to to show it .
D: Right .
G: And also I just wanted to mention briefly that , um , uh Andreas and I called up Dan Ellis who 's still stuck in Switzerland ,
G: and we were gonna ask him if if there 're you know , what 's out there in terms of echo cancellation and things like that .
G: Not that we were gonna do it ,
G: but we wanted to know what would need to be done .
D: And he said , " Lots lots lots lots . "
G: And he We 've given him the data we have so far ,
G: so these sychronous cases where there are overlap .
A: Yep .
G: And he 's gonna look into trying to run some things that are out there
G: and see how well it can do
G: because right now we 're not able to actually report on recognition in a real paper ,
B: So G: like a Eurospeech paper ,
G: because it would look sort of premature .
B: So B: So the idea is that you would take this big hunk where somebody 's only speaking a small amount in it ,
B: and then try to figure out where they 're speaking <comment> based on the other peopl
G: Right . Or who 's At any point in time who 's the foreground speaker ,
G: who 's the background speaker .
B: I thought we were just gonna move the boundaries in .
A: So yeah G: So .
A: Yeah , should it D: Well that 's with the hand stuff .
G: So there 's like D: But how would you do that automatically ?
B: Right .
A: Uh , I 've actually done some experiments with cross - correlation
G: Well ther there 's A: and it seems to work pretty well to to get rid of those those overlaps ,
F: Mm - hmm .
D: I mean that that 's the sort of thing that you would do .
G: Yeah .
G: Yeah . Exactly , so it 's it 's a D: So .
A: yeah .
B: So why do you want to do echo cancellation ?
G: Um , it would be techniques used from adaptive adaptive echo cancellation
G: which I don't know enough about to talk about .
B: Uh - huh .
F: It just it just to r to remove cross - talk .
G: Um .
G: But ,
C: Yeah .
F: Yeah .
G: right , um , and that would be similar to what you 're also trying to do ,
G: but using um , you know , more than energy G: I I don't know what exactly would go into it .
A: Yeah .
A: Yeah , sure .
G: So the idea is to basically run this on the whole meeting .
B: So it would be G: and get the locations , which gives you also the time boundaries of the individual speak
B: OK .
B: So do sort of what he 's already what he 's trying to do .
G: Right . Except that there are many techniques for the kinds of cues , um , that you can use to do that .