PhD A: but the rest would be constrained . So , I think we have a version that 's pretty good for the native speakers . I don't know yet about the non - native speakers . And , um , we basically also made noise models for the different sort of grouped some of the mouth noises together . Um , so , and then there 's a background speech model . And we also There was some neat or , interesting cases , like there 's one meeting where , um , Jose 's giving a presentation and he 's talking about , um , the word " mixed signal " and someone didn't understand , uh , that you were saying " mixed " I think , Morgan . And so your speech - ch was s saying something about mixed signal .
PhD H: Yeah , yeah .
PhD A: And the next turn was a lot of people saying " mixed " , like " he means mixed signal " or " I think it 's mixed " . And the word " mixed " in this segment occurs , like , a bunch of times .
PhD H: Sh
PhD A: And Chuck 's on the lapel here , and he also says " mixed " but it 's at the last one , and of course the aligner th aligns it everywhere else to everybody else 's " mixed " ,
PhD H: Yeah .
PhD A: cuz there 's no adaptation yet . So there 's  I think there 's some issues about u We probably want to adapt at least the foreground speaker . But , I guess Andreas tried adapting both the foreground and a background generic speaker , and that 's actually a little bit of a f funky model . Like , it gives you some weird alignments , just because often the background speakers match better to the foreground than the foreground speaker .
PhD F: Oh PhD D: Yeah .
PhD A: So there 's some things there ,
PhD H: Oh .
PhD A: especially when you get lots of the same words , uh , occurring in the PhD F: Well , the I I think you can do better by uh , cloning so we have a reject phone . And you and what we wanted to try with you know , once we have this paper written and have a little more time , uh , t cloning that reject model and then one copy of it would be adapted to the foreground speaker to capture the rejects in the foreground , like fragments and stuff , and the other copy would be adapted to the background speaker .
PhD A: Right . I mean , in general we actually PhD F: And PhD A: Right now the words like partial words are reject models and you normally allow those to match to any word .
PhD F: Mm - hmm .
PhD A: But then the background speech was also a reject model , and so this constraint of not allowing rejects in between you know , it needs to differentiate between the two . So just sort of working through a bunch of debugging kinds of issues .
PhD F: Right .
PhD A: And another one is turns , like people starting with " well I think " and someone else is " well how about " . So the word " well " is in this in this segment multiple times , and as soon as it occurs usually the aligner will try to align it to the first person who says it . But then that constraint of sort of uh , proximity constraint will push it over to the person who really said it in general .
Grad E: Is the proximity constraint a hard constraint , or did you do some sort of probabilistic weighting distance , or ?
PhD F: We we didn't PhD A: Right now it 's a kluge .
PhD F: No . We w OK . We it 's straightforward to actually just have a a penalty that doesn't completely disallows it but discourages it . But , um , we just didn't have time to play with , you know , tuning yet another yet another parameter .
Grad E: The ve level . Yeah .
PhD A: Yeah .
PhD F: And really the reason we can't do it is just that we don't have a we don't have ground truth for these . So , we would need a hand - marked , um , word - level alignments or at least sort of the boundaries of the speech betw you know , between the speakers . Um , and then use that as a reference and tune the parameters of the of the model , uh , to op to get the best performance .
PhD A: Yeah .
Professor B: G given I I mean , I wa I wa I was gonna ask you anyway , uh , how you assessed that things were better .
PhD F: Mm - hmm .
PhD A: I looked at them . I spent two days um , in Waves Professor B: OK .
PhD A: Oh , it was painful because the thing is , you know the alignments share a lot in common , so And you 're yo you 're looking at these segments where there 's a lot of speech . I mean , a lot of them have a lot of words . Not by every speaker
Professor B: Yeah .
PhD A: but by some speaker there 's a lot of words . No , not Professor B: Yeah .
PhD A: I mean that if you look at the individual segments from just one person you don't see a lot of words ,
PhD H: Ju
Professor B: Yeah .
PhD A: but altogether you 'll see a lot of words up there .
Professor B: Yeah .
PhD F: Mm - hmm .
PhD D: Yeah .
PhD A: And so the reject is also mapping and pauses So I looked at them all in Waves and just lined up all the alignments , and , at first it sort of looked like a mess and then the more I looked at it , I thought " OK , well it 's moving these words leftward and " You know , it wasn't that bad . It was just doing certain things wrong . So But , I don't , you know , have time to l {comment} to look at all of them and it would be really useful to have , like , a a transcriber who could use Waves , um , just mark , like , the beginning and end of the foreground speaker 's real words like , the beginning of the first word , the end of the last word and then we could , you know , do some adjustments .
Postdoc C: Yeah . I OK . I have to ask you something , is i does it have to be Waves ? Because if we could benefit from what you did , incorporate that into the present transcripts , {comment} that would help .
PhD F: No .
Postdoc C: And then , um , the other thing is , I believe that I did hand So . One of these transcripts was gone over by a transcriber and then I hand - marked it myself so that we do have , uh , the beginning and ending of individual utterances . Um , I didn't do it word level ,
PhD F: Mm - hmm .
Postdoc C: but but in terms PhD A: Mm - hmm .
Postdoc C: So I so for for one of the N S A groups . And also I went back to the original one that I first transcribed and and did it w uh , w uh , utterance by utterance for that particular one . So I think you do have if that 's a sufficient unit , I think that you do have hand - marking for that . But it 'd be wonderful to be able to benefit from your Waves stuff .
PhD A: Mm - hmm .
PhD F: We don't care what what tool you use .
PhD A: Yeah . I mean , if if you can , um if you wanna Postdoc C: OK . I used it in Transcriber
PhD F: U uh Postdoc C: and it 's it 's in the PhD A: well , Jane and I were just in terms of the tool , talking about this . I guess Sue had had some reactions . You know , interface - wise if you 're looking at speech , you wanna be able to know really where the words are . And so , we can give you some examples of sort of what this output looks like ,
Postdoc C: Yeah , that 's right . Middle of the word , or PhD A: um , and see if you can in maybe incorporate it into the Transcriber tool some way , or Postdoc C: Well , I th I 'm thinking just ch e e incorporating it into the representation .
PhD A: Um .
Postdoc C: I mean , if it 's if it 's PhD A: You mean like Yeah , word start insights .
Postdoc C: if you have start points , if you have , like , time tags ,
PhD A: Right .
Postdoc C: which is what I assume . Isn't that what what you ? Well , see , Adam would be PhD F: Yeah , whatever you use .
PhD A: Yeah .
PhD F: I mean , we convert it to this format that the , um , NIST scoring tool unders uh , CTM . Conversation Time - Marked file . And and then that 's the that 's what the Grad E: I think Transcriber , uh , outputs CTM .
Postdoc C: If it ? OK .
PhD A: Yeah .
Postdoc C: So you would know this more than I would .
Grad E: I think so .
PhD A: So , I mean Postdoc C: It seems like she if she 's g if she 's moving time marks around ,
PhD F: Right .
Postdoc C: since our representation in Transcriber uses time marks , it seems like there should be some way of of using that benefitting from that .
Grad E: Right .
PhD A: Yeah , it wou the advantage would just be that when you brought up a bin you would be able if you were zoomed in enough in Transcriber to see all the words ,
Professor B: Mm - hmm .
PhD A: you would be able to , like , have the words sort of located in time , if you wanted to do that .
Professor B: So so if we e e even just had a a It sounds like w we  we almost do .
PhD A: So .
Professor B: Uh , if we We have two .
Postdoc C: We have two .
Professor B: Yeah . Just ha uh , trying out the alignment procedure that you have on that
PhD A: Mm - hmm .
Professor B: you could actually get something , um uh , uh , get an objective measure . Uh PhD F: Mm - hmm .
PhD A: You mean on on the hand - marked , um So we we only r hav I only looked at actually alignments from one meeting that we chose ,
Professor B: Yeah .
PhD A: I think MR four , just randomly , um And PhD F: Actually , not randomly .
PhD A: Not randomly PhD F: We knew we knew that it had these insertion errors from PhD A: It had sort of average recognition performance in a bunch of speakers
PhD F: Yeah . Yeah .
PhD A: and it was a Meeting Recorder meeting . Um . But , yeah , we should try to use what you have . I did re - run recognition on your new version of MR one .
Postdoc C: Oh , good .
PhD A: I I mean the the one with Dan Ellis in it and Eric .
Postdoc C: Good ! Uh - huh . Yeah , exactly . Yeah . Yeah .
Grad G: I don't think that was the new version .
PhD A: Um That Yeah , actually it wasn't the new new , it was the medium new .
Postdoc C: OK .
PhD A: But but we would we should do the the latest version .
Postdoc C: OK .
Grad G: Yeah .
PhD A: It was the one from last week .
Grad G: You did you adjust the the utterance times , um , for each channel ?
Postdoc C: Yes . Yes , I did . And furthermore , I found that there were a certain number where  not not a lot , but several times I actually moved an utterance from Adam 's channel to Dan 's or from Dan 's to Adam 's . So there was some speaker identif And the reason was because I transcribed that at a point before  uh , before we had the multiple audio available f so I couldn't switch between the audio . I I transcribed it off of the mixed channel entirely , which meant in overlaps , I was at a at a terrific disadvantage .
PhD A: Right . Right .
Postdoc C: In addition it was before the channelized , uh , possibility was there . And finally I did it using the speakers of my , um  of you know , off the CPU on my on my machine cuz I didn't have a headphone .
PhD A: Right .
Postdoc C: So it @ @ , like , I mean Yeah , I I mean , i in retrospect it would 've been good to ha have got I should 've gotten a headphone . But in any case , um , thi this is this was transcribed in a in a , uh , less optimal way than than the ones that came after it , and I was able to you know , an and this meant that there were some speaker identif identifications which were changes .
Grad G: Well , I know there were some speaker labelling problems , um , after interruptions .
Postdoc C: Yeah . Fixed that .
Grad G: Is that what you 're referring to ? I mean , cuz there 's this one instance when , for example , you 're running down the stairs .
Postdoc C: Oh , well Grad G: I remember this meeting really well .
PhD D: Yeah .
PhD A: Don Don has had  He knows he can just read it like a play .
Grad G: Right . It 's a Yeah , I 've I 've I 'm very well acquainted with this meeting .
PhD D: Yeah .
Grad G: Yeah , I can s
PhD A: " And then she said , and then he said . "
Grad G: Yeah , I know it by heart . So , um , there 's one point when you 're running down the stairs .
Postdoc C: Uh - oh .
Grad G: Right ? And , like , there 's an interruption . You interrupt somebody , but then there 's no line after that . For example , there 's no speaker identification after that line .
Postdoc C: Uh - huh .
Grad G: Is that what you 're talking about ? Or were there mislabellings as far as , like , the a Adam was ?
Postdoc C: That was fixed , um , before i i i I think I I think I understood that pretty Grad G: Yeah . Cuz I thought I let you know about that .
Postdoc C: Thank you for mentioning . Yeah , no , tha that That I think went away a couple of versions ago ,
Grad G: Yeah . OK .
Postdoc C: but it 's good to know .
Grad G: But you 're actually saying that certain , uh , speakers were mis mis - identified .
Postdoc C: Yeah . So , with under um , uh , listening to the mixed channel , there were times when , as surprising as that is , I got Adam 's voice confused with Dan 's and vice versa Grad G: OK .
Postdoc C: not for long utterances ,
Grad G: OK .
PhD A: Yeah .
Postdoc C: but jus just a couple of places ,
Professor B: Mm - hmm .
Postdoc C: and embedde embedded in overlaps . The other thing that was w interesting to me was that I picked up a lot of , um , backchannels which were hidden in the mixed signal ,
PhD A: Right .
Postdoc C: which , you know , I mean , you c not not too surprising . But the other thing that I I hadn't thought about this , but I thou I wanted to raise this when you were uh , with respect to also a strategy which might help with the alignments potentially , but that 's When I was looking at these backchannels , they were turning up usually  very often in w well , I won't say " usually " but anyway , very often , I picked them up in a channel w which was the person who had asked a question . S so , like , someone says " an and have you done the so - and - so ? " And then there would be backchannels , but it would be the person who asked the question . Other people weren't really doing much backchannelling . And , you know , sometimes you have the Yeah , uh - huh .
PhD A: Well , that 's interesting . Yeah .
Postdoc C: I mean , i it wouldn't be perfect , but but it does seem more natural to give a backchannel when when you 're somehow involved in the topic ,
PhD A: No , that 's really interesting .
Professor B: Mm - hmm .
Postdoc C: and the most natural way is for you to have initiated the topic by asking a question .
PhD F: Well ,
PhD A: That 's interesting .
PhD F: I think No . I think it 's actually I think what 's going on is backchannelling is something that happens in two - party conversations .
Postdoc C: Mm - hmm .
PhD F: And if you ask someone a question , you essentially initiating a little two - party conversation .
Postdoc C: Yeah .
PhD A: Well , actu Yeah , when we looked at this Postdoc C: Exactly .
PhD F: So then you 're so and then you 're expected to backchannel because the person is addressing you directly and not everybody .
Postdoc C: Exactly . Exactly my point . An - and so this is the expectation thing that uh , uh ,
PhD F: Yeah . Yeah .
PhD A: Mm - hmm .
PhD F: Right .
Postdoc C: just the dyadic PhD F: Right .
Postdoc C: But in addition , you know , if someone has done this analysis himself and isn't involved in the dyad , but they might also give backchannels to verify what what the answer is that this that the the answerer 's given Professor B: H
PhD A: Right .
Professor B: I tell you , I say I say " uh - huh " a lot ,
PhD A: It 's Postdoc C: There you go .
PhD A: Well , but it 's interesting cuz , uh Professor B: while people are talking to each other .
PhD A: But there are fewer I think there are fewer " uh - huhs " .
Postdoc C: There you go . Yeah . Yeah .
PhD A: I mean , just from We were looking at word frequency lists to try to find the cases that we would allow to have the reject words in between in doing the alignment . You know the ones we wouldn't constrain to be next to the other words .
Postdoc C: Oh , yeah .
PhD A: And " uh - huh " is not as frequent as it sort of would be in Switchboard , if you looked at just a word frequency list of one - word short utterances . And " yeah " is way up there , but not " uh - huh " . And so I was thinking thi it 's not like you 're being encouraged by everybody else to keep talking in the meeting . And uh , that 's all , I I 'll stop there , cuz I I think what you say makes a lot of sense .
Postdoc C: Well , that 's right . And that would PhD A: But it was sort of Postdoc C: Well , an And what you say is the is the re uh , o other side of this , which is that , you know , so th there are lots of channels where you don't have these backchannels , w when a question has been asked and and these PhD A: Right . There 's just probably less backchannelling in general ,
Postdoc C: Mm - hmm . So that 's good news , really .
PhD A: even if you consider every other person altogether one person in the meeting , but we 'll find out anyway . We were I guess the other thing we 're we 're I should say is that we 're gonna , um try compare this type of overlap analysis to Switchboard , where PhD F: And
PhD A: and CallHome , where we have both sides , so that we can try to answer this question of , you know , is there really more overlap in meetings or is it just because we don't have the other channel in Switchboard
Professor B: Mm - hmm .
Grad E: Mm - hmm .
PhD A: and we don't know what people are doing . Try to create a paper out of that .
Professor B: Yeah . I mean , y y you folks have probably already told me , but were were you intending to do a Eurospeech submission , or ?
PhD A: Um , you mean the one due tomorrow ?
Professor B: Yeah .
PhD A: Yeah . Well , we 're still , like , writing the scripts for doing the research , and we will Yes , we 're gonna try .
Postdoc C: Mm - hmm .
PhD A: And I was telling Don , do not take this as an example of how people should work .
Professor B: Do as I say ,
Grad G: That 's r
PhD A: So , {comment} we will try .
Professor B: don't do as I do . Yeah .
PhD A: It 'll probably be a little late ,
Grad E: Well PhD A: but I 'm gonna try it .
Grad E: It is different . In previous years , Eurospeech only had the abstract due by now , not the full paper .
PhD A: Right .
Grad G: Right .
Grad E: And so all our timing was off . I 've given up on trying to do digits . I just don't think that what I have so far makes a Eurospeech paper .
PhD A: Well , I 'm no We may be in the same position , and I figured we 'll try , because that 'll at least get us to the point where we have We have this really nice database format that Andreas and I were working out that It it 's not very fancy . It 's just a ASCII line by line format , but it does give you information PhD F: It 's the it 's the spurt format .
PhD A: It Yeah , we 're calling these " spurts " after Chafe . I was trying to find what 's a word for a continuous region with pauses around it ?
Postdoc C: Hmm .
Professor B: Yeah . I know that th the Telecom people use use " spurt " for that .
Postdoc C: Good .
PhD A: They do ? Oh !
Professor B: Yes .
PhD F: Oh .
PhD A: Oh .
Professor B: And that 's I mean , I I was using that for a while when I was doing the rate of speech stuff ,
PhD A: I would jus
Professor B: because I because I looked up in some books and I found OK , I wanna find a spurt in which PhD A: Ah , right ! It 's just , like , defined by the acoustics .
Professor B: and an because cuz it 's another question about how many pauses they put in between them .
Grad E: Horrible .
PhD A: Right .
Professor B: But how fast do they do the words within the spurt ?
PhD A: Right .
Professor B: Yeah .
PhD A: Well , that 's what we were calling spurt ,
Grad E: It 's gonna Grad G: you know " Burst " also ?
Grad E: Burst .
Grad G: Isn't " burst " is used also ?
PhD A: so Grad E: Spurt has the horrible name overloading with other with hardware at ICSI .
Professor B: Here . Just very locally , yeah .
PhD A: Well , well , Chafe had this wor I think it was Chafe , or somebody had a the word " spurt " originally ,
Professor B: But but that just PhD H: Here @ @ PhD A: and so I But tha that 's good to know .
Postdoc C: Actually PhD A: Was thi it 's Chafe ?
Postdoc C: Well , see , I know S Sue wrote about spurts of development .
PhD F: So maybe we should talk PhD A: Maybe it was Sue ? Y
Postdoc C: But , in any case , I think it 's a good term ,
PhD A: So we have spurts and we have spurt - ify dot shell and spurt - ify
Professor B: Yeah .
Postdoc C: and , uh Grad E: Hmm !
Professor B: Yeah .
Postdoc C: And ma maybe maybe Chafe did .
PhD F: Uh .
PhD A: And then it 's got all it 's a verb now .
Postdoc C: I know I know Ch - Chafe dealt with PhD F: So s
Grad G: That 's cool .
PhD F: W uh , w
Postdoc C: Chafe speaks about intonation units .
PhD A: Yes . Right .
Postdoc C: But maybe he speaks about spurts as well
PhD F: We
Postdoc C: and I just don't know . Yeah , go ahead .
Grad E: I 've heard " burst " also .
PhD F: So what we 're doing uh , this this is just maybe someone has s some some ideas about how to do it better ,
Grad G: Mmm .
PhD F: but we So we 're taking these , uh , alignments from the individual channels . We 're from each alignment we 're producing , uh , one of these CTM files ,
Postdoc C: Great .
PhD F: which essentially has it 's just a linear sequence of words with the begin times for every word and the duration .
PhD A: It looks like a Waves label file almost . Right ?
PhD F: And and and of course PhD A: It 's just PhD F: Right . But it has one the first column has the meeting name , so it could actually contain several meetings . Um . And the second column is the channel . Third column is the , um , start times of the words and the fourth column is the duration of the words . And then we 're , um OK . Then we have a messy alignment process where we actually insert into the sequence of words the , uh , tags for , like , where where sentence ends of sentence , question marks , um , various other things .
PhD A: Yeah . These are things that we had Don PhD F: Uh .
PhD A: So , Don sort of , um , propagated the punctuation from the original transcriber PhD F: Right .
PhD A: so whether it was , like , question mark or period or , um , you know , comma and things like that , and we kept the and disfluency dashes uh , kept those in because we sort of wanna know where those are relative to the spurt overlaps PhD F: Mm - hmm . Right .
PhD A: sp overlaps ,
PhD F: So so those are actually sort of retro - fitted into the time alignment .
PhD A: or PhD F: And then we merge all the alignments from the various channels and we sort them by time . And then there 's a then there 's a process where you now determine the spurts . That is Actually , no , you do that before you merge the various channels . So you you id identify by some criterion , which is pause length you identify the beginnings and ends of these spurts , and you put another set of tags in there to keep those straight .
Professor B: Mm - hmm .
PhD F: And then you merge everything in terms of , you know , linearizing the sequence based on the time marks . And then you extract the individual channels again , but this time you know where the other people start and end talking you know , where their spurts start and end . And so you extract the individual channels , uh , one sp spurt by spurt as it were . Um , and inside the words or between the words you now have begin and end tags for overlaps . So , you you basically have everything sort of lined up and in a form where you can look at the individual speakers and how their speech relates to the other speakers ' speech .
Grad E: Right .
PhD A: Uh , I mean , I think that 's actually really u useful also
PhD F: And PhD A: because even if you weren't studying overlaps , if you wanna get a transcription for the far - field mikes , how are you gonna know which words from which speakers occurred at which times relative to each other ? You have to be able to get a transcript like like this anyway , just for doing far - field recognition . So , you know , it 's it 's sort of PhD F: Yeah .
PhD A: I thi it 's just an issue we haven't dealt with before , how you time - align things that are overlapping anyway .
Postdoc C: That 's wonderful .
PhD F: So PhD A: I mean , i I never thought about it before ,
Grad E: Well PhD F: And and we PhD A: but Grad E: Y yes .
PhD F: In Grad E: I mean , s when I came up with the original data suggested data format based on the transcription graph , there 's capability of doing that sort of thing in there .
PhD A: Right . But you can't get it directly from the transcription .
Postdoc C: Mm - hmm . Yeah , that 's right .
PhD F: Right . Well , this is this is just PhD A: Yeah , this is like a poor man 's ver formatting version . But it 's , you know It 's clean , it 's just not fancy .
Grad E: Right .
PhD A: Um .
PhD F: Well , there 's lots of little things . It 's like there 're twelve different scripts which you run and then at the end you have what you want . But , um , at the very last stage we throw away the actual time information . All we care about is whether that there 's a certain word was overlapped by someone else 's word . So you sort of at that point , you discretize things into just having overlap or no overlap . Because we figure that 's about the level of analysis that we want to do for this paper .
Grad E: Mm - hmm .
PhD F: But if you wanted to do a more fine - grained analysis and say , you know , how far into the word is the overlap , you could do that .
PhD A: Yeah .
PhD F: It 's just it 'll just require more PhD A: Just sort of huge .
PhD F: you know , slightly different Postdoc C: What 's interesting is it 's exactly what , um , i in discussing with , um , Sue about this ,
PhD A: Yeah .
Postdoc C: um , she , um , i i i indicated that that you know , that 's very important for overlap analysis .
PhD A: Yeah . It 's it 's nice to know ,
PhD F: Right .
PhD A: and also I think as a human , like , I don't always hear these in the actual order that they occur . So I can have two foreground speakers , you know , Morgan an and um , Adam and Jane could all be talking , and I could align each of them to be starting their utterance at the correct time , and then look where they are relative to each other , and that 's not really what I heard .
Postdoc C: And that 's another thing she said .
PhD A: Cuz it 's just hard to do .
Postdoc C: This is This is Bever 's Bever 's effect ,
PhD A: Y Yeah .
Postdoc C: when where In psy ps psycho - linguistics you have these experiments where people have perceptual biases a as to what they hear ,
PhD A: It 's sort of Yeah , you sort of move things around until you get to a low information point
Postdoc C: that that Not the best PhD A: and yo then you can bring in the other person . So it 's actually not even possible , I think , for any person to listen to a mixed signal , even equalize , and make sure that they have all the words in the right order . So , I guess , we 'll try to write this Eurospeech paper .
Postdoc C: Mm - hmm . Superb .
PhD A: I mean , we will write it . Whether they accept it late or not , I don't know . Um , and the good thing is that we have It 's sort of a beginning of what Don can use to link the prosodic features from each file to each other .
PhD F: Yeah .
Professor B: Yeah . That 's the good thing about these pape
PhD A: So . i You know , might as well .
PhD F: Plus , mayb
PhD H: Hmm ?
PhD A: We - I ju Otherwise we won't get the work done {comment} on our deadline .
PhD F: I don't know , m
Professor B: Yeah .
PhD F: I mean , u u Jane likes to look at data . Maybe , you know , you could you could look at this format and see if you find anything interesting .
Professor B: Yeah .
PhD F: I don't know .
PhD A: Yeah .
Professor B: No , it 's that 's the good thing about these pape paper deadlines and , uh , you know , class projects , and  and things like that ,
Postdoc C: Well , what I 'm thinking is PhD F: Yeah .
Postdoc C: Yeah .
PhD A: Right .
PhD F: Mm - hmm .
Postdoc C: Well , my PhD F: Well th th the other thing that that that yo that you usually don't tell your graduate students is that these deadlines are actually not that , um , you know , strictly enforced ,
Professor B: because you you really get g
PhD A: Forces you to do the work .
Postdoc C: Yeah .
Professor B: Yeah .
PhD A: Exactly .
Grad E: Strict .
PhD F: because the Professor B: Oh , now it 's out in the public , this this this secret information .
PhD F: because PhD A: Right .
Professor B: Yeah .
Postdoc C: I think we can ha
PhD F: bec b Nah PhD A: So Grad E: No .
Professor B: No .
Postdoc C: Nah .
PhD F: i Because these the conference organizers actually have an interest in getting lots of submissions .
PhD A: Right .
Grad E: Right .
PhD F: I mean , a a monetary interest .
Professor B: Yeah .
PhD F: So  Um .
Professor B: Th - that 's that 's true .
Postdoc C: And good ones , good ones , which sometimes means a little extra time .
PhD F: And good submission
Professor B: That 's PhD F: Right .
Professor B: That 's true .
PhD F: Well That 's another issue ,
Professor B: By th by the way , this is totally unfair , you may you may feel ,
PhD F: but Professor B: but the the , uh the morning meeting folks actually have an an extra month or so .
PhD F: Mm - hmm .
PhD D: Yep .
Grad E: Yep . The Aurora there 's a special Aurora PhD A: Uh PhD F: When Professor B: There 's a special Aurora session
PhD A: Oh .
Professor B: and the Aurora pe people involved in Aurora have till Ma - uh , early May or something to turn in their paper .
PhD F: Mmm .
PhD A: Oh .
PhD F: Mmm .
PhD A: Oh , well maybe we 'll submit to s {comment} Actually PhD F: Well , then you can just Maybe you can submit the digits paper on e for the Aurora session .
PhD H: Yeah .
PhD A: Yeah .
PhD D: Yeah .
Grad E: Oh , I could !
PhD A: Yeah .
Professor B: I if it w
Grad E: I could submit that to Aurora .
Professor B: Well Grad E: That would be pretty pretty PhD F: Yeah .
Professor B: i it has PhD A: Yeah .
Professor B :
Grad E: S That wouldn't work .
Professor B: No , it wouldn't work .
Grad E: It 's not Aurora .
Professor B: It 's it 's not the Aurora I mean , it it 's it 's actually the Aurora task .
PhD A: Maybe they 'll get s
Grad E: Aurora 's very specific .
Professor B: It
PhD A: Well , maybe it won't be after this deadline extension .
PhD F: But but the people I mean , a a paper that is not on Aurora would probably be more interesting at that point
PhD A: Maybe they 'll PhD F: because everybody 's so sick and tired of the Aurora task .
PhD D: Yeah .
Grad E: Oh , I thought you meant this was just the digits section . I didn't know you meant it was Aurora digits .
Professor B: Yeah .
PhD F: Well , no . If you if you have it 's to if you discuss some relation to the Aurora task , like if you use the same Professor B: This is not the Aurora task . So they just do a little grep for PhD A: Do uh , d d Do not do not we are not setting a good example .
PhD F: Um . Well , a relation other than negation , maybe ,
PhD A: This is not a PhD F: um . So .
PhD A: Anyway .
PhD F: I don't know .
PhD A: But the good thing is this does Grad E: Well , I I don't know . I mean , you could you could do a paper on what 's wrong with the Aurora task by comparing it to other ways of doing it .
PhD F: How well does an Aurora system do on on you know , on digits collected in a in this environment ?
PhD H :
Grad E: Different way . Yeah .
PhD F: Yeah .
Professor B: Maybe .
PhD F: Maybe .
Grad E: Pretty hokey .
Professor B: I think it 's a littl little far - fetched . Nah , I mean , the thing is Aurora 's pretty closed community .
Grad E: Yep .
Professor B: I mean , you know , the people who were involved in the  the only people who are allowed to test on that are people who who made it above a certain threshold in the first round ,
PhD F: Mm - hmm .
Grad E: It 's very specific .
Professor B: uh w in ninety - nine and it 's it 's sort of a it 's not like a PhD F: Well , that 's maybe why they don't f know that they have a crummy system . I mean , a crummy back - end . No , I mean I mean , seriously , if you if you have a very No , I 'm sorry .
PhD A: Uh , {comment} " beep " " bee "
Grad E: I mean , th
PhD F: No . I didn't mean anybody any particular system . I meant this H T K back - end .
Professor B: Oh , you don't like HTK ?
PhD F: If they PhD H: Yeah .
PhD F: I don't h I don't have any stock in HTK or Entropic or anything .
Professor B: No . I mean , this it it 's the HTK that is trained on a very limited amount of data .
Grad E: It 's d it 's very specific .
PhD F: Right .
Professor B: Yeah .
PhD F: But so , if you But maybe you should , you know , consider more using more data , or I mean Professor B: Oh , yeah . I I really think that that 's true . And they i i
PhD F: If yo if you sort of hermetically stay within one task and don't look left and right , then you 're gonna Grad E: But they they had Professor B: i But Grad E: They had something very specific in mind when they designed it . Right ?
Professor B: Well , u i
PhD F: Right .
Grad E: And so so you can you can argue about maybe that wasn't the right thing to do , but , you know , they they they had something specific .
Professor B: But , one of the reasons I have Chuck 's messing around with with the back - end that you 're not supposed to touch I mean , for the evaluations , yes , we 'll run a version that hasn't been touched .
PhD F: Mm - hmm . Mm - hmm .
Professor B: But , uh , one of the reasons I have him messing around with that , because I think it 's sort of an open question that we don't know the answer to . People always say very glibly that i if you s show improvement on a bad system , that doesn't mean anything , cuz it may not be  show uh , because , you know , it doesn't tell you anything about the good system .
PhD F: Mm - hmm .
Professor B: And I I 've always sort of felt that that depends . You know , that if some peopl If you 're actually are getting at something that has some conceptual substance to it , it will port .
PhD F: Mm - hmm .
Professor B: And in fact , most methods that people now use were originally tried with something that was not their absolute best system at some level . But of course , sometimes it doesn't , uh , port . So I think that 's that 's an interesting question . If we 're getting three percent error on , uh , u uh , English , uh , nati native speakers , um , using the Aurora system , and we do some improvements and bring it from three to two , do those same improvements bring , uh , th you know , the SRI system from one point three to you know , to point eight ?
PhD F: Hmm . Mm - hmm .
Grad E: Zero .
Professor B: Well . You know , so that 's that 's something we can test .
PhD F: Mmm . Right .
Professor B: So . Anyway .
PhD F: OK .
Professor B: I think we 've  we 've covered that one up extremely well .
Postdoc C: Mm - hmm .
PhD F: Whew !
Professor B: OK . So , um Yeah . So tha so we 'll you know , maybe you guys 'll have have one . Uh , you you and , uh and Dan have have a paper that that 's going in .
PhD D: Yeah .
Professor B: You know , that 's that 's pretty solid , on the segmentation stuff .
PhD D: Yeah . Yeah . I will send you the the final version ,