professor f: We can say the word " zero " all we want , phd g: I 'm doing some professor f: but just phd g: square brackets , coffee sipping , square brackets . phd d: There 's gonna be some zeros from this morning 's meeting because I noticed that professor f: u phd d: Barry , I think maybe you turned your mike off before the digits were Oh , was it during digits ? Oh , so it doesn't matter . phd b: So it 's not it 's not that bad if it 's at the end , but it 's in the beginning , it 's bad . grad a: Yeah , you wanna you wanna keep them on so you get good noise noise floors , through the whole meeting . Yeah I did have to run , but grad e: Is there any way to change that in the software ? grad a: Change what in the software ? grad e: Where like you just don't like if you if it starts catching zeros , like in the driver or something in the card , or somewhere in the hardware Where if you start seeing zeros on w across one channel , you just add some random , @ @ noise floor like a small noise floor . grad a:  certainly we could do that , but I don't think that 's a good idea . professor f: Well , I u I actually don't know what the default is anymore as to how we 're using the the front - end stuff but for for when we use the ICSI front - end , grad a: As an argument . professor f: but  , there is an there is an o an option in in RASTA , which , in when I first put it in , back in the days when I actually wrote things , I did actually put in a random bit or so that was in it , grad e: OK . professor f: but then I realized that putting in a random bit was equivalent to adding  adding flat spectrum , grad e: Right . Gee ! Here we all are ! grad a: so the only agenda items were Jane was Jane wanted to talk about some of the IBM transcription process . professor f: There 's an agenda ? grad a: I sort of condensed the three things you said into that . And then just I only have like , this afternoon and maybe tomorrow morning to get anything done before I go to Japan for ten days . So if there 's anything that n absolutely , desperately needs to be done , you should let me know now . professor f: Yeah , I guess you first you have to do the first one , grad a: Yeah . professor f: Oh they they had some extension that they announced or something ? phd g: Well yeah . Liz had sent them a note saying " could we please have another " I don't know , " three days " or something , and they said yes . phd d: And then she said " Did I say three ? grad a: Oh , phd d: I meant four . " grad a: that was the other thing  , phd g: But u grad a: Dave Gelbart sent me email , I think he sent it to you too , that  , there 's a special topic , section in si in Eurospeech on new , corp corpors corpora . grad e: Huh ! grad a: And  , professor f: Oh ! phd b: I got this mail from grad a: I s forwarded it to Jane as I thought being the most relevant person .  So , I thought it was highly relevant postdoc c: Yeah I 'm professor f: That 's grad a: have you did you look at the URL ? postdoc c: Yeah . phd b: Was this SmartKom message ? I think Christoph Draxler sent this , postdoc c: Yeah . grad a: but obviously I can't , really do , most of it , postdoc c: Yeah . For instance that Morgan  , accounted for fifty - six percent of the Robustness meetings in terms of number of words . postdoc c: because is it partly , eh , c correctly identified words ? Or is it or just overall volume ? phd g: No . I think it 's he 's he 's in all of them , postdoc c: Oh . phd g: we didn't mention Morgan by name grad a: and he talks a lot . professor f: Well we have now , but phd g: We we we something about grad a: Did you identify him as a senior member ? phd g: No , we as identify him as the person dominating the conversation . professor f:  I get these AARP things , but I 'm not se really senior yet , but phd g: Right professor f: phd g:  . professor f: but  , other than that delightful result , what was the rest of the paper about ? phd g: well it was about it had three sections professor f: You sent it to me but I haven't seen it yet .  , the one was that the just the the amount of overlap grad a: The good , the bad , and the ugly . phd g: s in terms of in terms of number of words and also we computed something called a " spurt " , which is essentially a stretch of speech with  , no pauses exceeding five hundred milliseconds .  , and we computed how many overlapped i  spurts there were and how many overlapped words there were .  , for four different corpora , the Meeting Recorder meetings , the Robustness meetings Switchboard and CallHome , and , found and sort of compared the numbers .  , and found that the , you know , as you might expect the Meeting Recorder meetings had the most overlap  , but next were Switchboard and CallHome , which both had roughly the same , almost identical in fact , and the Robustness meetings were had the least , so One sort of unexpected result there is that  two - party telephone conversations have about the same amount of overlap , grad a: I 'm surprised . phd g: sort of in gen you know order of magnitude - wise as ,  as face - to - face meetings with multiple grad a: I have I had better start changing all my slides ! phd g: Yeah . Also , I in the Levinson , the pragmatics book , in you know , textbook , there 's I found this great quote where he says you know you know , how people it talks about how  how how people are so good at turn taking , postdoc c:   phd g: and so they 're so good that generally , u the overlapped speech does not is less than five percent . grad e: Did he mean face like face - to - face ? Or ? phd g: Well , in real conversations , grad e:  . phd b: But postdoc c: Well , of course , no , it doesn't necessarily go against what he said , cuz he said " generally speaking " . phd b: And in f phd g: Well , he he made a claim grad a: Well phd g: Well grad a:  phd b: But professor f: Yeah , we we have pretty limited sample here . phd b: Five percent of time or five percent of what ? grad a: Yeah , I was gonna ask that too . phd b: Yeah , so postdoc c: It 's i it 's not against his conclusion , phd g: So but still but still u postdoc c: it just says that it 's a bi bell curve , and that , you have something that has a nice range , in your sampling . So there are slight There are differences in how you measure it , but still it 's You know , the difference between  between that number and what we have in meetings , which is more like , you know , close to in meetings like these ,  you know , close to twenty percent . professor f: But what was it like , say , in the Robustness meeting , for instance ? phd g: That grad a: But phd g: Robustness meeting ? It was about half of the r So , in terms of number of words , it 's like seventeen or eigh eighteen percent for the Meeting Recorder meetings and about half that for , the Robustness . professor f: Maybe ten percent ? grad a: But I don't know if that 's really a fair way of comparing between , multi - party , conversations and two - party conversations . phd b: Then then then you have to grad a:  that 's just something phd d: Yeah , I just wonder if you have to normalize by the numbers of speakers or something . phd b: Then Yeah , then normalize by by something like that , postdoc c: Yeah , that 's a good point . phd g: but this obvious thing to see if if there 's a dependence on the number of  participants . grad a: You have a lot of a lot of two - party , subsets within the meeting . phd g: And and and then and we also d computed this both with and without backchannels , postdoc c:   phd g: so you might think that backchannels have a special status because they 're essentially just grad a:  - huh . So , did we all said "  - huh " and nodded at the same time , phd g: R right . phd g: But , even if you take out all the backchannels so basically you treat backchannels l as nonspeech , as pauses , grad a:   You know , it goes down from maybe For Switchboard it goes down from I don't know f  I don't know f fourteen percent of the words to maybe  I don't know , eleven percent or something it 's it 's not a dramatic change , grad a:   phd g: so it 's Anyway , so it 's  That was that was one set of results , and then the second one was just basically the the stuff we had in the in the HLT paper on how overlaps effect the recognition performance .  , but mostly we added one one number , which was what if you  , basically score ignoring all So so the the conjecture from the HLT results was that most of the added recognition error is from insertions due to background speech . So , we scored all the recognition results , in such a way that the  grad a: Oh by the way , who 's on channel four ? You 're getting a lot of breath . phd g: OK , so so if you have the foreground speaker speaking here , and then there 's some background speech , may be overlapping it somehow , and this is the time bin that we used , then of course you 're gonna get insertion errors here and here . phd g: Right ? So we scored everything , and I must say the NIST scoring tools are pretty nice for this , where you just basically ignore everything outside of the , region that was deemed to be foreground speech . And where that was we had to use the t forced alignment , results from s for so That 's somewhat that 's somewhat subject to error , but still we we  , Don did some ha hand - checking and and we think that based on that , we think that the results are you know , valid , although of course , some error is gonna be in there . But basically what we found is after we take out these regions so we only score the regions that were certified as foreground speech , the recognition error went down to almost  , the level of the non - overlapped speech . So that means that even if you do have background speech , if you can somehow separate out or find where it is , the recognizer does a good job , grad a: That 's great . phd g: even though there is this back grad a: Yeah , I guess that doesn't surprise me , because , with the close - talking mikes , the the signal will be so much stronger .  , grad a: What what sort of normalization do you do ? phd g: so  , well , we just @ @ we do u you know , vit grad a:  in you recognizer , in the SRI recognizer . phd g: Well , we do  , VTL vocal tract length normalization , w and we  you know , we we  , make all the features have zero mean and unit variance . grad a: Over an entire utterance ? professor f: And grad a: Or windowed ? phd g: Over over the entire c over the entire channel . We just took the old So this is actually a sub - optimal way of doing it , grad a: Right . So the recognizer didn't have the benefit of knowing where the foreground speech a start professor f: Were you including the the lapel in this ? phd g: Yes . professor f: And did the did did the la did the the problems with the lapel go away also ? Or phd g: it Yeah . professor f: fray for for insertions ? phd g: It u not per  , not completely , but yes , professor f: Less so . So we have to  professor f: you still phd g: Well I should bring the should bring the table with results . professor f: I would presume that you still would have somewhat higher error with the lapel for insertions than phd g: Yes . professor f: Cuz again , looking forward to the non - close miked case , I think that we s still phd g:   And then , the third thing was , we looked at , what we call " interrupts " , although that 's that may be a misnomer , but basically we looked at cases where  , so we we used the punctuation from the original transcripts and we inferred the beginnings and ends of sentences . So , you know postdoc c: Di - did you use upper - lower case also , or not ? phd g:  postdoc c: U upper lower case or no ? phd g:  ? postdoc c: OK . phd g: No , we only used , you know ,  periods , question marks and exclamation . And we know that there 's th that 's not a very g  , we miss a lot of them , postdoc c: Yeah . That 's OK but phd g: but but it 's f i i postdoc c: Comma also or not ? phd g: No commas . And then we looked at locations where , if you have overlapping speech and someone else starts a sentence , you know , where do these where do other people start their turns not turns really , but you know , sentences , phd b: Ah . phd g:  So we only looked at cases where there was a foreground speaker and then at the to at the so the the foreground speaker started into their sentence and then someone else started later . phd b: Somewhere in between the start and the end ? phd g: OK ? And so what phd b: OK . phd g: Sorry ? phd b: Somewhere in between the start and the end of the foreground ? phd g: Yes . phd g: So , the the question was how can we what can we say about the places where the second or or actually , several second speakers ,  start their " interrupts " , as we call them . phd g: w And we looked at this in terms of  grad a: On T - closures , only . phd g: So so we had we had  u to for for the purposes of this analysis , we tagged the word sequences , and and we time - aligned them .  , and we considered it interrupt if it occurred in the middle of a word , we basically you know , considered that to be a interrupt as if it were at at the beginning of the word . phd g: And then we looked at the the locatio the , you know , the features that the tags because we had tagged these word strings , that that occurred right before these these  , interrupt locations . phd b: Tag by  phd g: And the tags we looked at are the spurt tag , which basically says or actually Sorry . So whether there was a pause essentially here , because spurts are a defined as being you know , five hundred milliseconds or longer pauses , and then we had things like discourse markers , backchannels , disfluencies .  , filled pauses So disfluen the D 's are for , the interruption points of a disfluency , so , where you hesitate , or where you start the repair there .  , repeated you know , repeated words is another of that kind of disfluencies and so forth . So we had both the beginnings and ends of these  so , the end of a filled pause and the end of a discourse marker . We just looked at the distribution of words , and so every " so yeah " , and " OK " , and "  - huh " were were the were deemed to be backchannels and " wow " and " so " and  " right " ,  were  Not " right " . But so , we sort of just based on the lexical  , identity of the words , we we tagged them as one of these things . So , and then we looked at the disti so we looked at the distribution of these different kinds of tags , overall  , and and and particularly at the interruption points . And  , we found that there is a marked difference so that for instance after so at the end after a discourse marker or after backchannel or after filled pause , you 're much more likely to be interrupted than before . So pauses are always an opportunity for So we have this little histogram which shows these distributions and , phd d: I wonder phd g: you know , it 's it 's it 's not No big surprises , but it is sort of interesting from grad a: It 's nice to actually measure it though . In other words  if you weren't going to pause you you will because you 're g being interrupted . professor f: But he yeah , he 's he 's right , y  maybe you weren't intending to pause at all , but You were intending to stop for fifty - seven milliseconds , phd g: Right . And and we so we wrote this and then , we found we were at six pages , and then we started cutting furiously phd b: Oops . phd g: and threw out half of the material again , and  played with the LaTeX stuff and grad a: Made the font smaller and the narrows longer . phd g: but we s we put Oh , I I grad a: Took out white space . phd g: you know the the gap between the two columns is like ten millimeters , phd b: Yeah . phd d: Wasn't there wasn't there some result , Andreas professor f: Yeah phd d: I I thought maybe Liz presented this at some conference a while ago about  , backchannels phd g:   Well phd d:  - i i do you rem phd g: y We didn't talk about , prosodic , properties at all , phd d: Right . But phd g: although that 's I I take it that 's something that  Don will will look at grad e: Yeah , we 're gonna be looking at that . postdoc c: Well , I didn't know about Liz 's finding on that , phd d: About postdoc c: but I know of another paper that talks about something phd d:  - huh . phd d: It made me think about a cool little device that could be built to  to handle those people that call you on the phone and just like to talk and talk and talk . And you just have this little detector that listens for these drops in pitch and gives them the backchannel . phd g: There 's actually  there 's this a former student of here from Berkeley , Nigel Nigel Ward . phd g: He did a system  , in he he lives in Japan now , and he did this backchanneling , automatic backchanneling system . phd g: It 's a very phd d: Oh ! phd g: So , exactly what you describe , phd d: Huh . And it 's apparently for Japa - in Japanese it 's really important that you backchannel . Actually for a lot of these people I think you could just sort of backchannel continuously and it would pretty much be fine . Where the barber who was afraid of scissors was playing a a tape of clipping sounds , and saying "  - huh " , " yeah " , " how about them sports teams ? " phd g: Anyway . So the paper 's on - line and y I I think I  I CC ' ed a message to Meeting Recorder with the URL so you can get it . So I I 'm actually about to send Brian Kingbury an email saying where he can find the the s the m the material he wanted for the s for the speech recognition experiment , so but I haven't sent it out yet because actually my desktop locked up , like I can't type anything .  b so if there 's any suggestions you have for that I was just gonna send him the phd d: Is it the same directory that you had suggested ? phd g: I made a directory . phd g: He does ? postdoc c: And he and he 's phd g: Yeah but but but he has to postdoc c: I 'd hafta add him to Meeting Recorder , I guess , phd g: he prefe he said he would prefer FTP postdoc c: but OK . phd g: and also , the other person that wants it There is one person at SRI who wants to look at the  , you know , the  the data we have so far , postdoc c: OK . So what I did is I  @ @ I made a n new directory after Chuck said that would c that was gonna be a good thing . Right ? The same the same as the mailing list , professor f: Yeah , phd g: and professor f: the No vowels .  , professor f: Yeah phd g: and then under there  actually Oh and this directory , is not readable . So , in other words , to access anything under there , you have to be told what the name is . phd g: So  , and the directory for this I call it I " ASR zero point one " because it 's sort of meant for recognition . professor f: So anyone who hears this meeting now knows the grad a: Beta ? phd g: And then then in there I have a file that lists all the other files , so that someone can get that file and then know the file names and therefore download them . If you don't know the file names you can't professor f: Is that a dash or a dot in there ? phd g:  you can grad a: Don't don't don't say . So all I all I was gonna do there was stick the the transcripts after we the way that we munged them for scoring , because that 's what he cares about , and  , and also and then the the waveforms that Don segmented .  , just basically tar them all up f  w for each meeting I tar them all into one tar file and G - zip them and stick them there . grad a: I  , put digits in my own home directory home FTP directory , phd g: And so . phd d: So we could point Mari to this also for her March O - one request ? phd g: OK . phd d: Or phd g: Oh ! phd d: You n Remember she was phd g: Oh she wanted that also ? phd d: Well she was saying that it would be nice if we had they had a Or was she talking Yeah . She was saying it would be nice if they had eh the same set , so that when they did experiments they could compare . grad e:  I phd g: But yeah , we can send I can CC Mari on this so that she knows phd d: Yeah . phd d: so I was gonna probably put it grad a: We can put it in the same place .  , so either we should regenerate the original versions , or  , we should just make a note of it . And so I but OK so but for the other meetings it 's the downsampled version that you have . Oh that 's th important to know , OK so we should probably  give them the non - downsampled versions . Alright , then I 'll hold off on that and I 'll wait for you  grad e: Probably by tomorrow phd g: gen grad e: I can I 'll send you an email . Yeah , definitely they should have the full bandwidth version , grad e: Yeah , because  I I think Liz decided to go ahead with the downsampled versions cuz we can There was no s like , r significant difference . grad e: It does take up less disk space , and apparently it did even better than the original than the original versions , phd g: Yeah . Good that Well , it 's a good thing that grad a: OK , I think we 're losing , Don and Andreas at three - thirty , right ? OK . professor f: So , that 's why it was good to have Andreas , say these things but So , we should probably talk about the IBM transcription process stuff that postdoc c: OK . So ,  you know that Adam created  , a b a script to generate the beep file ? professor f:  . But but you were gonna to use the originally transcribed file because I tightened the time bins and that 's also the one that they had already in trying to debug the first stage of this . And  , my understanding was that ,  I haven't I haven't listened to it yet , grad a:   postdoc c: but it sounded very good and and I understand that you guys were going to have a meeting today , before this meeting .  , just so that while I 'm gone , you can regenerate it if you decide to do it a different way . So  , Chuck and Thilo should , now more or less know how to generate the file postdoc c: Excellent . grad a: and , the other thing Chuck pointed out is that , since this one is hand - marked , there are discourse boundaries . So what what we 're probably gonna do is just write a script , that if two , chunks are very close to each other on the same channel we 'll just merge them . grad a: So , and that will get around the problem of , the , you know " one word beep , one word beep , one word beep , one word beep " . phd d: Yeah , in fact after our meeting  , this morning Thilo came in and said that  , there could be other differences between the  already transcribed meeting with the beeps in it and one that has just r been run through his process . phd d: So tomorrow , when we go to make the   , chunked file for IBM , we 're going to actually compare the two . So he 's gonna run his process on that same meeting , postdoc c: Great idea ! phd d: and then we 're gonna do the beep - ify on both , and listen to them and see if we notice any real differences . phd g: Beep - ify ! postdoc c: OK , now one thing that prevented us from apply you you from applying Exactly . Wel -  we just wanna if if there 're any major differences between doing it on the hand postdoc c:  - huh . phd g: So this training meeting ,  w un is that  some data where we have  very  , you know , accurate time marks ? for postdoc c: I went back and hand - marked the ba the bins , I ment I mentioned that last week . phd d: But the but there 's yeah , but there is this one issue with them in that there 're there are time boundaries in there that occur in the middle of speech . phd g: Because phd d: So Like when we went t to  When I was listening to the original file that Adam had , it 's like you you hear a word then you hear a beep and then you hear the continuation of what is the same sentence . grad a: It 's i phd d: So there are these chunks that look like  that have  grad a:  that 's not gonna be true of the foreground speaker . So you 'll you 'll have a chunk of , you know , channel A which starts at zero and ends at ten , and then the same channel starting at eleven , ending at fifteen , and then again , starting at sixteen , ending at twenty . Right , so that 's three chunks where actually we w can just make one chunk out of that which is A , zero , twenty . postdoc c: Yeah , I thought that was phd d: So if you were to use these , you have to be careful not to pull out these individual postdoc c: Yeah . phd g: Oh !  it Right ,  w  what I would I was interested in is having a se having time marks for the beginnings and ends of speech by each speaker . phd g: So  , it I don't care that you know , there 's actually abutting segments that we have to join together . phd g: But what we do care about is that the beginnings and ends  are actually close to the speech inside of that phd d: Yeah , I think Jane tightened these up by hand . phd g: OK , so what is the sort of how tight are they ? professor f: it looks much better . I just wanted to get it so tha So that if you have like " yeah " in a swimming in a big bin , then it 's phd g: No , no ! I don grad a: Let me make a note on yours . phd g: I it 's f That 's fine because we don't want to th that 's perfectly fine . You always want to have a little bit of pause or nonspeech around the speech , say for recognition purposes .  , but just just u w you know get an id I just wanted to have an idea of the of how much extra you allowed  so that I can interpret the numbers if I compared that with a forced alignment segmentation . postdoc c: but but my main goal was  , in these areas where you have a three - way overlap and one of the overlaps involves " yeah " , and it 's swimming in this huge bin , I wanted to get it so that it was clo more closely localized . But are we talking about , I don't know , a tenth of a second ? a ? You know ? How how much how much extra would you allow at most postdoc c: I I wanted to I wanted it to be able to l he be heard normally , phd g:   postdoc c: so that if you if you play back that bin and have it in the mode where it stops at the boundary , it sounds like a normal word . postdoc c: Now sometimes you know , it 's these are involved in places where there was no time . postdoc c:  some cases , there 're some people  , who who have very long segments of discourse where , you know , they 'll they 'll breath and then I put a break . postdoc c: But other than that , it 's really pretty continuous and this includes things like going from one sentence into the u one utterance into the next , one sentence into the next , w without really stopping .  i they , i you know in writing you have this two spaces and a big gap phd g:   postdoc c: But but  i some people are planning and , you know , a lot we always are planning what we 're going to say next . postdoc c: But  , in which case , the gap between these two complete syntactic units , which of course n spoken things are not always complete syntactically , but but it would be a shorter p shorter break than maybe you might like . postdoc c: But the goal there was to not have the text be so so crudely parsed in a time bin .  , because from a discourse m purpose it 's it 's more it 's more useful to be able to see and also you know , from a speech recognition purpose my impression is that if you have too long a unit , it 's it doesn't help you very much either , cuz of the memory . postdoc c: So , that means that the amount of time after something is variable depending partly on context , but my general goal when there was sufficient space , room , pause after it to have it be kind of a natural feeling gap . postdoc c: Which I c I don't know what it would be quantified as . You know , Wally Chafe says that  , in producing narratives , the spurts that people use tend to be , that the the what would be a pause might be something like two two seconds . postdoc c: I was interested that you chose  , you know  , the you know that you use cuz I think that 's a unit that would be more consistent with sociolinguistics . phd g: Well we chose  , you know , half a second because if if you go much larger , you have a y you know , your your statement about how much overlap there is becomes less , precise , postdoc c:   Yeah , I also used I think something around zero point five seconds for the speech - nonspeech detector phd g: and it 's also based  Liz suggested that value based on the distribution of pause times that you see in Switchboard and and other corpora . postdoc c: In any case , this this  , meeting that I hand I I hand - adjusted two of them I mentioned before , phd g:   postdoc c: and I sent I sent email , phd g: OK , postdoc c: so phd g: So so at some point we will try to fine - tune our forced alignment postdoc c: And I sent the path . phd g: maybe using those as references because you know , what you would do is you would play with different parameters . And to get an object You need an objective measure of how closely you can align the models to the actual speech . So , I will  phd b: Yeah and hopefully the new meetings which will start from the channelized version will will have better time boundaries and alignments . postdoc c: But I like this idea of  , for our purposes for the for the IBM preparation , n having these joined together , phd b: Yeah . phd b: And and in in the in the previous version where in the n which is used now , there , the backchannel would would be in - between there somewhere , so . Well , phd b: but postdoc c: that 's that 's right , but you know , thi this brings me to the other f stage of this which I discussed with you earlier today , phd b: Yeah . postdoc c: which is the second stage is  , w what to do in terms of the transcribers adjustment of these data .  , the tr so the idea initially was , we would get  , for the new meetings , so the e EDU meetings , that Thilo ha has now presegmented all of them for us , on a channel by channel basis . And  , so , I 've assigned I 've I 've assigned them to our transcribers and  , so far I 've discussed it with one , with  And I had a about an hour discussion with her about this yesterday , we went through  EDU - one , at some extent . And it occurred to me that  that basically what we have in this kind of a format is you could consider it as a staggered mixed file , we had some discussion over the weekend a about at at this other meeting that we were all a at  , about whether the tran the IBM transcribers should hear a single channel audio , or a mixed channel audio . And  , in in a way , by by having this this chunk and then the backchannel after it , it 's like a stagal staggered mixed channel . And  , it occurred to me in my discussion with her yesterday that  , the the the maximal gain , it 's from the IBM people , may be in long stretches of connected speech . So it 's basically a whole bunch of words which they can really do , because of the continuity within that person 's turn . So , what I 'm thinking , and it may be that not all meetings will be good for this , but but what I 'm thinking is that in the EDU meetings , they tend to be driven by a couple of dominant speakers . And , if the chunked files focused on the dominant speakers , then , when when it got s patched together when it comes back from IBM , we can add the backchannels . It seems to me that  , you know , the backchannels per - se wouldn't be so hard , but then there 's this question of the time @ @  , marking , and whether the beeps would be  y y y And I 'm not exactly sure how that how that would work with the with the backchannels . And , so  And certainly things that are intrusions of multiple words , taken out of context and displaced in time from where they occurred , that would be hard . So , m my thought is i I 'm having this transcriber go through the EDU - one meeting , and indicate a start time f for each dominant speaker , endpoi end time for each dominant speaker , and the idea that these units would be generated for the dominant speakers , and maybe not for the other channels . grad a: Yeah the only , disadvantage of that is , then it 's hard to use an automatic method to do that . The advantage is that it 's probably faster to do that than it is to use the automated method and correct it . I think I I think  , you know , the original plan was that the transcriber would adjust the t the boundaries , and all that for all the channels but , you know , that is so time - consuming , and since we have a bottleneck here , we want to get IBM things that are usable s as soon as possible , then this seemed to me it 'd be a way of gett to get them a flood of data , which would be useful when it comes back to us . postdoc c: Oh also , at the same time she when she goes through this , she 'll be  If there 's anything that was encoded as a pause , but really has something transcribable in it , then she 's going to  , make a mark w  , so you know , so that that bin would be marked as it as double dots and she 'll just add an S . And in the other in the other case , if it 's marked as speech , and really there 's nothing transcribable in it , then she 's going to put a s dash , and I 'll go through and it and  , you know , with a with a substitution command , get it so that it 's clear that those are the other category . But  , the transcribable events that  , I 'm considering in this , continue to be laugh , as well as speech , and cough and things like that , so I 'm not stripping out anything , just just you know , being very lenient in what 's considered speech . Yeah ? phd d: Jane ? In terms of the this new procedure you 're suggesting , u what is the grad a: It 's not that different . phd d: So I 'm a little confused , because how do we know where to put beeps ? Is it i d y is it postdoc c: Oh , OK . postdoc c: So what it what it what it involves is is really a s  , the original pr procedure , but only applied to  , a certain strategically chosen s aspect of the data . grad a: We pick the easy parts of the data basically , postdoc c: So grad a: and transcriber marks it by hand . grad a: And because phd d: But after we 've done Thilo 's thing . Oh , OK , postdoc c: Yes ! grad a: I didn't I didn't understand that . phd g: So ,  grad a: OK , leave the mikes on , and just put them on the table . postdoc c: We start with the presegmented version grad a: Let me mark you as no digits . phd b: You start with the presegmentation , r yeah ? postdoc c: Yeah . And then  , the transcriber , instead of going painstakingly through all the channels and moving the boundaries around , and deciding if it 's speech or not , but not transcribing anything . OK ? Instead of doing that , which was our original plan , the tra They focus on the dominant speaker phd d:   So what they do is they identify who 's the di dominant speaker , and when the speaker starts . postdoc c: So  , you 're still gonna phd b: And you just postdoc c: So we 're It 's based on your se presegmentation , that 's the basic thing . phd b: and you just use the s the segments of the dominant speaker then ? For for sending to to IBM or ? postdoc c: Yeah . phd d: So , now Jane , my question is when they 're all done adjusting the w time boundaries for the dominant speaker , have they then also erased the time boundaries for the other ones ? postdoc c:   postdoc c: That 's that 's why she 's notating the start and end points of the dominant speakers . So , on a you know , so i in EDU - one , i as far as I listened to it , you start off with a a s section by Jerry . So Jerry starts at minute so - and - so , and goes until minute so - and - so . And he starts at minute such - and - such , and goes on till minute so - and - so . And then meanwhile , she 's listening to both of these guys ' channels , determining if there 're any cases of misclassification of speech as nothing , and nothing as speech , phd d:   phd d: So she does the adjustments on those guys ? postdoc c: But you know , I wanted to say , his segmentation is so good , that  , the part that I listened to with her yesterday didn't need any adjustments of the bins . So this is not gonna be a major part of the process , at least least not in not on ones that that really phd d: So if you don't have to adjust the bins , why not just do what it for all the channels ? postdoc c:  -  ? phd d: Why not just throw all the channels to IBM ? postdoc c: Well there 's the question o of whether Well , OK . She i It 's a question of how much time we want our transcriber to invest here when she 's gonna have to invest that when it comes back from IBM anyway . postdoc c: So if it 's only inserting "  -  "s here and there , then , wouldn't that be something that would be just as efficient to do at this end , instead of having it go through I B M , then be patched together , then be double checked here . But But then we could just use the the output of the detector , and do the beeping on it , and send it to I B phd d: Without having her check anything . postdoc c: Well , I guess grad a: I think we just we just have to listen to it and see how good they are . phd b: For some meetings , I 'm I 'm sure it i n postdoc c: I 'm I 'm open to that , it was professor f: Yeah , if it 's working well , phd b: That 's And some on some meetings it 's good . professor f: that sounds like a good idea since as you say you have to do stuff with the other end anyway .  the detector , this phd d: Yeah ,  we have to fix it when it comes back anyhow . postdoc c: Now , you were saying that they they differ in how well they work depending on channel s sys systems and stuff . So we should perhaps just select meetings on which the speech - nonspeech detection works well , postdoc c: But EDU is great . phd b: and just use , those meetings to to to send to IBM and , do the other ones .  , my my my impression is that it 's better for meetings with fewer speakers , and it 's better for for meetings where nobody is breathing . phd d: So in fact this might suggest an alternative sort of a a c a hybrid between these two things . Yeah ? phd d: So the the one suggestion is you know we we run Thilo 's thing and then we have somebody go and adjust all the time boundaries phd b: Yeah . phd d: There 's a a another possibility if we find that there are some problems , phd b: Yeah . phd d: and that is if we go ahead and we just run his , and we generate the beeps file , then we have somebody listen beeps file . And erase phd d: And they listen to each section and say " yes , no " whether that section is phd b: Yeah . And it just You know , there 's a little interface which will for all the " yes " - es it then that will be the final beep file . postdoc c: That 's interesting ! Cuz that 's that 's directly related to the e end task .  it wouldn't be that much fun for a transcriber to sit there , hear it , beep , yes or no . I don't know , I I think I 'm I 'm really tending towards grad a: One and a half times real time . professor f: what 's the worst that happens ? Do the transcribers  as long as th on the other end they can say there 's there 's something conventions so that they say " huh ? " phd d: Yeah . professor f: i i It i phd d: We can just catch it at the catch everything at this side . phd d: Well maybe that 's the best way to go , postdoc c: How interesting ! phd d: just grad a:  it just depends on how postdoc c: Well EDU phd b: Yeah , grad a: Sorry , go ahead . phd b: u u u postdoc c: So I was gonna say , EDU - one is good enough , phd b: Yeah . postdoc c: maybe we could include it in this in this set of  , this stuff we send . phd b: Yeah there 's I I think there are some meetings where it would would It 's possible like this . grad a: Yeah I I think , we won't know until we generate a bunch of beep files automatically , listen to them and see how bad they are . phd d: We won't be able to s include it with this first thing , grad a: If postdoc c:  . phd d: because there 's a part of the process of the beep file which requires knowing the normalization coefficients . phd d: Right , except I don't think that the c the instructions for doing that was in that directory , right ? I I didn't see where you had gener grad a: No , but it 's easy enough to do . phd b: What professor f: But I but I have a phd b: Doing the gain ? It 's no problem . professor f: But but but I I I have another suggestion on that , which is , since , really what this is , is is is trying to in the large , send the right thing to them and there is gonna be this this post - processing step , why don't we check through a bunch of things by sampling it ? phd d:   professor f: Right ? In other words , rather than , saying we 're gonna listen to everything grad a: I didn't mean listen to everything , I meant , just see if they 're any good . So y you do a bunch of meetings , you listen to to a little bit here and there , phd d: Yeah . professor f: if it sounds like it 's almost always right and there 's not any big problem you send it to them . professor f: And , you know , then they 'll send us back what we w what what they send back to us , postdoc c: Oh , that 'd be great . professor f: and we 'll we 'll fix things up and some meetings will cost more time to fix up than others . grad a: And we should just double - check with Brian on a few simple conventions on how they should mark things . grad a: Yeah , cuz @ @  what I had originally said to Brian was well they 'll have to mark , when they can't distinguish between the foreground and background , professor f: Yeah . But if we send them without editing , then we 're also gonna hafta have m  , notations for words that are cut off , phd d:   phd d: And they may just guess at what those cut - off words are , postdoc c: Yeah . phd d: but w  we 're gonna adjust everything when we come back grad a: But what what we would like them to do is be conservative so that they should only write down the transcript if they 're sure . postdoc c: which professor f: i Can I maybe have have an order of it 's probably in your paper that I haven't looked at lately , but postdoc c: Certainty . professor f: an order of magnitude notion of of how on a good meeting , how often  , do you get segments that come in the middle of words and so forth , and  in a bad meeting how often ? phd b:  . postdoc c: Was is it in a in a what what is the t professor f: Well he 's saying , you know , that the the EDU meeting was a good good meeting , postdoc c: In a good meeting , what ? phd b: Yeah . professor f: right ? postdoc c: Oh I see , professor f: and so so so it was almost it was almost always doing the right thing . And then ,  in a bad meeting , or p some meetings where he said oh he 's had some problems , what does that mean ? postdoc c:  - huh . professor f: So  does one of the does it mean one percent and ten percent ? Or does it mean five percent and fifty percent ? postdoc c: OK . professor f:  phd b: So professor f: Or Maybe percentage isn't the right word , postdoc c: Just phd b: Yeah th professor f: but you know how many how many per minute , or You know . phd b: Yeah , the the problem is that , nnn , the numbers Ian gave in the paper is just  , some frame error rate . So that 's that 's not really What will be effective for for the transcribers , is They have to yeah , in in they have to insure that that 's a real s spurt or something . So the speech the amount of speech that is missed by the detector , for a good meeting , I th is around or under one percent , I would say . For yeah , but there can be more There 's There 's more amount speech  , more amount of Yeah well , the detector says there is speech , but there is none . Now what about in a meeting that you said we 've you 've had some more trouble with ? phd b: I can't really hhh , Tsk . That 's really I I did this on on four meetings and only five minutes of of every meet of of these meetings so , it 's not not that representative , but , it 's perhaps , Fff .  Yeah , it 's perhaps then it 's perhaps five percent of something , which s  the the frames speech frames which are which are missed , but  , I can't can't really tell . So I So i Sometime , we might wanna go back and look at it more in terms of how many times is there a spurt that 's that 's  , interrupted ? phd b: Yeah . professor f: Something like that ? postdoc c: The other problem is , that when it when it  d i on the breathy ones , where you get breathing , inti indicated as speech . professor f: And phd b: So postdoc c: And I guess we could just indicate to the transcribers not to encode that if they We could still do the beep file . professor f: Yeah again I I think that that is probably less of a problem because if you 're if there 's If if a if a word is is split , then they might have to listen to it a few times to really understand that they can't quite get it . professor f: Whereas if they listen to it and there 's don't hear any speech I think they 'd probably just listen to it once . professor f: So there 'd you 'd think there 'd be a a factor of three or four in in , cost function , postdoc c: OK . phd b: Yeah , so but I think that 's n that really doesn't happen very often that that that a word is cut in the middle or something . professor f: So so what you 're saying is that nearly always what happens when there 's a problem is that is that  , there 's some  ,  nonspeech that  that is b interpreted as speech . phd b: Yeah , it 's professor f: You know , if they they hear you know , a dog bark and they say what was the word , they you know , they phd b: Yeah , I als I professor f: Ruff ruff ! phd b: Yeah I also thought of there there are really some channels where it is almost  , only bre breathing in it . I 've got a a P - a method with loops into the cross - correlation with the PZM mike , and then to reject everything which which seems to be breath . phd b: So , I could run this on those breathy channels , and perhaps throw out grad a: That 's a good idea . But I think I th Again , I think that sort of that that would be good , phd b: Yeah . professor f: But I think none of this is stuff that really needs somebody doing these these  , explicit markings . Oh , I 'd be delighted with that , I I was very impressed with the with the result . professor f: Yeah , cuz the other thing that was concerning me about it was that it seemed kind of specialized to the EDU meeting , and and that then when you get a meeting like this or something , phd b: Yeah . professor f: and and you have a b a bunch of different dominant speakers postdoc c: Oh yeah , interesting . professor f: Whereas this sounds like a more general solution postdoc c: Oh yeah , I pr I much prefer this , professor f: is postdoc c: I was just trying to find a way Cuz I I don't think the staggered mixed channel is awfully good as a way of handling overlaps . phd d: And we can just , you know , get the meeting , process it , put the beeps file , send it off to IBM . I would phd d: Do what ? phd b: listen to it , and then grad a: Or at least sample it . professor f: make sure you don't send them three hours of " bzzz " or something . phd b: And there 's there 's one point which I  yeah , which which I r we covered when I when I r listened to one of the EDU meetings , professor f: Great . grad a:  - huh phd b: And i the speech - nonspeech detector just assigns randomly the speech to to one of the channels , so .  - I haven't - I didn't think of of s of this before , grad a: What can you do ? phd b: but what what shall we do about s things like this ? postdoc c: Well you were suggesting You suggested maybe just not sending that part of the meeting . postdoc c: But phd b: But , sometimes the the the laptop is in the background and some somebody is is talking , and , that 's really a little bit confusing , but grad a: It 's a little bit confusing . phd b: that 's that 's a second question , " what what will different transcribers do with with the laptop sound ? " postdoc c: Would you would professor f: What was the l what was the laptop sound ? postdoc c: Yeah , go ahead . postdoc c: Well , so  So my standard approach has been if it 's not someone close - miked , then , they don't end up on one of the close - miked channels . phd b: when thi when this is sent to to the I M - eh , I B M transcribers , I don't know if if they can tell that 's really postdoc c: Yeah , that 's right . grad a:  postdoc c: Well , they have a convention , in their own procedures , which is for a background sound . grad a: Right , but , in general I don't think we want them transcribing the background , cuz that would be too much work . grad a: Right ? For it because in the overlap sections , then they 'll phd d: Well I don't think Jane 's saying they 're gonna transcribe it , but they 'll just mark it as being there 's some background stuff there , grad a: But that 's gonna be all over the place . phd d: right ? grad a: How w how will they tell the difference between that sort of background and the dormal normal background of two people talking at once ? phd b: Yeah . postdoc c: Oh , I think I think it 'd be easy to to say " background laptop " . grad a: How would they know that ? phd d: But wait a minute , why would they treat them differently ? phd b: Yeah . postdoc c: Well because one of them grad a: Because otherwise it 's gonna be too much work for them to mark it . postdoc c: Oh , I s background laptop or , background LT wouldn't take any time . grad a: Sure , but how are they gonna tell bet the difference between that and two people just talking at the same time ? postdoc c: And phd b: Yeah . Acoustically , can't you tell ? phd b: It 's really good sound , so postdoc c: Oh is it ? Oh ! professor f: Well , isn't there a category something like  , " sounds for someone for whom there is no i close mike " ? phd b: Yeah that would be very important , grad a: But how do we d how do we do that for the I B M folks ? postdoc c: Yeah . grad a: How can they tell that ? phd d: Well we may just have to do it when it gets back here . grad a: And they 'll just mark it however they mark it , postdoc c: That sounds good . postdoc c: Well , as it comes back , we have a  when we can use the channelized interface for encoding it , then it 'll be easy for us to handle . postdoc c: But but if if out of context , they can't tell if it 's a channeled speak  , you know , a close - miked speaker or not , then that would be confusing to them . postdoc c: I don't know , I it doesn't I don't Either way would be fine with me , I don't really care . Do you think we should send the  that whole meeting to them and not worry about pre - processing it ? professor f: Yes ma ' postdoc c: Or  , what  is we we should leave the part with the audio in the  , beep file that we send to IBM for that one , or should we start after the that part of the meeting is over in what we send . professor f: Which part ? phd b: With postdoc c: So , the part where they 're using sounds from their from their laptops . phd b: with the laptop sound , or ? just postdoc c: w If we have speech from the laptop should we just  , excise that from what we send to IBM , or should we i give it to them and let them do with it what they can ? phd d: I think we should just it it 's gonna be too much work if we hafta worry about that I think . postdoc c: And give them freedom to to indicate if it 's just not workable . professor f: Cuz , I wouldn't don't think we would mind having that transcribed , if they did it . grad a: I think phd d: Yeah , e grad a: As I say , we 'll just have to listen to it and see how horrible it is . grad a:  -  , and and they 're very it 's very audible ? on the close - talking channels ? phd b: What what I would Yeah . postdoc c: OK , so we read the transcript number first , right ? grad a: Are we gonna do it altogether or separately ? phd b: So What time is it ? professor f: why don't we do it together , postdoc c: quarter to four . professor f: One , two , three , go ! postdoc c: It 's kind of interesting if there 're any more errors in these , than we had the first set . phd d: Do you guys plug your ears when you do it ? grad a: I do . professor f: I haven't been , phd d: How can you do that ? professor f: no . phd b: Perhaps there are lots of errors in it phd d: Gah ! grad a: Total concentration . Are you guys ready ? phd d: You hate to have your ears plugged ? professor f: Yeah