grad e: I think it 's actually phd d: What is what is that ? grad e: it depends on if the temp files are there or not , that at least that 's my current working hypothesis , phd d: Ah . grad e: that I think what happens is it tries to clear the temp files and if they 're too big , it crashes . phd b: When the power went out the other day and I restarted it , it crashed the first time . grad e: no , it doesn't it doesn't clear those necessarily , phd d: Oh wait It it doesn't clear them , OK . grad e: It 's i they 're called temp files , but they 're not actually in the temp directory they 're in the scratch , so . phd d: But that 's usually the meeting that I recorded , and it neve it doesn't crash on me . phd b: Well this wasn't Actually , this wasn't a before your meeting , this was , Tuesday afternoon when , Robert just wanted to do a little recording , grad e: Oh well . professor c: I don't know when would be a good excuse for it , but I just can't wait to be giving a talk t and and and use the example from last week with everybody t doing the digits at once . You know ? You wanted to pe keep people from listening in , you could like have that playing outside the room . Everybody give the reports about what they were doing at exactly the same time , phd d: And we 'll just all leave , phd b: And then we 'll we 'll go back later and review the individual channels , professor c: yeah . phd b: If you wanna know what professor c: Actually isn't that what we have been doing ? phd d: Yeah . professor c: What are we doing ? grad e: I Since I 've been gone all week , I didn't send out a reminder for an agenda , so . professor c: Yeah , and I 'm just grad e: Do we have anything to talk about or should we just read digits and go ? phd b: I wouldn't mind hearing how the conference was . professor c: What conference ? phd d: I had one question about grad e: Yeah , really . The next , phd d: Next weekend ? grad e: Next weekend , week from phd f: right ? professor c: That is right . phd d: Sorry , not not not the days coming up , but phd f: It 's like the grad e: A week from Saturday . phd d: So , are we do we have like an agenda or anything that we should be professor c: No , but that would be a good idea . professor c: Why don't we w phd f: So so the deal is that I can , I can be available after , like ten thirty or something . I don't know how s how early you wanted to professor c: They 're not even gonna be here until eleven or so . grad e: Eurospeech is due on Friday and then I 'm going down to San  , San Jose Friday night , so , if you know , if we start nice and late Saturday that 's a good thing . grad e: They 're flying from somewhere to somewhere , professor c: Yeah , and they 'll end up here . professor c: So , i I I will be  , he 's taking a very early flight phd f: Oh . professor c: and we do have the time work difference running the right way , but I still think that there 's no way we could start before eleven . grad e: But , yeah maybe an agenda , or at least some things to talk about would be a good idea . professor c: Well we can start gathering those those ideas , but then we we should firm it up by next next Thursday 's meeting . postdoc a: Will we have time to , to prepare something that we in the format we were planning for the IBM transcribers by then , or ? grad e: Oh yeah . grad e: So have you heard back from Brian about that , Chuck ? phd b: Yes , he 's I I 'm sorry , I should have forwarded that along .  , oh I I think I mentioned at the last meeting , he said that , he talked to them and it was fine with the beeps they would be That 's easy for them to do . So , oh , though Thi - Thilo isn't here , but , I I have the program to insert the beeps . What I don't have is something to parse the output of the channelized transcripts to find out where to put the beeps , but that should be really easy to do . So do we have a meeting that that 's been done with , postdoc a: He 's he 's grad e: that we 've tightened it up to the point where we can actually give it to IBM and have them try it out ? postdoc a: He generated , a channel - wise presegmented version of a meeting , but it was Robustness rather than EDU so I guess depends on whether we 're willing to use Robustness ? phd b: Well for this experiment I think we can use pre pretty much anything . phd b: This experiment of just grad e: Well we had we had talked about doing maybe EDU as a good choice , though . phd b: Well we 've talked about that as being the next ones we wanted to transcribe . phd b: But for the purpose of sending him a sample one to f grad e: Yeah , maybe it doesn't matter . phd b: I I don't think it matte postdoc a: I 'll I 'll I 'll , get make that available . grad e: OK , and has it been corrected ? postdoc a: Oh , well , wait .  grad e: Hand - checked ? Cuz that was one of the processes we were talking about as well . phd b: Right , so we need to run Thilo 's thing on it , postdoc a: That 's right . postdoc a: I think they 're coming phd b: And we probably don't have to do necessarily a whole meeting for that if we just wanna send them a sample to try . What would be a good number of minutes ? phd b: I don't know , maybe we can figure out how long it 'll take @ @ to to do . grad e: I don't know , it seems to me w we probably should go ahead and do a whole meeting because we 'll have to transcribe the whole meeting anyway sometime . professor c: Yes except that if they had if there was a choice between having fifteen minutes that was fully the way you wanted it , and having a whole meeting that didn't get at what you wanted for them It 's just dependent of how much grad e: Like I  I guess if we have to do it again anyway , but ,  professor c: Yeah . phd b: I guess , the only thing I 'm not sure about is , how quickly can the transcribers scan over and fix the boundaries , postdoc a:   phd b: and  , is it pretty easy ? grad e: I think it 's gonna be one or two times real time at Wow , excuse me , two or more times real time , right ? Cuz they have to at least listen to it . professor c: Can we pipeline it so that say there 's , the transcriber gets done with a quarter of the meeting and then we you run it through this other other stuff ?  , grad e: Well the other stuff is I B I 'm just thinking that from a data keeping - track - of - the - data point of view , it may be best to send them whole meetings at a time and not try to send them bits and pieces . So the first thing is the automatic thing , and then it 's then it 's then it 's the transcribers tightening stuff up , grad e: Right . professor c: OK , so you might as well ha run the automatic thing over the entire meeting , and then and then , you would give IBM whatever was fixed . postdoc a: And have them fix it over the entire meeting too ? grad e: Right . professor c: Well , yeah , but start from the beginning and go to the end , right ? So if they were only half way through then that 's what you 'd give IBM . The I guess the question on my mind is do we wait for the transcribers to adjust the marks for the whole meeting before we give anything to IBM , or do we go ahead and send them a sample ? Let their professor c: Why wouldn't we s @ @ w i if they were going sequentially through it , why wouldn't we give them  i are we trying to get something done by the time Brian comes ? phd b: Well I I  , I don't know . professor c: So if we if we were , then it seems like giving them something , whatever they had gotten up to , would be better than nothing . grad e: Well , I don't think  , h they they typically work for what , four hours , something like that ? postdoc a: I gue  . grad e: I think the they should be able to get through a whole meeting in one sitting . I would think , unless it 's a lot harder than we think it is , which it could be , certainly . postdoc a: If it 's got like for speakers then I guess  if phd b: We 're just doing the individual channels , grad e: Or seven or eight . phd b: So it 's gonna be , depending on the number of people in the meeting , postdoc a: I guess there is this issue of , you know , if if the segmenter thought there was no speech on on a particular stretch , on a particular channel , grad e: Well postdoc a: and there really was , then , if it didn't show up in a mixed signal to verify , then it might be overlooked , so , the question is " should should a transcriber listen to the entire thing or can it g can it be based on the mixed signal ? " And I th eh so far as I 'm concerned it 's fine to base it on the mixed signal at this point , and grad e: That 's what it seems to me too , in that if they need to , just like in the other cases , they can listen to the individual , if they need to . postdoc a: What what aspect ? professor c: So you 're talking about tightening up time boundaries ? phd b: Yeah . professor c: So how do you grad e: So , they have the normal channeltrans interface where they have each individual speaker has their own line , phd b: Yeah . grad e: but you 're listening to the mixed signal and you 're tightening the boundaries , correcting the boundaries . grad e: Right , so so you 'll have to I phd d: It will miss them . It will it will miss grad e:  - huh ! phd d: Yeah , you have to say "  - huh " more slowly to to get c grad e: Sorry . phd d: So it will miss stuff like that which phd b: I grad e: Well , so so that 's something that the transcribers will have to have to do . postdoc a: Yeah , but presumably , most of those they should be able to hear from the mixed signal unless they 're embedded in the heavil heavy overlap section when in which case they 'd be listening to the channels anyway . phd b: That 's that 's what I 'm I 'm concerned about the part . phd b: Can't we  couldn't we just have , I don't know , maybe this just doesn't fit with the software , but I guess if I didn't know anything about Transcriber and I was gonna make something to let them adjust boundaries , I would just show them one channel at a time , with the marks , and let them adju postdoc a: Oh they can grad e: Well , but then they have to do but then they for this meeting they would have to do seven times real time , and it would probably be more than that . grad e: Right ? Because they 'd have to at least listen to each channel all the way through . postdoc a: And if phd b: But i but it 's very quick , postdoc a:  - huh . postdoc a: w Well , the other problem is the breaths grad e: I just don't think postdoc a: cuz you also see the breaths on the waveform . I 've I 've looked at the int  , s I 've tried to do that with a single channel , and and you do see all sorts of other stuff besides just the voice . grad e: Yeah , and I I think that they 're going much more on acoustics than they are on visuals . postdoc a: What you the digital what the digital task that you had your interface ?  , I know for a fact that one of those sh she could really well she could judge what th what the number was based on the on the waveform . Yeah , I found the same thing that when I was scanning through the wave form I could see when someone started to read digits just by the shapes . professor c: So I don't I 'm I 'm now entirely confused about what they do . grad e: But professor c: So , they 're they 're looking at a mixed signal , or they 're looking what what are they looking at visually ? postdoc a: Well , they have a choice . But I 've I 've tried looking at the single signal and and in order to judge when it when it was speech and when it wasn't , grad e: Oh . postdoc a: but the problem is then you have breaths which which show up on the signal . professor c: But the procedure that you 're imagining , people vary from this , is that they have the mixed signal wave form in front of them , postdoc a: Yes . professor c: and they have multiple , well , let 's see , there isn't we don't have transcription yet . So but there 's markers of some sort that have been happening automatically , postdoc a: Yes . professor c: and those show up on the mixed signal ? postdoc a: Oh , professor c: There 's a @ @ clicks ? grad e: N the t postdoc a: they show up on the separate ribbons . postdoc a: and and i i it 'll be because it 's being segmented as channel at a time with his with Thilo 's new procedure , then you don't have the correspondence of the times across the bins  across the ribbons  you could have professor c: And is there a line moving across the waveform as it goes ? grad e: Yes . professor c: OK , so The way you 're imaging is they kind of play it , and they see oh this happened , then this happened , then and if it 's about right , they just sort of let it slide , postdoc a: Yeah . professor c: and if it if it there 's a question on something , they stop and maybe look at the individual wave form . professor c: They they might look at it , right ? grad e: Well , the problem is that the the interface doesn't really allow you to switch visuals . grad e: The problem is that that the Tcl - TK interface with the visuals , it 's very slow to load waveforms . grad e: And so when I tried that that was the first thing I tried when I first started it , postdoc a: Oh , oh . You can you can switch quickly between the audio , grad e: right ? postdoc a: but you just can't get the visual display to show quickly . So you have to It takes , I don't know , three , four minutes to Well , it takes it takes long enough phd d: Yeah , it 's very slow to do that . postdoc a: It takes long enough cuz it has to reload the I I don't know exactly what it 's doing frankly cuz but it t it takes long enough that it 's just not a practical alternative . phd d: That w grad e: Well it it does some sort of shape pre - computation so that it can then scroll it quickly , grad g: But you can cancel that . grad g: Oh , really ? postdoc a: Now you could set up multiple windows , each one with a different signal showing , and then look between the windows . grad e: we we could do different interfaces , grad g: What if you preload them all ? grad e: right ?  , so so we could use like X Waves instead of Transcriber , postdoc a: Yeah . grad g: What if you were to preload all the channels or or initially grad e: Well that 's what I tried originally . grad g: like doesn't grad e: So I I actually before , Dave Gelbart did this , I did an interface which showed each waveform and ea a ribbon for each waveform , grad g:   grad e: but the problem with it is even with just three waveforms it was just painfully slow to scroll . So you just scroll a screen and it would , you know go " kur - chunk ! " grad g: Oh , OK . postdoc a: You know , I am thinking if we have a meeting with only four speakers and , you know , you could fire up a Transcriber interface for , y you know , in different windows , multiple ones , one for each channel . And it 's sort of a a hack but  it would be one way of seeing the visual form . grad e: I think that if we decide that we need that they need to see the visuals , we need to change the interface so that they can do that . professor c: So phd d: That 's actually what I thought of , loading the chopped up waveforms , you know , that that would make it faster grad e: An But isn't grad g:  . phd b: The problem is if if anything 's cut off , you can't expand it from the chopped up phd d: So . phd d: Right , but if you a at some point grad e: And wouldn't that be the same as the mixed signal ? phd d: No ,  the individual channels that were chopped up that it 'd be nice to be able to go back and forth between those short segments . phd d: Cuz you don't really nee like nine tenths of the time you 're throwing most of them out , but what you need are tho that particular channel , or that particular location , grad e: Yeah . phd d: might be nice , cuz we save those out already , to be able to do that . But it won't work for IBM of course , it only works here cuz they 're not saving out the individual channels . postdoc a: Well , I I do think that this this will be a doable procedure , professor c: Yeah . postdoc a: and , then when they get into overlaps , just have them systematically check all the channels to be sure that there isn't something hidden from from audio view . Yeah , hopefully ,  The mixed signal , the overlaps are pretty audible because it is volume equalized . The only problem is is , you know , counting how many and if they 're really correct or not . phd d: I don't know that you can locate them very well from the mixed signal , grad e: Right but but once once you know that they happen , you can at least listen to the close talking , phd d: but you would know that they were there , and then you would switch . professor c: But right now , to do this limitation , the switching is going to be switching of the audio ? Is what she 's saying . professor c: So grad e: Right , so so professor c: so they 're using their ears to do these markings anyway . grad e: did Dave Did Dave do that change where you can actually just click rather than having to go up to the menu to listen to the individual channels ? postdoc a: Yes . postdoc a: I 'm not sure what click what click on the ribbon ? Yeah , you can get that grad e: Yeah . postdoc a: oh , oh , get you can get the ,  you can get it to switch audio ?  , not last I tried , grad e: Yeah . grad e: We should get him to do that because , I think that would be much , much faster than going to the menu . There 's a reason I disagree , and that is that , you it 's very good to have a dissociation between the visual and the audio . There 're times when I wanna hear the mixed signal , bu but I want to transcribe on the single channel . postdoc a: Maybe , I just don't I don't see that it 's a grad e: Just something so that it 's not in the menu option so that you can do it much faster . postdoc a: Well , that 's the i I I think that might be a personal style thing . grad e: Well it just seems to me that if you wanna quickly " well was that Jane , no , was that Chuck , no , was that Morgan " , right now , you have to go up to the menu , and each time , go up to the menu , select it , listen to that channel then click below , and then go back to the menu , select the next one , and then click below . postdoc a: Yeah , it could be faster , but , you know , th in the ideal world Yeah . grad e: What ? postdoc a: No I I agree that 'd be nice . professor c: So , Done with that ?  Does any I forget , does anybody , working on any any Eurospeech submission related to this ? grad e: I would like to try to do something on digits but I just don't know if we have time . Yeah there was that we that 's right , we had that one conversation about , what what what did it mean for , one of those speakers to be pathological , was it a grad e: Right , and I haven't had s chance to sit down and listen . phd f: Oh , I haven't I haven't listened to them either , grad e: I was going to do that this afternoon . phd f: but there must be something wrong , grad e: Well , Morgan and I were were having a debate about that . phd f: unless our grad e: Whereas I think it it 's probably something pathologic and actually Stephane 's results , I think confirm that . He s he did the Aurora system also got very lousy average error , like fifteen or or , fifteen to twenty percent average ? But then he ran it just on the lapel , and got about five or six percent word error ? So that that means to me that somewhere in the other recordings there are some pathological cases . So I 'll I 'll listen to it and find out since you 'd actually split it up by segment . phd b: Did you run the Andreas the r SRI recognizer on the digits ? grad e: Oh , I thought he had sent that around to everyone , phd f: Yeah . grad e: did you just sent that to me ? phd f: No , I d I didn't . phd b: I it wasn't phd f: But , yeah , if you take grad e: It was bimodal . phd f: So if you Yeah , it 's actually , it  it was trimodal , actually grad e: Oh , was it trimodal , OK . phd f: trimodal , so professor c: There 's zero , a little bit , and a lot . phd f: there were t there was there was one h one bump at ze around zero , which were the native speakers , professor c: Yeah . phd b: This is error you 're talking about ? professor c: Oh was it fifteen ? phd f: whe phd b: OK . And then there was another distinct bump at , like , a hundred , which must have been some problem . phd f: I can't imagine that grad g: What is patho what do you mean by pathological ? grad e: Just just something really wrong with grad g: I 'm sorry , I don't grad e: A bug is what  , phd f: In the recording grad g: Oh . phd f: And there was this one meeting , I forget which one it was , where like , six out of the eight channels were all , like had a hundred percent error . grad e: Which probably means like there was a th the recording interface crashed , grad g: Right . grad e: or there was a short you know , someone was jiggling with a cord phd f: But grad e: or , I extracted it incorrectly , phd f: But grad e: it was labeled grad g:   grad e: it was transcribed incorrectly , something really bad happened , and I just haven't listened to it yet to find out what it was . phd f: So , if I excluded the pathological ones , by definition , those that had like over ninety - five percent error rate , and the non - natives , then the average error rate was like one point four or something , professor c: What we 're calling . phd f: which which seemed reasonable given that , you know , the models weren't tuned for for it . phd b: And it didn't matter whether it was the lapel or whether it was the phd f: It was just a @ @ . I haven't split it up that way , phd d: But there 's no overlap during the digit readings , so it shouldn't really matter . professor c: No , but there 's a little difference , phd f: So it should grad e: There 's a lot . professor c: And so , cuz because what he was what I was saying when I looked at those things is it it I was almost gonna call it quadrimodal because because there was a whole lot of cases where it was zero percent . But if you p if you actually histogrammed it , and it was a nice  , you know , it it was zero was the most of them , professor c: Yeah . phd f: And then there was the bump for the non - natives and then the pathological ones , professor c: I see . postdoc a: You did you have , something in the report about , about , for f  , forced alignment ? professor c: Yeah . postdoc a: Have you have you started on that ? phd f: Oh , well , yeah , so I 've been struggling with the forced alignments . So the scheme that I drew on the board last time where we tried to ,  allow reject models for the s speech from other speakers , most of the time it doesn't work very well . So , and the I haven't done  , the only way to check this right now was for me to actually load these into X Waves and , you know , plus the alignments , and s play them and see where the professor c:  . phd f: And it looks And so I looked at all of the utterances from you , Chuck , in that one conversation , I don't know which You probably know which one  , it 's where you were on the lapel and Morgan was sitting next to you and we can hear everything Morgan says . phd f: But and and some of what you  , you also appear quite a bit in that cross - talk . So , I actually went through all of those , there were I think fifty - five segments , in in X Waves , and and sort of did a crude check , and more often than not , it it gets it wrong . So there 's either the beginning , mostly the beginning word , where th you , you know , Chuck talks somewhere into the segment , but the first , word of what he says , often " I " but it 's very reduced " I , " that 's just aligned to the beginning of someone else 's speech ,  in that segment , which is cross - talk . So , I 'm still tinkering with it , but it might well be that we can't get clean alignments out of this out of those , channels , so . phd d: Right , but that 's  , that was our plan , phd f: Yeah , right . phd d: but it 's clear from Dan that this is not something you can do in a short amount of time . phd d: So so we you know , we had spent a lot of time , writing up the HLT paper and we wanted to use that , kind of analysis , professor c: Yeah . phd d: but the HLT paper has , you know , it 's a very crude measure of overlap . It 's not really something you could scientifically say is overlap , it 's just whether or not the , the segments that were all synchronized , whether there was some overlap somewhere . phd d: And , you know , that pointed out some differences , so he thought well if we can do something quick and dirty because Dan said the cross - cancellation , it 's not straight - forward . If it were straight - forward then we would try it , but so , it 's sort of good to hear that it was not straight - forward , thinking if we can get decent forced alignments , then at least we can do sort of a overall report of what happens with actual overlap in time , but ,  phd b: I didn't think that his message said it wasn't straight - forward . grad e: Well if we 'd just professor c: Well phd b: I thought he 's just saying you have to look over a longer time window when you do it . phd d: and the but there are some issues of this timing , in the recordings professor c: Yeah . phd d: and phd b: So you just have to look over longer time when you 're trying to align the things , you can't you can't just look grad e: Well . are you talking about the fact that the recording software doesn't do time - synchronous ? Is that what you 're referring to ? professor c:  grad e: That seems to me you can do that over the entire file and get a very accurate phd f: I don't thi I d I don't think that was the issue . phd f: The issue was that you have to you have have you first have to have a pretty good speech detection on the individual channels . phd d: And it 's dynamic , so I guess it was more dynamic than some simple models would be able t to so so there are some things available , and I don't know too much about this area where if people aren't moving around much than you could apply them , and it should work pretty well if you took care of this recording time difference . phd d: but then if you add the dynamic aspect of adapting distances , then it wasn't I guess it just wasn't something that he could do quickly and not in time for us to be able to do something by two weeks from now , so . So  , so I don't know what we can do if anything , that 's sort of worth , you know , a Eurospeech paper at this point . phd b: Well , Andreas , how well did it work on the non - lapel stuff ? grad e: Yeah . phd f: we would really need , ideally , a transcriber to time mark the you know , the be at least the beginning and s ends of contiguous speech .  , and , you know , then with the time marks , you can do an automatic comparison of your of your forced alignments . phd b: Because really the the at least in terms of how we were gonna use this in our system was to get an ideal an idea , for each channel about the start and end boundaries . phd b: We don't really care about like intermediate word boundaries , so phd f: No , that 's how I 've been looking at it . phd f: I don't care that the individual words are aligned correctly , phd b: Yeah . phd f: but you don't wanna , infer from the alignment that someone spoke who didn't . So that 's why I was wondering if it phd f: so , so phd b: maybe if it doesn't work for lapel stuff , we can just not use that phd f: Yeah . phd b: and phd f: I haven't I ha just haven't had the time to , do the same procedure on one of the so I would need a k I would need a channel that has a speaker whose who has a lot of overlap but s you know , is a non - lapel mike . grad e:  ! phd f: So , I grad e: So a meeting with me in it . phd f: maybe someone can help me find a good candidate and then I would be willing to phd b: We c you know what ? Maybe the best way to find that would be to look through these . phd f: you know , hand phd b: Cuz you can see the seat numbers , and then you can see what type of mike they were using . And so we just look for , you know , somebody sitting next to Adam at one of the meetings phd d: Actually y we can tell from the data that we have , phd f: From the insertions , maybe ? phd d: yeah , there 's a way to tell . phd f: fr fr from the phd d: It might not be a single person who 's always overlapping that person but any number of people , phd f: Right . phd d: and , if you align the two hypothesis files across the channels , you know , just word alignment , you 'd be able to find that . So so I guess that 's sort of a last ther there 're sort of a few things we could do . Another one was to try to get somehow align Thilo 's energy segmentations with what we have . But then you have the problem of not knowing where the words are because these meetings were done before that segmentation . phd b: What what is why do you need the , the forced alignment for the HLT  for the Eurospeech paper ? phd d: Well , I guess I I wanted to just do something not on recognition experiments because that 's ju way too early , but to be able to report , you know , actual numbers . Like if we if we had hand - transcribed pe good alignments or hand - checked alignments , then we could do this paper . But without knowing where the real words are , in time phd b: So it was to get it was to get more data and better to to squeeze the boundaries in . phd d: To to know what an overlap really if it 's really an overlap , or if it 's just a a a segment correlated with an overlap , phd b: Ah , OK . phd d: and I guess that 's the difference to me between like a real paper and a sort of , promissory paper . So , if we d it might be possible to take Thilo 's output and like if you have , like right now these meetings are all , grad e: Ugh ! I forgot the digital camera again . phd d: grad e: Every meeting ! phd d: you know , they 're time - aligned , so if these are two different channels and somebody 's talking here and somebody else is talking here , just that word , if Thilo can tell us that there 're boundaries here , we should be able to figure that out grad e:   phd d: Yeah , if you have two and they 're at the edges , it 's like here and here , and there 's speech here , then it doesn't really help you , so ,  phd b: Thilo 's won't put down two separate marks in that case phd d: Well it w it would , but , we don't know exactly where the words are because the transcriber gave us two words in this time bin grad e: Thilo 's will . phd d: and we don't really know , postdoc a: Well it 's a merging problem . If you had a if you had a s if you had a script which would phd d: yeah it 's postdoc a: I 've thought about this , and I 've discussed I 've discussed it with Thilo , phd d: if you have any ideas . I would postdoc a: the , I I in principle I could imagine writing a script which would approximate it to some degree , but there is this problem of slippage , grad e: Well maybe Maybe that will get enough of the cases to be useful . grad e: You know s cuz it seemed like most of the cases are in fact the single word sorts , or at least a single phrase postdoc a: Well they they can be stretched . postdoc a: I wouldn't make that generalization cuz sometimes people will say , " And then I " and there 's a long pause phd d: Yeah . postdoc a: and finish the sentence and and sometimes it looks coherent and and the  it 's it 's not a simple problem . But it 's really And then it 's coupled with the problem that sometimes , you know , with with a fricative you might get the beginning of the word cut off and so it 's coupled with the problem that Thilo 's isn't perfect either .  , we 've i th it 's like you have a merging problem plus so merging plus this problem of , not grad e: Right .  ! postdoc a: y i i if the speech - nonspeech were perfect to begin with , the detector , that would already be an improvement , but that 's impossible , you know , i that 's too much to ask . postdoc a: And so i and may you know , it 's I think that there always th there would have to be some hand - tweaking , but it 's possible that a script could be written to merge those two types of things . I 've I 've discussed it with Thilo and  in terms of not him doing it , but we we discussed some of the parameters of that and how hard it would be to in principle to write something that would do that . phd d: I guess in the future it won't be as much as an issue if transcribers are using the tightened boundaries to start with , then we have a good idea of where the forced alignment is constrained to . postdoc a: Well , it 's just , you know , a matter of we had the revolution we had the revolution of improved , interface , one month too late , phd d: So I 'm no I don't know if this grad e: Oh . postdoc a: but it 's like , you know , it 's wonderful to have the revolution , phd d: Oh it 's it 's a postdoc a: so it 's just a matter of of , you know , from now on we 'll be able to have things channelized to begin with . grad e: I was just thinking about the fact that if Thilo 's missed these short segments , that might be quite time - consuming for them to insert them . phd d: But he he also can adjust this minimum time duration constraint and then what you get is noises mostly , postdoc a: Yeah . phd d: but that might be OK , an grad e: It might be easier to delete something that 's wrong than to insert something that 's missing . And you can also see in the waveform exac grad e: What do you think , Jane ? phd d: yeah . professor c: If you can feel confident that what the yeah , that there 's actually something phd d: Yeah . Cuz then then you just delete it , and you don't have to pick a time . phd d: I think it 's postdoc a: Well the problem is I you know I I it 's a it 's a really good question , and I really find it a pain in the neck to delete things because you have to get the mouse up there on the t on the text line and i and otherwise you just use an arrow to get down  , i it depends on how lar th there 's so many extra things that would make it one of them harder than the other , or or vice versa . But , you know , in principle , like , you know , if one of them is easier then to bias it towards whichever one 's easier . grad e: Yeah , I guess the semantics aren't clear when you delete a segment , right ? Because you would say You would have to determine what the surroundings were . phd d: You could just say it 's a noise , though , and write , you know , a post - processor will just all you have to do is just grad e: If it 's really a noise . phd d: or just say it 's just put " X , " you know , like " not speech " or something , postdoc a: I think it 's easier to add than delete , frankly , phd d: and then you can get Yeah , or postdoc a: because you have to , maneuver around on the on both windows then . grad e: But I think it 's the semantics that are that are questionable to me , that you delete something So let 's say someone is talking to here , and then you have a little segment here . Well , is that part of the speech ? Is it part of the nonspeech ?  , w what do you embed it in ? phd d: There 's something nice , though , about keeping , and this is probably another discussion , keeping the stuff that Thilo 's detector detected as possible speech and just marking it as not speech than deleting it . Because then when you align it , then the alignment can you can put a reject model or whatever , grad e: Oh , I see . So then they could just like put Oh that 's what you meant by just put an " X " there . phd d: and you 're consistent with th the automatic system , grad e: that 's an interesting idea . phd d: whereas if you delete it grad e: So so all they So that all they would have to do is put like an " X " there . phd d: Yeah , or some , you know , dummy reject mod grad e: So blank for blank for silence , " S " " S " for speech , " X " " X " for something else . That 's actually a better way to do it cuz the a the forced alignment will probably be more consistent than postdoc a: Well , like , I think there 's a complication which is that that you can have speech and noise in s phd d:  if it 's just as easy , but postdoc a: you know , on the same channel , the same speaker , so now sometimes you get a ni microphone pop and , there 're these fuzzy hybrid cases , and then the problem with the boundaries that have to be shifted around . phd d: Anyway , quick question , though , at a high level do people think , let 's just say that we 're moving to this new era of like using the , pre - segmented t you know , non - synchronous conversations , does it make sense to try to take what we have now , which are the ones that , you know , we have recognition on which are synchronous and not time - tightened , and try to get something out of those for sort of purposes of illustrating the structure and the nature of the meetings , or is it better to just , you know , forget that and tr  , it 's grad e: Well , I think we 'll have to , eventually . grad e: But if we can't phd d: And maybe we can for the non - lapel , but grad e: But if we can't , then maybe we just have to phd d: is it worth if we can't then we can fake it even if we 're we report , you know , we 're wrong twenty percent of the time or ten percent of the time . grad e: Well , I 'm thinking are you talking about for a paper , or are talking about for the corpus . grad e:  cuz for the corpus it would be nice if everything were phd d: Actually that 's a good question because we 'd have to completely redo those meetings , and we have like ten of them now . grad e: We wouldn't have to re - do them , we would just have to edit them . postdoc a: Well , and also , I still haven't I still haven't given up on forced alignment . phd d: No , you 're right , actually postdoc a: I think that when Brian comes , this 'll be  an interesting aspect to ask him as well b grad e: When postdoc a: when Brian Kingsbury comes . And it 's like , " Who 's Ryan ? " postdoc a: Yeah , good question . phd d: no , that 's a good point , though , because for feature extraction like for prosody or something , the meetings we have now , it 's a good chunk of data grad e: Yep . postdoc a: That 's what my hope has been , phd d: So we should at least try it even if we can't , postdoc a: and that 's what that 's what you know , ever since the the February meeting that I transcribed from last year , forced alignment has been on the on the table as a way of cleaning them up later . phd d: right ? grad e: On the table , right ? postdoc a: And and so I 'm hopeful that that 's possible . I know that there 's complication in the overlap sections and with the lapel mikes , phd f: There 's postdoc a: but phd f: Yeah . phd d: we might be able , at the very worst , we can get transcribers to correct the cases where  , you sort of have a good estimate where these places are because the recognition 's so poor . Right ? phd b: Yeah , we were never just gonna go with these as the final alignments . phd d: So we need some way to push these first chunk of meetings into a state where we get good alignments . phd f: I 'm probably going to spend another day or so trying to improve things by , by using , acoustic adaptation .  , the Right now I 'm using the unadapted models for the forced alignments , and it 's possible that you get considerably better results if you , manage to adapt the , phone models to the speaker and the reject model to the to to all the other speech .  , so phd b: Could you could you at the same time adapt the reject model to the speech from all the other channels ? professor c: That 's what he just said . phd b: Oh , not just the speech from that of the other people from that channel , phd f: Right . I don't think that would work , phd f: No , it grad e: right ? Because you 'd A lot of it 's dominated by channel properties . phd d: But what you do wanna do is take the , even if it 's klugey , take the segments the synchronous segments , the ones from the HLT paper , where only that speaker was talking . phd f: So you want to u phd d: Use those for adaptation , cuz if you if you use everything , then you get all the cross - talk in the adaptation , and it 's just sort of blurred . I thought it was higher than that , that 's pr phd d: It really it depends a lot . professor c: Well I know what we 're not turning in to Eurospeech , a redo of the HLT paper . professor c: That I don't wanna do that , grad e: Yeah , I 'm doing that for AVIOS . phd d: I think Morgan 's talk went very well it woke postdoc a: Excellent . phd d: you know , it was really a well presented and got people laughing postdoc a: Yeah . phd f: Some good jokes in it ? grad e: Especially the batteried meter popping up , phd d: Yeah . professor c: You know , that wa that was the battery meter saying that it was fully charged , grad e: It 's full . grad e: He he he was onto the bullet points about talking about the you know the little hand - held , and trying to get lower power and so on , phd f: Po - low power grad e: and Microsoft pops up a little window saying " Your batteries are now fully charged . grad e: I 'm thinking about scripting that for my talk , you know , put put a little script in there to say " Your batteries are low " right when I 'm saying that . No  , i in in your case , you were joking about it , but , your case the fact that your talking about similar things at a couple of conferences , it 's not these are conferences that have d really different emphases . Whereas HLT and and Eurospeech , pretty pretty pretty similar , so I I I can't see really just putting in the same thing , grad e: Are too close , yeah . phd d: No , I d I don't think that paper is really professor c: but phd d: the HLT paper is really more of a introduction - to - the - project paper , and ,  professor c: Yeah . phd d: Well , yeah , it it 's probably wouldn't make sense , professor c: Or some or some  , I would see Eurospeech if we have some Eurospeech papers , these will be paper p p  , submissions . phd d: but professor c: These will be things that are particular things , aspects of it that we 're looking at , rather than , you know , attempt at a global paper about it . I had , one of the transcribers go through and tighten up the bins on one of the , NSA meetings , and then I went through afterwards and double - checked it so that one is really very very accurate . I sent You know that one ? phd d: Oh , so grad g: The which one ? I 'm sorry . postdoc a: I 'm trying to remember I don't remember the number off hand . grad e: Those are all postdoc a: It 's one of the NSA 's . postdoc a: I 'm sure that that one 's accurate , I 've been through it myself . phd f: oh , Darn ! grad g: Yeah , that 's the problem with the NSA speakers . phd d: And e and e and extremely hard to follow , like word - wise , grad e: So . phd d: I bet the transcri  , I have no idea what they 're talking about , grad g: Yeah . phd d: so , postdoc a: I 'm sure that , they 're they 're accurate now . grad e: oh , before you l go I guess it 's alright for you to talk a little without the mike I noticed you adjusting the mike a lot , did it not fit you well ? Oh . postdoc a: Well I won I noticed when you turned your head , it would it would tilt . grad e: Maybe it wasn't just tightened enough , or phd d: Maybe the yeah , the s thing that you have tightened @ @ , phd b: Actually if if you have a larger head , that mike 's gotta go farther away which means the the balance is gonna make it wanna tip down . grad e: Cuz , I 'm just thinking , you know , we were we 're we 've been talking about changing the mikes , for a while , grad g:  postdoc a: Yeah . grad e: and if these aren't acoustically they seem really good , but if they 're not comfortable , we have the same problems we have with these stupid things . postdoc a: I think it 's com This is the first time I 've worn this , I find it very comfortable . grad e: I find it very comfortable too , but , it looked like Andreas was having problems , and I think Morgan was saying it professor c: Well , but I had it on I had it on this morning and it was fine . phd b: Can I see that ? grad e: Oh , oh you did wear it this morning ? professor c: Yeah . phd b: I yeah , I don't want it on , I just I just want to , say what I think is a problem with this . If you are wearing this over your ears and you 've got it all the way out here , then the balance is gonna want to pull it this way . phd b: Where as if somebody with a smaller head has it back here , grad e: It 's more balanced . postdoc a: Oh ! phd b: Then it then it falls back this way so it 's phd d: So we have to grad e: Well wh what it 's supposed to do is the backstrap is supposed to be under your crown , and so that should be should be postdoc a: Ah . grad e: if it 's right against your head there , which is what it 's supposed to be , that balances it so it doesn't slide up . grad e: Yep , right right below if you feel the back of your head , you feel a little lump , phd b: Yeah . phd d: So I 'm not saying anything about bias towards small headsize , grad e: About heads ? phd d: but does seem ,  phd b: It would be an advantage . postdoc a: Well , wonder if it 's if if he was wearing it over his hair instead of under his hair . professor c: Well , we should We shou we should work on compressing the heads , and grad e: I think probably it was Yeah .  , so the directions do talk about bending it to your size , which is not really what we want . phd b: The other thing that would do it would be to hang a five pound weight off the back . professor c: that 's good ! postdoc a: What did you say ? phd d: A little , grad e: wh professor c: Hang a five pound weight off the off the back . grad e: We at Boeing I used I was doing augmented reality so they had head - mounts on , and we we had a little jury - rigged one with a welder 's helmet , phd b: Counter - balance . grad e: and we had just a bag with a bunch of marbles in it as a counter - balance . If people those who talk a lot have to wear heavier weights or something , and grad e: Yeah ! professor c: and  , grad e: Anyway . professor c: so , what was I gonna say ? Oh , yeah , I was gonna say , I had these , conversations with NIST folks also while I was there and and , so they they have their their plan for a room , with , mikes in the middle of the table , and , close - mounted mikes , grad e: Yep . professor c: and they 're talking about close - mounted and lapels , just cuz phd d: And arrays , professor c: sort of and the array . grad e: And arrays , professor c: Yeah , so they were phd d: which is the i interesting grad e: yep . professor c: And yeah , like multiple multiple video cameras coverin covering every everybody every place in the room , phd d: and video , right . professor c: the yeah the the mikes in the middle , the head - mounted mikes , the lapel mikes , the array , with well , there 's some discussion of fifty - nine , grad e: Fifty - nine elements . professor c: they might go down to fifty - seven Because , there is , some pressure from a couple people at the meeting for them to use a KEMAR head . professor c: but what it is is it 's dummy head that is very specially designed , grad e: Oh , that 's right . professor c: and and and , so what they 're actually doing is they 're really there 's really two recording systems . professor c: So they may not be precisely synchronous , but the but there 's two two recording systems , one with , I think , twenty - four channels , and one with sixty - four channels . And the sixty - four channel one is for the array , but they 've got some empty channels there , and anyway they like they 're saying they may give up a couple or something if for for the KEMAR head if they go go with that . grad e: Yeah , h  , J Jonathan Fiscus did say that , they have lots of software for doing calibration for skew and offset between channels phd d:  -  grad e: and that they 've found that 's just not a big deal . I was thinking phd d: But they 're still planning to do like fake grad e: Scenario - based . grad e: But it sounded like they were pretty well thought out phd d: Yeah , th that 's true . grad e: and they 're they 're gonna be real meetings , postdoc a:   grad e: it 's just that they 're with str with people who would not be meeting otherwise . phd b: Did did they give a talk on this or was this informal ? phd d:   phd b: What was the ,  the paper by , Lori Lamel that you mentioned ? professor c: yeah , we sh we should just have you have you read it , but , I mea ba i i  , we 've all got these little proceedings , postdoc a: Mmm , yeah . professor c: but , basically , it was about , going to a new task where you have insufficient data and using using data from something else , and adapting , and how well that works .  , so in in fact it was pretty related to what Liz and Andreas did , except that this was not with meeting stuff , it was with grad e: Right . professor c: like I think they s didn't they start off with Broadcast News system ? And then they went to grad e: The - their Broadcast News was their acoustic models and then all the other tasks were much simpler . phd b: What was their rough what was their conclusion ? grad e: Yeah , read Wall Street Journal . phd d: Well , it 's it 's a good paper ,  professor c: Yeah , yeah . phd d: Bring the grad e: That It not only works , in some cases it was better , which I thought was pretty interesting , but that 's cuz they didn't control for parameters . phd b: Did they ever try going going the other direction from simpler task to more complicated tasks , grad e: acoustic models were a lot more complex . grad e: Yeah , well , one of the big problems with that is is often the simpler task isn't fully doesn't have all the phones in it , professor c: Yeah . grad e: like for the spr speech proxy thing that I did ? That 's what I did . Yeah , and they have  they have better adaptation than we had than that that system , grad e: Yep . professor c: yeah , we should probably what would actually what we should do , I haven't said anything about this , but probably the five of us should pick out a paper or two that that , you know , got our interest , and we should go around the room at one of the Tuesday lunch meetings and say , you know , what what was good about the conference , grad e: Present . phd d: Well , the summarization stuff was interesting , I don't know anything about that field , but for this proposal on meeting summarization , it 's sort of a far cry because they weren't working with meeting type data , but he got sort of an overview on some of the different approaches , grad e: Right . phd b: Do you remember who the groups were that we 're doing ? phd d: so . phd d: but , there 's that 's a huge field and probably the groups there may not be representative of the field , I I don't know exactly that everyone submits to this particular conference , phd b: Was were there folks from BBN presenting ? phd d: but yet there was , let 's see , this was on the last day , Mitre , BBN , and , Prager grad e: Mitre , BBN , IBM . phd d: no it was grad e: Wasn't Who who who did the order one ? phd d: this was Wednesday morning . The sentence ordering one , was that Barselou , and these guys ? grad e: Ugh ! I 'm just so bad at that . phd d: Anyway , I I it 's in the program , I should have read it to remind myself , but that 's sort of useful and I think like when Mari and Katrin and Jeff are here it 'd be good to figure out some kinds of things that we can start doing maybe just on the transcripts cuz we already have professor c:   postdoc a: Well , I like the idea that Adam had of of , z maybe generating minutes based on some of these things that we have because it would be easy to to to do that just , you know , and and phd d: Right . postdoc a: it has to be , though , someone from this group because of the technical nature of the thing . grad e: Someone who actually does take notes , I 'm very bad at note - taking . phd d: But I think what 's interesting is there 's all these different evaluations , like just , you know , how do you evaluate whether the summary is good or not , grad e: I always write down the wrong things . phd d: and that 's what 's was sort of interesting to me is that there 's different ways to do it , grad e: A judge . phd d: and phd b: Was SRA one of the groups talking about summarization , no ? grad e: Yep . One of those w grad e: And as I said , I like the Microsoft talk on scaling issues in , word sense disambiguation , phd d: Yeah . professor c: Yeah , that was an interesting discussion , grad e: The professor c: I grad e: It it it was the only one It was the only one that had any sort of real disagreement about . phd d: The data issue comes up all the ti professor c: Well , I didn't have as much disagreement as I would have liked , grad e: So . professor c: but I didn't wanna I wouldn I didn't wanna get into it because , you know , it was the application was one I didn't know anything about , grad e: Yep . professor c: it just would have been , you know , me getting up to be argumentative , but but , the missing thi so so what they were saying it 's one of these things is you know , all you need is more data , sort of But I mea i wh it @ @ that 's that 's dissing it , improperly , it was a nice study .  , they were doing this it wasn't word - sense disambiguation , it was phd d: Yeah yeah yeah grad e: Well , it sort of was . grad e: But it was it was a very simple case of " to " versus " too " versus " two " and " there " , " their " , " they 're " phd d: And there and their and professor c: Yeah , yeah . phd d: and that you could do better with more data , that 's clearly statistically professor c: Right . professor c: And so , what they did was they had these different kinds of learning machines , and they had different amounts of data , and so they did like , you know , eight different methods that everybody , you know , argues about about , " Oh my my kind of learning machine is better than your kind of learning machine . " And , they were started off with a million words that they used , which was evidently a number that a lot of people doing that particular kind of task had been using . And then they had this log scale showing a you know , and and naturally everything gets grad e: Them being beep , they went off to a billion . professor c: they well , it 's a big company , I didn't I didn't mean it as a ne anything negative , grad e: Yeah . professor c: but i i i phd d: You mean the bigger the company the more words they use for training ? grad e: Well , I think the reason they can do that , is that they assumed that text that they get off the web , like from Wall Street Journal , is correct , and edit it . Of course there was the kind of effect that , you know , one would expect that  that you got better and better performance with more and more data .  , but the the real point was that the the different learning machines are sort of all over the place , and and by by going up significantly in data you can have much bigger effect then by switching learning machines and furthermore which learning machine was on top kind of depended on where you were in this picture , so , phd b: This was my concern about the recognizer in Aurora . professor c: That phd b: That the differences we 're seeing in the front - end is b professor c: Yeah . phd d: If you add more data ? Or phd b: You know ? professor c: Yeah . So so , that was that was kind of , you know , it 's a good point , but the problem I had with it was that the implications out of this was that , the kind of choices you make about learning machines were therefore irrelevant which is not at n t as for as I know in in tasks I 'm more familiar with @ @ is not at all true . What i what is is true is that different learning machines have different properties , and you wanna know what those properties are . And someone else sort of implied that well we s you know , a all the study of learning machine we still don't know what those properties are . We don't know them perfectly , but we know that some kinds use more memory and and some other kinds use more computation and some are are hav have limited kind of discrimination , but are just easy to use , and others are phd b: But doesn't their conclusion just sort of you could have guessed that before they even started ? Because if you assume that these learning things get better and better and better , professor c: You would guess phd b: then as you approach there 's a point where you can't get any better , right ? You get everything right . phd d: It 's just no grad e: But phd b: So they 're all approaching . phd b: But what I 'm saying is that th they have to , as they all get better , they have to get closer together . phd b: But they 're all going the same way , right ? So you have to get closer . phd b: Oh they didn't ? professor c: Well grad e: They just switched position . professor c: well that 's getting cl  , yeah , the spread was still pretty wide that 's th that 's true , grad e: Yep . professor c: but but , I think it would be irntu intu intuition that this would be the case , but , to really see it and to have the intuition is quite different , I think somebody w w let 's see who was talking about earlier that the effect of having a lot more data is quite different in Switchboard than it is in in Broadcast News , phd d: Well it 's different for different tasks . phd d: So it depends a lot on whether , you know , it disambiguation is exactly the case where more data is better , right ? You 're you 're you can assume similar distributions , professor c: Yeah . phd d: but if you wanted to do disambiguation on a different type of , test data then your training data , then that extra data wouldn't generalize , grad e: Right . w  , I think one of them was that " Well , maybe simpler algorithms and more data are is better " . Right ? Because their simplest , most brain - dead algorithm did pretty darn well professor c:   Why are you sticking with a million words ? "  , their point was that this million - word corpus that everyone uses is apparently ten or fifteen years old . But anyway , I I I think it 's it 's just the the i it 's it 's it 's not really the conclusion they came to so much , as the conclusion that some of the , commenters in the crowd came up with grad e: But we could talk about this stuff , I think this would be fun to do . professor c: that , you know , this therefore is further evidence that , you know , more data is really all you should care about , and that I thought was just kind of going too far the other way , grad e: Machine - learning . professor c: and and the the , one one person ga g g got up and made a a brief defense , but it was a different kind of grounds , it was that that , i w the reason people were not using so much data before was not because they were stupid or didn't realize data was important , but in fact th they didn't have it available .  , but the other point to make a again is that , machine learning still does matter , but it it matters more in some situations than in others , and it and also there 's there 's not just mattering or not mattering , but there 's mattering in different ways .  , you might be in some situation where you care how much memory you 're using , or you care , you know , what recall time is , grad e: Right . professor c: or you care , you know , and and grad e: Or you only have a million words for your some new task . professor c: Yeah , or or ,  phd d: Or done another language , or  , you so there 's papers on portability and rapid prototyping and blah - blah - blah , grad e: Yep . " professor c: And there 's cost ! phd d: So , these are like two different religions , basically . professor c: you know , so so these ,  th the in the in the speech side , the thing that @ @ always occurs to me is that if you if you  one person has a system that requires ten thousand hours to train on , and the other only requires a hundred , and they both do about the same because the hundred hour one was smarter , that 's that 's gonna be better . because people , there isn't gonna be just one system that people train on grad e: Yep . She put this up , and it was like this is this p people kept saying , " Can I see that slide again ? " professor c: Yeah . phd d: Yeah , postdoc a: and then they 'd make a comment , and one person said , well - known person said , you know , " Before you dismiss forty - five years including my work " phd d: yeah . phd d: But th you know , the same thing has happened in computational linguistics , right ? You look at the ACL papers coming out , and now there 's sort of a turn back towards , OK we 've learned statistic you know , we 're basically getting what we expect out of some statistical methods , and , you know , the there 's arguments on both sides , grad e: Yep . grad e: Is that all all of them are based on all the others , right ? Just , you you can't say phd b: Maybe they should have said " focus " or something . And I 'm saying the same thing happened with speech recognition , right ? For a long time people were hand - c coding linguistic rules and then they discovered machine - learning worked better . And now they 're throwing more and more data and worrying perhaps worrying less and less about , the exact details of the algorithms . phd d: And and then you hit this grad e: Except when they have a Eurospeech paper . grad e: Shall we read some digits ? Are we gonna do one at a time ? Or should we read them all agai at once again