E: That 's an interesting idea .
E: Hey !
A: We 've we 've thought about doing that
A: but the recognition quality is gonna be horrendous .
D: Well , a couple things .
D: First of all the time marking you 'd get you could get by a tool .
B: Wow .
B: That 's true .
D: And so if the if if the issue really
E: That 's interesting .
D: uh , I 'm think about the close caption that you see running by on on live news casts .
D: You know , yo
A: Most of those are done by a person .
E: Yeah ,
D: I know I know that .
E: I
D: No , I understand .
D: And in a lot of them you see typos and things like that ,
A: Mm - hmm .
D: but it but it occurs to me that it may be a lot easier to correct things than it is to do things from scratch , no matter how wonderful the tool is .
A: Yeah .
A: Yeah , we D: But if if there was a way to merge the two C: Well , I mean , but sometimes it 's easier to type out something instead of going through and figuring out which is the right E: That 'd be fun .
A: I mean , we 've talked about it
A: but C: I mean , it depends on the error rate ,
C: right ?
D: Well s but but again the timing is for fr should be for free .
D: The timing should be C: But we don't care about the timing of the words .
D: Well I thought you just that 's said that was a critical issue .
E: No ,
A: We don't care about the timing of the words , just of the utterances .
E: uh the the boundary C: We cut it s s
B: We don't we don't know , actually .
E: boundary .
B: We haven't decided which which time we care about ,
B: and that 's kind of one of the things that you 're saying , is like  you have the option to put in more or less timing data B: and , uh , be in the absence of more specific instructions , we 're trying to figure out what the most convenient thing to do is .
A: Yeah ,
A: so so what what she 's done so far , is sort of more or less breath g not breath groups , <comment> sort of phrases , continuous phrases .
B: Yeah .
A: And so , um , that 's nice because you you separate when you do an extract , you get a little silence on either end .
A: So that seems to work really well .
E: That 's ideal .
A: Um .
E: Although I was I you know , the alternative , which I was sort of experimenting with before I ran out of time , recently was , um  that , you know , ev if it were like an arbitrary segment of time i t pre - marked
B: Yeah .
E: cuz it does take time to put those markings in .
B: Yeah .
E: It 's really the i the interface is wonderful
E: because , you know , the time it takes is you listen to it , and then you press the return key .
E: But then , you know , it 's like , uh , you press the tab key to stop the flow and  and , uh , the return key to p to put in a marking of the boundary .
E: But , you know , obviously there 's a lag between when you hear it and when you can press the return key
B: Yeah .
E: so it 's slightly delayed ,
E: so then you you listen to it a second time and move it over to here .
E: So that takes time .
D: a
D: i a
E: Now if it could all be pre - marked at some , l you know , good D: ar but
D: Are are those d delays adjustable ?
A: Hmm .
D: Those delays adjustable ?
D: See a lot of people who actually build stuff with human computer interfaces understand that delay ,
D: and and so when you by the time you click it it 'll be right on because it 'll go back in time to put the B: Yeah .
E: Yeah .
E: Yeah ,
E: uh , not in this case .
B: It could do that
B: couldn't it .
A: We could program that pretty easily ,
E: It has other A: couldn't we Dan ?
A: Yeah , mis Mister TCL ?
B: Yeah .
E: Oh , interesting point .
B: I would have thought so ,
B: yeah .
E: Ah ! <comment> Interesting point .
E: OK , that would make a difference .
B: Mmm .
A: But , um E: I mean , it 's not bad
A: But , if we tried to do automatic speaker ID .
E: but it does take twice .
A: I mean , cuz primarily the markings are at speaker change .
B: Yeah , yeah ,
B: but B: But we 've got we 've got the most channel data .
A: But that would be B: We 'd have to do it from your signal .
B: Right .
E: Oh , good point !
B: I mean , we 've we 've got we 've got a lot of data .
E: Ah !
E: We 've got volume .
A: Yeah , I guess the question is how much time will it really save us versus the time to write all the tools to do it .
B: Right .
B: but the chances are if we if we 're talking about collecting ten or a hundred hours , which is going to take a hundred or a thousand hours to transcribe D: If A: But D: if we can go from ten X to five X we 're doing a big A: We 're gonna need we 're gonna need ten to a hundred hours to train the tools , and validate the tools the do the d to to do all this anyway .
C: Right .
C: So maybe E: Wow .
E: But but it op
B: If we 're just doing silence detection A: I knew you were gonna do that .
A: Just saw it coming .
E: I 'm sorry .
E: I wish you had told me E: wish you 'd told me .
D: Put put it on your sweater .
E: At what part ?
E: OK ,
E: I 'm alright .
B: Um , i it seems like B: Well ,
B: uh , I don't know .
B: Yeah .
B: I mean , it it 's it 's maybe like  a week 's work to get to do something like this .
B: So forty or fifty hours .
E: Could you get it so that with so it would it would detect volume on a channel and insert a marker ?
C: Right .
E: And the the format 's really transparent .
B: Sure .
C: Yeah .
E: It 's just a matter of a very c clear E: it 's XML , isn't it ?
A: Mm - hmm .
E: It 's very I mean , I looked at the the file format and it 's just it has a t a time  a time indication and then something or other , and then an end time or something or other .
C: So
C: maybe maybe we could try the following experiment .
C: Take the data that you 've already transcribed
E: Mm - hmm .
C: and D: Is this already in the past or already in the future ?
C: Already in the past .
D: You 've already you 've already done some ?
C: She she 's done one she 's one E: Yes I have .
A: She 's she 's done about half a meeting .
D: Oh - Oh , I see .
C: Right .
D: OK ,
C: Right .
D: good .
A: Right ?
A: About half ?
C: I 'm go
E: S I 'm not sure if it 's that 's much
E: but anyway , enough to work with .
C: Right .
B: Several minutes .
C: Um ,
C: and and throw out the words ,
C: but keep the time markings .
C: And then go through I mean , and go through and and try and re - transcribe it , given that we had perfect boundary detection .
E: OK .
E: Good idea .
C: And see if it see if it see if it feels easier to you .
E: OK .
D: And forgetting all the words because you 've been thr
E: Yeah , that 's what I was thinking .
E: I 'd I 'd be cheating a little bit g with familiarity effect .
C: Yeah , I mean uh , that 's part of the problem is , is that what we really need is somebody else to come along .
B: Well , no , you should do it you should do it  Do it again from scratch
B: and then do it again at the boundaries .
B: So you do the whole thing three times
B: and then we get C: Yeah .
E: No .
E: Now , there 's a plan .
D: And then then w since we need some statistics do it three more .
E: OK .
D: And so you 'll get you 'll get down to one point two X by the time you get done .
E: Oh , yeah .
E: I 'll do that tomorrow .
E: I should have it finished by the end of the day .
D: No , but the thing is the fact that she 's she 's did it before just might give a lower bound .
D: That 's all .
E: Exactly .
D: Uh , which is fine .
B: Yeah .
C: Right .
D: It 's E: Yeah .
D: And if the lower bound is nine X then w it 's a waste of time .
C: Right .
E: Well , uh but there 's an extra problem which is that I didn't really keep accurate E: uh , it wasn't a pure task the first time ,
B: Oh !
E: so  uh , it 's gonna be an upper bound in in that case .
B: Yeah .
E: And it 's not really strictly comparable .
E: So I think though it 's a good proposal to be used on a new a new batch of text that I haven't yet done yet in the same meeting . Could use it on the next segment of the text .
B: The point we where do we get the the the oracle boundaries from ?
C: Right .
B: Or the boundaries .
A: Yeah , one person would have to assign the boundaries and the and the other person would have to E: Well , but couldn't I do it for the next B: We we we could get fake E: Oh , I see what you mean .
A: I mean that 's easy enough .
A: I could do that .
E: Well , but the oracle boundaries would come from volume on a partic specific channel wouldn't they ?
B: That would be the automatic boundaries .
C: No , no , no , no .
A: No , no .
C: You wanna know given Given a perfect human segmentation , I mean , you wanna know how well E: Yeah .
C: I mean , the the question is , is it worth giving you the segmentation ?
E: Oh , I see what you mean .
E: Yeah .
A: I mean , that that 's easy enough .
C: Right .
A: I could generate the segmentation and and you could do the words , and time yourself on it .
D: A little double - blind - ear kind of thing .
A: So .
A: Yep .
E: I see .
E: OK .
A: So it that might be worth doing .
E: That 's good .
E: I like that .
A: That would at least tell us whether it 's worth spending a week or two trying to get a tool , that will compute the segmentations .
D: And the thing to keep in mind too about this tool , guys is that sure , you can do the computation for what we 're gonna do in the future
C: Right .
D: but if if UW 's talking about doing two , or three , or five times as much stuff and they can use the same tool , then obviously there 's a real multiplier there .
A: Right .
E: And the other thing too is with with speaker identification ,
E: if if that could handle speaker identification that 's a big deal .
B: Well it w
D: Well , use it .
C: Right .
E: OK .
D: Yeah , that 's why we s bought the expensive microphones .
E: Yeah , I mean , that 's a nice feature .
A: Yep .
B: Yeah , yeah .
E: That 's a major that 's like , one of the two things that C: I mean , there 's gonna there 's gonna be in the meeting , like the reading group meeting that we had the other day , that 's it 's gonna be a bit of a problem
B: OK .
C: because , like , I wasn't wearing a microphone
C: f and there were other people that weren't wearing microphones .
B: Yes .
D: But you didn't say anything worth while anyway , right ?
A: That B: Right .
A: That 'll s
E: Yeah .
C: That 's pretty much true
C: but A: it might save ninety percent of the work though .
C: but , yes .
A: So .
B: So I I need to we need to look at what what the final output is
B: but it seems like we it doesn't it seems like it 's not really not that hard to have an automatic tool to generate the phrase marks , and the speaker , and speaker identity without putting in the words .
A: Yeah .
A: I 've already become pretty familiar with the format ,
E: That 'd be so great .
A: so
E: Yeah .
B: Mm - hmm .
A: it would be easy .
E: Yeah .
E: We didn't finish the the part of work already completed on this ,
A: If you 'd tell me where it is , huh ?
E: did we ?
E: I mean , you  you talked a little bit about the transcription conventions ,
A: Mm - hmm .
E: and , I guess you 've mentioned in your progress report , or status report , that you had written a script to convert it into E: So , I when I i the it 's quickest for me in terms of the transcription part to say something like , you know , if  if Adam spoke to , um to just say , " A colon " ,
E: Like who could be , you know , I mean at the beginning of the line .
B: Mmm .
E: and E colon instead of entering the interface for speaker identification and clicking on the thing , uh , indicating the speaker ID .
E: So , and then he has a script that will convert it into the the thing that , uh , would indicate speaker ID .
E: If that 's clear .
A: It 's pretty cute .
C: OK .
A: But at any rate .
E: It 's Perl script .
A: So , um ,
A: Right .
A: So so I think the guess at ten X seems to be pretty standard .
A: Everyone more or less everyone you talk to says about ten times for hard technical transcription .
E: Mm - hmm .
E: Yeah .
D: Using wh using stone age
D: using stone age tools .
E: That 's right .
A: Using using stone age tools .
E: Yeah , well that 's true ,
A: I mean , I looked at Cyber Transcriber
E: but A: which is a service that you send an audio file ,
A: they do a first - pass speech recognition .
A: And then they they do a clean up .
A: But it 's gonna be horrible .
A: They 're never gonna be able to do a meeting like this .
E: What i just approximately , what did you find out in terms of price or or whatever ?
D: Right .
B: No .
A: Well , for Cyber Transcriber they don't quote a price .
A: They want you to call and and talk .
A: So for other services , um , they were about thirty dollars an hour .
E: Of of tape ?
E: Or of action ?
A: Thirty A: So , yeah .
A: For thirty dollars an hour for of their work .
E: OK .
E: OK .
E: Oh , of their E: Oh !
A: So so if it 's ten times it 's three hundred dollars an hour .
C: So that 's three that 's three hours .
D: D did you talk to anybody that does closed captioning for for uh , TV ?
E: OK .
C: Right .
A: No .
D: Cuz they a usually at the end of the show they 'll tell what the name of the company is , the captioning company that 's doing it .
A: Mm - hmm .
E: Interesting .
A: Yeah ,
A: so so my my search was pretty cursory .
A: It was just a net search .
A: And , uh , so it was only people who have web pages and are doing stuff through that .
D: Well , you know , the the thing the thing about this is thinking kind of , maybe a little more globally than I should here
D: but <comment> that really this could be a big contribution we could make .
D: Uh , I mean , we 've been through the STP thing ,
D: we know what it what it 's like to to manage the manage the process ,
D: and admittedly they might have been looking for more detail than what we 're looking for here
D: but it was a it was a big hassle ,
D: right ?
D: I mean , uh , you know , they they constantly could 've reminding people and going over it .
B: Yeah .
D: And clearly some new stuff needs to be done here .
D: And it 's it 's only our time , where " our " of course includes Dan , Dan and you guys .
D: It doesn't include me at all .
D: Uh .
D: j Just seems like B: Yeah , I mean I don't know if we 'd be able to do any thing f to help STP type problems .
B: But certainly for this problem we can do a lot better than D: Bec
D: Why ?
D: Because they wanted a lot more detail ?
B: No .
A: Right .
B: Because they had because they only had two speakers , right ?
B: I mean , the the segmentation problem is D: Only had two .
A: Trivial .
A: They had two speakers over the telephone .
D: Oh , I see .
D: So what took them so long ?
A: Um , mostly because they were doing much lower level time .
B: Yeah .
A: So they were doing phone and syllable transcription , as well as , uh , word transcription .
D: Right .
D: Right .
C: Right .
E: Mm - hmm .
A: And so we 're w we decided early on that we were not gonna do that .
D: I see .
D: But there 's still the same issue of managing the process , of  of reviewing and keeping the files straight , and all this stuff , that which is clearly a hassle .
A: Yep .
B: Yeah .
A: Right .
A: And so so what I 'm saying is that if we hire an external service I think we can expect three hundred dollars an hour .
B: Yeah .
A: I think that 's the ball park .
A: There were several different companies that  and the the range was very tight for technical documents .
A: Twenty - eight to thirty - two dollars an hour .
C: And who
C: who knows if they 're gonna be able to m manage multal multiple channel data ?
B: Yeah , they won't .
A: They won't .
B: They w they 'll refuse to do it .
E: Mm - hmm .
A: We 'll have to mix them .
C: Right .
B: Yeah .
E: And then there 's the problem also that B: No , but I mean , they they they won't they won't they will refuse to transcribe this kind of material .
B: That 's not what they 're d quoting for , right ?
D: Well , they might  they might quote it A: Yes , it is .
B: For quoting meetings ?
A: Sev - several of them say that they 'll do meetings , and conferences , and s and so on .
B: Wow .
A: None of them specifically said that they would do speaker ID , or speaker change mark .
B: Yeah .
D: Th - th the th there may be just multiplier for five people costs twice as much and for ten people co <comment> Something like that .
A: They all just said transcription .
B: Yeah , yeah , yeah .
A: Well , the the way it worked is it it was scaled .
A: So what they had is , if it 's an easy task it costs twenty - four dollars an hour
A: and it will take maybe five or six times real time .
A: And what they said is for the hardest tasks , bad acoustics , meeting settings , it 's thirty - two dollars an hour
A: and it takes about ten times real time .
B: I see .
A: So I think that we can count on that being about what they would do .
B: Yeah .
E: Yeah .
A: It would probably be a little more
B: Right .
A: because we 're gonna want them to do speaker marking .
D: A lot of companies I 've worked for y the , uh the person leading the meeting , the executive or whatever , would sort of go around the room and and mentally calculate h how many dollars per hour this meeting was costing ,
A: So .
D: right ?
D: In university atmosphere you get a little different thing .
D: But
D: you know , it 's a lot like , " he 's worth fifty an hour ,
D: he 's worth "
D: And so he so here we 're thinking , " well let 's see , if the meeting goes another hour it 's going to be another thousand dollars . "
D: You know ?
D: It 's A: Yep ,
D: So ch So every everybody ta Talk really fast .
A: we have to have a short meeting .
E: That 's very interesting .
A: Stop talking !
B: Yeah .
D: Let 's get it over with .
E: Talk slowly but with few words .
A: And clearly .
D: And
B: That 's right .
D: only talk when you 're pointed to .
E: There you go .
A: Content words only .
E: We could have some telegraphic meetings .
E: That might be interesting .
B: Yeah , it 'd be cheap .
B: Cheap to transcribe .
A: So .
A: But at any rate , so we we have a ballpark on how much it would cost if we send it out .
D: And we 're talking about do doing how many hours worth of meetings ?
A: Thirty or forty .