phd a: Why is it so cold in here ? professor b: so , we haven't sent around the agenda . So , i  , any agenda items anybody has , wants to talk about , what 's going on ? postdoc g: I c I could talk about the meeting .  phd a: Well , I had a just a quick question but I know there was discussion of it at a previous meeting that I missed , but just about the the wish list item of getting good quality close - talking mikes on every speaker . professor b: OK , so let 's let 's So let 's just do agenda building right now . You were gonna starting to say something ? postdoc g: Well , you you , already know about the meeting that 's coming up and I don't know if if this is appropriate for this . phd e: What meeting ? professor b: We can so we can ta so n NIST is NIST folks are coming by next week postdoc g: OK . professor b: I think phd e: Who 's coming ? professor b: John Fiscus postdoc g: An - anything else , strike anybody ? phd a: we started running recognition on one conversation but it 's the r isn't working yet . phd e: Wha phd a: the main thing would be if anyone has , knowledge about ways to , post - process the wave forms that would give us better recognition , that would be helpful to know about . professor b: Yeah , so ,  phd e: What about , is there anything new with the speech , nonspeech stuff ? phd c: Yeah , we 're working more on it but , it 's not finished . postdoc g: I had thought under my topic that I would mention the , four items that I I , put out for being on the agenda f on that meeting , which includes like the pre - segmentation and the and the developments in multitrans . professor b: Alright , why don't we start off with this , u u I guess the order we brought them up seems fine . So the one issue was that the the , lapel mike , isn't as good as you would like . Right ? phd a: Ri -  , professor b: Is that is that basically the point ? phd a: yeah , the And actually in addition to that , that the the close talking mikes are worn in such a way as to best capture the signal . And the reason here is just that for the people doing work not on microphones but on sort of like dialogue and so forth ,  or and even on prosody , which Don is gonna be working on soon , it adds this extra , you know , vari variable for each speaker to to deal with when the microphones aren't similar . phd a: So And I also talked to Mari this morning and she also had a strong preference for doing that . And in fact she said that that 's useful for them to know in starting to collect their data too . Right , so one th grad h: Well , so professor b: well one thing I was gonna say was that , i we could get more , of the head mounted microphones even beyond the number of radio channels we have because I think whether it 's radio or wire is probably second - order . grad h: So it 's towards the corner of your mouth so that breath sounds don't get on it . grad h: And then just sort of about , a thumb or a thumb and a half away from your from your mouth . phd a: But we have more than one type of professor b: How am I d phd a: for instance , you 're phd c: Yeah . phd a: But if we could actually standardize , you know , the the microphones , as much as possible that would be really helpful . professor b: Well ,  it doesn't hurt to have a few extra microphones around , phd d: Yeah . professor b: so why don't we just go out and and get an order of of if this microphone seems OK to people , I 'd just get a half dozen of these things . grad h: Well the onl the only problem with that is right now , some of the Jimlets aren't working . grad h: And so , w  , I 've only been able to find three jacks that are working . phd e: Can we get these , wireless ? grad h: So professor b: No , but my point is phd a: But y we could just record these signals separately and time align them with the start of the meeting . professor b: R r right grad h: I I 'm not sure I 'm follow . Say that again ? professor b: Right now , we 've got , two microphones in the room , that are not quote - unquote standard . professor b: Also what we 've talked before about getting another , radio , grad h: Right . professor b: So ,  so we should go out to our full complement of whatever we can do , but have them all be the same mike . I think the original reason that it was done the other way was because , it w it was sort of an experimental thing and I don't think anybody knew whether people would rather have more variety or or , more uniformity , phd a: Right . phd a: Well , for short term research it 's just there 's just so much effort that would have to be done up front n  , professor b: Yeah . phd e: Is it because You you 're saying the for dialogue purposes , so that means that the transcribers are having trouble with those mikes ? Is that what you mean ? phd a: Well Jane would know more about the transcribers . postdoc g: a couple times , so , yeah , the transcribers notice And in fact there 're some where ,  ugh well ,  there 's it 's the double thing . postdoc g: And he 's always they always they just rave about how wonderful Adam 's Adam 's channel is .  , " Baaah ! " phd a: Even if if you 're talking on someone else 's mike it 's still you w postdoc g: Yeah , but  it 's not just that , it 's also you know you professor b: Yeah . postdoc g: It 's also like n no breathing , no You know , it 's like it 's it 's  , professor b: Yeah . postdoc g: it 's really it makes a big difference from the transcribers ' point of view grad h: Yeah , it 's an advantage when you don't breath . professor b: When we 're doing grad h: Yeah , I think that the point of doing the close talking mike is to get a good quality signal . professor b: Yeah , probably yeah , to the store we talked about and that grad h: Yep . postdoc g: And there was some talk about , maybe the h headphones that are uncomfortable for people , to grad h: Yep . So , as as I said , we 'll do a field trip and see if we can get all of the same mike that 's more comfortable than than these things , which I think are horrible . grad h: And , you know , we 're researchers , so we all have big heads . OK , so , Jonathan Fiscus is coming on the second of February and I 've spoken with , u u a lot of people here , not everyone .  , and , he expressed an interest in seeing the room and in , seeing a demonstration of the modified multitrans , which I 'll mention in a second , and also , he was interested in the pre - segmentation and then he 's also interested in the transcription conventions . postdoc g: And ,  So , it seems to me in terms of like , i i it wou You know , OK . So the room , it 's things like the audio and c and audi audio and acoustic acoustic properties of the room and how it how the recordings are done , and that kind of thing . OK , in terms of the multi - trans , well that that 's being modified by Dave Gelbart to , handle multi - channel recording . grad h: Oh , I should 've I was just thinking I should have invited him to this meeting . I I , @ @ didn't didn't see it , yesterday but I 'm going to see it today . And , that 's that will enable us to do nice  , tight time marking of the beginning and ending of overlapping segments . In terms of , like , pre - segmentation , that that continues to be , a terrific asset to the to the transcribers . What what I 'm doing right now is I 'm trying to include some information about which channel , there 's some speech in . I 'm just trying to do this by comparing energies ,  normalizing energies and comparing energies of the different channels . phd c: And so to to give the transcribers some information in which channel there 's there 's speech in addition to to the thing we we did now which is just , speech - nonspeech detection on the mixed file . So I 'm I 'm relying on on the segmentation of the mixed file postdoc g: This is good . phd c: but I 'm I 'm trying to subdivide the speech portions into different portions if there is some activity in in different channels . postdoc g: Excellent , so this 'd be like w e providing also speaker ID potentially . professor b: something I guess I didn't put in the list but , on that , same day later on in or maybe it 's No , actually it 's this week , Dave Gelbart and I will be , visiting with John Canny who i you know , is a CS professor , postdoc g: Oh . You know , maybe they 'd wanna stick an array mike here when we 're doing things phd e: That would be cool . professor b: or or maybe it 's it 's not a specific array microphone they want phd d: Yeah . professor b: but they might wanna just , you know , you could imagine them taking the four signals from these these table mikes and trying to do something with them  , I also had a discussion So , w  , we 'll be over over there talking with him , after class on Friday . I had a , discussion today with , Birger Kollmeier who 's a , a German , scientist who 's got a fair sized group doing a range of things . But but , he does stuff with auditory models and he 's very interested in directionality , and location , and and , head models and microphone things . And so , he 's he and possibly a student , there w there 's , a student of his who gave a talk here last year , may come here , in the fall for , sort of a five month , sabbatical . phd e: That that reminds me , I had a a thought of an interesting project that somebody could try to do with the data from here , either using , you know , the the mikes on the table or using signal energies from the head worn mikes , phd d:   phd e: and that is to try to construct a map of where people were sitting , professor b: Right . phd e: And so you could plot out who was sitting next to who professor b: A little bit , phd e: and professor b: he didn't do a very extreme thing but just it was just sort of phd d: Yeah , yeah . professor b: e e given that , the the the block of wood with the the the two mikes on either side , grad h:   professor b: if I 'm speaking , or if you 're speaking , or someone over there is speaking , it if you look at cross - correlation functions , you end up with a phd d: Yeah . professor b: if if someone who was on the axis between the two is talking , then you you get a big peak there . And if if someone 's talking on on on , one side or the other , it goes the other way . professor b: And then , it it it even looks different if th t if the two two people on either side are talking than if one in the middle . Well I was just thinking , you know , as I was sitting here next to Thilo that  , when he 's talking , my mike probably picks it up better than your guys 's mikes . phd e: So if you just looked at grad h: Oh , that 's another cl cue , phd d: Yeah . phd e: yeah , looked at the energy on my mike and you could get an idea about who 's closest to who . professor b: Yeah , well you have to the appropriate normalizations are tricky , and and and are probably the key . phd a: You just search for Adam 's voice on each individual microphone , you pretty much know where everybody 's sitting . postdoc g: Can I ask one thing ?  , so , Jonathan Fiscus expressed an interest in , microphone arrays . postdoc g: is there  b And I also want to say , his he can't stay all day . He needs to  , leave for  , from here to make a two forty - five flight grad h: Oh , so just morning . postdoc g: So it makes the scheduling a little bit tight but do you think that ,  that , i John Canny should be involved in this somehow or not . professor b: Probably not but I I 'll I 'll I 'll know better after I see him this Friday what what kind of level he wants to get involved . professor b: he might be excited to and it might be very appropriate for him to , or he might have no interest whatsoever . grad h: Is he involved in Ach ! I 'm blanking on the name of the project . NIST has has done a big meeting room instrumented meeting room with video and microphone arrays , and very elaborate software . Is is he the one working on that ? professor b: Well that 's what they 're starting up . professor b: well I think they 've instrumented a room but I don't think they they haven't started recordings yet . They don't have the phd e: Are they going to do video as well ? grad h:  . grad h: Oh , cuz what what I had read was , they had a  very large amount of software infrastructure for coordinating all this , both in terms of recording and also live room where you 're interacting the participants are interacting with the computer , and with the video , and lots of other stuff . professor b: All all I know is that they 've been talking to me about a project that they 're going to start up recording people meet in meetings . professor b: And one one notable difference u u actually I can't remember whether they were going to routinely collect video or not , but one one , difference from the audio side was that they are interested in using array mikes . The reason I didn't go for that here was because , the focus , both of my interest and of Adam 's interest was  , in impromptu situations . And we 're not recording a bunch of impromptu situations but that 's because it 's different to get data for research than to actually apply it . professor b: And so , for scientific reasons we thought it was good to instrument this room as we wanted it . But the thing we ultimately wanted to aim at was a situation where you were talking with , one or more other people i  , in in an p impromptu way , where you didn't didn't actually know what the situation was going to be . And therefore it would not it 'd be highly unlikely that room would be outfitted with with some very carefully designed array of microphones . It was just , you know , yet another piece of research and it seemed like we had enough troubles just phd e: So there 's no like portable array of mikes ? professor b: No . So there 's there 's  , there 's a whole range of things there 's a whole array of things , that people do on this . professor b: So , the ,  the big arrays , places , like  , Rutgers , and Brown , and other other places , they have , big arrays with , I don't know , a hundred hundred mikes or something . professor b: And it 's and , in fact at one point we had a a proposal in with Rutgers where we were gonna do some of the sort of per channel signal - processing and they were gonna do the multi - channel stuff , but it d it d we ended up not doing it . grad h: And then they have little ones too professor b: And then they had the little ones , yeah . grad h: but  but they don't have our block of wood , right ? professor b: Yeah , our block of wood is unique . professor b: But the But the No , there are these commercial things now you can buy that have four mikes or something phd a:   professor b: and and ,  So , yeah , there 's there 's there 's a range of things that people do . professor b: so if we connected up with somebody who was interested in doing that sort of thing that 's that 's a good thing to do .  , whenever I 've described this to other people who are interested on the with the acoustic side that 's invariably the question they ask . Just like someone who is interested in the general dialogue thing will always ask "  , are you recording video ? " phd a: Right , professor b: right ? phd a: right . professor b: And and the acoustic people will always say , " well are you doing , array microphones ? " So it 's it 's a good thing to do , but it doesn't solve the problem of how do you solve things when there 's one mike or at best two mikes in in this imagined PDA that we have .  , I know that having an array of  , I would imagine it would be more expensive to have a an array of microphones . But couldn't you kind of approximate the natural sis situation by just shutting off  , channels when you 're later on ?  , it seems like if the microphones don't effect each other then couldn't you just , you know , record them with an array and then just not use all the data ? grad h: It 's it 's just a lot of infrastructure that for our particular purpose we felt we didn't need to set up . professor b: Yeah , if ninety - nine percent of what you 're doing is c is shutting off most of the mikes , then going through the postdoc g: OK . professor b: But if you get somebody who 's who who has that as a primary interest then that put then that drives it in that direction . grad h: That 's right ,  if someone if someone came in and said we really want to do it , phd a: Right . That would be fine , phd e: So to save that data you You have to have one channel recording per mike in the array ? grad h: Buy more disk space . professor b: But then , you know , there 's it there 's phd e: What you save , if you 're going to do research with it . yeah professor b: There 's I I don't know what they 're going to do and I don't know how big their array is . Obviously if you were gonna save all of those channels for later research you 'd use up a lot of space . grad h: Well their software infrastructure had a very elaborate design for plugging in filters , and mixers , and all sorts of processing . But I think in practical situations you would have some engine of some sort doing some processing to reduce this to some to the equivalent of a single microphone that was very directional . professor b: Right ? phd a: it seems professor b: So phd e: Sort of saving the result of the beam - forming . phd a: it seems to me that there 's you know , there are good political reasons for for doing this , just getting the data , because there 's a number of sites like right now SRI is probably gonna invest a lot of internal funding into recording meetings also , which is good , but they 'll be recording with video and they 'll be You know , it 'd be nice if we can have at least , make use of the data that we 're recording as we go since it 's sort of this is the first site that has really collected these really impromptu meetings , and just have this other information available . So , if we can get the investment in just for the infra infrastructure and then , I don't know , save it out or have whoever 's interested save that data out , transfer it there , it 'd be g it 'd be good to have have the recording . grad h: You mean to to actually get a microphone array and do that ? phd a: Well , if Even if we 're not grad h: And video and phd a: I 'm not sure about video . That 's sort of an video has a little different nature since right n right now we 're all being recorded but we 're not being taped .  , but it definitely in the case of microphone arrays , since if there was a community interested in this , then grad h: Well , but I think we need a researcher here who 's interested in it . professor b: See the problem is it it took , it took at least six months for Dan to get together the hardware and the software , and debug stuff in in the microphones , and in the boxes . And so I think we could get a microphone array in here pretty easily and , have it mixed to to one channel of some sort . professor b: But , e I think for  , how we 're gonna decide For for maximum flexibility later you really don't want to end up with just one channel that 's pointed in the direction of the the the p the person with the maximum energy or something like that .  , you you want actually to you want actually to have multiple channels being recorded so that you can And to do that , it we 're going to end up greatly increasing the disk space that we use up , we also only have boards that will take up to sixteen channels and in this meeting , we 've got eight people and and six mikes . phd a: Well if there 's a way to say time to sort of solve each of these f those professor b: Yeah . phd a: So suppose you can get an array in because there 's some person at Berkeley who 's interested and has some equipment , and suppose we can as we save it we can , you know , transfer it off to some other place that that holds this this data , who 's interested , and even if ICSI it itself isn't .  , and it it seems like as long as we can time align the beginning , do we need to mix it with the rest ? I don't know . phd a:  it 's just it 's worth considering as sort of grad h: And y it 'd certainly gets skew . phd a: once you make the up front investment and can sort of save it out each time , and and not have to worry about the disk space factor , then it mi it might be worth having the data . I mentioned that , b as a practical matter , grad h: Just professor b: but the real issue is that , there is no way to do a recording extended to what we have now with low skew . professor b: which would mean that the sampling times and so forth would be all over the place compared to this . So it would depend on the level of pr processing you were doing later , but if you 're d i the kind of person who 's doing array processing you actually care about funny little times . And and so you actually wou would want to have a completely different set up than we have , phd a: I see . But what we could do is if there was someone else who 's interested they could have a separate set up which they wouldn't be trying to synch with ours which might be useful for for them . phd a: Right ,  at least they 'd have the data and the transcripts , professor b: And then we can offer up the room , phd a: and Right . professor b: Yeah , we can o offer the meetings , and the physical space , and and yeah , the transcripts , and so on . phd a: But it 's if it 's impossible or if it 's a lot of effort then you have to just balance the two , professor b: Well I thi phd a: so professor b: yeah , the thing will be , u u in in again , in talking to these other people to see what you know , what what we can do . phd e: Is there an interest in getting video recordings for these meetings ?  professor b: Right , so we have we phd e:  grad h: Yes , absolutely . But it 's exactly the same problem , that you have an infrastructure problem , you have a problem with people not wanting to be video taped , and you have the problem that no one who 's currently involved in the project is really hot to do it . Internally , but I know there is interest from other places that are interested in looking at meeting data and having the video . So it 's just postdoc g: Yeah , w although I m I I have to u u mention the human subjects problems , that i increase with video . There 's the fact that then  , if i I I 've heard comments about this before , " why don't you just put on a video camera ? " But you know , it 's sort of like saying , "  , well we 're primarily interested in in some dialogue things , but , why don't we just throw a microphone out there . "  , the thing is , once you actually have serious interest in any of these things then you actually have to put a lot of effort in . professor b: So I think NIST or LDC , or somebody like that I think is much better shape to do all that . postdoc g: I it it occurred to me , has Don signed a human subject 's form ? grad h: Oh ! Probably not . postdoc g: A permission form ? grad h: Has Don have you s did you si I thought you did actually . grad h: Didn't you read a digit string ? phd e: You were here at a meeting before . postdoc g: Did you sign a form ? grad f: Did I ? I don't know . professor b: Yeah , we we postdoc g: But I just grad f: Can I verbally consent ? postdoc g: you know . You 're on recor you 're being recorded postdoc g: o grad f: Yeah . phd a: and professor b: we don't we don't perform electro - shock during these meetings , grad f: I don't care . phd e: How big is the data set ? postdoc g: Oh , it 's what  is one meeting . I didn't want to have any conflicts of , you know , of of when to stop transcribing this one or So I wanted to keep it clear whose data were whose , and and and so phd e:  - huh . postdoc g: And , meetings , you know , I think that they 're they go as long as a almost two hours in some in some cases . So , you know , that means you know , if we 've got two already finished and they 're working on  , right now all eight of them have differe  , additional data sets . postdoc g: And , also Dan Ellis 's innovation of the ,  the multi - channel to here really helped a r a lot in terms of clearing clearing up h hearings that involve overlaps . But , just out of curiosity I asked one of them how long it was taking her , one of these two who has already finished her data set .  , these still , when they 're finished , that means that they 're finished with their pass through . They still need to be edited and all but But it 's word level , speaker change , the things that were mentioned . OK , now I wanted to mention the , teleconference I had with , Jonathan Fiscus . postdoc g: He , he in indicated to me that they 've that he 's been , looking , spending a lot of time with I 'm not quite sure the connection , but spending a lot of time with the ATLAS system . But it looks to me like that 's the name that has developed for the system that Bird and Liberman developed for the annotated graphs approach . postdoc g: So what he wants me to do and what we what we will do and  , is to provide them with the u already transcribed meeting for him to be able to experiment with in this ATLAS System . And they do have some sort of software , at least that 's my impression , related to ATLAS and that he wants to experiment with taking our data and putting them in that format , and see how that works out . I I I explained to him in in detail the , conventions that we 're using here in this in this word level transcript . And , you know , I I explained , you know , the reasons that that we were not coding more elaborately and and the focus on reliability . He 's he 's very  , independently he asked , " well what about reliability ? " So , he 's interested in the consistency of the encoding and that sort of thing . OK ,  phd a: Sorry , can you explain what the ATLAS I 'm not familiar with this ATLAS system . postdoc g: Well , you know , at this point I think  , well Adam 's read more in more detail than I have on this . But , there there is a way of viewing  , whenever you have coding categories , and you 're dealing with  , a taxonomy , then you can have branches that that have alternative , choices that you could use for each each of them . grad h: Is is Is ATLAS the his annotated transcription graph stuff ? I don't remember the acronym . The the one the what I think you 're referring to , they they have this concept of an an annotated transcription graph representation . grad h: And that 's basically what I based the format that I did I based it on their work almost directly , in combination with the TEI stuff . And so it 's it 's a data representation and a set of tools for manipulating transcription graphs of various types . phd e: Is this the project that 's sort of , between , NIST and and , a couple of other places ? postdoc g:   And I looked through them and they mainly had to do with this , this , tree structure , annotated tree diagram thing . postdoc g: So ,  and , you know , in terms of like the conventions that I 'm a that I 've adopted , it there there 's no conflict at all . And , " oh , and how 'd you handle this ? " And I said , " well , you know , this way " and And and we had a really nice conversation .  , OK , now I also wanted to say in a different a different direction is , Brian Kingsbury . I told him he could SSH on and use multi - trans , and have a look at the already done , transcription . And what he said was that , what they 'll be providing is will not be as fine grained in terms of the time information . And , that 's ,  You know , I need to get back to him and and , you know , explore that a little bit more and see what they 'll be giving us in specific , phd a:  . phd e: The p the people postdoc g: but I just haven't had time yet . phd e: The the folks that they 're , subcontracting out the transcription to , are they like court reporters postdoc g: Sorry , what ? Yes . phd e: or postdoc g: Apparently Well , I get the sense they 're kind of like that . Like it 's like a pool of of somewhat  , secretarial I don't think that they 're court reporters . Like medical transcriptionist type people grad h: Nu - it 's mostly it 's for their speech recognition products , phd e: But aren't they 're postdoc g: Yep . grad h: Well they they do send it out but my understanding is that that 's all this company does is transcriptions for IBM for their speech product . grad h: So most of it 's ViaVoice , people reading their training material for that . postdoc g: Up to now it 's been monologues , as far my understood . postdoc g: Brian himself downloaded So So , Adam sent them a CD and Brian himself downloaded  , cuz , you know , we wanted to have it so that they were in familiar f terms with what they wanted to do . postdoc g: And then what he 's going to do is check it , a before they go be beyond the first one . professor b: So if they hear something off in the distance they don't they just go phd e: OK . grad h: Well , but that 's OK , because , you know , you 'll do all them and then combine them . phd e: Well if you 're tran if you got that channel right there postdoc g: d  , in my case phd c: Yeah . postdoc g: Yeah , I I would think that it would be kind of hard to come out with Yeah . phd a: I I think it 's sort of hard just playing the you know , just having played the individual files . phd a: there are a lot of words that are so reduced phonetically that make sense when you know what the person was saying before . phd e: Yeah , that 's phd a: it sort of depends where you are in postdoc g: And especially since a lot of these phd d: Yeah . grad h: And the answer is we don't actually know the answer because we haven't tried both ways . postdoc g: Well , except I can say that my transcribers use the mixed signal mostly grad h: So . postdoc g: unless there 's a huge disparity in terms of the volume on on the mix . In which case , you know , they they wouldn't be able to catch anything except the prominent channel , grad h: Right . postdoc g: Yeah , well phd a: Actually , are th so are they giving any time markings ? grad h: Right . phd a: In other words , if postdoc g: Well , I have to ask him . postdoc g: But but the ,  I did want to say that it 's hard to follow one channel of a conversation even if you know the people , and if you 're dealing furthermore with highly abstract network concepts you 've never heard of So , you know , one of these people was was transcribing the , networks group talk and she said , " I don't really know what a lot of these abbreviations are , " " but I just put them in parentheses cuz that 's the that 's the convention and I just " Cuz you know , if you don't know grad h: Oh , I 'd be curious to to look at that . grad h: The networks group meetings are all phd e: Given all of the effort that is going on here in transcribing why do we have I B M doing it ? Why not just do it all ourselves ? professor b: it 's historical .  , some point ago we thought that  , it " boy , we 'd really have to ramp up to do that " , phd c:  - huh . phd d: No , just professor b: you know , like we just did , and , here 's , a a , collaborating institution that 's volunteered to do it . professor b: And it still might be a good thing phd e: I 'm just wondering now phd a: Actu yeah , Mar - Mari asked me the same question as sort of professor b: but phd e: Well , I 'm I 'm wondering now if it 's grad h: Well we can talk about more details later . Let 's see what comes out of it , and and , you know , have some more discussions with them . It 's very a real benefit having Brian involved because of his knowledge of what the how the data need to be used and so what 's useful to have in the format . grad h: So , Liz , with with the SRI recognizer , can it make use of some time marks ? phd a: OK , so this is a , grad h: I I guess I don't know what that means . phd a: and actually I should say this is what Don has b  , he 's already been really helpful in , chopping up these So so first of all you  , for the SRI front - end , we really need to chop things up into pieces that are f not too huge .  , but second of all ,  in general because some of these channels , I 'd say , like , I don't know , at least half of them probably on average are g are ha are have a lot of cross - ta sorry , some of the segments have a lot of cross - talk .  , it 's good to get sort of short segments if you 're gonna do recognition , especially forced alignment . So , Don has been taking a first stab actually using Jane 's first the fir the meeting that Jane transcribed which we did have some problems with , and Thilo , I think told me why this was , but that people were switching microphones around in the very beginning , so the SRI re phd c: No , th Yeah . They they were not switching them but what they were they were adjusting them , phd a: and they They were not phd c: so . phd a: So we have to sort of normalize the front - end and so forth , and have these small segments . phd a: So we 've taken that and chopped it into pieces based always on your your , cuts that you made on the mixed signal . Right ? phd a: the problem is if we have no time marks , then for forced alignment we actually don't know where you know , in the signal the transcriber heard that word . And so grad h: Oh , I see , phd a: if if it 's a whole conversation and we get a long , you know , par paragraph of of talk , grad h: it 's for the length . phd a: And , I think with phd e: Well you would need to like a forced alignment before you did the chopping , right ? phd a: No , we used the fact that So when Jane transcribes them the way she has transcribers doing this , whether it 's with the pre - segmentation or not , grad h: It 's already chunked . And maybe they choose the chunk or now they use a pre - segmentation and then correct it if necessary . That 's great , cuz the recognizer can grad h: it 's all pretty good sized for the recognizer also . phd a: Right , and it it helps that it 's made based on sort of heuristics and human ear I think . phd a: Th - but there 's going to be a real problem , even if we chop up based on speech silence these , the transcripts from I B M , we don't actually know where the words were , which segment they belonged to . phd e: Why not do a a a forced alignment ? grad h: That 's what she 's saying , is that you can't . phd a: If you do a forced alignment on something really grad h: Got  six sixty minutes of phd a: well even if you do it on something really long you need to know you can always chop it up but you need to have a reference of which words went with which , chop . postdoc g: Now wasn't I thought that one of the proposals was that IBM was going to do an initial forced alignment , phd a: So postdoc g: after they grad h: Yeah , but professor b: I I think that they are , grad h: We 'll have to talk to Brian . professor b: yeah , I 'm sure they will and so we we have to have a dialogue with them about it . professor b: it sounds like Liz has some concerns phd a: Maybe they have some you know , maybe actually there is some , even if they 're not fine grained , maybe the transcribers professor b: and phd a: I don't know , maybe it 's saved out in pieces or or something . postdoc g: I just you know , it 's like I got over - taxed with the timing . But the it is true that the segments I haven't tried the segments that Thilo gave you but the segments that in your first meeting are great . phd a: Right , cuz postdoc g: Well , I I was thinking it would be fun to to  , if if you wouldn't mind , to give us a pre - segmentation . postdoc g: maybe you have one already of that first m of the meeting that  , the first transcribed meeting , the one that I transcribed . phd c: I 'm sure I have some postdoc g: Do you have a could you generate a pre - segmentation ? grad h: February sixteenth I think . phd c: but but that 's the one where we 're , trai training on , so that 's a little bit grad h: Oh . phd c: It 's a little bit at odd to postdoc g: Oh , darn . phd a: And actually as you get transcripts just , for new meetings , we can try postdoc g:  - huh . So it 'd be good for just to know as transcriptions are coming through the pipeline from the transcribers , just to sort of we 're playing around with sort of  , parameters f on the recognizer , postdoc g:   phd a: The first meeting had I think just four people , phd c: Four speakers , yeah . postdoc g: Yeah , Liz and I spoke d w at some length on Tuesday and and I and I was planning to do just a a preliminary look over of the two that are finished and then give them to you . I guess the other thing , I I can't remember if we discussed this in the meeting but , I know you and I talked about this a little bit , there was an issue of , suppose we get in the , I guess it 's enviable position although maybe it 's just saying where the weak link is in the chain , where we we , we have all the data transcribed and we have these transcribers and we were we 're the we 're still a bit slow on feeding at that point we 've caught up and the the the , the weak link is is recording meetings . OK , two questions come , is you know what how how do we  , it 's not really a problem at the moment cuz we haven't reached that point but how do we step out the recorded meetings ? And the other one is , is there some good use that we can make of the transcribers to do other things ? So , I I can't remember how much we talked about this in this meeting but there was grad h: We had spoken with them about it . postdoc g: And there is one use that that also we discussed which was when , Dave finishes the and maybe it 's already finished the the modification to multi - trans which will allow fine grained encoding of overlaps .  , then it would be very these people would be very good to shift over to finer grain encoding of overlaps . It 's just a matter of , you know , providing So if right now you have two overlapping segments in the same time bin , well with with the improvement in the database in in the , sorry , in the interface , it 'd be possible to , you know , just do a click and drag thing , and get the  , the specific place of each of those , the time tag associated with the beginning and end of of each segment . professor b: One one was  , we had s had some discussion in the past about some very high level labelings , postdoc g: Yeah . The types of overlaps professor b: types of overlaps , and so forth that that someone could do . And the third one is is , just a completely wild hair brained idea that I have which is that , if ,  if we have time and people are able to do it , to take some subset of the data and do some very fine grained analysis of the speech . For instance , marking in some overlapping potentially overlapping fashion , the value of , ar articulatory features . professor b: You know , just sort of say , OK , it 's voiced from here to here , there 's it 's nasal from here to here , and so forth .  , as opposed to doing phonetic  , you know , phonemic and the phonetic analysis , grad h:  professor b: and , assuming , articulatory feature values for those those things . postdoc g: Also if you 're dealing with consonants that would be easier than vowels , wouldn't it ?  , I would think that that , being able to code that there 's a a fricative extending from here to here would be a lot easier than classifying precisely which vowel that was . professor b: but I think also it 's just the issue that that when you look at the u w u u when you look at Switchboard for instance very close up there are places where whether it 's a consonant or a vowel you still have trouble calling it a particular phone postdoc g:   grad h: Yeah , but but just saying what the professor b: because it 's you know , there 's this movement from here to here postdoc g: Yeah , I 'm sure . professor b: and and and it 's so I phd e: You 're saying r sort of remove the high level constraints and go bottom - up . Maybe there 's there 's even a better way to do it but it but but that 's , you know , sort of a traditional way of describing these things , phd e:   professor b: and  , actually this might be a g neat thing to talk to postdoc g: That 's nice . professor b: some sort of categories but but something that allows for overlapping change of these things and then this would give some more ground work for people who were building statistical models that allowed for overlapping changes , different timing changes as opposed to just " click , you 're now in this state , which corresponds to this speech sound " and so on . phd a: So this is like gestural  , these g professor b: Yeah , something like that . professor b: actually if we get into that it might be good to , haul John Ohala into this phd a: Right . phd a: But is is the goal there to have this on meeting data , postdoc g: Excellent . phd a: like so that you can do far field studies of those gestures or  , or is it because you think there 's a different kind of actual production in meetings that people use ? Or ? professor b: No , I think I think it 's for for for that purpose I 'm just viewing meetings as being a a neat way to get people talking naturally . And then you have i and then and then it 's natural in all senses , phd e: Just a source of data ? phd a: I see . professor b: in the sense that you have microphones that are at a distance that you know , one might have , and you have the close mikes , and you have people talking naturally . And the overlap is just indicative of the fact that people are talking naturally , phd a:  - huh . professor b: right ? So so I think that given that it 's that kind of corpus , phd d: Yeah . professor b: if it 's gonna be a very useful corpus  , if you say w OK , we 've limited the use by some of our , censored choices , we don't have the video , we don't and so forth , but there 's a lot of use that we could make of it by expanding the annotation choices . professor b: And , most of the things we 've talked about have been fairly high level , and being kind of a bottom - up person I thought maybe we 'd , do some of the others . professor b: people have made a lot of use of of TIMIT and ,  w due to its markings , and then the Switchboard transcription thing , well I think has been very useful for a lot of people . professor b: So phd a: I guess I wanted to , sort of make a pitch for trying to collect more meetings . phd a: I actually I talked to Chuck Fillmore and I think they 've what , vehemently said no before but this time he wasn't vehement and he said you know , " well , Liz , come to the meeting tomorrow professor b: Yeah . Go to their meeting tomorrow and see if we can try , to convince them postdoc g:   professor b: Cuz they have something like three or four different meetings , phd a: because they have And they have very interesting meetings from the point of view of a very different type of of talk than we have here professor b: right ? postdoc g:    phd e: You mean in terms of the topic topics ? phd a: Well , yes and in terms of the the fact that they 're describing abstract things and , just dialogue - wise , professor b:   And then the other thing is , I don't know if this is at all useful , but I asked Lila if I can maybe go around and talk to the different departments in this building to see if there 's any groups that , for a free lunch , professor b: Yes . grad h: You mean non - ICSI ? phd a: non - ICSI , non - academic , grad h: Yeah , I guess you you can try phd a: you know , like government people , grad h: but phd a: I don't know .  , it seems like we we had this idea before of having like linguistics students brought down for free lunches grad h: Well , tha I think that 's her point . phd a: Right , and then we could also we might try advertising again because I think it 'd be good if if we can get a few different sort of non - internal types of meetings postdoc g: Yeah . phd e: Does does John Ohala have weekly phonetics lab meetings ? postdoc g:   grad h: And I think , if we could get phd a: So I actually wrote to him and he answered , " great , that sounds really interesting " . grad h: But , it would be nice if we got someone other than me who knew how to set it up and could do the recording phd a: So grad h: so u I didn't have to do it each time . phd a: and I was thinking professor b: He - he 's supposed he 's supposed to be trained to do it .  , the other thing is that there was a number of things at the transcription side that , transcribers can do , like dialogue act tagging , grad h: It 's not that hard . phd a: disfluency tagging , things that are in the speech that are actually something we 're y working on for language modeling . So if you wanna process a utterance and the first thing they say is , " well " , and that " well " is coded as some kind of interrupt u tag .  , and things like that , th postdoc g: Of course some of that can be li done lexically . phd a: A lot of it can be done postdoc g: And I also they are doing disfluency tagging to some degree already . phd a: I think there 's a second pass and I don't really know what would exist in it . But there 's definitely a second pass worth doing to maybe encode some kinds of , you know , is it a question or not , postdoc g:   postdoc g: I wanted to whi while we 're  , so , to return just briefly to this question of more meeting data ,  I have two questions . One of them is , Jerry Feldman 's group , they they , are they I know that they recorded one meeting . professor b: I think there 's we should go beyond , ICSI but , there 's a lot of stuff happening at ICSI that we 're not getting now that we could . So th there was the thing in Fillmore 's group but even there he hadn't What he 'd said " no " to was for the main meeting . phd e: Well , and and the other thing too is when they originally said " no " they didn't know about this post - editing capability thing . professor b: there 's there 's , the networks group , I don't Do they still meeting regularly or ? grad h: Well , I don't know if they meet regularly or not but they are no longer recording . professor b: But  , ha ha have they said they don't want to anymore or ? grad h: ugh , what was his name ? professor b: i i postdoc g: Joe Sokol ? grad h: Yeah . professor b: OK , so they 're down to three or four people grad h: They and they stopped Yeah . postdoc g: We might be able to get the administration grad h: Well he was sort of my contact , so I just need to find out who 's running it now . phd a: Yeah , it One thing that would be nice postdoc g: I don't know phd a: and this it sounds bizarre but , I 'd really like to look at to get some meetings where there 's a little bit of heated discussion , like ar arguments and or emotion , and things like that . Some group , " yes , we must " grad h: Who 's willing to get recorded and distributed ? phd a: Well , you know , something phd c: Yeah . grad f: Yeah , I don't think the more political argumentative ones would be willing to phd a:  professor b: Yeah , with with with potential use from the defense department . phd a: No , but maybe stu student , groups or , film - makers , or som Something a little bit colorful . Yeah , th there 's a problem there in terms of , the  commercial value of of st  , postdoc g: Yeah , of course there is this problem though , that if we give them the chance to excise later we e might end up with like five minutes out of a f of m one hour phd d: Film - maker . phd a: And I don't mean that they 're angry phd d: Is postdoc g: of Yes . phd a: but just something with some more variation in prosodic contours and so forth would be neat . So if anyone has ideas , I 'm willing to do the leg work to go try to talk to people but I don't really know which groups are worth pursuing . postdoc g: Well there was this K P F A grad h: No that 's postdoc g: but OK . phd a: Or postdoc g: And I had one other one other aspect of this which is , Jonathan Fiscus expressed primar  y a major interest in having meetings which were all English speakers . phd e: Did he mean ,  did he mean and non - British ? grad h: Well phd c: The all native . phd e: He said British was OK ? postdoc g: But but Sure , sure , sure . professor b: Why ? grad h: British is English ? phd c:  postdoc g: Yeah . professor b: Well , I don't I don't I don't think if he didn't say that postdoc g: Native speaking . grad h: So , why would he care ? phd e: Knowing the application phd a: That 's professor b: I remember wh I I remember a study phd a: I was thinking , knowing the , n National Institute of Standards , it is all professor b: I remember a study that BBN did where they trained on this was in Wall Street Journal days or something , they trained on American English and then they tested on , different native speakers from different areas . professor b: it was Swiss w Yeah , so it 's so I think , you know , if he 's if he 's thinking in terms of recognition kind of technology I I I think he would probably want ,  American English , postdoc g: All America , OK . It it yeah , unless we 're gonna train with a whole bunch of postdoc g: I think that the Feldman 's meetings tend to be more that way , aren't they ?  , I sort of feel like they have professor b: I think so , grad h: Maybe . grad h: And maybe there are a few of with us where it was professor b: Yeah . grad h: you know , Dan wasn't there and before Jose started coming , professor b: Yeah . professor b: So , what about what about people who involved in some artistic endeavor ? phd d: Yeah . phd a: Exactly , that 's what I was professor b: You 'd think like they would be phd d: A film - maker . phd a: something where there there is actually discussion where there 's no right or wrong answer but but it 's a matter of opinion kind of thing . phd a: Yeah , we could phd e: A any department that calls itself science phd d: Department . phd d: Computer sci grad h: That postdoc g: We could get Julia Child . phd a: I 'm I 'm actually serious grad h: That 's phd a: because , you know , we have the set up here grad h: Got a ticket . grad f: I know grad h: I could phd a: Not not professor b: Put a little ad up saying , " come here and argue " . grad h: and they they they expressed willingness back when they thought they would be doing more stuff with speech . grad h: But when they lost interest in speech they also stopped answering my email about other stuff , so . phd a: Or people who are really h professor b: They could have a discussion about te grad f: I grad h: We should probably bleep that out . grad f: I heard that at Cal Tech they have a special room someone said that they had a special room to get all your frustrations out that you can go to and like throw things and break things . professor b: Yeah , now that is not actually what we grad f: So we can like post a grad h: Th - that 's not what we want . grad f: No , not to that extent phd a: Well , far field mikes can pick up where they threw stuff on the wall . professor b: Yeah , but we don't want them to throw the far field mikes is the thing . postdoc g: It 'd be fun to get like a a p visit from the grad h: There was a dorm room at Tech that , someone had coated the walls and the ceiling , and , the floor with mattresses . professor b: What did we mean by that ? Remember @ @ ? grad h: Liz wanted to talk about methods of improving accuracy by doing pre - processing . phd a: Well I think that that was just sort of I I already asked Thilo professor b: Oh , you already did that . phd a: but that , it would be helpful if I can stay in the loop somehow with , people who are doing any kind of post - processing , whether it 's to separate speakers or to improve the signal - to - noise ratio , or both , that we can sort of try out as we 're running recognition .  , so , i is that Who else is work I guess Dan Ellis and you phd c: Dan , yeah . professor b: he 's he 's interested in in fact we 're look starting to look at some echo cancellation kind of things . professor b: Which  grad h: I am not sure how much that 's an issue with the close talking mikes , professor b:  ? grad h: but who knows ? professor b: Well , let 's w i isn't that what what you want phd a: I don't know . I 'm bad professor b: t No , so No , i w wha what you what you want when you 're saying improving the wave form you want the close talking microphone to be better . professor b: And the question is to w to what extent is it getting hurt by ,  by any room acoustics or is it just  , given that it 's close it 's not a problem ? phd a: It doesn't seem like big room acoustics problems to my ear professor b:  phd a: but I 'm not an expert . grad h: e I bet with the lapel mike there 's plenty , room acoustic phd a: That that may be true . phd a: But I don't know how good it can get either by those the those methods grad h: Yeah . grad h: So I I think it 's just , phd a: Oh , I don't know . phd a: All I meant is just that as sort of as this pipeline of research is going on we 're also experimenting with different ASR , techniques . phd e: So the problem is like , on the microphone of somebody who 's not talking they 're picking up signals from other people and that 's causing problems ? phd a: R right , although if they 're not talking , using the the inhouse transcriptions , were sort of O K because the t no one transcribed any words there and we throw it out . phd a: But if they 're talking at all and they 're not talking the whole time , so you get some speech and then a "  -  " , and some more speech , so that whole thing is one chunk . And the person in the middle who said only a little bit is picking up the speech around it , that 's where it 's a big problem . postdoc g: You know , this does like seem like it would relate to some of what Jose 's been working on as well , the encoding of the phd d: Yeah . postdoc g: And and he also , he was phd a: The energy , phd d: Yeah , phd a: right . postdoc g: I was t I was trying to remember , you have this interface where you i you ha you showed us one time on your laptop that you you had different visual displays as speech and nonspeech events . Because , eh , it 's possible , eh , eh , in a simp sample view , to , nnn , to compare with c with the segment , the the kind of assessment what happened with the the different parameters . And only with a different bands of color for the , few situation , eh , I consider for acoustic event is enough to @ @ . phd d: I I I see that , eh , you are considering now , eh , a very sophisticated , eh , ehm , eh , @ @ set of , eh , graphic s eh , eh , ehm , si symbols to to transcribe . No ? Because , before , you you are talking about the the possibility to include in the Transcriber program eh , a set of symbols , of graphic symbol to t to mark the different situations during the transcription postdoc g: Oh , I w  - huh . No ? postdoc g: Well , you 're saying So , symbols for differences between laugh , and sigh , and and and slam the door and stuff ? phd d: Yeah . postdoc g: Or some other kind of thing ? phd d: No ? To to mark postdoc g: Well , I wouldn't say symbols so much . The the main change that I that I see in the interface is is just that we 'll be able to more finely c  , time things . postdoc g: But I I also st there was another aspect of your work that I was thinking about when I was talking to you phd a:  . postdoc g: which is that it sounded to me , Liz , as though you and , maybe I didn't q understand this , but it sounded to me as though part of the analysis that you 're doing involves taking segments which are of a particular type and putting them together . postdoc g: And th so if you have like a p a s you know , speech from one speaker , then you cut out the part that 's not that speaker , phd a:   postdoc g: and you combine segments from that same speaker to and run them through the recognizer . Is that right ? phd a: Well we try to find as close of start and end time of as we can to the speech from an individual speaker , postdoc g:   phd a: because then we we 're more guaranteed that the recognizer will for the forced alignment which is just to give us the time boundaries , because from those time boundaries then the plan is to compute prosodic features . phd a: And the sort of more space you have that isn't the thing you 're trying to align the more errors we have .  , so , you know , that that it would help to have either pre - processing of a signal that creates very good signal - to - noise ratio , postdoc g:   phd a: which I don't know how possible this is for the lapel , or to have very to have closer , time you know , synch times , basically , around the speech that gets transcribed in it , or both . So I just wanted to see , you know , on the transcribing end from here things look good . And then the issue of like global processing of some signal and then , you know , before we chop it up is is yet another way we can improve things in that . You can ,   The problem is just that the acoustic when the signal - to - noise ratio is too low , you you 'll get , a  an alignment with the wrong duration pattern or it phd e: Oh , so that 's the problem , is the the signal - to - noise ratio . It 's not the fact that you have like  , what he did is allow you to have , words that were in another segment move over to the at the edges of of segmentations . phd a: Right , things things near the boundaries where if you got your alignment wrong phd e:   It 's that there are problems even in inside the alignments , because of the fact that there 's enough acoustic signal there t for the recognizer to to eat , as part of a word . So , yeah , bottom bottom line is just I wanted to make sure I can be aware of whoever 's working on these signal - processing techniques for , detecting energies , phd d: Yeah . professor b: O K ,  tea has started out there I suggest we c run through our digits and , postdoc g: OK