grad h: Yeah , so if  So if anyone hasn't signed the consent form , please do so . phd a: Now you won't be able to walk or ride your bike , huh ? professor d: OK . grad h: And  , shall I go ahead and do some digits ? professor d: we were gonna do that at the end , remember ? grad h: OK , whatever you want . Just just to be consistent , from here on in at least , that that we 'll do it at the end . professor d: OK  Well , it ju  it might be that someone here has to go , postdoc f: Testing , one , two , three . So ,  I had asked actually anybody who had any ideas for an agenda to send it to me and no one did . professor d: postdoc f: From last time I wanted to  The An iss  one topic from last time . professor d: Right , s OK , so one item for an agenda is  Jane has some   some research to talk about , research issues . professor d: I have a list of things that I think were done over the last three months I was supposed to send off ,  and ,  I I sent a note about it to  to Adam and Jane but I think I 'll just run through it also and see if someone thinks it 's inaccurate or  insufficient . phd a: A list that you have to send off to who ? professor d: to   , IBM .  , And , Anything else ? anyone wants to talk about ? phd a: What about the ,  your trip , yesterday ? professor d: No . professor d: Cuz that 's Cuz that was all all about the ,  I I I can chat with you about that off - line .  , And , Anything else ? Nothing else ?  , there 's a  , there is a a ,   telephone call tomorrow , which will be a conference call that some of us are involved in for  a possible proposal .  , we 'll talk we 'll talk about it next week if if something grad h: Do you want me to be there for that ? I noticed you C C ' ed me , but I wasn't actually a recipient . So it sounds like the the three main things that we have to talk about are ,  this list ,  Jane and Jane and Adam have some research items , and , other than that , anything , as usual , anything goes beyond that . OK , Jane , since since you were sort of cut off last time why don't we start with yours , make sure we get to it . postdoc f: OK , it 's it 's very eh it 's very brief ,  just let me just hand these out . So , if you 've looked at this you 've seen it before , so Basically ,  as you know ,  part of the encoding includes a mark that indicates an overlap . It 's not indicated with , tight precision , it 's just indicated that OK , so , It 's indicated to to so the people know what parts of sp which which stretches of speech were in the clear , versus being overlapped by others . So , I used this mark and ,  and , divided the I wrote a script which divides things into individual minutes , of which we ended up with forty five , and a little bit . postdoc f: And ,  What you can see is the number of overlaps and then to the right , whether they involve two speakers , three speakers , or more than three speakers . And ,  and , what I was looking for sp sp specifically was the question of whether they 're distributed evenly throughout or whether they 're bursts of them . And it looked to me as though  , you know y this is just eh eh , this would this is not statistically verified , but it did look to me as though there are bursts throughout , rather than being localized to a particular region . The part down there , where there 's the maximum number of of ,  overlaps is an area where we were discussing whether or not it would be useful to indi to s to code stress , sentence stress as possible indication of ,  information retrieval . professor d: What was what 's the the parenthesized stuff that says , like e the first one that says six overlaps and then two point eight ? postdoc f: Oh , th That 's the per cent . postdoc f: So , six is ,  two point eight percent of the total number of overlaps in the session . postdoc f: At the very end , this is when people were , you know , packing up to go basically , there 's this final stuff , I think we I don't remember where the digits fell . postdoc f: So , i it seems like it goes through bursts but ,  that 's kind of it . postdoc f: Now , Another question is is there are there individual differences in whether you 're likely to be overlapped with or to overlap with others . And , again I want to emphasize this is just one particular  one particular meeting , and also there 's been no statistical testing of it all , but I ,  I took the coding of the I , you know , my I had this script figure out ,  who was the first speaker , who was the second speaker involved in a two - person overlap , I didn't look at the ones involving three or more . And ,  this is how it breaks down in the individual cells of who tended to be overlapping most often with who who else , and if you look at the marginal totals , which is the ones on the right side and across the bottom , you get the totals for an individual . So ,  If you look at the bottom , those are the ,  numbers of overlaps in which  Adam was involved as the person doing the overlapping and if you look I 'm sorry , but you 're o alphabetical , that 's why I 'm choosing you And then if you look across the right , then that 's where he was the person who was the sp first speaker in the pair and got overlap overlapped with by somebody . postdoc f: And , then if you look down in the summary table , then you see that ,  th they 're differences in whether a person got overlapped with or overlapped by . phd e: Yeah Yeah postdoc f: Yes , very true very true grad h: Normalized to how much postdoc f: it would be good to normalize with respect to that . Now on the table I did take one step toward ,  away from the raw frequencies by putting ,  percentages . So that the percentage of time of the of the times that a person spoke , what percentage eh , w so . Of the times a person spoke and furthermore was involved in a two two - person overlap , what percentage of the time were they the overlapper and what percent of the time were they th the overlappee ? And there , it looks like you see some differences , that some people tend to be overlapped with more often than they 're overlapped , but , of course ,  i e this is just one meeting ,  there 's no statistical testing involved , and that would be required for a for a finding of any kind of scientific reliability . professor d: S so , i it would be statistically incorrect to conclude from this that Adam talked too much or something . grad h: No no actually , that would be actually statistically correct , professor d: Yeah , yeah . And I 'm you know , I 'm I don't see a point of singling people out , professor d: B I I I rather enjoyed it , but but this postdoc f: now , this is a case where obviously phd a: But the numbers speak for themselves . postdoc f: Well , you know , it 's like I 'm not I 'm not saying on the tape who did better or worse grad h: Yes , that 's right , so you don't nee OK . postdoc f: because I don't think that it 's I you know , and and th here 's a case where of course , human subjects people would say be sure that you anonymize the results , and and , so , might as well do this . grad h: Yeah , when this is what This is actually when Jane sent this email first , is what caused me to start thinking about anonymizing the data . postdoc f: And actually , you know , the point is not about an individual , it 's the point about tendencies toward you know , different styles , different speaker styles . postdoc f: And it would be , you know of course , there 's also the question of what type of overlap was this , and w what were they , and i and I and I know that I can distinguish at least three types and , probably more , the general cultural idea which w  , the conversation analysts originally started with in the seventies was that we have this strict model where politeness involves that you let the person finish th before you start talking , and and you know , w we know that an and they 've loosened up on that too s in the intervening time , that that that 's that 's viewed as being a culturally - relative thing , that you have the high - involvement style from the East Coast where people will overlap often as an indication of interest in what the other person is saying . phd b: Exactly ! postdoc f: Yeah , exactly ! phd e: Yeah postdoc f: Well , there you go . And and , you know , in contrast , so Deborah d and also Deborah Tannen 's thesis she talked about differences of these types , that they 're just different styles , and it 's  you you can't impose a model of there of the ideal being no overlaps , and you know , conversational analysts also agree with that , so it 's now , universally a ag agreed with . And and , als  , I can't say universally , but anyway , the people who used to say it was strict ,  now ,  don't .  they they also you know , ack acknowledge the influence of sub of subcultural norms and cross - cultural norms and things . So ,  Then it beco though so just just superficially to give  a couple ideas of the types of overlaps involved , I have at the bottom several that I noticed . So , there are backchannels , like what Adam just did now and , anticipating the end of a question and simply answering it earlier , and there are several of those in this in these data where phd b:   postdoc f: because we 're people who 've talked to each other ,  we know basically what the topic is , what the possibilities are and w and we 've spoken with each other so we know basically what the other person 's style is likely to be and so and t there are a number of places where someone just answered early . And places also which I thought were interesting , where two or more people gave exactly th the same answer in unison different words of course but you know , the basically , you know everyone 's saying " yes " or you know , or ev even more sp specific than that . So , the point is that ,  overlap 's not necessarily a bad thing and that it would be im i useful to subdivide these further and see if there are individual differences in styles with respect to the types involved . professor d: Well , of course th the biggest ,  result here , which is one we 've we 've talked about many times and isn't new to us , but which I think would be interesting to show someone who isn't familiar with this is just the sheer number of overlaps . professor d: That that Right ? that that ,  phd e: Yes , yes ! postdoc f: Oh , OK interesting . professor d: here 's a relatively short meeting , it 's a forty forty plus minute meeting , and not only were there two hundred and fifteen overlaps but ,  I think there 's one one minute there where there where where there wasn't any overlap ? grad h: Hundred ninety - seven . professor d: it 's  throughout this thing ? phd a: It 'd be interesting professor d: It 's You have postdoc f: Well , at the bottom , you have the bottom three . grad h: S n are postdoc f: So four four minutes all together with none none . phd a: But it w professor d: Oh , so the bottom three did have s stuff going on ? There was speech ? postdoc f: Yes ,  - huh . professor d: OK , so if the this phd a: It 'd be interesting to see what the total amount of time is in the overlaps , versus postdoc f: Yes , exactly and that 's that 's where Jose 's pro project comes in . phd e: Yeah , yeah , I h I have this that infor I have th that information now . professor d: Oh , about how much is it ? phd e: The the duration of eh of each of the overlaps . professor d: O oh , what 's what 's the what 's the average length ? phd e: M I I haven't averaged it now but ,  I I will ,  I will do the the study of the with the with the program with the  , the different ,  the , nnn , distribution of the duration of the overlaps . professor d: You don't know ? OK , you you don you don't have a feeling for roughly how much it is ? Yeah . phd e: The duration is ,  the variation the variation of the duration is  , very big on the dat phd a:   postdoc f: I suspect that it will also differ , depending on the type of overlap involved . postdoc f: So backchannels will be very brief phd e: Because , on your surface eh a bit of zone of overlapping with the duration eh , overlapped and another very very short . phd e: i probably it 's very difficult to to because the the overlap is ,  on is only the in the final " S " of the of the the fin the the end the end word of the ,  previous speaker with the the next word of the the new speaker . phd e: I considered that 's an overlap but it 's very short , it 's an " X " with a and the idea is probably , eh when eh when eh , we studied th th that zone , eh eh , we h we have eh eh confusion with eh eh noise . phd e: With eh that fricative sounds , but  I have new information but I have to to study . phd g: You split this by minute ,  so if an overlap straddles the boundary between two minutes , that counts towards both of those minutes .  , so le let 's think about the case where A starts speaking and then B overlaps with A , and then the minute boundary happens . And let 's say that after that minute boundary ,  B is still speaking , and A overlaps with B , that would be a new overlap . But otherwise  , let 's say B comes to the conclusion of of that turn without anyone overlapping with him or her , in which case there would be no overlap counted in that second minute . phd g: No , but suppose they both talk simultaneously both a a portion of it is in minute one and another portion of minute two . In that case ,  my c the coding that I was using since we haven't ,  incorporated Adam 's ,  coding of overlap yets , the coding of Yeah , " yets " is not a word .  since we haven't incorporated Adam 's method of handling overl overlaps yet  then that would have fallen through the cra cracks . It would be an underestimate of the number of overlaps because ,  I wou I wouldn't be able to pick it up from the way it was encoded so far . professor d: I I postdoc f: We just haven't done th the precise second to sec you know , second to second coding of when they occur . professor d: Let 's say that in in second fifty - seven of one minute , you start talking and I start talking and we ignore each other and keep on talking for six seconds . professor d: So we go over So we were we were talking over one another , and it 's just in each case , it 's just sort of one interval . Right ? postdoc f:  -  ? professor d: So ,  we talked over the minute boundary . professor d: OK , so that 's good , i I think , in the sense that I think Andreas meant the question , phd b: That 's that 's good , yeah , cuz the overall rate is phd c:  phd g: Yeah . postdoc f: I should also say I did a simplifying ,  count in that if A was speaking B overlapped with A and then A came back again and overlapped with B again , I I didn't count that as a three - person overlap , I counted that as a two - person overlap , and it was A being overlapped with by D . postdoc f: Because the idea was the first speaker had the floor and the second person started speaking and then the f the first person reasserted the floor kind of thing . postdoc f: These are simplifying assumptions , didn't happen very often , there may be like three overlaps affected that way in the whole thing . grad h: Cuz i i I find it interesting that there were a large number of overlaps and they were all two - speaker . grad h:  what I thought what I would have thought in is that when there were a large number of overlaps , it was because everyone was talking at once , but  apparently not . postdoc f: Yeah , there 's a lot of backchannel , a lot o a lot of grad h: This is really interesting data . phd b: I think what 's really interesting though , it is before d saying " yes , meetings have a lot of overlaps " is to actually find out how many more we have than two - party . postdoc f: I think so too , I think phd b: Cuz in two - party conversations , like Switchboard , there 's an awful lot too if you just look at backchannels , if you consider those overlaps ? it 's also ver it 's huge . It 's just that people haven't been looking at that because they 've been doing single - channel processing for speech recognition . professor d:  -  ? phd b: So , the question is , you know , how many more overlaps do you have of , say the two - person type , by adding more people . professor d: Well , but see , I find it interesting even if it wasn't any more , phd b: So . professor d: because since we were dealing with this full duplex sort of thing in Switchboard where it was just all separated out we just everything was just nice , phd b:  -  ? professor d: so that so the issue is in in a situation where th that 's phd b: Well , it 's not really " nice " . So if you were actually having ,  depends what you 're doing , if Right now we 're do we have individual mikes on the people in this meeting . So the question is , you know " are there really more overlaps happening than there would be in a two - person party " . professor d:  -  ? phd b: And and there well may be , but professor d: Let let m let me rephrase what I 'm saying cuz I don't think I 'm getting it across . What what I what I shouldn't use words like " nice " because maybe that 's too i too imprecise . But what  is that ,  in Switchboard , despite the many many other problems that we have , one problem that we 're not considering is overlap . And what we 're doing now is , aside from the many other differences in the task , we are considering overlap and one of the reasons that we 're considering it , you know , one of them not all of them , one of them is that w  at least , you know I 'm very interested in the scenario in which ,  both people talking are pretty much equally audible , and from a single microphone . And so , in that case , it does get mixed in , and it 's pretty hard to jus to just ignore it , to just do processing on one and not on the other . phd b: I I agree that it 's an issue here but it 's also an issue for Switchboard and if you think of meetings being recorded over the telephone , which I think , you know , this whole point of studying meetings isn't just to have people in a room but to also have meetings over different phone lines . phd b: Maybe far field mike people wouldn't be interested in that but all the dialogue issues still apply , professor d:   phd b: so if each of us was calling and having a meeting that way you kn you know like a conference call . And , just the question is , y you know , in Switchboard you would think that 's the simplest case of a meeting of more than one person , professor d:   phd b: and I 'm wondering how much more overlap of the types that that Jane described happen with more people present . So it may be that having three people is very different from having two people or it may not be . professor d: I think what I 'm All I 'm s really saying is that I don't think we were considering that in Switchboard . But  but but professor d: Were you ? grad h: Though it wasn't in the design . professor d: Were you were you were you were you measuring it ?  , w w were phd b: There there 's actually to tell you the truth , the reason why it 's hard to measure is because of so , from the point of view of studying dialogue , which Dan Jurafsky and Andreas and I had some projects on , you want to know the sequence of turns . phd b: So what happens is if you 're talking and I have a backchannel in the middle of your turn , and then you keep going what it looks like in a dialogue model is your turn and then my backchannel , professor d: Yeah . professor d: Yeah ? phd b: So , for things like language modeling or dialogue modeling it 's We know that that 's wrong in real time . professor d: Yeah ? phd b: But , because of the acoustic segmentations that were done and the fact that some of the acoustic data in Switchboard were missing , people couldn't study it , but that doesn't mean in the real world that people don't talk that way . phd b: Well , we 've als professor d: And and and , you you maybe wanted to look at it before but , for these various technical reasons in terms of how the data was you weren't . professor d: So that 's why it 's coming to us as new even though it may well be you know , if your if your hypothes The hypothesis you were offering eh phd b:  . professor d: Right ? if it 's the null poth hypothesis , and if actually you have as much overlap in a two - person , we don't know the answer to that . The reason we don't know the answer to is cuz it wasn't studied and it wasn't studied because it wasn't set up . Right ? phd b: Yeah , all I meant is that if you 're asking the question from the point of view of what 's different about a meeting , studying meetings of , say , more than two people versus what kinds of questions you could ask with a two - person meeting . professor d:  -  ? phd b: It 's important to distinguish that , you know , this project is getting a lot of overlap but other projects were too , but we just couldn't study them . Right ? phd b: Well , there is a high rate , professor d: We do kn we don't know the numbers . It 's but I don't know how high , in fact phd a: Well , here I have a question . professor d: See , i i le let me t  , my point was just if you wanted to say to somebody , " what have we learned about overlaps here ? " just never mind comparison with something else , phd b:   professor d: what we 've learned about is overlaps in this situation , is that the first the first - order thing I would say is that there 's a lot of them . professor d: In in the sense that i if you said if i i i phd b: Yeah , I I don't di I agree with that . professor d: In a way , I guess what I 'm comparing to is more the common sense notion of how how much people overlap .  you know the fact that when when when , Adam was looking for a stretch of of speech before , that didn't have any overlaps , and he w he was having such a hard time and now I look at this and I go , " well , I can see why he was having such a hard time " . All I 'm saying is that from the professor d: I 'm saying if I I 'm saying if I have this complicated thing in front of me , and we sh which , you know we 're gonna get much more sophisticated about when we get lots more data , But Then , if I was gonna describe to somebody what did you learn right here , about , you know , the the modest amount of data that was analyzed I 'd say , " Well , the first - order thing was there was a lot of overlaps " . In fact and it 's not just an overlap bunch of overlaps second - order thing is it 's not just a bunch of overlaps in one particular point , but that there 's overlaps ,  throughout the thing . phd b: I 'm just saying that it may the reason you get overlaps may or may not be due to sort of the number of people in the meeting . phd b: because some of the data say Switchboard , which isn't exactly the same kind of context ,  these are two people who don't know each other and so forth , But we should still be able to somehow say what what is the added contra contribution to sort of overlap time of each additional person , or something like that . professor d: Yeah , that would be good to know , phd a: What professor d: but w we postdoc f: OK , now . postdoc f: And the reason is because I think there 's a limit there 's an upper bound on how many you can have , simply from the standpoint of audibility . postdoc f: children don't adjust so well , if a truck goes rolling past , adults will well , depending , but mostly , adults will will will hold off to what to finish the end of the sentence till the till the noise is past . postdoc f: And I think we generally do monitor things like that , about whether we whether our utterance will be in the clear or not . postdoc f: And partly it 's related to rhythmic structure in conversation , so , you know , you you t Yeah , this is d also  , people tend to time their their their ,  when they come into the conversation based on the overall rhythmic , ambient thing . And and , just to finish this , that  That I think that there may be an upper bound on how many overlaps you can have , simply from the standpoint of audibility and how loud the other people are who are already in the fray . Now if it 's just backchannels , people may be doing that with less intention of being heard , just sort of spontaneously doing backchannels , in which case that those might there may be no upper bound on those . phd g: I I have a feeling that backchannels , which are the vast majority of overlaps in Switchboard , don't play as big a role here , because it 's very unnatural I think , to backchannel if in a multi - audience you know , in a multi - person audience . It 's interesting , so if you watch people are going like Right right , like this here , phd g: Right . professor d: u phd g: But but , it 's sort of odd if one person 's speaking and everybody 's listening , and it 's unusual to have everybody going "  - huh ,  - huh " professor d: Actually , I think I 've done it a fair number of times today . So so actually ,  That 's in part because the nodding , if you have visual contact , the nodding has the same function , but on the phone , in Switchboard you you that wouldn't work . Your mike is phd a: So , in the two - person conversations , when there 's backchannel , is there a great deal of overlap in the speech ? grad h: That is an earphone , so if you just put it so it 's on your ear . phd a: or Cuz my impression is sometimes it happens when there 's a pause , phd b: Yes . phd a: you know , like you you get a lot of backchannel , when somebody 's pausing phd b: Yes . phd b: Sorry , what were you saying ? phd a: It 's hard to do both , huh ?  no , when when when there 's backchannel , just I was just listening , and and when there 's two people talking and there 's backchannel it seems like ,  the backchannel happens when , you know , the pitch drops and the first person phd b: Oh . phd a: and a lot of times , the first person actually stops talking and then there 's a backchannel and then they start up again , and so I 'm wondering about h I just wonder how much overlap there is . Is there a lot ? phd b: I think there 's a lot of the kind that Jose was talking about , where  , this is called " precision timing " in conversation analysis , where they come in overlapping , but at a point where the information is mostly complete . So all you 're missing is some last syllables or something or the last word or some highly predictable words . phd a: But maybe a just a small overlap ? phd b: But you know , from information flow point of view it 's not an overlap in the predictable information . phd g: So so phd b: Well , that 's exactly , exactly why we wanted to study the precise timing of overlaps ins in  Switchboard , professor d: Yeah .  , to distinguish between , say , backchannels precision timing Sort of you know , benevolent overlaps , and and and w and and sort of ,  I don't know , hostile overlaps , where someone is trying to grab the floor from someone else . I ju I I think that in this meeting I really had the feeling that wasn't happening , that the hostile hostile type . phd g: I could imagine that as there 's a fair number of  cases where , and this is sort of , not really hostile , but sort of competitive , where one person is finishing something and you have , like , two or three people jumping trying to trying to trying to ,  grab the next turn . phd g: And so it 's not against the person who talks first because actually we 're all waiting for that person to finish . professor d: I have a feeling most of these things are that that are not a benevolent kind are are are ,   are are competitive as opposed to real really really hostile . phd a:  postdoc f: Well , there are various things , you you have the professor d:  a vote vote in Florida . phd e: Voting for professor d: o one thing I I wanted to or you can tell a good joke and then everybody 's laughing and you get a chance to g break in . You know , the other thing I was thinking was that ,  these all these interesting questions are , of course , pretty hard to answer with ,  u you know , a small amount of data . professor d: So ,  I wonder if what you 're saying suggests that we should make a conscious attempt to have ,  a a fair number of meetings with ,  a smaller number of people . Right ?  we most of our meetings are  , meetings currently with say five , six , seven , eight people Should we really try to have some two - person meetings , or some three - person meetings and re record them just to to to beef up the the statistics on that ? postdoc f: That 's a control . Well , it seems like there are two possibilities there ,  i it seems like if you have just two people it 's not really , y like a meeting , w is not as similar as the rest of the of the sample . It depends on what you 're after , of course , but It seems like that would be more a case of the control condition , compared to ,  an experimental condition , with more than two . professor d: Well , Liz was raising the question of of whether i it 's the number there 's a relationship between the number of people and the number of overlaps or type of overlaps there , postdoc f:   professor d: and ,  If you had two people meeting in this kind of circumstance then you 'd still have the visuals . professor d: Yeah , acoustic is fine , but phd g: If if the goal were to just look at overlap you would you could serve yourself save yourself a lot of time but not even transcri transcribe the words . phd b: Well , I was thinking you should be able to do this from the acoustics , on the close - talking mikes , grad h: Yep . grad h: Well , that 's the that was my my status report , phd b: right ? postdoc f: You 've been working on that .  , not as well as what  , you wouldn't be able to have any kind of typology , obviously , professor d: Yeah . professor d: But what what do you think about that ? Do you think that would be useful ? I 'm just thinking that as an action item of whether we should try to record some two - person meetings or something . phd b: I guess my my first comment was ,  only that  we should n not attribute overlaps only to meetings , but maybe that 's obvious , maybe everybody knew that , professor d: Yeah . phd b: but that in normal conversation with two people there 's an awful lot of the same kinds of overlap , and that it would be interesting to look at whether there are these kinds of constraints that Jane mentioned , that what maybe the additional people add to this competition that happens right after a turn , professor d:   phd b: you know , because now you can have five people trying to grab the turn , but pretty quickly there 're they back off and you go back to this sort of only one person at a time with one person interrupting at a time . To answer your question I it I don't think it 's crucial to have controls but I think it 's worth recording all the meetings we can . phd b: D I wouldn't not record a two - person meeting just because it only has two people . phd g: Could we Could we ,  we have have in the past and I think continue will continue to have a fair number of  phone conference calls . phd g: And , and as a to ,  as another c c comparison condition , we could  see what what what happens in terms of overlap , when you don't have visual contact . phd g: So ,  phd b: Can we actually record ? grad h: It just seems like that 's a very different thing than what we 're doing . phd g: Or , this is getting a little extravagant , we could put up some kind of blinds or something to to remove ,  visual contact . grad h: Barriers ! phd b: That 's what they did on Map Task , you know , this Map Task corpus ? They ran exactly the same pairs of people with and without visual cues and it 's quite interesting . professor d: Well , we we record this meeting so regularly it wouldn't be that  a little strange . That would professor d: Oh , th that was the other thing , weren't we gonna take a picture at the beginning of each of these meetings ? grad h: what I had thought we were gonna do is just take pictures of the whiteboards . phd a: Why why do we want to have a picture of the meeting ? phd b: Ee - you mean , transc no postdoc f: The because you get then the spatial relationship of the speakers . postdoc f: And that could be phd g: Well , you could do that by just noting on the enrollment sheet the the seat number . phd a: We can can't you figure it out from the mike number ? grad h: No . And even the jacks , I 'm sitting here and the jack is over in front of you . phd g: Yeah , but It 's it would be trivial grad h: It would be another task . grad h: So I 'm gonna put little labels on all the chairs with the seat number . grad h: But ,  phd g: Put them Like , put them on the table where they phd e: The chair Yeah . postdoc f: But you know , they the s the linguistic anthropologists would say it would be good to have a digital picture anyway , phd a: Just remembered a joke . Posture , and we could like , you know , block out the person 's face or whatever phd g: What people were wearing . postdoc f: but but , you know , these are important cues , phd g: Oh , Andreas was phd a: How big their heads are . postdoc f:  the the how a person is sitting is professor d: But if you just f But from one picture , I don't know that you really get that . postdoc f: It 'd be better than nothing , is is i Just from a single picture I think you can tell some aspects . professor d: Think so ? postdoc f:  I I could tell you  , if I if I 'm in certain meetings I notice that there are certain people who really do eh The body language is very  is very interesting in terms of the dominance aspect . postdoc f: But it 's just , you know , the the body phd a: He agreed . postdoc f: you know ? grad h: Of course , the where we sit at the table , I find is very interesting , that we do tend to cong to gravitate to the same place each time . I 'm sitting here so that I can run into the room if the hardware starts , you know , catching fire or something . phd g: Oh , no , you you just like to be in charge , that 's why you 're sitting grad h: I just want to be at the head of the table . grad h: Yeah , I 've been playing with , using the close - talking mike to do to try to figure out who 's speaking . So my first attempt was just using thresholding and filtering , that we talked about about two weeks ago , and so I played with that a little bit , and it works O K , except that it 's very sensitive to your choice of your filter width and your threshold . So if you fiddle around with it a little bit and you get good numbers you can actually do a pretty good job of segmenting when someone 's talking and when they 're not . But if you try to use the same paramenters on another speaker , it doesn't work anymore , even if you normalize it based on the absolute loudness . phd b: But does it work for that one speaker throughout the whole meeting ? grad h: It does work for the one speaker throughout the whole meeting . grad h: What do you mean ? phd a: wh what was the grad h: The algorithm was ,  take o every frame that 's over the threshold , and then median - filter it , and then look for runs . grad h: So there was a minimum run length , phd a: Every frame that 's over what threshold ? grad h: so that A threshold that you pick . grad h: So you take a each frame , and you compute the energy and if it 's over the threshold you set it to one , and if it 's under the threshold you set it to zero , so now you have a bit stream of zeros and ones .  well , actually I guess depends on what you mean by long , you know , tenth of a second sorts of numbers .  and that 's to average out you know , pitch , you know , the pitch contours , and things like that . postdoc f: OK grad h: And that works O K , if you fil if you tune the filter parameters , if you tune how long your median filter is and how high you 're looking for your thresholds . phd a: Did you ever try running the filter before you pick a threshold ? grad h: No . OK and then the other thing I did , was I took Javier 's speaker - change detector acoustic - change detector , and I implemented that with the close - talking mikes , and unfortunately that 's not working real well , and it looks like it 's the problem is he does it in two passes , the first pass is to find candidate places to do a break . And he does that using a neural net doing broad phone classification and he has the the ,  one of the phone classes is silence . And what looks like it 's happening is that the even on the close - talking mike the broad phone class classifier 's doing a really bad job . Does an do you remember , Morgan , was it Broadcast News ? professor d: I think so , yeah . grad h:  So , at any rate , my next attempt , which I 'm in the midst of and haven't quite finished yet was actually using the  , thresholding as the way of generating the candidates . Because one of the things that definitely happens is if you put the threshold low you get lots of breaks . But , like , it could be someone who isn't the person here , but the person over there or it can be the person breathing . But all of this is close - talking mike , so it 's ,  just just trying to get some ground truth . phd e: Only with eh  , but eh I I I think , eh when when , y I I saw the the the the speech from PDA and , eh close talker . phd e:  but eh I but eh I I  that eh eh in the in the mixed file you can find ,  zone with , eh great different , eh level of energy . not phd e:  I I think for , eh algorithm based on energy , eh , that  h mmm , more or less , eh , like eh eh , mmm , first sound energy detector . When y you the detect the the the first at at the end of of the detector of , ehm princ  . What is the the name in English ? the the , mmm , the de detector of , ehm of a word in the in the s in an isolated word in in the background That ,  grad h: I 'm I 'm not sure what you 're saying , can you try phd e:  that when when you use , eh eh any phd a: I think he 's saying the onset detector . phd e: I I think it 's probably to work well eh , because , eh you have eh , in the mixed files a great level of energy . And probably is not so easy when you use the the PDA , eh that Because the signal is , eh the in the e energy level . I think that ,  grad h: Ah , in the clo in the P D A , you mean ? phd e: In the PDA . I think the , nnn , the , nnn , that the idea of using a neural network to to get a broad class of phonetic , eh from , eh  a candidate from the the the speech signal . If you have , eh  , I 'm considering , only because Javier , eh only consider , eh like candidate , the , nnn , eh the silence , because it is the the only model , eh eh , he used that , eh eh nnn , to detect the the possibility of a a change between the between the speaker , grad h: Right . phd e:  another another research thing , different groups , eh working , eh on Broadcast News prefer to , eh to consider hypothesis eh between each phoneme . phd e: Because , I I I think it 's more realistic that ,  only consider the the the the silence between the speaker . phd e: I I found that the , eh silence in in many occasions in the in the speech file , but , eh when you have , eh eh , two speakers together without enough silence between between them , eh I think eh is better to use the acoustic change detector basically and I I I IX or , mmm , BIC criterion for consider all the frames in my opinion . Yeah , the you know , the reason that he ,  just used silence was not because he thought it was better , it was it was it was the place he was starting . professor d: and ,  e e you know , as as as is in your case , if you 're here for only a modest number of months you try to pick a realistic goal , phd e: Yeah , yeah , yeah , yeah . professor d: But his his goal was always to proceed from there to then allow broad category change also . But , eh do do you think that if you consider all the frames to apply the the , eh the BIC criterion to detect the the the different acoustic change , eh between speaker , without ,  with ,  silence or with overlapping , I think like like , eh eh a general , eh eh way of process the the acoustic change . phd e: An - and then , eh eh without considering the you you you ,  you can consider the energy like a another parameter in the in the feature vector , eh . And if , if you do that , eh eh , with a BIC  criterion for example , or with another kind of , eh of distance in a first step , and then you , eh you get the , eh the hypothesis to the this change acoustic , eh to po process grad h: Right . phd e: Because , eh eh , probably you you can find the the eh a small gap of silence between speaker with eh eh a ga mmm , small duration Less than , eh two hundred milliseconds for example professor d:   phd e: and apply another another algorithm , another approach like , eh eh detector of ene , eh detector of bass - tone energy to to consider that , eh that , eh zone . of s a small silence between speaker , or another algorithm to to process , eh the the segment between marks eh founded by the the the BIC criterion and applied for for each frame . phd e: I think is , eh nnn , it will be a an an a more general approach the if we compare with use , eh a neural net or another , eh speech recognizer with a broad class or or narrow class , because , in my opinion eh it 's in my opinion , eh if you if you change the condition of the speech , if you adjust to your algorithm with a mixed speech file and to , eh to , eh adapt the neural net , eh used by Javier with a mixed file . phd e:  With a m mixed file , grad h: With the what file ? phd a: " Mixed " . And and then you you , eh you try to to apply that , eh , eh , eh , speech recognizer to that signal , to the PDA , eh speech file , I I think you will have problems , because the the the the condition you you will need t t I I suppose that you will need to to to retrain it . professor d: u Look , I I think this is a One once It 's a I used to work , like , on voiced on voice silence detection , you know , and this is this kind of thing . professor d:  If you have somebody who has some experience with this sort of thing , and they work on it for a couple months , they can come up with something that gets most of the cases fairly easily . Then you say , " OK , I don't just wanna get most of the cases I want it to be really accurate . So , the p the problem is is that if you say , " Well I I have these other data over here , that I learn things from , either explicit training of neural nets or of Gaussian mixture models or whatever . Well , what does that mean ? That that means you set some thresholds somewhere or something , phd e: Yeah . professor d: right ? and and so where do you get your thresholds from ? phd e: Yeah . So you always have this problem , you 're going to new data  H how are you going to adapt whatever you can very quickly learn about the new data ?  , if it 's gonna be different from old data that you have ? And I think that 's a problem with this . grad h: Well , also what I 'm doing right now is not intended to be an acoustic change detector for far - field mikes . What I 'm doing is trying to use the close - talking mike and just use Can - and just generate candidate and just try to get a first pass at something that sort of works . phd g: Or grad h: and I haven't spent a lot of time on it and I 'm not intending to spend a lot of time on it . phd g: but ,  I can imagine  building a  model of speaker change detection that takes into account both the far - field and the  actually , not just the close - talking mike for that speaker , but actually for all of th for all of the speakers . phd g:  If you model the the effect that me speaking has on your microphone and everybody else 's microphone , as well as on that , and you build ,  basically I think you 'd you would build a an  that has as a state space all of the possible speaker combinations grad h: All the Yep . professor d: But Actually , Andreas may maybe maybe just something simpler but but along the lines of what you 're saying , grad h: Anyway . professor d: I was just realizing , I used to know this guy who used to build , mike mixers automatic mike mixers where , you know , t in order to able to turn up the gain , you know ,  as much as you can , you you you lower the gain on on the mikes of people who aren't talking , phd g: Mmm . professor d: right ? And then he had some sort of reasonable way of doing that , phd g:   professor d: but  , what if you were just looking at very simple measures like energy measures but you don't just compare it to some threshold overall but you compare it to the energy in the other microphones . grad h: I was thinking about doing that originally to find out who 's the loudest , and that person is certainly talking . I I have found that when when I I analyzed the the speech files from the , eh mike , eh from the eh close eh microphone , eh I found zones with a a different level of energy . because , eh eh depend on the position of the of the microph of the each speaker to , eh , to get more o or less energy i in the mixed sign in the signal . and then , if you consider energy to to detect overlapping in in , and you process the the in the the the speech file from the the the mixed signals . I I think it 's it 's difficult ,  only to en with energy to to consider that in that zone We have eh , eh , overlapping zone Eh , if you process only the the energy of the , of each frame . professor d: Well , it 's probably harder , but I I think what I was s nnn noting just when he when Andreas raised that , was that there 's other information to be gained from looking at all of the microphones and you may not need to look at very sophisticated things , phd e: Yeah . professor d: because if there 's if most of the overlaps you know , this doesn't cover , say , three , but if most of the overlaps , say , are two , if the distribution looks like there 's a couple high ones and and the rest of them are low , phd e: Yeah . professor d: by the way , I had an idea with while I was watching Chuck nodding at a lot of these things , is that we can all wear little bells on our heads , so that then you 'd know that phd e: Yeah . That 's cute ! phd b: I think that 'd be really interesting too , with blindfolds . The question is , like whether grad h: " what are you nodding about ? " phd b: Well , trying with and with and without , yeah . phd a: Actually , I saw a  a woman at the bus stop the other day who , was talking on her cell phone speaking Japanese , and was bowing . phd b: It 's very difficult if you try while you 're trying , say , to convince somebody on the phone it 's difficult not to move your hands . I still think we should try a a meeting or two with the blindfolds , at least of this meeting that we have lots of recordings of grad h:   phd b: maybe for part of the meeting , we don't have to do it the whole meeting . It 'll be too hard to make barriers , I was thinking because they have to go all the way professor d: W Yeah . postdoc f: Actually well also I I can say I made barr barriers for so that the stuff I was doing with Collin wha which just used ,  this kind of foam board . You can you can masking tape it together , these are you know , pretty l large partitions . phd b: But then we also have these mikes , is the other thing I was thinking , so we need a barrier that doesn't disturb the sound , postdoc f: It 's true , it would disturb the ,  the the long - range grad h: The acoustics . phd b: it sounds weird but but you know it 's it 's cheap and ,  Be interesting to have the camera going . grad h: Yeah , that 's right , we didn't tell them we would be blindfolding . professor d: That 's postdoc f: " Do you mind being blindfolded while you 're interviewed ? " professor d: that 's that 's that 's the one that we videotape .  I did have this other agenda item which is ,  @ @ it 's  a list which I sent to  a couple folks , but  I wanted to get broader input on it , So this is the things that I think we did in the last three months obviously not everything we did but but sort of highlights that I can can tell s some outside person , you know , what what were you actually working on .  in no particular order  , one , ten more hours of meeting r meetings recorded , something like that , you know from from ,  three months ago .  , pilot data put together and sent to IBM for transcription ,  next batch of recorded data put together on the CD - ROMs for shipment to IBM , grad h: Hasn't been sent yet , but It 's getting ready .  human subjects approval on campus ,  and release forms worked out so the meeting participants have a chance to request audio pixelization of selected parts of the spee their speech .  preliminary analysis of overlaps in the pilot data we have transcribed , and exploratory analysis of long - distance inferences for topic coherence , that was I was wasn't sure if those were the right way that was the right way to describe that because of that little exercise that that you and and Lokendra did . postdoc f: What was that called ? professor d: I well , I I 'm probably saying this wrong , but what I said was exploratory analysis of long - distance inferences for topic coherence .  so ,  I a lot of that was from , you know , what what what you two were doing so I I sent it to you , and you know , please mail me , you know , the corrections or suggestions for changing grad h:   professor d: I I don't want to make this twice it 's length but but you know , just im improve it . professor d: " Bunch of stuff for s " OK , maybe maybe send me a sentence that 's a little thought through about that . grad h: So , OK , I 'll send you a sentence that doesn't just say " a bunch of " ? professor d: " Bunch of stuff " , yeah , " stuff " is probably bad too , grad h: Yep .  and and you know , I sort of threw in what you did with what Jane did on in under the ,   preliminary analysis of overlaps .   Thilo , can you tell us about all the work you 've done on this project in the last ,  last three months ? phd e: Yeah . Wh - what is " audio pixelization " ? professor d: audio pix wh he did it , so why don't you explain it quickly ? grad h: It 's just ,  beeping out parts that you don't want included in the meeting so , you know you can say things like , " Well , this should probably not be on the record , but beep " phd c: OK , OK . We we we spent a a a fair amount of time early on just talk dealing with this issue about op w e e we realized , " well , people are speaking in an impromptu way and they might say something that would embarrass them or others later " , and , how do you get around that phd c:  OK . professor d: so in the consent form it says , well you we will look at the transcripts later and if there 's something that you 're unhappy with , yeah . professor d: But you don't want to just totally excise it because   , well you have to be careful about excising it , how how you excise it keeping the timing right and so forth so that at the moment tho th the idea we 're running with is is h putting the beep over it . grad h: if someone is listening to it , there 's no mistake that it 's been beeped out , phd c: Yeah . You can you know , you could make a m as long as you keep using the same beep , people could make a model of that beep , postdoc f:  . And I use it 's it 's ,  it 's an A below middle C beep , phd b: I think the beep is a really good idea . phd b: Also phd a: Yeah , it 's more obvious that there was something there than if there 's just silence . professor d: Yeah , that  , he 's he 's removing the old thing phd e: Yeah professor d: and and and grad h: Yep . But  if you just replaced it with silence , it 's not clear whether that 's really silence or grad h: Yeah , it 's not phd c: Yeah . postdoc f: Well , the other thing that you know ,  the the alternative might be to s grad h: Yeah . grad h: Sorry , and then the DARPA meeting I just excised completely , postdoc f: Yeah . professor d: Alright , so ,  I think we should , go on to the digits ? postdoc f: I have one concept a t I I want to say , which is that I think it 's nice that you 're preserving the time relations , grad h: OK . postdoc f: s so you 're you 're not just cutting you 're not doing scissor snips . phd b: So I guess if there 's an overlap , like , if I 'm saying something that 's bleepable and somebody else overlaps during it they also get bleeped , too ? professor d: Yeah .  I d I did before we do the digits , I did also wanna remind people ,  please do send me , you know ,  thoughts for an agenda , grad h: Agenda ? professor d: yeah that that would be that 'd be good . professor d: Eh So that , people 's ideas don't get grad h: Thursday crept up on me this week . professor d: yeah , well it does creep up , doesn't it ? phd b: And , I wanted to say , I think this is really interesting analysis . grad h: I was gonna say " can you do that for the other meetings , phd b: It 's neat . phd a: Actually actually I I thought that 's what you were giving us was another meeting and I was like , " Oh , OK ! " phd b: Does it take postdoc f: Thank you . I have the script now , so , it can work off the ,  other thing , grad h: It 's As soon as we get labels , yep . phd a: But it has to be hand - labeled first ? postdoc f: but  , well , yeah . postdoc f: But I I just worked off of my phd b: It 's really neat . I think what I what this has , caused me so this discussion caused me to wanna subdivide these further . I 'm gonna take a look at the ,  backchannels , how much we have anal I hope to have that for next time . grad h: Yeah , my my algorithm worked great actually on these , but when you wear it like that or with the  , lapel or if you have it very far from your face , that 's when it starts failing . phd b: Well , I can wear it ,  if you grad h: It doesn't matter . grad h: right ? I I don't want to change the way we do the meeting . grad h: It 's  so , it was just a comment on the software , not a comment on prescriptions on how you wear microphones . grad h: Get the bolts , " whh whh " postdoc f: Let 's do it . postdoc f: Do you want us to put a mark on the bottom of these when they 've actually been read , or do you just i i the only one that wasn't read is is known , so we don't do it