B: and then over a period of time as the research happens on this we might develop some of these categories that would be generally useful
D: Some more .
B: and it it sounds like a good thing to do .
F: And D: OK .
F: Interesting . Good .
F: And systematize also maybe the encoding of it if as we find some are useful and some are not so useful .
D: Nope .
D: Yeah , so what I have right now is all pairwise ,
C: Mm - hmm .
D: so that 's probably not sufficient .
F: I think you don't need to have that much structure built in , and that would be less time - consuming to do it this way first , and then find out what structure 's needed .
D: Yeah , maybe not .
D: Yep . OK .
E: what about all of the meetings that have been recorded so far that without this ?
E: Will Will somebody go back and fill that in , or we just won't have it , or D: Well , I think that in a lot of the meetings people won't be filling this out anyway .
D: So really what I want to have is a place that if a researcher wants that information , they can add it .
E: Mmm .
B: Almost all of the meetings that we have are recorded with roughly this group
D: Right .
B: uh and uh you know and a fair minority with the group that we have in the morning ,
B: so this is not a lot of people .
B: We could easily There 's just a couple other meetings that
D: Right .
D: So it We can regenerate a lot of this information .
E: So it wouldn't be that hard to go back and E: Yeah .
B: Yeah .
D: But Even if we can't , generate it , I think it 's alright to have some of the sections in your database blank , that this information was not collected for this meeting .
E: Mm - hmm .
D: And there 's just not gonna be a way around that .
D: That 's just gonna happen
D: like the N S A meeting , the networks meeting .
D: I don't think they 're gonna want to fill out that information .
E: Yeah .
D: So .
D: OK , enough on forms .
B: OK , so we 're running kind of late ,
C: OK .
B: so we 'll probably wanna zip through some other things , maybe do some more time on on Jose 's stuff later ,
B: um uh Status of Transcription ,
B: Uh , I guess , you know , we we recently woke up um our friends
B: and uh
B: so I I guess you 've sent off this the uh CD - ROMs
B: and you got some response from from uh IBM about D: They haven't received them yet ,
D: but I did get a response from them saying " Yes we 're still alive ,
D: we haven't done anything yet . " basically .
B: Yeah .
D: And uh B: Was there some suggestion that they might soon , or D: Yeah .
B: Yeah .
D: So so , they 've transferred the responsibility to someone else .
D: I can't remember his name McCoond ?
B: McCoond
D: McCoond ?
B: Yeah .
D: Something like that ,
D: but uh D: And so he sent me email and said he 's gonna be starting to work on it ,
D: and I haven't received anything else .
B: Great , OK .
D: So .
B: Well , that was that one .
C: Mm - hmm .
B: Um ,
D: So tomorrow I 'll send a email and just ask if he received it ,
D: he should be getting it today .
D: Actually he should have gotten it yesterday .
D: But . Give him a day .
B: um , and I think we should just sort of look at this thing from you , off - line Jose
B: cuz it 's getting kind of late
C: Yeah .
B: but I mean I think the bottom line just glancing through it is that there 's there 's a lot of overlap , in just energy related things
C: Yeah .
B: and so you need something else .
B: I mean , that 's sort of C: OK .
B: right ?
B: And uh what the so I 'd E: What about that error that that uh the the supposed lub
B: The residual error ?
E: no , the supposed bug that was that we were E: We were guessing you had a bug before
E: because you went through and you concatenated all of these things together F: Mm - hmm .
C: Ah , yeah yeah .
C: I I found the error , and I repeated the experiments
C: eh and here , eh you can find the new results eh with E: Ah .
B: OK , so this is post bug fix .
C: Yeah , yeah .
C: OK ?
B: Good , OK .
B: So um so I had two thought just glancing through this .
B: So , one is one is that um that uh we might wanna talk about normalization .
B: I mean , I don't know what normalizations you use .
B: Sometimes things have more overlap before you 've done some kind of normalization
B: and we might think about what different kinds of normalizations are possible .
C: Yeah .
B: And the other thing is that , if that isn't @ @ uh so much of the issue , then probably C: Yeah .
B: I mean , residual LPC energy and just plain energy are very closely related , even though they 're different
B: and um so um you really may need to go to something that looks at i in some sense harmonicity , or um um <nonvocalsound> relationship to or fit fit to pitch tracks on uh on either side of of it , for instance ?
C: Yeah .
D: Hmm .
C: Yeah .
B: You know , so if if there 's uh a really bad fit between where the pitch is going in this suppose uh hypothesized region of overlap , and the the and one or the other uh side , you know , then this this this might uh , suggest a C: Another side .
C: an alteration in th in the tracking way .
B: Excuse me ?
C: You mean that eh you ehm mmm C: I understand that eh you you mean eh to to study the alteration between the the tracking of the pitch eh if we compare the overlapping zone with a speaker zone .
B: I if you have two if you have two if you have uh two people speaking uh and there 's an overlap , then I mean the first thing is that there should be a mixture of harmonics uh during overlapped voice sections anyway .
C: in the co in the context .
C: Yeah .
C: Mm - hmm .
B: And um the any a measure that you have of how much of the energy is due to your best guess at a particular harmonic sequence , it 's also going to be a smaller fraction .
B: This is , you know , related to the harmonicity sort of thing .
C: Uh - huh .
B: And then the other thing I 'm saying is that if you look at the temporal evolution of the pitch there should be something like a discontinuity there .
C: Yeah .
C: Yeah .
B: Now , there 's other places where you 'll have discontinuities in pitch and speech ,
B: but but maybe it 'll have something to have a different character .
B: But it seems like energy energy tells you something ,
B: but uh if unless this is a normalization problem , it looks from your results like there 's so much overlap that certainly you need something that isn't energy - like , in addition to it , at least .
C: Uh - huh .
C: OK .
E: Are Are the things in the C: I don't normalicized eh in these er experiments . by the moment .
C: I don't normalicize .
B: OK , so the question is is would some kind of normalization help .
C: Yeah ?
B: You know , uh , i i it could be that if you normalized uh by the overall um energy uh for uh you know some some longer period of time or something , uh that that there would be more of a distinction .
C: Nnn .
C: Yeah .
C: The different zone .
B: uh , I I don't know .
E: So , uh , from from these charts , are C: Yeah .
E: hmm , I 'm just trying to remember where we were ,
E: but uh , were you were interested in finding out if you can tell the difference between overlap and single speakers by looking at the e energy ,
E: like say , for example , the mean of the energy .
E: Was that the idea ?
C: Yeah .
E: OK .
C: Yeah .
E: So , if you look at on this page that has uh frame energy ,
E: you look at the mean for the overlap , that 's forty - four point sixty - two versus the mean for the speakers , thirty - nine point sixty - two ,
C: Yeah .
C: Yeah .
E: so there seems to be more energy when there 's overlapping which makes sense ,
B: Which is the intuition ,
C: Yeah .
B: un unfortunately there 's a standard deviation of like ten ,
C: Yeah .
E: but C: Yeah .
E: Yeah ,
F: Oh .
B: so  For each one ,
C: The product of the deviation is the vari is the variance .
E: different than standard deviation .
B: yeah .
E: Yeah .
A: But are we assuming that you could know the energy for a speaker when there 's no overlap ?
A: I mean I can imagine a model where you say , OK , you know I 'm I 'm gonna give myself D: And    And And did is this from the mixed signal ?
A: it 's cheating ,
A: but not completely if they 're often not overlapping and normalized for the speaker
C: Yeah .
A: or is that not allowed .
A: I mean , what 's the B: Well , that 's what I was suggesting , that y you wanna do some kind of longer time normalization eh with the C: Yeah .
C: Yeah .
C: But B: even if you make a mistake occasionally , that that i is roughly i is there some corresponding to that speaker , so that if you , if if uh  if you
A: right
C: Yeah .
B: because that that 's gonna spread out these distributions really a lot , to not do any kind of normalization ,
A: right
B: and so that that could mask the effect somewhat .
E: so the um just one more clarifying question E: On the overlap category , is that overlapping of speakers only ?
C: Yeah .
E: It doesn't include some of the other sounds ?
C: No .
E: OK .
C: It 's a pu pure overlapping zone .
B: So that 's why it doesn't add up down C: I mean that eh overlapping between two , three , four eh eh speaker .
E: OK .
E: OK .
C: Th
C: Yeah .
D: OK , so there is some normalization already , in that they 've been volume equalized uh over the entire signal .
A: Right , right .
C: Yeah .
A: Right .
A: That 's what makes it seem difficult ,
C: Because they they 're A: because you 're already left with a high standard deviation
C: Yeah .
C: Yeah .
A: and they 're already roughly
A: so that each person 's hearable ,
A: right ? A: so sort of equally loud , perceptually anyway .
D: Right .
D: Right .
F: What about doing it with just the single channels ?
C: Sorry ?
F: What about ,
F: uh
C: Say this graphic ?
F: Well , I I 'm thinking so they 're raising the question about the fact that if you use the mixed signal , there 's already been a C: Yeah .
C: I use the mixed signal without normalization without normalization ,
D: Well , but the mixed signal already has a normalization in it .
F: I I 'm wo
C: but but
F: Exactly .
F: So what I 'm wondering is what about doing some of this with the single channel recordings .
C: Ah .
C: I I don't study the single channel yet
C: but eh is eh is an idea .
D: Well , I think it 'll be worse ,
D: right ,
D: I think the mixed C: to to begin to work with the eh with the one single channel . Now .
B: I have another another thought .
B: Um , this is this is frame energy ,
B: of course frame frames have a lot of variance to them
B: because different sounds are different loudnesses .
B: So I mean , what if you took uh one second chunks or something like that .
C: Yeah .
B: Or or maybe not even a second ,
B: maybe a quarter of a second or something ,
B: so that you would typically have two or three basically syllable kind of length A: Yeah , like two hundred milliseconds
B: Yeah .
A: or twenty to twenty - five frames ,
B: Right .
D: Or window it A: or
C: Yeah .
B: So then you you 'll you know D: hamming window .
B: Something ,
C: Yeah .
C: Yeah .
B: yeah .
B: So so I mean , so you know two hundred , three hundred four hundred mill
B: some longer chunk of time ,
B: and then you looked at the amount of energy in that .
B: and how does that vary over these different cases .
C: Uh - huh .
A: or voiced regions ,
A: I mean can we tell ?
B: or voiced regions .
F: Interesting .
A: can we sorry I mean , is it known whether something 's voiced here , or is it roughly estimate estimateable ?
B: Well , I mean , you could do a you could first determine that something was voiced or not .
A: I mean like if one speaker 's voicing and the other one isn't and they 're overlapping , do we know that the first speaker 's voicing , or not ?
B: Well , I mean the overlaps are typically more than just a tiny little bit of time
C: Ah .
B: and so uh there could be voiced and non - voiced pieces during the overlap .
D: Huh .
A: Mm - hmm .
A: But will we be able t to detect the voicing when there 's overlap ?
A: That 's what I just wondered .
B: I don't know ,
B: because it 's it 's this this has to do with these questions of harmonicity , and so forth ,
A: OK .
A: So if we could , or if you could do some of it ,
C: Yeah .
C: Yeah .
A: yeah ,
C: Yeah .
B: right ?
A: then um , voiced regions would be , like , probably more informative , for energy
B: Yeah .
A: because B: Right , so you could do something like , uh
A: and pi and pitch for that matter .
B: but B: Sure .
B: But I think what will be easy for him to do would be to do this thing of looking over a large enough region of time .
B: Cuz if you look over a large enough region of time , most of the time you 'll have some voiced a fair amount of voiced uh energy in it .
D: Right .
B: And so I think that would be D: And he has on his graph something like forty percent .
D: It looked like fifty percent or more are point two seconds or longer overlap
D: It 's li I think the last page .
C: Yeah .
B: Is it ?
F: That 'll also lessen the standard deviation .
C: Here in the yeah it it 's a new  new a new diapositive with information about the context for overlapping zone .
D: Nope , not the last page .
C: Um .
D: Where was it ?
D: It was one of the tables .
B: What ?
C: How define ?
B: Are you using a pitch detector in this yet , in your experiments ?
C: I Eh B: Do you have a pitch detector that you 're using ?
C: j my ?
B: Do you do you have a a pitch detector that you are using in these experiments ?
F: Pitch .
C: eh , I I 'm working now with pitch detector .
A: Well , it 's the fifth page , I guess .
B: Yeah .
C: I 'm working now , but eh I haven't eh results yet .
D: Fifth page .
C: But eh at this moment I I I prepare , I am I am preparing the the pitch eh eh tracking . pitch tracker eh algorithm that I have
D: Oh , it 's seventy - five percent !
B: Does does it have a voiced , unvoiced uh detector in it ,
C: but eh
B: uh ?
C: No .
B: It doesn't .
B: it just finds the pi it just finds the pitch even when it 's unvoiced ?
B: I mean , it must have some It has to tell you something ,
C: eh , I I A: It has to tell you .
A: It has to give you at least a zero , one ,
C: Yeah .
B: right ?
A: I mean uh d binary B: Yeah .
C: It 's based on uh a correlation between frame the utterance
B: Yeah .
C: but eh I don't know what is ehm eh B: Right .
C: because it 's no my my algorithm .
B: Yeah . So ordinarily in these things , it does tell you that it 's it 's voiced with a pitch of such - and - such .
C: Is B: And it will i i it will say something like zero
C: Uh - huh .
B: or you know it 'll have something that it says when it 's when it can't find a good good uh uh period ,
C: Uh - huh .
B: right ?
B: So it 's looking for some periodic behavior ,
B: and if it can't find a period it tells you for for most things like that .
C: Uh - huh .
C: For most @ @ .
B: So I mean , this is , getting back to what Liz was suggesting
B: which I think actually is good , the more I think of it ,
B: that that um for normalization you could do something like take voiced sections and normalize to equalize the energy in that .
B: I 'm just still going back to normalization , that even though it 's roughly normalized , uh , for overall gain , I think it it 's it it may not be normalized enough .
C: Uh - huh .
C: For b for both , for eh the frame energy and eh residular LPC eh energy too .
B: Uh , for both ,
B: yeah .
C: For both .
B: Yeah .
B: Yeah .
C: but eh I test different normalization
C: I eh
C: What kind of normalization ?
C: I I B: OK , so so I think I mean , @ @ this is something to to experimentally determine ,
C: This is C: Yeah .
B: but I mean , if um u
C: Yeah .
B: as I understand it , you have you have some regions that uh um are marked , as B: You have training data that you 're trying to find a threshold ,
B: right ?
B: and so um um if you take um all the things all the uh speak things marked " SPK " , and uh look for the voiced uh energy in them ,
C: Uh - huh .
C: Uh - huh .
B: and uh
B: well , just in in each case , take a take a take a frame ,
B: take the
B: uh uh
B: if it 's if it 's voiced , include it , B: Just separate out from the from the normal things ,
A: Mm - hmm , mm - hmm .
B: separate out voiced .
C: Yeah .
B: Right ?
B: Then you 're gonna end up with uh a a uh a mean uh standard devia deviation , and so forth .
B: And then do your normalization based on that .
C: Yeah .
C: Yeah .
B: And once you do that just for voiced ,
B: um um
B: I guess you 'd A: You can also look at just the voiced regions later .
A: I mean , that would be the most sensitive measure .
A: because you know , if you 're normalizing for voiced , and then you look at regions that are voiced with the same robustness in estimation of what voicing is , um , you should be much more sensitive to the to the overlaps then if you C: Yeah .
A: you know if you normalize but you look everywhere at at at fricatives or something you won't really know anything .
B: Yeah .
A: So I mean assuming if it 's true that the overlap regions are sort of long enough that there 's some voicing in each of them , by each of the speakers , then you should do very well if you do this .
B: Right .
B: Right .
B: Right .
A: But you 'll only know that in the re voiced regions
C: Hmm .
A: but those are close enough to the A: You know , there 's enough voicing going on and off uh that if you can capture the voicing and if you can capture it when there 's overlap then you should do quite well there .
C: Uh - huh .