professor g: Alright ! grad a: So , sorry about not professor g: We 're not crashing . postdoc f: I don't know if you 're professor g: Yeah , that 's right . postdoc f: maybe raise the issue of microphone ,  procedures with reference to the cleanliness of the recordings . professor g: OK , transcription , microphone issues postdoc f: And then maybe ask , th  , these guys . The we have great great , p steps forward in terms of the nonspeech - speech pre - segmenting of the signal . phd d: use grad a: but I 'm not sure if that 's of general interest or not . professor g:  phd b: Since ,  since I have to leave as usual at three - thirty , can we do the interesting stuff first ? postdoc f: I beg your pardon ? professor g: Well phd c: Which is ? grad a: What 's the interesting stuff ? postdoc f: I beg your pardon ? phd d: Yeah . professor g: But phd b: Well , I guess the work that 's been done on segmentation would be most phd c: Yeah .  , and , the other thing , which I 'll just say very briefly that maybe relates to that a little bit , which is that , one of the suggestions that came up in a brief meeting I had the other day when I was in Spain with , Manolo Pardo and Javier , Ferreiros , who was here before , was , why not start with what they had before but add in the non - silence boundaries . So , in what Javier did before when they were doing ,  h he was looking for , speaker change points . As a simplification , he originally did this only using silence as , a putative , speaker change point . professor g: And , he did not , say , look at points where you were changing broad sp  , phonetic class , for instance . professor g: And , so one of the things that they were pushing in d in discussing with me is , w why are you spending so much time , on the , feature issue , when perhaps if you sort of deal with what you were using before phd d:  - huh . professor g: and then just broadened it a bit , instead of just ta using silence as putative change point also ? phd d: Nnn , yeah . professor g: So then you 've got you already have the super - structure with Gaussians and H - you know , simple H M Ms and so forth . And you you might So there was a there was a little bit of a a a a difference of opinion because I I thought that it was it 's interesting to look at what features are useful . professor g: But , on the other hand I saw that the they had a good point that , if we had something that worked for many cases before , maybe starting from there a little bit Because ultimately we 're gonna end up with some s su kind of structure like that , phd d: Yeah . professor g: where you have some kind of simple  and you 're testing the hypothesis that , there is a change . Yeah , why don't we do the speech - nonspeech discussion ? postdoc f: Yeah . Do I I hear you you didn't phd c: Speech - nonspeech ? OK . phd c: so , what we basically did so far was using the mixed file to to detect s speech or nonspeech portions in that . phd c: And what I did so far is I just used our old Munich system , which is an  - ba based system with Gaussian mixtures for s speech and nonspeech . And it was a system which used only one Gaussian for silence and one Gaussian for speech . phd c: Adam , Dave , and I , we did , for that dialogue and I trained it on that . They they can use it or ? postdoc f: they they think it 's a terrific improvement . postdoc f: And , y you also did some something in addition which was , for those in which there was , quiet speakers in the mix . phd c: And so I did two mixtures , one for the loud speakers and one for the quiet speakers . grad a: And did you hand - label who was loud and who was quiet , or did you just ? phd c: I did that for for five minutes of one dialogue grad a: Right . phd c: It 's just our our old Munich , loudness - based spectrum on mel scale twenty twenty critical bands and then loudness . phd c: And four additional features , which is energy , loudness , modified loudness , and zero crossing rate . And so I did some some modifications in those parameters , basically changing the minimum minimum length for s for silence to have , er to have ,  yeah to have more or less , silence portions in inserted . grad a: But for overlap I imagine that doesn't work at all , phd c: Yeah . But it it saves so much time the the transcribers professor g:  grad a: Yep . W w we originally we did that professor g: Just phd c: but we saw , when we used it , f for our close - talking microphone , which yeah , for our for our recognizer in Munich we saw that w it 's it 's not it 's not so necessary . professor g: Yeah , I don't think it 's a big deal for this application , phd c: Yeah . But then there 's another thing that also Thilo 's involved with , which is ,  OK , and and also Da - Dave Gelbart .  we ,  regarding the representation of overlaps , because at present , because of the limitations of th the interface we 're using , overlaps are , not being encoded by the transcribers in as complete and , detailed a way as it might be , and as might be desired I think would be desired in the corpus ultimately . postdoc f: So we don't have start and end points at each point where there 's an overlap . So @ @ the limits of the over of of the interface are such that we were at this meeting we were entertaining how we might either expand the the interface or find other tools which already do what would be useful . Because what would ultimately be , ideal in my my view and I think  , I had the sense that it was consensus , is that , a thorough - going musical score notation would be the best way to go . Because you can have multiple channels , there 's a single time - line , it 's very clear , flexible , and all those nice things . So , I spoke I had a meeting with Dave Gelbart on on and he had , excellent ideas on how the interface could be modified to to do this kind of representation . But , he in the meantime you were checking into the existence of already , existing interfaces which might already have these properties .  , I talked with , Munich guys from from Ludwi - Ludwig Maximilians University , who do a lot of transcribing and transliterations . phd c: And they basically said they have they have , a tool they developed themselves and they can't give away , f it 's too error - prone , and had it 's not supported , a a a and professor g: Yeah . phd c: But , Susanne Bur - Burger , who is at se CMU , he wa who was formally at in Munich and w and is now at with CMU , she said she has something which she uses to do eight channels , trans transliterations , eight channels simultaneously , professor g: Excuse me . grad a: Well , maybe we should get it and if it 's good enough we 'll arrange Windows machines to be available . postdoc f: I also wanted to be sure  , I 've I 've seen the this this is called Praat , PRAAT , which I guess means spee speech in Dutch or something . phd c: Yeah , but then I 'm not sure that 's the right thing for us . professor g: The other thing , to keep in mind , we 've been very concerned to get all this rolling so that we would actually have data , postdoc f: Mmm , yeah . professor g: but , I think our outside sponsor is actually gonna kick in postdoc f:   So I don't know if we have a long - term need to do lots and lots of transcribing . I think we had a very quick need to get something out and we 'd like to be able to do some later because just it 's inter it 's interesting . But as far a you know , with with any luck we 'll be able to wind down the larger project . phd b: But you s grad a: What our decision was is that we 'll go ahead with what we have with a not very fine time scale on the overlaps . postdoc f: And and I was just thinking that , if it were possible to bring that in , like , you know , this week , then when they 're encoding the overlaps it would be nice for them to be able to specify when you know , the start points and end points of overlaps . postdoc f: And , so my my goal was w m my charge was to get eleven hours by the end of the month . And it 'll be I 'm I 'm I 'm clear that we 'll be able to do that . postdoc f: I sent it to ,  who did I send that to ? I sent it to a list and I thought I sent it to the e to the local list . postdoc f: You saw that ? So Brian did tell me that in fact what you said , that ,  that our that they are making progress and that he 's going that they 're going he 's gonna check the f the output of the first transcription and and professor g: basically it 's it 's all the difference in the world .  , basically it 's just saying that one of our one of our best people is on it , postdoc f: Yeah . So phd b: But about the need for transcription , postdoc f: Isn't that great ? phd b: don't we didn't we previously decide that the IBM transcripts would have to be checked anyway and possibly augmented ? professor g: So . grad a: Yeah , and Dave Gelbart did volunteer , postdoc f: Good . grad a: and since he 's not here , I 'll repeat it to at least modify Transcriber , which , if we don't have something else that works , I think that 's a pretty good way of going . My approach originally , and I 've already hacked on it a little bit it was too slow because I was trying to display all the waveforms . grad a: That if you just display the mix waveform and then have a user interface for editing the different channels , that 's perfectly sufficient . And and , Dan Ellis 's hack already allows them to be able to display different waveforms to clarify overlaps and things , grad a: No . They can only display one , postdoc f: so that 's already grad a: but they can listen to different ones . postdoc f: Oh , yes , but Well , yes , but what  is that , from the transcriber 's perspective , those two functions are separate . And Dan Ellis 's hack handles the , choice the ability to choose different waveforms from moment to moment . postdoc f:  grad a: The waveform you 're looking at doesn't change . postdoc f: Yeah , but that 's that 's OK , cuz they 're they 're , you know , they 're focused on the ear anyway . postdoc f: the hack to preserve the overlaps better would be one which creates different output files for each channel , grad a: Right . postdoc f: which then would also serve Liz 's request of having , you know , a single channel , separable , cleanly , easily separable , professor g:   Well , holidays may have interrupted things , cuz in in in They seem to want to get absolutely clear on standards for transcription standards and so forth with with us . I just think I told them to contact Jane that , if they had a grad a: Oh , OK . So is it cuz with any luck there 'll actually be a a there 'll be collections at Columbia , collections at at UW  Dan Dan is very interested in doing some other things , grad a: Right . grad a: Well , I think it 's important both for the notation and the machine representation to be the same . postdoc f: N there was also this , email from Dan regarding the speech - non nonspeech segmentation thing . postdoc f: I don't know if , we wanna ,  and Dan Gel - and Dave Gelbart is interested in pursuing the aspect of using amplitude as a a a as a basis for the separation . He was talking he was talking  , we he had postdoc f: Cross professor g: Yeah , cross - correlation . phd c: Cross professor g: I had mentioned this a couple times before , the c the commercial devices that do , voice ,  you know , active miking , postdoc f:  - huh . professor g: So , by doing that , you know , rather than setting any , absolute threshold , you actually can do pretty good , selection of who who 's talking . professor g:  And those those systems work very well , by the way , so people use them in panel discussions and so forth with sound reinforcement differing in in sort of , phd d:  - huh . professor g:  and , those if Boy , the guy I knew who built them , built them like twenty twenty years ago , grad a:  . Cuz there is one thing that we don't have right now and that is the automatic , channel identifier . postdoc f: That that , you know , that would g help in terms of encoding of overlaps . postdoc f: The the transcribers would have less , disentangling to do if that were available . So I think , you know , basically you can look at some p you have to play around a little bit , to figure out what the right statistic is , postdoc f: But . professor g: but you compare each microphone to some statistic based on the on the overall phd c: Yeah . So that , you cou yo grad a: Yeah , although the the using the close - talking I think would be much better . professor g: I just it 'd be If I was actually working on it , I 'd sit there and and play around with it , and and get a feeling for it .  , the the the ,  But , you certainly wanna use the close - talking , as a at least . professor g: I don't know if the other would would add some other helpful dimension or not . What what are the different , classes to to code , the the overlap , you will use ? postdoc f: to code d phd d: What you you postdoc f: so types of overlap ? phd d: Yeah . postdoc f: so at a meeting that wasn't transcribed , we worked up a a typology . postdoc f: And ,  phd d: Look like , you t you explaining in the blackboard ? The ? Yeah ? Yeah . So it i the it 's basically a two - tiered structure where the first one is whether the person who 's interrupted continues or not . And then below that there 're subcategories , that have more to do with , you know , is it , simply backchannel phd d:   postdoc f: or is it , someone completing someone else 's thought , or is it someone in introducing a new thought . And I hope that if we do a forced alignment with the close - talking mike , that will be enough to recover at least some of the time the time information of when the overlap occurred .  , I I I I 've phd b: So who 's gonna do that ? Who 's gonna do forced alignment ? grad a: Well , u  , IBM was going to . grad a: and I imagine they still plan to but but , you know , I haven't spoken with them about that recently . postdoc f: It occurs to me one of my transcribers t told me today that she 'll be finished with one meeting , by professor g:   postdoc f: well , she said tomorrow but then she said you know , but the , you know let 's let 's just , say professor g:   I know these are er , I could send him that if it would be possible , or a good idea or not , to try to do a s forced alignment on what we 're on the way we 're encoding overlaps now . professor g: you know , basically he 's he just studies , he 's a colleague , a friend , and , postdoc f: Yeah ! professor g: they and and , you know , the the organization always did wanna help us . professor g: It was just a question of getting , you know , the right people connected in , who had the time . professor g: So , eh grad a: Is he on the mailing list ? The Meeting Recorder mailing li ? postdoc f: Oh ! grad a: We should add him . phd e: Did something happen , Morgan , that he got put on this , or was he already on it , grad a: Add him . phd e: or ? professor g: No , I , eh , eh , p It it oc I h it 's Yeah , something happened . So , where are we ? Maybe , brief Well , let 's why don't we talk about microphone issues ? postdoc f: Yeah . professor g: That was that was a grad a: so one thing is that I did look on Sony 's for a replacement for the mikes for the head m head - worn ones cuz they 're so uncomfortable . But I think I need someone who knows more about mikes than I do , because I couldn't find a single other model that seemed like it would fit the connector , which seems really unlikely to me . Does anyone , like , know stores or know about mikes who who would know the right questions to ask ? professor g: Oh , I probably would . phd e: You couldn't you couldn't find the right connector to go into these things ? grad a: Yep . When I looked , i they listed one microphone and that 's it phd e: Huh ! grad a: as having that type of connector . Well , let 's look at it together grad a: it seems it seems really unlikely to me that there 's only one . professor g: and postdoc f: And there 's no adaptor for it ? phd c: Yeah . professor g: Who who are we buying these from ? grad a: professor g: That 'd be grad a: I have it downstairs . grad a: And then ,  just in terms of how you wear them  , I had thought about this before .  , when when when you use a product like DragonDictate , they have a very extensive description about how to wear the microphone and so on . grad a: But I felt that in a real situation we were very seldom gonna get people to really do it and maybe it wasn't worth concentrating on . But professor g: Well , I think that that 's that 's a good back - off position . That 's what I was saying earlier , th that , you know , we are gonna get some recordings that are imperfect and , hey , that 's life . But I I think that it it doesn't hurt , the naturalness of the situation to try to have people wear the microphones properly , if possible , grad a:   professor g: because , the natural situation is really what we have with the microphones on the table . professor g: I think , you know , in the target applications that we 're talking about , people aren't gonna be wearing head - mounted mikes anyway . professor g: So this is just for u these head - mounted mikes are just for use with research . professor g: And , it 's gonna make You know , if if An - Andreas plays around with language modeling , he 's not gonna be m wanna be messed up by people breathing into the microphone . professor g: So it 's it 's ,  grad a: Well , I 'll dig through the documentation to DragonDictate and ste s see if they still have the little form . professor g: Right ?  , and any phd b: It 's interesting , I talked to some IBM guys , last January , I think , I was there . phd b: And they said , the breathing is really a a terrible problem for them , to to not recognize breathing as speech . grad a: Well , that 's the It seemed to me when I was using Dragon that it was really microphone placement helped an in ,  an enormous amount . grad a: So you want it enough to the side so that when you exhale through your nose , it doesn't the wind doesn't hit the mike . professor g: I remember when I was when I I I I used , a prominent laboratory 's , speech recognizer about ,  This was , boy , this was a while ago , this was about twelve twelve years ago or something . And , they were they were perturbed with me because I was breathing in instead of breathing out . And they had models for they they had Markov models for br breathing out but they didn't have them for breathing in . Well , what I wondered is whether it 's possible to have to maybe use the display at the beginning grad a: Yeah . postdoc f: to be able to to judge how how correctly  , have someone do some routine whatever , and and then see if when they 're breathing it 's showing . postdoc f: I don't know if the if it 's professor g: I grad a: You can definitely see it . grad a: And so , you know , I 've I 've sat here and watched sometimes the breathing , professor g: I grad a: and the bar going up and down , and I 'm thinking , I could say something , but professor g: I think grad a: I don't want to make people self - conscious . And you can do some , you know , first - order thing about it , which is to have people move it , a away from being just directly in front of the middle phd d: Yeah . postdoc f: Yeah , i professor g: And then , you know , I think there 's not much Because you can't al you know , interfere w you can't fine tune the meeting that much , I think . It just seems like i if something l simple like that can be tweaked and the quality goes , you know , dramatically up , then it might be worth doing . My my feedback from the transcribers is he is always close to crystal clear and and just fan fantastic to phd c: Yeah . postdoc f: I could say something about about the Well , I don't know what you wanna do . professor g: About what ? postdoc f: About the transcribers or anything or ? I don't know . professor g: Well , the other phd b: But , just to to ,  professor g: why don't we do that ? phd b: One more remark , concerning the SRI recognizer . It is useful to transcribe and then ultimately train models for things like breath , and also laughter is very , very frequent and important to to model . phd b: So , grad a: So , phd b: if you can in your transcripts mark grad a: mark them ? phd b: mark very audible breaths and laughter especially , phd c: Mmm . postdoc f: They 're putting Eh , so in curly brackets they put " inhale " or " breath " . Now they 're they 're not being awfully precise , m So they 're two types of laughter that are not being distinguished . postdoc f: One is when sometimes s someone will start laughing when they 're in the middle of a sentence . postdoc f: And and then the other one is when they finish the sentence and then they laugh . So , I I did s I did some double checking to look through  , you 'd need to have extra  e extra complications , like time tags indicating the beginning and ending of of the laughing through the utterance . phd b: It 's not so I don't think it 's ,  postdoc f: And that and what they 're doing is in both cases just saying " curly brackets laughing " a after the unit . phd b: As as long as there is an indication that there was laughter somewhere between two words I think that 's sufficient , phd c: Yeah . phd b: So as long as you can stick a you know , a t a tag in there that that indicates that there was laughter , grad a: Oh , I didn't know that . grad a: That would be a really interesting prosodic feature , postdoc f: Then phd d: Yeah . postdoc f: So , if they laugh between two words , you you 'd get it in between the two words . postdoc f: But if they laugh across three or four words you you get it after those four words . phd b: Well , the thing that you is hard to deal with is whe when they speak while laughing . phd b: But , that 's not as frequent as just laughing between speaking , postdoc f: OK . grad a: So are do you treat breath and laughter as phonetically , or as word models , or what ? phd b: so professor g:  is it ? phd d: Huh . There was a there 's actually a word for  , it 's not just breathing but all kinds of mouth grad a:   grad a: So train a phone in the neural net ? phd b: Same thing ? Yeah . professor g: No grad a: Oh , it does ? phd b: So the so the the mouth noise , word has just a single phone , that is for that . professor g: Right ?  , you could you could say well , let  we now think that laughter should have three sub sub sub - units in the the three states ,  different states . professor g: And then you would have three  , you know , eh , eh , it 's u grad a: Do whatever you want . phd b: And the the pronun the pronunciations the pronunciations are l are somewhat non - standard . phd b: They actually are  , it 's just a single , s  , you know , a single phone in the pronunciation , but it has a self - loop on it , so it can grad a: To go on forever ? phd b: r can go on forever . grad a: And how do you handle it in the language model ? phd b: It 's just a it 's just a word . We also tried , absorbing these  , both laughter and and actually also noise , and ,  phd d: Yeah . We also tried absorbing that into the pause model  , the the the model that that matches the stuff between words . You you did get me to thinking about I I 'm not really sure which is more frequent , whether f f laughing I think it may be an individual thing . grad a: I was noticing that with Dan in the one that we ,  we hand tran hand - segmented , professor g: Yeah . And and one thing that c that we 're not doing , of course , is we 're not claiming to , get be getting a representation of mankind in these recordings . Why don why don't we just since we 're on this vein , why don't we just continue with , what you were gonna say about the transcriptions postdoc f: OK . professor g: and ? postdoc f: the I I 'm really very for I 'm extremely fortunate with the people who , applied and who are transcribing for us . They are ,  really perceptive and very ,  and I 'm not just saying that cuz they might be hearing this . grad a: Cuz they 're gonna be transcribing it in a few days . So I , e you know , I I brought them in and , trained them in pairs because I think people can raise questions grad a: That 's a good idea . postdoc f: you know , i i the they think about different things and they think of different and  , I trained them to , f on about a minute or two of the one that was already transcribed . This also gives me a sense of You know , I can I can use that later , with reference to inter - coder reliability kind of issues . But the main thing was to get them used to the conventions and , you know , the idea of the th th the size of the unit versus how long it takes to play it back so these th sort of calibration issues . And then , I just set them loose and they 're they all have e a already background in using computers . postdoc f: Well , they they 're very perce they 'll So one of them said " well , you know , he really said " n " , not really " and " , phd d: Yeah . postdoc f: so what what should I do with that ? " grad a: Yeah . If it 's an a noncanonical p " That one , I think we you know , with Eric 's work , I sort of figure we we can just treat that as a variant . But I told them if if there 's an obvious speech error , like I said in one thing , professor g: OK . postdoc f: and I gave my my example , like I said , " microfon " in instead of " microphone " . But it but I thought it 's not worth fixing cuz often when you 're speaking everybody knows what what you mean . postdoc f: But I have a convention that if it 's obviously a noncanonical pronunciation a speech error with you know , wi within the realm of resolution that you can tell in this native English American English speaker , you know that I didn't mean to say " microfon . " Then you 'd put a little tick at the beginning of the word , professor g: Yeah . postdoc f: and that just signals that , this is not standard , and then in curly brackets " pron error " . Well Well , you know , it might be something we 'd wanna do with some , s small subset of the whole thing . Where were they when we needed them ? postdoc f: I think professor g: We certainly wouldn't wanna do it with everything .  , if ,  so I I told them that , we don't know if this will continue past the end of the month professor g:  - huh . postdoc f: and I also m I think they know that the data p source is limited and I may not be able to keep them employed till the end of the month even , although I hope to . professor g: The other thing we could do , actually , is , use them for a more detailed analysis of the overlaps . professor g: Right ? grad a: We could get a very detailed overlap if they were willing to transcribe each meeting four or five times . professor g: But I 've been saying the other thing is just go through it for the overlaps . professor g: Right ? postdoc f: And with the right in interface professor g: Given that y and and do so instead of doing phonetic , transcription for the whole thing , phd d: Yeah . professor g: which we know from the Steve 's experience with the Switchboard transcription is , you know , very , very time - consuming . And and you know , it took them I don't know how many months to do to get four hours . But , the other thing is since we 've been spending so much time thinking about overlaps is is maybe get a much more detailed analysis of the overlaps . professor g: I 'm open to every consideration of what are some other kinds of detailed analysis that would be most useful . professor g: It 's a we have we have due to @ @ variations in funding we have we seem to be doing , very well on m money for this this year , and next year we may have have much less . grad a: Is you mean two thousand one ? professor g: So I don't wanna hire a grad a: Calendar year or ? professor g: calendar year two thousand one . So it 's  , it 's we don't wanna hire a bunch of people , a long - term staff , grad a: Full - time . professor g: because the the funding that we 've gotten is sort of a big chunk for this year . But having temporary people doing some specific thing that we need is actually a perfect match to that kind of , funding . phd e: Are they working full - time now , or ? postdoc f: But Some of them are . But what  is Oh , I shouldn't say it that way because that does sound like forty - hour weeks . I th I I would say they 're probably they don't have o they don't have other things that are taking away their time . professor g: I postdoc f: I I I haven't checked them all , but just spot - checking . grad a: I think it would be professor g: I remember when we were transcribing BeRP , Ron Kay , volunteered to to do some of that . Well , you know , and I also thought , y Liz has this , eh , you know , and I do also , this this interest in the types of overlaps that are involved . These people would be great choices for doing coding of that type if we wanted , grad a: We 'd have to mark them . grad a: I think it would also be interesting to have , a couple of the meetings have more than one transcriber do , professor g: Yeah . postdoc f: You know , there 's also , the e In my mind , I think A An - Andreas was leading to this topic , the idea that , we haven't yet seen the the type of transcript that we get from IBM , and it may just be , you know , pristine . But on the other hand , given the lesser interface Cuz this is , you know we 've got a good interface , we 've got great headphones , m  professor g: It could be that they will  theirs will end up being a kind of fir first pass or something . professor g: Maybe an elaborate one , cuz again they probably are gonna do these alignments , which will also clear things up . Al - although you have to s Don't you have to start with a close enough approximation of the of the verbal part to be able to ? professor g: Well , tha that 's that 's debatable . professor g: Right ?  , so the so the argument is that if your statistical system is good it will in fact , clean things up . professor g: And , so in principle you could start up with something that was kind of rough  , to give an example of , something we used to do , at one point , back back when Chuck was here in early times , is we would take , da take a word and , have a canonical pronunciation and , if there was five phones in a word , you 'd break up the word , into five equal - length pieces which is completely gross . professor g: Right ?  , th the timing is off all over the place in just about any word . You start off with that and the statistical system then aligns things , and eventually you get something that doesn't really look too bad . professor g: So so I think using a a good aligner , actually can can help a lot . If you have a good alignment , it helps the , th the human in in taking less time to correct things . I guess there 's another aspect , too , and I don't know  , this this is very possibly a different , topic . So like in a you know , the topics that are covered during a meeting with reference to the other , uses of the data , professor g:   postdoc f: so being able to find where so - and - so talked about such - and - such , then ,  e  , I I I did sort of a a rough pass on encoding , like , episode - like level things on the , transcribed meeting professor g:   postdoc f: where that i if that 's something that we wanna do with each meeting , sort of like a ,  it 's like a manifest , when you get a box full of stuff , or or if that 's ,  professor g:   postdoc f: i I I don't know what  , level of detail would be most useful . I don't know i if that 's something that I should do when I look over it , or if we want someone else to do , or whatever . professor g: eh , was p Well , you know , the thing I 'm concerned about is we wanted to do these digits postdoc f: Oh , yeah . But I think , do you , maybe , eh ? Did you prepare some whole thing you wanted us just to see ? phd d: Yeah . professor g: how long a ? phd d: I I think it 's it 's fast , because , I have the results , eh , of the study of different energy without the law length . Eh , eh , in the in the measurement , the average , dividing by the by the , variance . phd d: the other ,  the the last w  , meeting eh , I don't know if you remain we have problem to with the with with the parameter with the representations of parameter , because the the valleys and the peaks in the signal , eh , look like , eh , it doesn't follow to the to the energy in the signal . professor g: No , that there 's no point in going through all of that if that 's the bottom line , really . professor g: So , I I think we have to start  , there there 's two suggestions , really , which is ,  what we said before is that , phd d: Mmm , yeah . professor g: it looks like , at least that you haven't found an obvious way to normalize so that the energy is anything like a reliable , indicator of the overlap . professor g: I I 'm I 'm still a little f think that 's a little funny . professor g: but but you don't want to keep ,  keep knocking at it if it 's if you 're not getting any any result with that . But , the other things that we talked about is , pitch - related things and harmonicity - related things , phd d: Yeah .  But , a completely different tack on it wou is the one that was suggested , by your colleagues in Spain , phd d: Yeah . professor g: That is to say , use , you know , as as you 're doing with the speech , nonspeech , use some very general features . professor g: You know , have a have a couple Markov models and and , try to indi try to determine , you know , w when is th when are you in an overlap , when are you not in an overlap . professor g: And let the , statistical system determine what 's the right way to look at the data . professor g: I I , I think it would be interesting to find individual features and put them together . professor g: But given the limitation in time and given the fact that Javier 's system already exists doing this sort of thing , phd d: Yeah . professor g: but , its main limitation is that , again , it 's only looking at silences which would phd d: Yeah . phd d: I I I think that , eh , the possibility , eh , can be that , eh , Thilo , eh , working , eh , with a new class , not only , eh , nonspeech and speech , but , eh , in in in the speech class , professor g:   phd d: dividing , eh , speech , eh , of from a speaker and overlapping , to try to to do , eh , eh , a fast a fast , eh , experiment to to prove that , nnn , this fea eh , general feature , eh , can solve the the the problem , professor g: Yeah . phd d: And I hope the the next week I will have , eh , some results and we we will show we will see , eh , the the parameter the pitch , eh , tracking in with the program . phd d: And , nnn , nnn professor g: Ha - h have you ever looked at the ,  Javier 's , speech segmenter ? phd c: No . professor g: Cuz again the idea is there the limitation there again was that he was he was only using it to look at silence as a as a as a as a p putative split point between speakers . professor g: But if you included , broadened classes then in principle maybe you can cover the overlap cases . phd c: Yeah , but I 'm not too sure if if we can really represent overlap with with the s detector I I I used up to now , phd d: Mmm , yeah . grad a: I think with phd c: the to speech - nonspeech as grad a: That 's right . But I think Javier 's phd c: it 's only speech or it 's it 's it 's nonspeech . professor g: N n grad a: It doesn't have the same Gaus -  , H M M modeling , phd c: Yeah . grad a: But ,  professor g: Well , it 's sort of has a simple one . grad a: Does it ? professor g: Right ? It 's it 's just it 's just a isn't it just a Gaussian phd d: Yeah . Oh , it doesn't have it doesn't have any temporal ,  ? grad a: Maybe I 'm misremembering , but I did not think it had a Markov professor g: I thought it Yeah . phd d: No , Javier di doesn't worked with , a Markov grad a: Yeah , I didn't think so . So he 's just he just computes a Gaussian over potential grad a: Yep . professor g: And and grad a: It 's just , that i it he has the two - pass issue that What he does is , as a first pass he he p he does , a guess at where the divisions might be and he overestimates . And that 's just a data reduction step , so that you 're not trying at every time interval . grad a: And right now he 's doing that with silence and that doesn't work with the Meeting Recorder . professor g: The other thing one could do is Couldn't  , it 's So you have two categories phd c: Yeah . Couldn't you have a third category ? So you have ,  you have , nonspeech , single - person speech , and multiple - person speech ? postdoc f: He has this on his board actually . Don't you have , like those those several different categories on the board ? professor g: Right ? And then you have a Markov model for each ? phd c:  I 'm not sure . But it 's not too easy , I think , the the transition between the different class , to model them in in the system I have now . professor g: the th the reason why , I was suggesting originally that we look at features is because I thought , well , we 're doing something we haven't done before , phd c: Yeah . professor g: It seems like if two people two or more people talk at once , it should get louder , phd c: Yeah . professor g: and , there should be some discontinuity in pitch contours , phd c: I had the impression . professor g: and , there should overall be a , smaller proportion of the total energy that is explained by any particular harmonic sequence in the spectrum . professor g: So far , Jose has has been By the way , I was told I should be calling you Pepe , but phd d: Yeah . professor g: the has has , been exploring , e largely the energy issue and , as with a lot of things , it is not  , like this , it 's not as simple as it sounds . professor g: And then there 's , you know Is it energy ? Is it log energy ? Is it LPC residual energy ? Is it is it is it , delta of those things ?  , what is it no Obviously , just a simple number absolute number isn't gonna work . So it should be with compared to what ? Should there be a long window for the normalizing factor and a short window for what you 're looking at ? phd c: Yeah . professor g: Or , you know , how b short should they be ? So , phd d:  . professor g: th he 's been playing around with a lot of these different things and and so far at least has not come up with any combination that really gave you an indicator . professor g: So I I still have a hunch that there 's it 's in there some place , but it may be given that you have a limited time here , it it just may not be the best thing to to to focus on for the remaining of it . professor g: So pitch - related and harmonic - related , I 'm I 'm somewhat more hopeful for it . professor g: But it seems like if we just wanna get something to work , phd c: Yeah . professor g: that , their suggestion of of Th - they were suggesting going to Markov models , but in addition there 's an expansion of what Javier did . professor g: even if the features that you give it are maybe not ideal for it , it 's just sort of this general filter bank phd c: Yeah . professor g: or or cepstrum or something ,  Eee it 's in there somewhere probably . phd d: But , eh , what did you think about the possibility of using the Javier software ? Eh , the ,  the ,  the BIC criterion , the the t to train the the Gaussian , eh , using the the mark , eh , by hand , eh , eh , to distinguish be mmm , to train overlapping zone and speech zone .  , eh , I I I think that an interesting , eh , experiment , eh , could be , th eh , to prove that , mmm , if s we suppose that , eh , the the first step  , the the classifier what were the classifier from Javier or classifier from Thilo ? W What happen with the second step ? I  , what what happen with the , eh the , clu the ,  the clu the clustering process ? grad a:   grad a: What do you mean ? phd d: I  , that is is enough is enough , eh , to work well , eh , to , eh , separate or to distinguish , eh , between overlapping zone and , eh , speaker zone ? Because th if if we if we , eh , nnn , develop an classifier and the second step doesn't work well , eh , we have another problem . I had tried doing it by hand at one point with a very short sample , phd d: N grad a: and it worked pretty well , but I haven't worked with it a lot . So what I d I d I took a hand - segmented sample phd d: Nnn , yeah . grad a: and I added ten times the amount of numbers at random , phd d: Yeah . But is is if grad a: But this was just very anecdotal sort of thing . phd d: But it 's possible with my segmentation by hand that we have information about the the overlapping , grad a: Right . So if we if we fed the hand - segmentation to Javier 's and it doesn't work , then we know something 's wrong . phd d: this kind o emph emphasises parameter and Gaussian grad a: Yeah . Y do you know where his software is ? Have you used it at all ? phd d: I yeah have