Professor C: Right .
PhD G: So the fact that they 're reading , first of all , whether they 're reading in a room of , people , or rea you know , just the fact that they 're reading will make a difference .
Professor C: Yeah .
PhD G: And , depends what you 're interested in .
Professor C: See , I don't know . So , may maybe the thing will be do to take some very small subset , I mean not have a big , program , but take a small set , uh , subset of the conversational speech and a small subset of the digits , and look and and just get a feeling for it . Um , just take a look . Really .
Postdoc B: H That could could be an interesting design , too , cuz then you 'd have the com the comparison of the , uh , predictable speech versus the less predictable speech
Professor C: Cuz I don't think anybody is , I at least , I don't know , of anybody , uh , well , I don't know , the answers .
PhD D: Hey .
Professor C: Yeah .
Postdoc B: and maybe you 'd find that it worked in , in the , case of the pr of the , uh , non - predictable .
Professor C: Yeah .
PhD D: Hafta think about , the particular acoustic features to mark , too , because , I mean , some things , they wouldn't be able to mark , like , uh , you know , uh , tense lax .
Professor C: Mm - hmm .
PhD D: Some things are really difficult . You know ,
Postdoc B: Well .
PhD D: just listening .
Grad F: M I think we can get Ohala in to , give us some advice on that .
PhD D: Yeah .
Postdoc B: Also I thought you were thinking of a much more restricted set of features , that Professor C: Yeah , but I I I I was , like he said , I was gonna bring John in and ask John what he thought .
Postdoc B: Yeah , sure . Sure . Yeah .
Professor C: Right . But I mean you want you want it be restrictive but you also want it to to to have coverage .
Grad F: Right .
Postdoc B: Yeah
Professor C: You know i you should . It should be such that if you , if you , uh , if you had o um , all of the features , determined that you that you were uh ch have chosen , that that would tell you , uh , in the steady - state case , uh , the phone . So , um .
Postdoc B: OK .
Grad F: Even , I guess with vowels that would be pretty hard , wouldn't it ? To identify actually , you know , which one it is ?
Postdoc B: It would seem to me that the points of articulation would be m more , g uh , I mean that 's I think about articulatory features , I think about , points of articulation , which means , uh , rather than vowels .
Grad F: Yeah .
PhD D: Points of articulation ? What do you mean ?
Postdoc B: So , is it , uh , bilabial or dental or is it , you know , palatal .
Professor C: Mm - hmm .
Postdoc B: Which which are all like where where your tongue comes to rest .
Professor C: Place , place .
PhD D: Place of ar place of articulation .
Grad F: Uvular .
PhD A: Place .
Postdoc B: Place . Thank you , what whatev whatever I s said , that 's Professor C: Yeah .
PhD D: OK .
Postdoc B: I really meant place .
Professor C: Yeah .
PhD D: OK , I see .
Professor C: Yeah . OK we got our jargon then , OK .
Postdoc B: Yeah .
Professor C: Uh .
PhD G: Well it 's also , there 's , really a difference between , the pronunciation models in the dictionary , and , the pronunciations that people produce . And , so , You get , some of that information from Steve 's work on the on the labeling
Professor C: Right .
Grad F: Right .
PhD G: and it really , I actually think that data should be used more . That maybe , although I think the meeting context is great , that he has transcriptions that give you the actual phone sequence . And you can go from not from that to the articulatory features , but that would be a better starting point for marking , the gestural features , then , data where you don't have that , because , we you wanna know , both about the way that they 're producing a certain sound , and what kinds of , you know what kinds of , phonemic , differences you get between these , transcribed , sequences and the dictionary ones .
Professor C: Well you might be right that mi might be the way at getting at , what I was talking about , but the particular reason why I was interested in doing that was because I remember , when that happened , and , John Ohala was over here and he was looking at the spectrograms of the more difficult ones . Uh , he didn't know what to say , about , what is the sequence of phones there . They came up with some compromise . Because that really wasn't what it look like . It didn't look like a sequence of phones
Grad F: Right .
PhD G: Right .
Professor C: it look like this blending thing happening here and here and here .
Grad F: Yeah , so you have this feature here , and , overlap , yeah .
PhD G: Right .
Professor C: Yeah .
PhD D: There was no name for that .
PhD G: But Right .
Professor C: Yeah .
PhD G: But it still is there 's a there are two steps . One you know , one is going from a dictionary pronunciation of something , like , " gonna see you tomorrow " ,
Grad F: And Or " gonta " .
Professor C: Right . Yeah .
PhD G: it could be " going to " or " gonna " or " gonta s " you know .
Professor C: Right .
PhD G: And , yeah . " Gonna see you tomorrow " , uh , " guh see you tomorrow " . And , that it would be nice to have these , intermediate , or these some these reduced pronunciations that those transcribers had marked or to have people mark those as well .
Professor C: Mm - hmm .
PhD G: Because , it 's not , um , that easy to go from the , dictionary , word pronuncia the dictionary phone pronunciation , to the gestural one without this intermediate or a syllable level kind of , representation .
Grad F: Well I don't think Morgan 's suggesting that we do that , though .
Professor C: Do you mean ,
PhD A: Yeah .
Professor C: Yeah , I mean , I I I 'm jus at the moment of course we 're just talking about what , to provide as a tool for people to do research who have different ideas about how to do it . So for instance , you might have someone who just has a wor has words with states , and has uh uh , comes from articulatory gestures to that . And someone else , might actually want some phonetic uh intermediate thing . So I think it would be be best to have all of it if we could . But um ,
Grad F: But What I 'm imagining is a score - like notation , where each line is a particular feature .
Professor C: Yeah .
Grad F: Right ,
Professor C: Yeah .
Grad F: so you would say , you know , it 's voiced through here , and so you have label here , and you have nas nasal here , and , they they could be overlapping in all sorts of bizarre ways that don't correspond to the timing on phones .
Professor C: I mean this is the kind of reason why I remember when at one of the Switchboard , workshops , that uh when we talked about doing the transcription project , Dave Talkin said , " can't be done " .
Grad F: Right .
Professor C: He was he was , what what he meant was that this isn't , you know , a sequence of phones , and when you actually look at Switchboard that 's , not what you see , and , you know . And . It ,
Grad F: And in in fact the inter - annotator agreement was not that good , right ? On the harder ones ?
Professor C: yeah I mean it was
PhD G: It depends how you look at it , and I I understand what you 're saying about this , kind of transcription exactly ,
Professor C: Yeah .
PhD G: because I 've seen you know , where does the voicing bar start and so forth .
Professor C: Yeah .
PhD G: All I 'm saying is that , it is useful to have that the transcription of what was really said , and which syllables were reduced . Uh , if you 're gonna add the features it 's also useful to have some level of representation which is , is a reduced it 's a pronunciation variant , that currently the dictionaries don't give you
Professor C: Mm - hmm .
PhD G: because if you add them to the dictionary and you run recognition , you , you add confusion .
Professor C: Right . Right .
PhD G: So people purposely don't add them . So it 's useful to know which variant was was produced , at least at the phone level .
PhD D: So it would be it would be great if we had , either these kind of , labelings on , the same portion of Switchboard that Steve marked , or , Steve 's type markings on this data , with these .
PhD G: Right . That 's all , I mean . Exactly .
Professor C: Yeah .
PhD G: Exactly .
Professor C: Yeah , no I I don't disagree with that .
PhD G: And Steve 's type is fairly it 's not that slow , uh , uh , I dunno exactly what the , timing was , but .
Professor C: Yeah u I don't disagree with it the on the only thing is that , What you actually will end en end up with is something , i it 's all compromised , right , so , the string that you end up with isn't , actually , what happened . But it 's it 's the best compromise that a group of people scratching their heads could come up with to describe what happened .
PhD D: And it 's more accurate than , phone labels .
PhD G: Mm - hmm .
Professor C: But . And it 's more accurate than the than the dictionary or , if you 've got a pronunciation uh lexicon that has three or four ,
Grad F: The word .
PhD D: Yeah .
Professor C: this might be have been the fifth one that you tr that you pruned or whatever ,
PhD D: So it 's like a continuum .
PhD G: Right .
Professor C: so sure .
PhD D: It 's you 're going all the way down ,
PhD G: Right . Right .
Professor C: Yeah .
PhD D: yeah .
PhD G: That 's what I meant is Professor C: Yeah .
PhD G: an and in some places it would fill in , So the kinds of gestural features are not everywhere .
Grad F: Well PhD D: Right .
PhD G: So there are some things that you don't have access to either from your ear or the spectrogram ,
PhD D: Mm - hmm .
PhD G: but you know what phone it was and that 's about all you can all you can say .
PhD D: Right .
PhD G: And then there are other cases where , nasality , voicing PhD D: It 's basically just having , multiple levels of of , information and marking , on the signal .
PhD G: Right . Right .
Professor C: Yeah .
Grad F: Well the other difference is that the the features , are not synchronous ,
PhD G: Right .
Grad F: right . They overlap each other in weird ways .
PhD D: Mm - hmm . Mm - hmm .
Grad F: So it 's not a strictly one - dimensional signal .
Professor C: Right .
Grad F: So I think that 's sorta qualitatively different .
PhD G: Right . You can add the features in , uh , but it 'll be underspecified .
Postdoc B: Hmm .
PhD G: Th - there 'll be no way for you to actually mark what was said completely by features .
Grad F: Well not with our current system but you could imagine designing a system , that the states were features , rather than phones .
PhD G: And i if you 're Well , we we 've probably have a separate , um , discussion of , uh of whether you can do that .
Postdoc B: That 's Well , isn't that I thought that was , well but that wasn't that kinda the direction ?
Grad F: Yeah .
Postdoc B: I thought
Professor C: Yeah , so I mean , what , what where this is , I mean , I I want would like to have something that 's useful to people other than those who are doing the specific kind of research I have in mind , so it should be something broader . But , The but uh where I 'm coming from is , uh , we 're coming off of stuff that Larry Saul did with with , um , uh , John Dalan and Muzim Rahim in which , uh , they , uh , have , um , a m a multi - band system that is , uh , trained through a combination of gradient learning an and EM , to um , estimate , uh , the , uh , value for m for for a particular feature . OK . And this is part of a larger , image that John Dalan has about how the human brain does it in which he 's sort of imagining that , individual frequency channels are coming up with their own estimate , of of these , these kinds of something like this . Might not be , you know , exact features that , Jakobson thought of or something . But I mean you know some , something like that . Some kind of low - level features , which are not , fully , you know , phone classification . And the the th this particular image , of how thi how it 's done , is that , then given all of these estimates at that level , there 's a level above it , then which is is making , some kind of sound unit classification such as , you know , phone and and , you know . You could argue what , what a sound unit should be , and and so forth . But that that 's sort of what I was imagining doing , um , and but it 's still open within that whether you would have an intermediate level in which it was actually phones , or not . You wouldn't necessarily have to . Um , but , Again , I wouldn't wanna , wouldn't want what we we produced to be so , know , local in perspective that it it was matched , what we were thinking of doing one week , And and , and , you know , what you 're saying is absolutely right . That , that if we , can we should put in , uh , another level of , of description there if we 're gonna get into some of this low - level stuff .
PhD D: Well , you know , um I mean if we 're talking about , having the , annotators annotate these kinds of features , it seems like , You know , you The the question is , do they do that on , meeting data ? Or do they do that on , Switchboard ?
Grad F: That 's what I was saying ,
Postdoc B: W Well it seems like you could do both .
Grad F: maybe meeting data isn't the right corpus .
Postdoc B: I mean , I was thinking that it would be interesting , to do it with respect to , parts of Switchboard anyway , in terms of ,
Professor C: Mm - hmm .
Postdoc B: uh partly to see , if you could , generate first guesses at what the articulatory feature would be , based on the phone representation at that lower level .
Professor C: Mm - hmm .
Postdoc B: It might be a time gain . But also in terms of comparability of , um ,
Professor C: Mm - hmm .
PhD D: Well cuz the yeah , and then also , if you did it on Switchboard , you would have , the full continuum of transcriptions .
Postdoc B: what you gain Yep .
PhD D: You 'd have it , from the lowest level , the ac acoustic features , then you 'd have the , you know , the phonetic level that Steve did ,
Postdoc B: Mm - hmm .
PhD G: Yeah that that 's all I was thinking about .
Postdoc B: And you could tell that PhD D: and , yeah .
PhD G: it is telephone band , so , the bandwidth might be PhD D: It 'd be a complete , set then .
Postdoc B: And you get the relative gain up ahead .
Professor C: It 's so it 's a little different . So I mean i we 'll see wha how much we can , uh , get the people to do , and how much money we 'll have and all this sort of thing ,
PhD G: Yeah .
Postdoc B: Mm - hmm . Mm - hmm .
PhD D: But it it might be good to do what Jane was saying uh , you know , seed it , with , guesses about what we think the features are , based on , you know , the phone or Steve 's transcriptions or something . to make it quicker .
Professor C: but , Might be do both .
Grad F: Alright , so based on the phone transcripts they would all be synchronous , but then you could imagine , nudging them here and there .
PhD D: Adjusting ? Yeah , exactly .
Postdoc B: Yeah .
PhD D: Scoot the voicing over a little , because Grad F: Right .
Professor C: Well I think what I mean I 'm I 'm a l little behind in what they 're doing , now , and , uh , the stuff they 're doing on Switchboard now . But I think that , Steve and the gang are doing , something with an automatic system first and then doing some adjustment . As I re as I recall . So I mean that 's probably the right way to go anyway , is to is to start off with an automatic system with a pretty rich pronunciation dictionary that , that , um , you know , tries , to label it all . And then , people go through and fix it .
Postdoc B: So in in our case you 'd think about us s starting with maybe the regular dictionary entry , and then ? Or would we Professor C: Well , regular dictionary , I mean , this is a pretty rich dictionary . It 's got , got a fair number of pronunciations in it
Postdoc B: But PhD D: Or you could start from the if we were gonna , do the same set , of sentences that Steve had , done , we could start with those transcriptions .
Postdoc B: Mm - hmm . So I was thinking Professor C: Yeah .
PhD G: That 's actually what I was thinking , is tha PhD D: Yeah .
Grad F: Right .
PhD G: the problem is when you run , uh , if you run a regular dictionary , um , even if you have variants , in there , which most people don't , you don't always get , out , the actual pronunciations ,
PhD D: Yeah .
PhD G: so that 's why the human transcriber 's giving you the that pronunciation ,
Postdoc B: Yeah . Oh .
Professor C: Actually maybe they 're using phone recognizers .
PhD G: and so y they they I thought that they were Professor C: Is that what they 're doing ?
Grad F: They are .
Professor C: Oh , OK .
PhD G: we should catch up on what Steve is ,
Professor C: Yeah .
PhD G: uh I think that would be a good i good idea .
Professor C: Yeah , so I think that i i we also don't have , I mean , we 've got a good start on it , but we don't have a really good , meeting , recorder or recognizer or transcriber or anything yet , so . So , I mean another way to look at this is to , is to , uh , do some stuff on Switchboard which has all this other , stuff to it .
PhD G: Yeah .
Professor C: And then , um , As we get , further down the road and we can do more things ahead of time , we can , do some of the same things to the meeting data .
PhD D: Mm - hmm .
Postdoc B: OK .
PhD D: Yeah .
Postdoc B: And I 'm and these people might they they are , s most of them are trained with IPA .
Professor C: Yeah
Postdoc B: They 'd be able to do phonetic - level coding , or articulatory .
PhD D: Are they busy for the next couple years , or ?
Postdoc B: Well , you know , I mean they , they they 're interested in continuing working with us , so I mean I , and this would be up their alley , so , we could when the when you d meet with , with John Ohala and find , you know what taxonomy you want to apply , then , they 'd be , good to train onto it .
Professor C: Yeah . Yeah . Anyway , this is , not an urgent thing at all ,
Postdoc B: Yeah .
Professor C: just it came up .
PhD D: It 'd be very interesting though , to have that data .
Postdoc B: I think so , too .
Grad F: I wonder , how would you do a forced alignment ?
PhD G: Yeah . Might Postdoc B: Interesting idea .
Grad F: To to I mean , you 'd wanna iterate , somehow . Yeah . It 's interesting thing to think about .
PhD D: Hmm .
PhD G: It might be Grad F: I mean you 'd you 'd want models for spreading .
PhD G: I was thinking it might be n
PhD D: Of the f acoustic features ?
Grad F: Yeah .
Professor C: Mm - hmm .
PhD D: Mm - hmm .
Professor C: Yeah .
PhD G: Well it might be neat to do some , phonetic , features on these , nonword words . Are are these kinds of words that people never the " huh "s and the " hmm "s and the " huh " and the uh These k No , I 'm serious . There are all these kinds of functional , uh , elements . I don't know what you call them . But not just fill pauses but all kinds of ways of interrupting {comment} and so forth .
Grad F: Uh - huh .
PhD G: And some of them are , yeah , " uh - huh "s , and " hmm "s , and , " hmm ! " " hmm " {comment} " OK " , " uh " {comment} Grunts , uh , that might be interesting .
Postdoc B: He 's got lip  lipsmacks .
PhD G: In the meetings .
Professor C: We should move on .
Postdoc B: Yeah .
Professor C: Uh , new version of , uh , presegmentation ?
PhD A: Uh , oh yeah , um , I worked a little bit on the on the presegmentation to to get another version which does channel - specific , uh , speech - nonspeech detection . And , what I did is I used some normalized features which , uh , look in into the which is normalized energy , uh , energy normalized by the mean over the channels and by the , minimum over the , other . within each channel . And to to , mm , to , yeah , to normalize also loudness and and modified loudness and things and that those special features actually are in my feature vector .
Grad F: Oh .
PhD A: And , and , therefore to be able to , uh , somewhat distinguish between foreground and background speech in in the different in each channel . And , eh , I tested it on on three or four meetings and it seems to work , well yeah , fairly well , I I would say . There are some problems with the lapel mike .
Grad F: Of course .
PhD A: Yeah . Uh , yeah .
Grad F: Wow that 's great .
PhD A: And .
Grad F: So I I understand that 's what you were saying about your problem with , minimum .
PhD A: Yeah . And . Yeah , and and I had I had , uh , specific problems with .
Grad F: I get it . So new use ninetieth quartile , rather than , minimum .
PhD A: Yeah . Yeah .
Postdoc B: Wow .
PhD A: Yeah yeah , then I I did some some some things like that ,
Postdoc B: Interesting .
PhD A: as there there are some some problems in , when , in the channel , there they the the speaker doesn't doesn't talk much or doesn't talk at all . Then , the , yeah , there are there are some problems with with with n with normalization , and , then , uh , there the system doesn't work at all . So , I 'm I 'm glad that there is the the digit part , where everybody is forced to say something ,
Professor C: Right .
PhD A: so , that 's that 's great for for my purpose . And , the thing is I I , then the evaluation of of the system is a little bit hard , as I don't have any references .
Grad F: Well we did the hand the one by hand .
PhD A: Yeah , that 's the one one wh where I do the training on so I can't do the evaluation on So the thing is , can the transcribers perhaps do some , some some meetings in in terms of speech - nonspeech in in the specific channels ?
Grad F: Uh .
Postdoc B: Well , I have PhD D: Well won't you have that from their transcriptions ?
Postdoc B: Well , OK , so , now we need Grad F: No , cuz we need is really tight .
PhD A: Yeah .
Postdoc B: so , um , I think I might have done what you 're requesting , though I did it in the service of a different thing .
PhD A: Oh , great .
Postdoc B: I have thirty minutes that I 've more tightly transcribed with reference to individual channels .
PhD A: OK . OK , that 's great . That 's great for me . Yeah , so .
Postdoc B: And I could And And Grad F: Hopefully that 's not the same meeting that we did .
Postdoc B: No , actually it 's a different meeting .
Grad F: Good .
PhD A: OK .
Postdoc B: So , um , e so the , you know , we have the , th they transcribe as if it 's one channel with these with the slashes to separate the overlapping parts .