C: Yeah , we had a long discussion about how much w how easy we want to make it for people to bleep things out .
C: So C: Morgan wants to make it hard .
D: It it doesn't C: Did did did it ?
C: I didn't even check yesterday whether it was moving .
D: It didn't move yesterday either when I started it .
C: So .
D: So I don't know if it doesn't like both of us C: Channel three ?
C: Channel three ?
D: You know , I discovered something yesterday on these , um , wireless ones .
B: Channel two .
C: Mm - hmm ?
D: You can tell if it 's picking up breath noise and stuff .
C: Yeah ,
C: it has a little indicator on it on the AF .
D: Mm - hmm .
D: So if you D: yeah , if you breathe under breathe and then you see AF go off , then you know it 's p picking up your mouth noise .
F: Oh , that 's good .
F: Cuz we have a lot of breath noises .
C: Yep .
C: Test .
F: In fact , if you listen to just the channels of people not talking , it 's like " @ @ " .
F: It 's very disgust
C: What ?
C: Did you see Hannibal recently or something ?
F: Sorry .
F: Exactly .
F: It 's very disconcerting .
F: OK .
F: So , um ,
F: I was gonna try to get out of here , like , in half an hour ,
F: um ,
F: cuz I really appreciate people coming ,
F: and the main thing that I was gonna ask people to help with today is to give input on what kinds of database format we should use in starting to link up things like word transcripts and annotations of word transcripts ,
F: so anything that transcribers or discourse coders or whatever put in the signal , with time - marks for , like , words and phone boundaries and all the stuff we get out of the forced alignments and the recognizer .
F: So , we have this , um F: I think a starting point is clearly the the channelized output of Dave Gelbart 's program ,
F: which Don brought a copy of ,
C: Yeah .
C: Yeah , I 'm I 'm familiar with that .
C: I mean , we I sort of already have developed an XML format for this sort of stuff .
F: um ,
F: which D: Can I see it ?
C: And so the only question is it the sort of thing that you want to use or not ?
C: Have you looked at that ?
C: I mean , I had a web page up .
F: Right .
F: So ,
F: I actually mostly need to be able to link up , or C: So F: I it 's it 's a question both of what the representation is and C: You mean , this C: I guess I am gonna be standing up and drawing on the board .
F: OK ,
F: yeah .
F: So you should , definitely .
C: Um , so so it definitely had that as a concept .
C: So tha it has a single time - line ,
F: Mm - hmm .
C: and then you can have lots of different sections ,
C: each of which have I Ds attached to it ,
C: and then you can refer from other sections to those I Ds ,
C: if you want to .
C: So that ,
C: um C: so that you start with with a time - line tag .
C: " Time - line " .
C: And then you have a bunch of times .
C: I don't e I don't remember exactly what my notation was ,
A: Oh , I remember seeing an example of this .
C: but it F: Right ,
F: right .
A: Yeah .
C: Yeah ,
C: " T equals one point three two " ,
C: uh C: And then I I also had optional things like accuracy ,
C: and then " ID equals T one , uh , one seven " .
C: And then , <nonvocalsound> I also wanted to to be i to be able to not specify specifically what the time was and just have a stamp .
F: Right .
C: Yeah , so these are arbitrary , assigned by a program ,
C: not not by a user .
C: So you have a whole bunch of those .
C: And then somewhere la further down you might have something like an utterance tag
C: which has " start equals T - seventeen ,
C: end equals T - eighteen " .
C: So what that 's saying is , we know it starts at this particular time .
C: We don't know when it ends .
F: OK .
C: Right ?
C: But it ends at this T - eighteen ,
C: which may be somewhere else .
C: We say there 's another utterance .
C: We don't know what the t time actually is
C: but we know that it 's the same time as this end time .
A: Mmm .
C: You know , thirty - eight ,
C: whatever you want .
A: So you 're essentially defining a lattice .
C: OK .
C: Yes ,
C: exactly .
A: Yeah .
C: And then , uh and then these also have I Ds .
C: Right ?
C: So you could you could have some sort of other other tag later in the file that would be something like , um , oh , I don't know , <comment> uh , <nonvocalsound> " noise - type equals <nonvocalsound> door - slam " .
C: You know ?
C: And then , uh , <nonvocalsound> you could either say " time equals a particular time - mark " or you could do other sorts of references .
C: So or or you might have a prosody C: " Prosody "
C: right ? D ? T ?
C: D ? T ?
C: T ?
F: It 's an O instead of an I ,
F: but the D is good .
C: You like the D ?
F: Yeah .
C: That 's a good D .
C: Um ,
C: you know , so you could have some sort of type here ,
C: and then you could have ,
C: um C: the utterance that it 's referring to could be U - seventeen or something like that .
F: OK .
F: So ,
F: I mean , that seems that seems g great for all of the encoding of things with time
F: and ,
C: Oh , well .
F: um F: I I guess my question is more , uh , what d what do you do with , say , a forced alignment ?
A: How - how
F: I mean you 've got all these phone labels ,
F: and what do you do if you just conceptually , if you get , um , transcriptions where the words are staying but the time boundaries are changing , cuz you 've got a new recognition output ,
F: or s sort of what 's the , um , sequence of going from the waveforms that stay the same , the transcripts that may or may not change , and then the utterance which where the time boundaries that may or may not change ?
A: Oh , that 's A: That 's actually very nicely handled here
F: Um .
A: because you could you could all you 'd have to change is the , um , time - stamps in the time - line without without , uh , changing the I Ds .
F: And you 'd be able to propagate all of the the information ?
C: Right .
C: That 's , the who that 's why you do that extra level of indirection .
C: So that you can just change the time - line .
A: Except the time - line is gonna be huge .
A: If you say C: Yes .
F: Yeah ,
F: yeah , especially at the phone - level .
A: suppose you have a phone - level alignment .
F: The we we have phone - level backtraces .
A: You 'd have you 'd have C: Yeah , this C: I don't think I would do this for phone - level .
F: Um C: I think for phone - level you want to use some sort of binary representation
C: because it 'll be too dense otherwise .
F: OK .
F: So , if you were doing that and you had this sort of companion , uh , thing that gets called up for phone - level , uh , what would that look like ?
F: How would you ?
C: I would use just an existing an existing way of doing it .
A: Why
A: Mmm .
A: But but why not use it for phone - level ?
F: H h
A: It 's just a matter of it 's just a matter of it being bigger .
A: But if you have A: you know , barring memory limitations , or uh I w I mean this is still the m
C: It 's parsing limitations .
C: I don't want to have this text file that you have to read in the whole thing to do something very simple for .
A: Oh , no .
A: You would use it only for purposes where you actually want the phone - level information , I 'd imagine .
F: So you could have some file that configures how much information you want in your in your XML or something .
C: Right .
C: I mean , you 'd y
F: Um ,
F: cuz th it does get very bush with C: I I am imagining you 'd have multiple versions of this depending on the information that you want .
A: You F: Right .
C: Um ,
C: I 'm just C: what I 'm wondering is whether C: I think for word - level , this would be OK .
F: Yeah .
F: Yeah .
C: For word - level , it 's alright .
F: Definitely .
A: Mm - hmm .
C: For lower than word - level , you 're talking about so much data that I just I don't know .
C: I don't know if that F: I mean , we actually have F: So , one thing that Don is doing , is we 're we 're running For every frame , you get a pitch value ,
D: Lattices are big , too .
F: and not only one pitch value but different kinds of pitch values
F: depending on C: Yeah , I mean , for something like that I would use P - file
C: or or any frame - level stuff I would use P - file .
F: Meaning ?
C: Uh , that 's a well , or something like it .
C: It 's ICS uh , ICSI has a format for frame - level representation of features .
F: OK .
C: Um .
F: That you could call that you would tie into this representation with like an ID .
C: Right .
C: Right .
C: Or or there 's a there 's a particular way in XML to refer to external resources .
F: And