C: a bunch of libraries , P - file utilities .
A: and so basically the uh the filesystem wouldn't C: Well , that 's gonna be a problem no matter what .
C: You have the two - gigabyte limit on the filesystem size .
C: And we definitely hit that with Broadcast News .
A: Maybe you could extend the API to , uh , support , uh , like splitting up , you know , conceptually one file into smaller files on disk
A: so that you can essentially , you know , have arbitrarily long f
C: Yep .
C: Most of the tools can handle that .
C: So that
A: Yeah .
C: we didn't do it at the API - level .
C: We did it at the t tool - level .
C: That that most many of them can s you can specify several P - files
C: and they 'll just be done sequentially .
A: OK .
C: So .
F: So , I guess ,
F: yeah ,
F: if if you and Don can if you can show him the P - file stuff and see .
C: Sure .
F: So this would be like for the F - zero C: I mean , if you do " man P - file " or " apropos P - file " , you 'll see a lot .
B: True .
B: I 've used the P - file , I think .
B: I 've looked at it at least , briefly , I think when we were doing s something .
A: What does the P stand for anyway ?
C: I have no idea .
B: Oh , in there .
C: I didn't de I didn't develop it .
C: You know , it was I think it was Dave Johnson .
C: So it 's all part of the Quicknet library .
C: It has all the utilities for it .
A: No , P - files were around way before Quicknet .
C: Oh , were they ?
A: P - files were were around when w with , um , RAP .
D: Mm - hmm .
A: Right ?
F: It 's like the history of ICSI .
A: You worked with P - files .
F: Like C: Mm - hmm .
D: No .
A: I worked with P - files .
F: Yeah ?
D: I don't remember what the " P " is , though .
A: No .
C: But there are ni they 're C: The Quicknet library has a bunch of things in it to handle P - files ,
A: Yeah .
C: so it works pretty well .
F: And that isn't really , I guess , as important as the the main I don't know what you call it , the the main sort of word - level C: Neither do I .
D: Probably stands for " Phil " .
D: Phil Kohn .
C: It 's a Phil file ?
D: Yeah .
D: That 's my guess .
F: Huh .
F: OK .
F: Well , that 's really useful .
F: I mean , this is exactly the kind of thing that I wanted to settle .
F: Um ,
F: so C: Yeah , I 've been meaning to look at the ATLAS stuff again anyway .
F: Great .
C: So , just keep F: Yeah .
F: I guess it 's also sort of a political deci
F: I mean , if if you feel like that 's a community that would be good to tie into anyway , then it 's sounds like it 's worth doing .
C: Yeah , I think it it w
A: j I think there 's C: And , w uh , as I said , I what I did with this stuff I based it on theirs .
C: It 's just they hadn't actually come up with an external format yet .
C: So now that they have come up with a format , it doesn't it seems pretty reasonable to use it .
A: Mmm .
C: But let me look at it again .
F: OK ,
F: great .
C: As I said , that F: Cuz we actually can start C: There 's one level there 's one more level of indirection
C: and I 'm just blanking on exactly how it works .
C: I gotta look at it again .
F: I mean , we can start with , um , I guess , this input from Dave 's ,
F: which you had printed out , the channelized input .
F: Cuz he has all of the channels ,
F: you know , with the channels in the tag and stuff like that .
C: Yeah , I 've seen it .
F: So that would be i directly ,
C: Yep .
F: um C: Easy easy to map .
F: Yeah .
F: And so then it would just be a matter of getting making sure to handle the annotations that are , you know , not at the word - level and , um , t to import the
B: Where are those annotations coming from ?
F: Well , right now , I g Jane would  would C: Mm - hmm .
F: Yeah .
E: Are you talking about the overlap a annotations ?
F: Yeah ,
F: any kind of annotation that , like , isn't already there .
F: Uh , you know , anything you can envision .
E: Yeah .
E: So what I was imagining was um , so Dave says we can have unlimited numbers of green ribbons .
E: And so put , uh , a a green ribbon on for an overlap code .
E: And since we w we E: I I think it 's important to remain flexible regarding the time bins for now .
E: And so it 's nice to have E: However , you know , you want to have it , uh , time time uh , located in the discourse .
E: So ,
E: um ,
E: if we if we tie the overlap code to the first word in the overlap , then you 'll have a time - marking .
E: It won't it 'll be independent of the time bins ,
E: however these e evolve , shrink , or whatever ,
E: increase ,
E: or E: Also , you could have different time bins for different purposes .
E: And having it tied to the first word in an overlap segment is unique , uh , you know , anchored , clear .
E: And it would just end up on a separate ribbon .
E: So the overlap coding is gonna be easy with respect to that .
C: Right .
E: You look puzzled .
D: I I just I don't quite understand what these things are .
E: OK .
E: What , the codes themselves ?
D: Uh .
D: Well , th overlap codes .
E: Or the ?
D: I 'm not sure what that @ @ C: Well , I mean , is that E: Well , we don't have to go into the codes .
D: It probably doesn't matter .
E: We don't have to go into the codes .
D: No , I d
C: I mean , it doesn't .
C: I mean , that not for the topic of this meeting .
E: But let me just E: No .
E: W the idea is just to have a separate green ribbon ,
E: you know ,
E: and and and let 's say that this is a time bin .
E: There 's a word here .
E: This is the first word of an overlapping segment of any length ,
E: overlapping with any other , uh , word uh , i segment of any length .
E: And , um ,
E: then you can indicate that this here was perhaps a ch a backchannel ,
E: or you can say that it was , um , a usurping of the turn ,
E: or you can you know , any any number of categories .
E: But the fact is , you have it time - tagged in a way that 's independent of the , uh , sp particular time bin that the word ends up in .
E: If it 's a large unit or a small unit , or
A: Mm - hmm .
E: we sh change the boundaries of the units ,
F: Right .
E: it 's still unique and and , uh , fits with the format ,
E: flexible , all that .
A: Um ,
A: it would be nice A: um ,
A: eh , gr this is sort of r regarding uh , uh it 's related but not directly germane to the topic of discussion ,
A: but , when it comes to annotations , um , you often find yourself in the situation where you have different annotations of the same , say , word sequence .
E: Yeah .
A: OK ?
E: Yeah .
A: And sometimes the word sequences even differ slightly because they were edited s at one place but not the other .
A: So , once this data gets out there , some people might start annotating this for , I don't know , dialogue acts or , um , you know , topics or what the heck .
A: You know ,
A: there 's a zillion things that people might annotate this for .
A: And the only thing that is really sort of common among all the versi the various versions of this data is the word sequence ,
E: Yep .
A: or approximately .
F: Or the time .
A: Or the times .
A: But , see , if you 'd annotate dialogue acts , you don't necessarily want to or topics you don't really want to be dealing with time - marks .
F: I guess .
A: You 'd it 's much more efficient for them to just see the word sequence , right ?
F: Mm - hmm .
A: I mean , most people aren't as sophisticated as as we are here with , you know , uh , time alignments and stuff .
A: So A: So the the the point is C: Should should we mention some names on the people who are n ?
A: Right .
A: So ,
A: um ,
A: the p my point is that you 're gonna end up with , uh , word sequences that are differently annotated .
A: And you want some tool , uh , that is able to sort of merge these different annotations back into a single , uh , version .
A: OK ?
A: Um , and we had this problem very massively , uh , at SRI when we worked , uh , a while back on , uh well , on dialogue acts as well as , uh , you know ,
A: um , what was it ?
A: uh ,
F: Well , all the Switchboard in it .
A: utterance types .
F: Yeah .
A: There 's , uh , automatic , uh , punctuation and stuff like that .
A: Because we had one set of annotations that were based on , uh , one version of the transcripts with a particular segmentation ,
A: and then we had another version that was based on , uh , a different s slightly edited version of the transcripts with a different segmentation .
A: So , we had these two different versions which were you know , you could tell they were from the same source but they weren't identical .
A: So it was extremely hard to reliably merge these two back together to correlate the information from the different annotations .
C: Yep .
C: I I don't see any way that file formats are gonna help us with that .
A: No .
C: It 's it 's all a question of semantic .
A: No .
A: But once you have a file format , I can imagine writing not personally , but someone writing a tool that is essentially an alignment tool , um , that mediates between various versions ,
F: Mm - hmm .
C: Yeah .
A: and uh , sort of like th uh , you know , you have this thing in UNIX where you have , uh , diff .
C: Diff .
F: W - diff or diff .
A: There 's the , uh , diff that actually tries to reconcile different two diffs f <comment> based on the same original .
F: Yeah .
E: Is it S - diff ?
C: Yep .
E: Mmm .
A: Something like that ,
A: um , but operating on these lattices that are really what 's behind this uh , this annotation format .
C: Yep .
A: So F: You could definitely do that with the C: There 's actually a diff library you can use to do things like that that so you have different formats .
A: So somewhere in the API you would like to have like a merge or some some function that merges two two versions .
C: Yeah , I think it 's gonna be very hard .
C: Any sort of structured anything when you try to merge is really , really hard
A: Right .
C: because you ha i
C: The hard part isn't the file format .
C: The hard part is specifying what you mean by " merge " .
A: Is A: Exactly .
F: But the one thing that would work here actually for i that is more reliable than the utterances is the the speaker ons and offs .
C: And that 's very difficult .
F: So if you have a good ,
F: um C: But this is exactly what I mean , is that that the problem i
F: Yeah .
F: You just have to know wha what to tie it to .
C: Yeah ,
C: exactly .
C: The problem is saying " what are the semantics ,
F: And C: what do you mean by " merge " ? "
F: Right , right .
A: Right .
C: So .
A: So so just to let you know what we where we kluged it by , uh , doing uh , by doing Hhh .
A: Both were based on words ,
A: so , bo we have two versions of the same words intersp you know , sprinkled with with different tags for annotations .
C: And then you did diff .
A: And we did diff .
C: Yeah ,
A: Exactly !
C: that 's just what I thought .
A: And that 's how C: That 's just wh how I would have done it .
A: Yeah .
A: But , you know , it had lots of errors
A: and things would end up in the wrong order , and so forth .
A: Uh ,
A: so ,
C: Yep .
A: um , if you had a more A: Uh ,
A: it it was a kluge
A: because it was basically reducing everything to uh , to uh , uh , to textual alignment .
C: A textual A: Um ,
A: so F: But ,
F: d isn't that something where whoever if  if the people who are making changes , say in the transcripts , cuz this all happened when the transcripts were different ye um , if they tie it to something , like if they tied it to the acoustic segment if they F: You know what I mean ?
F: Then Or if they tied it to an acoustic segment and we had the time - marks , that would help .
F: But the problem is exactly as Adam said , that you get , you know , y you don't have that information or it 's lost in the merge somehow ,
C: Yep .
F: so E: Well , can I ask one question ?
E: It it seems to me that , um , we will have o an official version of the corpus , which will be only one one version in terms of the words where the words are concerned .
E: We 'd still have the the merging issue maybe if coding were done independently of the A: And you 're gonna get that
E: But but A: because if the data gets out , people will do all kinds of things to it .
A: And , uh , s you know , several years from now you might want to look into , um , the prosody of referring expressions .
A: And someone at the university of who knows where has annotated the referring expressions .
C: Right .
A: So you want to get that annotation and bring it back in line with your data .
C: But unfortunately they 've also hand - edited it .
A: OK ?
E: OK , then F: But they 've also F: Exactly .
F: And so that 's exactly what we should somehow when you distribute the data , say that you know , that have some way of knowing how to merge it back in and asking people to try to do that .
A: Yeah .
C: Yep .
A: Right .
E: Well , then the D: What 's what 's wrong with doing times ?
E: I agree .
D: I E: That was what I was wondering .
F: Uh , yeah , time is the E: Time is unique .
C: Well ,
E: You were saying that you didn't think we should F: Time is passing !
A: Time time times are ephemeral .
E: Andreas was saying E: Yeah .
C: what if they haven't notated with them , times ?
F: Yeah .
F: He he 's a language modeling person , though .
A: Um C: So so imagine C: I think his his example is a good one .
C: Imagine that this person who developed the corpus of the referring expressions didn't include time .
A: Mm - hmm .
A: Yeah .
C: He included references to words .
E: Ach !
E: Well , then C: He said that at this word is when when it happened .
A: Yeah .
A: Or she .
E: But then couldn't you just indirectly figure out the time tied to the word ?
C: Or she .
F: But still they F: Exactly .
C: Sure .
F: Yeah .
C: But what if what if they change the words ?
E: Not E: Well ,
E: but you 'd have some anchoring point .
E: He couldn't have changed all the words .
D: But can they change the words without changing the time of the word ?
C: Sure .
C: But they could have changed it a little .
C: The the point is , that that they may have annotated it off a word transcript that isn't the same as our word transcript ,
C: so how do you merge it back in ?
C: I understand what you 're saying .
A: Mmm .
A: Mm - hmm .
C: And I I guess the answer is , um , it 's gonna be different every time .
C: It 's j it 's just gonna be F: Yeah .
E: Yeah .
F: You only know the boundaries of the C: I it 's exactly what I said before ,
C: which is that " what do you mean by " merge " ? "
C: So in this case where you have the words and you don't have the times , well , what do you mean by " merge " ?
F: Right .
C: If you tell me what you mean , I can write a program to do it .
F: Right .
F: You can merge at the level of the representation that the other person preserved and that 's it .
C: Right .
C: And that 's about all you can do .
F: And beyond that , all you know is is relative ordering
F: and sometimes even that is wrong .
C: So so in so in this one you would have to do a best match between the word sequences ,
F: So .
A: Mm - hmm .
C: extract the times f from the best match of theirs to yours ,
C: and use that .
F: And then infer that their time - marks are somewhere in between .
C: Right .
F: Yeah , exactly .
E: But it could be that they just uh , I mean , it could be that they chunked they they lost certain utterances and all that stuff ,
C: Right ,
C: exactly .
C: So it could get very , very ugly .
E: or F: Definitely .
F: Definitely .
E: Yeah .
F: Alright .
F: Well , I guess , w I I didn't want to keep people too long
E: That 's interesting .
F: and Adam wanted t people F: I 'll read the digits .
F: If anyone else offers to , that 'd be great .
F: And
A: Ah , well .
F: if not , I guess C: Yeah .
C: More digits , the better .
A: For th for the <nonvocalsound> for the benefit of science we 'll read the digits .
C: OK ,
F: Thanks thanks a lot .
C: this is
F: It 's really helpful .
F: I mean , Adam and Don <nonvocalsound> will sort of meet
F: and I think that 's great .
F: Very useful .
F: Go next .
D: Scratch that .
E: O three
C: Oh , right .