F: OK .
C: So you would say " refer to this external file " .
C: Um ,
C: so that external file wouldn't be in F: So that might that might work .
D: But what what 's the advantage of doing that versus just putting it into this format ?
C: More compact ,
C: which I think is is better .
D: Uh - huh .
C: I mean , if you did it at this F: I mean these are long meetings
F: and with for every frame ,
C: You don't want to do it with that F: um C: Anything at frame - level you had better encode binary
C: or it 's gonna be really painful .
A: Or you just compre
A: I mean , I like text formats .
A: Um ,
A: b you can always , uh , G - zip them ,
A: and , um , you know , c decompress them on the fly if y if space is really a concern .
D: Yeah , I was thi
D: I was thinking the advantage is that we can share this with other people .
C: Well , but if you 're talking about one per frame , you 're talking about gigabyte - size files .
C: You 're gonna actually run out of space in your filesystem for one file .
F: These are big files .
F: These are really F: I mean C: Right ?
C: Because you have a two - gigabyte limit on most O Ss .
A: Right ,
A: OK .
A: I would say A: OK , so frame - level is probably not a good idea .
F: And th it 's A: But for phone - level stuff it 's perfectly A: Like phones , or syllables , or anything like that .
F: Phones are every five frames though ,
F: so .
F: Or something like that .
A: But but but most of the frames are actually not speech .
A: So ,
A: you know , people don't F: Yeah ,
A: v Look at it , words times the average The average number of phones in an English word is , I don't know , <comment> five maybe ?
F: but we actually A: So , look at it , t number of words times five .
F: Oh , so you mean pause phones take up a lot of the A: That 's not that not F: long pause phones .
C: Yep .
A: Exactly .
F: Yeah .
F: OK .
A: Yeah .
F: That 's true .
F: But you do have to keep them in there .
F: Y yeah .
C: So I think it it 's debatable whether you want to do phone - level in the same thing .
F: OK .
C: But I think , a anything at frame - level , even P - file , is too verbose .
F: OK .
F: So C: I would use something tighter than P - files .
F: Do you Are you familiar with it ?
C: So .
F: I haven't seen this particular format ,
A: I mean , I 've I 've used them .
F: but A: I don't know what their structure is .
F: OK .
A: I 've forgot what the str
D: But , wait a minute ,
D: P - file for each frame is storing a vector of cepstral or PLP values ,
D: right ?
C: It 's whatever you want , actually .
D: Right .
C: So that what 's nice about the P - file It i
C: Built into it is the concept of frames , utterances , sentences , that sort of thing , that structure .
C: And then also attached to it is an arbitrary vector of values .
F: Oh .
C: And it can take different types .
C: So it th they don't all have to be floats .
C: You know , you can have integers and you can have doubles , and all that sort of stuff .
F: So that that sounds that sounds about what I w
C: Um .
C: Right ?
C: And it has a header it has a header format that describes it to some extent .
C: So ,
C: the only problem with it is it 's actually storing the utterance numbers and the frame numbers in the file ,
C: even though they 're always sequential .
C: And so it does waste a lot of space .
A: Hmm .
C: But it 's still a lot tighter than than ASCII .
C: And we have a lot of tools already to deal with it .
F: You do ?
F: OK .
F: Is there some documentation on this somewhere ?
C: Yeah ,
C: there 's a ton of it .
F: OK ,
F: great .
C: Man - pages and , uh , source code , and me .
F: So ,
F: I mean , that sounds good .
F: I I was just looking for something F: I 'm not a database person ,
F: but something sort of standard enough that , you know , if we start using this we can give it out ,
F: other people can work on it ,
C: Yeah , it 's not standard .
F: or <comment> Is it ?
C: I mean , it 's something that we developed at ICSI .
C: But , uh F: But it 's been used here
C: But it 's been used here
F: and people 've C: and and , you know , we have a well - configured system that you can distribute for free ,
C: and D: I mean , it must be the equivalent of whatever you guys used to store feat your computed features in , right ?
F: OK .
A: Yeah ,
A: th we have A: Actually , we we use a generalization of the the Sphere format .
D: Mmm .
A: Um ,
A: but A: Yeah , so there is something like that
A: but it 's , um , probably not as sophist
D: And I think there 's C: Well , what does H T K do for features ?
C: Or does it even have a concept of features ?
A: They ha it has its own A: I mean , Entropic has their own feature format that 's called , like , S - SD or some so SF or something like that .
F: Yeah .
D: Yeah .
C: I 'm just wondering , would it be worth while to use that instead ?
A: Hmm ?
F: Yeah .
F: Th - this is exactly the kind of decision It 's just whatever D: But , I mean , people don't typically share this kind of stuff , right ?
A: Right .
D: I mean C: They generate their own .
F: Actually , I I just you know , we we 've done this stuff on prosodics
D: Yeah .
F: and three or four places have asked for those prosodic files ,
F: and we just have an ASCII , uh , output of frame - by - frame .
C: Ah , right .
F: Which is fine ,
F: but it gets unwieldy to go in and and query these files with really huge files .
C: Right .
F: I mean , we could do it .
F: I was just thinking if there 's something that where all the frame values are C: And a and again , if you have a F: Hmm ?
C: if you have a two - hour - long meeting , that 's gonna F: They 're they 're fair they 're quite large .
F: And these are for ten - minute Switchboard conversations ,
C: Yeah , I mean , they 'd be emo enormous .
C: Right .
F: and F: So it 's doable ,
F: it 's just that you can only store a feature vector at frame - by - frame
F: and it doesn't have any kind of ,
F: um D: Is is the sharing part of this a pretty important consideration
D: or does that just sort of , uh a nice thing to have ?
F: I I don't know enough about what we 're gonna do with the data .
F: But I thought it would be good to get something that we can that other people can use or adopt for their own kinds of encoding .
F: And just , I mean we have to use some we have to make some decision about what to do .
C: Yeah .
F: And especially for the prosody work , what what it ends up being is you get features from the signal ,
F: and of course those change every time your alignments change .
F: So you re - run a recognizer ,
F: you want to recompute your features , um , and then keep the database up to date .
C: Right .
F: Or you change a word , or you change a utterance boundary segment , which is gonna happen a lot .
F: And so I wanted something where all of this can be done in a elegant way
F: and that if somebody wants to try something or compute something else , that it can be done flexibly .
F: Um ,
F: it doesn't have to be pretty ,
F: it just has to be , you know , easy to use , and C: Yeah , the other thing C: We should look at ATLAS ,
C: the NIST thing ,
F: Oh .
A: Mmm .
C: and see if they have anything at that level .
F: Uh C: I mean , I 'm not sure what to do about this with ATLAS ,
C: because they chose a different route .
C: I chose something that C: Th - there are sort of two choices .
C: Your your file format can know about know that you 're talking about language and speech , which is what I chose , and time ,
C: or your file format can just be a graph representation .
C: And then the application has to impose the structure on top .
C: So what it looked like ATLAS chose is , they chose the other way ,
C: which was their file format is just nodes and links ,
C: and you have to interpret what they mean yourself .
F: And why did you not choose that type of approach ?
C: Uh , because I knew that we were doing speech ,
C: and I thought it was better if you 're looking at a raw file to be t for the tags to say " it 's an utterance " , as opposed to the tag to say " it 's a link " .
F: OK .
F: OK .
F: But other than that , are they compatible ?
C: So , but F: I mean , you could sort of C: Yeah ,
C: they 're reasonably compatible .
F: I mean , you you could D: You could probably translate between them .
F: Yeah , that 's w
C: Yep .
F: So ,
C: So ,
C: well , the other thing is if we choose to use ATLAS , which maybe we should just do , we should just throw this out before we invest a lot of time in it .
F: OK .
F: I don't F: So this is what the meeting 's about ,
F: just sort of how to C: Yeah .
F: Um , cuz we need to come up with a database like this just to do our work .
F: And I actually don't care , as long as it 's something useful to other people , what we choose .
C: Yeah .
F: So maybe it 's maybe oth you know ,
F: if if you have any idea of how to choose , cuz I don't .
C: The only thing C: Yeah .
A: Do they already have tools ?
C: I mean , I I chose this for a couple reasons .
C: One of them is that it 's easy to parse .
C: You don't need a full XML parser .
C: It 's very easy to just write a Perl script to parse it .
A: As long as uh each tag is on one line .