G: Mmm .
F: Because because the what they 're asking for is is a feature set .
F: Right ?
F: And so , uh , we 're the ones who have been weird by by doing this training .
G: Yeah .
F: But if we say , " No , you have to have a different feature set for each language , " I think this is ver gonna be very bad .
C: Oh .
F: So G: You think so .
E: Oh .
C: That 's G: Mmm .
F: Oh yeah .
F: Yeah . I mean , in principle , I mean conceptually , it 's sort of like they want a re @ @ <comment> well , they want a replacement for mel cepstra .
G: Mmm .
F: So , we say " OK , this is the year two thousand ,
F: we 've got something much better than mel cepstra .
F: It 's , you know , gobbledy - gook . "
F: OK ? And so we give them these gobbledy - gook features
F: but these gobbledy - gook features are supposed to be good for any language .
G: Hmm .
F: Cuz you don't know who 's gonna call ,
F: and you know , I mean so it 's it 's it 's , uh ,
F: uh how do you know what language it is ?
F: Somebody picks up the phone .
F: So thi this is their image .
G: Well , I <comment> chh F: Someone picks up the phone , right ?
F: And and he he picks up the ph
G: Yeah , but the the application is G: there is a target language for the application .
G: So , if a F: Yeah . y y y
F: Well .
G: Well .
F: But , no but , y you you pick up the phone ,
G: Yeah ?
F: you talk on the phone ,
F: and it sends features out .
F: OK ,
F: so the phone doesn't know what a what what your language is .
G: Yeah ,
G: if G: Yeah .
G: If it 's th in the phone ,
G: but G: well , it that that could be th at the server 's side ,
F: But that 's the image that they have .
G: and , well . Mmm , yeah .
F: It could be ,
F: but that 's the image they have ,
F: right ?
F: So that 's that 's F: I mean , one could argue all over the place about how things really will be in ten years .
F: But the particular image that the cellular industry has right now is that it 's distributed speech recognition , where the , uh , uh , probabilistic part , and and s semantics and so forth are all on the servers , and you compute features of the uh , on the phone .
F: So that 's that 's what we 're involved in .
F: We might might or might not agree that that 's the way it will be in ten years ,
F: but that 's that 's that 's what they 're asking for .
F: So so I think that th th it is an important issue whether it works cross - language .
F: Now , it 's the OGI , uh , folks ' perspective right now that probably that 's not the biggest deal .
F: And that the biggest deal is the , um envir acoustic - environment mismatch .
F: And they may very well be right ,
F: but I I was hoping we could just do a test and determine if that was true .
F: If that 's true , we don't need to worry so much .
F: Maybe maybe we have a couple languages in the training set and that gives us enough breadth uh , uh , that that that the rest doesn't matter .
F: Um , the other thing is , uh , this notion of training to uh which I I guess they 're starting to look at up there , <comment> training to something more like articulatory features .
F: Uh , and if you have something that 's just good for distinguishing different articulatory features that should just be good across , you know , a wide range of languages .
G: Yeah .
F: Uh , but F: Yeah , so I don't th I know unfortunately I don't I see what you 're comi where you 're coming from , I think ,
F: but I don't think we can ignore it .
G: So we we really have to do test with a real cross - language .
G: I mean , tr for instance training on English and testing on Italian ,
G: or Or we can train or else , uh , can we train a net on , uh , a range of languages and which can include the test the test @ @ the target language ,
C: Test on an unseen .
G: or F: Yeah ,
F: so , um ,
F: there 's there 's , uh F: This is complex .
F: So , ultimately , uh , as I was saying , I think it doesn't fit within their image that you switch nets based on language .
G: Yeah .
F: Now , can you include , uh , the the target language ?
F: Um , from a purist 's standpoint it 'd be nice not to
F: because then you can say when F: because surely someone is going to say at some point , " OK , so you put in the German and the Finnish .
G: Mmm .
F: Uh , now , what do you do , uh , when somebody has Portuguese ? "
F: you know ?
F: Um , and Uh , however , you aren't it isn't actually a constraint in this evaluation .
F: So I would say if it looks like there 's a big difference to put it in , then we 'd make note of it , and then we probably put in the other ,
F: because we have so many other problems in trying to get things to work well here that that , you know , it 's not so bad as long as we we note it and say , " Look , we did do this " .
G: Mmm ?
A: And so , ideally , what you 'd wanna do is you 'd wanna run it with and without the target language and the training set for a wide range of languages .
F: Uh .
F: Yeah .
G: Yeah , perhaps .
G: Yeah .
G: Yeah .
F: Yeah .
A: And that way you can say , " Well , " you know , " we 're gonna build it for what we think are the most common ones " ,
A: but if that somebody uses it with a different language , you know , " here 's what 's you 're l here 's what 's likely to happen . "
F: Yeah , cuz the truth is , is that it 's it 's not like there are I mean , al although there are thousands of languages , uh , from uh , uh , the point of view of cellular companies , there aren't .
F: There 's  you know , there 's fifty or something ,
A: Right .
F: you know ?
F: So ,
F: uh , an and they aren't you know , with the exception of Finnish , which I guess it 's pretty different from most most things .
F: uh , it 's it 's , uh most of them are like at least some of the others .
F: And so , our guess that Spanish is like Italian , and and so on .
F: I guess Finnish is a is is a little bit like Hungarian , supposedly ,
F: right ?
F: Or is F: I think A: I don't know anything about Finnish .
F: well , I kn oh , well I know that H uh , H I mean , I 'm not a linguist , but I guess Hungarian and Finnish and one of the one of the languages from the former Soviet Union are in this sort of same family .
F: But they 're just these , you know , uh countries that are pretty far apart from one another , have I guess , people rode in on horses and brought their A: Hmm .
A: Hmm .
F: OK .
G: The G: Yeah .
F: Your turn .
C: Oh , my turn .
C: Oh , OK .
C: Um , Let 's see , I I spent the last week , uh , looking over Stephane 's shoulder . And  and understanding some of the data .
C: I re - installed , um , um , HTK , the free version ,
C: so , um , everybody 's now using three point O ,
C: which is the same version that , uh , OGI is using .
F: Oh , good .
C: Yeah .
C: So ,
C: without without any licensing big deals , or anything like that .
C: And , um , so we 've been talking about this this , uh , cube thing ,
C: and it 's beginning more and more looking like the , uh , the Borge cube thing .
C: It 's really gargantuan .
C: Um ,
C: but I I 'm Am I F: So are are you going to be assimilated ?
A: Resistance is futile .
C: Exactly .
C: Um , yeah , so I I 've been looking at , uh , uh , TIMIT stuff .
C: Um , the the stuff that we 've been working on with TIMIT , trying to get a , um a labels file so we can , uh , train up a train up a net on TIMIT and test , um , the difference between this net trained on TIMIT and a net trained on digits alone .
F: Mm - hmm .
C: Um , and seeing if if it hurts or helps .
F: And again , when y just to clarify , when you 're talking about training up a net , you 're talking about training up a net for a tandem approach ?
C: Anyway .
C: Yeah , yeah .
C: Um .
F: And and the inputs are PLP and delta and that sort of thing ,
C: Mm - hmm .
C: Well , the inputs are one dimension of the cube ,
F: or ?
C: which , um , we 've talked about it being , uh , PLP , um , M F C Cs , um , J - JRASTA , JRASTA - LDA G: Hmm .
F: Yeah , but your initial things you 're making one choice there ,
C: Yeah ,
F: right ?
F: Which is PLP , or something ?
C: right .
F: Yeah .
C: Um , I I haven't I haven't decided on on the initial thing .
C: Probably probably something like PLP .
C: Yeah .
G: Hmm .
F: Yeah . Um ,
F: so so you take PLP and you you , uh , do it uh , you you , uh , use HTK with it with the transformed features using a neural net that 's trained .
F: And the training could either be from Digits itself or from TIMIT .
F: And that 's the C: Right .
F: and , and th and then the testing would be these other things which which which might be foreign language .
C: Right .
F: I see .
C: Right .
F: I I I get in the picture about the cube .
C: Yeah .
F: OK .
C: Maybe C: OK .
F: OK .
C: Uh - huh .
F: Um ,
F: I mean , those listening to this will not have a picture either ,
F: so , um , I guess I 'm I 'm not any worse off .
F: But
F: but at some point somebody should just show me the cube .
F: It sounds s
F: I I get I think I get the general idea of it ,
C: Yeah , yeah ,
F: yeah .
C: b May
A: So , when you said that you were getting the labels for TIMIT , <comment> um , are y what do you mean by that ?
C: Mm - hmm .
C: Oh , I 'm just I 'm just , uh , transforming them from the , um , the standard TIMIT transcriptions into into a nice long huge P - file to do training .
A: Mmm .
A: Were the digits , um , hand - labeled for phones ?
C: Um , the the digits A: Or were they those labels automatically derived ?
C: Oh yeah ,
C: those were those were automatically derived by by Dan using , um , embedded embedded training and alignment .
A: Mmm .
F: Ah , but which Dan ?
C: Uh , Ellis .
F: OK .
C: Right ?
F: OK .
C: Yeah . So .
A: I was just wondering because that test you 're t
C: Uh - huh .
A: I I think you 're doing this test because you want to determine whether or not , uh , having s general speech performs as well as having specific speech .
C: That 's right .
F: Well , especially when you go over the different languages again ,
F: because you 'd the different languages have different words for the different digits ,
A: Mm - hmm .
A: And I was F: so it 's A: yeah ,
A: so I was just wondering if the fact that TIMIT you 're using the hand - labeled stuff from TIMIT might be confuse the results that you get .
F: I I think it would ,
F: but but on the other hand it might be better .
A: Right , but if it 's better , it may be better because it was hand - labeled .
F: Oh yeah ,
F: but still @ @ probably use it .
A: Yeah . OK .
F: I mean ,
F: you know , I I I guess I 'm sounding cavalier ,
F: but I mean , I think the point is you have , uh , a bunch of labels and and they 're han hand uh hand - marked .
F: Uh , I guess , actually , TIMIT was not entirely hand - marked .
F: It was automatically first , and then hand hand - corrected .
F: But but , um , uh , it it , um , it might be a better source .
A: Oh , OK .
F: So , i it 's F: you 're right .
F: It would be another interesting scientific question to ask , " Is it because it 's a broad source or because it was , you know , carefully ? "
F: uh . And that 's something you could ask ,
A: Mm - hmm .
F: but given limited time , I think the main thing is if it 's a better thing for going across languages on this training tandem system ,
A: Yeah .
F: then it 's probably A: Right .
A: What about the differences in the phone sets ?
C: Uh , between languages ?
A: No , between TIMIT and the the digits .
C: Oh ,
C: um , right .
C: Well , there 's a mapping from the sixty - one phonemes in TIMIT to to fifty - six , the ICSI fifty - six .
E: Sixty - one .
A: Oh , OK .
A: I see .
C: And then the digits phonemes , um , there 's about twenty twenty - two or twenty - four of them ?
C: Is that right ?
A: Out of that fifty - six ?
G: Yep .
C: Out of that fifty - six .
A: Oh , OK .
C: Yeah .
C: So ,
C: it 's it 's definitely broader ,
C: yeah .
G: But , actually , the issue of phoneti phon uh phone phoneme mappings will arise when we will do severa use several languages
E: Yeah .
G: because you Well , some phonemes are not , uh , in every languages ,
G: and So we plan to develop a subset of the phonemes , uh , that includes , uh , all the phonemes of our training languages ,
G: and use a network with kind of one hundred outputs or something like that .
A: Mm - hmm .
F: Mm - hmm .
F: You mean a superset , sort of .
G: Uh , yeah ,
G: superset ,
F: Yeah . Yeah .
G: yeah .
E: Yeah .
E: I th I looks the SAMPA SAMPA phone .
G: Yeah .
E: SAMPA phone ? For English uh American English , and the the the language who have more phone are the English .
A: Mmm .
E: Of the these language .
E: But n for example , in Spain , the Spanish have several phone that d doesn't appear in the E English and we thought to complete .
E: But for that , it needs we must r h do a lot of work because we need to generate new tran transcription for the database that we have .
F: Mm - hmm .
F: Mm - hmm .
B: Other than the language , is there a reason not to use the TIMIT phone set ?
B: Cuz it 's larger ?
B: As opposed to the ICSI phone set ?
C: Oh ,
C: you mean why map the sixty - one to the fifty - six ?
B: Yeah .
C: I don't know .
C: I have F: Um , I forget if that happened starting with you , or was it o or if it was Eric , afterwards who did that .
F: But I think , basically , there were several of the phones that were just hardly ever there .
A: Yeah ,
A: and I think some of them , they were making distinctions between silence at the end and silence at the beginning ,
B: Oh .
A: when really they 're both silence .
A: I th I think it was things like that that got it mapped down to fifty - six .
B: OK .
F: Yeah , especially in a system like ours , which is a discriminative system .
F: You know , you 're really asking this net to learn .
B: Yeah .
A: Yeah .
F: It 's it 's kind of hard .
A: There 's not much difference , really .
A: And the ones that are gone , I think are I think there was they also in TIMIT had like a glottal stop ,
A: which was basically a short period of silence ,
B: Mm - hmm .
A: and so .
B: Well , we have that now , too , right ?
A: I don't know .
B: Yeah .
F: i It 's actually pretty common that a lot of the recognition systems people use have things like like , say thirty - nine , phone symbols ,
A: So .
F: right ?
F: Uh , and then they get the variety by by bringing in the context , the phonetic context .
F: Uh . So we actually have an unusually large number in in what we tend to use here .
F: Um .
F: So , a a actually maybe now you 've got me sort of intrigued .
F: What there 's F: Can you describe what what 's on the cube ?
C: Yeah ,
F: I mean C: w I th I think that 's a good idea
F: Yeah , yeah .
C: to to talk about the whole cube
E: Yeah . Yeah .
C: and maybe we could sections in the cube for people to work on .
F: Yeah .
F: Yeah .
C: Um ,
C: OK .
F: OK ,
C: Uh , do you wanna do it ?
F: so even even though the meeting recorder doesn't doesn't , uh F: and since you 're not running a video camera we won't get this ,
F: but if you use a board it 'll help us anyway .
C: OK .
F: Uh , point out one of the limitations of this medium ,
C: OK .
F: but you 've got the wireless on ,
F: right ?
C: Yeah ,
C: I have the wireless .
F: Yeah ,
F: so you can walk around .
C: OK .
C: Can y can you walk around too ?
C: No .
C: OK ,
C: well ,
F: Uh , he can't , actually ,
C: um ,
F: but F: He 's tethered .
C: s
C: basically , the the cube will have three dimensions .
C: The first dimension is the the features that we 're going to use .
C: And the second dimension , um , is the training corpus .
C: And that 's the training on the discriminant neural net .
C: Um
C: and
C: the last dimension happens to be F: Yeah and again F: Yeah . So the the training for HTK is always that 's always set up for the individual test ,
F: right ?
F: That there 's some training data and some test data .
F: So that 's different than this .
C: Right , right .
C: This is this is for for ANN only .
C: And , yeah , the training for the HTK models is always , uh , fixed for whatever language you 're testing on .
F: Right .
C: And then , there 's the testing corpus .
C: So , then I think it 's probably instructive to go and and and show you the features that we were talking about .
C: Um ,
C: so , let 's see .
C: Help me out with G: PLP .
C: With what ?
G: PLP .
C: PLP ?
C: OK .
G: MSG .
C: MSG .
G: Uh , JRASTA .
C: JRASTA .
G: And JRASTA - LDA .
C: JRASTA - LDA .
G: Um ,
G: multi - band .
C: Multi - band .
G: So there would be multi - band before , um before our network , I mean .
C: Yeah ,
C: just the multi - band features , right ?
G: And G: Yeah .
C: Yeah .
G: So , something like , uh , s TCT within bands
F: Uh - huh . Ah . Ah .
G: and Well .
G: And then multi - band after networks . Meaning
G: that we would have , uh , neural networks , uh , discriminant neural networks for each band .
G: Uh , yeah .
G: And using the the outputs of these networks or the linear outputs or something like that .
G: Uh ,
G: yeah .
A: What about mel cepstrum ?
C: Oh , um A: Or is that A: you don't include that because it 's part of the base or something ?
F: Well , y you do have a baseline system that 's m that 's mel cepstra ,
E: Yeah databases .
E: Yeah .
F: right ?
F: So .
E: Mm - hmm .
G: But , uh ,
G: well ,
G: not for the the ANN .
G: I mean F: OK .
G: So , yeah ,
G: we could we could add MFCC also .
C: We could add F: Probably should .
F: I mean at least at least conceptually , you know , it doesn't meant you actually have to do it ,
G: Yeah .
E: Yeah .
F: but
F: conceptually it makes sense as a as a base line .
A: It 'd be an interesting test just to have just to do MFCC with the neural net
E: Without the E: Yeah .
A: and everything else the same .
A: Compare that with just M - MFCC without the the net .
G: Yeah .
E: Mm - hmm .
C: I think I think Dan did some of that .
A: Oh .
C: Um , in his previous Aurora experiments .
C: And with the net it 's it 's wonderful .
C: Without the net it 's just baseline .
F: Um , I think OGI folks have been doing that , too .
F: D Because I think that for a bunch of their experiments they used , uh , mel cepstra , actually .