three two three four seven six five five three one six two four one six seven seven eight nine zero nine four zero zero three zero one five eight one seven three five three two six eight zero three six two four three zero seven four five zero six nine four seven four eight five seven nine six one five O seven eight O two zero nine six zero four zero zero one two  Yeah , you don't actually n need to say the name . grad c: OK , this is Barry Chen and I am reading transcript professor f: That 'll probably be bleeped out . professor f:   not that there 's anything defamatory about  eight five seven or or anything , but grad c: OK .  so here 's what I have for I I was just jotting down things I think th w that we should do today .  This is what I have for an agenda so far  , We should talk a little bit about the plans for the  the field trip next week .  a number of us are doing a field trip to   OGI And  mostly  First though about the logistics for it . professor f: in and kind of go around see what people have been doing talk about that , a r progress report . If we find some holes in some things that that someone could use some help with , he 's he 's volunteering to help . So , and  , then  , talk a little bit about about disks and resource resource issues that that 's starting to get worked out . And then , anything else anybody has that isn't in that list ?  grad d: I was just wondering , does this mean the battery 's dying and I should change it ? professor f:  I think that means the battery 's O K . professor f: d do you grad d: Oh OK , so th phd a: Yeah , that 's good .   I I it was it was kind of my bright idea to have us take a plane that leaves at seven twenty in the morning .  this is  The reason I did it  was because otherwise for those of us who have to come back the same day it is really not much of a of a visit .  So  the issue is how how how would we ever accomplish that ?  what what what part of town do you live in ? grad c: I live in , the corner of campus . OK , so would it be easier those of you who are not , you know , used to this area , it can be very tricky to get to the airport at at  , you know , six thirty . Would it be easier for you if you came here and I drove you ? Yeah ? Yeah , yeah , OK . It 'll take it it it won't be bad traffic that time of day and and  phd a: I guess once you get past the bridge that that would be the worst . professor f: And then Martin Luther King to nine - eighty to eight - eighty , phd a: Yeah . phd a: Oh , I professor f: So that leaves us fifty minutes before the plane it 'll just yeah . So Great , OK so that 'll It 's  , it 's still not going to be really easy but well Particularly for for  for Barry and me , we 're not we 're not staying overnight so we don't need to bring anything particularly except for  a pad of paper and So , and ,  you , two have to bring a little bit grad c: OK . professor f: but  you know , don't don't bring a footlocker and we 'll be OK So .  I 'll I 'll I 'll I 'll give you my phone number , If I 'm not here for a few m after a few minutes then grad c: Wake you up . I just ,  it for me it just means getting up a half an hour earlier than I usually do .  , I I I figured maybe wait on the potential goals for the meeting  until we talk about wh what 's been going on . phd g: So , it means that  , well , it is , a digit French database of microphone speech , downsampled to eight kilohertz and I 've added noise to one part , with the actually the Aurora - two noises . professor f: OK  , So the HTK base lines so this is using mel cepstra and so on , or ? Yeah . professor f: And again , I guess the p the plan is , to  then given this What 's the plan again ? phd g: The plan with these data ? professor f: With So So Does i Just remind me of what what you were going to do with the what what what what 's y You just described what you 've been doing . phd g:  we actually we want to , mmm , analyze three dimensions , the feature dimension , the training data dimension , and the test data dimension . So we have the  , TI - digit task , the Italian task , the French task and the Finnish task . professor f: Yeah ? phd g: So we have numbers with  systems   neural networks trained on the task data . And then to have systems with neural networks trained on , data from the same language , if possible , with , well , using a more generic database , which is phonetically phonetically balanced , and . professor f: So - so we had talked I guess we had talked at one point about maybe , the language ID corpus ? phd g: Yeah . professor f: Is that a possibility for that ? phd g: Ye -  Yeah , but ,  these corpus , w w there is a CallHome and a CallFriend also , The CallFriend is for language ind identification . This could be a a problem for Why ? Because  , the the SpeechDat databases are not telephone speech . That 's really funny isn't it ?  cuz th this whole thing is for developing new standards for the telephone . phd g: Yeah , but the the idea is to compute the feature before the before sending them to the Well , you don't do not send speech , you send features , computed on th the the device , professor f:   professor f: Oh I see , so your point is that it 's it 's it 's  the features are computed locally , and so they aren't necessarily telephone bandwidth ,  or telephone distortions . phd a: Did you happen to find out anything about the OGI multilingual database ? professor f: Yeah , that 's wh that 's wh that 's what I meant . phd g: Yeah , it 's professor f: I said @ @ , there 's there 's there 's an OGI language ID , not the not the ,  the CallFriend is a is a , LDC w thing , right ? phd g: Yea - Yeah , there are also two other databases . One they call the multi - language database , and another one is a twenty - two language , something like that . professor f: But I 'm not sure phd g: So professor f: we ' r e e The bandwidth shouldn't be such an issue right ? Because e e this is downsampled and and filtered , right ? So it 's just the fact that it 's not telephone .  some of this stuff 's recorded in the car , and some of it 's  there 's there 's many different acoustic differences .  , unless we 're going to include a bunch of car recordings in the in the training database , I 'm not sure if it 's completely rules it out phd g: Yeah . professor f: if our if we if our major goal is to have phonetic context and you figure that there 's gonna be a mismatch in acoustic conditions does it make it much worse f to sort of add another mismatch , if you will . professor f: i i I I guess the question is how important is it to for us to get multiple languages  , in there . Well , actually , for the moment if we w do not want to use these phone databases , we we already have  English , Spanish and French  , with microphone speech . professor f: So that 's what you 're thinking of using is sort of the multi the equivalent of the multiple ? phd g: Well . phd g: So w f for for  Italian , which is close to Spanish , French and , i i  , TI - digits we have both  , digits training data and also more general training data . professor f: Well , we also have this Broadcast News that we were talking about taking off the disk , which is is microphone data for for English . phd g: Yeah , and perhaps ,  We were thinking that perhaps the cross - language issue is not , so big of a issue . u So that th Well , the the guy who has to develop an application with one language can use the net trained o on that language , or a generic net , professor f: depen it depen it depends how you mean " using the net " . phd g: but not trained on a professor f: So , if you 're talking about for producing these discriminative features that we 're talking about you can't do that . But if we say , " No , you have to have a different feature set for each language , " I think this is ver gonna be very bad .  , in principle ,  conceptually , it 's sort of like they want a re @ @ well , they want a replacement for mel cepstra . professor f: So , we say " OK , this is the year two thousand , we 've got something much better than mel cepstra . " OK ? And so we give them these gobbledy - gook features but these gobbledy - gook features are supposed to be good for any language . professor f: Cuz you don't know who 's gonna call , and you know ,  so it 's it 's it 's ,  how do you know what language it is ? Somebody picks up the phone . Someone picks up the phone , right ? phd g: Well , I chh professor f: And and he he picks up the ph phd g: Yeah , but the the application is there is a target language for the application . professor f: you talk on the phone , phd g: Yeah ? professor f: and it sends features out . If it 's th in the phone , but professor f: But that 's the image that they have . phd g: well , it that that could be th at the server 's side , professor f: It could be , phd g: and , well . professor f: but that 's the image they have , right ? So that 's that 's  , one could argue all over the place about how things really will be in ten years . But the particular image that the cellular industry has right now is that it 's distributed speech recognition , where the , probabilistic part , and and s semantics and so forth are all on the servers , and you compute features of the  , on the phone . We might might or might not agree that that 's the way it will be in ten years , but that 's that 's that 's what they 're asking for . Now , it 's the OGI , folks ' perspective right now that probably that 's not the biggest deal . And they may very well be right , but I I was hoping we could just do a test and determine if that was true . Maybe maybe we have a couple languages in the training set and that gives us enough breadth  , that that that the rest doesn't matter .  , the other thing is , this notion of training to  which I I guess they 're starting to look at up there , training to something more like articulatory features .  , and if you have something that 's just good for distinguishing different articulatory features that should just be good across , you know , a wide range of languages . professor f: but Yeah , so I don't th I know unfortunately I don't I see what you 're comi where you 're coming from , I think , but I don't think we can ignore it .  , tr for instance training on English and testing on Italian , or Or we can train or else , can we train a net on , a range of languages and which can include the test the test @ @ the target language , grad c: Test on an unseen . phd g: or professor f: Yeah , so , there 's there 's ,  This is complex . So , ultimately , as I was saying , I think it doesn't fit within their image that you switch nets based on language . professor f: from a purist 's standpoint it 'd be nice not to because then you can say when because surely someone is going to say at some point , " OK , so you put in the German and the Finnish . professor f: now , what do you do , when somebody has Portuguese ? " you know ?  , and  , however , you aren't it isn't actually a constraint in this evaluation . So I would say if it looks like there 's a big difference to put it in , then we 'd make note of it , and then we probably put in the other , because we have so many other problems in trying to get things to work well here that that , you know , it 's not so bad as long as we we note it and say , " Look , we did do this " . phd g: Mmm ? phd a: And so , ideally , what you 'd wanna do is you 'd wanna run it with and without the target language and the training set for a wide range of languages . phd a: And that way you can say , " Well , " you know , " we 're gonna build it for what we think are the most common ones " , professor f: Yeah . phd a: but if that somebody uses it with a different language , you know , " here 's what 's you 're l here 's what 's likely to happen . " professor f: Yeah , cuz the truth is , is that it 's it 's not like there are  , al although there are thousands of languages , from  , the point of view of cellular companies , there aren't . professor f: There 's you know , there 's fifty or something , you know ? So , an and they aren't you know , with the exception of Finnish , which I guess it 's pretty different from most most things . I guess Finnish is a is is a little bit like Hungarian , supposedly , right ? phd a: I don't know anything about Finnish . professor f: Or is I think well , I kn oh , well I know that H  , H  , I 'm not a linguist , but I guess Hungarian and Finnish and one of the one of the languages from the former Soviet Union are in this sort of same family . professor f: But they 're just these , you know ,  countries that are pretty far apart from one another , have I guess , people rode in on horses and brought their phd a:  . I re - installed , HTK , the free version , so , everybody 's now using three point O , which is the same version that , OGI is using . And , so we 've been talking about this this , cube thing , and it 's beginning more and more looking like the , the Borge cube thing .  , but I I 'm Am I professor f: So are are you going to be assimilated ? phd a: Resistance is futile .  , the the stuff that we 've been working on with TIMIT , trying to get a ,  a labels file so we can , train up a train up a net on TIMIT and test , the difference between this net trained on TIMIT and a net trained on digits alone . professor f: And again , when y just to clarify , when you 're talking about training up a net , you 're talking about training up a net for a tandem approach ? grad c: Yeah , yeah . professor f: And and the inputs are PLP and delta and that sort of thing , grad c: Well , the inputs are one dimension of the cube , professor f: or ? grad c: which , we 've talked about it being , PLP , M F C Cs , J - JRASTA , JRASTA - LDA phd g:  . professor f: Yeah , but your initial things you 're making one choice there , grad c: Yeah , professor f: right ? grad c: right . professor f: Which is PLP , or something ? grad c: I I haven't I haven't decided on on the initial thing .  , so so you take PLP and you you , do it  , you you , use HTK with it with the transformed features using a neural net that 's trained . professor f: And that 's the and , and th and then the testing would be these other things which which which might be foreign language .  , those listening to this will not have a picture either , so , I guess I 'm I 'm not any worse off . It sounds s I I get I think I get the general idea of it , grad c: Yeah , yeah , professor f: yeah . phd a: So , when you said that you were getting the labels for TIMIT , are y what do you mean by that ? grad c: b May   Oh , I 'm just I 'm just , transforming them from the , the standard TIMIT transcriptions into into a nice long huge P - file to do training . Were the digits , hand - labeled for phones ? grad c: the the digits phd a: Or were they those labels automatically derived ? grad c: Oh yeah , those were those were automatically derived by by Dan using , embedded embedded training and alignment . phd a: I was just wondering because that test you 're t grad c:  - huh . phd a: I I think you 're doing this test because you want to determine whether or not , having s general speech performs as well as having specific speech . professor f: Well , especially when you go over the different languages again , because you 'd the different languages have different words for the different digits , phd a:   And I was professor f: so it 's phd a: yeah , so I was just wondering if the fact that TIMIT you 're using the hand - labeled stuff from TIMIT might be confuse the results that you get . phd a: Right , but if it 's better , it may be better because it was hand - labeled . professor f: you know , I I I guess I 'm sounding cavalier , but  , I think the point is you have , a bunch of labels and and they 're han hand  hand - marked . It would be another interesting scientific question to ask , " Is it because it 's a broad source or because it was , you know , carefully ? " phd a:   And that 's something you could ask , but given limited time , I think the main thing is if it 's a better thing for going across languages on this training tandem system , phd a: Yeah . professor f: then it 's probably phd a: What about the differences in the phone sets ? grad c: between languages ? phd a: No , between TIMIT and the the digits . Well , there 's a mapping from the sixty - one phonemes in TIMIT to to fifty - six , the ICSI fifty - six . grad c: And then the digits phonemes , there 's about twenty twenty - two or twenty - four of them ? Is that right ? phd a: Out of that fifty - six ? phd g: Yep . phd g: But , actually , the issue of phoneti phon  phone phoneme mappings will arise when we will do severa use several languages phd e: Yeah . phd g: because you Well , some phonemes are not , in every languages , and So we plan to develop a subset of the phonemes , that includes , all the phonemes of our training languages , phd a:   phd e: SAMPA phone ? For English  American English , and the the the language who have more phone are the English . But n for example , in Spain , the Spanish have several phone that d doesn't appear in the E English and we thought to complete . But for that , it needs we must r h do a lot of work because we need to generate new tran transcription for the database that we have . phd b: Other than the language , is there a reason not to use the TIMIT phone set ? Cuz it 's larger ? As opposed to the ICSI phone set ? grad c: Oh , you mean why map the sixty - one to the fifty - six ? phd b: Yeah . I have professor f: I forget if that happened starting with you , or was it o or if it was Eric , afterwards who did that . phd a: Yeah , and I think some of them , they were making distinctions between silence at the end and silence at the beginning , when really they 're both silence . phd a: I th I think it was things like that that got it mapped down to fifty - six . professor f: Yeah , especially in a system like ours , which is a discriminative system . And the ones that are gone , I think are I think there was they also in TIMIT had like a glottal stop , which was basically a short period of silence , phd b:   phd b: Well , we have that now , too , right ? phd a: I don't know . professor f: i It 's actually pretty common that a lot of the recognition systems people use have things like like , say thirty - nine , phone symbols , right ?  , and then they get the variety by by bringing in the context , the phonetic context . What there 's Can you describe what what 's on the cube ? grad c: Yeah , w I th I think that 's a good idea professor f:  grad c: to to talk about the whole cube professor f: Yeah , yeah .  , do you wanna do it ? professor f: OK , so even even though the meeting recorder doesn't doesn't ,  and since you 're not running a video camera we won't get this , but if you use a board it 'll help us anyway . professor f: but you 've got the wireless on , grad c: Yeah , I have the wireless . OK , well , professor f: he can't , actually , but grad c: s basically , the the cube will have three dimensions . So the the training for HTK is always that 's always set up for the individual test , right ? That there 's some training data and some test data . And , yeah , the training for the HTK models is always , fixed for whatever language you 're testing on . So , then I think it 's probably instructive to go and and and show you the features that we were talking about . grad c: Yeah , just the multi - band features , right ? phd g: And grad c: Yeah . phd a: What about mel cepstrum ? Or is that grad c: Oh ,  phd a: you don't include that because it 's part of the base or something ? phd e: Yeah databases . professor f: Well , y you do have a baseline system that 's m that 's mel cepstra , phd e: Yeah .  at least at least conceptually , you know , it doesn't meant you actually have to do it , phd g: Yeah . phd a: It 'd be an interesting test just to have just to do MFCC with the neural net phd e: Without the phd a: and everything else the same . D Because I think that for a bunch of their experiments they used , mel cepstra , actually . phd a: Is that Was that distributed with Aurora , or ? grad c: One L or two L 's ? phd a: Where did that ? professor f: The newer one . And , oh yeah , and professor f: Is it French French or Belgian French ? There 's a phd g: It 's , French French . Yeah , Herve always insists that Belgian is i is absolutely pure French , has nothing to do with but he says those those those Parisians talk funny . grad c: right ? Spanish Oh , Spanish stories ? phd e: Albayzin is the name . phd a: What about TI - digits ? grad c: TI - digits  all these Aurora f d data p data is from is derived from TI - digits . grad c: basically , they they corrupted it with , different kinds of noises at different SNR levels . professor f: y And I think Stephane was saying there 's there 's some broader s material in the French also ? phd g: Yeah , we cou we could use grad c: OK . phd b: Did the Aurora people actually corrupt it themselves , or just specify the signal and the signal - t grad c: They they corrupted it , themselves , phd b: OK . grad c: but they also included the the noise files for us , right ? Or phd g: Yeah . professor f: I 'm just curious , Carmen  , I couldn't tell if you were joking or i Is it is it Mexican Spanish , phd e: No no no no . phd g: Yeah , the No , the French is f yeah , from , Paris , grad c: Oh , from Paris , OK . And , with within the training corporas  , we 're , thinking about , training with noise . So , incorporating the same kinds of noises that , Aurora is in incorporating in their ,  in their training corpus .  , I don't think we we 're given the ,  the unseen noise conditions , though , right ? professor f: I think what they were saying was that , for this next test there 's gonna be some of the cases where they have the same type of noise as you were given before hand and some cases where you 're not . professor f: So , presumably , that 'll be part of the topic of analysis of the the test results , is how well you do when it 's matching noise and how well you do where it 's not . grad c: So , I guess we can't train on on the the unseen noise conditions .  , i i i i it does seem to me that a lot of times when you train with something that 's at least a little bit noisy it can it can help you out in other kinds of noise even if it 's not matching just because there 's some more variance that you 've built into things . professor f: exactly how well it will work will depend on how near it is to what you had ahead of time . professor f: and then your testing corpus ? grad c: the testing corporas are , just , the same ones as Aurora testing . grad c: we ' r we 're gonna get German , right ? Ge - At the final test will have German . professor f: Well , so , yeah , the final test , on a guess , is supposed to be German and Danish , phd g: yeah . professor f: Oh , there 's a there 's Spanish testing in the Aurora ? phd g: not yet , but , yeah , e phd e: Yeah , it 's preparing . phd g: and , well , according to Hynek it will be we will have this at the end of November , or  . phd g: Yeah professor f: So that 's , three hundred and forty - three , different systems that are going to be developed . grad d: What a what about noise conditions ? professor f: What ? grad d: w Don't we need to put in the column for noise conditions ? professor f: Are you just trying to be difficult ? grad d: No , I just don't understand . grad c: Well , th  , when when I put these testings on there , I 'm assumi professor f: I 'm just kidding . And they 're all they 're all gonna be test tested , with one training of the HTK system . grad d: And do we do all our training on clean data ? grad c: no , no , phd e: Also , we can clean that . grad c: we 're we 're gonna be , training on the noise files that we do have . professor f: So ,  Yeah , so I guess the question is how long does it take to do a a training ?  , it 's not totally crazy t  , these are a lot of these are built - in things and we know we have programs that compute PLP , we have MSG , we have JRA you know , a lot of these things will just kind of happen , won't take  a huge amount of development , it 's just trying it out . professor f: But how how long does it take , do we think , for one of these trainings ? grad c: That 's a good question .  , cuz , so , for instance , I think the major advantage of MSG grad c: Oh ! professor f: Yeah , grad c: Och ! professor f: good point . A major advantage of MSG , I see , th that we 've seen in the past is combined with PLP . grad c: Now , this is turning into a four - dimensional cube ? phd a: Well , you just select multiple things on the one dimension . professor f: Yeah , so , you don't wanna ,  Let 's see , seven choose two would be , twenty - one different combinations . phd b: It 's not a complete set of combinations , though , professor f: Probably phd b: right ? It 's not a complete set of combinations , though , professor f: What ? phd b: right ? grad c: No . Yeah , there 's grad c: That would be professor f: yeah , so PLP and MSG I think we definitely wanna try cuz we 've had a lot of good experience with putting those together . phd a: When you do that , you 're increasing the size of the inputs to the net . Do you have to reduce the hidden layer , or something ? professor f: Well , so  , so i it doesn't increase the number of trainings . phd a: No , no , I 'm I 'm just wondering about number of parameters in the net . Do you have to worry about keeping that the same , or ? professor f: I don't think so . phd b: There 's a computation limit , though , isn't there ? professor f: Yeah , it 's just more compu Excuse me ? phd b: Isn't there like a limit on the computation load , or d latency , or something like that for Aurora task ? professor f: Oh yeah , we haven't talked about any of that at all , have we ? grad c: No . What it is is that there 's there 's ,  it 's just penalty , you know ? That that if you 're using , a megabyte , then they 'll say that 's very nice , but , of course , it will never go on a cheap cell phone .  , and , expensive cell phones , exa expensive hand - helds , and so forth , are gonna have lots of memory . So it 's just that , these people see the the cheap cell phones as being still the biggest market , so . But , yeah , I was just realizing that , actually , it doesn't explode out ,  It 's not really two to the seventh . But it 's but but i i it doesn't really explode out the number of trainings cuz these were all trained individually . Right ? So , if you have all of these nets trained some place , then , you can combine their outputs and do the KL transformation and so forth grad c:   phd a: But wh what about a net that 's trained on multiple languages , though ? professor f: Well , you gotta do the KL transformation , phd g: Eight y professor f: but phd a: Is that just separate nets for each language then combined , or is that actually one net trained on ? phd e: Necessary to put in . Right ? phd g: So , in the broader training corpus we can we can use , the three , or , a combination of of two two languages . professor f: Yeah , so , I guess the first thing is if w if we know how much a how long a a training takes , if we can train up all these these combinations , then we can start working on testing of them individually , and in combination . professor f: Because the putting them in combination , I think , is not as much computationally as the r training of the nets in the first place . But there is the testing also , which implies training , the HTK models phd e: The the model the HTK model . professor f: How long does it take for an , HTK training ? phd g: It 's around six hours , I think . professor f: No , I 'm sorry , ru running on what machine ? phd e: Ravioli . phd g: Yeah , I I think it 's - it 's - it 's not so long because , well , the TI - digits test data is about ,  how many hours ?  , th  , thirty hours of speech , I think , professor f: It 's a few hours . professor f: so , clearly , there there 's no way we can even begin to do an any significant amount here unless we use multiple machines . professor f: Right ? So so w we  there 's plenty of machines here and they 're n they 're often not in in a great great deal of use . So , I think it 's it 's key that that the that you look at , you know , what machines are fast , what machines are used a lot  , are we still using P - make ? Is that ? grad c: Oh , I don't know how w how we would P - make this , though . professor f: Well , you have a  , once you get the basic thing set up , you have just all the  , a all these combinations , grad c: Yeah . It 's it 's let 's say it 's six hours or eight hours , or something for the training of HTK . How long is it for training of of , the neural net ? grad c: The neural net ?  . phd b: How big is the net ? phd e: For Albayzin I trained on neural network , was , one day also . professor f: And I think there there there 's I think you folks are probably go the ones using them right now . phd a: Is it faster to do it on the SPERT , or ? professor f: don't know . grad c: It 's it 's still a little faster on the professor f: Used to be . Or either Adam or or Dan did some testing and they found that the SPERT board 's still still faster . grad c: And the benefits is that , you know , you run out of SPERT and then you can do other things on your your computer , professor f:   You could set up , you know , ten different jobs , or something , to run on SPERT different SPERT boards and and have ten other jobs running on different computers . So , it 's got to take that sort of thing , or or we 're not going to get through any significant number of these . professor f:  So this is Yeah , I kind of like this because what it No grad c: OK . professor f: no , what I like about it is we we we do have a problem that we have very limited time . You know , so , with very limited time , we actually have really quite a quite a bit of computational resource available if you , you know , get a look across the institute and how little things are being used . And  , on the other hand , almost anything that really i you know , is is new , where we 're saying , " Well , let 's look at , like we were talking before about , voiced - unvoiced - silence detection features and all those sort " that 's phd e: Yeah . But if it 's new , then we have this development and and and learning process t to to go through on top of just the the all the all the work . So what I like about this is you basically have listed all the things that we already know how to do . And , you 're just saying let 's look at the outer product of all of these things and see if we can calculate them . a a Am I am I interpreting this correctly ? Is this sort of what what you 're thinking of doing in the short term ? phd g: Mmm . professor f: So so then I think it 's just the the missing piece is that you need to , you know you know , talk to talk to , Chuck , talk to , Adam , sort out about , what 's the best way to really , you know , attack this as a as a as a mass problem in terms of using many machines .  , and  , then , you know , set it up in terms of scripts and so forth , and  , in in kind o some kind of structured way .  , and , you know , when we go to , OGI next week , we can then present to them , you know , what it is that we 're doing . And , we can pull things out of this list that we think they are doing sufficiently , grad c: Mmm . professor f: that , you know , we 're not we won't be contributing that much . phd b: How big are the nets you 're using ? grad c: for the for nets trained on digits , we have been using , four hundred order hidden units . And , for the broader class nets we 're we 're going to increase that because the , the digits nets only correspond to about twenty phonemes .  , w we 're gonna professor f: Oh , it 's not actually broader class , it 's actually finer class , but you mean y You mean more classes . professor f: Carmen , did you do you have something else to add ? We you haven't talked too much , and phd e: D I begin to work with the Italian database to nnn , to with the f front - end and with the HTK program and the @ @ . And I trained eh , with the Spanish two neural network with PLP and with LogRASTA PLP . professor f: Well , JRASTA has the potential to do better , but it doesn't always . It 's it 's ,  instead of doing RASTA with a log , you 're doing RASTA with a log - like function that varies depending on a J parameter , which is supposed to be sensitive to the amount of noise there is . So , it 's sort of like the right transformation to do the filtering in , is dependent on how much noise there is . It 's a little complicated because once you do that , you end up in some funny domain and you end up having to do a transformation afterwards , which requires some tables . professor f: so it 's it 's it 's a little messier , there 's more ways that it can go wrong , but if if if you 're careful with it , it can do better . phd e: and I think to to to recognize the Italian digits with the neural netw Spanish neural network , and also to train another neural network with the Spanish digits , the database of Spanish digits . Was for me , n it was a difficult work last week with the labels because the the program with the label obtained that I have , the Albayzin , is different w to the label to train the neural network . professor f: I 'm sorry , phd e: I 'm sorry , professor f: I have a p I had a problem with the pronunciation . phd e: Oh , also that professor f: So , OK , so let 's start over . professor f: So , TI TIMI TIMIT 's hand - labeled , and and you 're saying about the Spanish ? phd e: The Spanish labels ? That was in different format , that the format for the em the program to train the neural network . Yeah , but n yes , because they have one program , Feacalc , but no , l LabeCut , l LabeCut , but don't doesn't , eh , include the HTK format to convert . I ask e even I ask to Dan Ellis what I can do that , and h they he say me that h he does doesn't any any s any form to to do that . And at the end , I think that with LabeCut I can transfer to ASCII format , and HTK is an ASCII format . And I m do another , one program to put ASCII format of HTK to ase ay ac ASCII format to Exceed professor f:   phd e: Actually that was complicated , professor f: So you phd e: but well , I know how we can did that do that . So it 's just usual kind of  sometimes say housekeeping , right ? To get these get these things sorted out . professor f: So it seems like there 's there 's some peculiarities of the ,  of each of these dimensions that are getting sorted out . And then , if if you work on getting the , assembly lines together , and then the the pieces sort of get ready to go into the assembly line and gradually can start , you know , start turning the crank , more or less . And , we have a lot more computational capability here than they do at OGI , so I think that i if What 's what 's great about this is it sets it up in a very systematic way , so that , once these all of these , you know , mundane but real problems get sorted out , we can just start turning the crank phd e:   professor f: and and push all of us through , and then finally figure out what 's best .  , the first thing was ,  we we actually had thought of this as sort of like ,  not not in stages , but more along the the time axis . grad c: je - je - je - je - je check out the results and and go that way . No , I 'm just saying , I 'm just thinking of it like loops , grad c:  - huh . professor f: right ? And so , y y y if you had three nested loops , that you have a choice for this , a choice for this , and a choice for that , grad c: Yeah . professor f: And , the thing is that once you get a better handle on how much you can realistically do , concurrently on different machines , different SPERTs , and so forth , and you see how long it takes on what machine and so forth , you can stand back from it and say , " OK , if we look at all these combinations we 're talking about , and combinations of combinations , and so forth , " you 'll probably find you can't do it all . professor f: OK , so then at that point , we should sort out which ones do we throw away . professor f: Which of the combinations across you know , what are the most likely ones , and And , I still think we could do a lot of them . But , probably when you include all the combinations , you 're actually talking about a thousand of them or something , and that 's probably more than we can do . And I know that , Stephane 's working from an NT machine , so his his home directory exists somewhere else . We 're over the next year or two , we 're gonna be upgrading the networks in this place , grad c:   So , it actually has reached the point where it 's a significant drag on the time for something to move the data from one place to another . professor f: So , you you don't w especially in something with repetitive computation where you 're going over it multiple times , you do don't want to have the the data that you 're working on distant from where it 's being where the computation 's being done if you can help it . Now , we are getting more disk for the central file server , which , since it 's not a computational server , would seem to be a contradiction to what I just said . But the idea is that , suppose you 're working with , this big bunch of multi multilingual databases . professor f: Then , when you 're working with something and accessing it many times , you copy the piece of it that you 're working with over to some place that 's close to where the computation is and then do all the work there . And then that way you you won't have the the network you won't be clogging the network for yourself and others . So , it 's gonna take us It may be too late for this , p precise crunch we 're in now , but , we 're ,  It 's gonna take us a couple weeks at least to get the , the amount of disk we 're gonna be getting . We 're actually gonna get , I think four more , thirty - six gigabyte drives and , put them on another another disk rack . We ran out of space on the disk rack that we had , so we 're getting another disk rack and four more drives to share between ,  primarily between this project and the Meetings Meetings Project . But , we 've put another I guess there 's another eighteen gigabytes that 's that 's in there now to help us with the immediate crunch . But , are you saying So I don't know where you 're Stephane , where you 're doing your computations . If i so , you 're on an NT machine , so you 're using some external machine phd g: Yeah , it ,  Well , to It 's Nutmeg and Mustard , I think ,  professor f: Do you know these yet ? phd g: I don't know what kind .  , are these are these , computational servers , or something ? I 'm I 've been kind of out of it . professor f: Unfortunately , these days my idea of running comput of computa doing computation is running a spread sheet . Yeah , I 'm not sure what 's available on is it you said Nutmeg and what was the other one ? phd g: Mustard . Yeah , so basically , Chuck will be the one who will be sorting out what disk needs to be where , and so on , and I 'll be the one who says , " OK , spend the money . Which , n these days , if you 're talking about scratch space , it doesn't increase the , need for backup , and , I think it 's not that big a d and the the disks themselves are not that expensive . Right now it 's phd a: What you can do , when you 're on that machine , is , just go to the slash - scratch directory , and do a DF minus K , and it 'll tell you if there 's space available . phd a: and if there is then ,  professor f: But wasn't it ,  I think Dave was saying that he preferred that people didn't put stuff in slash - scratch . It 's more putting in d s XA or XB or , phd a: Well , there 's different there , there 's professor f: right ? phd a: Right . So there 's the slash - X - whatever disks , and then there 's slash - scratch . And if it 's called " slash - scratch " , it means it 's probably an internal disk to the machine . And so that 's the kind of thing where , like if  , OK , if you don't have an NT , but you have a a a Unix workstation , and they attach an external disk , it 'll be called " slash - X - something "  , if it 's not backed up and it 'll be " slash - D - something " if it is backed up . And if it 's inside the machine on the desk , it 's called " slash - scratch " . It 's easy to unhook the external disks , put them back on the new machine , but then your slash - scratch is gone . So , you don't wanna put anything in slash - scratch that you wanna keep around for a long period of time . But if it 's a copy of , say , some data that 's on a server , you can put it on slash - scratch because , first of all it 's not backed up , and second it doesn't matter if that machine disappears and you get a new machine because you just recopy it to slash - scratch . So tha that 's why I was saying you could check slash - scratch on those on on , Mustard and and Nutmeg to see if if there 's space that you could use there . Yeah , and we do have  , yeah , so so you yeah , it 's better to have things local if you 're gonna run over them lots of times so you don't have to go to the network . professor f: Right , so es so especially if you 're right , if you 're if you 're taking some piece of the training corpus , which usually resides in where Chuck is putting it all on the on the , file server , then , yeah , it 's fine if it 's not backed up because if it g g gets wiped out or something , y  it is backed up on the other disk . phd a: Yeah , so , one of the things that I need to I 've started looking at  , is this the appropriate time to talk about the disk space stuff ? professor f: Sure . Dan David , put a new , drive onto Abbott , that 's an X disk , which means it 's not backed up . So , I 've been going through and copying data that is , you know , some kind of corpus stuff usually , that that we 've got on a CD - ROM or something , onto that new disk to free up space on other disks . We haven't deleted them off of the slash - DC disk that they 're on right now in Abbott , but we I would like to go through sit down with you about some of these other ones and see if we can move them onto , this new disk also . So , anything that that you don't need backed up , we can put on this new disk .  , but if it 's experiments and you 're creating files and things that you 're gonna need , you probably wanna have those on a disk that 's backed up , just in case something goes wrong .  So far I 've I 've copied a couple of things , but I haven't deleted anything off of the old disk to make room yet . So I I guess I 'll need to get together with you and see what data we can move onto the new disk . professor f: yeah , I I just an another question occurred to me is is what were you folks planning to do about normalization ? phd g:  . So that this could be another dimension , but we think perhaps we can use the the best , normalization scheme as OGI is using , so , with parameters that they use there ,  professor f: Yeah , I think that 's a good idea . phd g: u u professor f:  it 's i i we we seem to have enough dimensions as it is . professor f: probably the on - line line normalization because then it it 's if we do anything else , we 're gonna end up having to do on - line normalization too , so we may as well just do on - line normalization . So , I guess , yeah , th the other topic I maybe we 're already there , or almost there , is goals for the for next week 's meeting . i i i it seems to me that we wanna do is flush out what you put on the board here . Like a s like a slide ? professor f: so w we can say what we 're doing , grad c: OK . And , also , if you have sorted out , this information about how long i roughly how long it takes to do on what and , you know , what we can how many of these trainings , and testings and so forth that we can realistically do , then one of the big goals of going there next week would be to to actually settle on which of them we 're gonna do . Anything else that I a a Actually started out this this field trip started off with with , Stephane talking to Hynek , so you may have you may have had other goals , for going up , and any anything else you can think of would be we should think about accomplishing ?  , I 'm just saying this because maybe there 's things we need to do in preparation . And  and the other the the last topic I had here was ,  d Dave 's fine offer to to , do something on this .  he 's doing he 's working on other things , but to to do something on this project . So the question is , " Where where could we , most use Dave 's help ? " phd g: yeah , I was thinking perhaps if , additionally to all these experiments , which is not really research , well  it 's , running programs professor f: Yeah . phd g: and , trying to have a closer look at the perhaps the , speech , noise detection or , voiced - sound - unvoiced - sound detection and Which could be important in i for noise noise phd a: I think that would be a I think that 's a big big deal . Because the you know , the thing that Sunil was talking about , with the labels , labeling the database when it got to the noisy stuff ? The That that really throws things off . You know , having the noise all of a sudden , your your , speech detector ,  the the ,  What was it ? What was happening with his thing ? professor f:  phd a: He was running through these models very quickly . professor f: The only problem  , maybe that 's the right thing the only problem I have with it is exactly the same reason why you thought it 'd be a good thing to do . But I think the first responsibility is sort of to figure out if there 's something that , an an additional  , that 's a good thing you remove the mike . professor f: over years , if he 's if he 's interested in , you know , voiced - unvoiced - silence , he could do a lot . But if there if in fact there 's something else that he could be doing , that would help us when we 're we 're sort of  strapped for time We have we we 've , you know , only , another another month or two to you know , with the holidays in the middle of it , to to get a lot done . If we can think of something some piece of this that 's going to be The very fact that it is sort of just work , and i and it 's running programs and so forth , is exactly why it 's possible that it some piece of could be handed to someone to do , because it 's not  , yeah , so that that 's the question . And we don't have to solve it right this s second , but if we could think of some some piece that 's that 's well defined , that he could help with , he 's expressing a will willingness to do that . phd e: Yes , maybe to , mmm , put together the the label the labels between TIMIT and Spanish or something like that . professor f: So what we were just saying is that that ,  I was arguing for , if possible , coming up with something that that really was development and wasn't research because we we 're we have a time crunch . And so , if there 's something that would would save some time that someone else could do on some other piece , then we should think of that first . See the thing with voiced - unvoiced - silence is I really think that that it 's to do to do a a a a poor job is is pretty quick , or , you know , a so - so job . You can you can you can throw in a couple fea we know what what kinds of features help with it . But I remember , in fact , when you were working on that , and you worked on for few months , as I recall , and you got to , say ninety - three percent , and getting to ninety - four really really hard . So ,  And th th the other tricky thing is , since we are , even though we 're not we don't have a strict prohibition on memory size , and and computational complexity , clearly there 's some limitation to it . So if we have to if we say we have to have a pitch detector , say , if we if we 're trying to incorporate pitch information , or at least some kind of harmonic harmonicity , or something , this is another whole thing , take a while to develop .  , one I think one of the a lot of people would say , and I think Dan would also , that one of the things wrong with current speech recognition is that we we really do throw away all the harmonicity information . Reason for doing that is that most of the information about the phonetic identity is in the spectral envelopes are not in the harmonic detail . So wh that so the the other suggestion that just came up was , well what about having him work on the , multilingual super f superset kind of thing .  , coming up with that and then , you know , training it training a net on that , say , from from ,  from TIMIT or something . What what would you what would you think it would wh what would this task consist of ? phd g: Yeah , it would consist in , well , creating the the superset , and , modifying the lab labels for matching the superset . professor f: creating a superset from looking at the multiple languages , phd g: Well , creating the mappings , actually . phd g: Yeah , yeah , with the @ @ three languages , phd e: Maybe for the other language because TIMIT have more phone . grad c: There 's ,  Carmen was talking about this SAMPA thing , and it 's , it 's an effort by linguists to come up with , a machine readable IPA , sort of thing , right ? And , they they have a web site that Stephane was showing us that has ,  has all the English phonemes and their SAMPA correspondent , phoneme , professor f: Yeah . grad c: and then , they have Spanish , they have German , they have all all sorts of languages , mapping mapping to the SAMPA phonemes , which phd e: Yeah , the tr the transcription , though , for Albayzin is n the transcription are of SAMPA the same , how you say , symbol that SAMPA appear . phd b: I was gonna say , does that mean IPA is not really international ? grad c: No , it 's it 's saying phd a: It uses special diacritics and stuff , which you can't do with ASCII characters . professor f: What ,  Has OGI done anything about this issue ? Do they have Do they have any kind of superset that they already have ? phd g: I don't think so . Well , they they they 're going actually the the other way , defining  , phoneme clusters , apparently . phd a: So they just throw the speech from all different languages together , then cluster it into sixty or fifty or whatever clusters ? phd g: I think they 've not done it , doing , multiple language yet , but what they did is to training , English nets with all the phonemes , and then training it in English nets with , kind of seventeen , I think it was seventeen , broad classes . But Hynek didn't add didn't have all the results when he showed me that , so , well . phd g: But professor f: Is there 's some way that we should tie into that with this . Right ?  , if if in fact that is a better thing to do , should we leverage that , rather than doing , our own . Right ? So , if i if if they s  , we have i we have the the trainings with our own categories . And now we 're saying , " Well , how do we handle cross - language ? " And one way is to come up with a superset , but they are als they 're trying coming up with clustered , and do we think there 's something wrong with that ? phd g: I think that there 's something wrong professor f: OK . What w phd g: or Well , because Well , for the moment we are testing on digits , and e i perhaps u using broad phoneme classes , it 's it 's OK for  ,  classifying the digits , but as soon as you will have more words , well , words can differ with only a single phoneme , and which could be the same , class . Although , you are not using this for the phd g: So , I 'm professor f: You 're using this for the feature generation , though , not the phd g: Yeah , but you will ask the net to put one for th th the phoneme class professor f: Yeah . phd a: So you 're saying that there may not be enough information coming out of the net to help you discriminate the words ? professor f: Yeah . phd b: Fact , most confusions are within the phone phone classes , right ? I think , Larry was saying like obstruents are only confused with other obstruents , et cetera , et cetera . grad c: So so , maybe we could look at articulatory type stuff , professor f: But that 's what I thought they were gonna grad c: right ? professor f: Did they not do that , or ? phd g: I don't think so . Well , professor f: So phd g: they were talking about , perhaps , but they d professor f: They 're talking about it , phd g: I d professor f: but that 's sort of a question whether they did phd g: w Yeah . professor f: Instead of the the the the superclass thing , which is to take So suppose y you don't really mark arti To really mark articulatory features , you really wanna look at the acoustics and and see where everything is , and we 're not gonna do that . So , the second class way of doing it is to look at the , phones that are labeled and translate them into acoustic  ,  articulatory , features . You won't really have these overlapping things and so forth , phd a: So the targets of the net are these ? professor f: but phd a: Articulatory features . phd a: But that implies that you can have more than one on at a time ? professor f: Right . And ,  I don't know if our software this if the qu versions of the Quicknet that we 're using allows for that . Do you know ? grad c: Allows for ? professor f: Multiple targets being one ? grad c: Oh , we have gotten soft targets to to work . professor f: is that we could we could , just translate instead of translating to a superset , just translate to articulatory features , some set of articulatory features and train with that . Now the fact even though it 's a smaller number , it 's still fine because you have the the , combinations . So , in fact , it has every , you know it had has has every distinction in it that you would have the other way . We could I don't know , if you had  the phone labels , you could replace them by their articulatory features and then feed in a vector with those  , things turned on based on what they 're supposed to be for each phone to see if it if you get a big win . phd a: So , if your net is gonna be outputting , a vector of basically of well , it 's gonna have probabilities , but let 's say that they were ones and zeros , then y and you know for each , I don't know if you know this for your testing data , but if you know for your test data , you know , what the string of phones is and and you have them aligned , then you can just instead of going through the net , just create the vector for each phone and feed that in to see if that data helps . Eh , eh , what made me think about this is , I was talking with Hynek and he said that there was a guy at A T - andT who spent eighteen months working on a single feature . And because they had done some cheating experiments professor f: This was the guy that we were just talking a that we saw on campus . phd a: Well , Hynek said that that , I guess before they had him work on this , they had done some experiment where if they could get that one feature right , it dramatically improved the result . phd a: So I was thinking , you know it made me think about this , that if it 'd be an interesting experiment just to see , you know , if you did get all of those right . So that 's that 's equivalent to saying that you 've got got all the phones right . professor f: Although , yeah , it would be make an interesting cheating experiment because we are using it in this funny way , phd a: Yeah . phd a: And then you also don't know what error they 've got on the HTK side . phd b: The soft training of the nets still requires the vector to sum to one , though , right ? grad c: To sum up to one . phd b: So you can't really feed it , like , two articulatory features that are on at the same time with ones cuz it 'll kind of normalize them down to one half or something like that , for instance . Is it always softmax grad c: it 's sig No , it 's actually sigmoid - X phd g: or ? Yeah . grad c: for the phd g: So if you choose sigmoid it 's o it 's OK ? grad c: You ,  professor f: Did we just run out of disk , grad c: I think I think apparently , the ,  professor f: or ? phd b: Why don't you just choose linear ? Right ? grad c: What 's that ? phd b: Linear outputs ? grad c: Linear outputs ? phd b: Isn't that what you 'll want ? grad c:  . Right , but during the training , we would train on sigmoid - X phd b: Oh , you Yeah ? grad c: and then at the end just chop off the final nonlinearity . professor f: So , we 're we 're we 're off the air , or ? About to be off the air .