We  we abandoned the lapel because they sort of were not too not too hot , not too cold , they were you know , they were  , far enough away that you got more background noise , and  and so forth phd a:  - huh . professor c: but they weren't so close that they got quite the you know , the really good No , th phd a: OK . They were not so far away that they were really good representative distant mikes , phd a:  - huh . professor c: but on the other hand they were not so close that they got rid of all the interference . On the other hand if you only had to have one mike in some ways you could argue the lapel was a good choice , precisely because it 's in the middle . professor c: There 's  , some kinds of junk that you get with these things that you don't get with the lapel  , little mouth clicks and breaths and so forth are worse with these than with the lapel , but given the choice we there seemed to be very strong opinions for  , getting rid of lapels . phd a: The mike number is professor c: So , phd f: your mike number 's written on the back of that unit there . phd f: It - it 's one less than what 's written on the back of your phd a: OK . professor c: And you should do a lot of talking so we get a lot more of your pronunciations . phd f: So what we usually do is  , we typically will have our meetings professor c: Yeah . One thing phd f: Sunil 's here for the summer ? professor c: Sunil 's here for the summer , right .  , so , one thing is to talk about a kick off meeting maybe  , and then just  , I guess  , progress reports individually , and then  , plans for where we go between now and then , pretty much . phd f: I could say a few words about  , some of the  , compute stuff that 's happening around here , so that people in the group know . professor c: Yeah ? phd f: We  So we just put in an order for about twelve new machines , to use as sort of a compute farm . And  , we ordered  , SUN - Blade - one - hundreds , and  , I 'm not sure exactly how long it 'll take for those to come in , but , in addition , we 're running So the plan for using these is , we 're running P - make and Customs here and Andreas has sort of gotten that all  , fixed up and up to speed . And he 's got a number of little utilities that make it very easy to  , run things using P - make and Customs . The simplest thing And I can send an email around or , maybe I should do an FAQ on the web site about it or something .  , professor c: How about an email that points to the FAQ , phd f: there 's a c professor c: you know what I 'm saying ? phd f: Yeah , yeah . And , if you say that and then some job that you want to execute , it will find the fastest currently available machine , and export your job to that machine , and  and run it there and it 'll duplicate your environment . So you can say " run dash command L S " , and , it 'll actually export that LS command to some machine in the institute , and  , do an LS on your current directory . So , substitute LS for whatever command you want to run , and  And that 's a simple way to get started using using this . And , so , soon , when we get all the new machines up , e then we 'll have lots more compute to use . Now th one of the nice things is that  , each machine that 's part of the P - make and Customs network has attributes associated with it .  , attributes like how much memory the machine has , what its speed is , what its operating system , and when you use something like " run command " , you can specify those attributes for your program . For example if you only want your thing to run under Linux , you can give it the Linux attribute , and then it will find the fastest available Linux machine and run it on that . You can control where your jobs go , to a certain extent , all the way down to an individual machine . If there 's already a job running , on some machine that you 're trying to select , your job will get queued up , and then when that resource , that machine becomes available , your job will get exported there . So , there 's a lot of nice features to it and it kinda helps to balance the load of the machines and  , right now Andreas and I have been the main ones using it and we 're  . professor c: So as I understand , you know , he 's using all the machines and you 're using all the machines , phd f: So . Yeah , you know , I I sort of got started using the recognizer just recently and  ,  I fired off a training job , and then I fired off a recognition job and I get this email about midnight from Andreas saying , "  , are you running two trainings simultaneously s my m my jobs are not getting run . grad g: I have I have a question about the  , parallelization ? phd f:   grad g: So , let 's say I have like , a thousand little little jobs to do ? phd f:   grad g: how do I do it with " run command " ?  do phd f: You could write a script  , which called run command on each sub - job grad g:  - huh . A thousand times ? phd f: right ? But you probably wanna be careful with that grad g: OK .  , so , you know , you should you should probably not run more than , say ten jobs yourself at any one time , just because then it would keep other people grad g: Oh , too much file transfer and stuff . phd f: Well it 's not that so much as that , you know , e with if everybody ran fifty jobs at once then it would just bring everything to a halt and , you know , people 's jobs would get delayed , so it 's sort of a sharing thing . phd f: so you should try to limit it to somet sometim some number around ten jobs at a time . So if you had a script for example that had a thousand things it needed to run , you 'd somehow need to put some logic in there if you were gonna use " run command " , to only have ten of those going at a time .  , professor c: I remember I I forget whether it was when the Rutgers or or Hopkins workshop , I remember one of the workshops I was at there were everybody was real excited cuz they got twenty - five machines and there was some kind of P - make like thing that sit sent things out . professor c: So all twenty - five people were sending things to all twenty - five machines phd f:   professor c: and and things were a lot less efficient than if you 'd just use your own machine . phd f: but  , you can also If you have that level of parallelization  , and you don't wanna have to worry about writing the logic in in a Perl script to take care of that , you can use  , P - make grad g: Just do P - make . phd f: and and you basically write a Make file that  , you know your final job depends on these one thousand things , grad g: s   phd f: and when you run P - make , on your Make file , you can give it the dash capital J and and then a number , grad g:   If you " Run command " , that I mentioned before , is doesn't know about other things that you might be running . phd f: So , it would be possible to run a hundred run jobs at once , phd d: Right . But if you use P - make , then , it knows about all the jobs that it has to run phd d:   professor c: So " run command " doesn't use P - make , or ? phd f: It uses " export " underlyingly . But , if you i It 's meant to be run one job at a time ? So you could fire off a thousand of those , and it doesn't know any one of those doesn't know about the other ones that are running . professor c: So why would one use that rather than P - make ? phd f: Well , if you have ,  Like , for example ,  if you didn't wanna write a P - make script and you just had a ,  an HTK training job that you know is gonna take  , six hours to run , and somebody 's using , the machine you typically use , you can say " run command " and your HTK thing and it 'll find another machine , the fastest currently available machine and and run your job there . professor c: Now , does it have the same sort of behavior as P - make , which is that , you know , if you run something on somebody 's machine and they come in and hit a key then it phd f: Yes . And if you specify that , in in one of your attribute lines , then it 'll go to a machine which your job won't be evicted from . phd f: But , the machines that don't have that attribute , if a job gets fired up on that , which could be somebody 's desktop machine , and and they were at lunch , professor c:   phd f: they come back from lunch and they start typing on the console , then your machine will get evicted your job will get evicted from their machine and be restarted on another machine . So which can cause you to lose time , right ? If you had a two hour job , and it got halfway through and then somebody came back to their machine and it got evicted . If you don't want your job to run on a machine where it could be evicted , then you give it the minus the attribute , you know , " no evict " , and it 'll pick a machine that it can't be evicted from . professor c: what what about I remember always used to be an issue , maybe it 's not anymore , that if you if something required if your machine required somebody hitting a key in order to evict things that are on it so you could work , but if you were logged into it from home ? phd f:   professor c: and you weren't hitting any keys ? cuz you were , home ? phd f: Yeah , I I 'm not sure how that works . I don't know whether it monitors the keyboard or actually looks at the console TTY , so maybe if you echoed something to the you know , dev dev console or something . phd f:  ? professor c:  you sort of you 're at home and you 're trying to log in , and it takes forever to even log you in , and you probably go , " screw this " , phd f: Yeah , yeah . phd a: I need a little orientation about this environment and  scr s how to run some jobs here because I never d did anything so far with this X emissions phd f: OK . Yeah , and and also  , Stephane 's a a really good resource for that if you can't find me . Well , why don't we  , Sunil since you 're haven't haven't been at one of these yet , why don't yo you tell us what 's what 's up with you ? Wh - what you 've been up to , hopefully . phd a: yeah , after the submission the what I 've been working on mainly was to take take other s submissions and then over their system , what they submitted , because we didn't have any speech enhancement system in in ours . And then I found that  , if if I combine it with LDA , it gives @ @ improvement over theirs . I just plug in I just take the cepstral coefficients coming from their system and then plug in LDA on top of that . phd a: What I did was I took the LDA filter 's design using clean speech , mainly because the speech is already cleaned up after the enhancement so , instead of using this , narrow narrow band LDA filter that we submitted  , I got new filters . And  , so then after after that I I added  , on - line normalization also on top of that . And that there there also I n I found that I have to make some changes to their time constant that I used because th it has a a mean and variance update time constant and which is not suitable for the enhanced speech , and whatever we try it on with proposal - one . But  , I didn't I didn't play with that time constant a lot , I just t g I just found that I have to reduce the value  , I have to increase the time constant , or reduce the value of the update value . And  , the other other thing what I tried was , I just  , took the baseline and then ran it with the endpoint inf  th information , just the Aurora baseline , to see that how much the baseline itself improves by just supplying the information of the  the w speech and nonspeech . And  , I found that the baseline itself improves by twenty - two percent by just giving the wuh . professor c: can you back up a second , I I I missed something , I guess my mind wandered . Ad - ad When you added the on - line normalization and so forth ,  things got better again ? phd a: Yeah . phd a: With the different time constant I found that  , I didn't get an improvement over not using on - line normalization , professor c: Oh . phd a: But I didn't play it with play play quite a bit to make it better than . The the other thing what I tried was the adding the  , endpoint information to the baseline and that itself gives like twenty - two percent because the the second the new phase is going to be with the endpointed speech . And just to get a feel of how much the baseline itself is going to change by adding this endpoint information , I just , use professor c:  . phd f: So people won't even have to worry about , doing speech - nonspeech then . professor c: G I guess the issue is that people do that anyway , phd f: I see . professor c: and they wanted to see , given that you 're doing that , what what are the best features that you should use . professor c: But but it might be  In some ways it might be better t to rather than giving the endpoints , to have a standard that everybody uses and then interacts with . phd f: So , are people supposed to assume that there is  Are are people not supposed to use any speech outside of those endpoints ? phd a:  phd f: Or can you then use speech outside of it for estimating background noise and things ? phd a: No . Like y you will you will You 'll be given the information about the beginning and the end of speech but the whole speech is available to you . professor c: So it should make the spectral subtraction style things work even better , phd a: Yeah . I found that in s one of the SpeechDat - Car cases , that like , the Spanish one improves by just fifty percent by just putting the endpoint . phd a: Yeah , so professor c: But but phd a: so that is when  , the the qualification criteria was reduced from fifty percent to something like twenty - five percent for well - matched . And  , Yeah , I guess after that , I just went home f I just had a vacation fo for four weeks . phd a: Ye Yeah , and I I came back and I started working on  , some other speech enhancement algorithm .  , so I from the submission what I found that people have tried spectral subtraction and Wiener filtering . phd a: so just to just to fill the space with some f few more speech enhancement algorithms to see whether it improves a lot , I I 've been working on this  , signal subspace approach for speech enhancement where you take the noisy signal and then decomposing the signal s and the noise subspace and then try to estimate the clean speech from the signal plus noise subspace . phd a: So , I 've been actually running some s So far I 've been trying it only on Matlab . phd a: and then I 'll p port it to C and I 'll update it with the repository once I find it it giving any some positive result . professor c: S So you s you So you said one thing I want to jump on for a second . So so now you 're you 're getting tuned into the repository thing that he has here phd a: Yeah . Cuz you did some stuff that you talked about last week , I guess ? phd d:   professor c: where you were also combining something both of you I guess were both combining something from the  , French Telecom system with the u  phd d: Right . professor c: I I don't know whether it was system one or system two , or ? phd d:   phd d: we The main thing that we did is just to take the spectral subtraction from the France Telecom , which provide us some speech samples that are  , with noise removed . So then , one distinction is that  , you were taking the actual France Telecom features and then applying something to phd a: no there is a slight different .   , which are extracted at the handset because they had another back - end blind equalization professor c: Yeah . professor c: What I meant was you had something like cepstra or something , right ? phd a: Yeah , yeah , yeah , yeah . But I guess it 's the s exactly the same thing because on the heads  , handset they just applied this Wiener filter and then compute cepstral features , phd a: Yeah , the cepstral f The difference is like There may be a slight difference in the way phd d: right ? or ? phd a: because they use exactly the baseline system for converting the cepstrum once you have the speech . But  , I th phd d: Yeah , well I think we should  , have a table with all the result because I don't know I  , I don't exactly know what are your results ? But , phd a: OK . Yeah , but so we did this , and another difference I guess is that we just applied  , proposal - one system after this without well , with our modification to reduce the delay of the the LDA filters , phd a:  - huh . phd d: and phd b: And the filter phd d: Well there are slight modifications , but it was the full proposal - one . In your case , if you tried just putting LDA , then maybe on - line normalization ? phd a: Only LDA . So we just tried directly to to just , keep the system as it was and , when we plug the spectral subtraction it improves  , signif significantly .  , but , what seems clear also is that we have to retune the time constants of the on - line normalization . phd d: Because if we keep the value that was submitted  , it doesn't help at all . But , you can still find some kind of optimum somewhere , and we don't know where exactly phd a: Yeah . professor c: So it sounds like you should look at some tables of results or something phd d: Right . professor c: and see where i where the where they were different and what we can learn from it . phd d: because we change it the system to have phd a: Oh yeah ,  the the new LDA filters . There are other things that we finally were shown to improve also like , the sixty - four hertz cut - off . phd d: And , right now if we look at the results , it 's , always better than it seems always better than France Telecom for mismatch and high - mismatch . But , the problem is that it 's not significant , but if you put this in the , mmm , spreadsheet , it 's still worse . I don't think it 's importa important because when they will change their metric , mainly because of  , when you p you plug the  , frame dropping in the baseline system , it will improve a lot HM , and  , phd a: Yeah . phd a: Because the your improvement on HM and  will also go down significantly in the spreadsheet so . phd a:  the well - matched may be the one which is least affected by adding the endpoint information . But they d the everything  is like , but there that 's how they reduce why they reduce the qualification to twenty - five percent or some something on . professor c: But are they changing the weighting ? phd a: no , I guess they are going ahead with the same weighting . professor c: I guess I I haven't been part of the discussion , so , it seems to me that the well - matched condition is gonna be unusual , phd a: Usual . professor c: Because , you don't actually have good matches ordinarily for what any @ @ particular person 's car is like , or phd a: Mmm . phd a: Yeah , but actually the well well the well - matched  ,  the the well - matched condition is not like , the one in TI - digits where  , you have all the training , conditions exactly like replicated in the testing condition also . The well - matched has also some some mismatch in that which is other than the professor c: The well wa matched has mismatch ? phd a: has has also some slight mismatches , unlike the TI - digits where it 's like prefectly matched phd f: Perfect to match . So remind me of what well - matched meant ? phd a: The the well - matched is like professor c: You 've told me many times . phd a: the the well - matched is defined like it 's seventy percent of the whole database is used for training and thirty percent for testing . phd a: It 's it 's phd d: Because it phd a: OK , it 's professor c: Yeah . So , yeah , unless they deliberately chose it to be different , which they didn't because they want it to be well - matched , it is pretty much You know , so it 's so it 's sort of saying if you phd f: It 's it 's not guaranteed though . phd a: the main mismatch is coming from the amount of noise and the silence frames and all those present in the database actually . professor c: Again , if you have enough if you have enough phd a: No yeah , yeah . professor c: So it 's sort of i i it 's sort of saying OK , so you much as you train your dictation machine for talking into your computer , you you have a car , and so you drive it around a bunch and and record noise conditions , or something , and then I don't think that 's very realistic ,  I th phd a:   professor c: I I you know , so I I I you know , I guess they 're saying that if you were a company that was selling the stuff commercially , that you would have a bunch of people driving around in a bunch of cars , and and you would have something that was roughly similar and maybe that 's the argument , but I 'm not sure I buy it , so . We are playing we are also playing , trying to put other spectral subtraction mmm , in the code .  , it would be a very simple spectral subtraction , on the  , mel energies which I already tested but without the  frame dropping actually , and I think it 's important to have frame dropping if you use spectral subtraction . phd f: Is it is spectral subtraction typically done on the after the mel , scaling or is it done on the FFT bins ? phd d: phd f: Does it matter , or ? phd d: I d I don't know . So - some of the proposal , we 're doing this on the bin on the FFT bins , phd f:  . You can do both , but I cannot tell you what 's which one might be better or I phd f:  . phd a: I guess if you want to reconstruct the speech , it may be a good idea to do it on FFT bins .  it may not be very different if you do it on mel warped or whether you do it on FFT . phd d: Well , it gives something different , but I don't know what are the , pros and cons of both . phd a: The other thing is like when you 're putting in a speech enhancement technique , is it like one stage speech enhancement ? Because everybody seems to have a mod two stages of speech enhancement in all the proposals , which is really giving them some improvement . phd a: And So , there 's something that is good about doing it  , to cleaning it up once more . phd a: so we can phd d: So maybe in my implementation I should also try to inspire me from this kind of thing phd a: Yeah . That 's what professor c: Well , the other thing would be to combine what you 're doing . professor c:  maybe one or one or the other of the things that you 're doing would benefit from the other happening first . So , professor c: Right , so he 's doing a signal subspace thing , maybe it would work better if you 'd already done some simple spectral subtraction , or maybe vi maybe the other way around , phd d: Yeah ,   professor c: you know ? phd a: So I 've been thinking about combining the Wiener filtering with signal subspace , phd d:   phd a:  just to see all some some such permutation combination to see whether it really helps or not . professor c: How is it I I guess I 'm ignorant about this , how does  , since Wiener filter also assumes that you 're that you 're adding together the two signals , how is how is that differ from signal subspace ? phd a: The signal subspace ? The professor c: Yeah . phd a: So , the the different the c the the advantage of combining two things is mainly coming from the signal subspace approach doesn't work very well if the SNR is very bad . So essentially you could do simple spectral subtraction , followed by a KL transform , followed by a phd a: Wiener filtering . Yeah , in general , you don't that 's right you don't wanna othorg orthogonalize if the things are noisy .  , that was something that  , Herve and I were talking about with  , the multi - band stuff , that if you 're converting things to from  , bands , groups of bands into cepstral coef you know , local sort of local cepstral coefficients that it 's not that great to do it if it 's noisy . phd a: So that that 's one reason maybe we could combine s some something to improve SNR a little bit , first stage , professor c: Yeah . phd d: What was your point about about colored noise there ? phd a: Oh , the colored noise  phd d: Yeah . phd a: the colored noise the the v the signal subspace approach has  , it it actually depends on inverting the matrices . phd a:  it has a it 's It doesn't behave very well if it is not positive definite ak It works very well with white noise because we know for sure that it has a positive definite . phd a: So the way they get around is like they do an inverse filtering , first of the colo colored noise professor c: Yeah . phd a: and then finally when you reconstruct the speech back , you do this filtering again .  if you sort of you do the s spectral subtraction , that also gets rid phd a: Yeah . professor c: and then you then then add a little bit l noise noise addition  , that sort of what J JRASTA does , in a way . professor c: If you look at what JRASTA doing essentially i i it 's equivalent to sort of adding a little adding a little noise , phd a: Huh ?  - huh . And maybe we well we find some people so that  , agree to maybe work with us , and they have implementation of VTS techniques so it 's  , Vector Taylor Series that are used to mmm ,  f to model the transformation between clean cepstra and noisy cepstra . Well , if you take the standard model of channel plus noise , it 's it 's a nonlinear eh  , transformation in the cepstral domain . phd d: And  , there is a way to approximate this using  , first - order or second - order Taylor Series and it can be used for  , getting rid of the noise and the channel effect . professor c: Who is doing this ? phd d:  w working in the cepstral domain ? So there is one guy in Grenada , phd b: Yeah , in Grenada one of my friend . professor c: Who 's the guy in Grenada ? phd d: phd b: Jose Carlos Segura . professor c: Yeah , so at any rate , you 're looking general , standing back from it , looking at ways to combine one form or another of  , noise removal , with with these other things we have , phd d:   But for sure there 's required to that requires to re - check everything else , and re - optimize the other things professor c: Oh yeah .  , professor c: Well one of the seems like one of the things to go through next week when Hari 's here , phd d: I professor c: cuz Hari 'll have his own ideas too or I guess not next week , phd d:  - huh . professor c: week and a half , will be sort of go through these alternatives , what we 've seen so far , and come up with some game plans .  one would be , you look at a few things very quickly , you pick on something that looks like it 's promising and then everybody works really hard on the same different aspects of the same thing . Another thing would be to have t to to pick two pol two plausible things , and and you know , have t sort of two working things for a while until we figure out what 's better , phd d:   professor c: and then , you know , but , w  , he 'll have some ideas on that too . phd a: The other thing is to ,  Most of the speech enhancement techniques have reported results on small vocabulary tasks . But we we going to address this Wall Street Journal in our next stage , which is also going to be a noisy task so s very few people have reported something on using some continuous speech at all . So , there are some  , I was looking at some literature on speech enhancement applied to large vocabulary tasks and spectral subtraction doesn't seems to be the thing to do for large vocabulary tasks . And it 's Always people have shown improvement with Wiener filtering and maybe subspace approach over spectral subtraction everywhere . But if we if we have to use simple spectral subtraction , we may have to do some optimization to make it work @ @ . professor c: So they 're making there Somebody 's generating Wall Street Journal with additive artificially added noise or something ? phd a: Yeah , yeah . professor c: Sort of a sort of like what they did with TI - digits , and ? phd a: Yeah . professor c: And then they 're they 're  , generating HTK scripts to phd a: Yeah . There are they have there is no I don't know if they are converging on HTK or are using some Mississippi State , professor c: Mis - Mississippi State maybe , phd a: yeah . professor c: well we 've Yeah , it 's true for the additive noise , y artificially added noise we 've always used small vocabulary too . But for n there 's been noisy speech this larv large vocabulary that we 've worked with in Broadcast News . professor c: and some of the focus conditions were noisy and and phd a: It had additive n professor c: But we but we didn't do spectral subtraction . We were doing our funny stuff , right ? We were doing multi multi  , multi - stream and and so forth . professor c: and and  , that we have  , for the  , the quote - unquote noisy data there is just noisy and reverberant actually . And that 's what most o again , most of our work has been done with that , with with  , connected digits . professor c: but  , we have recognition now with some of the continuous speech , large vocabulary continuous speech , using Switchboard  , Switchboard recognizer , phd a: Yeah . You just take the Switchboard trained ? Yeah , professor c: That 's that 's what we 're doing , phd a: yeah . professor c: but we 're hop  , actually  , Dave and I were just talking earlier today about maybe at some point not that distant future , trying some of the techniques that we 've talked about on , some of the large vocabulary data .  , I guess no one had done yet done test one on the distant mike using  , the SRI recognizer and , phd f: I don't not that I know of . professor c: You 'll see a little smoke coming up from the the CPU or something trying to trying to do it , phd f: That 's right professor c: but  , yeah . But , you 're right that that that 's a real good point , that  , we we don't know yeah , what if any of these ta I guess that 's why they 're pushing that in the  in the evaluation . Anything else going on ? at you guys ' end , phd b: I don't have good result , with the inc including the new parameters , professor c: or ? phd b: I don't have good result . phd a: With what what other new p new parameter ? grad g: You 're talking about your voicing ? professor c: Yeah . professor c: seeing as how Sunil , phd b: I tried to include another new parameter to the traditional parameter , professor c: yeah . phd b: that , like , the auto - correlation , the R - zero and R - one over R - zero phd a:   phd b: and another estimation of the var the variance of the difference for of the spec si  , spectrum of the signal and and the spectrum of time after filt mel filter bank . phd b: and you have the on the other side you have the output of the mel filter bank . phd b: You can extend the coefficient of the mel filter bank and obtain an approximation of the spectrum of the signal . phd b: because , suppose we we think that if the variance is high , maybe you have n  , noise . phd b: To to To The idea is to found another feature for discriminate between voice sound and unvoice sound . And I did experiment I need to change to obtain this new feature I need to change the size the window size size . phd b: And I do I did two type of experiment to include this feature directly with the with the other feature and to train a neural network to select it voice - unvoice - silence silence phd a: Unvoiced . It 's neve e e sometime it 's worse , sometime it 's a little bit better , but not significantly . phd a: is it with TI - digits , or with ? phd b: And No , I work with eh , Italian and Spanish basically . phd b: And if I don't y use the neural network , and use directly the feature the results are worse . professor c:  we 've had these discussions before , and and one of the things that struck me was that  , about this line of thought that was particularly interesting to me was that we  whenever you condense things , in an irreversible way , you throw away some information . And , that 's mostly viewed on as a good thing , in the way we use it , because we wanna suppress things that will cause variability for  particular , phonetic units . And so the question is , can we figure out if there 's something we 've thrown away that we shouldn't have . So , when they were looking at the difference between the filter bank and the FFT that was going into the filter bank , I was thinking " oh , OK , so they 're picking on something they 're looking on it to figure out noise , or voice voiced property whatever . But for me sort of the interesting thing was , " well , but is there just something in that difference which is useful ? " So another way of doing it , maybe , would be just to take the FFT  , power spectrum , and feed it into a neural network , phd b: To know professor c: and then use it , you know , in combination , or alone , or or whatever phd f: Wi - with what targets ? phd a: Voiced , unvoiced is like professor c: no . professor c: No the just the same same way we 're using  , the same way that we 're using the filter bank . And , you know , maybe if it 's used in combination , it will get at something that we 're missing . And maybe , you know , using , orth you know , KLT , or  , adding probabilities , all th all the different ways that we 've been playing with , that we would let the essentially let the neural network determine what is it that 's useful , that we 're missing here . professor c: Well , that 's probably why y i it would be unlikely to work as well by itself , but it might help in combination . professor c: But I I I have to tell you , I can't remember the conference , but , I think it 's about ten years ago , I remember going to one of the speech conferences and and  , I saw within very short distance of one another a couple different posters that showed about the wonders of some auditory inspired front - end or something , and a couple posters away it was somebody who compared one to  , just putting in the FFT and the FFT did slightly better . professor c: but again we have these wonderful statistical mechanisms for quantifying that a that variability , and you know , doing something reasonable with it . professor c: So , It - it 's same , you know , argument that 's gone both ways about  , you know , we have these data driven filters , in LDA , and on the other hand , if it 's data driven it means it 's driven by things that have lots of variability , and that are necessarily not necessarily gonna be the same in training and test , so , in some ways it 's good to have data driven things , and in some ways it 's bad to have data driven things . So , phd a: Yeah , d professor c: part of what we 're discovering , is ways to combine things that are data driven than are not . professor c: so anyway , it 's just a thought , that that if we if we had that maybe it 's just a baseline  , which would show us " well , what are we really getting out of the filters " , or maybe i i probably not by itself , but in combination , phd d:   professor c: you know , maybe there 's something to be gained from it , and let the But , you know , y you 've only worked with us for a short time , maybe in a year or two you w you will actually come up with the right set of things to extract from this information . phd a: What one one  p one thing is like what before we started using this VAD in this Aurora , the th what we did was like , I I guess most of you know about this , adding this additional speech - silence bit to the cepstrum and training the  on that . phd a: That is just a binary feature and that seems to be improving a lot on the SpeechDat - Car where there is a lot of noise but not much on the TI - digits . So , a adding an additional feature to distin to discriminate between speech and nonspeech was helping . phd d: Wait I I 'm sorry ? phd a: Yeah , we actually added an additional binary feature to the cepstrum , just the baseline . Well , in in the case of TI - digits it didn't actually give us anything , because there wasn't any f anything to discriminate between speech , phd d: Yeah . But anyway the question is even more , is within speech , can we get some features ? Are we drop dropping information that can might be useful within speech , phd a: OK . professor c: And it 's particularly more relevant now since we 're gonna be given the endpoints . phd a: There was a paper in ICASSP this ICASSP over the  extracting some higher - order  , information from the cepstral coefficients and I forgot the name . Some is some harmonics I don't know , I can I can pull that paper out from ICASSP . phd a: It wa it was taking the ,  It was about finding the higher - order moments of Yeah . professor c: Yeah , phd a: And I 'm not sure about whether it is the higher - order moments , or professor c: cumulants , yeah . professor c: Yeah , but again You could argue that th that 's exactly what the neural network does . professor c: So n neural network  , is in some sense equivalent to computing , you know , higher - order moments of what you phd a: trying to f to Moments , yeah . I can I can just  , share a little bit Sunil hasn't hasn't heard about  , what I 've been doing . grad g: so , I told you I was I was I was getting prepared to take this qualifier exam . So basically that 's just , trying to propose  , your next your your following years of of your PHD work , trying trying to find a project to to define and and to work on . So , I 've been , looking into , doing something about r  , speech recognition using acoustic events . So , the idea is you have all these these different events , for example voicing , nasality , R - coloring , you know burst or noise , frication , that kinda stuff , building robust  , primary detectors for these acoustic events , and using the outputs of these robust detectors to do speech recognition .  , and , these these primary detectors , will be , inspired by , you know , multi - band techniques , doing things , similar to Larry Saul 's work on , graphical models to to detect these these , acoustic events . And , so I I been I been thinking about that and some of the issues that I 've been running into are , exactly what what kind of acoustic events I need , what  , what acoustic events will provide a a good enough coverage to in order to do the later recognition steps . And , also , once I decide a set of acoustic events , h how do I how do I get labels ? Training data for for these acoustic events . And , then later on down the line , I can start playing with the the models themselves , the the primary detectors .  , so , I kinda see like , after after building the primary detectors I see  , myself taking the outputs and feeding them in , sorta tandem style into into a  , Gaussian mixtures  back - end , and doing recognition . grad g: professor c: By by the way , the voiced - unvoiced version of that for instance could tie right in to what Carmen was looking at . professor c: you know , if you if a multi - band approach was helpful as as I think it is , it seems to be helpful for determining voiced - unvoiced , grad g:   And so , this this past week  , I 've been  , looking a little bit into  , TRAPS  , and doing doing TRAPS on on these e events too , just , seeing seeing if that 's possible .  , and  , other than that , I was kicked out of I - house for living there for four years . professor c: or , no ? grad g: well , s s som something like that . professor c: Suni - i d ' you v did  did you find a place ? phd a: no professor c: Is that out of the way ? phd a: not yet .  , yesterday I called up a lady who ha who will have a vacant room from May thirtieth and she said she 's interviewing two more people . And then , you 're coming back  phd a: i  , I I p I plan to be here on thirty - first . professor c: Thirty - first , phd a: Yeah , well if there 's a house available or place to professor c: OK . professor c: They 're available , and they 'll be able to get you something , so worst comes to worst we 'll put you up in a hotel for for for a while phd a: Yeah . grad e: You know , if you 're in a desperate situation and you need a place to stay , you could stay with me for a while . professor c: Do y wanna say anything about You you actually been  , last week you were doing this stuff with Pierre , you were you were mentioning . Is that that something worth talking about , or ? grad e: it 's Well , it I don't think it directly relates .  , well , so , I was helping a speech researcher named Pierre Divenyi and he 's int He wanted to  , look at  , how people respond to formant changes , I think . So he he created a lot of synthetic audio files of vowel - to - vowel transitions , and then he wanted a psycho - acoustic  , spectrum . And he wanted to look at  , how the energy is moving over time in that spectrum and compare that to the to the listener tests . And to  he he t wanted to track the peaks so he could look at how they 're moving . I found the roots of the  , LPC polynomial to , track the peaks in the , PLP LPC spectra . phd a: well there is aligned spectral pairs , is like the the Is that the aligned s professor c: It 's a r root LPC , of some sort . phd a: Oh , it 's like line sp professor c: Except I think what they call line spectral pairs they push it towards the unit circle , don't they , phd a: Yeah , yeah , yeah , yeah . But what we 'd used to do w when I did synthesis at National Semiconductor twenty years ago , the technique we were playing with initially was was taking the LPC polynomial and and  , finding the roots . It wasn't PLP cuz Hynek hadn't invented it yet , but it was just LPC , and  , we found the roots of the polynomial , And th When you do that , sometimes they 're f they 're what most people call formants , sometimes they 're not . professor c:  Formant tracking with it can be a little tricky cuz you get these funny values in in real speech , phd f: So you just You typically just get a few roots ? professor c: but . phd f: something like that ? professor c: And it depends on the order that you 're doing , but . So , if @ @ Every root that 's Since it 's a real signal , the LPC polynomial 's gonna have real coefficients . So I think that means that every root that is not a real root is gonna be a c complex pair , phd f:   So for each And if you look at that on the unit circle , one of these one of the members of the pair will be a positive frequency , one will be a negative frequency , I think . So I just So , f for the I 'm using an eighth - order polynomial and I 'll get three or four of these pairs professor c: Yeah .  for real speech in real then what you end up having is , like I say , funny little things that are don't exactly fit your notion of formants all that well . grad e: Mmm , professor c: And and what  in in what we were doing , which was not so much looking at things , it was OK phd d: I professor c: because it was just a question of quantization .  , we were just you know , storing It was We were doing , stored speech , quantization . professor c: But but  , in your case  , you know phd d: Actually you have peaks that are not at the formant 's positions , but they are lower in energy grad e: But there 's some of that , yes . phd f: If this is synthetic speech can't you just get the formants directly ?  h how is the speech created ? grad e: It was created from a synthesizer , and  phd f: Wasn't a formant synthesizer was it ? professor c: I bet it it might have may have been grad e: I d d this professor c: but maybe he didn't have control over it or something ? grad e: In in fact w we we could get , formant frequencies out of the synthesizer , as well . And , w one thing that the , LPC approach will hopefully give me in addition , is that I I might be able to find the b the bandwidths of these humps as well . grad e: but I don't think there 's a g a really good reason not to  , get the formant frequencies from the synthesizer instead . professor c: Yeah , so the actual So you 're not getting the actual formants per se . professor c: You 're getting something that is is  , af strongly affected by the PLP model . So it 's a little It 's It 's It 's sort of sort of a different thing . i Ordinarily , in a formant synthesizer , the bandwidths as well as the ban  , formant centers are phd f: Yeah . professor c: that 's Somewhere in the synthesizer that was put in , as as what you grad e:   professor c: But but yeah , you view each complex pair as essentially a second - order section , which has , band center and band width , and  ,  But . So , yeah , you 're going back today and then back in a week I guess , phd a: Yeah