professor b: So ne next week we 'll have , both Birger and , Mike Michael Michael Kleinschmidt and Birger Kollmeier will join us . professor b: and you 're you 're probably gonna go up in a couple three weeks or so ? When d when are you thinking of going up to , OGI ? phd d: Yeah , like , not next week but maybe the week after . So at least we 'll have one meeting with yo with you still around , and and phd d:  - huh . So there was this conference call this morning , and the only topic on the agenda was just to discuss a and to come at  , to get a decision about this latency problem . professor b: No , this I 'm sorry , this is a conference call between different Aurora people or just ? phd d: yeah . There were like two hours of discussions , and then suddenly , people were tired , I guess , and they decided on a number , two hundred and twenty , included e including everything . W It 's it 's p d primary primarily determined by the VAD at this point , phd d:  . We probably should do that pretty soon so that we don't get used to it being a certain way . We have , like , a system that gives sixty - two percent improvement , but if you want to stick to the this latency Well , it has a latency of two thirty , but if you want also to stick to the number of features that limit it to sixty , then we go a little bit down but it 's still sixty - one percent . And i is the tandem network , small enough that it will fit on the terminal size in terms of ? phd d: no , I don't think so . phd d: It 's still in terms of computation , if we use , like , their way of computing the the maps the the MIPs , I think it fits , professor b:   phd d: and I don't know how much this can be discussed or not , because it 's it could be in ROM , so it 's maybe not that expensive . But professor b: Ho - how much memory d ? H how many ? phd d: I d I d  , I I don't kn remember exactly , but  . I 'd like to see that , cuz maybe I could think a little bit about it , cuz we maybe we could make it a little smaller or  , it 'd be it 'd be neat if we could fit it all . professor b: But I guess it 's still within their rules to have have it on the , t  , server side .  , mmm , and one of the trick was to , use some kind of hierarchical structure where the silence probability is not computed by the final tandem network but by the VAD network .  , so apparently it looks better when , we use the silence probability from the VAD network professor b: Huh . So it 's some kind of hierarchical thing , that Sunil also tried , on SPINE and apparently it helps a little bit also . Yeah , the reason w why why we did that with the silence probability was that ,  professor b: Could ?  , I 'm I 'm really sorry . So there is the tandem network that e e e estimates the phone probabilities professor b: Yeah . phd d: And things get better when , instead of using the silence probability computed by the tandem network , we use the silence probability , given by the VAD network , professor b: Oh . phd d: professor b: The VAD network is ? phd d: Which is smaller , but maybe ,  So we have a network for the VAD which has one hundred hidden units , and the tandem network has five hundred . phd d: Maybe it 's has something to do to the fact that we don't have infinite training data and professor b: We don't ? phd d: Well ! And so Well , things are not optimal professor b: Yeah . phd d: and Mmm grad e: Are you you were going to say why what made you wh what led you to do that .  , there was a p problem that we observed , that there was there were , like , many insertions in the in the system . phd d: Actually plugging in the tandem network was increasing , I I I think , the number of insertions . professor b: So , you know , in a way what it might i it 's it 's a little bit like combining knowledge sources . phd d:  professor b: Right ? Because the fact that you have these two nets that are different sizes means they behave a little differently , phd d:   And , if you have ,  f the distribution that you have from , f speech sounds is w sort of one source of knowledge . professor b: And this is and rather than just taking one minus that to get the other , which is essentially what 's happening , you have this other source of knowledge that you 're putting in there . Well , there are other things that we should do but , it requires time and We have ideas , like so , these things are like hav having a better VAD . Of course we have ideas on this also , but w we need to try different things and  , but their noise estimation ,   professor b: back on the second stream , that 's something we 've talked about for a while . professor b: so we have this this default idea about just using some sort of purely spectral thing ? phd d: yeah . professor b: for a second stream ? phd d: But , we we did a first try with this , and it it clearly hurts . So , if you just had a second stream that was just spectral and had another neural net and combined there , that that , might be good .  Yeah , and the other thing , that noise estimation and th  , maybe try to train  , the training data for the t tandem network , right now , is like i is using the noises from the Aurora task and I think that people might , try to argue about that because then in some cases we have the same noises in for training the network than the noises that are used for testing , professor b: Right . Well , that 's something I would like to investigate further , but , I did , like ,  I did , listen to the m most noisy utterances of the SpeechDat - Car Italian and tried to transcribe them . phd d: but but still , what happens is is that , the digit error rate on this is around one percent , professor b: Yeah .  , but what happens also is that if I listen to the ,  a re - synthesized version of the speech and I re - synthesized this using a white noise that 's filtered by a LPC , filter professor b: Yeah . phd d: well , you can argue , that ,  that this is not speech , professor b: Yeah . But s actually it sound like whispering , so we are professor b: Well , it 's phd d: eh professor b: There 's two problems there .   , so so the first is that by doing LPC - twelve with synthesized speech w like you 're saying , it 's i i you 're you 're adding other degradation . professor b: Right ? So it 's not just the noise but you 're adding in fact some degradation because it 's only an approximation .  , and the second thing is which is m maybe more interesting is that , if you do it with whispered speech , you get this number . What if you had done analysis re - synthesis and taken the pitch as well ? Alright ? So now you put the pitch in . professor b: What would the percentage be then ? phd d:  professor b: See , that 's the question . So , you see , if it 's if it 's if it 's ,  Let 's say it 's back down to one percent again . professor b: That would say at least for people , having the pitch is really , really important , which would be interesting in itself . But professor b: if i on the other hand , if it stayed up near five percent , then I 'd say " boy , LPC n twelve is pretty crummy " . professor b: So I I I 'm not sure I 'm not sure how we can conclude from this anything about that our system is close to the human performance . Well , the point is that eh l ey the point is that , what I what I listened to when I re - synthesized the LP - the LPC - twelve spectrum is in a way what the system , is hearing , cuz @ @ all the all the , excitation all the well , the excitation is is not taken into account . And professor b: Well , you 're not doing the LPC phd d: in this case professor b: so so what if you did a phd d: Well , it 's not LPC , sure , professor b: What if you did LPC - twenty ? phd d: but LPC ? professor b: Twenty . professor b: So , all I 'm saying is that you have in addition to the w the , removal of pitch , you also are doing , a particular parameterization , phd d:   professor b: so , let 's see , how would you do ? So , fo phd d: But that 's that 's what we do with our systems . Actually , we d we we don't , because we do we do , mel filter bank , for instance . Right ? phd d: Yeah , but is it that is it that different ,  ? professor b: I don't know what mel , based synthesis would sound like , phd d: I professor b: but certainly the spectra are quite different . phd a: Couldn't you t couldn't you , test the human performance on just the original audio ? phd d:   So , y  , your performance was one percent , and then when you re - synthesize with LPC - twelve it went to five . professor b:  We were we were j It it it 's a little bit still apples and oranges because we are choosing these features in order to be the best for recognition . professor b: And , i if you listen to them they still might not be very Even if you made something closer to what we 're gonna i it might not sound very good . professor b: and i the degradation from that might might actually make it even harder , to understand than the LPC - twelve . So all I 'm saying is that the LPC - twelve puts in synthesis puts in some degradation that 's not what we 're used to hearing , phd d:  - huh . professor b: and is ,  It 's not it 's not just a question of how much information is there , as if you will always take maximum advantage of any information that 's presented to you . And so it it isn't phd a: But professor b: But , I agree that it says that , the kind of information that we 're feeding it is probably , a little bit , minimal . And that 's why I was saying it might be interesting if you an interesting test of this would be if you if you actually put the pitch back in . So , you just extract it from the actual speech and put it back in , and see does that is that does that make the difference ? If that if that takes it down to one percent again , then you 'd say " OK , it 's it 's in fact having , not just the spectral envelope but also the also the the pitch that , @ @ has the information that people can use , anyway . phd a: But from this it 's pretty safe to say that the system is with either two to seven percent away from the performance of a human . professor b: Well , or it 's it 's phd a: Two two to six percent . professor b: Yeah , so It 's it 's one point four times , to , seven times the error , phd d: To f seven times , yeah . phd d: But but professor b: But that 's that 's what that 's the first thing that I would be curious about , is , you know , i i when you we phd d: But the signal itself is like a mix of  , of a a periodic sound and , @ @  , unvoiced sound , and the noise professor b:   So , what what do you mean exactly by putting back the pitch in ? Because phd a: In the LPC synthesis ? I think professor b: Yeah . professor b: So ,  and you did it with a noise source , rather than with with a s periodic source . professor b: Right ? So if you actually did real re - synthesis like you do in an LPC synthesizer , where it 's unvoiced you use noise , where it 's voiced you use , periodic pulses . professor b: Right ? phd d: Yeah , but it 's neither purely voiced or purely unvoiced . professor b: Well , it might be hard to do it phd d: So professor b: but it but but the thing is that if you  , if you detect that there 's periodic s strong periodic components , then you can use a voiced voice thing . professor b: But I 'm I 'm just saying , at least as a thought experiment , that 's what I would wanna test . professor b: I wan would wanna drive it with a a a two - source system rather than a than a one - source system . professor b: And then that would tell you whether in fact it 's Cuz we 've talked about , like , this harmonic tunneling or other things that people have done based on pitch , maybe that 's really a key element . Maybe maybe , without that , it 's it 's not possible to do a whole lot better than we 're doing . Evi professor b: But , other than that , I don't think it 's  , other than the pitch de information , it 's hard to imagine that there 's a whole lot more in the signal that that ,  that we 're throwing away that 's important . professor b: Right ?  , we 're using a fair number of filters in the filter bank and  phd d:   If somebody was paying really close attention , you might get I would actually think that if , you looked at people on various times of the day and different amounts of attention , you might actually get up to three or four percent error on digits . professor b: So it 's you know , we 're not we 're not incredibly far off . On the other hand , with any of these numbers except maybe the one percent , it 's st it 's not actually usable in a commercial system with a full telephone number or something .  , while we 're still on Aurora stuff maybe you can talk a little about the status with the , Wall Street Journal things for it . They wrote some scripts that sort of make it easy to run the system on the Wall Street Journal , data .  , I 'm waiting there was one problem with part of it and I wrote a note to Joe asking him about it .  , they on their web site they , did this little table of where their system performs relative to other systems that have done this this task . So they 're professor b: This is on clean test set ? phd a: This is on clean on clean stuff . They they 've started a table where they 're showing their results on various different noise conditions but they they don't have a whole lot of it filled in and and I didn't notice until after I 'd printed it out that , they don't say here what these different testing conditions are . professor b:  phd a: You actually have to click on it on the web site to see them . professor b: What kind of numbers are they getting on these on the test conditions ? phd a: Well , see , I was a little confused because on this table , I 'm the they 're showing word error rate . But on this one , I I don't know if these are word error rates because they 're really big . professor b:  phd a: So m I guess maybe they 're error rates but they 're ,  they 're really high . professor b: I I I don't find that surpri phd a: So professor b: we W what 's what 's some of the lower error rates on on on  , some of the higher error rates on , some of these w  , highly mismatched difficult conditions ? What 's a ? phd d:  . phd a: Correct ? phd d: And the baseline , eh phd a: Accuracy ? phd d: error rate . professor b: So if you 're doing so if you 're doing , phd d: and phd a: Yeah . professor b: Yeah , and if you 're saying sixty - thousand word recognition , getting sixty percent error on some of these noise condition not at all surprising . phd d: The baseline is sixty percent also on digits , phd a: Oh , is it ? phd d: on the m more mismatched conditions . professor b: It 's a bad sign when you looking at the numbers , you can't tell whether it 's accuracy or error rate .  , they 're I I 'm still waiting for them to release the , multi - CPU version of their scripts , cuz right now their script only handles processing on a single CPU , which will take a really long time to run . But their s professor b: This is for the training ? phd a:  I beli Yes , for the training also . So , as soon as they get that , then I 'll I 'll grab those too professor b: OK . I 'll go ahead and try to run it though with just the single CPU one , professor b: if the phd a: and I they they , released like a smaller data set that you can use that only takes like sixteen hours to train and stuff . So I can I can run it on that just to make sure that the the thing works and everything . Is that about right you think ? phd d: we don't know yet , I I think . Did they say anything on the conference call about , how the Wall Street Journal part of the test was going to be run ? Because I I thought I remembered hearing that some sites were saying that they didn't have the compute to be able to run the Wall Street Journal stuff at their place , phd d: No . Well , this first , this was not the point at all of this the meeting today phd a: Oh , OK . phd d: and , professor b: Some phd d: frankly , I don't know because I d didn't read also the most recent mails about the large - vocabulary task . But , did you do you still , get the mails ? You 're not on the mailing list or what ? phd a:   I I don't get any mail about professor b: I have to say , there 's  something funny - sounding about saying that one of these big companies doesn't have enough cup compute power do that , so they 're having to have it done by Mississippi State . phd a: because there 's this whole issue about , you know , simple tuning parameters , like word insertion penalties . If you change your front - end , you know , the scale is completely can be completely different , so . But phd d: You didn't get any answer from Joe ? phd a: I did , but Joe said , you know , " what you 're saying makes sense phd d:  - huh . phd a: that 's th We had this back and forth a little bit about , you know , are sites gonna are you gonna run this data for different sites ? And , well , if if Mississippi State runs it , then maybe they 'll do a little optimization on that parameter , and ,  But then he wasn't asked to run it for anybody . phd a: he 's been putting this stuff out on their web site and for people to grab but I haven't heard too much about what 's happening . professor b: So it could be  , Chuck and I had actually talked about this a couple times , and and over some lunches , I think , that , one thing that we might wanna do The - there 's this question about , you know , what do you wanna scale ? Suppose y you can't adjust these word insertion penalties and so forth , so you have to do everything at the level of the features . What could you do ? And , one thing I had suggested at an earlier time was maybe some sort of scaling , some sort of root or or something of the , features . But the problem with that is that isn't quite the same , it occurred to me later , because what you really want to do is scale the , @ @ the range of the likelihoods rather than phd d: Nnn , the dist Yeah . professor b: But , what might get at something similar , it just occurred to me , is kind of an intermediate thing is because we do this strange thing that we do with the tandem system , at least in that system what you could do is take the , values that come out of the net , which are something like log probabilities , and scale those . But but , you know So because what we 're doing is pretty strange and complicated , we don't really know what the effect is at the other end . professor b: So , my thought was maybe  , they 're not used as probabilities , but the log probabilities we 're taking advantage of the fact that something like log probabilities has more of a Gaussian shape than Gaus - than probabilities , and so we can model them better . So , in a way we 're taking advantage of the fact that they 're probabilities , because they 're this quantity that looks kind of Gaussian when you take it 's log . And we may end up being in a situation where we just you know really can't change the word insertion penalty . But the other thing we could do is also we could  , this this may not help us , in the evaluation but it might help us in our understanding at least . We might , just run it with different insper insertion penalties , and show that , " well , OK , not changing it , playing the rules the way you wanted , we did this . " phd a: I wonder if it it might be possible to , simulate the back - end with some other system . So we we get our f front - end features , and then , as part of the process of figuring out the scaling of these features , you know , if we 're gonna take it to a root or to a power or something , we have some back - end that we attach onto our features that sort of simulates what would be happening . phd a: professor b: And just adjust it until it 's the best number ? phd a: and just adjust it until that our l version of the back - end , decides that that professor b: Well , we can probably use the real thing , can't we ? And then jus just , use it on a reduced test set or something . So  , I I think that that 's a reasonable thing to do and the only question is what 's the actual knob that we use ? phd a:   professor b: And the knob that we use should  , unfortunately , like I say , I don't know the analytic solution to this cuz what we really want to do is change the scale of the likelihoods , phd a:   grad e: Out of curiosity , what what kind of recognizer is the one from Mississippi State ? phd a: w what do you mean when you say " what kind " ? grad e: Is it ?  , is it like a Gaussian mixture model ? phd a: Yeah . phd a: It 's the same system that they use when they participate in the Hub - five evals .  , they started off with  , when they were building their system they were always comparing to HTK to make sure they were getting similar results . And so , it 's a Gaussian mixture system ,  professor b: Do they have the same sort of mix - down sort of procedure , where they start off with a small number of some things phd a: I don't know . professor b: D Do you know what kind of tying they use ? Are they they sort of some sort of a bunch of Gaussians that they share across everything ? Or or if it 's ? phd a: Yeah , th I have I I I don't have it up here but I have a the whole system description , that describes exactly what their system is professor b: OK . phd a: It 's some kind of a mixture of Gaussians and , clustering and ,  They 're they 're trying to put in sort of all of the standard features that people use nowadays . professor b: So the other , Aurora thing maybe is I I dunno if any of this is gonna come in in time to be relevant , but , we had talked about , Guenter playing around , over in Germany phd d:   professor b: and and , @ @  , possibly coming up with something that would , fit in later .  , I saw that other mail where he said that he  , it wasn't going to work for him to do CVS . professor b: So if he 'll he might work on improving the noise estimate or on some histogram things , or phd d: Yeah . I just saw the Eurospeech We we didn't talk about it at our meeting but I just saw the just read the paper . Someone , I forget the name , and and Ney , about histogram equalization ? Did you see that one ? phd d: it was a poster .  It was something similar to n on - line normalization finally  , in the idea of of normalizing professor b: Yeah . But it 's a little more it it 's a little finer , right ? So they had like ten quantiles phd d: Yeah . professor b: So you you have the distributions from the training set , phd d: N professor b: and then ,  So this is just a a histogram of of the amplitudes , I guess . And and and then , when you get a new new thing that you you want to adjust to be better in some way , you adjust it so that the histogram of the new data looks like the old data . professor b: You do this kind of piece - wise linear or , some kind of piece - wise approximation . They did a  one version that was piece - wise linear and another that had a power law thing between them between the points . And , they said they s they sort of see it in a way as s for the speech case as being kind of a generalization of spectral subtraction in a way , because , you know , in spectral subtraction you 're trying to get rid of this excess energy . And then they have s they have some kind of , a floor or something , grad e:  . phd a: So is this a histogram across different frequency bins ? professor b: and phd a: Or ? professor b: I think this i You know , I don't remember that . I  Something like one per frequency band , professor b: One phd a: So , one histogram per frequency bin . phd a: So th professor b: And I don't remember whether it was filter bank things phd a: Oh . phd d: and I didn't professor b: or whether it was FFT bins phd a: Huh . professor b: or phd a: And and that that , histogram represents the different energy levels that have been seen at that frequency ? professor b: I don't remember that . And they do they said that they could do it for the test So you don't have to change the training . phd a: So they ,  Is the idea that you you run a test utterance through some histogram generation thing and then you compare the histograms and that tells you what to do to the utterance to make it more like ? professor b: I guess in pri Yeah . professor b: whether it was some , on - line thing , or whether it was a second pass , or what . We 're sort of curious about , what are some things that are , u u  , @ @ conceptually quite different from what we 've done . professor b: Cuz we you know , one thing that w that , Stephane and Sunil seemed to find , was , you know , they could actually make a unified piece of software that handled a range of different things that people were talking about , and it was really just sort of setting of different constants . So , we 're not making any use of pitch , which again , might might be important , because the stuff between the harmonics is probably a schmutz . And and ,  And we there 's this overall idea of really sort of matching the the hi distributions somehow . So I guess , Guenter 's gonna play around with some of these things now over this next period , phd d: I dunno . professor b: or ? phd d: I don't have feedback from him , but professor b: Yeah . phd d: I guess he 's gonna , maybe professor b: Well , he 's got it anyway , so he can . professor b: So potentially if he came up with something that was useful , like a diff a better noise estimation module or something , he could ship it to you guys u up there phd d:  - huh . So , why don't we just ,  I think starting starting a w couple weeks from now , especially if you 're not gonna be around for a while , we 'll we 'll be shifting more over to some other other territory . But , n not not so much in this meeting about Aurora , but but , maybe just , quickly today about maybe you could just say a little bit about what you 've been talking about with Michael . So Michael Kleinschmidt , who 's a PHD student from Germany , showed up this week . And he 's done some work using an auditory model of , human hearing , and using that f  , to generate speech recognition features . And he did work back in Germany with , a toy recognition system using , isolated digit recognition as the task .  , and he tried that on I think on some Aurora data and got results that he thought seemed respectable . And he w he 's coming here to u u use it on a  , a real speech recognition system . And , maybe I should say a little more about these features , although I don't understand them that well . And I 'm - I 'm not sure what we have @ @ in there that isn't already modeled in something like , PLP . And then the second stage is , the most different thing , I think , from what we usually do . It 's ,  it computes features which are , based on sort of like based on diffe different w  , wavelet basis functions used to analyze the input .  So th he uses analysis functions called Gabor functions , which have a certain extent , in time and in frequency . And the idea is these are used to sample , the signal in a represented as a time - frequency representation . And , that , is is interesting , cuz , @ @ for for one thing , you could use it , in a a multi - scale way . You could have these instead of having everything like we use a twenty - five millisecond or so analysis window , typically , and that 's our time scale for features , but you could using this , basis function idea , you could have some basis functions which have a lot longer time scale and , some which have a lot shorter , and so it would be like a set of multi - scale features . So he 's interested in ,  Th - this is because it 's ,  there are these different parameters for the shape of these basis functions ,  there are a lot of different possible basis functions . And so he he actually does an optimization procedure to choose an an optimal set of basis functions out of all the possible ones . H What does he do to choose those ? grad c: The method he uses is kind of funny is , he starts with he has a set of M of them .  , he and then he uses that to classify  , he t he tries , using just M minus one of them . grad c: Whichever sub - vector , works the the best , I guess , he says the the fe feature that didn't use was the most useless feature , professor b: Y yeah . grad c: so we 'll throw it out and we 're gonna randomly select another feature from the set of possible basis functions . phd a: So it 's a professor b: So i so it 's actuall phd a: it 's a little bit like a genetic algorithm or something in a way . grad e: It 's like a greedy professor b: But it 's but it 's  , it 's there 's a lot number of things I like about it , let me just say .  , i i in truth , both pieces of this are have their analogies in stuff we already do . But it 's a different take at how to approach it and potentially one that 's m maybe a bit more systematic than what we 've done , and a b a bit more inspiration from from auditory things . The primary features , are in fact Yeah , essentially , it 's it 's , you know , PLP or or mel cepstrum , or something like that . We always have some you know , the the the kind of filter bank with a kind of quasi - log scaling .  , if you put in if you also include the RASTA in it i RASTA the filtering being done in the log domain has an AGC - like , characteristic , which , you know , people typi typically put in these kind of , auditory front - ends .  , I would agree that the second one is is somewhat more different but , it 's mainly different in that the things that we have been doing like that have been  , had a different kind of motivation and have ended up with different kinds of constraints . So , for instance , if you look at the LDA RASTA stuff , you know , basically what they do is they they look at the different eigenvectors out of the LDA and they form filters out of it . But , they 're not sort of systematically multi - scale , like " let 's start here and go to there , and go to there , and go to there " , and so forth . It 's more like , you run it on this , you do discriminant analysis , and you find out what 's helpful .  grad c: I it 's multi - scale because you use several of these in parallel , professor b: Yeah .  , but it 's also ,   Hyn - when Hynek 's had people do this kind of LDA analysis , they 've done it on frequency direction and they 've done it on the time direction . I think he may have had people sometimes doing it on both simultaneously some two - D and that would be the closest to these Gabor function kind of things . And , the other thing that 's interesting the the ,  the feature selection thing , it 's a simple method , but I kinda like it .  , eh , I remember people referring to it as old when I was playing with it twenty years ago , so I know it 's pretty old , called Stepwise Linear Discriminant Analysis in which you which I think it 's used in social sciences a lot . And what what Michael 's describing seems to me much , much better , because the problem with the stepwise discriminant analysis is that you don't know that you know , if you 've picked the right set of features . So , here at least you 're starting off with all of them , and you 're throwing out useless features .  , so the only thing is , of course , there 's this this artificial question of of , exactly how you how you a how you assess it and if if your order had been different in throwing them out .  , it still isn't necessarily really optimal , but it seems like a pretty good heuristic . professor b: And and and , the thing that I wanted to to add to it also was to have us use this in a multi - stream way . professor b: so so that , when you come up with these different things , and these different functions , you don't necessarily just put them all into one huge vector , but perhaps you have some of them in one stream and some of them in another stream , and so forth . And ,  And we 've also talked a little bit about , Shihab Shamma 's stuff , in which you the way you look at it is that there 's these different mappings and some of them emphasize , upward moving , energy and fre and frequency . But , I think we 're sorta gonna start off with what he , came here with and branch out branch out from there . grad e: As as we were talking about this I was thinking , whether there 's a relationship between  , between Michael 's approach to , some some sort of optimal brain damage or optimal brain surgeon on the neural nets . grad e: So , like , if we have ,  we have our we have our RASTA features and and presumably the neural nets are are learning some sort of a nonlinear mapping , from the the the features to to this this probability posterior space . grad e: Right ? And ,  and each of the hidden units is learning some sort of some sort of some sort of pattern . And then when you 're looking at the the , the best features , you know , you can take out you can do the do this , brain surgery by taking out , hidden units that don't really help at all . grad e: And this is k sorta like professor b: Right ? grad e: Yeah . professor b: y actually , you make me think a a very important point here is that , if we a again try to look at how is this different from what we 're already doing , there 's a a ,  a nasty argument that could be made th that it 's it 's not different at at all , because ,  if you ignore the the selection part because we are going into a a very powerful , nonlinearity that , in fact is combining over time and frequency , and is coming up with its own you know , better than Gabor functions its , you know , neural net functions , grad e:   grad c:  professor b: so you could argue that in fact it But I I don't actually believe that argument because I know that , you can ,  computing features is useful , even though in principle you haven't added anything in fact , you subtracted something , from the original waveform You know , if you 've you 've processed it in some way you 've typically lost something some information . And so , you 've lost information and yet it does better with with features than it does with the waveform . So that 's why it really seems like the constraint in in all this stuff it 's the constraints that are actually what matters . Because if it wasn't the constraints that mattered , then we would 've completely solved this problem long ago , because long ago we already knew how to put waveforms into powerful statistical mechanisms . Well , if we had infinite processing power and data , I guess , using the waveform could grad e: Right . But but , i it 's With finite of those things  , we we have done experiments where we literally have put waveforms in and and and , phd d:   professor b: we kept the number of parameters the same and so forth , and it used a lot of training data . And it and it it ,  not infinite but a lot , and then compared to the number parameters and it it ,  it just doesn't do nearly as well . professor b: it 's not just having the maximum information , you want to suppress , the aspects of the input signal that are not helpful for for the discrimination you 're trying to make . So maybe just briefly ,  grad e: Well , that sort of segues into what what I 'm doing . grad e: so , the big picture is k  , come up with a set of , intermediate categories , then build intermediate category classifiers , then do recognition , and , improve speech recognition in that way .  , so right now I 'm in in the phase where I 'm looking at at , deciding on a initial set of intermediate categories . And I 'm looking for data data - driven methods that can help me find , a set of intermediate categories of speech that , will help me to discriminate later down the line . And one of the ideas , that was to take a take a neural net train train an ordinary neural net to  , to learn the posterior probabilities of phones . And so , at the end of the day you have this neural net and it has hidden hidden units .  , and I 'm gonna to try to to look at those patterns to to see , from those patterns  , presumably those are important patterns for discriminating between phone classes . And maybe maybe some , intermediate categories can come from just looking at the patterns of  , that the neural net learns . professor b: Be - before you get on the next part l let me just point out that s there 's there 's a a pretty nice relationship between what you 're talking about doing and what you 're talking about doing there . professor b: So , it seems to me that , you know , if you take away the the the difference of this primary features , and , say , you use as we had talked about maybe doing you use P - RASTA - PLP or something for the the primary features , then this feature discovery , thing is just what he 's talking about doing , too , except that he 's talking about doing them in order to discover intermediate categories that correspond to these  , what these sub - features are are are are showing you . And , the other difference is that , he 's doing this in a in a multi - band setting , which means that he 's constraining himself to look across time in some f relatively limited , spectral extent . Right ? And whereas in in this case you 're saying " let 's just do it unconstrained " . So they 're they 're really pretty related and maybe they 'll be at some point where we 'll see the the connections a little better and connect them . Yeah , so so that 's the that 's the first part  , one one of the ideas to get at some some patterns of intermediate categories .  , the other one was , to , come up with a a a model  , a graphical model , that treats the intermediate categories as hidden hidden variables , latent variables , that we don't know anything about , but that through , s statistical training and the EM algorithm , at the end of the day , we have ,  we have learned something about these these latent ,  latent variables which happen to correspond to intermediate categories . Yeah , and so those are the the two directions that I 'm I 'm looking into right now . Should we do our digits and get ou get our treats ? grad e: Oh , tea time ? professor b: Yeah . It 's kind of like , you know , the little rats with the little thing dropping down to them . phd a: That 's ri professor b: We do the digits and then we get our treats