Professor D: I mean one of the things about that is that um it 's e u the ri I guess the major risk you have there of being is being dependent on very dependent on the kind of noise and and so forth .
PhD A: Yeah . f But , yeah .
Professor D: Uh . But it 's another thing to try .
PhD A: So , this is w w i wa wa this is one thing , this this could be could help could help perhaps to reduce language dependency and for the noise part um we could combine this with other approaches , like , well , the Kleinschmidt approach . So the d the idea of putting all the noise that we can find inside a database . I think Kleinschmidt was using more than fifty different noises to train his network ,
PhD B: Yeah .
Professor D: Mm - hmm .
PhD A: and So this is one approach and the other is multi - band  uh , that I think is more robust to the noisy changes .
Professor D: Mm - hmm . Mm - hmm .
PhD A: So perhaps , I think something like multi - band trained on a lot of noises with uh , features - based targets could could could help .
Professor D: Yeah , if you i i It 's interesting thought maybe if you just trained up I mean w yeah , one one fantasy would be you have something like articulatory targets and you have um some reasonable database , um but then which is um copied over many times with a range of different noises ,
PhD A: Mm - hmm .
Professor D: And uh  If Cuz what you 're trying to do is come up with a a core , reasonable feature set which is then gonna be used uh , by the the uh HMM system .
PhD A: Mm - hmm .
Professor D: So . Yeah , OK .
PhD A: So , um , yeah . The future work is , well , try to connect to the to make to plug in the system to the OGI system . Um , there are still open questions there , where to put the MLP basically .
Professor D: Mm - hmm .
PhD A: Um .
Professor D: And I guess , you know , the the the real open question , I mean , e u there 's lots of open questions , but one of the core quote {comment} " open questions " for that is um , um , if we take the uh you know , the best ones here , maybe not just the best one , but the best few or something You want the most promising group from these other experiments . Um , how well do they do over a range of these different tests , not just the Italian ?
PhD A: Mmm ,
Professor D: Um . And y
PhD A: Yeah , yeah .
Professor D: y Right ? And then um then see , again , how We know that there 's a mis there 's a uh a a loss in performance when the neural net is trained on conditions that are different than than , uh we 're gonna test on , but well , if you look over a range of these different tests um , how well do these different ways of combining the straight features with the MLP features , uh stand up over that range ?
PhD B: Mm - hmm .
Professor D: That 's that that seems like the the the real question . And if you know that So if you just take PLP with uh , the double - deltas . Assume that 's the p the feature . look at these different ways of combining it . And uh , take let 's say , just take uh multi - English cause that works pretty well for the training .
PhD A: Mm - hmm .
Professor D: And just look take that case and then look over all the different things . How does that How does that compare between the PhD A: So all the all the test sets you mean , yeah .
PhD B: Yeah .
Professor D: All the different test sets ,
PhD A: And Professor D: and for and for the couple different ways that you have of of of combining them .
PhD A: Yeah .
Professor D: Um . How well do they stand up , over the PhD A: Mmm . And perhaps doing this for cha changing the variance of the streams and so on getting different scaling PhD B: Mm - hmm .
Professor D: That 's another possibility if you have time , yeah . Yeah .
PhD A: Um . Yeah , so thi this sh would be more working on the MLP as an additional path instead of an insert to the to their diagram .
Professor D :
PhD A: Cuz Yeah . Perhaps the insert idea is kind of strange because nnn , they they make LDA and then we will again add a network does discriminate anal nnn , that discriminates ,
Professor D: Yeah . It 's a little strange
PhD A: or ? Mmm ?
Professor D: but on the other hand they did it before .
PhD A: Mmm . And and and
Professor D: Um the
PhD A: yeah . And because also perhaps we know that the when we have very good features the MLP doesn't help . So . I don't know .
Professor D: Um , the other thing , though , is that um So . Uh , we we wanna get their path running here , right ? If so , we can add this other stuff .
PhD A: Um .
Professor D: as an additional path right ?
PhD A: Yeah , the the way we want to do Professor D: Cuz they 're doing LDA RASTA .
PhD A: The d What ?
Professor D: They 're doing LDA RASTA ,
PhD A: Yeah , the way we want to do it perhaps is to just to get the VAD labels and the final features .
Professor D: yeah ?
PhD A: So they will send us the Well , provide us with the feature files ,
Professor D: I see . I see .
PhD A: and with VAD uh , binary labels so that we can uh , get our MLP features and filter them with the VAD and then combine them with their f feature stream . So .
Professor D: I see . So we So . First thing of course we 'd wanna do there is to make sure that when we get those labels of final features is that we get the same results as them . Without putting in a second path .
PhD A: Uh . You mean Oh , yeah ! Just re re retraining r retraining the HTK ?
Professor D: Yeah just th w i i Just to make sure that we have we understand properly what things are , our very first thing to do is to is to double check that we get the exact same results as them on HTK .
PhD A: Oh yeah . Yeah , OK . Mmm .
PhD B: Yeah .
Professor D: Uh , I mean , I don't know that we need to r
PhD A: Yeah .
Professor D: Um Do we need to retrain I mean we can just take the re their training files also . But . But , uh just for the testing , jus just make sure that we get the same results so we can duplicate it before we add in another PhD A: Mmm . OK .
Professor D: Cuz otherwise , you know , we won't know what things mean .
PhD A: Oh , yeah . OK . And um . Yeah , so fff , LogRASTA , I don't know if we want to We can try networks with LogRASTA filtered features .
Professor D: Maybe .
PhD A: Mmm . I 'm sorry ? Yeah . Well Yeah . But Professor D: Oh ! You know , the other thing is when you say comb I 'm I 'm sorry , I 'm interrupting . {comment} that u Um , uh , when you 're talking about combining multiple features , um Suppose we said , " OK , we 've got these different features and so forth , but PLP seems pretty good . " If we take the approach that Mike did and have PhD A: Mm - hmm .
Professor D: I mean , one of the situations we have is we have these different conditions . We have different languages , we have different  different noises , Um If we have some drastically different conditions and we just train up different M L Ps with them .
PhD A :
Professor D: And put put them together . What what What Mike found , for the reverberation case at least , I mean I mean , who knows if it 'll work for these other ones . That you did have nice interpolative effects . That is , that yes , if you knew what the reverberation condition was gonna be and you trained for that , then you got the best results . But if you had , say , a heavily - reverberation ca heavy - reverberation case and a no - reverberation case , uh , and then you fed the thing , uh something that was a modest amount of reverberation then you 'd get some result in between the two . So it was sort of behaved reasonably . Is tha that a fair Yeah .
PhD A: Yeah . So you you think it 's perhaps better to have several M L Yeah but Professor D: It works better if what ?
PhD A: Yea
Professor D: I see . Well , see , i oc You were doing some something that was So maybe the analogy isn't quite right . You were doing something that was in way a little better behaved . You had reverb for a single variable which was re uh , uh , reverberation . Here the problem seems to be is that we don't have a hug a really huge net with a really huge amount of training data . But we have s f for this kind of task , I would think , sort of a modest amount . I mean , a million frames actually isn't that much . We have a modest amount of of uh training data from a couple different conditions , and then uh in yeah , that and the real situation is that there 's enormous variability that we anticipate in the test set in terms of language , and noise type uh , and uh , uh , channel characteristic , sort of all over the map . A bunch of different dimensions . And so , I 'm just concerned that we don't really have um , the data to train up I mean one of the things that we were seeing is that when we added in we still don't have a good explanation for this , but we are seeing that we 're adding in uh , a fe few different databases and uh the performance is getting worse and uh , when we just take one of those databases that 's a pretty good one , it actually is is is is is better . And uh that says to me , yes , that , you know , there might be some problems with the pronunciation models that some of the databases we 're adding in or something like that . But one way or another we don't have uh , seemingly , the ability to represent , in the neural net of the size that we have , um , all of the variability that we 're gonna be covering . So that I 'm I 'm I 'm hoping that um , this is another take on the efficiency argument you 're making , which is I 'm hoping that with moderate size neural nets , uh , that uh if we if they look at more constrained conditions they they 'll have enough parameters to really represent them . Mm - hmm . Mm - hmm . Mm - hmm . Yeah .
PhD A: So doing both is is not is not right , you mean , or ? Yeah .
Professor D: Yeah . I I just sort of have a feeling PhD A: But Yeah . Mm - hmm .
Professor D: Yeah . I mean  i i e The um I think it 's true that the OGI folk found that using LDA RASTA , which is a kind of LogRASTA , it 's just that they have the I mean it 's done in the log domain , as I recall , and it 's it uh it 's just that they d it 's trained up , right ? That that um benefitted from on - line normalization . So they did At least in their case , it did seem to be somewhat complimentary . So will it be in our case , where we 're using the neural net ? I mean they they were not not using the neural net . Uh I don't know . OK , so the other things you have here are uh , trying to improve results from a single Yeah . Make stuff better . OK . Uh . Yeah . And CPU memory issues . Yeah . We 've been sort of ignoring that , haven't we ?
PhD A: Yeah , so I don't know .
Professor D: But PhD A: But we have to address the problem of CPU and memory we Professor D: Yeah , but I li Well , I think My impression You you folks have been looking at this more than me . But my impression was that uh , there was a a a a strict constraint on the delay ,
PhD B: Yeah .
Professor D: but beyond that it was kind of that uh using less memory was better , and using less CPU was better . Something like that ,
PhD A: Yeah , but Professor D: right ?
PhD A: Yeah . So , yeah , but we 've I don't know . We have to get some reference point to where we Well , what 's a reasonable number ? Perhaps be because if it 's if it 's too large or large or @ @ Professor D: Um , well I don't think we 're um completely off the wall . I mean I think that if we if we have Uh , I mean the ultimate fall back that we could do If we find uh I mean we may find that we we 're not really gonna worry about the M L You know , if the MLP ultimately , after all is said and done , doesn't really help then we won't have it in .
PhD A: Mmm .
Professor D: If the MLP does , we find , help us enough in some conditions , uh , we might even have more than one MLP . We could simply say that is uh , done on the uh , server .
PhD A: Mmm .
Professor D: And it 's uh We do the other manipulations that we 're doing before that . So , I I I think I think that 's  that 's OK .
PhD A: And Yeah .
Professor D: So I think the key thing was um , this plug into OGI . Um , what what are they What are they gonna be working Do we know what they 're gonna be working on while we take their features ,
PhD A: They 're They 're starting to wor work on some kind of multi - band .
Professor D: and ?
PhD A: So . Um This that was Pratibha . Sunil , what was he doing , do you remember ?
PhD B: Sunil ?
PhD A: Yeah . He was doing something new or ?
PhD B: I I don't re I didn't remember . Maybe he 's working with neural network .
PhD A: I don't think so . Trying to tune wha networks ?
PhD B: Yeah , I think so .
PhD A: I think they were also mainly , well , working a little bit of new things , like networks and multi - band , but mainly trying to tune their their system as it is now to just trying to get the best from this this architecture .
PhD B: Yeah .
PhD A :
Professor D: OK . So I guess the way it would work is that you 'd get There 'd be some point where you say , " OK , this is their version - one " or whatever , and we get these VAD labels and features and so forth for all these test sets from them ,
PhD A: Mm - hmm .
Professor D: and then um , uh , that 's what we work with . We have a certain level we try to improve it with this other path and then um , uh , when it gets to be uh , January some point uh , we say , " OK we we have shown that we can improve this , in this way . So now uh um what 's your newest version ? " And then maybe they 'll have something that 's better and then we we 'd combine it . This is always hard . I mean I I I used to work with uh folks who were trying to improve a good uh , HMM system with uh with a neural net system and uh , it was a common problem that you 'd Oh , and this Actually , this is true not just for neural nets but just for in general if people were working with uh , rescoring uh , N - best lists or lattices that come came from uh , a mainstream recognizer . Uh , You get something from the the other site at one point and you work really hard on making it better with rescoring . But they 're working really hard , too . So by the time you have uh , improved their score , they have also improved their score
PhD A: Mmm .
Professor D: and now there isn't any difference ,
PhD A: Yeah .
Professor D: because the other PhD B: Yeah .
Professor D: So , um , I guess at some point we 'll have to
PhD A: So it 's Professor D: uh {comment} Uh , I I don't know . I think we 're we 're integrated a little more tightly than happens in a lot of those cases . I think at the moment they they say that they have a better thing we can we e e
PhD A: Mmm .
Professor D: What takes all the time here is that th we 're trying so many things , presumably uh , in a in a day we could turn around uh , taking a new set of things from them and and rescoring it ,
PhD A: Mmm . Yeah . Yeah , perhaps we could .
Professor D: right ? So . Yeah . Well , OK . No , this is I think this is good . I think that the most wide open thing is the issues about the uh , you know , different trainings . You know , da training targets and noises and so forth .
PhD A: Mmm . So we we can for we c we can forget combining multiple features and MLG perhaps ,
Professor D: That 's sort of wide open .
PhD A: or focus more on the targets and on the training data and ?
Professor D: Yeah , I think for right now um , I th I I really liked MSG . And I think that , you know , one of the things I liked about it is has such different temporal properties . And um , I think that there is ultimately a really good uh , potential for , you know , bringing in things with different temporal properties . Um , but um , uh , we only have limited time and there 's a lot of other things we have to look at .
PhD A: Mmm .
Professor D: And it seems like much more core questions are issues about the training set and the training targets , and fitting in uh what we 're doing with what they 're doing , and , you know , with limited time . Yeah . I think we have to start cutting down .
PhD A: Mmm .
Professor D: So uh I think so , yeah . And then , you know , once we Um , having gone through this process and trying many different things , I would imagine that certain things uh , come up that you are curious about uh , that you 'd not getting to and so when the dust settles from the evaluation uh , I think that would time to go back and take whatever intrigued you most , you know , got you most interested uh and uh and and work with it , you know , for the next round . Uh , as you can tell from these numbers uh , nothing that any of us is gonna do is actually gonna completely solve the problem .
PhD A: Mmm .
Professor D: So . So , {comment} there 'll still be plenty to do . Barry , you 've been pretty quiet .
Grad C: Just listening .
Professor D: Well I figured that , but  That what what what were you involved in in this primarily ?
Grad C: Um , helping out uh , preparing Well , they 've been kind of running all the experiments and stuff and I 've been uh , uh w doing some work on the on the preparing all all the data for them to to um , train and to test on . Um Yeah . Right now , I 'm I 'm focusing mainly on this final project I 'm working on in Jordan 's class .
Professor D: Ah !
Grad C: Yeah .
Professor D: I see . Right . What 's what 's that ?
Grad C: Um , I 'm trying to um So there was a paper in ICSLP about um this this multi - band um , belief - net structure . {comment} This guy did Professor D: Mm - hmm .
Grad C: uh basically it was two H M Ms with with a with a dependency arrow between the two H M
Professor D: Uh - huh .
Grad C: And so I wanna try try coupling them instead of t having an arrow that that flows from one sub - band to another sub - band . I wanna try having the arrows go both ways . And um , I 'm just gonna see if if that that better models um , uh asynchrony in any way or um  Yeah .
Professor D: Oh ! OK . Well , that sounds interesting .
Grad C: Yeah .
Professor D: OK . Alright . Anything to  you wanted to No . OK . Silent partner in the  in the meeting . Oh , we got a laugh out of him , that 's good . OK , everyone h must contribute to the our our sound  sound files here . OK , so speaking of which , if we don't have anything else that we need You happy with where we are ?
PhD A: Mmm .
Professor D: Know know wher know where we 're going ? Uh PhD A: I think so , yeah .
Professor D: Yeah , yeah . You you happy ?
PhD B :
Professor D: You 're happy . OK everyone should be happy . OK . You don't have to be happy . You 're almost done . Yeah , yeah . OK .
Grad E: Al - actually I should mention So if {comment} um , about the Linux machine " Swede . "
Professor D: Yeah .
Grad E: So it looks like the um , neural net tools are installed there .
PhD A: Mmm .
Grad E: And um Dan Ellis {comment} I believe knows something about using that machine so
PhD A: Mmm .
Grad E: If people are interested in in getting jobs running on that maybe I could help with that .
PhD A: Yeah , but I don't know if we really need now a lot of machines . Well . we could start computing another huge table but yeah , we Professor D: Well . Yeah , I think we want a different table , at least
PhD A: Yeah , sure .
Professor D: Right ? I mean there 's there 's some different things that we 're trying to get at now .
PhD A: But Professor D: But PhD A: Yeah . Mmm .
Professor D: So . Yeah , as far as you can tell , you 're actually OK on C - on CPU uh , for training and so on ? Yeah .
PhD A: Ah yeah . I think so . Well , more is always better , but mmm , I don't think we have to train a lot of networks , now that we know We just select what works fine
Professor D: OK . OK .
PhD A: and try to improve this
PhD B: Yeah . to work
Professor D: And we 're OK on And we 're OK on disk ?
PhD A: and It 's OK , yeah . Well sometimes we have some problems .
PhD B: Some problems with the Professor D: But they 're correctable , uh problems .
PhD A: Yeah , restarting the script basically
PhD B: You know .
PhD A: and Professor D: Yes . Yeah , I 'm familiar with that one , OK . Alright , so uh , {comment} since uh , we didn't ha get a channel on for you , {comment} you don't have to read any digits but the rest of us will . Uh , is it on ? Well . We didn't uh I think I won't touch anything cuz I 'm afraid of making the driver crash which it seems to do , pretty easily . OK , thanks . OK , so we 'll uh I 'll start off the uh um connect the PhD A: My battery is low .
Professor D: Well , let 's hope it works . Maybe you should go first and see so that you 're OK .
PhD B: batteries ?
Grad C: Yeah , your battery 's going down too .
Professor D: Transcript uh two Grad C: Carmen 's battery is d going down too .
Professor D: Oh , OK . Yeah . Why don't you go next then . OK . Guess we 're done . OK , uh so . Just finished digits . Yeah , so . Uh Well , it 's good . I think I guess we can turn off our microphones now .
Grad C: Just pull the batteries out .