PhD F: OK . Why don't you go ahead , Dave ?
Grad C: OK . Um , so , yeah , the this past week I 've been main mainly occupied with , um , getting some results , u from the SRI system trained on this short Hub - five training set for the mean subtraction method . And , um , I ran some tests last night . But , um , c the results are suspicious . Um , it 's , um , cuz they 're the baseline results are worse than , um , Andreas than results Andreas got previously . And it could have something to do with , um PhD F: That 's on digits ?
Grad C: That 's on digits . It c it it could h it could have something to do with , um , downsampling .
PhD F: Hmm .
Grad C: That 's that 's worth looking into . Um , d and , um , ap ap apart from that , I guess the the main thing I have t ta I have to talk is , um , where I 'm planning to go over the next week . Um . So I 've been working on integrating this mean subtraction approach into the SmartKom system . And there 's this question of , well , so , um , in my tests before with HTK I found it worked it worked the best with about twelve seconds of data used to estimate the mean , but , we 'll often have less {comment} in the SmartKom system . Um . So I think we 'll use as much data as we have at a particular time , and we 'll  we 'll concatenate utterances together , um , to get as much data as we possibly can from the user . But , um , there 's a question of how to set up the models . So um , we could train the models . If we think twelve seconds is ideal we could train the models using twelve seconds to calculate the mean , to mean subtract the training data . Or we could , um , use some other amount . So like I did an experiment where I , um , was using six seconds in test , um , but , for I tried twelve seconds in train . And I tried , um , um , the same in train I 'm a I tried six seconds in train . And six seconds in train was about point three percent better . Um , and  um , it 's not clear to me yet whether that 's something significant . So I wanna do some tests and , um , actually make some plots of , um for a particular amount of data and test what happens if you vary the amount of data in train .
PhD F: Mm - hmm .
Professor D: Uh , Guenter , I don't know if you t followed this stuff but this is , uh , a uh , uh , long - term long - term window F F Yeah . Yeah , he you talked about it .
Professor G: Yeah , we we spoke about it already ,
Professor D: Oh , OK . So you know what he 's doing .
Professor G: yeah .
Professor D: Alright .
Grad C: y s so I was I actually ran the experiments mostly and I I was I was hoping to have the plots with me today . I just didn't get to it . But , um yeah , I wou I would be curious about people 's feedback on this cuz I 'm  @ @ {comment} I p I think there are some I think it 's it 's kind of like a a bit of a tricky engineering problem . I 'm trying to figure out what 's the optimal way to set this up . So , um , I 'll try to make the plots and then put some postscript up on my on my web page . And I 'll mention it in my status report if people wanna take a look .
Professor D: You could clarify something for me . You 're saying point three percent , you take a point three percent hit , when the training and testing links are don't match or something ?
PhD E: Hello .
Professor D: Is that what it is ?
Grad C: w Well , it c
Professor D: Or ?
Grad C: I I don't think it it 's just for any mismatch you take a hit .
Professor D: Yeah .
Grad C: i In some cases it might be u better to have a mismatch . Like I think I saw something like like if you only have two seconds in test , or , um , maybe it was something like four seconds , you actually do a little better if you , um , train on six seconds than if you train on four seconds .
Professor D: Yeah . Right .
Grad C: Um , but the case , uh with the point three percent hit was using six seconds in test , um , comparing train on twelve seconds {comment} versus train on six seconds .
Professor D: And which was worse ?
Grad C: The train on twelve seconds .
Professor D: OK . But point three percent , uh , w from what to what ? That 's point three percent Grad C: On The the the accuracies w went from it was something vaguely like ninety - five point six accuracy , um , improved to ninety - five point nine wh when I Professor D: So four point four to four point one .
Grad C: OK .
Professor D: So yeah . So about a about an eight percent , uh , seven or eight percent relative ?
Grad C: OK .
Professor D: Uh , Yeah . Well , I think in a p You know , if if you were going for an evaluation system you 'd care . But if you were doing a live system that people were actually using nobody would notice . It 's uh , I think the thing is to get something that 's practical , that that you could really use .
Grad C: Huh . That 's that 's interesting . Alright , the e uh , I see your point . I guess I was thinking of it as , um , an interesting research problem . The how to g I was thinking that for the ASRU paper we could have a section saying , " For SmartKom , we we d in we tried this approach in , uh , interactive system " , which I don't think has been done before .
Professor D: Yeah . Mm - hmm .
Grad C: And and then there was two research questions from that .
Professor D: Mm - hmm .
Grad C: And one is the k does it still work if you just use the past history ?
Professor D: Mm - hmm .
Grad C: Alright , and the other was this question of , um what I was just talking about now . So I guess that 's why I thought it was interesting .
Professor D: I mean , a short - time FFT short - time cepstrum calculation , uh , mean u mean calculation work that people have in commercial systems , they do this all the time . They the they calculate it from previous utterances and then use it , you know .
Grad C: Yeah , um .
Professor D: But but , uh , as you say , there hasn't been that much with this long long - time , uh , spectra work .
Grad C: Oh , o Oh , OK .
Professor D: Uh ,
Grad C: So that 's that 's that 's standard . Um Professor D: Yeah . Pretty common .
Grad C: OK .
Professor D: Yeah . Um , but , u uh , yes . No , it is interesting . And the other thing is , I mean , there 's two sides to these really small , uh , gradations in performance . Um , I mean , on the one hand in a practical system if something is , uh , four point four percent error , four point one percent error , people won't really tell be able to tell the difference . On the other hand , when you 're doing , uh , research , you may , eh you might find that the way that you build up a change from a ninety - five percent accurate system to a ninety - eight percent accurate system is through ten or twelve little things that you do that each are point three percent . So so the they they it 's I don't mean to say that they 're they 're irrelevant . Uh , they are relevant . But , um , i for a demo , you won't see it .
Grad C: Mm - hmm . Right . OK .
Professor D: Yeah .
Grad C: And , um , Let 's l let 's see . Um , OK . And then there 's um , another thing I wanna start looking at , um , wi is , um , the choice of the analysis window length . So I 've just been using two seconds just because that 's what Carlos did before . Uh , I wrote to him asking about he chose the two seconds . And it seemed like he chose it a bit informally . So , um , with the with the HTK set - up I should be able to do some experiments , on just varying that length , say between one and three seconds , in a few different reverberation conditions , um , say this room and also a few of the artificial impulse responses we have for reverberation , just , um , making some plots and seeing how they look . And , um , so , with the the sampling rate I was using , one second or two seconds or four seconds is at a power of two um , number of samples and , um , I 'll I 'll jus f for the ones in between I guess I 'll just zero - pad .
Professor D: Mm - hmm . I guess one thing that might also be an issue , uh , cuz part of what you 're doing is you 're getting a a spectrum over a bunch of different kinds of speech sounds . Um , and so it might matter how fast someone was talking for instance .
Grad C: Oh .
Professor D: You know , if you if if if there 's a lot of phones in one second maybe you 'll get a a really good sampling of all these different things , and  and , uh , on the other hand if someone 's talking slowly maybe you 'd need more . So Grad C: Huh .
Professor D: I don't know if you have some samples of faster or slower speech but it might make a difference . I don't know .
Grad C: Uh , yeah , I don't I don't think the TI - digits data that I have , um , i is would be appropriate for that .
Professor D: Yeah , probably not . Yeah .
Grad C: But what do you What about if I w I fed it through some kind of , um , speech processing algorithm that changed the speech rate ?
Professor D: Yeah , but then you 'll have the degradation of of , uh , whatever you do uh , added onto that . But maybe . Yeah , maybe if you get something that sounds that that 's does a pretty job at that .
Grad C: Yeah . Well , uh , just if you think it 's worth looking into .
Professor D: You could imagine that .
Grad C: I mean , it it is getting a little away from reverberation .
Professor D: Um , yeah . It 's just that you 're making a choice uh , I was thinking more from the system aspect , if you 're making a choice for SmartKom , that that that it might be that it 's it c the optimal number could be different , depending on Grad C: Yeah . Right .
Professor D: Could be . I don't know .
Grad C: And and th the third thing , um , uh , is , um , Barry explained LDA filtering to me yesterday . And so , um , Mike Shire in his thesis um , did a a series of experiments , um , training LDA filters in d on different conditions . And you were interested in having me repeat this for for this mean subtraction approach ? Is is that right ? Or for these long analysis windows , I guess , is the right way to put it .
Professor D: I guess , the the the issue I was the general issue I was bringing up was that if you 're have a moving  moving window , uh , a wa a a set of weights times things that , uh , move along , shift along in time , that you have in fact a linear time invariant filter . And you just happened to have picked a particular one by setting all the weights to be equal . And so the issue is what are some other filters that you could use , uh , in that sense of " filter " ?
Grad C: Mm - hmm .
Professor D: And , um , as I was saying , I think the simplest thing to do is not to train anything , but just to do some sort of , uh , uh , hamming or Hanning , uh , kind of window , kind of thing ,
Grad C: Right . Mm - hmm .
Professor D: just sort of to de - emphasize the jarring . So I think that would sort of be the first thing to do . But then , yeah , the LDA i uh , is interesting because it would sort of say well , suppose you actually trained this up to do the best you could by some criterion , what would the filter look like then ?
Grad C: Uh - huh .
Professor D: Uh , and , um , that 's sort of what we 're doing in this Aur - Aurora stuff . And , uh , it 's still not clear to me in the long run whether the best thing to do would be to do that or to have some stylized version of the filter that looks like these things you 've trained up , because you always have the problem that it 's trained up for one condition and it isn't quite right for another . So . uh that 's that 's why that 's why RASTA filter has actually ended up lasting a long time , people still using it quite a bit , because y you don't change it . So doesn't get any worse . Uh ,
Grad C: Huh .
Professor D: Anyway .
Grad C: o OK . So , um , a actually I was just thinking about what I was asking about earlier , wi which is about having less than say twelve seconds in the SmartKom system to do the mean subtraction . You said in systems where you use cepstral mean subtraction , they concatenate utterances and , do you know how they address this issue of , um , testing versus training ? Can Professor D: Go ahead .
Professor G: I think what they do is they do it always on - line , I mean , that you just take what you have from the past , that you calculate the mean of this and subtract the mean .
Grad C: OK . Um Professor G: And then you can yeah , you you can increase your window whi while you get while you are getting more samples .
Grad C: OK , um , and , um , so so in tha in that case , wh what do they do when they 're t um , performing the cepstral mean subtraction on the training data ? So because you 'd have hours and hours of training data . So do they cut it off and start over ? At intervals ? Or ?
Professor G: So do you have uh , you you mean you have files which are hours of hours long ? Or ?
Grad C: Oh , well , no . I guess not . But Professor G: Yeah . I mean , usually you have in the training set you have similar conditions , I mean , file lengths are , I guess the same order or in the same size as for test data , or aren't they ?
Grad C: OK . But it 's OK . So if someone 's interacting with the system , though , uh , Morgan uh , Morgan said that you would tend to , um , chain utterances together um , r
Professor D: Well , I think what I was s I thought what I was saying was that , um , at any given point you are gonna start off with what you had from before .
Grad C: Oh .
Professor D: From and so if you 're splitting things up into utterances So , for instance , in a dialogue system , {comment} where you 're gonna be asking , uh , you know , th for some information , there 's some initial th something . And , you know , the first time out you you might have some general average . But you you d you don't have very much information yet . But at after they 've given one utterance you 've got something . You can compute your mean cepstra from that ,
Grad C: Mm - hmm .
Professor D: and then can use it for the next thing that they say , uh , so that , you know , the performance should be better that second time . Um ,  and I think the heuristics of exactly how people handle that and how they handle their training I 'm sure vary from place to place . But I think the ideally , it seems to me anyway , that you you would wanna do the same thing in training as you do in test . But that 's that 's just , uh , a prejudice . And I think anybody working on this with some particular task would experiment .
Grad C: Right . I g I guess the question I had was , um , amount of data e u was the amount of data that you 'd give it to , um update this estimate . Because say you if you have say five thousand utterances in your training set , um , and you you keep the mean from the last utterance , by the time it gets to the five thousandth utterance Professor D: No , but those are all different people with different I mean , i in y So for instance , in in the in a telephone task , these are different phone calls . So you don't wanna @ @ {comment} chain it together from a from a different phone call .
Grad C: OK , so so so they would g s
Professor D: So it 's within speaker , within phone call ,
Professor G: Yeah .
Professor D: if it 's a dialogue system , it 's within whatever this characteristic you 're trying to get rid of is expected to be consistent over ,
Professor G: Hmm .
Grad C: r and it Professor D: right ?
Grad C: right . OK , so you 'd you and so in training you would start over at at every new phone call or at every new speaker . Yeah ,
Professor D: Yeah .
Grad C: OK .
Professor D: Yeah . Now , you know , maybe you 'd use something from the others just because at the beginning of a call you don't know anything , and so you might have some kind of general thing that 's your best guess to start with . But So , s I I you know , a lot of these things are proprietary so we 're doing a little bit of guesswork here . I mean , what do comp what do people do who really face these problems in the field ? Well , they have companies and they don't tell other people exactly what they do .
Grad C: R right .
Professor D: But but I mean , when you the the hints that you get from what they when they talk about it are that they do they all do something like this .
Grad C: Right , OK . I see . Bec - because I so this SmartKom task first off , it 's this TV and movie information system .
Professor D: Yeah , but you might have somebody who 's using it
Grad C: And Yeah .
Professor D: and then later you might have somebody else who 's using it .
Grad C: Yeah . Right . Right . I I see .
Professor D: And so you 'd wanna set some Grad C: I was I was about to say . So if if you ask it " What what movies are on TV tonight ? " ,
Professor D: Yeah . Yeah .
Grad C: if I look at my wristwatch when I say that it 's about two seconds . The way I currently have the mean subtraction , um , set up , the the analysis window is two seconds .
Professor D: Yeah .
Grad C: So what you just said , about what do you start with , raises a question of what do I start with then ?
Professor D: Mm - hmm .
Grad C: I guess it because Professor D: Well , w OK , so in that situation , though , th maybe what 's a little different there , is I think you 're talking about there 's only one it it it also depends we 're getting a little off track here .
Grad C: Oh , right .
Professor D: r But but but Uh , there 's been some discussion about whether the work we 're doing in that project is gonna be for the kiosk or for the mobile or for both . And I think for this kind of discussion it matters . If it 's in the kiosk , then the physical situation is the same . It 's gonna you know , the exact interaction of the microphone 's gonna differ depending on the person and so forth . But at least the basic acoustics are gonna be the same . So f if it 's really in one kiosk , then I think that you could just chain together and and you know , as much as much speech as possible to because what you 're really trying to get at is the is the reverberation characteristic .
Grad C: Yeah .
Professor D: But in in the case of the mobile , uh , {comment} presumably the acoustic 's changing all over the place .
Grad C: Right .
Professor D: And in that case you probably don't wanna have it be endless because you wanna have some sort of it 's it 's not a question of how long do you think it 's you can get an approximation to a stationary something , given that it 's not really stationary .
Grad C: Right . Right .
Professor D: So .
Professor G: Hmm .
Grad C: And I I g I guess I s just started thinking of another question , which is , for for the very first frame , w what what do I do if I 'm if I take if I use that frame to calculate the mean , then I 'm just gonna get n nothing .
Professor D: Mm - hmm .
Grad C: Um ,
Professor D: Right .
Grad C: so I should probably have some kind of default mean for the first f couple of frames ?
Professor D: Yeah . Yeah .
Grad C: OK .
Professor D: Yeah . Or subtract nothing . I mean , it 's Grad C: Or subtract nothing . And and that 's that 's I guess that 's something that 's p people have figured out how to deal with in cepstral mean subtraction as well ?
Professor D: Yeah , yeah . Yeah , people do something . They they , uh , they have some , um , uh , in in cepstral mean subtraction , for short - term window analysis windows , as is usually done , you 're trying to get rid of some very general characteristic . And so , uh , if you have any other information about what a general kind of characteristic would be , then you you can do it there .
PhD F: You can also you can also reflect the data . So you take , uh you know , I 'm not sure how many frames you need .
Grad C: Uh - huh .
PhD F: But you take that many from the front and flip it around to a as the negative value .
Professor D: Yeah , that 's Yeah .
PhD F: So you can always Professor D: The other thing is that and and I I remember B B N doing this , is that if you have a multi - pass system , um , if the first pass ta it takes most of the computation , the second and the third pass could be very , very quick ,
Grad C: Mmm .
Professor D: just looking at a relatively small n small , uh , space of hypotheses .
Grad C: Uh - huh .
Professor D: Then you can do your first pass without any subtraction at all .
Grad C: Oh .
Professor D: And then your second pass , uh , uh , eliminates those most of those hypotheses by , uh by having an improved improved version o of the analysis .
Grad C: OK . OK .
Professor D: So .
Grad C: OK . So that was all I had , for now .
Professor D: Yeah .