PhD A: OK , we 're on .
Professor C: OK , what are we talking about today ?
PhD B: I don't know . Do you have news from the conference talk ? Uh , that was programmed for yesterday I guess .
Professor C: Uh PhD D: Yesterday
Professor C: Uh PhD D: Yesterday morning on video conference .
Professor C: Uh ,
PhD B: Well
Professor C: oh , I 'm sorry .
Grad E: Oh . Conference call .
Professor C: I know now I know what you 're talking about . No , nobody 's told me anything .
PhD B: Alright .
PhD A: Oh , this was the , uh , talk where they were supposed to try to decide PhD B: To to decide what to do ,
PhD A: Ah , right .
PhD B: yeah .
PhD D: Yeah .
Professor C: Yeah . No , that would have been a good thing to find out before this meeting , that 's . No , I have no I have no idea . Um , Uh , so I mean , let 's let 's assume for right now that we 're just kind of plugging on ahead ,
PhD B: Yeah .
Professor C: because even if they tell us that , uh , the rules are different , uh , we 're still interested in doing what we 're doing . So what are you doing ?
PhD B: Mm - hmm . Uh , well , we 've a little bit worked on trying to see , uh , what were the bugs and the problem with the latencies .
PhD D: To improve PhD B: So , We took first we took the LDA filters and , uh , we designed new filters , using uh recursive filters actually .
Professor C: So when you say " we " , is that something Sunil is doing or is that ?
PhD B: I 'm sorry ?
Professor C: Who is doing that ?
PhD B: Uh , us . Yeah .
Professor C: Oh , oh . Oh , OK .
PhD B: So we took the filters the FIR filters and we {comment} designed , uh , IIR filters that have the same frequency response .
PhD D: But Professor C: Mm - hmm .
PhD B: Well , similar , but that have shorter delays .
Professor C: Mm - hmm .
PhD B: So they had two filters , one for the low frequency bands and another for the high frequency bands . And so we redesigned two filters . And the low frequency band has sixty - four milliseconds of delay , and the high frequency band filter has something like eleven milliseconds compared to the two hundred milliseconds of the IIR filters . But it 's not yet test . So we have the filters but we still have to implement a routine that does recursive filtering
Professor C: OK .
PhD B: and Professor C: You you had a discussion with Sunil about this though ?
PhD B: No . No .
Professor C: Uh - huh . Yeah , you should talk with him .
PhD B: Yeah , yeah .
Professor C: Yeah . No , I mean , because the the the the whole problem that happened before was coordination ,
PhD B: Mm - hmm .
Professor C: right ? So so you need to discuss with him what we 're doing ,
PhD B: Yeah .
Professor C: uh , cuz they could be doing the same thing and or something .
PhD B: Mm - hmm . Uh , I yeah , I don't know if th that 's what they were trying to They were trying to do something different like taking , uh well , using filter that takes only a past
Professor C: Right .
PhD B: and this is just a little bit different . But I will I will send him an email and tell him exactly what we are doing , so .
Professor C: Yeah , yeah . Um ,
PhD B: Um ,
Professor C: I mean We just we just have to be in contact more . I think that the the fact that we we did that with had that thing with the latencies was indicative of the fact that there wasn't enough communication .
PhD B: Mm - hmm .
Professor C: So .
PhD B: Alright .
Professor C: OK .
PhD B: Um , Yeah . Well , there is w one , um , remark about these filters , that they don't have a linear phase . So ,
Professor C: Right .
PhD B: Well , I don't know , perhaps it perhaps it doesn't hurt because the phase is almost linear but . Um , and so , yeah , for the delay I gave you here , it 's it 's , uh , computed on the five hertz modulation frequency , which is the mmm , well , the most important for speech so . Uh , this is the first thing .
Professor C: So that would be , uh , a reduction of a hundred and thirty - six milliseconds ,
PhD D: The low f f
PhD B: Yeah .
Professor C: which , uh What was the total we ended up with through the whole system ?
PhD B: Three hundred and thirty .
Professor C: So that would be within ?
PhD B: Yeah , but there are other points actually , uh , which will perhaps add some more delay . Is that some other other stuff in the process were perhaps not very um perf well , not very correct , like the downsampling which w was simply dropping frames .
Professor C: Yeah .
PhD B: Um , so we will try also to add a nice downsampling having a filter that that Professor C: Uh - huh .
PhD B: well , a low - pass filter at at twenty - five hertz . Uh , because wh when when we look at the LDA filters , well , they are basically low - pass but they leave a lot of what 's above twenty - five hertz .
Professor C: Yeah .
PhD B: Um , and so , yeah , this will be another filter which would add ten milliseconds again .
Professor C: Yeah .
PhD B: Um , yeah , and then there 's a third thing , is that , um , basically the way on - line normalization was done uh , is just using this recursion on on the um , um , on the feature stream ,
Professor C: Yeah .
PhD B: and but this is a filter , so it has also a delay . Uh , and when we look at this filter actually it has a delay of eighty - five milliseconds . So if we Professor C: Eighty - five .
PhD B: Yeah . If we want to be very correct , so if we want to the estimation of the mean t t to to be well , the right estimation of the mean , we have to t to take eighty - five milliseconds in the future . Mmm .
Professor C: Hmm ! That 's a little bit of a problem .
PhD B: Yeah . Um , But , well , when we add up everything it 's it will be alright . We would be at six so , sixty - five , plus ten , plus for the downsampling , plus eighty - five for the on - line normalization . So it 's
Professor C: Uh ,
PhD B: plus plus eighty for the neural net and PCA .
Professor C: yeah , but then there 's Oh .
PhD B: So it would be around two hundred and forty so , well ,
Professor C: Just just barely in there .
PhD B: plus plus the frames , but it 's OK .
PhD A: What 's the allowable ?
Professor C: Two - fifty , unless they changed the rules .
PhD B: Hmm .
Professor C: Which there is there 's some discussion of .
PhD A: What were they thinking of changing it to ?
Professor C: But PhD B: Yeah .
Professor C: Uh , well the people who had very low latency want it to be low uh , very  very very narrow , uh , latency bound . And the people who have longer latency don't . So .
PhD A: Huh .
PhD B: So , yeah .
Professor C: Unfortunately we 're the main ones with long latency , but
PhD A: Ah !
Professor C: But , uh ,
PhD B: Yeah , and basically the best proposal had something like thirty or forty milliseconds of latency .
Professor C: you know , it 's Yeah .
PhD B: So . Well .
Professor C: Yeah , so they were basically I mean , they were more or less trading computation for performance and we were , uh , trading latency for performance . And they were dealing with noise explicitly and we weren't , and so I think of it as complementary , that if we can put the PhD A: Think of it as what ?
Professor C: Complementary .
PhD A: Hmm .
Professor C: I think the best systems so , uh , everything that we did in in a way it was it was just adamantly insisting on going in with a brain damaged system , which is something actually , we 've done a lot over the last thirteen years . Uh , which is we say , well this is the way we should do it . And then we do it . And then someone else does something that 's straight forward . So , w th w this was a test that largely had additive noise and we did we adde did absolutely nothing explicitly to handle ad additive noise .
PhD A: Right .
Professor C: We just , uh , you know , trained up systems to be more discriminant . And , uh , we did this , uh , RASTA - like filtering which was done in the log domain and was tending to handle convolutional noise . We did we actually did nothing about additive noise . So , um , the , uh , spectral sub subtraction schemes a couple places did seem to seem to do a nice job . And so , uh , we 're talking about putting putting some of that in while still keeping some of our stuff . I think you should be able to end up with a system that 's better than both but clearly the way that we 're operating for this other stuff does involved some latency to to get rid of most of that latency . To get down to forty or fifty milliseconds we 'd have to throw out most of what we 're doing . And and , uh , I don't think there 's any good reason for it in the application actually . I mean , you 're you 're you 're speaking to a recognizer on a remote server and , uh , having a a a quarter second for some processing to clean it up . It doesn't seem like it 's that big a deal .
PhD A: Mm - hmm .
Professor C: These aren't large vocabulary things so the decoder shouldn't take a really long time , and .
PhD A: And I don't think anybody 's gonna notice the difference between a quarter of a second of latency and thirty milliseconds of latency .
Professor C: So . No . What what does wa was your experience when you were doing this stuff with , uh , the the the surgical , uh , uh , microscopes and so forth . Um , how long was it from when somebody , uh , finished an utterance to when , uh , something started happening ?
PhD A: Um , we had a silence detector , so we would look for the end of an utterance based on the silence detector .
Professor C: Mm - hmm .
PhD A: And I I can't remember now off the top of my head how many frames of silence we had to detect before we would declare it to be the end of an utterance .
Professor C: Mm - hmm . Mm - hmm .
PhD A: Um , but it was , uh , I would say it was probably around the order of two hundred and fifty milliseconds .
Professor C: Yeah , and that 's when you 'd start doing things .
PhD A: Yeah , we did the back trace at that point to get the answer .
Professor C: Yeah . Of course that didn't take too long at that point .
PhD A: No , no it was pretty quick .
Professor C: Yeah .
PhD A: So Professor C: Yeah , so you you so you had a
PhD A: this w
Professor C: so you had a a quarter second delay before , uh , plus some little processing time ,
PhD A: Right .
Professor C: and then the the microscope would start moving or something .
PhD A: Right .
Professor C: Yeah .
PhD A: Right .
Professor C: And there 's physical inertia there , so probably the the motion itself was all PhD A: And it felt to , uh , the users that it was instantaneous . I mean , as fast as talking to a person . It th I don't think anybody ever complained about the delay .
Professor C: Yeah , so you would think as long as it 's under half a second or something .
PhD A: Yeah .
Professor C: Uh , I 'm not an expert on that
PhD A: Yeah .
Professor C: but .
PhD A: I don't remember the exact numbers but it was something like that .
Professor C: Yeah .
PhD A: I don't think you can really tell . A person I don't think a person can tell the difference between , uh , you know , a quarter of a second and a hundred milliseconds , and I 'm not even sure if we can tell the difference between a quarter of a second and half a second .
Professor C: Yeah .
PhD A: I mean it just it feels so quick .
Professor C: Yeah . I mean , basically if you yeah , if you said , uh , um , " what 's the , uh , uh what 's the shortest route to the opera ? " and it took half a second to get back to you ,
PhD A: Yeah .
Professor C: I mean , it would be f I mean , it might even be too abrupt . You might have to put in a s a s a delay .
PhD A: Yeah . I mean , it may feel different than talking to a person
Professor C: Yeah .
PhD A: because when we talk to each other we tend to step on each other 's utterances . So like if I 'm asking you a question , you may start answering before I 'm even done .
Professor C: Yeah .
PhD A: So it it would probably feel different
Professor C: Right .
PhD A: but I don't think it would feel slow .
Professor C: Right . Well , anyway , I mean , I think we could cut we know what else , we could cut down on the neural net time by by , uh , playing around a little bit , going more into the past , or something like that . We t we talked about that .
PhD A: So is the latency from the neural net caused by how far ahead you 're looking ?
Professor C: Mm - hmm .
PhD B: Mm - hmm .
Professor C: And there 's also well , there 's the neural net and there 's also this , uh , uh , multi - frame , uh , uh , KLT .
PhD A: Wasn't there Was it in the , uh , recurrent neural nets where they weren't looking ahead at all ?
Professor C: They weren't looking ahead much . They p they looked ahead a little bit .
PhD A: A little bit . OK .
Professor C: Yeah . Yeah , I mean , you could do this with a recurrent net . And and then But you also could just , um , I mean , we haven't experimented with this but I imagine you could , um , uh , predict a , uh um , a label , uh , from more in the past than in than than in the future . I mean , we 've d we 've done some stuff with that before . I think it it works OK .
PhD B: Mm - hmm .
PhD A: We 've always had usually we used the symmetric windows
Professor C: So .
PhD A: but I don't think Professor C: Yeah , but we 've but we played a little bit with with asymmetric , guys .
PhD A: Yeah .
Professor C: You can do it . So . So , that 's what that 's what you 're busy with , s messing around with this ,
PhD B: Uh , yeah .
Professor C: yeah . And , uh ,

Summary: The team was looking at existing models and thinking about the various bugs that they had to deal with. The frequency differences in the FIR filters was one. Another thing they wanted to look into was handling additive noise, which they had not dealt with explicitly.