A: so what is different between this and and what you C: So the previous syst
C: It 's based on the system that has a fifty - three point sixty - six percent improvement .
C: It 's the same system .
C: The only thing that changed is the n a p eh a es the estimation of the silence probabilities .
A: Ah .
A: OK .
C: Which now is based on , uh , cleaned features .
B: And , it 's a l it 's a lot better .
A: Wow .
C: Yeah .
C: Um B: That 's great .
C: So it 's it 's not bad ,
C: but the problem is still that the latency is too large .
B: What 's the latency ?
C: Because C: um C: the the latency of the VAD is two hundred and twenty milliseconds .
C: And , uh , the VAD is used uh , i for on - line normalization ,
C: and it 's used before the delta computation .
C: So if you add these components it goes t to a hundred and seventy ,
C: right ?
B: I I 'm confused .
B: You started off with two - twenty and you ended up with one - seventy ?
C: With two an two hundred and seventy .
B: Two - seventy .
C: If C: Yeah ,
C: if you add the c delta comp delta computation
B: Oh .
C: which is done afterwards .
C: Um B: So it 's two - twenty .
B: I the is this are these twenty - millisecond frames ?
B: Is that why ?
B: Is it after downsampling ?
C: The two - twenty is one hundred milliseconds for the um B: or C: No , it 's forty milliseconds for t for the , uh , uh , cleaning of the speech .
C: Um C: then there is , um , the neural network which use nine frames .
C: So it adds forty milliseconds .
B: a
B: OK .
C: Um ,
C: after that , um , you have the um , filtering of the silence probabilities .
C: Which is a million filter
C: it ,
C: and it creates a one hundred milliseconds delay .
C: So , um D: Plus there is a delta at the input .
C: Yeah ,
C: and there is the delta at the input
C: which is ,
B: One hundred milliseconds for smoothing .
C: um C: So it 's @ @ B: Uh ,
D: It 's like forty plus forty plus B: median .
C: Mmm .
C: Forty B: And then forty C: This forty plus twenty , plus one hundred .
B: forty p
C: Uh D: So it 's two hundred actually .
C: Yeah , there are twenty that comes from C: There is ten that comes from the LDA filters also .
D: Oh , OK .
C: Right ?
C: Uh , so it 's two hundred and ten ,
C: yeah .
D: If you are using B: Uh C: Plus the frame ,
D: t If you are using three frames C: so it 's two - twenty .
D: If you are phrasing f <comment> using three frames , it is thirty here for delta .
C: Yeah , I think it 's it 's five frames ,
D: So five frames , that 's twenty .
C: but .
D: OK ,
D: so it 's who un <comment> two hundred and ten .
B: Uh , p Wait a minute .
B: It 's forty  forty for the for the cleaning of the speech ,
C: So .
C: Forty cleaning .
B: forty for the I N ANN ,
B: a hundred for the smoothing .
C: Yeah .
B: Well , but at ten ,
C: Twenty for the delta .
D: At th <nonvocalsound> At the input .
B: Twenty for delta .
D: I mean , that 's at the input to the net .
C: Yeah .
B: Delta at input to net ?
D: And there i
C: Yeah .
D: Yeah .
D: So it 's like s five , six cepstrum plus delta
D: at nine nine frames of B: And then ten milliseconds for D: Fi - There 's an LDA filter .
B: ten milliseconds for LDA filter ,
B: and t and ten another ten milliseconds you said for the frame ?
C: For the frame I guess .
C: I computed two - twenty C: Yeah , well , it 's C: I guess it 's for the fr the B: OK .
B: And then there 's delta besides that ?
C: So this is the features that are used by our network
C: and then afterwards , you have to compute the delta on the , uh , main feature stream ,
C: which is um , delta and double - deltas ,
B: OK .
C: which is fifty milliseconds .
B: Yeah .
B: No , I mean , the B: after the noise part , the forty the the other hundred and eighty B: Well , I mean ,
B: Wait a minute .
B: Some of this is , uh is , uh is in parallel , isn't it ?
B: I mean , the LDA B: Oh , you have the LDA as part of the V D - uh , VAD ?
B: Or C: The VAD use , uh , LDA filtered features also .
B: Oh , it does ?
C: Mm - hmm .
B: Ah .
B: So in that case there isn't too much in parallel .
B: Uh C: No .
C: There is , um , just downsampling , upsampling , and the LDA .
B: Um , so the delta at the end is how much ?
C: It 's fifty .
D: It 's B: Fifty .
B: Alright .
B: So C: But well , we could probably put the delta , um , before on - line normalization .
C: It should not that make a big difference ,
A: What if you used a smaller window for the delta ?
C: because A: Could that help a little bit ?
A: I mean , I guess there 's a lot of things you could do to C: Yeah .
B: Yeah .
C: Yeah ,
C: but , nnn B: So
B: Yeah .
B: So if you if you put the delta before the , uh , ana on - line If C: Mm - hmm .
B: Yeah B: uh then then it could go in parallel .
C: Cuz i
B: And then y then you don't have that additive D: Yep .
C: Yeah ,
C: cuz the time constant of the on - line normalization is pretty long compared to the delta window ,
B: OK .
C: so .
C: It should not make B: OK .
B: And you ought to be able to shove tw , uh sh uh pull off twenty milliseconds from somewhere else to get it under two hundred ,
B: right ?
B: I mean C: Mm - hmm .
A: Is two hundred the d
B: The hundred milla
B: mill a hundred milliseconds for smoothing is sort of an arbitrary amount .
B: It could be eighty
B: and and probably do @ @ C: Yeah ,
C: yeah .
A: i a hun
A: uh A: Wh - what 's the baseline you need to be under ?
A: Two hundred ?
B: Well , we don't know .
B: They 're still arguing about it .
A: Oh .
B: I mean , if it 's two if if it 's , uh if it 's two - fifty , then we could keep the delta where it is if we shaved off twenty .
B: If it 's two hundred , if we shaved off twenty , we could we could , uh , meet it by moving the delta back .
A: So , how do you know that what you have is too much if they 're still deciding ?
B: Uh , we don't ,
B: but it 's just B: I mean , the main thing is that since that we got burned last time , and you know , by not worrying about it very much , we 're just staying conscious of it .
A: Uh - huh .
A: Oh , OK ,
A: I see .
B: And so , th
B: I mean , if if if a week before we have to be done someone says , " Well , you have to have fifty milliseconds less than you have now " , it would be pretty frantic around here .
B: So A: Ah , OK .
B: Uh A: But still , that 's that 's a pretty big , uh , win .
A: And it doesn't seem like you 're in terms of your delay , you 're , uh , that B: He added a bit on ,
C: Hmm .
B: I guess , because before we were we were had were able to have the noise , uh , stuff , uh , and the LVA be in parallel .
B: And now he 's he 's requiring it to be done first .
C: Well , but I think the main thing , maybe , is the cleaning of the speech , which takes forty milliseconds or so .
B: Right .
B: Well , so you say C: And B: let 's say ten milliseconds seconds for the LDA .
C: and but the LDA is , well , pretty short right now .
B: Well , ten .
C: Yeah .
B: And then forty for the other .
D: Yeah , the LDA LDA we don't know , is , like is it very crucial for the features , right ?
C: No .
C: I just D: Yeah .
C: This is the first try .
C: I mean , I maybe the LDA 's not very useful then .
B: Right ,
D: S s h
B: so you could start pulling back ,
B: but D: Yeah ,
D: l
B: But I think you have B: I mean , you have twenty for delta computation
B: which y now you 're sort of doing twice ,
B: right ?
B: But yo w were you doing that before ?
C: Mmm .
D: On the in the C: Well , in the proposal , um , the input of the VAD network were just three frames , I think .
D: Mm - hmm .
D: Just D: Yeah , just the static , no delta .
C: Uh , static features .
B: Right .
B: So , what you have now is fort uh , forty for the the noise , twenty for the delta , and ten for the LDA .
B: That 's seventy milliseconds of stuff which was formerly in parallel ,
B: right ?
B: So I think ,
C: Mm - hmm .
B: you know , that 's that 's the difference as far as the timing ,
C: Yeah .
B: right ?
B: Um ,
B: and you could experiment with cutting various pieces of these back a bit ,
B: but B: I mean , we 're s we 're not we 're not in terrible shape .
A: Yeah ,
A: that 's what it seems like to me .
C: Mm - hmm .
B: Yeah .
A: It 's pretty good .
B: It 's it 's not like it 's adding up to four hundred milliseconds or something .
A: Where where is this where is this fifty - seven point O two in in comparison to the last evaluation ?
B: Well , it 's I think it 's better than anything , uh , anybody got .
C: Yeah .
A: Oh , is that right ?
C: The best was fifty - four point five .
B: Yeah .
D: Point s
A: Oh .
B: Yeah .
C: And our system was forty - nine ,
B: Uh
C: but with the neural network .
A: Wow .
A: So this is almost ten percent .
B: With the f with the neural net .
B: Yeah ,
D: Yeah ,
B: and r and C: It would
D: so this is this is like the first proposal .
D: The proposal - one .
D: It was forty - four , actually .
B: Yeah .
B: Yeah .
B: And we still don't have the neural net in .
B: So so it 's A: Wow .
B: You know .
B: So it 's B: We 're we 're doing better .
B: I mean , we 're getting better recognition .
A: This is this is really good .
B: I mean , I 'm sure other people working on this are not sitting still either ,
B: but B: but A: Yeah .
B: but , uh B: Uh ,
B: I mean , the important thing is that we learn how to do this better ,
B: and , you know .
B: So .
B: Um ,
B: Yeah .
B: So , our ,
B: um B: Yeah , you can see the kind of kind of numbers that we 're having , say , on SpeechDat - Car
B: which is a hard task ,
B: cuz it 's really , um I think it 's just sort of sort of reasonable numbers , starting to be .
C: Mm - hmm .
B: I mean , it 's still terri
C: Yeah , even for a well - matched case it 's sixty percent error rate reduction ,
C: which is B: Yeah .
B: Yeah .
B: Probably half .
B: Good !
C: Um ,
C: Yeah .
C: So actually , this is in between what we had with the previous VAD and what Sunil did with an IDL VAD .
C: Which gave sixty - two percent improvement , right ?
D: Yeah , it 's almost that .
C: So D: It 's almost an average
D: somewhere around C: Yeah .
D: Yeah .
A: What was that ?
A: Say that last part again ?
C: So , if you use , like , an IDL VAD , uh , for dropping the frames ,
D: o o
D: Or the best we can get .
C: the best that we can get i That means that we estimate the silence probability on the clean version of the utterances .
C: Then you can go up to sixty - two percent error rate reduction , globally .
A: Mmm .
C: Mmm C: Yeah .
A: So that would be even That wouldn't change this number down here to sixty - two ?
C: Yeah .
B: Yeah .
B: So you you were get
C: If you add a g good v very good VAD , that works as well as a VAD working on clean speech ,
A: Yeah .
A: Yeah .
C: then you wou you would go A: So that 's sort of the best you could hope for .
C: Mm - hmm .
B: Probably . Yeah .
A: I see .
B: So fi si fifty - three is what you were getting with the old VAD .
C: Yeah .
B: And , uh B: and sixty - two with the the , you know , quote , unquote , cheating VAD .
B: And fifty - seven is what you got with the real VAD .
C: Mm - hmm .
B: That 's great .
C: Uh , yeah , the next thing is , I started to play C: Well , I don't want to worry too much about the delay ,
C: no .
C: Maybe it 's better to wait
B: OK .
C: for the decision
B: Yeah .
C: from the committee .
C: Uh , but I started to play with the , um ,  uh , tandem neural network .
C: Mmm
C: I just did the configuration that 's very similar to what we did for the February proposal .
C: And C: Um .
C: So . There is a f a first feature stream that use uh straight MFCC features .
B: Mm - hmm .
C: Well , these features actually .
C: And the other stream is the output of a neural network , using as input , also , these , um , cleaned MFCC .
C: Um C: I don't have the comp
A: Those are th those are th what is going into the tandem net ?
C: Mmm ?
A: Those two ?
C: So there is just this feature stream , <comment> the fifteen MFCC plus delta and double - delta .
B: No .
A: Yeah ?
C: Um , so it 's makes forty - five features <comment> that are used as input to the HTK .
C: And then , there is there are more inputs that comes from the tandem MLP .
A: Oh , oh .
A: OK .
B: Yeah ,
A: I see .
B: h he likes to use them both ,
B: cuz then it has one part that 's discriminative ,
A: Uh - huh .
C: Yeah .
C: Um B: one part that 's not .
A: Right .
A: OK .
C: So , um ,
C: uh , yeah .
C: Right now it seems that i I just tested on SpeechDat - Car while the experiment are running on your on TI - digits .
C: Well , it improves on the well - matched and the mismatched conditions ,
C: but it get worse on the highly mismatched .
C: Um ,
A: Compared to these numbers ?
C: Compared to these numbers , yeah .
C: Um ,
C: like , on the well - match and medium mismatch , the gain is around five percent relative ,
B: y
C: but it goes down a lot more , like fifteen percent on the HM case .
B: You 're just using the full ninety features ?
C: The B: Y you have ninety features ?
C: i
C: I have ,
C: um C: From the networks , it 's twenty - eight .
C: So B: And from the other side it 's forty - five .
C: So , d i It 's forty - five .
B: So it 's you have seventy - three features ,
C: Yeah .
B: and you 're just feeding them like that .
C: Yeah .
C: Mm - hmm .
B: There isn't any KLT or anything ?
C: There 's a KLT after the neural network , as as before .
A: That 's how you get down to twenty - eight ?
C: Yeah .
A: Why twenty - eight ?
C: I don't know .
C: Uh .
A: Oh .
C: It 's i i i It 's because it 's what we did for the first proposal .
C: We tested , uh , trying to go down
A: Ah .
B: It 's a multiple of seven .
C: and
D: Yeah .
C: Yeah .
C: So C: Um .
D: Yeah .
C: I wanted to do something very similar to the proposal as a first first try .
D: Yeah .
A: I see .
B: Yeah .
A: Yeah .
A: That makes sense .
C: But we have to for sure , we have to go down ,
C: because the limit is now sixty features .
C: So ,
B: Yeah .
C: uh ,
C: we have to find a way to decrease the number of features .
C: Um A: So , it seems funny that I don't know , maybe I don't u quite understand everything , <comment> but that adding features A: I guess I guess if you 're keeping the back - end fixed .
A: Maybe that 's it .
A: Because it seems like just adding information shouldn't give worse results .
A: But I guess if you 're keeping the number of Gaussians fixed in the recognizer , then B: Well , yeah .
C: Mmm .
B: But , I mean , just in general , adding information B: Suppose the information you added , well , was a really terrible feature and all it brought in was noise .
A: Yeah .
B: Right ?
B: So so , um B: Or or suppose it wasn't completely terrible ,
B: but it was completely equivalent to another one feature that you had ,
B: except it was noisier .
A: Uh - huh .
B: Right ?
B: In that case you wouldn't necessarily expect it to be better at all .
A: Oh , yeah , I wasn't necessarily saying it should be better .
A: I 'm just surprised that you 're getting fifteen percent relative worse on the wel
B: Uh - huh .
C: But it 's worse .
B: On the highly mismatched condition .
A: On the highly mismatch .
C: Yeah , I A: Yeah .
B: So , " highly mismatched condition " means that in fact your training is a bad estimate of your test .
C: Uh - huh .
B: So having having , uh , a g a l a greater number of features , if they aren't maybe the right features that you use , certainly can e can easily , uh , make things worse .
B: I mean , you 're right .
B: If you have if you have , uh , lots and lots of data , and you have and your your your training is representative of your test , then getting more sources of information should just help .
B: But but it 's It doesn't necessarily work that way .
A: Huh .
C: Mm - hmm .
B: So I wonder ,
B: um ,
B: Well , what 's your what 's your thought about what to do next with it ?
C: Um ,
C: I don't know .
C: I 'm surprised ,
C: because I expected the neural net to help more when there is more mismatch , as it was the case for the B: Mm - hmm .
D: So , was the training set same as the p the February proposal ?
D: OK .
C: Yeah , it 's the same training set ,
C: so it 's TIMIT with the TI - digits ' , uh , noises , uh , added .
B: Mm - hmm .
C: Um B: Well , we might uh , we might have to experiment with , uh better training sets .
B: Again .
C: Mm - hmm .
B: But ,
B: I The other thing is , I mean , before you found that was the best configuration , but you might have to retest those things now that we have different The rest of it is different ,
B: right ?
B: So ,
B: um ,
B: uh ,
B: For instance , what 's the effect of just putting the neural net on without the o other other path ?
C: Mm - hmm .
B: I mean , you know what the straight features do .
C: Yeah .
B: That gives you this .
C: Mm - hmm .
B: You know what it does in combination .
B: You don't necessarily know what A: What if you did the A: Would it make sense to do the KLT on the full set of combined features ?
A: Instead of just on the C: Yeah .
C: I g I guess .
C: Um .
C: The reason I did it this ways is that in February , it we we tested different things like that ,
C: so , having two KLT , having just a KLT for a network , or having a global KLT .
A: Oh , I see .
C: And A: So you tried the global KLT before
C: Well C: Yeah .
A: and it didn't really C: And , uh , th
C: Yeah .
C: The differences between these configurations were not huge ,
A: I see .
C: but it was marginally better with this configuration .
A: Uh - huh .
A: Uh - huh .
B: But , yeah , that 's obviously another thing to try ,