B: So you were just trying B: I think you r
B: I mean , this is it doesn't have the LDA aspect
B: but th as far as the orthogonalizing transformation , you were trying that at one point , right ?
C: Mm - hmm .
C: Mm - hmm .
B: I think you were .
C: Yeah .
B: Does something .
B: It doesn't work as well .
B: Yeah .
B: Yeah .
D: So , yeah ,
D: I 've been exploring a parallel VAD without neural network
D: with , like , less latency using SNR and energy , um , after the cleaning up .
D: So what I 'd been trying was , um ,
D: uh D: After the b after the noise compensation , n I was trying t to f find a f feature based on the ratio of the energies , that is , cl after clean and before clean .
D: So that if if they are , like , pretty c close to one , which means it 's speech .
D: And if it is n if it is close to zero , which is So it 's like a scale @ @ probability value .
D: So I was trying , uh , with full band and multiple bands ,
D: m ps uh separating them to different frequency bands
D: and deriving separate decisions on each bands , and trying to combine them .
D: Uh ,
D: the advantage being like it doesn't have the latency of the neural net if it if it can
B: Mm - hmm .
D: g And it gave me like , uh , one point One more than one percent relative improvement .
D: So , from fifty - three point six it went to fifty f four point eight .
D: So it 's , like , only slightly more than a percent improvement ,
B: Mm - hmm .
D: just like D: Which means that it 's it 's doing a slightly better job than the previous VAD ,
B: Mm - hmm .
D: uh , at a l lower delay .
B: Mm - hmm .
D: Um ,
D: so , um D: so u
B: But B: i d I 'm sorry ,
B: does it still have the median filter stuff ?
D: It still has the median filter .
B: So it still has most of the delay ,
D: So B: it just doesn't D: Yeah ,
D: so d with the delay , that 's gone is the input , which is the sixty millisecond .
D: The forty plus twenty .
D: At the input of the neural net you have this , uh , f nine frames of context plus the delta .
B: Well , w i
C: Mm - hmm .
B: Oh , plus the delta ,
B: right .
D: Yeah .
B: OK .
D: So that delay , plus the LDA .
B: Mm - hmm .
D: Uh , so the delay is only the forty millisecond of the noise cleaning , plus the hundred millisecond smoothing at the output .
B: Mm - hmm .
B: Mm - hmm .
D: Um .
D: So . Yeah .
D: So the the di the biggest D: The problem f for me was to find a consistent threshold that works well across the different databases ,
D: because I t I try to make it work on tr SpeechDat - Car
B: Mm - hmm .
D: and it fails on TI - digits ,
D: or if I try to make it work on that it 's just the Italian or something , it doesn't work on the Finnish .
B: Mm - hmm .
D: So ,
D: um .
D: So there are there was , like , some problem in balancing the deletions and insertions when I try different thresholds .
B: Mm - hmm .
D: So D: The D: I 'm still trying to make it better by using some other features from the after the p clean up D: maybe , some , uh , correlation auto - correlation or some s additional features of to mainly the improvement of the VAD . I 've been trying .
B: Now this this this , uh , " before and after clean " , it sounds like you think that 's a good feature .
B: That that , it you th think that the , uh the i it appears to be a good feature , right ?
D: Mm - hmm .
D: Yeah .
B: What about using it in the neural net ?
C: Yeah ,
C: eventually we could could just
D: Yeah ,
D: so Yeah ,
D: so that 's the D: Yeah .
D: So we 've been thinking about putting it into the neural net also .
B: Yeah .
D: Because they did that itself C: Then you don't have to worry about the thresholds and D: There 's a threshold
D: and Yeah .
D: Yeah .
B: Yeah .
D: So that that 's , uh C: but just B: Yeah .
B: So if we if we can live with the latency or cut the latencies elsewhere , then then that would be a , uh , good thing .
D: Yeah .
D: Yeah .
B: Um , anybody has anybody you guys or or Naren , uh , somebody , tried the , uh , um , second th second stream thing ?
D: Oh , I just I just h put the second stream in place and , uh ran one experiment ,
B: Uh .
D: but just like just to know that everything is fine .
B: Uh - huh .
D: So it was like , uh , forty - five cepstrum plus twenty - three mel log mel .
B: Yeah .
D: And and , just , like , it gave me the baseline performance of the Aurora ,
D: which is like zero improvement .
B: Yeah .
B: Yeah .
D: So I just tried it on Italian just to know that everything is D: But I I didn't export anything out of it
D: because it was , like , a weird feature set .
B: Yeah .
D: So .
B: Yeah .
B: Well , what I think , you know , would be more what you 'd want to do is is is , uh , put it into another neural net .
D: Yeah , yeah , yeah , yeah .
C: Mm - hmm .
B: Right ?
B: And then B: But ,
B: yeah , we 're we 're not quite there yet .
B: So we have to figure out the neural nets , I guess .
C: Yeah .
D: The uh , other thing I was wondering was , um , if the neural net , um , has any because of the different noise con unseen noise conditions for the neural net ,
D: where , like , you train it on those four noise conditions , while you are feeding it with , like , a additional some four plus some f few more conditions which it hasn't seen , actually ,
C: Mm - hmm .
D: from the f f while testing .
C: Yeah , yeah .
C: Right .
D: Um D: instead of just h having c uh , those cleaned up t cepstrum , sh should we feed some additional information , like The the D: We have the VAD flag .
D: I mean , should we f feed the VAD flag , also , at the input so that it it has some additional discriminating information at the input ?
C: Hmm - hmm !
C: Um B: Wh - uh , the the VAD what ?
D: We have the VAD information also available at the back - end .
B: Uh - huh .
D: So if it is something the neural net is not able to discriminate the classes B: Yeah .
D: I mean D: Because most of it is sil
D: I mean , we have dropped some silence f We have dropped so silence frames ?
B: Mm - hmm .
D: No , we haven't dropped silence frames still .
C: Uh , still not .
C: Yeah .
D: Yeah .
D: So D: the b b biggest classification would be the speech and silence .
C: Th
D: So , by having an additional , uh , feature which says " this is speech and this is nonspeech " , I mean , it certainly helps in some unseen noise conditions for the neural net .
A: What A: Do y do you have that feature available for the test data ?
D: Well , I mean , we have we are transferring the VAD to the back - end D: feature to the back - end .
D: Because we are dropping it at the back - end after everything all the features are computed .
A: Oh , oh , I see .
D: So A: I see .
D: so the neural D: so that is coming from a separate neural net or some VAD .
A: OK .
A: OK .
D: Which is which is certainly giving a
A: So you 're saying , feed that , also , into the neural net .
D: to D: Yeah . So it it 's an additional discriminating information .
A: Yeah .
A: Yeah .
A: Right .
D: So that B: You could feed it into the neural net .
B: The other thing <comment> you could do is just , um , p modify the , uh , output probabilities of the of the , uh , uh , um , neural net , tandem neural net , <comment> based on the fact that you have a silence probability .
D: Mm - hmm .
B: Right ?
C: Mm - hmm .
B: So you have an independent estimator of what the silence probability is ,
B: and you could multiply the two things , and renormalize .
B: Uh , I mean , you 'd have to do the nonlinearity part and deal with that .
C: Yeah .
B: Uh , I mean , go backwards from what the nonlinearity would , you know would be .
D: Through t to the soft max .
B: But but , uh C: Yeah , so C: maybe , yeah , when A: But in principle wouldn't it be better to feed it in ?
A: And let the net do that ?
B: Well , u Not sure .
B: I mean ,
A: Hmm .
B: let 's put it this way .
B: I mean , y you you have this complicated system with thousands and thousand parameters
A: Yeah .
B: and you can tell it , uh , " Learn this thing . "
B: Or you can say , " It 's silence !
B: Go away ! "
B: I mean ,
B: I mean , i
B: Doesn't ?
B: I think I think the second one sounds a lot more direct .
A: What A: what if you B: Uh .
A: Right .
A: So , what if you then , uh since you know this , what if you only use the neural net on the speech portions ?
B: Well , uh ,
C: That 's what A: Well , I guess that 's the same .
A: Uh , that 's similar .
B: Yeah ,
B: I mean , y you 'd have to actually run it continuously ,
B: but it 's @ @ A: But I mean I mean , train the net only on B: Well , no ,
B: you want to train on on the nonspeech also ,
B: because that 's part of what you 're learning in it ,
B: to to to generate , that it 's it has to distinguish between .
D: Speech .
A: But I mean , if you 're gonna if you 're going to multiply the output of the net by this other decision , uh , would then you don't care about whether the net makes that distinction , right ?
B: Well , yeah .
B: But this other thing isn't perfect .
A: Ah .
B: So that you bring in some information from the net itself .
A: Right ,
A: OK .
A: That 's a good point .
B: Yeah .
B: Now the only thing that that bothers me about all this is that I I I The the fact B: i i It 's sort of bothersome that you 're getting more deletions .
C: Yeah .
C: But C: So I might maybe look at ,
C: is it due to the fact that um , the probability of the silence at the output of the network , is ,
C: uh ,
B: Is too high .
C: too too high
C: or B: Yeah .
B: So maybe C: If it 's the case , then multiplying it again by i by something ?
B: So D: It may not be it B: Yeah .
D: Yeah ,
C: Mm - hmm .
D: it it may be too it 's too high in a sense , like , everything is more like a , um , flat probability .
B: Yeah .
C: Oh - eee - hhh .
D: So , like , it 's not really doing any distinction between speech and nonspeech C: Uh , yeah .
D: or , I mean , different among classes .
B: Yeah .
C: Mm - hmm .
A: Be interesting to look at the A: Yeah ,
A: for the A: I wonder if you could do this .
A: But if you look at the , um , highly mism high mismat the output of the net on the high mismatch case and just look at , you know , the distribution versus the the other ones , do you do you see more peaks or something ?
C: Yeah .
C: Yeah , like the entropy of the the output ,
A: Yeah .
C: or B: Yeah , for instance .
B: But I bu
C: It it seems that the VAD network doesn't Well , it doesn't drop , uh , too many frames
C: because the dele the number of deletion is reasonable .
C: But it 's just when we add the tandem , the final MLP , and then B: Yeah .
B: Now the only problem is you don't want to ta I guess wait for the output of the VAD before you can put something into the other system ,
C: u
B: cuz that 'll shoot up the latency a lot ,
B: right ?
B: Am I missing something here ?
C: But D: Mm - hmm .
C: Yeah .
C: Right .
B: Yeah .
B: So that 's maybe a problem with what I was just saying .
B: But B: but I I guess A: But if you were gonna put it in as a feature it means you already have it by the time you get to the tandem net ,
A: right ?
D: Um ,
D: well . We w we don't have it , actually ,
D: because it 's it has a high rate energy B: No .
D: the VAD has a A: Ah .
B: Yeah .
A: OK .
B: It 's kind of done in B: I mean , some of the things are , not in parallel ,
B: but certainly , it would be in parallel with the with a tandem net .
A: Right .
B: In time .
B: So maybe , if that doesn't work ,
B: um B: But it would be interesting to see if that was the problem , anyway .
B: And and and then I guess another alternative would be to take the feature that you 're feeding into the VAD , and feeding it into the other one as well .
C: Mm - hmm .
C: Mm - hmm .
B: And then maybe it would just learn learn it better .
B: Um B: But that 's B: Yeah , that 's an interesting thing to try to see , if what 's going on is that in the highly mismatched condition , it 's , um , causing deletions by having this silence probability up up too high ,
C: Mm - hmm .
B: at some point where the VAD is saying it 's actually speech .
C: Yeah .
C: So , m
B: Which is probably true .
B: Cuz B: Well , the V A
B: if the VAD said B: since the VAD is is is right a lot ,
B: uh C: Yeah .
B: Hmm .
B: Anyway .
B: Might be .
C: Mm - hmm .
B: Yeah .
B: Well , we just started working with it .
B: But these are these are some good ideas I think .
C: Mm - hmm .
C: Yeah , and the other thing C: Well , there are other issues maybe for the tandem ,
C: like , uh , well , do we want to , w uh n Do we want to work on the targets ?
C: Or , like , instead of using phonemes , using more context dependent units ?
A: For the tandem net you mean ?
C: Well , I 'm C: Yeah .
A: Hmm .
C: I 'm thinking , also , a w about Dan 's work where he he trained a network , not on phoneme targets but on the HMM state targets .
C: And C: it was giving s slightly better results .
B: Problem is , if you are going to run this on different m test sets , including large vocabulary ,
C: Yeah .
C: Yeah .
B: um ,
C: Uh C: Mmm .
B: I think C: I was just thinking maybe about , like , generalized diphones ,
C: and come up with a a reasonable , not too large , set of context dependent units ,
C: and C: and C: Yeah .
C: And then anyway we would have to reduce this with the KLT .
B: Yeah .
C: So .
C: But C: I don't know .
B: Yeah .
B: Well , maybe .
C: Mm - hmm .
B: But I d I d it it i it 's all worth looking at ,
B: but it sounds to me like , uh , looking at the relationship between this and the speech noise stuff is is is probably a key thing .
C: Mm - hmm .
B: That and the correlation between stuff .
A: So if ,
A: uh A: if the , uh , high mismatch case had been more like the , uh , the other two cases <comment> in terms of giving you just a better performance , <comment> how would this number have changed ?
C: Mm - hmm .
C: Oh , it would be C: Yeah .
C: Around five percent better , I guess .
C: If if i
A: y Like sixty ?
B: Well , we don't know what 's it 's gonna be the TI - digits yet .
B: He hasn't got the results back yet .
C: Yeah .
C: If you extrapolate the SpeechDat - Car well - matched and medium - mismatch , it 's around , yeah , maybe five .
A: Uh - huh .
A: Yeah .
A: So this would be sixty - two ?
B: Sixty - two .
C: Sixty - two ,
B: Yeah .
D: Somewhere around sixty , must be .
A: Which is C: yeah .
D: Right ?
D: Yeah .
C: Well , it 's around five percent , because it 's s
D: Yeah .
C: Right ?
C: If everything is five percent .
D: Yeah .
C: Mm - hmm .
A: All the other ones were five percent ,
A: the C: I d I d I just have the SpeechDat - Car right now ,
B: Yeah .
C: so A: Yeah .
C: It 's running C: it shou we should have the results today during the afternoon ,
A: Hmm .
C: but C: Well .
B: Hmm .
B: Well B: Um B: So I won't be here for A: When A: When do you leave ?
B: Uh , I 'm leaving next Wednesday .
B: May or may not be in in the morning .
B: I leave in the afternoon .
B: Um ,
B: so I A: But you 're A: are you A: you 're not gonna be around this afternoon ?
B: Yeah .
A: Oh .
B: Oh , well .
B: I 'm talking about next week .
B: I 'm leaving leaving next Wednesday .
A: Uh - huh .
B: This afternoon uh B: Oh , right ,
B: for the Meeting meeting ?
B: Yeah , that 's just cuz of something on campus .
A: Ah ,
A: OK ,
A: OK .
B: Yeah .
B: But , um ,
B: yeah ,
B: so next week I won't ,
B: and the week after I won't ,
B: cuz I 'll be in Finland .
B: And the week after that I won't .
B: By that time you 'll be <comment> Uh , you 'll both be gone from here .
B: So there 'll be no definitely no meeting on on September sixth .
B: Uh ,
B: and A: What 's September sixth ?
B: Uh , that 's during Eurospeech .
A: Oh , oh , right .
A: OK .
B: So , uh , Sunil will be in Oregon .
B: Uh , Stephane and I will be in Denmark .
B: Uh B: Right ?
B: So it 'll be a few weeks , really , before we have a meeting of the same cast of characters .
B: Um ,
B: but , uh B: I guess , just B: I mean , you guys should probably meet .