PhD D: g And it gave me like , uh , one point One more than one percent relative improvement . So , from fifty - three point six it went to fifty f four point eight . So it 's , like , only slightly more than a percent improvement ,
Professor B: Mm - hmm .
PhD D: just like Which means that it 's it 's doing a slightly better job than the previous VAD ,
Professor B: Mm - hmm .
PhD D: uh , at a l lower delay .
Professor B: Mm - hmm .
PhD D: Um , so , um Professor B: But i d I 'm sorry ,
PhD D: so u
Professor B: does it still have the median filter stuff ?
PhD D: It still has the median filter .
Professor B: So it still has most of the delay ,
PhD D: So Professor B: it just doesn't PhD D: Yeah , so d with the delay , that 's gone is the input , which is the sixty millisecond . The forty plus twenty .
Professor B: Well , w i
PhD D: At the input of the neural net you have this , uh , f nine frames of context plus the delta .
Professor B: Oh , plus the delta ,
PhD C: Mm - hmm .
Professor B: right . OK .
PhD D: Yeah . So that delay , plus the LDA .
Professor B: Mm - hmm .
PhD D: Uh , so the delay is only the forty millisecond of the noise cleaning , plus the hundred millisecond smoothing at the output .
Professor B: Mm - hmm . Mm - hmm .
PhD D: Um . So . Yeah . So the the di the biggest The problem f for me was to find a consistent threshold that works well across the different databases , because I t I try to make it work on tr SpeechDat - Car
Professor B: Mm - hmm .
PhD D: and it fails on TI - digits , or if I try to make it work on that it 's just the Italian or something , it doesn't work on the Finnish .
Professor B: Mm - hmm .
PhD D: So , um . So there are there was , like , some problem in balancing the deletions and insertions when I try different thresholds .
Professor B: Mm - hmm .
PhD D: So The I 'm still trying to make it better by using some other features from the after the p clean up maybe , some , uh , correlation auto - correlation or some s additional features of to mainly the improvement of the VAD . I 've been trying .
Professor B: Now this this this , uh , " before and after clean " , it sounds like you think that 's a good feature . That that , it you th think that the , uh the i it appears to be a good feature , right ?
PhD D: Mm - hmm .
Professor B: What about using it in the neural net ?
PhD D: Yeah .
PhD C: Yeah , eventually we could could just
PhD D: Yeah , so Yeah , so that 's the Yeah . So we 've been thinking about putting it into the neural net also .
Professor B: Yeah .
PhD D: Because they did that itself PhD C: Then you don't have to worry about the thresholds and PhD D: There 's a threshold and Yeah .
Professor B: Yeah .
PhD C: but just PhD D: Yeah . So that that 's , uh Professor B: Yeah . So if we if we can live with the latency or cut the latencies elsewhere , then then that would be a , uh , good thing .
PhD D: Yeah . Yeah .
Professor B: Um , anybody has anybody you guys or or Naren , uh , somebody , tried the , uh , um , second th second stream thing ? Uh .
PhD D: Oh , I just I just h put the second stream in place and , uh ran one experiment , but just like just to know that everything is fine .
Professor B: Uh - huh .
PhD D: So it was like , uh , forty - five cepstrum plus twenty - three mel log mel .
Professor B: Yeah .
PhD D: And and , just , like , it gave me the baseline performance of the Aurora , which is like zero improvement .
Professor B: Yeah . Yeah .
PhD D: So I just tried it on Italian just to know that everything is But I I didn't export anything out of it because it was , like , a weird feature set .
Professor B: Yeah .
PhD D: So .
Professor B: Yeah . Well , what I think , you know , would be more what you 'd want to do is is is , uh , put it into another neural net . Right ?
PhD C: Mm - hmm .
PhD D: Yeah , yeah , yeah , yeah .
Professor B: And then But , yeah , we 're we 're not quite there yet . So we have to figure out the neural nets , I guess .
PhD C: Yeah .
PhD D: The uh , other thing I was wondering was , um , if the neural net , um , has any because of the different noise con unseen noise conditions for the neural net , where , like , you train it on those four noise conditions , while you are feeding it with , like , a additional some four plus some f few more conditions which it hasn't seen , actually ,
PhD C: Mm - hmm .
PhD D: from the f f while testing .
PhD C: Yeah , yeah . Right .
PhD D: Um instead of just h having c uh , those cleaned up t cepstrum , sh should we feed some additional information , like The the We have the VAD flag . I mean , should we f feed the VAD flag , also , at the input so that it it has some additional discriminating information at the input ?
PhD C: Hmm - hmm ! Um Professor B: Wh - uh , the the VAD what ?
PhD D: We have the VAD information also available at the back - end .
Professor B: Uh - huh .
PhD D: So if it is something the neural net is not able to discriminate the classes Professor B: Yeah .
PhD D: I mean Because most of it is sil I mean , we have dropped some silence f We have dropped so silence frames ?
Professor B: Mm - hmm .
PhD D: No , we haven't dropped silence frames still .
PhD C: Uh , still not . Yeah .
PhD D: Yeah . So PhD C: Th
PhD D: the b b biggest classification would be the speech and silence . So , by having an additional , uh , feature which says " this is speech and this is nonspeech " , I mean , it certainly helps in some unseen noise conditions for the neural net .
PhD A: What Do y do you have that feature available for the test data ?
PhD D: Well , I mean , we have we are transferring the VAD to the back - end feature to the back - end . Because we are dropping it at the back - end after everything all the features are computed .
PhD A: Oh , oh , I see .
PhD D: So PhD A: I see .
PhD D: so the neural so that is coming from a separate neural net or some VAD .
PhD A: OK . OK .
PhD D: Which is which is certainly giving a
PhD A: So you 're saying , feed that , also , into the neural net .
PhD D: to Yeah . So it it 's an additional discriminating information .
PhD A: Yeah . Yeah . Right .
PhD D: So that Professor B: You could feed it into the neural net . The other thing {comment} you could do is just , um , p modify the , uh , output probabilities of the of the , uh , uh , um , neural net , tandem neural net , {comment} based on the fact that you have a silence probability .
PhD D: Mm - hmm .
Professor B: Right ?
PhD C: Mm - hmm .
Professor B: So you have an independent estimator of what the silence probability is , and you could multiply the two things , and renormalize .
PhD C: Yeah .
Professor B: Uh , I mean , you 'd have to do the nonlinearity part and deal with that . Uh , I mean , go backwards from what the nonlinearity would , you know would be .
PhD D: Through t to the soft max .
Professor B: But but , uh PhD C: Yeah , so maybe , yeah , when PhD A: But in principle wouldn't it be better to feed it in ? And let the net do that ?
Professor B: Well , u Not sure .
PhD A: Hmm .
Professor B: I mean , let 's put it this way . I mean , y you you have this complicated system with thousands and thousand parameters
PhD A: Yeah .
Professor B: and you can tell it , uh , " Learn this thing . " Or you can say , " It 's silence ! Go away ! " I mean , I mean , i Doesn't ? I think I think the second one sounds a lot more direct .
PhD A: What what if you Professor B: Uh .
PhD A: Right . So , what if you then , uh since you know this , what if you only use the neural net on the speech portions ?
Professor B: Well , uh ,
PhD C: That 's what PhD A: Well , I guess that 's the same . Uh , that 's similar .
Professor B: Yeah , I mean , y you 'd have to actually run it continuously ,
PhD A: But I mean I mean , train the net only on Professor B: but it 's @ @ Well , no , you want to train on on the nonspeech also , because that 's part of what you 're learning in it , to to to generate , that it 's it has to distinguish between .
PhD D: Speech .
PhD A: But I mean , if you 're gonna if you 're going to multiply the output of the net by this other decision , uh , would then you don't care about whether the net makes that distinction , right ?
Professor B: Well , yeah . But this other thing isn't perfect .
PhD A: Ah .
Professor B: So that you bring in some information from the net itself .
PhD A: Right , OK . That 's a good point .
Professor B: Yeah . Now the only thing that that bothers me about all this is that I I I The the fact i i It 's sort of bothersome that you 're getting more deletions .
PhD C: Yeah . But So I might maybe look at , is it due to the fact that um , the probability of the silence at the output of the network , is , uh ,
Professor B: Is too high .
PhD C: too too high or Professor B: Yeah . So maybe So PhD C: If it 's the case , then multiplying it again by i by something ?
PhD D: It may not be it Professor B: Yeah .
PhD C: Mm - hmm .
PhD D: Yeah , it it may be too it 's too high in a sense , like , everything is more like a , um , flat probability .
Professor B: Yeah .
PhD C: Oh - eee - hhh .
PhD D: So , like , it 's not really doing any distinction between speech and nonspeech PhD C: Uh , yeah .
PhD D: or , I mean , different among classes .
Professor B: Yeah .
PhD C: Mm - hmm .
PhD A: Be interesting to look at the Yeah , for the I wonder if you could do this . But if you look at the , um , highly mism high mismat the output of the net on the high mismatch case and just look at , you know , the distribution versus the the other ones , do you do you see more peaks or something ?
PhD C: Yeah . Yeah , like the entropy of the the output ,
PhD A: Yeah .
Professor B: Yeah , for instance .
PhD C: or Professor B: But I bu
PhD C: It it seems that the VAD network doesn't Well , it doesn't drop , uh , too many frames because the dele the number of deletion is reasonable . But it 's just when we add the tandem , the final MLP , and then Professor B: Yeah . Now the only problem is you don't want to ta I guess wait for the output of the VAD before you can put something into the other system ,
PhD C: u
Professor B: cuz that 'll shoot up the latency a lot , right ? Am I missing something here ?
PhD C: But PhD D: Mm - hmm .
PhD C: Yeah . Right .
Professor B: Yeah . So that 's maybe a problem with what I was just saying . But but I I guess PhD A: But if you were gonna put it in as a feature it means you already have it by the time you get to the tandem net , right ?
PhD D: Um , well . We w we don't have it , actually ,
Professor B: No .
PhD D: because it 's it has a high rate energy PhD A: Ah .
PhD D: the VAD has a Professor B: Yeah .
PhD A: OK .
Professor B: It 's kind of done in I mean , some of the things are , not in parallel , but certainly , it would be in parallel with the with a tandem net .
PhD A: Right .
Professor B: In time . So maybe , if that doesn't work , um But it would be interesting to see if that was the problem , anyway . And and and then I guess another alternative would be to take the feature that you 're feeding into the VAD , and feeding it into the other one as well .
PhD C: Mm - hmm .
Professor B: And then maybe it would just learn learn it better .
PhD C: Mm - hmm .
Professor B: Um But that 's Yeah , that 's an interesting thing to try to see , if what 's going on is that in the highly mismatched condition , it 's , um , causing deletions by having this silence probability up up too high ,
PhD C: Mm - hmm .
Professor B: at some point where the VAD is saying it 's actually speech .
PhD C: Yeah .
Professor B: Which is probably true .
PhD C: So , m
Professor B: Cuz Well , the V A if the VAD said since the VAD is is is right a lot , uh PhD C: Yeah .
Professor B: Hmm . Anyway . Might be .
PhD C: Mm - hmm .
Professor B: Yeah . Well , we just started working with it . But these are these are some good ideas I think .
PhD C: Mm - hmm . Yeah , and the other thing Well , there are other issues maybe for the tandem , like , uh , well , do we want to , w uh n Do we want to work on the targets ? Or , like , instead of using phonemes , using more context dependent units ?
PhD A: For the tandem net you mean ?
PhD C: Well , I 'm Yeah .
PhD A: Hmm .
PhD C: I 'm thinking , also , a w about Dan 's work where he he trained a network , not on phoneme targets but on the HMM state targets . And it was giving s slightly better results .
Professor B: Problem is , if you are going to run this on different m test sets , including large vocabulary ,
PhD C: Yeah . Yeah .
Professor B: um ,
PhD C: Uh Professor B: I think PhD C: Mmm . I was just thinking maybe about , like , generalized diphones , and come up with a a reasonable , not too large , set of context dependent units , and and Yeah . And then anyway we would have to reduce this with the KLT .
Professor B: Yeah .
PhD C: So . But I don't know .
Professor B: Yeah . Well , maybe . But I d I d it it i it 's all worth looking at ,
PhD C: Mm - hmm .
Professor B: but it sounds to me like , uh , looking at the relationship between this and the speech noise stuff is is is probably a key thing .
PhD C: Mm - hmm .
Professor B: That and the correlation between stuff .
PhD A: So if , uh if the , uh , high mismatch case had been more like the , uh , the other two cases {comment} in terms of giving you just a better performance , {comment} how would this number have changed ?
PhD C: Mm - hmm . Oh , it would be Yeah . Around five percent better , I guess . If if i
PhD A: y Like sixty ?
Professor B: Well , we don't know what 's it 's gonna be the TI - digits yet . He hasn't got the results back yet .
PhD C: Yeah . If you extrapolate the SpeechDat - Car well - matched and medium - mismatch , it 's around , yeah , maybe five .
PhD A: Uh - huh . Yeah . So this would be sixty - two ?
Professor B: Sixty - two .
PhD A: Which is Professor B: Yeah .
PhD C: Sixty - two , yeah .
PhD D: Somewhere around sixty , must be . Right ? Yeah .
PhD C: Well , it 's around five percent , because it 's s Right ? If everything is five percent .
PhD D: Yeah . Yeah .
PhD A: All the other ones were five percent ,
PhD C: Mm - hmm .
PhD A: the Professor B: Yeah .
PhD C: I d I d I just have the SpeechDat - Car right now , so PhD A: Yeah .
PhD C: It 's running it shou we should have the results today during the afternoon ,
PhD A: Hmm .
PhD C: but Well .
Professor B: Hmm . Well Um So I won't be here for PhD A: When When do you leave ?
Professor B: Uh , I 'm leaving next Wednesday . May or may not be in in the morning . I leave in the afternoon . Um ,
PhD A: But you 're Professor B: so I PhD A: are you you 're not gonna be around this afternoon ?
Professor B: Yeah .
PhD A: Oh .
Professor B: Oh , well . I 'm talking about next week . I 'm leaving leaving next Wednesday .
PhD A: Uh - huh .
Professor B: This afternoon uh Oh , right , for the Meeting meeting ? Yeah , that 's just cuz of something on campus .
PhD A: Ah , OK , OK .
Professor B: Yeah . But , um , yeah , so next week I won't , and the week after I won't , cuz I 'll be in Finland . And the week after that I won't . By that time you 'll be {comment} Uh , you 'll both be gone from here . So there 'll be no definitely no meeting on on September sixth . Uh ,
PhD A: What 's September sixth ?
Professor B: and Uh , that 's during Eurospeech .
PhD A: Oh , oh , right . OK .
Professor B: So , uh , Sunil will be in Oregon . Uh , Stephane and I will be in Denmark . Uh Right ? So it 'll be a few weeks , really , before we have a meeting of the same cast of characters . Um , but , uh I guess , just I mean , you guys should probably meet . And maybe Barry Barry will be around . And and then uh , uh , we 'll start up again with Dave and Dave and Barry and Stephane and us on the , uh , twentieth . No . Thirteenth ? About a month ?
PhD A: So , uh , you 're gonna be gone for the next three weeks or something ?
Professor B: I 'm gone for two and a half weeks starting starting next Wed - late next Wednesday .
PhD A: So that 's you won't be at the next three of these meetings . Is that right ?
Professor B: Uh , I won't it 's probably four because of is it three ? Let 's see , twenty - third , thirtieth , sixth . That 's right , next three . And the the third one won't probably won't be a meeting , cuz cuz , uh , Su - Sunil , Stephane , and I will all not be here .
PhD A: Oh , right . Right .
Professor B: Um Mmm . {comment} So it 's just , uh , the next two where there will be there , you know , may as well be meetings ,
PhD A: OK .
Professor B: but I just won't be at them . And then starting up on the thirteenth , {nonvocalsound} uh , we 'll have meetings again but we 'll have to do without Sunil here somehow .
PhD A: When do you go back ?
Professor B: So .
PhD D: Thirty - first , August .
Professor B: Yeah . Yeah . So . Cool .
PhD A: When is the evaluation ? November , or something ?
Professor B: Yeah , it was supposed to be November fifteenth . Has anybody heard anything different ?
PhD C: I don't know . The meeting in is the five and six of December . So PhD D: p s It 's like Yeah , it 's tentatively all full . Yeah .
PhD C: Mm - hmm .
PhD D: Uh , that 's a proposed date , I guess .
PhD C: Yeah , um so the evaluation should be on a week before or PhD A: Yeah .
Professor B: Yep . But , no , this is good progress . So . Uh OK .
PhD A: Should we do digits ?
Professor B: Guess we 're done . Digits ? Yep .
PhD A: OK .
Professor B: It 's a wrap .