Professor C: OK . So uh , he 's not here ,
PhD D: So .
Professor C: so you get to PhD D: Yeah , I will try to explain the thing that I did this this week during this week .
Professor C: Yeah .
PhD D: Well eh you know that I work I begin to work with a new feature to detect voice - unvoice .
PhD E: Mm - hmm .
PhD D: What I trying two MLP to to the with this new feature and the fifteen feature uh from the eh bus base system
PhD E: The the mel cepstrum ?
PhD D: No , satly the mes the Mel Cepstrum , the new base system the new base system .
PhD E: Oh the PhD D: Yeah , we PhD E: OK , the Aurora system .
PhD D: yeah the Aurora system with the new filter , VAD or something like that .
PhD E: OK .
PhD D: And I 'm trying two MLP , one one that only have t three output , voice , unvoice , and silence ,
Professor C: Mm - hmm .
PhD D: and other one that have fifty - six output . The probabilities of the allophone . And I tried to do some experiment of recognition with that and only have result with with the MLP with the three output . And I put together the fifteen features and the three MLP output . And , well , the result are li a little bit better , but more or less similar .
Professor C: Uh , I I 'm I 'm slightly confused .
PhD E: Hmm .
Professor C: What what feeds the uh the three - output net ?
PhD D: Voice , unvoice , and si
Professor C: No no , what feeds it ? What features does it see ?
PhD D: The feature the input ? The inputs are the fifteen the fifteen uh bases feature .
Professor C: Uh - huh .
PhD D: the with the new code . And the other three features are R , the variance of the difference between the two spectrum ,
Professor C: Uh - huh .
PhD D: the variance of the auto - correlation function , except the the first point , because half the height value is R - zero
Professor C: Mm - hmm . Mm - hmm . Mm - hmm . Mm - hmm .
PhD D: and also R - zero , the first coefficient of the auto - correlation function . That is like the energy with these three feature ,
Professor C: Right .
PhD D: also these three feature .
Professor C: You wouldn't do like R - one over R - zero or something like that ? I mean usually for voiced - unvoiced you 'd do yeah , you 'd do something you 'd do energy
PhD D: Yeah .
Professor C: but then you have something like spectral slope , which is you get like R - one ov over R - zero or something like that .
PhD D: Uh yeah .
PhD E: What are the R 's ?
Professor C: R correlations .
PhD E: I 'm sorry I missed it .
PhD D: No , R c No .
PhD E: Oh .
PhD D: Auto - correlation ? Yes , yes , the variance of the auto - correlation function that uses that
Professor C: Ye - Well that 's the variance , but if you just say " what is " I mean , to first order , um yeah one of the differences between voiced , unvoiced and silence is energy . Another one is but the other one is the spectral shape .
PhD D: Yeah , I I 'll The spectral shape ,
Professor C: Yeah , and so R - one over R - zero is what you typically use for that .
PhD D: yeah . No , I don't use that I can't use Professor C: No , I 'm saying that 's what people us typically use .
PhD D: Mmm .
Professor C: See , because it because this is this is just like a single number to tell you um " does the spectrum look like that or does it look like that " .
PhD D: Mm - hmm .
Grad A: Oh . R R R - zero .
Professor C: Right ?
PhD D: Mm - hmm .
Professor C: So if it 's if it 's um if it 's low energy uh but the but the spectrum looks like that or like that , it 's probably silence .
PhD D: Mm - hmm .
Professor C: Uh but if it 's low energy and the spectrum looks like that , it 's probably unvoiced .
PhD D: Yeah .
Professor C: So if you just if you just had to pick two features to determine voiced - unvoiced , you 'd pick something about the spectrum like uh R - one over R - zero , um and R - zero
PhD D: Mm - hmm , OK .
Professor C: or i i you know you 'd have some other energy measure and like in the old days people did like uh zero crossing counts .
PhD D: Yeah , yeah .
Professor C: Right . S S
PhD D: Well , I can also th use this .
Professor C: Yeah . Um ,
PhD D: Bec - because the result are a little bit better but we have in a point that everything is more or less the similar more or less similar .
Professor C: Yeah . But um
PhD D: It 's not quite better .
Professor C: Right , but it seemed to me that what you were what you were getting at before was that there is something about the difference between the original signal or the original FFT and with the filter which is what and the variance was one take uh on it .
PhD D: Yeah , I used this too .
Professor C: Right . But it it could be something else . Suppose you didn't have anything like that . Then in that case , if you have two nets , Alright , and this one has three outputs , and this one has f
PhD D: Mm - hmm .
Professor C: whatever , fifty - six , or something ,
PhD D: Mm - hmm .
Professor C: if you were to sum up the probabilities for the voiced and for the unvoiced and for the silence here , we 've found in the past you 'll do better at voiced - unvoiced - silence than you do with this one . So just having the three output thing doesn't doesn't really buy you anything . The issue is what you feed it .
PhD D: Yeah . Yeah , I have yeah .
Professor C: So uh
PhD D: No PhD E: So you 're saying take the features that go into the voiced - unvoiced - silence net and feed those into the other one , as additional inputs , rather than having a separate Professor C: w W well that 's another way .
PhD D: Yeah .
Professor C: That wasn't what I was saying but yeah that 's certainly another thing to do . No I was just trying to say if you b if you bring this into the picture over this , what more does it buy you ?
PhD E: Mmm .
Professor C: And what I was saying is that the only thing I think that it buys you is um based on whether you feed it something different . And something different in some fundamental way . And so the kind of thing that that she was talking about before , was looking at something uh ab um something uh about the difference between the the uh um log FFT uh log power uh and the log magnitude uh F F - spectrum uh and the um uh filter bank .
PhD D: Yeah .
Professor C: And so the filter bank is chosen in fact to sort of integrate out the effects of pitch and she 's saying you know trying So the particular measure that she chose was the variance of this m of this difference , but that might not be the right number .
PhD D: Mm - hmm . Maybe .
Professor C: Right ? I mean maybe there 's something about the variance that 's that 's not enough or maybe there 's something else that that one could use , but I think that , for me , the thing that that struck me was that uh you wanna get something back here , so here 's here 's an idea . uh What about it you skip all the all the really clever things , and just fed the log magnitude spectrum into this ?
PhD D: Ah I 'm sorry .
Professor C: This is f You have the log magnitude spectrum , and you were looking at that and the difference between the filter bank and and c c computing the variance .
PhD D: Yeah . Mm - hmm .
Professor C: That 's a clever thing to do .
PhD D: Mm - hmm .
Professor C: What if you stopped being clever ? And you just took this thing in here because it 's a neural net and neural nets are wonderful
PhD D: Mm - hmm .
Professor C: and figure out what they can what they most need from things , and I mean that 's what they 're good at .
PhD D: Yeah .
Professor C: So I mean you 're you 're you 're trying to be clever and say what 's the statistic that should we should get about this difference but uh in fact , you know maybe just feeding this in or or feeding both of them in
PhD E: Hmm .
Professor C: you know , another way , saying let it figure out what 's the what is the interaction , especially if you do this over multiple frames ?
PhD D: Mm - hmm .
Professor C: Then you have this over time , and and both kinds of measures and uh you might get uh something better .
PhD D: Mm - hmm .
Professor C: Um .
PhD E: So so don't uh don't do the division , but let the net have everything .
Professor C: That 's another thing you could do yeah . Yeah .
PhD D: Yeah .
Professor C: Um . I mean , it seems to me , if you have exactly the right thing then it 's better to do it without the net because otherwise you 're asking the net to learn this you know , say if you wanted to learn how to do multiplication .
PhD E: Mm - hmm .
Professor C: I mean you could feed it a bunch of s you could feed two numbers that you wanted to multiply into a net and have a bunch of nonlinearities in the middle and train it to get the product of the output and it would work . But , it 's kind of crazy , cuz we know how to multiply and you you 'd be you know much lower error usually if you just multiplied it out . But suppose you don't really know what the right thing is . And that 's what these sort of dumb machine learning methods are good at . So . Um . Anyway . It 's just a thought .
PhD E: How long does it take , Carmen , to train up one of these nets ?
PhD D: Oh , not too much .
PhD E: Yeah .
PhD D: Mmm , one day or less .
PhD E: Hmm .
Professor C: Yeah , it 's probably worth it .
Grad A: What are what are your f uh frame error rates for for this ?
PhD D: Eh fifty - f six uh no , the frame error rate ?
Grad A: O
PhD D: Fifty - six I think .
Professor C: Is that maybe that 's accuracy ?
PhD D: Percent .
Grad A: Fif - fifty - six percent accurate for v voice - unvoice
PhD D: The accuracy . Mm - hmm . No for , yes f I don't remember for voice - unvoice ,
Grad A: Oh , OK .
PhD D: maybe for the other one .
Grad A: OK .
Professor C: Yeah , voiced - unvoiced hopefully would be a lot better .
PhD D: for voiced . I don't reme
Grad A: Should be in nineties somewhere .
PhD D: Better . Maybe for voice - unvoice .
Grad A: Right .
PhD D: This is for the other one . I should I can't show that .
Grad A: OK .
PhD D: But I think that fifty - five was for the when the output are the fifty - six phone .
Grad A: Mm - hmm .
PhD D: That I look in the with the other nnn the other MLP that we have are more or less the same number . Silence will be better but more or less the same .
Professor C: I think at the frame level for fifty - six that was the kind of number we were getting for for uh um reduced band width uh stuff .
PhD D: I think that I I I think that for the other one , for the three output , is sixty sixty - two , sixty three more or less .
Grad A: Mm - hmm .
Professor C: That 's all ?
PhD D: It 's Yeah .
Professor C: That 's pretty bad .
PhD D: Yeah , because it 's noise also .
Grad A: Oh yeah .
Professor C: Aha !
PhD D: And we have
Professor C: Aha ! Yeah . Yeah . OK .
PhD D: I know .
Professor C: But even i in Oh yeah , in training . Still , Uh . Well actually , so this is a test that you should do then . Um , if you 're getting fifty - six percent over here , uh that 's in noise also , right ?
PhD D: Yeah , yeah , yeah .
Professor C: Oh OK . If you 're getting fifty - six here , try adding together the probabilities of all of the voiced phones here and all of the unvoiced phones
PhD D: will be Professor C: and see what you get then .
PhD D: Yeah .
Professor C: I bet you get better than sixty - three .
PhD D: Well I don't know , but I th I I think that we I have the result more or less . Maybe . I don't know . I don't I 'm not sure but I remember @ @ that I can't show that .
Professor C: OK , but that 's a That is a a good check point , you should do that anyway ,
PhD D: Yeah .
Professor C: OK ? Given this this uh regular old net that 's just for choosing for other purposes , uh add up the probabilities of the different subclasses and see see how well you do . Uh and that you know anything that you do over here should be at least as good as that .
PhD D: Mm - hmm .
Professor C: OK .
PhD D: I will do that . But PhD E: The targets for the neural net , uh , they come from forced alignments ?
PhD D: Uh , {comment} no .
Grad A: TIMIT canonical ma mappings .
PhD D: TIMIT .
Professor C: Oh . So , this is trained on TIMIT .
PhD E: Ah ! OK .
Grad A: Yeah , noisy TIMIT .
PhD D: Yeah .
Professor C: OK .
PhD D: Yeah this for TIMIT .
Professor C: But noisy TIMIT ?
Grad A: Right .
PhD D: Noisy TIMIT . We have noisy TIMIT with the noise of the the TI - digits . And now we have another noisy TIMIT also with the noise of uh Italian database .
Professor C: I see . Yeah . Well there 's gonna be it looks like there 's gonna be a noisy uh some large vocabulary noisy stuff too . Somebody 's preparing .
PhD E: Really ?
Professor C: Yeah . I forget what it 'll be , resource management , Wall Street Journal , something . Some some read task actually , that they 're preparing .
Grad A: Hmm !
PhD E: For what For Aurora ?
Professor C: Yeah .
PhD E: Oh !
Professor C: Yeah , so the uh Uh , the issue is whether people make a decision now based on what they 've already seen , or they make it later . And one of the arguments for making it later is let 's make sure that whatever techniques that we 're using work for something more than than connected digits .
PhD E: Hmm .
Professor C: So .
PhD E: When are they planning When would they do that ?
Professor C: Mmm , I think late uh I think in the summer sometime .
PhD E: Hmm .
Professor C: So . OK , thanks .

Summary: The existing net for voice-unvoice had three outputs, voice, unvoice, and silence. It took fifteen features as inputs. The team discussed how energy measures could be incorporated to improve performance on this task. The nets took around a day to train, so the team could run more experiments. The current performance on the task was unsatisfactory.