Professor B: OK . There was a start of some effort on something related to voicing or something . Is that ?
PhD A: Yeah . Um , yeah . So basically we try to ,  uh , find good features that could be used for voicing detection , uh , but it 's still , uh on the , um t
PhD F: Oh , well , I have the picture .
PhD A: we w basically we are still playing with Matlab to  to look at at what happened ,
PhD C: What sorts of PhD F: Yeah .
PhD A: and PhD C: what sorts of features are you looking at ?
PhD F: We have some PhD A: So we would be looking at , um , the variance of the spectrum of the excitation ,
PhD F: uh , um , this , this , and this .
PhD A: something like this , which is should be high for voiced sounds . Uh , we PhD C: Wait a minute . I what does that mean ? The variance of the spectrum of excitation .
PhD A: Yeah . So the So basically the spectrum of the excitation for a purely periodic sig signal shou sh
Professor B: OK . Yeah , w what yo what you 're calling the excitation , as I recall , is you 're subtracting the the , um the mel mel  mel filter , uh , spectrum from the FFT spectrum .
PhD A: e That 's right . Yeah . So Professor B: Right .
PhD A: Yeah .
PhD F: Mm - hmm .
PhD A: So we have the mel f filter bank , we have the FFT , so we just Professor B: So it 's it 's not really an excitation ,
PhD A: No .
Professor B: but it 's something that hopefully tells you something about the excitation .
PhD A: Yeah , that 's right .
Professor B: Yeah , yeah .
PhD A: Um Yeah .
PhD F: We have here some histogram ,
PhD A: E yeah ,
PhD F: but they have a lot of overlap .
PhD A: but it 's it 's still Yeah . So , well , for unvoiced portion we have something tha that has a mean around O point three , and for voiced portion the mean is O point fifty - nine . But the variance seem quite high .
PhD C: How do you know ?
PhD A: So Mmm .
PhD C: How did you get your voiced and unvoiced truth data ?
PhD A: We used , uh , TIMIT and we used canonical mappings between the phones
PhD F: Yeah . We , uh , use TIMIT on this ,
PhD A: and
PhD F: for PhD A: th Yeah .
PhD F: But if we look at it in one sentence , it apparently it 's good , I think .
PhD A: Yeah , but Yeah . Uh , so it 's noisy TIMIT . That 's right . Yeah .
Grad E: It 's noisy TIMIT .
PhD F: Yeah .
PhD A: It seems quite robust to noise , so when we take we draw its parameters across time for a clean sentence and then nois the same noisy sentence , it 's very close .
Professor B: Mm - hmm .
PhD A: Yeah . So there are there is this . There could be also the , um  something like the maximum of the auto - correlation function or which PhD C: Is this a a s a trained system ? Or is it a system where you just pick some thresholds ? Ho - how does it work ?
PhD A: Right now we just are trying to find some features . And ,
PhD C: Mm - hmm .
PhD A: uh Yeah . Hopefully , I think what we want to have is to put these features in s some kind of , um well , to to obtain a statistical model on these features and to or just to use a neural network and hopefully these features w would help PhD C: Because it seems like what you said about the mean of the the voiced and the unvoiced {comment} that seemed pretty encouraging .
PhD A: Mm - hmm .
Professor B: Well , yeah , except the variance was big .
PhD C: Right ?
PhD A: Yeah . Except the variance is quite high .
Professor B: Right ?
PhD C: Well , y
PhD A: Yeah .
PhD C: Well , y I I don't know that I would trust that so much because you 're doing these canonical mappings from TIMIT labellings .
PhD A: Uh - huh .
PhD C: Right ? So , really that 's sort of a cartoon picture about what 's voiced and unvoiced . So that could be giving you a lot of variance .
PhD A: Yeah .
PhD C: I mean , i it it may be that that you 're finding something good and that the variance is sort of artificial because of how you 're getting your truth .
PhD A: Mm - hmm .
Professor B: Yeah . But another way of looking at it might be that I mean , what w we we are coming up with feature sets after all . So another way of looking at it is that um , the mel cepstru mel spectrum , mel cepstrum , any of these variants , um , give you the smooth spectrum . It 's the spectral envelope . By going back to the FFT , you 're getting something that is more like the raw data . So the question is , what characterization and you 're playing around with this another way of looking at it is what characterization of the difference between the raw data and this smooth version is something that you 're missing that could help ? So , I mean , looking at different statistical measures of that difference , coming up with some things and just trying them out and seeing if you add them onto the feature vector does that make things better or worse in noise , where you 're really just i i the way I 'm looking at it is not so much you 're trying to f find the best the world 's best voiced - unvoiced , uh , uh , classifier ,
PhD C: Mm - hmm .
PhD A: Mmm .
Professor B: but it 's more that , you know , uh , uh , try some different statistical characterizations of that difference back to the raw data
PhD C: Right .
Professor B: and and m maybe there 's something there that the system can use .
PhD C: Right .
PhD A: Yeah . Yeah , but ther more obvious is that Yeah . The the more obvious is that that well , using the th the FFT , um , you just it gives you just information about if it 's voiced or not voiced , ma mainly , I mean . But So ,
Professor B: Yeah .
PhD A: this is why we we started to look by having sort of voiced phonemes
Professor B: Well , that 's the rea w w what I 'm arguing is that 's Yeah . I mean , uh , what I 'm arguing is that that that 's givi you gives you your intuition .
PhD A: and Mm - hmm .
Professor B: But in in reality , it 's you know , there 's all of this this overlap and so forth ,
Grad E: Oh , sorry .
Professor B: and But what I 'm saying is that may be OK , because what you 're really getting is not actually voiced versus unvoiced , both for the fac the reason of the overlap and and then , uh , th you know , structural reasons , uh , uh , like the one that Chuck said , that that in fact , well , the data itself is  that you 're working with is not perfect .
PhD A: Yeah . Mm - hmm .
Professor B: So , what I 'm saying is maybe that 's not a killer because you 're just getting some characterization , one that 's driven by your intuition about voiced - unvoiced certainly ,
PhD A: Mm - hmm .
Professor B: but it 's just some characterization of something back in the in the in the almost raw data , rather than the smooth version .
PhD A: Mm - hmm .
Professor B: And your intuition is driving you towards particular kinds of , uh , statistical characterizations of , um , what 's missing from the spectral envelope .
PhD A: Mm - hmm .
Professor B: Um , obviously you have something about the excitation , um , and what is it about the excitation , and , you know and you 're not getting the excitation anyway , you know . So so I I would almost take a uh , especially if if these trainings and so forth are faster , I would almost just take a uh , a scattershot at a few different ways of look of characterizing that difference and , uh , you could have one of them but and and see , you know , which of them helps .
PhD A: Mm - hmm . OK .
PhD C: So i is the idea that you 're going to take whatever features you develop and and just add them onto the future vector ? Or , what 's the use of the the voiced - unvoiced detector ?
PhD A: Uh , I guess we don't know exactly yet . But , um Yeah . Th
PhD C: It 's not part of a VAD system that you 're doing ?
PhD F: No .
PhD A: Uh , no . No .
PhD C: Oh , OK .
PhD A: No , the idea was , I guess , to to use them as as features .
PhD C: Features . I see .
PhD A: Uh Yeah , it could be , uh it could be a neural network that does voiced and unvoiced detection ,
PhD C: Mm - hmm .
PhD A: but it could be in the also the big neural network that does phoneme classification .
PhD C: Mm - hmm .
PhD A: Mmm . Yeah .
Professor B: But each one of the mixture components I mean , you have , uh , uh , variance only , so it 's kind of like you 're just multiplying together these , um , probabilities from the individual features within each mixture . So it 's so , uh , it seems l you know PhD C: I think it 's a neat thing . Uh , it seems like a good idea .
Professor B: Yeah . Um . Yeah . I mean , I know that , um , people doing some robustness things a ways back were were just doing just being gross and just throwing in the FFT and actually it wasn't wasn't wasn't so bad . Uh , so it would s and and you know that i it 's gotta hurt you a little bit to not have a  a spectral , uh a s a smooth spectral envelope , so there must be something else that you get in return for that PhD A: Mm - hmm .
Professor B: that , uh uh So .
PhD C: So how does uh , maybe I 'm going in too much detail , but how exactly do you make the difference between the FFT and the smoothed spectral envelope ? Wha - wh i i uh , how is that , uh ?
PhD A: Um , we just How did we do it up again ?
PhD F: Uh , we distend the we have the twenty - three coefficient af after the mel f filter ,
PhD A: Mm - hmm .
PhD F: and we extend these coefficient between the all the frequency range .
PhD C: Mm - hmm .
PhD F: And i the interpolation i between the point is give for the triang triangular filter , the value of the triangular filter and of this way we obtained this mode this model speech .
PhD A: S
Professor B: So you essentially take the values that th that you get from the triangular filter and extend them to sor sort of like a rectangle , that 's at that m value .
PhD F: Yeah .
PhD A: Yeah . I think we have linear interpolation .
PhD F: Mm - hmm .
PhD A: So we have we have one point for one energy for each filter bank ,
PhD F: mmm Yeah , it 's linear .
PhD C: Mmm .
Professor B: Oh .
PhD A: which is the energy that 's centered on on on the triangle PhD F: Yeah . At the n at the center of the filter PhD C: So you you end up with a vector that 's the same length as the FFT vector ?
PhD A: Yeah . That 's right .
PhD F: Yeah .
PhD C: And then you just , uh , compute differences
PhD F: Yeah . I have here one example if you if you want see something like that .
PhD A: Then we compute the difference .
PhD C: and ,
PhD A: Yeah . Uh - huh .
Professor B: OK .
PhD C: uh , sum the differences ?
PhD A: So . And I think the variance is computed only from , like , two hundred hertz to one to fifteen hundred .
PhD C: Oh ! OK .
Professor B: Mm - hmm .
PhD F: Two thou two {comment} fifteen hundred ?
Professor B: Mm - hmm .
PhD A: Because PhD F: No .
Professor B: Right .
PhD F: Two hundred and fifty thousand .
PhD A: Fifteen hundred . Because Yeah .
PhD F: Yeah . Two thousand and fifteen hundred .
PhD A: Above , um  it seems that Well , some voiced sound can have also , like , a noisy part on high frequencies , and But Professor B: Yeah .
PhD A: Well , it 's just Professor B: No , it 's makes sense to look at low frequencies .
PhD C: So this is uh , basically this is comparing an original version of the signal to a smoothed version of the same signal ?
PhD F: Yeah .
Professor B: Right . So i so i i this is I mean , i you could argue about whether it should be linear interpolation or or or or zeroeth order , but but
PhD C: Uh - huh .
Professor B: at any rate something like this is what you 're feeding your recognizer , typically .
PhD C: Like which of the ?
Professor B: No . Uh , so the mel cepstrum is the is the is the cepstrum of this  this , uh , spectrum or log spectrum ,
PhD A: So this is Yeah .
PhD C: Yeah . Right , right .
Professor B: whatever it You - you 're subtracting in in in power domain or log domain ?
PhD A: In log domain . Yeah .
PhD F: Log domain .
Professor B: OK . So it 's sort of like division , when you do the yeah , the spectra .
PhD F: Yeah .
PhD A: Uh , yeah .
PhD C: It 's the ratio .
Professor B: Um . Yeah . But , anyway , um and that 's PhD C: So what 's th uh , what 's the intuition behind this kind of a thing ? I I don't know really know the signal - processing well enough to understand what  what is that doing .
PhD A: So . Yeah . What happen if what we have have what we would like to have is some spectrum of the excitation signal ,
Professor B: Yeah . I guess that makes sense . Yeah .
PhD A: which is for voiced sound ideally a a pulse train
PhD C: Uh - huh .
PhD A: and for unvoiced it 's something that 's more flat .
PhD C: Uh - huh . Right .
PhD A: And the way to do this is that well , we have the we have the FFT because it 's computed in in the in the system , and we have the mel filter banks ,
PhD C: Mm - hmm . Mm - hmm .
PhD A: and so if we if we , like , remove the mel filter bank from the FFT , we have something that 's close to the excitation signal .
Grad E: Oh .
PhD A: It 's something that 's like a a a train of p a pulse train for voiced sound
PhD C: OK .
Professor B: Yeah .
PhD C: Oh ! OK . Yeah .
PhD A: and that 's that should be flat for Professor B: Yeah .
PhD C: I see . So do you have a picture that sh ?
PhD A: So - It 's Y
PhD C: Is this for a voiced segment ,
PhD A: yeah .
PhD C: this picture ? What does it look like for unvoiced ?
PhD F: Yeah .
PhD A: You have several some unvoiced ?
PhD F: The dif No . Unvoiced , I don't have
PhD A: Oh .
PhD F: for unvoiced .
Professor B: Yeah . So , you know , all PhD F: I 'm sorry .
PhD A: But Yeah .
Professor B: Yeah .
PhD F: Yeah . This is the between PhD A: This is another voiced example . Yeah .
PhD F: No . But it 's this ,
PhD A: Oh , yeah . This is PhD F: but between the frequency that we are considered for the excitation PhD A: Right . Mm - hmm .
PhD F: for the difference and this is the difference .
PhD A: Yeah .
PhD C: This is the difference . OK .
PhD A: So , of course , it 's around zero ,
Professor B: Yeah .
Grad E: Sure looks PhD A: but Grad E: Hmm .
PhD A: Well , no .
PhD C: Hmm .
PhD A: It is PhD F: Yeah . Because we begin , uh , in fifteen point the fifteen point .
PhD C: So , does does the periodicity of this signal say something about the the PhD F: Fifteen p
PhD A: So it 's Yeah .
Professor B: Pitch .
PhD A: It 's the pitch .
PhD C: the pitch ?
PhD A: Yeah . Mm - hmm .
Professor B: Yeah .
PhD C: OK .
Professor B: That 's like fundamental frequency .
PhD A: Mm - hmm .
Professor B: So , I mean , i t t
PhD C: OK . I see .
Professor B: I mean , to first order what you 'd what you 're doing I mean , ignore all the details and all the ways which is that these are complete lies . Uh , the the you know , what you 're doing in feature extraction for speech recognition is you have , uh , in your head a a a a simplified production model for speech ,
PhD C: Mm - hmm .
Professor B: in which you have a periodic or aperiodic source that 's driving some filters .
PhD C: Mm - hmm .
PhD F: Yeah . This is the the auto - correlation the R - zero energy .
PhD A: Do you have the mean do you have the mean for the auto - correlation ?
Professor B: Uh , first order for speech recognition , you say " I don't care about the source " .
PhD F: For Yeah .
PhD A: Well , I mean for the the energy .
PhD F: I have the mean .
Professor B: Right ?
PhD C: Right .
Professor B: And so you just want to find out what the filters are .
PhD C: Right .
PhD F: Yeah .
Professor B: The filters roughly act like a , um  a , uh  a an overall resonant you know , f some resonances and so forth that th that 's processing excitation .
PhD F: Here .
PhD A: They should be more close .
PhD F: Ah , no . This is this ? More close . Is this ? And this .
PhD C: Mm - hmm .
PhD A: Yeah .
PhD C: Mm - hmm .
PhD A: So they are this is there is less difference .
PhD F: Mm - hmm .
Professor B: So if you look at the spectral envelope , just the very smooth properties of it , you get something closer to that .
PhD A: This is less it 's less robust .
PhD F: Less robust . Yeah .
PhD A: Oh , yeah .
Professor B: And the notion is if you have the full spectrum , with all the little nitty - gritty details , that that has the effect of both ,
PhD C: Yeah .
Professor B: and it would be a multiplication in in frequency domain
PhD C: Mm - hmm .
Professor B: so that would be like an addition in log  power spectrum domain .
PhD C: Mm - hmm . Mm - hmm .
Professor B: And so this is saying , well , if you really do have that sort of vocal tract envelope , and you subtract that off , what you get is the excitation . And I call that lies because you don't really have that , you just have some kind of signal - processing trickery to get something that 's kind of smooth . It 's not really what 's happening in the vocal tract
PhD C: Yeah .
Professor B: so you 're not really getting the vocal excitation .
PhD C: Right .
Professor B: That 's why I was going to the why I was referring to it in a more  a more , uh , uh , conservative way , when I was saying " well , it 's yeah , it 's the excitation " . But it 's not really the excitation . It 's whatever it is that 's different between PhD C: Oh . This moved in the Professor B: So so , stand standing back from that , you sort of say there 's this very detailed representation .
PhD C: Yeah .
Professor B: You go to a smooth representation .
PhD C: Mm - hmm .
Professor B: You go to a smooth representation cuz this typically generalizes better .
PhD C: Mm - hmm .
Professor B: Um , but whenever you smooth you lose something , so the question is have you lost something you can you use ?
PhD C: Right .
Professor B: Um , probably you wouldn't want to go to the extreme of just ta saying " OK , our feature set will be the FFT " , cuz we really think we do gain something in robustness from going to something smoother , but maybe there 's something that we missed .
PhD C: Mm - hmm .
Professor B: So what is it ?
PhD C: Yeah .
Professor B: And then you go back to the intuition that , well , you don't really get the excitation , but you get something related to it .
PhD C: Mm - hmm .
Professor B: And it and as you can see from those pictures , you do get something that shows some periodicity , uh , in frequency ,
PhD C: Mm - hmm .
Professor B: you know , and and and also in time .
PhD C: Hmm .
Professor B: So PhD C: That 's that 's really neat .
Professor B: so ,
PhD C: So you don't have one for unvoiced picture ?
PhD F: Uh , not here .
PhD C: Oh .
PhD F: No , I have s
PhD A: Mm - hmm .
Professor B: Yeah .
PhD F: But not here .
Professor B: But presumably you 'll see something that won't have this kind of , uh , uh , uh , regularity in frequency , uh , in the PhD A: But Yeah . Well .
PhD F: Not here .
PhD C: I would li I would like to see those pictures .
PhD F: Well , so .
Professor B: Yeah .
PhD F: I can't see you {comment} now .
Professor B: Yeah .
PhD C: Yeah .
Professor B: Yeah .
PhD A: Mm - hmm .
PhD F: I don't have .
PhD C: And so you said this is pretty doing this kind of thing is pretty robust to noise ?
PhD A: It seems , yeah . Um ,
PhD C: Huh .
PhD F: Pfft . Oops . The mean is different with it , because the  the histogram for the  the classifica
PhD A: No , no , no . But th the kind of robustness to noise PhD F: Oh !
PhD A: So if if you take this frame , uh , from the noisy utterance and the same frame from the clean utterance PhD F: Hmm .
PhD C: You end up with a similar difference
PhD A: Y y y yeah . We end up with PhD C: over here ?
PhD A: Yeah .
PhD C: OK . Cool !
PhD F: I have here the same frame for the clean speech PhD C: Oh , that 's clean .
PhD F: the same cle
PhD C: Oh , OK
PhD F: But they are a difference .
PhD A: Yeah , that 's PhD F: Because here the FFT is only with two hundred fifty - six point
PhD C: Oh .
PhD F: and this is with five hundred twelve .
PhD A: Yeah . This is kind of inter interesting also
PhD C: OK .
PhD A: because if we use the standard , uh , frame length of of , like , twenty - five milliseconds , um , what happens is that for low - pitched voiced , because of the frame length , y you don't really have  you don't clearly see this periodic structure ,
Professor B: Mm - hmm .
PhD A: because of the first lobe of of each each of the harmonics .
PhD C: So this one inclu is a longer Ah .
PhD A: So , this is like yeah , fifty milliseconds or something like that .
PhD F: Fifty millis Yeah .
PhD A: Yeah , but it 's the same frame and PhD C: Oh , it 's that time - frequency trade - off thing .
PhD A: Yeah .
PhD C: Right ? I see . Yeah .
PhD A: So , yeah .
Professor B: Mm - hmm .
PhD C: Oh . Oh , so this i is this the difference here , for that ?
PhD F: No . This is the signal . This is the signal .
PhD A: I see that . Oh , yeah .
PhD F: The frame .
PhD C: Oh , that 's the f the original .
PhD A: Yeah .
PhD F: This is the fra the original frame .
PhD A: So with a short frame basically you have only two periods
PhD C: Yeah .
PhD A: and it 's not not enough to to have this kind of neat things .
PhD C: Mm - hmm .
PhD F: Mm - hmm .
PhD C: Yeah .
PhD A: But PhD F: And here No , well .
PhD A: Yeah . So probably we 'll have to use , like , long f long frames . Mm - hmm .
PhD C: Mm - hmm .
Grad E: Hmm .
PhD C: Oh .
Professor B: Mmm .
PhD C: That 's interesting .
Professor B: Yeah , maybe . Well , I mean it looks better , but , I mean , the thing is if if , uh if you 're actually asking you know , if you actually j uh , need to do place along an FFT , it may be it may be pushing things .
PhD A: Yeah .
Professor B: And and , uh PhD C: Would you would you wanna do this kind of , uh , difference thing after you do spectral subtraction ?
PhD A: Uh , maybe .
PhD F: No . Maybe we can do that .
PhD A: Mmm .
Professor B: Hmm . The spectral subtraction is being done at what level ? Is it being done at the level of FFT bins or at the level of , uh , mel spectrum or something ?
PhD A: Um , I guess it depends .
Professor B: I mean , how are they doing it ?
PhD A: How they 're doing it ? Yeah . Um , I guess Ericsson is on the , um , filter bank ,
PhD F: FFT . Filter bank ,
PhD A: no ? It 's on the filter bank ,
PhD F: yeah .
PhD A: so . So , yeah , probably I i it Yeah .
Professor B: So in that case , it might not make much difference at all .
PhD C: Seems like you 'd wanna do it on the FFT bins .
Professor B: Maybe . I mean , certainly it 'd be better .
PhD C: I I mean , if you were gonna uh , for for this purpose , that is .
PhD A: Mm - hmm .
Professor B: Yeah .
PhD A: Mm - hmm .
Professor B: Yeah . OK .
PhD A: Mmm .
Professor B: What else ?
PhD A: Uh . Yeah , that 's all . So we 'll perhaps   try to convince OGI people to use the new  the new filters and Yeah .
Professor B: OK . Uh , has has anything happened yet on this business of having some sort of standard , uh , source ,
PhD A: Uh , not yet
Professor B: or ?
PhD A: but I wi I will call them and Professor B: OK .
PhD A: now they are I think they have more time because they have this well , Eurospeech deadline is over