C: uh , we 'll have to figure ways without it .
C: Um ,
A: If they 're going to provide a , uh , voice activity detector that will tell you the boundaries of the speech , then , couldn't you just go outside those boundaries and do your estimate there ?
C: Oh , yeah .
C: You bet .
C: Yeah .
C: So I I imagine that 's what they 're doing ,
C: right ?
C: Is they 're C: they 're probably looking in nonspeech sections
C: and getting some , uh B: Yeah ,
B: they have some kind of threshold on on the previous estimate ,
B: and B: So .
B: Yeah .
B: I think . Yeah , I think Ericsson used this kind of threshold . Yeah ,
B: so , they h they have an estimate of the noise level
B: and they put a threshold like six or ten DB above ,
B: and
B: what 's under this threshold is used to update the estimate .
B: Is is that right
D: Yeah .
B: or ?
D: I think so .
D: I have not here the proposal .
B: So it 's it 's B: Yeah .
C: Does France Telecom do this B: It 's like saying what 's under the threshold is silence ,
B: and E: Hmm .
C: Does France Telecom do th do the same thing ?
C: More or less ?
B: I d I B: Y you know , perhaps ?
D: No .
D: I do I have not here the proposal .
C: OK .
C: Um ,
C: OK ,
C: if we 're we 're done done with that ,
C: uh ,
C: let 's see .
C: Uh , maybe we can talk about a couple other things briefly ,
C: just , uh , things that that we 've been chatting about
C: but haven't made it into these meetings yet .
C: So you 're coming up with your quals proposal ,
C: and , uh Wanna just give a two three minute summary of what you 're planning on doing ?
E: Oh ,
E: um ,
E: two , three ,
E: it can be shorter than that .
E: Um .
C: Yeah .
E: Well , I 've I 've talked to some of you already .
E: Um , but I 'm , uh , looking into extending the work done by Larry Saul and John Allen and uh Mazin Rahim .
E: Um , they they have a system that 's , uh , a multi - band , um , system
E: but their multi - band is is a little different than the way that we 've been doing multi - band in the past ,
E: where um Where we 've been @ @ <comment> uh taking um  sub - band features and i training up these neural nets and on on phonetic targets ,
E: and then combining them some somehow down the line ,
E: um ,
E: they 're they 're taking sub - band features and , um , training up a detector that detects for , um , these phonetic features
E: for example , um , he presents um , uh , a detector to detect sonorance .
E: And so what what it basically is is , um it 's there 's E: at the lowest level , there it 's it 's an OR ga I mean , it 's an AND gate .
E: So , uh , on each sub - band you have several independent tests ,
E: to test whether um , there 's the existence of sonorance in a sub - band .
E: And then , um , it c it 's combined by a soft AND gate .
E: And at the at the higher level ,
E: for every if ,
E: um E: The higher level there 's a soft OR gate .
E: Uh , so if if this detector detects um , the presence of of sonorance in any of the sub - bands , then the detect uh , the OR gate at the top says , " OK , well this frame has evidence of sonorance . "
E: And these are all A: What are what are some of the low level detectors that they use ?
E: Oh , OK .
E: Well , the low level detectors are logistic regressions .
E: Um ,
E: and
E: the , uh E: the one o
C: So that , by the way , basically is a is one of the units in our in our our neural network .
C: So that 's all it is .
C: It 's a sig it 's a sigmoid ,
E: Yeah .
C: uh , with weighted sum at the input ,
A: Hmm .
E: Right .
C: which you train by gradient descent .
E: Yeah ,
E: so he uses , um , an EM algorithm to to um train up these um parameters for the logistic regression .
E: The C: Well , actually ,
C: yeah ,
C: so I was using EM to get the targets .
C: So so you have this this this AND gate what we were calling an AND gate , but it 's a product product rule thing at the output .
C: And then he uses , uh , i u and then feeding into that are C: I 'm sorry ,
C: there 's it 's an OR at the output , isn't it ?
C: Yeah ,
E: Mm - hmm .
C: so that 's the product .
C: And then ,
C: um ,
C: then he has each of these AND things .
C: And ,
C: um ,
C: but C: so they 're little neural neural units .
C: Um ,
C: and , um ,
C: they have to have targets .
C: And so the targets come from EM .
A: And so are each of these , low level detectors <comment> are they ,
A: uh A: are these something that you decide ahead of time ,
A: like " I 'm going to look for this particular feature
A: or I 'm going to look at this frequency , "
A: or A: What what what are they looking at ?
E: Um A: What are their inputs ?
E: Uh
E: Right ,
E: so the E: OK ,
E: so at each for each sub - band <comment> there are basically , uh , several measures of SNR and and correlation .
A: Ah ,
A: OK ,
E: Um ,
A: OK .
E: um and he said there 's like twenty of these per per sub - band .
E: Um ,
E: and for for every s every sub - band , e you you just pick ahead of time , um , " I 'm going to have like five i independent logistic tests . "
A: Mm - hmm .
E: And you initialize these parameters , um , in some some way
E: and use EM to come up with your training targets for a for the the low - level detectors .
A: Mm - hmm .
E: And then , once you get that done , you you you train the whole whole thing on maximum likelihood .
E: Um ,
E: and h he shows that using this this method to detect sonorance is it 's very robust compared to , um  to typical , uh , full - band Gaussian mixtures um estimations of of sonorance .
A: Mm - hmm .
A: Mm - hmm .
E: And , uh
E: so E: so that 's just that 's just one detector .
E: So you can imagine building many of these detectors on different features .
E: You get enough of these detectors together , um , then you have enough information to do , um , higher level discrimination ,
E: for example , discriminating between phones
A: Mm - hmm .
E: and then you keep working your way up until you you build a full recognizer .
A: Mm - hmm .
E: So , um , that 's that 's the direction which I 'm I 'm thinking about going in my quals .
A: Cool .
C: You know , it has a number of properties that I really liked .
C: I mean , one is the going towards , um , using narrow band information for , uh , ph phonetic features of some sort
C: rather than just , uh , immediately going for the the typical sound units .
A: Right .
C: Another thing I like about it is that you t this thing is going to be trained explicitly trained for a product of errors rule ,
C: which is what , uh , Allen keeps pointing out that Fletcher observed in the twenties ,
A: Mm - hmm .
C: uh , for people listening to narrow band stuff . That 's Friday 's talk , by the way .
C: And then , um ,
C: Uh , the third thing I like about it is ,
C: uh , and we 've played around with this in a different kind of way a little bit
C: but it hasn't been our dominant way of of operating anything ,
C: um , this issue of where the targets come from .
C: So in our case when we 've been training it multi - band things , the way we get the targets for the individual bands is , uh , that we get the phonetic label for the sound there
A: Mm - hmm .
C: and we say , " OK , we train every "
C: What this is saying is , OK , that 's maybe what our ultimate goal is C: or not ultimate but penultimate goal is getting these these small sound units .
C: But but , um ,
C: along the way how much should we , uh C: uh , what should we be training these intermediate things for ?
C: I mean , because , uh , we don't know uh , that this is a particularly good feature .
C: I mean , there 's no way , uh C: someone in the audience yesterday was asking , " well couldn't you have people go through and mark the individual bands and say where the where it was sonorant or not ? "
A: Mm - hmm .
C: But , you know , I think having a bunch of people listening to critical band wide , uh , chunks of speech trying to determine whether <comment> I think it 'd be impossible .
E: Ouch .
C: It 's all gonna sound like like sine waves to you , more or less .
A: Mm - hmm .
C: I mean C: Well not
C: I mean , it 's g all g
C: narrow band
C: uh ,
C: i I m I think it 's very hard for someone to to a person to make that determination .
C: So , um ,
C: um , we don't really know how those should be labeled .
C: It could sh be that you should , um , not be paying that much attention to , uh , certain bands for certain sounds , uh , in order to get the best result .
A: Mm - hmm .
C: So , um , what we have been doing there , just sort of mixing it all together , is certainly much much cruder than that .
C: We trained these things up on the on the , uh the final label .
C: Now we have I guess done experiments C: you 've probably done stuff where you have , um , done separate , uh , Viterbis on the different E: Yeah .
E: Forced alignment on the sub - band labels ?
C: Yeah .
E: Yeah .
C: You 've done that .
C: Did did that help at all ?
E: Um , it helps for one or t one iteration
E: but um , anything after that it doesn't help .
C: So so that may or may t it that aspect of what he 's doing may or may not be helpful
C: because in a sense that 's the same sort of thing . You 're taking global information and determining what you how you should C: But this is this is , uh , I th I think a little more direct .
A: How did they measure the performance of their detector ?
C: And C: Well , he 's look he 's just actually looking at , uh , the confusions between sonorant and non - sonorant .
A: Mm - hmm .
C: So he hasn't applied it to recognition
C: or if he did he didn't talk about it .
C: It 's it 's just C: And one of the concerns in the audience , actually , was that that , um , the , uh ,
C: uh he he did a comparison to , uh , you know , our old foil ,
C: the the nasty old standard recognizer with mel mel filter bank at the front , and H M Ms , and and so forth .
C: And , um , it didn't do nearly as well ,
C: especially in in noise .
C: But the C: one of the good questions in the audience was , well , yeah , but that wasn't trained for that .
C: I mean , this use of a very smooth , uh , spectral envelope is something that , you know , has evolved as being generally a good thing for speech recognition
C: but if you knew that what you were gonna do is detect sonorants or not C: So sonorants and non - sonorants is is is almost like voiced - unvoiced , except I guess that the voiced stops are are also called " obstruents " .
C: Uh ,
C: so it 's C: it 's uh , but with the exception of the stops I guess it 's pretty much the same as voiced - unvoiced ,
C: right ?
A: Mm - hmm .
C: So so C: Um .
C: So , um ,
C: if you knew you were doing that ,
C: if you were doing something say for a a , uh a a Vocoder , you wouldn't use the same kind of features .
C: You would use something that was sensitive to the periodicity and and not just the envelope .
C: Uh , and so in that sense it was an unfair test .
C: Um ,
C: so I think that the questioner was right .
C: It it was in that sense an unfair test .
C: Nonetheless , it was one that was interesting because , uh , this is what we are actually using for speech recognition ,
C: these smooth envelopes .
C: And this says that perhaps even , you know , trying to use them in the best way that we can , that that that we ordinarily do , with , you know , Gaussian mixtures and H M Ms <comment> and so forth , you you don't , uh , actually do that well on determining whether something is sonorant or not .
A: Didn't they C: Which means you 're gonna make errors between similar sounds that are son sonorant or obstruent .
A: Didn't they also do some kind of an oracle experiment
A: where they said " if we could detect the sonorants perfectly and then show how it would improve speech recognition ?
A: I thought I remember hearing about an experiment like that .
C: The - these same people ?
A: Mm - hmm .
C: I don't remember that .
A: Hmm .
C: That would that 's C: you 're right ,
C: that 's exactly the question to follow up this discussion , is
C: suppose you did that , uh , got that right .
C: Um ,
C: Yeah .
A: Hmm .
B: What could be the other low level detectors , I mean , for <comment> Other kind of features ,
B: or ?
B: in addition to detecting sonorants
B: or ?
E: Um B: Th - that 's what you want to to to go for also
E: What t
B: or ?
E: Oh , build other other detectors on different phonetic features ?
B: Other low level detectors ?
B: Yeah .
E: Um ,
E: uh Let 's see ,
E: um ,
E: Yeah ,
E: I d I don't know .
E: e Um ,
E: um ,
E: I mean , w easiest thing would be to go go do some voicing stuff
E: but that 's very similar to sonorance .
B: Mm - hmm .
E: Um ,
A: When we when we talked with John Ohala the other day we made a list of some of the things that w
E: Yeah .
E: Oh ! OK .
A: like frication ,
E: Mm - hmm .
A: abrupt closure ,
E: Mm - hmm .
A: R - coloring ,
A: nasality ,
A: voicing C: Yeah ,
C: so there 's a half dozen like that that are A: Uh .
E: Yeah ,
E: nasality .
C: Now this was coming at it from a different angle
C: but maybe it 's a good way to start .
C: Uh , these are things which , uh , John felt that a a , uh a human annotator would be able to reliably mark .
E: Oh , OK .
C: So the sort of things he felt would be difficult for a human annotator to reliably mark would be tongue position kinds of things .
E: Placing stuff ,
A: Mm - hmm .
C: Yeah .
E: yeah .
C: Uh A: There 's also things like stress .
A: You can look at stress .
E: Mm - hmm .
C: But stress doesn't , uh , fit in this thing of coming up with features that will distinguish words from one another ,
C: right ?
C: It 's a it 's a good thing to mark
C: and will probably help us ultimate with recognition
A: Yeah ,
A: there 's a few cases where it can like permit <comment> and permit .
C: but A: But that 's not very common in English .
A: In other languages it 's more uh , important .
C: Well , yeah ,
C: but i either case you 'd write PERMIT ,
C: right ?
C: So you 'd get the word right .
A: No , I 'm saying , i i e I thought you were saying that stress doesn't help you distinguish between words .
C: Um ,
A: Oh , I see what you 're saying .
A: As long as you get The sequence ,
C: We 're g if we 're doing if we 're talking about transcription as opposed to something else A: right ?
A: Yeah .
A: Yeah , yeah , yeah . Yeah .
C: Yeah .
A: Right .
A: So where it could help is maybe at a higher level .
E: Like a understanding application .
C: Right .
A: Yeah .
A: Understanding ,
A: yeah .
C: Yeah .
E: Yeah .
A: Exactly .
C: But that 's this afternoon 's meeting .
C: Yeah .
C: We don't understand anything in this meeting .
C: Yeah ,
C: so that 's C: yeah ,
C: that 's , you know , a neat neat thing
C: and C: and , uh E: S so , um , Ohala 's going to help do these , uh transcriptions of the meeting data ?
C: So .
A: Uh ,
A: well I don't know .
A: We d we sort of didn't get that far .
A: Um , we just talked about some possible features that could be marked by humans
A: and , um ,
E: Hmm .
A: because of having maybe some extra transcriber time we thought we could go through and mark some portion of the data for that .
A: And , uh E: Hmm .
C: Yeah ,
C: I mean , that 's not an immediate problem , that we don't immediately have a lot of extra transcriber time .
A: Yeah ,
C: But but , uh , in the long term I guess Chuck is gonna continue the dialogue with John
A: right .
C: and and , uh ,
C: and , we 'll we 'll end up doing some I think .
A: I 'm definitely interested in this area , too ,
A: f uh , acoustic feature stuff .
C: Uh - huh .
E: OK .
A: So .
C: Yeah , I think it 's an interesting interesting way to go .
E: Cool .
C: Um ,
C: I say it like " said - int " . I think it has a number of good things .
C: Um ,
C: so , uh , y you want to talk maybe a c two or three minutes about what we 've been talking about today and other days ?
F: Ri
F: Yeah ,
F: OK ,
F: so , um ,
F: we 're interested in , um , methods for far mike speech recognition ,
F: um , mainly , uh , methods that deal with the reverberation in the far mike signal .
F: So , um ,
F: one approach would be , um , say MSG and PLP , like was used in Aurora one
F: and , um , there are other approaches which actually attempt to remove the reverberation , instead of being robust to it like MSG .
F: And so we 're interested in , um , comparing the performance of um , a robust approach like MSG with these , um , speech enhancement or de - reverber de - reverberation approaches .
B: Mm - hmm .
F: And , um , it looks like we 're gonna use the Meeting Recorder digits data for that .
B: And the de - reverberation algorithm ,
B: do you have can you give some more details on this
B: or ? Does it use one microphone ?
F: o
B: Several microphones ?
F: o
B: Does it ?
F: OK ,
F: well , um ,
F: there was something that was done by , um , a guy named Carlos , I forget his last name , <comment> who worked with Hynek , who , um ,
C: Avendano .
F: OK .
F: Who , um ,
C: Yeah .
B: Mm - hmm .
F: um , it was like RASTA in the sense that of it was , um , de - convolution by filtering
F: um , except he used a longer time window ,
B: Mm - hmm .
F: like a second maybe .
F: And the reason for that is RASTA 's time window is too short to , um include the whole , um , reverberation