Meeting transcript:
speaker: A
text: Eh , we should be going .
speaker: B
text: So ne next week we 'll have , uh , both Birger <pause> and , uh , Mike <disfmarker> Michael <disfmarker>
speaker: B
text: Michael Kleinschmidt
speaker: D
text: Uh - huh .
speaker: B
text: and Birger Kollmeier will join us .
speaker: B
text: Um ,
speaker: B
text: and you 're <disfmarker> <vocalsound> you 're probably gonna go up in a couple <disfmarker> three weeks or so ?
speaker: B
text: When d when are you thinking of going up to , uh , OGI ?
speaker: D
text: Yeah , like , uh , not next week
speaker: D
text: but maybe the week after .
speaker: B
text: OK .
speaker: B
text: Good . So at least we 'll have one meeting with <vocalsound> yo with you still around ,
speaker: D
text: Uh - huh .
speaker: B
text: and <disfmarker> and <disfmarker>
speaker: B
text: That 's good .
speaker: D
text: Um ,
speaker: D
text: Yeah . Well , <vocalsound> maybe we can start with this .
speaker: D
text: Mmm .
speaker: B
text: All today , huh ?
speaker: D
text: Yeah .
speaker: B
text: Oh .
speaker: D
text: Um .
speaker: D
text: Yeah . So there was this conference call this morning ,
speaker: D
text: um ,
speaker: D
text: and the only topic on the agenda was just to discuss
speaker: D
text: a and to come at <disfmarker> uh , to get a decision about this latency problem .
speaker: B
text: No , this <disfmarker> I 'm sorry , this is a conference call between different Aurora people
speaker: D
text: Uh , yeah . It 's the conference call between the Aurora , <vocalsound> uh , group .
speaker: B
text: or just <disfmarker> ?
speaker: B
text: It 's the main conference call .
speaker: B
text: OK .
speaker: D
text: Uh , yeah . There were like two hours of <pause> discussions ,
speaker: D
text: and then suddenly , <vocalsound> uh , people were tired , I guess ,
speaker: D
text: and they decided on <nonvocalsound> a number ,
speaker: D
text: two hundred and twenty ,
speaker: D
text: um ,
speaker: D
text: included e including everything .
speaker: D
text: Uh , it means that it 's like eighty milliseconds <pause> less than before .
speaker: D
text: Um .
speaker: B
text: And what are we sitting at currently ?
speaker: D
text: So , currently d uh , we have system that has two hundred and thirty .
speaker: B
text: Yeah .
speaker: D
text: So , that 's fine .
speaker: B
text: Two thirty .
speaker: D
text: Yeah .
speaker: D
text: So that 's the system that 's described on the second point of <pause> this <vocalsound> document .
speaker: B
text: So it 's <disfmarker>
speaker: B
text: we have to reduce it by ten milliseconds somehow .
speaker: D
text: Yeah . But that 's <disfmarker>
speaker: D
text: Yeah .
speaker: D
text: That 's not a problem , I <disfmarker> I guess .
speaker: B
text: OK .
speaker: D
text: Um .
speaker: B
text: W It 's <disfmarker> it 's p d primary <disfmarker> primarily determined by the VAD at this point ,
speaker: B
text: right ?
speaker: D
text: Yeah .
speaker: D
text: Yeah . At this point ,
speaker: B
text: S so we can make the VAD a little shorter .
speaker: D
text: yeah .
speaker: B
text: That 's <disfmarker>
speaker: D
text: Yeah , uh - huh .
speaker: B
text: Yeah . We probably should do that pretty soon so that we don't get used to it being a certain way .
speaker: D
text: Uh - huh .
speaker: B
text: Yeah .
speaker: D
text: Um .
speaker: B
text: Was Hari on the <disfmarker> on the phone ?
speaker: D
text: Yeah , sure .
speaker: B
text: OK .
speaker: D
text: Well , it was mainly a discussion <vocalsound> between Hari and <vocalsound> David ,
speaker: B
text: Hmm .
speaker: D
text: who was like <disfmarker>
speaker: B
text: Yeah .
speaker: D
text: Uh ,
speaker: B
text: OK .
speaker: D
text: mmm <disfmarker>
speaker: D
text: Uh , yeah . So , the second thing is the system that we have currently .
speaker: D
text: Oh , yes . We have , like , a system that gives sixty - two percent improvement ,
speaker: D
text: but <vocalsound> if you want to stick to the <disfmarker> <vocalsound> this latency <disfmarker>
speaker: D
text: Well , it has a latency of two thirty ,
speaker: D
text: but <vocalsound> if you want also to stick to the number <vocalsound> of features that <disfmarker> limit it to sixty , <vocalsound> then we go a little bit down
speaker: D
text: but it 's still sixty - one percent .
speaker: D
text: Uh , and if we drop the tandem network , then we have fifty - seven percent .
speaker: B
text: Uh , but th the two th two thirty includes the tandem network ?
speaker: D
text: Yeah .
speaker: B
text: OK .
speaker: B
text: And i is the tandem network , uh , small enough that it will fit on the terminal size
speaker: D
text: Uh , no , I don't think so .
speaker: B
text: in terms of <disfmarker> ?
speaker: B
text: No .
speaker: D
text: No .
speaker: B
text: OK .
speaker: D
text: It 's still <disfmarker> in terms of computation , if we use , like , their way of computing the <disfmarker> the maps <disfmarker> the <disfmarker> the MIPs , <vocalsound> I think it fits ,
speaker: B
text: Mm - hmm .
speaker: B
text: Mm - hmm .
speaker: D
text: but it 's , uh , m mainly a problem of memory .
speaker: B
text: Right .
speaker: D
text: Um ,
speaker: D
text: and I don't know how much <pause> this can be discussed or not ,
speaker: D
text: because it 's <disfmarker> it could be in ROM ,
speaker: D
text: so it 's maybe not that expensive .
speaker: D
text: But <disfmarker>
speaker: B
text: Ho - how much memory d ? H how many <disfmarker> ?
speaker: D
text: I d I d uh , I <disfmarker> I don't kn remember exactly ,
speaker: D
text: but <disfmarker> <vocalsound> Uh .
speaker: D
text: Yeah , I c I <disfmarker> I have to check that .
speaker: B
text: Yeah . I 'd like to <pause> see that ,
speaker: B
text: cuz maybe I could think a little bit about it ,
speaker: B
text: cuz we <vocalsound> maybe we could make it a little smaller
speaker: B
text: or <disfmarker> I mean , it 'd be <disfmarker> it 'd be neat if we could fit it all .
speaker: D
text: Uh - huh .
speaker: B
text: Uh , I 'd like to see how far off we are .
speaker: D
text: Mm - hmm .
speaker: B
text: But I guess it 's still within their rules to have <disfmarker> have it on the , uh , t uh , server side .
speaker: D
text: Yeah . Yeah .
speaker: B
text: Right ?
speaker: B
text: OK .
speaker: D
text: Mmm .
speaker: B
text: And this is still <disfmarker> ?
speaker: B
text: Uh , well , y you 're saying here .
speaker: B
text: I c I should just let you go on .
speaker: D
text: Yeah , there were small tricks to make this tandem network work .
speaker: D
text: Uh , <vocalsound> mmm ,
speaker: D
text: and one of the trick was to , <vocalsound> um , use <vocalsound> some kind of hierarchical structure
speaker: D
text: where <pause> the silence probability is not computed by <pause> the final tandem network but by the VAD network .
speaker: D
text: Um ,
speaker: D
text: so apparently it looks better when , <vocalsound> uh , we use the silence probability from the VAD network
speaker: D
text: and we re - scale the other probabilities by one minus the silence probability .
speaker: B
text: Huh .
speaker: D
text: Um .
speaker: D
text: So it 's some kind of hierarchical thing , <vocalsound> uh , that Sunil also tried , um , <vocalsound> <vocalsound> on SPINE
speaker: D
text: and apparently it helps a little bit also .
speaker: D
text: Mmm .
speaker: D
text: And .
speaker: D
text: Yeah , the reason w why <disfmarker> why we did that with the silence probability was that , <vocalsound> um <disfmarker>
speaker: B
text: Could <disfmarker> ? Uh , uh ,
speaker: B
text: I 'm <disfmarker> I 'm really sorry .
speaker: B
text: Can you repeat what you were saying about the silence probability ?
speaker: D
text: Mm - hmm .
speaker: B
text: I only <disfmarker>
speaker: D
text: Yeah .
speaker: B
text: My mind was some <disfmarker>
speaker: D
text: So there is the tandem network that e e e estimates the phone probabilities
speaker: B
text: Yeah .
speaker: B
text: Yeah .
speaker: D
text: and the silence probabilities also .
speaker: B
text: Right .
speaker: D
text: And <vocalsound> things get better when , instead of using the silence probability computed by the tandem network , we use the silence probability , uh , given by the VAD network ,
speaker: B
text: Oh .
speaker: D
text: um ,
speaker: B
text: The VAD network is <disfmarker> ?
speaker: D
text: Which is smaller ,
speaker: D
text: but maybe ,
speaker: D
text: um <disfmarker>
speaker: D
text: So we have a network for the VAD which has one hundred hidden units ,
speaker: D
text: and the tandem network has five hundred .
speaker: D
text: Um .
speaker: D
text: So it 's smaller
speaker: D
text: but th the silence probability <pause> from this network seems , uh , better .
speaker: B
text: OK .
speaker: D
text: Mmm .
speaker: D
text: Uh .
speaker: D
text: Well , it looks strange ,
speaker: D
text: but <disfmarker>
speaker: B
text: Yeah . But <disfmarker>
speaker: D
text: but it
speaker: B
text: OK .
speaker: D
text: Maybe it 's <disfmarker> has something to do to <vocalsound> the fact that <vocalsound> we don't have infinite training data
speaker: D
text: and <disfmarker>
speaker: B
text: We don't ?
speaker: D
text: Well !
speaker: D
text: And
speaker: D
text: so <disfmarker> Well , things are not optimal
speaker: D
text: and <disfmarker>
speaker: B
text: Yeah .
speaker: D
text: Mmm <disfmarker>
speaker: E
text: Are you <disfmarker> you were going to say why <disfmarker> what made you <disfmarker> wh what led you to do that .
speaker: D
text: Yeah . Uh , there was a p <comment> problem that we observed , um , <vocalsound> <vocalsound> that there was <disfmarker> there were , like , many insertions in the <disfmarker> in the system .
speaker: B
text: Mm - hmm .
speaker: D
text: Mmm .
speaker: B
text: Hmm .
speaker: D
text: Actually plugging in the tandem network was increasing , I <disfmarker> I <disfmarker> I think , the number of insertions .
speaker: B
text: Mm - hmm .
speaker: D
text: And , <vocalsound> um <disfmarker>
speaker: D
text: So it looked strange
speaker: D
text: and then just using the <disfmarker> the other silence probability helps .
speaker: D
text: Mmm .
speaker: D
text: Um <disfmarker>
speaker: D
text: Yeah . The next thing we will do is train this tandem on more data .
speaker: D
text: Um <disfmarker>
speaker: B
text: So , you know , in a way what it might <disfmarker> i it 's <disfmarker> it 's a little bit like <vocalsound> combining knowledge sources .
speaker: B
text: Right ?
speaker: B
text: Because <vocalsound> the fact that you have these two nets that are different sizes <pause> means they behave a little differently ,
speaker: D
text: Mm - hmm .
speaker: B
text: they find different <pause> things .
speaker: B
text: And , um ,
speaker: B
text: if you have , um <disfmarker> f the distribution that you have from , uh , f speech sounds is w <comment> sort of one source of knowledge .
speaker: D
text: Mm - hmm .
speaker: B
text: And this is <disfmarker>
speaker: B
text: and rather than just taking one minus that to get the other ,
speaker: B
text: which is essentially what 's happening ,
speaker: B
text: you have this other source of knowledge that you 're putting in there .
speaker: B
text: So you make use of both of them <vocalsound> in <disfmarker> in <pause> what you 're ending up with .
speaker: B
text: Maybe it 's better .
speaker: D
text: Yeah .
speaker: B
text: Anyway , you can probably justify anything if what 's use
speaker: D
text: Yeah .
speaker: D
text: And <disfmarker> and the features are different also .
speaker: B
text: Yeah .
speaker: D
text: I mean , the VAD doesn't use the same features there are .
speaker: B
text: Mm - hmm .
speaker: E
text: Hmm .
speaker: B
text: Oh !
speaker: D
text: Um <disfmarker>
speaker: B
text: That might be the key , actually .
speaker: D
text: Mm - hmm .
speaker: B
text: Cuz you were really thinking about speech versus nonspeech for that .
speaker: D
text: Mm - hmm .
speaker: B
text: That 's a good point .
speaker: D
text: Mmm .
speaker: D
text: Uh . Well , there are other things that <vocalsound> we should do
speaker: D
text: but , <vocalsound> um , <vocalsound> it requires time
speaker: D
text: and <disfmarker> <vocalsound> We have ideas ,
speaker: D
text: like <disfmarker> so , these things are like hav having a better VAD .
speaker: D
text: Uh , we have some ideas about that .
speaker: D
text: It would <disfmarker> <vocalsound> probably implies working a little bit on features that are more <vocalsound> suited to a voice activity detection .
speaker: B
text: Mm - hmm .
speaker: D
text: Working on the second stream .
speaker: D
text: Of course we have ideas on this also ,
speaker: D
text: but <disfmarker> <vocalsound> w we need to try different things
speaker: D
text: and <disfmarker>
speaker: D
text: Uh , but their noise estimation ,
speaker: D
text: um <disfmarker> <vocalsound> uh <disfmarker>
speaker: B
text: I mean , back on the second stream ,
speaker: B
text: I mean , that 's something we 've talked about for a while .
speaker: B
text: I mean , I think <nonvocalsound> that 's certainly a high hope .
speaker: D
text: Yeah . <vocalsound> Mmm .
speaker: B
text: Um ,
speaker: B
text: so we have this <disfmarker> this default idea about just using some sort of purely spectral thing ?
speaker: D
text: Uh , yeah .
speaker: B
text: for a second stream ?
speaker: D
text: But , um , we <disfmarker> we did a first try with this ,
speaker: D
text: and it <disfmarker> it <vocalsound> clearly hurts .
speaker: B
text: But , uh , how was the stream combined ?
speaker: D
text: Uh . <vocalsound> It was c it was just combined , um , by the acoustic model .
speaker: D
text: So there was , no neural network for the moment .
speaker: B
text: Right . So , I mean , if you just had a second stream that was just spectral and had another neural net and combined there , that <disfmarker> that , uh , <vocalsound> might be good .
speaker: D
text: Mm - hmm .
speaker: D
text: Yeah . Mm - hmm .
speaker: D
text: Mm - hmm .
speaker: D
text: Mmm .
speaker: D
text: Yeah .
speaker: D
text: Um <disfmarker>
speaker: D
text: Yeah , and the other thing , that noise estimation
speaker: D
text: and th um , maybe try to train <disfmarker> uh , the training data for the t
speaker: D
text: tandem network , right now , is like <disfmarker> i is using the noises from the Aurora task
speaker: D
text: and <vocalsound> <vocalsound> I think that people might , <vocalsound> um , try to argue about that
speaker: D
text: because <vocalsound> then in some cases we have the same noises in <disfmarker> for training the network <pause> than the noises that are used for testing ,
speaker: B
text: Right .
speaker: D
text: and <disfmarker>
speaker: D
text: So we have t n uh , to try to get rid of these <disfmarker> <vocalsound> this problem .
speaker: B
text: Yeah . Maybe you just put in some other noise ,
speaker: D
text: Mm - hmm . <vocalsound> Yeah .
speaker: B
text: something that 's different .
speaker: B
text: I mean , it <disfmarker> it 's probably helpful to have <disfmarker> have a little noise there .
speaker: D
text: Uh - huh .
speaker: B
text: But it may be something else
speaker: B
text: th at least you could say it was .
speaker: D
text: Yeah .
speaker: B
text: And then <disfmarker> if it doesn't hurt too much , though .
speaker: D
text: Uh - huh .
speaker: B
text: Yeah . That 's a good idea .
speaker: D
text: Um .
speaker: D
text: Yeah . The last thing is that I think we are getting close to human performance .
speaker: D
text: Well , that 's something I would like to investigate further ,
speaker: D
text: but ,
speaker: D
text: um ,
speaker: D
text: I did , like , um <disfmarker> I did , uh , listen to the m most noisy utterances of the SpeechDat - Car Italian
speaker: D
text: and tried to transcribe them .
speaker: D
text: And , um <disfmarker>
speaker: B
text: So this is a particular human .
speaker: B
text: This is <disfmarker> this i this is Stephane .
speaker: D
text: Yeah . So that 's <disfmarker> that 's <disfmarker>
speaker: E
text: St - Stephane .
speaker: B
text: Yeah .
speaker: D
text: that 's the <disfmarker> the flaw of the experiment .
speaker: D
text: This is just <disfmarker> i j <comment> <vocalsound> <vocalsound> it 's just one subject ,
speaker: B
text: Yeah .
speaker: E
text: Getting close .
speaker: D
text: but <disfmarker>
speaker: D
text: but still , uh , <vocalsound> what happens is <disfmarker> is that , <vocalsound> uh , the digit error rate on this is around one percent ,
speaker: B
text: Yeah .
speaker: D
text: while our system is currently at seven percent .
speaker: D
text: Um , but what happens also is that if I listen to the , um <disfmarker> <nonvocalsound> a re - synthesized version of the speech
speaker: D
text: and <pause> I re - synthesized this using a white noise that 's filtered by a LPC , uh , filter <disfmarker>
speaker: B
text: Yeah .
speaker: D
text: Um ,
speaker: D
text: well , you can argue , that , uh <disfmarker> that this is not speech ,
speaker: B
text: Yeah .
speaker: D
text: so the ear is not trained to recognize this .
speaker: D
text: But s actually it sound like <pause> whispering ,
speaker: D
text: so we are <disfmarker>
speaker: B
text: Well , I mean , it 's <disfmarker>
speaker: D
text: eh <disfmarker>
speaker: B
text: There 's two problems there .
speaker: B
text: I mean <disfmarker> I mean , so <disfmarker> so the first is <vocalsound> that by doing LPC - twelve with synthesized speech w like you 're saying , uh , it 's <disfmarker> <vocalsound> i i you 're <disfmarker> you 're adding other degradation .
speaker: D
text: Uh - huh .
speaker: B
text: Right ?
speaker: B
text: So it 's not just the noise
speaker: B
text: but you 're adding in fact some degradation
speaker: B
text: because it 's only an approximation .
speaker: B
text: Um ,
speaker: B
text: and the second thing is <disfmarker> which is m maybe more interesting <disfmarker> is that , um , <comment> <vocalsound> if you do it with whispered speech , you get this number .
speaker: B
text: What if you had <pause> done analysis <comment> re - synthesis and taken the pitch as well ?
speaker: B
text: Alright ?
speaker: B
text: So now you put the pitch in .
speaker: D
text: Uh - huh .
speaker: B
text: What would the percentage be then ?
speaker: D
text: Um <disfmarker>
speaker: B
text: See , that 's the question .
speaker: B
text: So , you see , if it 's <disfmarker> if it 's <disfmarker> if it 's , uh <disfmarker> Let 's say it 's <pause> back down to one percent again .
speaker: D
text: Uh - huh .
speaker: B
text: That would say at least for people , having the pitch is really , really important ,
speaker: B
text: which would be interesting in itself .
speaker: D
text: Uh , yeah . But <disfmarker>
speaker: B
text: Um ,
speaker: B
text: if i on the other hand , if it stayed up <pause> near five percent , <vocalsound> then I 'd say " boy , LPC n twelve is pretty crummy " .
speaker: B
text: You know ?
speaker: D
text: Uh - huh .
speaker: B
text: So I I I 'm not sure <disfmarker> I 'm not sure how we can conclude from this anything about <disfmarker> that our system is close to <vocalsound> the human performance .
speaker: D
text: Ye
speaker: D
text: Yeah . Well , the point is that eh l ey <disfmarker> the point is that , um , <vocalsound> what I <disfmarker> what I listened to when I re - synthesized the LP - the LPC - twelve <pause> spectrum <vocalsound> is in a way what the system , uh , is hearing ,
speaker: D
text: cuz @ @ <disfmarker> all the <disfmarker> all the , um , excitation <disfmarker> all the <disfmarker>
speaker: D
text: well , the excitation is <disfmarker> is not taken into account .
speaker: D
text: That 's what we do with our system .
speaker: D
text: And
speaker: B
text: Well , you 're not doing the LPC <disfmarker>
speaker: D
text: in this case <disfmarker>
speaker: B
text: I mean , so <disfmarker> so what if you did a <disfmarker>
speaker: D
text: Well , it 's not LPC , sure ,
speaker: D
text: but <disfmarker>
speaker: B
text: What if you did LPC - twenty ?
speaker: D
text: LPC <disfmarker> ?
speaker: B
text: Twenty .
speaker: B
text: Right ?
speaker: B
text: I mean , th the thing is LPC is not a <disfmarker> a really great representation of speech .
speaker: D
text: Mm - hmm .
speaker: D
text: Mm - hmm .
speaker: B
text: So , all I 'm saying is that you have in addition to the w the , uh , removal of pitch , <vocalsound> you also are doing , uh , a particular parameterization ,
speaker: D
text: Mm - hmm .
speaker: B
text: which ,
speaker: B
text: um , uh <disfmarker>
speaker: D
text: Mmm .
speaker: B
text: Uh , so , let 's see ,
speaker: B
text: how would you do <disfmarker> ? So , fo
speaker: D
text: But that 's <disfmarker> that 's what we do with our systems .
speaker: D
text: And <disfmarker>
speaker: B
text: No . Actually , we d we <disfmarker> we don't ,
speaker: B
text: because we do <disfmarker> we do , uh , <vocalsound> uh , mel filter bank , for instance .
speaker: D
text: Yeah , but is it that <disfmarker> is it that different , I mean ?
speaker: B
text: Right ?
speaker: B
text: Um , <vocalsound> I don't know what mel , <pause> uh , based synthesis would sound like ,
speaker: D
text: I
speaker: D
text: Mm - hmm .
speaker: B
text: but certainly the spectra are quite different .
speaker: D
text: Mm - hmm .
speaker: A
text: Couldn't you t couldn't you , um , test the human performance on just the original <pause> audio ?
speaker: D
text: This is the one percent number .
speaker: B
text: Yeah , it 's one percent .
speaker: D
text: Mm - hmm .
speaker: B
text: He 's trying to remove the pitch information
speaker: A
text: Oh , oh . OK ,
speaker: D
text: Mm - hmm .
speaker: B
text: and make it closer to what <disfmarker> to what we 're seeing as the feature vectors .
speaker: A
text: I see .
speaker: A
text: OK . So , y uh , your performance was one percent ,
speaker: D
text: Uh - huh .
speaker: A
text: and then when you re - synthesize with LPC - twelve it went to five .
speaker: D
text: Yeah .
speaker: A
text: OK .
speaker: B
text: I mean <disfmarker> We were <disfmarker> we were j It <disfmarker> it <disfmarker> it 's a little bit still apples and oranges
speaker: B
text: because we are choosing these features in order to be the best for recognition .
speaker: D
text: Uh - huh .
speaker: B
text: And , um ,
speaker: B
text: i if you listen to them they still might not be very <disfmarker> Even if you made something closer to what we 're gonna <disfmarker> i it might not sound very good .
speaker: D
text: Yeah .
speaker: B
text: Uh , and i the degradation from that might <disfmarker> might actually make it even harder , <vocalsound> uh , to understand than the LPC - twelve .
speaker: B
text: So all I 'm saying is that the LPC - twelve <vocalsound> puts in <disfmarker> synthesis
speaker: B
text: puts in some degradation
speaker: D
text: Uh - huh .
speaker: B
text: that 's not what we 're used to hearing ,
speaker: B
text: and is , um <disfmarker>
speaker: B
text: It 's not <disfmarker> it 's not just a question of how much information is there , as if you will always take maximum <vocalsound> advantage of any information that 's presented to you .
speaker: D
text: Mm - hmm .
speaker: B
text: In fact , you <vocalsound> hear some things better than others .
speaker: B
text: And so it <disfmarker> it isn't <disfmarker>
speaker: A
text: But <disfmarker>
speaker: B
text: But , <vocalsound> I agree that it says that , uh , the kind of information that we 're feeding it is probably , <vocalsound> um , um , a little bit , um , minimal .
speaker: B
text: There 's definitely some things that we 've thrown away .
speaker: B
text: And that 's why I was saying it might be interesting if you <disfmarker> <vocalsound> an interesting test of this would be if you <disfmarker> if you actually put the pitch back in .
speaker: B
text: So , you just extract it from the actual speech and put it back in ,
speaker: B
text: and see does that <disfmarker> is that <disfmarker> does that make the difference ?
speaker: D
text: Uh - huh .
speaker: B
text: If that <disfmarker> if that takes it down to one percent again , <vocalsound> then you 'd say " OK , it 's <disfmarker> it 's in fact having , um , <vocalsound> not just the spectral envelope but also the <disfmarker> also the <disfmarker> the pitch <vocalsound> that , uh , <comment> @ @ <comment> has the information that people can use , anyway . "
speaker: D
text: Mmm .
speaker: A
text: But from this it 's pretty safe to say that the system is with either <vocalsound> two to seven percent away from <pause> the performance of a human .
speaker: A
text: Right ?
speaker: A
text: So it 's somewhere in that range .
speaker: B
text: Well , or it 's <disfmarker> it 's <disfmarker>
speaker: B
text: Yeah , so <disfmarker>
speaker: A
text: Two <disfmarker> two to six percent .
speaker: B
text: It 's <disfmarker> it 's one point four times , uh , to , uh , seven times the error ,
speaker: D
text: To f seven times , yeah .
speaker: B
text: for Stephane .
speaker: D
text: Um .
speaker: B
text: So , uh <disfmarker>
speaker: B
text: uh , but i I don't know .
speaker: D
text: But <disfmarker> <comment> but <disfmarker>
speaker: B
text: I do don't wanna take you away from other things .
speaker: B
text: But that 's <disfmarker> <vocalsound> that 's what <disfmarker> that 's the first thing that I would be curious about , is , you know , i i <vocalsound> when you we
speaker: D
text: But the signal itself is like a mix of <disfmarker> um , of a <disfmarker> a periodic sound and , <pause> @ @ <comment> uh , unvoiced sound , and the noise
speaker: B
text: Mm - hmm .
speaker: D
text: which is mostly , <vocalsound> uh , noise .
speaker: D
text: I mean not <pause> periodic .
speaker: D
text: So , <pause> what <disfmarker> what do you mean exactly by putting back the pitch in ?
speaker: D
text: Because <disfmarker>
speaker: A
text: In the LPC synthesis ?
speaker: B
text: Yeah . You did LPC re - synthesis <disfmarker>
speaker: A
text: I think <disfmarker>
speaker: D
text: I
speaker: D
text: Uh - huh .
speaker: B
text: L PC re - synthesis .
speaker: B
text: So , <vocalsound> uh <disfmarker> and you did it with a noise source ,
speaker: D
text: Mm - hmm .
speaker: B
text: rather than with <disfmarker> with a s periodic source .
speaker: B
text: Right ?
speaker: B
text: So if you actually did real re - synthesis like you do in an LPC synthesizer , where it 's unvoiced you use noise ,
speaker: B
text: where it 's voiced you use , <vocalsound> uh , periodic pulses .
speaker: D
text: Um .
speaker: B
text: Right ?
speaker: D
text: Yeah , but it 's neither <pause> purely voiced or purely unvoiced .
speaker: D
text: Esp - especially because there is noise .
speaker: B
text: Well , it might be hard to do it
speaker: D
text: So <disfmarker>
speaker: B
text: but it but <disfmarker> but the thing is that if you <disfmarker> <vocalsound> um , if you detect that there 's periodic <disfmarker> s strong periodic components , then you can use a voiced <disfmarker> voice thing .
speaker: D
text: Oh .
speaker: D
text: Uh - huh .
speaker: D
text: Yeah .
speaker: B
text: Yeah . I mean , it 's probably not worth your time .
speaker: B
text: It 's <disfmarker> it 's a side thing
speaker: B
text: and <disfmarker> and <disfmarker> and there 's a lot to do .
speaker: D
text: Uh - huh , yeah .
speaker: B
text: But I 'm <disfmarker> I 'm just saying , at least as a thought experiment , <vocalsound> that 's what I would wanna test .
speaker: D
text: Mm - hmm .
speaker: B
text: Uh , I wan would wanna drive it with a <disfmarker> a <disfmarker> a two - source system rather than a <disfmarker> than a one - source system .
speaker: D
text: Mm - hmm .
speaker: D
text: Mm - hmm .
speaker: B
text: And then that would tell you whether in fact it 's <disfmarker>
speaker: B
text: Cuz we 've talked about , like , this harmonic tunneling or <vocalsound> other things that people have done based on pitch ,
speaker: B
text: maybe that 's really a key element .
speaker: B
text: Maybe <disfmarker> maybe , uh , <vocalsound> uh , without that , it 's <disfmarker> it 's not possible to do a whole lot better than we 're doing .
speaker: B
text: That <disfmarker> that could be .
speaker: D
text: Yeah .
speaker: D
text: That 's what I was thinking by doing this es experiment ,
speaker: D
text: like <disfmarker>
speaker: B
text: Yeah .
speaker: D
text: Mmm . <vocalsound> Evi
speaker: B
text: But , I mean , other than that , I don't think it 's <disfmarker>
speaker: B
text: I mean , other than the pitch de information , <vocalsound> it 's hard to imagine that there 's a whole lot more <vocalsound> in the signal that <disfmarker> that , uh <disfmarker> that we 're throwing away that 's important .
speaker: D
text: Yeah , but <disfmarker>
speaker: D
text: Yeah . <vocalsound> Mm - hmm . Yeah , right .
speaker: B
text: Right ? I mean , we 're using <vocalsound> a fair number of filters in the filter bank
speaker: B
text: and <disfmarker> uh <disfmarker>
speaker: D
text: Mm - hmm .
speaker: D
text: Uh , yeah .
speaker: B
text: Hmm .
speaker: B
text: Yeah .
speaker: D
text: Um .
speaker: D
text: Yeah , that 's it .
speaker: B
text: Yeah . That look
speaker: B
text: Yeah .
speaker: B
text: That 's <disfmarker> that 's <disfmarker> I mean , one <disfmarker> one percent is sort of what I would <disfmarker> I would figure .
speaker: B
text: If somebody was paying really close attention , you might get <disfmarker>
speaker: B
text: I would actually think that if , <vocalsound> you looked at people on various times of the day and different amounts of attention , you might actually get up to three or four percent error on digits .
speaker: D
text: Mm - hmm .
speaker: B
text: Uh , <vocalsound> uh <disfmarker>
speaker: D
text: Um .
speaker: B
text: So it 's <disfmarker>
speaker: B
text: you know , we 're not <disfmarker> we 're not incredibly far off .
speaker: B
text: On the other hand , with any of these numbers except maybe the one percent , it 's st it 's not actually usable in a commercial system with a full telephone number or something .
speaker: D
text: Uh - huh .
speaker: D
text: Yeah . At these noise levels .
speaker: D
text: Yeah . Mm - hmm .
speaker: B
text: Yeah .
speaker: B
text: Right .
speaker: D
text: Well , yeah . These numbers , I mean .
speaker: D
text: Mmm .
speaker: B
text: Good .
speaker: B
text: Um , while we 're still on Aurora stuff <pause> maybe you can talk a little about the status with the , uh , <vocalsound> Wall Street Journal <vocalsound> things for it .
speaker: A
text: So I 've , um , downloaded , uh , a couple of things from Mississippi State .
speaker: A
text: Um , one is their <vocalsound> software <disfmarker>
speaker: A
text: their , uh , LVCSR system .
speaker: A
text: Downloaded the latest version of that .
speaker: A
text: Got it compiled and everything .
speaker: A
text: Um , downloaded the scripts .
speaker: A
text: They wrote some scripts that sort of make it easy to run <vocalsound> the system on the Wall Street Journal , uh , data .
speaker: A
text: Um , so I haven't run the scripts yet .
speaker: A
text: Uh , I 'm waiting <disfmarker> there was one problem with part of it
speaker: A
text: and I wrote a note to Joe asking him about it .
speaker: A
text: So I 'm waiting to hear from him .
speaker: A
text: But , um , I did print something out just to give you an idea about where the system is .
speaker: A
text: Uh , <vocalsound> they <disfmarker> on their web site they , uh , did this little table of where their system performs relative to other systems that have done this <disfmarker> this task .
speaker: A
text: And , um , the Mississippi State system <vocalsound> using a bigram grammar , uh , is at about eight point two percent .
speaker: A
text: Other comparable systems from , uh <disfmarker> <vocalsound> were getting from , uh , like six point nine , six point eight percent .
speaker: A
text: So they 're <disfmarker>
speaker: B
text: This is on clean test set ?
speaker: A
text: This is on clean <disfmarker> on clean stuff . Yeah .
speaker: A
text: They <disfmarker> they 've started a table <vocalsound> where they 're showing their results on various different noise conditions
speaker: A
text: but they <disfmarker> they don't have a whole lot of it filled in
speaker: A
text: and <disfmarker> <vocalsound> and I didn't notice until after I 'd printed it out that , um , <vocalsound> they don't say here <pause> what these different testing conditions are .
speaker: A
text: You actually have to click on it on the web site to see them .
speaker: A
text: So I <disfmarker> I don't know what those <pause> numbers really mean .
speaker: B
text: What kind of numbers are they getting on these <disfmarker> on the test conditions ?
speaker: A
text: Well , see , I was a little confused
speaker: A
text: because on this table , I 'm <disfmarker> the they 're showing word error rate .
speaker: A
text: But on this one ,
speaker: A
text: I <disfmarker> I don't know if these are word error rates
speaker: A
text: because they 're really big .
speaker: A
text: So , <vocalsound> under condition one here it 's ten percent .
speaker: A
text: Then under three it goes to sixty - four point six percent .
speaker: B
text: Yeah , that 's probably Aurora .
speaker: B
text: I mean <disfmarker>
speaker: A
text: Yeah .
speaker: A
text: So m I guess maybe they 're error rates
speaker: A
text: but they 're , uh <disfmarker> they 're really high .
speaker: B
text: I <disfmarker> I <disfmarker> I don't find that surpri
speaker: A
text: So <disfmarker>
speaker: B
text: I mean , we <disfmarker>
speaker: B
text: W what 's <disfmarker> what 's some of the lower error rates on <disfmarker> on <disfmarker> on <disfmarker> uh , some of the higher error rates on , uh , <vocalsound> some of these w uh , uh , highly mismatched difficult conditions ?
speaker: B
text: What 's a <disfmarker> ?
speaker: D
text: Uh . Yeah , it 's around fifteen to twenty percent .
speaker: A
text: Correct ?
speaker: D
text: And the baseline , eh <disfmarker>
speaker: A
text: Accuracy ?
speaker: D
text: Uh , error rate .
speaker: B
text: Yeah .
speaker: D
text: Twenty percent error rate ,
speaker: B
text: Yeah . So twenty percent error rate on digits .
speaker: D
text: and <disfmarker>
speaker: B
text: So if you 're doing <disfmarker> so if you 're doing ,
speaker: D
text: and <disfmarker>
speaker: A
text: Oh , oh , on digits .
speaker: A
text: Yeah .
speaker: D
text: On digits .
speaker: D
text: And this is so <disfmarker> so <disfmarker> still the baseline .
speaker: A
text: OK .
speaker: B
text: you know ,
speaker: B
text: sixty - thousand <disfmarker>
speaker: D
text: Right ?
speaker: A
text: Yeah .
speaker: B
text: Yeah ,
speaker: B
text: and if you 're saying sixty - thousand word recognition , getting sixty percent error on some of these noise condition not at all surprising .
speaker: A
text: Yeah .
speaker: D
text: The baseline is sixty percent also on digits ,
speaker: A
text: Oh , is it ?
speaker: D
text: on the m more <pause> mismatched conditions .
speaker: A
text: OK .
speaker: D
text: So .
speaker: B
text: Yeah .
speaker: A
text: So , yeah , that 's probably what it is then .
speaker: A
text: Yeah . So they have a lot of different conditions that they 're gonna be filling out .
speaker: B
text: It 's a bad sign when you <disfmarker> looking at the numbers , you can't tell whether it 's accuracy or error rate .
speaker: A
text: Yeah .
speaker: A
text: Yeah . It 's <disfmarker> it 's gonna be hard .
speaker: A
text: Um ,
speaker: A
text: they 're <disfmarker> I I 'm still waiting for them to <pause> release the , um , <vocalsound> multi - CPU version of their scripts ,
speaker: A
text: cuz right now their script only handles processing on a single CPU ,
speaker: A
text: which will take a really long time to run .
speaker: A
text: So .
speaker: B
text: This is for the training ?
speaker: A
text: But their s
speaker: A
text: Uh <disfmarker> I beli
speaker: A
text: Yes , for the training <pause> also .
speaker: B
text: OK .
speaker: A
text: And , um , they 're supposed to be coming out with it any time ,
speaker: A
text: the multi - CPU one .
speaker: B
text: OK .
speaker: A
text: So , as soon as they get that , then I 'll <disfmarker> I 'll grab those too
speaker: A
text: and so w
speaker: B
text: Yeah . Cuz we have to get started ,
speaker: B
text: cuz it 's <disfmarker> cuz , uh ,
speaker: A
text: Yeah .
speaker: A
text: Yeah . I 'll go ahead and try to run it though with just the single CPU one ,
speaker: B
text: if the <disfmarker>
speaker: A
text: and <disfmarker> I <disfmarker> they <disfmarker> they , <vocalsound> um , released like a smaller data set that you can use that only takes like sixteen hours to train and stuff .
speaker: A
text: So I can <disfmarker> I can run it on that just to make sure that the <disfmarker> <vocalsound> the thing works and everything .
speaker: B
text: Oh ! Good .
speaker: B
text: Yeah .
speaker: E
text: Hmm .
speaker: B
text: Cuz we 'll <disfmarker>
speaker: B
text: I guess the actual evaluation will be in six weeks or something .
speaker: B
text: So .
speaker: B
text: Is that about right <pause> you think ?
speaker: D
text: Uh , we don't know yet , I <disfmarker> I think .
speaker: B
text: Really , we don't know ?
speaker: D
text: Uh - huh .
speaker: D
text: Um .
speaker: B
text: Hmm .
speaker: A
text: It wasn't on the conference call this morning ?
speaker: D
text: No .
speaker: A
text: Hmm .
speaker: A
text: Did they say anything on the conference call <pause> about , um , how the <pause> Wall Street Journal part of the test was going to be <pause> run ?
speaker: A
text: Because I <disfmarker> I thought I remembered hearing that some sites <vocalsound> were saying that they didn't have the compute to be able to run the Wall Street Journal stuff at their place ,
speaker: D
text: No .
speaker: D
text: Mmm .
speaker: A
text: so there was some talk about having Mississippi State run <pause> the systems for them .
speaker: A
text: And I <disfmarker>
speaker: A
text: Did <disfmarker> did that come up at all ?
speaker: D
text: Uh , no .
speaker: D
text: Well , this <disfmarker> first , this was not the point at all of this <disfmarker> the meeting today
speaker: A
text: Oh , OK .
speaker: D
text: and ,
speaker: D
text: uh , frankly , I don't know
speaker: B
text: Some
speaker: D
text: because I d <comment> didn't read also the <pause> most recent mails about <vocalsound> the large - vocabulary task .
speaker: D
text: But , <vocalsound> uh , did you <disfmarker> do you still , uh , get the mails ?
speaker: D
text: You 're not on the mailing list or what ?
speaker: A
text: Hmm - mm .
speaker: A
text: The only , um , mail I get is from Mississippi State <disfmarker>
speaker: D
text: Uh - huh .
speaker: A
text: so <disfmarker>
speaker: D
text: Oh , yeah . So we should have a look at this .
speaker: A
text: about their system .
speaker: A
text: I <disfmarker> I don't get any <pause> mail about <disfmarker>
speaker: B
text: I have to say , there 's uh something funny - sounding about saying that one of these big companies doesn't have enough cup compute power do that ,
speaker: B
text: so they 're having to have it done by Mississippi State .
speaker: A
text: Yeah .
speaker: B
text: It just <disfmarker> <vocalsound> just sounds funny .
speaker: A
text: Yeah . It does .
speaker: B
text: But ,
speaker: B
text: anyway .
speaker: A
text: Yeah . I 'm <disfmarker> I 'm wondering about that
speaker: A
text: because there 's this whole issue about , you know , simple tuning parameters , like word insertion penalties .
speaker: D
text: Mm - hmm .
speaker: A
text: And <pause> whether or not those are going to be tuned or not ,
speaker: A
text: and <disfmarker> <comment> So .
speaker: D
text: Mm - hmm .
speaker: A
text: I mean , it makes a big difference .
speaker: A
text: If you change your front - end , you know , the scale is completely <disfmarker> can be completely different ,
speaker: A
text: so .
speaker: A
text: It seems reasonable that that at least should be tweaked to match the front - end .
speaker: A
text: But <disfmarker>
speaker: D
text: You didn't get any answer from <pause> Joe ?
speaker: A
text: I did ,
speaker: A
text: but Joe <pause> said , you know , " what you 're saying makes sense
speaker: D
text: Uh - huh .
speaker: A
text: and <pause> I don't know " .
speaker: D
text: Uh - huh .
speaker: A
text: So he doesn't know what the answer is .
speaker: A
text: I mean , that 's th We had this back and forth a little bit about , <vocalsound> you know , are sites gonna <disfmarker> are you gonna run this data for different sites ?
speaker: A
text: And , well , if <disfmarker> if Mississippi State runs it , then maybe they 'll do a little optimization on that <pause> parameter ,
speaker: A
text: and , uh <disfmarker>
speaker: A
text: But then he wasn't asked to run it for anybody .
speaker: A
text: So i it 's <disfmarker> it 's just not clear yet what 's gonna happen .
speaker: D
text: Mm - hmm .
speaker: A
text: Uh , he 's been putting this stuff out on their web site and <disfmarker> for people to grab
speaker: A
text: but I haven't heard too much about what 's happening .
speaker: B
text: So it could be <disfmarker> I mean , Chuck and I had actually talked about this a couple times , and <disfmarker> and <disfmarker> over some lunches , I think , <vocalsound> that , um , <vocalsound> one thing that we might wanna do <disfmarker>
speaker: B
text: The - there 's this question about , you know , what do you wanna scale ?
speaker: B
text: Suppose y you can't adjust <vocalsound> these word insertion penalties and so forth ,
speaker: B
text: so you have to do everything at the level of the features .
speaker: B
text: What could you do ?
speaker: B
text: And , uh , one thing I had suggested at an earlier time was maybe some sort of scaling ,
speaker: B
text: some sort of root or <disfmarker> or something of the , um , <vocalsound> uh , features .
speaker: B
text: But the problem with that is that isn't quite the same ,
speaker: B
text: it occurred to me later ,
speaker: B
text: because what you really want to do is scale the , uh , @ @ <comment> the range of the likelihoods rather than <disfmarker>
speaker: D
text: Nnn , the dist
speaker: D
text: Yeah .
speaker: B
text: But , <vocalsound> what might get at something similar , it just occurred to me , is kind of an intermediate thing <disfmarker>
speaker: B
text: is because we do this strange thing that we do with the tandem system , at least in that system what you could do <vocalsound> is take the , um , <vocalsound> uh , values that come out of the net ,
speaker: B
text: which are something like log probabilities ,
speaker: B
text: and scale those .
speaker: B
text: And then , uh , um <disfmarker> <pause> then at least those things would have the right values
speaker: B
text: or the right <disfmarker> the right range .
speaker: B
text: And then that goes into the rest of it and then that 's used as observations .
speaker: B
text: So it 's <disfmarker> it 's , <vocalsound> um , another way to do it .
speaker: D
text: Mm - hmm .
speaker: D
text: Mm - hmm .
speaker: D
text: But , these values are not directly used as probabilities anyway .
speaker: B
text: I know they 're not .
speaker: D
text: So there are <disfmarker> there is <disfmarker>
speaker: B
text: I know they 're not .
speaker: B
text: But <disfmarker> but ,
speaker: B
text: you know <disfmarker>
speaker: D
text: Uh - huh .
speaker: B
text: So because what we 're doing is pretty strange and complicated , we don't really know what the effect is <pause> at the other end .
speaker: D
text: Mm - hmm .
speaker: B
text: So , <vocalsound> um , <pause> my thought was maybe <disfmarker>
speaker: B
text: I mean , they 're not used as probabilities ,
speaker: B
text: but the log probabilities <disfmarker>
speaker: B
text: we 're taking advantage of the fact that something like log probabilities has more of a Gaussian shape than Gaus - than <vocalsound> probabilities ,
speaker: B
text: and so we can model them better .
speaker: B
text: So , <pause> in a way we 're taking advantage of the fact that they 're probabilities ,
speaker: B
text: because they 're this quantity that looks kind of Gaussian when you take it 's log .
speaker: B
text: So , <comment> <vocalsound> uh , maybe <disfmarker> maybe it would have a <disfmarker> a reasonable effect to do that .
speaker: D
text: Mm - hmm .
speaker: B
text: I d I don't know .
speaker: B
text: But , <pause> I mean , I guess we still haven't had a <disfmarker> <vocalsound> a ruling back on this .
speaker: B
text: And we may end up being in a situation where we just you know really can't change the <vocalsound> word insertion penalty .
speaker: B
text: But the other thing we could do <vocalsound> is <disfmarker> also we could <disfmarker>
speaker: B
text: I mean , this <disfmarker> this may not help us , <vocalsound> uh , in the evaluation
speaker: B
text: but it might help us in our understanding at least .
speaker: B
text: We might , <vocalsound> just run it with different insper insertion penalties ,
speaker: B
text: and show that , uh , " well , OK , not changing it , <vocalsound> playing the rules the way you wanted , we did this . But in fact if we did that , it made a <disfmarker> <pause> a big difference . "
speaker: A
text: I wonder if it <disfmarker> it might be possible to , uh , simulate the back - end with some other system .
speaker: A
text: So we <disfmarker> we get our f front - end features ,
speaker: A
text: and then ,
speaker: A
text: uh ,
speaker: A
text: as part of the process of figuring out the scaling of these features , <comment> you know , if we 're gonna take it to a root or to a power or something , <comment> <vocalsound> we have some back - end that we attach onto our features that sort of simulates what would be happening .
speaker: B
text: Mm - hmm .
speaker: A
text: Um ,
speaker: B
text: And just adjust it until it 's the best number ?
speaker: A
text: and just adjust it until that <disfmarker> our l version of the back - end , uh , decides that <disfmarker> that <disfmarker>
speaker: B
text: Well , we can probably use the real thing ,
speaker: B
text: can't we ?
speaker: B
text: And then jus just , uh , <vocalsound> use it on a reduced test set or something .
speaker: A
text: Yeah . Oh , yeah .
speaker: A
text: That 's true .
speaker: B
text: Yeah .
speaker: A
text: And then we just use that to determine some scaling factor that we use .
speaker: B
text: Yeah . So I mean , I I think that that 's a reasonable thing to do
speaker: B
text: and the only question is what 's the actual knob that we use ?
speaker: B
text: And the knob that we use should <disfmarker>
speaker: A
text: Mm - hmm .
speaker: B
text: uh , uh , unfortunately , like I say , I don't know the analytic solution to this
speaker: B
text: cuz what we really want to do is change the scale of the likelihoods ,
speaker: B
text: not the cha not the scale of the <disfmarker> <vocalsound> the <pause> observations .
speaker: A
text: Mm - hmm .
speaker: B
text: But <disfmarker> but , uh <disfmarker>
speaker: D
text: Mm - hmm .
speaker: A
text: Yeah .
speaker: E
text: Out of curiosity , what <disfmarker> what kind of recognizer <pause> is the one from Mississippi State ?
speaker: A
text: Uh , w what do you mean when you say " what kind " ?
speaker: E
text: Is it <disfmarker> ?
speaker: E
text: Um , is it like a <pause> Gaussian mixture model ?
speaker: A
text: Yeah .
speaker: A
text: Gaussian mixture model .
speaker: E
text: OK .
speaker: A
text: It 's the same system that they use <pause> when they participate in the Hub - five evals .
speaker: A
text: It 's a , <vocalsound> um <disfmarker> sort of <pause> came out of , uh <disfmarker> uh , looking a lot like HTK .
speaker: A
text: I mean , they started off with <disfmarker> um , when they were building their system they were always comparing to HTK to make sure they were getting similar results .
speaker: A
text: And so , <vocalsound> it 's a Gaussian mixture system ,
speaker: A
text: uh <disfmarker>
speaker: B
text: Do they have the same sort of mix - down sort of procedure , where they <vocalsound> start off with a small number of some things
speaker: A
text: I don't know .
speaker: A
text: Yeah . And then <pause> divide the mixtures in half .
speaker: B
text: and <disfmarker> ?
speaker: B
text: Yeah .
speaker: A
text: I don't know if they do that .
speaker: A
text: I 'm not really sure .
speaker: B
text: Yeah .
speaker: E
text: Hmm .
speaker: B
text: D Do you know what kind of tying they use ?
speaker: B
text: Are they <disfmarker> they sort of <disfmarker> some sort of <disfmarker> a bunch of Gaussians that they share across everything ?
speaker: B
text: Or <disfmarker> <vocalsound> or if it 's <disfmarker> ?
speaker: A
text: Yeah , th I have <disfmarker> I <disfmarker> I <disfmarker> I don't have it up here
speaker: A
text: but I have a <disfmarker> <pause> the whole system description , that describes exactly what their <pause> system is
speaker: B
text: OK .
speaker: A
text: and I <disfmarker> I 'm not sure .
speaker: A
text: But , um <disfmarker>
speaker: B
text: OK .
speaker: A
text: It 's some kind of a mixture of Gaussians and , <vocalsound> uh , clustering
speaker: A
text: and , uh <disfmarker>
speaker: A
text: They 're <disfmarker> they 're trying to put in sort of all of the standard features that people use nowadays .
speaker: E
text: Mm - hmm .
speaker: B
text: So the other , uh , Aurora thing maybe is <disfmarker>
speaker: B
text: I I dunno if any of this is gonna <vocalsound> <pause> come in in time to be relevant ,
speaker: B
text: but , uh , we had talked about , uh , <comment> Guenter <vocalsound> playing around , uh , uh , over in Germany
speaker: D
text: Mm - hmm .
speaker: B
text: and <disfmarker> and , @ @ <comment> uh , <pause> possibly coming up with something <vocalsound> that would , uh , <pause> uh , fit in later .
speaker: B
text: Uh , I saw that other mail where he said that he <disfmarker> <vocalsound> uh , it wasn't going to work for him to do CVS .
speaker: D
text: Yeah .
speaker: D
text: Yeah . So now he has a version of the software .
speaker: B
text: So he just has it all sitting there .
speaker: B
text: Yeah .
speaker: D
text: Yeah .
speaker: D
text: Um <disfmarker>
speaker: D
text: Mm - hmm .
speaker: B
text: So if he 'll <disfmarker>
speaker: B
text: he might work on improving the noise estimate
speaker: B
text: or on <vocalsound> some histogram things ,
speaker: D
text: Yeah .
speaker: B
text: or <disfmarker>
speaker: D
text: Mm - hmm .
speaker: B
text: Yeah . I just saw the Eurospeech <disfmarker>
speaker: B
text: We <disfmarker> we didn't talk about it at our meeting
speaker: B
text: but I just saw the <disfmarker> just read the paper .
speaker: B
text: Someone , I forget the name , <comment> and <disfmarker> and Ney , uh , about histogram equalization ?
speaker: B
text: Did you see that one ?
speaker: D
text: Um , it was a poster .
speaker: D
text: Or <disfmarker>
speaker: B
text: Yeah . I mean , I just read the paper .
speaker: D
text: Yeah .
speaker: B
text: I didn't see the poster .
speaker: D
text: Yeah .
speaker: D
text: Um <disfmarker> <vocalsound> It was something <pause> similar to n <vocalsound> on - line normalization finally <disfmarker>
speaker: D
text: I mean , in <vocalsound> the idea of <disfmarker> of normalizing <disfmarker>
speaker: B
text: Yeah . But it 's a little more <disfmarker> it <disfmarker> it 's a little finer ,
speaker: B
text: right ?
speaker: D
text: Yeah .
speaker: B
text: So they had like ten quantiles
speaker: B
text: and <disfmarker> <vocalsound> and they adjust the distribution .
speaker: D
text: Right .
speaker: B
text: So you <disfmarker> you have the distributions from the training set ,
speaker: D
text: N
speaker: B
text: and then , uh <disfmarker>
speaker: B
text: So this is just a <disfmarker> a histogram of <disfmarker> of <vocalsound> the amplitudes , I guess .
speaker: B
text: Right ?
speaker: B
text: And then <disfmarker> <vocalsound> Um , people do this in image processing some .
speaker: D
text: Mm - hmm .
speaker: B
text: You have this kind of <disfmarker> <vocalsound> of histogram of <disfmarker> of levels of brightness or whatever .
speaker: B
text: And <disfmarker> and <disfmarker> and then , <vocalsound> when you get a new <disfmarker> new thing that you <disfmarker> you want to adjust to be <pause> better in some way , <vocalsound> you adjust it so that the histogram of the new data looks like the old data .
speaker: A
text: Hmm .
speaker: B
text: You do this kind of <vocalsound> piece - wise linear or , <vocalsound> uh , some kind of piece - wise approximation .
speaker: B
text: They did a <disfmarker> uh one version that was piece - wise linear and another that had a power law thing between them <disfmarker> <vocalsound> between the <pause> points .
speaker: B
text: And , uh ,
speaker: B
text: they said they s they sort of see it in a way as s for the speech case <comment> <disfmarker> as being kind of a generalization of spectral subtraction in a way ,
speaker: B
text: because , you know , in spectral subtraction you 're trying to <vocalsound> get rid of this excess energy .
speaker: B
text: Uh , you know , it 's not supposed to be there .
speaker: B
text: Uh <disfmarker> <vocalsound> and , uh ,
speaker: B
text: this is sort of <pause> <vocalsound> adjusting it for <disfmarker> for a lot of different levels .
speaker: B
text: And then they have s they have some kind of , <vocalsound> uh , <pause> a floor or something ,
speaker: E
text: Hmm .
speaker: B
text: so if it gets too low you don't <disfmarker> don't do it .
speaker: A
text: Hmm .
speaker: B
text: And they <disfmarker> they claimed very nice results ,
speaker: D
text: Mm - hmm .
speaker: B
text: and <disfmarker>
speaker: A
text: So is this a histogram across different frequency bins ?
speaker: A
text: Or <disfmarker> ?
speaker: B
text: Um , I think this i
speaker: B
text: You know , I don't remember that .
speaker: B
text: Do you remember <disfmarker> ?
speaker: D
text: I think they have , yeah , different histograms .
speaker: D
text: I uh <disfmarker>
speaker: D
text: Something like one per <pause> frequency band ,
speaker: B
text: One <disfmarker>
speaker: D
text: or <disfmarker>
speaker: B
text: One per critical <disfmarker>
speaker: A
text: So , one histogram per frequency bin .
speaker: D
text: But I did <disfmarker>
speaker: D
text: Yeah , I guess .
speaker: D
text: But I should read the paper .
speaker: A
text: And that 's <disfmarker>
speaker: D
text: I just went <pause> through the poster quickly ,
speaker: B
text: Yeah .
speaker: B
text: And I don't remember whether it was <pause> filter bank things
speaker: A
text: So th
speaker: D
text: and I didn't <disfmarker>
speaker: A
text: Oh .
speaker: B
text: or whether it was FFT bins
speaker: B
text: or <disfmarker>
speaker: A
text: Huh .
speaker: A
text: And <disfmarker> and that <disfmarker> that , um , <pause> histogram represents <pause> the <pause> different energy levels that have been seen at that <pause> frequency ?
speaker: B
text: I don't remember that .
speaker: B
text: And how often they <disfmarker> you 've seen them . Yeah .
speaker: E
text: Hmm .
speaker: A
text: Uh - huh .
speaker: B
text: Yeah . And they do <disfmarker> they said that they could do it for the test <disfmarker>
speaker: B
text: So you don't have to change the training .
speaker: B
text: You just do a measurement over the training .
speaker: B
text: And then , uh , for testing , uh , you can do it for one per utterance .
speaker: B
text: Even relatively short utterances .
speaker: B
text: And they claim it <disfmarker> it works pretty well .
speaker: A
text: So they , uh <disfmarker>
speaker: A
text: Is the idea that you <disfmarker> you run a test utterance through some histogram generation thing
speaker: A
text: and then you compare the histograms and that tells you <vocalsound> what to do to the utterance to make it more like <disfmarker> ?
speaker: B
text: I guess in pri
speaker: B
text: Yeah .
speaker: B
text: In principle .
speaker: B
text: I didn't read carefully how they actually implemented it ,
speaker: A
text: I see .
speaker: A
text: Hmm .
speaker: A
text: Yeah .
speaker: B
text: whether it was some , <vocalsound> uh , on - line thing , or whether it was a second pass , or what .
speaker: B
text: But <disfmarker> but they <disfmarker> <vocalsound> That <disfmarker> that was sort of the idea .
speaker: A
text: Hmm .
speaker: B
text: So that <disfmarker> that seemed , you know , different .
speaker: B
text: We 're sort of curious about , uh , what are some things that are , u u um , <vocalsound> @ @ <comment> <pause> conceptually quite different from what we 've done .
speaker: B
text: Cuz we <disfmarker> you know , one thing that w that ,
speaker: A
text: Mm - hmm .
speaker: B
text: uh , Stephane and Sunil seemed to find , <vocalsound> uh , was , you know , they could actually make a unified piece of software that handled a range of different things that people were talking about ,
speaker: B
text: and it was really just sort of setting of different <pause> constants .
speaker: B
text: And it would turn , you know , one thing into another .
speaker: B
text: It 'd turn Wiener filtering into spectral subtraction , or whatever .
speaker: B
text: But there 's other things that we 're not doing .
speaker: B
text: So , we 're not making any use of pitch ,
speaker: B
text: uh , uh , which again , might <disfmarker> might be important ,
speaker: B
text: uh , because the stuff between the harmonics is probably a schmutz .
speaker: B
text: And <disfmarker> and the , <vocalsound> uh , transcribers will have fun with that .
speaker: B
text: Uh <disfmarker> <vocalsound> And , um ,
speaker: B
text: the , uh , stuff at the harmonics isn't so much .
speaker: B
text: And <disfmarker> and , uh <disfmarker>
speaker: B
text: And we there 's this overall idea of really sort of matching the <disfmarker> the hi distributions somehow .
speaker: B
text: Uh , not just ,
speaker: B
text: um , <vocalsound> um <disfmarker>
speaker: B
text: not just subtracting off your estimate of the noise .
speaker: B
text: So .
speaker: B
text: So I guess , uh , <vocalsound> Guenter 's gonna play around with some of these things now over this next <pause> period ,
speaker: D
text: Uh , I dunno .
speaker: B
text: or <disfmarker> ?
speaker: D
text: I don't have feedback from him ,
speaker: D
text: but
speaker: B
text: Yeah .
speaker: D
text: I guess he 's gonna , maybe <disfmarker>
speaker: B
text: Well , he 's got it anyway ,
speaker: B
text: so he can .
speaker: D
text: Yeah .
speaker: D
text: Uh - huh .
speaker: B
text: So potentially if he came up with something that was useful , like a diff a better noise estimation module or something , he could ship it to you guys u up there
speaker: D
text: Yeah .
speaker: B
text: and
speaker: B
text: we could put it in .
speaker: D
text: Mm - hmm . <vocalsound> Mm - hmm .
speaker: B
text: Yeah .
speaker: B
text: Yeah .
speaker: B
text: So ,
speaker: B
text: that 's good .
speaker: B
text: So , why don't we just , uh , um <disfmarker>
speaker: B
text: I think starting <disfmarker> <pause> starting a w couple weeks from now , especially if you 're not gonna be around for a while , we 'll <disfmarker> we 'll be shifting more over to some other <disfmarker> <vocalsound> other territory .
speaker: B
text: But , uh , uh , <comment> uh ,
speaker: B
text: n not <disfmarker> not so much in this meeting about Aurora ,
speaker: B
text: but <disfmarker> but , uh ,
speaker: B
text: uh , maybe just , uh , quickly today about <disfmarker> maybe you could just say a little bit about what you 've been talking about with Michael .
speaker: B
text: And <disfmarker>
speaker: B
text: and then Barry can say something about <pause> what <comment> <disfmarker> what we 're talking about .
speaker: C
text: OK .
speaker: C
text: So Michael Kleinschmidt , who 's a PHD student from Germany , <vocalsound> showed up this week .
speaker: C
text: He 'll be here for about six months .
speaker: C
text: And he 's done some work using <vocalsound> an auditory model <pause> of , um , <vocalsound> human hearing ,
speaker: C
text: and <pause> using that f uh , to generate speech recognition features .
speaker: C
text: And <pause> he did <vocalsound> work back in Germany <vocalsound> with , um , a toy recognition system <vocalsound> using , um , isolated <vocalsound> digit recognition <vocalsound> as the task .
speaker: C
text: It was actually just a single - layer neural network <vocalsound> that classified words <disfmarker>
speaker: C
text: classified digits , <vocalsound> in fact .
speaker: C
text: Um , and <pause> he tried that on <disfmarker> I think on some Aurora data and got results that he thought <pause> seemed respectable .
speaker: C
text: And
speaker: C
text: he w he 's coming here to u u use it on a <vocalsound> uh , a real speech recognition system .
speaker: C
text: So I 'll be working with him on that .
speaker: C
text: And , um ,
speaker: C
text: maybe I should say a little more about these features ,
speaker: C
text: although I don't understand them that well .
speaker: C
text: The <disfmarker> I think it 's a two - stage idea .
speaker: C
text: And , um , <vocalsound> the first stage of these features correspond to what 's called the peripheral <vocalsound> auditory system .
speaker: C
text: And <vocalsound> I guess that is like <vocalsound> a filter bank with a compressive nonlinearity .
speaker: C
text: And <vocalsound> I 'm - I 'm not sure what we have @ @ in there that isn't already modeled in something like , <vocalsound> um , <pause> PLP .
speaker: C
text: I should learn more about that .
speaker: C
text: And then <vocalsound> the second stage <pause> is , um , <vocalsound> the most different thing , I think , from what we usually do .
speaker: C
text: It 's , um <disfmarker> <vocalsound> <vocalsound> it computes features which are , <vocalsound> um , <vocalsound> based on <disfmarker> sort of like based on diffe different w um , wavelet basis functions <vocalsound> used to analyze <vocalsound> the input .
speaker: C
text: So th he uses analysis functions called <vocalsound> Gabor functions ,
speaker: C
text: um , <vocalsound> which have a certain <vocalsound> extent , um , <vocalsound> in time and in frequency .
speaker: C
text: And <vocalsound> the idea is these are used to sample , <vocalsound> um , the signal in a represented as a time - frequency representation .
speaker: C
text: So you 're <pause> sampling some piece of this time - frequency plane .
speaker: C
text: And , um , <vocalsound> that , <vocalsound> um , is <disfmarker> is interesting ,
speaker: C
text: cuz , <vocalsound> @ @ for <disfmarker> for one thing , you could use it , <vocalsound> um , in a <disfmarker> a multi - scale way .
speaker: C
text: You could have these <disfmarker>
speaker: C
text: instead of having everything <disfmarker> like we use a twenty - five millisecond or so analysis window , <vocalsound> typically ,
speaker: C
text: um , and that 's our time scale for features ,
speaker: C
text: but you could <disfmarker> <vocalsound> using this , um , basis function idea , you could have some basis functions
speaker: C
text: which have a lot longer time scale
speaker: C
text: and , um , some which have a lot shorter ,
speaker: C
text: and <vocalsound> so it would be like <pause> a set of multi - scale features .
speaker: C
text: So he 's interested in , um <disfmarker>
speaker: C
text: Th - this is <disfmarker> because it 's , um <disfmarker> there are these different parameters for the shape of these <vocalsound> basis functions , <vocalsound> um <disfmarker> <vocalsound> there are a lot of different possible basis functions .
speaker: C
text: And so he <disfmarker> <vocalsound> he actually does <vocalsound> an optimization procedure to choose an <disfmarker> <vocalsound> an optimal set of basis functions out of all the possible ones .
speaker: A
text: Hmm .
speaker: A
text: H
speaker: A
text: What does he do to choose those ?
speaker: C
text: The method he uses is kind of funny <disfmarker>
speaker: C
text: is , <comment> <vocalsound> um , <vocalsound> he starts with <disfmarker> he has a set of M of them .
speaker: C
text: Um ,
speaker: C
text: he <disfmarker> and then <pause> he uses that to classify <disfmarker>
speaker: C
text: I mean , he t he tries , um , <vocalsound> using <pause> just M minus one of them .
speaker: C
text: So there are M possible subsets of this <vocalsound> length - M vector .
speaker: C
text: He tries classifying , using each of the M <vocalsound> possible sub - vectors .
speaker: D
text: Hmm .
speaker: C
text: Whichever sub - vector , <vocalsound> um , works the <disfmarker> the best , I guess , he says <disfmarker> <vocalsound> the <disfmarker> the fe feature that didn't use was the most useless feature ,
speaker: B
text: Y yeah .
speaker: B
text: Gets thrown out .
speaker: B
text: Yeah .
speaker: C
text: so we 'll throw it out
speaker: C
text: and we 're gonna randomly select another feature <pause> from the set of possible basis functions .
speaker: A
text: Hmm !
speaker: B
text: Yeah .
speaker: A
text: So it 's a <disfmarker>
speaker: B
text: So i so it 's actuall
speaker: A
text: it 's a little bit like a genetic algorithm or something in a way .
speaker: E
text: It 's like a greedy <disfmarker>
speaker: B
text: Well , it 's <disfmarker> it 's much simpler .
speaker: B
text: But it 's <disfmarker> but it 's <disfmarker> uh , it 's <disfmarker> there 's a lot <disfmarker> number of things I like about it , let me just say .
speaker: A
text: Greedy .
speaker: B
text: So , first thing , well , you 're absolutely right .
speaker: B
text: I mean , <vocalsound> i i <nonvocalsound> in truth , <pause> both pieces of this are <disfmarker> have their analogies in stuff we already do .
speaker: B
text: But it 's a different take <vocalsound> at how to approach it
speaker: B
text: and potentially one that 's m maybe a bit more systematic than what we 've done ,
speaker: B
text: uh , and a b a bit more inspiration from <disfmarker> from auditory things .
speaker: B
text: So it 's <disfmarker> so I think it 's a neat thing to try .
speaker: B
text: The primary features , <vocalsound> um , are in fact <disfmarker>
speaker: B
text: Yeah , essentially , it 's <disfmarker> it 's , uh , you know , PLP or <disfmarker> or mel cepstrum , or something like that .
speaker: B
text: You 've <disfmarker> you 've got some , <vocalsound> uh , compression .
speaker: B
text: We always have some compression .
speaker: B
text: We always have some <disfmarker> you know , the <disfmarker> the <disfmarker> the kind of filter bank with a kind of <vocalsound> <vocalsound> quasi - log scaling .
speaker: B
text: Um , <vocalsound> if you put in <disfmarker> if you also include the RASTA in it <disfmarker>
speaker: B
text: i RASTA <disfmarker> the filtering being done in the log domain <vocalsound> has an AGC - like , uh , characteristic , which , you know , people typi typically put in these kind of , <vocalsound> uh , <pause> um , <vocalsound> uh , auditory front - ends .
speaker: B
text: So it 's very , very similar ,
speaker: B
text: uh , but it 's not exactly the same .
speaker: B
text: Um ,
speaker: B
text: I would agree that the second one is <disfmarker> is somewhat more different
speaker: B
text: but , <vocalsound> um , it 's mainly different in that the things that we have been doing like that have been <disfmarker> <vocalsound> um , had a different kind of motivation and have ended up with different kinds of constraints .
speaker: B
text: So , for instance , if you look at the LDA RASTA stuff , <vocalsound> you know , basically what they do is they <disfmarker> they look at the different eigenvectors out of the LDA and they form filters out of it . Right ?
speaker: B
text: And those <pause> filters have different , uh , kinds of temporal extents and temporal characteristics .
speaker: B
text: And so in fact they 're multi - scale .
speaker: B
text: But , they 're not sort of systematically multi - scale , like " let 's start here and go to there , and go to there , and go to there " , and so forth .
speaker: B
text: It 's more like , <vocalsound> you run it on this , you do discriminant analysis , and you find out what 's helpful .
speaker: C
text: I it 's multi - scale because you use several of these in parallel ,
speaker: C
text: is that right ?
speaker: B
text: Yeah . They use several of them .
speaker: C
text: Of <disfmarker>
speaker: C
text: OK .
speaker: B
text: Yeah .
speaker: B
text: Uh , I mean , you don't have to
speaker: B
text: but <disfmarker> but <disfmarker> but , uh , Hynek has .
speaker: B
text: Um ,
speaker: B
text: but it 's also , uh <disfmarker>
speaker: B
text: Hyn - when Hynek 's had people do this kind of LDA analysis , they 've done it on frequency direction
speaker: B
text: and they 've done it on the time direction .
speaker: B
text: I think he may have had people sometimes doing it on both simultaneously <disfmarker>
speaker: B
text: some two - D <disfmarker>
speaker: B
text: and that would be the closest to these Gabor function kind of things .
speaker: B
text: Uh , but I don't think they 've done that much of that .
speaker: B
text: And , uh , the other thing that 's interesting <disfmarker> the <disfmarker> the , uh <disfmarker> the feature selection thing ,
speaker: B
text: it 's a simple method ,
speaker: B
text: but I kinda like it .
speaker: B
text: Um , <vocalsound> there 's a <disfmarker> <pause> a old , old method for feature selection .
speaker: B
text: I mean , <pause> eh , uh , I remember people referring to it as old when I was playing with it twenty years ago ,
speaker: B
text: so I know it 's pretty old ,
speaker: B
text: uh , called Stepwise Linear Discriminant Analysis
speaker: B
text: in which you <disfmarker> which <disfmarker>
speaker: B
text: I think it 's used in social sciences a lot .
speaker: B
text: So , you <disfmarker> you <disfmarker> you <disfmarker> you pick the best feature .
speaker: B
text: And then <vocalsound> you take <disfmarker> y you find the next feature that 's the best in combination with it .
speaker: B
text: And then so on and so on .
speaker: B
text: And what <disfmarker> what Michael 's describing seems to me much , much better ,
speaker: B
text: because the problem with the stepwise discriminant analysis is that you don't know that <disfmarker> you know , if you 've <vocalsound> picked the right set of features .
speaker: B
text: Just because something 's a good feature doesn't mean that you should be adding it .
speaker: B
text: So , <vocalsound> um , <pause> uh ,
speaker: B
text: here at least you 're starting off with all of them ,
speaker: B
text: and you 're <vocalsound> throwing out useless features .
speaker: B
text: I think that 's <disfmarker> that seems , uh <disfmarker> <vocalsound> that seems like a lot better idea .
speaker: B
text: Uh , you 're always looking at things in combination with other features .
speaker: B
text: Um ,
speaker: B
text: so the only thing is , of course , there 's this <disfmarker> this artificial question of <disfmarker> of , uh , <vocalsound> exactly how you <disfmarker> how you a how you assess it
speaker: B
text: and if <disfmarker> if your order had been different in throwing them out .
speaker: B
text: I mean , it still isn't necessarily really optimal ,
speaker: B
text: but it seems like a pretty good heuristic .
speaker: E
text: Hmm .
speaker: B
text: So I th I think it 's <disfmarker> it 's <disfmarker> I think it 's kinda neat stuff .
speaker: B
text: And <disfmarker> and <disfmarker> and , uh ,
speaker: B
text: the thing that I wanted to <disfmarker> to add to it also was to have us use this in a multi - stream way .
speaker: E
text: Hmm .
speaker: B
text: Um ,
speaker: B
text: so <disfmarker> so that , um , <vocalsound> when you come up with these different things , <vocalsound> and these different functions , <vocalsound> you don't necessarily just put them all into one huge vector ,
speaker: B
text: but perhaps <vocalsound> you <vocalsound> have some of them in one stream and some of them in another stream , and so forth .
speaker: B
text: And , um , um , <comment> um <disfmarker>
speaker: B
text: And we 've also talked a little bit about , uh , <vocalsound> uh , Shihab Shamma 's stuff ,
speaker: B
text: in which <vocalsound> you <disfmarker> the way you look at it is that there 's these different mappings
speaker: B
text: and some of them emphasize , uh , upward moving , <vocalsound> uh , energy and fre and frequency .
speaker: B
text: And some are emphasizing downward
speaker: B
text: and <vocalsound> fast things and slow things and <disfmarker> and <pause> so forth .
speaker: B
text: So .
speaker: B
text: So there 's a bunch of stuff to look at .
speaker: B
text: But , uh , I think we 're sorta gonna start off with what <vocalsound> he , uh , came here with
speaker: B
text: and branch out <disfmarker> <vocalsound> branch out from there .
speaker: B
text: And his advisor is here , too , <vocalsound> at the same time .
speaker: B
text: So ,
speaker: B
text: he 'll be another <pause> interesting source of <pause> wisdom .
speaker: E
text: Hmm .
speaker: B
text: So .
speaker: E
text: As <disfmarker> as we were talking about this I was thinking , <vocalsound> um , <vocalsound> whether there 's a relationship between <disfmarker> <vocalsound> um , <vocalsound> <vocalsound> between Michael 's approach to , uh , some <disfmarker> some sort of optimal brain damage or optimal brain surgeon on the neural nets .
speaker: B
text: Yeah .
speaker: E
text: So , like , if we have ,
speaker: C
text: Hmm .
speaker: E
text: um <disfmarker>
speaker: E
text: we have our <disfmarker> we have our RASTA features and <disfmarker>
speaker: E
text: and presumably the neural nets are <disfmarker> are learning some sort of a nonlinear mapping , <vocalsound> uh , from the <disfmarker> the <disfmarker> the features <vocalsound> to <disfmarker> to this <disfmarker> this probability posterior space .
speaker: B
text: Mm - hmm .
speaker: E
text: Right ?
speaker: E
text: And , um <disfmarker> <vocalsound> <vocalsound> and each of the hidden units is learning some sort of <disfmarker> some sort of <disfmarker> some sort of pattern .
speaker: E
text: Right ?
speaker: E
text: And it could be , like <disfmarker> <vocalsound> like these , um <disfmarker> these auditory patterns that Michael <pause> is looking at .
speaker: E
text: And then when you 're looking at the <disfmarker> <vocalsound> the , uh , <pause> um , <vocalsound> the best features , <vocalsound> you know , you can take out <disfmarker> you can do the <disfmarker> do this , uh , brain surgery by taking out , <vocalsound> um , hidden units that don't really help at all .
speaker: B
text: Mm - hmm .
speaker: E
text: And this is k sorta like <disfmarker>
speaker: B
text: Or the <disfmarker> or features .
speaker: B
text: Right ?
speaker: E
text: Yeah .
speaker: B
text: I mean , y actually , you make me think a <disfmarker> a very important point here is that , um , <vocalsound> if we a again try to look at how is this different from what we 're already doing , <vocalsound> uh , there 's a <disfmarker> a , uh <disfmarker> <vocalsound> a nasty argument that could be made th that it 's <disfmarker> it 's not different at <disfmarker> at all ,
speaker: B
text: because , uh <disfmarker> if you ignore the <disfmarker> the selection part
speaker: B
text: because we are going into a <disfmarker> a very powerful , <vocalsound> uh , nonlinearity
speaker: B
text: that , uh , in fact is combining over time and frequency ,
speaker: B
text: and is coming up with its own <disfmarker> you know , better than Gabor functions
speaker: E
text: Mm - hmm .
speaker: B
text: its , you know , neural net functions ,
speaker: B
text: its <disfmarker> <comment> <vocalsound> whatever it finds to be best .
speaker: B
text: Um , so you could argue that in fact it <disfmarker>
speaker: B
text: But I <disfmarker> I don't actually believe that argument
speaker: B
text: because I know that , um , <vocalsound> you can , uh <disfmarker>
speaker: B
text: computing features is useful ,
speaker: B
text: even though <pause> in principle you haven't <pause> <vocalsound> added anything <disfmarker>
speaker: B
text: in fact , you subtracted something , from the original waveform <disfmarker>
speaker: B
text: You know , uh , if you 've <disfmarker> you 've processed it in some way you 've typically lost something <disfmarker> some information .
speaker: B
text: And so , <vocalsound> you 've lost information and yet it does better with <disfmarker> <vocalsound> with features than it does with the waveform .
speaker: B
text: So ,
speaker: B
text: uh , I <disfmarker> I know that i sometimes it 's useful to <disfmarker> <pause> to constrain things .
speaker: B
text: So that 's <vocalsound> why it really seems like the constraint <disfmarker> in <disfmarker> in all this stuff it 's the constraints that are actually what matters .
speaker: B
text: Because if it wasn't <pause> the constraints that mattered , then we would 've completely solved this problem long ago ,
speaker: B
text: because long ago we already knew how to put waveforms into powerful statistical mechanisms .
speaker: B
text: So .
speaker: D
text: Yeah . Well , if we had infinite processing power and <pause> data , <comment> I guess , using the waveform could <disfmarker>
speaker: E
text: Right .
speaker: B
text: Yeah
speaker: B
text: Uh ,
speaker: B
text: then it would work .
speaker: B
text: Yeah , I agree .
speaker: B
text: Yeah . There 's the problem .
speaker: D
text: So , that 's <disfmarker>
speaker: B
text: Yeah . Then it would work .
speaker: B
text: But <disfmarker> but , I mean , i it 's <disfmarker> <vocalsound> With finite <pause> of those things <disfmarker>
speaker: B
text: I mean , uh , we <disfmarker> we have done experiments where we literally have put waveforms in
speaker: B
text: and <disfmarker> and <disfmarker> and , uh ,
speaker: D
text: Mm - hmm .
speaker: B
text: we kept the number of parameters the same and so forth ,
speaker: B
text: and it used a lot of training data .
speaker: B
text: And it <disfmarker> and it <disfmarker> it , uh <disfmarker>
speaker: B
text: not infinite
speaker: B
text: but a lot , and then compared to the number parameters <disfmarker>
speaker: B
text: and it <disfmarker> it , uh <disfmarker> it just doesn't do nearly as well .
speaker: D
text: Mm - hmm .
speaker: B
text: So , anyway the point is that you want to suppress <disfmarker>
speaker: B
text: it 's not just having the maximum information ,
speaker: B
text: you want to suppress , <vocalsound> uh , the aspects of the input signal that are not helpful for <disfmarker> for the discrimination you 're trying to make .
speaker: B
text: So .
speaker: B
text: So maybe just briefly , uh <disfmarker>
speaker: E
text: Well , that sort of segues into <pause> what <disfmarker> what I 'm doing .
speaker: B
text: Yeah .
speaker: E
text: Um , <vocalsound> so , uh , the big picture is k um , <vocalsound> come up with a set of , <vocalsound> uh , intermediate categories ,
speaker: E
text: then build intermediate category classifiers , then do recognition ,
speaker: E
text: and , um , improve speech recognition in that way .
speaker: E
text: Um , so right now I 'm in <disfmarker> in the phase where <vocalsound> I 'm looking at <disfmarker> at , um , deciding on a initial set of intermediate categories .
speaker: E
text: And <vocalsound> I 'm looking <vocalsound> for data data - driven <pause> methods that can help me find , <vocalsound> um , a set of intermediate categories <vocalsound> of speech that , uh , will help me to discriminate <pause> later down the line .
speaker: E
text: And one of the ideas , <vocalsound> um , that was to take a <disfmarker> take a neural net <disfmarker>
speaker: E
text: train <disfmarker> train an ordinary neural net <vocalsound> to <disfmarker> <vocalsound> uh , to learn the posterior probabilities of phones .
speaker: E
text: And so ,
speaker: E
text: um , at the end of the day you have this neural net
speaker: E
text: and it has hidden <disfmarker> <vocalsound> hidden units .
speaker: E
text: And each of these hidden units is <disfmarker> <vocalsound> um , is learning some sort of pattern .
speaker: E
text: And so , um , what <disfmarker> what are these patterns ?
speaker: A
text: Hmm .
speaker: E
text: I don't know .
speaker: E
text: Um , and I 'm gonna to try to <disfmarker> <vocalsound> to look at those patterns <vocalsound> to <disfmarker> to see , <vocalsound> um , <vocalsound> from those patterns <disfmarker>
speaker: E
text: uh , presumably those are important patterns for discriminating between phone classes .
speaker: E
text: And maybe <disfmarker> <vocalsound> maybe some , uh , intermediate categories can come from <vocalsound> just looking at the patterns of <disfmarker> <vocalsound> um , that the neural net learns .
speaker: B
text: Be - before you get on the next part l let me just point out that s there 's <disfmarker> there 's a <disfmarker> a pretty nice <comment> <vocalsound> relationship between what you 're talking about doing and what you 're talking about doing there . Right ?
speaker: E
text: Yeah .
speaker: B
text: So , <vocalsound> it seems to me that , you know , if you take away the <disfmarker> the <disfmarker> <pause> the difference of this <pause> primary features , <vocalsound> and , say , you use <disfmarker> as we had talked about maybe doing <disfmarker> you use P - RASTA - PLP or something for the <disfmarker> the primary features , <vocalsound> um , then this feature discovery , <pause> uh , uh , thing <vocalsound> is just what he 's talking about doing , too ,
speaker: B
text: except that he 's talking about doing them in order to discover <pause> intermediate categories that correspond <vocalsound> to these <disfmarker> uh , uh , what these sub - features are <disfmarker> are <disfmarker> are <disfmarker> are showing you .
speaker: B
text: And , um , <vocalsound> the other difference is that , um , <vocalsound> he 's doing this in a <disfmarker> in a multi - band setting ,
speaker: B
text: which means that he 's constraining himself <vocalsound> to look across time in some f relatively limited , uh , uh , spectral extent . Right ?
speaker: B
text: And whereas in <disfmarker> in this case you 're saying " let 's just do it unconstrained " .
speaker: B
text: So they 're <disfmarker> they 're really pretty related
speaker: B
text: and maybe they 'll be <disfmarker> at some point where we 'll see the <disfmarker> the connections a little better
speaker: C
text: Hmm .
speaker: B
text: and <vocalsound> connect them .
speaker: E
text: Mm - hmm .
speaker: E
text: Um .
speaker: E
text: Yeah , so <disfmarker> so that 's the <disfmarker> that 's the first part <disfmarker> uh , one <disfmarker> one of the ideas to get at some <disfmarker> <vocalsound> some patterns of intermediate categories .
speaker: E
text: Um , <vocalsound> the other one <pause> was , <vocalsound> um , to , <vocalsound> uh , come up with a <disfmarker> a <disfmarker> a model <disfmarker> <comment> um , a graphical model , <vocalsound> that treats <pause> the intermediate categories <vocalsound> as hidden <disfmarker> hidden variables , latent variables , that we don't know anything about ,
speaker: E
text: but that through , <vocalsound> um , s statistical training and the EM algorithm , <vocalsound> um , at the end of the day , <vocalsound> we have , um <disfmarker> we have learned something about these <disfmarker> these latent , um <disfmarker> latent variables
speaker: E
text: which happen to correspond to <vocalsound> intermediate categories .
speaker: E
text: Um . <vocalsound> <nonvocalsound> Yeah , and so those are the <disfmarker> the two directions that I 'm <disfmarker> I 'm looking into right now .
speaker: E
text: And , uh , <vocalsound> um <disfmarker> <vocalsound> <vocalsound> Yeah .
speaker: E
text: I guess that 's <disfmarker> that 's it .
speaker: B
text: OK .
speaker: B
text: Should we do our digits
speaker: B
text: and get ou get our treats ?
speaker: E
text: Oh , tea time ?
speaker: B
text: Yeah . It 's kind of like , you know , the little rats with the little thing dropping down to them .
speaker: A
text: That 's ri
speaker: B
text: We do the digits and then we get our treats .
speaker: E
text: Oops .
speaker: A
text: OK .
speaker: B
text: OK .

Meeting summary:
text: ICSI's Meeting Recorder Group met to discuss their progress in various aspects of the Aurora Project, but also to hear more about other developments relevant to the group.
text: On the Aurora Project, there were reports on a project conference call, the status of the tandem neural networks, and progress with the Mississippi State recognizer.
text: The latency limit has been set, and the group's system is performing very well, but is a little over.
text: On the larger vocabulary task, there are still a few issues to resolve before work can really get started.
text: The group heard of the plan of one of it's member's work into intermediate classifiers, and also of how a visiting research student's work into auditory models can be applied to their work.
text: Speaker me013 wants to know how much memory the tandem network takes up.
text: It is only a minor problem that the latency limit has been set below the current systems level, and also keeping the number of features within limits only drops performance a little.
text: A more significant problem is that the tandem approach may not fit in the memory space allowed, and removing it drops performance more.
text: Some of the group had issues with mn007's approach to human performance testing, but this was considered more of a side issue.
text: Speaker mn007 has been working on the tandem network approach, and the current results are good.
text: He has found a good way of calculating the silence probabilities, that does not increase insertions.
text: He also attempted to transcribe data himself, to establish a human performance level.
text: Speaker me018 is working on Mississippi Sate recognizer for dealing with the Wall Street Journal Data.