Meeting transcript:
speaker: D
text: OK .
speaker: D
text: So , uh
speaker: D
text: You can fill those out , uh <pause> after , actually ,
speaker: D
text: so
speaker: D
text: So , I got , uh <pause> these results from , uh , Stephane .
speaker: D
text: Also , um , I think that , uh <pause> um <pause> we might hear later today , about other results .
speaker: D
text: I think s that , uh , there were some other very good results that we 're gonna wanna compare to .
speaker: D
text: But , <vocalsound> r our results from other <disfmarker> other places ,
speaker: D
text: yeah .
speaker: A
text: I I 'm sorry ?
speaker: A
text: I didn't
speaker: D
text: Um , I got this from you
speaker: A
text: Yeah .
speaker: D
text: and then I sent a note to Sunil about the <disfmarker> cuz he has been running some other systems
speaker: D
text: other than the <disfmarker> the ICSI OGI one .
speaker: A
text: Mm - hmm .
speaker: A
text: Oh yeah .
speaker: D
text: So <pause> um , I wan wanna <disfmarker> wanna see what that is .
speaker: D
text: But , uh , you know , so we 'll see what it is comparatively later .
speaker: D
text: But <pause> it looks like , um
speaker: A
text: M yeah .
speaker: D
text: You know most of the time , even <disfmarker> I mean even though it 's true that the overall number for Danish <disfmarker> we didn't improve it
speaker: D
text: If you look at it individually , what it really says is that there 's , um , uh
speaker: D
text: Looks like out of the six cases , between the different kinds of , uh , matching conditions <pause> out of the six cases , there 's basically , um , a couple where it stays about the same ,
speaker: D
text: uh , three where it gets better , and one where it gets worse .
speaker: A
text: Yeah .
speaker: D
text: Uh , go ahead .
speaker: A
text: Y Actually , uh , um , for the Danish , there 's still some kind of mystery
speaker: A
text: because , um , um , when we use the straight features , we are not able to get these nice number
speaker: A
text: with the ICSI OGI one , I mean .
speaker: A
text: We don't have this ninety - three seventy - eight , we have
speaker: A
text: eight
speaker: E
text: Eighty - nine forty - four .
speaker: A
text: yeah .
speaker: A
text: Uh , so , uh , that 's probably something wrong with the features that we get from OGI .
speaker: A
text: Uh , and Sunil is working on <disfmarker> on trying to <disfmarker> to check everything .
speaker: D
text: Oh ,
speaker: D
text: and <disfmarker> and we have a little time on that <disfmarker> and <disfmarker> actually
speaker: D
text: so
speaker: A
text: Hmm ?
speaker: D
text: We have a little bit of time on that , actually .
speaker: A
text: Yeah .
speaker: D
text: We have a day or so ,
speaker: D
text: so
speaker: D
text: When <disfmarker> when <disfmarker> when do you folks leave ?
speaker: A
text: Uh , Sunday .
speaker: D
text: Sunday ?
speaker: D
text: So
speaker: D
text: So , uh
speaker: D
text: Yeah ,
speaker: D
text: until Saturday midnight , or something , we have
speaker: D
text: W we <disfmarker> we have time ,
speaker: D
text: yeah .
speaker: D
text: Well , that would be good .
speaker: D
text: That 'd be good .
speaker: A
text: Yeah .
speaker: D
text: Yeah .
speaker: D
text: Uh , and , you know , i u when whenever anybody figures it out they should also , for sure , email Hynek
speaker: D
text: because Hynek will be over there <vocalsound> telling people <vocalsound> what we did ,
speaker: D
text: so he should know .
speaker: A
text: Mmm .
speaker: D
text: Good ,
speaker: A
text: Yeah .
speaker: D
text: OK .
speaker: D
text: So , um
speaker: D
text: So , we 'll <disfmarker> we 'll hold off on that a little bit .
speaker: D
text: I mean , even with these results as they are , it 's <disfmarker> it 's <disfmarker> it 's really not that bad .
speaker: D
text: But <disfmarker> but , uh , um
speaker: D
text: And it looks like the overall result as they are now , even without , you know , any <disfmarker> any bugs being fixed is that , uh , on the <disfmarker> the other tasks , we had this average of , uh , forty uh <disfmarker> nine percent , or so , improvement .
speaker: D
text: And here we have somewhat better than that than the Danish , and somewhat worse than that on the German ,
speaker: D
text: but I mean , it sounds like , uh , one way or another , the methods that we 're doing can reduce the error rate from <disfmarker> from mel ceptrum <pause> down by , you know <pause> a fourth of them to , uh , a half of them .
speaker: D
text: Somewhere in there , depending on the <pause> exact case .
speaker: D
text: So
speaker: D
text: So that 's good .
speaker: D
text: I mean , I think that , uh , one of the things that Hynek was talking about was understanding what was in the other really good proposals
speaker: D
text: and <disfmarker>
speaker: D
text: and trying to see if what should ultimately be proposed is some , uh , combination of things .
speaker: D
text: Um , if , uh <disfmarker>
speaker: D
text: Cuz there 's things that they are doing <pause> there that we certainly are not doing .
speaker: D
text: And there 's things that we 're doing that <pause> they 're not doing .
speaker: D
text: And <disfmarker> and they all seem like good things .
speaker: A
text: Yeah .
speaker: E
text: Mmm , yeah .
speaker: D
text: So
speaker: D
text: So
speaker: C
text: How much <disfmarker> how much better was the best system than ours ?
speaker: D
text: Well , we don't know yet .
speaker: C
text: Mmm .
speaker: D
text: Uh , I mean , first place , there 's still this thing to <disfmarker> to work out ,
speaker: D
text: and second place <disfmarker> second thing is that the only results that we have so far from before were really development set results .
speaker: C
text: Oh , OK .
speaker: D
text: So , I think in this community that 's of interest .
speaker: D
text: It 's not like everything is being pinned on the evaluation set .
speaker: D
text: But , um , for the development set , our best result was a little bit short of fifty percent .
speaker: D
text: And the best result of any system was about fifty - four ,
speaker: D
text: where these numbers are the , uh , relative , uh , reduction in , uh , word error rate .
speaker: C
text: Oh ,
speaker: C
text: OK .
speaker: D
text: And , um , the other systems were , uh , somewhat lower than that .
speaker: D
text: There was actually <disfmarker> there was much less of a huge range than there was in Aurora one .
speaker: D
text: In Aurora one there were <disfmarker> there were systems that ba basically didn't improve things .
speaker: C
text: Hmm .
speaker: D
text: And here the <disfmarker> the worst system <pause> still reduced the error rate by thirty - three percent , or something , in development set .
speaker: C
text: Oh , wow .
speaker: D
text: So <disfmarker> so , you know , sort of everybody is doing things between , well , roughly a third of the errors , and half the errors being eliminated , <vocalsound> uh , and varying on different test sets and so forth .
speaker: C
text: Mm - hmm .
speaker: D
text: So I think
speaker: D
text: Um <pause> It 's probably a good time to look at what 's really going on and seeing if there 's a <disfmarker> there 's a way to combine the best ideas
speaker: D
text: while at the same time not blowing up the amount of , uh , resources used ,
speaker: D
text: cuz that 's <disfmarker> that 's critical for this <disfmarker> this test .
speaker: C
text: Do we know anything about <disfmarker>
speaker: C
text: who <disfmarker> who 's was it that had the lowest on the dev set ?
speaker: D
text: Um , uh , the , uh , the there were two systems that were put forth by a combination of <disfmarker> of , uh , French Telecom and Alcatel .
speaker: D
text: And , um they <disfmarker> they differed in some respects ,
speaker: D
text: but they e em one was called the French Telecom Alcatel System the other was called the Alcatel French Telecom System , <vocalsound> uh , which is the biggest difference , I think .
speaker: D
text: But <disfmarker> but there 're <disfmarker> there 're <disfmarker> there 're some other differences , too .
speaker: D
text: Uh , and <disfmarker> and , uh , they both did very well ,
speaker: C
text: Uh - huh .
speaker: D
text: you know ?
speaker: D
text: So , <vocalsound> um , my impression is they also did very well on <disfmarker> on the <disfmarker> the , uh , evaluation set ,
speaker: D
text: but , um , I <disfmarker> I we haven't seen <disfmarker>
speaker: D
text: you 've - you haven't seen any final results for that
speaker: D
text: yeah .
speaker: C
text: And they used <disfmarker> the main thing that <disfmarker> that they used was spectral subtraction ?
speaker: C
text: Or
speaker: D
text: There is a couple pieces to it .
speaker: D
text: There 's a spectral subtraction style piece <disfmarker> it was basically , you know , Wiener filtering .
speaker: D
text: And then <disfmarker> then there was some p some modification of the cepstral parameters , where they <disfmarker>
speaker: A
text: Yeah ,
speaker: A
text: actually , something that 's close to cepstral mean subtraction .
speaker: A
text: But , uh , the way the mean is adapted <disfmarker> um , it 's signal dependent .
speaker: A
text: I 'm <disfmarker> I 'm , uh
speaker: A
text: So , basically , the mean is adapted during speech and not during silence .
speaker: D
text: Yeah .
speaker: A
text: But it 's very close to <disfmarker> to cepstral mean subtraction .
speaker: D
text: But some people have done <vocalsound> <pause> exactly that sort of thing ,
speaker: D
text: of <disfmarker> of <disfmarker> and the <disfmarker> I mean it 's not <disfmarker> To <disfmarker> to look in <pause> speech only , to try to m to measure these things during speech ,
speaker: A
text: Yeah ,
speaker: A
text: yeah .
speaker: D
text: that 's p that 's not that uncommon .
speaker: D
text: But i it it <disfmarker> so it looks like they did some <disfmarker> some , uh , reasonable things ,
speaker: D
text: uh , and they 're not things that we did , precisely .
speaker: D
text: We did unreasonable things , <vocalsound> which <disfmarker> because we like to try strange things ,
speaker: D
text: and <disfmarker> and , uh , and our things worked too .
speaker: C
text: Hmm .
speaker: D
text: And so , um ,
speaker: D
text: uh , it 's possible that some combination of these different things that were done would be the best thing to do .
speaker: D
text: But the only caveat to that is that everybody 's being real conscious of how much memory and how much CPU they 're using
speaker: D
text: because these , <vocalsound> <vocalsound> <vocalsound> uh , standards are supposed to go on cell phones with m moderate resources in both respects .
speaker: C
text: Mm - hmm .
speaker: C
text: Did anybody , uh , do anything with the models as a <disfmarker> an experiment ?
speaker: C
text: Or
speaker: D
text: Uh , they didn't report it , if they did .
speaker: C
text: N nobody reported it ?
speaker: D
text: Yeah .
speaker: D
text: I think everybody was focused elsewhere .
speaker: D
text: Um , now , one of the things that 's nice about what we did is , we do have a <disfmarker> a , uh <disfmarker> a filtering , which leads to a <disfmarker> a , uh <disfmarker> a reduction in the bandwidth in the modulation spectrum , which allows us to downsample .
speaker: D
text: So , uh , as a result of that we have a reduced , um , transmission rate for the bits .
speaker: C
text: Mm - hmm .
speaker: D
text: That was misreported the first time out .
speaker: D
text: It <disfmarker> it said the same amount because for convenience sake in the particular way that this is being tested , uh , they were repeating the packets .
speaker: D
text: So it was <disfmarker> they were s they <disfmarker> they had twenty - four hundred bits per second , but they were literally creating forty - eight hundred bits per second , <vocalsound> um , even though y it was just repeated .
speaker: C
text: Oh .
speaker: C
text: Mm - hmm .
speaker: C
text: Right .
speaker: D
text: So , uh , in practice
speaker: C
text: So you could 've had a repeat count in there or something .
speaker: D
text: Well , n I mean , this was just a ph phoney thing just to <disfmarker> to fit into the <disfmarker> the software that was testing the errors <disfmarker> channel errors and so on .
speaker: C
text: Oh .
speaker: D
text: So <disfmarker>
speaker: C
text: Oh .
speaker: D
text: so in reality , if you put this <disfmarker> this system in into , uh , the field , it would be twenty - four hundred bits per second , not forty - eight hundred .
speaker: D
text: So ,
speaker: D
text: um , so that 's a nice feature of what <disfmarker> what we did .
speaker: D
text: Um ,
speaker: D
text: but , um , well , we still have to see how it all comes out .
speaker: C
text: Hmm .
speaker: D
text: Um , and then there 's the whole standards process , which is another thing altogether .
speaker: C
text: When is the development set <disfmarker> I mean , the , uh , uh , test set results due ?
speaker: C
text: Like the day before you leave or something ?
speaker: D
text: Uh , probably the day after they leave ,
speaker: D
text: but we 'll have to <disfmarker> <vocalsound> we 'll have to stop it the day before <comment> we leave .
speaker: A
text: Yeah , yeah .
speaker: A
text: So
speaker: C
text: Huh .
speaker: D
text: I think tha I think the <disfmarker> the meeting is on the thirteenth or something .
speaker: A
text: Yeah ,
speaker: A
text: this Tuesday ,
speaker: A
text: yeah .
speaker: D
text: And , uh ,
speaker: D
text: they , uh
speaker: D
text: Right .
speaker: D
text: And the <disfmarker> the , uh , results are due like the day before the meeting or something .
speaker: D
text: So
speaker: A
text: Yeah ,
speaker: A
text: probably ,
speaker: A
text: well
speaker: D
text: I th I think <disfmarker> I I think they are ,
speaker: A
text: Yeah , well
speaker: D
text: yeah .
speaker: D
text: So <pause> <vocalsound> um , since we have a bit farther to travel than <vocalsound> some of the others , <vocalsound> uh , we 'll have to get done a little quicker .
speaker: D
text: But , um , I mean , it 's just tracing down these bugs .
speaker: D
text: I mean , just exactly this sort of thing of , you know , why <disfmarker> why these features seem to be behaving differently , uh , in California than in Oregon .
speaker: C
text: Hmm .
speaker: D
text: Might have something to do with electricity shortage .
speaker: D
text: Uh , we didn't <disfmarker> we didn't have enough electrons here
speaker: D
text: and
speaker: D
text: Uh , but , um
speaker: D
text: Uh , I think , you know , the main reason for having <disfmarker>
speaker: D
text: I mean , it only takes w to run the <disfmarker> the two test sets in <disfmarker> just in computer time is just a day or so ,
speaker: D
text: right ?
speaker: D
text: So
speaker: A
text: Yeah ,
speaker: A
text: it 's very short interval .
speaker: D
text: yeah .
speaker: D
text: So , I think the who the whole reason for having as long as we have , which was <pause> like a week and a half , is <disfmarker> is because of bugs like that .
speaker: D
text: So
speaker: D
text: Huh
speaker: D
text: So , we 're gonna end up with these same kind of sheets that have the <pause> the percentages and so on just for the <disfmarker>
speaker: A
text: Yeah , so there are two more columns in the sheets ,
speaker: D
text: Oh , I guess it 's the same sheets ,
speaker: D
text: yeah ,
speaker: A
text: two . Yeah ,
speaker: A
text: it 's the same sheets ,
speaker: D
text: yeah <disfmarker>
speaker: D
text: just with the missing columns filled in .
speaker: A
text: yeah .
speaker: A
text: Yeah .
speaker: D
text: Yeah .
speaker: D
text: Well , that 'll be good .
speaker: D
text: So , I 'll dis I 'll disregard these numbers .
speaker: D
text: That 's <disfmarker> that 's <disfmarker> that 's good .
speaker: A
text: So , Hynek will try to push for trying to combine , uh , different things ?
speaker: A
text: Or
speaker: D
text: Uh , well that 's <pause> um
speaker: A
text: Hmm ?
speaker: D
text: yeah
speaker: D
text: I mean , I think the question is " Is there <disfmarker> is there some advantage ? "
speaker: D
text: I mean , you could just take the best system and say that 's the standard .
speaker: D
text: But the thing is that if different systems are getting at good things , um , a again within the constraint of the resources , if there 's something simple that you can do
speaker: D
text: Now for instance , uh , it 's , I think , very reasonable to have a standard for the terminal 's side and then for the server 's side say , " Here 's a number of things that could be done . "
speaker: D
text: So , um , everything that we did could probably just be added on to what Alcatel did ,
speaker: D
text: and i it 'd probably work pretty well with them , too .
speaker: D
text: So ,
speaker: D
text: um , uh , that 's one <disfmarker> one aspect of it .
speaker: D
text: And then on the terminal 's side , I don't know how much , um , memory and <disfmarker> and CPU it takes , but it seems like the filtering <pause> Uh ,
speaker: D
text: I mean , the VAD stuff they both had ,
speaker: D
text: right ?
speaker: D
text: And , um , so <disfmarker> and they both had some kind of on - line normalization ,
speaker: D
text: right ?
speaker: A
text: Uh , yeah .
speaker: D
text: Of sorts ,
speaker: D
text: yeah ?
speaker: D
text: So <disfmarker>
speaker: D
text: so , it seems like the main different there is the <disfmarker> is the , uh , filtering .
speaker: D
text: And the filtering <disfmarker> I think if you can <disfmarker>
speaker: D
text: shouldn't take a lot of memory to do that Uh , and I also wouldn't think the CPU , uh , would be much either for that part .
speaker: D
text: So , if you can <disfmarker> if you can add those in <pause> um <pause> then , uh , you can cut the data rate in half .
speaker: A
text: Yeah .
speaker: D
text: So it seems like the right thing to do is to <disfmarker> on the <disfmarker> on the terminal 's side , take what they did , if it <disfmarker> if it does seem to generalize well to German and Danish ,
speaker: D
text: uh , take what they did add in a filter , and add in some stuff on the server 's side
speaker: D
text: and <disfmarker> and <disfmarker> and that 's probably a reasonable standard .
speaker: D
text: Um <pause> Uh
speaker: A
text: They are working on this already ?
speaker: A
text: Because <disfmarker> yeah , Su - Sunil told me that he was trying already to put some kind of , uh , filtering in the <vocalsound> <pause> France Telecom .
speaker: D
text: Yeah , so that 's <disfmarker> that 's <disfmarker> that 's what
speaker: D
text: That would be ideal <disfmarker> would be is that they could , you know , they could actually show that , in fact , a combination of some sort , <vocalsound> uh , would work even better than what <disfmarker> what any of the systems had .
speaker: D
text: And , um , then it would <disfmarker> it would , uh <pause> be something to <disfmarker> to discuss in the meeting .
speaker: D
text: But , uh , not clear what will go on .
speaker: D
text: Um , I mean , on the one hand , um , sometimes people are just anxious to get a standard out there .
speaker: D
text: I mean , you can always have another standard after that ,
speaker: D
text: but <vocalsound> this process has gone on for a while on <disfmarker> already
speaker: D
text: and <disfmarker> and people might just wanna pick something and say , " OK , this is it . "
speaker: D
text: And then , that 's a standard .
speaker: D
text: Uh , standards are always optional .
speaker: D
text: It 's just that , uh , if you disobey them , then you risk not being able to sell your product ,
speaker: D
text: or <pause> <vocalsound> Uh <pause> um
speaker: D
text: And people often work on new standards while an old standard is in place and so on .
speaker: D
text: So it 's not final even if they declared a standard .
speaker: D
text: The other hand , they might just say they just don't know enough yet to <disfmarker> to declare a standard .
speaker: D
text: So
speaker: D
text: you <disfmarker> you <disfmarker> you will be <disfmarker> you will become experts on this and know more <disfmarker> far more than me about the tha this particular standards process once you <disfmarker> you go to this meeting .
speaker: D
text: So ,
speaker: D
text: be interested in hearing .
speaker: D
text: So , uh , I 'd be , uh , interested in hearing , uh , your thoughts now
speaker: D
text: I mean you 're almost done .
speaker: D
text: I mean , you 're done in the sense that , um , you may be able to get some new features from Sunil , and we 'll re - run it .
speaker: D
text: Uh , but other than that , you 're <disfmarker> you 're basically done ,
speaker: D
text: right ?
speaker: D
text: So , uh , I 'm interested in hearing <disfmarker> hearing your thoughts about <pause> where you think we should go from this .
speaker: A
text: Yeah .
speaker: D
text: I mean , we tried a lot of things in a hurry ,
speaker: D
text: and , uh , if we can back off from this now and sort of take our time with something , and not have doing things quickly be quite so much the constraint , what <disfmarker> what you think would be the best thing to do .
speaker: A
text: Uh , well
speaker: A
text: Hmm
speaker: A
text: Well , first , uh , to really have a look at <disfmarker> at the speech <pause> <vocalsound> from these databases
speaker: A
text: because , well , we tried several thing ,
speaker: A
text: but we did not really look <vocalsound> at what what 's happening , and <vocalsound> where is the noise , and
speaker: D
text: OK .
speaker: A
text: Eh
speaker: D
text: It 's a novel idea .
speaker: D
text: Look at the data .
speaker: D
text: OK .
speaker: A
text: Yeah .
speaker: D
text: Or more generally , I guess , what <disfmarker> what is causing the degradation .
speaker: A
text: Yeah ,
speaker: A
text: yeah .
speaker: A
text: Actually , there is one thing that <disfmarker> well <pause> Um , generally we <disfmarker> we think that <vocalsound> most of the errors are within phoneme classes ,
speaker: A
text: and
speaker: A
text: so I think it could be interesting to <disfmarker> to see if it <disfmarker> I don't think it 's still true when we add noise ,
speaker: A
text: and <vocalsound> so we have <disfmarker> I <disfmarker> I guess the confusion ma the confusion matrices are very different when <disfmarker> when we have noise , and when it 's clean speech .
speaker: A
text: And probably , there is much more <pause> between classes errors for noisy speech .
speaker: D
text: Mm - hmm .
speaker: A
text: And <vocalsound> so , um
speaker: A
text: Yeah , so perhaps we could have a <disfmarker> a large gain , eh , just by looking at improving the , uh , recognition , not of phonemes , but of phoneme classes , simply .
speaker: D
text: Mm - hmm .
speaker: A
text: And <vocalsound> which is a s a s a simpler problem , perhaps , but <disfmarker> which is perhaps important for noisy speech .
speaker: D
text: The other thing that strikes me , just looking at these numbers is , just taking the best cases ,
speaker: D
text: I mean , some of these , of course , even with all of our <disfmarker> our wonderful processing , still are horrible kinds of numbers .
speaker: D
text: But just take the best case , the well - matched <pause> uh , German case after <disfmarker> er well - matched Danish after we <disfmarker>
speaker: A
text: Mm - hmm .
speaker: D
text: the kind of numbers we 're getting are about eight or nine <pause> uh <pause> p percent <pause> error <pause> per digit .
speaker: A
text: Mm - hmm .
speaker: A
text: Yeah .
speaker: D
text: This is obviously not usable ,
speaker: D
text: right ?
speaker: A
text: No .
speaker: D
text: I mean , if you have ten digits for a phone number <comment> I mean , every now and then you 'll get it right .
speaker: A
text: Sure .
speaker: D
text: I mean , it 's <disfmarker> it 's , uh , <vocalsound> um
speaker: D
text: So , I mean , the other thing is that , uh <disfmarker>
speaker: D
text: And <disfmarker> and <disfmarker> a and <disfmarker> and also , um <pause> part of what 's nice about this is that this is , uh , <vocalsound> um <pause> a realistic <disfmarker> almost realistic database .
speaker: D
text: I mean , it 's still not people who are really trying to accomplish something ,
speaker: D
text: but <disfmarker> but , uh , within the artificial setup , it isn't noise artificially added , you know , simulated , uh , additive noise .
speaker: A
text: Mm - hmm .
speaker: D
text: It 's real noise condition .
speaker: D
text: And , um , <vocalsound> the <disfmarker> the training <disfmarker> the training , I guess , is always done on the close talking
speaker: A
text: No ,
speaker: A
text: actually <disfmarker> actually the well - matched condition <pause> is <pause> still quite di still quite difficult .
speaker: D
text: No ?
speaker: A
text: I mean , it 's <disfmarker> they have all these data from the close mike and from the distant mike , <vocalsound> from different driving condition , open window , closed window ,
speaker: D
text: Yeah .
speaker: A
text: and they take all of this
speaker: A
text: and they take seventy percent , I think , for training and thirty percent for testing .
speaker: E
text: Mm - hmm .
speaker: A
text: So , training is done <vocalsound> on different conditions and different microphones ,
speaker: A
text: and testing also is done <pause> on different microphone and conditions .
speaker: A
text: So , probably if we only take the close microphones , <vocalsound> I guess the results should be much much better than this .
speaker: D
text: I see .
speaker: A
text: Mmm .
speaker: D
text: Oh , OK ,
speaker: D
text: that explains it partially .
speaker: A
text: Uh
speaker: D
text: Wha - what about
speaker: D
text: i in <disfmarker> so the <disfmarker> the <disfmarker>
speaker: A
text: Yeah ,
speaker: A
text: so <disfmarker> there is this , the mismatched is , um <pause> the same kind of thing ,
speaker: D
text: go ahead .
speaker: A
text: but <pause> the driving conditions , I mean the speed and the kind of road , is different for training and testing ,
speaker: A
text: is that right ?
speaker: E
text: Yeah .
speaker: A
text: And the last condition is close microphone for training and distant for testing .
speaker: A
text: Yeah .
speaker: D
text: Uh , OK ,
speaker: D
text: so
speaker: A
text: So <disfmarker> <vocalsound> s so <disfmarker>
speaker: D
text: I see .
speaker: D
text: So , yeah ,
speaker: D
text: so the high <disfmarker> so the <disfmarker> right <disfmarker> so the highly mismatched <vocalsound> case <pause> is in some sense a good model for what we 've been , you know , typically talking about when we talk about additive noise in <disfmarker>
speaker: D
text: And so <disfmarker> and i i k it does correspond to a realistic situation in the sense that , <vocalsound> um , people might really be trying to , uh , call out telephone numbers or some or something like that , in <disfmarker> in their cars
speaker: A
text: Yeah .
speaker: D
text: and they 're trying to connect to something .
speaker: A
text: Mmm .
speaker: D
text: Um
speaker: A
text: Actually , yeah , it 's very close to clean speech training because , well , because the close microphone <vocalsound> and noisy speech testing ,
speaker: D
text: Yeah .
speaker: D
text: Yeah .
speaker: D
text: Yeah .
speaker: A
text: yeah .
speaker: A
text: Mmm .
speaker: D
text: And the well - matched condition <pause> is what you might imagine that you might be able to approach , if you know that this is the application .
speaker: D
text: You 're gonna record a bunch on people in cars and so forth , and do these training .
speaker: D
text: And then , uh , when y you sell it to somebody , they will be a different person with a different car , and so on .
speaker: D
text: So it 's <disfmarker> this is a an optim somewhat optimistic view on it ,
speaker: D
text: uh , so , you know , the real thing is somewhere in between the two .
speaker: D
text: Uh ,
speaker: A
text: Yeah .
speaker: D
text: uh , but
speaker: A
text: But the <disfmarker> I mean , the <pause> th th
speaker: D
text: Even the optimistic one is
speaker: A
text: it doesn't work .
speaker: D
text: Yeah ,
speaker: D
text: right .
speaker: A
text: It <disfmarker>
speaker: D
text: Right ,
speaker: D
text: it doesn't work .
speaker: D
text: So , in a way , that 's , you know , that 's sort of the dominant thing
speaker: D
text: is that even , say on the development set stuff that we saw , the , uh , the numbers that , uh , that Alcatel was getting when choosing out the best single numbers , <vocalsound> it was just <disfmarker> you know , it wasn't good enough for <disfmarker> for <pause> a <disfmarker> a <disfmarker> for a real system .
speaker: A
text: Mmm .
speaker: A
text: Mm - hmm .
speaker: D
text: You <disfmarker> you <disfmarker>
speaker: D
text: you , <vocalsound> um
speaker: D
text: So , uh , we still have stuff to do .
speaker: A
text: Yeah .
speaker: D
text: Uh , and , uh
speaker: D
text: I don't know
speaker: D
text: So , looking at the data , where , you know <disfmarker> what 's the <disfmarker> what 's <disfmarker> what 's th what 's characteristic
speaker: D
text: i e yeah , I think that 's <disfmarker> that 's a good thing .
speaker: D
text: Does a any you have any thoughts about what else <vocalsound> y you 're thinking that you didn't get to that you would like to do if you had more time ?
speaker: D
text: Uh
speaker: E
text: Oh , f a lot of thing .
speaker: E
text: Because we trying a lot of s <pause> thing ,
speaker: E
text: and we doesn't work , <vocalsound> we remove these .
speaker: E
text: Maybe <vocalsound> we trying again with the articulatory feature .
speaker: E
text: I don't know exactly
speaker: E
text: because we tried <disfmarker> we <disfmarker> some <disfmarker> one experiment that doesn't work .
speaker: E
text: Um , forgot it , something <pause> I don't know exactly
speaker: D
text: Mm - hmm .
speaker: E
text: because , tsk <comment> <vocalsound> maybe do better some step the general , <vocalsound> eh , diagram .
speaker: D
text: Mm - hmm .
speaker: E
text: I don't know exactly s to think what we can improve .
speaker: D
text: Yeah , cuz a lot of time it 's true ,
speaker: D
text: there were a lot of times when we 've tried something and it didn't work right away , even though we had an intuition that there should be something there .
speaker: D
text: And so then we would just stop it .
speaker: D
text: Um
speaker: D
text: And , uh , one of the things <disfmarker> I don't remember the details on , but I remember at some point , when you were working with a second stream , and you tried a low - pass filtering to cepstrum , in some case you got <disfmarker>
speaker: E
text: MSG Yeah .
speaker: D
text: Well , but it was <comment> an MSG - like thing , but it wasn't MSG ,
speaker: D
text: right ?
speaker: D
text: Uh , you <disfmarker> y I think in some case you got some little improvement ,
speaker: D
text: but it was , you know , sort of a small improvement ,
speaker: D
text: and it was a <disfmarker> a big added complication , so you dropped it .
speaker: D
text: But , um , that was just sort of one try ,
speaker: D
text: right ?
speaker: D
text: You just took one filter , threw it there ,
speaker: A
text: Yeah ,
speaker: D
text: right ?
speaker: D
text: And it seems to me that , um , if that is an important idea , which , you know , might be , that one could work at it for a while , as you 're saying .
speaker: A
text: Hmm .
speaker: D
text: And , uh
speaker: D
text: Uh , and you had , you know , you had the multi - band things also , and , you know , there was issue of that .
speaker: A
text: Yeah ,
speaker: E
text: Mm - hmm .
speaker: D
text: Um , Barry 's going to be , uh , continuing working on multi - band things as well .
speaker: D
text: We were just talking about , um , <vocalsound> some , uh , some work that we 're interested in .
speaker: D
text: Kind of inspired by the stuff by Larry Saul with the , uh <pause> uh , learning articulatory feature in <disfmarker> I think , in the case of his paper <disfmarker> with sonorance based on , uh , multi - band information where you have a <disfmarker> a combination of gradient learning an and , uh , EM .
speaker: A
text: Mm - hmm .
speaker: D
text: Um , and <pause> <vocalsound> <vocalsound> Um ,
speaker: D
text: so , I think that , you know , this is a , uh <disfmarker> this is a neat data set .
speaker: D
text: Um , and then , uh , as we mentioned before , we also have the <disfmarker> the new , uh , digit set coming up from recordings in this room .
speaker: D
text: So , there 's a lot of things to work with .
speaker: D
text: Um
speaker: D
text: and , uh
speaker: D
text: what I like about it , in a way , is that , uh , the results are still so terrible .
speaker: D
text: Uh <pause> <vocalsound> Uh <pause> <vocalsound> I mean , they 're much better than they were , you know .
speaker: D
text: We 're talking about thirty to sixty percent , uh , error rate reduction .
speaker: D
text: That 's <disfmarker> that 's really great stuff to <disfmarker> to do that in relatively short time .
speaker: D
text: But even after that it 's still , you know , so poor that <disfmarker> that , uh , no one could really use it .
speaker: D
text: So , um
speaker: D
text: I think that 's great that <disfmarker> because <disfmarker> and y also because again , it 's not something <disfmarker>
speaker: D
text: sometimes we 've gotten terrible results by taking some data , and artificially , you know , convolving it with some room response , or something <disfmarker>
speaker: D
text: we take a very <disfmarker>
speaker: D
text: Uh , at one point , uh , Brian and I went downstairs into the <disfmarker> the basement where it was <disfmarker> it was in a hallway where it was very reverberant
speaker: D
text: and we <disfmarker> we made some recordings there .
speaker: D
text: And then we <vocalsound> <disfmarker> we , uh <disfmarker> uh , made a simulation of the <disfmarker> of the room acoustics there and <disfmarker> and applied it to other things ,
speaker: A
text: Mm - hmm .
speaker: D
text: and uh
speaker: D
text: But it was all pretty artificial ,
speaker: D
text: and <disfmarker> and , you know , how often would you really try to have your most crucial conversations in this very reverberant hallway ?
speaker: D
text: Um <pause> So ,
speaker: D
text: uh <pause> This is what 's nice about the Aurora data and the data here , is that <disfmarker> is that it 's sort of a realistic room situation <pause> uh , acoustics <disfmarker> acoustic situation , both terms in noise and reflections , and so on
speaker: D
text: and n n And , uh ,
speaker: D
text: uh , with something that 's still relatively realistic , it 's still very very hard to do very well .
speaker: D
text: So
speaker: D
text: Yeah .
speaker: A
text: Yeah ,
speaker: A
text: so d
speaker: A
text: well
speaker: A
text: Actually , this is <disfmarker> tha that 's why we <disfmarker>
speaker: A
text: well , it 's a different kind of data .
speaker: A
text: We 're not <disfmarker> we 're not used to work with this kind of data .
speaker: D
text: Yeah .
speaker: A
text: That 's why we should have a loo more closer look at what 's going on .
speaker: E
text: Mm - hmm .
speaker: A
text: Um
speaker: A
text: Yeah .
speaker: A
text: So this would be the first thing ,
speaker: A
text: and then , of course , try to <disfmarker> well , <vocalsound> kind of debug what was wrong , eh , when we do Aurora test on the MSG <pause> particularly , and on the multi - band .
speaker: D
text: Yeah .
speaker: D
text: Yeah .
speaker: D
text: Yeah .
speaker: A
text: Uh
speaker: D
text: Yeah .
speaker: D
text: Yeah .
speaker: D
text: No , I <disfmarker> I think there 's lots of <disfmarker> lots of good things to do with this .
speaker: D
text: So
speaker: D
text: Um
speaker: D
text: So let 's <disfmarker>
speaker: D
text: I guess <pause> You were gonna say something else ?
speaker: D
text: Oh , OK .
speaker: D
text: What do you think ?
speaker: C
text: About
speaker: D
text: Anything
speaker: C
text: About other experiments ?
speaker: C
text: Uh , now , I 'm interested in , um , uh <pause> looking at the experiments where you use , um <pause> uh , data from multiple languages to train the neural net .
speaker: C
text: And I don't know how far , or if you guys even had a chance to try that , but <pause> that would be some it 'd be interesting to me .
speaker: A
text: Yeah , but
speaker: D
text: S b
speaker: A
text: Again , it 's the kind of <disfmarker> of thing that , uh , we were thin thinking <disfmarker> thinking that it would work , but it didn't work .
speaker: A
text: And , eh , so there is kind of <disfmarker> of <pause> not a bug , but something wrong in what we are doing , perhaps .
speaker: D
text: Yeah .
speaker: C
text: Right .
speaker: C
text: Right .
speaker: C
text: Right .
speaker: A
text: Uh , something wrong , perhaps in the <disfmarker> just in the <disfmarker> the fact that the labels are <disfmarker>
speaker: A
text: well
speaker: C
text: Mm - hmm .
speaker: A
text: What worked best is the hand - labeled data .
speaker: C
text: Mm - hmm .
speaker: A
text: Um
speaker: A
text: Uh , so , yeah .
speaker: A
text: I don't know if we can get some hand - labeled data from other languages .
speaker: C
text: Yeah .
speaker: A
text: It 's not so easy to find .
speaker: C
text: Right .
speaker: A
text: But <pause> that would be something interesting t to <disfmarker> to see .
speaker: C
text: Yeah ,
speaker: C
text: yeah .
speaker: D
text: Yeah .
speaker: D
text: Also , uh , <vocalsound> I mean , there was just the whole notion of having multiple nets that were trained on different data .
speaker: D
text: So one form of different data was <disfmarker> is from different languages , but the other
speaker: D
text: Well , i in fact , uh , m in those experiments it wasn't so much combining multiple nets , it was a single net that had different
speaker: A
text: Yeah .
speaker: D
text: So , first thing is would it be better if they were multiple nets , for some reason ?
speaker: D
text: Second thing is , never mind the different languages , just having acoustic conditions rather than training them all up in one ,
speaker: D
text: would it be helpful to have different ones ?
speaker: D
text: So , um
speaker: D
text: That was a question that was kind of raised by Mike Shire 's thesis , and on <disfmarker> in that case in terms of reverberation .
speaker: D
text: Right ?
speaker: D
text: That <disfmarker> that sometimes it might be better to do that .
speaker: D
text: But , um , <vocalsound> I don't think we know for sure .
speaker: D
text: So , um
speaker: D
text: Right .
speaker: D
text: So , next week , we , uh , won't meet because you 'll be in Europe .
speaker: D
text: Whe - when are you two getting back ?
speaker: E
text: Um , I 'm
speaker: A
text: You on Friday or S on Saturday
speaker: E
text: Sunday
speaker: A
text: or <pause> ?
speaker: E
text: because it 's <disfmarker> it 's less expensive , the price <disfmarker> the price the ticket .
speaker: A
text: S oh yeah , Sunday , yeah .
speaker: D
text: Yeah , that 's right .
speaker: D
text: You 've gotta S have a Saturday overnight ,
speaker: D
text: right ?
speaker: A
text: I 'll be back on Tuesday .
speaker: D
text: Tuesday .
speaker: C
text: Where <disfmarker> where 's the meeting ?
speaker: D
text: Uh , Amsterdam , I think ,
speaker: A
text: Yeah , Amsterdam .
speaker: D
text: yeah ?
speaker: C
text: Uh - huh .
speaker: D
text: Yeah .
speaker: D
text: Yeah , yeah .
speaker: D
text: Yep .
speaker: D
text: Um <pause> So , we 'll skip next week , and we 'll meet two weeks from now .
speaker: D
text: And , uh , I guess the main topic will be , uh , you telling us what happened .
speaker: A
text: Yeah .
speaker: E
text: Yeah .
speaker: D
text: Uh , so
speaker: D
text: Yeah ,
speaker: D
text: well , if we don't have an anything else to discuss , we should , uh , turn off the machine and then say the real nasty things .
speaker: C
text: Should we do digits first ?
speaker: B
text: Oh , yeah , digits .
speaker: D
text: Oh yeah , digits !
speaker: A
text: Yeah .
speaker: D
text: Yeah .
speaker: D
text: Good point .
speaker: D
text: Yeah ,
speaker: D
text: good thinking .
speaker: D
text: Why don't you go ahead .
speaker: C
text: OK .
speaker: C
text: OK .

Meeting summary:
text: The Berkley Meeting Recorder Group discussed the most recent progress with their current project, a digit recognition system for use in cell phones.
text: This included some discussion of results, comparing various other groups' systems, issues involving the set up, and plans for future work.
text: Results are required for an upcoming meeting, but since some group members will be away, results need to be in sooner.
text: Although error rates have been greatly reduced, current rates are still unusable in a practical situation.
text: There is a problem replicating some results found by partner OGI, but it is unclear why.
text: mn007 and fn002 have been working with the new Danish and German databases, making improvements, though more so with results on the Danish.
text: The results are reasonable, but still not good enough.
text: However, it has not been possible to compare results to the best, as still development system.
text: There are a number of things that the group wishes to consider for looking for further improvements.
text: There are various techniques that various groups have tried, and they should all be considered for possible combinations of systems.
text: One suitable candidate for combination is the group's own filtering which reduces bandwidth to half bit transmission rate.