PhD A: OK .
PhD C: Um Well , I I sent out an email s couple hours ago so um with Andreas ' help um Andreas put together a sort of no frills recognizer which is uh gender - dependent but like no adaptation , no cross - word models , no trigrams a bigram recognizer and that 's trained on Switchboard which is telephone conversations . Um and thanks to Don 's help wh who Don took the first meeting that Jane had transcribed and um you know separated used the individual channels we segmented it in into the segments that Jane had used and uh Don sampled that so so eight K um and then we ran up to I guess the first twenty minutes , up to synch time of one two zero zero so is that that 's twenty minutes or so ? Um yeah because I guess there 's some ,
Grad E: Or so .
PhD C: and Don can talk to Jane about this , there 's some bug in the actual synch time file that ah uh I 'm we 're not sure where it came from but stuff after that was a little messier . Anyway so it 's twenty minutes and I actually
Grad E: Hmm .
PhD C: um
Grad E: I was that did that did that recording have the glitch in the middle ?
Postdoc G: I 'm puzzled by that . I oh oh , I see .
PhD C: There 's there 's a Postdoc G: Oh there was a glitch somewhere .
PhD C: yeah , so that actually um
Grad F: Was it twenty minutes in ,
PhD C: if it was twenty minutes in then I don't know
Postdoc G: I forgot about that .
Grad F: I thought PhD A: Well it was interesting ,
Postdoc G: Well , I mean , they PhD A: suddenly the the overall error rate when we first ran it was like eighty percent
Grad E: I don't remember when it is .
Postdoc G: but I was able to can transcribe
PhD A: but i looking at the first sentences looked much better than that and then suddenly it turned very bad and then we noticed that the reference was always one off with the it was actually recognized
PhD C: Wel
Grad E: Oh no .
Grad F: Yeah , that might be that might be that might be my fault .
Postdoc G: Wow .
PhD A: so
Grad E: Oh so that was just a parsing mismatch .
Grad F: I 'm not PhD A: OK .
PhD C: No actually it was yeah i it was a complicated bug because they were sometimes one off and then sometimes totally random so um
Grad F: yeah , I was pretty certain that it worked up until that time ,
Postdoc G: Oh . That 's not good .
PhD C: Yeah
PhD A: OK .
PhD C: so that 's what we have
Grad E: Alright .
Grad F: so
PhD C: but that that will be completely gone if this synch time problem
Postdoc G: Yeah .
Grad E: The the glitch
PhD A: So so we have everything recognized but we scored only the first uh whatever , up to that time to
Postdoc G: And the only glitch Grad E: yeah .
Postdoc G: yeah .
PhD C: So you guys know .
Professor B: S sorry I haven't seen the email ,
PhD C: Yeah .
Grad E: Th - the
Postdoc G: The the well wait
Professor B: what was the score ?
PhD C: So here 's the actual copy of the email
Postdoc G: we should say something about the glitch . He he can say something about the glitch .
PhD C: um oh OK
Grad E: yeah .
Postdoc G: Cuz it 's it 's it 's h it 's it 's very small PhD C: so does this glitch occur at other Grad E: There there there 's an acoustic glitch that occurs where um the channels get slightly asynchronized
Postdoc G: very small . Yep .
PhD C: Oh .
PhD A: Mmm .
PhD C: Right .
Grad E: so the that that problem has gone away in the original driver believe it or not when the SSH key gen ran the driver paused for a fraction of a second
Professor B: Hmm .
Grad F: Hmm .
Grad E: and so the channels get a little asynchronous and so if you listen to it in the middle there 's a little part where it starts doing doing click sounds .
Professor B: So PhD C: And is it only once that that happens ?
Grad E: But yeah
PhD C: OK .
Grad E: it right once in the middle .
PhD C: There 's the previous page has some more information about sort of what was wrong
Professor B: so so un unsurprisingly Adam is the golden voice ,
PhD C: but
Grad E: Um But that shouldn't affect anything
PhD C: OK so that 's actually
Postdoc G: S and it Professor B: you see this here ?
PhD C: It y it 's Grad E: yeah yeah " bah "
PhD C: OK no PhD A: Oh , and PhD C: What happens is it actually affects the script that Don PhD D: Huh .
PhD C: I mean if we know about it then I guess it could always be checked for it
Grad E: Well the acoustic one shouldn't do anything .
PhD C: but they
Grad F: Yeah , I don't know exactly what affected it
Postdoc G: I agree . I agree .
PhD A: I I have Grad F: but I 'll I 'll talk to you about it ,
PhD A: Yeah .
Grad E: But I I do remember PhD C: Yeah .
Grad F: I 'll show you the point .
Postdoc G: Yeah . It it had no effect on my transcription ,
PhD A: Mmm .
Postdoc G: you know , I mean I I had no trouble hearing it and and having time bins
Grad E: I do remember seeing once the transcriber produce an incorrect XML file where one of the synch numbers was incorrect .
Postdoc G: but there was a Oh .
PhD C: Well , the the synch time the synch numbers have more significant digits than they should ,
Grad F: That 's what happened .
Postdoc G: Oh .
PhD H: Yeah .
Grad E: Where where they weren't monotonic .
Grad F: There was yeah , I mean PhD C: right ? There 's things that are l in smaller increments than a frame .
PhD H: Yeah .
Postdoc G: Oh , interesting .
PhD C: And so then , I mean you look at that and it 's got you know more than three significant digits in a synch time then that can't be right
Grad E: Oh OK so that 's
Grad F: Hmm .
Postdoc G: Oh .
PhD A: Mmm .
PhD C: so anyway it 's it 's just Grad E: yeah sounds like a bug .
Postdoc G: Yeah .
PhD C: that 's why we only have twenty minutes but there 's a significant amount of Grad F: Non - zero ? Um there are like more cuz there 's a lot of zeros I tacked on just because of the way the script ran ,
Grad E: The other one I saw was that it yeah .
Grad F: I mean but there were there was a point .
PhD C: Yeah that was fine . That that was OK .
Grad E: The other one I saw was non non - monotonic synch times
Grad F: OK .
Grad E: and that definitely indicra indicates a bug .
Grad F: Uh .
PhD C: Well that would really be a problem , yeah . So anyway these are just the ones that are the prebug for one meeting .
Grad F: Yeah .
PhD C: um and what 's which Grad E: So that 's very encouraging .
PhD C: this is really encouraging cuz this is free recognition ,
Professor B: Hmm .
PhD H: Yeah .
Professor B: Cool .
PhD A: Mmm .
PhD C: there 's no I mean the language model for Switchboard is totally different so you can see some like this Trent Lott which
PhD D: Trent Lott .
PhD C: um I mean these are sort of funny ones ,
PhD D: It 'll get those though .
PhD C: there 's a lot of perfect ones and good ones and all the references , I mean you can read them and when we get more results you can look through and see
Grad E: I and as I said I would like to look at the lattices
PhD A: Mm - hmm .
PhD C: but um it 's pretty good .
Grad E: because it sounded like even the ones it got wrong it sort of got it right ?
PhD C: Well so I guess we can generate
Grad E: Sounds likes ?
PhD A: There are a fair number of errors that are , you know where got the plural S wrong or the inflection on the verb wrong .
Postdoc G: Mm - hmm .
PhD C: um
Grad E: Yeah , and who cares ? And and there were lots of of course the " uh uh " - s , " in on " - s " of uh " - s .
PhD A: Mmm , so if PhD C: there 's No those are actually
PhD A: Yeah .
PhD C: a lot of the errors I think are out of vocabulary ,
PhD A: Mm - hmm .
PhD C: so is it like PZM is three words , it 's PZM ,
PhD A: Mm - hmm .
PhD C: I mean there 's nothing There 's no language model for PZM or
Grad E: Right . Ri - ri right .
PhD C: um
Grad E: Did you say there 's no language for PZM ?
PhD C: No language model , I mean those Grad E: Do you mean so every time someone says PZM it 's an error ? Maybe we shouldn't say PZM in these meetings .
PhD C: Well well there 's all kinds of other stuff like Jimlet and I mean um anyway there Grad E: Yeah , that 's right , Jimlet .
Professor B: Well , we don't even know what that means ,
PhD C: so but this is really encouraging because
Professor B: so I
Grad E: Yeah , that 's right .
PhD C: so , I mean the bottom line is even though it 's not a huge amount of data um it should be uh reasonable to actually run recognition and be like within the scope of of r reasonable s you know Switchboard this is like h about how well we do on Switchboard - two data with the Switchboard - one trained mostly trained recognizer
Grad E: Right .
PhD C: and Switchboard - two is got sort of a different population of speakers and a different topic
Grad E: Excellent .
PhD C: and they 're talking about things in the news that happened after Switchboard - one so there was @ @ so that 's great .
Professor B: Yeah . Yeah so we 're in better shape than we were say when we did had the ninety - three workshop
PhD C: Um
Professor B: and we were all getting like seventy percent error on Switchboard .
PhD A: Mm - hmm .
PhD C: Oh yeah
Professor B: you know
PhD C: I mean this is really ,
PhD A: Mmm .
PhD C: and thanks to Andreas who , I mean this is a
PhD A: Mmm .
Grad E: Well especially for the very first run , I mean you PhD A: Oh it 's the Professor B: Yeah .
PhD C: eh um
Professor B: Yeah .
PhD C: yeah
Grad E: the first run I ran of Switchboard I got a hundred twenty percent word error but
PhD C: So and what al also this means is that
Postdoc G: Right .
PhD C: um
Grad E: Not Switchboard ,
PhD A: Well it 's PhD C: I mean there 's a bunch of things in this note to various people
Grad E: uh Broadcast News .
PhD C: especially I guess um with Jane that that would help for since we have this new data now uh in order to go from the transcripts more easily to um just the words that the recognizer would use for scoring . I had to deal with some of it by hand but I think a lot of it can be automated s by Professor B: Oh one thing I guess I didn't get so you know the language model was straight from from bigram from Switchboard the acoustic models were also from Switchboard or or
PhD A: Yeah .
PhD C: Yeah .
Professor B: So they didn't have anything from this acoustic data in yet ?
Postdoc G: That 's amazing .
Grad E: Yeah , so that 's great .
PhD C: No .
Professor B: OK .
PhD C: And actually we actually um used Switchboard telephone bandwidth models
Postdoc G: That 's amazing .
PhD A: Well that 's those are the only we ones there are ,
Professor B: Yeah .
PhD C: which I guess
PhD D: I was just gonna say ,
PhD C: so that 's the on that 's the only acoustic training data that we have a lot of
PhD D: yeah .
PhD A: I mean
Grad E: Yeah .
PhD A: Right .
PhD C: and I guess Ramana , so a guy at SRI said that um there 's not a huge amount of difference going from Professor B: Right .
PhD C: it 's it 's not like we probably lose a huge amount but we won't know because we don't have any full band models for s conversational speech .
PhD D: It 's probably not as bad as going f using full band models on telephone band speech
PhD C: So .
PhD A: Oh yeah .
PhD C: Right .
PhD D: right ?
PhD A: Yeah .
Professor B: Yeah ,
PhD C: Right , so it 's so
Professor B: but for Broadcast News when we we played around between the two there wasn't a huge loss .
Grad E: Right , it was not a big deal .
PhD C: Yeah
PhD A: I should I should say that the language model is not just Switchboard
PhD C: so I wou so that 's good .
Grad E: Although combining em worked well .
PhD A: it 's also I mean there 's uh actually more data is from Broadcast News but with a little less weight
PhD C: Yeah .
PhD A: uh because
Professor B: Uh - huh .
PhD C: Like Trent Lott must have been from
PhD A: mm - hmm , right .
PhD C: I guess Switchboard was before
PhD A: Um By the way just for fun we also ran ,
PhD C: uh .
Professor B: Good point .
PhD A: I mean our complete system starts by doing ge a gender detection
Professor B: Mm - hmm .
PhD A: so just for the heck of it I ran that
Grad E: And it said a hundred percent male ?
PhD A: um and it might be reassuring for everybody to know that it got all the genders right .
PhD C: The j
PhD A: Yeah so
Grad E: Oh it did ?
Postdoc G: Oh that 's I 'm glad .
Grad E: It got all two genders ?
PhD C: Yeah but you know Jane and Adam have you kn about equal performance
PhD A: Yeah . Yes .
PhD C: and uh and that 's interesting cuz I think the their language models are quite different so and I I 'm pretty sure from listening to Eric that , you know given the words he was saying and given his pronunciation that the reason that he 's so much worse is the lapel .
Professor B: Yeah .
Grad E: Right .
Postdoc G: That makes a lot of sense ,
PhD C: So it 's nice now if we can just sort of eliminate the lapel one when when we get new microphones
Postdoc G: yeah . Very possible .
Professor B: Yeah I I I would bet on that too
PhD C: that would be worth it
Professor B: cuz he certainly in that when as a as a burp user he was he was a pretty uh strong one .
PhD C: um Yeah
Grad E: Sheep .
PhD C: he he he sounded to me just from he sounded like a ,
Professor B: Yeah .
PhD C: what 's it a sheep or a goat ?
Professor B: Sheep .
Grad E: A sheep .
PhD C: Sheep ,
Grad E: Baah .
Professor B: Yeah . Sheep is good .
PhD C: right . Sounded good .
Professor B: Yeah .
PhD C: Right so um so I guess the good news is that
Postdoc G: Mm - hmm .
PhD C: and and again this is without a lot of the sort of bells and whistles that we c can do with the SRI system and we 'll have more data and we can also start to maybe adapt the language models once we have enough meetings . So this is only twenty minutes of one meeting with no no tailoring at all .
PhD A: I mean clearly there are um with just a small amount of uh actual meeting transcriptions uh thrown into the language model you can probably do quite a bit better because the PhD C: Yeah . The voca the vocabulary especially
Grad E: Or just dictionary .
PhD C: yeah .
PhD A: Not that much the vocabulary actually
PhD C: Yeah , so .
PhD A: I think um well we have to see but it 's uh PhD C: Yeah . It 's pretty good um so then
Professor B: Have to add PZM and so on
Grad E: And I have to try it on the far field mike
Professor B: but
PhD C: PZM
Grad E: yeah .
PhD C: and then there 's things like for the transcription I got when someone has a digit in the transcript I don't know if they said , you know one one or eleven and I don't know if they said Tcl or TCL . there 's things like that where , you know the um we 'll probably have to ask the transcribers to indicate some of those kinds of things but in general it was really good and I 'm hoping and this is this is good news because that means the force alignments should be good and if the force alignments , I mean it 's good news anyway but if the force alignments are good we can get all kinds of information . For example about , you know prosodic information and speaker overlaps and so forth directly from the aligned times . Um so that 'll be something that actually in order to assess the forced alignment um we need s some linguists or some people to look at it and say are these boundaries in about the right place . Because it 's just gonna give us time marks
PhD D: But you know Grad E: Well we 've done that for one meeting .
PhD C: so . For forced alignment .
Grad E: Uh oh oh f not for words
PhD C: Ye - right .
Grad E: I 'm sorry just for overlaps is we did it for not not for words .
PhD C: Right . So this would be like if you take the words um you know and force align them on all the individual close talk uh close talking mikes then how good are these sort of in reality
Grad E: Right .
PhD C: and then I was thinking it Grad E: So we might want to take twenty minutes and do a closer word level transcription . Maybe actually mark the word boundaries .
PhD C: Oh or i have someone look at the alignments uh maybe a linguist who can say um you know roughly if these are OK and how far away they are .
Professor B: Yeah .
PhD C: Um but I think it 's gotta be pretty good because otherwise the word recognition would be really b crummy .
Grad E: Right , right .
PhD C: It wouldn't necessarily be the other way around , if the wor word recognition was crummy the alignment might be OK but if the word recognition is this good the alignment should be pretty good . So that 's about it .
Professor B: I r
PhD D: I wonder if this is a good thing or a bad thing though , I mean if we 're pr
Grad E: That we 're starting so well ?
PhD D: yeah if we 're producing a database that everybody 's gonna do well on
Professor B: Oh
Grad E: Don't worry about it w d that 's that 's the close talking mikes . Try it on the P Z Ms and and
Professor B: Yeah , which I would which well n n n n
PhD D: So the real value of the database is these ?
PhD H: Yeah , yeah , yeah , yeah .
Grad E: Yeah , abso well no but
Professor B: I mean there 's still just the w the percentages and , I mean they 're not a as we 've talked about before there 's probably overlaps
PhD C: This i yeah . This is not that good .
Professor B: there 's probably overlaps in in uh in fair number in Switchboard as well so but but there 's other phenomena , it 's a meeting , it 's a different thing and there 's lots of stuff to learn with the close talking mikes but uh yeah certainly I 'd like to see as soon as we could , I mean maybe get some of the glitches out of the way but soon as we could how well it does with say with the P Z Ms or maybe even one of the
PhD C: Right .
Professor B: and uh see if it 's , you know is it a hundred twenty percent or maybe it 's not maybe if with some adaptation you get this down to fifty percent or forty - five percent or something and and then if for the PZM it 's seventy or something like that that 's actually something we could sort of work with a little bit