it doesn't necessarily go against what he said cuz he said generally speaking In order to Hmm to go against Well f made claim claim Well big Well canvassing Yeah we we But have pretty limited Five sample time or five percent Yeah what gonna ask that Well too Exactly So i Yeah so not still against but his still it just says u that a bi bell curve and that you have something that has a nice range in Yeah your So sampling slight There are differences in how you measure it but still You know the difference between um between that number and what we have in meetings which is more like you know close to in meetings like these uh you know close to twenty percent But what was it That like say in the Robustness meeting for instance Robustness meeting It was about half of the r So in Maybe terms ten of percent words like seventeen or eigh eighteen percent for the Meeting Recorder meetings and about half that for uh But the I Robustness really a fair way of comparing between conversations and conversations Then Yeah then I then I have don't to I mean just something Yeah I just wonder if you have Yeah to normalize by the numbers Yeah speakers normalize something Well by didn't something point that that Yeah but this obvious thing to see Good if idea a dependence on the number of I mean uh participants I bet a weak dependence Yeah sure Right not a real strong one Right Cuz Because everybody Right talks Right Yeah You have a lot of a lot of Right subsets within the meeting Well regardless So interesting result regardless and Yes and then and we also d computed this both with and without backchannels so you might think that backchannels have a special status because essentially just did we all said R nodded at the same time But even if you take out all the backchannels so basically you treat backchannels l as nonspeech as pauses you still have significant overlap You know it goes down from maybe For Switchboard it goes down from I don't know f um I don't know f fourteen percent of the words to maybe uh I don't know eleven percent or something not a dramatic change so Anyway so uh That was that was one set of results and then the second one was just basically the the stuff we had in the in the HLT paper on how overlaps effect the recognition performance Right And we rescored things um a little bit more carefully We also fixed the transcripts in in numerous ways Uh but mostly we added one one number which was what if you uh basically score ignoring all So so the the conjecture from the HLT results was that most of the added recognition error is from insertions due to background speech So we scored all the recognition results uh in such a way that the uh Oh by the way on channel four Yeah getting I lot j of was breath just wondering Yeah me uh well been working hard right OK so so if you have the foreground speaker speaking here and then some background speech may be overlapping it somehow um and this is the time bin that we used then of course gonna get insertion errors here and here Right Right So we scored everything and I must say the NIST scoring tools are pretty nice for this where you just basically ignore everything outside of the uh region that was deemed to be foreground speech And where that was we had to use the t forced alignment uh results from s for so somewhat somewhat subject to error but still we we Uh Don did some ha and and we think that based on that we think that the results are you know valid although of course some error is gonna be in there But basically what we found is after we take out these regions so we only score the regions that were certified as foreground speech the recognition error went down to almost uh the level of the speech So that means that even great if you do have background speech if you can somehow separate out or find where it is Yeah uh the recognizer does a good job even though there is Yeah this guess back that doesn't surprise me because with the mikes Right the the signal will be so much Right stronger Um so sort of normalization do you do Uh well we just we do u you know vit recognizer in the SRI recognizer Well we do uh VTL vocal tract length normalization w and we uh you know we we uh make all the features have zero mean and unit variance And Over an entire utterance Over windowed entire c over the entire channel train Over the but Hmm know Um now we didn't the recognizer for this We just took the old So this is actually a way of doing Right right Right So we took the old recognition output and we just scored it differently So the recognizer didn't have the benefit of knowing where the foreground speech Were you including start the the lapel in this Yes And did the did did the la did the the problems with the lapel go away also Um Or it Yeah fray It for u not for per I insertions completely but yes dramatically So we have to um I mean you still I should bring the should bring the table with results Maybe we can look at it I would presume that you still would have somewhat higher Yes error with the lapel for insertions than Yes Yeah Yeah Cuz again looking forward to the miked case I think that we s still looking forward to it i the high ratio Right here u that s that Right helps you So so that was number that was the second set of uh the second section And then the third thing was we looked at uh what we call interrupts although that may be a misnomer but basically we looked at cases where Uh so we we used the punctuation from the original transcripts and we inferred the beginnings and ends of sentences So you know Di Um use case also or not Hmm U upper lower case No no we OK only used you know uh periods uh question marks and exclamation And we know that th not a very g I mean we miss a lot of them Yeah but OK but f Comma also or not No commas No And then we looked at locations where uh if you have overlapping speech and someone else starts a sentence you know where do these where do other people start their Ah turns not turns really but you know sentences um So we only looked at cases where there was a foreground speaker and then at the to at the so the the foreground speaker started into their sentence and then someone else started later OK between the