A: but the rest would be constrained .
A: So , I think we have a version that 's pretty good for the native speakers .
A: I don't know yet about the non - native speakers .
A: And , um ,
A: we basically also made noise models for the different sort of grouped some of the mouth noises together .
A: Um , so , and then there 's a background speech model .
A: And we also A: There was some neat or , interesting cases ,
A: like there 's one meeting where , um , Jose 's giving a presentation
A: and he 's talking about , um , the word " mixed signal "
A: and someone didn't understand , uh , that you were saying " mixed " A: I think , Morgan .
H: Yeah , yeah .
A: And so your speech - ch was s saying something about mixed signal .
A: And the next turn was a lot of people saying " mixed " ,
A: like " he means mixed signal " or " I think it 's mixed " .
A: And the word " mixed " in this segment occurs , like , a bunch of times .
H: Sh
A: And Chuck 's on the lapel here ,
A: and he also says " mixed "
A: but it 's at the last one ,
A: and of course the aligner th aligns it everywhere else to everybody else 's " mixed " ,
H: Yeah .
A: cuz there 's no adaptation yet .
A: So there 's  I think there 's some issues about u
A: We probably want to adapt at least the foreground speaker .
A: But , I guess Andreas tried adapting both the foreground and a background generic speaker ,
A: and that 's actually a little bit of a f funky model .
A: Like , it gives you some weird alignments ,
A: just because often the background speakers match better to the foreground than the foreground speaker .
F: Oh D: Yeah .
H: Oh .
A: So there 's some things there ,
A: especially when you get lots of the same words , uh , occurring in the F: Well , the F: I I think you can do better by uh , cloning F: so we have a reject phone .
F: And you and what we wanted to try with you know , once we have this paper written and have a little more time , uh , t cloning that reject model
F: and then one copy of it would be adapted to the foreground speaker to capture the rejects in the foreground ,
F: like fragments and stuff ,
F: and the other copy would be adapted to the background speaker .
A: Right . I mean , in general we actually F: And A: Right now the words like partial words are reject models
F: Mm - hmm .
A: and you normally allow those to match to any word .
A: But then the background speech was also a reject model ,
A: and so this constraint of not allowing rejects in between A: you know , it needs to differentiate between the two .
A: So just sort of working through a bunch of debugging kinds of issues .
F: Right .
A: And another one is turns , like people starting with " well I think "
A: and someone else is " well how about " .
A: So the word " well " is in this in this segment multiple times ,
A: and as soon as it occurs usually the aligner will try to align it to the first person who says it .
A: But then that constraint of sort of uh , proximity constraint will push it over to the person who really said it in general .
E: Is the proximity constraint a hard constraint ,
E: or did you do some sort of probabilistic weighting distance , or ?
F: We we didn't F: No .
A: Right now it 's a kluge .
F: We w
F: OK .
F: We it 's straightforward to actually just have a a penalty that doesn't completely disallows it but discourages it .
F: But , um , we just didn't have time to play with , you know , tuning yet another yet another parameter .
E: The ve level .
E: Yeah .
A: Yeah .
F: And really the reason we can't do it is just that we don't have a we don't have ground truth for these .
F: So , we would need a hand - marked , um , word - level alignments
F: or at least sort of the boundaries of the speech betw you know , between the speakers .
F: Um , and then use that as a reference
F: and tune the parameters of the of the model , uh , to op to get the best performance .
A: Yeah .
B: G given I I mean , I wa I wa I was gonna ask you anyway , uh , how you assessed that things were better .
F: Mm - hmm .
A: I looked at them .
A: I spent two days um , in Waves B: OK .
A: Oh , it was painful
A: because the thing is , you know the alignments share a lot in common ,
A: so A: And you 're yo you 're looking at these segments where there 's a lot of speech .
A: I mean , a lot of them have a lot of words .
B: Yeah .
A: Not by every speaker
A: but by some speaker there 's a lot of words .
B: Yeah .
H: Ju
A: No , not A: I mean that if you look at the individual segments from just one person you don't see a lot of words ,
B: Yeah .
F: Mm - hmm .
B: Yeah .
A: but altogether you 'll see a lot of words up there .
A: And so the reject is also mapping and pauses D: Yeah .
A: So I looked at them all in Waves
A: and just lined up all the alignments ,
A: and , at first it sort of looked like a mess
A: and then the more I looked at it , I thought " OK , well it 's moving these words leftward
A: and " You know , it wasn't that bad .
A: It was just doing certain things wrong .
A: So A: But , I don't , you know , have time to l <comment> to look at all of them
A: and it would be really useful to have , like , a a transcriber who could use Waves ,
A: um , just mark , like , the beginning and end of the foreground speaker 's real words A: like , the beginning of the first word , the end of the last word C: Yeah .
A: and then we could , you know , do some adjustments .
C: I OK .
C: I have to ask you something ,
C: is i does it have to be Waves ? Because if we could benefit from what you did , incorporate that into the present transcripts , <comment> that would help .
F: No .
C: And then , um , the other thing is , I believe that I did hand
C: So . One of these transcripts was gone over by a transcriber
C: and then I hand - marked it myself so that we do have , uh , the beginning and ending of individual utterances .
F: Mm - hmm .
C: Um , I didn't do it word level ,
C: but but in terms C: So I so for for one of the N S A groups .
A: Mm - hmm .
C: And also I went back to the original one that I first transcribed and and did it w uh , w uh , utterance by utterance for that particular one .
C: So I think you do have C: if that 's a sufficient unit , I think that you do have hand - marking for that .
C: But it 'd be wonderful to be able to benefit from your Waves stuff .
A: Mm - hmm .
F: We don't care what what tool you use .
A: Yeah . I mean , if if you can , um if you wanna C: OK .
C: I used it in Transcriber
F: U uh C: and it 's it 's in the A: well , Jane and I were just in terms of the tool , talking about this .
A: I guess Sue had had some reactions .
A: You know , interface - wise if you 're looking at speech , you wanna be able to know really where the words are .
C: Yeah , that 's right .
A: And so , we can give you some examples of sort of what this output looks like ,
C: Middle of the word , or A: um , and see if you can in maybe incorporate it into the Transcriber tool some way ,
A: or C: Well , I th I 'm thinking just ch e e incorporating it into the representation .
A: Um .
C: I mean , if it 's if it 's A: You mean like Yeah , word start insights .
C: if you have start points , if you have , like , time tags ,
A: Right .
C: which is what I assume .
C: Isn't that what what you ?
C: Well , see , Adam would be F: Yeah , whatever you use .
F: I mean , we convert it to this format that the , um , NIST scoring tool unders uh , CTM . Conversation Time - Marked file .
A: Yeah .
F: And and then that 's the that 's what the E: I think Transcriber , uh , outputs CTM .
C: If it ?
C: OK .
E: I think so .
C: So you would know this more than I would .
A: Yeah .
F: Right .
A: So , I mean C: It seems like she if she 's g if she 's moving time marks around ,
C: since our representation in Transcriber uses time marks , it seems like there should be some way of of using that benefitting from that .
E: Right .
A: Yeah , it wou the advantage would just be that when you brought up a bin you would be able if you were zoomed in enough in Transcriber to see all the words ,
B: Mm - hmm .
A: you would be able to , like , have the words sort of located in time ,
A: if you wanted to do that .
B: So so if we e e even just had a a It sounds like w we  we almost do .
A: So .
B: Uh , if we We have two .
C: We have two .
B: Yeah .
B: Just ha uh , trying out the alignment procedure that you have on that
A: Mm - hmm .
B: you could actually get something , um uh , uh , get an objective measure .
F: Mm - hmm .
B: Uh A: You mean on on the hand - marked , um A: So we we only r hav I only looked at actually alignments from one meeting that we chose ,
B: Yeah .
A: I think MR four ,
A: just randomly , um A: And F: Actually , not randomly .
A: Not randomly F: We knew we knew that it had these insertion errors from A: It had sort of average recognition performance in a bunch of speakers
F: Yeah .
F: Yeah .
A: and it was a Meeting Recorder meeting .
A: Um .
A: But , yeah , we should try to use what you have .
A: I did re - run recognition on your new version of MR one .
C: Oh , good .
A: I I mean the the one with Dan Ellis in it and Eric .
C: Good !
C: Uh - huh . Yeah , exactly . Yeah . Yeah .
G: I don't think that was the new version .
A: Um A: That A: Yeah , actually it wasn't the new new ,
C: OK .
A: it was the medium new .
A: But but we would we should do the the latest version .
C: OK .
G: Yeah .
G: You did you adjust the the utterance times , um , for each channel ?
A: It was the one from last week .
C: Yes .
C: Yes , I did .
C: And furthermore , I found that there were a certain number where  not not a lot , but several times I actually moved an utterance from Adam 's channel to Dan 's or from Dan 's to Adam 's .
C: So there was some speaker identif
C: And the reason was because I transcribed that at a point before  uh , before we had the multiple audio available
C: f
C: so I couldn't switch between the audio .
C: I I transcribed it off of the mixed channel entirely ,
C: which meant in overlaps , I was at a at a terrific disadvantage .
A: Right .
A: Right .
C: In addition it was before the channelized , uh , possibility was there .
C: And finally I did it using the speakers of my , um  of you know , off the CPU on my on my machine
C: cuz I didn't have a headphone .
C: So it @ @ , like , I mean A: Right .
C: Yeah , I I mean , i in retrospect it would 've been good to ha have got I should 've gotten a headphone .
C: But in any case , um , thi this is this was transcribed in a in a , uh , less optimal way than than the ones that came after it ,
C: and I was able to you know , an and this meant that there were some speaker identif identifications
G: Well , I know there were some speaker labelling problems , um , after interruptions .
C: which were changes .
C: Yeah .
C: Fixed that .
G: Is that what you 're referring to ?
G: I mean , cuz there 's this one instance when , for example , you 're running down the stairs .
C: Oh , well G: I remember this meeting really well .
D: Yeah .
G: Right .
A: Don Don has had  He knows he can just read it like a play .
G: It 's a G: Yeah , I 've I 've I 'm very well acquainted with this meeting .
G: Yeah , I can s
D: Yeah .
A: " And then she said , and then he said . "
G: Yeah , I know it by heart .
G: So , um , there 's one point when you 're running down the stairs .
C: Uh - oh .
G: Right ?
G: And , like , there 's an interruption .
G: You interrupt somebody ,
G: but then there 's no line after that .
G: For example , there 's no speaker identification after that line .
C: Uh - huh .
G: Is that what you 're talking about ?
G: Or were there mislabellings as far as , like , the a Adam was ?
C: That was fixed , um , before C: i i i I think I I think I understood that pretty G: Yeah . Cuz I thought I let you know about that .
C: Thank you for mentioning .
G: Yeah .
C: Yeah , no , tha that That I think went away a couple of versions ago ,
G: OK .
G: But you 're actually saying that certain , uh , speakers were mis mis - identified .
C: but it 's good to know .
C: Yeah . So , with under um , uh , listening to the mixed channel , there were times when , as surprising as that is , I got Adam 's voice confused with Dan 's and vice versa G: OK .
C: not for long utterances ,
G: OK .
A: Yeah .
C: but jus just a couple of places ,
B: Mm - hmm .
C: and embedde embedded in overlaps .
C: The other thing that was w interesting to me was that I picked up a lot of , um , backchannels which were hidden in the mixed signal ,
C: which , you know , I mean , you c not not too surprising .
A: Right .
C: But the other thing that C: I I hadn't thought about this ,
C: but I thou I wanted to raise this when you were uh , with respect to also a strategy which might help with the alignments potentially ,
C: but that 's C: When I was looking at these backchannels , they were turning up usually  very often in w well , I won't say " usually " but anyway , very often , I picked them up in a channel w which was the person who had asked a question .
C: S so , like , someone says " an and have you done the so - and - so ? "
C: And then there would be backchannels ,
C: but it would be the person who asked the question .
C: Other people weren't really doing much backchannelling .
C: And , you know , sometimes you have the Yeah , uh - huh .
A: Well , that 's interesting .
C: I mean , i it wouldn't be perfect ,
A: Yeah .
C: but but it does seem more natural to give a backchannel when when you 're somehow involved in the topic ,
A: No , that 's really interesting .
C: and the most natural way is for you to have initiated the topic by asking a question .
B: Mm - hmm .
F: Well ,
F: I think A: That 's interesting .
F: No . I think it 's actually I think what 's going on is backchannelling is something that happens in two - party conversations .
C: Mm - hmm .
F: And if you ask someone a question , you essentially initiating a little two - party conversation .
C: Yeah .
A: Well , actu Yeah , when we looked at this F: So then you 're so and then you 're expected to backchannel
C: Exactly .
F: because the person is addressing you directly and not everybody .
C: Exactly .
C: Exactly my point .
F: Yeah .
C: An - and so this is the expectation thing that uh , uh ,
F: Yeah .
A: Mm - hmm .
F: Right .
F: Right .
C: just the dyadic C: But in addition , you know , if someone has done this analysis himself and isn't involved in the dyad , but they might also give backchannels to verify what what the answer is that this that the the answerer 's given B: H
B: I tell you , I say I say " uh - huh " a lot ,
A: Right .
A: It 's C: There you go .
A: Well , but it 's interesting
A: cuz , uh B: while people are talking to each other .
C: There you go .
A: But there are fewer I think there are fewer " uh - huhs " .
C: Yeah .
C: Yeah .
A: I mean , just from We were looking at word frequency lists to try to find the cases that we would allow to have the reject words in between in doing the alignment .
A: You know the ones we wouldn't constrain to be next to the other words .
C: Oh , yeah .
A: And " uh - huh " is not as frequent
A: as it sort of would be in Switchboard ,
A: if you looked at just a word frequency list of one - word short utterances .
A: And " yeah " is way up there ,
A: but not " uh - huh " .
A: And so I was thinking
A: thi it 's not like you 're being encouraged by everybody else to keep talking in the meeting .
A: And uh , that 's all , I I 'll stop there ,
A: cuz I I think what you say makes a lot of sense .
C: Well , that 's right .
C: And that would C: Well , an
A: But it was sort of C: And what you say is the is the re uh , o other side of this ,
C: which is that , you know , so th there are lots of channels where you don't have these backchannels , w when a question has been asked
C: and and these A: Right .
A: There 's just probably less backchannelling in general ,
C: Mm - hmm .
C: So that 's good news , really .
A: even if you consider every other person altogether one person in the meeting ,
A: but we 'll find out anyway .
A: We were I guess the other thing we 're we 're I should say is that we 're gonna , um try compare this type of overlap analysis to Switchboard ,
F: And
A: where A: and CallHome ,
A: where we have both sides , so that we can try to answer this question of , you know , is there really more overlap in meetings or is it just because we don't have the other channel in Switchboard
E: Mm - hmm .
B: Mm - hmm .
A: and we don't know what people are doing .
A: Try to create a paper out of that .
B: Yeah . I mean , y y you folks have probably already told me ,
B: but were were you intending to do a Eurospeech submission ,
B: or ?
A: Um , you mean the one due tomorrow ?
B: Yeah .
A: Yeah . Well , we 're still , like , writing the scripts for doing the research ,
A: and we will Yes , we 're gonna try .
C: Mm - hmm .
A: And I was telling Don , do not take this as an example of how people should work .
B: Do as I say ,
G: That 's r
A: So , <comment> we will try .
B: don't do as I do .
B: Yeah .
E: Well A: It 'll probably be a little late ,
E: It is different .
A: but I 'm gonna try it .
E: In previous years , Eurospeech only had the abstract due by now ,
E: not the full paper .
G: Right .
A: Right .
E: And so all our timing was off .