C: And so I think it was it was it was very important for people then who were doing something diagnostic to say " hey , I know I 'm doing something that isn't kosher " .
I: Hmm .
C: And to make it really clear that they knew it .
C: So that was , I think , why why it became a popular term .
F: Yeah , it just seems like , you know , i if it were " bootstrapping " , or if it were I mean , there are other ways to maybe get the point across .
C: But it isn't bootstrapping .
C: Uh , it 's it 's really cheating .
A: It 's us it 's using information you wouldn't normally have .
C: It 's just ng
C: Yeah . Yeah .
F: Hu - Well . OK .
C: But it 's cheating in a way that 's pril It 's it 's announcing to everybody " hey , I 'm cheating , by doing this . "
A: So .
C: It 's saying so it 's all above the table .
C: I 'm actually using this other thing .
F: OK .
C: That 's that 's that 's the reason .
F: OK . Alright .
C: Bootstrapping would imply it was actually legitimate in some kind of way .
A: Hmm .
F: Well , I I think C: But A: We 're not de - legitimizing the data .
A: We 're de - legitimizing the experiment .
A: We 're not saying that the data is cheating data .
A: We 're saying th we are cheating by using this data .
C: Yeah .
F: OK .
C: Yeah .
A: Because normally you wouldn't have that data available .
F: OK . OK .
C: Yeah .
C: Anyway .
F: Does seem to me that it that it carries over some baggage with it . That , uh , i that F: I can understand it in context as you described it .
F: But , it it seems to me that u um , it does import some negative evaluation , that , um , would maybe not be good if C: It has some shock value .
F: It has shock value ,
F: and
C: Which is part of why it 's used .
F: to the wr to the wrong audience , I think that that might be t a negative shock .
H: Yeah . I thi I think to the wrong audience , I agree that , um , p
C: Uh - huh .
I: It 's like a disclaimer .
H: to the wrong audie we should just explain what it what it means here , and that it 's a common term ,
F: Yeah . We started with hand - marked data , or with , uh , t you know , hand - transcribed data ,
H: and
H: we 're F: or A: Shoot .
F: Yeah .
F: Did you break the microphone ?
H: Yeah .
A: Just the clip .
F: Nnn . OK .
I: Now we have to l delete that expletive .
F: OK . Well , I feel better now .
F: Thank you very much .
H: Hmm . The first time I heard that , I I thought , you know , the same thing
E: Thank you , Andreas .
H: and I guess you ju after a while it becomes almost a It 's a bu bit of a a humbling thing when somebody says that .
A: It 's p it it really is part of the jargon .
C: But it 's also is H: If they get good results but we were cheating on this feature because we took it for granted even though we can't really assume that , then it 's actually the opposite .
C: Right .
H: It 's F: The trouble is , that F: you know , I understand it in that context ,
F: but it but it is almost resentful .
F: It 's almost like , you know , resentful of the data , resentful of the , um , the hard work that 's going into preparing the data
A: Well F: is C: No , no . It has I don't think you 're saying the data is cheating .
H: Well , it 's it 's C: I think you 're saying " in my experiment , I cheated in this way " .
A: That 's what I was saying .
D: Yeah .
C: I mean , another is what Liz was talking about , how in the Switchboard tests in all the Switchboard tests we 've been g doing , we 've been making the same er uh , same uh , uh , standard using the same standard way of of getting the data to test on .
C: Which means that we weren't actually running it on on , uh , on data that had had no speech .
C: And , in a sense , that was cheating .
F: OK .
C: So i so it it 's it 's a good wake - up call if people " well , we have this performance but you have to keep in mind , we 're doing we 're we 're not doing the whole real task in this way and this way and this way .
C: But , I 'll convince you that that it 's still important for you to listen to what I have to say next because of this and this . "
F: OK . OK .
C: And so it 's just a way of putting it all out on the table , uh , as opposed to A: Uh , and it 's used for a lot of different types of data .
A: So , segment whether you have segmentation or not , is it male or female or not  Um , do you know the signal - to - noise ?
C: Yeah .
F: Mmm .
A: Like , that 's another one I see all the time , where you assume it 's known .
C: Yeah . Sure .
F: Oh , interesting .
C: Yeah .
A: And you say it 's cheating because you don't actually compute it .
I: Hmm .
F: I didn't know that .
F: Huh !
C: In C: In the multi - band experiments , in the first one , we really wanted to find out , " what if you knew which band was really noisy ? "
F: Uh - huh !
A: N Right .
C: I mean , suppose you just know that .
A: Same cheating .
C: And then then , even if you know that , can that help you ?
C: I mean , what can what strategy can you do , to do well without that particular band in the spectrum ?
I: Hmm .
C: And so that was important to know as a baseline .
C: And and then , once you knew that , then s you go " well , now how do I know that that 's noisy ? "
I: Hmm .
F: OK .
C: And and and , uh C: But , y you know , I I should also say that there 's a lot of It 's not just the word " cheating " . But there 's lots of other things that we talk about , which , as soon as you go outside of your narrow little group , it gets very , very confusing to people .
A: Mm - hmm .
F: Oh , sure . Terminology is always context dependent .
C: And I i i i
F: No question about it .
F: It 's just that it seems like the the point without the negative evaluation would be F: However , you know , if i I understand your point . That it th it has a long tradition in this field A: It 's not even really negative .
F: I didn't realize .
F: And is used F: This is interesting what what Adam said about it being used also for a bunch of other , uh , dimensions .
C: Well , the example I was thinking of , also , was this w thing that that Herve and and Hynek and I made about , uh , increasing the error rate .
C: And , so we we did a number of papers , and talks , and so forth about , uh w uh , i the the virtues of increasing the error rate .
F: Oh .
C: OK .
C: And I mean , and the whole point of it was , not that it was good to increase your error rate , but it was good to be willing to risk increasing your error rate by trying risky things , and trying Because there 's this notion of a local minimum , that if you just have some system that 's very complex ,
C: and you t you  you turn some knobs to try to make it better and better , you 'll never get out of this local minimum .
A: Fine - tune little bits .
F: Mm - hmm .
C: You have to be willing to jump to something that 's quite different .
I: Mmm .
F: Interesting .
C: And the first time you jump to something quite different , or maybe the first ten times or a hundred times you do , it 's gonna be much much worse ,
C: because you 've optimized the other system .
C: So the effect the immediate effect is gonna be to increase your error rate .
F: Interesting .
C: And so it was we had a couple papers like " Towards increasing the error rate in speech " and and so on .
A: Hmm .
F: Huh !
C: And we really did get feedback from a few people , some of whom were , you know , fairly senior , that that , um , " Well , you know , you sh you you c We 're c really concerned about you misleading people into thinking they should be increasing the error rate . "
I: Hmm .
F: Yeah .
C: And , uh , and we go we thought " Oh , come on . "
C: But .
A: Did you read the paper ?
I: But weren't you cheating in those experiments ?
F: Yeah .
C: That 's C: we weren't .
C: That 's why we increased the error rate .
F: Interesting .
H: Actually , I think of cheating as a way to do some work , where you can't address all of the computational tasks .
H: Like , if we wanna study , um , speaker habits , but we can't do speaker detection .
H: But we wanna assume Let 's say , we know this is Jane and we know this is Chuck ,
H: even though automatically to look at the habits , we would need to also first figure that out .
C: Mm - hmm .
H: But we can sort of cheat on that factor
H: because it 's somebody else 's research .
H: And then we just H: So it 's Then we wanna make a proof of concept for A: What what this w
A: Right . What would this be like if w it were perfect ?
H: If someone else A: If this component were perfect ?
H: Right . Eh or w But we really can't work on that problem .
D: Yep .
H: We don't have time ,
H: we 're not interested , or w whatever