D: I I would imagine that the data would be really different .
D: Um , so it 's just B: Yeah .
B: But It was never th th the goal of that data collection to to serve for sat for such a purpose .
B: So that 's why for example the tasks were not differentiated by intentionality ,
D: Mm - hmm .
B: there was n there was no label ,
D: Mm - hmm .
D: Right .
B: you know , intention A , intention B , intention C .
B: Or task A , B , C .
B: Um
B: I 'm sure we can produce some if we need it ,
D: Mm - hmm .
B: um , that that will help us along those lines .
B: But , you know , you gotta leave something for other people to model .
B: So , to Finding out what , you know , situational con what the contextual factors of the situation really are , you know is an interesting s interesting thing .
D: Mm - hmm .
D: Mm - hmm .
B: u u Sort of I 'm , at the moment , curious and I 'm I 'm s w want to approach it from the end where we can s sort of start with this toy system that we can play around with ,
D: Mm - hmm .
B: so that we get a clearer notion of what input we need for that ,
D: Mm - hmm .
B: what suffices and what doesn't .
B: And then we can start worrying about where to get this input ,
B: what what do we need , you know B: Ultimately once we are all experts in changing that parser , for example , maybe , there 's just a couple three things we need to do
B: and then we get more whatever , part of speech and more construction - type - like stuff out of it .
D: Mm - hmm .
D: Hmm .
B: It 's a m pragmatic approach , uh , at the moment .
E: How exactly does the data collection work ?
E: Do they have a map ,
E: and then you give them a scenario of some sort ?
B: OK .
B: Imagine you 're the the subject .
B: You 're gonna be in here ,
B: and somebody B: And and you see , uh , either th the three - D model , or uh , a QuickTime animation of standing u in a square in Heidelberg .
B: So you actually see that .
B: Um . The uh ,
B: um , first thing is you have to read a text about Heidelberg .
B: So , just off a textbook , uh , tourist guide , to familiarize , uh , yourself with that sort of odd - sounding German street names , like Fischergasse and so forth .
B: So that 's part one .
B: Part two is , you 're told that this huge new , wonderful computer system exists , that can y tell you everything you want to know ,
B: and it understands you completely .
B: And so you 're gonna pick up that phone ,
B: dial a number ,
B: and you get a certain amount of tasks that you have to solve .
B: First you have to know find out how to get to that place ,
B: maybe with the intention of buying stamps in there .
B: Maybe So , the next task is to get to a certain place and take a picture for your grandchild .
B: The third one is to get information on the history of an object .
B: The fourth one B: And then the g system breaks down .
B: It crashes ,
D: a At the third ?
B: And D: Right then ?
B: After the third task .
D: OK .
B: And then B: Or after the fourth .
B: Some find @ @ <comment> Forget that for now .
B: And then , a human operator comes on , and and exp apologizes that the system has crashed ,
B: but , you know , urges you to continue , you know ?
B: now with a human operator .
B: And so , you have basically the same tasks again ,
B: just with different objects ,
B: and you go through it again ,
B: and that was it .
B: Oh , and one one little bit B: w And uh , the computer you are you are being told the computer system knows exactly where you are , via GPS .
B: When the human operator comes on , um , that person does not know .
B: So the GPS is crashed as well .
B: So the person first has to ask you " Where are you ? " .
B: And so you have to do some s tell the person sort of where you are , depending on what you see there .
B: Um , this is a a a a a bit that I d I don't think we B: Did we discuss that bit ?
B: Uh , I just sort of squeezed that in now .
B: But it 's something , uh , that would provide some very interesting data for some people I know .
B: So .
D: So , in the display you can Oh , you said that you cou you might have a display that shows , like , the B: Yeah .
B: a Additionally , y you have a a a sort of a map type display .
D: a w your perspective ? sort of ?
D: And so , as you B: Uh , two - D .
B: n
D: Oh , two - D .
D: OK .
B: Two - D .
D: So as you move through it that 's - they just track it on the for themselves
B: Yeah .
B: b y
B: You don't D: there .
B: That 's B: I don't know .
B: I but y I don't think you really move ,
D: OK .
B: sort of .
D: So
B: Yeah ?
B: I mean that would be an an an enormous technical effort ,
B: unless we would B: We can show it walks to , you know .
B: We can have movies of walking ,
B: you walking through through Heidelberg , and u ultimately arriving there .
D: Mm - hmm .
B: Maybe we wanna do that .
B: Yeah .
D: Uh , I was just trying to figure out how how ambitious the system is .
B: The map was sort of intended to B: You want to go to that place .
D: Mm - hmm .
B: You know , and it 's sort of there .
B: And you see the label of the name B: So we get those names , pronunciation stuff , and so forth ,
D: Mm - hmm .
B: and we can change that .
D: Mm - hmm .
D: So your tasks don't require you to D: I mean , uh yo you 're told So when your task is , I don't know , " Go buy stamps " or something like that ?
D: So , do you have to respond ?
D: or does your D: Uh , what are you ste what are you supposed to be telling the system ?
D: Like , w
D: what you 're doing now ?
D: or B: Well , we 'll see what people do .
D: There 's no D: OK , so it 's just like , " Let 's figure out what they would say under the circumstances " .
B: Yeah , and and we will record both sides .
B: I mean , we will record the Wi - the Wizard D: Uh - huh .
D: Uh - huh .
B: I mean , in both cases it 's gonna be a human , in the computer , and in the operator case .
B: And we will re there will be some dialogue , you know ?
B: So , you first have to do this , and that ,
D: Yep .
B: and and D: Mm - hmm .
B: see wh what they say .
B: We can ins instruct the , uh , wizard in how expressive and talkative he should be .
B: But um ,
B: maybe the maybe what you 're suggesting B: Is what you 're suggesting that it might be too poor , the data , if we sort of limit it to this ping pong one t uh , task results in a question and then there 's an answer and that 's the end of the task ?
B: You wanna m have it more more steps , sort of ?
D: Yeah , I I don't know how much direction is given to the subject about what their interaction D: I mean , th they 're unfamiliar w with interacting with the system .
D: All they know is it 's this great system that could do s stuff .
B: Mm - hmm .
B: Mm - hmm .
C: Oh yeah , but to some extent this is a different discussion .
D: Right ?
D: So C: OK ?
C: So . Uh , we we have to have this discussion of th the experiment , and the data collection , and all that sorta stuff
D: Uh - huh .
C: and we do have , um , a student who is a candidate for wizard .
C: Uh , she 's gonna get in touch with me .
C: It 's a student of Eve 's .
C: FEY ,
C: Fey ?
C: Spelled FEY .
C: Do you do you D: Oh , Fey Parrill .
C: You know her ?
D: Yeah .
D: Uh - huh .
C: OK .
C: Sh - Is sh
D: She started taking the class last year
D: and then didn't um , you know , didn't continue .
D: I g She 's a g
D: Is she an undergradua
C: She 's graduated .
D: She is a graduate , OK .
C: Yeah .
D: Yeah , I m I know her very , very briefly .
D: I know she was inter you know , interested in aspect and stuff like that .
C: OK .
C: So , anyway , she 's looking for some more part time work w while she 's waiting actually for graduate school .
C: And she 'll be in touch .
C: So we may have someone , uh , to do this ,
C: and she 's got you know , some background in in all this stuff .
C: And is a linguist st
C: and , so
C: So . That 's So , Nancy , we 'll have an At some point we 'll have another discussion on exactly wha t t you know , how that 's gonna go .
D: Mm - hmm .
C: And um , Jane , but also , uh , Liz have offered to help us do this , uh , data collection and design and stuff .
D: Mm - hmm .
D: Mmm .
C: So , when we get to that we 'll have some people doing it that know what they 're doing .
D: OK .
D: I guess the reason I was asking about the sort of the de the details of this kind of thing is that , um , it 's one thing to collect data for , I don't know , speech recognition or various other tasks that have pretty c clear correct answers ,
D: but with intention um , obviously , as you point out , there 's a lot of di other factors
D: and I 'm not really sure , um , how how e the question of how to make it a t appropriate toy version of that Um , it 's ju it 's just hard .
D: So , I mean , obviously it 's a E: Yeah , uh , actually I guess that was my question .