Grad B: Yeah .
Grad C: OK ? And um And then after three tasks the system breaks down . And Fey comes on the phone as a human operator . And says " Sorry the system broke down but let 's continue . " And we sort of get the idea what people do when they s think they speak to a machine and what people say when they think they speak to a human , or know , or assume they speak to a human .
Grad A: OK . Huh .
Grad B: Mm - hmm . Mm - hmm .
Grad C: That 's the data collection . And um And Fey has some thirty subjects lined up ? Something ?
Undergrad E: Yeah .
Grad C: And um And they 're r ready uh to roll .
Undergrad E: And more and more every day .
Grad C: And we 're gonna start tomorrow at three ? four ? one ?
Undergrad E: Tomorrow , well we don't know for sure . Because we don't know whether that person is coming or not ,
Grad C: OK . Around four - ish .
Undergrad E: but Grad C: And um we 're still l looking for a room on the sixth floor because they stole away that conference room . Um behind our backs . But Professor D: Well , there are these uh uh oh , I see , we have to Yeah , it 's tricky . We 'll let 's let we 'll do that off - line , OK .
Grad C: Yeah , but I i i it 's happening . David and and Jane and and Lila are working on that as we speak .
Professor D: OK .
Grad C: OK . That was the uh the data collection in a nutshell . And um I can report a so I did this but I also tried to do this so if I click on here , Isn't this wonderful ? we get to the uh uh belief - net just focusing on on the g Go - there node . uh Analogously this would be sort of the reason node and the timing node and so forth .
Grad B: Mm - hmm .
Grad C: And what w what happened is that um design - wise I 'd sort of n noticed that we can we still get a lot of errors from a lot of points to one of these sub Go - there User Go - there Situation nodes . So I came up with a couple of additional nodes here where um whether the user is thrifty or not , and what his budget is currently like , is going to result in some financial state of the user . How much will he is he willing to spend ? Or can spend . Being the same at this just the money available , which may influence us , whether he wants to go there if it is you know charging tons of dollars for admission or its gonna g cost a lot of t e whatever . Twenty - two million to fly to International Space Station , you know . just Not all people can do that .
Professor D: Right .
Grad C: So , and this actually turned out to be pretty key , because having specified sort of these uh this this intermediate level Um and sort of noticing that everything that happens here let 's go to our favorite endpoint one is again more or less we have um then the situation nodes contributing to the the endpoint situation node , which contributes to the endpoint and so forth . um I can now sort of draw straight lines from these to here , meaning it g of course goes where the sub - S everything that comes from situation , everything that comes from user goes with the sub - U , and whatever we specify for the so - called " Keith node " , or the discourse , what comes from the um parser , construction parser , um will contribute to the D and the ontology to the sub - O node . And um one just s sort of has to watch which also final decision node so it doesn't make sense t to figure out whether he wants to enter , view or approach an object if he never wants to go there in the first place . But this makes the design thing fairly simple . And um now all w that 's left to do then is the CPG 's , the conditional probabilities , for the likelihood of a person having enough money , actually wanting to go a place if it costs , you know this or that . And um OK . and once um Bhaskara has finished his classwork that 's where we 're gonna end up doing . You get involved in that process too . And um And for now uh the the question is " How much of these decisions do we want to build in explicitly into our data collection ? " So Um , one could sort of think of you know we could call the z see or you know , people who visit the zoo we could s call it " Visit the zoo tomorrow " , so we have an intention of seeing something , but not now but later .
Professor D: Right . Yeah . Yeah , so let 's s uh s see I th I think that from one point of view , Uh , um , all these places are the same , so that d d That , um in terms of the linguistics and stuff , there may be a few different kinds of places , so I th i it seems to me that We ought to decide you know , what things are k are actually going to matter to us . And um , so the zoo , and the university and the castle , et cetera . Um are all big - ish things that um you know have different parts to them , and one of them might be fine .
Grad C: Hmm . Hmm , hmm . Yeah The the reason why we did it that way , as a as a reminder , is uh no person is gonna do all of them .
Professor D: And Grad C: They 're just gonna select u um , according to their preferences .
Professor D: Yeah , yeah .
Grad C: " Ah , yeah , I usually visit zoos , or I usually visit castles , or I usually " And then you pick that one .
Professor D: Right , no no , but but s th point is to to y to build a system that 's got everything in it that might happen you do one thing .
Undergrad E: They 're redundant .
Professor D: T to build a system that um had the most data on a relatively confined set of things , you do something else . And the speech people , for example , are gonna do better if they if things come up uh repeatedly . Now , of course , if everybody says exactly the same thing then it 's not interesting . So , all I 'm saying is i th there 's there 's a kind of question of what we 're trying t to accomplish . and I think my temptation for the data gathering would be to uh , you know And each person is only gonna do it once , so you don't have to worry about them being bored , so if if it 's one service , one luxury item , you know , one big - ish place , and so forth and so on , um then my guess is that that the data is going to be easier to handle . Now of course you have this I guess possible danger that somehow there 're certain constructions that people use uh when talking about a museum that they wouldn't talk about with a university and stuff , um but I guess I 'm I uh m my temptation is to go for simpler . You know , less variation . But I don't know what other people think about this in terms of Grad B: So I don't exactly understand Professor D: uh Grad B: like I I I guess we 're trying to limit the detail of our ontology or types of places that someone could go , right ? But who is it that has to care about this , or what component of the system ?
Professor D: Oh , well , uh th I think there are two places where it comes up . One is uh in the th these people who are gonna take this and and try to do speech with it .
Grad B: Mm - hmm .
Professor D: uh Lots of pronunciations of th of the same thing are going to give you better data than l you know , a few pronunciations of lots more things .
Grad B: OK .
Professor D: That 's one .
Grad B: So we would rather just ask uh have a bunch of people talk about the zoo , uh and assume that that will that the constructions that they use there will give us everything we need to know about these sort of zoo , castle , whatever type things , these bigger places .
Professor D: Bigger Y yeah thi well this is a question for Grad B: And that way you get the speech data of people saying " zoo " over and over again or whatever too .
Professor D: Yeah . Yeah .
Grad B: OK .
Professor D: Yeah . So this is a question for you ,
Grad B: Mm - hmm .
Professor D: and , you know , if we if we do , and we probably will , actually try to uh build a prototype , uh probably we could get by with the prototype only handling a few of them anyway . So , Um Grad C: Yeah , the this was sort of these are all different sort of activities . Um But I think y I I got the point and I think I like it . We can do put them in a more hierarchical fashion . So , " Go to place " and then give them a choice , you know either they 're the symphony type or opera type or the tourist site guide type or the nightclub disco type person and they say " yeah this is on that " go to big - ish place " ,
Grad B: Mm - hmm .
Grad C: this is what I would do . " And then we have the " Fix " thing , and then maybe " Do something the other day " thing , so . My question is I guess , to some extent , we should y we just have to try it out and see if it works . It would be challenging , in in a sense , to try to make it so so complex that they even really should schedule , or to plan it , uh , a more complex thing in terms of OK , you know , they should get the feeling that there are these s six things they have to do and they sh can be done maybe in two days .
Professor D: Well yeah .
Grad C: So they make these decisions ,
Professor D: Well I think th th
Grad C: " Can I go there tomorrow ? "
Professor D: yeah .
Grad C: or you know influences
Grad B: Mm - hmm .
Professor D: Yeah . Well , I think it 's easy enough to set that up if that 's your expectation . So , the uh system could say , " Well , uh we 'd like to to set up your program for two days in Heidelberg , you know , let 's first think about all the things you might like to do . So there th i i in I mean in I th I I 'm sure that if that 's what you did then they would start telling you about that , and then you could get into um various things about ordering , if you wanted .
Grad C: Mm - hmm . Yeah . Yeah , but I think this is part of the instructor 's job . And that can be done , sort of to say , " OK now we 've picked these six tasks . " " Now you have you can call the system and you have two days . "
Professor D: I 'm sorry .
Grad C: And th w
Professor D: No , we have to help we have to decide . Fey will p carry out whatever we decide . But we have to decide you know , what is the appropriate scenario . That 's what we 're gonna talk about t yeah .
Grad C: Yep , yep .
PhD F: But these are two different scenarios entirely . I mean , one is a planner The other , it kind of give you instructions on the spot
Grad C: Yeah , but th the I don't I 'm not really interested in sort of " Phase planning " capabilities . But it 's more the how do people phrase these planning requests ? So are we gonna masquerade the system as this as you said simple response system , " I have one question I get one response " , or should we allow for a certain level of complexity . And a I w think the data would be nicer if we get temporal references .
Professor D: Well , so Keith , what do you think ?
Grad B: Well , um it seems that Yeah , I mean , off the top of my head it kinda seems like you would probably just want , you know , richer data , more complex stuff going on , people trying to do more complex sets of things . I mean you know , if our goal is to really sort of be able to handle a whole bunch of different stuff , then throwing harder situations at people will get them to do more linguistic more interesting linguistic stuff . But I mean I 'm I 'm not really sure Uh , because I don't fully understand like what our choices are of ways to do this here yet .
Grad C: I mean w we have tested this and a y have you heard listen to the f first two or th as a matter of fact the second person is uh is was faced with exactly this kind of setup .
Grad B: I started to listen to one and it was just like , um , uh , sort of depressing .
Grad C: And Grad B: I thought I 'd just sort of listen to the beginning part and the person was just sort of reading off her script or something . And .
Grad C: Oh , OK . That was the first subject .
Professor D: Yeah .
Grad B: Yeah .
Professor D: First one wasn't very good .
Grad B: Yeah .
Grad C: Yeah .
Grad B: So um , I Grad C: Um , it is already with this it got pretty with this setup and that particular subject it got pretty complex .
Undergrad E: Although Grad B: Mm - hmm .
Grad C: Maybe I suggest we make some fine tuning of these , get sort of run through ten or so subjects
Grad B: Mm - hmm .
Grad C: and then take a breather , and see whether we wanna make it more complex or not , depending on what what sort of results we 're getting .
Grad B: Right . Yeah . It In fact , um , I am just you know today , next couple days gonna start really diving into this data . I 've basically looked at one of the files you know one of these l y y y you gave me those dozens of files and I looked at one of them which was about ten sentences , found fifteen , twenty different construction types that we would have to look for and so on and like , " alright , well , let 's start here . " Um . So I haven't really gone into the , you know looked at all of the stuff that 's going on . So I don't really Right , I mean , once I start doing that I 'll have more to say about this kind of thing .
Professor D: OK .
Grad C: And y and always Professor D: But well th but you did say something important , which is that um you can probably keep yourself fairly well occupied uh with the simple cases for quite a while .
Grad B: Yeah .
Professor D: Although , obviously th so so that sa s does suggest that Uh , now , I have looked at all the data , and it 's pre it 's actually at least to an amateur , quite redundant .
Grad B: Yeah , Yeah .
Professor D: That that it was it was very stylized , and quite a lot of people said more or less the same thing .
Grad B: I um I did sort of scan it at first and noticed that , and then looked in detail at one of them .
Professor D: Yeah .
Grad B: But yeah , yeah I noticed that , too .
Professor D: So , we we we wanna do more than that .
Grad C: And with this we 're getting more . No question .
Professor D: OK . Right . So Grad C: uh w do we wanna get going beyond more , which is sort of the Professor D: Well , OK , so let 's let 's take let 's I I think your suggestion is good , which is we 'll do a b uh a batch . OK . And , uh , Fey , How long is it gonna be till you have ten subjects ? Couple days ? Or thr f a A week ? Or I don't I don't have a feel for th
Undergrad E: Um I can Yeah , I mean I s I think can probably schedule ten people , uh , whenever .
Professor D: Well , it 's it 's up to you , I mean I j I uh e We don't have any huge time pressure . It 's just when you have t
Undergrad E: How long will it be ?
Professor D: Yeah .
Undergrad E: Um I I would say maybe two weeks .
Professor D: Oh , OK . So let 's do this . Let 's plan next Monday , OK , to have a review of what we have so far .
Grad C: This means audio , but Professor D: and Huh ?
Grad C: no transcriptions of course , yeah .
Professor D: No , we won't have the transcriptions , but what we should be able to do and I don't know if , Fey , if you will have time to do this , but it would be great if you could , um , not transcribe it all , but pick out uh , some stuff . I mean we could lis uh just sit here and listen to it all . Are you gonna have the audio on the web site ? OK .
Grad C: Until we reach the gigabyte thing and David Johnson s ki kills me . And we 're gonna put it on the web site . Yeah .
Professor D: Oh , we could get I mean , you can buy another disk for two hundred dollars , right ? I mean it 's it 's not like OK . So , we 'll take care of David Johnson .
Grad C: No , he uh , he he has been solving all our problems or is wonderful ,
Professor D: OK .
Undergrad E: Take care of him .
Professor D: OK .
Grad C: so s
Professor D: Alright . So we 'll buy a disk . But anyway , so , um , If you if you can think of a way to uh , point us to th to interesting things , sort of as you 're doing this or or something uh , make your make notes or something that that this is , you know , something worth looking at . And other than that , yeah I guess we 'll just have to uh , listen although I guess it 's only ten minutes each , right ? Roughly .
Undergrad E: Well , I guess . I 'm not sure how long it 's actually going to take .
Grad C: The reading task is a lot shorter . That was cut by fifty percent . And the reading , nobody 's interested in that except for the speech people .
Professor D: Right . No , we don't care about that at all .
Grad C: So . It 's actually like five minutes dialogue .
Professor D: I b My guess is it 's gonna be ten .
Grad C: Ten minutes is long .
Professor D: People I understand , but people people you know uh Undergrad E: It feels like a long time
Grad C: Yeah .
Undergrad E: but .
Grad C: It feels like forever when you 're doing it ,
Professor D: Yeah .
Grad C: but then it turns out to be three minutes and forty five seconds .
Professor D: Yeah .
Grad B: Yeah .
Professor D: Could be . OK . I was thinking people would , you know , hesitate and Whatever . Whatever it is we 'll we 'll deal with it .
Grad C: Yeah , it 's not And it 's fun .
Professor D: OK , so that 'll be that 'll be um on on the web page .
Grad C: OK .
Professor D: That 's great . Um But anyway yeah , so I think it 's a good idea to start with the sort of relatively straight forward res just response system . And then if we want to uh get them to start doing uh multiple step planning with a whole bunch of things and then organize them an um tell them which things are near each other and you know , any of that stuff . uh You know , " Which things would you like to do Tuesday morning ? "
Grad C: Yeah .
Professor D: So yeah I th that seems pretty straight forward .
Undergrad E: But were you saying that Grad C: I need those back by the way .
Grad B: OK .
Professor D: OK .
Undergrad E: Yeah .
Grad C: That 's for Professor D: I 'm sorry , Fey , what ?
Undergrad E: That w maybe one thing we should do is go through this list and sort of select things that are categories and then o offer only one member of that category ?
Professor D: That 's what I was suggesting for the first round , yeah .
Undergrad E: OK .
Grad B: So rather than having zoo and castle .
Undergrad E: And then , I mean , they could be alternate versions of the same If you wanted data on different constructions .
Professor D: They could , but i but i uh tha eh they c yeah , but uh but Undergrad E: Like one person gets the version with the zoo as a choice , and the other person gets the Professor D: You could , but i but I I I think in the short run , Grad C: And no , th the per the person don't get it . I mean , this is why we did it , because when we gave them just three tasks for w part - A and three tasks for part - B a
Professor D: Right . Yeah .
Undergrad E: Well no , they could still choose . They just wouldn't be able to choose both zoo and say , touring the castle .
Grad C: Exactly . This is limiting the choices , but yeah . Right . OK , sorry . But um I I think this approach will very well work , but the person was able to look at it and say " OK , This is what I would actually do . "
Undergrad E: Yeah .
Grad C: Yeah .
Professor D: OK .
Grad C: OK .
Undergrad E: He was vicious .
Grad C: OK , we gotta we gotta disallow uh traveling to zoos and uh castles at the same time , sort of Undergrad E: I mean there they are significantly different , but .
Grad C: But no , they 're I mean they 're sort of this is where tour becomes you know tourists maybe a bit different
Undergrad E: Yeah , I guess so .
Grad C: and , um , these are just places where you you enter um , much like here .
Professor D: Yeah .
Grad C: But we can uh Professor D: Yeah , in fact if y if y if you use the right verb for each in common , like at you know , " attend a theater , symphony or opera " is is a group , and " tour the university , castle or zoo " ,
Grad C: mm - hmm Yeah .
Professor D: all of these d do have this kind of " tour " um aspect about the way you would go to them . And uh , the movie theater is probably also uh e is a " attend " et cetera .
Grad C: Attend , yeah .
Professor D: So it may turn out to be not so many different kinds of things ,
Grad C: Hmm , mm - hmm .
Professor D: and then , what one would expect is that that the sentence types would uh their responses would tend to be grouped according to the kind of activity , you would expect .
Grad B: Mm - hmm .
PhD F: But I mean i it seem that um there is a difference between going to see something , and things like " exchange money " or " dine out "
Professor D: Oh , absolutely . Yeah .
PhD F: uh @ @ function , yeah .
Grad C: Yeah , this is where yeah th the function stuff is definitely different and the getting information or g stuff yeah . OK . But this is open . So since people gonna still pick something , we we 're not gonna get any significant amount of redundancy . And for reasons , we don't want it , really , in that sense . And um we would be ultimately more interested in getting all the possible ways of people asking , oh , for different things with or with a computer . And so if you can think of any other sort of high level tasks a tourist may do just always just m mail them to us and we 'll sneak them into the collection . We 're not gonna do much statistical stuff with it .
Professor D: We don't have enough .
Grad C: No . But it seems like since we since we are getting towards uh subject uh fifty subjects and if we can keep it up um to a uh sort of five four - ish per week rate , we may even reach the one hundred before Fey t takes off to Chicago .