Goal-driven Answers in the Cards Dialogue Corpus

The resolvedness conditions for questions are not semantically invariant, but rather heavily influenced by the goals and intentions of the discourse participants. One of the chief innovations of recent questionand preference-driven models of pragmatics is to characterize how these high-level contextual features influence language production and comprehension. The goal of this paper is to develop and motivate general techniques for quantitatively and qualitatively assessing such models. The basis for my exploration is the Cards corpus of task-oriented dialogues, a highly structured resource that allows us not only to rigorously interpret the discourse participants’ utterances but also to track their evolving goals and subgoals. The paper’s central experiments provide evidence that the semantic variability of answers to Where are you? is governed by the particular demands of the goal that the question engages. © 2012 Christopher Potts Cascadilla Proceedings Project Completed July 10, 2012 Proceedings of the 30th West Coast Conference on Formal Linguistics, ed. Nathan Arnett and Ryan Bennett, 1-20. Somerville, MA: Cascadilla Proceedings Project. Goal-driven Answers in the Cards Dialogue Corpus Christopher Potts Stanford University 1. Resolvedness and the task The starting point for this paper is Ginzburg’s (1995a) observation that the resolvedness conditions for questions are not semantically invariant, but rather “fixed in a particular context to a level identified by the goal” of the conversation (p. 466; see also Clark 1979; Clark & Schober 1992; Perrault & Allen 1980; Groenendijk & Stokhof 1984:II; Gibbs & Bryant 2007). Such goal-orientation is especially clear for questions like Where are you? , in which the appropriate level of granularity and sense of location are so highly variable; each of my office, California, the U.S., Linguistics, and still working on the introduction is resolving in some contexts but non-resolving (even downright unhelpful) in others. One of the chief innovations of models like Ginzburg’s is to bring these goals (issues, plans, decision problems) into the pragmatic model, so that they can play a direct role in resolving underspecification, setting bounds on vagueness, and guiding inferences about the discourse participants’ intentions. This is a leading idea of all the recent task-driven and question-driven models of contextual dynamics (Allen, 1991; Hobbs et al., 1993; Roberts, 1996, 2004; Ginzburg, 1995b, 1996; Groenendijk, 1999; Beaver, 2002; Büring, 2003; Stone et al., 2007; Beaver & Clark, 2008; Groenendijk & Roelofsen, 2009) as well as their game-theoretic and decision-theoretic counterparts (Clark, 1996; Merin, 1997, 1999; Parikh, 2000, 2001; van Rooy, 2003; Benz, 2005; Franke, 2009; Jäger, To appear; Frank & Goodman, 2012). The goal of this paper is to develop and motivate general techniques for identifying and characterizing this kind of task dependence in real task-oriented dialogue, where the phenomena can be both quantitatively and qualitatively assessed. The basis for my exploration is the Cards corpus, a large collection of dialogues derived from a two-person collaborative game. The most noteworthy feature of this corpus for present purposes is that its transcripts record not just the dialogue exchanged by the players, but also all their actions in the game world, with high enough fidelity that we can faithfully replay the games in their entirety. Of course, we don’t have access to the players’ beliefs and intentions, but we do have enough linguistic and behavioral cues to make confident inferences about them. My focus is on the corpus’s 800+ answers to the (implicit or explicit) questionWhere are you? Even in the highly constrained Cards world, these answers come at numerous levels of granularity, frommiddle of the board to 2 spaces to the left of the gap underneath the entrance to the middle room. Intuitively, the preferred granularity is governed by the players’ subgoal at that point in the game. For example, where they want to divide up the board for general exploration, very general answers are preferred. When they need to meet or find specific things, only highly specific answers are resolving. After describing the Cards corpus in some detail (sec. 2), I look more closely at the nature of these locative answers (sec. 3), and then begin the process of setting up an experiment to test the hypothesis of goal-orientation. This involves some semantic annotation (sec. 4) and, more interestingly, a technique for interpreting these annotations in the Cards world itself (sec. 5). With these preliminaries in place, it is straightforward to define a precise notion of granularity and show that it correlates with the players’ subgoals (sec. 6). The paper closes with a more precise theoretical explanation for this correlation (sec. 7) and some suggestions for additional experiments along these lines that could be conducted fairly easily ⇤ Thanks to Pranav Anand, Mike Frank, Noah Goodman, Jesse Harris, Dan Lassiter, Sven Lauer, Adam Vogel, and audiences at WCCFL 30, NYU, and the University of Rochester for valuable comments, suggestions, and discussion. Thanks also to Alex Djalali, Sven Lauer, Karl Schultz, and the whole SUBTLE team for help with corpus collection. This research was supported in part by ONR grant No. N00014-10-1-0109 and ARO grant No. W911NF-07-1-0216. © 2012 Christopher Potts Cascadilla Proceedings Project Completed July 10, 2012 You are on 2D Yellow boxes mark cards in your line of sight. Task description: Six consecutive cards of the same suit TYPE HERE The cards you are holding Move with the arrow keys or these buttons. Figure 1: An annotated version of the Cards gameboard. with the Cards corpus (sec. 8). The corpus is available at http://cardscorpus.christopherpotts. net/. The distribution includes the transcripts and starter code for working with them in Python and R, and the site provides a search and visualization function. I have also posted the annotations and R functions for extracting and visualizing the denotations used in this paper, and for running the experiments of sec. 6.