An Empirical Investigation of Proposals in Collaborative Dialogues

We describe a corpus-based investigation of proposals in dialogue. First, we describe our DRI compliant coding scheme and report our inter-coder reliability results. Next, we test several hypotheses about what constitutes a well-formed proposal. 1 I n t r o d u c t i o n Our project's long-range goal (see http://www.isp. p i t t .edu/in tgen/) is to create a unified architecture for collaborative discourse, accommodating both interpretation and generation. Our computational approach (Thomason and Hobbs, 1997) uses a form of weighted abduction as the reasoning mechanism (Hobbs et al., 1993) and modal operators to model context. In this paper, we describe the corpus study portion of our project, which is an integral part of our investigation into recognizing how conversational participants coordinate agreement. From our first annotation trials, we found that the recognition of "classical" speech acts (Austin, 1962; Searle, 1975) by coders is fairly reliable, while recognizing contextual relationships (e.g., whether an utterance accepts a proposal) is not as reliable. Thus, we explore other features that can help us recognize how participants coordinate agreement. Our corpus study also provides a preliminary assessment of the Discourse Resource Initiative (DRI) tagging scheme. The DRI is an international "grassroots" effort that seeks to share corpora that have been tagged with the core features of interest to the discourse community. In order to use the core scheme, it is anticipated that each group will need to refine it for their particular purposes. A usable draft core scheme is now available for experimentation (see http://www.georgetown.edu/luperfoy/DiscourseTreebank/dri-home.html). Whereas several groups are working with the unadapted core DRI scheme (Core and Allen, 1997; Poesio and Traum, 1997), we have at tempted to adapt it to our corpus and particular research questions. First we describe our corpus, and the issue of tracking agreement. Next we describe our coding scheme and our intercoder reliability outcomes. Last we report our findings ,on tracking agreement. 2 T r a c k i n g A g r e e m e n t Our corpus consists of 24 computer-mediated dialogues 1 in which two participants collaborate on a simple task of buying furniture for the living and dining rooms of a house (a variant of the task in (Walker, 1993)). The participants' main goal is to negotiate purchases; the items of highest priority are a sofa for the living room and a table and four chairs for the dining room. The problem solving task is complicated by several secondary goals: 1) Match colors within a room, 2) Buy as much furniture as you can, 3) Spend all your money. A point system is used to motivate participants to try to achieve as many goals as possible. Each subject has a budget and inventory of furniture that lists the quantities, colors, and prices for each available item. By sharing this initially private information, the participants can combine budgets and select furniture from either's inventory. The problem is collaborative in that all decisions have to be consensual; funds are shared and purchasing decisions are joint. In this context, we characterize an agreement as accepting a partner's suggestion to include a specific furniture item in the solution. In this paper we will focus on the issue of recognizing that a suggestion has been made (i.e. a proposal). The problem is not easy, since, as speech act theory points out (Austin, 1962; Searle, 1975), surface form is not a clear indicator of speaker intentions. Consider excerpt (1): 2 (I) A: [35]: i have a blue sofa for 300. [36]: it's my cheapest one. B: [37]: I have I sofa for 350 [38]: that is yellow [39]: which is my cheapest, [40]: yours sounds good. [35] is the first mention of a sofa in the conversa1participants work in separate rooms and communicate via the computer interface. The interface prevents interruptions. 2We broke the dialogues into utterances, partly following the algorithm in (Passonneau, 1994).