The Keystone Role Played by Questions in Debate (2022)

Abstract: Building on the recent results of a study into the roles that are played by questions in argumentative dialogue (Hautli-Janisz et al.,2022a), we expand the analysis to investigate a newly released corpus that constitutes the largest extant corpus of closely annotated debate. Questions play a critical role in driving dialogical discourse forward; in combative or critical discursive environments, they not only provide a range of discourse management techniques, they also scaffold the semantic structure of the positions that interlocutors develop. The boundaries, however, between providing substantive answers to questions, merely responding to questions, and evading questions entirely, are fuzzy and the way in which answers, responses and evasions affect the subsequent development of dialogue and argumentation structure are poorly understood. In this paper, we explore how questions have ramifications on the large-scale structure of a debate using as our substrate the BBC television programme Question Time, the foremost topical debate show in the UK. Analysis of the data demonstrates not only that questioning plays a particularly prominent role in such debate, but also that its repercussions can reverberate through a discourse.

## introduction whether employed to garner votes, determine policy, air grievances or test theories, debates rely upon questioning; questions provide the driving force, a rhythmic sparking of the engine of debate. they do not merely provide impetus for a moment or a turn, however: they can have far-reaching ramifications, scaffolding large structures of subsequent discursive interaction. we intuitively recognise this keystone role of questions, when, for example, we criticise politicians for evading questions, or are frustrated when our own are ignored. (the us 2016 presidential debates offer a good example, wherein clinton was roundly criticised by the press at both ends of the political spectrum for failing to give direct answers to the questions she was asked). our intuitions, however, have heretofore not been matched by a theoretical understanding. how do different types of questions lead to different interactional and argumentative structures? what patterns of ramifications do different classes of reactions to those questions have? how can questions be used strategically to open up and limit the space of information and outcome? for the purpose of exploring these questions in naturally occurring argumentation at scale, we investigate a new dataset, qt30 @xcite , comprising analysis of thirty episodes of broadcast topical debate from the uk bbc television programme question time. this is three times the size of the previously largest dataset and offers a rich environment in which to test the theories from @xcite . by comparing the analysis of qt30 with that of the earlier datasets, we can establish a robust grounding for our conclusions about the role that information-seeking, assertive and rhetorical questions play across domains of argumentation. we also show that the response space across these three question types remains consistent, i.e., whether questions are ignored, reacted to, responded to, or in fact answered. ## related work categorising the function of questions has been a notoriously tricky issue, not only in theoretical but also in computational approaches to discourse. the field is therefore confronted with "a trade-off between usefulness and ease or consistency of coding" @xcite . the majority of approaches represent and model informationseeking (or so-called 'canonical') questions, as for instance in friendsqa @xcite . in the maptask coding scheme @xcite , rhetorical questions do not form a discrete category. the switchboard corpus @xcite en-codes information-seeking questions according to their syntactic property (e.g. yes-no-questions, whquestions), non-information seeking questions are summed up in the category 'rhetorical questions'. @xcite propose seven different 'social actions' of questions, among them 'request for information', 'outloud' ("delivered to no one in particular"), and 'rhetorical question' ("questions that may seek a response but do not seek an answer"). in terms of characterising the response space, @xcite use the four categories of 'non-response', 'non-answer response', 'answer', and 'can't determine' (p. 2624) for spontaneous, naturally occurring conversation, showing some overlap with the six-way distinction of @xcite with 'possible answers', 'indirect answers', 'confessions of ignorance', 'clarification requests', 'evasive replies' and 'miscellaneous' (targeted at nursery school child conversation). @xcite propose a taxonomy for query responses, bearing some resemblance to @xcite . for the task-based interactions in maptask, the bee corpora and informal conversations in the bnc corpus, @xcite propose a two-way distinction of responses into 'answers' and 'nonanswers', with more fine-grained categories subsumed under them. our goal in this paper is to combine the strengths of these approaches whilst maintaining a simplicity in taxonomy that enables us to focus specifically upon the impact of questions in argumentative discourse. for our investigation, we pair structures of question-answering with an analysis of how argumentation is invoked or 'anchored' in dialogue. it is only in this combination that we are able to derive insights into how these two realms of structures impact surrounding discourse. the only framework that allows us to investigate both realms of discourse in parallel is inference anchoring theory (iat) @xcite , a pragmatically-driven account of dialogical argumentation. ## inference anchoring theory and the question/response space the central motivation of inference anchoring theory (iat) @xcite is to provide a theoretical framework for analysing dialogical argumentation. as such, iat sets out to answer the question of where argumentation comes from in dialogical interaction and acts as a theory-neutral scaffolding that integrates different communicative structures, namely dialogue structure, argument structure (including inference understood in the logical manner) and illocutionary forces such as asserting, suggesting or promising, to anchor argument structure in dialogue structure. one of the primary loci of argumentation is precisely in the asking of, and responding to, questions of particular sorts, and it is upon these questions that we focus here, rather than questions of clarification @xcite ; of recovery from nonunderstanding @xcite ; or in many cases of personal information @xcite , inter alia. an example of how questions can lead to argumentative structure is shown in (1) in which question time moderator fiona bruce uses questions as a tool that allows her to forward and control the debate: a. fiona bruce [00:48:45] [...] nadhim, should the uk be doing more to help find a long lasting solution? because who knows how long this particular ceasefire will last. b. nadhim zahawi [00:49:05] the simple answer is yes, and we have been doing. the foreign secretary dominic raab and the us secretary of state antony blinken have worked tirelessly, one, to deescalate the situation. . . obviously president biden has made a huge difference, as has our prime minister to urge both sides. . . c. fiona bruce [00:49:30] but there is no peace process at the moment. is there anything the uk can do with the international community to try and foster a situation where that can start again? in (1-a) fiona bruce asks one of the panel members a question about a long-lasting resolution of the israeli-palestinian conflict. she then supports her question by expressing her concerns regarding the current ceasefire ('because who knows how long this particular ceasefire will last'). panel member nadhim zahawi answers fiona bruce's question directly with 'yes' in (1-b) and further elaborates his answer with 'we have been doing [more]'. he then proceeds to support his claim by mentioning previous work on the de-escalation of the conflict by foreign secretary dominic raab and the us secretary of state antony blinken as well as president biden and the uk prime minister. however, by mentioning the past work of the uk and us politicians, he avoids going into detail regarding the future steps of the uk in the conflict. in (1 -c ) fiona bruce picks up on that and steers the conversation back to the current israeli-palestinian conflict by stating that 'there is no peace process at the moment'. she then uses this as a reason to ask nadhim zahawi another and more specific question on whether the uk can play a role in restarting the peace talks, pressing him to give a more relevant answer. this second question leads to further discussion, which is not illustrated in this example. in the following we go through the components of analysis that are needed to investigate the role of questions in argumentative dialogue. ## data the combined dataset underlying our investigation consists of thirty six corpora in four argumentative genres: election debate, moral debate, public deliberation and topical debate. all corpora are annotated with full iat. in total, our dataset consists of 414,726 words (tokens), and 51,993 illocutionary structures. for the 'election debate' genre, we use the us2016tv foot_1 corpus @xcite , which includes the transcripts of all televised debates in the us 2016 presidential election. the 'election debate' dataset has 39,694 words, and 6,570 illocutionary structures and an inter-annotator agreement of cohen's κ = 0.61 (substantial agreement) @xcite . for the genre 'moral debate', we include two sub-corpora. mm2012 foot_2 , a collection of transcripts of bbc radio 4's 'the moral maze', a series of debates about moral dilemmas and bbc2017 foot_3 , which consists of a special issue of 'the moral maze' and a related tv debate about the morality of abortion.the 'moral debate' dataset has 60,273 words, and 5,488 illocutionary structures. for 'public deliberation', we include three subcorpora. uscd2011 foot_4 consists of several transcripts of public meetings and hearings from departments of transportation (dots) across the usa @xcite , @xcite . dedd2019 foot_5 is a set of two experimental 4-party deliberations on whether or not to allow fracking in germany. ukdd2019 foot_6 is a public deliberation organised by the royal society of edinburgh in the context of an inquiry about the future of energy in scotland. the 'public deliberation' dataset has 34,759 words and 3,931 illocutionary structures. the corpora in the genres 'public deliberation', 'moral debate' and 'election debate' were annotated by small teams of two to ten analysts. these three genres provide background and context for our main study which focuses upon the genre of 'topical debate', for which we use the qt30 corpus (hautli-janisz et al., 2022b). 8 this unique dataset includes transcripts of 30 episodes aired between june 2020 and november 2021 of bbc's question time, one of the most viewed political talk shows in the uk. the corpus consists of more than 29 hours of transcribed broadcast debate and comprises 280,000 words and 36,004 illocutionary structures by more than 400 participants, making it the largest corpus of analysed broadcast topical debate published to date. the annotation was conducted by 38 students of linguistics, philosophy, literature and computer science across europe. inter-annotator agreement for qt30 is at a combined argument similarity score (cass) @xcite of 0.56, signaling moderate agreement. ## conclusion our analysis here extends upon and refines the findings reported in hautli-janisz et al. (2022a), which illustrates the strategic role questions play in formulating various argumentative structures across genres of debate. for the first time, we are able to inspect the role of questions at scale, in the largest annotated corpus of dialogical argumentation, qt30 @xcite . the results reported here scale up previous work in two distinct ways. first of all, there is explicit annotation of relationships diachronically through each hour, allowing tracking of ramifications and consequences of questions long after they have been uttered. secondly, we also scale up the robustness of analysis by considering more debates -thereby covering more speakers (so we are at less risk of merely picking up on speaker idiosyncrasy), and more themes (so we are less at risk of merely picking up on thematic specificity). some of the observations presented in the analysis are to be expected -that pure questions elicit more answers and that rhetorical questions are usually answered by the same speaker. some are a little less straightforward -that assertive questions and rhetorical questions often present material that is used to support further argumentation, whereas the contents of pure questions typically end up being the conclusions of arguments. that assertive and rhetorical questions are used to provide an evidential basis for further reasoning further supports the hypothesis that such illocutions provide a rhetorical means to lower the interpersonal cost of making an assertion -a gambit to reduce dialogical risk. sometimes, though, this gambit fails, as suggested by the fact that assertive questions in particular very often lie at the heart of conflict. the most striking new insight from this analysis is that across both inferential and conflict structures by which argumentation is driven forward, almost a quarter are triggered, either immediately or eventually, by questioning. questions are important to debate and reasoning: from journalistic punditry, from personal experience, from scholarly study across philosophy of science to rhetoric, from practical political training; wherever humans debate, the role of questions and question-answering is placed front and centre. this work, however, is the first time that it has become possible to quantify that role, and pin down exactly how it is, and to what extent, questions are keystone in the development of reasoning in debate.