On the Impact of Reconstruction and Context for Argument Prediction in Natural Debate (2023)

Abstract: Debate naturalness ranges on a scale from small, highly structured, and topically focused settings to larger, more spontaneous and less constrained environments. The more unconstrained a debate, the more spontaneous speakers act: they build on contextual knowledge and use anaphora or ellipses to construct their arguments. They also use rhetorical devices such as questions and imperatives to support or attack claims. In this paper, we study how the reconstruction of the actual debate contributions, i.e., utterances which contain pronouns, ellipses and fuzzy language, into full-fledged propositions which are interpretable without context impacts the prediction of argument relations and investigate the effect of incorporating contextual information for the task. We work with highly complex spontaneous debates with more than 10 speakers on a wide variety of topics. We find that in contrast to our initial hypothesis, reconstruction does not improve predictions and context only improves them when used in combination with propositions.

## introduction spontaneous natural debate is anything but easy to track: it contains anaphora, elliptical constructions, fragments, a fuzzy linguistic surface and a wide variety of rhetorical structures. the waters get even murkier when 10+ speakers contribute, multiple, possibly divergent, topics are covered in one debate, the stakes of the interlocutors are high, and debate constraints are low. so far, debates at this scale of naturalness have been largely ignored in computational argumentation: either the number of participants was restricted and debates were highly constrained @xcite , there was only one topic per debate @xcite , or the setting was structured and consisted of monological speaker utterances @xcite . most striking is the difference in the underlying data: argument mining on natural debate has either taken propositions as argumentative units of analysis, i.e., fully reconstructed records of speaker contributions that do not need context for interpretation @xcite or like @xcite removed claims that contain, for instance, unresolved demonstratives. another common approach, however, is to take transcripts as is, without any edits or restrictions @xcite . our hypothesis is that using fully reconstructed material, i.e., propositions, increases the performance of argument relation prediction. in the case of locutions, where for instance anaphora and ellipses are not reconstructed, we assume that some of the information relevant to reconstruction is contained within the preceding context, like in an example from the corpus foot_0 where an anaphoric pronoun 'she' from the locution 'she's looking at what happened' can be resolved as 'sue grey' in a proposition 'sue gray is looking at what happened' using preceding context 'sue grey is doing this investigation'. however, the task of completely reconstructing propositions from locutions, i.e., the actual, skeletal contributions in the debate, is costly: manual reconstruction requires an extensive amount of effort @xcite , while automatic approach struggles with unresolved non-personal anaphora and omitted verb phrases @xcite . our contributions in this paper are as follows: (1) we provide more insight into model performance in a realistic debate mining setting where only skeletal locutions and not fully reconstructed propositions are available, using the best-performing model on a dataset that is closest in nature to the debates here. our results indicate that despite the notable structural differences between locutions and propositions, we achieve comparable performance in argument relation prediction for both. (2) we perform a detailed error analysis and show that performance across argument relations varies noticeably and that context does not help in solving the issue of using only skeletal locutions but improves the predictions when used in combination with propositions. ## related work in computational argumentation in the debate genre, one strand of research focuses on mostly monological speech, either produced by professional debaters @xcite @xcite or by political actors @xcite . a slightly different variety of debate concerns heavily structured oxford-style debates @xcite where conversational flow is important. in the case of more natural but still highly moderated debates, the focus is mostly on the political genre with some work on the identification of the central and divisive elements of the debate @xcite and prediction of the argument relations using support and attack annotation scheme @xcite as well as more complex categories @xcite . there is more research on the us 2016 elections @xcite as well as the uk prime-ministerial elections from 2015 @xcite with both papers focusing on the detection of argument components such as claims and premises/evidence. in terms of segmentation, we are similar to most other work in debate mining: @xcite and @xcite also assume sentential (or potentially sub-sentential) segments between which argument relations can hold, in contrast to @xcite who seem to take utterances to be the minimal units of analysis. given the significant amount of argument relations within one utterance, we are confident that the former approach is what captures this genre most appropriately. ## error analysis context only helps sometimes a closer inspection of the results (for details see appendix a, when we compare the results for propositions with and without context (for details see appendix a,