V ives D ebate-Speech: A Corpus of Spoken Argumentation to Leverage Audio Features for Argument Mining (2023)

Abstract: In this paper, we describe VivesDebate-Speech, a corpus of spoken argumentation created to leverage audio features for argument mining tasks. The creation of this corpus represents an important contribution to the intersection of speech processing and argument mining communities, and one of the most complete publicly available resources in this topic. Moreover, we have performed a set of first-of-their-kind experiments which show an improvement when integrating audio features into the argument mining pipeline. The provided results can be used as a baseline for future research.

## introduction the automatic analysis of argumentation in human debates is a complex problem that encompasses different challenges such as mining, computationally representing, or automatically assessing natural language arguments. furthermore, human argumentation is present in different mediums and domains such as argumentative monologues (e.g., essays) and dialogues (e.g., debates), argumentation in text (e.g., opinion pieces or social network discussions) and speech (e.g., debate tournaments or political speeches), and domains such as the political @xcite , legal @xcite , financial @xcite , or scientific @xcite among others. thus, human argumentation presents a linguistically heterogeneous nature that requires us to carefully investigate and analyse all these variables in order to propose and develop argumentation systems which are robust to these variations in language. in addition to this heterogeneity, it is worth mentioning that a vast majority of the publicly available resources for argumentation-based natural language processing (nlp) have been created considering text features only, even if their original source comes from speech @xcite . this is a substantial limitation, not only for our knowledge on the impact that speech may directly have when approaching argument-based nlp tasks, but because of the significant loss of information that happens when we only take into account the text transcript of spoken argumentation. in this work, we will focus on the initial steps of argument analysis considering acoustic features, namely, the automatic identification of natural language arguments. argument mining is the area of research that studies this first step in the analysis of natural language argumentative discourses, and it is defined as the task of automatically identifying arguments and their structures from natural language inputs. as surveyed in @xcite , argument mining can be divided into three main sub-tasks: first, the segmentation of natural language spans relevant for argumentative reasoning (typically defined as argumentative discourse units adus @xcite ); second, the classification of these units into finer-grained argumentative classes (e.g., major claims, claims, or premises @xcite ); and third, the identification of argumentative structures and relations existing between these units (e.g., inference, conflict, or rephrase @xcite ). therefore, our contribution is twofold. first, we create a new publicly available resource for argument mining research that enables the use of audio features for argumentative purposes. second, we present first-of-their-kind experiments showing that the use of acoustic information improves the performance of segmenting adus from natural language inputs (both audio and text). ## the vivesdebate-speech corpus the first step in our research was the creation of a new natural language argumentative corpus. in this work, we present vivesdebate-speech, an argumentative corpus created to leverage audio features for 2071 argument mining tasks. the vivesdebate-speech has been created taking the previously annotated vivesdebate corpus @xcite as a starting point. the vivesdebate corpus contains 29 professional debates in catalan, where each debate has been comprehensively annotated. this way, it is possible to capture longer-range dependencies between natural language adus, and to keep the chronological order of the complete debate. although the nature of the debates was speech-based argumentation, the vivesdebate corpus was published considering only the textual features included in the transcriptions of the debates that were used during the annotation process. in this paper, we have extended the vivesdebate corpus with its corresponding argumentative speeches in audio format. in addition to the speech features, we also created and released the bio (i.e., beginning, inside, outside) files for approaching the task of automatically identifying adus from natural language inputs (i.e., both textual and speech). the bio files allow us to determine whether a word is the beginning, it belongs inside, or it is outside an adu. the vivesdebate-speech corpus is, to the best of our knowledge, the largest publicly available resource for audio-based argument mining. furthermore, combined with the original vivesdebate corpus, a wider range of nlp tasks can be approached taking the new audio features into consideration (e.g., argument evaluation or argument summarisation) in addition to the standard argument mining tasks such as adu segmentation, argument classification, and the identification of argumentative relations. compared to the size of the few previously available audio-based argumentative corpora @xcite (i.e., 2 and 7 hours respectively), the vivesdebate-speech represents a significant leap forward (i.e., more than 12 hours) for the research community not just in size (see ## problem description we approached the identification of natural language adus in two different ways: (i) as a token classification problem, and (ii) as a sequence classification problem. for the first approach, we analyse our information at the token level. each token is assigned a bio label and we model the probabilities of a token belonging to each of these specific label considering the n-length closest natural language contextual tokens. for the second approach, the information is analysed at the sentence level. in order to address the adu identification task as a sequence classification problem we need to have a set of previously segmented natural language sequences. then, the problem is approached as a 2-class classification task, discriminating argumentative relevant from non-argumentative natural language sequences. ## proposed method the use of audio information for argument mining presents significant advantages across 3 axes: efficiency, information and error propagation. firstly, the segmentation of the raw audio into independent units is a pre-requisite for most automatic speech recognition (asr) system. if the segmentation produced in the asr step is incorporated into the argument mining pipeline, we remove the need for a specific text-segmentation step, which brings significant computational and complexity savings. secondly, the use of audio features allows us to take into account relevant prosodic features such as intonation and pauses which are critical for discourse segmentation, but that are missing from a text-only representation. lastly, the use of asr transcriptions introduces noise into the pipeline as a result of recognition errors, which can hamper downstream performance. working directly with the speech signal allows us to avoid this source of error propagation. two different methods to leverage audio features for argument mining are explored in this paper. first, a standard end-to-end (e2e) approach that takes the text-based transcription of the spoken debate produced by the asr as an input, and directly outputs the segmentation of this text into argumentative units. second, we propose a cascaded model composed of two sub-tasks: argument segmentation and argument classification. in the first sub-task, the discourse is segmented into independent units, and then for each unit it is determined if it contains argumentative information or not. both approaches produce an equivalent output, a sequence of bio tags which is then compared against the reference. this work investigates how audio features can be best incorporated into the previously described process. an overview of the proposed cascaded method is shown in figure 1 . as we can observe, a segmentation step follows the asr step, which segments either the whole audio (a-seg) or the whole text (t-seg) into (potential) argumentative segments. a classification step then detects if the segment contains an argumentative unit, by using either the audio (a-clf) or the text (t-clf) contained in each segment. if efficiency is a major concern, the segmentation produced by the asr step can be re-used instead of an specific segmentation step tailored for argumentation, but this could decrease the quality of the results. this process significantly differs from the e2e approach where the bio tags are directly generated from the output of the asr step. this way, our cascaded model is interesting because it makes possible to analyse different combinations of audio and text features. the cascaded method has one significant advantage, which is that audio segmentation is a widely studied problem in asr and speech translation (st) for which significant breakthroughs have been achieved in the last few years. currently, one of the best performing audio segmentation methods is shas @xcite , which uses a probabilistic divide and conquer (dac) algorithm to obtain optimal segments. furthermore, we have compared shas with a voice activity detection (vad) baseline, as well as with the nonprobabilistic vad method @xcite using a wav2vec2 pause predictor @xcite , which performs asr inference and then splits based on detected word boundaries (see appendix a.1). to complete our proposal, we have also explored text-only segmentation methods in which a transformer-based model is trained to detect boundaries between natural language segments. this way, each word can belong to two different classes, boundary or not boundary. the second stage of our cascaded method is an argument detection classifier, that decides, for each segment, if it includes argumentative content and should be kept, or be discarded otherwise. in the case that our classifier detects argumentative content within a segment, its first word is assigned the b label (i.e., begin) and the rest of its words are assigned the i label (i.e., inside). differently, if the classifier does not detect argumentative content within a segment, all the words belonging to this segment are assigned the o label (i.e., outside). the code used to implement all the experiments reported in this paper can be publicly accessed from github foot_2 . furthermore, the weights of all the textbased models resulting from our experiments can be publicly downloaded from the huggingface foot_3 repository. ## results once the hyperparameters of each individual model have been optimised considering the experimental setup defined in the appendix a.2, the best results for each system combination are reported on for the cascade model, we test both audio and text segmenters and classifiers. similarly to the e2e case, the use of audio segmentation consistently improves the results. for the text classifier, moving from text segmentation (t-seg + t-clf ) to audio segmentation (a-seg + t-clf ) increases test macro-f1 from 0.49 to 0.51. likewise, when using an audio classifier (a-clf), audio segmentation improves the test results from 0.41 to 0.43 macro-f1. however, the relatively mediocre performance of the audio classification models with respect to its text counterparts stands out. we believe this could be caused due to the fact that speech classification is a harder task than text classification in our setup, because the audio classifier deals with the raw audio, whereas the text classifier uses the reference transcriptions as input. additionally, the pretrained audio models might not be capable enough for some transfer learning tasks, as they have been trained with a significantly lower number of tokens, which surely hampers language modelling capabilities. ## conclusions we have presented the vivesdebate-speech corpus, which is currently the largest speech-based argument corpus. furthermore, the experiments have shown how having access to audio information can be a source of significant improvement. specifically, using audio segmentation instead of text-based segmentation consistently improves performance, both for the text and audio classifiers used in the cascade approach, as well as in the endto-end scenario, where audio segmentation is used as a decoding constraint for the model.