D ebate KG – Automatic Policy Debate Case Creation with Semantic Knowledge Graphs (2023)

Abstract: Recent work within the Argument Mining community has shown the applicability of Natural Language Processing systems for solving problems found within competitive debate. One of the most important tasks within competitive debate is for debaters to create high quality debate cases. We show that effective debate cases can be constructed using constrained shortest path traversals on Argumentative Semantic Knowledge Graphs. We study this potential in the context of a type of American Competitive Debate, called “Policy Debate”, which already has a large scale dataset targeting it called “DebateSum”. We significantly improve upon DebateSum by introducing 53180 new examples, as well as further useful metadata for every example, to the dataset. We leverage the txtai semantic search and knowledge graph toolchain to produce and contribute 9 semantic knowledge graphs built on this dataset. We create a unique method for evaluating which knowledge graphs are better in the context of producing policy debate cases. A demo which automatically generates debate cases, along with all other code and the Knowledge Graphs, are open-sourced and made available to the public here: https://huggingface.co/spaces/Hellisotherpeople/DebateKG

## introduction 1.policy debate persuasion has been of interest to humans since we first began communicating with each other. the formal process of using argumentation and rhetoric to convince others to see in one's own way is known as "debate". with varying levels of formality and intensity, these debates happen all around us every day. more formalized, competitive forms of debate are both highly educational and integral to the formation of a lawful and just society. there is a long and time-honored tradition of academic institutions and news organizations facilitating competitive debate. many organizations and associations organize debate tournaments according to their differing traditions and rule sets. some types of debate are more suited to be assisted with natural language processing systems than others. a popular form of competitive debate done predominantly within united states high schools and universities is called "policy debate". policy debate maintains one extremely broad and open-ended topic over a whole year, and challenges teams to be ready to either affirm any plan which implements the topic, or to be ready to explain why the opposing teams plan is a bad idea. policy debate is a highly technical form of debate, which puts relatively little emphasis on the aesthetic quality of the speech act, and correspondingly strong emphasis on the quality of the delivered evidence and the delivered argumentation around it. for this reason, policy debate rewards teams who can present the maximum amount of evidence possible during their limited speaking time. this leads to a peculiar phenomenon known as "speed reading" or "spreading" which is normalized among most serious competitors. while policy debate idiosyncrasies may end up making it less amicable for the general public to watch than other forms, those very same traits make it a uniquely good source of data for nlp systems which generate high quality debate cases. ## innovations introduced in this work, we introduce several innovations related to automatic policy debate case generation. ## prior work many others have looked at the relationships between graph methods and argumentation. the closest prior work to our own comes from ibm project debater @xcite . they created a full debating system which they prominently pitted against champion parliamentary debaters. they defined a custom tailored, "simplified version" of the parliamentary debate style. parliamentary debate has dramatic differences compared to policy debate, namely that the topics are only known to each side 15 minutes ahead of time. as a result, parliamentary debate relies far less on evidence, usually only including small snippets as part of a larger speech. in policy debate, the vast majority of most of the opening speeches is recitation of extractive summaries of evidence for or against a position. this dramatically simplifies the required system for policy debate case generation. project debater utilizes many closed source models models, a massive but generalized corpus and requires significantly more compute resources than debatekg to run. finally, policy debate is considered to be a more "rigorous style" of debate at its highest level than parliamentary debate, which requires dramatically more effort to participate in. an example of this can be found in the 2014-2015 national parliamentary tournament of excellence (npda) tournament, the largest american college level parliamentary debate tournament, where the winning team had no prior parliamentary debate experience and was otherwise a good but not champion policy debate team foot_1 . their defeated opponents had been undefeated for the prior 3 years that they competed in the national tournament. further work coming from ibm exists about knowledge graphs directly being used for argument generation @xcite . their work explores how to utilize kg encoded knowledge to fine-tune gpt-2 to generate arguments. our system is extractive in nature, as it creates debate cases by chaining together evidence from debatesum utilizing graph traversals. extractive systems are far more appropriate for policy debate. there is fascinating work that applies the idea of graph neural networks for predicting the way that each member of a legislative branch will vote on an input motion @xcite . our work does not try to predict how judges will vote based on any inputs, but instead generates debate cases given input arguments. their work is in the context of elected officials, whereas ours is in the context high school and collegic competitive debate. there is also work related to trying to understand the arguments made within these legislative parliamentary debates @xcite knowledge graphs have been utilized for fact checked arguments. claimskg @xcite is an example, which indexes a wide variety of fact checking websites and annotates them. debatesum and its contributed kgs do not have fact checking information directly since it is considered the debaters job to convince the judge of the truth of each presented piece of evidence. debatesum and debatekg are also significantly larger in size than claimskg and its training corpus. work related to automatically evaluating the quality of arguments using knowledge graphs exists @xcite . in their work, they leverage a dataset of debate, the vivesdebate corpus, to identify if an argument is likely to "win". they also recognized the potential for graph traversals to form arguments, or whole debate cases (see figures 2 and 3 from their work). vivesdebate is significantly smaller and less encompassing than debatesum, and debatesum does not have information about how successful the arguments within it are. other work, which recognizes the potential for paths within knowledge graphs to form arguments, exists @xcite . the idea of using "debate dynamics" to present evidence for graph classification has been extensively explored @xcite . they imagine triple classification and link prediction in graphs as a figurative "debate game" between two reinforcement learning agents who extract "arguments" (paths) which support or oppose a hypothesis. a final binary classifier "judge" votes based on the presented "arguments". they show parallels within graph analysis algorithm development to the ideas that we present, but they evaluate this algorithm on non-argumentative datasets. to our knowledge, we are the first work to explore "arguments" (constrained paths) within knowledge graphs on an argumentative dataset. ## details the debatekg demo is hosted on huggingface foot_2 . in this section, we describe the details of debatekg and its underlying semantic knowledge graphs. ## evaluation debatesum does not include any data indicating if an argument is "strong", or if it is likely to win or not. it also does not have similarity labels between each example or even between pairs of samples. this means that it is challenging to compare the ## conclusion in this paper, we significantly expanded and improved an existing large scale argument mining dataset called "debatesum". we created 9 semantic knowledge graphs using the "txtai" semantic ai toolkit. we showed how constrained shortest path traversals on these graphs can be used to create policy debate cases. we created a system demonstration of this called debatekg which is a "space" webapp hosted on huggingface. we discuss implementation details of this system. we propose a way for policy debaters to decide which graph is better for their needs, and evaluate our systems using this technique. we open source all data, code, and graphs.