Project D ebater API s: D ecomposing the AI Grand Challenge (2021)

Abstract: Project Debater was revealed in 2019 as the first AI system that can debate human experts on complex topics. Engaging in a live debate requires a diverse set of skills, and Project Debater has been developed accordingly as a collection of components, each designed to perform a specific subtask. Project Debater APIs provide access to many of these capabilities, as well as to more recently developed ones. This diverse set of web services, publicly available for academic use, includes core NLP services, argument mining and analysis capabilities, and higher-level services for content summarization. We describe these APIs and their performance, and demonstrate how they can be used for building practical solutions. In particular, we will focus on Key Point Analysis, a novel technology that identifies the main points and their prevalence in a collection of texts such as survey responses and user reviews.

## introduction argumentation and debating are fundamental capabilities of human intelligence. they are essential for a wide range of everyday activities that involve reasoning, decision making or persuasion. over the last few years, there has been growing interest in computational argumentation, defined as "the application of computational methods for analyzing and synthesizing argumentation and human debate" @xcite . a recent milestone in this field is project debater, which was revealed in 2019 as the first ai system that can debate human experts on complex topics 1 . project debater is the third in the series of ibm research ai's grand challenges, following deep blue and watson. it has been developed for over six years by a large team of researchers and engineers, and its live demonstration in february 2019 received massive media attention. in our recent paper, "an autonomous debating system", published in the nature magazine @xcite , we describe project debater's architecture and evaluate its performance. to debate humans, an ai must be equipped with a diverse set of skills. it has to be able to pinpoint relevant arguments for a given debate topic in a massive corpus, detect the stance of arguments and assess their quality. it also has to identify principled, recurring arguments that are relevant for the specific topic, organize the different types of arguments into a compelling narrative, recognize the arguments made by the human opponent, and make a rebuttal. accordingly, project debater has been developed as a collection of components, each designed to perform a specific subtask. over the years, we published more than 50 papers describing these components and released many related datasets for academic use. successfully engaging in a debate requires high level of accuracy from each component. for example, failing to detect the argument's stance may result in arguing in favor of your opponent -a dire situation in a debate. a crucial part of developing highly accurate models was the collection of uniquely large scale, high-quality labeled datasets for training each component. the evidence detection classifier, for instance, was trained using 200k labeled examples, and was able to achieve a remarkable precision of 95% for top 40 candidates @xcite . another major challenge was scalability. one example is applying wikification @xcite to our 10 billion sentences corpus, a task that was infeasible for any of the available tools. we therefore developed a novel, fast wikification algorithm, which can be applied to massive corpora while achieving competitive accuracy @xcite . project debater apis give access to selected capabilities originally developed for the live debating system, as well as related technologies we have developed more recently. we provide free access for academic use to these apis, as well as trial and licensing options for developers. the apis can be divided into three main groups: several demonstrations of argument mining capabilities have been previously published @xcite @xcite , some of which also provide access to their capabilities via apis. however, project debater apis offer a much broader set of services, trained on unique large-scale, high quality datasets, which have been developed over many years of research. the next sections describe each of the apis and their performance assessment, and how they can be accessed and used via the debater early access program. we then describe several examples of using and combining these apis in practical applications. ## services overview in this section we provide a short description for each service, and point to its related publications, and other relevant resources. all the training datasets for these services have been developed as part of project debater. ## assessment the text clustering assessment is the only one that is not performed over a project debater dataset, but over a standard benchmark -the widely-used 20 newsgroups dataset, which contains about 18,000 news posts on 20 topics @xcite . we clustered these posts into 20 clusters, and compared the results with the original partition. we report adjusted mutual information (ami) and adjusted rand index (ari) measures. our results (@xmath0.595 and @xmath1.466) are considerably bet-ter than those obtained with k-means (@xmath2.228 and @xmath3.071). overall, the results confirm the high quality of our services. ## the debater early access program the 12 project debater apis are offered via the ibm debater early access program. the goal of this program is to make core capabilities from project debater available as building blocks for a variety of text understanding applications. the early access program is freely available for academic use on the ibm cloud, and can also be licensed for commercial use. as part of the program, both python and java sdks are available. all the services are rest-based, which enables their usage by any desired programming language. in order to access these apis, an api key is required. we supply such api keys freely for non-commercial use. the early access website, shown in figure 2 , contains various resources foot_1 . the main tab includes a detailed description of all the services with python, java, and curl code examples. the description also includes links to related publications. in addition, it contains a demo ui, which allows interacting with the apis online. the examples tab contains step-by-step tutorials, which demonstrate how the apis can be applied in complex scenarios, to solve real-world problems. the data sets tab contains a link to all project debater datasets. finally, there is a tab for the speech by crowd application. speech by crowd is a web application that enables the user to collect and analyze opinions on a desired controversial topic. the application is free for non-commercial use. the application has been demonstrated in several events, as we discuss in the next section. ## conclusion we introduced project debater apis, which provide access to many of the core capabilities of the project debater grand challenge, as well as more recent technologies such as key point analysis. the evaluation we presented confirms the high quality of these services. we discussed different use cases for these apis, in particular for analyzing and summarizing various types of opinionated texts. we believe that this diverse set of services may be used as building blocks in many text understanding applications, and may be relevant for a broad audience in the nlp community.