UKP - SQ u ARE : An Interactive Tool for Teaching Question Answering (2023)

Abstract: The exponential growth of question answering (QA) has made it an indispensable topic in any Natural Language Processing (NLP) course. Additionally, the breadth of QA derived from this exponential growth makes it an ideal scenario for teaching related NLP topics such as information retrieval, explainability, and adversarial attacks among others. In this paper, we introduce UKP-SQuARE as a platform for QA education. This platform provides an interactive environment where students can run, compare, and analyze various QA models from different perspectives, such as general behavior, explainability, and robustness. Therefore, students can get a first-hand experience in different QA techniques during the class. Thanks to this, we propose a learner-centered approach for QA education in which students proactively learn theoretical concepts and acquire problem-solving skills through interactive exploration, experimentation, and practical assignments, rather than solely relying on traditional lectures. To evaluate the effectiveness of UKP-SQuARE in teaching scenarios, we adopted it in a postgraduate NLP course and surveyed the students after the course. Their positive feedback shows the platform’s effectiveness in their course and invites a wider adoption.

## introduction question answering (qa) is one of the overarching research topics in natural language processing (nlp). qa pipelines have been developed to address different types of questions, knowledge sources, and answer formats, including extractive, abstractive, knowledge base, multiple-choice, generative, and open-domain qa. such a massive number of qa systems and relevant nlp techniques are making qa lectures more important in nlp courses. however, despite qa being an application-oriented topic (e.g., chatbots, virtual assistants, etc.), classes are usually theoretically driven. thus, in this paper, we propose the use of the ukp-square platform as a tool for qa education. this platform integrates most qa formats, popular models, datasets, and analysis tools, such as explainability, adversarial attacks, and graph visualizations. compared with conventional teacher-led classes, we propose a learner-centered class following the flipped classroom @xcite with ukp-square as the driving tool of the lecture. this tool provides an interface for users to interact with different qa models and analysis tools. therefore, students can actively learn about qa systems and get hands-on experience by interacting with models on the platform. concretely, students can flexibly compare multiple architectures that model different qa formats, analyze their outputs with explainability tools, and even analyze their robustness against adversarial attacks. prior studies have shown that flipped classroom lectures improve the learning process of students in programming courses @xcite . thus, we believe that teaching and learning qa through a live demo with this platform can also make nlp lectures more engaging, drawing students' attention, and interest in the topics. to investigate the effectiveness of ukp-square in qa education, we adopted it for the first time in a postgraduate nlp course foot_0 and conducted a survey afterward. the positive feedback from the students encourages us to continue adopting this platform and education method in more nlp courses. the contributions of this paper are: i) a novel interactive learner-centered methodology to teach qa and relevant nlp topics, ii) extending the ukp-square platform for teaching qa, and iii) the design of a syllabus for interactive qa lectures. ## ukp-square ukp-square @xcite @xcite is an extendable and interactive qa platform that integrates numerous popular qa models such as deeepset's roberta-base-squad2 foot_1 , spanbert @xcite for hotpotqa, and qagnn @xcite . it provides an ecosystem for qa research, including comparing different models, explaining model outputs, adversarial attacks, graph visualizations, behavioral tests, and multi-agent models. in addition, this platform provides a user-friendly interface foot_2 that enables users to interact. users can run available models, deploy new ones, compare their behaviors, and explain outputs. ## learning question answering with ukp-square in this section, we present the syllabus of a lecture focused on qa and relevant nlp topics that use the platform ukp-square following the flipped classroom methodology @xcite . the flipped classroom is an effective learnercentered educational methodology in which students study pre-recorded lectures and materials in advance to engage in more interactive and collaborative learning activities in class. ukp-square can be the driving tool for the flipped classroom in qa education. with our platform, lecturers can introduce the topics by interacting with the students and then proceed to an in-depth explanation of the technical details behind the methods of each topic. we propose dividing the lecture into three topics in the qa field: basic qa concepts, trustworthy qa, and multi-agent qa systems. with these topics, students can learn about qa and related nlp topics such as information extraction, explainability, adversarial attacks, and multi-agent systems. ## related work the most relevant tool is the allennlp demo 7 , which provides a user interface to the main components of the allennlp library @xcite . this website includes an interface where users can interact with five extractive qa models. however, their goal is to have a showcase of their library rather than an extensive platform for teaching qa. thus, their functionalities are limited. most of their deployed models are outdated, only cover extractive qa settings, and do not provide information retrieval methods. moreover, their explainability and adversarial attacks are not compatible with their transformer-based model. furthermore, they do not provide graph-based models, which can be useful to explain graph neural networks and explainability methods based on graphs. additionally, it cannot be used for our homework assignment because users cannot deploy and analyze their own models with explainability and adversarial attack tools as in our platform. however, they do provide demos for other nlp topics, such as open information extraction and named entity recognition, and parsing trees, among others. ## conclusion in this paper, we present a novel method to teach question-answering to postgraduate nlp students following the learner-centered method of flipped classrooms. we propose to provide reading materials to the students before the class and use the ukp-square platform as a driving tool to conduct the class. this platform integrates the most popular qa pipelines and an ecosystem of tools to analyze the available models. these tools include explainability methods, behavioral tests, adversarial attacks, and graph visualizations. we provide a series of use cases for teaching based on the provided models and methods by ukp-square, showing that classes can become much more interactive by using ukp-square than in conventional lectures. to evaluate the effectiveness of the platform and our methodology, we conducted a survey to collect feedback from students who took our class. the results show that most of the students think ukp-square accelerates their learning process and reduces the overhead to get hands-on experience. we plan to extend our platform to support prompting large language models, and therefore, we leave as future work creating a curriculum to teach prompting methods.