D eep B lue AI at T ext G raphs 2021 Shared Task: Treating Multi-Hop Inference Explanation Regeneration as A Ranking Problem (2021)

Abstract: This paper describes the winning system for TextGraphs 2021 shared task: Multi-hop inference explanation regeneration. Given a question and its corresponding correct answer, this task aims to select the facts that can explain why the answer is correct for that question and answering (QA) from a large knowledge base. To address this problem and accelerate training as well, our strategy includes two steps. First, fine-tuning pre-trained language models (PLMs) with triplet loss to recall top-K relevant facts for each question and answer pair. Then, adopting the same architecture to train the re-ranking model to rank the top-K candidates. To further improve the performance, we average the results from models based on different PLMs (e.g., RoBERTa) and different parameter settings to make the final predictions. The official evaluation shows that, our system can outperform the second best system by 4.93 points, which proves the effectiveness of our system. Our code has been open source, address is https://github.com/DeepBlueAI/TextGraphs-15

## introduction multi-hop inference is the task of doing inference by combining more than one piece of information, such as question answering @xcite . the textgraphs 2021 shared task on multi-hop inference explanation regeneration focuses on the theme of determining relevance versus completeness in large multi-hop explanations, which asks participants to rank how likely table row sentences are to be a part of a given explanation. concretely, given an elementary science question and its corresponding correct answer, the system need to perform the multi-hop inference and rank a set of explanatory facts that are expected to explain why the answer is correct from a large knowledge base. an example is shown in figure 1 . a number of contemporary challenges exist in performing multi-hop inference for question answering @xcite , including semantic drift, long inference chains, etc. several multi-hop inference shared tasks have been conducted in the past few years @xcite , and methods based on large pretrained language models (plms) such as bert @xcite , roberta @xcite and ernie @xcite are proposed. in this paper, we describe the system that we submitted to the textgraphs 2021 shared task on multi-hop inference explanation regeneration. there are two main parts of our system. first, we use a pre-trained language model-based method to recall the top-k relevant explanations for each question. second, we adopt the same model architecture to re-rank the top-k candidates to do the final prediction. when determine whether an explanation sentence is relevant to the question, the previous works @xcite constructed a pair of explanations with the qa (questions with corresponding answers) sentence as the input of the plms. to reduce the amount of calculation and accelerate training, instead of using all the explanations from the given table, we propose to fine-tune plms with triplet loss @xcite , a loss function where a baseline (anchor) input is compared to a positive (true) input and a negative (false) input. for choosing samples as the negative input, we design several ways which will be introduced in section 3. experiments on the given dataset show the effectiveness of our model and we rank first in this task. task definition the explanation regeneration task supplies models with questions, their correct answers, the gold explanation authored by human annotators, as well as a knowledge base of explanations. from this, for a given question and its correct answer, the model must select a set of explanations from the knowledge base that explain why the answer is correct. ## model architectures our system consists of two major components. the first part is the retrieval procedure, which utilize the plms fine-tuned with triplet loss to recall top-k (@xmath0. the second part is the re-ranking procedure, which use the same model architecture to rank the top-k candidates. the model architecture is shown in figure 2 . ## conclusion in this paper, we propose a top performing approach for the task of multi-hop inference explanation regeneration. we fine-tune pre-trained language models with the triplet loss to accelerate training and design different ways for negative sampling. the same model architecture is utilized to recall the top-k candidates from all the facts and to re-rank the top-k relevant explanations for the final prediction. experimental results show the effectiveness of the proposed method and we win the first place for the task.