Hierarchical Representation-based Dynamic Reasoning Network for Biomedical Question Answering (2022)

Abstract: Recently, Biomedical Question Answering (BQA) has attracted growing attention due to its application value and technical challenges. Most existing works treat it as a semantic matching task that predicts answers by computing confidence among questions, options and evidence sentences, which is insufficient for scenarios that require complex reasoning based on a deep understanding of biomedical evidences. We propose a novel model termed Hierarchical Representation-based Dynamic Reasoning Network (HDRN) to tackle this problem. It first constructs the hierarchical representations for biomedical evidences to learn semantics within and among evidences. It then performs dynamic reasoning based on the hierarchical representations of evidences to solve complex biomedical problems. Against the existing state-of-the-art model, the proposed model significantly improves more than 4.5%, 3% and 1.3% on three mainstream BQA datasets, PubMedQA, MedQA-USMLE and NLPEC. The ablation study demonstrates the superiority of each improvement of our model. The code will be released after the paper is published.

## introduction machine reading comprehension (mrc) @xcite @xcite tasks are often used to evaluate the intelligence degree of a system, and many recent large-scale pre-trained language models @xcite have surpassed the human performance on open-domain mrc. in recent years, biomedical question answering (bioqa) @xcite @xcite question：a 35-year-old caucasian female presents to the hospital alarmed by her recent truncal weight gain, facial hair growth, and thinning skin. during the physical exam, the physician finds that the patient is hypertensive. serum analysis reveals hyperglycemia. the physician suspects a pituitary adenoma. which dexamethasone test result would help confirm the physician's suspicions? options： a: low-dose, increased acth; high-dose, decreased acth b: low-dose, decrease in acth; high-dose, no change in acth c: low-dose, no change in acth; high-dose, no change in acth d: low-dose, no change in acth; high-dose, decreased acth e1: if cortisol production is driven by an acth producing pituitary adenoma dexamethasone suppression is ineffective at low doses but usually induces suppression at high doses. inappropriately low acth levels in the setting of low cortisol levels are characteristic of diminished acth reserve. e2: high dose dexamethasone suppresses acth production by a pituitary adenoma serum cortisol is lowered but does not suppress ectopic acth production serum cortisol remains high. cortisol stimulates gluconeogenesis and insulin resistance resulting in hyperglycemia as well as muscle cell protein breakdown and lipolysis to provide sub strates for hepatic gluconeogenesis. ## background bioqa is a classification task that uses accuracy as the evaluation metric. specifically, given a natural language question q and evidences c, it requires the intelligent system to predict the correct option ô from the candidate set ω o based on the understanding of the evidences c. θ is set of the model parameters. as figure 2 (a) shows, most previous works treat bioqa as a semantic matching task. they first concatenate all information together, including questions and evidences, then encode them using a language model, and finally predict the best option by computing the semantic matching score with a multi-layer perceptron. the previous works accomplish both representation learning and reasoning using a single model. there are two main drawbacks: (i). lack of deep understanding of biomedical evidences. the hierarchical information among evidences is easily lost when all the information is mixed together for encoding. (ii). lack of strong reasoning capability. single-step implicit reasoning does not cope well with complex questions and evidences. in terms of human experience, the reasoning is usually an iterative process that requires multi-step to solve complex questions. as figure 2 (b) shows, to address the problem, we propose hdrn, a network for representing learning and reasoning for bioqa. it first constructs hierarchical representations to obtain a deep understanding of the biomedical evidences, and then performs multi-step dynamic reasoning to solve complex questions. ## method in this section, we describe the detail of the proposed method. the overall architecture is shown in figure 3 , which consists of two components: (a) dynamic reasoning mechanism, which can better solve complex biomedical questions by conducting multi-step dynamic reasoning. (b) hierarchical representation learning, which can better understand the biomedical evidences by learning the semantics within and among the them. ## conclusion this paper proposes hdrn, a novel model for representation learning and reasoning for biomedical question answering. first, we construct hierarchical representations to obtain a deep understanding of the biomedical evidences. then, we perform multi-step dynamic reasoning to solve complex biomedical questions. we evaluate our model on three bioqa datasets and achieve new state-of-theart performances.