P ipe N et: Question Answering with Semantic Pruning over Knowledge Graphs (2024)

Abstract: It is well acknowledged that incorporating explicit knowledge graphs (KGs) can benefit question answering. Existing approaches typically follow a grounding-reasoning pipeline in which entity nodes are first grounded for the query (question and candidate answers), and then a reasoning module reasons over the matched multi-hop subgraph for answer prediction. Although the pipeline largely alleviates the issue of extracting essential information from giant KGs, efficiency is still an open challenge when scaling up hops in grounding the subgraphs. In this paper, we target at finding semantically related entity nodes in the subgraph to improve the efficiency of graph reasoning with KG. We propose a grounding-pruning-reasoning pipeline to prune noisy nodes, remarkably reducing the computation cost and memory usage while also obtaining decent subgraph representation. In detail, the pruning module first scores concept nodes based on the dependency distance between matched spans and then prunes the nodes according to score ranks. To facilitate the evaluation of pruned subgraphs, we also propose a graph attention network (GAT) based module to reason with the subgraph data. Experimental results on CommonsenseQA and OpenBookQA demonstrate the effectiveness of our method.

## introduction question answering requires related background knowledge. a line of research resorts to combining pre-trained language models (lms) and knowledge graphs (kg) to utilize both the implicit knowledge in lms and explicit knowledge in structured kgs @xcite @xcite . the researches towards utilizing knowledge from kgs typically follow a grounding-andreasoning pipeline, namely schema graph grounding and schema graph reasoning @xcite . in the grounding module, multi-hop neighbors of matched concept nodes in the query from kg form a subgraph. recent works focus on improving reasoning ability by enhancing the representation of multi-hop nodes in grounded subgraphs with graph neural networks (gnns) @xcite or interaction between representations of query context and subgraphs @xcite . while pre-trained lms are powerful at extracting plain text features for the query context, the quality of subgraph feature extracted from gnns is still prone to noisy nodes in grounded subgraphs. specifically, there are two challenges in fusing kgs with gnns. first, the computation and memory cost would increase with the hops increase. second, the noisy nodes induced with increasing hops deteriorate the quality of the subgraph feature, and further decrease the performance of the reasoning module. in this paper, we tackle the problems brought by noisy nodes with a grounding-pruning-reasoning pipeline framework, pipenet. previous researches show that the improvement of graph-based reasoning systems is minor, though with the number of grounded hops increasing, many more new nodes are induced @xcite @xcite . as shown in figure 1 , many of them are the same for different candidate answers, especially near the question concepts. diverse nodes are mainly brought in due to the difference in answer concepts. this diversity is critical to the subgraph representation learning with gnns. our pruning module prunes noisy nodes before the reasoning module, reducing the computation cost and memory usage while keeping the diversity of subgraphs in the meantime. specifically, we propose a dependency structure based pruning method to prune the nodes with dependency parsing (dp) tools. the dp-pruning strategy is inspired by relation extraction in automatic ontology building, in which the dependency tree is applied to find possible relations between concepts according to the distance on the tree @xcite @xcite . similarly, we assume the dependency tree provides reasonable linguistic links between grounded concepts in a natural language context. we further convert the dependency distances between grounded concepts into concept node scores and propagate the node scores onto the grounded multi-hop subgraph to prune external noisy nodes. to facilitate the evaluation of pruned subgraph, we also propose a simplified version of gat @xcite for graph representation learning. we redesign the message passing mechanism in @xcite . our contributions are as follows: experiments on two standard benchmarks, com-monsenseqa @xcite and open-bookqa @xcite , demonstrate the effectiveness of our proposed method. the code is open-sourced foot_0 . ## methodology our grounding-pruning-reasoning framework, pipenet, consists of three stages: subgraph grounding, subgraph pruning, and reasoning. the overall framework is shown in figure 2 . ## experiments our experiments are conducted on two standard question answering benchmarks, common-senseqa (csqa) and openbookqa (obqa). we also introduce details of baselines and implementations in this section. ## results in this section, we first present of main results of pipenet as well as pipenet with dp pruning strategy on standard benchmarks. then we analyze the time and memory efficiency improvement brought by dp-pruning strategy. finally, we conduct an ablation study over pruning strategy. ## conclusion in this work, we propose pipenet, a groundingpruning-reasoning pipeline for question answering 367 with knowledge graph. the pruning strategy utilizes the dependency structure of query context to prune noisy entity nodes in the grounded subgraph, benefiting the subgraph representation learning with gnns. we further design a gat-based module for the subgraph representation learning with simplified message flow. experiment results on two standard benchmarks demonstrate the effectiveness of semantic dependency of concept items benefits the subgraph representation learning.