Triggering Multi-Hop Reasoning for Question Answering in Language Models using Soft Prompts and Random Walks (2023)

Abstract: Despite readily memorizing world knowledge about entities, pre-trained language models (LMs) struggle to compose together two or more facts to perform multi-hop reasoning in question-answering tasks. In this work, we propose techniques that improve upon this limitation by relying on random-walks over structured knowledge graphs. Specifically, we use soft-prompts to guide LMs to chain together their encoded knowledge by learning to map multi-hop questions to random-walk paths that lead to the answer. Applying our methods on two T5 LMs shows substantial improvements over standard tuning approaches in answering questions that require multi-hop reasoning.

## introduction performing multi-hop reasoning to answer questions such as where was david beckham's daughter born? requires two fundamental capacities: c1: possessing pre-requisite knowledge (david beckham's daughter is harper beckham, harper beckham was born in los angeles), and c2: ability to compose internalized knowledge. contemporary pre-trained language models (lms) such as bert @xcite and t5 @xcite have been shown to be adept at encoding factual knowledge @xcite @xcite , an ability that can be further boosted by explicitly integrating them with knowledge about entities and relations @xcite @xcite . at the same time, these lms often struggle to compose the knowledge they encode @xcite @xcite , and therefore do not satisfy c2. to overcome this limitation, previous works have proposed methods that decompose multi-hop questions into single hop sub-questions that models can more easily answer f work done during an internship at google research. @xcite . however, such methods require training entirely separate models, or make use of human-annotations @xcite . furthermore, they focus on tasks where models explicitly receive additional text containing relevant facts, which makes it unclear if they can truly compose the knowledge that they have internalized. in this work, we aim to improve the standalone, self-contained ability of lms to perform multi-hop reasoning. we posit that random walks-paths between entity nodes sampled from structured knowledge graphs-can provide a useful training signal for lms to compose entity knowledge. to test this, we perform a case-study on two t5 models (large and xxl, @xcite . specifically, we first integrate within the lms the single-hop knowledge that is required to answer multi-hop questions (effectively guaranteeing c1 is met). we show that this alone is not enough to demonstrate substantial improvements on questions requiring 2-hop reasoning. we then adapt the knowledge integrated t5 models by training soft prompts @xcite on random walks over the structured knowledge that they have encoded, and devise two methods that trigger this ability in the lms given a multi-hop question as input. the first method, parse-then-hop (path), uses two specialized soft prompts: one to parse entities and relations from the question, and another to generate a path to the answer, resembling the outputs of a random walk. the second method, mixhop, trains a single prompt on a mixture that combines the qa task with the random walk training, so as to allow the model to implicitly learn path's task. both these soft prompt methods use the same underlying lm (kept frozen), and guide it to compose its internalized entity knowledge. our experiments suggest that integrating random walks in the t5 models using our proposed techniques can substantially improve their ability to t5 david beckham ; daughter harper beckham question: where was the director of violet tendencies born? david beckham ; daughter ; place of birth 🔥 knit5 knowledge integration random walk training ❄ method 1: parse-then-hop (path) pp ⨁ question knit5 violet tendencies ; director; place of birth knit5 ❄ violet tendencies ; director ; casper andreas ; place of birth ; sweden answer entity-centric 2-hop questions @xcite at larger model sizes. briefly, on t5-xxl our methods show improvements over previously proposed prompt-tuning approaches @xcite as well as full model finetuning, with path and mixhop demonstrating gains of ⇠16 and ⇠9.6 points in exact match scores over fine-tuning the entire model, respectively. in the case of t5-large, our methods demonstrate improvements over standard prompt-tuning methods, but fall short of the performance achieved using fine-tuning, suggesting that larger modelswith up to 11b parameters-are more conducive to leveraging the training signal provided by random walks via soft prompts. ## experiments and findings 2 we report and summarize our results as follows: integration of 1-hop knowledge only results in marginal improvements on 2-hop questions we begin by first establishing the extent to which t5 models encode and compose 1-hop knowledge required to answer 2-hop questions, and whether additional knowledge integration (via knit5) can improve both these abilities. from tables 1 and 3 , we observe that the t5 models struggle to answer both 1-hop as well as 2-hop questions, suggesting that they critically lack the precise 1-hop entity knowledge required to demonstrate success on the 2-hop questions. the knit5 lms overcome this limitation, by showing substantial gains on 1wik-ihopqa over their t5 counterparts-they show improvements of ⇠16.5 and ⇠34.8 points in ex- ## conclusion we show that composition of memorized world knowledge can be triggered in lms with up to 11b parameters (t5-xxl) to a desirable extent by leveraging training signal from random walks over structured knowledge using approaches based on prompt-tuning @xcite . doing so leads to substantial improvements in the lms' ability to answer 2-hop questions, even beyond standard, full model fine-tuning.