Leveraging Structured Information for Explainable Multi-hop Question Answering and Reasoning
Authors: Ruosen Li
Xinya Du
Venue: EMNLP 2023 Findings (2023)
Keywords: Natural Language Processing
Multi-hop Question Answering

Abstract: Neural models, including large language models (LLMs), achieve superior performance on multi-hop question-answering. To elicit reasoning capabilities from LLMs, recent works propose using the chain-of-thought (CoT) mechanism to generate both the reasoning chain and the answer, which enhances the model's capabilities in conducting multi-hop reasoning. However, several challenges still remain: such as struggling with inaccurate reasoning, hallucinations, and lack of interpretability. On the other hand, information extraction (IE) identifies entities, relations, and events grounded to the text. The extracted structured information can be easily interpreted by humans and machines (Grishman, 2019). In this work, we investigate constructing and leveraging extracted semantic structures (graphs) for multi-hop question answering, especially the reasoning process. Empirical results and human evaluations show that our framework: generates more faithful reasoning chains and substantially improves the QA performance on two benchmark datasets. Moreover, the extracted structures themselves naturally provide grounded explanations that are preferred by humans, as compared to the generated reasoning chains and saliency-based explanations.

Leveraging Structured Information for Explainable Multi-hop Question Answering and Reasoning
Ruosen Li and Xinya Du Department of Computer Science
University of Texas at Dallas
ruosen ruosen. wwt.nnniuuuttdllaaaaaaammaaaaaaa edu xinya. du@ut edu
Abstract
et al., 2020) have applied graph neural networks (GNN) to this task. In recent years, with the grow- ing capabilities of large language models (LLMs), several works propose using prompting to address this task in a few- or zero-shot way (Wei et al.. 2022; Ho et al., 2023).
Neural models, including large language mod- els (LLMs), achieve superior performance on To elicit reason- ing capabilities from LLMs, recent works pro- pose using the chain-of-thought (CoT)) mecha- nism to generate both the reasoning chain and the answer, which enhances the model's l's ca- pabilities in conducting multi-hop reasoning. However, several challenges still remain: such struggling with inaccurate reasoning, hallu- cinations, and lack interpretability. On the other hand, information extraction (IE) iden- tifies entities, relations, and events grounded the text. The extracted structured infor- mation be easily interpreted by humans and machines (Grishman, 2019). In this work, we investigate constructing and leveraging ex- tracted semantic structures (graphs) for multi- hop question answering, especially the reason- ing process. Empirical results and human eval- uations show that our framework: generates more faithful reasoning chains and substantially improves the QA performance two bench- mark datasets. Morcover, the extracted struc- tures themselves naturally provide grounded explanations that are preferred by humans, as compared to the generated reasoning chains and saliency-based explanations."
multi-hop prestton--nsweeren
LLMs have achieved superior performance reasoning-based question-answering tasks. These models have the potential to "reason" and connect multiple pieces of information to generate the an- swers. However, they still struggle with complex multi-hop questions; and the end-to-end generation nature makes their answering process not explain- able. To elicit the reasoning capabilities of LLMss, recent research introduced the chain-of-thought (CoT) mechanism and its variants (Wei et al., 2022; Wang et al., 2023). The CoT mechanism enables models to not only generates better answers but the reasoning chain (process) in natural language, improving the LLM's performance in conducting multi-hop reasoning.
Despite the progress made, several challenges persist under this paradigm, One is the neural mod- els' limitations in tackling compositional problems that require striet multi-hop reasoning to derive correct answers (Dziri et al., 2023). CoT-based methods still conduct generations an end-to-end way, which not a strict "symbolic derivation" and often causes inaccurate reasoning (e.g., gener- ating answers that don't exist or two-hops away). Instead, we propose multi-step approach (Fig- ure 1) to tackle this problem: firstly, constructing the semantic graph structures (SG) with informa- tion extraction (IE) and then leveraging this sym- bolic information (including entities and semantic relations) for strictly guiding the model's reasoning process.
1 Introduction
Multi-hop question answering (QA) involves an- swering questions that require reasoning over mul- tiple pieces of information, often spread across dif- ferent parts of document multiple documents. It looks like "hopping" over various facts to arrive at a conclusion or answer. In multi-hop question answering, the system needs to understand the con- text, maintain the sequence of information, and utilize this combined information to generate cor- rect and comprehensive answer. Traditionally, re- searchers (Qiu et al., 2019; Tu et al., 2019; Fang
The second challenge is that, although the cur- rent gener-egnerrated reasoning chain provides ex- planations, they are only surface-level interpreta- tions, and there is no guarantee that they are fully
'Code is available at https: // thub. com/ Bedn1p/Sttuucturee

---

Model Input
Model Input
Documents:
Documents:
Wkkipedia Anne
Wikipedia Tille: Anne
Anne loves..... She was the daughter of Jean...
Anne loves ... She was the daughter of Jeen
Information
Anne
daughter of
Extraction
Wikipedia Title: Jean Jean was great- grandson Louis. He also the third and youngest child of Robert and Bath baught by Philips.
Wikipedia
Jeen was greal-grandeon of Louie ... He is also the third son and youngest child of Robert and Jean Robert taught by Philips
Jean
Robert
youngest child of
Semantic Structure:
Q. Answer the following question slep-by-step. the teacher paternal grandfiathr?
daughter.
<Jean, youngeet child. Robert-
Bequentalize
<lean, graat grändsonnnnnnnn Louis>
taught
Philips
<Robrri, taught Phillos>
Q Answer the following question step-by-step.
Who the leacher of Anne's palernel
grandfather?
Model Oulput)
(Model Outpul)
A: was daughter youngest child of Rabert. Robert was the great- grandson of Louis. The document does say who laught
was
daughter Jean. Jean the youngest child of Robert. Therefare, Robert the paternal grandfather of Anne. Robert is: Philips.
unknown.
laught by Philipe.
We first extract the semantic graph and add to the model input prompt. Then they are fed into the LLM to generate the reasoning chain and answer. The semantic structures help improve question answering.
Figure I:
grounded context (Narang et al., 2020). For example, the model could make up a reasonable- sounding explanation that is not faithfulness for its decision-making (hallucinations). While our ex- tracted graphs from the IE step naturally provide explanations grounded to the given context (Fig- ure 4), which overwhelmingly preferred by hu- man evaluators, compared to saliency- and NLG- based explanations. Plus, we also find that by lever- aging our extracted SGs, the large language model also generates more faithful reasoning chains (Sec- tion 4.3),
method (Asai et al. (2020); Qi et al. (2021)), and miscellancous techniques (Chen et al. (2020); Ho et al. (2023)).
In these various directions, the graph-based method is branch that solves the problem by build- ing graphs and inferring on graph neural networks (GNN) to predict answers. Graphs can be cate- gorized entity-only graphs (Qiu al. (2019); De Cao al. (2019)) and multi-level node graphs (Tu et al. (2019): Ye et al. (2019); Fang al. (2020)). Nodes those graphs represent entities, sentences, paragraphs, and documents. Edges con- nect entities and include no semantic information. In our method, we build semantic graphs based on the definition in Section 3.2. Compared to previ- ous work graph building, our constructed graphs include key semantic relations edges, which provides important fine- fine-grained information.
To sum up. make the following contribu- tions in this work: (1) We propose creating and utilizing semantic graphs to enhance models' ca- pability in multi-hop reasoning and achieve bet- ter performance on the task; (2) We demonstrate that leveraging semantic graphs in the prompting process help models generate higher-quality and more faithful reasoning chains; (3) We show that semantic graphs-based explanations are grounded context and help improve the interpretability of the answer- preeicccg process, which human evalu- ators prefer.
In addition, under our prompting-baut based QA set- ting, we sequentialize the graphs by concatenating the triples, which simplifies the process of encod- ing the semantic graphs. Specifically, our approach does not require the fine-tuning process since we apply the pooppt-hase method over LLMs, which simplifies the reasoning process.
2 Related Work
Multi-hop Question Answering Traditional ap- proaches to solving the multi-hop QA problem can be mainly categorized as question decomposition (Perez et al. (2020); Fu et al. (2021); Sun et al. (2021); Deng et al. (2022)), graph- based method (Qiu et al. (2019); Fang et al. (2020)), iterative
This generative format is closely related and con- current the prompting-basee method that is cently applied to the multi-hop QA task (Trivedi et al. (2022); Khot et al. (2023); Yoran et. (2023)). We introduce the details and their variance the paragraph below.

---

Prompting and Reasoning The Chain-of- Thought (CoT) prompting strategy (Wei et al., 2022), as a variation of LLM few-shot prompting, significantly improves the reasoning ability of LLMs. Compared to traditional few-shot prompt- ing, it turns complex implicit reasoning chains into explicit long natural language text and adds it to the demonstrations, which helps LLMs conduct better reasoning. Kojima et al. (2023) introduce zero-shot-CoT by using a simpler prompt ("Let's think step-by-step") to instruct L.Mss. Wang et al. (2023) introduce the self-consistency strategy to improve the performance of the CoT strategy.
3.1 Task and Settings
Given a question and candidate paragraphs, we aim to obtain answer that involves multi-hop rea- soning, We tackle the task with in-context learning from two different settings: (1) generate the answer as well as the explanations. one representative method is chain-of- ofhingo reasoning (Wei et al.. 2022). We name it "CoT setting" here. - it is our default setting and of more importance since we fo- cus on studying the faithfulness and interpretability of explanations; (2) directly generate the answer from the context and question ("fewshot setting").
When applying prompting strategy to the multi- hop QA task, Trivedi et al. (2022) introduces an iterative information retrieval method to provide more relevant context. Khot et al. (2023) decom- poses prompts to simple solvable tasks. In our approach, we leverage semantic graphs extracted from documents, which guide reasoning processes and help accurately find the answer node. Those graphs are extracted from context, sequentialized into text form, and added back into context.
3.2 Entity and Relation Extraction with Prompting
Drawing insights from the literature on information extraction, we propose two paradigms on entity and relation extractions: (1) treating them as separate tasks and conducting multi-step prompting for ex- tracting entities first, and then relations in between them (Zelenko et al., 2003); (2) joint extraction (Li et al., 2013; Miwa and Bansal, 2016) of the entity and relations, thus building the structured semantic graph in an end-to-end way.
Generating faithful reasoning chains is difficult with the vanilla CoT strategy, as generated natural language contents may contain irrelevant informa- tion and cause hallucinations. Lyu et al. (2023) try to solve by translating natural languages to symbolic languages and deducing results by de- terministic external solvers. But, the process of translation itself remains opaque and cannot guar- antee the prevention of hallucinations.
For entity (node), we define it as short text span/sequence that clearly describes an ob- ject/event/truth in a sentence. This formulation has broader coverage than traditional named entity recognition (NER) based on fixed schema. For example, "<Windermere, is popular for, its prox- imity to the lake>" an important triple in the semantic graph extracted from "Tourism is popular in Windermere mainly for its proximity to the lake." Traditional NER tools can only extract "Tourism", "Windermere", and "lake", real-world objects, but not "its proximity to the lake", a description of truth. We find in the prelim study that using tradi- tional NER doesn't provide improvements, mainly because they are uninformative. For relations (edges), we borrow ideas from Open IE (Fader et al., 2011). Traditional relation extraction clas- sifies relations into sets of categories, for exam- ple, OWNERSHIP, LOCATED, FAMILY) Thus, the number of closed relations is limited. They can hardly be used to present complex relations between entities that are defined above. OpenIE ex- tracts any strings in the sentence that can describe relations between two entities. For our case, we extract the document-level relations described in
Instead of directly generating symbolic presenta- tions from contexts, we introduce semantic graphs prevent hallucinations and improve interpretabil- ity. Since graphs are extracted from contexts, they are strictly grounded to context. Edges in the graphs can be seen as potential reasoning steps in the whole reasoning process.
3 Methodology
In this section, we first give formal introduction to the multi-hop QA task and two different settings tackle the problem. We then describe our technique for extracting semantic graphs, which will be used in our framework. Third, we introduce our overall and specific prompting for generating the reasoning chains and final answers.
PIn total, 17 relations are used in ACE relation extraction evaluations; and Wikidata has 11 11,008 relations.

---

Documents:
Document
Wikipedia Title: Princess Anne
Princess Anne She the daughter of Prince
paval
Jean
Wikipedia Title: Prince Jean
Pisson Aree
Aaagtar df
Prince Jann
pausgam
Piicce Rosert
Loaa
Prince Jean of was great-grandson of Lauis Philippe also third son and youngest child of Prince Robert and
Joint Extraction
Enlity Extraction
Relation Extraction
O,O
Document Princess Anne
Document 2:
Prince Jean Prince Robert Louis Philippe
Figure 2: We propose to use single-step prompting for jointly extracting entity entity-relation graphs; well as using multi-step prompting for extracting the entities and relations separately. The extracted semantic graph captures the inter-document and intra-document dependencies between entities.
natural language in the text.
Semantic Relation Extraction To extract the se- mantic relations between entities, we propose two variations, one uses the multi-step extraction idea, and the second conduct joint extraction. Specifi- cally, for multi-step extraction, the model extracts the entities first (as mentioned above), then append them to the prompt of relation extraction, and fi- nally the prompt ends with Graph:,
Finally, to illustrate in the semantic graph, we represent the entities as nodes and semantic rela- tions as edges (c.g., Figure 2). For example, the node "Princess Anne" is connected to the node "Prince Jean" via the edge "daughter of".
Entity Extraction The input we use as the prompt for the entity extraction step basically con- sists of one document, including Wikipedia para- graph. It starts with the word "Document: :"
Document:
Wikipedia Title: <Page Title> <Paragraph Text>
Then, it is followed by paragraph with title. Finally, we add the prompt Entities:. ask- ing it to start extracting list of entities:
<Entity i> An<Entity k>
Graph:
The output to be generated would look like this:
Document:
Mikipedia Title: <Page Title>
(<Entity a>, <R 1>, <Entity b>) (<Entity b>, <R 2>, <Entity <>)
<Paragraph Text>
Entities:
We also conduct a few-shot ICL. In this graph structure, not all pairs of entities have relations. We extract relations for pairs of entities if it exists.
The output to be generated by the LLM is sup- posed to be all relevant entities listed line-by-line. Overall, it looks like:
Joint Extraction of Entity and Relation Triples Different from the multi-step of extracting seman- tic graphs above, we also try direct prompting. Namely, skip the entity generation process and directly generate the (subject, relation, object) triples with prompting. Basically, we directly add Graph: by the end of the paragraph context.
<Entity 1>
<Entity 2>
<Entity k>
To better instruct the model following the out- put format, utilize few-shot in-context learning (ICL) to prompt the LLMs (Brown et al., 2020). Specifically, we add four document--ntities pair as the demonstration.
Document:
Wikipedia Title:
<Page Title>
<Paragraph Text>
Graph:

---

We name the semantic graph generated through multi-step prompting "SG-Multi"; the semantic graph generated through the single step (joint ex- traction) "SG-One".
For the example question in Figure 1, the reasoning steps would be "Princess Anne was the daughter of Prince Jean. Prince Jean was the youngest child of Prince Robert. Prince Robert is the paternal grandfather of Princess Anne." One interesting dif- ference from our reasoning chain (as specified in the demonstration) to the default CoT (Wei et al., 2022) tasks' is that, under the multi-hop setting, we specific format for the multi-hop question reasoning process. More specifically, we define reasoning chain a list of short sentences in the correct logical order, and each sentence only represents relation between two entities. The set of sentences naturally leads to the answer entity, Under this format, the reasoning chain can be more consistent, facilitating more fair and automatic eval- uations.
To achieve a comprehensive understanding of how the graph structure affects the QA and rea- soning process. We also build fully connected version of the graph based all the entities ex- tracted in Step More specifically, suppose there are k extracted entities, there would be k²2 tuples, the graph would look like:
(<Entity 1>, <Entity 2>) (<Entity 1>, <Entity 3>)
(<Entity k-1>, <Entity k>)
Since the fully connected graph does not leverage semantic information, we call it "G-Full".
Finally, we post-process the generated <answer> by applying a regex expression extract answers from the <answer> after is generated - they might have a slightly different format or length to the real/gold answer. For the simple few-shot setting (non-CoT), we remove "Answer the question by ..." and "Reasoning Steps" from the few-shot examplevdeemonstatton...
3.3
The Overall Prompt for the Question Answering Task
For each question, we have multiple associated passages context. For each passage, seman- tic graph is generated as described above. Then, all the graphs combined together form the overall prompt template. Thus, the overall prompt template includes three components: documents, an extracted graph, and question prompt.
Experiments
4.1 Datasets and Configuration
The input part consists of multiple evidence doc- uments followed by their corresponding graphs. Then all of these structures are concatenated. Since our default setting is chain-of offthoughthh suggested by Trivedi al. (2022), add "Answer the fol- lowing question by reasoning step-by-step" ahead the real question.
We evaluate our model on two multi-hop question- answering datasets: HotpotQA (Yang et al., 2018) and 2WikiMuliHHpQA (Ho et al., 2020). Both datasets have ten candidate paragraphs for each question and originally have supporting facts. We use the golden paragraphs as context for all ques- tions. Following Trivedi et al. (2022), for each dataset, randomly sample 100 questions from the original development set and use it as our de- velopment set for tuning hyperparameters. We ran- domly sample another 500 questions from the de- velopment set as our test set.
Documents:
Mikipedia Title: <Page Iitle 1>
<Paragraph Text 1>
<graph 1>...
Wikipedia Title: <Page Title n>
We use the model GPT-3.53 for the entity, rela- tion, and graph extractions, well as the final QA task. For entity and graph extractions, we add four demonstrations, For the question answering, we add demonstrations, We set the maximum num- ber of generation tokens for the entity and graph extraction process to be 300. There no num- ber limitation generating the reasoning chain and
<Paragraph Text n>
<graph n>
Q: "Answer the question by reasoning step-by-step" <question> A:
Then the output part would be like below:
prttps://plie form..oppee com/ ccm/docs/ docs/ models httpss//pttt33 3-5
<Reasoning Steps>: <answer>

---

EM
F1
Precision
Recall
EM
F1
Precision
Recall
Methods
HotpotQA
2WikiMuliHHoPAA
Base
0.59
0.733
0.745
0.779
0.43
0.532
0.501
0.724
G-Full prompt
0.62
0.763
0.777
0.813
0.46
0.565
0.534
0.757
SG-Multi prompt
0.64
0.785
0.791
0.849
0.49
0.601
0.573
0.772
SG-One prompt
0.61
0.749
0.741
0.823
0.47
0.579
0.563
0.748
Table 1: CoT setting results. SG-Multi and SG-One generally outperforms G-Full and the base prompt. This demonstrates the importance of explicitly extract and encoding the semantic relations.
F1
Precision
Recall
F1
Precision
Recall
Methods
CoT Setting
Few-shot Setting
Base
0.716
0.729
0.830
0.6280
0.5970
0.7821
G-Full prompt
0.714
0.732
0.805
0.6065
0.5910
0.7802
SG-Multi prompt
0.728
0.731
0.872
0.5925
0.5596
0.8205
Table 2: Few-shot Setting Recall Results on HotpotQA.
answer span. But, the generation stops when changes line. The "temperature" parameter set to when generating all predictions.
goes up both the G-Full, SG-Multi, and SG-One settings. After manual analysis and comparisons to the Hotpot, we find that the graphs are smaller and of higher quality, which leads to a more faith- ful chain and final results. This will be discussed further in Section 4.3.
exact match (EM), F1, Precision, and Recall. We the official evalu- ation scripts of each dataset get the output scores.
Regarding metrics,
Further, we also present the recall performances for the few-shot setting (w/o CoT) Table 2. We see recall improved by adding the structures, es- pecially using SG-Multi for both HotPotQA and 2WikiHop. We will mention below why recall is a better metric for the multi-hop QA task when comparing prompting-basee methods.
4.2 Evaluation on the QA fask
We first run experiments on both datasets under the CoT setting. We report results in Table 1.
On HotpotQA, we observe that all prompting- based methods achieve decent performance, and by adding graph structures, across all four metrics. Especially, the "SG-Multi" prompt improves the recall by 4%. When the model generates reasoning chains and answers, SG en- ables the LLM to "hop through" the structured graph and context instead of inferring relations be- tween entities purely from context. The reason that the recall score goes down (from 0.83 to 0.81) by using G-Full is that it sometimes provides spurious implicit related information (i.e., all entity pairs). Then the sequentialized graph would be too long and doesn't 't fit into the LLMs' input length con- straint. Then is harder for the model to capture an entire picture of the relations needed to answer the question.
improvement
Recall, precision and other metries We argue that recall a better metric for this task among the four metrics under the generative QA setting, where LLMs are used to generate the reasoning chains and answers.
Different from extractive QA setting based fine-tuned discriminative models: (1) generated answer's length varies, both wider (lower preci- sion) or shorter (lower recall) answers can cor- rect; (2) LLMs sometimes generate tokens that are not shown/grounded to the original documents. They can be in longer-form (Fan et al., 2019) that includes tokens outside golden answers but don't 'tn't exactly match (EM). From the human's judg- ment, most of those predicted answers may be cor- rect. For example, the gold answer the question "Where was the father of Knut Hedemann born?" is "Stange". But "Stange, Norway" is also correct, which provides extra information. this specific
On 2WikiMultiHopOAA we see similar trend of improvements. The results of exact match, Fl, and precision increase substantially after adding the graphs. They are improved by 9%, 6%, and 6% on average as compared to the base prompt. Recall

---

example, the recall metric is more lenient and pro- vides the same score. But the precision is punished. Without fine-tuning, text generation is less control- lable than text span extraction. Thus, the precision metric is less suitable for the generated answers. Since Fl is affected by precision, it also punished slightly longer but correct answers.
based on the two requirements. During our study, our two annotators fully agree.
A Although reliable, human evaluation is expen- sive. Motivated by this, recent work has also inves- tigated automatic metrics for NL explanations eval- uation, including word overlap or embedding based similarly with human written explanations (Clinciu et al., 2021). We consider two candidate automatic metrics: ROUGE measures n-gram overlap be- tween generated and reference texts based on recall of important content. BERTScore (Zhang et al.) uses BERT embeddings to capture contextual infor- mation and computes a similarity score based on cosine similarity between generated and reference text embeddings. We first check the correlations between automatic metrics and human evaluations on whether the reasoning process is faithful not. For each metric, they provide a score between 0 to (with as the perfect match). We calculate Spear- man (p) and Kendall-Tau (r) correlations. We find that ROUGE is a slightly better metric for auto-eval (with being around 0.32 and т around 0.27), as compared to BERTScore (with p being around 0.24 and t around 0.20). Overall, this shows moderate agreement with human evaluations.
EM
F1
Prec
Recall
P
0.51
0.77
0.76
0.88
0.51
0.69
0.68
0.87
Table 3: Correlations of metrics and human judgments.
To empirically verify, we conduct a brief man- ual study. Firstly, we manually check if generated answers are correct/acceptable under human judg- ment. If we believe an answer is correct, we assign as its score; otherwise, O. Then we calculate both Spearman (p) and Kendall-Taa (T) correlation to scores provided by each metric. As shown in Table 3, recall corresponds more human evaluation. Apart from recall, according to Table 1, we find that precision gets slightly increased (with hurting recall) by adding the SG-Multi (or SG-Onc), es- pecially on the 2WikiMultiHopQA. As discussed previously, precision is highly influenced by the length of predicted answers. By adding graphs, the QA and reasoning process is more likely to refer entity-relation triples in the graphs rather than starting from scratch from the long documents. Ac- cording to our entity definition above and answer format, they are brief and informative. Thus, the nal predicted answers are sometimes more accurate, which helps improve the precision and Fl score.
On our sampled questions, Table 4 shows the performance of generated reasoning chains by each method based on corresponding prompts. We show both human and automatic evaluation scores. We see that based on strong LLMs, all three prompting- based methods achieve good scores, but the prompt augmented with rich semantic structured (SG- Multi) is of better quality. Below we look into examples and provide more fine-grained evalua- tion and analysis.
More manual evaluation and analysis We sam- ple and compare 100 pairs of reasoning chains generated by three prompting-based methods on the 2WikiMultiHopQA dataset. There are mainly three kinds of benefits/difffrences that our SG- augmented prompts bring,
4.3 Evaluation on the Reasoning Chain
Apart from evaluating the quality of answers, it is also important to evaluate free-form reasoning chains under our CoT setting. We sample 100 ques- tions in the 2WikiMultHHopQ4 dataset and manu- ally annotate their reasoning chains (explanations) according to the format defined in Section 3.3. For the predicted reasoning chains, we conduct both human and automatic evaluations. We make the human correctness judgments based solely on factuality and hallucinations/aaaittflness are also the two most important considerations in Golovneva et al. (2022). Human judges decide whether one reasoning process is "correct/good"
Improvement of the faithfulness of reasoning chains. In Figure 3, there is comparing ex- ample on the difference. Both reasoning chain starts with truthful facts (around two sentences). Later, the base prompt-based method generates Upper sing, which is either a city ("Grafelfing") or a state name ("Upper Bavaria"), which causes the error of the final answer as well.
They
*Coherence and grammar mistakes rarely occur.
Help the model navigate through and conduct

---

method
Human
ROUGE-1
ROUGE-2
ROUGE-L
Base Prompt
0.81
0.680
0.520
0.614
G-Full prompt
0.87
0.687
0.536
0.624
SG-Multi prompt
0.88
0.689
0.543
0.628
Table 4: Evaluations of Generated Reasoning Chains.
fected. Most of the time, the poor quality is caused by the incompleteness of the generated graph. Out of the 100 questions, about three examples exhibit such problems. Basically, the extracted SG is incomplete because of the output length limitation of the LLM. Thus the informa- tion is lost in the extracted graphs. This prompts the model to output "The graph does not provide information about the question." instead of the reasoning chain.
Wikipedia Title: Marcel (song)
"Marcel" was the German entry in the Eurovision Song Contest 1963, performed German by Heidi Brühl
Wikipedia Title: Heidi Brühl
Heidi Rosemarie Brühl (30 January 1942
Gräfelfing, Upper Bavaria - 8 June 1991 in
Starnberg) was German singer and actress [..]]
Graph:
("Marcel", performed by, Heidi Brühl)
(Heidi Rosemarie Brühl, was born in, Grätelfing) (Grätelfing, located in, Upper Bavaria)
To summarize the above, we find that generation of reasoning chains relies SG's quality. With better-quality graphs, both final answers and rea- soning chains can be more accurate.
Question:
What is the place of birth of the performer of song Marcel (Song)?
Reasoning chian (base prompt):
The performer of the song "Marcel" is Heidi Brühl. According to the Wikipedia page about Heidi Brühl, she was born Gräfelfing, Upper Sing in Germany. So the answer is: Germany Sing.
Further Qualitative Analysis
Error Analysis We further investigate errors made by SG-based framework. There four major categories: (1) lack of external commonsense knowledge (e.g. one's uncle is the brother of father), most the wrong answers are caused by this. (2) poor quality graph (limita- tion of input length); (3) other problems such as metaphors/ppraphrraing (e.g. relations between "rests" and "death"") and mathematical knowledge. (4) unanswerable questions based on the given context, which is problem of dataset creation. Around 5% of questions have this problem.
Reasoning Chain (SG-One prompt):
The performer of the song Marcel is Heidi Brühl. Heidi Rosemarie Brühl was bom on January 30, 1942, in Grafelfing, which is located in Upper Bavaria. So the answer is: Gräfelfing. Upper Bavaria.
Figure 3: Reasoning chain and answer generated by our framework are more faithful. The base prompt leads to "Germany Sing" which didn't appear in context.
more accurate reasoning (and know when to stop). For the example in Figure 4, in the ques- tion "Who is the paternal grandfather of Princess Anne of Orléans?", the reasoning chain gener- ated by the based prompt has "over-reasoning" phenomenon (conducting more hops of reason- ing). - generates a name that is the great-great- grandfather of Princess Anne of Orléans, "Louis Philippe I". While the reasoning chain gener- ated by inputting SG-Multi prompt, the reasoning process "stop" at the correct answer (i.e., "Prince Robert, Duke of Chartres").
More about Explamabitiy/Ittepprtaaiii extracted semantic graph naturally provides an explanation of the reasoning process. Current work mainly relies on the generated reasoning chain, which is NLG-bbased and contains hallu- cinations (Golovneva et al., 2022). Our SG is fully grounded to the input context, as shown in the upper part of Figure 4. The red parts are the extracted nodes, and the green parts denote the relations/edges. Apart from comparing to NLG- based (reasoning chains in our setting) explana- tions (Narang al., 2020). We also compare the prevalent saliency- saliency-based based method, which obtains explanations that are also grounded to the context
Our
When the quality of the extracted semantic graph is low, the reasoning chain's 's quality will be af-

---

Conclusion
Documents:
Wikipedia Title: Princess Anne of Orléans
Princess Anne of Orleans... She was the daughter of Prince Jean, Duke of Guise and Princess Isabelle of Oriaans.
We investigate how extracted semantic graphs can contribute to explainable multi-hop question an- swering. We propose prompting-based method that leverages the extracted and sequentialized graph into a prompt context. Through compre- hensive experiments from multiple aspects, we find that leveraging the SGs substantially im- proves LLMs' reasoning capabilities in answering multiple-hop questions - with more faithful reason- ing chains and better accuracy scores. Meanwhile, human studies show that the extracted graph, which itself is an interpretation that is strictly grounded to context, is preferred. As compared to the NLG- based explanations such as the generated reasoning chain and saliency- -bsse explanations.
Wikipedia Title: Prince Jean, Duke of Guise
Prince Jean of Orléans. Duke of Guise [..]]. was the third son and youngest child of Prince Robert, Duke Chartres (1840 1910). grandson of Prince Ferdinand Philippe and great-grandsse of Louis Philippe King of the French....
Graph:
(Princess Anne of Orléans, daughter of, Prince Jean) (Prince Jean, grandson of Prince Ferdinand
Philippe)
(Prince Jean, was, great-grandson of Louis Philippe
[..]
Question:
Who the paterall grandfather of Princess Anne Of Orléans?
Documents:
Wikipedia Title: Princess Anne of Orléans
Limitations
Princess Anne of Orleans... She was the daughter of Prince Jean, Duke of Guise and Princess Isabelle of Orléans.
Since framework adds extra step of extracting the semantic graphs, the total in- ference time of the large language model is longer, and the cost is more.
Wikipedia Title: Prince Jean, Duke of Guise Prince [...] youngest child of Prince Robert, Duke of Chartres(...] Philippe and great- grandson of Louis Phillppee1 [...]
Sometimes, the extracted semantic graph's quality unsatisfactory. It's mainly because of the limitation of context window size, which causes the graph to be incomplete. This affects the generation of reasoning chains well as answering the questions.
Question:
Who the patemnal grandfather of Princess Anne Orléans?
Figure 4: Our extracted triples are highlighted (up- per), which naturally provided grounded interpreta- tions. While the saliency- based method provides non- informative ones (bottom). Answers are underlined.
References
Akari Asai, Kazuma Hashimoto, Hannaneh Hajishirzi, Richard Socher, and Caiming Xiong. 2020. Learning to retrieve reasoning paths over wikipedia graph for question answering.
(bottom of Figurer 4). It's mainly done by using attention scores to highlight the most important spans/words in the context (Lei et al.. 2016). We conduct rigorous human preference studies. We invite a group of over four volunteers (with least bachelor's 's degrees) make pairwise comparisons explanations
Tom Brown, Benjamin Mann, Nick Ryder, Melanic Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, al. 2020. Language models are few-shot learners. Advances neural information processing systems, 13:1877-1901.
blind way. They are only pro- vided with ten questions, the corresponding con- texts, and the highlighted words in the context (that models deem as most important for answering ques- tions). Volunteers overwhelmingly vote for our SG-based extractive graphs/cxplanations most grounded and informative. Beyonds, results also show large preference for grounded explanations as compared to the reasoning chains, mainly be- cause they have to verify the facuulity/fuithullesss the generated explanations.
Wenhu Chen, Hanwen Zha, Zhiyu Chen, Wenhan Xiong- Hong Wang, and William Yang Wang. 2020. Hy- bridQA: dataset of multi-hop question answer- ing over tabular and textual (attayavuz-etal-202 modeling. In Findingy of the Association for Com- putational Linguistics: EMNLP 2020, pages 1026- 1036, Online. Association for Computational Linguis- tics.
Miruna-Adriana Clinciu, Arash Eshghi, and Helen Hastie, 2021. A study of automatic metrics for the

---

evaluation of natural language explanations. Pro- ceedings of the 16th Conference of the European Chapter of the Association for Computational Lin- guistics: Main Volume, pages 2376-2387, Online. Association for Computational Linguistics.
Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara. and Akiko Aizawa. 2023. Analyzing the effective- ness of the underlying reasoning tasks in multi-hop question answering. In Findings of the Associa- fion for Computational Linguistics: EACL 2023, pages 1163-1 180, Dubrovnik, Croatia. Association for Computational Linguisties.
Nicola De Cao, Wilker Aziz, and Ivan Titov. 2019. Question answering by reasoning across documents with graph convolutional networks. In Proceedings of the 2019 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers), pages 2306-2317, Minneapolis, Min- nesota. Association for Computational Linguistics.
Xanh Ho, Anh-Khou Duong Nguyen, Saku Sugawara. and Akiko Aizawa. 2020. Constructing A multi-hop QA dataset for comprchensive evaluation of reason- ing steps. In Proceedings of the 28th International Conference on Compulational Linguistics, COLING 2020, Barcelona, Spain (Online), December 8-13, 2020, pages (609-6625, International Committee on Computational Linguistics.
Zhenyun Deng, Yonghua Zhu, Yang Chen, Michael Wit- brock, and Patricia Riddle. 2022. Interpretable amr- based question decomposition for multi-hop question answering.
Tushar Khot, Harsh Trivedi, Matthew Finlayson, Yao Fu, Kyle Richardson, Peter Clark, and Ashish Sabharwal. 2023. Decomposed prompting: A modular approach for solving complex tasks.
Nouha Dziri, Ximing Lu, Melanie Sclar, Xiang Lorraine Li, Liwei Jian, Bill Yuchen Lin, Peter West, Chandra Bhagavatula, Ronan Le Bras, Jena D Hwang, et al. 2023. Faith and fate: Limits of transformers on com- positionality. arXiv preprint arXiv:2305.18654.
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yu- taka Matsuo, and Yusuke Iwasawa. 2023. Large lan- guage models are zero-shot reasoners.
Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2016. Rationalizing neural predictions. In Proceedings of the 2016 Conference Empirical Methods in Natu- ral Language Processing, pages 107-117.
Anthony Fader, Stephen Soderland, and Oren Etzioni.
2011. Identifying relations for open information ex- traction. In Proceedings of the 2011 conference on empirical methods natural language processing, pages 1545-1545.
Qi Li, Heng Ji, and L Liang Huang. 2013. Joint event extraction via structured prediction with global fea- tures. In Proceedings of the 51 Anmual Meeting of the Association for Computational Linguistics ( (Vol- ume 1: Long Pupers), pages 73-82, Sofia, Bulgaria. Association for Computational Linguisties.
Angela Fan, Yacine Jernite, Ethan Perez, David Grang- ier, Jason Weston, and Michael Auli. 2019. Eli5:
Long form question answering. In Proceedings of the 57th Annual Meeting of the Association for Com- putational Linguistics, pages 3558-333 5556-567.
Qing Lyu, Shreya Havaldar, Adam Stein, Li Zhang, Delip Rao, Eric Wong, Marianna Apidianaki, and Chris Callison-Burch. 2023. Faithful chain-of- thought reasoning. arXiv preprint arXiv:2301. 13379.
Yuwei Fang, Siqi Sun, Zhe Gan, Rohit Pillai, Shuo- hang Wang, and Jingjing Liu. 2020. Hierarchical graph network for multi-hop question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8823-8838, Online. Association for Computa- tional Linguistics.
Makoto Miwa and Mohit Bansal. 2016. End-to-end relation extraction using Istms on sequences and tree structures. In Proceedings of the 54th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages I 1105-1110 116.
Ruiliu Fu, Han Wang, Xuejun Zhang, Jun Zhou, and Yonghong Yan. 2021. Decomposing complex ques- tions makes multi-hop QA easier and more inter- pretable. In Findings of the Association for Compu- tational Linguistics: EMNLP 2021, pages 169-180. Punta Cana, Dominican Republic. Association for Computational Linguistics.
Sharan Narang, Colin Raffel, Katherine Lee, Adam Roberts, Noah Fiedel, and Karishma Malkan. 2020. Wt5?! training text-to-text models to explain their predictions. arXiv preprint arXiv:2004. 14546.
Ethan Perez, Patrick Lewis, Wen-tau Yih, Kyunghyun Cho, and Douwe Kiela. 2020. Unsupervised question decomposition for question answering. In Proceed- ings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8864 8880, Online. Association for Computational Linguistics.
Olga Golovneva, Moya Chen, Spencer Poff, Martin Corredor, Luke Zettlemoyer, Maryam Zarand,, and Asli Celikyilmaz. 2022. Roscoe: suite of metrics for scoring step-hy-step reasoning. arXiv preprint arXiv:2212.0799
Ralph Grishman. 2019.
Twenty-five years of infor- mation extraction. Natural Language Engineering, 2576)6677-692.
Peng Qi, Haejun Lee, Tg Sido, and Christopher Man- ning. 2021. Answering open-domain questions of varying reasoning steps from text. In Proceedings of

---

the 2021 Conference on Empirical Methods in Natu- ral Language Processing, pages 3599-3614, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Wein- berger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. In International Conference Learning Representations.
Lin Qiu, Yunxuan Xiao, Yanru Qu, Hao Zhou, Lei Li, Weinan Zhang, and Yong Yu. 2019. Dynamically fused graph network for multi-hop reasoning. In Proceedings of the 57th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 6140- 61500 Florence, Italy. Association for Computational Linguistics.
Haitian Sun, William W. Cohen, and Ruslan Salakhutdi- nov. 2021. Iterative hierarchical attention for answer- ing complex questions over long documents.
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. Interleav- ing retrieval with chain-of-thought reasoning for knowledge-intensivee multi-step questions. arXiv preprint arXiv:2212.10509
Ming Tu, Guangtao Wang, Jing Huang, Yun Tang, Xi- aodong He, and Bowen Zhou. 2019. Multi-hop read- ing comprehension across multiple documents by soning over heterogeneous graphs. Proceedings of the 57th Annual Meeting of the Association for Com- putational Linguistics, pages 7704-2713, Florence, Italy, Association for Computational Linguisties.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quac Lee, Fd Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-consistency improves chain of thought reasoning in language models.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed H Chi, Quoe Le. Denny Zhou, et al. 2022. Chain-of-ther of-thought prompting elicits rea- soning in large language models. In Advances in Neural Information Processing Systems.
Zhilin Yang. Peng Qi. Saizheng Zhang, Yoshua Ben- gio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. Hotpotqa: A dataset for diverse, explainable multi-hop -hop question answer- ing. In Proceedings of the 2018 Conference on Em- pirical Methods in Natural Language Processing, Brussels, Belgium, October 31 November 4, 2018, pages 2369-2388. Association for Computational Linguistics.
Deming Yc, Yankai Lin, Zhenghao Liu, Zhiyuan Liu, and Maosong Sun. 2019. Multi-paragraph reasoning with knowledge-enhance
graph neural network.
Ori Yoran, Tomer Wolfson, Ben Bogin, Uri Katz, Daniel Deutch, and Jonathan Berant. 2023, Answering questions by meta-reasoning of thought.
multiple chains
Dmitry Zelenko, Chinatsu Aone, and Anthony Richardella. 2003. Kernel methods for relation extraction. Journal of machine learning research, 1083-1


Reviews:
[{"review_id": "jRP61nbnRj", "reviewer": "Reviewer_ns5s", "summary": "", "questions": "Q1 Despite G-Full not utilizing semantic relations (edges) and connecting all entities indiscriminately, the results are remarkably similar to SG-multi. Why is this similarity observed, given the inclusion of potentially irrelevant data? Shouldn't the performance be significantly lower due to the infusion of ineffective information?\n\nQ2: How do the results change if we use only the graph and the question in the prompt?\n\nQ3: How do you address coreferences, pronouns, and same mentions? For instance, when two mentions refer to the same entity, what approach do you employ to handle such cases?\n\nQ4: Why does Table 2 only display the recall on hotpotQA? What about the results for other benchmarks and metrics to offer a more comprehensive evaluation also on the few-shot setting?\n\nQ5: In line 372, it is mentioned something about tunning the hyperparameters, but after that, it is not mentioned why and when this tuning happened. ", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your reviews. We address your doubts and answer your questions below:\n\n**W1: Different GPT3.5 variants may exhibit superior performance in different aspects.**\n- We tried different variants and there are no significant differences in terms of results. We will specify this in the final version.\n\n**W2: The size of semantic graphs is large, which impedes the applicability of the approach in scenarios where dealing with large-scale graphs is essential.**\n- The limitation of G-Full is mentioned in section 4.2. Long graphs may not fit into the LLM’s input length constraint. With high information retrieval quality, the number of relevant paragraphs will not be large. Thus, SG-Multi or SG-One will not be large and can fit into LLM’s input (>90%).\n\n**Q1: Shouldn't the performance be significantly lower due to the infusion of ineffective information?**\n- Since we use golden paragraphs, most graphs do not exceed the graph size upper limit. Reasoning chains are implicitly included. In the paper, we mentioned that some graphs of G-Full may not fit the LLM’s input size limitation. So, its performance is still slightly above baseline and is lower than the other two SG-based methods.\n\n**Q2: How do the results change if we only include the graph and the question in the prompt?**\n- If we only input graphs and questions, some details from the original text are ignored (e.g. one exact day). It leads to a worse performance. Documents contain details that graphs may not cover, which may affect the results.\n\n**Q3: How do you address coreferences, pronouns, and same mentions?**\n- According to our human evaluation and analysis on semantic graphs, co-referential entities can be implicitly recognized by LLMs (often with help from the SGs). \n\n**Q4: Why does Table 2 only display the recall on HotpotQA?**\n- Recall is the most important metric among all four metrics. We have discussed it in section 4.2. In this table, we show that the semantic graph-building method brings the most improvement in recall. Other metrics also describe the same trend as recall. However, their improvements are not as significant as on the recall metric.\n\n**Q5: In line 372, it is mentioned something about tunning the hyperparameters, but after that, it is not mentioned why and when this tuning happened.**\n- We follow the settings in the paper Trivedi et al., 2023$^1$. The parameters for generation are required by OpenAI API, such as temperature, top_p for sampling, etc. Most parameters are default values. The temperature for generation is set to 0 for diversity control.\n\n**Reference**:\n1. Trivedi, H., Balasubramanian, N., Khot, T., & Sabharwal, A. (2022). Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. ACL 2023", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "S1: Employing a straightforward yet highly efficient prompt engineering method to integrate data into the Language Model (LLM) during in-context learning, enhancing LLM performance in multi-hop reasoning tasks.\n\nS2: Assessing the quality of predicted CoT following the infusion of Graph information by incorporating human judgment.\n\nS3: Investigating the model prediction's explainability through a series of experiments involving both human evaluators and the models themselves.", "reasons_to_reject": "W1: The text states that GPT3.5 was employed for all experiments. Although we acknowledge that each variant of GPT3.5 possesses distinct capabilities, it is crucial to specify the exact model used. For instance, InstructGPT(DaVinci-003) exhibits superior performance in reasoning and instruction-based tasks, whereas Turing and DaVinci perform comparably in extraction tasks. If ChatGPT(Turing) were utilized for reasoning, DaVinci might yield better results or vice versa. Thus, clarifying the specific GPT3.5 variant utilized can significantly impact the outcomes and conclusions drawn from the experiments.\n\nW2: In the experiments of this paper, only the gold paragraph is utilized (line 372). However, as indicated, the size of the graph (SG-full) in certain instances becomes huge and cannot be accommodated within the prompt. Consequently, employing this method in the primary challenge, which involves considering all paragraphs, or in real-world scenarios, may prove impractical due to the substantial size of the constructed graphs, even for SG-multi and SG-one variants. This limitation may impede the applicability and feasibility of the approach in scenarios where dealing with large-scale graphs is essential.", "questions_to_authors": "Q1 Despite G-Full not utilizing semantic relations (edges) and connecting all entities indiscriminately, the results are remarkably similar to SG-multi. Why is this similarity observed, given the inclusion of potentially irrelevant data? Shouldn't the performance be significantly lower due to the infusion of ineffective information?\n\nQ2: How do the results change if we use only the graph and the question in the prompt?\n\nQ3: How do you address coreferences, pronouns, and same mentions? For instance, when two mentions refer to the same entity, what approach do you employ to handle such cases?\n\nQ4: Why does Table 2 only display the recall on hotpotQA? What about the results for other benchmarks and metrics to offer a more comprehensive evaluation also on the few-shot setting?\n\nQ5: In line 372, it is mentioned something about tunning the hyperparameters, but after that, it is not mentioned why and when this tuning happened. ", "paper_topic_and_main_contributions": "This paper focuses on using structured information for explainable multi-hop question answering and reasoning. It addresses the challenge neural models face, including large language models (LLMs), in performing multi-hop reasoning and providing well-grounded explanations. To address this issue, the paper proposes a novel framework that leverages extracted semantic structures to enhance the models' multi-hop reasoning capabilities, leading to improved task performance.\nThe main contributions of this work are as follows:\n\n1) Introducing the concept of semantic graphs and their application to enhance the models' multi-hop reasoning abilities, resulting in improved task performance.\n\n2) Demonstrating that incorporating semantic graphs in the prompting process leads to the generation of higher-quality and more accurate reasoning chains.\n\n3) Showing that semantic graphs-based explanations contribute to context-grounded interpretations of the predicting process, which human evaluators prefer.\n\nEmpirical results and human evaluations are provided to assess the effectiveness of the proposed approach, comparing it against alternative methods in terms of generating faithful reasoning chains and delivering grounded explanations.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "T1: The naming of CoT and Few-shot is confusing in this paper. While citing Wei,2022, in line 197 for CoT-setting, it's important to note that in this paper, CoT is included in the prompt. In contrast, your approach only asks the model to include CoT in the prediction. Both of your approaches (CoT-setting and few-shot setting) are actually few-shot prompting with additional graph information. To avoid confusion, consider using a different or clearer term that does not overlap with the CoT method from the original paper.\n\nT2: Maybe an example in line 235 can make it more clear. \n\nT3: In many cases, you have both G-Full and SG-Full (e.g., Captions and body of Table 1 and 2). Please only keep one of them."}}, {"review_id": "RAYZlGdJdB", "reviewer": "Reviewer_ke1q", "summary": "", "questions": "A. Instead of using the fully connected graph (SG-Full) to understand the effect of the semantic graph structure, did you try using a random subgraph of this graph? \n\nB. The fully connected graph (G-Full) does not contain relation information. Did you try dropping the relations/edges from the actual semantic graph and noting the drop in performance to understand the role of relations? In combination with A, this could tell you if the entities/nodes in the semantic graph are more important than a random set of entities.\n\nC. The evaluations are mostly done on the reasoning chains and how grounded they are in context. Can the generated SG itself be evaluated?\n\nD. It would be interesting to see an analysis of the error cases where the model was previously correct (w/o semantic graph), but not after. In such cases, what is the quality of the generated reasoning chain before and after?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your reviews. We address your doubts and answer your questions below:\n\n**W1: Does better reasoning faithfulness correlate with better model performance?**\n- Thanks for the good question. We focus on faithfulness, which is as important as performance. Graphs provide grounded structure information that helps improve the faithfulness of answers and reasoning chains. Based on our experiment, most of the time better reasoning faithfulness/correctness improves performance. – There is a very strong correlation (across our test examples).\n\n**Q1: How does using a random subgraph affect the performance?**\n- Random sub-graphs have a large chance of missing important triples. We have tried adding random SGs and run the method multiple times. The performance dropped dramatically.\n\n**Q2: Did you try dropping the relations/edges from the actual SG and noting the drop in performance to understand the role of relations?**\n- By dropping relations in SG-Multi, the performance drops and looks similar to the performance of G-Full. Relations help LLMs more precisely jump a hop from one entity to another entity.\n\n**Q3: Can the generated SG itself be evaluated**\n- We conduct the evaluation of semantic graphs based on the method in Jurafsky & Martin 2022$^1$. Currently, Semantic graphs are evaluated by humans. Human evaluators report over the 90% precision on triples in SG. \n\n**Q4: It would be interesting to see an analysis of the error cases where the model was previously correct (w/o semantic graph), but not after. In such cases, what is the quality of the generated reasoning chain before and after?**\n- In our observations based on the human analysis, most of the cases when adding SG leads to the error are because of the incomplete SG. – It leads to worse/unfaithful reasoning processes and incorrect answers. We will list several qualitative examples in the final version. \n\nThank you again for the insightful suggestions, we will report the findings/results in the final version.\n\n**Reference**\n1. Speech and Language Processing (3rd edition), Jurafsky & Martin 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper is well-written and the claims are strongly supported.\n- Producing reasoning chains that are grounded in context is essential and this work proves that by extensive human evaluation and finds that this can also improve model performance\n", "reasons_to_reject": "- It is unknown if models perform reasoning in a manner that is similar to humans. In other words, a model producing more faithful reasoning might not always lead to better model performance. It is important to understand why model performance improves and whether there are spurious correlations in the data that are responsible for the improved performance. ", "questions_to_authors": "A. Instead of using the fully connected graph (SG-Full) to understand the effect of the semantic graph structure, did you try using a random subgraph of this graph? \n\nB. The fully connected graph (G-Full) does not contain relation information. Did you try dropping the relations/edges from the actual semantic graph and noting the drop in performance to understand the role of relations? In combination with A, this could tell you if the entities/nodes in the semantic graph are more important than a random set of entities.\n\nC. The evaluations are mostly done on the reasoning chains and how grounded they are in context. Can the generated SG itself be evaluated?\n\nD. It would be interesting to see an analysis of the error cases where the model was previously correct (w/o semantic graph), but not after. In such cases, what is the quality of the generated reasoning chain before and after?", "paper_topic_and_main_contributions": "This work proposes a way to construct and leverage semantic graphs in the prompting setting to enhance multi-hop reasoning and improve reasoning chains. Results show that the framework introduced in this work generates more faithful reasoning chains and substantially improves the QA performance on two benchmark datasets. The extracted structures provide grounded explanations preferred by humans, as compared to the generated reasoning chains and saliency-based explanations.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "- Typos: 609, 583, 498\n- 534: it should be \"example comparing\"?\n- Fig 3: typo in reasoning \"chian\"\n"}, "extra_text": {}}, {"review_id": "TIQqHigY85", "reviewer": "Reviewer_3aEg", "summary": "", "questions": "1. Since the LLM is grounded to structured information during reasoning, I believe the performance of  IE is very important. Could you test the LLM in terms of IE capability? Such as evaluating some NER or relation extraction benchmarks.\n2. In line 87, the explanations generated by your method are grounded to the extracted information. How do you verify this? I did not find any experiments to support this claim. Your method can improve the performance of QA, but it does not mean that the model really grounded to the extracted information.", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your reviews. We address your doubts and answer your questions below:\n\n**W1: The size of the test set.**\n- We test it on a new set including 400 questions on both HotpotQA and 2WikiMultihopQA (same as Trivedi et al., 2022$^1$). The performance is consistent with what we showed in the paper. The trend is the same on all semantic graph-based approaches.\n- CoT setting results on 400 random questions on HotpotQA:\n\n    | Methods         | EM     | F1     | Precision | Recall |\n    |-----------------|--------|--------|-----------|--------|\n    | Base            | 0.5925 | 0.7327 | 0.7448    | 0.7791 |\n    | G-Full prompt   | 0.6175 | 0.7634 | 0.7768    | 0.8129 |\n    | SG-Multi prompt | 0.6375 | **0.7846** | 0.7912    | **0.8489** |\n\n- CoT setting results on 400 random questions on 2WikiMultihopQA:\n\n    | Methods         | EM     | F1     | Precision | Recall |\n    |-----------------|--------|--------|-----------|--------|\n    | Base            | 0.4250 | 0.5317 | 0.5009    | 0.7238 |\n    | G-Full prompt   | 0.4625 | 0.5650 | 0.5339    | 0.7564 |\n    | SG-Multi prompt | 0.4900 | **0.6005** | 0.5728    | **0.7723** |\n\n- We sample 100 new questions in the 2WikiMultihopQA dataset and manually annotate their reasoning chains according to the format defined in Section 3.3. The result is shown in the table:\n\n    | Method          | Human | ROUGE-1 | ROUGE-2 | ROUGE-L |\n    |-----------------|-------|---------|---------|---------|\n    | Base Prompt     | 0.81  | 0.680   | 0.520   | 0.614   |\n    | G-Full prompt   | 0.87  | 0.687   | 0.536   | 0.624   |\n    | SG-Multi prompt | 0.88  | 0.689   | 0.543   | 0.628   |\n\n**W2: The significance of performance improvement and the long prompt**\n- We also focus on the faithfulness of the reasoning process, which is arguably as important as the QA performance. The prompt including the SGs (although longer) provides grounded structure information that helps improve the faithfulness of the answer and reasoning chain.\n- We do the t-test on the recall and F1 metrics on samples we select from both HotpotQA and 2WikiMultihopQA to check whether the improvement is significant. The results show that F1 and recall are significantly better (with p-value < 0.05) on the SG-Multi prompt method.\n\n**W3: The effectiveness of extracted information from LLM is a concern due to the lack of IE outcomes.**\n- We conduct the evaluation of semantic graphs based on the method in Jurafsky & Martin 2022$^2$. Currently, Semantic graphs are evaluated by humans. Human evaluators report over the 90% precision on triples in SG. -- we will add this to the final version.\n\n**Q1. Could you test the LLM in terms of IE capability? Such as evaluating some NER or relation extraction benchmarks.**\n- Our work does not rely on close-domain IE (schema-based). Thus we do test on a close domain IE banchmark (e.g. CoNLL, ACE). The human evaluation shows that our open-IE based SG is good, i.e., achieving over 90% precision (as mentioned above). We will formally report this in the final version.\n\n**Q2: The explanations generated by the method are grounded to the extracted information. How do you verify this?**\n- In our work, explanations are semantic graphs but not reasoning chains. SGs are *extracted* from documents with the *generative* approach, which makes them fully grounded to **the document context**.\n- As for the generated reasoning chains/processes, we evaluate them separately and measure their faithfulness/correctness (Figure 3).\n\n\n**References:**\n1. Trivedi, H., Balasubramanian, N., Khot, T., & Sabharwal, A. (2022). Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. ACL 2023\n2. Jurafsky & Martin 2022. Speech and Language Processing (3rd edition).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The motivation of this paper is clear. That is, ask the model to pay attention to the structured information from input documents and make faithful reasoning.\n2. This method improves the quality of reasoning chains generated by the LLM.", "reasons_to_reject": "1. Although the methodology presented in this study seems to be interesting, the overall evaluation lacks rigor. The authors conducted experiments on two multi-hop question-answering datasets, each containing a mere 100 questions. Furthermore, the human evaluation was carried out with only 25 questions. Such a limited scope of evaluation is insufficient to derive reliable conclusions.\n2. The performance improvement is not significant. Is it worthwhile to consume a lot of computation to process such a long prompt for only a marginal improvement?\n3. As indicated in this paper, the primary contribution of this work is the demonstration that LLM grounded to structured information can enhance the reasoning process. Consequently, the performance of LLM concerning Information Extraction (IE) capability holds significant importance. Nevertheless, this article does not present the IE outcomes of LLM, raising concerns regarding the effectiveness of the extracted information in genuinely providing assistance.", "questions_to_authors": "1. Since the LLM is grounded to structured information during reasoning, I believe the performance of  IE is very important. Could you test the LLM in terms of IE capability? Such as evaluating some NER or relation extraction benchmarks.\n2. In line 87, the explanations generated by your method are grounded to the extracted information. How do you verify this? I did not find any experiments to support this claim. Your method can improve the performance of QA, but it does not mean that the model really grounded to the extracted information.", "paper_topic_and_main_contributions": "This paper leverages structured information like entities and relations to enhance LLM with better CoT capability for solving multi-hop QA tasks. This work prompts the LLM to additionally to extract the entities and their relations in the input documents. By grounding on the extracted structured information, the model can generate higher-quality and more faithful reasoning chains and hence improve the performance of multi-hop QA tasks.\nThe main contribution lies in the better prompt algorithm and NLP engineering experiments.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The work suggests to additionally feed semantic graphs that represent input documents as part of the prompt when answering multi-hop questions. The authors show that on two multi-hop QA datasets, feeding the semantic graphs helps the model to predict more faithful reasoning chains and more accurate answers. The work is sound as agreed by all reviewers (3,4,3) and relatively exciting (4,4,3). I also find it appealing as it introduces a simple way to use structure in LLMs with CoT reasoning, that shows some empirical benefit. Given the above, I think it should be accepted to the main conference or Findings.
Decision: Accept-Findings