Evidence-Enhanced Triplet Generation Framework for Hallucination Alleviation in Generative Question Answering

## Abstract

Abstract To
address the hallucination in generative question answering (GQA) where the answer can not be derived from the document, we propose a novel evidence-enhanced triplet generation framework,
EATQA, encouraging the model to
predict all the combinations of ⟨Question, Evidence, Answer⟩ triplet
by flipping the source pair and the target label
to understand their logical relationships, i.e.,
predict Answer(A), Question(Q), and Evidence(E) given a QE, EA, and QA
pairs, respectively. Furthermore, we bridge the distribution gap to distill the knowledge from evidence in inference stage. Our framework ensures the model to learn the logical relation between query, evidence and answer, which simultaneously improves the evidence generation and query answering. In this paper, we apply EATQA to LLama and it outperforms other LLMs-based methods and hallucination mitigation approaches on two challenging GQA benchmarks. Further analysis shows that our method not only keeps prior knowledge within LLM, but also mitigates hallucination and generates faithful answers.

## 1 Introduction

Large language models (LLM) represent a significant milestone in the development of general artificial intelligence Brown et al. (); Touvron et al. (); Chowdhery et al. (). While these
models have demonstrated unprecedented performance across various general tasks, they still face a series of challenges, including
issues such as hallucination Tonmoy et al. ()
and handling long contexts Jin et al. (). In document-based generative question answering (GQA) Lewis and Fan (), model may generate answers which are inconsistent with the document or mismatch the query, well known as hallucinations Gunjal et al. (); Liu et al. ().
Recent works utilize an external model to retrieve relevant information and agument the factuality of generation. However, the intrinsic discrepancy between retriever and LLM may introduce information which is only surface relevant but not helpful to answer the question Salemi and Zamani ().

Figure: Figure 1: One example from MultiRC dataset. Red denotes supporting evidence and green denotes misleading sentences.
Refer to caption: x1.png

To enhance the logical reasoning and avoid the misleading information, we highlight the supporting evidence of the answer in document-based QA. Instead of the retrieve-then-read pipeline, we let the LLM to tackle the generation of evidence and answer in a unified triplet generation framework, where each two of <question, evidence, answer> serve as the input in the respective instruction to generate the other. The evidence is utilized to reconstruct the question to ensure the model captures its logical relations to question and answer instead of superficial relevance.

We take an example in MultiRC Khashabi et al. () dataset in Figure 1. The question is “After the Osprey resumed flights how long did it take for the Air Force to begin using the aircraft?
” and the answer can not be extracted from a single sentence in the document. To answer this question, the model needs to find the multiple evidence “Osprey resumed flights in 2002”
and “Air Force began using Ospreys in 2008 after testing the aircraft in 2006”, and decide the answer is “4 years”. If the model is distracted by the incorrect evidence “Marines develop the aircraft in Iraq in 2007”, it will derive the wrong answer “5 years”. On the other hand, based on the correct evidence, the model can reconstruct the original question because the correct evidence contains the enough information. However, with the incorrect evidence, the reconstructed question is “How long did it take for the Marines to begin using the aircraft $\cdots$ ” which is not consistent with the original question. It shows the correct evidence is crucial for the question answering and the reconstruction of the question based on evidence and answer can reflect the correctness of the evidence.

To alleviate the hallucination and enhance the logical reasoning between the question, evidence and answers, we propose our Evidence enhAnced Triplet generation framework (EATQA), which includes three instruction tuning tasks to predict all the combinations of ⟨Question, Evidence, Answer⟩ triplet
by flipping the source pair and the target label
to understand their logical relationships, i.e.,
predict A(Answer), Q(Question), and E(Evidence) given a QE, EA, and QA
pairs, respectively. We close the QA distribution distance between evidence-aware and evidence-absent settings (distribution bridging) to distill the knowledge from evidence to mitigate the gap in inference stage when the evidence sentences can not be derived.

We conduct experiments in two widespread document-based GQA datasets with diverse answer types, MultiRC and QASPER, based on different sizes of LLMs. Compared with different sizes of the backbone model, our unified triplet generation framework shows significant improvement on the two datasets, becoming the new state-of-the-art. Further analysis demonstrates the ability of our approach to tackle longer document with more sentences. Moreover, we observe the positive correlation among the performance of 3 subtasks in the triplet generation framework, which shows the effectiveness of unifying the generation of the three parts with one LLM in the triplet generation framework.

We conclude our contributions as follows:
1. We highlight the evidence retrieval to alleviate hallucinations of LLM in GQA task. Instead of utilizing another LM as the retriever, which may introduce misleading information, we propose the unified triplet generation framework including three instruction tuning tasks to improve the logical reasoning ability of LLM for GQA task. Specifically, we train the LLM to generate the Answer(A), Question(Q), and Evidence(E) based on the information of QE, EA, and QA
pairs respectively. The distribution bridging is utilized to distill knowledge from evidence for inference stage.
2. We conduct experiments on two widespread document-based QA datasets MultiRC and QASPER with different sizes of LLM, and achieve the new state-of-the-art on both datasets. 3. Further experiments show the effectiveness of unified triplet generation framework on both evidence retrieval and question answering. Moreover, our method not only keeps the prior knowledge within LLM, but also mitigates the hallucination for questions beyond the internal knowledge.

## 2 Related Work

Generative question answering (GQA) aims to generate an abstractive answer rather than extract an answer to a given question
from provided passages Fan et al. (); Li et al. ().
Early works on GQA mostly tried to improve the faithfulness of the answer by investigating
reliable external knowledge sources or incorporating multiple information sources. Yin et al. ()
propose Neural Generative Question Answering, an end-to-end model that generates
answers to simple factoid questions based on the knowledge base, while Bi et al. () propose the
Knowledge-Enriched Answer Generator (KEAG) to generate a natural answer by integrating facts
from four different information sources, namely, questions, passages, vocabulary, and knowledge.

Recent works focus more on the conditional generation model.
Li et al. ()
propose Rationale-Enriched Answer Generator (REAG), in which they add an extraction task to
obtain the rationale for an answer at the encoding stage, and the decoder is expected to generate the answer based on both the extracted rationale and original input. Su et al. () propose a framework named RBG (read before generate), to jointly models answer generation with machine reading. They augment the generation model with fine-grained, answer-related salient information predicted by the MRC module, to enhance answer faithfulness. Such methods can exploit and utilize the information in the original input better, while they require the extra effort of building models to extract that information.
CAD Shi et al. () follows a contrastive
output distribution that amplifies the difference between
the output probabilities when a model is
used with and without context.
RHO Ji et al. () introduce
local and global knowledge-grounding techniques
into dialogue generation and further utilize a conversational
reasoning model to re-rank the generated
responses.

Our approach differs from these methods in 3 folds:
1. Our model does not need the discourse structure including paragraphs and sections, and focus on the sentence-level evidence retrieval. 2. We tackle the logical relationship between question, evidence and answer in the unified triplet generation framework with the vanilla LLM, which prevents the intrinsic discrepancy of external retriever. 3. We claim correct evidence with answers contains the enough information to reconstruct the question, and utilize this characteristic to alleviate the generation of relevant but not needed evidence.

## 3 Methodology

In this part, first we introduce the document-based GQA task and the model architecture of Triplet-QA. Then we present the unified triplet generation framework, which predicts all the combinations of ⟨Question, Evidence, Answer⟩ triplet
by flipping the source pair and the target label
to learn their logical relationships.
Specifically, our method contains 3 subtasks, answer-aware evidence retrieval, evidence-aware query answering and evidence-aware query restoration, which is shown in Figure 2 from upper to down.

Figure: Figure 2: Model overview of EATQA.
Refer to caption: x2.png

The motivation of triplet generation framework comes from that the posterior probability of question answering is positively proportional to the probability of evidence generation and question recovery by Bayesian formulation:

$$ $\displaystyle\mathbf{P_{M}}(a|q,e,d)=\frac{\mathbf{P_{M}}(a,q,e,d)}{\mathbf{P_ {M}}{(q,e,d)}}$ (1) $\displaystyle=\frac{\mathbf{P_{M}}(a,d)\mathbf{P_{M}}(e|a,d)\mathbf{P_{M}}(q|e ,a,d)}{\mathbf{P_{M}}{(q,e,d)}}$ $$

where $d$, $q$, $e$, $a$ denote the document, question, evidence and answer. It shows the accuracy of QA should be proportional to the accuracy of evidence extraction and question restoration. We assume the evidence sentences contain the sufficient information to reconstruct the question, i.e. $\mathbf{P_{M}}(q|e,a)$ = $\mathbf{P_{M}}(q|e,a,d)$.

As the basis of our framework, we explain the feasibility of our method. Take query restoration as an example: in Figure 1, only from the answer “4 years”, the model is hard to restore the query because there may be multiple sentences involving “4 years” in the document. However, given the evidence sentences which point to the key events “Osprey resumed flights” and “Air Force begin using the aircraft”, the model has derived the basic part of the query and it enables our query restoration module reasonable, which improves the model ability to arrange the information to restore the query.

### 3.1 Preliminary

Document-base GQA task aims to generate the answer of the natural text question based on the document including multiple sentences:

$$ $\displaystyle D=[s_{1},s_{2},\cdots,s_{n}]$ (2) $$

where $D$ denotes the document, $s_{i}$ denotes the i-th sentence in the document and $n$ denotes the number of sentences.
The model can be formulated as a function of

$$ $\mathbf{f_{M}(a)}=\prod_{i=1}^{n}P(a_{i}|a_{1},a_{2},\cdots,a_{i-1},\mathbf{q} ,\mathbf{D})$ (3) $$

where $n$ denotes the answer length, $q$ denotes the query and $a_{0}$ denotes the begin-of-speech (BOS) token.
Generally the answer has flexible forms which can not be directly extracted from the document.

### 3.2 Model Architecture

EATQA is built on the widespread LLM, LLama Touvron et al. () with a few additional
learnable parameters. we additionally adopt several trainable adapter
tokens $p=[p_{1},p_{2},\cdots,p_{N_{p}}]$ which are prepended
to the key and value of each self-attention layer,
where $N_{p}$ is the number of adapter tokens.
So the number of trainable parameters
of EATQA 7B is 4.5M, only 0.06% of total
parameters of LLama 7B. With such a few
trainable parameters, EATQA effectively preserves
LLMs’ prior knowledge and the casual reasoning ability to
understand the logical relations between the question, evidence and answer.
EATQA consists of three objectives: answer-aware evidence generation, evidence-enhanced query answering and evidence-aware query restoration.

#### 3.2.1 Answer-Aware Evidence Generation (QA->E)

In this part, we model the probability of supporting evidence extraction for the query-answer pair $\mathbf{P_{M}}(e|a,d)$. We design the instruction and ask the LLM to generate the evidence which supports the query and the corresponding answers. Therefore, the input to model is the instruction, source document, the query and the corresponding answer. The output of model is the supporting evidence. The specific instruction is “generate the relevant evidence from the document to answer the following question” and we insert the document, question and answers into the template in Figure 5.

As for the loss function, by Bayesian Formula Mises () we derive

$$ $\displaystyle\log(P(e,q))=\log\int P(e,q,a)d_{a}$ (4) $\displaystyle=\log\int q(a|e,q)\frac{P(e,q,a)}{q(a|e,q)}d_{a}$ $\displaystyle\geq\int q(a|e,q)\log(\frac{P(e,q,a)}{q(a|e,q)})d_{a}$ $\displaystyle=E_{q(a|e,q)}\log(\frac{P(e,q,a)}{q(a|e,q)})$ $\displaystyle=E_{q(a|e,q)}\log(\frac{P(a,q)P(e|a,q)}{q(a|e,q)})$ $\displaystyle=E_{q(a|e,q)}\log(P(e|a,q))+E_{q(a|e,q)}\log(\frac{P(a,q)}{q(a|e, q)})$ $\displaystyle=E_{q(a|e,q)}\log(P(e|a,q))-KL(P(a,q)||q(a|e,q))$ $$

where $q(a|e,q)$ denotes the probability of answer $a$ to the question $q$ holds based on the evidence $e$, which is produced by the same backbone in our method with specific prompt, $KL$ denotes Kullback-Leibler divergence Van Erven and Harremos (). To maximize the evidence extraction probability, we should maximize the probability of evidence supporting the question-answer pair $P(e|a,q)$ and minimize the distribution distance between question answering with or without evidence $KL(P(a,q)||q(a|e,q))$. Considering we can not get access to the golden evidence during inference stage, the second term $KL(P(a,q)||q(a|e,q))$, named as “distribution bridging”, narrows down the gap between training and inference for answer generation.
we utilize cross-entropy loss function to optimize the probability $P(e|a,q)$:

$$ $\displaystyle\mathcal{L}_{\mathrm{QAE}}$ $\displaystyle=-\log P(\mathbf{e}\mid\mathbf{D},\mathbf{q},\mathbf{a})$ (5) $\displaystyle=-\sum_{t=0}^{N_{e}-1}\log P\left(e_{t+1}\mid\mathbf{D},\mathbf{q },\mathbf{a},e_{\leq t}\right)$ $$

where $D$ denotes the document, $N_{e}$ denotes the length of the evidence, $P\left(e_{1}\mid\mathbf{D},\mathbf{q},\mathbf{a},e_{\leq 0}\right):=P\left(e_{
1}\mid\mathbf{D},\mathbf{q},\mathbf{a}\right)$.

#### 3.2.2 Evidence-Enhanced Question Answering (QE->A)

In this part, we ask the LLM to generate the answers based on the corresponding question and the relevant evidence. The instruction is “generate the correct answers for the following question based on the document and the evidence support the answers to the question.”, and we insert the instruction, document, question and evidence into the template in Figure 5, as inputs into the LLM.
The objective function formulated as:

$$ $\displaystyle\mathcal{L}_{\mathrm{seq}}$ $\displaystyle=-\log P(\mathbf{a}\mid\mathbf{D},\mathbf{q},\mathbf{e})$ (6) $\displaystyle=-\sum_{t=0}^{N_{a}-1}\log P\left(a_{t+1}\mid\mathbf{D},\mathbf{q },\mathbf{e},a_{\leq t}\right)$ $$

where $N_{a}$ denotes the length of the answers, $P\left(a_{1}\mid\mathbf{D},\mathbf{q},\mathbf{e},a_{\leq 0}\right):=P\left(a_{
1}\mid\mathbf{D},\mathbf{q},\mathbf{e}\right)$.
This task can be seen as the main task of EATQA and enables the model to derive the answers based on the question and evidence.
On the other hand, to narrow the gap between training and inference, we minimize second term of Eq.4: $KL(P(a,q)||q(a|e,q))$. When the evidence provided is incomplete or has misleading information, the model learns to resort to the original document for the answer, which improves the robustness of training stage.
Therefore, the loss function of this part is:

$$ $\displaystyle\mathcal{L}_{\mathrm{QEA}}$ $\displaystyle=\mathcal{L}_{\mathrm{Seq}}+\alpha_{kl}\cdot\mathbf{KL}(\mathbf{P }(a,q)||\mathbf{q}(a|e,q))$ (7) $$

where $\alpha_{kl}$ denotes the hyper-parameter to tune.

#### 3.2.3 Evidence-Aware Question Restoration (EA->Q)

In this part, we aim to model the probability of $\mathbf{P_{M}}(q|e,a)$ and ask the LLM to recover the question based on the evidence-answer pair. The prompt is “reconstruct the question based on the answers and corresponding supporting evidence”, and we insert the prompt, document, evidence and answers into the template in Figure 5.
The objective function is formulated as:

$$ $\displaystyle\mathcal{L}_{\mathrm{EAQ}}$ $\displaystyle=-\log P(\mathbf{q}\mid\mathbf{D},\mathbf{e},\mathbf{a})$ (8) $\displaystyle=-\sum_{t=0}^{N_{q}-1}\log P\left(q_{t+1}\mid\mathbf{D},\mathbf{e },\mathbf{a},q_{\leq t}\right)$ $$

where $N_{q}$ denotes the length of the question, $P\left(q_{1}\mid\mathbf{D},\mathbf{e},\mathbf{a},q_{\leq 0}\right):=P\left(q_{
1}\mid\mathbf{D},\mathbf{e},\mathbf{a}\right)$.
Considering the incorrect evidence does not contain the full information of the original question, this objective helps to enhance the casual relations between evidence and answers.

**Table 1: Results on MultiRC dataset compared with competitive LLM methods. “#Para.” denotes the parameter number in the model. GPT-3 reports few shot results with 32 examples in the prompt without parameter updating, and other methods are finetuning results from the original paper. We conduct 5 experiments with different random seeds and our method significantly beats the prior SOTA, with p-value less than 0.001.**
| Methods | EM | F1 | #Para. |
| --- | --- | --- | --- |
| GPT-3 (32 shot) | 30.5 | 75.4 | 175B |
| Flan-T5 | - | 83.4 | 137B |
| T5 | 63.1 | 88.1 | 11B |
| Vega-2 | 62.4 | 88.2 | 6B |
| ERNIE-3.0 | 63.2 | 88.6 | 10B |
| PALM | 63.6 | 88.7 | 540B |
| LLama2-7B | 57.2 | 86.1 | 7B |
| RAG | 58.1 | 86.7 | 7B |
| CAD | 58.2 | 87.2 | 7B |
| RHO | 59.4 | 87.3 | 7B |
| EAT-QA-7B | 61.9 | 88.1 | 7B |
| LLama2-13B | 62.0 | 87.9 | 13B |
| RAG | 63.1 | 88.1 | 13B |
| CAD | 63.5 | 88.3 | 13B |
| RHO | 64.2 | 88.4 | 13B |
| EATQA-13B | 65.5 | 89.8 | 13B |

**Table 2: Results on Qasper dataset compared with competitive LLM methods. “#Para.” denotes the parameter number in the model. The results are cited from the original paper. LLama2-7B-PI denotes using Position Interpolation with LLama2-7B.**
| Methods | F1 | #Para. |
| --- | --- | --- |
| VCC-3B | 38.2 | 3B |
| SE-Mistral-7B | 39.3 | 7B |
| TOVA-7B | 42.0 | 7B |
| ChatGLM3-6B-32k | 43.3 | 6B |
| LLama2-7B-PI | 42.4 | 7B |
| RAG | 43.9 | 13B |
| CAD | 43.1 | 13B |
| RHO | 43.2 | 13B |
| EATQA-7B | 45.1 | 7B |

### 3.3 Training and Inference

With the unified optimization of all three EATQA objectives, our model captures the logical relations between question, evidence and answers. The overall objective is the weighted accumulation:

$$ $L_{Triplet}=\alpha_{1}L_{QAE}+\alpha_{2}L_{QEA}+\alpha_{3}L_{EAQ}$ (9) $$

where $\alpha_{1}$, $\alpha_{2}$ and $\alpha_{3}$ are tuneable hyper-parameters.

Because of the design of distribution bridging, we do not need to first generate the evidence based on the question and then construct QE->A template in the inference stage. Instead, we directly instruct the model to generate the answer from the original document, which keeps the inference efficiency.

## 4 Experiments

**Table 3: Ablation results with LLama2 from 7B to 13B on MultiRC dataset.**
| Methods | EM | F1 | #Para. |
| --- | --- | --- | --- |
| w/ LLama2-7B |  |  |  |
| backbone | 57.2 | 86.1 | 7B |
| -Question Restoration | 60.2 | 87.1 | 7B |
| -Evidence Generation | 60.8 | 87.7 | 7B |
| -KL | 61.0 | 87.6 | 7B |
| EATQA-7B | 61.9 | 88.1 | 7B |
| w/ LLama2-13B |  |  |  |
| backbone | 62.0 | 87.9 | 13B |
| -Query Restoration | 64.2 | 88.7 | 13B |
| -Evidence Generation | 64.5 | 88.6 | 13B |
| -KL | 64.6 | 89.1 | 13B |
| EATQA-13B | 65.5 | 89.8 | 13B |

### 4.1 Datasets

We evaluate on two widespread benchmark GQA datasets, MultiRC Khashabi et al. () and QASPER Dasigi et al. (), across different
domains.
MultiRC creates multi-domain multi-hop questions, where documents across various domains are selected from multiple datasets.
Each instance consists of a document including about 15 sentences. All instances were constructed
such that it is not possible to answer a question
correctly without gathering information from multiple sentences.
Qasper includes 5049 questions over 1585 Natural Language Processing papers in the academic
research domain focusing on entire papers, which is designed to facilitate document-grounded, information-seeking QA. Qasper contains a variety of answer types,
including extractive, abstractive, yes/no, and unanswerable questions.

We utilize Exact Match (EM) and F1 scores Opitz and Burst () to evaluate our method. The F1 score measures the overlap of answer tokens between the predicted and ground-truth answer. EM is more strict which awards point if any of the annotated answers is generated exactly.

### 4.2 Implementation Details

We conduct experiments with LLama2 Touvron et al. () from 7B to 13B as the LLM. To reduce computation cost and keep prior knowledge in LLM, we use LoRA Hu et al. (), which freezes the pretrained model weights and injects trainable rank decomposition matrices into each
layer of the LLM. The parameters $\alpha_{1}$, $\alpha_{2}$, and $\alpha_{3}$ in Eq. 9 are tuned from [0.1, 0.3, 0.5, 0.7, 1.0], and set to 0.3, 1.0 and 0.3 in our method. We use AdamW as optimizer and the initial learning rate is set to 3e-5.
Because the maximum input length of LLama2 is 4096 and the average context length of QASPER is about 16K, we utilize position interpolation Chen et al. () to extend the context length to 32K.

### 4.3 Baselines

We compare our method with existing widespread LLMs including T5-11B Raffel et al. (), Flan-137B Wei et al. (), Vega2-6B Zhong et al. (), GPT-3 (few shot) Brown et al. (), LoRAMoE Dou et al. (), PaLM 540B Anil et al. () for MultiRC.
For Qasper, we compare our method with LLM-based long context methods, AttenWalker Nie et al. (), ChatGLM3-6B-32k Du et al. (), SE-Mistral-7B Jin et al. (), VCC-3B Zeng et al. () and TOVA-7B Oren et al. ()
For hallucination mitigation methods, we compare our approach against RAG with Dense Passage Retriever (DPR) Karpukhin et al. (), CAD Shi et al. (), RHO Ji et al. () using the same backbone.

### 4.4 Results

From Table 1, compared with the backbone, our method improves by 3.7 EM and 2.0 F1 on 7B-scale model as well as 2.3 EM and 1.5 F1 on 13B-scale model. It demonstrates the effectiveness of our evidence enhanced triplet generation framework on document based GQA. Moreover, our method with 13B parameters outperforms the 540B PaLM finetuning by 0.7 EM and 1.0 F1, becoming the new state-of-the-art. Our method with 7B-scale model has achieved the comparable performance on F1 with larger models like T5-xxl and ERNIE-3.0.

From Table 2
compared with the backbone, our method improves by 2.7 F1 on 7B-scale model. Qasper contains more rigorous samples and existing hallucination mitigation methods struggle to improve the performance. It demonstrates the effectiveness of our method on challenging long document QA.

## 5 Analysis

**Table 4: Prior knowledge mitigation and hallucination mitigation. $Y_{A|Q}$ denotes the answer generated based on the vanilla query by QA model, which reflects the prior knowledge of LLM. $Y_{A|Q,D}$ denotes the answer generated based on the query and document. $\hat{Y}$ denotes the golden answer.**
| Probability | LLama2 | EATQA |
| --- | --- | --- |
| $P(Y_{A|Q}=\hat{Y})$ | 34.8 | 37.1 |
| $P(Y_{A|Q,D}=\hat{Y}|Y_{A|Q}=\hat{Y})$ | 88.8 | 85.8 |
| $P(Y_{A|Q,D}=\hat{Y}|Y_{A|Q}\neq\hat{Y})$ | 48.7 | 52.2 |

Figure: Figure 3: Performance relevance between 3 modules in our method with 13B backbone. QEA denotes evidence-aware question answering, EAQ denotes evidence-grounded query restoration and QAE denotes answer-aware evidence retrieval.
Refer to caption: x3.png

**Table 5: Results on MultiRC dataset grouped by different document lengths. Groups are indexed by the ascending order of document length, i.e., Group 1 denotes cases in the percentile interval 0-0.25 of the full dataset and Group 4 denotes cases in the percentile interval 0.75-1.0. Therefore, groups 3 and 4 have longer documents than groups 1 and 2. “length” denotes the average document length in the specific percentile interval. We utilize F1 to evaluate the model performance.**
| Group | 1 | 2 | 3 | 4 |
| --- | --- | --- | --- | --- |
| Length | 379 | 486 | 587 | 726 |
| LLama2 | 88.3 | 90.7 | 82.9 | 87.8 |
| EATQA | 90.5 | 91.9 | 86.4 | 89.3 |

### 5.1 Ablation

In this part, we investigate the effectiveness of different modules in our method, including
QA->E, EA->Q and the distribution bridging.

##### Does question restoration matter?

In this ablation, we remove the module of question restoration and investigate its effect on question answering. In table 3, removing question restoration will drop 1.7 EM and 1.0 F1 with 7B model, as well as 1.3 EM and 1.1 F1 with 13B model. Considering the context is not inputted into model in the query restoration module, the model has to utilize the information in evidence to recover the question. This module enhances the ability to arrange multiple pieces of information in evidence sentences, and understand logical relation between query, answer and evidence for LLM, which shows the effectiveness for GQA.

##### Does evidence generation matter?

In this ablation, we remove the module of evidence generation and investigate its effect on GQA.
In Table 3, removing evidence restoration will drop 2.1 EM and 0.4 F1 with 7B model, as well as 1.0 EM and 1.2 F1 with 13B model. Evidence extraction encourages the model to reason for the supporting facts that entail the question-answer pair, which enhances the understanding of logical relation among query, answer and evidence. Removing evidence generation decreases the attention of model pays to the important facts in the document.

##### Should we narrow down the distance between P ⁢ ( a , q ) 𝑃 𝑎 𝑞 P(a,q) italic_P ( italic_a , italic_q ) and q ⁢ ( a | e , q ) 𝑞 conditional 𝑎 𝑒 𝑞 q(a|e,q) italic_q ( italic_a | italic_e , italic_q ) ?

In this ablation, we remove the KL-divergence loss in Eq.4 in training. In inference stage, we input the predicted evidence and the query to derive the answer.
In Table 3, removing KL loss will drop 0.9 EM and 0.5 F1 with 7B model, as well as 0.9 EM and 0.7 F1 with 13B model. Though keeping effective performance, the distribution bridging distills the knowledge of evidence and narrows down the gap between training and inference, avoiding first retrieving the evidence and then inputting the evidence alongside the query into model to reason for the answer.

**Table 6: Results on MultiRC dataset grouped by different sentence numbers in the document. Groups are indexed by the ascending order of sentence number. ‘number” denotes the average sentence number in the specific percentile interval. We utilize F1 to evaluate the model performance.**
| Group | 1 | 2 | 3 | 4 |
| --- | --- | --- | --- | --- |
| number | 10.8 | 13.5 | 16.0 | 18.3 |
| LLama2 | 87.5 | 85.1 | 85.8 | 89.0 |
| EATQA | 90.7 | 84.3 | 89.2 | 91.7 |

### 5.2 Different document lengths and sentence number

In this part, we investigate our performance on cases with different document lengths and sentence numbers comparing with the backbone.
We classify the MultiRC development set into 4 groups based on the document length and sentence number respectively and apply F1 to evaluate the performance of different models.

Figure: Figure 4: Attention weights about different layers with 13B backbone. The left graph denotes the attention weights of query to document and evidence in Evidence-Enhanced Question Answering stage; the right denotes the attention weights of generated query to evidence and answer in Evidence-Aware Question Restoration stage.
Refer to caption: x6.png

Generally, our model derives significant improvement over LLama2-13B in groups with different document lengths and sentence numbers. It demonstrates the effectiveness of our evidence enhanced triplet generation framework on document-based GQA. In Table 5, EATQA outperforms LLama2 by 3.5 and 1.5 F1 in groups 3 and 4, as well as 1.8 and 1.2 F1 in groups 1 and 2. In Table 6, EATQA outperforms LLama2 by 3.4 and 2.7 F1 in groups 3 and 4. Longer context brings the difficulty for model to capture important information about the query and derive the correct answer. Our method enhances the logical relation between evidence, query and answer, which mitigates the hallucination about distracting information in the document.

### 5.3 Performance on Evidence Generation

**Table 7: Performance on evidence generation in MultiRC dataset. We utilize token-level F1 score as the evaluation metric. “LLama2” denotes instructing the LLM to generate the evidence only. “Joint Decoding” denotes sequentially generating evidence and answer.**
| Model | 7B | 13B |
| --- | --- | --- |
| LLama2 | 59.8 | 62.7 |
| Joint decoding | 60.3 | 63.1 |
| EATQA | 63.4 | 65.6 |

Not only deriving effectiveness on GQA, our method also shows improvement on evidence generation. In Table 7, comparing with sequentially generating evidence and answer, our method outperforms by 3.1 on 7B and 2.5 F1 on 13B. It shows our model captures the logical relation of query, evidence and answer in the unified generation framework.

### 5.4 Hallucination Mitigation

Considering the prior knowledge within LLM, we observe for some “already-known” questions, the model can generate the correct answer without the document, such as “What is gravity’s role in space?”. We utilize $P(Y_{A|Q}=\hat{Y})$ to evaluate the internal knowledge of model. When the model can not generate the correct answer without the document, the model resorts to the document rather than internal knowledge. We utilize the probability of generating faithful answer based on the document (without hallucinations) $P(Y_{A|Q,D}=\hat{Y}|Y_{A|Q}\neq\hat{Y})$ to evaluate the hallucination mitigation of the model.
In Table 4, our model significantly mitigates the hallucination while keeping prior knowledge to solve the “already-known” questions.

### 5.5 Correlation between Different Modules

In this part, we investigate the correlation of model performance in query answering (QEA), evidence generation (QAE) and query restoration (EAQ) on data samples. To mitigate the bias of extreme sample, we classify the samples in development set into 50 groups with same size based on the QEA F1. We take the average F1 score of all samples in the group as its overall F1 score. We respectively draw the scatter plot of each pair of QEA, QAE, EAQ score versus the other and fit with linear function. In Figure 3. we find the QAE score and EAQ score are directly proportional to QEA score. In our triplet generation framework, with better performance in evidence generation and query restoration, the model derives better performance in query answering. This shows the effectiveness of our EATQA, which enhances the understanding of LLM
about logical relations between query, evidence and answer.

### 5.6 Attention Weights

In this part, we compute the average attention weights about query to document and evidence in the query answering task in respective layers of the transformer block. We conduct statistics on the development set of the MultiRC dataset with 13B model. In evidence-aware query answering, the model assigns about 2 times attention weights to evidence token than context token. It shows the evidence contains denser information to derive the answer. Our introduction of distribution bridging distills the abundant information in evidence to evidence-absent query answering in inference phrase. In the EA->Q, the token-average attention weights of generated query paid to evidence are comparable to answer texts. Considering the evidence contains more tokens than the answer, It shows the evidence sentences are crucial for the feasibility of EA->Q task.

## 6 Conclusion

In this paper, we propose the unified triplet generation framework including three instruction tuning tasks to improve the logical reasoning ability of LLM for GQA task.
We conduct experiments on two widespread document-based QA datasets MultiRC and QASPER with different sizes of LLM, and achieve the new state-of-the-art on both datasets.

## Limitations

In this paper, we propose the unified triplet generation framework to improve the logical reasoning ability of LLM for GQA task. We conduct experiments with different sizes of LLama and achieve the new SOTA. We will conduct experiments on other LLMs with different architectures in further research.

## Appendix A Appendix

Input templates of different modules in EATQA.

Figure: Figure 5: Input templates of EATQA.
Refer to caption: x8.png

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 3
presentation: 2
questions: See the weaknesses section.
soundness: 2
strengths: 1. The paper presents a comprehensive methodology and demonstrates a strong experimental setup. EATQA's effectiveness is validated across two benchmarks, MultiRC and QASPER, where it outperforms prior state-of-the-art models. The paper provides detailed comparisons with competitive LLMs, proving the reliability and effectiveness of the proposed method. Ablation studies further establish the significance of each component in the framework, such as the impact of removing evidence generation or query restoration on performance.
2. The authors provide a clear exposition of EATQA’s architecture and its underlying principles. The paper is well-organized, with clear definitions of the three primary tasks (evidence generation, question answering, and question restoration). Figures, such as the model overview and template instructions, aid in visualizing the complex relationships within the triplet generation framework. Additionally, the equations and methodological breakdown make it accessible to readers familiar with GQA and hallucination mitigation research.
summary: The paper proposes EATQA (Evidence-Enhanced Triplet Generation Framework), designed to reduce hallucinations in Generative Question Answering (GQA). EATQA leverages a structured approach by generating triplets of Question, Evidence, and Answer (QEA) and using these to reinforce logical consistency. The model is trained on three main tasks: evidence generation, question answering, and query restoration, which improve the alignment between evidence and answers. Tested on MultiRC and QASPER datasets, EATQA achieves state-of-the-art results, effectively reducing hallucination and enhancing answer fidelity by distilling knowledge directly from evidence during inference.
weaknesses: 1. Limited innovation: The paper's proposed three training losses lack technical depth, and this multi-task approach has already been proposed and used in many scenarios. Although there are improvements on two benchmarks, the method does not provide new insights or thoughts for the readers.
2. Insufficient baseline models: The discussion of baseline models for retrieval-enhanced methods in the paper is not comprehensive enough.
3. Limited generalizability: The paper does not conduct experiments on a broader range of datasets, making it difficult to demonstrate the method's generalizability, especially in scenarios where large models are fine-tuned, such as in different types of multi-hop QA scenarios like NQ, TQ, StrategyQA, and MusiQA.
4. Non-standard writing format: There are many citation format errors, images are not in vector format, and there are issues with the image formatting.

---

confidence: 3
contribution: 2
initial_rating: 6
presentation: 3
questions: How does the method perform on datasets without gold evidence annotations?
soundness: 3
strengths: The method is well-motivated and the paper is easy to follow. The experiments show the proposed method has great improvements.
summary: This paper proposes EATQA to address hallucination issues in GQA. It is an unified triplet generation approach that can capture logical relationships between question, evidence, and answer.
weaknesses: 1. The method is based on gold evidence annotations when training. It may limit its applicability to datasets without such annotations.

2. The improvement margins on some baselines, e.g., CAD and RHO, are relatively modest.

3. Is the computational costs and inference time comparison to baselines missing?

---

confidence: 3
contribution: 3
initial_rating: 5
presentation: 2
questions: 1. What was the value for a hyperparameter \alpha_{kl} and how did the authors fix it?
soundness: 2
strengths: The proposed triplet generation framework showed significant improvement on two widespread document-based GQA datasets, MultiRC and QASPER, yielding state-of-the-art performance on the datasets.
summary: The paper proposed an evidence-enhanced triplet generation framework, EATQA, to address a hallucination issue in generative question answering (GQA). The EATQA encourages the model to predict Answer (A), Question (Q), and Evidence (E), given QE, EA, and QA pairs, respectively. that is, all the combinations of ⟨Question, Evidence, Answer⟩. to understand their relationships. The paper applied it to LLama, that outperformed other LLM-based methods and hallucination mitigation approaches on two GQA benchmarks.
weaknesses: 1. First, the paper was not necessarily written in good English. It should receive a native check. Further, it is partly difficult to understand. The authors incorrectly used LaTeX cite commands, that makes the draft more difficult to read. It is better to check the whole draft more carefully again. 

2.  While the proposed framework could yield better performance in GQA tasks, the evaluation in hallucination alleviation was not necessarily thorough enough, that makes it difficult to judge whether the proposed framework is really good in the hallucination alleviation.  The analysis in Sec. 5.4 did not necessarily directly evaluate the degree of hallucination alleviation. Furthermore, no comparisons with previous related work were shown. It is better to show how well the proposed framework can alleviate hallucination directly and clearly, in comparison with related work.

3. In the analysis in Sec. 5.3, no explanation was provided for the performance in Table 6. If it is the evaluation for generated evidences, how reference evidences can be obtained because it was mentioned that evidence annotation is unavailable in the datasets? it is also not described how the scores were calculated. 

4. The analysis in Sec. 5.2 seems to contribute to fewer useful findings. In my understanding, since the document length is proportional to the number of sentences, just a table might be enough from Tables 4 and 5.

5. It is better to clearly describe how the authors fixed hyperparameters in the experiments.

Meta-review: The paper introduces a method designed to alleviate hallucinations in Generative Question Answering by employing a structured approach. This involves the creation of triplets consisting of a Question, Evidence, and Answer, which are used to enhance logical consistency.

While the authors have addressed several points raised during the review process, significant concerns remain unresolved after the rebuttal:

- Limited novelty and technical depth: The proposed contributions rely on training loss functions and multi-task learning approaches. These techniques has been proposed and applied in various scenarios. As such, the method lacks fresh insights or contributions that would advance the field.

- Clarity and writing issues: Two reviewers noted that the paper is partially difficult to follow. More critically, it contains errors in writing format, including improper citation formatting and image formatting.

Given these major unresolved issues, we all agree that the submission does not currently meet the standards required for acceptance at ICLR. We hope the feedbacks provided will help the authors improve the paper in future revisions.

Decision: Reject