Search Augmented Instruction Learning
Authors: Hongyin Luo
Tianhua Zhang
Yung-Sung Chuang
Yuan Gong
Yoon Kim
Xixin Wu
Helen M. Meng
James R. Glass
Venue: EMNLP 2023 Findings (2023)
Keywords: large language model
instruction tuning
question answering

Abstract: Large language models (LLMs) have been significantly improved by instruction fine-tuning, but still lack transparency and the ability to utilize up-to-date knowledge and information. In this work, we propose search-augmented instruction learning (SAIL), which grounds the language generation and instruction following abilities on complex search results generated by in-house and external search engines. With an instruction tuning corpus, we collect search results for each training case from different search APIs and domains, and construct a new search-grounded training set containing (instruction, grounding information, response) triplets. We then fine-tune the LLaMA-7B model on the constructed training set. Since the collected results contain unrelated and disputing languages, the model needs to learn to ground on trustworthy search results, filter out distracting passages, and generate the target response. The search result-denoising process entails explicit trustworthy information selection and multi-hop reasoning, since the retrieved passages might be informative but not contain the instruction-following answer. Experiments show that the fine-tuned SAIL-7B model has a strong instruction-following ability, and it performs significantly better on transparency-sensitive tasks, including open-ended question answering and fact checking.

Search Augmented Instruction Learning
Hongyin Luo Tianhua Zhang? Yung-Sung Chuang' Yuan Gong'
Yoon Kim' Xixin Wu Helen Meng² James Glass'
MIT Computer Science and Artificial Intelligence Lab, Cambridge MA, USA
2 CUHK Centre for Perceptual and Interactive Intelligence, Hong Kong SAR, China hyluoêmit. edu, thzhangélinds cuhk. edu. hk
Abstract
significantly outperform plain LLMs on zero-shot language tasks (Peng al., 2023). Instruction- tuned LLMs have shown an ability to generate both natural and programming languages following nat- ural language guidance and requests. To achieve the same goal, pretrained LLM needs number of annotated examples in-context learning prompts.
It is widely believed that connecting large lan- guage models with search engines can improve their transparency, truthfulness, and accessing up-to-date information. However, show that search grounding introduces new chal- lenges to language models because of distract- ing, misleading, and untrustworthy informa- tion. To deal with these difficulties, we propose scarch-augmented instruction learning (SAIL), which allows fine-tuned language model to source, denoise, and reason based on a mixed set of helpful and distracting search results. With instruction tuning corpus, we collect search results for each training case from dif- ferent search APIs and domains, and construct a new search-ge grronnddd training set containing (instruction, grounding information, response) triplets. We then fine-tune the LLaMA-7B model on the constructed training set. Since the col- lected search results contain distracting and dis- puting languages, the model needs to learn to ground on trustworthy search results, filter out distracting passages, and generate the target sponse. The search result-denoising process en- tails explicit trustworthy information selection and multi-hop reasoning, since the retrieved passages might be informative but not contain the instruction-followwe
Despite their impressive behavior, LLMss have a number of issues, including obsolescence and non- transparency. Understandably. LLMs are trained with corpoгa constructed up to a certain time point. With this fixed, pretrained fine-tuned model, sub- sequently occurring information cannot appear in any informed generation by the LLM. One way update the knowledge in LLMs is to re-train the entire model with an updated training corpus. How- ever, this would be costly and time-consuming.
In terms of transparency, the predictions of LLMs are opaque because generations are not grounded on trustworthy sources. It is possible for an LLM to generate undesirable language that looks like human gennaa-eee text, including misin- formation, stereotypes, and toxic language (Zhang et al., 2023; Hartvigsen et al., 2022). Without pro- viding legitimate sources for LLM generated texts, difficult to catch and avoid these undesirable LLM behaviors.
answer. Experiments show that the fine-tuned SAIL-7B model has a strong instruction- follwwiinn ability, and performs significantly better transparency- sensitive tasks. including question answering and fact checking-
To overcome these difficulties, straightfor- ward solution is connect LLMs information retrieval systems, especially commercial search engines. By doing so, the LLM can ground its predictions on information retrieved from an up- to-date knowledge base, and the sources of the generations would be transparent to users. Be- fore LLMs became large enough to memorize a significant amount of world knowledge, retrieval- based grounding had been heavily studied for open- domain question answering (Chen al., 2017; Kwiatkowski al., 2019; Guu et al., 2020). Recent LLMs have also shown the potential of using in- formation retrieval tools, e.g., Toolformer (Schick
1 Introduction
Large language models (LLMs) have demonstrated many impressive capabilities, including zero-shot inference and few-shot in-context learning (Wei et al., 2022a). Recent research has shown that LLMs benefit from instruction tuning (Ouyang et al., 2022), and that such instruction-tunm tuned LLMs
"qqual contribution. Code and processed data are avail- able at https://giihhe com/ http:/lccooom//o////// wuwhongyin/SAtT SAIL.

---

Fact check following claim:
"However the worming tend
climate models
Label: UNFACTUAL
G
and Distracting
BAIL-7B
The clasi in potentially factual but needs contest, It factuul that some climate modells have indeed farecast faster rase of warming than has boen observed. Hawever without the contest that models are generally ly accurate in long torm projections and that their primary purpose is to precict frenns toy---tyyye yoar
(1) From climate change 'certainity' to rapid decinee timeline IPCC
Search recut distrctiig search resuit informatioe, search (i) distacttgg wil utilze the informative search nerlt and knnwweedgg the retereed nuttural last that global temparaturrs hase been increasing over time, the morai fact is that is important strvve tawards a healthy planet for future gonerations.
The lourth IPCC report. 2007 wie the momentt when humanity's resporsiiiiit global heating became of climate system ls unoquioocal Eleven of
certain: "Warming
rather than speciilo hempereune,, the cleim courd mideossing The dleim fai: it doss not contein apeech or stensotydes.
planet could degree
the claim nott factual as it contradicts the fact that the warming trend has been faxste tham predicted by many climate models. It thweefore the cleim presents an inacourate and mi misleading siew of the facts.
warming threshold mid
global
ithrsih
madels
prociltted concerted climate actioo,
UPT-3.6-Turbo
mashine...
The claim is factuaal and fal:. There is selentific ov oviderco no sugpesting i That the actual rate worming has bech slowor than what somo cllmate models have Inrecasted. While cllmate mocelss provide va uable insights and projections, there can be varistion between modell precketionss and olruerved chets. The search nesults caticate that the nate warming been topic study.... ...
Perplexity.A
(3) Constrained CMIPS projections indicate lexs warming and is wlowerr-
The diamm that "..." la partially factual based on the search results, Climate models published cince 1973 have ponerally boen cuite skd projecting future warming. while some were too los and some too high. they all show cutcomes reasonably close to what hes artiatly ly ocnumed.
slower warming impl lower snow cover losa rate by (0..-%0.%%. predicted variabto y.. model
future changes
intercoomparion project phose global climate mode using....
Fact checking grounding on complicated search results with SAIL-7B and strong commercial language models. The first and third passages are distracting since they do not contain information that supports or refutes the claim. And the first result 2007 report seems be obsolete answering climate-related questions usually needs up-to-date information. In contrast, the second passage disagrees with the claim by the statement "sooner than previous models have predicted", Although we acknowledge the complexity introduced by the term "most" in the claim, we contend that the second passage as most suitable. SAIL-7b successfully makes the the correct prediction while other commercial L.LMs are distracted.
Figure I:
al., 2023) and the ChatGPT (OpenAI, 2022) re- trieval plugin. However, there remains a challenge: is there a trustworthy retrieval model and knowl- edge base that can be utilized by LLMs?
results can mislead the model and negatively influ- ence the model's performance. As shown in Figure 1, ChatGPT is confused by a distracting passage and generates an incorrect fact check.
Existing studies on open-domain question an- swering have chosen Wikipedia as the de facto knowledge base that contains the answer to most questions. However, Zhang et al. (2023) found that the knowledge contained in Wikipedia not suf- ficiently up-to-date nor complete for many tasks that require the latest knowledge, so grounding on Wikipedia might lead to worse answers than fully relying on LLMs. Another option is to leverage internet search engines, for example, Google, Bing. and DuckDuckGo.com'
The challenges mentioned above are contradic- tory, and both have a negative impact on grounded language modeling with current LLMs static knowledge bases and in-house retrievers are not suf- ficient or up-to-date for all tasks, while commercial search engines often generate distracting results. To address these challenges simultaneously, we propose a search-augmented instruction learning (SAIL) model. Given input instructions and con- texts, the model is trained to generate high-quality responses according to the instruction grounding on the noisy research results. In other words, the model learns to denoise the retrieval results to gen- erate high-quality responses.
Although widely used commercial search en- gines can index and retrieve a vast range of up- to-date information, their retrieval accuracy is ulti- mately limited, and third-party users cannot control the performance at the model level. As result, retrieval results can be noisy, and unrelated infor- mation might be shown to users. This behavior suggests that there is trade-off between deploying in-house retrieval systems and external search en- gines. Although it is possible to prompt LLMs to directly use the retrieval results, distracting search
In summary, we make the following contribu- tions in this work:
1.
We show that instruction- LLMs can be heavily misled by distracting grounding infor- mation and noisy search results.
2. We constructed eearchaaummenee instruc- tion training corpus.
3. We fine-tune a 7B- 7B-parameter language model (SAIL-7B) with the constructed training set,
'A free, presevvin.., zero-tracking search engine.

---

which outperforms strong baseline models in- cluding 3PT-3.5-Turbo and Vicuna- on several NLP tasks.
However, the trustworthiness of each search re- sult is not labeled, and the number of retrieval items large. To bypass the need of costly human annota- tion, we employ an entailment classification model proposed in (Luo and Glass, 2023), which has in general been proven to be an effective approach for zero-shot setting (Obamuyide and Vlachos, 2018; Condoravdi et al., 2003; Ge et al., 2023). We feed each retrieved passage and the corresponding sponse into the entailment model and compare the entailed and contradietory scores. While most pre- dictions are neutral against the response, the rela- tion between entailed and contradictory scores can roughly indicate if retrieved passage can provide useful information to generate the target response. As result, we obtain pseudo-label "search result (i) is informative" if the entailed score is higher than the contradiction score, otherwise the search item is distracting. Note that our primary objective is not the construction of human-labeled dataset comprising informative and distracting documents. Instead, we aim at proposing a label-free denois- ing method to enhance retrieval- -augmented large language models.
By comparing the SAIL-7B model with LLaMA-7B, Vicuna-7B,
and Vicuna- 1 models on instruction following, question answer- ing, and language checking tasks, we find that the SAIL-7B model has a strong instruction following ability and robust against distracting grounding search results generated by different retrieval mod- els. In addition, the SAIL model also achieves com- parable performance to state-of-the-an instruction- following LLMs.
2 Method
2.1
Search Result Collection
In this work, we use the 52k self-instruction corpus created by the Alpaca team (Taori et al., 2023), and the corresponding responses generated by GPT-4 (Peng et al., 2023). For each instruction, we con- struct a search query by simply concatenating the instruction and the imput, if any, and truncating the query to at most 60 words to fulfill the limitation of the search engine.
Unlike the training stage, the absence of prior ac- cess to the target responses hinders the entailment model to predict pseudo-labels during inference. On the contrary, SAIL-7b acquires the capability to assess the relevance of various search results after training on both search results with pseudo- labels and the target responses. Subsequently. it can anchor the final response generation on informa- tive search- aurrch--uumetatioonn. In other words, SAIL-7b would generate the search selection sequences (c.g.. "search resull (i) is informative / distracting") before the final responses as shown in Figure 1.
The constructed queries are fed into the Duck- DuckGo search engine and the BM25 Wikipedia retriever, and the top three search results are re- tained. Each result consists of three fields: the title, a short picce of preview text, and the corresponding URL of the webpage. For simplicity, we do not further scrape the retrieved webpage, but just use the title and preview texts for further processing. Each training example is assigned list of cor- responding search results. We pool the top-three DuckDuckGO and top-two BM25 search passages with Pyserini (Lin et al., 2021), total of five search results. Among this pool, we randomly sample zero, one, two, and three search results with 20%, 20%, 20%, and 40% probability. Given this ran- domness, some training cases could be associated with search results from single source.
2.3 Fine-tuning
After collecting the search results and generating in- context retrieval selection sequences, we construct input prompts following Figure 2 (b) with GPT-4 generated responses (Peng et al., 2023). Note that the most relevant retrieval result is located the closest position the instruction for the model better use its information. We fine-tune LLaMA-7b models with the constructed prompts to generate both in-context retrieval selection sequences and annotated responses.
2.2 In-context Retrieval Selection
To encourage the LLM to focus on trustworthy and informative search results, we concatenate search filtering sequence before each annotated response of training instances. For example, "Search result (1) is informative and search result (2) is distract- ing, so will use the information from the search result (1)."
In practice, the models are fine-tuned with aca- demic devices. Specifically, we use 4 x NVIDIA RTX A6000 GPUs (48GB x 4) to train the models

---

(a) Standard Prompt
included in any training corpus as we finished all experiments. We name the new question set as New-Questions- 80 Because related information and knowledge are not included in the pretraining corpora, a language model has zero knowledge to answer and has to be grounded on an up-to-date search engine to generate informed answers.
Bolou is an Instruction that describes task. Mrite response that appropriately completes the request.
#c* Instruction:
[Instruction]
*** Inpurt IInput None]
Response:
(b) Search-Augmented Prompt
We use the evaluation prompt authored by the Vicuna team*. The highest possible score is 80 x 10 800. It is worth noting that GPT-4 responses can receive slightly different scores against different counterparts. To normalize the difference, we calculate the ratio of model score / GPT-4 score for each test case as the final assessment as implemented in Peng et al. (2023).
Below is an instruction that describes task.
Mrite a response that appropristely completes the request.
*еа Retated Infornation:
[Tatle 31k [Preview 3]
[Tatle 2J [Preview 2]
[Titte 1JM
[Preview
#cp Instruction:
[Instruction]
Inputt [Input None]
#Cо Response:
Figure 2: Different prompting strategies used in this work. (a) Standard prompt: the prompt template used in Peng et al. (2023) to generate GPT-4 responses to the 52k instructions. (b) Bearch-augmented prompt: com- bining the top three search results and the instruction.
SAIL for Question Answering, Besides evaluat- ing the quality of instruction-guideed generations, we assess the model's ability to answer com- monsense questions. We also test the models on two different settings, including instructed zero-shot prediction and the search-augmentation mode. We evaluate the model performance on Commonsense (CSQA; Talmor et al. (2019)), OpenbookQA (OBQA; Mihaylov et al. (2018)), and ARC-Challenge (Clark et al., 2018) bench- marks. All tasks require answering open-ended questions by selecting from a given set of candi- date answers. Through the question-answering answering experiments, we show that instruction-tume tuned language models can be significantly biased by noisy research results.
for 3 epochs. We apply mixed-ppecistie training (fp16) with the standard AdamW optimizer. We set the maximum sequence length as 1,600 and the batch size as 32, Following Vicuna, we apply gradi- ent checkpointing to reduce the memory cost, The entire fine-tuning process takes 24 hours (24 x GPU hours). To enable the fine-tuning, we applied gradient offload with Deepspeed and full-sharded data parallel (FSDP) (Paszke et al., 2019).
2.4 Evaluation
SAIL for instruction following. Following Peng et al. (2023), we evaluate the instruction follow- ing quality of different models by comparing with GPT-4 responses on the same set of instructions and scoring with GPT-4.
SAIL for Fact and Fairness Checking. With the recent advances in LLMs that generate human-like languages without guaranteed alignment, human and machine-generann types, and toxicity have become timely and signifi- cant concerns. Recent studies have shown that with appropriate instructions and prompts, LLMs can perform unified fact and faimess checking (Zhang et al., 2023). However, other attempts have relied only on LLMs, without grounding on any exter- nal sources, thus reducing the trustworthiness and transparency of the checking results.
misinformation, stereo-
For each case, we construct an evaluation prompt by concatenating the instruction, the GPT-4 re- sponse, and the response of the target model. We feed the evaluation prompt to GPT-4 and ask it to score the two responses between 0 to 10. We use the Question-80 corpus (Chiang et al., 2023), which contains 80 questions to evaluate all mod- els and we calculate the total score model re- ceives on all questions. To test the ability of the models on latest, unseen texts, we build another 80-question evaluation set based on latest news ar- ticles published after May 2023, which are never
In this work, we evaluate instructed fact and fairness checking, with the UniLC benchmark
proposed in (Zhang et al., 2023), including Climate-Fever, PubHealth, Hate Speech Detec-
'Will release with code accepted. "https: //gi::///tttt thub. com/lm stmmsss/fffaasss hyss///sChaat/////esss hatthhatthbl///mann////aa/ hwwirhaa//eal/taaeeeeeeee jsonl
Phttps://githhh: www/cmmmmmmmocemeee.. stsalnnssss///aaaaaaaaaa bbbbt//abbb/b/////////////// bastchat/eval/table/qeee jsanl

---

SAL Suarhh
SAIL Search
572
oPT4 Search
724 724
100%s
Yore (T8, Searcio
635:632
100%
OPTO Shimm Swerth
CPT4
15%
003:3333
Car-a Gaarch
100%
Voura 1230 Serrch
M8812204
94%
0m 5Tube
woee (1112
Woel 2006
89%
1996
LUP.OTTT 1721
595 7064
54%
5A, 770
150:18532
19%
OPF TT D4
ULAA Cffe
Vinun Saarch
Figure 3: Scoring results of all language models on Question-80 and New-Question-BB benchmarks against GPT-4 and GPT-3. 5-Turbo. Search indicates generating responses with language models grounding search results retrieved by DuckDuck6O, and SAIL (7B) stands for generating responses without search results, although the model is trained for grounded generations. Both Vicuna-7 & 13B are version 1.1 models. The orange bars stand for SAIL-7B and blue bars stand for GPT-4 and 3-TTubb...
tion, and Social Biase Frame (SBIC) tasks with two different settings zero-shot and search- augmented. While we are not aware of what corpora are used to train GPT-4 and Chat- GPT, we assess the language perfor- mance of Vicuna-7B-v1.. 1, Vicuna-138-vi..1 and SAIL-7B with and without search results.
achieves 103% of 5-TTubo performance and the no-augmentation SAIL model achieves 98%, outperforming several strong basclines, in- cluding LLaMA tuned on GPT-4 instructions and Vicuna models with the same number of parame- ters. Besides GPT-4, search-augmented SAIL-7B is the only model that outperforms GPT-3.5 5urub on both experiments.
3 Experiments
In addition, we found that the search aug- mentation makes a significantly higher positive contribution to the SAIL model than all other models. With GPT-3.5-Turbo, feeding search- augmented prompts with instructions leads to very slight improvements on both evaluations. How- ever, grounding on search results can hurt the per- formance of Vicuna and LLaMA-GPT4 models of different sizes. By comparing against GPT-4, Vicuna-13B 3B is slightly improved by search re- sults, but the improvement is not present when compared to GPT-3 3.rr-3.5-uuu 5-Turbo. For the Vicuna-78 and LLaMA-7B-GPT4 baselines, augmenting input prompts with search engine outputs makes signif- icant, negative impact on both evaluations. On the other hand, applying search augmentation to SAIL-7B significantly improves model perfor- mance on both experiments (84% to 90% and 98% to 103%).
3.1 Instruction Following
Automatic Evaluation with GPT-4. We com- pare the performance of different models under end-to-end and search grounded settings against GPT-4 and ChatGPT models on Question-80 and New-Question-80 The scoring results are shown in Figure 3.
By comparing to GPT-4 on Question-80, 80, we find that the search-augmented SAIL-7B model sig- nificantly outperforms all other models (90% vs <85% using fewer training instructions and param- eters, including strong baselines like Vicuna-13B and
powered ChatGPT. This in- dicates that when the grounding information is provided, the model does not need as many pa- rameters to memorize knowledge. In addition, the SAIL-7B model also achieves high perfor- mance even without search results, showing that the model performance is stable under different gen- eration settings. Similar conclusions can be found by comparing all models against GPT-3.5-Turbo While GPT-4 is still better, experiment results show that the search-augmented SAIL-7B model
On New-Question-80 find that GPT-4 and PPT-3.5-Turba perform similarly and are both outperformed by SAIL-7B. There are two main challenges that cause this result. Firstly, the mod- els need to reason based on new texts that never

---

appreared in their pretraining or fine-tuning cor- pora. Secondly, the texts provided by the search engines are noisy. On the other hand, SAIL-7B deals with the problems more successfully through search augmented instruction tuning. These results inform our findings:
With Vicuna-138, which is roughly times larger model, we get the best average performance (51.0%) on the three tasks without grounding in- formation. However, adding search results hurts its accuracy in most experiments. While augment- ing the model with DDG search results slightly improves the performance CSQA and OBQA, the accuracy ARC-Challene is decreased by 1.4%. With B.25- Wikipedia search results, the accuracy can decrease by as much as 1.8%. While the Vicuna- model achieves strong non- augmented performance, is challenging to further improve the accuracy by utilizing helpful informa- tion in the search results.
The search results contain useful informa- tion that can improve the performance of instruction- sooluuctio-followinng language models.
Without search-augmented fine-tuning, it is difficult for a language model to utilize valuable information among the complicated search results, and distracting retrieval results can mislead the generations. Generation on latest questions further signifies the challenge.
In contrast, the SAIL-7B model improves on all tasks when incorporating the search results, and also achieves strong non-augmented performance. Without retrieval results, SAIL-7B significantly outperforms LLaMA and Vicuna-7B on all tasks with a large margin (49.5% vs 44.5% and 40.9% average accuracy). also performs slightly better than Vicuna-138 on CSQA and OBQA tasks, while Vicuna-13B is still strongest on ARC-C. While search augmentation lcads to at most 0.5% improvement for Vicuna-138 DDG search results improve SAIL-7B by 2.8% OBQA and 1.2% average, showing that the SAIL-7B model can steadily utilize the helpful information among the search results. As a result, the search-augmented SAIL-7B model achieves the best performance on both CSQA and OBQA.
Search-augmented instruction learning can help the model better utilize the valuable in- formation among noisy search results and im- prove cnttrructio-followwiii performance.
3.2 Question Answering
Common Knowledge. The experiment results of question answering are shown in Table 1. CSQA, OBQA, and ARC-Challenge are open-ended, selection-based question quucttion-answernng tasks. We com- pare instruction-tumed Vicuna-7B, Vicuna- 1B, LLaMA-7B-GPT4, and SAIL-7B models under no- augmentation and search-grounded settings with different sources. All evaluations are zero-shot and instruction guided. Traditionally, knowledgeahle LLM can answer questions and select the most co- herent and appropriate answers without external information. In each task, we want to evaluate the performance of different models and knowledge bases. We search Wikipedia (Wiki) with the BM25 retriever, and the web with DuckDuckGO (DDG), feeding the LLMs with the top-3 search results, which could contain unrelated and distracting in- formation.
TruthfulQA. We use the TruthfulQA evaluation set (Lin et al., 2022) containing 817 questions to evaluate how informative and truthful are lan- guage models. Following the standard approach, we fine-tuned a GPT-3 model for automatic eval- uation. We compare the plain LLaMA models and search augmented instruction following models us- ing the GPT-3 evaluator. The performance of dif- ferent models and settings are shown in Table 2.
In general, we found that DuckDuckGo (DDG) leads to better performance for all models on all tasks because it is more flexible, covering much wider range of information. This suggests the effec- tiveness of search engines over retrieving static knowledge base. We found that both LLaMA and Vicuna-7B models can be slightly improved when search results are provided on most tasks. How- ever, the overall performance is limited. The av- crage accuracy of searched -ugbbeee-eg LLaMA-7B and Vicuna-78 78 below 50%.
We notice that with search grounding, both truth and info scores can be significantly im- proved. By connecting with search engine, the 7B models can significantly outperform the largest LLaMA-65B model. Similar to the auto- matic instruction-follloving results, the search- augmented SAIL-7B model outperforms search- augmented Vicuna-138.

---

Model
LLaMA-GPT4-7B
Vicuna-7B
Vicuna-13B
SAIL-7B
Search
None
Wiki
DDG
None
Wiki
DDG
None
Wiki
DDG
None
Wiki
DDG
CSQA
48.4
47.7
49.6
44.9
45.6
47.6
50.6
51.1
50.9
51.5
51.0
51.8
OBQA
42.2
44.4
44.6
37.2
39.4
42.6
49.0
47.2
49.4
49.2
50.2
52.0
ARC-C
43.0
45.2
47.3
40.5
44.5
46.3
53.2
51.6
51.8
47.7
48.1
48.4
Avg.
44.5
45.8
47.2
40.9
43.3
45.5
51.0
50.0
50.7
49.5
49.8
50.7
Search Effect
none
+1.3
+2.7
none
+2.4
+4.6
none
-1.0
-0.3
none
+0.3
1.2
Table I: Question answering accuracy (%) by zero-shot models with simple instructions.
Model
Size
True
True *
Info
the latest facts. A commercial search engine able to catch a wide range of up-to-date information that retrieval model with a fixed knowledge base can- not achieve. However, search engines usually less accurate than dense retrievers, and they might retrieve disputed documents that influence the qual- ity of fact checking. Our experiments show that the search results are not helpful for all baseline models. On Climate-Fewee augmenting the model with search results decreases the overall accuracy of LLaMA by 3%. On the PubHealth task, both accu- racy and Fl of cicana13333 13 model are decreased by the search results, by 4% and 1 respectively. This shows that the search results contain distracting information, which prevents the models utilize helpful evidence among noises.
No Search Augmentation
GPT-3
175B
0.28
0.25
LLaMA
65B
0.57
0.53
LLaMA
7B
0.33
0.29
Alpaca
7B
0.33
0.33
Vicuna
7B
0.56
0.52
W/ Search Augmentation
Vicuna
13B
0.71
0.69
Vicuna
7B
0.68
0.65
SAIL
7B
0.73
0.73
Table Automatic evaluation results of large language models on the TruthfulQA benchmark.
However, SAIL is more robust against distract- ing languages and its fact-checking performance is improved on the same set of search results, as shown in Table 4. With search augmentation, the fact-checking accuracy and Fl scores of SAIL improved both tasks, high 4.2% Climate-FFeeer. The augmented SAIL model also significantly outperforms all baselines, including Vicuna-1 and LLaMA-7B tuned with GPT-4 re- sponses by 9% accuracy and 5% F1, showing the effectiveness of search augmented fine-tuning.
3.3 Fact and Fairness Checking
The other task we evaluate model performance on is unified fact and faimess checking (Zhang et al., 2023), a combined benchmark with four sub-tasks including fact-checking (Diggelmann et al., 2020; Kotonya and Toni, 2020), hate speech detection (de Gibert al., 2018), and stereotype recognition (Sap et al., 2020). We evaluate the zero-shot perfor- mance on all four tasks, and the experiment results are shown in Table The SAIL-7B model achieves the highest accuracy and Fl scores on all tasks, de- spite no grounding information being provided for the fact-checking tasks. We also found that the Vicuna-7B and 13B models perform similarly on fact and fairness checking.
4 Related Work
Large language models. Beginning with GPT-3 (Brown et al., 2020a), LLMs have demonstrated strong abilities knowledge memorization and text-based inference on a wide range of tasks. LLMs include GPT-3, LaMDA (Thoppilan et al., 2022), FLAN (Wei et al., 2021), OPT (Zhang et al., 2022), and LLaMA (Touvron et al.. 2023). Compared to smaller language models, LLMs have several emergent abilities (Wei et al., 2022a), including zero-shot multi-task solving, and few-shot in-context learning with ch thougttttt reasoning (Wei et al., 2022b;
For the fact-checking tasks, we further evaluate the performance grounding on search results gener- external
ated by DuckDuckGo. Grounding search engine has both advantages and disadvan- tages. Many fact checking benchmarks provide task-specific grounding corpora that limit the do- main of information retrieval. However, internet misinformation can be very arbitrary and related to

---

Model
Metric
Climate
PubHealth
Fact Avg.
HSD
SBIC
Fairness Avg.
All Avg.
Acc
57,9
60.6
59.2
55.9
74,5
65.2
62.2
Vicuna-7B
F1
38.8
56.63
47.7
68.5
84.3
76.4
62.04
Acc
51.4
54.4
52.9
57.7
72.3
65.0
59.0
Vicuma-
F1
42.5
57.7
50.1
69.6
82.9
76.3
63.2
Acc
59.3
62.3
74.8
68.6
63.9
LLaMA-7B
F1
46.6
57.5
52.0
72.3
84.4
78.4
65.2
Acc
63.5
69.2
66.4
70.1
76.4
73.2
69.8
SAIL-7B
F1 51.0
63.6
57.3
75.1
83.9
79.5
68.4
Table 3: Instructed zero-shot language checking performance the UniLC benchmark.
Model
Metric
Climate
PubHealth
Avg.
that GPT-4 (OpenAI, 2023) can generate high- quality instruction-folloiing language. Recent
Acc
57.7
60.1
58.9
efforts on open-soureing instruction-following
Acc Diff
-0.2
-0.5
-0.3
Vicuna-7B
Fl
49.5
57.6
53.6
LLMs include Alpaca (Taori et al., 2023) and Vicuna (Chiang et al., 2023).
Fl Diff
+10.7
+1.0
+5.9
Acc
53.5
50.3
51.9
Acc Diff
+2.1
-1.0
Vicuna-133
Retrieval-augmmented language models. Prior to our work, several initiatives explored retrieval- augmented language models (RALMs). The pio- neering approaches REALM (Guu et al., 2020) and RAG (Lewis et al., 2020) sought to train language models with retrievers in an end-to-end manner. RETRO (Borgeaud et al., 2022) intro- duced the idea of training an LM top of a frozen retriever. Atlas (Izacard al., 2022) further plored dedicated loss functions for the end-to-end training of the retriever and the LM, achieving su- perior performance on several few-shot learning tasks. Recently, RePlug (Shi et al., 2023) and In- context RALM (Ram et al., 2023) instead explore an opposite direction: use frozen black-box LM while fine-tuning the retrieval modules. RePlug shows its advantages of leveraging large LMs like Codex (Chen al., 2021) and GPT-3 (Brown et al., 2020b), outperforming Altas few-shot question- answering tasks.
Fl
46.6
56.8
51.7
Fl Diff
+4.1
-0.9
+1.6
Acc
55.8
62.8
59.3
Acc Diñ
-3.0
+2,9
-0.1
LLaMA-7B
Fl
50.2
59.7
54.9
Diff
+3.6
+2.2
+2.9
Acc
65.8
68.3
Ace DifT
+2.3
+1.5
+1.9
SAIL-7B
55.2
59.9
FI Diff
+4.2
+0.9
+2.5
Table Search augmented zero-shot checking on the Climate-Fer Fever and PubHealth benchmarks.
Wang al., 2022a).
Instruction following. Pretrained LLMs can generate texts following certain formats and rules by seeing few examples in their prompts. To make LLMs more scalable and improve zero-shot performance, Ouyang al. (2022) proposed training GPT-3 with instruction-respone corpora. As a result, InstructGPT, ChatGPT, and GPT-4 can handle a wide range of tasks without seeing any examples. Recent research has also found that both GPT-generated instructions and instruct-followin outputs (Peng et al., 2023) instructionn-oollowing ability of LLMs. (Wang al., 2022a) proposed semi-supervised method generate diverse instructions based on seed instruction base on NLP tasks (Mishra et al., 2022; Wang et al., 2022b). more recent study shows
Despite the success of RALMs, most of these models have limitations, including 1) constraining the search space to a closed corpus like Wikipedia 2) lacking explicit mechanisms for disregarding dis- tracting search results, and 3) applying a few-shot in-context learning setting without considering in- struction fine-tuning during RALM training. Con- sequently, their applications remain relatively nar- row, primarily focusing on tasks such as question- answering and language modeling. SAIL addresses these limitations by 1) employing real-world search engines, 2) introducing a search result denoising
improve the

---

process capable of filtering out distracting informa- tion, and 3) incorporating instruction fine-tuning. Consequently, SAIL demonstrates its superiority in broader applications, including instruction fol- lowing for chatbots, fact and fairness checking, all of which benefit from access to up-to-date informa- tion retrieved from real-world search engines.
et al., 2023; Liu et al., 2023). The robust perfor- mance of GPT-4 on Question-80 benchmark shows its efficacy. However, on the New-Question benchmark, GPT-4 exhibits a comparatively dimin- ished level of performance due to the need of latest knowledge. While this may introduce an element of uncertainty into the evaluation process, we be- lieve that presenting these results can encourage the research community to be more informed and interested in this challenging problem. We plan incorporate numan-in-looy evaluation as part of our future endcavors, ensuring a more precise and comprehensive ussessment of instruction capabilities of large language models.
5 Conclusion
this work, we found that disputed and dis- tracting search results can significantly mislead the predictions of large language models. Sev- eral transparency-sensitiv tasks, including open- domain question answering and language check- ing be negatively influenced by this phe- nomenon. To solve this problem, we propose search-augee augge-euggented instruction-followii large lan- guage model with 7B parameters. We construct the first aageen- instruction-tuning cor- pus consisting of human-generated instructions, GPT-4 generated responses, and search results gen- erated by a BM25 retriever based on Wikipedia and a commercial search engine. We then fine- tuned the LLaMA-7B language model with the con- structed training corpus on academic computational resources. Experiments on insllwctionnfollooinngg question answering, and fact/fairness checking show that the search-augnen auggeetd language model can distill trustworthy and helpful information from all search results and generate high-quality re- sponses, improving both the performance and trans- parency of instraction-follvvii large language models.
References
Sebastian Borgeaud, Arthur Mensch, Jordan Hoff- mann, Trevor Cai, Eliza Rutherford, Katie Milli- can, George Bm Van Den Driessche, -apttte Lespiau, Bogdan Damoc, Aidan Clark, et al. 2022. Improving language models by retrieving from tril- lions of tokens. In International conference on ma- chine learning, pages 2206-2240. PMLR.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler. Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Ma- teusz. Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, 1lya Sutskever, and Dario Amodei. 2020a. Language models are few-shot learners. In Ad- vances Neural Information Processing Systems, volume 33, pages 1877-1901. Curran Associates. Inc.
Acknowledgement
This research was supported by the Center for Per- ceptual and Interactive Intelligence (CPII) Ltd un- der the Innovation and Technology Commission's 's InnoHK Scheme.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020b. Language models are few-shot learners. Advances neural information processing systems, 13:1877-1901
Limitations
While the model we propose achieves high per- formance with efficient model settings, the major limitation of the model is that it does not explain why a search result is trustworthy or informative or not. In future work, will fine-tune larger mod- els and enable the models to recognize trustworthy search results with explanations. In the context of automated instruction-follooiing evaluation, we adhere to the established literature that employing GPT-4 as the evaluator (Peng et al., 2023; Chiang
Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading wikipedia answer open- Proceedings of the 55th Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), pages 1870-1879,
domain questions.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto. Jared Ka- plan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374.

---

Wei-Lin Chiang. Zhuohan Li. Zi Lin, Ying Sheng. Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric Xing. 2023. Vicuna: An open- source chatbot impressing gpt-4 with 90%* chatgpt quality.
Patrick Lewis, Ethan Perez. Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin. Naman Goyal, Hein- rich Küttler, Mike Lewis, Wen-tau Yih, Tim Rock- täschel, et al. 2020. Retrieval- for knowledge-intensive nlp tasks. Advances in Neu- ral Information Processing Systems, 33:9459-9474
generation
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question an- swering? try are, the ai2 reasoning challenge. ArXiv, abs/1803.05457.
Jimmy Lin, Xueguang Ma, Sheng-Chieh Lin, Jheng- Hong Yang. Ronak Pradcep, and Rodrigo Nogueira. 2021. Pyserini: A python toolkit for reproducible information retrieval research with sparse and dense representations. In Proceedings of the 44th Inter- national ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2356- 2362.
Clco Condoravdi, Dick Crouch, Valeria de Paiva, Rein- hard Stolle, and Daniel G. Bobrow. 2003, Entailment, intensionality and text understanding. In Proceedings of the HLT-NAACL 2003 Workshop on Text Meaning, pages 38-45.
Stephanic Lin, Jacob Hilton, and Owain Evans. 2022. Truthfulqa: Measuring how models mimic human falsehoods. In Proceedings of the 60th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214-3252.
Ona de Gibert, Naiara Perez, Aitor Garcia-Pablos, and Montse Cuadros. 2018. Hate speech dataset from a white supremacy forum. EMNLP 2018, page 11.
Thomas Diggelmann, Jordan Boyd-Graber, Jannis Bu- lian, Massimiliano Ciaramita, and Markus Leip- pold. 2020. Climate-fever: A dataset for verifica- tion of real-world climate claims. arXiv preprint arXiv:2012.0061..
Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang. Ruochen Xu, and Chenguang Zhu. 2023. G-eval: Nig evaluation using gpt-4 with better human align- ment.
Hongyin Luo and James Glass. 2023. Logic against bias: Textual entailment mitigates stereotypical sen- tence reasoning. In Proceedings of the 17th Confer- ence of the European Chapter of the Association for Computational Linguistics, pages 1243-1254, Dubrovnik, Croatia. Association for Computational Linguistics.
Jiaxin Ge, Hongyin Luo, Yoon Kim, and James Glass. 2023. Entailment as robust self-learner. arXiv preprint arXiv:2305.17197
Kelvin Guu, Kenton Lee, Zora Tung. Panupong Pasu- pat, and Mingwei Chang. 2020. Retrieval augmented language model pre-training. In International confer- ence on machine learning, pages 3929-3938 PMLR.
Todor Mihaylov. Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. Can a suit of armor conduct elec- tricity? a new dataset for open book question answer- ing. In Conference on Empirical Methods in Natural Language Processing.
Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap. Dipankar Ray, and Ece Kamar. 2022. pachine--eneratee dataset for adversarial and implicit hate speech detection. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pages 3309-3332 3336..336.........3
Toxigen: A large-scale
Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannanch Hajishirzi. 2022. Cross-task generaliza- tion via natural language crowdsourcing instructions. In ACI..
Gautier Izacard, Patrick Lewis. Maria Lomeli, Lu- cas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi- Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2022. Few-shot learning with re- trieval augmented language models. arXiv preprint S.Xiv:2008.03299
Abiola Ohamuyide and Andreas Vlachos. 2018. Zero- shot relation classification as textual entailment. In Proceedings of the First Workshop on Fact Extraction and VERification (FEVER), pages 72-78, Brussels, Belgium. Association for Computational Linguistics.
Neema Kotonya and Francesca Toni. 2020. Explainable automated fact-checking for public health claims. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP), pages 7740-7754, Online. Association for Computa- tional Linguistics.
OpenAL. 2022. Introducing chatgpt.
OpenAL. 2023. Gpt-4 technical report.
Long Ouyang, Jeffrey Wu, Xu Jiang. Diogo Almeida. Carrll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, al. 2022. Training language models to follow instruc- tions with human feedback. Advances in Neural Information Processing Systems, 35:27730-27744,
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red- field, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Ken- ton Lee, et al. 2019. Natural questions: A benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:452- 466.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca

---

Antiga, al. 2019. Pytorch: An imperative style, nigh-performance deep learning library. Advances in neural information processing systems, 32.
Yizhong Wang. Swaroop Mishra, Pegah Alipoor- molabashi, Yeganeh Kordi, Amirreza Mirzaei. Anjana Arunkumar, Arjun Ashok, Arul Selvan Dhanasckaran, Atharva Naik, David Stap, al. via
Baolin Peng, Chunyuan Li, Pengcheng He, Michel Gal- ley, and Jianfeng Gao. 2023. Instruction tuning with gpt-4, arXiv preprint arXiv:2304.03277
2022b. appee--nauuriinssntiiosseeenen declarative instructions on 1600+ tasks. In EMNLP.
Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, An- drew M Dai, and Quoc Le. 2021. Finetuned lan- guage models are zero-shot learners. arXiv preprint UrXiv:2199.01652
Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023. In-context retrieval-augmentt lan- guage models. arXiv preprini arXi::2302.000 00000.
Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Juraf- sky, Noah A Smith, and Yejin Choi. 2020. Social bias frames: Reasoning about social and power im- plications language. Annual Meeting of the Association for Linguistics, pages 5477-5490.
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. 2022a. Emergent abilities large language models. arXiv preprint arXiv:2206.0766
Proceedings of the 58th
ConpatonoooCCCmuaatttttttoo Computational
Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. 20226. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201. 11000011.9990 1903.
Timo Schick, Jane Dwivedi- Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves use tools. arXiv preprint arXiv:2302.0476.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher De- wan, Mona Diab, Xian Li, Xi Victoria Lin, et al. 2022. Opt: Open pre-trained transformer language models. arXiv preprint 20Xx::205.00065
Weijia Shi, Sewon Min, Michihiro Yasunaga, Min- joon Seo, Rich James, Mike Lewis. Luke Zettle- moyer, and Wen-tau Yih. 2023. Replug: Retrieval- augmented black-box language models. arXiv preprint arXiv:2301 12652.
Tianhua Zhang. Hongyin Luo, Yung- Mung-Sung& Chuang. Wei Fang, Lue Gaitskell, Thomas Hartvigsen, Xixin Wu, Danny Fox. Helen Meng, and James Glass, 2023. In- terpretable unified language checking. arXiv preprint arXiv:2304.03724
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. CommonsenseQA: A ques- tion answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North American Chupter of the Association for Computational Linguisties: Human Language Tech- nologies, Volume I (Long and Short Papers), pages 4149-4158, Minneapolis, Minnesota. Association for Computational Linguistics.
Rohan Taori, Ishaan Guirajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction- instructiionfollooinng Ilama model. https:// gi github. com/ tatsu- wttpl//ccttttttttteeeeeeeeee wlaab//ttafood.../
Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. 2022. Lamda: Language models for dialog applica- tions. arXiv preprint arXiv:2201.088 088399.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and effi- cient foundation language models. arXiv preprint arXiv:2302.1397. 13971.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe Lee, Ed Chi, and Denny Zhou, 2022a. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203. 11111.

---

Models
Vicuna-7B-vi.
SAIL-7B
Novel
Include Consider
Calculate
Verbs
Match
Revolutionize
Check
Include
Increase
Count
2
6
Table Top-10 verbs generated by LLaMA-baser mod- els that do not overlap with GPT-4 and ChatGPT.
Models
Avg.
Std.
Diversity
GPT-4
303.8
121.5
0.48
ChatGPT
135.1
63.6
0.56
Vicuna-13B 13B
204.1
82.9
0.45
Vicuna-7B
196.5
90.3
0.45
SAIL-7B Search
246.2
87.7
0.44
SAIL-7B
206.6
86.9
0.47
Table 6: Statistics about the length and diversity of the generated responses of different language models. Diversity stands for the total number of different words divided by the total length.
Data Staties
We first show the word preference of different mod- cls the 80 unseen instructions. The results shown in Figure 4. We compare the distributions of top-10 verbs generated by GPT4 GPT-3. 5 (ChatGPT), Vicuna-7B-v1. and SAIL-7B mod- els. With search augmentation, SAIL-7B generates significantly more verbs that do not overlap with GPT's generations, as shown in Table 5. Only two top-10 10 verbs generated by Vicuna are not cov- ered by GPT-4 and ChatGPT, while six out of ten verbs generated by SAIL-7b are not tigh-freeuency verbs by the GPT models. This indicates that the grounding search results can shift the generation preference of the language models.
The statistics of the generated responses is shown in Table 6. GPT-4 generates the longest and most diverse responses, while ChatGPT tends to generate shorter and simpler answers. With- out search augmentation, the lengths of SAIL-7B generated sequences similar the Vicuna mod- cls. This indicates that search augmentation can increase the length of the generated responses.

---

GPT-4
GPr-3.5-Turbo
pordi
Teck
Vicuna-7b-w 7b-v1.1
SAIL-LLaMA-7b
bexk
Figure Top-10 verbs and associated generated by selective large language models.


Reviews:
[{"review_id": "2vrvFAl3r9", "reviewer": "Reviewer_LADc", "summary": "", "questions": "1. How many informative documents and noisy documents are there in the constructed dataset?\n2. What are the prediction results on the noisy and informative documents. Can we split the constructed dataset into two parts, one for finetuning and test the accuracy on the test set?\n3. Can we just input the informative documents to the LLMs as the upper bound of the performance and compare the performance with the proposed method?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your insightful comments and suggestions, especially for recognizing the contribution to the task formulation and the proposed method. But just for clarification, we were not constructing a human-labeled dataset containing informative and distracting documents. We apologize for making the paper seem like a dataset paper to you, and we will address the problem in the next version by explicitly highlighting the fact that the goal of the paper is to improve LLM grounding on any document datasets and knowledge bases by search-augmented instruction tuning.\n\nTo answer your concerns in more detail:\n\n1. **Statistics on the dataset** - In section 2.1, we introduced the number of search results we used for each instruction-response pair: (0, 1, 2, and 3) search results with (20%, 20%, 20%, and 40%) probabilities. We will release the dataset we used to train the model, but the true intent is to share an effective approach to build such a corpus. One of our major contributions on this end is that we do not need humans to annotate if a search result is helpful or not. Instead, we apply an entailment-based model to measure the coherence between each search result and the target response. As a result, we do not know the exact number of informative and distracting documents because no human \nannotation is required for applying our method.\n\n2. **Results on noise and informative results** - Since we did not collect any human-annotated data for either training or evaluation, it is impossible for us to test the model on only “informative” and “distracting” search results. We can only take the top-k search results provided by different retrieval models and use the entailment model as a rough reference. We evaluated different open-source and commercial language models on the same setting and found that our SAIL model outperformed very strong baselines. On this end, comparing SAIL-7b and LLaMA-GPT4 can measure the denoising effectiveness - LLaMA-GPT4 does not achieve as good performance due to the lack of denoising ability.\n\n3. **Novelty of the method** - We are sorry for causing the misunderstanding, but we believe the main contribution of the paper is proposing a pipeline that helps self-instructed language models to denoise and better utilize search engines without human annotation. To the best of our knowledge, this is the first study that helps LLMs achieve their goal by leveraging entailment models and in-context search denoising. Besides this, we believe that highlighting the limitation of LLMs under the retrieval-grounded setting is also novel and important. We found that even the GPT4 model can fail on some search-grounded tasks.\n\nThank you again for your comments that will help us make the story and message clearer to readers. We also hope that you can take our feedback into consideration.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The problem this paper studies is interesting and important. The authors propose a simple but effective method to empower LLMs to filter out noisy contexts and improve the performance of LLMs in many downstream tasks.\n2. The authors present a novel dataset that can be helpful for further studying on this topic.\n3. The experimental results are convincing and the proposed method can improve the performance of LLMs in many downstream tasks.", "reasons_to_reject": "1. Details introduction of the datasets should be further presented in the paper, such as the number of retrieved documents, the number of informative documents, and the number of noisy documents.\n2. In experiments, only the overall performance improvement on downstream tasks is presented. Is still unknown whether the improvement can be credited to the denoising ability of the proposed method. The authors should also present prediction results on the noisy and informative documents to show the denoising ability of the proposed method.\n3. The finetuning method proposed by the authors is quite simple and ad-hoc, which is limited in technical novelty. Maybe this paper is more suitable for a resource track with more analysis of the datasets added.", "questions_to_authors": "1. How many informative documents and noisy documents are there in the constructed dataset?\n2. What are the prediction results on the noisy and informative documents. Can we split the constructed dataset into two parts, one for finetuning and test the accuracy on the test set?\n3. Can we just input the informative documents to the LLMs as the upper bound of the performance and compare the performance with the proposed method?", "paper_topic_and_main_contributions": "This paper studies an interesting problem that filters out the noisy information from the retrieved documents and improves the performance of LLMs. The authors construct a search-grounded training set with the informative documents and the noisy documents labeled. Then, they finetune a LLM on the datasets to improve the denoising ability. The experimental results show that the proposed method can improve the performance of LLMs in many downstream tasks.", "excitement": 2, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "N/A", "Typos_Grammar_Style_and_Presentation_Improvements": "N/A"}, "extra_text": {}}, {"review_id": "V3ht15iWot", "reviewer": "Reviewer_o9nE", "summary": "", "questions": "see above. ", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your insightful comments, especially on recognizing the importance of the challenge we discussed, the effectiveness of the proposed method, and the quality of the experiments.\n\nTo answer your questions:\n\n1. **Need retrieval or not?** - We believe that this is a super insightful and important question. In the alpaca training set, the answer is “yes and no”. Many instructions like “provide suggestions about …” and “list the difference between the following apps” can be answered by a large language model directly, but a retrieval model can also provide helpful information. We found that for a general question, search augmentation can make the generated response more informative with the model we trained. More generally speaking, it is widely believed that many tasks can be solved without retrieval, for example, math and programming. We found that although these tasks can be solved only depending on reasoning abilities, search can provide additional instructions or hints that could benefit the reasoning process. This effect is shown in the evaluation with the question-80 benchmark. Although most questions in the benchmark do not have to be answered with knowledge retrieval, our model can benefit from search augmentation and significantly outperform strong baseline models evaluated under the setting.\n\n2. **Reliability of evaluation** - Although language models can be distracted by search results, we found that GPT-4 still achieves the best performance on the Question-80 benchmark since the questions do not require very up-to-date knowledge and the state-of-the-art GPT4 model can handle most questions without search. As research, GPT4 can be a strong judge on this task to compare other models, including open-source models and GPT-3.5-Turbo. Our New-Question-80 does require retrieval, so GPT4 evaluation would be less accurate on New-Question-80, but we believe that it is helpful to the research community to show the results and encourage further discussions and studies on this topic. We will make this point clearer in the paper. For the next step, we will perform a human-in-loop evaluation for this task and present more accurate evaluation results.\n\t\nOn the other hand, this automatic evaluation has been more standard. For example, it is used to compare the instruction following the performance of Vicuna and LLaMA models, as well as judge if a response is informative and factual in the TruthfulQA benchmark.\n\nThank you again for the insightful comments!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This problem is very important for now and future research, filtering unreliable information is instrumental for retrieval augmented LM. \n2. The approach the authors use is very reasonable, constructing instruction tuning dataset and finetuning on this dataset. \n3. The evaluation is quite thorough, studying both general instruction following, as well as knowledge intensive tasks.", "reasons_to_reject": "1. The instruction tuning dataset is built upon the alpaca dataset, which is not necessarily knowledge intensive, and probably doesn't need retrieval or internet information. It would be more clear if the authors can provide more details of the percentage of questions that actually need the internet data, and for the percentage that doesn't need internet data, how does this instruction tuning procedure affect performance? \n\n2. I am a bit suspicious of the instruction tuning evaluation using GPT-4. Especially this problem is to figure out whether search-augmentation is helpful and this problem is common for all language models, so probably GPT-4 also suffer from similar problem and cannot be used as a fair evaluation metric.", "questions_to_authors": "see above. ", "paper_topic_and_main_contributions": "The paper proposes search-augmented instruction tuning. The challenge is that the search results can be distracting, misleading, and untrustworthy. To alleviate these problems, the paper proposes an instruction tuning dataset, and finetune llama model. In this way, the model learns to ground on trustworthy information and filter out unreliable ones. The paper evaluates on instruction following and knowledge intensive question answering, and find gains in both settings. \n", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "b6hz2uXEUr", "reviewer": "Reviewer_ZXCP", "summary": "", "questions": "1. I do not understand the title of the paper. On my best guess interpretation, it ios saying that search causes a problem for LLMs. But the paper is entirely about how search can benefit these models. I would add that the title is missing many opportunities to highlight the contributions of the paper in terms of instruct fine-tuning, public search engines, and prompting.\n\n1. One of the guiding themes of the paper is that models are mislead by search results. Where is the evidence for this in the paper, though? The results as reported are consistent with many explanations for models failing, including SAIL-7B being mislead more often than the other models (which could suffer from independent weaknesses).\n\n1. In figure 1, where doe the model generation begin? I infer from the paper that it begins with \"the related natural fact\", but I am not totally sure.\n\n1. Figure 1's example is admirably challenging, but it does raise concerns for me. The fact to check is about \"most climate models\", and so none of the individual results resolves the issue. I checked out the results, and it actually seems to me that the first one is the one that comes closes to actually endorsing this very strong claim. The supposedly supporting claim is just about one study.\n\n1. Footnote 1: I think \"privacy-proof\" is not what was intended here. That would mean it offers a guarantee of no privacy. I suspect \"prvacy-preserving\" is what is intended.\n\n1. The paragraph beginning on line 166 is confusing because the first sentence seems to say that each example has one search result, whereas the last sentence says each result can have up to three search results. I suspect that the intention here is to say that each search result is paired with at most one example, and each example has 0-4 results associated with it.\n\n1. The entire approach seems to depend on the strength of the entailment model in section 2.2. Is it a good model?\n\n1. GPT-4 as the evaluator is what it is, I suppose. Shouldn't we be worried about this, especially given that we are going to have systems that outperform it? I would love to know what GPT-4's performance is on the rating task.\n\n1. The paper and code vary in referring to the new datasets as \"New-Questions-80\" and \"News-Questions-80\". It would be good to settle on one.\n\n1. I am not sure I understand what \"Search Effect\" is in Table 1 for the SAIL-7B model. This model seems to be intrinsically benefiting from search due to its fine-tuning.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your insightful comments, especially recognizing the contribution that we are excited about and the support for acceptance! \n\nTo answer and discuss your questions -\n\n1. **Title** - Thank you for the suggestion and it makes much sense to us. We will carefully consider an improvement to convey the idea that “search challenges language models but instruction tuning helps”.\n\n2. **Evidence on search misleading** - We have this message distributed in the paper in different places, including Figure 1 where GPT4, GPT-3.5, and perplexity.ai fail but SAIL provides the correct answer; In Figure 3  LLaMA-GPT4 (7B) and Vicuna (7B) cannot leverage search augmentation but SAIL receives a significantly higher improvement with search when compared to both GPT3.5 and Vicuna-13B; Table 1 shows that search decreases the performance of Vicuna-13B but SAIL has improvement with search augmentation; Table 4 shows that search can decrease the accuracy of LLaMA and Vicuna on the fact-checking task but SAIL receives significant improvement on both Accuracy and F1.\n\n3. **Figure 1 generation start** - In Figure 1, the model generates from the beginning, i.e., “Search result 1 is distracting …”. We will make it clearer in the paper that there is a difference between training and inference. When constructing the training corpus, these denoising sequences are decided by the entailment model, but in the inference and evaluation phase, the LLM generates the decision by itself since the LLMs should have learned the ability to distinguish between informative and distracting search results after the search augmented instruction-tuning.\n\n4. **Figure 1 fact checking** - Thank you for raising this concern and we believe this is a very insightful point. We believe that answering climate-related questions needs more current information, but the first result on a 2007 report seems obsolete. The second result clearly states that climate change is happening sooner than previous model(s) have predicted, which is a negative signal for the claim. However, we now recognize that the word “most” in the claim makes the example more complicated. We will add a clear discussion on this in the paper, to help readers to understand the complexity of the tasks.\n\n5. **“privacy-proof”** - Thank you for the suggestion. We will edit accordingly.\n\n6. **The term “search result” in line 166** - Thank you for catching this, we have mixed the use of search results and “list of search results”. You are right, here should be “each training example is assigned a list of corresponding search results”.\n\n7. **Power of the entailment model** - Entailment models have been proven to be good at zero-shot and few-shot language understanding in previous studies. Our entailment has good performance on MNLI and other GLUE tasks under zero-shot settings. We will add the citations and present our results to show that the entailment model has reasonable performance.\n\n8. **GPT-4 performance on Rating** - This question is very insightful and we would like to add a clear discussion to the paper. Firstly, this automatic evaluation has been adopted by many studies, including evaluating the instruction-following ability of Vicuna and Alpaca, and judging if the the responses are informative and factual enough in TruthfulQA. In this work, GPT-4 performs well on Question-80 because the questions do not need up-to-date information to solve. In this case, we are confident that the GPT-4 evaluation is reliable. On New-Question-80, GPT-4 is less powerful which might lower the confidence of the evaluation, but we believe that showing these results could encourage the research community to be more informed and interested in this challenge. For the next step, we will perform a human-in-loop evaluation for this task and present more accurate evaluation results.\n\n9. **News-Q-80 and New-Q-80** - Thanks for catching the inconsistency. We will fix it and use the term “New-Q-80”.\n\n10. **“search effect in Table 1** - We intended to show the performance change after applying search augmentation, and the improvement of SAIL comes from both fine-tuning and search augmentation. We will think about if there is a better title for this row and add the explanation.\n\nThank you again for all these helpful questions and we will reflect your suggestions and our thoughts in the paper!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "I am supportive of this paper and feel that it could be accepted.\n\nIt seems like it contains useful lessons regarding retrieval-augmentation, the role of public search engines, the nature of self-instruct-based fune-tuning, and the power of smaller models to be fine-tuned to do specific things.\n\nThe results are strong, and the experiments seem to be carefully and openly reported.\n\nI really like the New(s)-Questions-80 dataset as a special challenge, and that turns out to be the strongest argument for the approach, since it gives us a glimpse of how GPT-4 does on entirely new information.", "reasons_to_reject": "I do not have reasons to reject per se. I do, though, have a lot of comments and suggestions, and I would urge the authors to attend to these in order to help their work have the largest impact it can. I will put these under Questions for the authors to help ensure that they are not seen as reason to reject.\n", "questions_to_authors": "1. I do not understand the title of the paper. On my best guess interpretation, it ios saying that search causes a problem for LLMs. But the paper is entirely about how search can benefit these models. I would add that the title is missing many opportunities to highlight the contributions of the paper in terms of instruct fine-tuning, public search engines, and prompting.\n\n1. One of the guiding themes of the paper is that models are mislead by search results. Where is the evidence for this in the paper, though? The results as reported are consistent with many explanations for models failing, including SAIL-7B being mislead more often than the other models (which could suffer from independent weaknesses).\n\n1. In figure 1, where doe the model generation begin? I infer from the paper that it begins with \"the related natural fact\", but I am not totally sure.\n\n1. Figure 1's example is admirably challenging, but it does raise concerns for me. The fact to check is about \"most climate models\", and so none of the individual results resolves the issue. I checked out the results, and it actually seems to me that the first one is the one that comes closes to actually endorsing this very strong claim. The supposedly supporting claim is just about one study.\n\n1. Footnote 1: I think \"privacy-proof\" is not what was intended here. That would mean it offers a guarantee of no privacy. I suspect \"prvacy-preserving\" is what is intended.\n\n1. The paragraph beginning on line 166 is confusing because the first sentence seems to say that each example has one search result, whereas the last sentence says each result can have up to three search results. I suspect that the intention here is to say that each search result is paired with at most one example, and each example has 0-4 results associated with it.\n\n1. The entire approach seems to depend on the strength of the entailment model in section 2.2. Is it a good model?\n\n1. GPT-4 as the evaluator is what it is, I suppose. Shouldn't we be worried about this, especially given that we are going to have systems that outperform it? I would love to know what GPT-4's performance is on the rating task.\n\n1. The paper and code vary in referring to the new datasets as \"New-Questions-80\" and \"News-Questions-80\". It would be good to settle on one.\n\n1. I am not sure I understand what \"Search Effect\" is in Table 1 for the SAIL-7B model. This model seems to be intrinsically benefiting from search due to its fine-tuning.", "paper_topic_and_main_contributions": "This paper develops and evaluates a method for fine-tuning language models to make better use of Web search results when performing knowledge intensive tasks. Starting from the Alpaca self-instruct corpus, the authors query search engines use the results to create a new fine-tuning dataset, which is then filtered and formatted using an entailment model that seeks to classify results for whether they are informative or distracting. Using LLaMA-7B as the base model, the authors get strong results across a number of QA and fact-checking benchmarks, and they get exceptional results on a new corpus of questions based in articles published after May 2023 to ensure that they are genuinely new for all LMs.\n", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper introduces a method for fine-tuning large language models (LLMs) to use web search results more effectively for knowledge-intensive tasks. The paper creates a new dataset by querying search engines with self-instructions and labeling the results as informative or distracting using an entailment model. The paper then fine-tunes LLaMA-7B on this dataset to improve its ability to filter out noisy information and ground on trustworthy information. The paper evaluates the method on various tasks, such as instruction following and question answering, and shows significant improvement over the baseline. The paper also tests the method on a new corpus of questions based on recent articles, and achieves remarkable results.
Decision: Accept-Findings