Thrust: Adaptively Propels Large Language Models with External Knowledge
Authors: Xinran Zhao
Hongming Zhang
Xiaoman Pan
Wenlin Yao
Dong Yu
Jianshu Chen
Venue: Submitted to ICLR 2023 (2023)
Keywords: knowledge-intensive natural language processing
pre-trained language models
instance-level adaptive knowledge usage

Abstract: Large-scale pre-trained language models (PTLM) have achieved great success in various natural language processing (NLP) tasks. Much evidence shows that PTLMs already encode rich knowledge themselves, but knowledge stored in PTLMs can be opaque and static, making external knowledge retrieval necessary. However, there are two major challenges when using external knowledge. First, knowledge indexing and retrieving on large-scale knowledge bases are time costly. Second, knowledge retrieved could be noisy and sometimes misleading. Motivated by the observation that external knowledge is not always required by PTLMs, we investigate an effective and efficient way to apply knowledge only when the knowledge is essential. Specifically, we propose instance-level adaptive propulsion of external knowledge (IAPEK), where we score each instance on whether the PTLMs need the support of external knowledge. To achieve this goal, we design a novel metric, Thrust, which leverages the distribution estimation on seen/training instances. Extensive experiments demonstrate that we can achieve significantly higher cost-efficiency through Thrust compared to the naive usage of external knowledge on 88% of the evaluated tasks with 26% average performance improvement. Such findings further shed light on the real-world practice of knowledge-enhanced LMs with a limited budget for knowledge seeking due to computation latency or costs. 

Under review as a conference paper at ICLR 2023
THRUST: ADAPTIVELY PROPELS LARGE LANGUAGE MODELS WITH EXTERNAL KNOWLEDGE
Anonymous authors Paper under double-blind review
ABSTRACT
Large-scale pre-trained language models (PTLM) have achieved great success in various natural language processing (NLP) tasks. Much evidence shows that PTLMs already encode rich knowledge themselves, but knowledge stored in PTLMs be opaque and static, making external knowledge retrieval neces- sary. However, there are two major challenges when using external knowledge. First, knowledge indexing and retrieving on large-scale knowledge bases are time costly. Second, knowledge retrieved could be noisy and sometimes misleading. Motivated by the observation that external knowledge is not always required by PTLMs, we investigate an effective and efficient way to apply knowledge only when the knowledge essential. Specifically, we propose instance-level adap- tive propulsion of external knowledge (IAPEK), where score each instance whether the PTLMs need the support of external knowledge. To achieve this goal, we design novel metric, Thrust, which leverages the distribution estimation seen/training instances. Extensive experiments demonstrate that we can achieve significantly higher cost-efficiency through Thrust compared to the naive usage of external knowledge on 88% of the evaluated tasks with 26% average perfor- mance improvement. Such findings further shed light on the real-world practice of knowledge-enhanee LMs with a limited budget for knowledge seeking due to computation latency or costs
INTRODUCTION
Knowledge plays an important role in solving natural language processing (NLP) tasks, where en- cyclopedic or commonsense knowledge is commonly required answer questions from various tasks (Yin et all (022). In recent years, the emergent advance of pre-trained language models (PTLM) has demonstrated great improvement on various tasks (Deylin et all 2019; Radford et all, 200, Lu all A Baire et all 202T; Brown et all 2020). Evidence also show that PTLMs con- tain rich encyclopedic (Perron alal M)) or commonsense (Kocijan et all M) knowledge them- selves. However, such implicit knowledge embedded in the model's hidden states can be opaque, static, and inefficient to utilize (Khattah et all 2022). These issues motivate the common practice on seeking external knowledge (Xu all 20211 Verga et all 2021: Paranjape et all 2022) in NLP. A typical line of work focuses on retrieval-based methods, where knowledge is retrieved by a stand- alone retriever from external knowledge bases and then used to augment the inference models (i.e.,
Reader) such as PTLMs (Karpukhın et all, 20211 Gao & Callan 2021: Khattab &r Zabare, (120)).
However, there are several limitations with the usage of external knowledge: (i) performance on the downstream tasks is not commonly revealed. Metrics of the common benchmarks (e.g., MS- MARCO (Nguyen et all, 2016), BEIR (Thakur et all, 2021)) measure the quality of retrieval (e.g., Recall @ 50, nDCG@ 10). Although retrieving the relevant content may positively relate to the down- stream performance, not reporting the downstream performance, especially for the out-of-domain tasks, limits the exploration of how to utilize the external knowledge in practice; (ii) the external knowledge can be noisy or unnecessary. On the retriever side, though concurrent retrievers achieve great performance various tasks, the noise can still exist. For instance, ColBERT v2 (Santbanam ct all 2022) achieved 68.9 Success @5 on Natural Question (Kwiatkowskı et all 2019), which sug- gests that gold documents do not appear in the top retrieved documents for 31.1 1% of the queries.
'The code and data will he released upon acceptance.

---

Under review as a conference paper at ICLR 2023
Query: Who is the first woman to get the Nobel prize in physics?
Answer: Marie Curie
Direct Prediction: Marie Curie
Knowledge #1: Wilhelm Conrad
Knowledge #2: Marie Curie was the first woman to be awarded Nobel prize. Curie and her husband declined to go to Stockholm
Rontgen discovered X-ray ... which made him the first winner of the Nobel
prize in physics in 1901
Prediction: Wilhelm Conrad Rontgen
Prediction: Marie Curie
Figure 1:
The predictions from OPT (175B version) with/without external knowledge retrieved by DPR (Karpukhun et all, (020)) from Wikipedia paragraphs. Although the top retrieved paragraphs are relevant, since the internal knowledge already sufficient, the external knowledge can either be misleading (potentially due to the effect of misprime (Kassner & Schlitze, 2020) or less useful.
Considering the limited input token length, the most useful documents may not be included dur- ing prediction. Not to mention that others can be noise to the model. On the model side, PTLMS with growing capacity, from millions (e.g., BERT (Devlin all, 2019) billions of parameters (e.g., OPT (Zhang et al, 20225)), may solve the queries directly without external knowledge, which makes unnecessary to seek external knowledge and signifies the noise issue. The instance shown in Figure II demonstrates the noise and inefficiency issues. OPT (the version with 175 billion pa- rameters) can directly give the correct answer without any external knowledge. However, with top external knowledge retrieved by DPR (Karpukhin et all 2020) from Wikipedia paragraphs, the ex- ternal knowledge can be useless or even lead to wrong predictions.
Intuitively, solution to the noise and inefficiency issues is only seeking external knowledge when it necessary. this work, we capture this intuition by proposing Instance-level Adaptive Propulsion of External Knowledge (LAPEK) to reduce the effect of noise in external knowledge and improve the cost-efficiency of knowledge augmentation. In detail, for each instance of a given task, we com- pute confidence score measuring how likely can be solved directly with respect and reject the use of external knowledge when the score high. We design a simple and lightweight metric Thrust to serve such purpose by leveraging the estimation of the instance distribution in the eyes of the target models. To comprehensively understand the effectiveness of Thrust, we first create large-scale benchmark examining the downstream performance of the fask-plus-knowledgee paradigm with (i) tasks with different formats and types (e.g., multiple-chice classification (MC classification) and open-domain question answering (open-domain QA)); (ii) knowledge with dif- ferent formats and from different resources (e.g., knowledge graphs, Wikipedia paragraphs, and human annotations). Next, with models that can utilize external knowledge, we evaluate the ef- fectiveness of Thrust by showing that it can boost the performance of various tasks under various settings, such as injecting external knowledge to different portions of the test instances.
given model
Extensive experiments show that Thrust can improve the cost-efficiency of seeking and using exter- nal knowledge on 88% cases with 26% average performance improvement through identifying the instances that mostly require knowledge. We can also observe that, with Thrust, we achieve higher performance than injecting external knowledge for all the instances, where models are bene- fited from both the performance and efficiency aspects.
2
OUR METHOD
2.1
INSTANCE-LEVEL ADAPTIVE PROPULSION OF KNOWLEDGE
We first define IAPEK as follows: for each query di in given test set = (1(1), g(2) (...)...) let f(g) denotes the scoring function of the necessity of external knowledge, we extract the corresponding scores S = ()) (()))
With S, we re-rank the test set into D' = (1(1), (((2) ...).
Given any threshold t R, sample subset D3. (g/ need, where for each de E Dx. f(x) > t. Empirically, we can set t as a particular percentile of S, e.g., top 25% of S. Next, for each instance in Dr. we seek for external knowledge pieces and
(1) .9k ..... (2.)...) as that with highest knowledge
2

---

Under review as a conference paper at ICLR 2023
close one cluster: knowledge exists
close to multiple clusters: controversial
far from any cluster: no knowledge
labele
labeln-1
Figure 2: Figurative illustration of the intuition behind proposing Thrust. We represent the instance query vectors as triangles and instance clusters with different labels as circles with different colors. In the controversial and no knowledge cases, the knowledge contained internally is less likely to answer the query successfully. contrast, the model finds the query similar various seen instances, the internal knowledge is likely to exist.
augment each query to q We combine updated D&+ and original unsampled instances D DR the new knowledge augmented dataset D+ to apply to inference models.
2.2 THRUST
We then introduce a novel empirical method, Thrust, perform the proposed instance-level adaptive propulsion of external knowledge (IAPEK). We design Thrust to measure how likely the given query can solved by the internal knowledge of the target model. Intuitively, there are two cases where models can fail to answer query with internal knowledge: (i) the model has relevant knowledge and not familiar with the query semantics or inference types; (ii) the model faces controversial knowledge, where the query may have similar semanties with different kinds of seen questions that potentially require different reasoning to solve.
As shown in Figure C the degree of difference of queries and clusters of seen instances measure how well the internal knowledge of the model covers such queries. Motivated by the intuition, the Thrust score is computed For QA tasks, we regard all instances as having a single dummy label. Then, we embed instances within each group with an embedding function f(.). Next, we form groups for each task T as (I()) (z)) y: = l}. With the groups in hand, we perform k-means clustering to proceed each group into clusters and instances of task T are represented by the clusters
follows: given task T, first group the instances their labels.
(g))) K j=1 for I = 1,2, ..., N, with m² as the corresponding centroids and m² (gt) the vectorized centroids pointing from 0 to the c
For some time input e T, where the label is unknown, let f(x) denote the vectorized features pointing from 0 to the embedded ((x), the directed distance between the instance and the clusters are then defined as
(g)) f(x).
Influence of each cluster towards the query is considered vector with the same direction of d and re-weighted by the size of the cluster over the square Fuclidean distance. Thrust score for an input x then computed as
N K
('( &(f,x,
SThrust
(2)
N.
1, 13
==1
We use the layer hidden states as the embedding function. For T5-based models, we use layers of the decoders. We empirically set the number of clusters as the maz( V V/133,,3) where Der denotes the training set of task T.

---

Under review as a conference paper at ICLR 2023
Dataset
Type
Source
Train
Test #
Query Len
Know, Len
Ans. Len
AGNews
MC
gold
120,000
7,600
35.9
1.0
e-SNLI
NLI
human
259,999
9,824
24,9
14.3
1.0
StaatgyQA
Binary
2,290
33.5
CIKQA
Binary
KG
4,818
604
18.2
28.0
1.0
BoolQ
Binary
retrieeer
9,437
3,270
9.8
113.8
ARC-E
MC
retriever
2,251
570
23.1
238.2
ARC-C C
MC
retriever
1,119
299
26.2
240.5
5.5
HotpoiQA
QA
gold
90,447
7,405
19.0
56.3
2.5
QA
retriever
6,468
588.9
Web Questions
QA
retriever
2,474
278
7.8
117.3
Curated TREC
QA
retriever
1,125
116
116.5
TriviaQA
QA
retriever
78,785
6,760
15.0
117.6
27.5
Table 1: Statistics of the selected datasets. ARC-E & ARC-C denote the easy and hard ARC datasets as defined in the original work. MC, NLI, Binary, QA denote the task types of multiple-choice clas- sification, natural language inference, binary classification, and question answering, respectively. Query/Know./Ans. Len denote the average numbers of words for the queries/tnowlelgg/nnswess
3 EXPERIMENT
3.1 SETUP
We evaluate T5 (Rate et all (020), GPT-J (Wang & Komatsuzaki 20211). OPT (Zhang et all 20226), and UnifiedQA (Khashabrettal al (T2O). We test both the zero-shot learning setting (models take the prompt with or without knowledge directly) and the transfer-learning setting (models are fine-tuned with instances containing external knowledge) to examine the proper way to external knowledge. For the zero-shot setting, we test on T5, GPT-J, and OPT (30 billion parameter version). For the transfer-learning setting, we test on UnifiedQA with different scales.
After that, we evaluate the effectiveness of Thrust, where we try to simulate the real-world case where we have limited bandwidth budget to retrieve external knowledge. Specifically, are only allowed to conduct the knowledge retrieval for 25%, 50%, and 75% of instances. As introduced in Section 21L. we use Thrust rank the instances by their need for external knowledge and select the instances from the most necessary ones. The threshold can be considered as expecting 100-X% savings (X = 25, 50, 75) in continuously incoming future test inputs. We compare the performance difference between Thrust and random sampling
For MC classification tasks, we follow previous works to use accuracy as the evaluation metric. For open-domain QA, we report the QA-FI scores of the models under different settings. The QA- F1 score for question answering measures the max uni -ram overlap between the model prediction and all gold answer candidates. Denoting the gold answer set as G and uni-gram tokens of each query and each corresponding gold answer as T, and Ty, the QA-FI score can then be written as QA-FI maxges (T, n (T)/(T U Ty)- We test two settings: without knowledge and with knowledge. For the former one, we directly pass the prompt-decorated queries to the model to retrieve the choices with the highest probability or the answer the questions. For the latter one, we add the prompt-decorated queries, one piece of knowledge, and Answer: into three lines and pass them all the models.
3.2 DATASETS
We first prepare benchmark containing knowledge-intensive tasks from two main types: MC clas- sification and open-domain question answering, with seven and five tasks, respectively. We unify the instances into the same format, where each contains: (i) query: a piece of text containing a question or the sentences to be classified; (ii) an answer: either the label words or the answers the questions in the query; (iii) knowledge: one piece of potentially helpful knowledge for the query, which is either inherently relevant due to the task design, annotated by humans, or retrieved from Wikipedia paragraphs with DPR. Details of the selected datasets and definitions of external knowledge for each task are follows:
We also involve BM25 alternative perform TAPEK. The comparison be found in Appendix.
4

---

Under review as a conference paper at ICLR 2023
Multiple-choice classification. For MC classification, each query q includes a sentence or ques- tion and requires models to extract one correct answer from a set of candidate answers (e.g., yes or no). The selected tasks are (i) AGNews (Zhang et all 20TS): AGNews is classic text clas- sification task in NLP, where models are required to classify if a piece of news belongs to which among political, sports, business, or technology categories. We regard the titles of the news as the queries since they may already contain sufficient information for classification (c.g., Olympic his- tory for India belongs Sports); Then, the content of the news is considered as the gold external knowledge extracted from the task design; (ii) e-SNLI (Camburu et all 2018) e-SNLI is a natural language inference (NLI) task exploiting the role of explanations for the task of textual entailment. We concatenate the original two sentences with blank and add question about if entailment exists to form the query. Naturally, the human- providing explanations are considered strong source of external knowledge; (iii) StrategyQA (Geva et al. 202TT): StrategyQA is a challenging multi-hop reasoning dataset that requires models to answer creative questions (c.g., Did Aristotle use a lap- top?) through strategical inference from implicit reasoning steps. We regard the original creative questions as queries and muman-written explicit facts (e.g., Aristorle was born in ...) as external knowledge, which is again powerful and expected to contain little noise; (iv) CIKQA (Zhang all, 2022a); CIKQA is commonsense inference task in the format of multiple-choice question an- swering, combining the tasks of pronoun coreference resolution, commonsense QA (Talmor all 2019). COPA (Roemmele et al. I), and questions mined from ATOMIC knowledge graph (Sap all 2018). We regard the original questions as queries and the supporting commonsense knowl- edge extracted from knowledge graphs (KGs) in the original work as the external knowledge; (v) BoolQ (Clark etlll 2019): BoolQ is a question that contains encyclopedic questions that require models to answer yes or no. Following (Khashabi et all 202(), we use the Wikipedia paragraphs retrieved by DPR as the external knowledge, which can be potentially noisy; (vi) ARC-E & ARC- C (Clark et all 2018): ARC is challenging multiple-choice question answering dataset that re- quires powerful knowledge understanding and reasoning, which is partitioned to an Easy set and a Challenge set (denoted ARC-E & ARC-C, respectively), where the Challenge set questions are answered incorrectly by the retrieval-basee or co-occurrence- based algorithms tested by the original authors. Similarly, we use the Wikipedia paragraphs retrieved by DPR as external knowledge.
Open-domain QA. For open-domain QA, each query q contains an open question that typically requires solving an encyclopedic commonsense inference. The generated answers can either be few phrases or single sentence. The involved datasets are HotpotQA (Yang et all, 201N), Natural Questions (NQ) Ikwiatkowski et all, 2019), Web Questions (Berant et ell 2013), Curated TREC (Bandis, 2015), and TriviaQA (loshi et all 20TT)) We use Wikipedia paragraphs retrieved by DPR as the external knowledge as a common practice (Y'n all 2022), except for HotpoiQA, where we use the passages the queries are generated from as a gold knowledge resource.
The statistics of the involved datasets are reported in Table We collect benchmark with var- ious datasets of different types, formats, and knowledge sources, where we will then evaluate the effectiveness of IAPEK.
3.3 USING EXTERNAL KNOWLEDGE
Table 2 presents the model performance on both the MC classification and open-domain QA tasks. For the MC classification tasks, we can observe that: (i) for the zero-shot setting, models do not con- sistently get benefit from external knowledge. In addition, the ability to utilize external knowledge is also not clearly improved as the parameter size grows, which indicates that simply using larger models may not be the solution for better using the knowledge; (ii) for the transfer-learning setting, although AGNews, e-SNLI CIKQA, and StrategyQA are not seen during the training of UnifiedQA models, we can observe that models achieve better performance than vanilla T5 models at differ- ent sizes. Under the with knowledge case, the UnifiedQA models achieve significant improvement for utilizing external knowledge compared to the zero-shot models, UnifiedQA-33 achieves the best performance on all the tasks, which indicates that models can learn and transfer the ability to utilize external knowledge with instances containing external knowledge.
For open-domain QA datasets, we can observe that: (i) similarly, models fail to get benefit from external knowledge in 11 out of 25 cases under the zero-shot setting. Surprisingly, the smallest model T5-base gets benefits for all the tasks, but the largest model (OPT-30b) gets worse perfor- mance with knowledge for all tasks. The reason behind this can be that, since T5-base does not
5

---

Under review as a conference paper at ICLR 2023
Model
parameters
AGNews
c-SNLI
CIKQA
Stranegy
BoolQ
ARC-E
ARC-C
Zero-shot
220M
65.1
51.5 51.8
38.9
27.8
15-large
770M
25.8
25.2
65.7
65.7
50.0
50.0
53.3
53.3
37.8
38.6
25.1
21.7
27.7
24.7
T5-36
3B
27.9
39.1
57.6
61.5
52.6
50.5
44.5
48.9
56.6
45.3
25.8
26.0
26.4
28.4
GPT-J
6B
25.1
26.9
40.8
37.0
49.8
50.7
47.2
55.9
60.2
47.2
25.4
29.5
28.4
27.1
OPT-306
30B
25.0
25.0
65.7
63.7
50.0
50.0
53.3
53.3
37.8
37.8
27.4
27.7
25.8
26.4
Transfer-learning
UnstiedQA-base
220M
46.6
35.7
38.5
70.2
56.0 59.6
48.5
57.2
60.4
80.8
50.2
61.6
44.8
45.2
UnifiedQA large
770M
71.0
67.9
42.8
74.2
59.6
62.1
48.5
66.4
59.8
84.5
64.0
66.0
55.2
49.5
UnifiedQA-3b
3B
75.7
84.5
89.6
61.3
66.9
57.6
83.4
61.5
87.8
73.7
76.5
parameters
Questions
HotpotQA
NQ
TriviaQA
Zero shot
T3-base
220M
6.7
8.7
2.5 3.8
6.0 |.1
1.9 6600
8,9 13.1
Ia-gar
3.0
2.7
TS-3b 3b
3B
4.9
4.0
2.0 1.0
4.9 668
1.7 6.6
8,3 5.6
GPT-J
4.3
2.7
5.1
10.9699 6.9
2.1
OPT-306
30B
18.3
6.3
16.0
11.42.
5.3 5.1
16.3 6.9
Transfer-learning
UnifiedQA -asse
220M
3.6
UnifeeQQA-large
770M
12.9
46.5
8.2
36.6
14.2
42.7
3.8
36.3
13.7
74.6
UnifiedQAA38 3b
3B
11.5
48.1
9.9
41.8
17.0
47.0
4.4
37.6
18.6
80.0
Table Performance of various models on the MC classification tasks (accuracy) and open-domain QA tasks (QA-F1). Performances without/with knowledge external knowledge are presented be- fore/after the vertical bar, respectively. UnifiedQA-X denotes T5 models with corresponding sizes fine-tuned on the UnifiedQA dataset.
UniiiedQA--an hase
UnifiedQA-Iaage
UnifiedQA.33
Dataset
25%
75%
75%
50%
AGNews
50.7
| 55.6
55.6
$2.8
56.3
55.0
56.8
70.2
69.1
69.4
70.2
68.7
70.6
77.9
78.4
80.1
880.4
82.3
82.3
e-SNLI
46.5 66.6
54.4
68.3
62.3
69.6
50.7
71.1
72.2
66.4
73.2
69.1
86.3
75.9
87.5
82.8
88.8
CIKQA
56.9 59.6
57.8
59.6
58.7
59.9
60.2
62.1
60.8
62.3
61.5
62.4
62.7
66.9
64.1
66.9
65.5
66.9
QuateggyA
50.7 55.6
55.0
56.8
$2.9
62.1
61.9
65.9
77.0
82.9
BoolQ
65.5 76.2
70.7
79.9
75.8
80.9
63.9
77.7
72.1
81.3
78.3
84.4
68.1
79.1
74.6
85.7
81.2
87.1
ARC-E
50.7 55.6
52.8
56.3
55.0
56.8
64.5
64.6
65.0
64.7
63.5
65.1
74.4
74.6
75.1
74.9
75.8
75.1
ARC-C
44.9 43.8
45.0
44.5
45.1
44.8
53.8
50.8
52.3
51.2
50.9
51.5
64.5
63.9
64.4
64.9
64.3
65.6
WQ
19.2 26.3
27.5
42.1
35.8
43.8
22.5
38.5
30.5
39.0
38.5
46.0
20.9
19.3
30.0
35.4
39.1
46.4
TREC
13.5 33.6
21.3
36.4
29.1
36.9
30.8
32.7
36.0
34.6
36.3
27.0
40.6
34,4
40.9
HatexQA
25.2 32.9
30.2
35.5
35.2
37.8
26.7
35.2
32.1
37.5
37.4
40.2
24.9
41.9
32,3
43.9
39.7
45.7
T
32.0
52.7
56.4
54.4
00.0
32.4
59.7
64.3
60.5
71.8
08.3
52.8
71.0
66.4
73.4
NQ
20.0
33.0
24.9
33.5
29.7
33.9
12.0
34.8
20.1
35.2
28.2
35.7
12.8
35.9
21.1
36.5
29.4
37.0
Table 3: Performance of IAPEK leveraging Thrust. With 25%, 50%, and 75% percent instances augmented with their corresponding knowledge, performances random/Thrust are presented be- fore/after the vertical bar. If performance increases with Thrust, the score will be marked in green and otherwise red. WQ, TREC denote the tasks of Web Questions, Curated TREC, respectively.
have enough internal knowledge, any relevant external knowledge can help. As a comparison, the OPT-30b model already contains rich knowledge, and thus the external knowledge may only troducing extra noise if the model does not learn utilize knowledge; (ii) under the transfer-learning setting, UnifiedQA-bent based models get significant benefit from the external knowledge, again showing the effectiveness of helping models to learn
knowledge.
In conclusion, we find that fine-tuning on instances containing external knowledge is an effective way to help models gain performance increase from external knowledge test time. Since the pre-condition of using IAPEK is that the model can utilize external knowledge well, we conduct experiments with UnifiedQA only when evaluating the performance of Thrust.

---

Under review as a conference paper at ICLR 2023
Model
Thrust > Full
Thrust Foll
UnifiedQA-basee
BoolQ(r) ARC-C(e)
CIKQAO)
StrategyQA(hh
e-SNIJ())
ARC-R(1)
WQ(?)
TriviaQA(e)*
AGNNewstg)
TREC(r)
HotpoiQA(g)*
NQU)
BoolQ
CIKQAG)
ARC-C(F)
C-SNLIh)
StrategyQACh"" WQGF)
AGNews(g)*
Uni
Holpo(QA(g)
ARC- E(C
TREC(r
TriviaQA(1)*
NQG!)"
Table 4: Comparison between using Thrust and the costly full knowledge usage for models with different scales. The knowledge type is noted in bracket, where denotes gold knowledge, h denotes human annotated knowledge, and denotes the knowledge retrieved from Wikipedia passages or knowledge graphs. If the performance difference is less than 1% accuracy or QA-FI, we denote the entry with * WQ, TREC denote the tasks of Web Questions, Curated TREC, respectively. We find that for many cases Thrust can outperform full knowledge usage with fewer instances augmented with external knowledge.
3.4 PERFORMANCE OF THRUST
From the results in Table EL we can observe that: (i) Thrust consistently contributes to the per- formance from the base to the 3B model. Through clustering the instances, acquire the whole instance distribution in the eyes of the models. Then with distance to the cluster, Thrust represents how well the model can categorize new query vector and find its similarity with others the task. Leveraging such information, Thrust identifies the no knowledge and controversial knowledge cases well and puts the knowledge into the most necessary ones; (ii) the gain is higher when the portion of augmented instances is smaller. For instance, for UnifiedQA-3b, the gains from Thrust with 25% instances augmented with knowledge are 6. 1%, 13.56% on MC classification and QA tasks, respec- tively, while for the 75% case, the gains are 2.8% and 6.8%. Such observation shows that Thrust is most effective on identifying the most necessary cases. One potential reason is that Thrust is sen- sitive to the distance change so the isolated instances (no knowledge case in Figure 2) can be easily identified; (iii) we also observe consistent failure case on ARC-C. The reason can be that the queries designed open questions, and the answers are usually about plans ideas, not facts, so that it is hard for the small-size models to extract useful information from the seemingly unrelated Wikipedia document. For instance, a query from ARC-C is: Juan and LaKeisha roll few objects down a ramp. They want to see which object rolls the farthest. What should they do so they can re- peat their investigation?. The correct and wrong options are Record the details of the investigation and Choose differeni objects to roll. For questions of this style, it is even hard for humans to find a relevant Wikipedia page that can help. The failure case further sheds light on the pre-condition of Thrust: we assume that external knowledge is useful.
As for efficiency, following the definition in Section MM at test time, the computation complexity for Thrust is O(N K), where N is the number of labels and K is the number of clusters, for a five-way classification task, N K 15. However, if we retrieve knowledge from external corpus, the com- putation complexity is O (MM. where M ~ 1, 000, 000 for DPR. In this case, expected performance N K), with a as the probability of rejecting the knowledge use for one example, which is 0.25 to 0.75 in our experiment. On the other hand, extracting knowledge from other sources (e.g., human annotations for e-SLLI can be potentially more consemnnnuingg
improvement (a
4 ANALYSIS
4.1
COMPARISON WITH FULL KNOWLEDGE USAGE
We denote simply using external knowledge for all instances as a costly but straightforward way of leveraging external knowledge. Since the big models might be sufficient for certain instances and the external knowledge might introduce extra noise, we hypothesize that, in some cases, Thrust can help identify instances requiring (or not) knowledge and achieve higher overall performance on the whole dataset compared to seeking and adding knowledge indiscriminately. Table g presents the compar- ison between adaptive and indiscriminate knowledge propulsion. Thrust here denotes the best per- formance achieved when less than 90% of instances use external knowledge. We could observe that, for 50% and 30% tasks for UnifiedQA-base and UnifiedQA-33 respectively, Thrust achieves better
7

---

solve, also Under review as a conference paper at ICLR 2023
task
bookq
cikga
holpolqa
triviaça
ng
Figure 3: Distribution of Thrust scores for various tasks with UnifiedQA-3b to create the instance representation. The distribution is normalized by Kernel Density Estimation. Low scores denote the cases where internal knowledge is not enough, vice versa. We find that Thrust scores predict that more cases from HotpotQA require external knowledge and more cases from e-SNLI do not require external knowledge compared to other tasks.
performance than using knowledge for all instances. Such results indicate that Thrust can help avoid potential noise. On the other hand, we can also observe that for the e-SNLI, Web Questions, and NQ, the full knowledge setting performs better than Thrust. The reason behind this be that external knowledge is essential and of high quality (e.g., the used knowledge is manually written rather than retrieved). As long as the model can comprehend the external knowledge, seeking and adding more high-quality knowledge always benefit the models.
4.2 DISTRIBUTION OF THRUST ACROSS TASKS
To further understand how Thrust leads adaptive knowledge propulsion, we select the top tasks with the highest improvement after applying IAPEK with Thrust from MC classification and open-domain QA tasks, respectively. The selected tasks are: e-SNLI, BoolQ, CIKQA, HolporQA, TriviaQA, and NQ. Figure B demonstrates the distribution of Thrust scores for each of the involved tasks. The scores are cast to [0, 1] with respect to the extremum, and the distribution is normalized by Kernel Density Estimation. From the figure, needs external knowledge) appear commonly for the open-domain QA tasks such as HotporQA and TriviaQA. CIKQA queries, which are designed to require commonsense knowledge need external knowledge for many cases, as predicted by Thrust. On the other hand, for e-SNLI and BoolQ, external knowledge is not always necessary. Such findings demonstrate the potential of using Thrust to investigate the characteristics of tasks from the knowledge aspect.
observe that low scores (i.c., the query
4.3 LAYER ABLATION
Since we cast instances into the representation space, a crucial contributing factor for Thrust the layer of the PTLM to use. To investigate the effect of which layer to use, we conduct experiments on UnifiedQA-33 with the same setting as in Section 3.T. Figure presents the performance of adding 25%, 50%, 75% knowledge-augmented instances with Thrust using the hidden states of different layers. We can observe that, for most tasks, there no significant difference across lay- ers, which shows the robustness of Thrust and potential capacity accelerate the computation of Thrust of using lower layers. However, for some tasks, such as StrategyQA and Web Questions, the middle-layer representation may worsen the overall performance. The reason be that: early layers in the model contain rich semantic information, and later layers contain task-specific informa- tion (Lovering et al, (02T), so that both can act as good representation of the instances. However, in the middle layers, rich semantic features are abandoned during extracting task-specific features and task-specific features are also not fully extracted and expressed yet.
RELATED WORK
PTLM with external knowledge. The paradigm of retrieving knowledge from knowledge bases, augmenting PTLMs, and solving downstream tasks has been widely explored in the community of NL.P (Lewis et all 2020: [Borgeaud all, 2022: Izacard et all 2022). The knowledge bases can
8

---

Under review as a conference paper at ICLR 2023
80
133k
bodiq
80
acbquestions
srategyea
50
curatedtrec
agnews
hotpotga
arc-hard
portion
30
portion
25
25
layer
layer
(a)
(b)
Figure 4: Layer-wiss ablation across tasks and portions of instances augmented with knowledge for (a) MC classification tasks and (b) open-domain QA tasks. The X-axis denotes the layer of Unified- 3b decoder used to cast instances into the representation space. We find that for most tasks, the layer number will not be significant factor. For some tasks (e.g.. Strategy QA), choosing middle layers for representation slightly worsens the performance.
range from knowledge graphs (Xu et all (021)).. documents (Paranjape et al, ((22).. pre- pre-peceesed vectors (Verga all (02II), other PTLMs (Shwartz all 2020), search engines (Nakano et all 2021), to Wikipedia documents as used in this work. To augment the PTLMs, common practice includes creating synthesizing datasets (Wu all (021), adding knowledge to the prompts (Weil 0022: Nye et all 2)2T),, create demonstrations (Brown el at (02I), and extending feature
vectors (Kaattab & Zabari ((2()).
The contribution of IAPEK is orthogonal to above work, gated framework to reject annotations or retrieval. Since Thrust allows test time estimation on the queries, without labels nor gold answers, the method can be extended to any of these aforementioned settings.
Hardness and Confidence Estimation in PTLMs. Much previous work studies the estimation of dataset hardness and model confidence under the context of PTLMs. For dataset hardness, RDA (Perez et all 2)2 measures the hardness as the cumulative area under the loss curves of cross-fold validation the test Point-wise V-Usable information (Ethayyraaj et al, (0zE)) com- putes the hardness as entropy difference between the feature-provided case and the blank feature case. Sensitivity Measurement (Hahn all, (02II measures the dataset difference by computing the variance of loss of the correct labels on a set of neighbor sentences extracted from generative models with masked original sentences as the inputs. These methods achieve great correlation with the model performance. However, all these methods focus on analyzing the test set performance, thus the test set labels are required and can be applied when predicting the answers. Another line of work focuses on estimating the expected calibration errors (ECE) for classification (Kong et all, 2021), QA (Jaag et all 2021), and math al 2022) datasets, as a reflection model certainty on the correct answers. ECE can be considered as orthogonal evaluation metric measure the model's 's capability of understanding the tasks, compared to common metrics such as accuracy.
Most of the previous work be considered posterior analysis of the model capability. In this work, instead, we estimate the pragmatic confidence at the test time to empirically increase the performance with limited budget or bandwidth acquire knowledge.
6 CONCLUSION
In this work, we propose Instance-level Adaptive Propulsion of External Knowledge (IAPEK) as solution to propel model performance when the external knowledge is useful but noisy. Accordingly, propose simple and effective mnstance-wise metric, Thrust, to perform the adaptive knowledge injection. Extensive experiments show that Thrust can improve the performance of utilizing external knowledge under various settings. Understanding the delicate usage of potentially noisy knowledge for PTLMs can further enable the models to conduct inference beyond the limitation of implicit internal knowledge.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Petr Baudiš. Yodaqa: a modular question answering system pipeline. In POSTER 2015-19th Inter- national Student Conference Electrical Engineering, pp. 1156-1165, 2015.
Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. Semantic parsing on Freebase from question-answer pairs. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1533-1544, Seattle, Washington, USA, October 2013. Association for Computational Linguistics. URL
Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Mil- lican, George Bm Van Den Driessche, Jean-Bape -aptistt Lespiau, Bogdan Damoc, Aidan Clark, Diego De Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Maggiore, Chris Jones, Albin Cassirer, Andy Brock, Michela Paganini, Geoffrey Irv- ing, Oriol Vinyals, Simon Osindero, Karen Simonyan, Jack Rae, Erich Elsen, and Laurent Sifre. Improving language models by retrieving from trillions of tokens. In Kamalika Chaud- huri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Pro- ceedings of the 39th International Conference Machine Learning, volume 162 of Proceed- ings of Machine Learning Research, pp. 2206-2240. PMLR, 17-23 Jul 2022. URL nEEps: l/proceedings. www.s.ppes........ V162 borgeaudzza. html.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar- wal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, MMrk Chen, Eric Sigler, Ma- teusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCan- dlish, Alec Radford, llya Sutskever, and Dario Amodei. Language models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Ad- vances in Neural Information Processing Systems, volume 33, pp. 1877-1901. Curran Asso- ciates, Inc., 2020. URL 1457c0d6bfcbb cb49 741 41111118111116666-20000 pdi.
neur iis. htcppapprr/2220.
Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. e- snli: Natural language inference with natural language explanations. In S. Ben- gio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Gamett (eds.), Advances in Neural Information Processing Systems, volume 31. Curran Asso- ciates, Inc., 2018. URL http:////.....aaeooo.o.r..reraar
pdd.
Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no questions. In NAACL, 2019.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. ArXiv, abs/ toss/880.00557, 2018.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HILT 2019, pp- 4171-4186, 2019.
Kawin Ethayarajh, Yejin Choi, and Swabha Swayamdipta. Understanding dataset difficulty with V-usable information. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.). Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learming Research, pp. -888-6008. PMLR, 17-23 Jul 2022. URL https: httpss//proceeeii ngs. wtt.m.r.ppp. w.w.pppe///... b2t. I.
Luyu Gao and Jamie Callan. Condenser: pre-training architecture for dense retrieval. In Proceed- ings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 981-993, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10. 10.18655/112222..cm mannl... URL hteps: http:s:///cceee www.aaannoooooooooooooooo
2021 emn wwpppaan...... 75.
10

---

Under review as a conference paper at ICLR 2023
Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies. Transac- tions of the Association for Computational Linguisties (TACL), 2021.
Michael Hahn, Dan Jurafsky, and Richard Futrell. Sensitivity as a Complexity Measure for Se- quence Classification Tasks. Transactions of the Association for Computational Linguistics, 9: 891-908, 08 2021. ISSN 2307-387X. doi: 10. 162/ttcl_aaaaaaaaaaa
URL https:/ //0 orq/
10.1162/tac1 00403
Gautier Izacard, Patrick Lewis, Maria Lomeli. Lucas Hosseini. Fabio Petroni, Timo Schick, Jane Dwivedi- Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. Few-shot learning with re- trieval augmented language models. arXiv preprint arXiv:2208.
2022.
Zhengbao Jiang, Jun Araki, Haibo Ding, and Graham Neubig. How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering. Transactions of the Association for Computational Linguisties, 9:962 977, 09 2021. ISSN 2307-387X. doi: 10.11622tacla Lacee_____000 004077 URL nEtps://GO1 www.oo.g///.. 1wt.:..111.aac.c.. a 00407.
Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. TriviaQA: A large scale dis- tantly supervised challenge dataset for reading comprehension. In Proceedings of the 55th An- mual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp- 1601-1611, Vancouver, Canada, July 2017. Association for Computational Linguistics. doi: 10. N0.18653//11PP17-1144 URL htt.::ttt:////t /tops//aace ac aatcaaanto.... ogy. www.oo//g////11111
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP), pp. 6769-6781, Online, November 2020. Association for Computational Linguistics. doi: 10. 10118855311/20000 1000000//-122200.mmmmmmmmm 2020 emn p- ww..maiin............m......m... 550.
URL https: /TEEp::/wwww.cc.. www. wwwwb...ww.................. azg.athoooy/
Nora Kassner and Hinrich Schütze. Negated and misprimed probes for pretrained language models: Birds can talk, but cannot fly. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 7811-7 Online, July 2020. Association for Computational URL bttps://aclantho
Linguistics. doi: 10.
ore/
2020 ac main. 6.88
D. Khashabi, S. Min, T. Khot, A. Sabhwaral, O. Tafjord, P. Clark, and H. Hajishirzi. Unifiedqa: Crossing format boundaries with single qa system. 2020.
Omar Khattab and Matei Zaharia. Colbert: Efficient and effective passage search via contextualized late interaction over bert. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '20. pp. 3948, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781450380164, 3401075. URL https: /ttp:::/ddoi.oooo 1...0..03/1...1553999131. 0.45 3401007
doi: 10.1145/3397271.
Omar Khattab, Christopher Potts, and Matei Zaharia. Building scalable, explainable, and www.bloodeeu/bllc//
adaptive nlp models with retrieval, 2022. URL
Vid Kocijan, Ana-Maria Cretu, Oana-Maria Camburu, Yordan Yordanov, and Thomas Lukasiewiez. A surprisingly robust trick for the Winograd schema challenge. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4842, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10. 10.11865/11/11111
URL https:
Lingkai Kong, Haoming Jiang, Yuchen Zhuang, Jie Lyu, Tuo Zhao, and Chao Zhang. Calibrated lan- guage model fnnettnnig for in- and oft-oof-dstrrbbun data. In Proceedings of the 2020 Confer- ence on Empirical Methods in Natural Language Processing (EMNLP), pp. 1326-1340, Online, November 2020. Association for Computational Linguistics. doi: 10.18653/v 1/2020.emnlp main. 102. URL https: //acl antho logy org 2000 emn p--ma ma i n. 102.
11

---

Under review as a conference paper at ICLR 2023
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-War Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le. and Slav Petrov. Natural questions: A benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:452 466, 2019. doi: 10. 10.1162/tacl_ a a_0766. URL
1026
Patrick Lewis, Ethan Perez. Alcksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retreeval-augmented generation for knowledge-intensive nip tasks. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS' 20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546.
Stephanie Lin, Jacob Hilton, and Owain Evans. Teaching models to express their uncertainty in words, 2022. URL https:
Yinhan Liu, Myle Out, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer. and Veselin Stoyanov. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907..a 1692, 2019.
Charles Lovering, Rohan Jha, Tal Linzen, and Ellic Pavlick. Predicting inductive biases of pre- trained models. In International Conference Learning Representations, 2021. URL nttps:
//openreview nwt//ett/uu/rte B
Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman. Webgpt: Browser-assiited question-answering with human feedback. CoRR, abs/2112.093322 2021. URL
https://arxiv: wwt./oo//////ooo
09332.
Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. MS MARCO: A human generated machine reading comprehension dataset. CoRR, abs/1611.099 1.09268, 2016. URL http://arxiv.orrt org.eivv.rrr/aaoo wtwp//////aaoo.oaooo.ooeaee wtw.b////1..... 09268.
Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, Charles Sutton, and Au- gustus Odena. Show your work: Scratchpads for intermediate computation with language models. CoRR, abs/2112.00114 2021.
00114.
Ashwin Paranjape, Omar Khattab, Christopher Potts, Matei Zaharia, and Christopher D Manning. Hindsight: Posterior-guide training of retrievers for improved open-ended generation. In In- ternational Conference on Learning Representations, 2022. URL https: https://oppnrevvess /opperevie......
het//ooum?id=vr_. BTww333
Ethan Perez, Douwe Kiela, and Kyunghyun Cho. Rissanen data analysis: Examining dataset char- acteristics via description length. In ICML, 2021. URL nttps::/aaxxiii wwwpwwwwwwww///////...3333
03872
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. Language models as knowledge bases? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Confer- ence on Natural Language Processing (EMNLP-IJCNLP)) pp. 2463-2473, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18653/v1/D19-122 URI. htt.ps://aclaatte wtw.:/laatttttoooooooooo.oooo wot..oog/.....11110/./
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. OpenAl Blog, 1(8):9, 2019.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with unified text-to- text transformer. Journal of Machine Learning Research, 21(140):1-67, 2020. URL kEEP:
7/iml wwwp://pppppppooeeooo //.///21/220 074 html.
12

---

Under review as a conference paper at ICLR 2023
Melissa Roemmele, Cosmin Adrian Bejan, and Andrew S Gordon. Choice of plausible altemaaties: An evaluation of commonsense causal reasoning. In AAAI spring symposium: logical formaliza- tions of commonsense reasoning. pp. 90-95, 2011.
Keshav Santhanam, Omar Khattab, Jon Saad-Falcon Christopher Potts, and Matei Zaharia. Col- V Effective and efficient retrieval via lightweight late interaction. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguis-
ties: Human Language Technologies, pp. 3715-3734, Seattle, United States, July 2022. Asso- ciation for Computational Linguistics. doi: w/tp://////22.aaa2lllllao maw.l...naacccc................. mataaac-mman......
URL nEERS:
/aaclancholoyy.crkt
Maarten Sap, Ronan LeBras, Emily Allaway, Chandra Bhagavatula, Nicholas Lourie, Hannah Rashkin, Brendan Roof, Noah A. Smith, and Yejin Choi ATOMIC: an atlas of machine com- monsense for if-then reasoning. CoRR, abs/1811.00146, 2018. URL attp://arxix
abs
Vered Shwartz, Peter West, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Unsupervised commonsense question answering with self-talk. In Proceedings of the 2020 Conference on Em- pirical Methods in Natural Language Processing (EMNLP), pp. 4615-4629, Online, November 2020. Association for Computational Linguistics. doi: 10. 111113/1122200.mm/ll. main.373. URL
https: antho ogy. org. 2020 emn p- 1 373.
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. CommonsenseQQ:: A ques- tion answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Con- ference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies. Volume (Long and Short Papers), pp. 4149-4158, Minneapolis, Min- nesota, June 2019. Association for Computational Linguistics. doi: 10.18653/11/11-11444 URL
Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In Thirty-fift Conference on Neural Information Processing Systems Datasets and Benchmarks Truck (Round 2), 2021. URL httphtt://ppeeeeeeeeeeee
net/forummid-wwe
jeJ.
Andrew Trotman, Antti Puurula, and Blake Burgess. Improvements to bm25 and language models examined. In Proceedings of the 2014 Australasian Document Computing Symposium, ADCS "1, pp. 5865, New York, NY, USA, 2014. Association for Computing Machinery. ISBN 9781450330008. doi: 10.1145/2682862.2622222 26828863.
URL
2642862. 2682H63
Pat Verga, Haitian Sun, Livio Baldini Soares, and William Cohen. Adaptable and interpretable neu- ral MemoryOver symbolic knowledge. In Proceedings of the 2021 Conference of the North Amer-
ican Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 3678-3691, Online, June 2021. Association for Computational Linguistics. Linguttttsss doi: 10. 18653vv// n02lln main.288. URL
wrg.oyyorrg/22110 naac 1m..lmmmmaa..........m..........
Ben Wang and Aran Komatsuzaki. GPT-J-6B; 6 Billion Parameter Autoregressive Language httt::///atcbbbb..o////oooooo//o htw.://oooooooommmeee---aanreerr 1ax, May 2021.
Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Ed Chi, Quoe Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. CoRR, 0bs/22011.19900 2022. URL Ettps: /ttp::///rri.v..rcc/b////22
Yuhuai Wu, Markus N Rabe, Wenda Li, Jimmy Ba, Roger B Grosse, and Christian Szegedy. Lime: Learning inductive bias for primitives of mathematical reasoning. In International Conference on Machine Learning, pp. 11251-11262 PMLR, 2021.
Yichong Xu, Chenguang Zhu, Shuohang Wang. Siqi Sun, Hao Cheng, Xiaodong Liu, Jianfeng Gao, Pengcheng He, Michael Zeng, and Xuedong Huang. Human parity commonsensequ: Augmenting self-attention with external attention. CoRR, abs/2112.03254, 2021. URL netps: 7/arxi sarxiv.ore wttps//orr//ab///1111 03254.
13

---

Under review as a conference paper at ICLR 2023
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. HotpotQA: A dataset for diverse, explainable multi-hop question an- swering. In Proceedings of the 2018 Conference Empirical Methods in Natural Language Processing, pp. 2369-2380, Brussels, Belgium, October-November 2018. Association for Com- putational Linguistics. doi: 10. 10.18653/11/D11--2235 URL https::/acllattee eqy.
D18- 1259
Da Yin, Li Dong, Hao Cheng, Xiaodong Liu, Kai- Wei Chang, Furu Wei, and Jianfeng Gao. A survey of knowledge-intensive nlp with pre-trained language models. ArXiv, abs/2202.08772 2022.
Hongming Zhang, Yintong Huo, Yanai Elazar, Yangqiu Song, Yoav Goldberg, and Dan Roth. CIKQA: Learning commonsense inference with unified snowledge-in-the-lodd QA paradigm. In ACL 2022 Workshop on Commonsense Representation and Reasoning, 2022a. URL neEbS:
/ooeerrvienn net. wttts//frmmmmmmmmm
1ztg 65.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christo- pher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang. and Luke Zettlemoyer. Opt: Open pre-trained transformer language models, 2022b.
Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. nett (eds.), Advances in Neural Information Processing Systems, volume 28. Curran Asso- ciates, Inc., 2015. URL htttpss://prooceedinnng meur https//eeeeesspcpccaaaa/////// pos.s//////aaae///1////111 45
Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Gar-
14

---

Under review as a conference paper at ICLR 2023
A
APPENDIX
14
task
agnews
12.
boolq
cikqa
10
esnli
strategyqa
arc-easy
arc-hard
webquestions
curatedtrec
hotpotqa
triviaqa
ng
0.
-250
-0.25
0.0
0.75
1.00
125
150
Brust
Figure 5: Distribution of Thrust scores for all involved tasks with UnifiedQA-3b to create the in- stance representation. The distribution normalized by Kernel Density Estimation. Low scores denote the cases where internal knowledge is not enough, vice versa.
BM25
Thrust
Dataset
25%
50%
25%
75%
AGNews
77.9
77.0
80.1
79.2
82.3
81.3
77.9
78.4
80.1
80.4
82.3
82.3
75.6
86.3
87.5
CIKQA
62,7
62.3
64.1
64.4
65.5
66.1
62.7
66.9
64.1
66.9
65.5
66.9
StrategyQA
63.3
70.5
68.3
77.0
78.1
74.3
70,5
81.4
82.9
BoolQ
68.1
68.4
74.6
75.9
81.2
82.0
68.1
79.1
74.6
85.7
81.2
87.1
ARC-E
74.4
74.9
75.1
75.3
75.8
76.3
74.4
74.6
75.1
74.9
75.8
75.1
ARC-C
64.5
65.2
644
66.2
64.3
66.6
64.5
63.9
64.4
64.9
64.3
65.6
WQ
20.9 19.0
30.0
28.2
39.1
37.3
20.9
19.3
30.0
35.4
39.1
46.4
27.0
28.1
HorpotQA
24.9
25.2
32.3
32.8
39.7
40.6
24.9
41.9
32.3
43.9
39.7
45.7
TriviaQA
39.2
34.2
52.8
50.0
66.4
65.4
39.2
08.3
528
71.0
73.4
NQ
12.8
12.9
21.1
21.1
29.4
29.6
12.8
35.9
21.1
36.5
29.4
37.0
Table 5: Performance of IAPEK with UnifiedQA-3b leveraging Thrust and BM25. With 25%, 50%, and 75% percent instances augmented with their corresponding knowledge, performances of random/BM25 and randon/Thrust are presented before/after the vertical bar. If performance increases with Thrust, the score will be marked in green and otherwise in red. WQ, TREC denote the tasks of Web Questions, Curated TREC, respectively. We can observe IAPEK performs good with BM25 as the difficulty score on QA tasks. However, the performance of BM25 is worse (on QA tasks) and less robust (on MC classification tasks) comparing to our Thrust.
We use BM25 (Troiman et all 20))), common approach evaluate the difficulty of queries, as an alternative to Thrust to perform TAPEK. Specifically, we regard each test input the query and all training data input as the corpus to extract the score. We use the average of the relevance score across the corpus to rank each test input. From Table EL we can observe that BM25 leads to performance improvement many cases, Thrust shows better (e.g., for QA tasks) and more robust improvement (e.g., s., for classification tasks).
15


Reviews:
[{"review_id": "jAn0CEQL4FQ", "reviewer": "Reviewer_sAvG", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work focuses on improving the use of knowledge in knowledge-augmented Pre-Trained Language Models (PTLM). Authors hypothesize that using knowledge for all instances can backfire and lead the model towards incorrect predictions, but simultaneously for some instances, external knowledge is required. To combat this issue, a new metric, Thrust, is proposed to select instances where external knowledge is more important. Authors test this new method for efficiency and performance improvements on several datasets and models.", "summary_of_the_review": "Authors have proposed a new metric to effectively select cases which require external knowledge. Authors have done a decent job of evaluating the method with current literature, however a few important evaluations are missing. Since this paper revolves around retrieval based LMs, authors should have done more evaluation on using thrust in conjunction with some of the well known retrieval based LMs.\nOverall, I think in it's current form this work requires more refinement in terms of more robust evaluation and I would vote for rejecting this paper.", "strength_and_weaknesses": "Strengths\n\n+ The core hypothesis of the work makes sense to me, especially for larger models.\n+ Proposed method is quite simple and works without further fine-tuning or training and offers efficiency gains.\n+ Authors show that using thrust is better at selecting instances which require knowledge than just random selection.\n\nWeaknesses / Questions\n\n- Evaluation with recent models like RETRO [1] and Atlas [2] is missing.\n- Table 2, if my understanding is correct, with knowledge case is presented after the vertical bar (|). If that's true, please fix the description of the model which states the opposite.  \n- Table 2, is the with knowledge same as using full knowledge?\n- Table 4, when comparing thrust with full knowledge, what are the exact performance numbers? Please mention the evaluation times for both cases as well. \n\nCitations:\n\n[1] Borgeaud, S., Mensch, A., Hoffmann, J., Cai, T., Rutherford, E., Millican, K., Driessche, G.V., Lespiau, J., Damoc, B., Clark, A., Casas, D.D., Guy, A., Menick, J., Ring, R., Hennigan, T.W., Huang, S., Maggiore, L., Jones, C., Cassirer, A., Brock, A., Paganini, M., Irving, G., Vinyals, O., Osindero, S., Simonyan, K., Rae, J.W., Elsen, E., & Sifre, L. (2022). Improving language models by retrieving from trillions of tokens. ICML.\n\n[2] Izacard, G., Lewis, P., Lomeli, M., Hosseini, L., Petroni, F., Schick, T., Yu, J.A., Joulin, A., Riedel, S., & Grave, E. (2022). Few-shot Learning with Retrieval Augmented Language Models. ArXiv, abs/2208.03299.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "cAwNe8GLlkH", "reviewer": "Reviewer_xs9i", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work aims to improve the efficiency and robustness of knowledge augmented LM.\nThe intuition is for LM to  decide when an external  knowledge source is needed.\nA metric Thrust is developed for this decision, based on the relationship between the query embedding and the clusters of instance embeddings.\n1) representation learning (the detail of which is not clear)\n2)  k-means clustering on training instance embeddings (the training data is not formally defined)\n3)  the Thrust score of a query is computed based on its distance to cluster centers and the length of individual instance embeddings. (the intuition is given in Figure 2, but I have hard time connecting that with the given formula)\n4) (I assume some procedure \"adaptively\" filter queries by their Thrust score, but I cannot find the description.)\n\nExperiment is conducted with several LMs (T5, GPT-J, OPT, UnifiedQA) under zero-shot and transfer-learning setttings for QA and classification tasks.\n\nRetrieval is conducted to the top 25%, 50% or 75% queries based on the Thrust score.\n", "summary_of_the_review": "This paper is very unclear.", "strength_and_weaknesses": "Overall the idea of this work is novel, but the description is very hard to follow.\nThe lack of formal definitions, make it hard to  understand how thrust is computed\ne.g., \"casting a set of instances ( the training data ) into the representation space \"\nThe writing is mostly sloppy, e.g., c_0 is used before it is defined\n\nThe experiment result also need more explanations.\nFor example in Table 3 it is good to see that using knowledge 25% of the time is as good as using it 75% of the time for many tasks. However, one might wonder\nwhy only the result of UnifiedQA is shown?\nwhy not also compare to the case using knowledge 100% of the time?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "YAVt8ZY4ynP", "reviewer": "Reviewer_xVMQ", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper uses large pre-trained generative language models to perform tasks of text classification or question answering.\nAs other works, in classification, they feed the text as prompt, and predict the class with the maximal next-token-probability by the model.\nThe authors argue and show that by adding to the query some relevant background knowledge in textual form (this text, which is obtained from various sources is referred to as `external knowledge'), performance often improves. \nAnd this leads to the following main claimed contribution of the paper. Rather than incorporate external knowledge for any query, the authors propose to assess the difficulty of addressing each query, and only model external knowledge for the hardest queries. To achieve this, they first cluster the training data within the embedding space, and measure the distance of the query representation to these clusters. \n\nThe experiments apply to several language models (including T5, GPT-J and a couple more), either in zero-shot learning setting, or when first presenting the models some labeled examples (fine tuning). Several tasks are evaluated, where the source of `external knowledge' differs per task. For example, in some cases, the document title is the query, and the document content is modeled as external knowledge; in other cases the knowledge pertains to human-authored explanations (on the task of detecting textual entailment between two sentences), relevant human-authored facts (question answering), or facts derived from a knowledge graph.\nResults are reported with and without external knowledge. In addition, results are reported when extending only the hardest 25/50/75 percent of the queries using the proposed approach for detecting hard queries. The results are overall positive.", "summary_of_the_review": "The ideas are interesting, but the writeup is not solid. The presentation of ideas and experiments lack comparison with existing works. \nSome terms are not well defined or misused. Reproducibility is a main issue.", "strength_and_weaknesses": "Strengths: \n- The topic and general ideas are interesting\n\nWeaknesses:\n- A main weakness is that the work is not reproducible, with respect to both the methods and datasets.\n- Clarity: some important terms are not defined at all or until later in the paper, including the key term of external knowledge (which commonly means something else actually, as in knowledge graphs).\n- The experiments miss some important baselines.\n- There are some technical inaccuracies, e.g., the definition of the F1 measure is incorrect.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Vm1CPaMeUX", "reviewer": "Reviewer_hhnt", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the authors point out two limitations in using external knowledge with pretrained language models (PLMs): (1) it is time-costly to index and retrieve on large-scale knowledge bases, and (2) retrieved knowledge could be noisy and misleading. With these limitations as the motivation, they propose a scoring metric named Thrust, which uses the weighted sum of the unit vectors pointing from the query vector (numeric vector for binary classification) to the centroids of the clustered vectors of the training instances which serve as the knowledge to retrieve. The weight is calculated as the size of each cluster over the square of the distance between the query and cluster centroids. The method of using external knowledge in this paper is to use the given knowledge instance as a part of the input through concatenation. As the main experiment to show the effectiveness of the suggested method, the authors came up with an experimental setup where only a portion (25%, 50%, 75%) of the queries could be answered with external knowledge, and used the scoring metric to evaluate all the queries and then choose the portion of the queries with the low Thrust scores. The proposed performs better than the method of randomly choosing the queries to use external knowledge.", "summary_of_the_review": "The authors’ motivation for this work is interesting, but their proposed method and experimental setup do not align well with their motivation. While they claim to simulate real-world cases, the experimental setup is artificial and highly restricted, making it far from real-world cases. The experiments are insufficient to show the effectiveness of the proposed method, making the usefulness of the algorithm questionable.", "strength_and_weaknesses": "**Strengths**\n\n- The proposed method to calculate the Thrust score seems novel.\n- It is interesting to use the Thrust score to see if the task requires external knowledge or not.\n\n**Weaknesses**\n\n- The motivation and suggested method are not well-aligned.\n- The authors claim that they simulate real-world cases, but the experimental setup is highly artificial, making the practicality of the algorithm dubious.\n- The experiments are insufficient to show the effectiveness of Thrust.\n\n**Explanations on Weaknesses**\n\n- The authors claim that they try to simulate the real-world case where they have limited bandwidth to retrieve external knowledge from the perspective of cost-efficiency, but the claim is not persuasive because the way they choose the portion of the queries to use external knowledge is through relative comparison between queries after scoring all other queries, and not through absolute scoring on each independent query, e.g., by applying a score threshold. In other words, the suggested method seems impractical because each query would come to the system independently in the real world, not as a batch as in the experiments. Therefore, if cost-effectiveness is a concern, the authors should have proposed a method that works on each query independently to determine whether that query itself requires external knowledge or not.\n- Comparing the results in Table 2 and 3, on many of the tasks, using Thrust to partially utilize the external knowledge drops the performance compared to when all queries are answered with external knowledge (e.g., 70.2 → 56.8 (75% of queries) on e-SNLI UnifiedQA-base, 80 → 73.4% (75% of queries) for TriviaQA UnifiedQA-3b). Therefore, the cost-effectiveness of Thrust, if it exists, comes with a sacrifice of accuracy for many cases, creating a trade-off. Meanwhile, it is also unclear whether the method is actually cost-effective because the calculation of the Thrust score would also take up some inference time. The authors should compare the average inference speed between when all queries use the external knowledge as the input, and when part of the queries use the external knowledge while the Thrust score is calculated for all queries.\n- The authors only compare Thrust with random baseline, but in order to show the effectiveness of Thrust as a scoring metric, they should compare the algorithm with baselines simpler than Thrust but better than random, e.g., using BM25 scores.\n- [Minor point] While the number of external knowledge instances to utilize and the method of utilizing knowledge might vary, the authors seem to test the effectiveness of Thrust with only one combination of them. The authors did not clearly indicate how many external knowledge instances are used for a query, but it seems like one by inferring from the context. Also, they test only the case of using external knowledge as part of the input and did not explore the cross-attention-based method (e.g., FiD).", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject