The Effect of Scaling, Retrieval Augmentation and Form on the Factual Consistency of Language Models
Authors: Lovisa Hagström
Denitsa Saynova
Tobias Norlund
Moa Johansson
Richard Johansson
Venue: EMNLP 2023 Main (2023)
Keywords: consistency
evaluation
large language models
retrieval-augmentation
causal analysis

Abstract: Large Language Models (LLMs) make natural interfaces to factual knowledge, but their usefulness is limited by their tendency to deliver inconsistent answers to semantically equivalent questions. For example, a model might supply the answer "Edinburgh" to "Anne Redpath passed away in X." and "London" to "Anne Redpath's life ended in X.". 
In this work, we identify potential causes of inconsistency and evaluate the effectiveness of two mitigation strategies: up-scaling and augmenting the LM with a passage retrieval database. Our results on the LLaMA and Atlas models show that both strategies reduce inconsistency but that retrieval augmentation is considerably more efficient. We further consider and disentangle the consistency contributions of different components of Atlas. For all LMs evaluated we find that syntactical form and task artifacts impact consistency. Taken together, our results provide a better understanding of the factors affecting the factual consistency of language models.

The Effect of Scaling, Retrieval Augmentation and Form on the Factual Consistency of Language Models
Lovisa Hagström' Denitsa Saynova' Tobias Norlund
Moa Johansson' Richard Johansson" 1.2
1 Chalmers University of Technology 2 University of Gothenburg {lovhag, saynova)@chalmers..ss
Abstract
subject
relation object
Fact Anne Redpath place of death Edinburgh
Large Language Models (LLMs) make natural interfaces to factual knowledge, but their use- fulness is limited by their tendency to deliver inconsistent answers to semantically equivalent questions. For example, model might predict both "Anne Redpath passed away in Edinburgh." and "Anne Redpath's 's life ended in London." In this work. we identify potential causes of in- consistency and evaluate the effectiveness of two mitigation strategies: up-scaling and aug- menting the LM with retrieval corpus. Our results the LLaMA and Atlas models show that both strategies reduce inconsistency while retrieval augmentation is considerably more ef- ficient. We further consider and disentangle the consistency contributions of different compo- nents of Atlas. For all LMs evaluated we find that syntactical form and other evaluation task artifacts impact consistency. Taken together, our results provide better understanding of the factors affecting the factual consistency of language models.
Template: <Y> expired at <X>
Queries 1, Anne Redpath expired <X>
2. Anne Redpath's 's life ended in <X>.
3. Anne Redpath passed away <x>.
Atlas
Retriever
LLaMA
Reader
Wikipedia
Answers
Southampton
2. London
3. Edinburgh
Figure 1: Overview of how consistency is computed in ParaRel for Atlas and LLaMA.
gests that performance benchmarks for factual information does not measure true factual knowl- edge, since models do not generalize well across rephrased prompts.
1 Introduction
Problems with inconsistency extend beyond fact- related accuracy, as they also indicate a tendency to hallucinate and a lack of factuality in foundational LMs (Ji et al., 2023). As inconsistent models may be fragile in unexpected manners, they are diffi- cult to interpret and are in worse position for providing limitations on what they may generate (Wang et al., 2022).
We have recently observed the development of sev- eral highly performant pretrained large language models (LL.Ms) that have expanded the boundaries of what we can expect from foundational language models. These recent successes have highlighted the potential of using language models as simpler interface factual knowledge (Petroni al., 2019; Dinan et al., 2019).
To promote desirable properties such as robust- ness and interpretability we may need to reconsider our model designs. As an alternative to up-scaling, we have seen the rise of model designs guided by inductive biases to promote various properties (Feder et al., 2022). Examples of such models are text
However, in fact critical settings we require not only high accuracy but also consistency, i.e. robust- ness to lexical variations in semantically equivalent queries. Recent LM developments have mainly improved accuracy, while the question of con- sistency has seen less attention. As exemplified in Figure 1, even SoTA LMs may produce differ- ent outputs depending on lexical variations in se- mantically equivalent queries (Elazar et al., 2021; Ribeiro et al., 2019: 99; Cao et al., 2021). This sug-
models that condition pre- dictions on retrieved text passages for improved adaptability, interpretability and efficiency (Izacard and Grave, 2021; Izacard et al., 2023; Wu et al., 2022), neurosymbolic models that condition predic-

---

tions on logic for reasoning capabilities (Pacheco and Goldwasser, 2021) and fact-injected LMs for interpretabilityy (Verga et al., 2021).
We investigate what extent the reader component depends on term frequencies in the retrieved result and find it to be more inconsistent when this dependence is weaker.
Our goal in this paper is to improve our under- standing of consistency in foundational LMs. We start by investigating methods to improve model consistency. We consider two potential solutions, retrieval
2 Data
1) model up-scaling and 2) Wikipedia augmentation, building on suitable inductive bi- ases related to factual consistency. Furthermore, we identify and investigate potential sources of inconsistency of LMs. Our contributions can be summarized as follows:
We describe the original ParaRel benchmark in Sec- tion 2.1 and our improvements in Section 2.2. Sec-
tion 2.3 explains the metries studied for ParaRel#.
2.1 ParaRel
ParaRel (Elazar et al., 2021) is based on LAMA (Petroni et al., 2019), an evaluation task based on Wikidata that measures factual knowledge stored in LMs through prompting for bubjecctrelation--oe object tuples. ParaRel adds a layer of semantically equiv- alent cloze-style prompts to LAMA, which in turn allows to measure the consistency of LMs with respect to the knowledge tuples represented by LAMA (see Figure 1). The idea that a model is consistent if is invariant to query paraphrases.
We investigate ParaRel, a benchmark for eval- uating consistency (Elazar et al., 2021) and develop an improved version it (Section 2). We refer to this version as ParaRel* through- out this paper. We have removed ambigu- ous fact duplicates and added four query-level metries to estimate inconsistency sources re- lated to evaluation task format, such as lin- guistic form effects (Section 4).
ParaRel measures consistency for N-1 relations, for which there is only one correct object alterna- tive for subject-relation pair, and plausibility for N-M relations, where there are several correct ob- jects for subject-relation pair. We only consider the N-1 I relations that measure consistency. Addi- tionally, following Elazar al. (2021), we simplify the task by restricting the candidate sets. The mod- els are only allowed to generate an answer from the alternatives in the ParaRel* data for each relation.
We investigate the hypothesis that model scal- ing mitigates inconsistency (Section 3.1). Our evaluation of varying sizes of the LLaMA and Atlas models (Touvron al., 2023; Izacard et al., 2023) show that scale improves consis- tency, but with diminishing returns.
We hypothesize that retrieval augmentation mitigates inconsistency and investigate this using Atlas (Izacard et al., 2023)- retrieval- augmented model - along with non-retrieval models of similar scale (Section 3.2). We show that Atlas outperforms all baselines on the ParaRel* task.
2.2 Analysis and Improvements
Analysis of the ParaRel data showed that contains exact duplicate tuples e.g. Audi R8 produced- by Audi This refers once to the Wikidata entity Q758775 (prototype race car) and once to Q758778 (sports model), which not distinguishable by name only. Additionally, duplicated subject- relation pairs (with a different object) also exist in the data. For example, SNES-CD produced-by Sony and SNES-CD produced-by Nintendo are both present in the data. Appendix F presents statistics for the number of such duplicates.
We also investigate causes and correlations related to consistency of Atlas (Section 5).
We define metrics for measuring retrieval consistency and find that the Atlas re- triever components generally is consis- tent and that retrieval consistency corre- lates with prediction consistency.
Through interventions on the retrieval component we find that the consistency and relevance of the retrieved result both affect the consistency and accuracy of the predictions.
Since we aim to use ParaRel to measure con- sistency for N-1 relations, relation instances that occur more than once in the dataset. Relation P37 dfficial-langueee duplicates out of 900 data entries and cannot be considered to be a N-1 relation, why we completely remove it from our updated P version.
remove all subject-
had 280
'Code and datascts available at tttps://ggihuu com/ dsaynova/parare

---

This removal of duplicates resulted in an updated ParaRel* of 30 relations with 21,830 data tuples in total, instead of the original 23,097. Some of the retained relations contained more duplicates than others, but never more than 10% of the data tuples.
alone. Retrieval augmentation from e.g. Wikipedia has been successful for several fact-intensive ques- tion answering tasks (Izacard et al., 2023; Lewis et al., 2020) and reduced hallucination (Shuster et al., 2021; Thoppilan et al., 2022).
Atlas is retrieval-augmentee LM optimized for few-shot knowledge-intensiin tasks. It was devel- oped by Izacard al. (2023) and retrieves pas- sages from Wikipedia and common crawl dump. The model consists of a dense retriever based on the Contriever architecture (Izacard et al., 2022) and reader based on the T5 seq2seq architec- ture (Raffel et al., 2020). The model performs on par with, or even better than, many larger fully- parametric language models on e.g. Natural Ques- tions (Kwiatkowski al., 2019).
2.3 Evaluation Metries
ParaRel provides several statistics that can be used to evaluate model performance. We mainly focus on three: Consistency pairwise agreement be- tween prompts for each tuple; Accuracy accuracy when using the original LAMA prompt: Consis- tent & Accurate what percentage of subjects get assigned the correct object for all prompts.
The consistency metric results measurements for all possible query pairs per tuple and relation. To get one consistency value per model evaluated, we calculate the micro-average of the consistency values across tuples per relation and then take the macro-average across relations, following Elazar al. (2021). The pairwise comparisons used to es- timate consistency imply that other metrics relating to consistency, by correlation or stratification, also need to be on a pairwise query-level.
We evaluate Atlas on ParaRel* and compare its performance to relevant baselines, i.e. non- retrieval-augnee -ugmented model counterparts. We mainly use the base version of Atlas with 330M parameters in our analysis and to some extent also the Atlas-large with 880M parameters, using the model weights released by the authors. We use Wikipedia 2017 passages as the retrieval corpus since this should match the temporal origin of the ParaRel* data and keep the retrieval indices fixed. We retrieve 20 passages per query.
3 Effect of Scaling and Retrieval Augmentation
Since Atlas has been pre-trained with MLM as the pretext task, we can evaluate it zero-shot with some adaptations (see Appendix A).
To investigate the effect of scaling and retrieval augmentation the consistency of LMs we evalu- ate LLaMA (Section 3.1) and Atlas (Section 3.2) on R and report the results in Section 3.3.
To properly investigate the effects of the different model development choices for Atlas on consistency, we need to compare Atlas to rea- sonable baselines. To reason about the benefits of the retrieval augmentation and additional training provided for Atlas, we compare against the fully- parametric T5-base model Atlas was initialized from. 3 To assess the benefits of retrieval augmenta- tion specifically, we evaluate Atlas in a closed-bood setting without augmentation, while we acknowl- edge that the model has not been adapted to this setting.
Baselines
3.1 Effect of Scaling
Model scaling has been a successful approach to improving performance on many NLP tasks. We evaluate LLaMA (7B, 13B, 33B and 65B parame- ters) (Touvron et al., 2023) on ParaRel* (see Ap- pendix A for details on how we do this) and mea- sure whether consistency improves with model size. LLaMA represents traditional state-of-the art LLMs, being decoder-only auto-regressive LMs trillion tokens. We also investigate this for eetgeetal-ugmmentedd models by comparing two sizes of the Atlas model.
trained
We also compare Atlas-bast and BERT-large to estimate the benefits of retrieval augmentation. The latter model should be good representative for the retrieval-free setup as it has an advantage in all aspects except for retrieval augmentation. BERT- large has slightly more parameters than Atlas-base better adapted to the encoder-based ParaRel* task
3.2 Effect of Retrieval Augmentation
We expect atrieval-augmenneed LMs to generally be more consistent than standard LMs. Given consistent retrieval component, the prediction of retrieval- auggeeted model is conditioned on some- thing bound to be more consistent than the query
hnttps://githa http://ttb..oofffaeeeeeeee// google/t5 base-la-addaaa

---

Model
Cons
Acc
C& A
both LLaMA and Atlas. The consistency increases with approximately 1% in LLaMA and 3% in Atlas per doubling in model size. Similar effects of up- scaling were also observed by Elazar et al. (2021). The larger LLaMA models also outperform BERT- large, while they do not outperform Atlas.
atlas-base
0.7410.15
0.80
0.42
atlas-large
0.77:10.14
0.81=0.14
0.17=0.22
atlas-base*
0.60110.23
0.41::0.24
0.2411022
15-base
0.5910.23
0.39
0.22
bert-base
0.58++022
0.46+0.26
0.27+0.24
bert-large
0.60=+023
0.48
0.29
Retrieval augmentation is more efficient than up-scaling in increasing consisteney Atlas-base (330M parameters) performs on par with LLaMA-
llama-7b
0.6710.18
0.6710.21
0.3610.28
Ilama-131 13b
0.68+0.17
0.70=0.21
0.39+0.28
1laba-30b
0.70
0.75=1018
0.42:1.28
65B despite being 90 times smaller. Evidently, retrieval augmentation with Atlas more efficient than up-scaling for increased model consistency. We also note that up-scaling LLaMA yields dimin- ishing returns with respect to consistency. Conse- quently, have found foundational NLP issue consistency for which model up-scaling is not the complete solution. Using model designs guided by inductive biases is a promising approach for robust and interpretable NLP systems.
1lama-65b
0.71+0.16
0.75=0.19 0.43+0.22
Table 1: ParaRel results of Atlas, LLaMA and base- lines averaged over all 30 N-1 relations. "=closed-bool
and has mainly been trained on Wikipedia, which could give it an advantage (Elazar et al., 2021).
3.3 Consistency Performance
Table shows the ParaRel# zero-shot results for all evaluated models for all 30 relations. Following the work by Elazar et al. (2021), all tables report the macro-average and standard deviation over the different relations. Our findings based on these results are as follows:
Effect of Evaluation Task Format
Apart from retrieval- aaggmettttugtttniiononnn and scaling, we also expect form and evaluation task format to have had an impact on the consistency results in Table 1. ParaRel#, like many other fact-critical evaluation sets (Petroni et al., 2019; Norlund et al., 2021; Sci- avolino et al., 2021), relies on automatic prompt generation by substituting different subjects into pre-defined templates. Additionally, both templates and testing data are extracted and constructed semi- automatically, Using synthetic data and automated steps in benchmarks generation is common NLP pipelines (He et al., 2022; Meng et al., 2022b). However, the convenience of this approach comes with the risk of producing disfluent queries. Pre- vious work has found that language models learn form and syntax faster than semantics and gen- eral natural language understanding (NLU) (Zhang et al., 2021). We hypothesize that this can lead a prioritization of text fluency and syntax over facts, influencing model consistency. Therefore, we also evaluate the sensitivity of our models to issues that may arise from the evaluation task format.
Retrieval augmentation has a sizeable effect on consistency and accuracy We find that the Atlas models are more consistent compared to all other evaluated models. Since Atlas-base outperforms BERT-large in spite of the disadvantages outlined in Section 3.2, we have good reason believe that retrieval augmentation leads to improved consis- tency and the main contributor to the superior performance of Atlas. Atsas-base and large are also more accurate on ParaRel*. Accuracy is not the main interest of this work, however we expect these effects to be interconnected.
While Atlas has superior consistency ParaRel* compared to the other models investi- gated, it does not achieve perfect consistency. De- spite being developed for fact-critical tasks and allowed to retrieve information from Wikipedia, the model can generate different answers to seman- tically equivalent factual questions. We investigate potential reasons related to the evaluation task for- mat and reader-retriever interaction in Section and Section 5.
4.1 Metrics to Estimate Inconsistency Sources from the Evaluation Task Format
Issues from the evaluation task format manifest four query-related effects, falling into three groups: semantic overlap in answers, unidiomatic language (on both template and object level), and subject- object similarity. While semantic overlap is caused
Model scaling has sizeable effect on consis- tency and accuracy We observe improved con- sistency and accuracy performance with size for

---

by the evaluation setup, the remaining issues relate larger issue of the effect of language form on model behaviour.
objects and templates respectively.
Subject and object similarity Several relations contain an overlap between the subject and object - for example- Nokia N9 produced-by by Nokia. We hypothesise that in these cases the model can be guided by simple heuristic rather than access- ing factual information, which may lead to an in- creased performance ParaRel#. We identify 9 relations that contain more than 20% of tuples with an overlap between subject and object stems (see Appendix H) and report pairwise consistency for queries with and without subject-object similarity.
Semantic overlap The exact string matching used in ParaRel to determine model success, taken together with the constrained decoding may lead to biased estimations of model performance. Some re- lations include more ambiguity than others, where the model is allowed to choose between seman- tically close answer alternatives, but only one of these is accepted as a correct answer. Relation P30 for example, only includes Africa, Americas, Antarctica, Asia, and Europe, while relation P101 field-ofwoork contains both bi- ology and science, and relation P19 born-in con- tains both Glasgow and Scotland. We hypothesise that this added ambiguity may affect consistency as measured by ParaRel#; and therefore one of the comparisons we perform is on the consistency of the 12 relations that have semantic overlap in the answer options (for a full list see Appendix H) and the remaining 18 relations that do not have lap.
4.2 Evaluation task format has an effeet on consistency
Salias-bbssulllammb5 - bert-large
0.8
Consistency
0.6
Unidiomatic language Certain template-answer combinations in ParaRel* result in disfluencies. This is caused by either an ill-fitting template or object, that, when combined with template, results in unidiomatic language.
0.4
Subj-Ohj
Unidiomatic Object
Unidionatic
Semantic Overlap
Similarity
Template
For example "Anne Redpath died Edinburgh" is much more natural than "Anne Redpath died at Edinburgh"" even though to human the meaning is clear in both. This affects templates in 6 rela- tions (for full list of unidiomatic templates see Appendix H).
Figure 2: Performance for query-related sources of in- consistency. Filled-in marks indicate performance on affected data, non-filled-in on data not affected.
Figure 2 indicates the consistency performance and analysis of model sensitivity to evaluation task format issues, including form. Detailed results are available in Appendix G. All evaluation task for- mat related phenomena discussed have effect consistency across all model settings. We see that unidiomatic objects and templates affect all models on similar scale, whereas some models are less susceptible than others to subject-object similarity and semantic overlap in the answer alternatives.
Furthermore, "Solar Mass named after the Sun" sounds more natural than the P entry "Solar Mass is named after Sun". This mainly affects ob- jects in three relations: P101 specializes-in, P138 named-after, and P361 part-of. A manual anno- tation of the answer alternatives can be found in Appendix H, where we identified objects that were expressed unnaturally within the template, typi- cally because they lacked a determiner or plural form. more systematic classification to refine these labels through larger-scale human annotation processing with a grammar checker is left for future work.
Our results show that all evaluated LMs are frag- ile to language usage, such that cannot expect them be robust in the face of less fluent queries. Judging whether this acceptable LM behaviour could be based on the severity of the disfluency. We could leverage our results to create better eval- uation tasks by filtering out samples that prevent
To evaluate the effect of these issues, we com- pare the pairwise consistency of affected relations between the cases with and without unidiomatic

---

syntax
sion (1) summarizes the expected behavior of an ideal retriever.
T
O₂
Y
query
R(F,T)3) LIF
(1)
F
R2
Second, the generated output of ideal retrieval- augmented model for fact-related query should be independent of the syntax, given the semantics of the query and corresponding perfect retrieved information. Expression (2) formalizes this idea. In the ideal case, the retrieved information adds an inductive bias to the generation process that conditions the prediction to be independent of the syntax of the query.
semanties
Figure 3: Causal effects on model consistency for a retrieval-augmee cerrevellaugmeeteed model. O; denotes the prediction made by the model given retrieved information R, and query, based on template 7i and invariant fact F. For each fact we have ny templates. Y is the consistency, i.e. fraction of equal O;, Og pairs. Dashed lines indicate weak effects.
O F. F,
(2)
us from measuring our quantity of interest, such as factual knowledge under correct language usage. However, we argue that in the face of the less se- vere linguistic variations measured in this work, it is desirable to have a LM that is robust and stays faithful to its factual predictions.
In practice, we do not expect Atlas to correspond perfectly the ideal case. Our experiments inves- tigate these dependencies and how they interact with the consistency. We measure retriever con- sistency (Section 5.1), the effect of the retrieved information on consistency (Section 5.2) and the dependence of the reader on the retrieved informa- tion (Section 5.3).
5 Effect of Retrieval Components
We found Atlas to be more consistent compared to LMs without retrieval augmentation. In this section we perform deeper analysis on the effects of letting a model retrieve over some corpus with focus on the retriever and reader components.
5.1 Retriever consistency
We hypothesize that the retriever is less dependent on syntax compared to a standard LM and thereby more consistent. To estimate the dependence on syntax of the Atlas retriever, we measure retriever consistency by estimating the pairwise retriever agreement for the prompt pairs for each tuple in ParaRel As no automatic method exists for esti- mating retriever consistency, we propose and eval- uate three novel metrics. For Atlas, the retrieved in- formation is in the format of 20 Wikipedia passages, so our metrics for pairwise retriever agreement rely on 1) id overlap between retrieved passage sets, 2) title overlap between retrieved passage sets and 3) retriever query embedding similarity.
We use the DAG depicted in Figure as start- ing point for our reasoning. We define a query to be the combination of fact tuple, F and tem- F
plate T, depicted in Figure 1. We can and T' as the semanties and syntax of a query re- spectively. A model prediction for certain query and retrieved information can thus be described as O R(F, O)), where and B are model parameters of the reader and retriever respectively.
The motivation for why reurieeal-agmmentedd LMs should be more consistent than standard LMs builds on two parts related the retrieval and infer- ence steps of the strrival-auggentee model. First, a perfect retrieval result should ideally be indepen- dent of the syntax (template) of a query conditioned on the semantics (facts) of the query. The retriever is trained on the more narrow task of identifying rel- evant factual aspects of query and matching those the information in the retrieval corpus, thereby becoming less dependent on syntactic variations. Furthermore, the retriever has more restricted output space and can only choose between already existing facts rather than create new ones. Expres-
For the two first metries, we calculate the and title overlap of the 20 retrieved documents (normalized to 1), estimating exact passage match and whether passages were retrieved from the same Wikipedia page respectively. These metrics do not account for the fact that different passages, coming from different Wikipedia pages, could contain the same information, especially in the current setting that queries for simple tuple-based facts. Thus, the metrics are strict and should give a lower bound on retriever consistency.

---

The embedding based retriever consistency met- rie is more soft. For this metric, we consider the em- beddings produced by the retriever for each query paraphrase and estimate their similarity. Since the Atlas retrieval is based on a search with respect to the similarity between embeddings of queries and passages, the query embedding similarities for paraphrases can be expected to reflect retriever con- sistency. We use the cosine similarity between retriever embeddings for paraphrase pairs as an estimate for retriever consistency.
not sharing the same id title. An average over- lap of 0.5 could thus be seen as quite consistent performance, especially if we compare them to the low values for the baselines. There are multiple factors that can account for the incomplete overlap e.g. the number of paragraphs that are related to a particular fact can vary.
For the metric based on embedding similarities, we observe significant increase in embedding sim- ilarity for queries with the same semantic meaning compared to for random pairs of queries. The triever embedding similarities also capture some relation information, as the embeddings for random queries sampled over subjects have an increased similarity compared to the random baseline.
To get a baseline performance of the retriever consistency metrics we also evaluate them on pairs of randomly sampled passages previously re- trieved by Atlas-base. We use two methods for random sampling, 1) sample passage/query pairs completely random across relations and subjects, denoted r-all, and 2) sample passage/query pairs from retrieval results for the same relations but different subjects, denoted as r-subject.
We conclude that the Atlas retriever quite con- sistent according to our metrics, especially if we compare against the randomized baselines. How- ever, its consistency is not perfect across para- phrases, indicating weak dependence on syntax. We note that there many nuances automatic metrics for retriever consistency that remain be explored. Future work could for example in- clude measuring the consistency among passages retrieved for the same query.
Model
Metric Similarity μ Similarity
r-all id 0.0010.00
0.0110001
title 0.00+0.00
0.02+0.01
emb 0.54+0.04
0.06+00001
r-subject id 0.01+0.01
0.02+0.02
title 0.0111001
0.0210.02
emb 0.64
0.0610.01
atlas-base id 0.51
0.1910.03
Reader and retriever consistencies are corre- lated To estimate the effect of the retriever con- sistency on the overall model consistency, we mea- sure the Pearson correlations between reader con- sistency and our retriever consistency metrics. We expect consistent predictions correlate with con- sistent retrieval results and vice versa, building on arguments related Expressions (1)-(2).
title 0.5810.10
0.20110.4
emb 0.88+0.04
0.05++000
atlas-large
id 0.52+0.10
0.19+0.03
title 0.59+0.09
0.20+0.04
emb 0.8810.04
0.05
Table 2: Similarity metrics used to estimate retriever consistency. r-all denotes random sampling over sub- jects and relations, and r-subject sampling over subjects with the relation fixed. refers to the distribution of the mean per relation and a refers the distribution of the standard deviation of the metrics per relation.
We observe weak correlation between all re- triever consistency metrics and the Atlas consis- tency. For both id and title the Pearson correlation is 0.16 + 0.08, while for query embedding simi- larity it's 0.14 + 0.13. As previously discussed, we expect retrieval augmentation to condition the prediction to be less syntax dependent, why we expected some form of correlation to exist, but we cannot fully explain why it not higher. One expla- nation imperfect metries for retriever consistency. Most likely, there are additional factors play that determine whether Atlas consistent. After all, the Atlas reader is only conditioned on and not controlled by the retrieved information.
We apply retriever consistency metrics the Atlas retriever and report the results in Table 2. Our findings are as follows:
The Atlas retriever exhibits weak dependency syntax For the id and title overlap metrics, we first set threshold in order label a retrieval re- sult as consistent. From inspection by example we found the retriever able to find many passages on the same topic, some with semantic overlap despite

---

Intervention
Cons
Acc
C& A
clude that relevant and consistent retrieved infor- mation leads to more consistent Atlas model, while not perfectly consistent. To some extent, this confirms our hypothesis related to Expression (2), while further investigations are needed to explain the absence of perfect consistency. As we saw in our previous results, the effect of form could play role.
none
0.74
0.8010.16
0.42-0.27
relevant 0.79=0.15
0.8010.16
0.48=0.28
irr cohesive
0.61+0.22
0.42+0.24
0.23=0.23
irr incohesive
0.58+0.224
0.42+0.24
0.22-0.22
Table 3: Atlas-base results for different interventions on the retrieval augmentation. relevant denotes a consistent and relevant retrieved result. irr cohesive consistent and unified result, but irrelevant. irr incohesive a con- sistent but completely random result for each of the 20 passages.
Another potential explanation for the lack of per- feet consistency is given by Longpre et al. (2021). They found that a context-augmmntted LM may ignore the provided context if it contradicts the knowledge stored in the model parameters, hypoth- esizing that fact popularity plays an important role. Future work could examine whether this phenom- ena also extends retrieval-augmented models.
5.2 Interventions to further investigate the effect of retrieval augmentation
We carry out intervention experiments on the re- trieved information to investigate its effect on Atlas. To test whether consistent retrieved information across paraphrases contributes to more consistent predictions, we set the retrieved information R& to a fixed value across paraphrases indexed by i in ParaRel meaning that Vi R2 = R. We then mea- sure the consistency of the reader conditioned on this consistent retrieved information.
Consistent while irrelevant retrieval augmen- tation does not result in more consistent pre- dictions We can also observe from Table how consistent but irrelevant retrieved information not related to the query at hand leads to significant decrease in both consistency and accuracy. We ob- serve similar behaviour regardless of whether the irrelevant retrieved information was cohesive or not (i.e. related to the same fact). Seemingly, the reader prediction is not only dependent consistent trieved information, and the reader may be able discriminate what retrieved information is relevant to the query. This corroborates our theory that ad- ditional inconsistency sources for Atlas cxist apart from retriever consistency. More assumptions are needed for the variables in Expression (2) before we can accurately describe the workings of Atlas.
Taking the effect of retrieved information quality into account, we test three different approaches for setting the fixed retrieved information for given fact = r.) First, given the corresponding query paraphrases for F indexed by i we take the retrieved passages for one of the paraphrases i' and use that as the retrieved information for the remain- ing paraphrases r). R( r,, making each r.), both relevant and consistent. Second, take the retrieved passages for a query paraphrase i' for another subject s' from the same relation r and set this as the fixed retrieved information R(xrr).: = B( r), making the retrieved informa- tion consistent and cohesive across the retrieved passages but irrelevant. Third, we use set of completely random passages Rrand as the retrieved information R(,,)), Rrand: making it consistent but irrelevant and incohesive. See Appendix for a sketched example. The two latter approaches let us disentangle the effect of relevance from the effect consistency for the retrieved information.
5.3 Reader dependence retrieval result
To further reason about the effect of retrieval aug- mentation on consistency, we would like to know to what extent the Atlas reader is dependent the retrieval result. We start by investigating sible heuristic namely, that the reader relies on frequencies in the retrieved passages.
pos-
We estimate the dependence of the Atlas reader on the retrieval result using the rank of the reader prediction according to term frequencies in the re- trieved passages. This is further described in Ap- pendix C. Since the Atlas reader may be more less dependent on the retrieved passages depend- ing on how well it reflects a correct answer the query we also measure the rank of the gold label following the same approach. We expect a useful result to rank the correct answer higher compared
We report the results from these interventions in Table 3. Our findings based on these results are as follows:
Consistent and relevant retrieval augmentation causes more consistent predictions We con-

---

Model Type
Rank
Match
No match
Related Work
base
pred
0.0910.06
0.06=0.04
0.1910.09
While there is a large body work on the fac- tual accuracy of LMs, the question of consistency has received less attention. This issue was investi- gated by Elazar et al. (2021), which introduced the ParaRel benchmark which forms the basis of our work here. On the accuracy side, there have been investigations that relate effects of the training data (primarily how frequently facts occur) to the cor- rectness of factual predictions (Mallen et al., 2023; Kandpal et al., 2023; Elazar et al., 2022).
gold
0.07+0.05
0.05=0.04
0.1320.07
large
pred
0.08++006
0.05+0.04
0.19+0.08
gold 0.07+00.5
0.05+0000 0.14+0.08
Table 4: "requency-basedd averaged rankings in the re- trieved result of the reader's predictions and the gold labels. The Pearson p between consistency, predic- tion and respectively gold rank were - -0.34 + 0.12 and -0.18 0.10.
The approach of viewing a complex LM as a causal system and carrying out interventions to in- vestigate the effect of components on the model's behavior was pioneered by Vig et al. (2020). Fur- ther work by Meng et al. (2022a) applies this methodology to examining fact associations and proposes hypothesis of how models recall factual information stored in their parameters.
to less useful result.
We expect consistent predictions be more re- liant on the retrieval result compared to inconsis- tent predictions and can measure this using our defined metrics. Moreover, we expect the reader to be more reliant on useful retrieval results, and thus more consistent for these, compared to for less useful results that are less successful in surfacing the correct answer.
Conclusions and Future Work
We estimate the dependence of the Atlas reader on the retrieval result using our ranking and report the results in Table 4. An average rank of O cor- responds the reader always predicting the top answer candidate according to the retriever. Our findings based on these results are as follows:
In this work, we investigate methods to improve factual consistency of LMs and further reason about potential sources of inconsistency. We find that up-scaling, represented by LLaMA, generally improves consistency, while it is inefficient and surpassed by retrieval augmentation, repre- sented by Atlas. Retrieval augmentation produce the largest consistency improvement. However. Atlas does not achieve perfect consistency even for perfectly consistent retrieved passages, this in- dicates that additional inconsistency sources are involved and persistent in spite of consistent con- ditioning. These are potentially related to form anomalies and changeable reader dependency on the retrieved information. In conclusion, we have identified and measured several potential sources of (in) connissssyy that should aid future work on improving the robustness of LMs, while much re- mains to be investigated.
Reader dependence on retrieval result correlates with consistency The average rank of a predic- tion is higher for consistent predictions, compared to inconsistent. It could be that the reader for some instances is less faithful to the frequency based sig- nal of the retriever and simply picks an answer less supported by the retrieval result. Drawing from results in Figure 2, we hypothesize that this could happen if the answer supported by the retrieved in- formation would produce an unidiomatic sentence. Alternatively, some samples may be more difficult for both reader and retriever to process, resulting in lower rankings of the predictions made.
Retriever frequency of the correct answer cor- relates with consistency The retriever generally promotes the correct answer frequency-wise Addi- tionally, similarly to our previous observation, the retriever is largely better at promoting the correct answer for samples for which the reader consis- tent. This indicates that there are samples for which the retriever struggles to find suitable passages and that the reader is less consistent for these.
Future work includes evaluating consistency of additional model designs, such as context- augmented L.MMs and models that build on knowl- edge graphs (Shu et al., 2022). We could also make use of methods similar to the ones proposed by Meng al. (2022a) to further investigate paramet- ric versus non-pan -arrmaraetrric memorization in the con- text of retrieval augmentation and its effects on consistency.

---

Limitations
but that may not serve as binary distinction be- tween a fact being represented or not in the model. The different level of difficulty of recalling a fact is a further possible confounder left for future work.
We reason about benefits related to up-scaling and retrieval augmentation based on our results from Atlas and LLaMA, but this may not give the com- plete picture of the performance of these model types in general.
Ethics Statement
Based on the performance of Atlas, we reason about the effect of retrieval augmentation on incon- sistency, while we cannol perfectly control for the confounding effect of different training data. Gen- erally, training data has been found to have big impact on model performance and especially when relating to accessing factual information. The best way to estimate the exact effect of retrieval aug- mentation would be to use a proper closed-book version of Atlas, trained without augmentation on the same data. However, since the weights for this baseline have not been released we resort to ad- dressing this by marginalization over several other relevant baselines.
Our work has the overall goal of improving consis- tency of LMs - goal that, if accomplished, could make AI generated text harder to identify with po- tential negative consequences. On the positive side, improved consistency of LMs should make them easier to explain and safer to use in fact critical situations. The ParaRel* data used in this work is not associated with any ethical complications.
Acknowledgements
We are grateful to Marco Kuhlmann, whose sup- port and insightful feedback was essential for the completion of this work.
This work was partially supported by the Wal- lenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation and by the Wallenberg AI, Autonomous Systems and Software Program Hu- manities and Society (WASP-HS) funded by the Marianne and Marcus Wallenberg Foundation and the Marcus and Amalia Wallenberg Foundation. The computations were enabled by resources pro- vided by the National Academic Infrastructure for Supercomputing in Sweden (NAISS) at Alvis par- tially funded by the Swedish Research Council through grant agreement no. 2022-06725, and by the Berzelius resources provided by the Knut and Alice Wallenberg Foundation at the National Su- percomputer Centre.
Similarly, we compare LLaMA to Atlas to reason about the benefits of LLMs versus re- trieval augmentation, while these two models have been trained on different datasets and we cannot marginalize over training data effects. Elazar et al. (2022) found that models trained larger extent on Wikipedia performed better on ParaRel, poten- tially because Wikipedia a more unified source of factual knowledge. In this sense, LLaMA might be at a disadvantage, due to requiring large amounts of training data that are not guaranteed to be unified or factual.
Furthermore, we base our analysis of consistency on a single benchmark dataset, which the best of out knowledge is the largest and most relevant dataset for these types of evaluations. However, fur- ther investigations into how these results generalize beyond ParaRel* and its specific design choices re- mains to be investigated.
References
Boxi Cao, Hongyu Lin, Xianpei Han, Le Sun, Lingy- ong Yan, Meng Liao, Tong Xue, and Jin Xu. 2021. Knowledgeable or educated guess? revisiting lan- guage models as knowledge bases. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1860-1874, Online. Association for Computational Linguistics.
We focus our study on cloze-style queries, which limits how well the test set can be handled by dif- ferent models. These prompts are more suited for masked language modelling than for auto- regressive models, so further investigations of the effects of the chosen task formulation is needed. Furthermore, models cannot express level of cer- tainty, so it is unclear to what extent the perfor- mance is tied to the (un)availability of facts in the model. Together with the constrained decoding method applied in this work, this may lead to mod- els being able to surface a fact in certain settings,
Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. 2019. Wizard conversational
of Wikipedia: Knowledge-powers agents. In Proceedings of the International Confer- ence on Learning Representations (ICLR).

---

Yanai Elazar, Nora Kassner, Shauli Ravfogel, Amir Feder, Abhilasha Ravichander, Marius Mosbach, Yonatan Belinkov, Hinrich Schütze, and Yoav Gold- berg. 2022. Measuring causal effects of data statis- tics on language model's 'factual' predictions. arXiv preprint arXiv:2207. (tto:2227.1425.11. 14251.
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red- field, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Ken- ton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Was Wei Chang. Andrew M. Dai, Jakob Uszkoreit, Quoe Le, and Slav Petrov. 2019. Natu- ral questions: A benchmark for question answering research. Transactions of the Association for Compu- tational Linguistics, 7:452 466.
Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhi- lasha Ravichander. Eduard Hovy, Hinrich Schütze, and Yoav Goldberg. 2021. Measuring and improving consistency in pretrained language models. Transac- tions of the Association for Computational Linguis- tics, 9:1012-1031.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Hein- rich Küttler, Mike Lewis, Wen-tau Yih, Tim Rock- täschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval- -augeettd generation for knowledge- intensive nip tasks. In Advances in Neural Infor- mation Processing Systems, volume 33, pages 9459- 9474. Curran Associates, Inc.
Amir Feder, Katherine A. Keith, Emaad Manzoor, Reid Pryzant, Dhanya Sridhar, Zach Wood- Wood-Doughty, Ja- cob Eisenstein, Justin Grimmer. Roi Reichart, Mar- garet E. Roberts, Brandon M. Stewart, Victor Veitch, and Diyi Yang. 2022. Causal inference in natural lan- guage processing: Estimation, prediction, interpreta- tion and beyond. Transactions of the Association for Computational Linguistics, 10:1 138-1158 158.
Shayne Longpre. Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh. 2021. Entity-based knowledge conflicts in question answering. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Process- ing, pages 7052-7063, Online and Punta Cana, Do- minican Republic. Association for Computational Linguistics.
Xuanli He, Islam Nassar, Jamie Kiros, Gholamreza Haf- fari, and Mohammad Norouzi. 2022. Generate, An- notate, and Learn: NLP with Synthetic Text. Trans- actions of the Association for Computational Linguis- tics, 10:826 842.
Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das. Daniel Khashabi, and Hannanch Hajishirzi. 2023. When not trust language models: Investigating effectiveness of parametric and non-parametric mem- ories. In Proceedings of the 61 Annual Meeting of the Association for Computational Linguistics ( (Vol- ume 1: Long Papers), pages 9802 9922,982,, Toronto, Canada. Association for Computational Linguistics.
Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebas- tian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave, 2022. Unsupervised dense informa- tion retrieval with contrastive learning. Transactions on Machine Learning Research.
Gautier Izacard and Edouard Grave. 2021. Leveraging passage retrieval with generative models for open do- main question answering. In Proceedings of the 16th Conference of the European Chapter of the Associ- ation for Computational Linguistics: Main Volume, pages 874-880, Online. Association for Computa- tional Linguistics.
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022a. Locating and editing factual asso- ciations in gpt. In Advances in Neural Information Processing Systems, volume 35, pages 17359-17372. Curran Associates, Inc.
Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi- Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2023. Atlas: Few-shot learning with retrieval augmented language models. Journal of Machine Learning Research, 24(251):1-43.
Yu Meng. Jiaxin Huang, Yu Zhang, and Jiawei Han. 2022b. Generating training data with language mod- els: Towards zero-shot language understanding. In Advances in Neural Information Processing Systems.
Tobias Norlund, Lovisa Hagström, and Richard Johans- son. 2021. Transferring knowledge from vision to language: How to achieve it and how to measure it? In Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP,, pages 149-162, Punta Cana, Dominican Repub- lic. Association for Computational Linguistics.
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of halluci- nation in natural language generation. ACM Comput- ing Surveys, 55(12):
Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. 2023. Large language models struggle to learn long-tail knowledge. In Proceedings of the 40th International Conference on Machine Learning. volume 202 of Proceedings of Machine Learning Research, pages 15796-15707 PMLR.
Maria Leonor Pacheco and Dan Goldwasser. 2021. Modeling content and context with deep relational learning. Transactions of the Association for Compu- tational Linguistics, 9:100-119.
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and

---

Alexander Miller. 2019. Language models as knowl- edge bases? In Proceedings of the 2019 Confer- ence on Empirical Methods in Natural Language Pro- cessing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2463-2473, Hong Kong, China, Association for Computational Linguistics.
Lamm, Viktoriya Kuzmina, Joe Fenton, Aaron Co- hen, Rachel Bernstein, Ray Kurzweil, Blaise Aguera- Arcas, Claire Cui, Marian Croak, Ed H. Chi, and Quoe Le. 2022. Lamda: Language models for dialog applications. CoRR, hass/2011.02399
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro. Faisal Azhar, et al. 2023. Llama: Open and effi- cient foundation language models. arXiv preprint a7Xi
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text trans- former. The Journal of Machine Learning Research, 21(1):5485-5551
Pat Verga, Haitian Sun, Livio Baldini Soares, and William Cohen. 2021. Adaptable and interpretable neural MemoryOver symbolic knowledge. In Pro- ceedings of the 2021 Conference of the North Amer- ican Chapter of the Association for Computational Linguisties: Human Language Technologies. pages 3678 1778-3691, Online. Association for Computational Linguistics.
Marco Tulio Ribeiro, Carlos Guestrin, and Sameer Singh. 2019. Are red roses red? evaluating consis- tency of question-answering answering models. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6174-6184, Flo- rence, Italy. Association for Computational Linguis- tics.
Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stu- art Shieber. 2020. Investigating gender bias lan- guage models using causal mediation analysis. In Advances in Neural Information Processing Systems, volume 33, pages 12388 12401. Curran Associates. Inc.
Christopher Sciavolino, Zexuan Zhong, Jinhyuk Lee, and Danqi Chen. 2021. Simple entity-centric ques- tions challenge dense retrievers. In Proceedings of the 2021 Conference on Empirical Methods in Natu- ral Language Processing, pages 1338-148,, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Xuezhi Wang, Haohan Wang, and Diyi Yang, 2022. Measure and improve robustness in NLP models: A survey. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, pages 4569-45866 Seattle, United States. Association for Computational Linguistics.
Yiheng Shu, Zhiwei Yu, Yuhan Li, Börje Karlsson, Tingting Ma. Yuzhong Qu, and Chin- Yew Lin. 2022. retrieval for robust question answering over large knowledge base. Proceed- ings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8108-8121, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
TIARA: Multi-grained
Yuxiang Wu, Yu Zhao, Baotian Hu, Pasquale Min- ervini, Pontus Stenetorp. and Sebastian Riedel. 2022. An efficient memory-augmented transformer for knowledge-intensive NLP tasks. In Proceedings of the 2022 Conference on Empirical Methods in Nat- ural Language Processing, pages 84-55196 Abu Dhabi, United Arab Emirates. Association for Com- putational Linguistics.
Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. 2021, Retrieval augmentation reduces hallucination in conversation. In Findings of the Association for Computational Linguistics: EMNLP 2021. pages 3784-3803, Punta Cana, Do- minican Republic. Association for Computational Linguistics.
Yian Zhang, Alex Warstadt, Xiaocheng Li, and Samuel R. Bowman, 2021. When do you need bil- lions of words of pretraining data? In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume I: Long Papers), pages 1112-1125, Online. Association for Computational Linguisties.
Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng- Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, YaGuang Li. Hongrae Lee. Huaixiu Steven Zheng, Amin Ghafouri, Marcelo Menegali, Yanping Huang, Maxim Krikun, Dmitry Lepikhin, James Qin, Dehao Chen, Yuanzhong Xu, Zhifeng Chen, Adam Roberts, Maarten Bosma, Yanqi Zhou, Chung-Ching Chang. Igor Krivokon, Will Rusch, Mare Pickett, Kathleen S. Meredith Ringel Morris, Tulsee Doshi, Renelito Delos Santos, Toju Duke, Johnny So- raker, Ben Zevenbergen, Vinodkumar Prabhakaran, Mark Diaz, Ben Hutchinson, Kristen Olson, Ale- jandra Molina, Erin Hoffman-John, Josh Lee, Lora Aroyo, Ravi Rajakumar, Alena Butryna, Matthew

---

A Generating predictions for ParaRel* with LLaMA and Atlas
To give an example how the metric works, con- sider the input query "Y located in [X]." and as- sume that we have the answer candidates Canada, Norway and Singapore. We generate the corre- sponding Atlas retrieval result with term frequen- cies as indicated in Table 6 and pass the passages to the reader to get the prediction Y-Singapore. According to the term frequencies, this prediction has rank 2, while Canada has rank 3 and Norway 1. After normalisation to values between 0 and 1, the rank of Y is 0.5 and this what we report.
To evaluate LLaMA on ParaRel we compute the log likelihoods of all answer options given a rela- tion, template and subject, and select the answer with the highest likelihood score.
Since Atlas is seq2seq model, we cannot use the same technique to restrict the prediction can- didates as for the BERT-like models. Instead, make use of an approach similar to the one used for LLaMA. For a given query, we get the latent rep- resentation from the Atlas reader encoder and give this as input to the Atlas reader decoder together with each answer option. The answer options are formatted as e.g. "<extra_id O>FFrance" to suit the T5 format. We then select the answer option with the highest likelihood score according to the reader decoder. To ascertain that the constrained decoding approach working, we compare the freely generated model answers to the constrained model answers and ensure that they match when the model happens to freely generate an accepted answer option.
D Knowledgeable Consistency
ParaRel also proposes two metrics for deeper anal- ysis of cases when the model is expected to know the fact and when the fact may not be in the model. They define knowledgeable consistency the pair- wise consistency for any fact, for which any of the prompts was able to retrieve the correct answer, and unknowledgeable consistency pairwise con- sistency for fact that no prompt is successful. We propose one more metric, which expands on the idea of knowledgeable consistency. Since the origi- nal metric in ParaRel counts any pair matches be- tween prompts, as long as one of them was correct, this can be misleading in estimating how consis- tent to the true fact the model is. We define the new metric as the pairwise consistency of prompts, where both prompts agree and are both correct.
B Interventions on Retrieval Augmentation
To analyse the effects of retrieval consistency, develop several strategies for setting consistent re- trieved passages: relevant, irrelevant but cohesive, irrelevant and incohesive. These refer to whether the we expect the information in the passages to be related to the questioned asked (relevant) or to the type of relation in the question (cohesive), Ta- ble 5 shows sketched example of how these are
E Additional results
We calculate the extended metrics described in Appendix D for the models we evaluate (See Ta- ble 7). We observe similar trends in performance between the models. One trend notice is that the unknowledgeable consistency plateaus for larger model sizes.
assigned.
C Measuring reader dependence on retrieval result
We measure reader dependence on the retrieval result by taking the frequency-based rank of the reader prediction compared to all other possible answer alternatives. This is done in two steps: I) we order the ParaRel* answer alternatives by fre- quency for each retrieved result and 2) estimate the ranking (normalised between 0 and 1) of the prediction within this order. To relate this metric to consistency, make the estimates prompt-pair level by taking the average of the two correspond- ing prediction rankings. This allows us to estimate both mean rank per relation, but also mean rank for consistent and inconsistent predictions separately.
Furthermore, we perform stratified measurement of consistency of retrieval passages depending on wheter the overall model prediction matched (was consistent) or not (See Table 8).
Finally, we perform analysis of our proposed metrics for measuring retrieval consistency and cal- culate how well they correlate with each other (See Table 9). Unsurprisingly, we find high correlation between id and title match (since id sub-level of title). We also find embedding distance correlates with and title.

---

Set of retrieved passages
query
intervention
relevant
irr cohesive
irr incohesive
Eibenstock is located in [Y].
(Pi, P2,
(Pi, P2, Pa)
(Po, Pio, Pu)
(Ps, Pio, Pua)
Eibenstock, which is located in [Y].
(M, Ps, Ps)
P2, Ps]
[Pa, Pio Pu]
{Ps, Pio Plo, Pia]
Eibenstock, located in [Y].
{12 Pa
(M1, Ps)
{Ps Pio Pu]
{Ps, Pi, Pud
Gleneree is located in [Y].
(Pa, Pio.P1)
Robert Bunsen specializes [Y]
(P12, P1s, Pua)
Table 5:
An example of intervention the retrieved passages when querying for the location of Fibenstock, We refer to the passages as Pt. Pz, Ps... and assume that 3 passages are retrieved for each query.
Candidate Frequency
Canada
2
Norway 7
Singapore 5
Table 6: Candidate frequencies for retrieval result,
as an example.
F Duplicated entries in ParaRel
See Table 10 for statistics for the duplicated data entries in ParaRel.
G Detailed results for evaluation task format effects consistency
Detailed results for evaluation task format related effects can be found in Tables to 14, where we report consistency per model and per group.
H ParaRel* flags indicating potential data issues
Some queries are identified to have semantically similar answer options, which may cause the model to be unfairly punished for predicting, for example, science instead of biology. For summary see Table 15.
Some queries are observed to result in unid- iomatic language due to the template structure. For details see Table 16.
Furthermore, resulting texts may unidiomatic due to the gold label that we want to predict. De- tailed list of those objects can be found in Table 17. Finally, Table 18 shows the full list of relations that contain some overlap between the subject in the prompt and the gold label we want to pre- dict, potentially allowing the model to use shallow heuristics for its prediction.

---

Model
Know Cons
K-know Cons
Unk Cons
atlas-bass
0.75 + 0.15
0.71 + 0.17
0.59 + 0.19
atlas-large
0.78
0.74 + 0.15
0.61 + 0.20
atlas-base*
0.65
0.57 + 0.28
0.51 0.20
15-base
0.23
0.56 + 0.26
0.52 0.23
bert-base
0.63 0.25
0.56 + 0.29
0.45 0.21
bert-large
0.64 + 0.24
0.57 0.28
0.47 0.20
Ilama-7b
0.69 + 0.18
0.63 0.22
0.48 0.16
1lama-13b
0.71 + 0.17
0.65 + 0.20
0.50 + 0.17
llama-30b
0.72 0.17
0.67 + 0.19
0.50 + 0.17
llama-65b
0.73 0.16
0.69 + 0.18
0.50 + 0.17
Table 7: ParaRel results of Atlas, LLaMA and some baselines averaged over all 30 N-1 relations with extra metries. #=closed-book.
Model
Metric
Match sim.
No match sim.
atlas-base
0.53 0.10
0.45 + 0.11
title
0.60 = 0.09
0.52 0.11
embedding
0.89 + 0.04
0.87 0.04
atlas-large
id
0.54 = 0.09
0.46
title
0.60 + 0.09
0.52
embedding
0.89 0.04
0.87 + 0.04
Table 8: The pairwise retriever similarity results stratified over whether the corresponding prediction made by the retriever was consistent. Complement to Table Match consistency refers to the retriever consistency for cases for which the final model prediction is consistent and match consistency refers cases for which the final model prediction is inconsistent.
id
title
embedding
id
1.00 + 0.00
0.82 + 0.20
0.20 + 0.12
title
0.82 + 0.20
1.00 + 0.00
0.21 + 0.11
embedding
0.20 + 0.12
0.21 + 0.11
1.00 + 0.00
Table 9: The correlations between the retriever consistency metrics for random sampling across relations and subjects ('all') averaged over ParaRel* relations. 1000 pairs were sampled per sampling method.

---

Relation
Name
#entries
#duplicates
#exact
P17
located-in
912
2
0
P19
born-in
779
0
0
P20
died-in
817
0
0
P27
citizen-of
958
0
0
P30
located-in-continent
959
4
0
P36
capital-of
471
P37
official-lnguuage
900
280
PIO1
specializes-in
571
52
P103
native-language
919
2
P106
-sro-yy-poffession
821
0
P127
owned-by
616
0
P131
located-in
775
0
P136
plays-music
859
2
P138
named-after
461
23
P140
dthliated-with-reliie
432
10
P159
headquarter-in
801
4
P176
produced-by
925
19
8
P178
dy
588
12
P264
represenied-by-mmuii-heeel
53
2
P276
located-in
764
74
P279
subclass-of
900
4
P361
part-of
746
64
P364
original-language
756
6
P407
written-in-language
857
31
P413
plays-in-position
952
0
P449
originally-aired-on
801
9
P495
created-in
905
2
0
P740
founded-in
843
0
0
P937
worked-in
853
21
0
P1376
capital-of
179
8
P1412
communicated-in
924
2
Total
23097
647
15
Table 10: We count the number of data tuples for each of the 31 ParaRel N-1 relations and indicate how many of these are duplicates with respect to subject and relation. We also indicate how many of these duplicates are exact duplicates, in the sense that
tuple occurs multiple times for relation.
Model
subject-object similarity
no subj-obj similarity
atlas-base
0.83 + 0.13
0.75 + 0.15
atlas large
0.88 + 0.10
0.79 + 0.13
bert-base-cased
0.79 + 0.14
0.50 + 0.16
bert-large-cased
0.81 + 0.11
0.55 + 0.14
1lama-07b
0.87 + 0.08
0.72 + 0.11
llama-65b
0.86 + 0.11
0.76 + 0.10
Table 11: Pairwise consistency for datapoints containing subject-object similarity and for those that do not. Results are based on relations that contain least 20% subject-object similarity.

---

Model
object issue
no object issue
atlas-base
0.60 = 0.22
0.74 = 0.16
atlas-large
0.55 0.23
0.71 = 0.16
bert-base-cased
0.39 + 0.26
0.58 0.40
bert- Sarttllage-cased
0.45 = 0.27
0.59 = 0.35
llama-
0.45 = 0.13
0.66 = 0.21
llama-65b 0.43 = 0.14
0.68 + 0.20
Table 12: Pairwise consistency for datapoints containing object level issues and for those that do not. Results are based on relations that have been labeled with object issues.
Model
template issue both
template issue
template issue none
atlas-base
0.43 + 0.71
0.68 + 0.13
0.83 + 0.10
atlas-large
0.49 + 0.73
0.73 = 0.13
0.84 + 0.03
bert-base
0.30 + 0.69
0.43 = 0.28
0.65 + 0.23
bert-large
0.32 + 0.70
0.48 + 0.26
0.72 + 0.21
llama-07b
0.35 + 0.68
0.56 + 0.20
0.76 + 0.18
llama-65b
0.39 + 0.69
0.62 + 0.17
0.77 +: 0.18
Table 13: Pairwise consistency for cases where one, both or none of the compared templates have a template issuc. Results based relations, where we identified template issues.
Model
semantic overlap
no semantic overlap
atlas-base
0.73 + 0.13
0.75 + 0.17
atlas-large
0.78 + 0.10
0.77 + 0.16
bert-base-cased
0.48 + 0.23
0.65 + 0.23
bert-large-cased
0.51 + 0.20
0.67 + 0.23
1lama-07b
0.63 + 0.17
0.69 + 0.19
1lama-65b
0.67 + 0.16
0.74 + 0.15
Table 14: ParaRel* results of Atlas and some baselines averaged over all 30 N-1 relations divided into relations that have semantic ovelap in the answer options (12 relations) and those that do not have an overlap (18 relations).
Relation
Name
Comment
P19
born-in
geographic (City/Country)
P20
died-in
geographic (City/Country)
P101
specializes-in
several general options (e.g. art science)
P106
is-a-by-profession
somcone could be multiple roles e.g a diplomat and politician geographic (City/Country)
P131
located-in
P140
shiliaatd-wwit-reeltt
includes (Catholicism, Christian, Christianity) and (Islam, Muslim)
P159
headquarter-in
geographic (City/Country)
P276
located-in
geographic (City/Country)
P279
subclass-of
several general options (e.g. and science)
P361
part-of
several general options (c.g. and science) geographic (City/Country)
P740
founded-in
P937
worked-in
geographic (City/Country)
Table 15: Relations where alternatives could have ambiguity due semantic overlap.

---

Relation
Template
Comment
P19 born-in
[X] is native to [Y].
egg. "Claude Arrieu is native to Paris"
X was native to [Y].
e.g.
"Claude Arrieu was native to Paris"
P20 died-in
[X died at [Y].
preposition
[X] passed away at [Y].
preposition
X] lost their life at [Y].
preposition
X] succumbed at [Y].
preposition
P27 citizen-of
X] is [Y] citizen.
[X] France citizen >>a French citizen
P106 as---by-pres --a-bb--rrffessoon
X] works as [Y].
may require afan
XJ. who works as [Y].
may require a/an
XX'ss ' occupation is [Y]
may require a/an may require a/an may require a/an
the occupation of [X] is [Y].
the profession of [X] is [Y].
P138 named-after
[X] is named in [Y]'s honor.
apostrophe
[X] was named [Y]'s honor.
apostrophe
[], named [Y]'s honor.
apostrophe
[]] which named [Y]'s honor.
apostrophe
[], which was named in [Y]'s honor.
apostrophe
P1376 capital-of
[Y]'s capital, [X].
apostrophe
[Y]'s capital city, [X].
apostrophe
[Y]'s capital is [X].
apostrophe
[Y]'s capital city [X].
apostrophe
Table 16: Relations containing unidiomatic templates.

---

P101 specialize in
P138 named after
P361 part of
Internet
Alps
Alps
car
foot
perfume
astronomer
Americas
Americas
cartridge
forest
pistol
bird
Arctic
Antarctic
castle
fruit
piston
car
Bible
BBC
cavalry
galaxy
port
cave
Moon
Bible
cell
gang radar
comedian
Netherlands
Caribbean
cemetery
gene saddle
diplomat
Sun
Caucasus
chromosome
genome
screw
economist
arrow
Internet
clergy
gospel
habitat
backpack
Nile
cloud
graph
seed
hotel
brake
Quran
cocktail
head
shield
icon
canon
airline coin
heart
skeleton
mathematician
cube
airport comer
kidney
skull
miniature
flower
ankle computer leaf
spacecraft
musical
glove
aquarium door
liver
stomach
musician
grape
army
lung sword
nightelub
horse
artillery
economist
matrix
track
novelist
hotel
atom ecosystem
molecule
trail
philosopher
liver
banana engine mosque tree
physician
mayor
battery enzyme
municipality triangle
physicist
mole
bicycle
eye
navy
turbine
priest
monastery
bird facade
neck
volcano
programmer
patent bow film
nerve
stock
patriarch
brain firearm orbit
stomach
red
breast fish
organism
virus
bridge
fleet
parish
website
candle flower
penis
Table 17: Relations containing unidiomatic objects.

---

Relation
Name
P36
capital-of
P127
owned-by
P131
located-in
P138
named-after
P176
produced-by
P178
de
P276
located-in
P279
subclass-of
P361
part-of
Table 18: Relations containing subject and object simi-
larity.


Reviews:
[{"review_id": "16XBinzT8j", "reviewer": "Reviewer_91nZ", "summary": "", "questions": "The authors conclude in section 6 that\n\n\"Retrieval augmentation is discovered as the best method for consistency improvement, while it does not achieve perfect consistency.\"\n\nGiven this statement, it would be interesting to learn what other approaches the authors could foresee which could provide perfect consistency. One area to explore would be the usage of knowledge graphs as a source of explicit factual knowledge. One could expect that a given query to the same version of the graph will allows provide the same results. ", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review. We have mainly based our rebuttal on your _Reasons To Reject_ and _Questions For The Authors_. Please let us know if there is anything we should address in addition to this.\n\n> The results that retrieval augmentation improves performance is not much of a surprise. The paper confirms this not with accuracy but with consistency as a benchmark criterion. \n\nIt is correct that it has already been proven that retrieval augmentation can improve _accuracy_ on several QA and NLU tasks. However, to the extent of our knowledge, there are no preceding published results that have found retrieval augmentation to improve on consistency or otherwise similar robustness metrics. While the reviewer may not find this surprising, such claims still ought to be empirically validated.\n\nAdditionally, we would like to argue that the main contributions of our paper are not only that we establish retrieval augmentation as a way to improve consistency. We move on to compare the strength of the effect of retrieval augmentation against other approaches that can be expected to perform equally well, such as scaling, and show that retrieval-augmentation outperforms scaling consistency-wise. We also investigate potential reasons for _why_ retrieval augmentation improves on consistency and why it does not achieve perfect consistency, considering e.g. retriever consistency and unidiomatic language usage. \n\n> In that way, the paper does not go far beyond the state-of-the-art, but mostly improves an existing benchmark about consistency (ParaRel).\n\nWe did not intend for our contributions to be to obtain state-of-the-art on general benchmarks that measure accuracy, or to improve benchmark results on ParaRel. Our intended contribution lies in diagnosing the factual consistency of modern LMs, for which we use ParaRel as the measurement tool. To our knowledge, there are no other existing datasets that can be used to measure factual consistency or similarly related robustness metrics for foundational LMs. If you know of any additional benchmarks suitable for consistency evaluations, we would be happy to hear of them.\n\n> Given this statement, it would be interesting to learn what other approaches the authors could foresee which could provide perfect consistency. One area to explore would be the usage of knowledge graphs as a source of explicit factual knowledge. One could expect that a given query to the same version of the graph will provide the same results.\n\nThis answer will have to be a pure speculation since we do not currently have empirical results to back up any claims. It is on the agenda to consider models that build on knowledge graphs, such as TIARA by Shu et al. (2022), but it remains to be seen whether they are more or less consistent (and accurate) than models based on text retrieval and we have no prior opinion on the outcome of such a comparison. Just as we have investigated in this paper for text retrieval, the accuracy and consistency of KG-based models will depend on the robustness of query representations under paraphrasing and how sensitive the retrieval component is to such variation.\n\nThe goal of the current paper was not to modify any existing system to achieve improved consistency, but rather to investigate the relative strengths and interplay of various factors that influence consistency in LMs. Focusing on the task of improving consistency, we do not expect any approach to achieve 100% consistency given the current state of the field. Apart from changing model setup, one potential avenue to investigate that would take us closer to perfect consistency is to consider model confidences, since there are probably situations for which a model does not “know” the necessary fact and therefore performs random guessing from the start. We expect that providing the models with the ability to skip when blank for a certain fact or phrasing should improve on consistency. TIARA has this ability for example.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper empirically analysis the effect of different methods on consistency of LLMs. ", "reasons_to_reject": "The results that retrieval augmentation improves performance is not much of a surprise. The paper confirms this not with accuracy but with consistency as a benchmark criterion. In that way, the paper does not go far beyond the state-of-the-art, but mostly improves an existing benchmark about consistency (ParaRel).", "questions_to_authors": "The authors conclude in section 6 that\n\n\"Retrieval augmentation is discovered as the best method for consistency improvement, while it does not achieve perfect consistency.\"\n\nGiven this statement, it would be interesting to learn what other approaches the authors could foresee which could provide perfect consistency. One area to explore would be the usage of knowledge graphs as a source of explicit factual knowledge. One could expect that a given query to the same version of the graph will allows provide the same results. ", "paper_topic_and_main_contributions": "The paper discusses two methodologies for addressing challenges of large language models with regards to inconsistencies: up-scaling the LLM, and the usage of retrieval databases.\n\nThe two approaches are evaluated with an improved version of the ParaRel benchmark. The results show that retrieval augmentation shows better performance compared to the up-scaling of LLMs, and is also more efficient. \n\nThe authors then further analyse the retrieval augmentation approach, e.g. with regards to metrics for measuring retrieval consistency.\n\nThe authors conclude that even with the retrieval augmentation approach, there is no full consistency. \n\n\n\n", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "IvDBbrvRXl", "reviewer": "Reviewer_Urmf", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review. We have mainly based our rebuttal on your _Reasons To Reject_. Please let us know if there is anything we should address in addition to this.\n\n> I would have liked to see if the results obtained from the experimentation generalize to more datasets.\n\nAs would we. But as mentioned in our response to reviewer 91nZ, we know of no other datasets that could help us measure that kind of generalizability with respect to factual consistency. To our knowledge, ParaRel is the largest and most relevant dataset for these types of evaluations. If you have recommendations on additional datasets to consider, we would be happy to receive them. \n\n> I would have also liked to see a retriever-reader baseline build on LLaMA or similar LLM to see how the combination of higher parameter count and retrieval-based enhancement impact the overall results.\n\nThat is definitely interesting future work. But we would firstly like to reiterate that the main goal of our paper is not to chase the new SoTA on ParaRel. Our focus lies on measuring and disentangling effects on consistency such that we can understand underlying mechanisms. \n\nSecondly, to develop a “fair” representation of a retrieval-augmented LLaMA it would be necessary to find a suitable information fusion method and train the reader to learn to use the retrieved information, similarly to how Izacard et al. (2022) developed Atlas. To successfully develop such a reader-retriever network is not a simple feat and should amount to its own paper, as was the case for Atlas. However, we would be happy to add this under a future work section.\n\nTo expand on our answer above, we would like to point out that even if we were to use the simplest possible approach for a retrieval-augmented LLaMA, there would be significant and time-consuming choices regarding the method design that would need to be evaluated. We could for example take the passages that were retrieved by Atlas and give them to LLaMA to form a baseline, but it would require figuring out a good way of doing the information fusion for 20 or so passages. Atlas uses FiD but LLaMA cannot do that since it does not build on the T5 model, so one would need to find an alternative approach for fusing large quantities of information into the reader. We could simply append as many retrieved passages as possible to the query (similar to the REALM setup by Guu et al. (2020)) but this is probably not SoTA. To conclude, we would either get something underdeveloped method-wise or need to write a new paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "+ The study of consistency in tasks related to factual knowledge has been limited compared to accuracy so far. This work is timely in that direction.\n+ The paper is well structured and its narrative, understood as the coherence between the research hypoteheses and the experimentation, flows naturally.\n+ The approach is well formalized.\n+ Experimentation on the ParaRel becnhmark is well-deisgned, abundant and offers convincing results.", "reasons_to_reject": "- I would have like to see if the results obtained from the experimentation generalize to more datasets.\n- I would have also liked to see a retriever-reader baseline build on LLaMA or similar LLM to see how the combination of higher parameter count and retrieval-based enhancement impact the overall results", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper studies the impact of two different strategies to mitigate inconsistencies during access to factual knowledge from LLMs, where such inconsistencies manifest in the form of variability in the results produced by semantically equivalent queries to the same model. The first strategy focuses on simply upscaling the model, i.e. replacing a given model with other versions of such model with an increasing parameter count. To this purpose, the authors use the LLaMA family of LLMs. The second strategy involves augmenting the LLM with information retrieved from a collection of documents, through a retrieval-reader architecture model like Atlas, which is used for experimentation and compared to non-retrieval models of similar scale. \n\nThe results show that although both strategies contribute to improve consistency, the latter does so with larger gains and more efficiently so. However interesting this result may be, the main contribution of the paper IMO is the extensive analysis carried out to drill down on the actual reasons behind this finding. To this purpose, the authors f thocus on the ParaRel benchmark, derived from Lama, adapting it for the task at hand. Such experimentation deal with the possible factors related to consistency along the following dimensions: consistency of the retrieval, correlation between retrieval consistency and prediction consistency, consistency and relevance of the retriever vs. consistency and accuracy of the predictions, and dependency of the reader on term frequency in the retrieved results.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "lIIzpq5T41", "reviewer": "Reviewer_Moz1", "summary": "", "questions": "See reasons to reject above (as QA, QB, QC, and QD)", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review. We have mainly based our rebuttal on your _Reasons To Reject_ and _Questions For The Authors_. Please let us know if there is anything we should address in addition to this.\n\n> A) The paper uses the ParaRel dataset which addresses consistency in a very specific task, which I feel should be reflected in the title of the paper. Else, the paper promises in its title more than it can deliver.\n\nFair point, we can revise the title. Would for example the title “The Effect of Scaling, Retrieval Augmentation and Form on the Factual Consistency of Language Models” be acceptable?\n\n> B) Some aspects/hypothesis of the study are very specific to (and are made to fit) the ParaRel dataset and quite idiosyncratic: I am thinking of the experiments on the effect of Form, whereby Form is taken to include an heterogeneous set of 'phenomena' such as use of incorrect preposition, some minor errors for unidiomatic objects, semantic overlap (in what way is this 'form'?).\n\nAgreed, our study is largely adapted to the use of ParaRel for measuring consistency and it could be argued that the query related effects we observe on consistency only are relevant to datasets created by automatic template based generation. However, ParaRel is not the only dataset to be created in this way (and it is probably not the last), so measuring the effect on factual consistency of slightly imperfect language usage should still be of value, e.g. when arguing for why ParaRel is not the best approach for measuring factual consistency and why we need better evaluation methods. Additionally, we do not know in what settings LMs will be used in the future and we cannot rule out that we might require our models to remain factually consistent regardless of whether a question is expressed as “Did Napoleon die in Paris?” or “Did Napoleon die at Paris?”, which could motivate an investigation of such effects of imperfect language usage.\n\nAbout our use of the word “form” for a wide array of phenomena. We are aware that the term is used a bit broadly and will try to refine the terminology and/or add a discussion on how those groups fit under the same category of phenomena.\n\n> C) I found the paper quite a hard read, requiring a lot of long term memory :-D and shuffling backwards and forwards, as all the settings and hypotheses of the (many) experiments are presented first and then the results are presented afterwards. I would prefer a different organization of the paper whereas each group of hypothesis/experiments is explained, and results of the experiment are shown immediately thereafter.\n\nWe understand the point of the reviewer, and would certainly consider reordering parts of the paper, should all reviewers agree this is preferable. If there were specific sections or experiments that were hard to follow, we would also appreciate it if you could point these out, alternatively let us know if the issue lies with the whole structure. \n\nAlso, if you have any examples of papers that followed your suggested setup in a nice way, we would be happy to receive them as inspiration.\n\n> D) The paper lacks an element of 'excitement' (it seems hardly surprising that generating a response given relevant retrieved documents will produce better results, or that the more parameters a model have, the better the results will be). \n\nThis comment is similar to one of Reviewer 91nZ’s so please also refer to our answer there. To summarize, we would like to argue that 1) just because something may be considered expected it should not mean that it should not be supported by empirical evidence and 2) we do more than to simply investigate whether scaling or retrieval augmentation improves on consistency. For example, we: \n1. perform a relative comparison of scaling vs. retrieval-augmentation, \n2. do a causal analysis on Atlas to find that providing perfectly consistent passages across query paraphrases does not lead to perfect consistency, indicating that retrieval-augmentation is not the full story for consistency, and \n3. find that scaling LLaMA has diminishing returns with respect to consistency (scaling from 30B to 65B parameters yields insignificant consistency improvements) and is therefore not the full solution. \n\nPoints two and three above should be of special interest, given the reviewer's claim that those augmentations should yield expected, and therefore unexciting, improvements. (2) shows us that perfect retrieval-augmentation only marginally improves consistency and (3) clearly contradicts the assumption that the more parameters a model has, the better the results will be.\n\n> Also the paper lacks vision outside the ParaRel experiment. The authors make some hints that things might be more complicated and not everything is explained by what they study, but it is very much left to that.\n\nWe agree and will expand on the Future Work section. Some examples on potential directions, mostly focused on exploring new model types, were given in the response to Reviewer 91nZ. Another direction we are interested in is to further investigate the interaction between reader and retriever for retrieval augmented models, for example when there are conflicts between retrieval corpus and reader weights, similarly to the work on “Entity-Based Knowledge Conflicts in Question Answering” by Longpre et al. (2021). This might explain why the reader could be inconsistent in spite of being provided with perfectly consistent passages, as observed in our paper. It would also be interesting to investigate the effect of the interaction between reader and retriever during training, for example if the reader to some extent memorizes less information as a result of receiving passages with the information, where one could use a method similar to “Locating and Editing Factual Associations in GPT” by Meng et al. (2023). Lastly, improved consistency and retrieval augmentation could highlight a path towards more explainable LMs, holding the promise of models that are explainable by design.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "A multi-experiment study focused on ways to improve consistency in LLMs and the identification of elements that affect consistency, using the ParaRel benchmark dataset.", "reasons_to_reject": "A) The paper uses the ParaRel dataset which addresses consistency in a very specific task, which I feel should be reflected in the title of the paper. Else, the paper promises in its title more than it can deliver.\nB) Some aspects/hypthesis of the study are very specific to (and are made to fit) the ParaRel dataset and quite idiosyncratic: I am thinking of the experiments on the effect of Form, whereby Form is taken to include an heterogeneous set of 'phenomena' such as use of incorrect preposition, some minor errors for unidiomatic objects, semantic overlap (in what way is this 'form'?).\nC) I found the paper quite a hard read, requiring a lot of long term memory :-D and shuffling backwards and forwards, as all the settings and hypothesis of the (many) experiments are presented first and then the results are presented afterwards. I would prefer a different organization of the paper whereas each group of hypothesis/experiments is explained, and results of the experiment are shown immediately thereafter.\nD) The paper lacks an element of 'excitement' (it seems hardly surprising that generating a response given relevant retrieved documents will produce better results, or that the more parameters a model have, the better the results will be). Also the paper lacks vision outside the ParaRel experiment. The authors make some hints that things might be more complicated and not everything is explained by what they study, but it is very much left to that.", "questions_to_authors": "See reasons to reject above (as QA, QB, QC, and QD)", "paper_topic_and_main_contributions": "This paper presents a set of experiments on the effect of scaling, retrieval augmentation and form on 'consistency' of language models, that is, the ability of a model to deliver consistent answers to semantically equivalent questions. To do so, the authors use the ParaRel dataset developed for precisely this problem (consistency) which consists in a set of semantically equivalent cloze-style English query paraphrases for 'triple' relations. The dataset is first cleaned by the authors, removing duplicates.\n\nThe authors find that up-scaling (using bigger models) somewhat improves consistency but not as much as retrieval augmentation (using the RAG model Atlas) which has a more significant effect on consistency. 'Form' which is a combination of different issues, specific to the dataset, is also shown to have an effect. The authors also investigate whether retrieved text similarity has an effect on consistency (it has), and whether using same retrieved text for all equivalent queries has an effect on response consistency (it has if it is relevant), amongst other experiments.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "N/A"}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "As mentioned in reasons to reject, a reoganization of the paper might improve its readability.\nAlso, providing some examples might be helpful at some points of the paper. For example the procedure explained in lines 450 to 462."}}]

Author rebuttal:

Meta-review: The paper studies the inconsistencies problem in LLMs. It compares two methodologies: up-scaling the LLM, and the usage of retrieval databases. With an improved version of the ParaRel benchmark the results show that retrieval augmentation performs better and is more efficient. However, even with the retrieval augmentation approach, there is no full consistency. extensive analysis drills down on the reasons behind this finding along various dimensions: consistency of the retrieval, correlation between retrieval consistency and prediction consistency, dependency of the reader on term frequency in the retrieved results, etc.

Strength:
1. Introduce an  improved benchmark (ParaRel) by considering consistency.
2. The approach is well formalized.
3. The result  is  abundant and convincing.

Weakness:
1. The results that retrieval augmentation improves performance is not much of a surprise.
2. The result is restricted to only one dataset
3. There is no result for the combination of higher parameter count and retrieval-based enhancement
Decision: Accept-Main