EpiK-Eval: Evaluation for Language Models as Epistemic Models
Authors: Gabriele Prato
Jerry Huang
Prasanna Parthasarathi
Shagun Sodhani
Sarath Chandar
Venue: EMNLP 2023 Main (2023)
Keywords: large language models
large language model
LLMs
LLM
language model
language models
LM
LMs
EpiK-Eval
knowledge consolidation
story
benchmark
knowledge-base
KB
theory-of-mind
epistemic
hallucination
hallucinate
dataset
task
scale
scaling
knowledge representation
reasoning
consolidation
knowledge
context
evaluation
limitations
limitation
narrative
narratives
training objective
causal language modeling
masked language modeling
TL;DR: First study to investigates LMs' capability to combine information seen in different training documents.

Abstract: In the age of artificial intelligence, the role of large language models (LLMs) is becoming increasingly central. Despite their growing prevalence, their capacity to consolidate knowledge from different training documents—a crucial ability in numerous applications—remains unexplored. This paper presents the first study examining the capability of LLMs to effectively combine such information within their parameter space. We introduce EpiK-Eval, a novel question-answering benchmark tailored to evaluate LLMs' proficiency in formulating a coherent and consistent knowledge representation from segmented narratives. Evaluations across various LLMs reveal significant weaknesses in this domain. We contend that these shortcomings stem from the intrinsic nature of prevailing training objectives. Consequently, we advocate for refining the approach towards knowledge consolidation, as it harbors the potential to dramatically improve their overall effectiveness and performance. The findings from this study offer insights for developing more robust and reliable LLMs. Our code and benchmark are available at https://github.com/chandar-lab/EpiK-Eval

EpiK-Eval: Evaluation for Language Models as Epistemic Models
Gabriele Prato
Jerry Huang
Prasannna Parthasarathi Shagun Sodhani! Sarath Chandar
Mila Meta AI Université de Montréal Polytechnique Montréal *CIFAR AI Chair
scabriele.prato jerry. huang Meaan))mmla Jembea.quebec
Abstract
Type Systems
In the age of artificial intelligence, the role of large language models (LLMs) is becoming in- creasingly central. Despite their growing preva- lence, their capacity to consolidate knowled ledge from different training documents-a crucial ability in numerous applications-remmee explored. This paper presents the first study examining the capability of LLMs to effec- tively combine such information within their parameter space. We introduce EpiK-Eval, Eval, novel guestion-answering benchmark tailored to evaluate LLMs' proficiency in formulating a coherent and consistent knowledge repre- sentation from segmented narratives. Evalu- ations across various LLMs reveal significant weaknesses in this domain. We contend that these shortcomings stem from the intrinsic na- ture prevailing training objectives. Conse- quently, advocate for refining the approach towards knowledge consolidation, as it harbors the potential to dramatically improve their over- all effectiveness and performance. The find- ings from this study offer insights for devel- oping more robust and reliable I.LMs. Our code and benchmark are available at https: //gi thub. com/ channaar-aab/ppi---ee
Tom apple
45%
apple
45%
Tom ohe peor:
Tom dle a pear.
Bolr ate crargge
10%
Tam ate onange
un-
What did Tom eat?
apple
100%
Tom ate pear:
apple and pear.
Bee ale on orange
Type lI Systems
Figure I: When training on samples (red), Type sys- tems process each sequence independently, unable to discern their interrelations. Presented with a question (gray), they are unable to consolidate their knowledge and instead assign a probability to each fact when an- swering (green). In contrast, Type II Systems can learn these relationships and possess a unified knowledge state, allowing them to answer accurately.
1 Introduction
linked this knowledge being diffused, state where the known information remains indepen- dent (AlKhamissi et al., 2022).
Developing systems that reason through lan- guage understanding has been cornerstone in natural language processing research. Recent progress (Devlin et al., 2019; Brown et al., 2020; Touvron et al., 2023) has showcased notable ad- vancements in a variety of reasoning tasks (Hwang et al., 2021; Cobbe et al., 2021; Yang et al., 2022; Han et al., 2022; Zelikman et al., 2022; Lampinen et al., 2022). Arguably, the ability of LMs to act knowledge bases (lzacard et al., 2022) has been large factor in these successes. How- ever, observed errors (Kim al., 2021; Zhang et al., 2023) on tasks which entail learning depen- dencies among multiple facts can be potentially
Meanwhile, humans maintain a consistent in- ternal representation of the world which they ac- tively use for reasoning (Nader, 2009; Johnson- Laird, 2010). This motivates language models to be equipped and evaluated to be knowledge consistent (Moghaddam and Honey, 2023; Hao et al., 2023), the lack of consistency and con- solidation in parametric knowledge could result in poor reasoning (Madsen al., 2022: Valmeekam et al., 2022; Zheng et al., 2023). Extrapolating from AlKhamissi et al. (2022), we focus on the behaviour of LMs as epistemic models (Rendsvig

---

and Symons, 2019 Osband et al., 2023) with consolidated and consistent retention of multiple learned facts in its parameters, a knowledge state.
100
T5
unsegmented sepmented
FlaniTs
80-
OPT
Accuracy (%)
When the facts are concatenated into a long con- text, the knowledge state can be constructed solely from this context. The success of in-context Iearn- ing. where LM infers over specific prompt de- scribing the task and few examples (Brown et al.. 2020; Lu al., 2021; Wu et al., 2022), primarily relies on the information in the input to be cor- rect (Liu et al., 2021). However, real-world sce- narios rarely adhere to this setting. For instance, a LM might have to recall information stored in its parameter space, but the information can originate from multiple sources encountered during training. Consequently, to maintain consolidated knowl- edge state, LMs must serve as epistemic models, effectively modeling knowledge dependencies. As LMS continue to establish themselves as fundamen- tal tools in machine learning research (Ahn et al., 2022; Huang et al., 2022: Hao et al., 2023), un- derstanding their knowledge structure becomes im- perative. The central question emerging from this exploration is whether the knowledge within these models exists as dispersed, standalone elements, or whether they possess the capacity to sustain an interconnected and consistent knowledge state.
60-
20-
10a
10p
Parameters
Figure 2: Performance on EpiK-Eval, measuring ac- curacy as the percentage of correct answers. Models struggle to answer questions that require consolidating knowledge from multiple training documents (orange). In comparison, they perform much better if the same in- formation can be found within single document (blue).
stories segmented throughout the training corpus, analogous news articles covering certain topics through time in large web corpora. These LMs are evaluated on their ability to consolidate the knowl- edge of the segmented narratives. Specifically, we test 7 different categories of reasoning involving complex yet explicit relations over the presented information. Although EpiK- Eval tasks require rea- soning beyond explicit factual knowledge, they do not need modeling of other agent's belief states. As such EpiK-Eval is positioned order of com- plexity above vanilla knowledge extraction tasks and an order below complex theory-of-mind tasks. We assess where LMs lie on the spectrum between Type and Type systems, based on their inferred knowledge state evaluated through aggregate per- formance on EpiK-Eval. Type systems maintain information independently across different observa- tions, whereas Type II systems are characterized by their ability to consolidate information from across those observations (example in Figure 1).
Thus far, assessing parametric knowledge rep- resentations has garnered interest on two ends of spectrum. On one side, the paradigm of LMs as knowledge bases hypothesizes that LMs store and retrieve knowledge when prompted, with improved efficiency possible by storing ever- increasing amounts of knowledge (Petroni et al., 2019; Wang et al., 2020; Heinzerling and Inui,
2020; Sung et al., 2021; Dhingra et al., 2022). Others (Gu et al., 2023; Sap et al., 2022: Ruis et al., 2022; Zhang et al., 2023; Moghaddam and Honey, 2023) evaluate theory-of-ming (Premack and Woodruff, 1978), the ability to impute mental states oneself and others, in LMs and show they fall short of having a consistent world belief state. abilities for LMs enhance their reasoning and applications, evaluating and equipping the LMs with a first-order knowledge state is a necessary next step from LMs merely being knowledge bases.
Overall, our findings indicate that LMs exhibit characteristics of Type I rather than Type II systems. Indeed, we observe a significant performance gap between LMs trained on these segmented narratives versus unsegmented ones (Figure 2). Specifically, these models struggle to recall and consolidate the proper information and hallucinate facts and events at higher rate than those trained on unsegmented stories. This pronounced disparity highlights an intrinsic shortcoming in existing LMs. We posit that this can be attributed to their training objective, suggesting a need for the development of novel
Although theory-of-minn
To this end, we propose the novel Epistemic Knowledge Evaluation (EpiK-Eval) benchmark, to evaluate this ability to leverage such consoli- dated knowledge state. EpiK-Eval trains LMs on

---

Story
Question
Answer
[Task 7] Alice's Day
[Task 7] Between going and making tea, does Alice read book?
Morning, Alice goes for a walk. Noon, Alice makes phone call. Afternoon, Alice makes tea. Evening. Alice reads a book. The answer is
Morning, Alice goes walk. Noon, Alice makes phone call. Aftemoon, Alice makes tea. Evening, Alice reads book.
[Task 9] Bob at the Restaurant Boh arrived the restaurant 6:00 PM. 2 minutes after arriving, Rob ordered drink. 10 minutes after ordering drink, Bob ordered a hamburger. 5 minutes after ordering hamburger. Bob asked the bill.
Bob arrived at the restaurant at 6:00 PM. minutes after arriving, Bob ordered a drink. 10 minutes after ordering a drink, Rob ordered a hamburger. 5 minutes after ordering a hamburger, Bob asked for the bill. 2 10 + 5 = 17. The answer is 6:17 PM.
[Task 9] At what time does Boh ask for the bill?
Table
Sample stories, questions and answers from our dataset. Additional examples can be found in Appendix
methods aimed towards improvements in knowl- edge consolidation. By investigating how LMs consolidate and reason from segmented knowledge, we aim to catalyze further research in the pursuit of more sophisticated, reliable, and knowledge- consistent machine learning systems.
riodic flow, it is necessary that they use such infor- mation appropriately during inference. While tech- niques like self- tfomptinnn and generation over self- retrieval are gaining popularity, the performance re- lies on the quality of the prompt, which adds to the robustness concerns on the performance of LMs on varying reasoning tasks. Inspired by epistemology, we design Epik-Bval to diagnose whether LMs comply with a first-order knowledge state follow- ing sequence of facts which holds consolidated summary of information during inference.
2 Epistemology & Language Models
Eppistmicc frameworks (Wang, 2015; Rendsvig and Symons, 2019) are formal systems used to rep- resent knowledge, belief and the uncertainty that entails what reasoning system knows and/or be- lieves. This is enabled through organizing the knowledge observed by the system. The rules to combine the knowledge in the abstract framework governs combining a new information to the cur- rent set of information, or when to ignore the new information, and using the current beliefs to antici- pate related events. While LMs behave as KBs to store known relations, epistemic logic provides us with the inspiration to describe how these models organize and update their knowledge.
3 EpiK-Eval
The EpiK-Eval benchmark presents a suite of novel, narrative- baativee-ased diagnostic tasks, meticulously de- signed to evaluate LM's capacity to construct a comprehensive, unified knowledge state.
Dataset: Our benchmark comprises 18 tasks. which are questions about relations between facts and events in stories, e.g., "Does x happen be- fore/after y?". Table 2 provides the full list of tasks. For each task, generate 100 stories fol- lowing a per task template. Task 2 for instance uses the following template:
Consider the example from Figure 1, where we have the knowledge x1: "Tom ate an apple.", ".,e""",", 22: "Tom ate pear." and Ø3: "Bob ate an orange.". Prompted with the question "What did Tom eat?". the model must recall knowledge from within its parameter space. It has connect 2 and 22 while also ignoring 23. To answer the query, system is expected to consolidate the information and retain a knowledge state over the information it had seen until then. However, an inability to draw the con- nections would leave the facts disconnected. We describe the model that struggles to consolidate as Type and one that is better it and infer over consolidated knowledge state as Type II.
[Task 21 {name)'s Vacation (name) went (activity) (day).
where the first line is the story title, the [name] is randomly sampled such that is unique to each story and the (activiry) and [day) in sentence are randomly sampled from the list ["fishngg", "hik- ing" "I and [ ("Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday"] re- spectively. The story can have random number of sentences, with the range pre-determined for each task, ex. Task 2 stories can have between 3 and 5 sentences. An example story for Task 2 is
With LMs being used in real-world scenarios where information is frequently presented as a pe-

---

Category
Description
Tasks
Counting
Tests proficiency in quantifying occureences and quantities.
How many times does x happen?
List the different x.
Tests ability identity and enumerate within given list.
Listing
Is the /'th on the list?
Among the list of x, is there
Tests understunding relutive amounts, frequency, and ranking.
Does happen more/less often than
Ranking
x the same as y?
Does happen before/after y?
When. x happens, does y happen??
Between and does z huppen?
Tests the model has learned temporal dependencies in the data, such what events follow each other.
Temporal
How much time has passed between and
At what time does happen based on y?
After how many does y huppen?
What is state of whea happeas?
Causal
Tests understanding cause-eflect.
x had/hadn' happened, would y have happened?
x the only time that y happens?
x'th time that y happens, what unique detail about y
Uniqueness
Tests understanding of exclusivity or uniqueness in the data.
compared to the other times?
Among the list of x. is there only
Consistency
Tests ability recognize consistency patterns states.
Every time happens, y always the same?
Table 2: The 18 tasks in the EpiK-Eval benchmark, categorized by type. Tasks aim to encompass a wide range of fact and event relationships.
[Task 2] Tom's Vacation
which itself has three parts: recall of the entire story, an optional reasoning part depending the task and the final answer. For example, question- answers pairs in Task 2 uses the following template
Tom went fishing on Monday.
Tom went hiking Wednesday. Tom went fishing on Saturday.
Thus, with 100 stories generated for each 18 tasks, there is total of 1800 stories, which re- ferred to as our dataset of unsegmented stories Dy {I 42, F1800) After generating these stories, we also generate a second dataset, con- sisting of the segmented version of these stories. For each given story, we segment it into individual sentences and add part number to the title. For example, given the previous story about Tom, would get the following three text sequences:
[Task 2J How many times did [name) go fishing? (story)
The answer is {counr).
with an example of a generated question-answer pair being
[Task 2] How many times did Tom go fishing? Tom went fishing on Monday. Tom went hiking on Wednesday. Tom went fishing on Saturday.
(Task 2] Tom's 's Vacation, Part 1/3 Tom went fishing on Monday.
The answer is 2.
A description of each task, its templates and examples are provided in Appendix B. A few ex- amples are also provided in Table
[Task 2] Tom's Vacation, Part 2/3 Tom went hiking on Wednesday.
[Task 2] Tom's 's Vacation, Part 3/3 Tom went fishing on Saturday.
Having generated one question per story, we have a total of 1800 question-answer pairs split randomly into two sets: the validation and the test set. For the models to learn the answer format, add question -nesstion-nsswr answer examples to the training set. We thus generate an additional 1800 stories and guestion-ansses pairs. We discard the stories and add these 1800 question-answer pairs to the training set, such that there are no overlaps between questions in the training, validation and test set.
We do this for all 1800 stories and get 6800 story segments, which form our dataset of story segments Ds = {81, $2, .... $6500
For each story, we also generate one question- answer pair. Questions
re- re-phrasings of the task. For example, for Task 2 "How many times does happen?", we have "How many times did [name] go fishing?"" The question-answe pairs are also generated following a template. The template al- ways consists of a question followed by the answer
Evaluation Process: To evaluate pre- pre-trained LMs for their ability to consolidate knowledge, given a

---

pre-trained language model we make two copies of it: My and Ms. We fine-tune My on the unseg- mented stories and Ms on the segmented stories. The prior setting ensures all necessary information for answering a given question to be found in a single text sequence without requiring the model to learn dependencies across multiple text sequences. The latter requires consolidating information from the narrative segments. Having both allows mea- sure the effect of information being spread across separate text sequences and the LMs' ability to con- solidate this knowledge at inference, by measuring the gap in performance between both models.
My serves as an upper-bound on the performance and any potential gap in performance between it and My showcases the added difficulty of consol- idating knowledge from the story segments. The number of correct responses over the total number of questions is referred to as the accuracy. We also measure an additional metric, which we refer to the hallucination rate. Given an answer, consider only the recall part of the answer and disregard the reasoning part and the final answer. The hallucina- tion rate is the number of recalled sentences that contain an error (does not match with the actual sentence in the narrative) over the total number of recalled sentences. This provides more fine- grained examination of the recall and knowledge consolidation capabilities of the model. We want to evaluate if the model is more likely to hallucinate facts, events or segments when recalling these from multiple training sequences (segmented setup) ver- sus a single training sequence (unsegmented setup). Once both models have been fine-tuned, we take the best performing checkpoint of each model on the validation set and evaluate these on the test set. This is done in the same manner as the validation, except that the questions are from the test set.
Mu and Ms are fine-tuned on their respective dataset, Du and Ds. as well as the training set of question-answer examples. Thus, one epoch for My consists of 3600 samples (1800 stories + 1800 q/a examples) and one epoch for Ms of 8600 sam- ples (6800 segments 1800 q/a examples). Sam- ples are shuffled such that batch may contain mix of stories and question-anss answer examples in the case of My or story segments and question- answer examples in the case of Mg. Models are fine-tuned with their respective pre-training objec- tive. Specifically, in the case of encoder-decoder style models, the story's title (first line in the text sequence) is fed to the encoder and the decoder is expected to output the rest of the story in the case of My or the story segment in the case of Ms. As for question-answer pairs, the question is fed to the encoder and the model is expected to output the answer. For causal language models, they are simply expected to predict the next token in the given sequence, as is standard procedure. Precisely, for My. text sequence is either entire story or question concatenated with its answer, while for Ms. text sequence is either story segment or question concatenated with its answer.
4 Experiments
We experiment with three different LLMs: T5 (Raf- fel et al., 2020), its instruction-tumed variant Flan- T5 (Chung et al., 2022), and OPT (Zhang et al., 2022). For T5 and Flan-T5 models, we benchmark sizes from Small to XL. For the OPT model, we benchmark sizes 125M, 350M, 1.3B and 2.7B pa- rameters. Unless otherwise stated, the reported performance is on the test set. Performance scores presented in this section are always averaged over the 18 tasks of our benchmark. Individual task per- formance can be found in Appendix B, and training details are provided in Appendix A.
During fine-tuning, both models are also peri- odically evaluated on the validation set. Models are run in inference mode as described in the pa- pers they were introduced in. We prompt models with questions from the validation set and model answers are compared to the target answers. For an answer to be deemed as correct, it must match the exact target answer. This is to capture poten- tial recall and reasoning errors as well as verify the final answer. This important for evaluating Mg's s ability to consolidate the separate story seg- ments, which is why we require the model to recall the entire story when answering a question. Here,
4.1 Are LMs Type I or Type II Systems?
Answering this question relies on 1) the model performing well in the unsegmented setting and 2) equal performance in the segmented setup.
Performance on our benchmark is shown in Fig- ure 2. There is noticeable decline in performance for models trained on segmented stories compared to unsegmented ones. This trend suggests that, regardless of size or training methodology, LMs struggle to consolidate knowledge from multiple sources, behavior characteristic of Type systems.

---

Recall
Reasoning
Final Answer
Accuracy (%)
40
Flan-T5
20-
unsegmented
segmented
10a
107 10P
109
10a
Parameters
Parameters
Parameters
Figure 3: Breakdown of model answers into three parts: story recall, reasoning and final answer. (Leff) percentage correct recalls. (Center) percentage of correct reasonings when recall is correct. (Right) percentage correct final answers when recall and reasoning are correct when recall is correct and task has no reasoning part. Recall performance is worse when models need to recollect information from multiple training documents (orange) versus from single documents (blue), but reasoning and final answer capabilities seem unaffected.
In the unsegmented setting, Flan-T5 surpasses T5. OPT, the other hand, starts behind both but matches T5's performance in its largest variant. Interestingly, in the segmented scenario, all models exhibit comparable performance.
Reasoning: When narrowing down to answers with correct recall, we analyze reasoning capabil- ities, depicted in the center plot of Figure 3. Noteworthy observations include:
Models trained on segmented stories perform slightly better than their unsegmented counter- parts, although this may be due to variance from the much smaller subset size for segmented sto- ries, rather than better reasoning capabilities.
When scaling the LMs, performance generally improves as LMs are scaled in both segmented and unsegmented setups. The only exception is TS when trained unsegmented stories.
4.2 In-Depth Answer Analysis
Among unsegmented models, T5 trails both Flan-T5 and OPT. While it's expected for Flan- T5 to outperform T5 due to its instruction tuning, OPT outperforming T5 is intriguing.
In order to better understand the models' behaviour, we take closer look the models' answers. We break these down into three parts: the recall of the story, the reasoning and the final answer.
For segmented models, performance is gener- ally uniform across all models. However, in the largest variants, both T5 and Flan-T5 experience a significant drop in performance.
Recall: We initially examine the models' recall capabilities. The left plot of Figure presents the percentage of correct recalls. We observe:
A consistent trend with Figure 2, models trained unsegmented stories greatly outperform those trained segmented ones.
In both the segmented and unsegmented setting, scaling doesn't enhance reasoning skills.
Final Answers: Focusing on answers with both correct recall and reasoning, or just correct recall for tasks without a reasoning component, we assess the correctness of the final answers (right plot Figure 3). We observe that:
Within the unsegmented setting, OPT lags slightly behind T5, while T5 and Flan-T5 show comparable recall capabilities. Scaling effects are more pronounced for OPT, while T5 and Flan-T5 show marginal improvements.
Segmented models show superior performance, but the variance argument remains relevant.
Models trained on segmented stories all demon- strate similar performance, with notable im- provements as they scale.
The performance of a given model seems to follow similar trend in both settings.
Analysis of model recall lengths compared to target distribution revealed similar patterns, indicating that segmentation doesn't t impact the recall span in terms of sentence numbers. See Appendix
As models scale, Flan-T5 and OPT both show improved performance in each setting. How- ever, T5's performance declines with its largest variant.

---

Train
Test
100
segmenled
Hallucination Rate. (%)
T5
RanTT
OPT
60
Parameters
Parameters
Figure 4: Model hallucination the training (left) and the test (right). Models which need to recall information from multiple documents seen during training (orange) are more prone to hallucinations during testing than models which only need recall information from single training document (blue).
Given these results, the drop in performance of T5-XL in Figure 2 can be explained by its poor reasoning and final answer performance rather than issues with recall.
els, the hallucination rate of T5, Flan-T5 and OPT decreases model size increases. Notably, these models exhibit slightly higher hallucination rate on the test set than on the training set. This could be attributed to the change in context, where the model prompted with a question instead of a story title. Interestingly, OPT models in the unseg- mented setting hallucinate more than T5 and Flan- T5 models on the test set, but not on the training set. This behavior might stem from OPT models overfitting training samples with positional embed- dings, affecting their performance when prompted with questions, which differ length from titles. Conversely, for the segmented-trained models, hallucination rates among different models are more similar and also decrease with scale. How- ever, whether this decline continues models in- crease in size is uncertain. To elucidate this, exper- iments with larger models are essential.
Overall, this reveals that while unsegmented- trained models may falter in recall, reasoning, providing the correct final answer, segmented- trained models predominantly grapple with recall errors. This shows that their main challenge is con- solidating knowledge effectively in order to solve the problem.
4.3 Hallucinations
Our next analysis of model behaviour looks at the tendency of these models to "hallucinate". Exam- ples of such hallucinations can be found in Ap- pendix C. Figure 4 showcases the hallucination rate, defined as the percentage of sentences in the recall part of an answer that aren't present in the target. This rate presented for both the training and test sets.
4.4 Effeet of Scale
For the training set, the hallucination rate re- mains nearly 0% for both segmented and unseg- mented stories, with the highest observed rate be- ing 0.2%. However, distinct difference emerges the test set. Models trained on segmented sto- ries display a significant gap in hallucination rates compared to those trained on unsegmented stories. This suggests that models recalling and consolidat- ing information from multiple training documents are more susceptible to hallucinations, which high- lights one of the potential reasons why hallucina- tions happen in LLMs.
Both key metries we use study knowledge con- solidation: recall performance and hallucination rate, seem to improve as model size increases. However, given the improvement in performance both the unsegmented and segmented settings, this is not conclusive evidence to knowledge consolida- tion happening with scale. To support the emergent behavior hypothesis (Wei et al., 2022b), the im- provement rate in the segmented setting should significantly outpace that in the unsegmented one. Additionally, it remains uncertain if performance in the segmented scenario will eventually platcau, perhaps before reaching the performance levels
Upon examining the unsegmented-traag -rriiieddddddd mod-

---

models trained on unsegmented stories. To truly gauge the impact of scale on knowledge consoli- dation, experiments with larger models are needed, but we unfortunately lack the compute to run them.
maintain state information, the authors probe for factual information that does not require consolida- tion. Unlike existing works, using Epik-Eval, we focus on studying the effect of information spread during a LM's training on the model's ability to recall and consolidate the knowledge at inference.
5 Related Work
Knowledge Representation: Results from prob- ing neural language models have shown models not only encoding facts (Petroni et al., 2019) or linguistic knowledge (Tenney al., 2019) in their parameters, but also using them in downstream tasks (Peters et al., 2018; Goldberg, 2019; Kazem- nejad et al., 2023). The amount of knowledge a model retains in the parameters (Dhingra et al., 2022; AlKhamissi et al., 2022; Roberts et al., 2020) is perceived as a reflection of the models' success in downstream tasks (Moghaddam and Honey, 2023).
6 Discussion
Consolidating Knowledge in Language Models: Our study delineates the limitations of language models in consolidating knowledge across different text sequences, compared to a noticeably stronger performance when working within a single text se- quence. We attribute this disparity primarily to the core objective of such models: to enhance word prediction within given sequences, while also us- ing knowledge from previously processed text quences, encoded in the model's parameters.
However, relying on parameters for knowledge has shown that language models can hallucinate (Ji al., 2023) and struggle to model less frequent data (Kandpal et al., 2022). Going further than the existing work, with the proposed E framework we attempt to understand LMs' behav- ior towards knowledge representation of segmented text chunks describing set of relation-categorin
Current pre-training objectives such as masked and causal language modeling (Devlin et al., 2019; Brown et al., 2020) potentially prioritize learn- ing dependencies within text sequences over those spanning across multiple ones. For instance, cause-and-effect relationship could exist between two sequences. However, if the content of the first does not explicitly help in predicting the second's s content, the model might not learn this relation. Consequently, numerous inter-sequence dependen- cies in the training corpus, which may hold sig- nificant importance in downstream tasks, may be ignored owing to their perceived irrelevance in the next-word prediction task. In contrast, the model can readily establish correlational dependencies within individual sequences which can even lead the direct memorization of text, a frequent occur- rence in LLMs (Carlini et al., 2020; McCoy et al., 2021; Tirumala et al.. 2022; Carlini et al., 2023). In light of these arguments and results, we assert the need to revisit the training objectives of lan- guage models. To utilize these models effectively, we should prioritize devising training methods that capture and consolidate the numerous information dependencies within the training corpus. A po- tential avenue to explore could be to guide these models in consolidating their knowledge via meth- ods such as RL-HH (Bai et al., 2022) or self-taught (Zelikman al., 2022).
Multi-Hop QA: In multi-hop question answering (QA) benchmarks (Welbl et al., 2017; Yang et al., 2018; Ho et ul., 2020; Mavi et al., 2022), models are tasked with answering questions by navigating through multiple documents and extracting relevant information from them. This process is pure infer- ence, with the model relying on external knowledge sourced directly from the documents.
Conversely, we focus on investigating how well these models can recall and consolidate the knowl- edge already embedded within their parameter prace-knowledge acquired during training (re- ferred to as "internal knowledge"). This contrasts with merely assessing the model's 's ability to con- duct document-based searches.
Artifacts of Reasoning in LMs: To utilize the stored knowledge, approaches such as prompting and in-context learning (Wei et al., 2022a,b.c; Liu et al., 2023) have gained popularity for tasks in- volving reasoning over a given context. While LMs have shown strong reasoning skills when in- formation is fully available in the context (Han et al., 2022; Zelikman et al., 2022), inconsistent results appear when such is not the case (Gu et al., 2023). While Li et al. (2021) demonstrate that LMs
Exploiting Longer Context vs Knowledge Con- solidation: In response to the knowledge consol- idation challenge faced by LMs, it could be ar-

---

gued that the inclusion of a comprehensive context through prompts could be an effective alternative to having the LM remember the necessary context autonomously. This proposition is emboldened by recent successes in extending the context window size (Xiong et al., 2022; Ratner et al., 2023; An- thropic. 2023) as well as the sequence length (Dai et al., 2019; Gu et al., 2022; Poli et al., 2023; Bertsch al., 2023). Such additional information can be supplied by either user or an auxiliary sys- tem (Nakano et al., 2022; Schick al., 2023; Patil et al., 2023; Paranjape et al., 2023).
bly significant. As we venture into the realm of real-world applications (OpenAI, 2023; Anil al., 2023; Touvron et al., 2023), there exist a wide ar- ray of settings that necessitate a LLM to recall and integrate data from multiple text sequences. Ac- cordingly, enhancing this ability can potentially elevate the efficiency, robustness and performance of such models, thereby redefining the landscape of complex language tasks.
One challenge with studying this problem at scale is distinguishing whether LLMs demonstrate an improved ability to model dependencies within their training corpus (emergent behavior) or if the dataset diversity enables the extraction of most de- pendencies of interest within single text sequences in the corpus. To probe for knowledge consolida- tion at scale, we propose the use of self-contained narratives such short stories books. These documents can segmented and dispersed within the training corpus of LLMs (Touvron et al., 2023; Computer, 2023) and evaluation can be performed in a similar fashion as EpiK-Eval, where questions can assess the understanding of the overall narrative and the various relations in the story. With complex enough naratives, this methodology should provide robust framework for examining the knowledge consolidation capabilities of LLMs.
Expecting humans to provide comprehensive context may, however, be impractical. Given the diverse range of specialist knowledge needed for various tasks, it's possible for a user to lack the necessary expertise. On the other hand, integrating auxiliary systems to provide these contexts presents a challenge analogous to that faced by LMs. To be useful, such an auxiliary system must understand and retain all relevant interdependencies within the training corpus related to problem-solving Unfor- tunately, current auxiliary systems, such as search engines or retrieval tools (Karpukhin et al., 2020; Guu et al., 2020; Lewis et al., 2020), fall short of this holistic understanding and recall of context.
Another strategy leveraging longer context win- dows can be train LMs on concatenated text se- quences with inherent relevance (Shi al., 2023). This approach, however, presents its own complex- ities. The innumerable ways texts can interrelate complicates the process of determining and train- ing on all possible combinations. Hence current solutions do not provide comprehensive solution to this issue
Conclusion
In this paper, we presented the EpiK-Eval bench- mark, a tool designed specifically to evaluate the proficiency of LMs in consolidating their knowl- edge for problcm-solving tasks. Our findings un- derscore the limitations of current LMs, which ap- pear to mostly maintain disjoint knowledge state of training observations. Further, we note sig- nificant performance gap and an increased rate of hallucinations for models trained on segmented nar- ratives compared to those trained on unsegmented ones. We attribute these discrepancies to the train- ing objectives of the models, which underscores the need to more effectively model the dependencies within the training corpus. By highlighting current limitations and opportunities for improving LMs, these results delineate paths for future research, hopefully enabling the growth of language models beyond simple knowledge bases.
Knowledge Consolidation at Scale: Our study un- derscores a substantial discrepancy in performance between models trained on segmented stories and those trained on unsegmented stories. If we as- sume that the recall performance for models in the segmented setting continues to improve without plateauing prematurely, our estimates (Caballero et al., 2022) suggest that a model with 172B param- eters, trained on our benchmark's s segmented sto- ries, would required to match the performance of an 80M parameter model trained on the unseg- mented stories
Although consolidating knowledge from frag- mented text sequences arguably poses a greater challenge than from a singular cohesive text, the margin for enhancement in this domain is possi-
Limitations
Ensuring that EpiK-Eval's data doesn't leak into the pre-training set of LLMs is a challenge. This

---

inclusion could skew the benchmark's results. One straightforward solution is to check if the data ex- ists within the pre-training set, though this method is computationally intensive. Another practical ap- proach is to generate and release a new version of the dataset periodically, for instance, annually. To further safeguard against potential leaks, we've en- crypted the data in the public release of the bench- mark. Users are required to decrypt it locally before use.
Rohan Anil, Andrew M. Dai, Orhan Firat, Melvin John- son, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, Eric Chu, Jonathan H. Clark, Laurent El Shafey, Yanping Huang. Kathy Hellsr-Hlll,tem, Gau- rav Mishra, Erica Moreira, Mark Omernick, Kevin Robinson, Sebastian Ruder, Yi Tay, Kefan Xiao, Yuanzhong Xu, Yujing Zhang, Gustavo Hernandez Abrego. Junwhan Ahn, Jacob Austin, Paul Barham, Jan Botha, James Bradbury, Siddhartha Brahma, Kevin Brooks, Michele Catasta, Yong Cheng, Colin Cherry, Christopher A. Choquette-Choo, Aakanksha Chowdhery, Clément Crepy, Shachi Dave, Mostafa Dehghani, Sunipa Dev, Jacob Devlin, Mark Díaz, Nan Du Ethan Dyer, Vlad Feinberg, Fangxiaoyu Feng, Vlad Fienber, Markus Freitag, Xavier Gar- cia, Sebastian Gehrmann, Lucas Gonzalez, Guy Gur- Ari, Steven Hand, Hadi Hashemi, Le Hou, Joshua Howland, Andrea Hu, Jeffrey Hui, Jeremy Hur- witz, Michael Isard, Abe Ittycheriah, Matthew Jagiel- ski, Wenhao Jia, Kathleen Kenealy, Maxim Krikun, Sneha Kudugunta, Chang Lan, Katherine Lee, Ben- jamin Lee, Eric Li, Music Li, Wei Li, YaGuang Li, Jian Li, Hyeontaek Lim, Hanzhao Lin, Zhongtao Liu, Frederick Liu, Marcello Maggioni, Aroma Mahendru, Joshua Maynez, Vedant Misra. Maysam Moussalem, Zachary Nado. John Nham, Eric Ni, Andrew Nys- trom, Alicia Parrish, Marie Pellat, Martin Polacek, Alex Polozov, Reiner Pope, Siyuan Qiao, Emily Reif, Bryan Richter, Parker Riley. Alex Castro Ros, Au- rko Roy, Brennan Saeta, Rajkumar Samuel, Renee Shelby, Ambrose Slone, Daniel Smilkov, David R. So. Daniel Sohn, Simon Tokumine, Dasha Valter. Vijay Vasudevan. Kiran Vodrahalli, Xuezhi Wang. Pidong Wang, Zirui Wang, Tao Wang, John Wiet- ing, Yuhuai Wu, Kelvin Xu, Yunhan Xu, Linting Xue, Pengcheng Yin, Jiahui Yu, Qiao Zhang, Steven Zheng, Ce Zheng, Weikang Zhou, Denny Zhou, Slav Petrov, and Yonghui Wu. 2023. PaLM 2 Technical Report. arXiv e-prints, page arXivv2305.100
Ethics Statement
This study employs machine learning algorithms, specifically large language models, which are trained on vast amounts of text data. While these models have shown remarkable predictive capabil- ities, it is important to underscore the ethical con- cern that arises from their training process. These models often learn from data that is intrinsically embedded with human biases, which can subse- quently be reflected in their outputs. Therefore, it is paramount to approach any output produced by these models with critical consideration of this potential for embedded bias.
Acknowledgements
Jerry Huang is supported by Natural Sciences and Engineering Research Council of Canada (NSERC) Canada Graduate Scholarship and Fonds de Recherche du Québec Nature et technologies (FRQNT) training scholarship.
Sarath Chandar is supported by the Canada CI- FAR Al Chairs program, the Canada Research Chair in Lifelong Machine Learning, and the NSERC Discovery Grant.
Anthropic. 2023. Introducing 100k context win- dows. https: htt////wwwtttpppc.ooo index/ gook-context-windonn
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kermion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds Danny Hernandez, Tristan Hume, Scott Johnston, Shauna Kravee, Liane Lovitt, Neel Nanda, Catherine Olsson, Dario Amodei, Tom Brown, Jack Clark, Sam McCandlish, Chris Olah, Ben Mann, and Jared Kaplan. 2022. Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback. arXiv e-prints, page arXiv:2204.0586..
This research was enabled in part by compute re- sources provided by Mila (mila.quebecc) and the Digital Research Alliance of Canada (alliance- can.ca).
References
Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, et al. 2022. Do as can, not as i say: Ground- ing language in robotic affordances. arXiv preprint arXiv:2204.01991.
Amanda Bertsch, Uri Alon, Graham Neubig, and Matthew R. Gormley, 2023. Unlimiformer: Long- Range Transformers with Unlimited Length Input. arXiv e-prints, page ar arXiv:2305.016
Badr AlKhamissi, Millicent Li, Asli Celikyilmaz, Mona Diab, and Marjan Ghazvininejad. 2022. A review on language models as knowledge bases.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie

---

Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz: Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. arXiv e-prints, page 2001::2005.14165.
Bhuwan Dhingra, Jeremy Cole, Julian Martin Eisenschlos, Daniel Gillick, Jacob Eisenstein, and William W. Cohen. 2022. Time-aware language mod- els as temporal knowledge bases. Transactions the Association for Computational Linguistics, 10:257- 273.
Yoav Goldberg. 2019. Assessing bert's 's syntactic abili- ties.
Albert Gu, Karan Goel, and Christopher Re. 2022. Ef- ficiently modeling long sequences with structured state spaces. In International Conference on Learn- ing Representations.
Ethan Caballero, Kshitij Gupta, Irina Rish, and David Krueger. 2022. Broken Neural Scaling Laws. arXiv e-prints, page arxiv:2210. 14891.
Yuling Gu, Bhavana Dalvi Mishra, and Peter Clark. 2023. Do language models have coherent mental models of everyday things?
Nicholas Carlini, Daphne Ippolito. Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. 2023. Quantifying memorization across neural lan- guage models.
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat. and Ming-Wei. Chang. 2020. REALM: Retrieval- Augmented Language Model Pre-Training, arXiv e-prints, page arXiv:2002.0890 089909
Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Xiaodong Song. Ülfar Erlingsson, Alina Oprea, and Colin Raf- fel. 2020. Extracting training data from large lan- guage models. In USENIX Security Symposium.
Simon Jerome Han, Keith Ransom, Andrew Perfors, and Charles Kemp. 2022. Human -ikan property induction is challenge for large language models.
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghami, Siddhartha Brahma, Al- bert Webson, Shixiang Shane Gu. Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdh- ery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang, Andrew Dai, Hongkun Yu, Slav Petrov, Fed H. Chi, Jeff Dean, Ja- cob Devlin, Adam Roberts, Denny Zhou, Quoe V. Le. and Jason Wei. 2022. Scaling instruction finetuned language models.
Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong. Zhen Wang. Daisy Zhe Wang, and Zhiting Hu. 2023. Reasoning with language model is planning with world model. arXiv preprint 2xXv:2335.11492
Benjamin Heinzerling and Kentaro Inui. 2020. Lan- guage models as knowledge bases: On entity repre- sentations, storage capacity, and paraphrased queries. arXiv preprint 2rXivv2008.0900
instructionffiettuned
Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sug-
awara, and Akiko Aizawa. 2020. Constructing A Multi-hop QA Dataset for Comprehensive Evalu- ation of Reasoning Steps. arXiv e-prints, page arXiv:201 20.00000000000.
Karl Cobbe, Vineet Kosaraju. Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv: 10. 14168.
Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang. Pete Florence, Andy Zeng, Jonathan Tomp- son, Igor Mordatch, Yevgen Chebotar, et al. 2022. Inner monologue: Embodied reasoning through planning with language models. arXiv preprint 2207.0200000000
Together Computer. 2023. Redpajama: An open source recipe to reproduce Ilama training dataset.
Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Car- bonell, Quoe V. Le, and Ruslan Salakhutdinov. 2019. Transformer-Xe XL: Attentive Language Models Be- yond a Fixed- Length Context. arXiv e-prints, page arXiv:1901 100..6000..0800.
Jena D. Hwang, Chandra Bhagavatula, Ronan Le Bras, Jeff Da Keisuke Sakaguchi. Antoine Bosselut, and Yejin Choi. 2021. Comet-atomic 2020: On sym- bolic and neural commonsense knowledge graphs. In AAAI.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume I (Long and Short Papers), pages 4171-48 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Gautier Izacard, Patrick Lewis, Maria Lomeli, Lu- cas Hosseini, Fabio Petroni, Timo Schick. Jane Dwivedi- Yu, Armand Joulin, Sehastian Riedel, and Edouard Grave. 2022. Few-shot learning with trieval augmented language models. arXiv preprint arXiv:2208.03299

---

Ziwei Ji. Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang. Andrea Madotto, and Pascale Fung. 2023. Survey of halluci- nation in natural language generation. ACM Comput. Surv. 55(12).
Andreas Madsen, Siva Reddy, and Sarath Chandar. 2022. Post-hoc interpretability for neural nlp: A survey. ACM Computing Surveys, 55(8):1-42.
Vaibhav Mavi, Anubhav Jangra, and Adam Jatowt. 2022.
A Survey on Multi-hop Question Answering and Gen- eration. arXiv e-prints, page arXiv:2204.09140)
Philip N Johnson-Laird. 2010. Mental models and human reasoning. Proceedings of the National Academy of Sciences, 187443):18243-1822
R. Thomas McCoy, Paul Smolensky, Tal Linzen, Jian- feng Gao, and Asli Celikyilmaz. 2021. How much do language models copy from their training data? evaluating linguistic novelty in text generation using raven.
Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. 2022. Large language models struggle to learn long-tail knowledge.
Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense Passage Retrieval for Open-Domain Question Answering. arXiv e-prints, page arXiv:2004.04900
Shima Rahimi Moghaddam and Christopher J Honey. 2023. Boosting theory-of mf- performance in large language models via prompting. arXiv preprint 2r34v 11490.
Karim Nader. 2009, Reconsolidation: A possible bridge between cognitive and neuroscientific views of mem- ory. The cognitive neurocciences, pages 691-703
Amirhossein Kazemnejad, Mehdi Rezagholizadeh, Prasanna Parthasarathi, and Sarath Chandar. 2023. Measuring the knowledge acquisiiin--tilizatin gap in pretrained language models. arXiv preprint arXiv:2305.141: 14775.
Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju. William Saunders. Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman. 2022. Webgpt: Browser- assisted question--aswerrig with human feedback.
Najoung Kim, Ellie Pavlick, Burcu Karagol Ayan, and Deepak Ramachandran. 2021. Which lin- guist invented the lightbulb? presupposition ver- ification for question-anse answerin.. arXiv preprint arXiv:2101.00391.
OpenAl. 2023, GPT-4 Technical Report. arXiv e-prints. page arXiv:2303.08774
Andrew K Lampinen, Ishita Dasgupta, Stephanie CY Chan, Kory Matthewson, Michael Henry Tessler, Antonia Creswell, James I. McClelland, Jane X Wang, and Felix Hill. 2022. Can language models learn from explanations in context? arXiv preprint arXiv:2204.02329.
Ian Osband, Seyed Mohammad Asghari, Benjamin Van Roy, Nat McAleese, John Aslanides, and Geoffrey Irving. 2023. Fine-tuning language models via epis- temic neural networks.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Hein- rich Küttler, Mike Lewis. Wen-tau Yih, Tim Rock- täschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-Augmenter Generation for Knowledge- Intensive NLP Tasks. arXiv e-prints, page arXiv:2005.11 1401.
Bhargavi Paranjape, Scott Lundberg, Sameer Singh, Hannaneh Hajishirzi, Luke Zettlemoyer, and Marco Tulio Ribeiro. 2023. Art: Automatic multi- step reasoning and tool-use for large language mod- els.
Shishir G. Patil, Tianjun Zhang, Xin Wang. and Joseph E. Gonzalez. 2023. Gorilla: Large language model connected with massive apis.
Belinda 7. Li, Maxwell Nye, and Jacob Andreas. 2021. Implicit representations of meaning in neural lan- guage models. arXiv preprint arXiv:2106.00737
Matthew E. Peters, Mark Neumann, Luke Zettlemoyer. and Wen-tau tau Yih. 2018. Dissecting contextual word embeddings: Architecture and representation. In Pro- ceedings of the 2018 Conference on Empirical Meth- ods in Natural Language Processing, pages 1499- 1509, Brussels, Belgium. Association for Computa- tional Linguistics.
Andy Liu, Hao Zhu, Emmy Liu, Yonatan Bisk, and Graham Neubig. 2023. Computational language ac- quisition with theory of mind. In The Eleventh Inter- national Conference on Learning Representations.
Jaahhang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. 2021. What makes good in-context examples for gpt-3? arXiv preprint 1xxi:2211100604
Fabio Petroni, Tim Rocktäschel. Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. Language models as knowl- edge bases? In Proceedings of the 2019 Confer- ence on Empirical Methods in Natural Language Pro- cessing and the 9th International Joint Conference Natural Language Processing (EMNLP-IJCNLP), pages 2463-2477 Hong Kong, China, Association for Computational Linguistics.
Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2021. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. arXiv preprint arXiv:2104.08786.

---

Michael Poli, Stefano Massaroli, Eric Nguyen, Daniel Y. Fu, Tri Dao, Stephen Baccus, Yoshua Bengio, Ste- fano Ermon, and Christopher Ré. 2023. Hyena hierar- chy: Towards larger convolutional language models.
Kushal Tirumala, Aram H. Markosyan, Luke Zettle- moyer, and Armen Aghajanyan. 2022. Memorization without overfitting: Analyzing the training dynamics of large language models. In Advances in Neural Information Processing Systems.
David Premack and G. Woodruff. 1978. Does the chim- panzee have a theory of mind? Behavioral and Brain Sciencex, 4(4):515-622)
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet. Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023, Llama: Open and efficient foundation language models.
Colin Raffel, Noam Shazeer, Adam Roberts, Kather- ine Lee. Sharan Narang, Michael Matena. Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1-67.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Al- bert, Amjad Almahairi, Yasmine Babaci, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu. Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedamuj Goswami, Naman Goyal, An- thony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Pumit Singh Koura, Marie-Anne Lachaux. Thibaut Lavril, Jenya Lee, Di- ana Liskovich, Yinghai Lu, Yuning. Mao, Xavier Mar- tinet, Todor Mihaylov, Pushkar Mishra, Igor Moly- bog. Yixin Nie. Andrew Poulton, Jeremy Reizen- stein, Rashi Rungta, Kalyan Saladi, Alan Schel- ten, Ruan Silva, Eric Michael Smith, Ranjan Sub- ramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, An- gela Fan, Melanie Kambadur, Sharan Narang, Aure- lien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023. Llama 2: Open Foundation and Fine- Tine-Tunes Tuned Chat Models. arXiv e-prints, page arXivv:3377. .09288.
Nir Ratner, Yoav Levine, Yonatan Belinkov, Ori Ram, Inbal Magar, Omri Abend, Ehud Karpas, Amnon Shashua, Kevin Leyton--rown and Yoav Shoham. 2023. Parallel context windows for large language models.
Rasmus Rendsvig and John Symons. 2019. Epistemic logic.
Adam Roberts, Colin Raffel, and Noam Shazeer. 2020. How much knowledge can you pack into the param- eters of language model? In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (ENNLP), pages 1418-5426, Online. Association for Computational Linguisties.
Laura Ruis, Akbir Khan, Stella Biderman, Sara Hooker, Tim Rocktäschel, and Edward Grefenstette. 2022. Large language models are not zero-shot communi- cators. arXiv preprint arXiv:2210. 14986.
Maarten Sap, Ronan LeBras, Daniel Fried, and Yejin Choi. 2022. Neural theory-of-mind? on the limits of social intelligence in large Ims. arXiv preprint arXiv:2210.133/2.
Karthik Valmeekam, Alberto Olmo, Sarath Sreedharan, and Subbarao Kambhampati, 2022. Large language models still can't a plan (a benchmark for Ilms on plan- ning and reasoning about change). arXiv preprint arXiv:2206. 1044999.
Timo Schick, Jane Dwivedi- Yu. Roberto Dessi, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools.
Chenguang Wang, Xiao Liu, and Dawn Song. 2020. Language models are open knowledge graphs. arXiv preprint arXiv:2010.11 11967.
Weijia Shi, Sewon Min, Maria Lomeli, Chunting Zhou, Margaret Li, Victoria Lin, Noah A. Smith, Luke Zettlemoyer, Scott Yih, and Mike Lewis. 2023. In-Context Pretraining: Language Modeling Be- yond Document Boundaries. arXiv e-prints, page arXiv:2310.1063:
Yanjing Wang. 2015. A logic of knowing how. Logic, Rationality, and Interaction: 5th International Workshop, LORI 2015, Taipei, Taiwan, October 28- 30, 2015. Proceedings 5, pages 392-405. Springer.
Mujeen Sung, Jinhyuk Lee, Sean Yi, Minji Jeon, Sung- dong Kim, and Jaewoo Kang. 2021. Can language models be biomedical knowledge bases? arXiv preprint arxiv: 100.:22499.
Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V Le. 2022a. Finetuned language models are zero-shot learners. In International Con- ference on Learning Representations.
Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak. R Thomas McCoy, Najoung Kim, Ben- jamin Van Durme, Sam Bowman, Dipanjan Das, and Fillie Pavlick. 2019. What do you learn from con- text? probing for sentence structure in contextualized word representations. In International Conference on Learning Representations.
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph. Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang. Jeff Dean, and William Fedus. 2022b. Emer- gent abilities of large language models. Tranxactions

---

on Machine Learning Research. Survey Certifica- tion.
which it remains constant. No weight decay gradient clipping is applied.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten
OPT: Except for the learning rate, we the same hyperparameters with T5 and Flan-T5. The base learning rates for different OPT model sizes are:
Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoc V Le, and Denny Zhou. 2022c. Chain of thought prompt- ing elicits reasoning in large language models. In Advances in Neural Information Processing Systems.
125M: 10-5
Johannes Welbi, Pontus Stenetorp, and Sebastian Riedel. 2017. Constructing Datasets for Multi-hop Reading Comprehension Across Documents, arXiv e-prints, page arXiv:1710.06481.
350M: 3 10-5
1.3B: 2 x 10-5
Zhiyong Wu, Yaoxiang Wang, Jiacheng Ye, and Ling- peng Kong, 2022. Self-adaptive in-context learning. arXiv preprint arXiv:2212.10375.
2.7B: 1.6 x 10-5
B Per Task Description & Results
Wenhan Xiong, Anchit Gupta, Shubham Toshniwal, Yashar Mehdad. and Wen tau Yih. 2022. Adapt- ing pretrained text-to-text models for long text se- quences.
We provide detailed description of each task in Ta- bles 3-20, along with the per task results in Figures 5-22.
Zhilin Yang, Peng Qi. Saizheng Zhang, Yoshua Ben- gio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering. arXiv e-prints, page arXiv:1809.000 099009000.
C Hallucination Examples
In Table 21 and Table 22, we present examples of hallucinations observed in models trained on segmented stories. Our analysis revealed no signif- icant differences in the patterns of hallucinations across various models. It's also worth noting that models trained on unsegmented stories exhibited similar hallucination patterns, albeit at reduced frequency (as shown in Figure 4).
Zonglin Yang, Li Dong, Xinya Du, Hao Cheng, Erik Cambria, Xiaodong Liu. Jianfeng Gao, and Furu Wei. 2022. Language models inductive reason- ers. arXiv preprint 2rXiv:2212. 10923.
Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Good- man. 2022. Star: Bootstrapping reasoning with rea- soning. Advances in Neural Information Processing Systems, 35:15476-15488
D Recall Length Distribution
We analyzed the length of story recalls relation to the target distribution to determine the impact of training on segmented versus unsegmented sto- ries. Figure 23 displays the distribution of the recall length, measured in number of sentences, for both the model and the target. For brevity, present results only for the largest variant of each model, noting that similar patterns were observed across all model sizes. Our analysis revealed no significant differences between these distributions, leading us to conclude that training on segmented stories does not influence the recall length of the models' out- puts.
Muru Zhang, Ofir Press, William Merrill, Alisa Liu, and Noah A Smith. 2023. How language model hallucinations can snowball. arXiv preprint vrXiv:2305.1334.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher De- wan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mi- haylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. 2022. OPT: Open Pre-trained Transformer Language Models. arXiv e-prints, page arXiv:2205.01065
Shen Zheng, Jie Huang, and Kevin Chen-Chuan Chang. 2023. Why does chatgpt fall short answering ques- tions faithfully? arXiv preprint arXiv:2304. 10513.
A Training Details
T5 & Flan-T5: All models are fine-tuned for 360,000 steps with batch size of 50. We use the Adam optimizer, setting a base learning rate x 10-4, The learning rate undergoes a lincar warmup for the initial 1% of training steps, after

---

Task 1: "List the different.x"
Category: Listing Description: The objective of this task is to identify and list the days on which a person worked from home.
Template
Example
Story:
Story:
[Task 1J [name]'s Work From Home Log [name) worked from home on [day).
[Task I] Tom's Work From Home Log
Tom worked from home on Monday. Tom worked from home on Friday.
[name) worked from home on {day)-
Question:
Question:
[Task IJ Which days did (name) work from home?
[Task IJ Which days did Tom work from home?
Answer:
Answer:
(story)
Tom worked from home on Monday.
The answer is (answer).
Tom worked from home on Friday.
The answer is Monday and Friday.
Details
{day):
Randomly sampled without replacement from ["Monday", "Wednesday". "Friday"].
[answer]: Comprises the days listed.
The story can span one to three sentences, excluding the title. Sentences are ordered chronologically based on [day).
Table 3: Templates for generating Task stories, questions, and answers, with an example provided.
Whole Answer
Train
Test
100
100
80
Accuracy (%)
Hallucination
60
10a
10a
10*
10a
10"
Parameters
Parameters
Parameters
Recall
Final Answer
Accuracy (86)
20
Parameters
Parameters
Flan-T5
OPT
unsegmented
segmented
Figure 5: Task results. Top left: percentage of correct answers. Top right: hallucination rate for both train and test Bottom: percentage of correct recalls (left) and final answers (right).

---

Task 2: "How many times does x happen?"
Category: Counting
Description: The task aims count the number of times the fishing activity occurred within the story.
Template
Example
Story:
Story:
[Task 2] [name]'s Vacation
[Task 2] Tom's Vacation
[name) went {activity) on {day).
Tom went fishing on Monday,
Tom went hiking on Wednesday,
[name) went (activity) on {day].
Tom went fishing on Thursday.
Tom went hiking on Saturday.
Tom went hiking on Sunday.
Question:
Question:
[Task 2] How many times did [name) go fishing?
[Task 2] How many times did Tom go fishing?
Answer:
Answer:
(story)
Tom went fishing on Monday.
The answer is (answer).
Tom went hiking on Wednesday.
Tom went fishing on Thursday.
Tom went hiking on Saturday. Tom went hiking on Sunday.
The answer is 2.
Details
(activity):
Randomly sampled with replacement from ("fishing", "hiking"].
[day]: Randomly sampled without replacement from the seven days of the week.
(answer]: A numeric value representing the count.
The story comprises 3 to 5 sentences, excluding the title. Sentences are ordered chronologically by [day].
Table 4: Templates for generating Task stories, questions, and answers, with an example provided.
Whole Answer
Train
Test
100
100
80
80
Accuracy (%)
60
Hallucination
60
10a
10a
10
10'
10a
20"
Parameters
Parameters
Parameters
Recall
Final Answer
Accuracy (%)
40
20-
10a
10a
Parameters
Parameters
Flan-T5
OPT
unsegmented
segmented
Figure Task 2 results. Top left: percentage of correct answers. Top right: hallucination rate for both train and test Bottom: percentage of correct recalls (left) and final answers (right).

---

Task 3: "Does x happen more/less often than
Category: Ranking
Description: The objective of this task is to determine whether the person has more meetings with Person
A or Person B.
Template
Example
Story:
Story:
[Task 3] (name)'s 's Afternoon
[Task 3] Tom's 's Afternoon
[time) [name) {activiry).
1:00 PM Tom has meeting with co-worker A.
2:00 PM Tom fills up some forms.
[time) [name) (activity).
3:00 PM Tom has meeting with co co-worker B.
4:00 PM Tom fills up some forms.
5:00 PM Toom has meeting with co-worker A.
Question:
Question:
[Task 3] Does {name) have more meetings with
[Task 3] Does Tom have more meetings with
co-worker A or B?
co-worker or B?
Answer:
Answer:
[story)
1:00 PM Tom has meeting with co-worker A.
The answer is (answer).
2:00 PM Tom fills up some forms.
3:00 PM Tom has meeting with co-worker B.
4:00 PM Tom fills up some forms.
5:00 PM Tom has meeting with co-worker A.
The answer is A.
Details
[time): Randomly sampled without replacement from ["1:00 PM", "2:00 PM", "3:00 PM", "4:00 PM",
"5:00 PM"].
{activity): Randomly sampled with replacement from ("has a meeting with co-worker A", "has
meeting with co-worker B", "fills up some forms"].
[answer): Either "A" "B", based on the frequency of the meetings.
The story consists of to 5 sentences, excluding the title. Sentences are chronologically ordered by (time).
Table 5: Templates for generating Task stories, questions, and answers, with example provided.
Test
Halluc Halluciiaton nation Patt (%6
82
82
Accurssy (%u
20-
20-
124
10
12
16
12P
10'
Parameters
Parameters
Parameters
Recall
Final Answer
Accuracy
50
40
10
30a
Parameters
Parameters
Flan-T5
OPT
unsegmented
segmented
Figure 7: Task 3 results. Top left: percentage of correct answers. Top right: hallucination rate for both train and test sets. Bottom: percentage correct recalls (left) and final answers (right).

---

Task 4: "Does x happen before/after y?
Category: Temporal
Description: The task is designed to ascertain whether specific event happened before or after another event. Additionally, reasoning based on the order of months is provided to justify the answer.
Template
Example
Story:
Story: [Task 4] Tom's Year
[Task 4] (name)'s 's Year
[name) [event] in [month).
Tom buys house in March.
Tom gocs vacation in June.
(name) [event] in [month).
Tom gets married in October.
Question:
Question:
[Task 4] Does {name) (event_a a (before/after)
[Task 4] Does Tom buy a house after they get married?
they (event_b))
Answer:
Answer:
(story)
Tom buys house in March.
[month_a] is (reasoning) (month_b)
Tom goes vacation in June.
The answer is (answer).
Tom gets married in October.
March is not after October.
The answer is no.
Details
(evenr):
Randomly sampled without replacement from ["buys a house", "goes on a vacation", "gets married"].
[month]: Randomly sampled without replacement from ("January", "March", "June", "August", "October"].
(event_a) and (event_b) are randomly drawn among the sampled (event).
a and (month_b) are associated with the corresponding (event_a) and (event_b). before/after): Randomly sampled between "before" and "after". reasoning): Explains the temporal relationship between (month_a) and (month_br b). Options include "before", "after", "not before", and "not after". (answer): A simple "yes" "no".
The story consists of sentences, excluding the title. Sentences are chronologically ordered by (month).
Table 6: Templates for generating Task stories, questions, and answers, with example provided.
Whole Answer
Train
Test
Actuncy (86)
62-
Mallucimation
63-
10-
23-
10
Parameters
Patemeters
Parameterss
Recall
Reasoning
Answer
100
Accuracy PN0
80-
63-
T5
Flan-TS
OPT
unsegmented
.UU.. segmented
Figure 8: Task 4 results. Top left: percentage of correct answers. Top right: hallucination rate for both train and test sets. Bottom: percentage correct recalls (left), reasoning (center), and final answers (right).

---

Task 5: "When x happens, does happen?"
Category:
Temporal
Description: This task aims to determine whether, on days when person A is in one specific location,
person B is in another specific location.
Template
Example
Story:
Story:
[Task 5] [name_a _ and [name_b]] __s's Travel Log (name_at was in (location) on [day).
[Task 5] Tom and Alice's Travel Log
Tom in Paris Monday.
Tom was in New York Tuesday.
(name a) was in (location) on [day). (name_ b} was in (location] on [day].
Alice was in Los Angeles Monday.
Alice was in Rome Tuesday.
(name_ b} was in (location) on [day).
Question:
Question:
[Task 5] When (name__d is in (location_al a), is
[Task 5] When Tom is in Paris, is Alice in Rome?
(name_b) b) in (location_b) _
Answer:
Answer:
[story]
Tom was in Paris on Monday.
Those (reasoning) days.
Tom in New York Tuesday.
The answer (answer).
Alice was in Los Angeles on Monday.
Alice was in Rome Tuesday. Those are different days.
The answer is no.
Details
{location) } for person A chosen without replacement from ["Paris", "New York", "Vancouver"). and for person B from ["Los Angeles", "Rome", "Tokyo"]]
[day) is picked without replacement from ["Monday", "Tuesday", "Wednesday"). llocation__ _ and (location_]] b) are randomly drawn from the sampled (location) ) for person A and B respectively.
reeaoningg]: Specifies whether the days of the events in question are "the same" or "different". [answer): I: A simple "yes" "no".
Each person's 's events ordered by (day)-person A's events listed first, followed by person B's events. There can be between and sentences per person.
Table Templates for generating Task stories, questions, and answers, an example provided.
47-
Aconnry (5)
Mullciiation
10-
Parameters Becall
Paramakers
Parameters
Beasaming
Finel Arswer
Flan-Ts
OFT
unsegmented
segmented
Figure 9: Task 5 results. Top left: percentage of correct answers. Top right: hallucination rate for both train and test percentage correct recalls (left), reasoning (center), and final answers (right).
sets. Bottom:

---

Task 6: "Isxx x the only time y happens?"
Category: Uniqueness
Description: Determine whether person engaged in specific activity only once during the week.
Template
Example
Story:
Story:
[Task 6] [name)' Holiday
[Task 6] Tom's 's Holiday
[name) [activity) on {day].
Tom goes hiking on Monday,
Tom goes fishing on Tuesday.
[name) (activity) on {day)-
Tom goes to the park on Wednesday.
Tom plays golf on Thursday.
Tom visits friend on Friday.
Question:
Question:
[Task 6] (name) (activity_a on (day_a). it the only time that week that [name) factivity_ayy aa}?
[Task 6J Tom goes fishing on Tuesday. Is it the only time that week that Tom goes fishing?
Answer:
Answer:
(story)
Tom goes hiking on Monday.
The anwer is {answer).
Tom goes fishing on Tuesday.
Tom goes to the park on Wednesday.
Tom plays golf on Thursday. Tom visits a friend on Friday.
The answer is yes.
Details
(activity) is chosen with replacement from ["goes hiking", "goes fishing", "goes to the park", "plays
golf", "visits a friend"].
(day) is picked without replacement from ["Monday", "Tuesday", "Wednesday", "Thursday", "Fri- day"].
factivity_a _a) is randomly selected (activity), and (day_a) a) is its corresponding day.
[answer] can be "yes" or "no".
The story contains to 5 sentences, excluding the title. Sentences are ordered by [day).
Table Templates for generating Task stories, questions, and answers, with example provided.
Whole Answer
Train
Test
100
100
80-
Accuracy
Hallucination
40-
40
0-
10
10a
Parameters
Parameters
Recall
Final Answer
100
Accuracy (56)
60-
0-
10a
Parameters
Parameters
T5
Flan-T5
OPT
unsegmented
segmented
Figure 10: Task 6 results. Top left: percentage of correct answers. Top right: hallucination rate for both train and test sets. Bottom: percentage of correct recalls (left) and final answers (right).

---

Task "Between x and y, does z happen?"
Category: Temporal
Description: Determine if person performs specific activity between two other distinct activitics during the day.
Template
Example
Story: [Task 7] [name]'s Day
Story:
[Task 7] Tom's Day
[time), {name} {activity).
Morning, Tom goes for a walk. Noon, Tom makes phone call. Afternoon, Tom makes tea.
[time), {name) (activity).
Evening, Tom reads a book.
Question:
Question:
[Task 7] Between (activity_od a) and (activity_b) b), does (name) (activity_c)?
[Task 7] Between going for walk and making tca,
does Tom read a book?
Answer:
Answer:
{story]
Morning, Tom goes for a walk. Noon, Tom makes phone call.
The answer is (answer).
Afternoon, Tom makes tca. Evening, Tom reads book.
The answer is no.
Details
(activity) is chosen without replacement from ["goes for a walk", "makes phone call", "makes tea", "reads book"].
[time) chosen without replacement from the sequential list ["Morning". "Noon", "Afternoon", "Evening"].
(activity_ar a) and (activity_be bb are randomly selected among the sampled (activity). lactiviy__c) is sampled from the list of activities but cannot be the same as {activity_at or (activity_br b). [ is cither "yes" or "no".
The story contains 3 to 4 sentences, excluding the title, with sentences ordered chronologically by (time).
Table 9: Templates for generating Task stories, questions, and answers, with an example provided.
Whole Answer
Train
Test
Accuracy 1963
60-
Hollucination Rote
80-
60
62-
40
16
10
18
Parameters
Parameters
Parameters
Racall
Final Answar
100
40-
20-
10p
10a
12'
Rarameters
Parameters
Flan-T5
OPT
unsegmented
segmented
Figure 11: Task 7 results. Top left: percentage of correct answers. Top right: hallucination rate for both train and test sets. Bottom: percentage of correct recalls (left) and final answers (right).

---

Task 8: "How much time has passed between & and
Category: Temporal Description: Determine the duration hours between two activities person engaged in.
Template
Example
Story:
Story:
[Task 8] [name]' Is Contact Log At [time), {name) (event).
['Task 8] Tom's Contact Log
At 2pm, Tom wrote a letter.
At 4pm, Tom sent an email.
At [time), {name) (event).
At 7pm. Tom made a phone call.
Question:
Question:
[Task 8] How much time passed between (name)
[Task 8] How much time passed between Tom
(event_a) and (event_b]?
wrote a letter and sent an email?
Answer:
Answer:
{story)
At 2pm, Tom wrote a letter.
(reasoning)
At 4pm, Tom sent an email.
The answer is (answer).
At 7pm, Tom made a phone call.
2
The answer is 2.
Details
[time) is selected without replacement from ["Ipm", "2pm", "3pm", "4pm", "Spm"].
(event) is selected without replacement from ["wrote a letter", "sent an email", "made phone call", "started video chat").
(event_a) _annt_aa___ and (event_b) bb} randomly chosen among the sampled (event), with (event_b) always occurring after {event_a]
(reaooning) describes the subtraction of the times corresponding to (event_a) from (event_b), repre- senting the duration in hours.
[answer] indicates the number of hours.
The story contains 3 to 4 sentences, excluding the title, arranged chronologically by (time).
Table 10: Templates for generating Task 8 stories, questions, and answers, with an example provided.
Whole Answer
Train
Test
Accuracy (%)
80-
60
Hollucination
40-
40
20-
20
107
10a
100
10
15
Parameters
Parameters
Parameters
Reasaning
100
Accuracy
60-
40
Parameters
Paramettrs
Paramoters
T5
Flan-T5
OPT
unsegmented
segmented
Figure 12: Task 8 results. Top left: percentage correct answers. Top right: hallucination rate both train and test sets. Bottom: percentage of correct recalls (left), reasoning (center), and final answers (right).

---

Task 9; "At what time does y happen based xnx
Category: Temporal
Description: Determine the time the person asks the bill based on prior events at restaurant.
Template
Example
Story:
Story:
[Task 9] [name]] J at the Restaurant [name] arrived at the restaurant at [time). (minute) minutes after arriving, [name] ordered (item__ [minute] after ordering a [item_] J.. (name) ordered a [item_2 2). [minute] minutes ordering a [item_2), ) (name) asked for the bill.
[Task 9J Tom at the Restaurant
Tom arrived at the restaurant 6:00 PM.
minutes after urriving. Tom ordered drink.
minutes after ordering drink. Tom ordered a hamburger.
minutes after ordering hamburger, Tom asked for the bill.
Question:
Question:
[Task what time does [name) the bill?
[Task what time does Tom ask for the bill?
Answer:
Answer:
[story)
Tom arrived at the restaurant at 6:00 PM.
(reasoning]
2 minutes after arriving, Tom ordered drink.
The answer (answer).
minutes after ordering drink. Tom ordered hamburger.
minutes after ordering hamburger, Tom asked the bill. (reasoving]
The answer is (answer).
Details
(time) is selected between "6:00 PM" and "6:30 PM," rounded to the nearest minute.
[minute] chosen with replacement from ["1", '2", "3"].
[item_/) can either "drink" or "coffee".
[item_2] 2 can be 'hamburger' or 'sandwich".
"Thee'ss 's a 50% chance [name] won't order (item_?). ). If so, the penultimate sentence is omitted, and
the last sentence references (item_/]
(reasoning) provides the total time elapsed from the arrival to the request for the bill. [answer] indicates the exact time.
The story contains sentences, counting title, depending on whether not {item_2 _ was ordered.
Table 11: Templates for generating Task stories, questions, and answers, with an example provided.
Whole Answer
Train
Test
100
Hallucination Rate (%))
80
80-
Accuracy (%%)
60-
60
40-
40
20
20
Parameters
Parameters
Parameters
Recall
Reasoning
Final Answer
100
80
Accuracy (%6
60-
10a
10P
100
102
Parameters
Parameters
Parameters
T5
Flan-T5
OPT
unsegmented
segmented
Figure 13: Task 9 results. Top left: percentage of correct answers. Top right: hallucination rate for both train and test sets. Bottom: percentage of correct recalls (left), reasoning (center), and final answers (right).

---

Task 10: "Thex'th time y happens, what is unique detail about compared to the other times?"
Category: Uniqueness Determine who accompanied the person the x x'th time they engaged in a specific activity.
Description:
Template
Example
Story:
Story:
[Tass 10] [name] Hunting and Canoeing Week [day), (name) went (activity) with (friend).
[Task 10] Tom's Hunting and Canoeing Week Monday, Tom went humting with Alice. Tuesday, Tom went canocing with Bob. Wednesday, Tom went hunting with Carl. Thursday, Tom went canoeing with James. Friday, Tom went canocing with Steve.
[day), (name) went (activity) with (friend).
Question:
Question:
[Task 10] The (xx] time that name)) went ig_activity), who else was there?
[Task 10] The second time that Tom went hunting, who else was there?
Answer:
Answer:
(story)
Monday, Tom went hunting with Alice. Tuesday, Tom went canoeing with Bob. Wednesday, Tom went hunting with Carl. Thursday, Tom went canoeing with James. Friday, Tom went canoeing with Steve. The answer is Carl.
The answer is {answer).
Details
[day] be any day of the week.
lactivity) in each statement can be either "canoeing" or "hiking". However, "hunting" must be picked
at least twice but no more than three times.
(friend) is randomly sampled from list names.
(q activity) can be either "canoeing" "hiking".
[x] is number between and the number of times lg_activityl occurs.
(answer) is the name of the person who was with [name] during the [x]'th 'th occurrence of the
(q_ l._activity).
The story comprises 4 or sentences, not including the title. Sentences are arranged by [day).
Table 12:
Templates for generating Task 10 stories, questions, and answers, with example provided.
Whole Answer
Train
Test
100
100
80-
Accuracy
Hollucinatioon
40-
40
0-
10*
10
16'
10f
Parameters
Parameters
Recall
Final Answer
100
Accuracy (50
60-
0-
10a
10
Parameters
Parameters
T5
Flan-T5
OPT
unsegmented
segmented
Figure 14: Task results. Top left: percentage of correct answers. Top right: hallucination rate for both train and test sets. Bottom: percentage of correct recalls (left) and final answers (right).

---

Task 11: "Every time happens, y always the same?"
Category: Consistency
Description: Determine if, every time a person travels to specific location, they consistently use the same type of vehicle.
Template
Example
Story:
Story:
[Task 11] [name)'s 's Car Choice
[Task 11] Tom's Car Choice
{day), [name) drives (place) in a (vehicle).
Monday, Tom drives the grocery store in minivan. Tuesday; Tom drives the pharmacy in minivan. Wednesday, Tom drives to the grocery store in SUV. Thursday, Tom drives to the grocery store in SUV.
{day), (name) drives to (place) in a (vehicle).
Question:
Question:
(Task 11] Every time (name) drives to (q_place), it always a (q_veiccee)?
[Task II] Every time Tom drives to the grocery
store, is it always in a minivan?
Answer:
Answer:
[story]
Monday, Tom drives to the grocery store in a minivan. Tuesday, Tom drives to the pharmacy in minivan. Wednesday, Tom drives to the grocery store in SUV. Thursday, Tom drives the grocery store in SUV.
The answer is {answer).
The answer is
Details
(day) is selected without replacement from I"Monday", "Tuesday", "Wednesday", "Thhrsday"].
place in each statement can be cither "the pharmacy" or "the grocery store".
place) is randomly chosen from the sampled (place).
vehicle) in each statement can be either "minivan" or "SUV".
q vehicle can either "minivan" or "SUV". (answer) is either "yes" or "no".
The story consists of to 4 sentences, excluding the title, and sentences are ordered by [day).
Table 13:
Templates for generating Task 11 stories, questions, and answers, with an example provided.
Whole Answer
Train
Test
100
1.00
Iallucination Rata (96)
80-
80-
Accuracy (9))
60-
60
40
40
20
Parameters
Parameters
Parameters
Recall
Final Answer
100
80
Accuracy (96)
20*
10*
10a
10*
Parameters
Parameters
T5
Flan-T5
OPT
unsegmented
segmented
Figure 15: Task II results. Top left: percentage of correct answers. Top right: hallucination rate for both train and percentage correct recalls (left) and final answers (right).
test sets. Bottom:

---

Task 12: "After how many yoes happen?"
Category: Temporal
Description: The goal of the task is to determine after how many days person joins person
Template
Example
Story:
Story:
[Task 12] [name]'s 's Company
[Task 12] Tom's Company
[day), [name] is alone.
Monday, Tom is alone.
Tuesday, Tom is alone. Wednesday, Alice arrives. Thursday, Tom with Alice.
[day), [company) arrives.
(day), [name) is with (company).
[day), (name) is with (company).
Question:
Question:
[Task 12] After how many days does [company]
[Task 12] After how many days does Alice join
join {name}?
Tom?
Answer:
Answer:
(story)
Monday, Tom is alone.
The answer is (answer).
Tucsday, Tom is alone. Wednesday, Alice arrives. Thursday, Tom with Alice. The answer is 2.
Details
[day] is sampled without replacement from ["Monday", "Tuesday", "Wednesday", "Thursday"J.
company) is randomly sampled name that is the same between statements.
[answer] is numeral indicating the number of days (name) was alone before being joined by [company].
The story comprises 3 to 4 sentences, excluding the title, and sentences ordered chronologically by (day).
Table 14: Templates for generating Task 12 stories, questions, and answers, with example provided.
Whole Answer
Train
Test
100
100
80-
Accuracy
Hallucination
40-
40
104
15
16
Porameters
Recall
Final Answer
100
Accuracy (56)
60
Parameters
Parameters
T5
Flan-T5
OPT
unsegmented
segmented
Figure 16: Task 12 results. Top left: percentage of correct answers. Top right: hallucination rate for both train and test sets. Bottom: percentage of correct recalls (left) and final answers (right).

---

Task 13: "Isx y'th in the list?"
Category: Listing
Description: The goal of the task is to determine if person B is the x'th person that person A meets.
Template
Example
Story:
Story:
[Task 13] [name)'s Friends
[Task 13] Tom's Friends
[name) meets (friend) the morning.
Tom meets Eve in the morning.
[name) meets (friend) at noon.
Tom meets Alice at noon.
[name) meets (friend) the afternoon.
Tom meets Bob the afternoon.
[name) meets (friend) in the evening.
Question:
Question:
[Task 13] (q_friend) the [x] person that [name]
[Task 13] Bob the second person that Tom
meets?
moets?
Answer:
Answer:
(story)
Tom meets Eve in the morning.
(q_friend) the (reaonnng)).
Tom meets Alice at noon.
The answer (answer).
Tom meets Bob in the afternoon.
Bob the third.
The answer is no.
Details
[friend] is randomly sampled name.
friend) is randomly selected from one of the friends that [name) meets.
(x) is randomly sampled from ["first". "second", "third", "Fourth"]. reasoning) indicates the actual position of (q_friend) with respect to ["first", "second", "third", "[ourth"].
[answer] is either "yes" or "no".
The story have between 3 and 4 sentences, excluding the title. If there are only sentences, the last one "(name) meets (friend) in the evening." is omitted.
Table 15:
Templates for generating Task 13 stories, questions, and answers, with example provided.
Whole Answer
Train
Test
100
100
(96)
80-
Accuracy
40-
40-
109
16'
Perameters
Recall
Reasoning
Final Answer
100
60-
40-
10'
Parameters
Parameters
Parameters
T5
Flan-T5
OPT
unsegmented
segmented
Figure 17: Task results. Top left: percentage of correct answers. Top right: hallucination rate for both train and test sets. Bottom: percentage of correct recalls (left), reasoning (center), and final answers (right).

---

Task 14: "Among the list of x, there y
Category: Listing
Description: The goal is to determine if a person atc a given fruit or not, among the list of fruits they atc.
Template
Example
Story:
Story:
[Task 14] (name)'s Snacks (name) ate (fruit) at (time).
[Task 14] Tom's Snacks
Tom ate an apple at 8am.
Tom ate a pear at 10am.
(name) ate (fruit) at (time).
Tom ate an orange at 2pm.
Question:
Question:
[Task 14J Among the snacks that (name) ) ate, is there (q_fruit)?
[Task 14] Among the snacks that Tom ate, is there a banana?
Answer:
Answer:
{story)
Tom atc an apple at 8am.
The answer is (answer).
Tom atc pear at 10am.
Tom atc an orange at 2pm.
The answer is no.
Details
[time) is sampled without replacement from ["8am", "10am", "12pm", "2pm").
[fruit] is randomly sampled from ["an applc", "a pear". "an orange". "a banana", "a cherry"] for each
statement,
{q fruir) is randomly sampled from ["an applc", "a "car", "an orange", "a banana", "a cherry"].
[ is "yes" or "no".
The story can have between 2 and 4 sentences, excluding the title. Sentences are ordered by {time).
Table 16: Templates for generating Task stories, questions, and answers, with an example provided.
Whole Answer
Train
Test
100
100
Accuracy (%)
Hallucination Rate
80
20
10a
10a
10a
10'
10a
10P
Parameters
Parameters
Parameters
Recall
Final Answer
Accuracy (96)
60
40
20-
10a
Parameters
Parameters
Flan-T5
OPT
unsegmented
segmented
Figure 18: Task results. Top left: percentage of correct answers. Top right: hallucination rate for both train and test sets. Bottom: percentage of correct recalls (left) and final answers (right).

---

Task 15: "Among the list of x, is there only y?
Category: Uniqueness
Description: The goal of the task to determine the person only received a given grade in either their science or language courses.
Template
Example
Story:
Story:
[Task 15] (name)'s Grades
[Task 15] Tom's Grades
(name) got {grade) in [language_course).
Tom got an A in English. Tom got an A in Spanish.
(name) got {grade) in (anguage__ourre (name) got {grade} in (science_course
Tom got an B in Biology.
Tom got an A in Physics.
(name) ) got {grade} in (science_course
Question:
Question:
[Task 15] Did (name) only get (q_grade) in
[Tass 15] Did Tom only get A in science courses?
(course_type _ype courses?
Answer:
Answer:
[story]
Tom got an A in English.
The answer is (answer).
Tom got an A in Spanish.
Tom got an B in Biology.
Tom got an A in Physics.
The answer is no.
Details
Songuage_courge is randomly sampled without replacement from ["English", "Spanish", "French"]. |science_come _otee___orreee is randomly sampled without replacement from ["Biology", "Physics", "Chemistry"]. (grade) is randomly chosen to be "A" or "B" for each statement.
grade) is randomly chosen to be "A" or "B" for each statement.
[corse_type] is randomly chosen to be "science" or "language". [ is either "yes" or "no".
The story can contain 2 to 3 sentences about language courses and 2 to 3 sentences about science courses.
Table 17: Templates for generating Task 15 stories, questions, and answers, with an example provided.
Answer
Train
130
100
form (44)
Accuracy
104
10
10
Parameters
Parameters
Parameters
Recall
Rinal
Accunscy (8)
20-
Porameters
Parameters
Flan
unsogmented
segmented
Figure 19; Task results. Top left: percentage of correct answers. Top right: hallucination rate for both train and test sets. Bottom: percentage of correct recalls (left) and final answers (right).

---

Task 16: "Isx the same as y
Category: Ranking
Description: The goal is to determine if the person went as many times to the beach as the cinema.
Template
Example
Story:
Story:
[Task 16] {name)'s s Activities
[Task 16] Tom's Activitics
[day], [name] went to {place)
Monday, Tom went to the beach.
Tuesday, Tom went to the beach.
[day], [name] went to {place)
cinema. Thursday, Tom went to the park. Friday, Tom went to the cinema.
Wednesday, Tom went
Question:
Question:
[Task 16] Did [name) go to the beach as many days as to the cinema?
[Task 16] Did Tom go to the beach as many days
as to the cinema?
Answer:
Answer:
(story)
Monday, Tom went to the beach. Tuesday, Tom went to the beach. Wednesday, Tom went to the cinema. Thursday, Tom went to the park. Friday, Tom went to the cinema.
The answer is (answer).
The answer is yes.
Details
[day] can be any day of the week.
{place) is randomly sampled with replacement from ("einema", "park", "bcach"], but "cinema" and "beach" must each be sampled at least once and no more than three times.
[ is either "yes" or "no".
The story can contain 4 to 5 sentences, excluding the title. Sentences ordered by (day).
Table 18:
Templates for generating Task 16 stories, questions, and answers, with an example provided.
Whole Answer
Train
Test
100
100
80
Accuracy (%)
Hallucination
60
10a
10a
10
10'
10a
10
Parameters
Parameters
Parameters
Recall
Final Answer
Accuracy (%)
40
20
10a
10
Parameters
Parameters
Flan-T5
OPT
unsegmented
segmented
Figure 20: Task results. Top left: percentage correct answers. Top right: hallucination rate for both train and test sets. Bottom: percentage of correct recalls (left) and final answers (right).

---

Task 17: "What is the state of x when happens?"
Category: Temporal
The objective of this task is to identify what the individual was wearing at the moment the storm began.
Description:
Template
Example
Story:
Story:
[Task 17] {name)'s Outfits
[Task 17] Tom's Outfits
[time), {name} } is wearing (clothing).
8am, Tom is wearing pyjama.
10am, Tom is wearing workout clothes.
[time), the storm starts.
12pm, Tom is wearing a bathrobe.
2pm, the storm starts.
[time), {name) is wearing {clothing).
4pm, Tom is wearing raincoat.
Question:
Question:
[Task 17] What was (name) wearing when the
[Task 17] What Tom wearing when the storm
storm started?
started?
Answer:
Answer:
(story)
8am, Tom is wearing a pyjama.
The answer is (answer).
10am, Tom is wearing workout clothes.
12pm, Tom is wearing bathrobe.
2pm, the storm starts.
4pm, Tom is wearing raincoat.
The answer is bathrobe.
Details
[time) is randomly sampled without replacement from ["8am", "9am". "10am", "Ilam", "12pm", "Ipm". "2pm", "3pm". "4pm". "5pm"].
(clothing] is randomly sampled without replacement from ("a pyjama", "workout clothes", "a bathrobe", "a raincoat"].
The statement "(time), the storm starts." is randomly positioned within the story but can appear anywhere from the first to the penultimate sentence.
[answer] the (clothing) mentioned in the sentence immediately preceding "[time), the storm starts".
The story will consist of 4 to sentences, excluding the tille, and sentences are sequenced according to [time).
Table 19: Templates for generating Task 17 stories, questions, and answers, with an example provided.
Answer
100
Accunsty (8)
Faata
62-
23-
Parameters
Parmmeters
Parameters
Recall
Final Answer
02-
Aeturaty 150
60-
60-
20-
107
Parameeters
Parameters
Plan-75
OPT
unsegmented
segmented
Figure 21: Task 17 results. Top left: percentage of correct answers. Top right: hallucination rate for both train and test sets. Bottom: percentage of correct recalls (left) and final answers (right).

---

Task 18: xIfx had/hadn' 't happened, would y have happened?"
Category:
Causal
Description: This task requires determining whether, specific day, a person would have had a certain amount of money had they not sold particular item.
Template
Example
Story:
Story:
[Task 18] (name) Money
[Task 18] Tom's Money
(day), (name) sold {item) for {price]S.
Monday, Tom sold pencil for 2$.
Tuesday, Tom sold an craser for 1$. Wednesday, Tom sold marker for 3$. Thursday, Tom sold staple for 1$.
{day), (name) sold {item) for {price]S.
Question:
Question:
[Task 18] If (name) hadn't 't sold (q_ [t_ttmm,, would they have (q_money)s on {q_day]?
[Task 18] If Tom hadn't 't sold staple, would they have 6$ on Wednesday?
Answer:
Answer:
[story]
Monday, Tom sold a pencil for 2$.
(reasoning)
Tuesday, Tom sold eraser for 1$. Wednesday, Tom sold marker for 3$. Thursday, Tom sold staple for 1$.
The answer is (answer).
2 + 6.
The answer is yes.
Details
{day) is randomly sampled without replacement from ["Monday", "Tuesday". "Wednesday". "Thurs-
day"].
(item) is randomly sampled without replacement from ["a pencil", "an eraser", "a marker", "a staple"].
(price) is randomly sampled integer value between and 3.
{q_item] is randomly sampled from ["a pencil", "an eraser", "a marker", "a staple"].
q q_money) is some integer value between 3 and 8.
{ is randomly selected among the sampled [day].
reasoning is the summation of the (price) to (and including) (g_day), but excludes the (price)
corresponding to {q_item).
[answer] is either "yes" or "no".
The story consists to 4 sentences, not counting the title, and sentences are listed chronologically by (day).
Table 20: Templates for generating Task stories, questions, and answers, with an example provided.
Whole Answer
Trin
Tarumittrrs
Taramitann
Phramatera
Recall
Reasening
Final Answer
80-
Penumatarr
Paramearrs
Parumizirs
Flan-T5
OFT
unsogmented
segmented
Figure 22: Task 18 results. Top left: percentage of correct answers. Top right: hallucination rate for both train and test sets. Bottom: percentage of correct recalls (left), reasoning (center), and final answers (right).

---

Question
Target Answer
Model Answer
Hailey worked from home on Wednesday. Hailey worked from home on Friday. The answer is Wednesday and Friday.
[TTask IJ Which days did Hailey work from home?
Hailey worked from home Wednesday. The answer is Wednesday.
Kozma went fishing on Monday. Kozma went hiking on Tuesday. Kozma went fishing on Wednesday. Kozma went fishing on Friday. Kozma went hiking on Saturday. The answer is 3.
Kozma went fishing on Monday. Kozma went hiking on Tuesday. Kozma went fishing on Wednesday. Kozma went hiking on Thursday. Kozma went fishing on Friday. The answer is 2.
(TTss 2] How many times did Kozma go fishing?
1:00 PM - Artemia fills up some forms. 2:00 PM Artemia has meeting with co-worker A. 3:00 PM- Artemia has a meeting with co- co-worker A. 4:00 PM Artemia has meeting with co-worker B. 5:00 PM- Artemia fills up some forms. The answer is A.
1:00 PM - Artemia fills up some forms. 2:00 PM Artemia has meeting with co-worker A. 3:00 PM Artemia fills up some forms. 4:00 PM Artemia has a meeting with co- worker B. The answer is neither.
(Tass 3] Does Artemia
have more meetings with co-worker A or B?
Wahoo goes on vacation in January. Wahoo buys house in March. Wahoo gets married in October. March is after January. The answer is yes.
('Task 4] Does Wahoo buy a house after they go vacation?
Wahoo goes on vacation in January. Wahoo buys house in March. March is after January. The answer is yes.
Mayr was in New York on Monday. Mayr was in Vancouver on Tuesday,
Mayr was in Vancouver on Monday. Mayr was New York on Tuesday. Mayr was in Paris Wednesday. Grant was in Rome on Monday. Grant was in Los Angeles on Tuesday, Those are the same days. The answer yes.
('Task 5] When Mayr is in Vancouver, is Grant in Rome?
Mayr was in Paris on Wednesday. Grant was in Rome Monday. Grant was in Tokyo on Tuesday. Grant was in Los Angeles on Wednesday. Those are different days. The answer is no.
Adlar goes to the park on Monday. Adlar visits friend on Tuesday. Adlar plays golf on Wednesday. Adlar goes to the park on Thursday. Adlar visits a friend on Friday. The answer is no.
Adlar goes to the park on Monday. Adlar visits friend on Tuesday. Adlar visits friend on Wednesday. Adlar plays golf on Thursday, Adlar goes to the park on Friday. The answer is no.
('Task 6] Adlar visits friend on Friday. Is it the only time that Adlar visits a friend?
Morning, Braili goes for a walk. Noon, Braili makes a phone call. Afternoon, Braili makes tea. Evening, Braili reads a book. The answer is yes.
[Taask 7] Between going for walk and making tea, does Braili read a book?
Morning, Braili goes for walk. Afternoon, Braili reads book. Evening, Braili makes tea. The answer is yes.
At Ipm, Damiyon sent an email. At 3pm, Damiyon started a video chat, At 4pm, Damiyon wrote letter. At 5pm, Damiyon made phone call. 5 = 4. The answer is 4.
At I Ipm, Damiyon sent an email. At 2pm, Damiyon made a phone call. At 4pm, Damiyon wrote letter. At 5pm, Damiyon started video chat. 2 I
(TTask 8] How much time passes between Damiyon made phone call and sent an email?
1. The answer is 1.
Tia arrived the restaurant 6:08 PM. 3 minutes after arriving, Tia ordered drink. 2 minutes after ordering a drink, Tia ordered a sandwich. 3 minutes after ordering sandwich, Tia asked for the bill. 3 + 2 3 = 8. The answer is 6:16 PM.
Tia arrived at the restaurant at 6:08 PM. 2 minutes after arriving. Tia ordered drink. 3 minutes after ordering a drink, Tia asked for the bill. 2 + 3+ 5, The answer is 6:13 PM.
[Task 9] At what time does Tia ask for the bill?
Table 21: Hallucination examples from models trained on segmented stories for Tasks I to 9. The Target Answer is provided for comparison, with ballucinations highlighted bold, For hallucination examples corresponding to Tasks 10 18, see Table 22.

---

Question
Target Answer
Model Answer
Monday, Demontre went canoeing with Emelita. Tuesday, Demontre went hunting with Taffa. Thursday, Demontre went canoeing with Maibelle. Friday, Demontre went hunting with Ebere. Sunday, Demontre went hunting with Amyty. The answer is Maibelle.
Tuesday, Demontre went canocing with Maibelle. Thursday, Demontre went canoeing with Xian. Friday, Demontre went hunting with Neria. Sunday, Demontre went hunting with Sidoney. The answer is Xian.
(Task 10] The second time that Demontre went canoeing, who else was there?
Monday, Lamya drives to pharmacy in SUV. Tuesday, Lamya drives pharmacy in a minivan. Wednesday, Lamya drives to grocery store a SUV. Thursday, Lamya drives to pharmacy in SUV. The answer is
Tuesday, Lamya drives to pharmacy in minivan. Wednesday, Lamya drives to pharmacy in a SUV. Thursday, Lamya drives to pharmacy a minivan. The answer is
(TTask 11] Every time Lamya drives to pharmacy, it always minivan?
Monday, Osmar alone. Tuesday, Osmar is alone. Wednesday, Osmar is alone. Thursday, Rooster arrives. The answer is
[TTas 12] After how many days does Rooster join Osmar?
Monday, Osmar is alone. Tuesday, Osmar is alone. Wednesday, Osmar is alone. The answer is never.
Marayna meets Person B in the morning. Marayna meets Person noon. Marayna meets Person D in the afternoon. Person D is the third. The answer is yes.
Marayna meets Person B in the morning. Marayna meets Person at noon. Marayna meets Person C in the afternoon. Person D is the first. The answer is no.
[Task 133 Is Person the third person Marayna meets?
[Task 14] Among the snacks that Kornelis ate, is there an orange?
Kornelis ate banana at 8am. Kornelis ate an apple at 10am. Kornelis ate a cherry at 12pm. The answer no.
Kornelis ate banana at 8am. Kornelis ate an apple at 2pm. The answer is no.
Corrine got an A in English. Corrine got Spanish. Corrine got A in French. Corrine got A in Biology. Corrine got an A in Physics. Corrine got an A in Chemistry, The answer is no.
Corrine got an A in English. Corrine got Spanish. Corrine got in French. Corrine got A in Biology. Corrine got a B in Physics. The answer no.
[TTask 15] Did Corrine only get B in language courses?
Monday, Trella went to the beach. Tuesday, Trella went to the cinema. Wednesday. Trella went to the beach. Thursday, Trella went to the park. Friday, Trella went to the cinema. The answer yes.
Monday, Trella went to the beach. Tuesday, Trella went to the cinema. Wednesday, Trella went to the park. Thursday, Trella went to the park. Friday, Trella went to the park. The answer no.
[TTask 16] Did Trella go the beach as many days to the cinema?
9am, Kyra is wearing a raincoat. 10am, Kyra is wearing pyjama. 12pm, the storm starts. 2pm, Kyra is wearing workout clothes. The answer is pyjama.
9am, Kyra is wearing a raincoat. 12pm, the storm starts. 2pm, Kyra is wearing workout clothes. 3pm, Kyra is wearing a bathrobe. The answer a raincoat.
[Tassk 17] What was Kyra wearing when the storm started?
[TTask 18] If Rheanna hadn't sold staple, would they have 8S on Wednesday?
Monday, Rheanna sold an eraser for 3$. Tuesday, Rheanna sold marker for 3$. Wednesday, Rheanna sold staple for 2S. 3 + = The answer is no.
Monday, Rheanna sold an eraser for 3$. Tuesday, Rheanna sold marker for 2$. Wednesday, Rheanna sold staple for 1S. 3 + = 5. The answer is no.
Table 22: Hallucination examples from models trained on segmented stories for Tasks 18. The Target Answer is provided for comparison, with hallucinations highlighted in bold, For hallucination examples corresponding to Tasks to 9, see Table 21.

---

400 TS-XL
Flan-TS-XL
OPT-2.7B
Unsegmented Segmented
Count
200
100
0
Number Sentences
Number of Sentences
Number Sentences
Figure 23: Comparison of the number sentences in the recall part answers from three models: T5-XL (left), Flan-T5-XI. (center), and OPT-2.7B (right). This compares the target distribution with models trained unsegmented and segmented stories. Similar patterns were observed for other model sizes. There no significant difference between these distributions, suggesting that training on segmented stories does not affect recall length.


Reviews:
[{"review_id": "oHg9T33NJO", "reviewer": "Reviewer_hoTF", "summary": "", "questions": "How are the prompts for the segmented narratives constructed? In which sense do they differ from the non-segmented narratives?\nWhat does Figure 3 mean? What is measured on X-axis in the graphs? Is is the size of the model (number of parameters)? \nWhat do the authors mean by partial observability and full observability?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to extend our sincere appreciation to the reviewers for their thorough and invaluable feedback. It brings us great joy to note that our work has captured the interest of the reviewers (R1 and R2), been recognized for its novelty (R1 and R3), and acknowledged as insightful (R1 and R2). Furthermore, we are delighted to learn that the matter of addressing knowledge consolidation in LLMs has been recognized as \"an important topic that needs to be address\" (R2). We have meticulously examined all the concerns raised by the reviewers and have diligently endeavored to address these issues, as detailed in the discussion below.\n\n**Reviewer’s Comment: “...CoT reasoning has been observed to be very susceptible to the examples presented [1,2] and the order of presentation of examples [3], but the failure to control for these and other important characteristics that could influence the performance of the models, yet they draw conclusions regarding the behaviour of such models.”**\n\n**Response:** There seems to be a misunderstanding here, as we are not using few-shot examples or the chain-of-thought reasoning.\n\n**Reviewer’s Comment: “The authors also do not clearly discuss how and why each task was chosen and the methodological aspects of creation/selection and validation of examples in the corpus, and whether they are good proxies for information consolidation and epistemic reasoning.”**\n\n**Response:** When selecting the tasks, we tried to cover as many different types of relationship between facts and events as possible. By reading stories and writing down questions, we came up with a comprehensive list of 18 tasks. These general questions can then be rephrased in many different ways. Finally, we grouped them in categories, for example tasks about temporal relationships. These categories aren't perfect though, as tasks from different categories sometimes share similar aspects, such as counting. Thus the categories do contain some subjective aspects, but as for the tasks themselves, we believe they cover a broad range of questions about dependencies between facts and events in a story. Note that this choice is secondary to the objective of measuring the knowledge consolidation capabilities of these models, where the focus is on the consolidation and recall capabilities. We have added this explanation to the paper.\n\n**Reviewer’s Comment: “The work limits itself to the English language, but in no moment the authors consider how (syntactic, semantic, pragmatic, etc) aspects of the language can impact the performance of the models - especially in a task for language-based reasoning.”**\n\n**Response:** We agree that the nature of the language (syntax, semantics etc) could play a role in the performance of the model, as the performance of a machine learning system does depend on the properties of the dataset it is trained on. However, in this case, the effect (if any) should be quite limited as we are finetuning the models on datasets from the same language. In that sense, the choice of the language is secondary.\n\n**Reviewer’s Comment: “... seem to fail to control aspects that may explain the performance of the models in their experiments.”**\n\n**Response:** We trained the models on our dataset in the same way as described in the papers they were introduced in, i.e. the models see the same stories multiple times during training in random order. We decided to replicate the pretraining methodology as much as possible, since the goal is to make some parallels about how LMs consolidate knowledge from various documents during training. This is so we do not introduce a distributional shift, which could become a confounder. At test time, there are no few-shot examples and no chain-of-thought reasoning is applied. This is a reasonable choice as the main objective is to study if these models are capable of piecing back the story in the proper order and not to evaluate the \"difficulty\" of the reasoning aspect of the benchmark..\n\n**Reviewer’s Comment: \"I find the authors' claims are not properly justified in the manuscript.\" and \"The authors seem to draw much deeper conclusions than their experiments allow…\"**\n\n**Response:** We are confused as to which claims the reviewer is talking about, as our paper’s main claim is that LM’s struggle to consolidate their knowledge. Our results back up this claim and the reviewer themselves agree with us that we provide insight into the knowledge consolidation capabilities of LMs: “To my knowledge, information consolidation capabilities in LLM have not been independently measured in any work, and this gives insights into the limitations of such models in LM as KB and reasoning settings in knowledge-rich scenarios.” Additionally, no other reviewer has mentioned any issues with unjustified claims. Could the reviewer elaborate on which claims exactly they felt were not well justified?\n\n**Reviewer’s Comment: “How are the prompts for the segmented narratives constructed? In which sense do they differ from the non-segmented narratives?”**\n\n**Response:** There are two types of prompts: stories to memorize and question-answers. The format of stories is as follows: For unsegmented stories:\n> <A title/header>\n>\n> <The story>\n\nSo for example you could have:\n> [Task 1] Tom's Afternoon\n>\n> 1 PM: Tom reads a book.\n> \n> 3 PM: Tom goes to the store.\n>\n> 5 PM: Tom is at the restaurant.\n\nThe segmented stories follow the same format, so for this example, we would have the three following text sequences instead of only one:\n> [Task 1] Tom's Afternoon, Part 1/3\n>\n> 1 PM: Tom reads a book.\n\n> [Task 1] Tom's Afternoon, Part 2/3\n>\n> 3 PM: Tom goes to the store.\n\n> [Task 1] Tom's Afternoon, Part 3/3\n>\n> 5 PM: Tom is at the restaurant.\n\nAs for the format of question-answer prompts, they follow this general format:\n> <Question>\n>\n> <Recall the story> <Reasoning part for certain tasks> <Final answer>\n\nFor example:\n> [Task 1] Did Tom go to the library during his afternoon?\n>\n> 1 PM: Tom reads a book. 3 PM: Tom goes to the store. 5 PM: Tom is at the restaurant. The answer is no.\n\nAs we explain in the paper, one version of the model is trained on the unsegmented version of the stories while the other version is trained on the segmented version. Both versions are also trained on question-answer examples, which refer to stories not included in the training set. Their purpose is to help the models learn the appropriate answer format.\n\n**Reviewer’s Comment: “What does Figure 3 mean? What is measured on X-axis in the graphs? Is it the size of the model (number of parameters)?”**\n\n**Response:** Figure 3 simply breaks down the accuracy of the models per category of tasks. In Figure 2, the shown accuracy is the global accuracy. In Figure 3 we simply break it down per category. We also only show the best and worst performance of models trained on unsegmented stories to simplify the plots. The x-axis is the number of parameters of these models, so it captures the size of the models.\n\n**Reviewer’s Comment: “What do the authors mean by partial observability and full observability?”**\n\n**Response:** The inspiration to use these terms comes from world modeling and reinforcement learning. In our work, they are equivalent terms for the segmented setup (partial observability) and the unsegmented setup (full observability), meaning that in one case models are trained on segmented narratives and in the other, models are trained on unsegmented narratives. To avoid any confusion in the paper though, we will replace this terminology with the unsegmented/segmented setup.\n\n**Reviewer’s Comment: “I have not been able to identify exactly the results achieved by the authors can be understood as probing of epistemic capabilities of language models in its interpretation of LLM as epistemic models.”**\n\n**Response:** There seems to be a misunderstanding here. When we claim that we want to study language models as epistemic models, we mean studying their knowledge consolidation capabilities, not the formal definition of epistemology. We have updated section 2 in order to clarify this, see below.\n\n## 2 Epistemology & Language Models\nEpistemic frameworks (Wang, 2015; Rendsvig and Symons, 2019) are formal systems used to represent knowledge, belief and the uncertainty that entails what a reasoning system knows and/or believes. This is enabled through organizing the knowledge observed by the system. The rules to combine the knowledge in the abstract framework governs combining a new information to the current set of information, or when to ignore the new information, and using the current beliefs to anticipate related events. While LMs behave as KBs to store known relations, epistemic logic provides us with the inspiration to describe how these models organize and update their knowledge.\n\nConsider the example from Figure 1, where we have the knowledge x_1: “Tom ate an apple.”, x_2: “Tom ate a pear.” and x_3: “Bob ate an orange.”. Prompted with the question “What did Tom eat?”, the model must recall knowledge from within its parameter space. It has to connect x_1 and x_2 while also ignoring x_3. To answer the query, a system is expected to consolidate the information and retain a knowledge state over the information it had seen until then. However, an inability to draw the connections would leave the facts disconnected. We describe the model that struggles to consolidate as Type I, and one that is better at it and infer over a consolidated knowledge state as Type II.\n\nAs LMs are used in many real-world scenarios that are frequently presented with a periodic flow of information, it is necessary for the LMs to use such information appropriately during inference. While techniques like self-prompting and generation over self-retrieval are gaining popularity, the performance relies on the quality of the prompt, which adds to the robustness concerns on the performance of LMs on varying reasoning tasks. Inspired by epistemology, we design Epik-Evalto to diagnose whether LMs comply with a first-order knowledge state following a sequence of facts which holds a consolidated summary of information during inference.\n\n**Reviewer’s Comment: “I have not been able to understand exactly how the in-context training with the segmented narrative texts works, as it was not particularly clear in the manuscript.”**\n\n**Response:** We have rewritten section 3 in order to clarify the description of the dataset, the details on its generation, and how the models are evaluated in detail. See the updated methodology section 3 that follows.\n\n## 3 EpiK-Eval\nThe EpiK-Eval benchmark presents a suite of novel, narrative-based diagnostic tasks, meticulously designed to evaluate a LM's capacity to construct a comprehensive, unified knowledge state.\n\n**Dataset:** Our benchmark comprises 18 tasks, which are questions about relations between facts and events in stories, for example: \"Does x happen before/after y?\". See Table 2 for the full list of tasks. For each of these, we generate 100 stories following a per task template. For example, Task 2 has the following template:\n> [Task 2] {name}'s Vacation\n> \n> {name} went {activity} on {day}.\n>\n> .\n> \n> .\n>\n> .\n\nwhere the first line is the story's title, the name is randomly sampled such that it is unique to each story and the activity and day in a sentence are randomly sampled from the list ['fishing', 'hiking'] and ['Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday', 'Sunday'] respectively. The story can have a random number of sentences, the range is pre-determined per task, for example, Task 2 stories can have between 3 and 5 sentences. Here is an example of a generated story for Task 2:\n> [Task 2] Tom's Vacation\n> \n> Tom went fishing on Monday.\n>\n> Tom went hiking on Wednesday.\n>\n> Tom went fishing on Saturday.\n\nThus, with a 100 stories generated for each 18 tasks, we get 1800 stories, which we refer to as our dataset of unsegmented stories D_U = {x_1, x_2, ..., x_1800}. After having generated these stories, we also generate a second dataset which consists of the segmented version of these stories. For each given story, we segment it into its individual sentences and add a part number to the title. For example, given the previous story about Tom, we would get the following three text sequences:\n> [Task 2] Tom's Vacation, Part 1/3\n> \n> Tom went fishing on Monday.\n\n> [Task 2] Tom's Vacation, Part 2/3\n>\n> Tom went hiking on Wednesday.\n\n> [Task 2] Tom's Vacation, Part 3/3\n>\n> Tom went fishing on Saturday.\n\nWe do this for all 1800 stories and get 6800 story segments, which form our dataset of story segments D_S = {s_1, s_2, ..., s_6800}.\n\nFor each story, we also generate one question-answer pair. The questions are rephrasings of the task. For example, for Task 2 \"How many times does x happen?\", we have \"How many times did {name} go fishing?\". The question-answer pairs are also generated following a template. The template always consists of a question followed by the answer which itself has three parts: recall of the entire story, an optional reasoning part depending on the task and the final answer. For example, the template for question-answers in Task 2 is the following:\n> [Task 2] How many times did {name} go fishing?\n> \n> {story} The answer is {count}.\n\nand here is an example of a generated question-answer pair:\n> [Task 2] How many times did Tom go fishing?\n>\n> Tom went fishing on Monday. Tom went hiking on Wednesday. Tom went fishing on Saturday. The answer is 2.\n\nWe also include in the Appendix a full list of templates as well as the values each placeholder variable (e.g., activities, days) can take, in order to generate stories and question-answer pairs for each task. Having generated one question per story, we have a total of 1800 question-answer pairs, which we randomly split into two sets: the validation and the test set. In order for the models to learn the answer format that they are expected to respect while answering the questions, we add question-answer examples to the training set. We thus generate an additional 1800 stories and question-answer pairs. We discard the stories and add these 1800 question-answer pairs to the training set, such that there are no overlaps between questions in the training, validation and test set.\n\n**Evaluation Process:** We evaluate pretrained LMs for their ability to consolidate knowledge. To do so, given a pretrained language model, we make two copies of it: M_U and M_S. We fine-tune M_U on the unsegmented stories and M_S on the segmented stories. The prior setting allows for all the information necessary to answer a given question to be found in a single text sequence without requiring the model to learn dependencies across multiple text sequences, while the latter setting requires consolidating the information from the narrative segments. Having both allows to measure the effect of information being spread across separate text sequences and the LMs' ability to consolidate this knowledge during inference, by measuring the gap in performance between both models.\n\nBoth models are fine-tuned on their respective dataset, D_U and D_S, as well as the training set of question-answer examples. Thus, one epoch for M_U consists of 3600 samples (1800 stories + 1800 q/a examples) and one epoch for M_S of 8600 samples (6800 segments + 1800 q/a examples). Samples are shuffled such that a batch may contain a mix of stories and question-answer examples in the case of M_U or story segments and question-answer examples in the case of M_S. Models are fine-tuned with their respective pretraining objective. Specifically, in the case of encoder-decoder style models, the story's title (first line in the text sequence) is fed to the encoder and the decoder is expected to output the rest of the story in the case of M_U or the story segment in the case of M_S. As for question-answer pairs, the question is fed to the encoder and the model is expected to output the answer. For causal language models, they are simply expected to predict the next token in the given sequence, as is standard procedure. Precisely, for M_U, a text sequence is either an entire story or a question concatenated with its answer, while for M_S, a text sequence is either a story segment or a question concatenated with its answer.\n\nDuring fine-tuning, both models are also periodically evaluated on the validation set. Models are run in inference mode as described in the papers they were introduced in. We prompt the models with the questions from the validation set and their answers are compared to the target answers. For an answer to be deemed as correct, it must match the exact target answer. This is to capture potential recall and reasoning errors as well as verifying the final answer. It is important for us to evaluate M_S's ability to consolidate the separate story segments, which is why we require the model to recall the entire story when answering a question. Here, M_U serves as an upper-bound on the performance and any potential gap in performance between it and M_S showcases the added difficulty of consolidating knowledge from the story segments. The number of correctly answered questions over the total number of questions is referred to as the accuracy. We also measure an additional metric, which we refer to as the hallucination rate. Given an answer, consider only the recall part of the answer and disregard the reasoning part and the final answer. Our hallucination rate is the number of recalled sentences that contains an error (does not match with the actual sentence in the narrative) over the total number of recalled sentences. This metric provides a more fine-grained examination of the recall and knowledge consolidation capabilities of the model. We want to evaluate if the model is more likely to hallucinate a fact, event or segments when recalling these from multiple training sequences (segmented setup) versus a single training sequence (unsegmented setup).\n\nOnce both models have been fine-tuned, we take the best performing checkpoint of each model on the validation set and evaluate these on the test set. This is done in the same manner as the validation, except that the questions are from the test set.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The proposed dataset is novel and can point to different avenues on how we can improve knowledge consolidation and encoding in LLM, especially for applications such as LM as KB and knowledge-rich tasks. Currently, complex structures have been employed such as graph NN and advanced neural machines to deal with the limitations of knowledge encoding and processing in neural models and information consolidation could indicate avenues to deal with such complex tasks in LLM in a more efficient manner.\n\nTL;DR\nInteresting new resource for probing capabilities of LLM;\nTo my knowledge, information consolidation capabilities in LLM have not been independently measured in any work, and this gives insights into the limitations of such models in LM as KB and reasoning settings in knowledge-rich scenarios.", "reasons_to_reject": "While interesting in principle, several aspects of the work are not completely clear. Also, I have not been able to identify exactly the results achieved by the authors can be understood as probing of epistemic capabilities of language models in its interpretation of LLM as epistemic models. While the resource gives insights on the limitation of these models in reasoning tasks and of LLm both as knowledge bases and epistemic models, the presented evaluation tasks and analysis don't strike me as epistemic in nature - for example, it is false that the expected answer to question 2 in Table 1 should be no unless we understand epistemic reasoning in a rather limited setting. This difficulty may also lie in the fact that I have not been able to understand exactly how the in-context training with the segmented narrative texts works, as it was not particularly clear in the manuscript.\nOverall, in terms of analysing the epistemic capabilities of LLM, I fail to see how that work informs us more than those regarding the neural theory of Mind, such as that of Sap et al. and Liu et al. Also, it is important to notice that CoT reasoning has been observed to be very susceptible to the examples presented [1,2] and the order of presentation of examples [3], but the failure to control for these and other important characteristics that could influence the performance of the models, yet they draw conclusions regarding the behaviour of such models. The authors also do not clearly discuss how and why each task was chosen and the methodological aspects of creation/selection and validation of examples in the corpus, and whether they are good proxies for information consolidation and epistemic reasoning. Without fully discussing the proposed resource, I hardly see how any of the authors' claims can be trusted. I do think the resource is interesting as a collection of different reasoning tasks that require information consolidation, and thus let us probe the consolidation powers of LLMs, but I find the authors' claims are not properly justified in the manuscript.\nThe work limits itself to the English language, but in no moment the authors consider how (syntactic, semantic, pragmatic, etc) aspects of the language can impact the performance of the models  - especially in a task for language-based reasoning.\n\nTL;DR\nThe authors fail to clearly present the resource and methodological aspects for its creation and validation and discussion of the corpus composition in order for the reader to understand the empirical study resented later;\nThe conclusions drawn by the authors are, in my opinion, unsubstantiated by the work presented. The authors seem to draw much deeper conclusions than their experiments allow and seem to fail to control aspects that may explain the performance of the models in their experiments, but no do discuss these limitations in Limited to the English language.", "questions_to_authors": "How are the prompts for the segmented narratives constructed? In which sense do they differ from the non-segmented narratives?\nWhat does Figure 3 mean? What is measured on X-axis in the graphs? Is is the size of the model (number of parameters)? \nWhat do the authors mean by partial observability and full observability?", "paper_topic_and_main_contributions": "The present paper proposes a dataset for evaluation of epistemic capabilities of language models by means of inspection of knowledge integration using several reasoning tasks of different. The work's main contribution lies in the proposal of the dataset, but also in the experimental analysis of the performance T5 and Flan-T5 models on the resource.\nThe authors propose a dataset composed of a collection of 1.8K narrative texts, transformed into 3.6K question-answer pair, for reasoning tasks of different nature probing the ability of models to consolidate information. The authors then employ this resource to evaluate the epistemic capabilities of T5 and Flan-T5 models of different size in two different training regimens. They strive for answering two guiding questions regarding the nature of knowledge representation and consolidation in these models the the effect of scaling on this capability. The authors further conduct analysis and discussion of their experimental results and pointing to how knowledge consolidation can be improved in language models.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "I would like to see better examples of how the training with segmented narratives works and some explanation of Figure 3, which I found very difficult to understand."}, "extra_text": {"Missing_References": "[1] Liu, J., Shen, D., Zhang, Y., Dolan, B., Carin, L., & Chen, W. (2021). What Makes Good In-Context Examples for GPT-$3 $?. arXiv preprint arXiv:2101.06804.\n[2]  Fu, Y., Peng, H., Sabharwal, A., Clark, P., & Khot, T. (2022). Complexity-based prompting for multi-step reasoning. arXiv preprint arXiv:2210.00720.\n[3] Lu, Y., Bartolo, M., Moore, A., Riedel, S., & Stenetorp, P. (2021). Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. arXiv preprint arXiv:2104.08786."}}, {"review_id": "JT0FmQt5YD", "reviewer": "Reviewer_1wi7", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to extend our sincere appreciation to the reviewers for their thorough and invaluable feedback. It brings us great joy to note that our work has captured the interest of the reviewers (R1 and R2), been recognized for its novelty (R1 and R3), and acknowledged as insightful (R1 and R2). Furthermore, we are delighted to learn that the matter of addressing knowledge consolidation in LLMs has been recognized as \"an important topic that needs to be address\" (R2). We have meticulously examined all the concerns raised by the reviewers and have diligently endeavored to address these issues, as detailed in the discussion below.\n\n**Reviewer’s Comment: “I wonder how the dataset and the task of knowledge consolidation relates to multi-hop question answering and datasets like HotpotQA. While Epik-Eval makes emphasis on a sequence of events with knowledge that needs to be consolidated in order to reach a final output, HotpotQA requires the model to extract information at each step of the reasoning chain that allows addressing such steps in order to answer the overall multi-hop question.”**\n\n**Response:** In the multi-hop question answering benchmarks, a model is given a question and can then search through multiple documents such that the model can use text from these documents  in order to answer the question. This is akin to providing entire documents in the prompt along with the question. Thus, the setup is pure inference and the model must consolidate external knowledge (information from the documents it is searching through). In contrast our work focuses on analyzing if these models can consolidate the knowledge stored in their parameter space (internal knowledge), that they have acquired through training. To do so, we train the models on documents, in order to memorize them, and then in inference mode, evaluate the models for their ability to consolidate and recall the necessary information from the multiple memorized documents in order to be able to answer the question. The focus is thus on the model's ability to consolidate the proper knowledge stored in the parameter space instead of evaluating its ability to search through documents. We will update the related work section to discuss these distinctions.\n\n**Reviewer’s Comment: “I would have liked to see more LLMs evaluated on the dataset in this paper”**\n\n**Response:** We have evaluated Llama-2 7b, which gets 49.56% accuracy when trained on unsegmented stories and 7.56% accuracy when trained on segmented stories. We are currently evaluating the other Llama-2 variants and will do the same with Llama 1 and the OPT models.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "+ Knowledge consolidation in LLMs is an important topic that needs to be address\n+ Evaluation indicates the benchmark is actually useful to hightlight LLMs' ability to deal with this task\n+ Interesting final recommendations ", "reasons_to_reject": "- I would have liked to see more LLMs evaluated on the dataset in this paper\n- Related work should probably look at work like multi-hop QA and existing datasets", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposes a benchmark called Epik-Eval that aims at measuring the ability of LLMs to consolidate knowledge and answer complex questions, which require to build a \"knowledge state\". The authors also propose two baselines based on T5, one of them trained on the non-segmented version of the dataset and the other one on the segmented train set.\n\nExperimentation shows how the segmented version of the LLM incurs in lower perfromance and hallucinates more. This behavior is amelliorated by the instruct-following versions of the model, but this improvement allegedly comes from a better understanding of the question rather thatn from an improved ability to aggregate knowledge. The paper concludes with a number of recommendations, which include adding this kind of tasks involving knowledge consolidation as a predictive goal of LLM pretraining.\n\nI wonder how the dataset and the task of knowledge consolidation relates to multi-hop question answering and datasets like HotpotQA. While Epik-Eval makes emphasis on a sequence of events with knowledge that needs to be consolidated in order to reach a final output, HotpotQA requires the model to extract information at each step of the reasoning chain that allows addressing such steps in order to answer the overall multi-hop question. ", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "L2yn6w0hTH", "reviewer": "Reviewer_tdge", "summary": "", "questions": "•\tIn Figure 3, the performance on the different QA types are compared. We can see a difference in performance for the non-baseline models on the different types. Can this difference be interpreted in any way?\n•\tHow are the stories produced?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to extend our sincere appreciation to the reviewers for their thorough and invaluable feedback. It brings us great joy to note that our work has captured the interest of the reviewers (R1 and R2), been recognized for its novelty (R1 and R3), and acknowledged as insightful (R1 and R2). Furthermore, we are delighted to learn that the matter of addressing knowledge consolidation in LLMs has been recognized as \"an important topic that needs to be address\" (R2). We have meticulously examined all the concerns raised by the reviewers and have diligently endeavored to address these issues, as detailed in the discussion below.\n\n**Reviewer’s Comment: “A more realistic scenario with better insights would be multiple fragmentation ratios, with sentence splitting being one extreme and no splitting being the other. For example, if stories are split into two, the model may be able to identify that the two fragments need to be combined during inference.”**\n\n**Response:** This is a very interesting point which we had not thought about. An approximation of this setup is to group performance by the number of sentences in the stories. We ran this setup and obtained the following results:\n\n#### Flan-T5 Accuracy (Unsegmented Stories)\n|           |    Small |   Base  |  Large  |  XL     |\n|---|---|---|---|---|\n| 1 sentences |   77.97% |  89.83% |  96.61% | 94.07%  |\n| 2 sentences |  100.00% | 100.00% | 100.00% | 93.33%  |\n| 3 sentences |   85.47% |  92.31% |  92.31% | 96.58%  |\n| 4 sentences |   84.00% |  92.00% |  96.00% | 96.00%  |\n| 5 sentences |   59.78% |  60.00% |  81.78% | 81.78%  |\n| 6 sentences |   62.00% |  70.00% |  69.00% | 78.00%  |\n| Total       |   69.11% |  73.44% |  85.56% | 86.67%  |\n\n#### Flan-T5 Accuracy (Segmented Stories)\n|           |    Small |   Base  |  Large  |  XL     |\n|---|---|---|---|---|\n| 1 sentences | 14.41% | 26.27% | 22.88% | 32.20% |\n| 2 sentences | 33.33% | 26.67% | 26.67% | 26.67% |\n| 3 sentences | 34.19% | 37.61% | 41.88% | 50.43% |\n| 4 sentences | 11.00% | 22.00% | 29.00% | 34.00% |\n| 5 sentences | 10.22% |  9.56% | 13.11% |  9.33% |\n| 6 sentences | 4.00% |  2.00% |  1.00% |  6.00% |\n| Total       | 13.67% | 16.22% | 18.78% | 20.33% |\n\n#### Flan-T5 Hallucination Rate (Unsegmented Stories)\n|           |    Small |   Base  |  Large  |  XL     |\n|---|---|---|---|---|\n| 1 sentences | 25.58% | 7.14% | 3.49% | 4.82%\n| 2 sentences |  0.00% | 0.00% | 0.00% | 0.00%\n| 3 sentences | 6.00% | 2.85% | 2.56% | 1.14%\n| 4 sentences | 10.75% | 7.00% | 3.00% | 2.50%\n| 5 sentences | 9.44% | 6.80% | 4.61% | 4.03%\n| 6 sentences | 6.19% | 1.37% | 0.69% | 0.00%\n| Total       | 9.04% | 5.54% | 3.56% | 2.94%\n\n#### Flan-T5 Hallucination Rate (Segmented Stories)\n|           |    Small |   Base  |  Large  |  XL     |\n|---|---|---|---|---|\n| 1 sentences | 72.55% | 72.55% | 64.36% | 55.88%\n| 2 sentences | 27.78% | 27.78% | 27.03% | 26.32%\n| 3 sentences | 22.25% | 22.25% | 19.54% | 15.32%\n| 4 sentences | 59.00% | 59.00% | 43.00% | 36.25%\n| 5 sentences | 62.48% | 62.48% | 57.19% | 57.44%\n| 6 sentences | 41.24% | 41.24% | 41.75% | 37.97%\n| Total       | 54.97% | 54.97% | 49.61% | 47.85%\n\nwhere Accuracy is as in our paper (Section 3): the percentage of correct answers, where a correct answer means that it exactly matches the target answer. Hallucination Rate is also as in the paper (Section 3): the rate of sentences in the model's answer that weren't in the target answer. We note that the accuracy does seem to lower in the 5-6 sentence range. This makes sense given that the model's answer must exactly match the target answer. Since the hallucination rate is not zero, as more sentences need to be recalled, the likelihood of making a mistake increases. As for the hallucination rate, interestingly it fluctuates. The rate starts relatively higher for 1 sentence, then lowers until 3 sentences, comes back up until 5 sentences and lowers a bit for 6 sentences. Since other variables can affect the performance though, for example the number of facts to recall in a given sentence might vary between different tasks, we cannot make strong conclusions between the hallucination rate and the number of sentences in the story. The ideal experiment will be to do what the reviewer has proposed, which is given a story, measure if the performance changes depending on how many times it is segmented. We plan on running this experiment and adding it to the paper.\n\n**Reviewer’s Comment: “In Figure 3, the performance on the different QA types are compared. We can see a difference in performance for the non-baseline models on the different types. Can this difference be interpreted in any way?”**\n\n**Response:** The goal of Figure 3 was twofold. First, we wanted to compare the difficulty of the different tasks. For example, some types of relationships such as temporal or causal might be more challenging for the models. The problem with Figure 3 right now is that we only look at whether or not the entire answer matches the target answer. So if some tasks have longer stories on average than others, those can be more difficult with respect to hallucinations because more sentences need to be recalled. Thus, in order to study solely the reasoning aspect of these tasks, we would need to only consider cases where the model has successfully recalled the story and then look at the reasoning performance. The models trained on the unsegmented stories are a good approximation of this, since they have a very high success rate at recalling the story without hallucinating any part. If we thus look at the performance of these models, we can guess that some categories seem more difficult than others specifically because the reasoning is more challenging. But instead of guessing, we can simply measure the reasoning performance of these models (number of reasoning and final answers that match the target) excluding answers with an issue in the recall part of the answer. Doing so, we get the following results:\n#### Flan-T5 Reasoning Accuracy per Category (Unsegmented Stories)\n|           |    Small |   Base  |  Large  |  XL     |\n|---|---|---|---|---|\nRetrieval | 91.15% | 100.00% | 100.00% | 100.00%\nComparison & Ranking | 80.71% |   0.00% |  62.50% |  90.00%\nTemporal | 82.53% |  96.00% |  95.83% |  90.00%\nParallel | 100.00% | 100.00% | 100.00% | 100.00%\nCausal | 100.00% | 100.00% | 100.00% | 100.00%\nExclusivity | 65.25% | 100.00% | 100.00% | 100.00%\nDetail Recognition | 60.61% | -       | -       | -\nConsistency | 100.00% | 100.00% | 100.00% | 100.00%\nInterrelation | 100.00% | 100.00% | 100.00% | 100.00%\n\nAs we can see, the larger models get very good reasoning accuracy in all categories, except comparison & ranking which seems a bit more challenging, but the XL model eventually performs well on it. A dash in the table means that the models failed to recall the story properly and thus the reasoning part is ignored since we only want to evaluate the reasoning when the model has recalled the correct information first. We will add this analysis to the paper.\n\nThe second goal of Figure 3 was to verify if for some categories/tasks, the models trained on segmented stories performed more closely to the models trained on unsegmented stories. This is true for certain categories. The reason for a difference in gap could be: because the stories are longer or contain more facts and thus it is more difficult for the model to recall them without making a mistake or because the reasoning is more challenging. If we look at the average number of sentences of stories per category, we get:\n| Category | Sentences on average |\n|---|---|\n| Retrieval |            3.33 |\n| Comparison & Ranking | 5.50 |\n| Temporal |             4.75 |\n| Parallel |             5.00 |\n| Causal |               5.00 |\n| Exclusivity |          5.5 |\n| Detail recognition |   5.00 |\n| Consistency |          3.00 |\n| Interrelation |        4.00 |\n\nwhich does not seem to correlate with the gap between the models trained on segmented and unsegmented stories. As we saw earlier, the reasoning aspect also does not seem to be a challenge for the larger models. Finally, we could look at the number of facts to recall per story to see if this correlates with the gap in performance:\n#### Flan-T5 Hallucination Rate (Unsegmented Stories)\n|           |    Small |   Base  |  Large  |  XL     |\n|---|---|---|---|---|\nRetrieval | 11.04% |  3.12% |  1.00% | 1.52%\nComparison & Ranking | 6.07% |  5.71% |  0.78% | 2.34%\nTemporal | 5.76% |  2.72% |  3.36% | 6.05% \nParallel | 7.20% |  0.00% |  0.00% | 0.00%\nCausal | 12.80% | 10.80% |  3.60% | 0.80%\nExclusivity | 7.30% |  1.57% |  3.14% | 2.40%\nDetail Recognition | 20.40% | 24.40% | 14.00% | 0.40%\nConsistency | 12.00% |  6.00% |  5.33% | 2.67%\nInterrelation | 21.50% | 14.00% |  6.00% | 5.00%\n\n#### Flan-T5 Reasoning Accuracy per Category (Segmented Stories)\n|           |    Small |   Base  |  Large  |  XL     |\n|---|---|---|---|---|\nRetrieval | 50.16% | 42.80% | 45.46% | 39.59%\nComparison & Ranking | 52.20% | 51.92% | 44.79% | 36.41%\nTemporal | 63.38% | 58.66% | 59.38% | 59.19%\nParallel | 43.60% | 50.80% | 46.00% | 60.00%\nCausal | 67.20% | 73.60% | 56.00% | 54.80%\nExclusivity | 51.94% | 56.74% | 51.00% | 52.00%\nDetail Recognition | 87.60% | 94.40% | 88.40% | 85.20%\nConsistency | 22.67% | 20.00% | 14.67% |  8.67%\nInterrelation | 62.00% | 52.00% | 31.00% | 20.00%\n\nThe hallucination rate does vary depending on the category. If we look at the consistency and interrelation results, the XL models trained on segmented stories hallucinate at a lower rate than in other categories and these same models perform relatively well compared to other categories in Figure 3. As for other categories, the pattern is less clear. To better understand the reason for the gap between models trained on segmented and unsegmented stories with respect to categories of relation, a per task analysis would be required as too many variables come into play. We believe though that this falls outside the scope of the paper, as the goal is to study the impact of segmentation on the performance of these models. The choice of task is somewhat irrelevant and the reasoning aspect certainly is. We will add these analyses and discussion to the paper.\n\n**Reviewer’s Comment: “The two colors in Figure 2 are difficult to distinguish.”**\n\n**Response:** We have switched the colors in Figure 2 from blue and light-blue to blue and orange.\n\n**Reviewer’s Comment: “How are the stories produced?”**\n\n**Response:** We have clarified this process in the updated section 3, which we have provided in the following response. Please refer to it. If any details are still unclear, the reviewer is more than welcome to let us know.\n\n**Reviewer’s Comment: “The corpus used for this study, EpiK-Eval, is extremely fragmented – the stories are split into their sentences. Even for a human it would be difficult to identity which of these fragments belong to the same story and need to be considered together to solve the QA task.”**\n\n**Response:** We have failed to acknowledge the following detail and apologize for it. Each story and story segment begins with the title of the story, such that the model can easily tell which story a given segment belongs to. For example, an unsegmented story will look like:\n> [Task 1] Tom's Afternoon\n> \n> 1 PM: Tom reads a book.\n> \n> 3 PM: Tom goes to the store.\n> \n> 5 PM: Tom is at the restaurant.\n\nAnd the story segments:\n> [Task 1] Tom's Afternoon, Part 1/3\n> \n> 1 PM: Tom reads a book.\n\n> [Task 1] Tom's Afternoon, Part 2/3\n> \n> 3 PM: Tom goes to the store.\n\n> [Task 1] Tom's Afternoon, Part 3/3\n> \n> 5 PM: Tom is at the restaurant.\n\nThe character's name in each story is also unique and appears in each sentence, thus the model can always relate the story parts to the character's name. We’ve updated section 3 with this clarification as well as clarifying the entire data generation and evaluation process. See our updated section 3 below.\n\n## 3 EpiK-Eval\nThe EpiK-Eval benchmark presents a suite of novel, narrative-based diagnostic tasks, meticulously designed to evaluate a LM's capacity to construct a comprehensive, unified knowledge state.\n\n**Dataset:** Our benchmark comprises 18 tasks, which are questions about relations between facts and events in stories, for example: \"Does x happen before/after y?\". See Table 2 for the full list of tasks. For each of these, we generate 100 stories following a per task template. For example, Task 2 has the following template:\n> [Task 2] {name}'s Vacation\n> \n> {name} went {activity} on {day}.\n>\n> .\n>\n> .\n>\n> .\n\nwhere the first line is the story's title, the name is randomly sampled such that it is unique to each story and the activity and day in a sentence are randomly sampled from the list ['fishing', 'hiking'] and ['Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday', 'Sunday'] respectively. The story can have a random number of sentences, the range is pre-determined per task, for example, Task 2 stories can have between 3 and 5 sentences. Here is an example of a generated story for Task 2:\n> [Task 2] Tom's Vacation\n> \n> Tom went fishing on Monday.\n> \n> Tom went hiking on Wednesday.\n> \n> Tom went fishing on Saturday.\n\nThus, with a 100 stories generated for each 18 tasks, we get 1800 stories, which we refer to as our dataset of unsegmented stories D_U = {x_1, x_2, ..., x_1800}. After having generated these stories, we also generate a second dataset which consists of the segmented version of these stories. For each given story, we segment it into its individual sentences and add a part number to the title. For example, given the previous story about Tom, we would get the following three text sequences:\n> [Task 2] Tom's Vacation, Part 1/3\n> \n> Tom went fishing on Monday.\n\n> [Task 2] Tom's Vacation, Part 2/3\n>\n> Tom went hiking on Wednesday.\n\n> [Task 2] Tom's Vacation, Part 3/3\n>\n> Tom went fishing on Saturday.\n\nWe do this for all 1800 stories and get 6800 story segments, which form our dataset of story segments D_S = {s_1, s_2, ..., s_6800}.\n\nFor each story, we also generate one question-answer pair. The questions are rephrasings of the task. For example, for Task 2 \"How many times does x happen?\", we have \"How many times did {name} go fishing?\". The question-answer pairs are also generated following a template. The template always consists of a question followed by the answer which itself has three parts: recall of the entire story, an optional reasoning part depending on the task and the final answer. For example, the template for question-answers in Task 2 is the following:\n> [Task 2] How many times did {name} go fishing?\n>\n> {story} The answer is {count}.\n\nand here is an example of a generated question-answer pair:\n> [Task 2] How many times did Tom go fishing?\n> \n> Tom went fishing on Monday. Tom went hiking on Wednesday. Tom went fishing on Saturday. The answer is 2.\n\nWe also include in the Appendix a full list of templates as well as the values each placeholder variable (e.g., activities, days) can take, in order to generate stories and question-answer pairs for each task. Having generated one question per story, we have a total of 1800 question-answer pairs, which we randomly split into two sets: the validation and the test set. In order for the models to learn the answer format that they are expected to respect while answering the questions, we add question-answer examples to the training set. We thus generate an additional 1800 stories and question-answer pairs. We discard the stories and add these 1800 question-answer pairs to the training set, such that there are no overlaps between questions in the training, validation and test set.\n\n**Evaluation Process:** We evaluate pretrained LMs for their ability to consolidate knowledge. To do so, given a pretrained language model, we make two copies of it: M_U and M_S. We fine-tune M_U on the unsegmented stories and M_S on the segmented stories. The prior setting allows for all the information necessary to answer a given question to be found in a single text sequence without requiring the model to learn dependencies across multiple text sequences, while the latter setting requires consolidating the information from the narrative segments. Having both allows to measure the effect of information being spread across separate text sequences and the LMs' ability to consolidate this knowledge during inference, by measuring the gap in performance between both models.\n\nBoth models are fine-tuned on their respective dataset, D_U and D_S, as well as the training set of question-answer examples. Thus, one epoch for M_U consists of 3600 samples (1800 stories + 1800 q/a examples) and one epoch for M_S of 8600 samples (6800 segments + 1800 q/a examples). Samples are shuffled such that a batch may contain a mix of stories and question-answer examples in the case of M_U or story segments and question-answer examples in the case of M_S. Models are fine-tuned with their respective pretraining objective. Specifically, in the case of encoder-decoder style models, the story's title (first line in the text sequence) is fed to the encoder and the decoder is expected to output the rest of the story in the case of M_U or the story segment in the case of M_S. As for question-answer pairs, the question is fed to the encoder and the model is expected to output the answer. For causal language models, they are simply expected to predict the next token in the given sequence, as is standard procedure. Precisely, for M_U, a text sequence is either an entire story or a question concatenated with its answer, while for M_S, a text sequence is either a story segment or a question concatenated with its answer.\n\nDuring fine-tuning, both models are also periodically evaluated on the validation set. Models are run in inference mode as described in the papers they were introduced in. We prompt the models with the questions from the validation set and their answers are compared to the target answers. For an answer to be deemed as correct, it must match the exact target answer. This is to capture potential recall and reasoning errors as well as verifying the final answer. It is important for us to evaluate M_S's ability to consolidate the separate story segments, which is why we require the model to recall the entire story when answering a question. Here, M_U serves as an upper-bound on the performance and any potential gap in performance between it and M_S showcases the added difficulty of consolidating knowledge from the story segments. The number of correctly answered questions over the total number of questions is referred to as the accuracy. We also measure an additional metric, which we refer to as the hallucination rate. Given an answer, consider only the recall part of the answer and disregard the reasoning part and the final answer. Our hallucination rate is the number of recalled sentences that contains an error (does not match with the actual sentence in the narrative) over the total number of recalled sentences. This metric provides a more fine-grained examination of the recall and knowledge consolidation capabilities of the model. We want to evaluate if the model is more likely to hallucinate a fact, event or segments when recalling these from multiple training sequences (segmented setup) versus a single training sequence (unsegmented setup).\n\nOnce both models have been fine-tuned, we take the best performing checkpoint of each model on the validation set and evaluate these on the test set. This is done in the same manner as the validation, except that the questions are from the test set.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The ability to combine information seen in different documents during training is crucial for many applications. This is the first study that investigates LMs’ capability to perform this information combination effectively.", "reasons_to_reject": "The corpus used for this study, EpiK-Eval, is extremely fragmented – the stories are split into their sentences. Even for a human it would be difficult to identity which of these fragments belong to the same story and need to be considered together to solve the QA task. A more realistic scenario with better insights would be multiple fragmentation ratios, with sentence splitting being one extreme and no splitting being the other. For example, if stories are split into two, the model may be able to identify that the two fragments need to be combined during inference.\n\nEDIT: The authors clarified that each fragment contains the story title, meaning that for a human it would indeed be an easy task to identify fragments belonging together.", "questions_to_authors": "•\tIn Figure 3, the performance on the different QA types are compared. We can see a difference in performance for the non-baseline models on the different types. Can this difference be interpreted in any way?\n•\tHow are the stories produced?", "paper_topic_and_main_contributions": "The authors investigate the ability of LMs to learn from fragmented information, in other words, to identify information belonging together and combining for consistent inference. For this investigation a corpus of stories is fragmented into sentences and LMs (T5 and Flan-T5) trained on the fragments and compared to the same models trained on the un-fragmented stories. The models are evaluated on QA tasks that requires the combination of multiple sentences in a story.\nThe authors find that LMs perform poorly on the QA tasks when learning from fragmented data. They conclude that the future work should investigate adapting the training objective of LMs to learn context beyond just one input document.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "•\tThe two colors in Figure 2 are difficult to distinguish."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The main conclusions of the reviews and the post-rebuttal discussions:
- 3/ 3 reviewers consider the paper sound (scores 3, 3, 3)
- 3/ 3 reviewers find the paper exciting (scores 4, 4, 3)

From reading the rebuttal and seeing the scores above, I find that the reviewers consider strong points for soundness the following:
- Novel dataset for evaluation of epistemic capabilities of language models using inspection of knowledge integration using several reasoning tasks. 
- Good experimental analysis of the performance T5 and Flan-T5 models on the dataset and final recommendations 
- Knowledge consolidation in LLMs is important. The ability to combine information seen in different documents during training is crucial for many applications. This is the first study that investigates LMs’ capability to perform this information combination effectively.
Decision: Accept-Main