Recollection versus Imagination: Exploring Human Memory and Cognition via Neural Language Models (2020)

Abstract: We investigate the use of NLP as a measure of the cognitive processes involved in storytelling, contrasting imagination and recollection of events. To facilitate this, we collect and release Hippocorpus, a dataset of 7,000 stories about imagined and recalled events. We introduce a measure of narrative flow and use this to examine the narratives for imagined and recalled events. Additionally, we measure the differential recruitment of knowledge attributed to semantic memory versus episodic memory (Tulving, 1972) for imagined and recalled storytelling by comparing the frequency of descriptions of general commonsense events with more specific realis events. Our analyses show that imagined stories have a substantially more linear narrative flow, compared to recalled stories in which adjacent sentences are more disconnected. In addition, while recalled stories rely more on autobiographical events based on episodic memory, imagined stories express more commonsense knowledge based on semantic memory. Finally, our measures reveal the effect of narrativization of memories in stories (e.g., stories about frequently recalled memories flow more linearly; Bartlett, 1932). Our findings highlight the potential of using NLP tools to study the traces of human cognition in language.

## introduction when telling stories, people draw from their own experiences (episodic knowledge; @xcite and from their general world knowledge (semantic knowledge; @xcite . for example, in figure 1 (top), a recalled story about a birth will likely recount concrete events from that day, relying heavily on the author's episodic memory @xcite . on the ….her husband called me and then drove her to the hospital. i joined her at the hospital. when we got the hospital things got complicated. her husband tried his best to be with her and to keep her strong. she eventually delivered perfectly. my daughter gave birth to her first child. she and her husband were overwhelmed by emotions. ## hippocorpus creation we construct hippocorpus, containing 6,854 stories ( ## measures to quantify the traces of imagination and recollection recruited during storytelling, we devise a measure of a story's narrative flow, and of the types of events it contains (concrete vs. general). ## imagining vs. remembering we summarize the differences between imagined and recalled stories in hippocorpus in imagined stories flow more linearly. we compare ∆ l , i.e., pairwise differences in nll for sentences when conditioned on the full history vs. no history (density plot shown in figure 3 ). when averaging ∆ l over the entire story, we find that sentences in imagined stories are substantially more predictable based on the context set by prior sentences than sentences in remembered stories. this effect is also present with varying history sizes (see figure 5 in appendix c.1). recalled stories are more event-dense. as seen in lexical differences. lexicon-based counts uncover additional differences between imagined and recalled stories. namely, imagined stories are more self-focused (i-words), more emotional 6 linear regressions use z-scored variables. we confirm that our findings hold with multivariate regressions as well as when adding participant random effects in appendix c.2. 7 note that simply using verb count instead of number of realis events yields the opposite effect, supporting our choice of measure. (tone, positive and negative emotion) and evoke more cognitive processes. 8 in contrast, recalled stories are more concrete and contain more logical or hierarchical descriptions (analytic). discussion. our interpretation of these findings is that the consolidated memory of the author's life experience permeates in a more holistic manner through the sentences in the recalled story. imagined stories are more fluent and contain more commonsense elaborations, which suggests that authors compose a story as a sequence, relying more on preceding sentences and commonsense knowledge to generate the story. while our findings on linearity hold when using different language models trained on wikipedia articles @xcite or english web text (mostly news articles; @xcite , a limitation of the findings is that gpt is trained on large corpus of fiction, which may boost linearity scores for imagined (vs. recalled) sentences. future work could explore the sensitivity of our results to changes in the language model's training domain or neural architecture. ## narrativization of recalled stories we further investigate how our narrative and commonsense measures can be used to uncover the narrativization of recalled events (in recalled and retold stories). these analyses aim to investigate the hypothesis that memories are narrativized over time @xcite , and that distant autobiographical memories are supplemented with semantic or commonsense knowledge @xcite @xcite . first, we compare the effects of recency of the event described (timesinceevent: a continuous variable representing the log time since the event). 9 then, we contrast recalled stories to their retold counterparts in pairwise comparisons. finally, we measure the effect of how frequently the experienced event is thought or talked about (frequencyofrecall: a continuous variable ranging from very rarely to very frequently). 10 as in §4, we holm-correct for multiple comparisons. temporal distance. first, we find that recalled and retold stories written about temporally distant events tend to contain more commonsense knowledge (|β| = 1.10, @xmath2.001). we found no other significant associations with timesinceevent. ## conclusion to investigate the use of nlp tools for studying the cognitive traces of recollection versus imagination in stories, we collect and release hip-pocorpus, a dataset of imagined and recalled stories. we introduce measures to characterize narrative flow and influence of semantic vs. episodic knowledge in stories. we show that imagined stories have a more linear flow and contain more commonsense knowledge, whereas recalled stories are less connected and contain more specific concrete events. additionally, we show that our measures can uncover the effect in language of narrativization of memories over time. we hope these findings bring attention to the feasibility of employing statistical natural language processing machinery as tools for exploring human cognition.