What is Wrong with Language Models that Can Not Tell a Story? (2023)

Abstract: In this position paper, we contend that advancing our understanding of narrative and the effective generation of longer, subjectively engaging texts is crucial for progress in modern Natural Language Processing (NLP) and potentially the broader field of Artificial Intelligence. We highlight the current lack of appropriate datasets, evaluation methods, and operational concepts necessary for initiating work on narrative processing.

## introduction since the linguistic turn in the early 20th century @xcite , human language has been considered fundamental to shaping human cognition. this notion positions language as a core aspect of intelligence, often equating intelligence with the ability to generate natural language. in @xcite , turing famously suggests that the capacity for meaningful natural language interaction is critical for artificial intelligence. while most contemporary researchers narrow the turing test's scope to day-to-day conversations, the original essay emphasizes that artificial intelligent agents should convincingly imitate humans in creative tasks expressed in natural language. framing the problem in turing's original terms reveals the current limitations of artificial systems, which can only partially imitate human dialogue in specific contexts and struggle to generate engaging stories @xcite or jokes @xcite . modern natural language generation (nlg) leverages increased computational power and vast training data @xcite @xcite @xcite , focusing on computation-heavy solutions rather than on statistical methods and mathematical models to qualitatively advance our understanding of language. a century after andrey markov developed his eponymous chains to analyze poetry, nlg concepts remain similar, and their limitations could hardly be overcome solely through quantitative means. this is particularly evident in narrative processing, where automatic generation of textual narratives often requires significant human intervention or relies on predefined narrative structures @xcite . efforts to generate longer text blocks foot_0 exist, such as (kedziorski, 2019) and @xcite , see figure 1 , but they succeed only under certain stylistic and topical constraints that preclude genuine narrative generation. while recent advancements have been made in suspense generation @xcite , narrative personalization @xcite , and short context-based narratives @xcite , generating extended stories remains a challenge @xcite . philosophers and linguists have attempted to conceptualize plot, narrative arc, action, and actor notions for nearly a century @xcite @xcite , but few of these concepts have proven useful for modern nlp. in @xcite , a machine comprehension corpus is presented for end-to-end script knowledge evaluation, revealing that existing machine comprehension models struggle with tasks humans find relatively easy. despite these setbacks, some progress in narrative generation has been made within the nlp community @xcite . however, narrative generation is still largely considered a fringe research topic. we argue that the concept of narrative is crucial for further nlp progress and should become a focal point within the nlp community. this paper raises vital questions for narrative processing to establish itself as a well-defined sub-field in nlp research. we begin by presenting several arguments for why breakthroughs in narrative pro-figure 1 : "copyrighted protein fiction may be deemed speculative propaganda" -a line from a generative art project "paranoid transformer -a diary of an artificial neural network", @xcite . the diary was generated end-to-end without any human post-processing and published as a hardcover book. this is one of the examples of long-form generated artistic text, however the text is devoid of narrative. we argue that narrative processing as a research field would significantly impact two other core aspects of natural language processing, which are essential for expanding the adoption of nlp products and technologies. the first aspect is causality and natural language inference. causal inference from natural language is crucial for further nlp progress, and current language models still underperform in this area. although narrative could be considered a sub-category within a broader family of causal texts, we contend that narrative generation is an ideal task for validating and testing hypotheses around natural language inference, paving the way for more explainable ai. the second area where narrative processing is indispensable is the continued development of human-machine interaction. people are known to remember stories more than facts @xcite , but nlp-based natural language interfaces exhibit the opposite tendency, processing and "remembering" facts more easily than stories. these factors make narrative essential for further nlp progress. another field in which narrative processing could prove pivotal is explainable ai. one could argue that a feasible path to explainable artificial intelligence involves a set of dedicated models trained to communicate with humans in natural language, clarifying specific aspects of a given decision. these models would necessarily need to be capable of causal inference in natural language. although this technically leads to the same bottleneck discussed earlier, we believe this field is so critical for the continued development and adoption of artificial intelligence in the industry that it warrants explicit mention here. this position paper aims to highlight critical gaps in our conceptual understanding, benchmarking, and evaluation within the field of narrative processing. we contend that these three significant layers require the immediate focus of the research community. in this section, we examine each of these layers in depth and propose potential avenues for progress. ## conclusion this position paper puts forth two primary assertions: we hope this paper stimulates further discussion on these topics and attracts the attention of the nlp and ai community towards the challenges surrounding narrative generation.