Data-to-text systems as writing environment (2022)

Abstract: Today, data-to-text systems are used as commercial solutions for automated text productionof large quantities of text. Therefore, they already represent a new technology of writing. This new technology requires the author, asan act of writing, both to configure a systemthat then takes over the transformation into areal text, but also to maintain strategies of traditional writing. What should an environmentlook like, where a human guides a machineto write texts? Based on a comparison of theNLG pipeline architecture with the results ofthe research on the human writing process, thispaper attempts to take an overview of whichtasks need to be solved and which strategiesare necessary to produce good texts in this environment. From this synopsis, principles for thedesign of data-to-text systems as a functioningwriting environment are then derived.

## introduction natural language generation (nlg) systems are computer systems that automatically generate texts in human languages, using advanced techniques from artificial intelligence and/or computational linguistics @xcite . non-academic nlg systems are used in different areas of text production and result in fundamental changes for content creation and publication processes: they form a new type of writing technology and create a new environment for humans in which texts are generated automatically, but humans still (co-)design the rules and specifications for this generation. while nlg systems based on pre-trained large language models function more as writing assistants for authors on an individual level, the nlg systems that are the subject of this study have a different aim: they are configured to be able to produce large amounts of text automatically. in this context, writing is regarded in a broader sense and means creating a blueprint for producing specific texts. so this new type of writing can be described as meta-writing: however, since the requirements of text structure, expression, and realisation of a communication goal cannot be solved on an abstract level only, many traditional writing tasks remain to be done by the author. @xcite have described this new condition as follows: "automated text production -when the author is not the writer". this observation raises the question, what a writing environment should be like in which a machine is guided by an author to write a text? in this research, we use the framework of creating a writing environment to set out the requirements for an nlg system. so, the human writer is considered here as the agent, while the software functions as the environment. this setting is due to the fact that writing, in general, is primarily perceived as an individual action, even though some instances of writing are performed in collaboration. but of course, it is not the only possible framework. the interaction between humans and machines has recently been discussed, especially in the communicative field of ai, where both humans and the instances of ai are seen as agents and the aspect of collaboration is much more prominent (for the field of journalism: @xcite ; for fiction writing: @xcite ; @xcite ). and indeed, it may be that statistical approaches and deep-learning methods, in particular, bring the software's autonomy more to the fore. autonomy is, after all, the distinctive property of the agent @xcite . this then would call for a reassessment of the situation, looking more closely at the requirements of collaboration within this described environment. however, data-to-text systems in real-world applications still require such a share of human configuration and control and the creative contribution share of the software, at least in the nlg systems focused on in this paper, is still so limited that it would not be adequate to claim creative autonomy of the software in the process. the environmental framework with its orientation towards the writing processes also offers the advantage of shifting the focus in the evaluation of nlg systems @xcite from the evaluation of the output to an evaluation of the processes, that @xcite recently postulated: "evaluating nlg tasks only through the lens of outputs is thus insufficient and we should strife (sic) to deliver a more fine-grained breakdown (...)". for traditional writing it is set that the principle of having control over the writing and editorial processes is the most effective method of influencing text quality @xcite . and we assume it remains valid also for working with nlg systems. thus, our approach could open up new perspectives for the evaluation of nlg systems. what perrin stated in 2002 for writing per se also applies to automated text production nowadays: "writing is thus changing from a field of largely intuitive language design to a language technology that becomes aware of its compositional principles and purposefully uses its means, tools, and strategies" (translated from german @xcite ). as a starting point to achieve such an awareness and methodology for this new kind of writing, including a system of rules, strategies, and cues that guide action, we want to make the action steps, tools and decisions within the processes explicit: in the basic reference work on nlg it is characterized as 'the subfield of artificial intelligence and computational linguistics that is concerned with the construction of computer systems that can produce understandable texts in english or other human languages from some underlying non-linguistic representation of information' @xcite . there are already a number of implemented applications for the data-to-text approach in different areas. they range from the media sector, where they have been a much-discussed topic as "robot journalism", to medical reports, business and finance reports or product descriptions in ecommerce. nlg systems are useful when large amounts of text are needed or information is only available in formats that are not easily understood (such as measurement data from medical examinations), and verbalisation facilitates or enables understanding. in this study, a further classification concerning the organization of nlg systems is to be discussed. on the one hand there are the so-called pipeline solutions that modularize the procedures and then execute the tasks (one after the other). the end-toend solutions on the other hand leave the modular approach behind and aim for end-to-end generation based on the successes of deep learning. they can be trained with (data, text) tuples that can be efficiently collected at scale @xcite . large pre-trained language models such as gpt-3 or bert can be integrated into all of these solutions. at present, end-to-end solutions are not yet suitable for commercial production of great amount of texts because they have two fundamental limitations: first, they are very domain-bound, so they can only generate texts for very limited segments. in addition, they lack semantic fidelity, this means how accurately the generated text conveys the meaning @xcite . as described, end-to-end systems based on deep learning combine all nlg steps in one function. this means that the only possible intervention is to select or edit the results @xcite . due to this too tight restriction of interaction these approaches fall out of consideration for this research. modular data-totext systems, on the other hand, offer more points of connection and reflect parallels between humans and systems in the text generation process. since this study analyses the application under real-life conditions, the focus is on implementable solutions, not on academic nlg projects. in the commercial sector, rule-based pipeline solutions are established first and foremost, which differ in handling, architectures and purposes. some of the solutions are offered as self-service, requiring limited or no programming skills. the leading companies in this fields are arria nlg, narrative science, ax semantics, yseop and automated insights @xcite . ## overview of the nlg pipeline there are different ways to structure the tasks and decisions of text generation. the most cited model for this is the nlg architecture constructed by ehud reiter and robert dale that performs tasks in sequence related to document planning, sentence planning and linguistic realization @xcite . ## a model of the human writing process from the best-known model that illustrates how human writing functions at the cognitive level -the so called flower-hayes-model @xcite -three important features can be derived that are characteristic of the human individual writing process: the three distinguishable cognitive processes are controlled by a monitor. this central executive directs attention and switches from one sub-process to another. the first process is planning of a text, where information is collected and thoughts are made about the form and structure of the text. what should the text achieve? whom does it address? what aspects, data, information should appear in it? it comprises three types of sub-operations: first there is generating, in which the writer retrieves information relevant to the writing task from longterm memory. then there is organising, during which the most useful of the retrieved elements are arranged in a plan; finally the writer sets further goals to guide the writing (goal setting). after the planning follows the phase of linguistic implementation (translating). while many ideas in the planning phase are not really linguistically available, a kind of translation process now takes place during which the thoughts are translated into language: one now decides on the concrete vocabulary. the third main process is reviewing with its two sub-operations editing and revising. now, the writer re-reads the text and aims to improve the quality of the written text by changing the text at the time it was written to correct errors, or fit the plans (editing). or they intentionally revise the text to look for problems and errors at all levels of the text (revising). ## requirements for writing software in general, technology and writing have always been interdependent: the writing tool and the writing medium influence writing in terms of how the problems at hand can be solved. in most writing settings today, the pen, pencil or typewriter has ceased to be the tool, and paper is no longer the medium. rather, computers, tablets and smartphones with input functions and screens are the extended writing environment today @xcite . the writing environment in the narrower sense is the associated software. there have been and still are approaches to investigate which conditions serve the authors to write without interference and receive the appropriate support during the writing process. the investigation of the results of writing process research played an important role in this context @xcite . it was criticised that the writing tool and the medium were not included in former research. the most important results of the critique are, first, sharples' @xcite re-evaluation of recursiveness and writing phases and second, the description of certain objects (external mental representations) as a bridge between the writer's ideas and the emerging text. he emphasises the biphasic nature of two activities within the writing process: engagement -this means the actual writing, where new material is created and reflection, the thinking (about the writing) where the generated material is revised. the two processes are separate and cannot occur simultaneously, forming cycles of engagement and reflection in writing @xcite . from these results, guidelines for the development of writing environment software were derived @xcite with elaborating the following aspects: today there are a handful of software tools that take this non-linear writer-centred approach as their starting point (such as pagefour, liquid story binder, roughdraft (discontinued), ulysses, scrivener), but they tend to be used for specific professionalised, often narrative, writing @xcite . however, functions are built into conventional text processors as well that support individual phases of the writing process, such as the outline view, comment functions, text and grammar checks @xcite . ## nlg systems in real life: writing on a meta level at this point, the phases of the human text production process and the modules of the nlg pipeline architecture are juxtaposed in order to find out which principles can be derived for an nlg system that is not designed for experts, but as a writing environment for the (automated) production of large numbers of texts. ## principles for designing nlg systems the following principles for the design of an nlg system can be derived from the observations presented above: and secondly, all changes within a task must be immediately passed on to all instances of the system. ## conclusion we showed that there are considerable similarities between the nlg modules and the writing phases of humans in terms of the tasks and decisions involved, which is a significant prerequisite for establishing these systems as a new extended writing technology. the analysis of these processes is of particular relevance in that quality assurance for data-to-text systems -whose goal is the mass generation of texts -is only attainable by optimizing the processes, since an evaluation of the entire output is not feasible. however, it also became clear that the human writing process has special features that need to be taken into account when designing nlg systems, especially the consistent and fast change between the processes and the distinctive cognitive activities that require access to different components of the human working memory (e.g. visio-spatial or phonological loop). to neglect these characteristics would mean confining the human involved in a linear process and to strict rules of formal language (i.e. code) to produce natural language texts. this kind of environment would impede the capacity of human writing and, with it, the quality of the text generated. in other words, it would stand in the way of a further successful development of the technology of writing which is to be expected in the course of adapting nlg systems in text production.