A Design Space for Writing Support Tools Using a Cognitive Process Model of Writing (2022)

Abstract: Improvements in language technology have led to an increasing interest in writing support tools. In this paper we propose a design space for such tools based on a cognitive process model of writing. We conduct a systematic review of recent computer science papers that present and/or study such tools, analyzing 30 papers from the last five years using the design space. Tools are plotted according to three distinct cognitive processes–planning, translating, and reviewing–and the level of constraint each process entails. Analyzing recent work with the design space shows that highly constrained planning and reviewing are under-studied areas that recent technology improvements may now be able to serve. Finally, we propose shared evaluation methodologies and tasks that may help the field mature.

## introduction the development of large-scale language models (sometimes called foundation models) is dramatically changing what technology can achieve and support @xcite . language models like gpt3 @xcite and meena @xcite have led to an increasing interest in how these new technologies may support writers, for instance by providing a journalist with text in the style of the new yorker @xcite or giving a novelist a new story ending @xcite . in this paper we seek to understand where research on writing support tools currently stands, and what areas of research may be important but currently under-served. computational approaches to writing support have a long and rich history, certainly dating back to before the introduction of modern computation, at least to the early 1900s with the cut-up method @xcite and 'plot genie' books @xcite , and likely even further back when considering the long history of generative traditions such as tarot cards @xcite . in more contemporary understandings of computation, technology developed by the natural language processing (nlp) community is often taken up as a writing tool. 1 we believe the advent of foundation models poses an exciting inflection point at which these technologies can be used to support the evergreen activity of writing in new ways. in this paper, we draw on a cognitive process model of writing that considers writing to be a goal-directed thinking process with three distinct and non-linear cognitive processes: planning, translating, and reviewing @xcite . we use this model to propose a design space for writing support tools. this allows us to understand what a writing support tool is attempting to support, and identify gaps or opportunities in the field. it provides a shared vocabulary for researchers, and we hope it will help the field mature and provide common goals and methodologies. to demonstrate the use of the design space, we perform a systematic literature review of research on writing support tools from the last five years @xcite @xcite @xcite . this shows areas of active research and under-served areas, as well as limitations of current technology to support different aspects of writing. we also use these papers to investigate how to evaluate writing support tools. the contributions of this paper are: ## writing goals design space flower and @xcite describe writing in the following way: the act of composing itself is a goaldirected thinking process, guided by the writer's own growing network of goals. these writing goals may be large, like to write up an experiment for an academic paper, or small, like to make a sentence sound more formal. they may be open-ended, like to come up with the name for a character, or quite limited, like to spell a word correctly. the goals may require imagining the reader, like to determine if a sentence is too confusing, or they may require diving deeper into what's already written, like to ensure a technical topic is discussed consistently throughout an article. writing goals may start as external motivators-someone may ask one to write something-but as one writes, writing goals are created by the writer and propel the writing process forward. we propose using this to structure a design space for writing support tools. whether we call them support tools, assistants, co-creators or machinesin-the-loop, we believe what unites these systems is that they take on goals inherent in the writing process. we propose two axes for the design space: we acknowledge that there can be some ambiguity in distinguishing between these processes. for instance, consider a tool that, upon request, completes a writer's sentence. this tool may be supporting translating, if the completion is intended to articulate what the writer already had in mind. or it could be supporting planning, if the completion is intended to provide the writer with new ideas or directions for their writing. when annotating papers, we rely on how the researchers describe the tool, though we acknowledge the ambiguities involved in this and that writers may use a tool in unexpected or unintended ways. 6 2. the amount of constraint the goal has. a highly constrained goal has very few possible solutions, like when writing a technical definition. a lightly constrained goal has many possible solutions, like when describing a newly introduced fictional character. the amount of constraint gives us a measure of how particular the support must be to achieve the goal. this may be considered a measure of difficulty-writing a technical definition is very constrained, and supporting this writing task requires a high level of world understanding from a system-but constraint doesn't always imply difficulty. a writing goal may be very constrained, for instance make a particular sentence more positive, but the support may be fairly straightforward, like providing a list of positive words. figure 2 shows some hypothetical writing support tools in this design space, to better understand the space. further details and descriptions of the design space can be found in the appendix. ## methodology we perform a preliminary, systematic literature review such that we can plot tools in the design space. this validates the utility of the design space and provides insights into the landscape of writing support tools. ## limitations our systematic review was limited in scope, as we focused only on the last five years, and our query for selecting papers may not have caught all relevant papers. for instance, one clear problem with using the acm digital library is that many nlp conferences are not included. future work should investigate more sources for papers, and look further into the archive. additionally, we did not include commercial or open source writing tools that exist outside of the academy, which likely would improve the findings of any large-scale, systematic review of writing support tools. there are also many more questions that could be asked about writing support tools. for instance, we found that user type was not widely reported, but user type may be implied by the writing task, or inferred by the evaluation methodology. relatedly, further analysis could be done on how much work is dedicated to fiction v. nonfiction or short v. longer writing. we hope that by making our selected papers easily accessible, others may use this to do their own investigations with other focuses. ## conclusion we present a design space for writing support tools based on a cognitive process model of writing. we perform a systematic literature review, reviewing 30 papers from the last five years @xcite @xcite @xcite . we find that highly constrained planning and reviewing are under-studied areas. we see that evaluation methodologies vary widely, and propose validated surveys and interaction measures as ways to make evaluations more comparable across systems. we also propose three shared tasks-storytelling, argumentative writing, and personal essays-to aid in propelling work on writing support tools forward. shaomei wu, lindsay reynolds, xian li, and francisco guzmán. 2019. design and evaluation of a social media writing support tool for people with dyslexia. in proceedings of the 2019 chi conference on human factors in computing systems, chi '19, page 1-14, new york, ny, usa. association for computing machinery.