Sparks: Inspiration for Science Writing using Language Models (2022)

Abstract: Large-scale language models are rapidly improving, performing well on a variety of tasks with little to no customization. In this work we investigate how language models can support science writing, a challenging writing task that is both open-ended and highly constrained. We present a system for generating “sparks”, sentences related to a scientific concept intended to inspire writers. We run a user study with 13 STEM graduate students and find three main use cases of sparks—inspiration, translation, and perspective—each of which correlates with a unique interaction pattern. We also find that while participants were more likely to select higher quality sparks, the overall quality of sparks seen by a given participant did not correlate with their satisfaction with the tool.

## introduction new developments in large-scale language models have produced models that are capable of generating coherent, convincing text in a wide variety of domains @xcite @xcite . their success has spurred improvements on many tasks, from classification and summarization @xcite to creative writing support @xcite . these improvements demonstrate that language models have the potential to support writers in real-world, highimpact domains. despite their successes, language models continue to exhibit known problems, such as generic outputs @xcite , lack of diversity in their outputs @xcite , and factually false or contradictory information @xcite . additionally, there remain many unknowns about how this technology will interface with people in real-world writing tasks, such as how language models can best contribute to different writ-ing forms @xcite and how to mitigate the bias that language models encode @xcite . in this work we study how language models can be applied to a real-world, high-impact writing task: science writing. this introduces challenges different to those in traditional creative writing tasks which tend to deal with common objects and relations. science writing requires a system to demonstrate proficiency within an area of expertise. we pose the following research question: how can language model outputs support writers in a creative but constrained writing task? as a test-bed, we use a science writing form called "tweetorials" @xcite . tweetorials are short, technical explanations of around 500 words written on twitter for a general audience; they have a low-barrier to entry and are gaining popularity as a science writing form @xcite . we present a system that aims to inspire writers when writing tweetorials on a topic of their expertise. this system provides what we call "sparks": sentences generated with a language model intended to spark ideas in the writer. we report on a study in which we have 13 graduate students from five stem disciplines write tweetorials with our system and report on how they thought about and made use of the sparks. we make the following contributions: