  Large Transformer-based language models can aid human authors by suggesting
plausible continuations of text written so far. However, current interactive
writing assistants do not allow authors to guide text generation in desired
topical directions. To address this limitation, we design a framework that
displays multiple candidate upcoming topics, of which a user can select a
subset to guide the generation. Our framework consists of two components: (1) a
method that produces a set of candidate topics by predicting the centers of
word clusters in the possible continuations, and (2) a text generation model
whose output adheres to the chosen topics. The training of both components is
self-supervised, using only unlabeled text. Our experiments demonstrate that
our topic options are better than those of standard clustering approaches, and
our framework often generates fluent sentences related to the chosen topics, as
judged by automated metrics and crowdsourced workers.
