  Providing pretrained language models with simple task descriptions or prompts
in natural language yields impressive few-shot results for a wide range of text
classification tasks when combined with gradient-based learning from examples.
In this paper, we show that the underlying idea can also be applied to text
generation tasks: We adapt Pattern-Exploiting Training (PET), a recently
proposed few-shot approach, for finetuning generative language models on text
generation tasks. On several text summarization and headline generation
datasets, our proposed variant of PET gives consistent improvements over a
strong baseline in few-shot settings.
