Plug-and-Play Recipe Generation with Content Planning (2022)

Abstract: Recent pre-trained language models have shown promising capability to generate fluent and realistic natural text. However, generating multi-sentence text with global content planning has been a long-existing research question. The current controlled text generation models cannot directly address this issue, as they usually condition on single known control attribute. We propose a low-cost yet effective framework that explicitly models content plans and optimizes the joint distribution of the natural sequence and the content plans in a plug-and-play post-processing manner. We evaluate our model with extensive automatic metrics and human evaluations and show that it achieves the state-of-the-art performance on the recipe generation task on Recipe1M+ dataset.

## introduction recent progress in large-scale language model pretraining has facilitated significant improvement in generating increasingly realistic natural language text. although this has been achieved on the surface-level fluency, it has been pointed that generating multi-sentence text with global constraints, or long-term planning is still far from being solved. typical examples of such task include story continuation with logical coherency @xcite , and recipe generation with stepby-step planning @xcite . as suggested by @xcite , the aforementioned issues cannot be ameliorated by simply increasing the size of model parameters or the scale of pre-training data. adding to this, current approaches for controlled text generation cannot directly tackle those issues either. for example, ctrl @xcite , which trains a classconditional language model, and pplm @xcite , which re-ranks the language model predictions by an attribute model. they usually share a common setup of optimizing conditional distributions p (y|a), where y is the text sequence and a is the desired single control attribute. examples of control attribute include sentiment @xcite , topic @xcite and formality @xcite . however, content planning requires controlling with consideration of global context, which is more sophisticated than the single control attributes. therefore, we identify the research gap for the current controlled text generation models to generate multi-sentence text with long-term content planning. motivated by previous research in cognitive science @xcite , @xcite pointed out that the reasoning of a neural-based model should consist of two systems, i.e. the system 1 makes intuitive and associative responses, and the system 2 makes deliberative and logical decisions. with greatly increased capabilities, large language models have become sufficiently competent to act as the system 1. however, we argue that, to address the aforementioned research gap, it is vital to empower the language models with the ability to make logical decisions, i.e. predict content plans. in contrast to the existing methods that optimize the conditional distributions, we propose a novel framework which explicitly models the content plan c and optimizes the joint distribution p (y, c) in a plug-and-play manner. figure 1 depicts an overview of our approach. specifically, our proposed framework consists of (i) a content planner which predicts the global content plan of the output text; and (ii) a sequence generator, based on pretrained language models, that generates the output following the content plan. the predicted content plan steers the generation of the sequence generator through a lightweight and plug-and-play style plan classifier. it worth emphasizing that the sequence generator does not need to be trained with planspecific data, which means adapting our framework to other natural language generation (nlg) tasks is cheap and efficient. we comprehensively evaluate our approach on the recipe generation task using the widely-used recipe1m+ benchmark @xcite . the experimental results demonstrate that our approach significantly outperforms previous state-of-the-art (sota) as judged by both automatic and human evaluations. in particular, the results show that the recipes generated by our model are more accurate and highly controllable. in summary, we conclude our contributions as two-fold: firstly, we identify the current research gap and propose a novel framework that generates text with content plans in a plug-and-play manner. secondly, we conduct extensive experiments to show that our framework achieves sota performance on the widely-used recipe1m+ benchmark. ## experiments in this section, we evaluate our method from three aspects: (i) the performance of planner module; we pre-processed the recipe1m+ dataset by firstly filtering out instructions that contain less than 3 words, e.g. 'combine all', as they are usually trivial. we also truncate recipes with too many instructions at the length of 15, because recipes with too many instructions usually include irrelevant information due to data scraping errors. by this way, about 9% of the original recipe1m+ are filtered out and the resulting dataset are used in our experiments. ## conclusion in this work, we first identify the research gap of the current controlled text generation models to generate text with sentence-level content planning. then we propose a framework that optimizes the joint distribution of the natural language sequence and the content plans in a lightweight as well as plugand-play manner. extensive automatic and human evaluations demonstrate that our model achieves a new state of the art on the recipe generation task and outperforms previous studies by significant margins. lastly, we show that our model can generate recipes that are more accurate and controllable by following the guidance of explicit content plans.