Before revision:  To alleviate these problems, we frame text generation as a reinforcement learning (RL) problem with expert demonstrations (i.e., the training data ), where the goal is to maximize quality given model-generated histories.
After revision:  To alleviate these problems, we frame text generation as an offline reinforcement learning (RL) problem with expert demonstrations (i.e., the training data ), where the goal is to maximize quality given model-generated histories.