Before revision:  To alleviate these problems, we frame text generation as a reinforcement learning (RL) problem with expert demonstrations (i.e., the training data ), where the goal is to maximize quality given model-generated histories.
After revision:  To alleviate these problems, we frame text generation as a reinforcement learning (RL) problem with expert demonstrations (i.e., the reference ), where the goal is to maximize quality given model-generated histories.