Before revision:  This value function is learned through off-policy reinforcement learning, using trajectory optimization as its behavior policy.
After revision:  We investigate various instantiations of this framework and demonstrate its benefit in three settings: online reinforcement learning, using trajectory optimization as its behavior policy.