Learning a neural response generation model on data synthesized under the adversarial training framework helps to explore more possible responses. However, most of the data synthesized de novo are of low quality due to the vast size of the response space . In this paper, we propose a counterfactual off-policy method to learn on a better synthesis of data. It takes advantage of a real responseto infer an alternative that was not takenusing a structural casual model. Learning on the counterfactual responses  helps to explore the high-reward area of the  response space. An empirical study on the DailyDialog dataset shows that our approach significantly outperforms the HRED model as well as the conventional adversarial training approaches.
Open-domain dialogue generation suffers from the data insufficiency problem due to the vast size of potential responses . In this paper, we propose to explore potential responses by counterfactual reasoning. Given an observed response, the counterfactual reasoning model automatically infers the outcome of an alternative policy that could have been taken. The resulting counterfactual response synthesized in hindsight is of higher quality than the response synthesized from scratch. Training on the counterfactual responses under the adversarial learning framework helps to explore the high-reward area of the potential response space. An empirical study on the DailyDialog dataset shows that our approach significantly outperforms the HRED model as well as the conventional adversarial learning approaches.
R
Learning a neural response generation model on data synthesized under the adversarial training framework helps to explore more possible responses. However, most of the data synthesized de novo are of low quality
Open-domain dialogue generation suffers from the data insufficiency problem
R
the response space
potential responses
R
a counterfactual off-policy method to learn on a better synthesis of data. It takes advantage of a real responseto infer an alternative that was not takenusing a structural casual model. Learning
to explore potential responses by counterfactual reasoning. Given an observed response, the counterfactual reasoning model automatically infers the outcome of an alternative policy that could have been taken. The resulting counterfactual response synthesized in hindsight is of higher quality than the response synthesized from scratch. Training
A
under the adversarial learning framework
A
potential
R
training
learning