PaddlePaddle/plato-mini
Introduction
Pre-training models have been proved effective for a wide range of natural language processing tasks. Inspired by this, we propose a novel dialogue generation pre-training framework to support various kinds of conversations, including chit-chat, knowledge grounded dialogues, and conversational question answering. In this framework, we adopt flexible attention mechanisms to fully leverage the bi-directional context and the uni-directional characteristic of language generation. We also introduce discrete latent variables to tackle the inherent one-to-many mapping problem in response generation. Two reciprocal tasks of response generation and latent act recognition are designed and carried out simultaneously within a shared network. Comprehensive experiments on three publicly available datasets verify the effectiveness and superiority of the proposed framework.
More detail: https://arxiv.org/abs/1910.07931
Available Models
- plato-mini, 6 layer, 12 heads, 768 hidden size
How to Use?
Click on the Use in paddlenlp button on the top right!
Citation Info
@article{ernie2.0,
title = {PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable},
author = {Bao, Siqi and He, Huang and Wang, Fan and Wu, Hua and Wang, Haifeng},
journal={arXiv preprint arXiv:1910.07931},
year = {2019},
}
- Downloads last month
- 12