  Paraphrase generation plays an essential role in natural language process
(NLP), and it has many downstream applications. However, training supervised
paraphrase models requires many annotated paraphrase pairs, which are usually
costly to obtain. On the other hand, the paraphrases generated by existing
unsupervised approaches are usually syntactically similar to the source
sentences and are limited in diversity. In this paper, we demonstrate that it
is possible to generate syntactically various paraphrases without the need for
annotated paraphrase pairs. We propose Syntactically controlled Paraphrase
Generator (SynPG), an encoder-decoder based model that learns to disentangle
the semantics and the syntax of a sentence from a collection of unannotated
texts. The disentanglement enables SynPG to control the syntax of output
paraphrases by manipulating the embedding in the syntactic space. Extensive
experiments using automatic metrics and human evaluation show that SynPG
performs better syntactic control than unsupervised baselines, while the
quality of the generated paraphrases is competitive. We also demonstrate that
the performance of SynPG is competitive or even better than supervised models
when the unannotated data is large. Finally, we show that the syntactically
controlled paraphrases generated by SynPG can be utilized for data augmentation
to improve the robustness of NLP models.
