Paper:
Domain transfer is an exciting and challenging branch of machine learning because models must learn to smoothly transfer between domains, preserving local variations and capturing many aspects of variation without labels. However, most successful applications to date require the two domains to be closely related (e.g., image-to-image, video-video), utilizing similar or shared networks to transform domain-specific properties like texture, coloring, and line shapes. Here, we demonstrate that it is possible to transfer across modalities (e.g., image-to-audio) by first abstracting the data with latent generative models and then learning transformations between latent spaces. We find that a simple variational autoencoder is able to learn a shared latent space to bridge between two generative models in an unsupervised fashion, and even between different types of models (e.g., variational autoencoder and a generative adversarial network). We can further impose desired semantic alignment of attributes with a linear classifier in the shared latent space. Through qualitative and quantitative evaluations, we demonstrate that the proposed model preserves both locality and semantic alignment through the transfer process. Finally, the hierarchical structure decouples the cost of training the base generative models and semantic alignments, enabling computationally efficient and data efficient retraining of personalized mapping functions.

Review:
In this paper, the authors have proposed a cross domain transferring methods, supervised by three category of losses. The experiments somewhat demonstrate the effective of this method. However, this paper still suffers from some drawbacks as below:
The paper is not well-organized, the structure of the paper need improving. For example, the related work is put almost at the end of the paper and the tables and figures are hard to follow sometimes.
The technical implementation of the proposition is somewhat trivial. Why the generative model should be pre-trained. Why not try in the end-to-end way. 
The experiments are not convincing. The authors argue that CycleGAN suffers from some drawback. Why do not the authors compare with CycleGAN in this paper? By the way, the authors also need to compare with more state-of-the-art methods, such as StarGAN.
Some implementation details are not clearly stated. For example, the authors say “Our goal can thus be stated as learning transformations that preserve locality and semantic alignment, while requiring as few labels from a user as possible.” So, how many labeled samples are used in Table 2?