  For unsupervised image-to-image translation, we propose a discriminator
architecture which focuses on the statistical features instead of individual
patches. The network is stabilized by distribution matching of key statistical
features at multiple scales. Unlike the existing methods which impose more and
more constraints on the generator, our method facilitates the shape deformation
and enhances the fine details with a greatly simplified framework. We show that
the proposed method outperforms the existing state-of-the-art models in various
challenging applications including selfie-to-anime, male-to-female and glasses
removal. The code will be made publicly available.
