iFlowGAN: An Invertible Flow-Based Generative Adversarial Network for Unsupervised Image-to-Image Translation

We propose iFlowGAN that learns an <italic>invertible flow</italic> (a sequence of invertible mappings) via <italic>adversarial learning</italic> and exploit it to transform a source distribution into a target distribution for <italic>unsupervised image-to-image translation</italic>. Existing GAN-based generative model such as CycleGAN [1], StarGAN [2], AGGAN [3] and CyCADA [4] needs to learn a highly under-constraint forward mapping <inline-formula><tex-math notation="LaTeX">$\mathcal {F}: X \rightarrow Y$</tex-math><alternatives><mml:math><mml:mrow><mml:mi mathvariant="script">F</mml:mi><mml:mo>:</mml:mo><mml:mi>X</mml:mi><mml:mo>→</mml:mo><mml:mi>Y</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="tang-ieq1-3062849.gif"/></alternatives></inline-formula> from a source domain <inline-formula><tex-math notation="LaTeX">$X$</tex-math><alternatives><mml:math><mml:mi>X</mml:mi></mml:math><inline-graphic xlink:href="tang-ieq2-3062849.gif"/></alternatives></inline-formula> to a target domain <inline-formula><tex-math notation="LaTeX">$Y$</tex-math><alternatives><mml:math><mml:mi>Y</mml:mi></mml:math><inline-graphic xlink:href="tang-ieq3-3062849.gif"/></alternatives></inline-formula>. Researchers do this by assuming there is a backward mapping <inline-formula><tex-math notation="LaTeX">$\mathcal {B}: Y \rightarrow X$</tex-math><alternatives><mml:math><mml:mrow><mml:mi mathvariant="script">B</mml:mi><mml:mo>:</mml:mo><mml:mi>Y</mml:mi><mml:mo>→</mml:mo><mml:mi>X</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="tang-ieq4-3062849.gif"/></alternatives></inline-formula> such that <inline-formula><tex-math notation="LaTeX">$\boldsymbol{x}$</tex-math><alternatives><mml:math><mml:mi mathvariant="bold">x</mml:mi></mml:math><inline-graphic xlink:href="tang-ieq5-3062849.gif"/></alternatives></inline-formula> and <inline-formula><tex-math notation="LaTeX">$\boldsymbol{y}$</tex-math><alternatives><mml:math><mml:mi mathvariant="bold">y</mml:mi></mml:math><inline-graphic xlink:href="tang-ieq6-3062849.gif"/></alternatives></inline-formula> are fixed points of the composite functions <inline-formula><tex-math notation="LaTeX">$\mathcal {B} \circ \mathcal {F}$</tex-math><alternatives><mml:math><mml:mrow><mml:mi mathvariant="script">B</mml:mi><mml:mo>∘</mml:mo><mml:mi mathvariant="script">F</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="tang-ieq7-3062849.gif"/></alternatives></inline-formula> and <inline-formula><tex-math notation="LaTeX">$\mathcal {F} \circ \mathcal {B}$</tex-math><alternatives><mml:math><mml:mrow><mml:mi mathvariant="script">F</mml:mi><mml:mo>∘</mml:mo><mml:mi mathvariant="script">B</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="tang-ieq8-3062849.gif"/></alternatives></inline-formula>. Inspired by zero-order reverse filtering [5], we (1) understand <inline-formula><tex-math notation="LaTeX">$\mathcal {F}$</tex-math><alternatives><mml:math><mml:mi mathvariant="script">F</mml:mi></mml:math><inline-graphic xlink:href="tang-ieq9-3062849.gif"/></alternatives></inline-formula> via contraction mappings on a metric space; (2) provide a simple yet effective algorithm to present <inline-formula><tex-math notation="LaTeX">$\mathcal {B}$</tex-math><alternatives><mml:math><mml:mi mathvariant="script">B</mml:mi></mml:math><inline-graphic xlink:href="tang-ieq10-3062849.gif"/></alternatives></inline-formula> via the parameters of <inline-formula><tex-math notation="LaTeX">$\mathcal {F}$</tex-math><alternatives><mml:math><mml:mi mathvariant="script">F</mml:mi></mml:math><inline-graphic xlink:href="tang-ieq11-3062849.gif"/></alternatives></inline-formula> in light of Banach fixed point theorem; (3) provide a Lipschitz-regularized network which indicates a general approach to compose the inverse for arbitrary Lipschitz-regularized networks via Banach fixed point theorem. This network is useful for image-to-image translation tasks because it could save the memory for the weights of <inline-formula><tex-math notation="LaTeX">$\mathcal {B}$</tex-math><alternatives><mml:math><mml:mi mathvariant="script">B</mml:mi></mml:math><inline-graphic xlink:href="tang-ieq12-3062849.gif"/></alternatives></inline-formula>. Although memory can also be saved by directly coupling the weights of the forward and backward mappings, the performance of the image-to-image translation network degrades significantly. This explains why current GAN-based generative models including CycleGAN must take different parameters to compose the forward and backward mappings instead of employing the same weights to build both mappings. Taking advantage of the Lipschitz-regularized network, we not only build iFlowGAN to solve the redundancy shortcoming of CycleGAN but also assemble the corresponding iFlowGAN versions of StarGAN, AGGAN and CyCADA without breaking their network architectures. Extensive experiments show that the iFlowGAN version could produce comparable results of the original implementation while saving half parameters.