# Image-to-Image Translation Using a Cross-Domain Auto-Encoder and Decoder
## 3 Method  
### Feature Space Mapping  
We think that the decoder part can have the same significant effect as the encoder part and propose to use not only the pre-trained encoder but also a pre-trained decoder for the image-to-image translation task. We can consider an auto-encoder that has pre-trained feature extraction and reconstruction for the domain X. The encoder in this network, \(\text{En}_{\text{x}}\), is responsible for representing the domain X to the latent space; \(\text{En}_{\text{x}}\): \(X\to Z_{\text{x}}\). Contrarily, the decoder \(\text{De}_{\text{x}}\) is responsible for representing the latent space to the domain X; \(\text{De}_{\text{x}}\): \(Z_{\text{x}}\to X\) (Figure 2). Our idea is using a \"cross-connection\" of two pairs of auto-encoders and decoders \(\text{En}_{\text{x}}-\text{De}_{\text{x}}\) and \(\text{En}_{\text{y}}-\text{De}_{\text{y}}\) for a generator of a GAN to improve image-to-image translation results. For example, we can connect \(\text{En}_{\text{x}}\) and \(\text{De}_{\text{y}}\) to make a generator to learn translation from domain X to domain Y. Since the encoder \(\text{En}_{\text{x}}\) has been pre-trained to represent the domain X to the latent space, it learns to extract the feature of domain X effectively in the image-to-image translation learning phase, and the decoder \(\text{De}_{\text{y}}\) also learns to reconstruct domain Y from the latent space while preserving common characteristics of Y with a pre-trained effect. This cross-connecting approach makes the resultsof GAN more stable and photorealistic. Recently, like UNIT, image-to-image translation methods using auto encoders and decoders have been proposed. These methods try to train extracting latent space and translating domains in one training stage, resulting in some problems, such as low-quality translation, as we described in Section 4.3.2. In contrast, we think our method performs better because pre-training the auto-encoder and decoders provide more flexible advantages for the characteristics of each domains. To train auto-encoders for each domain, we follow the basic equation of the convolutional auto-encoder [11]. The representation of the k-th latent feature map extraction for input I is denoted as:  
\[\mathrm{h^{k}}=\sigma\,(\mathrm{I}\,^{*}\,\mathrm{W^{k}}+\mathrm{b^{k}}), \tag{1}\]  
where sigma is an activation function, \(\mathrm{W_{k}}\) is a weight, and \(\mathrm{bk}\) is a bias, and \({}^{*}\) denotes the 2D convolution operation. The reconstruction of latent representation is obtained using  
\[\mathrm{y}=\sigma\,(\int_{\mathrm{k}\in\mathrm{H}}\mathrm{h^{k}}\,{}^{*}\, \mathrm{\hat{W}}+\mathrm{c}), \tag{2}\]  
where \(\mathrm{c}\) is a bias for input channel. \(\mathrm{H}\) identifies the list of feature maps and \(\mathrm{\hat{W}}\) represents the 2D transposing convolution operation. Mean absolute error (MAE) or mean squared error (MSE) is used for a reconstruction loss between an output of the autoencoder and a ground-truth. In addition to this, we construct an additional layer to combine two auto-encoder and decoder pairs. This layer we add learns to fill the \"gap\" between the space \(Z_{\mathrm{x}}\) and \(Z_{\mathrm{y}}\). As we mentioned earlier, \(\mathrm{En_{x}}\) learns to represent X relative to the latent space \(Z_{\mathrm{x}}\), and \(\mathrm{De_{y}}\) reconstructs Y from space \(Z_{\mathrm{y}}\), so it does not guarantee that \(Z_{\mathrm{x}}\) and \(Z_{\mathrm{y}}\), which are independently trained results of two different auto-encoders, lie in the same dimensional space. The added layer fills this gap to learn how to map two latent spaces properly. After the cross-connection is complete (Figure 3), the entire network is fine-tuned by the learning method according to the model used to apply the cross-connected auto-encoder and decoder. Figure 3: The concept of cross-connecting pre-trained autoencoders and decoders for feature space transfer (including a feature mapping layer).