# Bootstrapped SSL CycleGAN for Asymmetric Domain Transfer
## 4 Elements of the Adopted Experimental Setup
### Network Architecture and Training Details  
For the purpose of experimental comparisons we have implemented the corresponding neural networks based on the best practices from the literature. Since the proposed semi-supervised learning is generally applicable to any variant of the original CycleGAN architecture, we have decided to rely on the referent implementations form the papers were the corresponding methods were originally introduced.  
Thus, for the main neural network architecture of the corresponding GANs we have adopted the architecture proposed in [17], where the CycleGAN model was originally proposed, and which was also utilized in [19]. This implementation of unsupervised domain translation was also the basis for implementation of the SSL+BTS method proposed in this work. In addition, since the original CycleGAN was introduced in the context of image-to-image translation tasks, the same type of domain transfer tasks was also chosen for the experimental comparisons in this work. One of the advantages of this type of experiments is also that they provide visual insights into the quality of obtained domain transfer results.  
The main characteristics of the adopted architecture are that it contains two stride-2 convolutions, as well as several residual blocks and 2 fractionally strided convolutions with stride \(\frac{1}{2}\). It uses 6 blocks for \(128\times 128\) and 9 blocks for \(256\times 256\) and higher-resolution type images. It also uses instance normalization as in [17; 23; 24].  
For the discriminator network, we have relied on \(70\times 70\) PatchGAN in order to classify whether the image patches are real or fake, as it was more efficient than the usual full-image network.  
When it comes to training details, besides the main algorithmic steps of the performed training procedures, which have already been described in detail in Sections 2.2 and 3, some of the specific settings and values of control parameters that were used in the experiments include the following. In the training procedure, as in [17], instead of negative log likelihood loss figuring in (1), we used more stable \(L_{2}\) loss approach consisting of training \(G\) to minimize \(E_{z\sim p(z)}[(D(G(z))-1)^{2}]\) as well as training discriminator \(D\) to minimize \(E_{x\sim p(x)}[(D(x)-1)^{2}]+E_{z\sim p(z)}[D(G(z))]\). Furthermore, similarly as in [17], we use the history of generated images (50 of them) in order to calculate the average score. We also keep \(\lambda_{cyc}=\lambda_{SSL}=10\), for all of our experiments. Sizes of minibatches \(m\) and \(m_{SSL}\) are kept equal and set to \(m=m_{SSL}=50\). Random initialization of the network parameterswas performed by using random samples drawn from normal distribution \(\mathcal{N}(0,0.02)\). All networks were trained using the learning rate of \(\eta=0.0002\) which is kept constant during the first 100 epochs and then linearly decays to zero during the next 100 epochs.  
In experiments with all used datasets we have controlled the size of the scarce domain, by changing the percentage \(\mathcal{S}_{X}\) of the domain used: 25%, 50% and 100%, and performed both the unsupervised, as well as the proposed SSL approach (loss function (12)), with and without the proposed bootstrapping of the statistic of the discriminator \(D_{X}\) from the scarce domain (variants SSL+BTS, and SSL). In all experiments, as all datasets used actually contain paired images, we use a fixed 20% of the scarce domain training paired examples for evaluation in the SSL manner. In each experiment, after initial \(K_{0}\) iterations, during the next \(k+K\)-th training iteration, where \(K=50\), 20% of randomly chosen examples from the target domain is transformed by \(G_{Y\to X}\) and added to the pool of the discriminator \(D_{X}\).