# Bootstrapped SSL CycleGAN for Asymmetric Domain Transfer
## 4 Elements of the Adopted Experimental Setup
### Utilized Measures for Results Comparison  
Evaluating the quality of synthesized images is an open and difficult problem. We use some classical measures such as Peak Signal-to-Noise Ratio (PSNR) as well as more advanced Structural Similarity Index Measure (SSIM), which is a perception-based model that considers image degradation as perceived change in structural information. It also incorporates important perceptual phenomena, including both luminance masking and contrast masking terms. The SSIM measure is much more appropriate for measuring image degradation than PSNR.  
The PSNR is evaluated as \(PSNR=20 log\left(MAX_{I}/\sqrt{MSE}\right)\), where \(MAX_{I}\) is the maximum possible pixel value of the ground truth images, while \(MSE\) is the squared Euclidean norm between the generated and ground truth images. The SSIM measure between images generated by the considered GAN algorithms and ground truth images is calculated on various windows \(x,y\) of an image, by using the following formula:  
\[SSIM(x,y)=\frac{(2\mu_{x}\mu_{y}+c_{1})(2\sigma_{xy}+c_{2})}{(\mu_{x}^{2}+\mu_ {y}^{2}+c_{1})(\sigma_{x}^{2}+\sigma_{y}^{2}+c_{2})} \tag{13}\]  
where \(\mu_{x}\) and \(\mu_{y}\) are average values of \(x\) and \(y\), while \(\sigma_{x}^{2}\) and \(\sigma_{y}^{2}\) are variances and \(c_{1}\) and \(c_{2}\) are constants, set as reported in [27].  
In addition to the mentioned classical image quality measures, we have also used some more specific measures, such as measures based on human judgments, which we call Perceptual Realistic Measure (PRM), as well as the FCN score measure proposed in [28], which is based on pre-trained semantic classifiers and measures how discriminative the generated output is. We then score the synthesized photos using the classification accuracy against the labels these photos were synthesized from There are various measures of FCN accuracy used in [28]: pixel accuracy evaluated as \(\sum_{i}n_{ii}/\sum_{i}t_{i}\), mean accuracy evaluated as \((1/n_{cl})\sum_{i}(\sum_{i}n_{ii}/t_{i})\) and mean region intersection over union accuracy evaluated as \((1/n_{cl})\sum_{i}n_{ii}/\left(t_{i}+\sum_{j}n_{ji}-n_{ii}\right)\). Term \(n_{ij}\) is the number of pixels of class \(i\) predicted to belong to class \(j\), \(n_{cl}\) is the number of classes and \(t_{i}=\sum_{j}n_{ij}\) is the total number of pixels belonging to the class \(i\).  
Concerning the PRM measure, we have asked student participants (10 of them) at the Faculty of Technical Sciences, University of Novi Sad, to evaluate the perceptual quality of images generated by the proposed method in comparison to baseline methods. We note that for us it was much more accessible to organize our own pool of human participants, than to use some internet service such as AMT perceptual studies. In the Map\(\leftrightarrow\) aerial photo experiments conducted on _Google Maps_, we follow the procedure reported in [17].  
The participants were shown pairs of images, each pair containing one real image (i.e., the ground truth) and one synthesized, generated by the proposed as well as baseline algorithms. Next, they were asked to choose an image they thought it was real. The first 5 trials of each session were conducted for practice and a feedback was given whether the participant's response was correct or incorrect. The remaining 15 trials were used to assess the rate at which each algorithm fooled the participants.  
In addition, the relative improvement of the proposed method against the CycleGAN baseline, denoted by \(\delta\), was computed for each of the described measures separately.