# Cyclic Consistent Image Style Transformation: From Model to System
## 5 Experiment
### Metric Comparison against SOTA Methods  
In this section, we delve into the performance of various image style transfer systems, focusing on key metrics such as Per-pixel Accuracy (PixelAcc), Per-class Accuracy (ClsAcc), and Mean Intersection over Union (mIoU). These metrics offer a comprehensive view of how well each model performs in translating image styles while preserving the semantic integrity of the content.  
Pixel Accuracy (PixelAccuracy). Pixel Accuracy measures the proportion of correctly predicted pixels for each class over the total number of pixels. It is calculated using the following formula:  
\[PixelAcc=\frac{\sum_{i}n_{ii}}{\sum_{i}t_{i}}\]  
Class Accuracy (ClsAcc). Class Accuracy provides the average accuracy across all classes. It is calculated as follows:  
\[ClsAcc=\frac{PixelAcc}{n_{cls}}\]  
\begin{table}
\begin{tabular}{c c c} \hline \hline  & **Map–Photo** & **Photo–Map** \\ \hline Method & Believed it to be the true & Believed it to be the true \\ CoGAN & 16.65\% & 15.12\% \\ SimGAN & 16.68\% & 18.56\% \\ BiGAN/ALI & 18.28\% & 17.95\% \\ L1+CNN & 20.88\% & 20.24\% \\ C3 GAN & **24.63\%** & **22.90\%** \\ \hline \hline \end{tabular}
\end{table}
Table 1: Summary of the scoring results of the same group of volunteers for the same picture.  
Mean Intersection over Union (mIoU). Mean IoU is a widely used metric in image segmentation that measures the overlap between the predicted segmentation and the ground truth. It is calculated as follows:  
\[mIoU=\frac{1}{n_{cls}}\sum_{i}\frac{n_{ii}}{t_{i}+\sum_{j}(n_{ji}-n_{ii})}\]  
where \(n_{ii}\) is the number of pixels predicted as class \(i\) that are actually class \(i\), \(t_{i}\) is the total number of pixels in class \(i\), \(n_{cls}\) is the number of classes, and \(n_{ji}\) is the number of pixels predicted as class \(i\) but are actually class \(j\).  
Our C3 GAN model, enhanced with cyclic consistency loss, demonstrates clear superiority across all evaluated metrics, highlighting its effectiveness in producing high-quality image style transfers. As shown in Table 2, C3 GAN achieved a remarkable PixelAcc of 0.76, outperforming all other methods, including the well-established Pix2 Pix baseline, which scored 0.62. This substantial improvement in PixelAcc indicates that C3 GAN excels in accurately predicting the pixel-level content of the transferred images.  
When examining ClsAcc, which reflects the average accuracy across all classes, C3 GAN again takes the lead with a score of 0.28. This is a notable enhancement over other methods like L1+cGAN and Pix2 Pix, which scored 0.22 and 0.21, respectively. The improved ClsAcc underscores C3 GAN's ability to maintain high accuracy across different classes within the images, suggesting that the model is particularly effective at preserving class-specific details during the style transfer process.  
The mIoU metric, which measures the overlap between predicted segmentations and ground truth, further demonstrates C3 GAN's strength. With an mIoU of 0.19, C3 GAN surpasses other approaches such as cGAN and L1+cGAN, both of which scored 0.16. This improvement indicates that C3 GAN not only preserves the overall structure of the images but also ensures a better match between the generated content and the actual semantic layout, leading to more coherent and contextually accurate style transfers.  
The consistent performance across these metrics can be attributed to C3 GAN's use of cyclic consistency, which effectively mitigates common issues in GAN-based models, such as mode collapse and training instability. By ensuring that the generated image can be accurately translated back to the original semantic tags, C3 GAN achieves higher fidelity in the style transfer process.