# Conditional Generative Adversarial Networks for Domain Transfer: A Survey
## 3 The Evaluation Method of CGAN
### Automatic Evaluation  
Automatic evaluation refers to the evaluation index that can be calculated directly by using the formula without human participation. Some indexes in the original GAN, such as Inception Score (IS) [56] and Frechet Inception Distance (FID) [57], can also be used to evaluate CGAN to measure the fidelity of the generated images [23; 45; 58; 59; 60]. Compared with the classical GAN, for CGAN, the expected image is usually given in some task, and the similarity between the generated image and the expected image is also an important index, such as the Target Attributes Recognition Rate (TARR), Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) [61; 62; 41].  
As noted, CGAN's task is to generate correct images rather than only arbitrary images. We have also mentioned the diversity of the CGAN loss function in Section 2.2.1, and each loss function corresponds to whether the generated image meets the requirements in some aspects. For example, category loss is used to monitor whether the generated image conforms to the category specified in the input conditions. Therefore, the classification accuracy of the generated image can also be used as an evaluation index for CGAN. The hypothesis that the classifier trained on the real image can be well applied to the classification of the generated image once the quality of the generated image is good enough is employed here. For example, the indicator Dice introduced by [35; 63] uses the pre-trained image segmentation model, FCN-8s, to segment the generated image and then calculates the distance between the segmentation result and the original semantic mask. The principles of [58; 61] are similar. In addition to the discriminant method developed by category loss, reference [64] calculates the average distance of a group of random samples in the feature space, and also the Cosine Similarity (CSIM) of embedded vectors proposed in [65] has a similar effect.  
For some cross-modal tasks, a more indirect approach is often required. For example, reference [45] mainly solves the task of converting text to image using R-precision. First, it needs 100 candidate texts, including \(R\) ground truth text and \(100-R\) random text. Then, it queries these candidate texts according to the generated images and counts the number of texts related to the generated images in the previous \(R\) text, that is \(r\), and the value \(r/R\) is namely the R-precision.  
The advantage of automatic evaluation is that it does not need manual intervention and can save on manpower. In addition, it can derive quantitative metrics for the quality of the generated data. However, its disadvantages are also obvious. On the one hand, due to the complexity of data generated, the evaluation content is diverse, and there is no absolutely reasonable index. For example, IS can reflect the proximity of the distribution between the generated data and the real data, but it cannot monitor the problem of overfitting. When the model can only generate data from the training set but cannot create data, IS may still remain at a high level. If one application relies too much on automatic evaluation, the problem of disconnection between the evaluation metrics and the actual effect will occur. On the other hand, as many evaluation metrics are involved in the pre-training model, the uncertainty of evaluation results will also increase.