# Review of Image Augmentation Used in Deep Learning-Based Material Microscopic Image Segmentation
## 2 Method Review
### Deep Learning-Based Image Augmentation Methods
#### 2.2.2 Image Generation for Image Augmentation  
The obvious drawback of basic augmentation and AutoAugment methods is that they can only learn isolated information from a small amount of training data, rather than grasp the fundamental patterns of the data distribution in the whole data sets. However, many differences have been created by using basic image augmentation methods [50].  
Since 2014, Generative Adversarial Net (GAN) [51] has gained more attention in the field of deep learning. In this case, [PERSON] et al. [52] used a convolutional neural network (CNN) combined with GAN to obtain deep convolution generative adversarial nets (DCGAN), which led to the explosion of realistic image generation. An innovative concept for image data augmentation is the use of synthetic images to boost the training data. Theoretically, synthetic images can be generated that closely resemble the distribution of real image data by using adversarial training [53]. The issue of overfitting caused by a lack of training samples is anticipated to be resolved. GAN has the ability to efficiently and rapidly generate unreal images that meet specific requirements and seamlessly integrate them with real training data to augment the data set.  
The issue with generating image inputs in GAN is that synthetic images are randomly sampled from the noise space, which cannot produce that are related to real images. In other words, there is no user control over the generation process. Conditional generative adversarial nets(CGAN) was proposed by [PERSON] and [PERSON] et al. [54] with real images as network input. [PERSON] et al. [55] in 2017 proposed the first general image transformation CGAN framework, Pix2 Pix, which uses aligned input images to learn the mapping relationship between real and synthetic images. Later, [PERSON] et al. [56] proposed cycle-consistent adversarial networks (CycleGAN), which eliminates the challenge of requiring precisely paired training samples in pix2 Pix and enables training with unpaired samples. CycleGAN expands the potential use of GAN in multiple domains. [PERSON] et al. [57] suggest that it is possible to supplement the segmentation task training with images generated by the Pix2 PixHD [58]. The output can be manually reconstructed, semantically segmented, labeled images.  
For the collaborative creation of paired images and masks for the image segmentation challenge, [PERSON] et al. [59] proposed a two-stage GAN framework. Instead of combining synthetic and real images for model training, [PERSON] and [PERSON] et al. [60] proposed a novel approach that enhances the performance of downstream MRI brain image segmentation tasks. A task-guided branching was integrated into the end of the generator with a learned low-dimensional latent space. To face the Unsupervised Part Segmentation challenge, [PERSON] et al. [61] used GANSeg to generate images conditional on potential masks. The masked images were synthesized without a limit on the number of masks that could be used to supplement training data. [PERSON] [13] et al. proposed a style transfer-based image augmentation technique that would migrate the styles of actual images to simulated images to generate synthetic images as an expansion of the data set, creating a bridge for the simulated data-assisted predictive analysis of real data.  
However, none of the above methods can also produce images consistent with their underlying structural information. Furthermore, training GAN-based image augmentation methods also needs an amount of data. As a result, the lack of sufficient data hinders the development of deep learning-based data augmentation methods.