Generative Adversarial Networks For Data Scarcity Industrial Positron Images With Attention
Authors: Mingwei Zhu
Min Zhao
Min Yao
Ruipeng Guo
Venue:  (2020)

Abstract: In the industrial field, the positron annihilation is not affected by complex environment, and the gamma-ray photon penetration is strong, so the nondestructive detection of industrial parts can be realized. Due to the poor image quality caused by gamma-ray photon scattering, attenuation and short sampling time in positron process, we propose the idea of combining deep learning to generate positron images with good quality and clear details by adversarial nets. The structure of the paper is as follows: firstly, we encode to get the hidden vectors of medical CT images based on transfer Learning, and use PCA to extract positron image features. Secondly, we construct a positron image memory based on attention mechanism as a whole input to the adversarial nets which uses medical hidden variables as a query. Finally, we train the whole model jointly and update the input parameters until convergence. Experiments have proved the possibility of generating rare positron images for industrial non-destructive testing using countermeasure networks, and good imaging results have been achieved.

Under review as a conference paper at ICLR 2020
GENERATIVE ADVERSARIAL NETWORKS FOR DATA SCARCITY INDUSTRIAL POSITRON IMAGES WITH ATTENTION
ABSTRACT
In the industrial field, the positron annihilation is not affected by complex envi- ronment, and the gamma-ray photon penetration is strong, so the nondestructive detection of industrial parts can be realized. Due to the poor image quality caused by gamma-ray photon scattering, attenuation and short sampling time in positron process, we propose the idea of combining deep learning to generate positron im- ages with good quality and clear details by adversarial nets. The structure of the paper is as follows: firstly, encode to get the hidden vectors of medical CT im- ages based on transfer Learning, and use PCA to extract positron image features. Secondly, construct positron image memory based attention mechanism as a whole input to the adversarial nets which uses medical hidden variables as query. Finally, we train the whole model jointly and update the input param- eters until convergence. Experiments have proved the possibility of generating rare positron images for industrial non-destructive testing using countermeasure networks, and good imaging results have been achieved.
INTRODUCTION
In recent years, with the advancement of science and technology. especially the rapid developmen- t of high-end manufacturing, the field of industrial non-destructive testing, in many cases, is necessary to perform defect detection without damaging or affecting the performance and inter- nal structure of the device under test. Therefore, there is an increasing demand for corresponding detection devices.
In complex industrial environments (such as aviation, internal combustion engines, chemical engi- neering, etc.), it is of great research significance to detect faults and defects in closed chambers. In this paper, the use of positron annihilation gamma photon imaging positron emission imaging technology for industrial nondestructive testing studied. The occurrence of positron annihilation is not affected by factors such as high temperature, high pressure, corrosion, etc., so it can penetrate the dense metal material cavity, realize the undisturbed and non-destructive trace imaging of the detected object, and obtain the detected object after processing. Describe the image and perform state analysis. Therefore, the quality of imaging technology directly affects the analysis of fault detection results.
Positron Emission Tomography (PET) was first used in medical imaging. The principle is that when radioactive positron nucleus decays, proton in the nucleus is converted into a neutron, and a positron and a neutron are released. The positron will quickly combine with the electrons in the material in very short time, causing a positron annihilation phenomenon, producing pair of gamma photon pairs with opposite directions and energy of 511.. Photon pairs are collected, identified, processed, and finally reconstructed to obtain medical images.
Commonly used PET reconstruction algorithms are analytic method (K) 22000) and statistical method (Shepp & Vardi 2007). The currently widely used algorithms MLEM and OSEM. At present, PET technology has been widely used in the clinical diagnosis of human diseases. The advantages quite obvious, the imaging quality is higher, and it shows great advantages in medical research.
The principle of positron emission in industrial non dentruttee fields is similar to medical imaging, but it has its own unique difficulties: the detection environment is more harsh, the sampling time is

---

Under review as a conference paper at ICLR 2020
short, and at the same time, due to the phenomenon of scattering and attenuation of photons, indus- trial positron imaging is obtained. The image quality is even worse. Therefore, the reconstructed image needs to be further processed to obtain higher quality image.
In this paper, we propose adversarial networks of positron image memory module based on attention mechanism. Using medical images basic data sets, introducing knowledge of migration learning, building memory module according to the contribution of detail features to images, a positron image generation network in the field of industrial non-destructive testing is obtained through joint training. thus achieving higher quality generation of industrial positron images.
In summary, main contributions in this paper are as follows:
We are the first to advocate an idea of using Generative Adversarial Networks to enhance the detail of the positron image in the industrial non-destructive field, and realize the generation and processing of the scarce image data in the professional field.
We use the medical CT image dataset the basic training sample of the network framework, which is based on the idea of migration learning, and then extract the features of a small number of indus- trial non-destructively detected positron images, which can improve the details of the generated im- ages, and make the network model have better applicability in the field of industrial non-destructive testing.
We combine the attention- mechanism in the professional domain image feature extraction. By constructing a memory module containing industrial positron image features, we can generate image generation in a specific domain, and finally conduct an industrial lossless positron image generation model.
We train the whole network jointly, through the discriminant network of the antagonistic genera- tion network, the front-end network was back- prop-apopag,,ed, the input parameters were updated, and the model was optimized. Finally, the convergence was achieved and The Turing test was passed successfully.
RELATED WORK
Model of GAN: Since GAN (Goodfellow et al. 2014) was proposed in 2014, it has become search hotspot in the field of unsupervised learning, and related researches continue to increase. For the improvement of network structure, it mainly focuses on training stability and mode collapse. Radford et al Radford et al. (015) combine CNN with GAN to realize image generation, which makes GAN move from theory to practice, and establishes the framework and training mode of the whole GAN model. Arjovsky et al. M et al. (2017) propose Wasserstein GAN (WGAN), which use Earth-Mowe Mover instead of JS divergence to measure the distance between the real sample and the generated sample distribution, and the problem of pattern collapse was well solved. Mirza proposed Conditional Generative Adversarial Nets (CGAN) in 2014M & S (2014), which transformed unsu- pervised tasks into supervised tasks, thus improving the stability of training. Mao, X al X et al. (2066) propose Least Squares Generative Adversarial Networks (LSGAN), which replace the loss function in the original network with the least squares loss function, alleviating the problem of train- ing instability and insufficient diversity of generated images in GAN. Karras T et al [ et al. (2017) of NVIDIA team propose progressive structure model to realize the transition from low-resolution to high-resolution image, and the generative model of high definition image can be trained smoothly. DeepMind team (A et al., 2018) introduce the idea of orthogonal regularization into GAN, which greatly improve the generation performance of GAN by truncating the input prior distribution Z in time, and contribute lot of experience of parameter adjustment in the process of training mode. EBGAN et al. 2016) regards the discriminative network in GAN as energy function and adds reconstruction error to it, which can solve the problem of mode collapse. Unrolled GAN {L et al., 2016) modifies the loss function of the generative network so that the generator can take into account the changes of discriminator after training K times and avoid the mode collapse caused by switching between different modes. DRAGAN (N et al. 2017) introduces the "no regret algorithm" in game theory, and construct gradient penalty scheme to avoid unwanted partial equilibrium. It can solve mode collapse problem and improve training efficiency.
2

---

Under review as a conference paper at ICLR 2020
Domain Application: The greatest advantage of GAN is that it can generate real-like samples with- out any explicit modeling of data distribution in the whole generation process. Therefore, GAN has achieved good results in many fields such as image, text, voice and so on. Image generation P
et al., 2016) IT et al. 2017 (Z. et al. 2017). per resolution processing of image et al. 2017). object
detection( al. [20T8)] et al. 2017), object transfiguration(S et al., (2017) (W H| joint im-
age generation (iu & Tuzel [ video veneration(Cheve et al. 20018) (Tulyakov et al. (017). text
to image(H et al. 2217 MA et al. 2017), text generation( et al. 2018), speech conversion( al.)
2017). domain adaptation et al. 2017). and so on. These researches provide more possibilities for the practical application of GAN in the field.
Medical Image Processing: Image generation in medical field can be divided into two categories: conditional and unconditional. (M et al., (018)) & J 2017 et al. 2017) uses DCGAN (Radford et ctal.,201 2015) to realize the batch generation of realistic medical images(prostatic lesion plaques, lung cancer cakes, retina), and the resolution reaches 16*16,56*56 and 64 64 respectively, the results of which can pass the Turing test successfully, (C et al. 201 8a) uses DCGAN to learn higher resolution MR images distribution from a small number of samples, and compare them with the real images, so to achieve the effect of "false and true". (C et al. (018b) uses PGGAN to synthesize skin le- sion images, and it successfully showed amazing and highly realistic synthetic images. Conditional medical image generation studies
follows: (D et al. (017) realizes CT image synthesis from MR images by means of 3D full convolution network. The loss function of the network is composed of reconstruction loss and gradient loss, and also trained by combining additional confronta- tion network to improve the reality of the synthesized images. (P et al. 2017) compress the images of vascular tree into a multivariate normal distribution by means of the antagonistic automatic en- coder. The normal distribution is sampled to synthesize any high resolution vascular tree image, and then the end-to-end framework for high resolution retinal image synthesis obtained by using image conversion model(P et al.) (016). et al.) (017) proposes two-stage GAN, the first one of which is to synthesize vessel tree images from noise training, and the second network uses image translation framework to generate real, high resouutooo pairs of groundtruth vessel segmentation and the corresponding eye fundus image.(W et al.) 2018) uses CGAN as the framework to synthesize 200*200 PET images by using CT image and binary label graph. The researchs contribution lies in the construction of a two-channel generation network to achieve a more realistic global output and it called multi-channel GAN. (F & D 2018) sets up a multi-stage generator to obtain speckle images, low-resolution images, and high-resolution images in turn by intra-vascular ultrasound simulation of tissue maps according to different generating networks. (A & G. 2017) conducts joint learning by adding a specific task network to CGAN, and then obtain network model that retain specific task characteristics. (M al. 2018) uses WGAN as the network framework. and use noise and attribute vectors as inputs to generating high-resolution 3D images.
In addition, in the aspects of image segmentation, reconstruction, detection, denoising, recognition, classification, etc, and the use of GAN model has also achieved some research results, which pro- vides feasibility for the application of GAN in professional and specific field. Traditional positron image processing in the field of industrial nondestructive processing focuses on the improvement of image reconstruction algorithm, but due to the inherent characteristics of small sampling points and short time, the quality of the obtained image is poor, and the stability of the image affected by the field is poor. Therefore, in this paper, we innovatively introduce the knowledge of neural network for positron image processing. And through the experimental verification, a high quality image is obtained.
3 METHOD
The structure of the whole positron image countermeasure generation network based on memory module is shown in the following figure
3.1 STRUCTURE OF GAN
The original GAN Goodfellow et al. 2014) model consists of two parts: the discriminative nets and the generative nets. Through training, the generator generates pseudo-data close as possible to the real data distribution. The initial input of the network is random noise signal z, then map it to new data space by generator to get the generated data G(z), after that the discriminator outputs
3

---

Under review as a conference paper at ICLR 2020
Rupposs atteus
Medical images
Medical images
Medical images
Concidence events
encoder
encoder
encodar
MLEM
Attertion Probabllin Distribution
Postron mmggs
PCA
Feature fusion
training
Generative Adversarial Nets
Medical CT images
images
Figure 1: Positron Image Generation Flow Chart.
a probability value through binary classification real data and generated data. The confidence level indicates the performance of the generator. Through continuous iteration optimization during the whole training process, the default optimization achieved when the two sets of data cannot be distinguished.
The basic idea the GAN comes from the zero-sum game theory. The two networks against each other, in which the discriminator aims distinguish the real distribution and the generative distri- bution as far as possible, while the generator aims to make the generated data consistent with the real data the feature space and make the discriminator indistinguishable, finally the whole model reaches the optimum. The mathematical model is expressed as formula equation
- min max ((G, D) = min max Egw log + - D(G(Z))
(I)
Where represents real images, represents noise to the generator, and G(Z) represents the gener- ated images, D(x) represents the probability of judging whether real image is true.
3.2 ENCODER
GAN is used to directly fit the distribution n-dimensional vectors in images, and sample with ran- dom noise, the whole training process is in unsupervised state, the direction of data generation uncontrollable unless the process traverses all initial distributions. At the same time, the generated images may not be the true expression of image meaning due the excessive pursuit of quality.

---

Under review as a conference paper at ICLR 2020
Therefore, when using the adversarial model to generate positron images in the industrial non- destructive testing, we choose to add a prior to restrict the data generation, so that the generation model can be trained for specific areas.
Based on the scarcity of industrial positron image data, in this paper, we introduce the knowledge of migration learning and use medical images training data to construct encoder, which is based on the variational auto-encoder.
Firstly, sample medical image data X get series of sample points 2 22; 2a, Ens which makes all the sample data in X fit successfully and obtains a distribution p(t), is described as formula equation 2
1 72
E
24 p(z)
In order achieve this goal, the distribution fitting of data sample X is finally realized with the help of implicit variable Z. It is assumed that p(r) describes probability distribution of X generated by Z, which satisfies the Gauss distribution. Therefore, the whole encoder can be expressed as sampling Z from the standard normal distribution. In the process, we can get Variance and Mean of Sample Data X Sampled by Neural Network. The clustering process can be parametrized as formula equation
μ = fi (Xx) log 0² = f2 (Xx)
(3)
p(2) = =x ((ZIX)p(X) = Ex - =
~(0,
Where the mean and variance of the normal distribution which is exclusive to Xg can be obtained. Then Z& can be sampled from this exclusive distribution.
3.3 FEATURE EXTRACTION-MEMOO MODULE
In order to obtain a more suitable generation model for positron image in the industrial field, we proposes an image feature memory module based on attention mechanism, which used extract domain image features. This network research focus of this chapter.
The basic flow of the whole network is follows:1) neural network to extract the feature of the rare positron images, and obtain the images feature vectors. 2) combine the vector and the hidden variable get in Section 3.2 based on attention mechanism obtain an image memory model. 3usee the memory model as the input of adversarial nets and train jointly with the whole network to obtain positron image generator for the industrial field.
Positron image feature extraction We use the principal component analysis(H et al, 2015) is used extract the positron sample data and the vector space transformation is used reduce the di- mensionality of higher dimensional positron data. Firstly, transform the original data into new coordinate system by projection according to the new coordinate vector. Secondly, the variance of the first principal component of the projection data in the coordinate system is the largest. As the dimension increases, the variance decreases in turn and the dimension decreases. is described as formula equation 4
yf
V1,1
1/12
V1.m
y/2
V2,1
V2,2
V/,n
Y
(4)
...
Vm,1
Vmn,2
Vnn.
mXX
m represents the positron sample, n represents the sample dimension and the sample = m X n
The data matrix Y is de-averaged so that the mean value of each dimension is 0. Then should find the most important feature vectors the images, that is, the data on the coordinate axis represented by the feature fluctuates the most and the of squares of all samples projected unit vector is
5

---

Under review as a conference paper at ICLR 2020
the largest. Then we get the value of μ using Lagrange theorem. The mathematical expression is as follows equation
u² arg max 1
( ( u argmax x² E Mey 2
Li=1
L(X, U.
u Eu + A (u
(5)
aL
= Eu Aw
du
= arg max u Eu = argmax Au
The nets use convolution neural netWork(CNN) construct image feature extraction network, the network structure is divided into three layers, namely two convolution layers and one non-linear output layer. Firstly, small image slices are extracted from sample images and the dimension of the slices is the same as convolution core. Then traverse all the pixels in them and perform two- level convolution operation. Finally, hashing operation and histogram statistics are carried out the output layer to print the feature vector.
Memory Module Based on Attention Mechanism The obtained positron eigenvector Y is fused with the hidden variable Z the medical image obtained based on the attention mechanism to get the input nets. The purpose is make the prior knowledge contained in the nets more focused on positron features, so that the features of scarce data can be more applied in the whole training process.
The basic idea is the global attention and the focus in our model is to extract all positron image features. The specific realization is align image data vectors, directly medical images as query vectors, and input posítron image feature vectors as hidden state to calculate their weights equation 6
(s) alig n (3t, exp(s core(zt.
Σ, core score (zt, W.D.
(6)
Where 2 is the medical image distribution, F, are feature vectors extracted from positron images, and score (2,. is the scoring criterion for the operation.
We get a constant and normalize it. And the contribution degree of each feature of positron image the network can be obtained. So the image feature can be fused according the weight ratio. Finally, the vector containing prior knowledge in the field is obtained as overall input of adversarial nets.
3.4 ADVERSARIAL NETS
Generative model The generative network is constructed based DenseNet (G al., 2017), and the positron image features can be requisitioned repeatedly in the model. The network can also strengthen the contribution of the characteristics of searce data that the generated images closer to real industrial positron images in detail.
The generative model is follows: the output of the memory model in chapter 3.3 whole input to the net, and the input of each layer is related to the output of all the previous layers, not only related upper layer. be expressed as formula equation 7
X1 Hi ([Xo. X1, .1-1))
[No: XI...., .X(n)) the concatenation the net. We can group all output feature maps from layer Xo to X(-1) according to different channels and the structure is used to reduce the parameters without losing features randomly, so that the initial input can enter each layers convolution calcula- tion realize the feature reuse. The basic structure is 33 convolution layer, Batch Normalization S & C 20d ReLU non-linear activation layer.
Feature maps of all previous layers need be cat in the network. In order perform the down sampling operation, the net is divided into several Denseblocks and transition layers are used be- tween different them. Referring the original network, consists of Batch Normalization layer,

---

Under review as a conference paper at ICLR 2020
1#1 convolution network and 2*2 average-pooling In the same Denseblock, the state of each lay- er is associated with all previous layers, and the training of each layer is aimed at the global state feedback of the network to update the parameters.
Discriminative model The discriminative net is used to discriminate specific images a specific domain, in which domain image features be used the evaluation criteria for network classifi cation as much as possible. The net uses Markov Model based on PatchGAN, which is composed of full convolution layers. The output is an n-dimensional matrix, and the mean of the matrix used as the output of the discriminative network, so that each receptive field in the image can be judged, which is equivalent the convolution discriminant in batches by layers, and finally fed back the whole network. The calculation of receptive field is equation
SinpCipuut (sizCoulput 1) * kistride + lisize
(8)
In the model, the real input samples are medical data sets. Therefore, in order make the generated data better characterize positron image features, need to add an additional attention perception loss function to the net. The loss function the whole net consists of two parts: Ag and L the attention loss function L PG' is used to characterize the feature contribution of positron image, which is realized to measure the distribution distance between the generated data and the positron images. The loss function is described as equation 9
LAPG Ex.amp((z,))
1D'())
W
W. represents number of elements in each layer, and s is the number of layers. The loss function of the whole net can be described as equation 10 and the AGAN is similar as the original GAN.
LGAN LGAN + LAPG
(10)
EXPERIMENTAL RESULTS
4.1
IMPLEMENTATION DETAILS
We design the model firstly by using encoder to obtain the medical images hidden vectors and using principal component analysis to reduce positron images dimensionality and extract the main fea- ture. Train memory module and adversarial nets jointly, and in the process of backpropagation, the identification network updates the parameters of the front-end network, so that the feature extrac- tion network extracts the features repeatedly until the whole network achieves the optimal model. Finally, the positron image generator for industrial non-destructive testing is obtained.
The discriminator refers to pixel and each batch is 70*70. The learning rate is 0.0002 in the whole net. The model are trained iteratively using Adam algorithm(=0.5).
4.2 DATASET
DeepLession The d al. 2018) contains more than 32,000 lesions from more than 10,000 case studies. The data is developed by National Institutes of Health Clinical Center(NIHHCC, and developed by mining historical medical data from image archives and communication systems. is now maybe the largest set of CT medical image data available to all. In the experiment, 150,000 images were selected for our training and the image pixels are 256*256.
Positron images The dataset is obtained by the Geant4 Application for Tomographic Emis- sion(GATE). GATE is a simulation software which can simulate the physical process of PET imag- ing system, such as geometric size, material composition, detector movement and so In the model design, the anisotropic tube made of aluminium metal is filled with positron nuclide solution. Its activity is 600 Bq. the number of detectors is 184 *64, the sampling time is 0.1 the energy resolution is 15 percent, the time resolution is 300 ps, the energy window is 350-650 keV, and the time window is 10 ns. The design sampling time is 0.1s meet the needs of rapid sampling in
7

---

Under review as a conference paper at ICLR 2020
Table 1: Evaluation results
METHOD MS-SSIM FID
VAE 0.0485
DCGAN 0.0522
15.2
WGAN 0,0567
27.5
Ours
0.0694
52.8
the industrial field. Using Maximum Likelihood Expectation Maximization (MLEM) iteration al- gorithm to realize image preliminary reconstruction and obtain positron defect image in industrial field.
4.3 EVALUATION
In this section, we compare our approach with the commonly used generation model, aiming at generation of industrial positron images. Quantitative results presented in Table [
The metrics to evaluate the performance our method, namely multi-scale structural similarity SSIM) (A al. 2017) and Frchet Inception Distance (FID) al. (017)). The SSIM is used to measure the similarity of two images and FID to measure the Frehet distance of two distributions in feature space.
By comparing the experimental data, we can clearly see that the confrontation network constructed in this paper has better effect the generation of positron images for professional fields, and the generated images are closer to the real images.
5 CONCLUSION
In this paper, we introduce an application of GAN in the field of nondestructive testing for specific industries. We combine the knowledge of transfer learning make up the problem of insufficien- t data. The key point is to introduce attention mechanism to construct a positron image feature memory module, which can reuse image features under the condition of scarce data. At the same time, attention loss function is added to the discriminative net to further improve the generator performance. Experiments show that compared with the sart-of-the-ar generation methods in deep learning, the model in our paper has an obvious improvement in the quality of industrial positron image generation.
In the future, our focus is to further study the application of generative adversarial networks in industrial positron image processing, and further improve the quality domain images.
ACKNOWLEDGMENTS
This research has been funded by the Natural Science Foundation of China No.51887229NN.6187311 the Aeronautical Science Foundation of China (No.20162D52036, the Fundamental Research Funds for the Central Universities (No. NS2019017).
REFERENCES
Gao K. Medical imaging system. Tsinghua University Press, 2000.
L. A. Shepp and Y. Vardi. Maximum likelihood reconstruction for emission tomography. IEEE Transactions Medical Imaging, 1(2):113-122 2007.
Ian Goodfellow, Abadie J. Pouget, and Mirza M. Generative adversarial nets. In Advances in neural information processing systems. MIT Press, 2014.
Radford, Metz L, and Chintala Unsupervised representation learning with deep convolutional generative adversarial networks. Computer Science, 2015.
8

---

Under review as a conference paper at ICLR 2020
Arjovsky M, Chintala S, and Bottou L. Wasserstein gan. 2017.
Mirza M and Osindero S. Conditional generative adversarial nets. 2014.
Mao X, Li Q. and Xie H. Least squares generative adversarial networks. 2016.
Karras T, Aila T, Laine S, and Lehtinen J. Progressive growing of gans for improved quality, stability, and variation. 2017.
Brock A, Donahue and Simonyan K. Large scale gan training for high fidelity natural image synthesis. 2018.
Zhao J, Mathieu M, and LeCun Y. Energy-based generative adversarial network. 2016.
Metz L, Poole B, and Pfau D. Unrolled generative adversarial networks. 2016.
Kodali N, Abernethy J, and Hays J. On convergence and stability of gans. 2017.
Isola P. Zhu J Y, and Zhou T. 2016.
Image-to-image translation with conditional adversarial networks.
Kim T, Cha M. and Kim H. Learning to discover cross-domain relations with generative adversarial networks. 2017.
Yi Z, Zhang H, and Tan P. Dualgan: Unsupervised dual learning for image-to-image translation. 2017.
Ledig C. Theis L L, and Huszr F. Photo-realistic single image super-resolution using generative adversarial network. 2017.
Xue Y, TT, and Zhang H. Segan:adversarial network with multi-scale 11 loss for medical image segmentation. Neuroinformatiic, 16(3-4):383-39) 2018.
Li J, Liang X, and Wei Perceptual generative adversarial networks for small object detection. 2017.
Zhou S, Xiao T, and Yang Y. Genegan: Learning object transfiguration and attribute subspace from umpaired data. 2017.
ZhangJ J Wu H, Zheng S. Gp-gan: Towards realistic high-resolution image blending. 2017.
Ming Yu Liu and Oncel Tuzel. Coupled generative adversarial networks. 2016.
Jiawei Chen, Janusz Konrad, and Prakash Ishwar. Vgan-based image representation learning for privacy-preservie facial expression recognition. 2018.
Tulyakov, Sergey, Ming Yu Liu, and Xiao dong Yang. Mocogan: Decomposing motion and content for video generation. 2017.
Zhang H. Xu T, and Li H. Stackgan: Text to photo-realistic image synthesis with stacked generative adversarial networks. 2017.
Dash A, Gamboa J C B, and Ahmed S. Tac-gan-text conditioned auxiliary classifier generative adversarial network. 2017.
Guo J, Lu S, and Cai H. Long text generation via adversarial training with leaked information. 2018.
Hsu C, H Hwang, and Wu Y C. Voice conversion from unaligned corpora using variational autoencoding wasserstein generative adversarial networks. 2017.
Hoffman J, Tzeng E, and Park T. Cycada: Cycle-consisn cycle-consistent adversarial domain adaptation. 2017.
Chuquicusma M M, Hussein S, and Burt J. How to fool radiologists with generative adversarial networks? a visual turing test for lung cancer diagnosis. 2018.
Kitchen A and Seah J. Deep generative adversarial neural networks for realistic prostate lesion mri synthesis. 2017.

---

Under review as a conference paper at ICLR 2020
Schlegi T, Seebek P, and Waldstein S M. Unsupervised anomaly detection with generative adversar- ial networks to guide marker discovery. 2017.
Bermudez C, Plassard J., and Davis T.T. Learning implicit brain mri manifolds with deep learning. 2018a.
Baur C, Albargouni S, and Navab N. Generating highly realistic images of skin lesions with gans. 2018b.
Nie D, Trullo R, and Lian J. Medical image synthesis with context-aware generative adversarial networks. 2017.
Costa P. Galdran A, and Meyer M End-to-end adversarial retinal image synthesis. transactions on medical imaging, 37(3):781-79,, 2017.
IEEE
Guibas T, Virdi T S. and Li S. Synthetic medical images from dual generative adversarial networks. 2017.
Wei W, Poirion E E,, and Bodini B. Learning myelin content in multiple sclerosis from multimodal mri through adversarial training. 201 8.
Tom F and Sheet D. Simulating patho-realistic ultrasound images using deep generative networks with adversarial learning. 2018.
BenTaich A and Hamarneh G. Adversarial stain transfer for histopathology image analysis. JEEE transactions medical imaging, 37(3):792-88) 802, 2017.
Wolterink M, Leiner T, and Isgum I. Blood vessel geometry synthesis using generative adversarial networks. 2018.
Chan T H, Jia K, and Gao S. Pcanet: A simple deep learning baseline for image classification? IEEE transactions image processing, 24(12),507755032 2015.
Huang G, Liu Z, and Van Der Maaten I.. Densely connected convolutional networks. 2017.
loffe S and Szegedy Batch normalization: Accelerating deep network training by reducing inter- nal covariate shift. 2015.
Yan K. Wang X, and Lu L. Deeplesion: automated mining of large-scale lesion annotations and universal lesion detection with deep learning. Journal of Medical Imaging, 5(3):036501, 2018.
Odena A, Olah and Shlens Conditional image synthesis with auxiliary classifier gans. In ICML, pp. 26422651, 2017.
Heusel M, Ramsauer H, and Unterthiner T. Gans trained by a two time-scale update rule converge to local nash equilibrium. In NIPS, pp. 66266637, 2017.
A APPENDIX
The Positron image are shown in figure 2 which are simulation diagram with defects. The picture shows one-to-one correspondence between the original images and the generated images.
10

---

Under review as a conference paper at ICLR 2020
(a)
(b)
(c)
(d)
Figure 2: positron images


Reviews:
[{"review_id": "SkgMSQgJ9S", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper applies GAN to the field of nondestructive testing for specific industries. This paper is more like a technical report rather than a formal paper. It seems to me that this paper should be submitted to other computer vision conferences of even specific areas while not in ICLR. \n\nIssues:\n\n* Bad format or organization. The authors are suggested that they should give a subtitle of each categories of work in the related work section. The table and picture in this paper are arbitrarily designed and take too much space. In some equations, you need to put commas to separate different equations.\n\n* Acknowledgements reveal personal information in the paper. It's not allowed in ICLR submission that would review the identity of the authors. This is a highly critical issue. \n\n* Bad writing. I can barely understand what the authors are talking about. For example, \"In this paper, we propose adversarial networks of positron image memory module based on attention mechanism\". Are you referring you proposed a new GAN model in the paper? Or you proposed a positron image memory module? ...\n\n* What are the functions of the metrics used in the experimental part? The higher, the better? Or the lower, the better? Besides, you need some analysis to illustrate the significance of your results.\n\nConsidering these issues demonstrated in the paper, I recommend rejection.", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "N/A", "review_assessment:_checking_correctness_of_derivations_and_theory": "I did not assess the derivations or theory."}, "extra_text": {}}, {"review_id": "H1xdnWUVtB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The topic of the paper is a GAN framework to enhance PET images in industrial inspection, as far as I understand by transfer learning from a medical PET database. Unfortunately, I am unable assess the paper due to serious language problems. The text is incoherent and not understandable, it is impossible to decipher what is actually proposed.\n\nAdditional Comments:\n\nThe text reads like a machine translation gone wrong, including weird terms like \"migration learning\" (transfer learning?), \"antagonistic generation network\" / \"countermeasure generation network\" / \"confrontation network\" (GAN?).\n\nReferences are also mangled and undecipherable. And seemingly also not always well-chosen - even if I cannot map it to a paper due to bibtex problems, it appears implausible to reference a 2015 paper for something as basic as principal component analysis.\n\nIt seems that the only experiments are on simulated PET data. In may view that would be insufficient for a largely empirical application paper.\n\nThe paper claims to be quantitatively better than the baselines, but has, by far, the highest Frechet Inception distance. To my understanding, FID is a distance, lower is better.\n\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "BkxnKw5pFr", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This work proposes a method to transfer information from PET imaging data from the medical domain, where data is highly available, to the industrial one, where data is scarce, in the context of non-destructive material quality evaluation. The basic idea seems interesting, but unfortunately in the present form he paper is very difficult to appreciate, as it lacks of important details concerning methodology, experimental results, and comparison with respect to the state-of-the-art. Moreover, the manuscript still appears in a draft form. Sentences are often broken, the text presents many typos and grammar mistakes, and the citations are not understandable. \n\nMissing methodological details are grouped in the following parts of this review.\n\nSection 3.2 Encoder\nParagraph 2: The authors claim that they introduce the knowledge of migration learning without explaining it. Where does this concept come from? Is there a literature about it or is this a new concept?\n\nMedical images are fitted though a variational auto encoder (VAE) (citation missing). The encoder description is minimal and lacks of implementation details (see Eq. 3), while the decoder description is missing throughout the paper. \n\nEq. (2): the authors write that they obtain a distribution p(x) according to Eq. (2), but this equation is just the formula for the sample mean, where p(x) is sampled and not computed.\n\nEq. (3): f1 and f2 are never made explicit in the paper so we do not know if they are linear or non-linear functions. The prior p(Z) is decomposed as a summation of posteriors p(Z|X) and the choice to have these posteriors equal to N(0,1) (1-dimensional, which is unusual) regardless of the data point X is not explained.\n\nSection 3.3 Feature extraction memory module\nFeature extraction from industrial images is done through principal component analysis (PCA). In the same paragraph is written that features are extracted through convolution neural networks (CNN). So it is not clear if there is an arbitrary choice to use PCA or CNN, and what are the conditions when this happens.\n\nEq. (6): the score between z_t (medical image distribution) and y_s (industrial image feature vector) is computed as scalar product dot(z_t , W_a * y_s). The key link is the linear mapping W_a, which is never made explicit in the paper. How do the authors compute W_a ?\n\nSection 4.1 Implementation details\nthe networks called “identification network” and “front-end network” are not well defined. They may refer to the  VAE, the CNN, the Adversarial Nets. There is too much ambiguity. A captioned figure can help in resolving the ambiguities.\n\nSection 4.2 Dataset\nIn the first paragraph the authors cite a dataset of CT images, while the main focus of the paper is on PET images. How the CT images comes into play?\n\nSection 4.3 Evaluation\nThe authors compare their method with respect to other three methods, namely VAE, DCGAN, and WGAN.\nThe implementation details of the competing methods are not described so we cannot be sure about the fairness of the comparison.\n\n\nOther considerations\nIntroduction, 4th paragraph: “imaging quality is higher” With respect to what? Usually PET images have the worst quality in the medical domain.\nIntroduction, 3rd-to-last paragraph: “We use the medical CT image ...”. Should be PET images\n\nRelated work: This section is a list of works and the relation with the current paper is not highlighted.\n“lung cancer cakes” what are they?\n\nParagraph 3.2: “excessive pursuit of quality” why is it bad? “migration learning” do they mean transfer learning? Equation 2 and 3 in relation to a clustering problem never pointed out before in the paper.\n\nParagraph 4.1. What is the meaning of “Adam algorithm(=0.5)” ?\n\nFigures 1 and 2 have very minimal captions. What do they represent?\n\nCitation formatting problem: name and surname are switched. Journal/conferences often omitted.", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper studies Positron Emission Tomography (PET) in medical imaging. The paper focuses on the challenges created by gamma-ray photon scattering, that results in poor image quality. To tackle this problem and enhance the image quality, the paper suggests using generative adversarial networks. Unfortunately due to poor writing and severe language issues, none of the three reviewers were able to properly assess the paper [see the reviews for multiple examples of this]. In addition, in places, some important implementation details were missing.

The authors chose not to response to reviewers' concerns. In its current form, the submission cannot be well understood by people interested in reading the paper, so it needs to be improved and resubmitted. 
Decision: Reject