Adversarial AutoAugment

1 INTRODUCTION . Massive amount of data have promoted the great success of deep learning in academia and industry . The performance of deep neural networks ( DNNs ) would be improved substantially when more supervised data is available or better data augmentation method is adapted . Data augmentation such as rotation , flipping , cropping , etc. , is a powerful technique to increase the amount and diversity of data . Experiments show that the generalization of a neural network can be efficiently improved through manually designing data augmentation policies . However , this needs lots of knowledge of human expert , and sometimes shows the weak transferability across different tasks and datasets in practical applications . Inspired by neural architecture search ( NAS ) ( Zoph & Le , 2016 ; Zoph et al. , 2017 ; Zhong et al. , 2018a ; b ; Guo et al. , 2018 ) , a reinforcement learning ( RL ) ( Williams , 1992 ) method called AutoAugment is proposed by Cubuk et al . ( 2019 ) , which can automatically learn the augmentation policy from data and provide an exciting performance improvement on image classification tasks . However , the computing cost is huge for training and evaluating thousands of sampled policies in the search process . Although proxy tasks , i.e. , smaller models and reduced datasets , are taken to accelerate the searching process , tens of thousands of GPU-hours of consumption are still required . In addition , these data augmentation policies optimized on proxy tasks are not guaranteed to be optimal on the target task , and the fixed augmentation policy is also sub-optimal for the whole training process . In this paper , we propose an efficient data augmentation method to address the problems mentioned above , which can directly search the best augmentation policy on the full dataset during training a target network , as shown in Figure 1 . We first organize the network training and augmentation policy search in an adversarial and online manner . The augmentation policy is dynamically changed along with the training state of the target network , rather than fixed throughout the whole training process like normal AutoAugment ( Cubuk et al. , 2019 ) . Due to reusing the computation in policy evaluation and dispensing with the retraining of the target network , the computing cost and time overhead are extremely reduced . Then , the augmentation policy network is taken as an adversary to explore the weakness of the target network . We augment the data of each min-batch with various adversarial policies in parallel , rather than the same data augmentation taken in batch augmentation ( BA ) ( Hoffer et al. , 2019 ) . Then , several augmented instances of each mini-batch are formed into a large batch for target network learning . As an indicator of the hardness of augmentation policies , the training losses of the target network are used to guide the policy network to generate more aggressive and efficient policies based on REINFORCE algorithm ( Williams , 1992 ) . Through adversarial learning , we can train the target network more efficiently and robustly . The contributions can be summarized as follows : • Our method can directly learn augmentation policies on target tasks , i.e. , target networks and full datasets , with a quite low computing cost and time overhead . The direct policy search avoids the performance degradation caused by the policy transfer from proxy tasks to target tasks . • We propose an adversarial framework to jointly optimize target network training and augmentation policy search . The harder samples augmented by adversarial policies are constantly fed into the target network to promote robust feature learning . Hence , the generalization of the target network can be significantly improved . • The experiment results show that our proposed method outperforms previous augmentation methods . For instance , we achieve a top-1 test error of 1.36 % with PyramidNet+ShakeDrop ( Yamada et al. , 2018 ) on CIFAR-10 , which is the state-of-the-art performance . On ImageNet , we improve the top-1 accuracy of ResNet-50 ( He et al. , 2016 ) from 76.3 % to 79.4 % without extra data , which is even 1.77 % better than AutoAugment ( Cubuk et al. , 2019 ) . 2 RELATED WORK . Common data augmentation , which can generate extra samples by some label-preserved transformations , is usually used to increase the size of datasets and improve the generalization of networks , such as on MINST , CIFAR-10 and ImageNet ( Krizhevsky et al. , 2012 ; Wan et al. , 2013 ; Szegedy et al. , 2015 ) . However , human-designed augmentation policies are specified for different datasets . For example , flipping , the widely used transformation on CIFAR-10/CIFAR-100 and ImageNet , is not suitable for MINST , which will destroy the property of original samples . Hence , several works ( Lemley et al. , 2017 ; Cubuk et al. , 2019 ; Lin et al. , 2019 ; Ho et al. , 2019 ) have attempted to automatically learn data augmentation policies . Lemley et al . ( 2017 ) propose a method called Smart Augmentation , which merges two or more samples of a class to improve the generalization of a target network . The result also indicates that an augmentation network can be learned when a target network is being training . Through well designing the search space of data augmentation policies , AutoAugment ( Cubuk et al. , 2019 ) takes a recurrent neural network ( RNN ) as a sample controller to find the best data augmentation policy for a selected dataset . To reduce the computing cost , the augmentation policy search is performed on proxy tasks . Population based augmentation ( PBA ) ( Ho et al. , 2019 ) replaces the fixed augmentation policy with a dynamic schedule of augmentation policy along with the training process , which is mostly related to our work . Inspired by population based training ( PBT ) ( Jaderberg et al. , 2017 ) , the augmentation policy search problem in PBA is modeled as a process of hyperparameter schedule learning . However , the augmentation schedule learning is still performed on proxy tasks . The learned policy schedule should be manually adjusted when the training process of a target network is non-matched with proxy tasks . Another related topic is Generative Adversarial Networks ( GANs ) ( Goodfellow et al. , 2014 ) , which has recently attracted lots of research attention due to its fascinating performance , and also been used to enlarge datasets through directly synthesizing new images ( Tran et al. , 2017 ; Perez & Wang , 2017 ; Antoniou et al. , 2017 ; Gurumurthy et al. , 2017 ; Frid-Adar et al. , 2018 ) . Although we formulate our proposed method as a Min-Max game , there exists an obvious difference with traditional GANs . We want to find the best augmentation policy to perform image transformation along with the training process , rather than synthesize new images . Peng et al . ( 2018 ) also take such an idea to optimize the training process of a target network in human pose estimation . 3 METHOD . In this section , we present the implementation of Adversarial AutoAugment . First , the motivation for the adversarial relation between network learning and augmentation policy is discussed . Then , we introduce the search space with the dynamic augmentation policy . Finally , the joint framework for network training and augmentation policy search is presented in detail . 3.1 MOTIVATIONS . Although some human-designed data augmentations have been used in the training of DNNs , such as randomly cropping and horizontally flipping on CIFAR-10/CIFAR-100 and ImageNet , limited randomness will make it very difficult to generate effective samples at the tail end of the training . To struggle with the problem , more randomness about image transformation is introduced into the search space of AutoAugment ( Cubuk et al. , 2019 ) ( described in Section 3.2 ) . However , the learned policy is fixed for the entire training process . All of possible instances of each example will be send to the target network repeatedly , which still results in an inevitable overfitting in a long-epoch training . This phenomenon indicates that the learned policy is not adaptive to the training process of a target network , especially found on proxy tasks . Hence , the dynamic and adversarial augmentation policy with the training process is considered as the crucial feature in our search space . Another consideration is how to improve the efficiency of the policy search . In AutoAugment ( Cubuk et al. , 2019 ) , to evaluate the performance of augmentation policies , a lot of child models should be trained from scratch nearly to convergence . The computation in training and evaluating the performance of different sampled policies can not be reused , which leads to huge waste of computation resources . In this paper , we propose a computing-efficient policy search framework through reusing prior computation in policy evaluation . Only one target network is used to evaluate the performance of different policies with the help of the training losses of corresponding augmented instances . The augmentation policy network is learned from the intermediate state of the target network , which makes generated augmentation policies more aggressive and adaptive . On the contrary , to combat harder examples augmented by adversarial policies , the target network has to learn more robust features , which makes the training more efficiently . 3.2 SEARCH SPACE . In this paper , the basic structure of the search space of AutoAugment ( Cubuk et al. , 2019 ) is reserved . An augmentation policy is defined as that it is composed by 5 sub-policies , each sub-policy contains two image operations to be applied orderly , each operation has two corresponding parameters , i.e. , the probability and magnitude of the operation . Finally , the 5 best policies are concatenated to form a single policy with 25 sub-policies . For each image in a mini-batch , only one sub-policy will be randomly selected to be applied . To compare with AutoAugment ( Cubuk et al. , 2019 ) conveniently , we just slightly modify the search space with removing the probability of each operation . This is because that we think the stochasticity of an operation with a probability requires a certain epochs to take effect , which will detain the feedback of the intermediate state of the target network . There are totally 16 image operations in our search space , including ShearX/Y , TranslateX/Y , Rotate , AutoContrast , Invert , Equalize , Solarize , Posterize , Contrast , Color , Brightness , Sharpness , Cutout ( Devries & Taylor , 2017 ) and Sample Pairing ( Inoue , 2018 ) . The range of the magnitude is also discretized uniformly into 10 values . To guarantee the convergence during adversarial learning , the magnitude of all the operations are set in a moderate range.1 Besides , the randomness during the training process is introduced into our search space . Hence , the search space of the policy in each epoch has |S| = ( 16×10 ) 10 ≈ 1.1×1022 possibilities . Considering the dynamic policy , the number of possible policies with the whole training process can be expressed as |S| # epochs . An example of dynamically learning the augmentation policy along with the training process is shown in Figure 2 . We observe that the magnitude ( an indication of difficulty ) gradually increases with the training process . 1The more details about the parameter setting please refer to AutoAugment ( Cubuk et al. , 2019 ) .

Summary: This paper describes a method to learn data augmentation policies using an adversarial loss. It builds on the AutoAugment method. In AutoAugment, an augmentation policy generator is trained by reinforcement learning. At each iteration, a classifier network is trained from scratch based on the current augmentation policy, and its validation accuracy is used as the reward signal. This is extremely costly because it requires to train a complete network for every training step of the policy generator. Instead, the current paper proposes to train the policy generator and the classifier simultaneously. The policy generator is trained adversarially to find augmentation policies that increase the loss of the classifier. This leads to a significant speedup compared to classic AutoAugment.