L OCAL P ATCH A UTO A UGMENT WITH M ULTI -A GENT
C
OLLABORATION
Anonymous authors
Paper under double-blind review
A BSTRACT
Data augmentation (DA) plays a critical role in improving the generalization of deep learning models.
Recent works on automatically searching for DA policies from data have achieved great success.
However, existing automated DA methods generally perform the search at the image level, which limits the exploration of diversity in local regions.
In this paper, we propose a more ﬁne-grained automated DA approach, dubbed Patch AutoAugment, to divide an image into a grid of patches and search for the joint optimal augmentation policies for the patches.
We formulate it as a multi-agent reinforcement learning (MARL) problem, where each agent learns an augmentation policy for each patch based on its content together with the semantics of the whole image.
The agents cooperate with each other to achieve the optimal augmentation effect of the entire image by sharing a team reward.
We show the effectiveness of our method on multiple benchmark datasets of image classiﬁcation, ﬁne-grained image recognition and object detection ( e .
g ., CIFAR-10, CIFAR-100, ImageNet, CUB-200-2011, Stanford Cars,
FGVC-Aircraft and Pascal VOC 2007).
Extensive experiments demonstrate that our method outperforms the state-of-the-art DA methods while requiring fewer computational resources.
I NTRODUCTION
[Figure]
Data Augmentation (DA) has been widely used to alleviate the overﬁtting risk in training deep neural networks by appropriately enriching the diversity of training data (Shorten & Khoshgoftaar, 2019;
Gontijo-Lopes et al., 2020).
Notable DA methods improve the performance and robustness of the neural networks, such as rotation, Mixup (Zhang et al., 2017) and Cutmix (Yun et al., 2019).
But these approaches are typically handcraft and require human prior knowledge, which causes weak transferability of DA across different datasets.
To relieve the dependence on manual design and further explore more adaptive augmentation, AutoAugment (AA) (Cubuk et al., 2018), as a new DA paradigm, is proposed to automate the search of the optimal DA policies ( i .
e .,
DA operation, probability and magnitude) from the training dataset.
To be speciﬁc, AA trains a proxy model with the augmentation policy generated by a controller, which is updated through reinforcement learning using validation accuracy as the reward signal.
In spite of the superior performance of
AA, its optimization procedure is computationally intensive due to the need to evaluate thousands of policies.
Therefore, to reduce the search costs, multiple automated DA approaches (Lim et al., 2019; Ho et al., 2019; Hataya et al., 2019; Zhang et al., 2019b; Lin et al., 2019) have been proposed.
For example, (Lim et al., 2019) employs density matching as a search method to accelerate the policy search, and (Zhang et al., 2019b) introduces adversarial learning to organize the target network training and augmentation policies search in an online manner.
[Figure]
Yet the aforementioned automated DA methods all search for policies at the image level.
They ignore the exploration of diversity in local regions, which may result in insufﬁcient diversity of dataset and limit the beneﬁts of DA (Gontijo-Lopes et al., 2020).
In addition, due to this coarse-grained augmentation, they may damage key semantic information and introduce ambiguity into the training process (see a simple example in Figure 1 row 2, column 2).
With those in mind, it is necessary to automatically search for the optimal augmentation policies for different regions by taking regional diversity into account.
One straightforward idea is to directly apply
image-wise automated
DA methods in different regions.
But such an intuitive solution ignores the contextual relationship between regions, which may lead to the non-globally optimal effectiveness of DA policies across the entire image.
Besides, it may encounter an extremely high computational cost due to the need of optimizing multiple policies for regions respectively.
In this paper, we propose a new approach, named Patch AutoAugment (PAA) (see the last row of
Figure 1), to address the above-mentioned problems.
We ﬁrst divide an image into a grid of patches to increase the ﬂexibility of representations for different regions and take “patch” as the basic control unit.
Then, we model the search for the augmentation policies of patches as a fully cooperative multiagent task, and we leverage a multi-agent reinforcement learning (MARL) algorithm where agents cooperatively learn the policies.
Speciﬁcally, based on the content of each patch and the semantics of the entire image, the agent searches for a policy in terms of choosing which transformation to apply out of the pre-deﬁned DA operations.
To encourage our policy networks to adaptively learn the beneﬁcial augmentation policies for the target network, we use the feedback of the target network as the team reward signal to guide the policy networks to learn on the ﬂy.
All agents wind up beneﬁting from two mechanisms ( i .
e ., parameter sharing and centralized training with decentralized execution) in MARL.
In this way, all agents collaboratively and parallelly learn policies to further achieve the joint optimal DA policy across the whole image and alleviate the computational cost.
In summary, our contributions can be summarized as follows: 1) We pinpoint that exploring diversity in local regions is important for automated learned DA approaches.
To our best knowledge, we are the ﬁrst to propose a more ﬁne-grained automated DA approach, that searches for the optimal policies for the patches according to the content of the patch and the semantics of the entire image.
2) To further achieve the joint optimal policy across the image, we model the DA policy search of patches as a fully cooperative multi-agent task, and adopt a multi-agent reinforcement learning algorithm for Patch AutoAugment by considering the contextual relationship between the patches.
Our visualization results provide some insights to the DA community on which augmentation operation should be chosen for patches with different content during the whole training process.
R ELATED W ORK 2. D ATA A UGMENTATION
Despite the remarkable performance of deep learning models in computer vision tasks, they often suffer from overﬁtting.
Data augmentation (DA) as an effective technique has been proved to im- prove the generalization ability of deep learning models (Shorten & Khoshgoftaar, 2019).
Previous works (Cubuk et al., 2020; Gontijo-Lopes et al., 2020) indicate that the main beneﬁt of DA arises from increasing the diversity of images.
Popular techniques, such as rotation, ﬂipping, color transformation, have been performed as commonly used augmentation methods.
Recently, thanks to the skillful design of human experts, many DA methods ( e . g .,
Cutout (DeVries & Taylor, 2017), Mixup (Zhang et al., 2017), CutMix (Yun et al., 2019)) have been proposed and show signiﬁcant performance.
However, these manually designed methods require additional human prior knowledge on the dataset and sometimes they are limited to certain datasets and target tasks.
Naturally, automatically ﬁnding DA methods from data have emerged to overcome the limitations of dependence on manual cumbersome exploration.
Some works use generative adversarial networks (GANs) to directly generate training data (Shrivastava et al., 2017; Tran et al., 2017).
Furthermore, recent studies aim to automate the search for augmentation policies that choose the optimal transformations for training images.
AutoAugment (AA) (Cubuk et al., 2018) adopts a controller to generate an augmentation policy which is used to train a proxy network, then gets the validation accuracy as the reward signal to update the controller using reinforcement learning.
Unfortunately, the evaluation of thousands of policies makes AA computationally expensive.
Therefore, multiple automated DA approaches focus on reducing the huge complexity and have achieved great progress.
For example, PBA (Ho et al., 2019) employs hyperparameter optimization, Fast AA (Lim et al., 2019) uses a density matching algorithm, Adv AA (Zhang et al., 2019b) proposes an adversarial framework to jointly optimize the target network and the augmentation network, and
DADA (Li et al., 2020) and DDAS (Liu et al., 2021a) target relaxing the discrete DA policy selection to a differentiable problem.
Besides, RandAugment (Cubuk et al., 2020) removes the separate search on a proxy task in the ofﬂine automated DA methods ( e .
g ., AA) while also outperforming them.
However, these automated DA methods perform the search at the image level, i .
e ., they use the same policy on the whole image.
It inevitably ignores the diversity of different regions in an image, which limits the diversity of data increased by data augmentation, and sometimes causes the damage of critical semantic information.
In contrast, our method takes diversity in local regions and contextual relationships into account to search for augmentation policies for multiple regions.
Our proposed method is conceptually orthogonal to most region-based DA methods where DA transformations are performed in a non-automated way.
For example, RandomErasing (Zhong et al., 2020), CutMix (Yun et al., 2019) perform cropping or replacement operation on a randomly selected rectangle region, Hard chips (Hong et al., 2019) extracts the object patches from an object pool and pastes them to an image to train the detector, and SECOND (Yan et al., 2018) performs
DA by pasting the objects into the 3D point cloud.
Some works further exploit class activation map (Zhou et al., 2016) or saliency map (Zhou et al., 2015) to select representative regions which are augmented by a randomly selected DA operation ( e .
g .,
SaliencyMix (Uddin et al., 2020), SnapMix (Huang et al., 2020)), AugMix (Hendrycks et al., 2019), KeepAugment (Gong et al., 2020) and
TrivialAugment (M ¨ uller & Hutter, 2021)).
Yet our proposed method automatically searches for the augmentation transformations based on the given datasets and tasks.
2. M ULTI -A GENT R EINFORCEMENT L EARNING
The most signiﬁcant characteristic of MARL is the cooperation between agents (Tampuu et al., 2017;
Ma & Cameron, 2008; Foerster et al., 2017) which is distinct from directly applying reinforcement learning (RL) algorithms to multi-agent systems.
Due to the limited observation and action of a single agent, cooperation is necessary in the reinforced multi-agent system to achieve the common goal.
Compared with independent agents, cooperative agents can improve the efﬁciency and robustness of the model (Neto, 2005; Zhang et al., 2019a;
Busoniu et al., 2008).
Many vision tasks use
MARL to interact with the public environment to make decisions, with the goal of maximizing the expected total return of all agents, such as image segmentation (Liao et al., 2020; Han et al., 2019;
Ma et al., 2020), image processing (Furuta et al., 2019).
P ROPOSED M ETHOD
As above mentioned, we aim to search augmentation policies for the patches to explore more augmentation diversity.
Learning the optimal DA policies often needs to interact with the target network, which aligns with the nature of RL ( i .
e ., RL agents learn to make decisions through interacting with the environment (Sutton & Barto, 2018; Arulkumaran et al., 2017)).
In addition, it is of great necessity to take the patch content and the contextual relationship between patches into account.
Therefore, we use multi-agent reinforcement learning (MARL) algorithm.
Speciﬁcally, the search for patch policies is based on the content of the patch together with the semantics of the image, and policies are encouraged to coordinate to achieve the joint optimal DA policy across the whole image.
In this section, we ﬁrst describe the preliminaries of MARL, then elaborate on our augmentation policy formulation and modelling.
Furthermore, we summarize the framework of Patch
AutoAugment.
3. P RELIMINARIES OF M ULTI -A GENT R EINFORCEMENT L EARNING
We ﬁrst introduce the preliminaries of reinforcement learning (RL).
RL models the decision-making problem as a Markov decision process (MDP) which is presented with a tuple ( S , A , P, R, γ, T ) .
In
RL framework, given the state s ∈ S , the agent takes an a ∈ A according to its policy π ( a | s ) :
S × A → [0 , 1] and then receives a reward r : S ×
A → R .
The environment moves to the next state with a transition function denoted as P : S ×
A × S → [0 , 1] .
γ ∈ (0 , 1] is a discount factor and T is a time horizon.
The agent aims to maximize the long-term reward R over T steps to learn the optimal policy π ∗ .
Furthermore, multi-agent reinforcement learning (MARL) considers a group of N agents, denoted as N , operating cooperatively in a shared environment towards a common goal.
It can be formulated as a multi-agent MDP (MAMDP) (Boutilier, 1996) represented with a tuple ( S , {O i } Ni =1 , {A i } Ni =1 , P, R, γ, T ) .
Here, S describes the shared state space and {O 1 , · · · , O N } is a set of observations for agents.
In the MARL conﬁguration, each agent receives a private observation correlated with part of state, i .
e .
, o i : S
→ O i .
According to the global state s and its observation o i , the agent takes its action a i ∈ A
i based on its policy π i ( a i
| o i , s ) ,
A i is the action space for the i -th agent and
A =
{A 1 , ··· , A N } denotes the joint action space.
Then, the joint action a = a 1 × · · · × a N produces next state according to transition function P : S ×
A × S → [0 , 1] and the environment gives the team reward r : S ×
A → R to agents N .
The objective of each agent is to maximize the total accumulative reward R to cooperatively learn the globally optimal policy π ∗ =
{ π ∗ 1 , · · · , π ∗ N } that consists of the optimal policy of each agent.
In this paper, as mentioned before, we employ MARL to search optimal policy for each patch, and achieve the optimal effectiveness of DA across the entire image to further improve the performance of the target network as possible.
3. P ATCH A UTO A UGMENT
In our proposed Patch AutoAugment (PAA), we formulate the task of policy search for the patches as a cooperative multi-agent decision-making problem and adopt multi-agent reinforcement learning (MARL) to solve it.
In the following, We clarify the detailed formulation i ) the state , observation and action modeling for the policy, ii ) an effective team reward function design and iii ) the detailed
MARL algorithm for policy learning) of Patch AutoAugment.
Policy Modeling.
As illustrated in Figure 2, given the original input batch x and the corresponding label y ( i .
e ., { x, y }
= { x j , y j } bj =
and b is the batch size), we divide an image x j into N equalsized and non-overlapping patches, denoted as x j = {P ij } Ni =1 where P jj is the i -th patch of the image x j .
In our proposed method, we aim to search the augmentation policy for each patch.
Therefore, in MARL formulation, the augmentation policy of the patch P i is controlled by an agent i ∈ N and we detail the state , observation and action for the augmentation policy as below.
State.
As aforementioned, the selection of augmentation operation for a patch is closely bound up with the contextual relationship between regions.
Therefore, the augmentation policy needs to perceive the image semantics and we take the deep features of the whole image extracted by a backbone ( e . g ., ImageNet pre-trained ResNet-
(He et al., 2016)) as the global state s which is visible to all agents.
We analyze the impact of feature extractor initialization in Appendix F.
Observation.
Apart from capturing state ( i .
e ., the global information), the agent only use their own observation ( i .
e
., the local information) which is invisible to other agents.
The augmentation policy seeks to choose augmentation operation based on the content of the patch and the observation is generally part of the state.
Considering all these factors, we utilize the deep features of the i -th patch as observation o i , which are extracted by the same backbone as the state extractor.
Action.
The augmentation policy is responsible for choosing which transformations to apply from pre-deﬁned operations.
Following the previous automated DA methods (Cubuk et al., 2018; Lim et al., 2019), we deﬁne the ﬁfteen operation functions ( i .
e ., ShearX/Y, TranslateX/Y, Rotate, Invert, Equalize, Solarize, Posterize, Contrast, Color, Brightness, Sharpness, RandomErasing, Cutout,
Mixup, Cutmix) to construct the action space A .
Given the state s and the observation o i , according to policy π i ( a i | o i , s ) , each agent i determines an action a i ( · ) ∈ A , which is the operation performed on the patch.
Each operation is associated with two hyperparameters: probability p and magnitude m .
In order to dramatically and effectively reduce the action space for augmentation policy, similar with (Cubuk et al., 2020; Ho et al., 2019; Lim et al., 2019), we take a ﬁxed probability and magnitude schedule.
Among them, the probability of applying the operation is sampled from the uniform distribution ( i .
e ., p i ∼ U (0 , 1) .
Following (Ho et al., 2019), we employ the same linear scale to be the magnitude schedule.
In summary, the processed i -th patch in image x
j is denoted as (cid:101) P ij = a i ( P ij ) with the probability p i and the magnitude of a i ( · ) is m i , otherwise (cid:101) P ij =
P ij .
Note that most operations are label-invariant, except Mixup (Zhang et al., 2017) and CutMix (Yun et al., 2019) are label-disturbing operations that combine different patches as well as their labels.
We take
Mixup as an example, then (cid:101) P ij = λ
P ij + (1 − λ ) P it with probability p i , where P it is a patch from another image x t , t (cid:54) =
j , λ ∼ Beta ( α, α ) , for α ∈ (0 , ∞ ) , and the one-hot label is modiﬁed as (cid:101) y
j
= λN y j
+ (1 − λ ) N y t .
Once all patches are processed by the corresponding operations chosen by the augmentation policies, we obtain the image augmented by our PAA, denoted as (cid:101) x
j = { (cid:101) P 1 j , ··· , (cid:101) P Nj } , and the ﬁnal label (cid:101)
y j .
More operation details are shown in Appendix A.
Reward Function.
The reward function is of importance to guide the agents to learn so that they follow desired behaviors.
The previous work, AdvAA (Zhang et al., 2019b), attempts to increase the training loss of the target network to generate harder augmentation policies and explore the weakness of the target network.
Inspired by AdvAA, we reformulate the reward design appropriately under our conﬁguration.
In our proposed PAA, the common objective of all agents is to improve the performance of the mainstream target task through enhancing the beneﬁts of DA.
Therefore, we compare the feedback of target network φ ( · ) on the augmented data processed by our proposed PAA (cid:101) x with the original data x and take their difference on the training losses as the reward for the policy in MARL, as in Eq.
[Equation]
where x and y denote raw inputs and labels in supervision tasks.
In our PAA model, all agents are encouraged to cooperate to achieve the common goal, thus we adopt the team reward function design ( i .
e ., the shared reward mechanism in MARL) as Eq.
(1) for all agents to make the joint augmentation policy achieve the optimal effectiveness.
Policy Learning.
Here, we introduce the training for the augmentation policies mentioned above.
Considering that the action space is discrete, we adopt multi-agent Advantage Actor-Critic algorithm (Lowe et al., 2017) to learn the augmentation policies and encourage the coordination behaviors.
In
MARL, the framework of centralized training with decentralized execution (Foerster et al., 2018;
Rashid et al., 2018) is generally adopted.
More concretely, each agent i has an actor which is to learn discrete policy π i ( a i
| o i , s ) and agents share a common critic which aims to estimate the value of global state V π ( s ) .
And we use the centralized critic to train decentralised actors.
Here, we reformulate it appropriately for our task.
We model the centralized action-value Q function that takes the actions of all agents in addition to state information s and outputs the Q-value for the team, formulated as:
[Equation]
where a is the joint action of all agents
a = { a i , · · · , a N } and R t = (cid:80) Tl =
γ l r t + l is the long-term discounted reward.
Then, the advantage function on the augmentation policy is given as follows:
[Equation]
The critic network is a Multi-layer Perceptron (MLP).
And we use ϕ to denote the its parameters.
We take the square value of the advantage function A π as the loss function to update ϕ :
[Equation]
[Table] [Table]
The global policy network is a fully convolutional network (FCN), denoted as θ .
Besides, to further achieve the ability to cooperate, similar to (Foerster et al., 2018; Rashid et al., 2018), the parameters of the actor networks of all agents are shared.
The loss function for updating θ is deﬁned as:
[Equation]
Framework Summary.
In this part, we summarize the overall framework of our proposed PAA.
As shown in Figure 2, PAA ﬁrst divides an image into a grid of patches.
Then, we use a feature extractor to obtain the deep features of the whole image as the state.
Each agent draws its individual observation, i .
e ., the deep features of the patch.
According to the global state ( i .
e ., the semantics of the entire image) and the local observation ( i .
e ., the content of the patch), the actor networks output the augmentation operations of patches to further construct the joint operation map performed on the whole image.
The augmented images are processed by our proposed PAA and then we input them to the target task network for parameters updating.
Moreover, the feedback of the target network is used as the team reward signal to update the policy network.
E XPERIMENTS 4. E XPERIMENT O VERVIEW
In this section, to study the effectiveness of Patch AutoAugment (PAA), we conduct experiment on image classiﬁcation, ﬁne-grained image recognition and local tasks where local information is of importance, e .
g ., object detection.
Speciﬁcally, we focus on CIFAR-10, CIFAR-100 (Krizhevsky et al., 2009) and ImageNet (Deng et al., 2009) datasets as well as three ﬁne-grained object recognition datasets, i .
e ., CUB-200-2011 (Wah et al., 2011), Stanford Cars (Krause et al., 2013), FGVCAircraft (Maji et al., 2013) and Pascal VOC 2007 (Everingham & Winn, 2009).
We describe the datasets in detail in Appendix B.
We compare PAA with baseline pre-processing, Cutout (DeVries & Taylor, 2017), Mixup (Zhang et al., 2017), Cutmix (Yun et al., 2019), Co-Mixup (Co-Mix)
(Kim et al., 2021), AutoAugment (AA) (Cubuk et al., 2018), Fast AutoAugment (FastAA) (Lim et al., 2019), RandAugment (RA) (Cubuk et al., 2020), DADA (Li et al., 2020) and Adversarial AA (AdvAA).
The baseline follows (Zoph et al., 2018;
Yamada et al., 2018; Gastaldi, 2017): standardizing the data, horizontally ﬂipping with 0.5 probability, zero-padding and random cropping.
Our policy model architecture design mainly follows (Chu et al., 2016), and we make some adjustments according to our task scenario.
More details about our policy network architectures and the hyperparameters of the target model and our policy model are supplied in Appendix C and D, respectively.
Moreover, we default the number of patches N to 16 except for CIFAR tasks N = 4 .
To ensure the reliability of our experiments, we run each experiment four times using different random seeds.
4. R ESULTS AND A NALYSIS Classiﬁcation
Results on CIFAR-10 and CIFAR-100.
For CIFAR-10 and CIFAR-100, we examine on Wide-ResNet-28-10 (WRN-28-10) (Zagoruyko & Komodakis, 2016), Shake-Shake (SS) (Gastaldi, 2017) and Pyramid-Net+ShakeDrop (Pyramid+SD) (Han et al., 2017; Yamada et al., 2018) models.
The results are reported in Table 1, which shows the proposed approach consistently outperforms several state-of-the-art DA methods.
We observe that the improvement of performance is relatively slight, due to the small image size of CIFAR which is 32 × 32 .
In the following, we further apply our proposed PAA on datasets with larger image sizes and other networks.
[Table] [Table]
Classiﬁcation Results on ImageNet.
As shown in Table 2, we evaluate our method on ResNetand ResNet-
(He et al., 2016) backbone on ImageNet, and our PAA signiﬁcantly improves the performance of the target networks.
The results further demonstrate that our proposed method is an effective DA technique for consistent and expressive beneﬁts for datasets with larger image sizes.
Effectiveness of Fine-grained Classiﬁcation.
Furthermore, we evaluate our proposed method on ﬁne-grained image recognition tasks.
According to previous work (Du et al., 2020; Chen et al., 2019), we take ResNet-50 and ResNet-101 as the backbones.
The results are shown in Table 4, which illustrates that the performance of PAA is consistently better than other methods and PAA has achieved remarkable performance on these challenging ﬁne-grained tasks.
Effectiveness of Local Tasks.
In order to further verify the effectiveness of our proposed method, we conduct experiments on the object detection task.
We adopt the mainstream detector Faster RCNN (Ren et al., 2015) (ResNet-50) on the dataset Pascal VOC 2007 (Everingham & Winn, 2009) and show the mAP results on the Table 5.
We observe that the proposed PAA achieves consistent improvements and PAA is indeed effective for the local tasks like object detection.
4. C OMPLEXITY A NALYSIS
In this section, in order to further demonstrate the performance of PAA in terms of complexity, we compare the policy search time and training time of PAA with AA (Cubuk et al., 2018), FastAA (Lim et al., 2019) and AdvAA (Zhang et al., 2019b), as illustrated in Table 3.
As shown in Table 3, compared to the previous works, PAA requires the fewest total computational resources, and the search time is almost negligible.
As for the parameters, the total number of PAA model parameters (about 0.23M) is less than 1 % of the target network ( e .
g ., ResNet50: about 25.5M).
In summary, the main methods to reduce the computational cost lie in three aspects: 1)
The augmentation policy network is jointly optimized with the target network, similar to (Zhang et al., 2019b;
Lin et al., 2019).
Namely, our proposed method searches for policies in an online manner, obviating thousands of policies validation on a small proxy network and the requirement for retraining the target network.
Besides, we use ﬁxed schedules for the two corresponding hyperparameters ( i .
e ., probability and magnitude) of each transformation to effectively reduce the search space, which makes it easier to search for effective policies.
By these means, PAA compresses most of the policy search time 2)
As mentioned before, a MARL algorithm is adopted, in which all agents parallelly learn the augmentation policies, to reduce the training time .
Compared with the previous DA methods using image-by-image sequential transformations, we perform parallel transformations on tensor.
Speciﬁcally, we pick the patches in a batch performing the same operation to reconstruct
[Table] [Table]
a new tensor.
And we use Kornia 1 to realize tensor transformations on GPU to further reduce the processing time which is included in the training time.
4. A BLATION S TUDY
In this section, we study the effectiveness of MARL and also discuss the design of patch numbers, through several ablation experiments.
Performance of random policies.
We randomize the augmentation policy for each patch, dubbed
Patch RandAugment and compare it with our proposed PAA.
To be speciﬁc, the augmentation policy is randomly selected from predeﬁned transformations with uniform probability.
As shown in Table 6, Patch RandAugment (PRA) leads to considerable appreciable improvements.
However, our Patch
AutoAugment with meaningful guidance signiﬁcantly surpasses the random patch policies.
Performance of the policies searched using single-agent RL.
Furthermore, we directly use the same single-agent RL to search for each patch’s policy, termed as Patch SingleAutoAugment (PSAA) and compare it to our PAA, where the search task is a cooperative multi-agent problem.
In short,
PSAA ignores the contextual relationship between patches, where the agents are independent and patches are non-cooperative.
The results (see Table 6) indicate that taking the contextual information into account with cooperative RL-agents has improved the joint effectiveness of DA.
Impact of the number of patches.
We explore the effect of the number of patches on the target model performance.
As shown in Table 7, we respectively set the number of patches
N = { 4 , 16 , 64 , 256 , 1024 } .
The results show that on ImageNet / CUB-200-2011, when patch numbers N increase, the performance accuracy ﬁrst increases and then decreases.
In addition, when
N = 16 , PAA achieves the best performance.
We analyze that the small number of patches may cause PAA to be unable to effectively explore local diversity, and its advantages would be limited.
In particular, when N = 1 , PAA degenerates into image-wise automated DA.
In contrast, under too larger values ( e . g ., the extreme case is to search for DA policy for each pixel), the local semantic consistency is broken and the beneﬁt brought by the consideration of contextual relationship between patches is gradually overtaken.
4. V ISUALIZATION
Policy Visualization.
Grad-CAM (Selvaraju et al., 2017) is used to localize the important regions in the image.
Therefore we can calculate the importance score of each patch, and we divide the importance scores into four bins, i .
e ., very important, important, normal and not important .
Then, we categorize patches into four groups and draw four stacked area charts showing the percentages of operations selected by PAA augmentation policies over time.
We take ResNet-50 backbone trained on CUB-200-2011 as an example.
Since our proposed PAA searches for patch policies in an online manner, the strategies change dynamically over time as shown in Figure 4.
At the beginning of the training process, the selected actions are messy since the MARL network is in the exploratory stage.
At the tail end of the training, the target network has converged, causing the percentage of all operations to be almost the same and the percentage to ﬂatten out.
In addition, we have some interesting ﬁndings which may provide some insights to the DA community.
i )
The optimal augmentation strategies of patches vary by their importance levels.
Therefore, it is necessary to take the image content into account when performing data augmentation.
ii )
In
[Table]
[Figure]
[Table]
[Figure] [Figure]
the middle of the training process, different types of patches prefer to select different augmentation operations.
Concretely, as illustrated in Figure 4, for the important patches, color transformation is mostly picked.
For the unimportant patches, RandomErasing and Cutout are usually chosen by
PAA.
Important patches commonly take along semantic information that is related to mainstream tasks.
It is better to choose the mild transformations ( e .
g ., color ) for them, which can effectively protect the semantic information from being damaged.
In contrast, unimportant patches typically carry unexpected features (Singla et al., 2021) which are causally unrelated to the desired class.
Severe transformations ( e . g ., Cutout , RandomErasing) could be chosen for them, which introduce noise and disturbance to reduce the impact of unexpected features on the target network learning.
Grad-CAM Visualization.
Here, we adopt Grad-CAM (Selvaraju et al., 2017) to visualize the learned features to intuitively show the impact of PAA, as shown in Figure 3.
We take the ResNetas the backbone which is trained with dataset processed by 1) Baseline 2) AutoAugment and 3) Patch
AutoAugment, respectively.
We observe that the model trained with PAA focus on more task-related areas rather than spurious correlations ( e .
g ., the branch where the bird stands) or overemphasized features ( e .
g ., birds’ claws).
More visualization results are provided in the Appendix E.
C ONCLUSION
In this paper, we propose Patch AutoAugment (PAA), a more ﬁne-grained automated data augmentation approach.
Our method adopts multi-agent reinforcement learning to automatically search for the optimal augmentation policies for patches, and encourages agents to cooperate with each other to further achieve the joint optimal policy across the entire image.
Extensive experiments demonstrate that PAA improves the target network performance with low computational cost in various tasks.
Meanwhile, we use visualization to show that PAA is beneﬁcial for the target network to localize more class-related cues.
Furthermore, we hope that our visual observations of policies will be useful for future development.
In future work, we will investigate different schemes on dividing different regions.
Additionally, our method is naturally aligned with the patch token mechanism in the current vision transformers (Dosovitskiy et al., 2020; Touvron et al., 2021; Liu et al., 2021b) and the data augmentation speciﬁc to vision transformers has not been extensively studied.
Therefore, we leave the automated data augmentation for vision transformers to future work, which may provide some interesting insights to the community.
A PPENDIX A O PERATIONS D ETAILS
Following (Cubuk et al., 2018; Ho et al., 2019; Lim et al., 2019; Zhang et al., 2019b), we deﬁne the ﬁfteen common augmentation operations to form the action space.
Here, we detail the description of these operations, as illustrated in Table 8.
In addition, we give magnitudes range of augmentation operations corresponding to hyperparameters of the functions in the Kornia PyTorch library.
Some operations, such as Cutout and CutMix, have no parameters.
Additionally, in our implementation, we pick out the patches that perform the same operation and put them into a new tensor.
Then, we speed up the process by performing parallel transformations on the tensor.
Tensor transformations on GPU can be realized by Kornia (Riba et al., 2020) to reduce the computational costs.
In particular, when performing the label-disturbing operations ( i .
e ., Mixup and CutMix), a patch needs to mix with another patch that is randomly selected from the new tensor.
Namely, a patch is mixed with another patch that performs the same operation.
B D ATASETS
We evaluate Patch AutoAugment (PAA) on the following datasets: CIFAR-10 (Krizhevsky et al., 2009), CIFAR-100 (Krizhevsky et al., 2009), ImageNet (Deng et al., 2009) and three ﬁne-grained
[Table]
image recognition datasets (CUB-200-2011 (Wah et al., 2011), Stanford Cars (Krause et al., 2013), FGVC-Aircraft (Maji et al., 2013)) and Pascal VOC (Everingham & Winn, 2009).
To be speciﬁc, both CIFAR-10 and CIFAR-100 have 50,000 training examples.
Each image of size 32 × 32 belongs to one of 10 categories.
ImageNet dataset has about 1.2 million training images and 50,000 validation images with 1000 classes.
Original ImageNet data have different sizes and we resize them to 224 × 224 .
In addition, we evaluate the performance of our proposed PAA on three standard ﬁne-grained object recognition datasets.
CUB-200-2011 (Wah et al., 2011) consists of 6,000 train and 5,800 test bird images distributed in 200 categories.
Stanford Cars (Krause et al., 2013) contains 16,185 images in 196 classes.
The FGVC-Aircraft dataset contains 10,200 images of aircraft, with 100 images for each of 102 different aircraft model variants, most of which are airplanes.
The image size in the above three datasets is 448 × 448 .
The Pascal VOC dataset consists of a set of images and each image has an annotation ﬁle giving a bounding box and object class label for each object in one of the twenty classes present in the image.
The data has been split into 50 % for training and 50 % for testing.
In total there are 9,963 images, containing 24,640 annotated objects.
C M ODEL A RCHITECTURE
Here, we provide the detailed model architecture for each component in our PAA augmentation policy model, including feature extractor network, actor network and critic network.
We use the pretrained on ImageNet ResNet-18 backbone (excluding the ﬁnal avgpool and softmax layer) to extract the deep features of the image and the patch, which are denoted as the state s and the observation o i respectively.
As for the global policy network and the critic network, the detailed model architectures are shown in Table 9.
The policy network is a fully convolutional neural network (FCN).
The parameters of the actor networks of all agents are shared.
Although they share parameters, each agent’s policy is not identical.
When determining an action, each agent’s policy relies on different activations of the global policy network.
D H YPERPARAMETERS
We detail various target models hyperparameters ( e .
g ., batch size, learning rate and training epochs) on CIFAR-10, CIFAR-100, ImageNet, CUB-200-2011, Stanford Cars and FGVC-Aircraft in Table 10.
We do not speciﬁcally tune these hyperparameters, and all of these are consistent with previous works (Cubuk et al., 2018;
Lim et al., 2019; Dabouei et al., 2021;
Du et al., 2020; Chen et al., 2019). [Table]
In addition, the time horizon T determines the times of augmentations performed sequentially on a patch.
Therefore the agent can output more than one augmentation for a patch by setting T > 1 .
In our PAA model, we set time horizon T = 1 , i .
e ., the augmentation policies take actions at every time step for saving computational cost and avoiding over-regularization.
Even in the single-step decision-making process ( i .
e ., T = 1 ), RL can still effectively search for the optimal policy as illustrated in (Mnih et al., 2016; Singh et al., 2000).
In addition, we use the SGD optimizer with an initial learning rate of 1e-4 to train the actor network and the critic network.
E M ORE G RAD -CAM V ISUALIZATION
In this section, we provide more Grad-CAM (Selvaraju et al., 2017) results of ResNet-50 models trained using baseline augmented data, AA (Cubuk et al., 2018) and our proposed PAA, respectively, as shown in Figure 5.
The visualization results demonstrate that our proposed PAA improves the localization ability of the target network and tends to help the target network focus on more parts of the foreground object.
In short, our proposed PAA make the target network focus on the important or representative regions closely related to the class within an image.
F I MPACT OF THE INITIALIZATION OF THE FEATURE EXTRACTOR
We further study the impact of the initialization of the feature extractor as shown in Table 11.
We ﬁrst observe that the performance of the Non-pretrained ResNet-18 scheme (using non-pretrained
ResNet-18 as the state/observation extractor) is still much better than the baseline.
In other words, our agents do not rely on the pre-trained ResNet to produce features to be used as the state or observation.
We also notice that the performance of the Non-pretrained ResNet-18 scheme is slightly lower than the Pretrained ResNet-18 .
This may be because that at the very early stage of the training, the features extracted by Non-pretrained ResNet-18 are not as good as Pretrained ResNet-18 in practice.
G D ETAILS OF COMPUTATIONAL COST
The computational cost of our proposed PAA is estimated on GeForce GTX 1080 Ti while AA (Cubuk et al., 2018) and AdvAA (Zhang et al., 2019b) are on NVIDIA Tesla P100.
Besides, FastAA (Lim et al., 2019) is estimated on NVIDIA Tesla V100.
The computational cost is mainly used for searching policies and training the target networks.
[Figure]