Improved Image Augmentation for Convolutional Neural Networks by Copyout and CopyPairing
Authors: Philip May
Venue:  (2020)
Keywords: image augmentation
cnn
images
augmentation

Abstract: Image augmentation is a widely used technique to improve the performance of convolutional neural networks (CNNs). In common image shifting, cropping, flipping, shearing and rotating are used for augmentation. But there are more advanced techniques like Cutout and SamplePairing.

In this work we present two improvements of the state-of-the-art Cutout and SamplePairing techniques. Our new method called Copyout takes a square patch of another random training image and copies it onto a random location of each image used for training. The second technique we discovered is called CopyPairing. It combines Copyout and SamplePairing for further augmentation and even better performance.

We apply different experiments with these augmentation techniques on the CIFAR-10 dataset to evaluate and compare them under different configurations. In our experiments we show that Copyout reduces the test error rate by 8.18% compared with Cutout and 4.27% compared with SamplePairing. CopyPairing reduces the test error rate by 11.97% compared with Cutout and 8.21% compared with SamplePairing.

Copyout and CopyPairing implementations are available at https://github.com/anonym/anonym.

Under review as a conference paper at ICLR 2020
IMPROVED IMAGE AUGMENTATION FOR CONVOLU- TIONAL NEURAL NETWORKS BY COPYOUT AND COPY- PAIRING
Anonymous authors Paper under double-blind review
ABSTRACT
Image augmentation is widely used technique improve the performance of convolutional neural networks (CNNs). In common image shifting, cropping, flipping, shearing and rotating advanced techniques like Cutout and SamplePairing.
used for augmentation. But there are more
In this work we present two improvements of the state-of-the-art Cutout and SamplePairing techniques. Our new method called Copyout takes a square patch of another random training image and copies it onto a random location each image used for training. The second technique we discovered is called CopyPairing, It combines Copyout and SamplePairing for further augmentation and even better performance.
We apply different experiments with these augmentation techniques on the CIFAR- dataset to evaluate and compare them under different configurations. In our experiments we show that Copyoul reduces the test error rate by 8. 8.18% compared with Cutout and 4.27% compared with SamplePairing. CopyPairing reduces the test error rate by 11.97% compared with Cutout and 8.21% compared with SamplePairing.
Copyout and CopyPairing httpss//githbb.ommmmmmymmm/mgn
implementations are available
INTRODUCTION
Image augmentation is data augmentation method that generates more training data from the existing training samples. This way the neural network is trained with more different images. This helps to generalize better on new and unknown images and leads to better performance with smaller datasets. Image augmentation is especially useful in domains where training data is limited or expensive to obtain like in biomedical applications.
Image augmentation is regularization technique like Dropout (Hinton al. 2012) and weight regularization (Ng) 2004). But different in one way. It is not part of the network itself which makes relatively easy and cheap apply. Images can be augmented runs the neural network, the CPU can do the augmentation for the next batch in an asynchronous manner.
runtime. While the GPU
Two state-of-the-ard techniques for advanced image augmentation are called Cutout Devriee & Taylor 2017) and SamplePairing (Inoue 2018). our work we show two improvements these techniques. Our main contributions are:
1. Copyout image augmentation described in section 2
2. CopyPairing image augmentation described in section 3
3. Several experiments to compare them in Section 4.3 ff.
4. An analysis of the reasons why these methods improve performance in Section

---

Under review as a conference paper at ICLR 2020
Figure 1: Examples of CIFAR-10
Krizhevsky 2009) Images augmented by Copyout.
2
COPYOUT
The Copyout image augmentation technique should be applied on runtime on each image before used build the mini-batch for training. The results must not be stored and reused. This would reduce the variance of the training images and reduce the performance of the CNN. The augmentation conducted in these three steps:
1. Select target training image to augment. On this target image select square area random location. By this way it is possible that the randomly selected square leaves the image on one or sides. This can effectively change the shape of the square rectangle. As stated by Devries & Taylor (2017) this is an important feature and improves performance. In our implementation the center of the square never leaves the target image.
2. Randomly select source image from the training set. On this source image also select area of the same extend as the one we selected on the target image. This second area is also selected on a random location but must be fully placed within the source image.
3. Copy the area of the source image to the target image.
The extend of the square is hyperparameter which must be set before training starts and does not change during the training process. Apparently the extend must be smaller than the training images. Devries Taylor (017)) found that for Cutout the extend of the square is more important than the shape. This is the reason why we also decided to use squares for simplicity.
Examples of images augmented by Copyout are given in Figure 1
COPYPAIRING
CopyPairing is mixture of Copyout and SamplePairing (Inoue 2018) and Appendix A.2p. CopyPair- ing follows the same three phases as SamplePairing. The only difference is that replaces the phases where SamplePairing disabled by using Copyout. CopyPairing conducted by the following three steps:
1. Only Copyout is enabled and SampleParring is completely disabled for the first epochs.
2. SamplePairing is enabled for a few epochs and then Copyout used for the next few epochs. This alternating for the majority of the epochs.
3. At the end of training only Copyout is enabled while SampleParring is completely disabled again. This also called fine-tuning.
EXPERIMENTS
4.1 EXPERIMENTAL SETUP
The CNN for experiments consists of six 2D convolution layers with Glorot uniform kernel initializer (Glorot& & Bengio 2010). Rectified Linear Unit (ReLU) activation (Glorot et al. 2011) and Batch Normalization (loffe & Szegedy 2015). Two dense layers applied the output
2

---

Under review as a conference paper at ICLR 2020
Copyout- Buf. 16 Median E. R.; 6.08 %
Copyout Buf. 128 Median E. 6.06
Capyout
Median E. R.; 6.16 %
Copyout- Buf. 10000 Median E. 6.17
Copyout - no Buf Median E. 6.56 %
5.8
6.4
Figure 2: Comparison of Copyout with different buffer sizes to store augmented images. "Copyout no Buf" selects the source images from the whole training set without buffer and augmentation. Method name with buffer size and median test error rate is given by the caption. Each experiment has been repeated at least 12 times to draw the boxplots.
and combined with two Dropout layers (Hinton et al. 2012). Although loffe & Szegedy (2015)
propose to place the Batch Normalization layer front of the activation function found out that this CNN performs better when activation and Batch Normalization are interchanged. Appendix A.] Listing presents the definition the concrete CNN with Keras (Chollet et al. 2015).
As the optimizer we use Adam with parameters as provided in the paper Kingma & Ba (0114) (learning rate 0.001 e.g.). The mini-batch size 128 and we train for 1200 epochs. The CIFAR- T0 Dataset (Krizhevsky 2009) is used for reference and we try to minimize the test error rate in experiments.
4.2 BASIC AUGMENTATION
For basic augmentation we randomly rotate the images by up to 15 deg. to the left and right, randomly flip them horizontally and randomly presents our implementation of basic augmentation with the Keras ImageDataGeneraate
them by factor of up 0.2. Appendix A. Listing
class.
It is important where and how basic augmentation is applied. When it is only applied to the target image and not the source image the performance will be reduced. It is important that the source image is also augmented. This be achieved when basic augmentation is applied after the square has been copied. But we selected another method.
Our basic augmentation is combined with Copyout and SamplePairing by using buffer. We apply basic augmentation on the target image and store it in a buffer. For Copyout and SamplePairing we then select the source image from this buffer randomly. This way have augmentation for source and target images.
Figure 2 displays different experiments with varying buffer sizes. shows that the buffer size can be small without reducing performance. It also shows that augmentation of source images is important. The "Copyout- no Buf" experiment selects the source images from the whole training set without taking them from the buffer and without applying basic augmentation on the source images. Its median test error rate is significantly increased.
4.3 METHOD COMPARISON
We compared the different augmentation techniques while using the setup described Section 4.1 All experiments have been combined with basic augmentation (see Section T]. Fach experiment was

---

Under review as a conference paper at ICLR 2020
Baseline
Median E. R.: 7.09 %
Cutout
Median R.:
SamplePairing 8:2 Median E. R. 6.33 %
Copyout
Median E. R.: 6.06 %
CopyPairing 1:1 Median R.:
5.6
5.8
6.0
6.2
6.4
6.6
6.8
7.0
7.2
Test Error Rate (%)
Figure 3: Comparison Cutout, SamplePairing, Copyout and CopyPairing. Also illustrated the baseline with just default image augmentation and without any further augmentation. Method name and median test error rate is given by the caption. Each experiment has been repeated at least 12 times to draw the boxplots.
repeated at least 12 times to get meaningful boxplots. Boxplots with test error rates are shown in
Figure 3
For the buseline experiment only used basic augmentation and reached median test error rate of 7.09%. With Cutout augmentation we reached median test error rate of 6.6%.
For the SamplePairing experiment we used the following configuration: In phase we disabled SamplePairing for the first 100 epochs. Phase lasted 900 epochs, SamplePairing was turned on for 8 epochs and turned off for epochs alternating. Phase 3 (fine-tuning) lasted the remaining 200 epochs. This corresponds to the setup of Inoue (2018). rate of 6.33%.
With SamplePairing we reached median test
For Copyout augmentation we selected a 16 pixel extend of the square as hyperparameter and reached median test eиrо rate of 6.06%. This is an 8.18 improvement compared with Cutout and a 4.27% 27% improvement compared with SamplePairing.
The experiment with CopyPairing part we selected a 16 pixel extend of the square as hyperparameter. In phase we disabled the SamplePairing part for the first 100 epochs with only Copyout enabled. Phase lasted 900 epochs with alternating SamplePairing and Copyout part for one epoch each. This was executed alternating. Phase (fine-tuning) lasted the remaining 200 epochs with the SamplePairing part turned off and only Copyout enabled.
done with the following configuration: For the Copyout
We reached a median test error rate of 5.81%. %. This an 11.97% improvement compared with Cutout and an 8.21% % improvement compared with SamplePairing Even compared with the Copyout augmentation we reached an improvement of 4.13%
4.4 SQUARE AREA EXTENT COMPARISON FOR COPYOUT
In our experiments with Copyout selected 16 pixel extend of the square as hyperparameter. To examine if the selection was the best choice we executed three different experiments. One with 16 pixel extend, one with 15 and 17 pixel extends. The result is that larger and smaller extend numbers for the square are lowering the performance for our experimental setting and that 16 pixel extend is the best choice for this setup (see Figure 4).
4

---

Under review as a conference paper at ICLR 2020
Copyout- Extend Median E. R.; 6.06 %
Copyout Extend 17 Median E. R.: 6.09 %
Copyout Extend 15 Median R.: 6.12 %
5.9
6.0
62
6.3
Test Error Rate (%)
Figure 4: Comparison of three different extends of the square area for Copyout. Method name with the selected extend and median test error rate is given by the caption. Each experiment has been repeated least 12 times to draw the boxplots.
4.5 COPYPAIRING HYPERPARAMETER COMPARISON
As with Copyout we also wanted compare the different hyperparameter settings for CopyPairing. In particular phase of CopyPairing provides many different options for configuration. Therefore we selected different ratio of epochs with SamplePairing and Copyout. The results presented in Figure 5 and show that a ratio of one epoch with SamplePairing and one epoch with Copyout is best (see CopyPairing 1:1).
As second experiment we investigated if the fine-tuning phase is needed. In this third phase (called fine-tuning) the SamplePairing normally turned off and just Copyout is used for the last epochs of the training. In this experiment we just skipped this fine-tuning phase and extended the second phase until the end training. The result is also shown in Figure and indicates that the fine-tuning phase should be retained as it improves the performance.
CONCLUSION
In this work the application of Copyout and CopyPairing is described. Considering the boxplots, we conclude that they are statistically significant improvement of the state-of-the-art image aug- mentation techniques Culout and SamplePuiring (see Section 4.3) But what are the reasons for this improvement?
We conducted several experiments to compare the average magnitude of feature activations as done by Devries & Taylor (017). But this gave us ambiguous results. That is the reason why we give intuitive explanations. Cutout and Copyout both randomly remove visual features of the training images and forces the network to focus on wider number of different features to distinguish between different images. This helps to generalize better on unknown pictures. Cutout and Copyout also help to handle occlusion. But what is the relevant distinction between Copyout and Cutout? Why does Copyout perform better?
Cutout performs the occlusion with just gray square which can be considered wasted space. Copyout fills this space with more visual information which can observed Figure [ One wing of the airplane on the right image is occluded by a frog. This way Copyout not only removes random visual features but also adds new random ones. These random features better correspond to the natural effect of occlusion. In real images for example truck more likely not occluded by grey box but by a car. This helps with further generalization. The small squares of the Copyout technique with additional visual features moreover forces the CNN differ between important and less important

---

Under review as a conference paper at ICLR 2020
CapyPairing 1:2 Median E. R.; 5.96 %
Pairing 1:1 Median E. 5.81
Pairing Median E. R.; 5.87 %
Cairiai 3:1 Median E. 5.87
CopyPairing 1:1 noFT Median E. R.: 5.88 %
5.6
5.7
6.0
(%)
Figure 5: Comparison of different CopyPairing experiments. The ratios of SamplePairing and Copyout epochs have been varied. For example 3:1 means epochs of SamplePairing followed by epoch of Copyout in phase 2 of the SamplePairing technique. CopyPairing 1:1 noFT shows the results of a 1:1 CopyPairing experiment without phase (fine-tuning). Method name with the selected ratio and median test error rate is given by the caption. Each experiment has been repeated at least 12 times to draw the boxplots.
features. The intuitive explanation is that the network learns to focus more closcly to the different details to distinguish between important features and misguiding details.
The best presented augmentation scheme is CopyPairing. But why does it improve performance compared to SamplePairing? The reason might be that SamplePairing is wasting augmentation potential when it is deactivated for few epochs in phase 2 and completely deactivated in phase for fine- tnn-tuning (also see Appendix A.2). A.22) This gap is filled in the CopyPairing technique with using Copyout in-between.
In summary we found two new augmentation techniques that improve the performance for image processing tasks with CNNs. This is especially useful in domains where training data is limited expensive to obtain as in biomedical applications.
REFERENCES
François Chollet et al. Keras. https::/keras. 2015.
Terrance Devries and Graham W. Taylor. Improved regularizationn of convolutional neural networks with cutout. arXiv.org, 2017. URL
04552
Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference artificial intelligence and statistics, pp. 249 2010. URL htthht::/// htala/ppp//mm...coo////redggggg pwtps//sggg/ppprs////a///// gglrott00//gloottt11
Xavier Glorot, Antoine Bordes, and Yoshua Bengio. Deep sparse rectifier neural networks. In Proceed- ings of the fourteenth international conference on artificial intelligence and statistics, pp. 315-323, 2011. URLp:ttt://ppoceeiiin
ml. http:/ppws.....o....eeeeeeaaaaaa
pdf
Geoffrey F. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R. Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors. arXiv.org, 2012. URL
https::/arae http://..rg/b//////o/o/ooee 0580
Hiroshi Inoue. Data augmentation by pairing samples for images classification. arXiv.org, 2018.
URL http:htt/////iiiee org 801 10tpp//1100119199999 02929

---

Under review as a conference paper at ICLR 2020
Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv.org, 2015. URL httpss://axxxw..rg////
03167
Diederik P. Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization, arXiv.org 2014. URL htttps://axxi.. wrw.:/o.rrrrooooo aba/aaas1111 6980
Alex Krizhevsky, Cifar-10 and cifar-100 100 datasets. http::/www..s..trrrtto..ddddd
cifar. html 2009.
Andrew Y. Ng. Feature selection, 11 12 regularization, and rotational invariance. In Proceedings of the Twenty-first International Conference on Machine Learning, ICML. '04, pp. 78-, New York, NY, USA, 2004. ACM. ISBN 1--8113-838-5. doi: 10.1145/1015330.101555 feature seteee-eeeeiiinn ion
URL http::/www.andreengg.oom www.aaeggggooooooooooooooooooo regurggaarizztion and rotational inv.riiiin---neiinneeeeee
A APPENDIX
A.1 KERAS CODE
Listing shows the CNN model definition we used in our experiments. Listing[|show, shows the code and parameters for our basic data augmentation. Both is implemented with the Keras neural networks API (Chollet et 2015).
Listing I: Keras CNN Model definition for our Experiments
input_tensor _eessereorrrrrro Input (shape- (32, 32, 3))
2
Livers..BtcchNommlllaaatt. () (input_tensor)
layers
(3, 3), padding same") ) (x)
4
layers. ('relu') (x)
Layeers.Batcchhormaliaaaaac () (x)
Layers.Conv2D(96,, (3, 3), padding same") (x)
Layers.Activvtiion"'eellll (x)
8 Jayeess.MaxFooo11ng22.... 2)) (x)
9 X = aoyers.PaatchNoommlliaaii. () (x)
10
11
Layers.Conv2D (3, 3), padding = ssame)) (x)
12 layers.Activattion ('relu" (x)
13 = Layerss.BatchNommallaaiiiioo () (x)
14 Laye:s,Conv2D1128,, (3, 3), padding 'same') ) (x)
15 layerss,kttivationn(reeel (x)
16 X layers.Naxfoooiing22(11. 2)) (x)
17 = fayerss.attchoommllizztzz (x)
18
19
Laye:s,Conv2D1125, (3, 3) padding samee") (x)
20 X layers.Astiivation('relll (x)
21 Layerss.RatchNormallzzz.ooo (x)
22 layers.Conv2D11122, (3, 3) padding = samee) (x)
23 layerssActivvation(Telll (x)
24 X layerss.MaxPoooling221122 2)) (x)
25
Layers.latcolommaliaa...oo (x)
26
layers.Flatted () (x)
28
= layers.Dropour (0.4) (x)
29
X Jayers.Dende (512, activation 'relu') (x)
30
= Bayers.Dropon (0.3) (x)
output_tensol layers aayers.Dense. (classes, activation = 'softmax') (x)
32
33
model = Model Linput_tensor, output_tensory
34
7

---

Under review as a conference paper at ICLR 2020
35
medel.compill (loss cattapprial_erooseettoo optimizer Adam (), metrics
37
[' ('aco'])
Listing 2: Keras basic Data Augmentation for our Experiments
train www.iiiaaagaaaaaee wwedddaaageeeeattoee
ImageDataCenerator
2
rescale 1./255,
rotation_range 15,
4
horizontai_flit flip True,
zoom zoom_range = 0.2,
preprocessing hgg_ function preprocessi
A.2 SAMPLEPAIRING OVERVIEW
Since the Copy augmentation is building SamplePairing (Inoue 2018 we give short overview of SamplePairing. takes the image it wants to augment, second random image from the training set and then averages each pixel of the two images. The augmentation is done in these three steps:
SamplePairing is completely disabled for the first epochs.
2. SamplePairing is enabled for a few epochs and then disabled again. This is alternating for the majority of the epochs.
3. At the end of training SamplePairing is completely disabled again. This is called fine-tuning.
The SamplePairing paper describes several experiments. For the CIFAR-I0 dataset the following configuration is applied: phase SamplePairing is disabled for the first 100 epochs. Phase 2 takes 700 epochs with SamplePairing enabled for 8 epochs and turned off for 2 epochs alternating. Phase (fine-tuning) lasts for the remaining 200 epochs.
8


Reviews:
[{"review_id": "ryg1OCnTtr", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "[Summary]\nThis paper proposes two data augmentation methods that combine cutout [1] and sample paring [2] for training CNNs, Copyout and Copyparing. The authors evaluate their methods on the CIFAR-10 dataset.\n\n[Pros]\n- Data augmentation is an important regularization method for training diverse NN models\n\n[Cons]\n- The main issue is novelty. What are the differences of the proposed methods from CutMix [3] and RICAP [4]?\n- Only CIFAR-10 was used for evaluation. The results on ImageNet-1k are essential.\n- In recent papers, data augmentation methods for training CNN backbones should be validated on various architectures and downstream tasks such as object detection and semantic segmentation.\n- The method description is not specific.\n\n[1] Devries and Taylor.  Improved regularization of convolutional neural networks with cutout, ArXiv 2017.\n[2] Inoue, Data augmentation by pairing samples for images classification, ArXiv 2018.\n[3] Yun et al. CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features, ArXiv 2019.\n[4] Takahasi et al. Data Augmentation using Random Image Cropping and Patching for Deep CNNs, ACML 2018.\n\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "SylmDA_-FS", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "I think this paper is not enough to accept in ICLR because\n- Lack of novelty.\n  - CutMix [1] is very similar to Copyout.\n  - To verify the novelty, a more sophisticated description and experimental supports should be required.\n- Insufficient experiments for supporting the effectiveness of the proposed method.\n  - 6-layer convolutional networks are used, but other architectures, e.g., ResNet, should be demonstrated.\n  - Other datasets, e.g., CIFAR100 and ImageNet, should be demonstrated.\n  - Various settings, e.g., the number of training samples is limited, should be demonstrated.\n  - Need comparison with other augmentation methods, e.g., Mixup, CutMix, AutoAugment.\n- Overall, the paper is awkwardly written.\n\n[1] Sangdoo Yun et al. \"Cutmix: Regularization strategy to train strong classifiers with localizable features.\" ICCV 2019.\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "SJgZAoMH9S", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper presents an extensions to the CutOut and SamplePairing techniques for image augmentation, CopyOut and CopyPairing. CutOut itself consists of randomly masking out a rectangular region of an image. In CopyOut one chooses a source and target images, and a rectangular region from the source image is copied into target image.\n\n\nThough the extensions seem to provide an improvement in performance, we feel there are a few improvements that prevent the paper from being accepted:\n - more thorough experimental verification with various CNN architectures and larger variety of datasets datasets is needed (the original paper conducted experiments on CIFAR-10, CIFAR-100, SVHN and STL-10).\n- better exposition needed - it would have been helpful to include more examples of the original methods CutOut and Sample pairing. Sample pairing though mentioned is not described. As CopyPairing is largely based on SamplePairing, it would have been helpful to include a diagram or thorough description of it.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers have issues with novelty and quality of exposition. I recommend rejection.
Decision: Reject