Paper:
Data augmentation is a widely used technique in many machine learning tasks, such as image classification, to virtually enlarge the training dataset size and avoid overfitting. Traditional data augmentation techniques for image classification tasks create new samples from the original training data by, for example, flipping, distorting, adding a small amount of noise to, or cropping a patch from an original image. In this paper, we introduce a simple but surprisingly effective data augmentation technique for image classification tasks. With our technique, named SamplePairing, we synthesize a new sample from one image by overlaying another image randomly chosen from the training data (i.e., taking an average of two images for each pixel). By using two images randomly selected from the training set, we can generate N 2 new samples from N training samples. This simple data augmentation technique significantly improved classification accuracy for all the tested datasets; for example, the top-1 error rate was reduced from 33.5% to 29.0% for the ILSVRC 2012 dataset with GoogLeNet and from 8.22% to 6.93% in the CIFAR-10 dataset. We also show that our SamplePairing technique largely improved accuracy when the number of samples in the training set was very small. Therefore, our technique is more valuable for tasks with a limited amount of training data, such as medical imaging tasks.

Review:
The paper investigates a method of data augmentation for image classification, where two images from the training set are averaged together as input, but the label from only one image is used as a target.  Since this scheme is asymmetric and uses quite unrealistic input images, a training scheme is used where the technique is only enabled in the middle of training (not very beginning or end), and in an alternating on-off fashion.  This improves classification performance nicely on a variety of datasets.
This is a simple technique, and the paper is concise and to the point.  However, I would have liked to see a few additional comparisons.
First, this augmentation technique seems to have two components:  One is the mixing of inputs, but another is the effective dropping of labels from one of the two images in the pair.  Which of these are more important, and can they be separated?  What if some of the images' labels are changed at random, for half the images in a minibatch, for example?  This would have the effect of random label changes, but without the input mixing.  Likewise, what if both labels in the pair are used as targets (with 0.5 assigned to each in the softmax target)?  This would mix the images, but keep targets intact.
Second, the bottom of p.3 says that multiple training procedures were evaluated, but I'd be interested to see the results of some of these.  In particular, is it important to alternate enabling and disabling SamplePairing, or does it also work to mix samples with and without it in each minibatch (e.g. 3/4 of the minibatch with pairing augmentation, and 1/4 without it)?
I liked the experiment mixing images from within a restricted training set composed of a subset of the CIFAR images, compared to mixing these images with CIFAR training set images outside the restricted sample (p.5 and Fig 5).  This suggests to me, however, that it's possible the label manipulations may play an important role.  Or, is an explanation why this performs not as well that the network will train these mixing images to random targets (that of the training image in the pair), and never see this example again, whereas by using the training set alone, the mixing image is likely to be repeated with its correct label?  Some more discussion on this would be nice.
Overall, I think this is an interesting technique that appears to achieve nice results.  It could be investigated deeper at some key points.