# GaussianMix: Rethinking Receptive Field for Efficient Data Augmentation
## 1 Introduction  
Deep Neural Networks (DNNs) have shown promising performance in various aspects of computer vision, including image classification [1, 2], object detection [3, 4], semantic segmentation [5, 6], and more. Modern DNNs typically consist of a significantly large number of parameters. However, with an insufficient number of training samples, these large models may suffer from overfitting problem and poor generalization ability [7, 8, 9]. Furthermore, the models usually focus on a small region of input images, which results in poor generalization capability on the part of the model. To mitigate such issues, data augmentation strategies have been identified as an effective solution [10, 11]. For instance, Cutout [12] achieves a similar effect by removing random regions of the input image. More specifically, Cutout replaces the removed image region with blank pixels; these unwanted image pixels in the augmented image can hurt the model's performance, as the blank pixels alter the training data distribution [13]. Instead of removing some image regions, CutMix [13] proposes mixing a randomly selected patch (source patch) from another training sample from the target image and also mixing their labels according to the proportion of mixed pixels. Unlike Cutout, this approach prevents the augmented image from having any uninformative image pixels [12]. However, random selection of the source patch may select the background part of the image, which may not correspond to the label of the source image. In order to solve this problem, some methods use saliency information to crop a representative source patch [14, 15, 16]. Selecting the source patch from aroundthe salient region ensures that the image information is relevant and helps to enhance the model's performance. However, saliency-based data augmentation methods suffer from two limitations: (i) performance largely depends on the quality of the underlying saliency detection method [17; 18; 19] and (ii) calculating saliency information introduces computational overhead. To reduce the computational complexity, one potential solution is to compute the saliency information for a single image per mini-batch, then use this information to guide patch selection for all images within that mini-batch. However, this approach may not guarantee selection of the most salient region for each image. Although there are methods for solving a dual optimization problem to ensure selection of salient patches [14], these involve large computational burden. To address the aforementioned challenges, we draw inspiration from the CNN receptive field concept. Studies have shown that the influence of pixels on a CNN's output follows a Gaussian distribution, with the central pixels contributing more significantly than those at the edges [20]. This suggests that salient information is naturally concentrated around the center of an image. Building on this insight, we propose GaussianMix, a data augmentation strategy that selects salient source patches using a center-biased Gaussian distribution. Notably, unlike traditional saliency-based methods, GaussianMix is independent of the performance of saliency detection algorithms and effectively identifies salient regions without incurring additional computational overhead. In addition, we performed an experiment to verify the correctness of source patch selection in comparison with SaliencyMix [15], a state-of-the-art method that crops the important source patch using saliency information. To this end, we counted the number of times a pixel was considered to be salient and accumulated the result for all the images in the CIFAR-100 [21] and ImageNet [22] datasets. The resulting heatmaps are presented in Figure 1. In the case of SaliencyMix on CIFAR-100, the result is a Gaussian distribution which seems somewhat tilted. In the case of SaliencyMix on ImageNet, the patches are highly skewed to the left side, which cannot satisfy the receptive field theory [20].