# Feature Mining: A Novel Training Strategy for Convolutional Neural Network
###### Abstract  
In this paper, we propose a novel training strategy named Feature Mining for convolutional neural networks (CNNs) that aims to strengthen the network's learning of the local features. Through experiments, we found that different parts of the feature contain different semantics, while the network will inevitably lose a large amount of local information during feedforward propagation. In order to enhance the learning of the local features, Feature Mining divides the complete feature into two complementary parts and reuses this divided feature to make the network capture different local information; we call the two steps Feature Segmentation and Feature Reusing. Feature Mining is a parameter-free method with a plug-and-play nature and can be applied to any CNN model. Extensive experiments demonstrated the wide applicability, versatility, and compatibility of our method. classification; convolutional neural network; training strategy; regularization +
Footnote †: journal:  
0000-0001-5000-0000-0000-0000-0000-0000-0000-0000-0000-0000-0000-0000-0000-0000-0000-0000-00000-00000-00000-00000-00000-0000-00000-0  
Our motivation stems from a phenomenon in the training process of CNNs: information loss inevitably occurs after going through the pooling layer, which may lead to the CNN's inadequate learning of the local features. We performed several visualization experiments [20] to explore the difference between the global features and the local features. The result empirically suggested that different parts of the feature contain different semantics, and the local features contain different semantics compared with the global features. However, the network will inevitably lose the local information during feedforward propagation. Making the network contain more local feature knowledge is a benefit to the training, but how to make the local features of images sufficiently mined by the network is a great challenge. Based on the above observations, we propose a parameter-free training method: Feature Mining, which aims to make the network efficiently learn the local features. This strategy requires two steps: Feature Segmentation and Feature Reusing. During Feature Segmentation, we divide the feature into two complementary parts by two binary masks. Feature Reusing denotes that these two parts of the feature will respectively pass through a global average pooling layer and a fully connected layer to generate two independent outputs, and the three cross-entropies will be added to form the final loss function. In this way, different local features can jointly participate in the network training with the global features, so that the network is able to learn more abundant feature expressions. Feature Mining can also be extended to any layer of the CNN. Specifically, adding Feature Mining in the last two layers of the network not only further improves the performance of the network, but also strengthens the shallow layers' learning ability. Due to its inherent simplicity, Feature Mining has a plug-and-play nature. Besides, we do not need to set any hyperparameters for Feature Mining, which can reduce the operating cost and maintain the stability of our method for performance improvement. We demonstrate the effectiveness of our simple, yet effective training method with different network structures, such as ResNet [4], DenseNet [21], Wide ResNet [22], and PyramidNet [23], for image classification tasks on various datasets including CIFAR100 [24], TinyImageNet, and ImageNet [2]. A great accuracy boost was obtained by Feature Mining, higher than the Dropout [15; 16; 17] and self-distillation methods [18; 19]. In addition, our training method has strong compatibility and can be superimposed on the mainstream data augmentation [12] and self-distillation methods. Besides, we verified the wide applicability of Feature Mining in fine-grained classification [25], object detection [26], and practical scenarios [27; 28].