Reconciling FeatureReuse and Overfitting in DenseNet with Specialized Dropout Recently convolutional neural networks (CNNs) achieve great accuracy in visual recognition tasks. DenseNets become one of the most popular CNN models due to its effectiveness in the featurereuse. However, like other CNN models, DenseNets also face the overfitting problem if not more severe. Existing dropout methods can be applied but not effective. In particular, the property of the featurereuse in DenseNets will be impeded, and the dropout effect will be weakened by the spatial correlation inside feature maps. To address these problems, we craft the design of a specialized dropout method from three aspects, the dropout location, the dropout granularity, and the dropout probability. The insights attained here could potentially be applied as a general approach for boosting the accuracy of other CNN models with similar shortcut connections. Experimental results show that DenseNets with our specialized dropout method yield better accuracies compared to vanilla DenseNets and stateoftheart CNN models, and such accuracy boost increases with the model depth.

The Evolution of Adjacency Matrices for Sparsity of Connection in DenseNets Image classification is a task in which convolutional neural networks hold stateoftheart results. DenseNets, in particular, have been shown to provide stateoftheart results with reduced model complexity when compared with other convolutional neural network structures. However, DenseNets themselves have opportunities for model simplification, which can not only make DenseNets smaller, but also improve the accuracy of resulting networks. This work develops a novel genetic algorithm, Matrix Evolution for sparse Dense Blocks (MESDB), to find such DenseNets, by removing unneeded connections within DenseNets. Our Sparse DenseNets improved the classification accuracy and reduced the number of trainable parameters by up to 33% on CIFAR10, while providing promising results on CIFAR100.