RotationOut as a Regularization Method for Neural Network
In this paper, we propose a novel regularization method, RotationOut, for neural networks.Different from Dropout that handles each neuron/channel independently, RotationOut regards its input layer as an entire vector and introduces regularization by randomly rotating the vector.RotationOut can also be used in convolutional layers and recurrent layers with a small modification.We further use a noise analysis method to interpret the difference between RotationOut and Dropout in co-adaptation reduction.Using this method, we also show how to use RotationOut/Dropout together with Batch Normalization.Extensive experiments in vision and language tasks are conducted to show the effectiveness of the proposed method.Codes will be available.
We propose a regularization method for neural network and a noise analysis methodThis paper proposes a new regularization method to mitigate the overfitting issue of deep neural networks by rotating features with a random rotation matrix to reduce co-adaptation.This paper proposes a novel regularization method for training neural networks, which adds noise neurons in an inter-dependent fashion.