Recurrent Convolutions: A Model Compression Point of View
Recurrent convolution shares the same convolutional kernels and unrolls them multiple times, which is originally proposed to model time-space signals.We suggest that RC can be viewed as a model compression strategy for deep convolutional neural networks.RC reduces the redundancy across layers and is complementary to most existing model compression approaches.However, the performance of an RC network cant match the performance of its corresponding standard one, i.e. with the same depth but independent convolutional kernels. ", This reduces the value of RC for model compression.In this paper, we propose a simple variant which improves RC networks: The batch normalization layers of an RC module are learned independently for different unrolling steps.We provide insights on why this works.Experiments on CIFAR show that unrolling a convolutional layer several steps can improve the performance, thus indirectly plays a role in model compression.
Recurrent convolution for model compression and a trick for training it, that is learning independent BN layres over steps.The author modifies the recurrent convolution neural network (RCNN) with independent batch normalization, with the experimental results on RCNN compatible with the ResNet neural network architecture when it contains the same number of layers.