  The ResNet and its variants have achieved remarkable successes in various
computer vision tasks. Despite its success in making gradient flow through
building blocks, the simple shortcut connection mechanism limits the ability of
re-exploring new potentially complementary features due to the additive
function. To address this issue, in this paper, we propose to introduce a
regulator module as a memory mechanism to extract complementary features, which
are further fed to the ResNet. In particular, the regulator module is composed
of convolutional RNNs (e.g., Convolutional LSTMs or Convolutional GRUs), which
are shown to be good at extracting Spatio-temporal information. We named the
new regulated networks as RegNet. The regulator module can be easily
implemented and appended to any ResNet architecture. We also apply the
regulator module for improving the Squeeze-and-Excitation ResNet to show the
generalization ability of our method. Experimental results on three image
classification datasets have demonstrated the promising performance of the
proposed architecture compared with the standard ResNet, SE-ResNet, and other
state-of-the-art architectures.
