  Convolutional neural network (CNN) is a class of artificial neural networks
widely used in computer vision tasks. Most CNNs achieve excellent performance
by stacking certain types of basic units. In addition to increasing the depth
and width of the network, designing more effective basic units has become an
important research topic. Inspired by the elastic collision model in physics,
we present a general structure which can be integrated into the existing CNNs
to improve their performance. We term it the "Inter-layer Collision" (IC)
structure. Compared to the traditional convolution structure, the IC structure
introduces nonlinearity and feature recalibration in the linear convolution
operation, which can capture more fine-grained features. In addition, a new
training method, namely weak logit distillation (WLD), is proposed to speed up
the training of IC networks by extracting knowledge from pre-trained basic
models. In the ImageNet experiment, we integrate the IC structure into
ResNet-50 and reduce the top-1 error from 22.38% to 21.75%, which also catches
up the top-1 error of ResNet-100 (21.75%) with nearly half of FLOPs.
