# Multi-Scale Receptive Fields Convolutional Network for Action Recognition
## 3 Method
### Multi-Scale Receptive Fields  
To address the challenge of handling moving objects of different sizes, we draw inspiration from the behavior of human observers. When observing a static object, humans tend to move closer or farther away from it in order to focus on specific parts or obtain a broader view. This behavior is linked to changes in the field of vision, which can be thought of as analogous to the concept of receptive fields in convolutional neural networks. Specifically, receptive fields refer to the size of the convolution kernels used in the network, and can be adjusted to handle objects of different sizes. By using large receptive fields, ConvNets can effectively capture information from large moving objects, while smaller receptive fields can be used to capture details from smaller objects. This approach allows for more effective handling of intra-class variability and improved performance on action recognition tasks. A large convolution kernel often has a large receptive field. However, a large kernel is not recommended for use because it is hard to optimize during training process. Besides, a large kernel also leads to the increase of parameters. In order to address this, [PERSON] et al. [35] proposed the use of very small \(3\times 3\) receptive fields throughout the  
Figure 1: Some video samples from UCF101. Figure 2: Some video samples from HMDB51 and Kinetics-400. whole network, and showed that stacking multiple \(3\times 3\) convolution layers can achieve the same effect as using larger kernels. They found that a stack of two \(3\times 3\) convolution layers has an effective receptive field of \(5\times 5\), and three such layers have a \(7\times 7\) effective receptive field, as shown in Figure 3. Additionally, a stack of three \(3\times 3\) convolution layers incorporates three non-linear rectification layers instead of a single one, making the decision function more discriminative. This approach also reduces the number of parameters compared with using larger kernels, and avoids the loss of texture details in the original feature map that can occur with the use of too many convolution layers. Dilated convolution offers a solution to enlarging the receptive field without increasing the size of convolution kernels. This technique introduces a parameter known as the \"dilation rate\", which specifies the spacing distance between each convolution point. As the dilation rate increases, the receptive field grows larger, as demonstrated in Figure 4.  
By adopting dilated convolution, this paper proposes a multi-scale receptive fields block. The block is modified on the basis of pseudo-3D residual block, as shown in Figure 5. The pseudo-3D residual block contains one \(3\times 1\times 1\) convolution layer, one \(1\times 3\times 3\) convolution layer, and one \(1\times 1\times 1\) convolution layer. The function of \(3\times 1\times 1\) convolution is to extract temporal features and reduce the number of channels of feature maps, which lowers the computational complexity. The \(1\times 3\times 3\) convolution is used for  
Figure 4: Systematic dilation supports exponential expansion of the receptive field without loss of resolution or coverage [36]. (**a**) A 1-dilated convolution; each output element has a receptive field of \(3\times 3\). (**b**) Dilated convolution with dilation rate being 2; each output element has a receptive field of \(5\times 5\). (**c**) Dilated convolution with dilation rate being 4; each output element has a receptive field of \(9\times 9\). The number of parameters of each above dialted convolution is identical. Figure 3: A stack of three \(3\times 3\) convolution layers has an effective receptive field of \(7\times 7\). extracting spatial features, and the last \(1\times 1\times 1\) convolution is applied to increase the number of channels to enrich the feature information. The proposed multi-scale receptive fields block is designed to tackle the challenge of varying receptive field sizes.