# A High-Density Crowd Counting Method Based on Convolutional Feature Fusion
## 2 Feature Fusion CNN for Crowd Counting
### Network Architecture  
The VGG [18] network has achieved outstanding results in the image classification competition, which has strong expansion and generalization performance. The network model used in this paper was based on the VGG-16 network, and the backbone of the network was constructed by repeatedly stacking the convolutional layer with 3 \(\times\) 3 convolutional kernel and the max pooling layer with 2 \(\times\) 2 filter. Since too much downsampling will result in smaller feature maps and lower resolutions, to obtain a higher-quality crowd density map, only the first four sets of convolutions in the VGG-16 network will be adopted, which includes three downsampling layers. The first four sets of convolutions in the VGG-16 network are shown in Figure 2:The experiments with CCF (Convolutional Channel Features) [26] demonstrated that, to extract richer features, the suggested typical accumulated downsampling coefficients are four and eight. Inspired by this conclusion, the corresponding feature maps of downsampling coefficients of four and eight (the feature maps of Conv3_3 and Conv4_3) were fused by deconvolution technique to get richer features and finally get higher-quality crowd density map.  
The constructed network architecture in this paper is shown in Figure 3. The convolutional layers from Conv1_1 to Conv4_3 use a small convolutional kernel of 3 \(\times\) 3. The convolutional step is fixed to 1 pixel, and a pixel padding with a value of 0 is performed around the input space of the convolutional layer, which may keep spatial resolution after convolution operation. A rectified linear unit (ReLU) [27] is applied to each convolutional layer as an activation function. All pooling layers use the max pooling operation, while the filter size is 2 \(\times\) 2 with a fixed step size of 1. The number of feature maps of the convolutional layer will be doubled after each downsampling. After twice max pooling operations with filter size 2 \(\times\) 2, the size of the feature map Conv3_3 is W/4 \(\times\) H/4, and Conv4_3 has undergone three times downsampling operations to obtain a feature map of size W/8 \(\times\) H/8. In order to achieve the fusion of Conv3_3 and Conv4_3 feature maps, a deconvolution layer is added after the Conv4_3 to sample the feature maps of Conv4_3 to the same size as the feature maps of Conv3_3. Conv5_1 and Conv5_2 of the convolution layer with 3 \(\times\) 3 filters are adopted in the upper layer of the network to reduce the number of feature maps and increase the nonlinearity of the model. Finally, a convolutional layer with 1 \(\times\) 1 filters is used to map it to the density map, which can be integrated to obtain the overall count.  
Since a deconvolution layer after the Conv4_3 layer is added, the size of the feature map of the Conv4_3 layer is sampled to the size of the feature map of Conv3_3 layer. Therefore, though there are three downsampling operations in the network, the final accumulated downsampling coefficient is four. It only needs to downsample the ground truth density map to 1/4 of the input image, while the resulted estimated density map size is also 1/4 of the original image. In addition, though FF-CNN is a fully convolutional network [28; 29], which may accept images with any sizes, since there is a concatenate layer in the network, the output sizes of Conv3_3 and Conv4_3 layers should be identical after convolutional and pooling layers, so the height and width of the input image should be a multiple of eight.  
Figure 3: FF-CNN network architecture. FF-CNN: Feature Fusion of Convolutional Neural Network.  
Figure 2: The first four sets of convolutions in the VGG-16 network.