## 3 Two-Stream UDN for Pedestrian Detection
### Two-Stream UDN Based on Improved UDN  
The key to pedestrian detection is to find the image area containing a human body. However, in an actual pedestrian detection scenario, some columnar objects such as trees and telephone poles are often misjudged as pedestrians, as shown in Figure 5.  
Due to the relatively minor change of posture of the pedestrian's head, the region including the head has better invariance and discrimination from the upper part of columnar objects such as trees and telephone poles. Therefore, we construct another branch of improved UDN to accept the upper 1/3 pedestrian image as input. This paper proposes a two-stream UDN network, which is shown in Figure 6. The two-stream UDN consists of two parallel branches: the Global network and the Local network. The Global network is an improved UDN described in Section 3.1, the details of each layer of the Local network are as follows:  
(1)Convolution layer 1: The input of the first convolution layer has three components \(X_{i}^{local}\in R^{28\times 28},\ i=1,2,3\), which are the upper 1/3 part region of the original input image corresponding to the lower input channels in Figure 6. There are 64 convolution  
Figure 5: Examples of misjudged images with improved UDN. Figure 6: The two-stream UDN. kernels \(\left\{W_{j}^{local}\middle|W_{j}^{local}\in R^{9\times 9\times 3},\;\mathrm{j}=1, \ldots,64\right\}\) that are utilized for extracting features. With each convolution kernel \(W_{j}^{local}\), the feature map \(A_{j}^{local}\in R^{20\times 20}\) is computed as follows: \[A_{j}^{local}=Softplus\left(b_{j}^{local}+\sum_{i=1}^{3}\left[W_{j}^{local} \right]_{;,;i}*X_{i}^{local}\right)\] (4) where \(\left[W_{j}^{local}\right]_{;,;,i}\) represents the \(i\)th slice of three-order tensor \(W_{j}^{local}\), \({}^{*}\) represents the convolution operator, and \(b_{j}^{local}\) represents the bias parameters for the local network, which is obtained by random initialization. Before training, \(W_{j}^{local}\) is initialized with a Gabor filter. 2. Pooling layer: Calculate the average value of pixels in each \(4\times 4\) neighborhood on each feature map \(A_{k}^{local}\) and obtain extracted feature maps \(\left\{B_{k}^{local}\middle|B_{k}^{local}\in R^{5\times 5},\;\mathrm{k}=1, \ldots,64\right\}\), which is computed as follows: \[B_{k}^{local}=AveragePool\left(A_{k}^{local},4,4\right)\] (5) where \(k\) represents the number of the extracted feature maps. 3. Convolution layer 2: The second convolution layer has 20 filters \(F_{n}^{local}\in R^{3\times 5\times 64},\;n=1,\ldots,20\). Since the input of the local network is the upper 1/3 region of the image, there is no need to convolve in parts as in the network based on the global input. The local feature map of the second convolution layer \(C_{n}^{local}\) is computed as follows: \[C_{n}^{local}=Softplus\left(u_{n}^{local}+\sum_{m=1}^{64}\left[F_{n}^{local} \right]_{;,;m}*B_{m}^{local}\right)\] (6) where \(n\) represents the number of the filtered data maps, \(\left[F_{n}^{local}\right]_{;,;m}\) represents the \(m\)th slice of three-order tensor \(F_{n}^{local}\), \({}^{*}\) represents the convolution operator, and \(u_{n}^{local}\) represents the bias parameters obtained by random initialization. Before training, \(F_{n}^{local}\) uses a Gabor filter for initialization. 4. Deformation layer: The deformation layer is same with the global network, which reduces the number of parameters and extracts head-shoulder feature scores \(s_{p}^{local},\left\{s_{p}^{local}\middle|p=1,\ldots,20\right\}\). 5. Classification layer: The visibility reasoning network [15] is used to estimate the label \(\mathrm{y}^{local}\) by learning the head-shoulder feature scores \(s_{p}^{local}\), which represents whether the upper 1/3 part contains a pedestrian head-shoulder or not.