<figure><img src="image_1.png" /></figure>

HAL
open science

# Accelerate High Resolution Image Pedestrian Detection With Non-Pedestrian Area Estimation

Haodi Zhang, Nut Ovidiu Pop, Alexandrina Rogozan, Abdelaziz Bensrhair

► To cite this version:

Haodi Zhang, Nut Ovidiu Pop, Alexandrina Rogozan, Abdelaziz Bensrhair. Accelerate High Resolution Image Pedestrian Detection With Non-Pedestrian Area Estimation. IEEE Access, 2021, 9, pp.8625-8636. 10.1109/ACCESS.2021.3049401. [hal-05293790](https://hal.science/hal-05293790v1)

HAL Id: hal-05293790

https://hal.science/hal-05293790v1

Submitted on 2 Oct 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Received December 21, 2020, accepted December 25, 2020, date of publication January 5, 2021, date of current version January 14, 2021.

Digital Object Identifier 10.1109/ACCESS.2021.3049401

# Accelerate High Resolution Image Pedestrian Detection With Non-Pedestrian Area Estimation

HAODI ZHANG<sup>1</sup>, DĂNUJ OVIDIU POP<sup>1,2</sup>, ALEXANDRINA ROGOZAN<sup>1</sup>, AND ABDELAZIZ BENSRHAIR<sup>1</sup>

Corresponding author: Haodi Zhang (haodi.zhang@insa-rouen.fr)

This work was supported by the China Scholarship Council (CSC) under grant 201806960147.

**ABSTRACT** Detecting pedestrians at high speed in high resolution (HR) images is important for preventing collisions in autonomous vehicle. The HR images provide more detail information while creating a heavy computational load, resulting in slower detection speed. We find that the pedestrians are usually not present in the upper part of the HR image, due to the camera vertical field of view and the distance between pedestrian and camera. We named this area a non-pedestrian area (NPA). Therefore, we are able to reduce the calculation time while maintaining the detection accuracy by removing NPA from the HR image. According to this concept, we propose a novel pedestrian detection acceleration algorithm called Non-Pedestrian Area Estimation (NPAE). The NPAE algorithm estimates and removes non-pedestrian areas of the image, followed by pedestrian detection of the NPAE output image. Two datasets with three different resolution images are used to test our proposed NPAE algorithm on both GPU and CPU platforms. The RetinaNet is chosen as the reference pedestrian detector. Our algorithm increases the detection speed with 42% on the GPU platform and with 52% speed increase on the CPU platform, for the test images with a resolution of 1920 × 1080 pixel. In terms of detection accuracy, NPAE improves by up to 37.94% compared to classical data optimization acceleration methods. The detection accuracy results are well preserved in both platforms.

**INDEX TERMS** Pedestrian detection, non-pedestrian area estimation, detection acceleration, autonomous vehicle.

## I. INTRODUCTION

The pedestrians are one of the vulnerable roles of public transportation since they tend to suffer more injuries when a collision occurred between vehicles and pedestrians [1]. According to the World Health Organization (WHO) 2020 Road Traffic Injury Report, more than 67,500 pedestrians, cyclists and motorcyclists die in road traffic accidents every year [2]. Injury risk factors are mainly caused by human error, such as speeding, driving under the influence of alcohol. Based on fast and accurate pedestrian detection, autonomous vehicle (AV) can avoid these kinds of human errors and significantly reduce traffic accidents and casualties.

Pedestrian detection technology has made great progress in the past decade [3] due to deep neural networks. As more and more layers are adopted, deep neural networks usually have higher detection accuracy. The increased complexity requires

The associate editor coordinating the review of this manuscript and approving it for publication was Shiping Wen<sup>1, ∗</sup>.

more computing resources and usually consumes additional computing time.

In an emergency, even if the braking system responds immediately, AV still needs a long braking distance. For example, the speed limit on French city roads is 50 km/h [4] and the braking distance is 14.5 m according to [5]. To ensure pedestrian safety, pedestrian detectors are required to locate targets at the farthest possible distance. Not only that, but the detector also needs to be fast enough to gain reaction time for the AV.

High-resolution (HR) image is required to detect pedestrian targets at long distances. HR image has many advantages, such as improved detection results [6]. It is also a basic need for autonomous driving [7]. The rich detail information provided by HR images improves the accuracy of long-distance detection. However, it increases the amount of computational consumption and time. Therefore, Pedestrian detection needs to be accelerated and kept accurate.

<footnote n="1">¹STI Team, LITIS Lab, INSA Rouen, 76800 Saint-Étienne-du-Rouvray, France</footnote>

<footnote n="2">²RITS Team, INRIA Paris, 75012 Paris, France</footnote>
Most of the advanced acceleration methods are summarized in [8], such as pruning, low-rank approximation, quantization, knowledge distillation and compact network design. All mentioned methods are based on optimizing deep neural networks for acceleration. These methods are incredible in thought and design, and have significant effects in practical.

However, we noticed that the input image is an under-optimized part of the detection process. There are two processing methods for the image input to the detector. One method is to compress the input image to a fixed size as in the [9], [10]. Another method crops part of the image area, such as [11]. Both of these image processing methods are very convenient. Nevertheless, the compress-image method results in a loss of detection accuracy. The crop-image method can only be used with an existing dataset and acquisition environment. Changes in camera model, spatial position, and scaling require researchers to redesign the system.

We observe that the HR images obtained by autonomous vehicles contain some areas without pedestrians. We named this area a Non-Pedestrian Area (NPA). We find that there is a relationship between the region size of NPA in the image and the environmental parameters. The environmental parameters contain camera height, camera pitch angle, camera vertical field of view (FOV), the distance between the pedestrian and the camera, pedestrian height and vertical resolution of the image. Therefore, we propose the Non-Pedestrian Area Estimation (NPAE) algorithm to describe the relationship between the two factors. Afterwards, the NPAE algorithm can estimate the NPA based on the environmental parameters. Once the NPA is acquired, we remove the non-pedestrian area from the HR image and then send the processed image to the subsequent pedestrian detector.

We first design an experiment to verify the correctness of the NPAE algorithm. Then, the algorithm acceleration performance is tested on the JAAD dataset both on the GPU platform and the CPU platform. Finally, the performance of the algorithm in extreme cases is tested on the Caltech dataset which has a plentiful variety of scenarios. Among them, the JAAD dataset contains two types of high-resolution images, 1920 × 1080 pixels and 1280 × 720 pixels. The Caltech data set contains a low-resolution 640 × 480 pixels image.

The NPAE algorithm is proposed for acceleration pedestrian detection. It works by estimating and cropping the Non-Pedestrian Area of the image. Due to the reduction of image size, the pedestrian detector can process faster. Moreover, the accuracy of the pedestrian detector can be maintained because of the pedestrian area of the image is not modified. The NPAE algorithm has several highlights: 1) The NPAE algorithm can calculate and crop the Non-Pedestrian Area uniformly for the images acquired under the same capture conditions. 2) The NPAE algorithm does not require training and is capable of accelerating a variety of state-of-the-art pedestrian detectors. 3) The proposed algorithm is an image data optimization method that can also be used in a variety of applications such as image segmentation, object tracking.

## II. RELATED WORK

### A. PEDESTRIAN DETECTORS

Integrate Channel Features (ICF) is a classic pedestrian detection method. The ICF detector involves channel feature pyramids and boosted classifiers. Subsequent work such as Aggregated Channel Features (ACF), Locally Decorrelated Channel Features (LDCF), and SquaresChnFtrs (SCF) are based on ICF for further improvement [12].

In recent years, machine learning methods such as SVM, ELM, ANN, CNN have gradually replaced manual design rules [13]. In the field of pedestrian detection, deep learning has been adopted by most algorithms with excellent performances. These algorithms based on deep neural networks have higher detection accuracy, but they still need huge calculation during training and testing. The following are several classic neural network detectors:

R-CNN [14] uses the selective search to extract a large number of region proposals. These regions are converted into fixed-size images and sent to convolutional neural networks to detect and classify. Fast R-CNN [15] extracts features only once to generate region proposals. This task solves the problem of R-CNN by repeatedly extracting features and increases the network detection speed. Faster R-CNN [16] further proposes a candidate region network (RPN), which can generate region proposals more efficiently.

YOLO [17] uses the entire image as the input of the network and directly returns the position of the bounding box and the class it belongs to in the output layer, which greatly speeds up the detection. After that, YOLOv2 [10] combined the advantages of RCNN-like algorithms to further improve the detection accuracy and speed.

RetinaNet [18] is a one-stage object detector. Usually, the one-stage detector has a faster detection speed, but the detection accuracy is not as good as the two-stage detector. RetinaNet adopts the Feature Pyramid Network (FPN) from [19] as the backbone network. The FPN fuses different levels of features through lateral connections. Due to this architecture, the network can effectively construct a rich multi-scale feature pyramid from the input image. RetinaNet also introduces a novel loss function called the focal loss to solve the foreground-background imbalance problem. These designs ensure that RetinaNet is an efficient and accurate detector. Therefore, we chose RetinaNet as the reference of the pedestrian detector to test the performance of the proposed NPAE algorithm.

### B. PEDESTRIAN DETECTION ACCELERATION

#### 1) NEURAL NETWORK OPTIMIZATION

The current mainstream idea for neural network acceleration is to optimize the neural network structure to reduce network complexity and calculation. According to different perspectives of network optimization strategies, it can be divided into the following five categories:
a: PRUNING [20]

The pruning method assumes that many neural network
parameters are not necessary and deletes them. The pruning
method preserves the structure of the neural network and
reduces the computation. However, the pruning process usu-
ally is not lossless to accuracy. As reported in [21], pruning
process causes degradation of neural network performance.

b: LOW-RANK APPROXIMATION [22]

Low-rank approximation reduces the dimension of the con-
volutional kernel which will facilitate computation.

c: QUANTIZATION [23]

This method can be divided into two categories: (1) Scalar and vector quantization: Quantize each weight of the network into a certain element in a finite set. (2) Fixed-point quantization: Convert floating-point numbers in neural networks to fixed-point numbers.

d: TEACHER-STUDENT NETWORK [24]

Generally, the teacher network is a large neural network or a
collection of neural networks, while the student network is a
compact and efficient neural network. By using the student
network to fit the teacher network, the knowledge learned by
the teacher network is transferred to the student network.

e: COMPACT NETWORK DESIGN [25]

By using methods such as 1 × 1 convolution and branching
strategies, a more compact network structure can be designed
to improve network performance. For instance, the use of
end-to-end design in [26], [27] reduces the computation of
intermediate transformations and improves the performance
of the algorithm. [28] uses a sparse convolutional design to
improve both the running speed and the performance of the
algorithm.

The methods described above are excellent optimization
methods. Nevertheless, these methods still have some dis-
advantages. A large amount of repeated training is required
when applying pruning methods to neural networks. The
low-rank method requires adequate design. Quantization is
not suitable for all tasks and needs to convert the network,
which may cause a loss of accuracy. The teacher-student
method requires a lot of training and requires a good teacher
network. The compact network design method requires a lot
of optimization.

2) IMAGE DATA OPTIMIZATION

There are two methods for accelerating detection by optimiz-
ing the input image: one is to compress the image, and the
other is to crop the image.

a: IMAGE COMPRESSION

In order to detect faster and reduce resource consumption,
most methods usually compress the image size. For exam-
ple, there are two similar network architectures in SSD,

SSD512 and SSD300. The SSD512 network uses images with
a resolution of 512 × 512 pixels and it has high detection
accuracy. However, the detection speed is slow due to the
high resolution of the images used. In order to accelerate the
detection speed, [9] proposes the SSD300 structure. SSD300
uses an image with a resolution of 300 × 300 pixels. From the
results, the SSD512 has a detection accuracy of 76.8% mean
average precision (mAP) and a speed of 22 frames per second
(fps). To increase detection speed, the detection accuracy of
SSD300 has been reduced by 2.5%.

The image compression methods increase the speed of
detection, but it also reduces the accuracy of detection. This
is because compressing the image causes some of the target
information to be lost, which interferes with detection.

b: IMAGE CROPPING

Image cropping is also a very widely used method. The usual
method is to crop the region of interest (RoI) in the image.
For example, the image area affected by pincushion distor-
tion effect is cropped in the KITTI dataset [11]. The image
resolution is cropped from 1392 × 512 pixels to 1242 × 375
pixels, a reduction of 34.65% pixels. Reducing the number of
pixels will reduce the amount of computation and therefore
accelerate the detection process.

The advantage of the image cropping method is that it
is quick and easy to understand. The disadvantage is that
when there is less cropping, the acceleration is less effective.
When there is too much cropping, the target will be cropped.
Therefore, it is necessary to get all the data first, count the
target distribution and then manually adjust the cropping area.
But it is difficult to meet this condition in practice

With different design concept, neural network optimization
and image data optimization are both designed to acceler-
ate pedestrian detection. The neural network optimization
methods are concerned with improving accuracy and speed
by finding a better detector structure. The image data opti-
mization methods allow neural network to process less data
by reducing the image size. The image data optimization
methods are used in combination with optimized neural net-
works to take advantage of both acceleration methods for
pedestrian detection. In [17], a neural network named YOLO
is proposed. The YOLO neural network first compresses the
size of the input images to 224 × 224 pixels. Then, high-
speed detection is achieved through a novel one-stage detec-
tor design. [9] introduces two architectures called SSD300
and SSD512, which compress the input images to 300 × 300
pixels and 512 × 512 pixels respectively. This design attempts
to make a trade-off between accuracy and speed of detection.

However, the existing image data optimization methods
lack accurate image processing. Thus, we propose the Non-
Pedestrian Area Estimation (NPAE) algorithm. The NPAE
algorithm is a method for image data optimization to accel-
erate pedestrian detection. Different from the classical image
data optimization methods, the NPAE algorithm estimates the
Non-Pedestrian Area (NPA) in the image based on the param-
eters of the image capture environment. The advantage of this
<figure><img src="image_3.png" /><figcaption>FIGURE 1. Definition of pedestrian bounding box (bbox). Bbox is jointly determined by point P1 and point P2.</figcaption></figure>

processing is that the image size is reduced, while the areas containing pedestrian are preserved. Therefore, the NPAE algorithm could accelerate pedestrian detection and maintain detection accuracy. Compared to neural network optimization methods, the NPAE algorithm has several advantages: 1) The proposed algorithm does not require any training process. 2) The optimization process does not affect the detection accuracy. 3) The NPAE algorithm could be used in the neural network optimization methods together to accelerate pedestrian detection.

### III. ALGORITHM DESIGN

#### A. PEDESTRIAN LOCATION ANALYSIS

To present the basic idea of this paper, we selected two datasets for observation and experimentation.

##### 1) JAAD DATASET

The dataset includes high-resolution (HR) images in two formats, 1920 × 1080 pixels and 1280 × 720 pixels. It is a representative dataset of HR images for pedestrian detection. The two resolutions allow us to observe the distribution of pedestrians in different situations.

##### 2) CALTECH DATASET

The dataset has the images with low resolution (LR) of 640 × 480 pixels. The Caltech dataset does not contain HR images. However, it contains a rich set of scenes and is widely adopted by existing methods. Thus this LR image dataset is used for comparison experiments. Both the JAAD dataset and the Caltech dataset have a frame rate of 30 fps.

For the selected dataset, we will count the distribution of pedestrians. Therefore, it is necessary to select one of the pedestrian's attribute parameters as the statistical object. We define that the pedestrian bounding box is located by two points, P1 and P2, as shown in Figure 1. Where $P1 = (P1_x, P1_y)$ indicates the top-left vertex, $P2 = (P2_x, P2_y)$ indicates the right bottom vertex.

Since pedestrians will appear at random locations, $P1_x \in [0, X_{max}]$, $P2_x \in [0, X_{max}]$. The range of $P1_y$ and $P2_y$ cannot be determined. We categorized the images into three groups by resolution: 1920 × 1080, 1280 × 720 and 640 × 480. All P1 and P2 points are plotted in the image space of the corresponding resolution, respectively, as shown in Figure 2.

The purpose of analyzing the three sets of points P1 and P2 is to determine whether the Non-Pedestrian Area (NPA) is widely present in the image.

$P1$ represents the top boundary of the pedestrian bounding box, so we need to find $NPA_{top}$ in the upper half of the image. $NPA_{top}$ can be found in the three sets of images with different resolutions in Figure 2(a), 2(c) and 2(e). In the HR image, the $NPA_{top}$ region is larger. In LR images, the region of $NPA_{top}$ is smaller. There are some exceptions to the $NPA_{top}$ region in LR images, which we will discuss in subsection III-C.

$P2$ represents the bottom boundary of the pedestrian bounding box, so we need to find $NPA_{bottom}$ in the lower half of the image. $NPA_{bottom}$ does not exist in the three sets of images with different resolutions in Figure 2(b), 2(d) and 2(f).

This situation is not a coincidence. For a better view, the camera is usually set up on top of the autonomous vehicle. Hence $h_c = h_v$, where $h_c$ is the camera height and $h_v$ is the vehicle height. In general, vehicle height $h_v$ and pedestrian height $h_p$ have $h_v > h_p/2$. Then $h_c > h_p/2$, which means that the area where pedestrian appears in the image is more towards the bottom. This is the reason why NPA only exists at the top of the image. In this paper, we use NPA to refer to $NPA_{top}$. From the analysis of the dataset, we can see that NPA is present. The next step is to analyze the environmental parameters related to the NPA and find an algorithm to estimate NPA.

#### B. NON-PEDESTRIAN AREA ESTIMATION FOR SINGLE IMAGE

Our goal is to find suitable non-pedestrian areas for all images in the dataset. The situation in the dataset is more complicated. It contains a large number of pedestrians, with different camera postures and different collection equipment. Therefore, we first try to find the optimal NPA for a single image. Then we will discuss how to find a common applicable NPA on the dataset.

To accurately estimate the non-pedestrian area in a single image, it is necessary to analyze the highest position of the pedestrian bounding box $P1$ point in the image. As shown in Figure 3(a), there are three pedestrians, $Ped_1$, $Ped_2$ and $Ped_3$. The distance between camera and pedestrian is $d_1$, $d_2$ and $d_3$, respectively. $Ped_1$ is on the vertical centre line of the image, $d_1$ is perpendicular to the plane of the image. Therefore $d_1 > d_2$, $d_3$. According to the imaging rule of the camera, the closer the pedestrian is, the larger the image is. Suppose that the two pedestrians of $Ped_1$ and $Ped_2$ are the same height. Since $Ped_1$ is closer to the camera, the $P1$ point of its bounding box is higher than $Ped_2$'s.
<figure><img src="image_4.png" /><figcaption>**FIGURE 2.** P1 is the upper left corner of the pedestrian bounding box. P2 is the lower right corner of the pedestrian bounding box. Non-pedestrian areas are present in the statistics at point P1 in all three datasets. NPA is not present in the statistics at point P2. Thus for the dataset, the non-pedestrian areas are predominantly present at the top of the image. There are special scenarios in the Caltech dataset that need to be analyzed, so non-pedestrian areas are marked as NPA*.</figcaption></figure>
<figure><img src="image_5.png" /><figcaption>FIGURE 3. Analyze and build NPAE algorithms for single images. (a) Analysis of non-pedestrian area at the top of the image containing multiple pedestrians. (b) Analysis of the environmental parameters involved in NPAE algorithm.</figcaption></figure>

Figure 3(b) establishes a model of pedestrian and camera. Assume that the pedestrian is located in the center of the image. At this position, the pedestrian's imaging area is the largest and has the highest P1 point. Several environmental parameters are used: $h_c$ is the camera height, $\gamma$ is the camera pitch angle, $\alpha$ is the camera vertical field of view (FOV), $d_{pc}$ is the distance between the pedestrian and the camera, $h_p$ is the pedestrian height, $px_v$ is the vertical resolution of the image, NPA is the Y-axis coordinate of the pedestrian bounding box P1 point according to the environmental parameters.

We first consider the scenario where the pitch angle $\gamma$ of a camera is 0. The goal of the NPAE algorithm is to calculate the vertical coordinate of a point P on the image. The height of the camera is $h_c$, the height of point P is $h_p$, and the distance of point P from the camera is $d$. According to the camera vertical FOV $\alpha$ and $d$, the observable length $l_d = 2d \tan(\alpha/2)$ can be calculated. Assume that the vertical coordinate of point P in the image is $Y$, and the vertical resolution of the image is $px_v$. According to the proportional relationship we can get: $Y : px_v = (l_d/2 - (h_p - h_c)) : (l_d/2)$. Thus $Y = px_v/2(1 - (h_p - h_c)/(d \tan(\alpha/2)))$. Here the $Y$ represents the Y-axis coordinate of the non-pedestrian area. According to the above analysis, the NPAE algorithm equation with $\gamma$ is shown in equation (1):

$$NPA = \frac{px_v}{2} \left\{ 1 - \frac{(h_p - h_c) \cos \gamma - d_{pc} \sin \gamma}{[d_{pc} \cos \gamma + (h_p - h_c) \sin \gamma] \tan \frac{\alpha}{2}} \right\} \quad (1)$$

In order to verify NPAE algorithm in equation (1), six environmental parameters are needed. After recording and inputting the required parameters of equation (1), the NPAE algorithm can be verified by comparing the calculated value of NPA with the actual value. Since the dataset does not contain the required environmental parameter, a validation experimental platform is built.

The parameters in the validation experimental platform are: The height of the pedestrian $h_p$ is 1.90 m. The camera height $h_c$ is 1.17 m. The vertical resolution of the image $px_v$

is 1080 pixels. The camera vertical FOV $\alpha$ is 52 degrees. The camera pitch angle $\gamma$ is set at 0.5 degrees, 3.5 degrees, and 10 degrees in sequence. The distance between the pedestrian and the camera $d_{pc}$ is set at 2.0 m, 2.5 m, 3.0 m, 3.5 m, and 4.0 m in sequence. The vertical coordinate of the P1 point of the pedestrian bounding box P1_y is manually measured. The experimental results are shown in Table 1.

It can be seen from Table 1 that the NPA value estimated by the NPAE algorithm is consistent with the P1_y. The Dissimilarity index are adopted to measure the similarity between NPA and P1_y. There are a lot of similarity calculation methods, such as Euclidean metric, Manhattan distance and Chebyshev distance. The outputs of these methods are the distance between the two sets of data, which does not visually show the degree of similarity. While the output of cosine similarity shows the similarity in the form of percentage. This makes it easier to determine the degree of similarity between NPA and P1_y. Thus, we choose to use cosine similarity to measure the degree of similarity between the two vectors. The formula of cosine similarity is:

$$sim(\mathbf{x}, \mathbf{y}) = \frac{\mathbf{x} \cdot \mathbf{y}}{|\mathbf{x}| |\mathbf{y}|} \quad (2)$$

where $\mathbf{x}$ and $\mathbf{y}$ are:

$$\begin{aligned} \mathbf{x} &= (P1_{y1}, P1_{y2}, \dots, P1_{y5}) \\ \mathbf{y} &= (NPA_1, NPA_2, \dots, NPA_5) \end{aligned} \quad (3)$$

and $|| \cdot ||$ is the Euclidean norm of vector:

$$\begin{aligned} ||\mathbf{x}|| &= \sqrt{P1_{y1}^2 + P1_{y2}^2 + \dots + P1_{y5}^2} \\ ||\mathbf{y}|| &= \sqrt{NPA_1^2 + NPA_2^2 + \dots + NPA_5^2} \end{aligned} \quad (4)$$

The measure computes the cosine of the angle between vectors $\mathbf{x}$ and $\mathbf{y}$. The closer the cosine value to 1, the smaller the angle and the greater the match between vectors [29]. We can then get the dissimilarity by:

$$dissim = 1 - sim(\mathbf{x}, \mathbf{y}) \quad (5)$$
<table><caption>**TABLE 1.** Verify the correctness of the NPAE algorithm. The dissimilarity is calculated by $P1_y$ and NPA. A smaller value means that the two vectors are more similar. The six environmental parameters used: camera height $h_c$, camera pitch angle $\gamma$, camera vertical view angle $\alpha$, pedestrian-camera distance $d_{pc}$, pedestrian height $h_p$, vertical resolution $px_v$. $P1_y$ is the $P1$ point Y-axis value of ground truth. NPA is the non-pedestrian area calculated by the NPAE algorithm.</caption><thead><tr><th>px<sub>v</sub> (pixels)</th><th>h<sub>p</sub> (m)</th><th>h<sub>c</sub> (m)</th><th>α (degree)</th><th>γ (degree)</th><th>d<sub>pc</sub> (m)</th><th>P1<sub>y</sub> (pixels)</th><th>NPA (pixels)</th><th>Dissimilarity</th></tr></thead><tbody><tr><td>1080</td><td>1.90</td><td>1.17</td><td>52</td><td>0.5</td><td>2.0</td><td><b>145</b></td><td><b>147</b></td><td rowspan="5">4.53 × 10<sup>-5</sup></td></tr><tr><td>1080</td><td>1.90</td><td>1.17</td><td>52</td><td>0.5</td><td>2.5</td><td><b>223</b></td><td><b>227</b></td></tr><tr><td>1080</td><td>1.90</td><td>1.17</td><td>52</td><td>0.5</td><td>3.0</td><td><b>277</b></td><td><b>281</b></td></tr><tr><td>1080</td><td>1.90</td><td>1.17</td><td>52</td><td>0.5</td><td>3.5</td><td><b>315</b></td><td><b>319</b></td></tr><tr><td>1080</td><td>1.90</td><td>1.17</td><td>52</td><td>0.5</td><td>4.0</td><td><b>350</b></td><td><b>348</b></td></tr><tr><td>1080</td><td>1.90</td><td>1.17</td><td>52</td><td>3.5</td><td>2.0</td><td><b>216</b></td><td><b>211</b></td><td rowspan="5">2.74 × 10<sup>-6</sup></td></tr><tr><td>1080</td><td>1.90</td><td>1.17</td><td>52</td><td>3.5</td><td>2.5</td><td><b>296</b></td><td><b>289</b></td></tr><tr><td>1080</td><td>1.90</td><td>1.17</td><td>52</td><td>3.5</td><td>3.0</td><td><b>349</b></td><td><b>341</b></td></tr><tr><td>1080</td><td>1.90</td><td>1.17</td><td>52</td><td>3.5</td><td>3.5</td><td><b>386</b></td><td><b>379</b></td></tr><tr><td>1080</td><td>1.90</td><td>1.17</td><td>52</td><td>3.5</td><td>4.0</td><td><b>415</b></td><td><b>407</b></td></tr><tr><td>1080</td><td>1.90</td><td>1.17</td><td>52</td><td>10.0</td><td>2.0</td><td><b>337</b></td><td><b>344</b></td><td rowspan="5">3.37 × 10<sup>-5</sup></td></tr><tr><td>1080</td><td>1.90</td><td>1.17</td><td>52</td><td>10.0</td><td>2.5</td><td><b>416</b></td><td><b>418</b></td></tr><tr><td>1080</td><td>1.90</td><td>1.17</td><td>52</td><td>10.0</td><td>3.0</td><td><b>468</b></td><td><b>469</b></td></tr><tr><td>1080</td><td>1.90</td><td>1.17</td><td>52</td><td>10.0</td><td>3.5</td><td><b>509</b></td><td><b>506</b></td></tr><tr><td>1080</td><td>1.90</td><td>1.17</td><td>52</td><td>10.0</td><td>4.0</td><td><b>536</b></td><td><b>533</b></td></tr></tbody></table>

Since the results of dissimilarity are very close to 0, that means the proposed Non-Pedestrian Area Estimation algorithm accurately calculates the non-pedestrian area.

### C. NON-PEDESTRIAN AREA ESTIMATION FOR MULTIPLE IMAGES

NPAE algorithm is designed to find the non-pedestrian area for the dataset. There is a difference between applying the NPAE algorithm to a dataset and applying the algorithm to a single image. When using the NPAE algorithm on a dataset, all six environment parameters need to be set to constants. In this way, the NPAE algorithm can uniformly remove non-pedestrian areas for the images in the dataset based on the set environmental parameters. However, the situation is further complicated by the large number of images contained in the dataset. The six environmental parameters used by NPAE need to be revisited.

#### 1) CAMERA VERTICAL FOV $\alpha$

The camera vertical FOV does not change for a dataset or an automated vehicle once the camera has been placed. Camera vertical FOV is constant.

#### 2) VERTICAL RESOLUTION $px_v$

Similar to the camera vertical FOV, the vertical resolution does not change after the camera is setup. Vertical resolution is constant.

#### 3) CAMERA HEIGHT $h_c$

As the vehicle runs, the camera height changes slightly under varying slope and vibration conditions. Since this change is instantaneous. Eventually it will return to its initial state. Therefore, camera height can be considered as a constant.

#### 4) CAMERA PITCH ANGLE $\gamma$

The camera pitch angle will slightly change in the car movement. Similar to the camera height, camera pitch angle can be also considered as a constant.

#### 5) PEDESTRIAN HEIGHT $h_p$

The pedestrian height in the dataset varies and is not available. According to subsection III-A, the highest pedestrian height determines the value of NPA when the pedestrians are at the same location. Therefore we refer to [30] and set $h_p$ using the largest value of the average country citizen height in the world, thus $h_p = 1.8$ m.

#### 6) PEDESTRIAN-CAMERA DISTANCE $d_{pc}$

According to subsection III-A, the smaller pedestrian-camera distance for pedestrians with the same height determines the value of NPA. Although pedestrians may appear at any location, the $d_{pc}$ will usually not be 0 due to the vehicle width. We refer to the average size of cars and set pedestrian-camera distance $d_{pc} = 1.2$ m.

The camera vertical FOV, vertical resolution, camera height, and camera pitch angle obtained from the dataset along with the predefined pedestrian height and pedestrian-camera distance are the environmental parameters required for the NPAE algorithm. The NPA value for the dataset can be obtained by entering these parameters into equation (1).

Taking into account the following special conditions may encounter: 1) The pedestrian height is higher than the assumed value. 2) The pedestrian distance from the camera is less than the assumed value. 3) The pedestrian is not in the same plane as the camera. These special pedestrian targets need to be carefully investigated. A discussion of special conditions follows.

The NPA is used to remove the horizontal pixels lines from 0 to the value of NPA. NPAE not only optimizes the image, but also changes the coordinates of the bounding boxes $P1$ and $P2$. The modified $P1_{new}$ and $P2_{new}$ can be calculated from equation (6).

$$ \begin{array}{l} P1_{new} \begin{cases} P1_{x\_new} = P1_x \\ P1_{y\_new} = P1_y - \text{NPA} \end{cases} \\ P2_{new} \begin{cases} P2_{x\_new} = P2_x \\ P2_{y\_new} = P2_y - \text{NPA} \end{cases} \end{array} \tag{6} $$
The X-axis coordinates of P1 and P2 have not changed, only the Y-axis coordinates have changed. There are three cases of P1<sub>y_new</sub> and P2<sub>y_new</sub>:

I: P1<sub>y_new</sub> > 0 and P2<sub>y_new</sub> > 0.

In this case, the pedestrians in the processed image are complete. The processing does not have any effect on the pedestrian detection results. This is the most common state.

II: P1<sub>y_new</sub> < 0 and P2<sub>y_new</sub> < 0.

In this case, some pedestrians in the original images will be removed. This type of pedestrian is usually located above certain objects. There is no way that the autonomous vehicle can hurt this type of pedestrian. We found this type of pedestrian in the Caltech dataset, as shown in Figure 4(a). Pedestrians on the overpass are not the targets we are interested in. Dealing with such pedestrian target is a waste of processing resources. Therefore, eliminating such pedestrian targets will not affect the pedestrian detection accuracy.

III: P1<sub>y_new</sub> < 0 and P2<sub>y_new</sub> > 0.

In this case, some part of the pedestrian will be removed. To cause this, pedestrians have to be close enough to the camera and the camera's angle of view should be small enough. We only found some samples in the Caltech dataset, as shown in Figure 4(b). The area above the pedestrian's head in the red box will be partially removed, but the remaining pedestrian image still occupies a large area in the image. Due to problems such as object occlusion and pedestrian overlap, the pedestrian detector should have the ability to detect part of the pedestrian. Missing some part of the pedestrian image will not cause the detection result to fail.

Based on the above analysis, we conclude that the NPAE algorithm does not have a negative impact on the pedestrian detection results.

# IV. EXPERIMENTS AND RESULTS

## A. EXPERIMENT PREPARATION

### 1) DATASETS

Two pedestrian datasets are adopted, JAAD dataset [31] and Caltech dataset [32].

#### a: JAAD

The JAAD dataset is developed by York University. It has various weather conditions such as sunny, cloud, rain and snow. It is collected from three cities with complex traffic scenarios and a large number of pedestrians. 2786 pedestrians are labelled in a total of 389545 bounding boxes. JAAD dataset has 336 videos with a resolution of 1920 × 1080 pixels and 10 videos with a resolution of 1280 × 720 pixels. The frame rate is 30 fps, which leads to a total of 82032 frames. There are 79812 frames with the resolution of 1920 × 1080 pixels. 63850 frames are used for training and 15962 frames are used for testing. There are 2220 frames with the resolution of 1280 × 720 pixels. 1776 frames are used for training and 444 frames are used for testing.

#### b: CALTECH

Caltech dataset is developed by the California Institute of Technology. The dataset contains a large number of image frames, challenging low resolution (LR) images and images of occluded targets. The dataset consists of 10 hours of video with 2300 pedestrians marked. There are over 350000 bounding boxes. The resolution of the image is 640 × 480 pixels and the frame rate is 30 fps. 146622 frames in total are adopted in the experiment. 122185 frames are used for training and 24437 frames are used for testing.

Although our goal is to accelerate high resolution image pedestrian detection, experiments on low resolution images are still valuable. The Caltech dataset has an equal number of pedestrian bounding boxes as the JAAD dataset. However, the situation in the Caltech dataset is more complex due to the different environmental parameters set. These complex environmental conditions could test the robustness of the proposed algorithm.

### 2) PARAMETERS FOR NPAE

There are six environmental parameters: camera height $h_c$, camera pitch angle $\gamma$, camera vertical FOV $\alpha$, pedestrian trian-camera distance $d_{pc}$, pedestrian height $h_p$, vertical resolution $px_v$.

According to [31], for JAAD with a resolution of 1920 × 1080: Vertical resolution $px_v$ = 1080 pixels. Camera vertical FOV $\alpha$ = 73°. Camera pitch angle $\gamma$ = 0°. Camera height $h_c$ = 1.6 m. For JAAD with a resolution of 1280 × 720: Vertical resolution $px_v$ = 720 pixels. Camera vertical FOV $\alpha$ = 73°. Camera pitch angle $\gamma$ = 0°. Camera height $h_c$ = 1.6 m. For Caltech with a resolution of 640 × 480: Vertical resolution $px_v$ = 480 pixels. Camera vertical FOV $\alpha$ = 27°. Camera pitch angle $\gamma$ = 0°. Camera height $h_c$ = 1.6 m. Based on the conclusions of the analysis in subsection III-C, we set pedestrian height $h_p$ = 1.8 m, pedestrian-camera distance $d_{pc}$ = 1.2 m.

Considering the car shudder, the Camera pitch angle $\gamma$ is varied between [-1°, 1°]. The adjusted environmental parameters can be used to derive the corresponding NPA for each dataset according to equation (1). The NPA interval for JAAD 1920 × 1080 is between $NPA_{\gamma=-1}$ and $NPA_{\gamma=1}$. We take the minimum value, so NPA = 405 pixels. To facilitate the experiment, the value of NPA is modified using equation (7).

$$NPA_{\text{mod}} = \text{NPA} - (\text{NPA} \mod 10) \quad (7)$$

Equation (7) is not obligatory. The purpose of using this operation is simply to make the experimental comparison clearer. The effect of this operation on detection performance is very slight. Using the same processing method, the NPA value of JAAD 1280 × 720 is 270 pixels. For the Caltech low resolution dataset, we want to verify the proposed NPAE algorithm under extreme situations. The $\gamma$ is not processed. We still applied equation (7). The NPA value of Caltech 640 × 480 is 70 pixels.
<figure><img src="image_6.png" /><figcaption>**FIGURE 4.** Two special pedestrian targets in red bounding boxes. (a) The pedestrian is not on the ground. (b) The camera angle of view is small and the distance between the pedestrian and the camera is short.</figcaption></figure>

### 3) PARAMETERS FOR PEDESTRIAN DETECTOR

The RetinaNet is adopted to detect pedestrians. There are several versions of RetinaNet, depending on the backbone used. We chose the resnet50 [25] as the backbone network. This network offers a better balance between detection performance and speed.

## B. EXPERIMENTS

Experiments are performed separately on high resolution (HR) images and low resolution (LR) images. The purpose of the experiments on HR images is to measure the performance of the NPAE algorithm acceleration. The experiments on LR images focus on the robustness of the NPAE algorithm in complex situations.

To compare the effects of detection acceleration, we designed comparison experiments for each resolution image. In general, the experiments are compared with two other image data optimization acceleration methods. The image compression method compresses the image to the same resolution as the NPAE-processed image. Since the NPAE algorithm is essentially an image cropping method, we compare the results of acceleration with different degrees of cropping. We also count the number of pedestrian targets affected by the image cropping method. There are two categories, targets that are partially cropped (PR) and targets that are completely removed (CR).

The evaluation metric used in this paper is the Average Precision (AP) [33]. It is a typical object detection task evaluation metric. We record the runtime of pedestrian detection accelerated by different image data optimization methods. The CPU experimental platform is Intel Xeon W-2104 with 16GB RAM. The GPU experimental platform is Nvidia Quadro P5000. Most of the current pedestrian detection algorithms are based on deep learning. The common platforms for deep learning frameworks are CPU and GPU, therefore our experiments focus on these two platforms. The exper-

imental results on the high resolution images are shown in Table 2 and Table 3.

We train RetinaNet using three resolution images obtained from two datasets. All images used for training are not cropped by the NPAE algorithm. We use the random rotation and random cropping to augment the input images. RetinaNet is trained for a total of 50 epochs with a learning rate of 1e-5.

After training, we combine the NPAE algorithm with RetinaNet for testing. During the test, the proposed algorithm accelerate pedestrian detection. The algorithm flow of using the NPAE algorithm to accelerate pedestrian detection is shown in Algorithm 1.

**Algorithm 1** Use NPAE Algorithm to Accelerate Pedestrian Detection

**Input:** *Environmental parameters*: Obtained from the input image collection environment.

*Images*: Data collected under the same camera settings.

**Output:** Pedestrian bounding boxes.

1. Calculate NPA value: Input *environmental parameters* to the NPAE algorithm.

2. Crop images: Crop all *images* according to the NPA value. The original image size is (H,W), and the cropped image size is (H-NPA, W).

3. Calculate pedestrian bounding box: The cropped images are sent to the pedestrian detector to get the pedestrian bounding boxes.

In Table 2, RetinaNet is the baseline. The detection result has 84.80% average precision (AP). It takes $RT_G = 175.7$ milliseconds to process an image on the GPU, which means 5.69 frames per second ($FPS_G$). On the CPU, the time to detect an image $RT_C$ is 4307.8 milliseconds, which means 0.23 frames per second ($FPS_C$). The proposed NPAE algorithm reduces the detection time of RetinaNet to 119.5 ms
<table><caption>**TABLE 2.** The experimental results on JAAD 1920 × 1080 pixels test data. In combination with the proposed NPAE algorithm (NPAE in the table) the detection speed on the GPU platform is 8.36 fps improved by 46.92%. The detection speed on the CPU platform is 0.35 fps, which is a 52.17% improvement. AP mean average precision. FPS mean frame per second on gpu platform. RT mean runtime. PedPC mean partially cropped pedestrian. PedCR mean completely removed pedestrian. G mean GPU platform. C mean CPU platform.</caption><thead><tr><th>Model</th><th>Resolution (pixels)</th><th>AP (%)</th><th>FPS<sub>G</sub></th><th>RT<sub>G</sub> (ms)</th><th>FPS<sub>C</sub></th><th>RT<sub>C</sub> (ms)</th><th>Ped<sub>PC</sub></th><th>Ped<sub>CR</sub></th></tr></thead><tbody><tr><td>RetinaNet</td><td>1920 × 1080</td><td>84.60</td><td>5.69</td><td>175.7</td><td>0.23</td><td>4307.8</td><td>0</td><td>0</td></tr><tr><td>Compress</td><td>1920 × 680</td><td>61.75</td><td>8.36</td><td>119.6</td><td>0.35</td><td>2819.9</td><td>0</td><td>0</td></tr><tr><td>Crop-150</td><td>1920 × 930</td><td>82.94</td><td>6.04</td><td>165.4</td><td>0.26</td><td>3769.1</td><td>0</td><td>0</td></tr><tr><td>Crop-200</td><td>1920 × 880</td><td>87.78</td><td>6.38</td><td>156.7</td><td>0.27</td><td>3584.9</td><td>0</td><td>0</td></tr><tr><td>Crop-250</td><td>1920 × 830</td><td>82.59</td><td>7.12</td><td>140.4</td><td>0.29</td><td>3381.8</td><td>0</td><td>0</td></tr><tr><td>Crop-300</td><td>1920 × 780</td><td>86.74</td><td>7.58</td><td>131.8</td><td>0.31</td><td>3191.8</td><td>0</td><td>0</td></tr><tr><td>Crop-350</td><td>1920 × 730</td><td>83.01</td><td>7.91</td><td>126.4</td><td>0.33</td><td>2994.3</td><td>2</td><td>0</td></tr><tr><td>NPAE(Crop-400)</td><td>1920 × 680</td><td><b>84.85</b></td><td><b>8.36</b></td><td><b>119.5</b></td><td><b>0.35</b></td><td><b>2813.2</b></td><td><b>33</b></td><td><b>0</b></td></tr><tr><td>Crop-450</td><td>1920 × 630</td><td>84.03</td><td>9.14</td><td>109.4</td><td>0.38</td><td>2625.2</td><td>348</td><td>0</td></tr><tr><td>Crop-500</td><td>1920 × 530</td><td>81.50</td><td>9.50</td><td>105.2</td><td>0.41</td><td>2421.7</td><td>2657</td><td>3</td></tr></tbody></table>

<table><caption>**TABLE 3.** The experimental results on JAAD 1280 × 720 pixels test data. In combination with the proposed NPAE algorithm (NPAE in the table) the detection speed on the GPU platform is 14.42 fps improved by 39.19%. The detection speed on the CPU platform is 0.82 fps, which is a 60.78% improvement. AP mean average precision. FPS mean frame per second on gpu platform. RT mean runtime. PedPC mean partially cropped pedestrian. PedCR mean completely removed pedestrian. G mean GPU platform. C mean CPU platform.</caption><thead><tr><th>Model</th><th>Resolution (pixels)</th><th>AP (%)</th><th>FPS<sub>G</sub></th><th>RT<sub>G</sub> (ms)</th><th>FPS<sub>C</sub></th><th>RT<sub>C</sub> (ms)</th><th>Ped<sub>PC</sub></th><th>Ped<sub>CR</sub></th></tr></thead><tbody><tr><td>RetinaNet</td><td>1280 × 720</td><td>60.35</td><td>10.23</td><td>97.7</td><td>0.51</td><td>1926.2</td><td>0</td><td>0</td></tr><tr><td>Compress</td><td>1280 × 450</td><td>22.97</td><td>14.40</td><td>69.4</td><td>0.83</td><td>1196.8</td><td>0</td><td>0</td></tr><tr><td>Crop-150</td><td>1280 × 570</td><td>59.61</td><td>11.96</td><td>83.6</td><td>0.63</td><td>1570.6</td><td>0</td><td>0</td></tr><tr><td>Crop-200</td><td>1280 × 520</td><td>61.23</td><td>12.69</td><td>78.8</td><td>0.70</td><td>1424.6</td><td>0</td><td>0</td></tr><tr><td>Crop-250</td><td>1280 × 470</td><td>59.25</td><td>14.06</td><td>71.1</td><td>0.76</td><td>1304.6</td><td>0</td><td>0</td></tr><tr><td>NPAE(Crop-270)</td><td>1280 × 450</td><td><b>60.91</b></td><td><b>14.24</b></td><td><b>70.2</b></td><td><b>0.82</b></td><td><b>1206.5</b></td><td><b>0</b></td><td><b>0</b></td></tr><tr><td>Crop-300</td><td>1280 × 420</td><td>58.81</td><td>14.90</td><td>67.1</td><td>0.84</td><td>1177.5</td><td>20</td><td>0</td></tr><tr><td>Crop-350</td><td>1280 × 370</td><td>51.71</td><td>16.07</td><td>62.2</td><td>0.98</td><td>1018.7</td><td>252</td><td>0</td></tr></tbody></table>

<table><caption>**TABLE 4.** The experimental results on Caltech 640 × 480 pixels test data. There is limited acceleration on low resolution images. There are pedestrian targets that are partially cropped after processing by the NPAE algorithm, but the detection accuracy is not much reduced. AP mean average precision. FPS mean frame per second on gpu platform. RT mean runtime. PedPC mean partially cropped pedestrian. PedCR mean completely removed pedestrian. G mean GPU platform. C mean CPU platform.</caption><thead><tr><th>Model</th><th>Resolution (pixels)</th><th>AP (%)</th><th>FPS<sub>G</sub></th><th>RT<sub>G</sub> (ms)</th><th>FPS<sub>C</sub></th><th>RT<sub>C</sub> (ms)</th><th>Ped<sub>PC</sub></th><th>Ped<sub>CR</sub></th></tr></thead><tbody><tr><td>RetinaNet</td><td>640 × 480</td><td>66.40</td><td>22.52</td><td>44.4</td><td>1.48</td><td>674.5</td><td>0</td><td>0</td></tr><tr><td>Compress</td><td>640 × 410</td><td>49.23</td><td>23.98</td><td>41.7</td><td>1.70</td><td>587.3</td><td>0</td><td>0</td></tr><tr><td>Crop-40</td><td>640 × 440</td><td>53.01</td><td>22.98</td><td>43.5</td><td>1.60</td><td>623.6</td><td>41</td><td>5</td></tr><tr><td>Crop-60</td><td>640 × 420</td><td>58.67</td><td>23.64</td><td>42.3</td><td>1.66</td><td>602.4</td><td>68</td><td>17</td></tr><tr><td>NPAE(Crop-70)</td><td>640 × 410</td><td><b>58.20</b></td><td><b>23.81</b></td><td><b>42.0</b></td><td><b>1.70</b></td><td><b>586.5</b></td><td><b>90</b></td><td><b>27</b></td></tr><tr><td>Crop80</td><td>640 × 400</td><td>43.11</td><td>23.42</td><td>41.7</td><td>1.75</td><td>568.7</td><td>177</td><td>36</td></tr></tbody></table>

and 2813.2 ms on two platforms. The detection speed is 8.36 fps on GPU and 0.35 fps on CPU. The detection speed is increased by 46.92% and 52.17% respectively. After NPAE acceleration, the AP of the detector RetinaNet remains at 84.85%.

Image compression method, as Compress mentioned in Table 2, can also accelerate detection. However, there is a significant loss of accuracy. The experimental detection accuracy AP is only 61.75%. The results can be further improved to a certain extent by extensive multi-scale training. Nevertheless, this improvement is limited and leads to a more complex process.

NPAE is an accurate image cropping method. We have also labeled it Crop-NPA. In order to compare the performance of NPAE, seven experiments with different cropping values are designed in Table 2. These experiments are named Crop-Value and Value is the number of rows of cropped pixels. In addition to analyzing the detection accuracy AP and detection speed FPS/RT, the pedestrian targets affected by the cropping should also be considered.

The partially cropped pedestrian target number is recorded as PedPC, and the completely removed pedestrian is labeled as PedCR.

In terms of acceleration, the speed increases as the degree of shearing increases. In terms of detection accuracy, it first remains constant and then decreases. The inflexion point occurs around the NPA value estimated by the NPAE algorithm. The PedPC and PedCR values also show an inflexion point around the NPA value. This pattern means that more and more pedestrian area information is lost as the cropping range exceeds the NPA value. This results in a decrease in detection accuracy.

The same conclusion can be drawn from the experimental results on the JAAD 1280 × 720 dataset in Table 3. On the GPU platform, the proposed NPAE algorithm increases the detection speed by 39.19% from 10.23 fps to 14.24 fps relative to baseline. On the CPU platform, detection is 60.78% faster, from 0.51 fps to 0.82 fps. The NPAE algorithm increases the speed significantly while the detection accuracy remains unaffected.
The experimental results on the low resolution images are shown in Table 4. There is little room for image data optimization on low resolution images. Nevertheless, the similar pattern as in Table 2 and Table 3 can also be seen. Despite the decrease in accuracy, the speed has increased. What is enlightening is the quantity of pedestrian partially cropped *PedPC* and pedestrian completely removed *PedCR*.

The detection accuracy of the accelerated detectors remains consistent from the *Crop-40* method to the NPAE algorithm. Although some pedestrians are partially cropped, it does not significantly affect the detection results. This means that the detector can still get the information it needs from *PedPC*. With further cropping, such as *Crop-80*, the detection accuracy decreases dramatically. At this point, the detector no longer has enough information to ensure the correctness of the detection results. There are 27 pedestrians that are completely removed indicated by *PedCR*. These targets are consistent with the situation discussed in subsection III-C.

From the experimental results, the proposed NPAE algorithm is suitable for optimizing high resolution images. The acceleration obtained for pedestrian detection is significant and the algorithm does not affect the detection accuracy. The NPAE algorithm can also be applied on low resolution images, but with ordinary results.

## V. CONCLUSION

In this paper, we present an algorithm for accelerating pedestrian detection with high resolution images, named Non-Pedestrian Area Estimation (NPAE). Based on six environmental parameters, the NPAE algorithm can accurately estimate the non-pedestrian areas in the dataset. Therefore, the proposed algorithm crops the image more accurately compared to other image data optimization methods. Because the image area is reduced and the target information is preserved, the pedestrian detector gains acceleration while maintaining accuracy.

Two high resolution images, 1920 × 1080 and 1280 × 720, and two different computing platforms, GPU and CPU, are used for the experiments. In the GPU platform, the detection speed is increased by 46.92% and 39.19%, respectively. In the CPU platform, the detection speed is increased by 52.17% and 60.78%, respectively. All experiments have shown that the proposed algorithm does not affect the detection accuracy. A low resolution dataset is used to confirm the robustness of the NPAE algorithm.

The NPAE algorithm performs better compared to traditional image data optimization methods. Image compression usually results in a loss of detection accuracy. Compared to NPAE algorithm, image compression loses 23.10% of average precision on 1920 × 1080 resolution, 37.94% on 1280 × 720, and 8.97% on 640 × 480. Image cropping cannot be calculated to obtain the optimal size of the cropping area, so it is difficult to accurately balance detection accuracy and detection speed.

The NPAE algorithm has four advantages: 1). It accelerates detection without loss of accuracy, 2). It can be used on a

variety of computing platforms, 3). It is easily integrated with other acceleration and detection methods. 4). it does not require any training.

In future work, we will explore the use of NPAE algorithm for pedestrian image position estimation by introducing depth information. At the same time, the NPAE algorithm can also be used to accelerate pedestrian detection of multi-modal fusions.

## ACKNOWLEDGMENT

The authors would like to thank the Centre Régional Informatique d'Applications Numériques de Normandie (CRI-ANN) for providing computational resources.

## REFERENCES

[1] G. Batouli, M. Guo, B. Janson, and W. Marshall, "Analysis of pedestrian-vehicle crash injury severity factors in Colorado 2006-2016," *Accident Anal. Prevention*, vol. 148, Dec. 2020, Art. no. 105782.

[2] A. Calvi, F. D'Amico, C. Ferrante, and L. B. Ciampoli, "Effectiveness of augmented reality warnings on driving behaviour whilst approaching pedestrian crossings: A driving simulator study," *Accident Anal. Prevention*, vol. 147, Nov. 2020, Art. no. 105760.

[3] A. Brunetti, D. Buongiorno, G. F. Trotta, and V. Bevilacqua, "Computer vision and deep learning techniques for pedestrian detection and tracking: A survey," *Neurocomputing*, vol. 300, pp. 17-33, Jul. 2018.

[4] F. Goverment. (2020). *Limitations De Vitesse*. [Online]. Available: https://www.service-public.fr/particuliers/vosdroits/F19460

[5] M. Sabri and A. Fauza, "Analysis of vehicle braking behaviour and distance stopping," in *Proc. IOP Conf. Ser., Mater. Sci. Eng.*, vol. 309, Feb. 2018, Art. no. 012020.

[6] B. Bosquet, M. Mucientes, and V. M. Brea, "STDnet: Exploiting high resolution feature maps for small object detection," *Eng. Appl. Artif. Intell.*, vol. 91, May 2020, Art. no. 103615.

[7] X. Huang, X. Cheng, Q. Geng, B. Cao, D. Zhou, P. Wang, Y. Lin, and R. Yang, "The ApolloScape dataset for autonomous driving," in *Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshops*, Jun. 2018, pp. 954-960.

[8] J. Cheng, P.-S. Wang, G. Li, Q.-H. Hu, and H.-Q. Lu, "Recent advances in efficient computation of deep convolutional neural networks," *Frontiers Inf. Technol. Electron. Eng.*, vol. 19, no. 1, pp. 64-77, Jan. 2018.

[9] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, "SSD: Single shot multibox detector," in *Proc. Eur. Conf. Comput. Vis. Springer*, 2016, pp. 21-37. [Online]. Available: https://link.springer.com/chapter/10.1007/978-3-319-46448-0_2

[10] J. Redmon and A. Farhadi, "YOLO9000: Better, faster, stronger," in *Proc. IEEE Conf. Comput. Vis. Pattern Recognit.*, Jul. 2017, pp. 7263-7271.

[11] A. Geiger, P. Lenz, C. Stiller, and R. Urtasun, "Vision meets robotics: The KITTI dataset," *Int. J. Robot. Res.*, vol. 32, no. 11, pp. 1231-1237, Sep. 2013.

[12] L. Zhang, L. Lin, X. Liang, and K. He, "Is faster R-CNN doing well for pedestrian detection?" in *Proc. Eur. Conf. Comput. Vis. Springer*, 2016, pp. 443-457. [Online]. Available: https://link.springer.com/chapter/10.1007/978-3-319-46475-6_28

[13] W.-J. Niu and Z.-K. Feng, "Evaluating the performances of several artificial intelligence methods in forecasting daily streamflow time series for sustainable water resources management," *Sustain. Cities Soc.*, vol. 64, Jan. 2021, Art. no. 102562.

[14] R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation," in *Proc. IEEE Conf. Comput. Vis. Pattern Recognit.*, Jun. 2014, pp. 580-587.

[15] R. Girshick, "Fast R-CNN," in *Proc. IEEE Int. Conf. Comput. Vis.*, Dec. 2015, pp. 1440-1448.

[16] S. Ren, K. He, R. Girshick, and J. Sun, "Faster R-CNN: Towards real-time object detection with region proposal networks," in *Proc. Adv. Neural Inf. Process. Syst.*, 2015, pp. 91-99.

[17] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You only look once: Unified, real-time object detection," in *Proc. IEEE Conf. Comput. Vis. Pattern Recognit.*, Jun. 2016, pp. 779-788.
[18] T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, “Focal loss for dense object detection,” in *Proc. IEEE Int. Conf. Comput. Vis.*, Oct. 2017, pp. 2980–2988.

[19] T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature pyramid networks for object detection,” in *Proc. IEEE Conf. Comput. Vis. Pattern Recognit.*, Jul. 2017, pp. 2117–2125.

[20] Y. Guo, A. Yao, and Y. Chen, “Dynamic network surgery for efficient DNNs,” in *Proc. 30th Int. Conf. Neural Inf. Process. Syst.*, 2016, pp. 1387–1395.

[21] S. Wen, J. Chen, Y. Wu, Z. Yan, Y. Cao, Y. Yang, and T. Huang, “CKFO: Convolution kernel first operated algorithm with applications in memristor-based convolutional neural network,” *IEEE Trans. Comput.-Aided Design Integr. Circuits Syst.*, early access, Sep. 4, 2020, doi: 10.1109/TCAD.2020.3019993.

[22] X. Zhang, J. Zou, K. He, and J. Sun, “Accelerating very deep convolutional networks for classification and detection,” *IEEE Trans. Pattern Anal. Mach. Intell.*, vol. 38, no. 10, pp. 1943–1955, Oct. 2016.

[23] J. Wu, C. Leng, Y. Wang, Q. Hu, and J. Cheng, “Quantized convolutional neural networks for mobile devices,” in *Proc. IEEE Conf. Comput. Vis. Pattern Recognit.*, Jun. 2016, pp. 4820–4828.

[24] J. Yim, D. Joo, J. Bae, and J. Kim, “A gift from knowledge distillation: Fast optimization, network minimization and transfer learning,” in *Proc. IEEE Conf. Comput. Vis. Pattern Recognit.*, Jul. 2017, pp. 4133–4141.

[25] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in *Proc. IEEE Conf. Comput. Vis. Pattern Recognit.*, Jun. 2016, pp. 770–778.

[26] S. Wen, M. Dong, Y. Yang, P. Zhou, T. Huang, and Y. Chen, “End-to-end detection-segmentation system for face labeling,” *IEEE Trans. Emerg. Topics Comput. Intell.*, early access, Nov. 6, 2019, doi: 10.1109/TETCI. 2019.2947319.

[27] S. Wen, W. Liu, Y. Yang, P. Zhou, Z. Guo, Z. Yan, Y. Chen, and T. Huang, “Multilabel image classification via feature/label co-projection,” *IEEE Trans. Syst., Man, Cybern., Syst.*, early access, Feb. 6, 2020, doi: 10.1109/ TSMC.2020.2967071.

[28] M. Dong, S. Wen, Z. Zeng, Z. Yan, and T. Huang, “Sparse fully convolutional network for face labeling,” *Neurocomputing*, vol. 331, pp. 465–472, Feb. 2019.

[29] J. Han, J. Pei, and M. Kamber, *Data Mining: Concepts and Techniques*. Amsterdam, The Netherlands: Elsevier, 2011.

[30] P. M. Visscher, “Sizing up human height variation,” *Nature Genet.*, vol. 40, no. 5, pp. 489–490, May 2008.

[31] A. Rasouli, I. Kotseruba, and J. K. Tsotsos, “Agreeing to cross: How drivers and pedestrians communicate,” in *Proc. IEEE Intell. Vehicles Symp. (IV)*, Jun. 2017, pp. 264–269.

[32] P. Dollar, C. Wojek, B. Schiele, and P. Perona, “Pedestrian detection: A benchmark,” in *Proc. IEEE Conf. Comput. Vis. Pattern Recognit.*, Jun. 2009, pp. 304–311.

[33] M. Zhu, “Recall, precision and average precision,” Dept. Statist. Actuarial Sci., Univ. Waterloo, Waterloo, ON, Canada, Tech. Rep., 2004, vol. 2, p. 30.

<figure><img src="image_7.png" /></figure>

**HAODI ZHANG** received the M.Sc. degree in engineering from Xidian University, Xi’an, Shaanxi, China, in 2019. He is currently pursuing the Ph.D. degree with the LITIS Lab National Institute of Applied Science Rouen, Rouen, France. His research interests include pedestrian recognition, detection, 3D perception and multi-modal fusion based on vision, lidar, and radar for autonomous vehicle.

<figure><img src="image_8.png" /></figure>

**DĂNUȚ OVIDIU POP** received the Ph.D. degree from INSA Rouen, Rouen, France, in 2019. He is currently working at the INRIA Paris, RITS Team, Paris, France, in collaboration with the INSA Rouen LITIS Laboratory. His research interests include classification, detection, actions prediction, and tracking of road users based on vision, radar, and sensors fusion methods for the intelligent vehicle.

<figure><img src="image_9.png" /></figure>

**ALEXANDRINA ROGOZAN** received the Ph.D. degree from Orsay University of Paris-Sud, in 1999. Since 2000, she has been an Associate Professor at INSA Rouen Normandie. She has published more than 70 articles in international journals, conferences, and book chapters. Her teaching activities include statistics and information theory. Her research interests include concern machine learning and multi-modal fusion techniques in the area of intelligent vehicles, medical imaging, text-image-emotion understanding, and speech recognition, among others.

<figure><img src="image_10.png" /></figure>

**ABDELAZIZ BENSRHAIR** received the M.Sc. degree in electrical engineering and the Ph.D. in computer science from the University of Rouen, France, in 1989 and 1992, respectively. He was the Head of the Intelligent Transportation Systems Division, from 2007 to 2012, and the Co-Director of the Computer Science, Information Processing, and Systems Laboratory (LITIS), National Institute of Applied Science Rouen (INSA), from 2002 to 2016, where he is currently a Professor with the Information Systems Architecture Department.