Anchor Free One Stage 3D Object Detection
By: Nabil Madali
Tags: Computer Vision, Object Detection, Point Cloud, Machine Learning

Efficient point cloud 3D object detection running on embedded systems is very important for many robotic applications, including autonomous driving. Most of the previous work tried to use Anchor-based detection methods to solve it, which has two shortcomings:

Post-processing is relatively complicated and computationally intensive.

Adjusting Anchor point parameters is very tricky

[1] propose an anchor free and NMS free one stage end-to-end point cloud 3D object detector (AFDet) with simple post-processing to solve these shortcomings.

The framework of anchor free one stage 3D detection (AFDet) system and detailed structure of anchor free detector. The whole pipeline consists of the point cloud encoder, the backbone and necks, and the anchor free detector. The number in the square brackets indicate the number of last convolution layer’s output channels. C is the number of categories used in the detection. Better viewed in color and zoomed in for details.Source[1]

The most important design here is the Anchor-Free detection head, which consists of five different heads to predict object centers in the BEV plane and to regress different attributes of the 3D bounding boxes, the outputs of the five heads are combined together to generate the detection results.

Generating Ground Truth boxes

For a 3D object k with category c_k, we parameterize its 3D ground truth bounding box as

Where x (k) , y (k) , θ (k) represent the center location in LiDAR coordinate system, w (k) , l (k) , h (k) are the width, length and height of the bounding box, and θ (k) is the yaw rotation around z-axis which is perpendicular to the ground.

Let [(back, front),(left, right)] denote the detection range in x-y plane. To be specific, back and front is along the x-axis and left and right is along the y-axis in the LiDAR coordinate system. In this work, the pillar in x-y plane is always a square. So let b denote the pillar side length, for each object center we have the keypoint

in BEV pseudo image coordinate.

Let,

be the equivalent in the keypoint heatmap where [] is the floor operation. The 2D bounding box in BEV could be expressed as

For each pixel (x , y) which are covered in the 2D bounding boxes in the pseudo image, we set its value in the heatmap following

where d is the Euclidean distance calculated between the bounding box center and the corresponding pixel in the discretized pseudo image coordinates.

Network Architecture

First they use PointPillars to encode the entire point cloud into pseudo images or image-like feature maps in Bird’s Eye View (BEV).However, AFDet can be used with any point cloud encoder which generates pseudo images or image-like 2D data.

We consider a three-dimensional point cloud with N points

With K-dimensional features

The original point cloud is converted into a stacked column tensor (pillar tensor) and a column index tensor (pillar index tensor).

Aggregate features from points in a pillar using a PointNet to get pillar-wise features, as follows:

Where,F_P (v_j ) gives the set of points in pillar vj.

Pillar-wise features are further transformed through an additional convolutional neural network (CNN) as follows:

To retrieve point-wise features from pillars, the pillar-to-point feature projection is given by

Where, FV (pi) returns the index j of pi ’s corresponding pillar vj .

Since P is the number of selected pillars, they are not one-to-one correspondent with the original pillars in the entire detection range. So the third step is to scatter the selected P pillars to their original location on the detection range. After that we can get a pseudo image

where W and H indicate the width and height, separately.

The backbone part is used to extract features while downsampling the spatial size through different blocks. The necks part is used to upsample the features to make sure all outputs from different blocks of the backbone have the same spatial size so that we can concatenate them along one axis.

Let define a block B(T, E, A) that consists of E convolution layers with A output channels, each followed by a BatchNorm and a ReLU. T is defined as the downsampling stride for this block.

And define the upsampling necks V(T, A), each upsampling neck contains one transposed convolution with A output channels and T upsampling stride followed by BatchNorm and ReLU.

The backbone and necks we used in KITTI detection. Different colors represent different operations with different parameters. The pseudo image is from point cloud encoder. t represents stride. F is the number of channels of the pseudo image. W and H are the width and height, separately. T. Conv is short for transposed convolution. Better viewed in color.Source[1]

The final backbone and necks consists of two blocks B(1, 7, 32) and B(2, 8, 64) followed by two upsampling necks V(1, 64), V(2, 64), separately.

The anchor free detector consists of five heads:

Keypoint heatmap head

Local offset head

z-axis location head

3D object size head

Orientation head

For heatmap head and offset head, they predict a keypoint heatmap

and a local offset regression map

Where C is the number of keypoint types. The keypoint heatmap is used to find where the object center is in BEV. The offset regression map is to help the heatmap to find the more accurate object centers in BEV and also help to recover the discretization error caused by the pillarization process.To be specific, once the heatmap predicts a wrong center which is several pixels away from the ground truth center, the offset head has the capability to mitigate and even eliminate several pixels’ error to the ground truth object center.

z-axis location regression. After the object localizations in BEV, we only have object x-y location. Thus we have the z-axis location head to regress the z-axis values. We directly regress z-value

Using L1 loss.

Size regression. Additionally, we regress the object sizes

directly. For each object, we have

Orientation prediction. Like the traditional method, the angle prediction is decomposed into two tasks:

Bin classification

Offset regression.

Specifically, it is divided into two bins:

For each bin, we predict

which are used for softmax classification and

which are used for calculating sin and cos value of the offset to the bin center γi .

Finally, the outputs of the five heads are combined together to generate the detection results.

The entire AFDet can be processed efficiently on a CNN accelerator or a GPU with the simplified post-processing. Without bells and whistles, the proposed AFDet performs competitively with other one stage anchor-based methods on KITTI validation set and Waymo Open Dataset validation set.

Reference

1.Ge, Runzhou, et al. “Afdet: Anchor free one stage 3d object detection.” arXiv preprint arXiv:2006.12671 (2020).