<<<:(<<<<<Why I quit on using Faster RCNN on KITTI Bird’s Eye View(BEV) Benchmark
By: Yunshuang Yuan
Tags: Object Detection, Faster Rcnn, Kitti

Faster RCNN was very popular, and has been used a lot. Because there are so many git repos that implemented the network, and most importantly, in different DL frameworks. But….if you are just like me, who is lazy on rewriting the code from scratch, it is better not to use Faster RCNN on BEV benchmarks. There is why:

If you want to detect boxes with heading angle, you nearly need to modify or rewrite most modules of the original Faster RCNN which only output bounding boxes. These modules are iou, nms, roi pooling, for which you might need to rewrite the cuda code if you need faster computation. But if you know some good implementations on this, please let me know. Thanks!

Comparing to the original point clouds, BEV itself losses a lot of information during the projection of point clouds into 2d images. The resulted grid maps mostly only contain some sparse points scattered on the 2d images. It is a waste of computation to do dense/conventional convolutions on them. Also, it is hard to decide which hand-crafted encoding method is better for generating grid maps.

The objects in BEV images are mostly small, especially the pedestrians and cyclists which appear in the BEV images with only several pixels and might also disappear after being down-sampled by a deep backbone encoder(eg. Resnet101, 16x down-sampling). Detector with pre-defined bounding boxes/anchors/proposals can hardly recall all these small objects. You may think about using less down-sampling, but this would reduce the size of receptive field of view for each pixel in the final feature map. How about dilated convolution? As described in [1], this could introduce checkerboard artifacts. Using more more layer but with less channels in each layer, FPN, or Unet-like structure may solve this problem, but that’s another story.

The two stage detection of Faster RCNN has more control flows(in perspective of programming) than other single shot detectors, which can be complex to have a very clear overview of the the whole processing pipeline. And I want to follow Occam’s Razor.

If you want to track the data flow intensively, for example, you want to deal with uncertainty estimation, it is better to use a network with simpler data flow structures.

Pre-defining anchors and then resorting and selecting a part from it in the intermediate stage of training or processing may make the network more incomprehensible by definition, which hurts the fluency of data flow and introduces uninterpretable or hard-interpretable features to uncertainty estimation. In contrast, simpler work/control flow may make the network generalize better on new data.

There are too many hyper parameters should be pre-defined. The anchor generation itself already needs about 6 or more parameters(ratios, anchor sizes, heading angles...), and each of other modules also need several parameters(number of proposals from the first stage to the second stage, nms thresholds, iou thresholds…). Tuning tens of parameters of this network on a brand new task can be very frustrating, even if you have some pre-experience on optimizing these parameters.

And…Uhmm, i just do not like it anymore! Because I love FCN, you are a simple guy! I like to play with simple and cute guys!

By the way, why does one people say, hey, we need bounding boxes! Then everybody needs it? Maybe it is time to get out from this trap!