Decoding: State Of The Art Object Detection | by Kb Pachauri | Towards Data Science

EfficientDet is a neural network architecture which achieves State-Of-The-Art (SOTA) results (~55.1 Average Precision) on object detection (Microsoft COCO dataset) task with much lesser (~4x-9x) complexity than the previous detectors [2]. The key building blocks of EfficientDet are
Compound ScalingBi-directional Feature Pyramid Network (BiFPN)
Compound Scaling
Bi-directional Feature Pyramid Network (BiFPN)
Scaling up ConvNets in width (b), depth(c), or resolution (d) is widely used to achieve better accuracy, but require tedious manual tuning when scaling is done in 2 or 3 dimensions and still led to sub-optimal accuracy and efficiency. Compound scaling (e) uses a single compound coefficient φ to scale network width, depth, and resolution in a principled way according to the below equations.
The compound scaling is formulated as an optimization problem given the baseline network and target flops find out the scaling coefficient for scaling the base network as shown in the above equations and is done in 2 steps:
“1. Fix φ = 1, assume 2x more resources available, do a grid search of ⍺, β, γ based on the above equations.
2. Fix ⍺, β, γ as constants and scale-up baseline network with different φ ” [3]
The parameter of above left equation Fi (Input Layer), Li (Number of layers), Hi (Height), Wi (Width), Ci (Number of channels) are predefined parameters of the baseline network as shown in the below Efficient network details table, and w, d, r are coefficient for scaling network width, depth, and resolution. The parameters of the above right equation φ control resources for model scaling, while ⍺, β, γ controls depth, width, or resolution scaling respectively. Since compound scaling does not change layer operators in the baseline network, having a strong baseline network is also critical. “The baseline network is selected by leveraging a multi-objective neural architecture search that optimizes accuracy and FLOPs”[3].
Multi-scale feature fusion aggregates feature at different resolutions and widely used practice to learn scale-invariant features. Feature Pyramid Network [FPN] (a) fuse features top-down from level 3 to 7 and is limited by one direction information flow, Path Aggregation Network [PANet] (b) adds bottom-up pathway on top of FPN, NAS-FPN (c) uses neural architecture search to find cross-scale feature network topology and required 1000+ hours of search. Also found network using NAS-FPN is irregular and difficult to interpret. [2]
BiFPN (d) integrates bi-directional cross-scale connections with weighted feature fusion. The below equation and block diagram describe the fused feature for level 6.
P7_in, P6_in (x, y, w) are the input features at level 6/7 respectively, P5_out is the out features at level-5, P6_out (x, y, w) is the out fused features at level-6. P7_in features are upsampled using nearest-neighbor interpolation and P5_out features are downsampled using max-pooling for the features fusion at level 6. The WeightedAddition layer implements fast normalized fusion, positive weights are ensured by applying Relu on each weight in the WeightedAddition layer.
EfficientDet architecture uses 4 major networks, EfficientNet backbone, BiFPN, Bounding box prediction, and class prediction network.
Input Image resolution: Since BiFPN uses features from level 3–7, input resolution must be divisible by pow(2, 7) = 128, and the input image is scaled according to the below equation
BiFPN network: Since depth needs to be rounded to the smallest integers, depth is scaled linearly. For width (#channels) exponentially grow BiFPN, for width grid search is performed and the optimal value of 1.35 was chosen
Backbone network: EfficientDet uses the EfficientNet network without any change, to reuse pre-trained weights of the imagenet model. The below table shows the efficientNet B0 and B7 network details.
EfficientNet uses mobile inverted bottleneck MBConv as a building block and relies on the idea of depthwise separable convolution and residual connection which enable faster training and better accuracy. These MBConv blocks as shown in the above image, are scaled according to the compound scaling coefficient φ (num_repeat) in the backbone network.
Box/class prediction network: The width of the box and class prediction network is scaled according to BiFPN network width and depth linearly scales according to the below equation.
The below table shows the scaling config of EfficientDet detectors according to the compound coefficient φ.
#Make sure python3 pip3 is installed and updatedsudo apt-get updatesudo apt install python3-pip sudo -H pip3 install -U pip (to upgrade the pip to latest version)#Clone EfficientDet git repogit clone https://github.com/google/automl.gitcd ~/automl/efficientdet#Install all the EfficientDet requirementspip3 install -r requirements.txt#Download the pretrained weights. Bold d0 represent the model version and can be in the range d0-d7.wget https://storage.googleapis.com/cloud-tpu-checkpoints/efficientdet/coco/efficientdet-d0.tar.gztar zxf efficientdet-d0.tar.gzmkdir -p ./savedmodels/efficient-d0# Export saved model.python3.6 model_inspect.py --runmode=saved_model --model_name=efficientdet-d0 --ckpt_path=./efficientdet-d0 --hparams="image_size=1920x1280" --saved_model_dir=./savedmodels/efficientdet-d0#Make output dir and do inferencing with the saved modelmkdir -p outdirpython3.6 model_inspect.py --runmode=saved_model_infer --model_name=efficientdet-d0 --saved_model_dir=./savedmodels/efficientdet-d0 --input_image=path_to_input_image --output_image_dir=./output --min_score_thresh=0.6
EfficientDet output looks to be better on the tested images with respect to the cloud vision APIs. A detailed comparative analysis is required to conclude EfficientDet quality with respect to production cloud vision APIs. EfficientDet with novel BiFPN and compound scaling will definitely serve as a new foundation of future object detection related research and will make object detection models practically useful for many more real-world applications.
Thanks for reading the article, I hope you found this to be helpful. If you did, please share it on your favorite social media so other folks can find it, too. Also, let us know in the comment section if something is not clear or incorrect.