Increasing Performance When Detecting Small Objects Using Data Crops
By: Peter Lebiedzinski
Tags: Small Object, Machine Learning, Crop, TensorFlow, Object Detection

The models using the sliced data perform 30 to 50 percent better than the base data, depending on the slice resolution chosen. For the tested crop resolution range of 480 to 640, the mAP at all thresholds except for 0.90 increases with the crop resolution. A possible explanation for this is that while the crop resolution increased, the tolerance was held constant at 10 %, allowing more freedom for the cropping tool to generate better crops of the data, and also more crops. Additionally, as the slice dimension increases past 640, lower resolution slices aren’t being generated for training to upsize them to 640 x 640. The 10 % tolerance allows for deviation in the positive or negative of the input slice dimension, allowing for 432 x 432 slices to be taken when specifying a slice dimension of 480. This effect completely disappears after a specified slice dimension over 710 is used with a 10 % tolerance.

The computer used for training is running Tensorflow-GPU and has the following specifications:

NVIDIA RTX 2060 Super (8 GB Vram)

AMD Ryzen 5 3600 6 Core Processor ( 3.6 GHz)

G.Skill Ripjaw V Series 32 GB 3200 MHz DDR 4 Ram

Samsung 970 EVO 500 GB SSD 3500/2500 MB/s Read/Write Speed

The average steps per second plotted on Tensorboard are used to compare training time between all three models, and is summarized below:

Steps Per Second using various slicing configurations, Image by Author

It appears that the various slicing configurations train at a similar rate, with the 480 x 480 configuration training 13 % slower than the 640 x 640 and base configurations.

The inference time was determined by running the various detection models on a video file of drone footage taken by the author. The Inference speed term is used synonymously with frames per second achieved by detecting objects in the video. The Inference speed remains fairly constant for all three of the models, and is summarized below:

Inference speed using various slicing configurations, Image by Author

Conclusion

The key take away is that taking data crops significantly increases the performance of an object detection model when detecting small objects, with little performance speed cost when inferencing.

All of the results have been tabulated for the same configuration parameters, so further performance upgrades can be made by tuning the training config parameters of the sliced data.

Extra

Similar performance metrics were measured for other datasets, and yielded similar results. Shown below are the results for an aerial image dataset taken in high resolution (4000 x 3000).

Mean Average Precision for various slicing configurations for secondary dataset, Image by Author

Data Cropping increases the Mean Average Precision by 4 to 5 times for this dataset. A reason why it is so effective for this dataset is that the objects are very high resolution in the images, allowing for distinct features to be recognized easily when cropped to a similar resolution as the training resolution. Additionally, when cropping the 4000 x 3000 base image to 640 x 640, the small objects become very difficult to detect.

Similar training time and inference speed trends were observed, shown below:

Steps Per Second using various slicing configurations for secondary dataset, Image by Author

Inference speed using various slicing configurations for secondary dataset, Image by Author

References

[1] Stanford Drone Dataset. In: Computational Vision and Geometry Lab. https://cvgl.stanford.edu/projects/uav_data/. Accessed 2 Dec 2020

[2] A. Robicquet, A. Sadeghian, A. Alahi, S. Savarese, Learning Social Etiquette: Human Trajectory Prediction In Crowded Scenes in European Conference on Computer Vision (ECCV), 2016.

[3] Tensorflow (2020) tensorflow/models. In: GitHub. https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md. Accessed 23 Nov 2020

[4] Rafaelpadilla rafaelpadilla/Object-Detection-Metrics. In: GitHub. https://github.com/rafaelpadilla/Object-Detection-Metrics. Accessed 23 Nov 2020

[5] Padilla R, Netto SL, da Silva AAB (2020) 2020 International Conference on Systems, Signals and Image Processing (IWSSIP), A Survey on Performance Metrics for Object-Detection Algorithms.