Query: Small object Detection

title: Recent advances in small object detection based on deep learning: A review
abstract: Small object detection is a challenging problem in computer vision. It has been widely applied in defense military, transportation, industry, etc. To facilitate in depth understanding of small object detection, we comprehensively review the existing small object detection methods based on deep learning from five aspects, including multi scale feature learning, data augmentation, training strategy, context based detection and GAN based detection. Then, we thoroughly analyze the performance of some typical small object detection algorithms on popular datasets, such as MS COCO, PASCAL VOC. Finally, the possible research directions in the future are pointed out from five perspectives: emerging small object detection datasets and benchmarks, multi task joint learning and optimization, information transmission, weakly supervised small object detection methods and framework for small object detection task.

title: UAV YOLO: Small Object Detection on Unmanned Aerial Vehicle Perspective
abstract: Object detection, as a fundamental task in computer vision, has been developed enormously, but is still challenging work, especially for Unmanned Aerial Vehicle (UAV) perspective due to small scale of the target. In this study, the authors develop a special detection method for small objects in UAV perspective. Based on YOLOv3, the Resblock in darknet is first optimized by concatenating two ResNet units that have the same width and height. Then, the entire darknet structure is improved by increasing convolution operation at an early layer to enrich spatial information. Both these two optimizations can enlarge the receptive filed. Furthermore, UAV viewed dataset is collected to UAV perspective or small object detection. An optimized training method is also proposed based on collected UAV viewed dataset. The experimental results on public dataset and our collected UAV viewed dataset show distinct performance improvement on small object detection with keeping the same level performance on normal dataset, which means our proposed method adapts to different kinds of conditions.

title: Small Object Detection in Unmanned Aerial Vehicle Images Using Feature Fusion and Scaling Based Single Shot Detector With Spatial Context Analysis
abstract: Objects in unmanned aerial vehicle (UAV) images are generally small due to the high photography altitude. Although many efforts have been made in object detection, how to accurately and quickly detect small objects is still one of the remaining open challenges. In this paper, we propose a feature fusion and scaling based single shot detector (FS SSD) for small object detection in the UAV images. The FS SSD is an enhancement based on FSSD, a variety of the original single shot multibox detector (SSD) We add an extra scaling branch of the deconvolution module with an average pooling operation to form a feature pyramid. The original feature fusion branch is adjusted to be better suited to the small object detection task. The two feature pyramids generated by the deconvolution module and feature fusion module are utilized to make predictions together. In addition to the deep features learned by the FS SSD, to further improve the detection accuracy, spatial context analysis is proposed to incorporate the object spatial relationships into object redetection. The interclass and intraclass distances between different object instances are computed as a spatial context, which proves effective for multiclass small object detection. Six experiments are conducted on the PASCAL VOC dataset and the two UAV image datasets. The experimental results demonstrate that the proposed method can achieve a comparable detection speed but an accuracy superior to those of the six state of the art methods.

title: STDnet: Exploiting high resolution feature maps for small object detection
abstract: The accuracy of small object detection with convolutional neural networks (ConvNets) lags behind that of larger objects. This can be observed in popular contests like MS COCO. This is in part caused by the lack of specific architectures and datasets with a sufficiently large number of small objects. Our work aims at these two issues. First, this paper introduces STDnet, a convolutional neural network focused on the detection of small objects that we defined as those under 16 x 16 pixels. The high performance of STDnet is built on a novel early visual attention mechanism, called Region Context Network (RCN) to choose the most promising regions, while discarding the rest of the input image. Processing only specific areas allows STDnet to keep high resolution feature maps in deeper layers providing low memory overhead and higher frame rates. High resolution feature maps were proved to be key to increasing localization accuracy in such small objects. Second, we also present USC GRAD STDdb, a video dataset with more than 56,000 annotated small objects in challenging scenarios. Experimental results over USC GRAD STDdb show that STDnet improves the AP 5 of the best state of the art object detectors for small target detection from 50.8% to 57.4% Performance has also been tested in MS COCO for objects under 16 x 16 pixels. In addition, a spatio temporal baseline network, STDnet bST, has been proposed to make use of the information of successive frames, increasing the AP 5 of STDnet in 2.3% Finally, optimizations have been carried out to be fit on embedded devices such as Jetson TX2.

title: Extended Feature Pyramid Network for Small Object Detection
abstract: Small object detection remains an unsolved challenge because it is hard to extract information of small objects with only a few pixels. While scale level corresponding detection in feature pyramid network alleviates this problem, we find feature coupling of various scales still impairs the performance of small objects. In this paper, we propose extended feature pyramid network (EFPN) with an extra high resolution pyramid level specialized for small object detection. Specifically, we design a novel module, named feature texture transfer (FTT) which is used to super resolve features and extract credible regional details simultaneously. Moreover, we design a foreground background balanced loss function to alleviate area imbalance of foreground and background. In our experiments, the proposed EFPN is efficient on both computation and memory, and yields state of the art results on small traffic sign dataset Tsinghua Tencent 100K and small category of general object detection dataset MS COCO.

title: Beyond context: Exploring semantic similarity for small object detection in crowded scenes
abstract: Small object detection in crowded scene aims to find those tiny targets with very limited resolution from crowded scenes. Due to very little information available on tiny objects, it is often not suitable to detect them merely based on the information presented inside their bounding boxes, resulting low accuracy. In this paper, we propose to exploit the semantic similarity among all predicted objects' candidates to boost the performance of detectors when handling tiny objects. For this purpose, we construct a pairwise constraint to depict such semantic similarity and propose a new framework based on Discriminative Learning and Graph Cut techniques. Experiments conducted on three widely used benchmark datasets demonstrate the improvement over the state of the art approaches gained by applying this idea.

title: An Evaluation of Deep Learning Methods for Small Object Detection
abstract: Small object detection is an interesting topic in computer vision. With the rapid development in deep learning, it has drawn attention of several researchers with innovations in approaches to join a race. These innovations proposed comprise region proposals, divided grid cell, multiscale feature maps, and new loss function. As a result, performance of object detection has recently had significant improvements. However, most of the state of the art detectors, both in one stage and two stage approaches, have struggled with detecting small objects. In this study, we evaluate current state of the art models based on deep learning in both approaches such as Fast RCNN, Faster RCNN, RetinaNet, and YOLOv3. We provide a profound assessment of the advantages and limitations of models. Specifically, we run models with different backbones on different datasets with multiscale objects to find out what types of objects are suitable for each model along with backbones. Extensive empirical evaluation was conducted on 2 standard datasets, namely, a small object dataset and a filtered dataset from PASCAL VOC 2007. Finally, comparative results and analyses are then presented.

title: Multi block SSD based on small object detection for UAV railway scene surveillance
abstract: A method of multi block Single Shot MultiBox Detector (SSD) based on small object detection is proposed to the railway scene of unmanned aerial vehicle surveillance. To address the limitation of small object detection, a multi block SSD mechanism, which consists of three steps, is designed. First, the original input images are segmented into several overlapped patches. Second, each patch is separately fed into an SSD to detect the objects. Third, the patches are merged together through two stages. In the first stage, the truncated object of the sub layer detection result is spliced. In the second stage, a sub layer suppression and filtering algorithm applying the concept of non maximum suppression is utilized to remove the overlapped boxes of sub layers. The boxes that are not detected in the main layer are retained. In addition, no sufficient labeled training samples of railway circumstance are available, thereby hindering the deployment of SSD. A two stage training strategy leveraging to transfer learning is adopted to solve this issue. The deep learning model is preliminarily trained using labeled data of numerous auxiliaries, and then it is refined using only a few samples of railway scene. A railway spot in China, which is easily damaged by landslides, is investigated as a case study. Experimental results show that the proposed multi block SSD method produces an overall accuracy of 96.6% and obtains an improvement of up to 9.2% compared with the traditional SSD.

title: Context Aware Block Net for Small Object Detection.
abstract: State of the art object detectors usually progressively downsample the input image until it is represented by small feature maps, which loses the spatial information and compromises the representation of small objects. In this article, we propose a context aware block net (CAB Net) to improve small object detection by building high resolution and strong semantic feature maps. To internally enhance the representation capacity of feature maps with high spatial resolution, we delicately design the context aware block (CAB) CAB exploits pyramidal dilated convolutions to incorporate multilevel contextual information without losing the original resolution of feature maps. Then, we assemble CAB to the end of the truncated backbone network (e.g. VGG16) with a relatively small downsampling factor (e.g. 8) and cast off all following layers. CAB Net can capture both basic visual patterns as well as semantical information of small objects, thus improving the performance of small object detection. Experiments conducted on the benchmark Tsinghua Tencent 100K and the Airport dataset show that CAB Net outperforms other top performing detectors by a large margin while keeping real time speed, which demonstrates the effectiveness of CAB Net for small object detection.

title: Perceptual Generative Adversarial Networks for Small Object Detection
abstract: Detecting small objects is notoriously challenging due to their low resolution and noisy representation. Existing object detection pipelines usually detect small objects through learning representations of all the objects at multiple scales. However, the performance gain of such ad hoc architectures is usually limited to pay off the computational cost. In this work, we address the small object detection problem by developing a single architecture that internally lifts representations of small objects to super resolved ones, achieving similar characteristics as large objects and thus more discriminative for detection. For this purpose, we propose a new Perceptual Generative Adversarial Network (Perceptual GAN) model that improves small object detection through narrowing representation difference of small objects from the large ones. Specifically, its generator learns to transfer perceived poor representations of the small objects to super resolved ones that are similar enough to real large objects to fool a competing discriminator. Meanwhile its discriminator competes with the generator to identify the generated representation and imposes an additional perceptual requirement generated representations of small objects must be beneficial for detection purpose on the generator. Extensive evaluations on the challenging Tsinghua Tencent 100K [45] and the Caltech [9] benchmark well demonstrate the superiority of Perceptual GAN in detecting small objects, including traffic signs and pedestrians, over well established state of the arts.