CONTEXT AUGMENTATION AND FEATURE REFINEMENT NETWORK FOR TINY OBJECT DETECTION

Abstract: Tiny objects are hard to detect due to their low resolution and small size. The poor detection performance of tiny objects is mainly caused by the limitation of network and the imbalance of training dataset. A new feature pyramid network is proposed to combine context augmentation and feature refinement. The features from multi-scale dilated convolution are fused and injected into feature pyramid network from top to bottom to supplement context information. The channel and spatial feature refinement mechanism is introduced to suppress the conflicting formation in multi-scale feature fusion and prevent tiny objects from being submerged in the conflict information. In addition, a data enhancement method called copy-reduce-paste is proposed, which can increase the contribution of tiny objects to loss during training, ensuring a more balanced training. Experimental results show that the mean average precision of target targets on the VOC dataset of the proposed network reaches 16.9% (IOU=0.5:0.95), which is 3.9% higher than YOLOV4, 7.7% higher than CenterNet, and 5.3% higher than RefineDet.

Reviews and author responses:

The paper aims at the tiny object detection and point out the issues are small context feature, semantic feature conflicts, and less tiny objects in training data.  To solve the aforementioned problems, authors introduce context augmentation module (CAM), design a feature refinement module, and adopt data-augmentation manner in training process. Experiments are conducted on PASCAL VOC dataset to validate the proposed methods and modules. ## Strength
### 1. The tiny object detection is a challenging and meaningful research topic in both theory and application.
## Weakness
### 1. The motivation of the work is not clear and the explanation is not convincing. In paper, the authors mention that "different densities directly will cause semantic conflicts", this should be justified experimentally or theoretically. 
### 2. The authors claim that the proposed CAM and FPM can solve the feature conflicts, this should extensively demonstrated and explained in-depth, not just some number and trivial feature visualization map.
### 3. The novelty of proposed method CAM, FPM, and data augmentation methods are slim. CAM is initially proposed Yu & Koltun, 2015. Data augmentation is a common operation to improve performance. If the authors want to justify the superiority of the proposed data augmentation method, they should systematically compare theirs with other data augmentation methods.
### 4. In order to validate the performance of the proposed method regarding tiny object, COCO dataset is more suitable than VOC.
### 5. Too many typos. Here some of them are listed. In abstract, "the precision of target targets" might be "the precision of tiny targets"
"great progress(Tonget al., 2020" should be "great progress (Tong et al., 2020", a space should be placed between text and bracket. In figure 1, the CEM should CAM.  Based on apparent weakness, the paper is nor ready to publish.

This paper proposed a method of combined context augmentation and feature refinement to eliminate the limitation of network and imbalance of dataset issues for tiny object detection. A data enhancement method is proposed to increase the contribution of tiny objects to loss during the training.  There are several technical issues of this work.

1) In context augmentation module, the method applies 1*1 convolutions to do the feature fusion. Although 1*1 convolution can improve the long-distance dependence to some extent, it requires much computations and overhead. So I
doubt if it is too heavy to apply so many 1*1 convolution in this module?

2) In the feature refinement module, the expression and the symbol of variables are too complex that may affect reader's understanding. The method is somehow similar to the dual attention network in [R1], which proposed both spatial and channel attention to do the segmentation. However that paper is not cited.

[R1] J. Fu, J. Liu, H. Tian, Y. Lin, Y. Bao, Z. Fang and H. Lu, “Dual Attention Network for Scene Segmentation,” In ICCV, 2019.

3) In the copy-reduce-paste data part, there is no description regarding how to avoid the overlap issue of pasting images, which I believe to be important. If the pasted object and previous one have some overlapping, the training performance will be definitely affected.

4) The paper lacks details on how data enhancement is performed, such as the selection of large objects to be copied and the scale to reduce the large objects. How to control such selections so that the dataset can be balanced?

5) To my understanding, the approach of [R2] is similar or highly related to the proposed work in solving the data balancing issue in tiny object detection. I suggest the authors to compare results with [R2]

[R2] M. Kisantal, Z. Wojna, J. Murawski, J. Naruniec and K. Cho, ”Augmentation for small object detection,” In CVPR 2019.

6) Detailed implementation details of the proposed method is lacking.

7) For the experiment, the paper utilized a different backbone that may make the comparison not fair.
Comparison results in Table 4 and Table 5 are not convincing. Table 4 compares the overall MAP on the VOC dataset (which is not designed specifically for tiny object detection). I think the authors should target datasets such as https://neurohive.io/en/news/a-summary-of-the-1st-tiny-object-detection-challenge/

Table 5 compares the detection performance of tiny objects, however all comparison methods are not designed specifically for tiny object detection.

9) As the title suggests, contexts are used for tiny object detection. However, there is no reference to relevant works that are specifically design to leverage contexts, such as 

[R3] J. Lim, M. Astrid, H. Yoon and S. Lee, ”Small Object Detection using
Context and Attention,” In ICAIIC 2021.
[R4] X. Tang, D. K. Du, Z. He and J. Liu, ”PyramidBox: A Context-assisted
Single Shot Face Detector,” In ECCV 2018.
[R5] H. Hu, J. Gu, Z. Zhang, J. Dai and Y. Wei, ”Relation Networks for Object
Detection,” In CVPR 2018.

10) There are some obvious written errors in the paper such as CEM and
FRM in Figure 1, which are supposed to be CAM and FRM. Both the technical method and evaluation results have many issues indicated above, so I recommend rejection.

This paper focuses on tiny object detection, and a new feature pyramid network is proposed to combine context augmentation and feature refinement. Experimental results show that the proposed method is better than YOLOV4, CenterNet, and RefineDet. Strengths:
1.	This paper mainly solve the problem of tiny object detection and obtain good results of tiny object detection on VOC dataset.
2.	A data augmentation method called copy-reduce-paste is proposed, which can increase the contribution of tiny objects to loss during training, ensuring a more balanced training.
Weaknesses:
1.	The innovation in the paper is limited, more like an assembly of existing work, such as DeepLabv3+, attention and spatial attention。
2.	The proposed method was not evaluated on other datasets, such as MS COCO dataset.
3.	The main focus of this paper is tiny object detection, but the analysis of small object is limited in the experimental results.

Question:
1.	What’s the ‘CEM’ and ‘FPM’ mean in Figure 1?
2.	The novelty of CAM is limited, A similar structure has been proposed in DeepLabv3+.
3.	The proposed FRM is a simple combination of channel attention and spatial attention. The innovative should be given in detail.
 The innovation of this article is limited, and the methods proposed exist in prior works, with minor changes. The originality is not strong. And the experiments are not enough with only a relatively simple VOC dataset.

Meta-review: This paper is proposed to address the tiny object detection with the help of the context augmentation module (CAM) and feature refinement module (FRM). To obtain rich context information for feature augmentation, CAM merges multi-scale dilated convolution features. The proposed method has been verified on the PASCAL VOC dataset with the considerable improvements over the latest baseline methods. The major concern of this paper is the novelty that similar ideas such as context augmentation and multi-scale have been commonly applied in previous works. And authors failed to provide the results on the COCO benchmark, which is more important than PASCAL VOC. Moreover, the authors have not offered any rebuttal to address the reviewers' concerns.

Decision: Reject