Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper addresses the crucial problem in computer vision: pedestrian detection. The issue of faster inference (i..e real-time performance) with competitive accuracy is still a major problem as stated by the author(s). To solve this major issue/problem, the author(s) proposes a single-stage anchor-free pedestrian detector with the deformable vision transformer to balance the issue of faster inference and high accuracy. Together with strong experiments, solid baselines, and evaluation criteria(s), the author(s) achieves competitive results on well-known datasets of Caltech and Citypersons. 
strengths_and_weaknesses: ++ Novelty

- The task formulation is concise, convincing, solid, and novel. A seemingly reasonable approach has been conducted in this paper. Compared to the existing single/two-stage detectors, the proposed single-stage detector based on the feature extractor and CSP strategy achieves competitive results.
- To the best of my knowledge, this simple strategy of incorporating the vision transformers on top of the backbone of pedestrian detection tasks is a solid contribution to the computer vision community. 

++ Evaluation

- The experiments are strong, sufficient, and very well presented. The analysis presented here showcases the author's efforts on how the small things have been noted down while performing such extensive experiments with solid evaluation criteria(s).
- The experimental evaluations demonstrate the effectiveness of the complete approach and showcase its practical value. The detailed analyses clarify the contribution of each component.

++ Clarity

 - The proposed solution by providing a multi-scale deformable feature extractor based on a deformable vision transformer is theoretically sound and well-written clearly describes the improvements and adequately contextualizes the contributions.
- The manuscript also provides a good description of related work and background, motivating the problem. Additionally, it provides a thorough description of the part of the pipeline it deals with.
questions: There is a major concern, I have on this manuscript's contribution that although the trade-off between higher accuracy and faster inference has been seriously taken by the author(s), there is little-to-less information has been provided on the occlusion part. Well, in general, if we narrow it down to common scenarios, we see an occluded person(s) in lot more cases and should be a concern to raise, whereas except for related work, I wasn't able to see major information on this issue. I appreciate the author's efforts in providing very clear information on their contribution, could I please ask the authors to also provide some insight on this case. 
limitations: There's a significant limitation in terms of providing insight on the occlusions/occluded person(s). However, I do not see any major limitations except the mentioned question, the proposed solution for a trade-off between high accuracy and faster inference using Vision Transformer is a pretty solid contribution, in my view to the computer vision community.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: Pedestrian detection is crucial in many applications, but the fluctuation of information from the images makes it difficult to do it accurately. CNN-based two-stage detectors are used for this task but are slow at inference and are difficult to train. Single-stage detectors solve these issues but have a higher miss rate, especially in the presence of occlusion. The authors suggest that these problems stem from information fed to the detection head: single-stage detectors get insufficient information compared to two-stage detectors. 

To solve the above-mentioned problems, the authors proposed to use a deformable vision transformer. The transformer enhances multi-scale features extracted from a backbone, thereby providing better information to the detection head. They also proposed to use center-and-scale prediction (CSP). The proposed method uses multi-scale features extracted from a CNN. These features are passed through a detection neck. These features are then passed through a deformable transformer to enhance them. These enhanced features are then used to find three outputs that are relevant to CSP. Comprehensive experiments are performed to show the performance of the method on two pedestrian datasets. A set of ablation studies are also conducted to show the effect of different components of the method.
strengths_and_weaknesses: ## Strengths

1. The paper is mostly well written and easy to follow.

2. The motivation, to bring the speed of inference in single-stage detectors by using features enhanced by transformers, is interesting. 

3. The well-motivated problem is then solved in a way that makes sense. 

4. Results (in Table 5, 6) show that enhanced feature maps do improve results, especially in the presence of occlusion.

5. Comprehensive experiments are performed to show not only the SOTA results but also the effect of individual components. 

Weaknesses are pointed out in the next section.
questions: ## Questions / Weaknesses

1. What is the main contribution of the paper. More specifically, what parts of the method are proposed. Or the main contribution is to show the effectiveness of vision transformers in overcoming the issues in single-stage detectors? Both contributions (proposing a new method or showing the effectiveness of existing ones) are equally important, but this should be highlighted clearly.

2. If I understand it correctly, the main proposed components are the use of deformable transformers to enhance features and the use of CSP.

3. If I'm not mistaken, similar architectures are also used in general object detection. It would be interesting to highlight the difference between general object detection and how the proposed method is different.

4. In the relevant work, it is noted that DETRs don't perform better than two-stage-detectors. Is this because DETR produces a certain number of encoded outputs? It would be great if the authors can comment on it. 
limitations: The limitations of the work are not highlighted clearly.  It would be interesting to see some cases where the proposed method fails.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: Due to the lack of spatially adaptive features, in this paper, the authors propose deformable vision transformer aiming to achieve the balance between speed and accuracy.
strengths_and_weaknesses: 1.	R an HO are not famous terminologies, but they explain these terms in Section 4.1. It hurts the readability.
2.	The parameters in loss function are not well-justified by experiments. The same issue is also for inference stage.
3.	Lack of simulations results in popular detection benchmarks. 
4.	We already have deformable vision transformer in CVPR2022 [A].
[A] Xia, Zhuofan, et al. "Vision transformer with deformable attention." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.
5. The performance is not very impressive.

questions: 1.	In page 4, line 153, what is the original work?
2.	In fig2’s caption, why don’t we need the key tensor?
3.	Can you explain the difference to [1]?
[A] Xia, Zhuofan, et al. "Vision transformer with deformable attention." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.
limitations: They should address their limitation in conclusion or discussion section.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The paper presents a deformable vision transformer for pedestrian detection. The vision transformer introduces multi-scale deformable attention [33] to enhance features for pedestrian classification and localization. The experiments on Caltech and CityPersons demonstrate that the enhanced features by the proposed deformable vision transformer help improve the pedestrian detection performance. 
strengths_and_weaknesses: Strenghts:
1. The introduced multi-scale deformable attention enhances features for pedestrian detection in one-stage pdestrian detector. The results in Tables 5 and 6 show that it brings performance improvements on both Caltech and CityPersons datasets.

2. Overall, the paper is easy to follow. 

Weaknesses:
1. The novelty and contributions of the paper are limited. It simply adopts multi-scale deformable attention [33] to enhance features for pedestrian detection. 
2. The performance of the proposed method is not impressive enough. On CityPersons, its peformance is only on par with state-of-the-art.  

questions: The paper needs careful proofreading. It contains some grammatical errors. For example, Line 3 the latter categorys --> the latter categories or the latter category.
  
limitations: No. See weaknesses.
review_title: null

Meta-review: metareview: This paper improves single-stage pedestrian detectors with deformable vision transformers. 
The paper initially received mixed reviews, *i.e.* two weak accept and two bordeline reject recommendations. The reviewers' main concerns were essentially related to the novelty and contributions of the submission and to the performance's assessment of the proposed approach. The rebuttal provided elements to clarify the scope and contributions of the paper, but R6hn5 still challenges the novelty.  

The AC's owns reading of the submission leads to the following analysis: 
- The AC agrees that the novelty can be challenged, in the sense that the submission uses known components, namely deformable transformers and Center and Scale Prediction (CSP), in the context of pedestrian detection. Although the paper is overall well written, a more coarse-to-fine presentation of the approach, starting from the description of the overall pipeline in Figure 3, and then describing the proposed multi-scale deformable attention (MSDA) and CSP would have help the reading flow in AC's opinion.
- On the other hand, the paper is clearly motivated and the adaptation of both components have not been successfully applied to pedestrian detection yet. The approach shows that specific architecture design, *e.g.* with a smaller number of encoders, is required to make the approach successful. The combination of MSDA and CSP in this context is also meaningful and experimentally validated in ablation studies. Finally, the absolute performances reached by the approach is convincing, by reducing the gap between single-stage approaches and two-step methods, and even outperforming state-of-the-art performances in several cases. 

Therefore, the AC recommends paper acceptance. It highly encourages the authors to take into account reviewers and AC remarks to improve the final paper.
confidence: Certain
Decision: Accept