Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper:   This paper proposes a large-scale Chinese cross-modal benchmark where one pre-training dataset and five fine-tuning datasets are involved. The new benchmark serves a significant contribution for the community. The paper proposes Global Contrastive Pre-Ranking and Fine-Grained Ranking to explore the matching relationship between image and text pairs. Besides, the paper employ distillation to overcome the problem of noisy labels.
strength_and_weaknesses: Strength: 
1. A new large-scale Chinese cross-modal benchmark which makes great contribution to the community.
2. Different schemes are designed to learn better feature representations and overcome the problems of noisy labels.
3. The paper is well written and easy to read.
Weaknesses:
1. There is no substantive modification of the ranking algorithm. These two losses are not novel somehow compared with previous contrastive loss.
2. The distillation loss is not novel both on the target level and feature level. Some technologies such as " adopt soft targets generated by momentum-updated encoders" are common and there are no essential progresses over published approaches.

clarity,_quality,_novelty_and_reproducibility: Paper is well written overall except some mistakes such as “cross-model” in page 2, etc. The approach is easy to reproduce. Novelty for the technical part is weak.
summary_of_the_review: Taking both the contributions from both technique and dataset, I recommend a borderline.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper presents a method of casting the problem of multi-person pose estimation as a two-level (human and keypoints) Explicit-box Detection problem. It is built on the DETR framework and its variants such as the Deformable DETR and DN-DETR.  The human detection component is a direct application of the existing DETR variant. The proposed human-to-keypoint decoder with the box-based keypoint formulation is the novel component. The proposed method is end-to-end trainable without resorting to post-processing, thanks to the DETR framework. It obtains better accuracy performance than the prior art on the COCO keypoint benchmark and the CrowdPose benchmark.  
strength_and_weaknesses: + The proposed method takes advantage of the fully end-to-end DETR framework for pose estimation in a straightforward yet effective way.
+ The proposed human-to-keypoint decoder  with the box-based keypoint formulation seems to be effective. 
+ The proposed method obtains strong performance on two challenging datasets. 

- Some details of the box-based keypoint formulation are missing. How are those bounding boxes determined for different keypoints of different persons in training to compute the keypoint box regression loss, L_k^l? Is the same method used in both the COCO and CrowdPose datasets? If not, how is the dataset prior (e.g., crowd vs not-so crowd) leveraged?  And, how sensitive is the proposed method to the keypoint boxes?

- The interactive learning part is not clearly presented. How is the human-to-human interaction computed, as well as the human-keypoint and keypoint-keypoint? 

- Two references that seem to be relevant are not discussed.  1) N. Xue et al Learning Local-Global Contextual Adaptation for Multi-Person Pose Estimation, CVPR22; 2) D. Wang and S. Zhang, Contextual Instance Decoupling for Robust Multi-Person Pose Estimation, CVPR22. 

- It may be interesting to test the COCO or CrowdPose trained model on the OCHuman dataset to see the transferability as done in 1) N. Xue's paper. It will also enable better understanding the impacts of the selection of the keypoint boxes. 

clarity,_quality,_novelty_and_reproducibility: Overall, the paper is well written and easy to follow.  The advantage of the proposed method is well shown. The experiments are thorough.  The code is provided (but not tested by the reviewer).  
summary_of_the_review: Overall, this is a good paper on multi-person pose estimation. It leverages the strong DETR framework with a new human-to-keypoint component.  The reviewer would like to see the rebuttal on the aforementioned weaknesses.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In summary, to deal with the task of multi-person pose estimation, this paper regards the task as two explicit box detection processes. This enables the model to be optimized in an end-to-end manner and achieves good result.
strength_and_weaknesses: Strength:
1. The paper gives a clear summary and analysis of existing methods, which motivates their method well.
2. Many experiments have been conducted and the results seem satisfactory.

Weakness:
1. My main concern with this paper is the proposed explicit box detection process. Specifically, I think a clearer definition of what is the so-called explicit box detection process should be given more precisely. In the current version, it seems that I can only guess what it means, which brings me reading difficulties. From my guess, does this explicit box detection process just mean to explicitly detect bounding boxes for both the human and the keypoints? I personally use this understanding to understand this paper and I am sorry if I misunderstand it. 
2. Then with such an understanding, I think the advantage of such a process should be mentioned in more detail. Only demonstrating the advantage of such a process through experiments seems to be not enough to me. Besides, I am not sure if this process has improved the results because it really helps with pose estimation, or because it is more relevant to the task of object detection and therefore fits DETR and other existing object detection methods better. Can the authors also explain this?
3. I also have some other minor worries w.r.t. the experiments. (1) I am not sure whether the authors overclaim their improvement a bit as it seems a large proportion of their improvement on Swin-L (1.5 -> 5.0 on crowdpose and 1.2 -> 2.7 on coco) is because they use an extra dataset Objects365. (2) I am not sure if it is allowed to put the result on COCO-test dev in the appendix only, as I am not sure whether this should be considered as a new dataset or not.
clarity,_quality,_novelty_and_reproducibility: I think the reproducibility of this paper is quite good as the authors provide their code. For the others, I still have some concern on the clarity and the quality (see the Strength And Weaknesses for more details).
summary_of_the_review: Overall, my main concern w.r.t. this submission is on the proposed explicit box detection process (see the Strength And Weaknesses for more details). Specifically, I think there are some things that should be made clearer before the acceptance of this submission.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a method for multi-person pose estimation method, which integrates human detection and human pose estimation into an end-to-end framework. The first part of this framework is a human detector based on deformable DETR and DAB-DETR, and outputs human-instance features and estimated human bounding boxes.  The second part, i.e. keypoint detector, is fed with a human-instance query and a set of keypoint queries which are initialized with its corresponding human-instance query. By doing so, the queries of keypoints belonging to one human instance can interact with each other while each keypoint query can also interact with its human-instance query. The keypoint locations are regressed by the keypoint detector. Experiments are performed on multiple datasets. the proposed method significantly outperforms one-stage SOTAs. When compared with two-stage methods, the proposed method performs better than bottom-up SOTAs, while performs comparably or worse than top-down SOTAs but with better efficiency and more succinct pipeline.  Besides, the ablation study also verifies the effectiveness of some proposed algorithm components.
strength_and_weaknesses: Strength:  
1. Paper is written clearly.
2. Global and local information are  reasonably combined by query initialization and human-to-keypoint query expansion.
3. Human detection and keypoint detection are integrated into an end-to-end and succinct framework. each part of this framework can benefit each other during training.
4. The proposed unified framework achieves promising accuracy and good efficiency.

Weaknesses:
The whole model architecture can be considered as the variant of  DETR
clarity,_quality,_novelty_and_reproducibility: This paper is written clearly. Novelty is good.
summary_of_the_review: The proposed one-stage end-to-end framework for multi-person pose estimation is succinct and efficient. The accuracy is also promising. The effective combinations of human detection and keypoint detection, global and local information are a novel extension of existing DETR architectures.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper presents an end-to-end approach for multi-person pose estimation. It unifies human detection and keypoint estimation in an one-stage framework. Based on transformer, human tokens and keypoint tokens are introduced and interacted to generate discriminate representations for human detection and keypoint estimation. The proposed aporoach achives state-of-the-art performance on COCO and CrowdPose datasets. Extensive experiments demonstrate the effectiveness of the the components in the proposed approach. 
strength_and_weaknesses: Strengths:
1. The proposed approach introduces human tokens to interact with keypoint tokens to resolve the ambiguous association between keypoints and humans in the transformer framework.The approach is reasonable and novel. Explicit human detection contributes significant performance improvement in the proposed framework. The interaction between human and keypoint tokens further improves the performance. 

2. The representation of (x, y, w, h) instead of (x, y) for keypoints provides more contextual information for keypoint estimation as demonstrated in Table 5.

3. The proposed approach achieves state-of-the-art performance on COCO and CrowdPose datasets with competing inference speed as shown in Tables 2 and 3. The proposed approach also has faster convergence speed than competing methods as shown in Figure 5.  

Weaknesses:
Some implementation details are missing: 
(1) The detailed losses are not presented in neither the paper nor the supplementary material. 
(2) Some details of the network structures are not provided (e.g. how many human detection decoders and human-to-keypoint detection decoders are used?)
 
clarity,_quality,_novelty_and_reproducibility: The paper is well writen and easy to follow. The proposed mehtod is novel and effective.
summary_of_the_review: Overall, the paper is well written. The novelty and contributions of the paper are sufficient.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper obtains 4 pieces of valid review reports, and the 5th one is invalid (comments are irrelevant to this paper). All the four reviewers are quite positive on this paper. The paper treats the task of multi-person pose estimation as two explicit box detection processes, which enables end-to-end training and achieves good pose estimation result. The method utilizes the DETR framework for pose estimation, and designed a human-to-keypoint decoder with the box-based key-point formulation, which seems to be interesting and effective. Authors have already revised some parts of the paper during rebuttal. Meanwhile, as pointed out by reviewers, more details need to be added to the paper and appendix. Thus, authors need to carefully incorporate the details and other comments into the camera ready version.
justification_for_why_not_higher_score: While the model seems to be interesting and quite effective, it is a variant of DETR for the multi-person pose task. Thus, AC recommends poster presentation.
justification_for_why_not_lower_score: Though DETR part is an application, the human-to-keypoint decoder with the box-based keypoint formulation is a novel part.
Decision: Accept: poster