Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: Heatmap detection-based method and integral regression-based method are widely used in the pose estimation task. The paper gave a detailed comparison and theoretical analysis between these two methods. A unified model is proposed to compare these two methods. Comprehensive experiments are conducted to support the theoretical analysis. 
main_review: ## Strengths

- The paper is well written and organized.
- A theoretical study on how these two pose estimation methods, detection-based and integral regression-based, work is missing in the literature. This work has well studied this. 
- Comprehensive experiments are conducted to support the theoretical analysis.


## Weaknesses
- How to apply the conclusion in this work to further improve the pose estimation is missing. This is not a big issue to me. However, it would be great if the author can give such a guide according to the theoretical analysis. 
summary_of_the_review: In the field of human pose estimation, the heatmap detection-based method and integral regression-based method are widely used. Previous works showed the performance and behavior differences between these two approaches. However, there are rarely works to theoretically study where the differences come from. This work proposed a unified model to analyze these two methods and is well supported by comprehensive experiments. It's great to see such a theoretical study on that. I'd like this paper, and I believe this work can help the research community in the field of human pose estimation to better understand the problem. Though the novelty of the paper is limited, I think we should encourage such efforts on how to theoretically analyze our methods.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper is an in-depth study of the Integral Pose Regression method that is very common for Keypoint Localization. Integral Pose Regression has been adopted by many methods, and it is probably one of the two main paradigms for Keypoint Localization, along with Heatmap Regression. Many works have compared the two approaches, and in some cases (e.g., Gu et al, 2021) they provide a more detailed empirical comparison, identifying the strong and weak points of each approach. This paper has a similar goal, but combines theoretical investigation with empirical results to highlight insights about the two methods.
main_review: - The paper considers a very interesting empirical setting. Keypoint Localization is a very popular task and many approaches rely on this. Getting more insight about each different method can be valuable to many people.

- The paper combines some theoretical exploration along with empirical validation/demonstration in both toy settings, but also real settings. I find very useful this combination of theory/experiments and toy examples/real world settings. For example, the toy experiment shows different behavior than the real world training, so it is important to look at both of them at the same time. On the contrary, the theoretical study is a little basic, but it is enough to demonstrate the insights of the authors.

- The insights and trade offs discussed are very useful for practitioners that use these methods. Some of these observations also exist in previous works, but it is helpful to have papers like that, that collect this type of observations (e.g., trade off for training speed, convergence, potential vanishing gradients, etc)

- I really appreciate that when the authors are making specific assumptions (i.e., about the form of the heatmaps), they also provide the empirical demonstration of the fact.

- In figure 4, the top red window (in third frame of bottom row) is not clear that it corresponds to the zoomed in version of the bottom window. I thought its goal was to indicate activations in locations other than the ankle. It would be good to indicate this to avoid confusion.

- Is there any exploration to be done for cases where these two approaches are combined? E.g., training with heatmap supervision in the beginning to warm up the network, and with integral supervision later to get the best performance for hard examples too?
summary_of_the_review: Overall, I am positive about this paper. Obviously, the novelty is limited, but I believe that its value is high, particularly for practitioners. Since the analysis done here is more basic, I will give it a Weak Accept rating, but I think it is important to accept it to encourage similar efforts.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper investigates the performance inconsistency of integral pose regression (IPR) methods on 'easy' pose samples and 'hard' pose samples (v.s. argmax methods). The authors find that the heatmap shrinkage caused by the IPR methods leads to the lower accuracy on 'easy' samples than argmax based methods. They give some toy examples and intuitively demonstrate how the shrinkage of heatmaps is caused by the gradient of the IPR method. They further propose a heatmap distribution prior loss to mitigate the shrinkage.
main_review: Pros:
1. This paper gives an interesting hypothesis on the cause of performance inconsistency in IPR methods. 
2. Some theoretical evidence and empirical results are provided to help understand and verify this hypothesis. 

Cons:
1. The authors claim that extremely localized heatmaps degrade performance (of IPR methods on easy samples). But this claim is not well supported by empirical results or theory.

    (1) To verify the impact of sigma on IPR methods, the authors introduce a distribution prior to prevent the shrinkage of the heatmap. However, this method may also introduce some other confounding factors. For example, what are the weights of the prior loss term and the integral loss term? What are the magnitudes of the gradients produced by these two loss functions? Would the prior loss term dominate the training process thus Table 1 mainly reflects the impact of sigma on detection methods? These are not revealed. 

   (2) As shown by Fig 2(b), not all samples converge to the extremely localized heatmaps at the end of the training. After introducing an appropriate sigma prior, the overall performance improves as in Table 1. However, the respective impact of sigma on the previously localized heatmaps and previously dispersed heatmaps are not clearly shown. It might be better if the authors would show the change of shrinkage and performance separately on these two kinds of samples.

   (3) Although theoretical evidence is provided to help understand the behavior of gradients of IPR methods, no theoretical evidence is provided to directly support the main claim: extremely localized heatmaps degrade performance.

2. The novelty of the proposed heatmap prior loss is limited. It has already been explored in the initial IPR paper [1]. The authors in [1] combine the heatmap loss with the integral loss and show that this improves over the pure integral loss. What is more, the proposed heatmap prior loss does not substantially improve the gradient behavior of the IPR process. It is more likely that the prior loss dominates the backward update. It might be better if the authors would give more details.

Minor Comments:

1. Abbreviation was used before detailed explanation.  e.g.  Introduction Section :  EPE
2. Typos:
    (1) Subsection 3.3  hte --> the
    (2) Subsection 4.2  E_{de} <= E_{re} -->  E_{de} >= E_{re}
    (3) Subsection 5.1  location factor --> value factor
3. Subsection 5.1: The detailed derivation of Eq10 is not shown in the Supplementary. 

[1] Sun et al. Integral human pose regression. ECCV 2018

summary_of_the_review: Because the main claims are not well supported and the proposed heatmap prior loss lacks novelty, I lean towards reject.
review_title: null

Meta-review: All reviewers recognized the contribution of performing a theoretical study to investigate how the two technique lines (detection-based methods and integral regression-based methods) work for pose estimation. The study in this paper could be valuable for the researchers in the pose estimation domain, for the further research. The AC agrees with the reviewers and recommends accept for this paper.
Decision: Accept (Poster)