Dive Deeper Into Integral Pose Regression
Authors: Kerui Gu
Linlin Yang
Angela Yao
Venue: ICLR 2022 Poster (2022)

Abstract: Integral pose regression combines an implicit heatmap with end-to-end training for human body and hand pose estimation. Unlike detection-based heatmap methods, which decode final joint positions from the heatmap with a non-differentiable argmax operation, integral regression methods apply a differentiable expectation operation. This paper offers a deep dive into the inference and back-propagation of integral pose regression to better understand the differences in performance and training compared to detection-based methods. For inference, we give theoretical support as to why expectation should always be better than the argmax operation, i.e. integral regression should always outperform detection.  Yet, in practice, this is observed only in hard cases because the heatmap activation for regression shrinks in easy cases. We then experimentally show that activation shrinkage is one of the leading causes for integral regression's inferior performance.  For back-propagation, we theoretically and empirically analyze the gradients to explain the slow training speed of integral regression.  Based on these findings, we incorporate the supervision of a spatial prior to speed up training and improve performance.

Published as a conference paper at ICLR 2022
DIVE DEEPER INTO INTEGRAL POSE REGRESSION
Kerui Gu', Linlin Yang 1. & Angela Yao'
National University of Singapore, Singapore
University of Bonn, Germany kkeruigu, yangll, ayao)0come nus. edu. sg
ABSTRACT
Integral pose regression combines implicit heatmap with end-to-end training for human body and hand pose estimation. Unlike detection- heatmap meth- ods, which decode final joint positions from the heatmap with non-differentiable argmax operation, integral regression methods apply a differentiable expectation operation. This paper offers a deep dive into the inference and back- pooppproaggttion of integral pose regression to better understand the differences in performance and training compared detection- based methods. For inference, we give theoretical support as to why expectation should always be better than the argmax opera- tion, i.e. integral regression should always outperform detection. Yet, in practice, this is observed only in hard cases because the heatmap activation for regression shrinks in easy cases. We then experimentally show that activation shrinkage is one of the leading causes for integral regression's 's inferior performance. For back- propagation, we theoretically and empirically analyze the gradients to explain the slow training speed of integral regression. Based on these findings, incorporate the supervision of spatial prior to speed up training and improve performance.
INTRODUCTION
2D human pose estimation aims to detect the image coordinates of the body and/or the hand. In recent years, detection-based methods (Newell et al., 2016; Xiao et al., 2018; Sun et al., 2019; Li et al., 2019) and integral pose regression methods (Sun al., 2018; Iqbal et al., 2018) have emerged as two common paradigms for human pose estimation. Both methods learn a heatmap representing pixel-wise likelihoods of the joint positions. The heatmap is learned explicitly for detection- methods but remains implicit or "latent" for integral regression methods. To decode the heatmaps to joint coordinates, detection methods use an argmax, while integral regression methods take the expected value. As the expectation operation differentiable, integral regression has the benefit of being end-to-end learnable, even though detection methods seem more competitive accuracy- wise (COC, 2020).
Gu et al. (2021)'s recent work showed a curious performance difference between integral regression and detection methods. With a nuanced split over the evaluation set, integral regression outperforms detection when test samples are "harder", i.e. with fewer keypoints present in the scene, under higher occlusion and lower bounding box resolutions. Given that both detection and integral regression methods work with fully convolutional feed-forward architectures, the questions naturally arise: What are the reasons behind this performance difference? Why is integral regression able excel in these hard cases, when its overall performance seems to lag behind detection methods (COC, 2020)? These questions serve as the motivation for our closer study and analysis of integral pose regression.
Detection and integral regression methods differ in two aspects in both the forward and backward pass (see Fig. for an overview). In the forward pass, the heatmap decoded with an argmax for detection whereas a softmax normalization and expectation are used for integral regression. In the backward pass, detection methods are supervised with an explicitly defined Gaussian heatmap centered on the ground truth joint, while integral regression is supervised by the joint coordinates directly. Through detailed theoretical analysis and experimentation on the decoding and back- propagation process, make the following findings and contributions:

---

Published as a conference paper at ICLR 2022
1. We propose unified model of the heatmap to interpret and compare detection and integral regression methods. We verify the model experimentally and show that as samples shift from hard to easy, the activation region on the heatmap shrinks for both detection and integral regression methods.
2. We demonstrate experimentally that degenerately small regions of activation degrade the accuracy of both detection and integral regression methods.
3. Integral regression methods, which decode the heatmap with an expectation operation, should result in lower expected end-point error than detection methods that decode with an argmax operation. In practice, this can be observed only for hard samples due to the shrinkage of the active region on the heatmap.
4. Direct supervision with the joint coordinates in integral regression, although end-to-end, suffers from gradient vanishing and provides less spatial cues for learning the heatmap than the explicit heatmap supervision of detection methods. As result, the training of integral regression is more inefficient and slower to converge than detection methods.
Our findings provide insight into integral pose regression, which has better theoretical performance, and show that the density of heatmaps plays an important role in decoding heatmaps to coordinates.
2 RELATED WORK
Since the concept of heatmaps proposed in (Tompson et al., 2014), detection-based methods have been top performers in human pose estimation. Existing detection- baacct works emphasize ex- tracting high-quality multi-resolution features. Particularly, Xiao et al. (2018) proposed to adapt ResNet with deconvolution layers while Hourglass (Newell et al., 2016) and Cascaded Pyramid Network (CPN) (Li al.. 2019) introduced cascaded network architectures with a coarse-to-fine design paradigm. High-Resolution Network (HRNet) (Sun et al., 2019) follows the coarse-to-fing paradigm and further improves performance by adopting more dense connections across different resolution representations. The integral regression method of Sun et al. (2018) 8) introduced compet- itive regression-bel based framework but is surpassed by more advanced detection based works (Xiao et al., 2018; Sun et al., 2019).
Numerical regression- methods, which directly regress joint coordinates, are commonly used in facial landmark detection (Feng et al., 2018; Zhu et al., 2020). These methods, however, are not as accurate as detection-baan -baee methods on human pose estimation. integral regression methods attempt to implicitly merge the spatial knowledge from the heatmap via a "latent" heatmap. Integral regression methods are especially preferred in hand pose estima- tion (Spurr et al., 2020; Yang et al., 2021) but still less common for human pose estimation (Sun et al., 2018; Nibali et al., 2018).
order to improve the accuracy,
Recently, two parallel lines of work have emerged analyse detection and integral regression meth- ods separately. For detection- based methods, Huang et al. (2020) identified a heatmap bias caused by the inconsistency the coordinate system transformations and solved it by redesigning the trans- formations during data processing. Additionally, as the predicted heatmaps during inference may violate the Gaussian assumption and worsen the performance, Huang al. (2020) proposed find the optimal Gaussian location based on the prediction while Zhang et al. (2020) modulated the predicted Gaussian distribution for decoding.
For integral regression methods, Nibali et al. (2018) experimentally compared different heatmap regularizers, heatmap normalization schemes and loss functions. They found that using Jensen- Shannon regularization on the heatmap with softmax normalization and L1 loss achieves the best performance for integral regression. More recently, Gu et al. (2021) performed systematic com- parison of detection and integral regression methods with a common backbone and discovered the performance advantage of integral regression on the "hard" samples. Furthermore, Gu et al. (2021) revealed a bias that arises from taking the expectation after the softmax normalization. As such, they proposed a compensation scheme to mitigate the bias, which in turn improves the overall per- formance of integral regression methods, making them competitive with detection methods.
2

---

Published as a conference paper at ICLR 2022
Encoding
IPR-based Decoding
Narman wommaallio aosattiooo
Fribebiition
Expestation
Coordinate
Detection-based Decoding
Ground truth
Liollhood
Mac
Coordinato
Argman
= VuJyy)
Heatmap ground Irth
Coordratt
UV Coorcinate
pasta
backwards paso
suppr aicn
anritattion
Comparison of the two decoding processes for detection-based methods (pink panel) and integral regression methods (green panel). They share the encoding (yellow panel) but have different representa- tions of the ground truth (gray panel).
Figure 1:
3 PRELIMINARIES ON HUMAN POSE ESTIMATION
In this work, we target the more commonly used 'top-down' form of pose estimation in which person detector already provides cropped image T of the person. For simplicity, focus discussion on one given joint out of the K total joints in the body. The pose estimation model outputs a heatmap H IR MxN where M and N are the dimensions of the spatial heatmap. Typically, M and N are scaled down by factor of 4 from the original input dimensions of I (Xiao et al., 2018; Sun al., 2019). The heatmap H represents (discrete) spatial likelihood P(J[T), where R 1x2 the 2D coordinates of the joint. In practice, all K heatmaps are predicted simultaneously by the same network, where each joint one channel. both detection and integral regression methods, the coordinates are decoded from H. where the two methods differ in the manner of decoding (see Sec. 3.1) and the form of supervision applied (see Sec. 3.2).
3.1 HEATMAP DECODING: MAX VERSUS EXPECTED VALUE
Defection methods apply an argmax on H indexed by to estimate the joint coordinates Jae:
Ja arg max H(p).
Taking argmax be interpreted as taking maximum likelihood the heatmap H. assuming that H is proportional to the likelihood. In practice, the final Jac value is determined as linear combination of the highest and second-highest response H as way account for quantization effects in the discrete heatmap (Newell et al., 2016). A more recent work, DARK (Zhang et al., 2020), approximates the true prediction by Taylor series evaluated the maximum activation of the heatmap and shows this to be more accurate.
Integral pose regression applies an expectation operation on H to give mean estimate of the joint coordinates. To do so, the heatmap must first be normalized to sum up to 1; the most common and effective approach (Nibali et al., 2018) is to apply softmax normalization. Afterwards, the predicted joint J,c with and y components J, and J, is determined by taking the expectation on the normalized heatmap H with elements h location p:
J J,
@h
J.c
Σp peB
h hp
where
Σ @he
(p))n
Here, is the domain of the heatmap and 33 is scaling parameter used in the softmax normalization. Note that softmax normalization assigns non-zero value to all pixels in H, even if it was originally
zero in H. These values also contribute to the expected value, resulting in center-biased estimated
For clarity, drop the subscript 're', refer to the individual components for integral regression only.

---

Published as a conference paper at ICLR 2022
joint coordinate Jee (Gu al., 2021). The smaller the 3, the stronger the bias. Although Gu et al. (2021) proposed a compensation scheme, for the purpose of our analysis, we will assume that / is sufficiently large such that the impact of the bias is negligible.
3.2 SUPERVISION: EXPLICIT HEATMAP VERSUS GROUND TRUTH COORDINATES
Detection methods are learned by providing supervision on the heatmap. The ground truth H is given circular Gaussian, with mean centered at the ground truth joint coordinate (see Fig. 1). The loss applied a pixel-wise MSE between the ground truth hp and the predicted hp:
de
H H
Σ0 (1p hp)²
PEΩ2
where Q is the domain of the heatmap. As the loss is defined in terms of the heatmap and not in terms of the predicted joint coordinates, detection-based methods are not end-to-end in their learning, and this is often cited drawback (Sun al., 2018; Zhang al., 2020).
Integral pose regression defines a loss based on the difference between the prediction J,c and the ground truth joint location J. The LI loss empirically performs better than L2 (Sun et al., 2018):
L = | Jeel1 = (IJ= Jzl + | - Jyl).
(4)
Integral regression methods are end-to-end because they provide supervision at the joint level. As H is learned only implicitly, some works refer to the heatmap as "latent" (Iqbal et al., 2018).
3.3 PERFORMANCE DIFFERENCES
Sun al. (2018) and Gu al. (2021) compared the performance of detection and integral regression methods on human pose estimation. Sun et al. (2018) found that integral regression is either as competitive better for 2D pose estimation, though results are not fully conclusive, as the backbone they used for integral regression is different from detection.
The more systematic and detailed comparison from Gu al. (2021) used a common backbone. also partitioned the data based on three factors of variation dominating current human pose estima- tion benchmarks (Ruggero Ronchi Perona, 2017): the number of joints present the scene, the percentage of occlusion and the largest dimension of the bounding box input. The "hard" samples are those with either 1-5 joints present, > 50% occlusion, or 32-64px input. For these samples, integral regression had, average, 15% lower end-point error (EPE) than detection. The "easy" samples are defined by 11-17 joints, <10% occlusion, or > ~22pp input. In these cases, detection methods were marginally better. The remaining "medium"" ; performance of both methods was roughly equivalent on these samples.
with factors in between
considered to be
ANALYSIS ON HEATMAP DECODING
4.1 LOCALIZED HEATMAP MODEL
To analyze the performance differences between detection and integral regression methods, we make the following assumptions on the heatmap itself. First and foremost, we assume that well-trained network will produce heatmap hp with significant or large values in a localized support region around J. We denote the region of support as and assume that outside of Φb the heatmap activation approximately zero (see Fig. 2 (a)).
Specifically, the normalized heatmap² H is modelled by some density distribution P. We make assumption on the form P, but assume that it fully captures the support for the joint location within & and that $ is centered on the expected value of P on location μ (μzz Myy with area of (2s + 1, 2s + 1), i.e.
11
Σ p h PEO
Σ p h (E
Σ
hij
(5)
imjt
For the purpose of discussion, may consider normalizing the heatmap for detection methods per Eq. 2, which will not affect the outcome since arg max (H) arg max(H).
4

---

Published as a conference paper at ICLR 2022
5..9
84
0.8
0.7
86
64
Regression
Detection
Regress on Detection
61
10
12
34
emy
meduu
hard
0.5 15 2.0 3.0 4.
(a) Illustration of +
(b) Activation sum of H
(c) Optimal a of predicted
(d) AP versus ground truth
heatmaps
Figure 2: (a) The heatmap is only activated within d; (b) more heatmap activations are bounded by a (2s + 1) x (2s 1) region for integral regression than detection; (c) if the heatmap is approximated as Gaussian, the resulting shrinks as samples shift from hard to easy; this shrinkage is faster and more extreme for integral regression compared detection; in detect methods, the optimal is around 2; smaller and larger hurt performance.
Experimental Verification of Localized Heatmap Model The assumption of localized activa- tions is reasonable for detection methods as their heatmaps are learned explicitly to match (local- ized) ground truth Gaussian. We empirically verify this model by tallying the activations in the nor- malized heatmap for both detection and integral regression methods based the casy/mecium/hard splits as specified in Sec. 3.3.
Specifically, we train both detection and an integral regression network for human pose estima- tion. For both networks, we use a Simple Baseline (SBL) (Xiao et al., 2018) architecture with ResNet50 (He et al., 2016) backbone. The two networks differ only in their manner of decoding and form of supervision as outlined in Section 3. We then apply the networks on the standard hu- man pose estimation benchmark MSCOCO (Lin al., 2014) using the same easy, medium and hard splits as Gu et al. (2021). For each produced heatmap of 64x48, we estimate an activation sum A by summing the normalized heatmap activations within a (2s 1) (2s 1) square around the ground truth coordinate, (Jz, Jy):
J, Σ
A(s)
Σ
heg-
-
From Fig. 2 (b), we show that with sufficiently large s, e.g. 9, the activation sum exceeds 95%.
4.2 EXTREME LOCALIZATION A CAUSE FOR PERFORMANCE DIFFERENCES
Expected End-Point Error (EPE) For detection methods, which decode the heatmap with an argmax, assume that each position p has some probability w(H,, s) of being the maximum acti- vation, i.e. arg max(P) ((,ss) w(H, s). Here, w(H, s) represents a radially symmetric distribution centered on μ with all non-zero support contained within 4. It follows that the expected EPE for one sample of a detection method can be defined as:
(a,( 5) = Σ
w(D) -
pE(((pt)))
where Jgr denotes the ground truth coordinates. For integral regression methods, which decode the heatmap with an expectation, the estimated joint coordinate aligns with the center of & by definition, i.e. J = μ. leading to the following expected EPE for single sample:
E = || 14/12-
can be verified that the Ea.: 2 Er (see proof in Appendix A.2). This result suggests that for some fixed (μ,, s), the expected EPE of regression should always be better than detection. However, the re- sults Gu et al. (2021) as summarized in Sec. 3.3 have clearly shown otherwise. This inconsistency can be better understood with the following two experiments.
Experimental Verification: Integral Regression Methods Are "More" Localized than Detec- tion The plot in Fig. (b) indicates that regression methods to be more localized than detec- tion methods. In particular, for smaller s, A is smaller for detection methods than integral regression methods.
5

---

Published as a conference paper at ICLR 2022
Model
Delection
Regression
+ +0.50.5
to
to ta=2
+o 33
to
AP
71.3
67.1
65.1
68.5
71.4
69.8
66.9
AP e/m
74.3
70.4
68.2
71.9
74.1
73.0
70.2
APa
43.2
44.7
43.9
45.2
47.4
47.6
46.9
A(s
2)
0,49
0.65
0.75
0.63
0.54
0.49
0.45
A(s = 2)a.
0.30
0.35
0.37
0.36
0.41
0.39
0.39
Table 1: Performance and activation sum of detection baseline, regression baseline, and regression baseline added KI.iv loss with different rss When = 0.5, 1, the results are not improved but even worsened. When = 2, the spread of heatmaps smoothed and the results even exceed detection haselines.
To further verify, we compare the heatmaps with idealized Gaussian heatmap of varying stan- dard deviations or a. We plot the a that gives the lowest Pearson Chi-square statistic, Lee. the highest similarity for the different levels of difficulty in Fig. 2 (c) (detailed values provided in Appendix A.3). As expected, the optimal a decreases as the samples progress from hard to easy; this result is in line with Fig. (b) and shows that heatmaps are more localized for easy samples. However, we also observe that the optimal a for intgral regression is much smaller for the medium and easy cases. We speculate that this difference arises because detection methods are trained with a Gaussian heatmap of a = 2 (Xiao et al., 2018; Sun et al., 2019). Integral regression methods, on the other hand, have no such explicit supervision, and therefore no restriction on the extent of localization.
Experimental Verification: Extremely Localized Heatmaps Degrade Performance Is it possi- ble that smaller hypothetical a, i.e. more localized heatmaps, causes poor performance? We verify this for detection methods by applying ground truth Gaussian heatmaps of varying for training the networks as per Sec. 4.1 and evaluate network performance using Average Precision (AP). A higher AP corresponds to more accurately located joints. Fig. 2 (d) confirms that 2 is optimal. smaller ø = 1 degrades the performance slightly, and there is a significant drop in the extreme case when 0.5.
To verify the impact of a on regression methods, we add a distribution prior to the heatmap to prevent apply a Kullback-Leibler Divergence (KLDiv) loss between the non-normalized heatmap H and Gaussian heatmap with varying a. From Table we see that the regression baseline is worse than detection baseline. However, when we add prior on the heatmap distribution to encourage a sufficiently large 4, i.e. (+o 1, 3), performance improves. At the optimal +o = 2, integral regression becomes competitive with and even exceeds the detection baseline with the optimal a. Most importantly, extreme a values cause performance to degenerate, especially in the case of the extremely small +o==-=-0- 0.5.
the shrink of the support region +. Specifically,,
We also demonstrate the performance and localization extent of easy/medium and hard cases by AP and activation sum A(s = 2). After adding priors (e.g. +o 2), A(s 2) decreases from 0.65 to 0.54, indicating the heatmaps are less localized, and the performance AP, is significantly improved. We provide the detailed training information regarding different loss weights for better understanding in Appendix A.4 and detailed changes in heatmap shrinkage in Appendix A.5.
ANALYSIS OF SUPERVISION AND LEARNING
Given the same heatmap, different methods of heatmap decoding not only affect the final coordi- nates but also determine the gradients that arise from the loss. The gradients in turn influence the learning and thereby change the generation of the heatmaps. In this section, explore the gradi- ents of heatmaps with respect to the loss functions for detection and integral regression methods and pinpoint the specific gradient components that slow down the learning of integral regression.

---

Published as a conference paper at ICLR 2022
5.1 HEATMAP GRADIENTS
Detection Methods The gradient of Lae (see Eq. 3) with respect to each pixel in the estimated
heatmap hp is straightforward:
aLde = 2(hp hp). ahp
The gradient in Eq. features the predicted heatmap value hp and the ground truth heatmap value hp- It provides explicit supervision at every pixel. False positives and false negatives are penalized by reducing wrong high likelihoods and raising incorrect low likelihoods, respectively. For each position p in the heatmap, if the predicted hi is smaller than the ground truth lp- the gradient negative and proportionally increases the value in the next iteration. The vice versa is true for the predicted hp larger than the ground truth.
Integral Regression The gradient of the loss Li (see Eq. 4) with respect to hp can be estimated based on the chain rule as (see detailed derivation in Appendix A.6)
V = aLi aLe aî Ohp Shi ai Shp Ohi
B hip (s(Je - (i - J.) + sJJ, Jy)(J jy))-
(10)
v, (valuc factor)
22 (location factor)
In Eq. 10, the gradient is split into two terms of interest: V, and V2, which we name as the value factor and the location factor, respectively. The nature of the value and location factors gives strong indication to the learning process for integral regression methods. Firstly, the gradient Vp is propor- tional to the normalized predicted heatmap value hp as given in the value factor. This factor makes it prone to gradient vanishing wherever locations p have small heatmap predictions. Secondly, the location factor is linear combination of i and j, and it manifests itself linear plane; far away points like corners are more likely to have large magnitudes. Though these two factors do not pre- vent the loss from decreasing during learning, they slow down training. We further elaborate this in Sec. 5.2.
5.2 A DETAILED ANALYSIS OF THE INTEGRAL REGRESSION HEATMAP
We now conduct detailed analysis of the gradients presented in Eq. 10 and show the typical cases that arise during learning. When referring to Ω the domain of the heatmap. we use a coordinate system with the origin at the upper-left corner and increasing coordinates moving the lower right corner (see Fig. 2 (a)). For simplicity of discussion, we assume that the ground truth joint Jgi is located somewhere in the lower-right quadrant of Ωt, though the analysis holds for Jg: in all other quadrants of Ω analogously. The four scenarios are visualized in Fig. 3.
We base our analysis on the assumption that a larger magnitude of Vp has a larger influence on the weights of the network and causes greater change the location the next iteration, i.e.
hp ha+1
hp VV.
(11)
where y represents the step size or learning rate (see sketch of proof in Appendix A.7) and n denotes the update iteration. As the value and location factors have different contributions on the gradient from Eq. 10, and they are in turn affected by the size and location μ of Φ can define four characteristic cases of & and outline how learning is affected in each case.
(1) s is large, + has uniformly random values, e.g. randomly initialized heatmap the start of the training. As Ao is in similar range for all p. the value factor V1 (see Eq. 10) of all pixels are approximately similar. As such, the distinction between one pixel's gradient and that of another is determined by the location factor V2. Given that the ground truth coordinates (Jz, Jy) are in the lower-right quadrant, as long as (Ja: Jy) are both greater than the predicted coordinate (Jz, Jy), Vp has gradation that increases progressively towards the bottom right corner. Generally speaking, the location factor pushes the activations of the heatmap towards the corner of the correct quadrant corners own the largest gradients in the resultant linear plane of the location factor Vz-
(2) s is small, μ is far from the ground truth J. e.g. when j is in the upper left quadrant of Ω while the ground truth J ge is at the bottom right. A typical scenario is if the heatmap activates around the
7

---

Published as a conference paper at ICLR 2022
Case
Case 2
Case
Case 4
Regression Update
Detection Update
Figure 3: Toy example of regression update (left) and detection update (right). The black triangle denotes the prediction (expectation value regression and max value for detection) while the red dot denotes ground truth position. The arrow shows the movement the prediction.
left ankle for the ground truth right ankle. In this case, for the pixels outside of o, the value factor approaches zero, i.e. hp - 0, and pushes Vp towards zero, hence the pixels receive very limited gradient updates. For pixels in @ the values decrease gradually until the value factor outside of h no longer dominates, Le. they reach the same scale, which point the heatmap returns to case (1).
(3) s is small, μ is in the corner of the same quadrant as the ground truth Jgt. Initially, this case similar case (2). In the process of all the elements becoming the same scale, all the activations move towards the diagonal side and the prediction approaches the ground truth coordinate.
(4) is small, f close to ground truth Jgt. This scenario occurs when the model is reasonably trained and can roughly localize the joint. The gradient at the ground truth pixel Jgr is always negative zero, i.e.
= -hh.. J 11, Jyll).
(12)
This non-positive value guides the network predict large heatmap response the ground truth location, i.e. a large h.J,...,); this in turn increases the gradient (JJ,.,,) This property makes the network more likely predict a few exceptionally large pixels, thus shrinking the support region + and leading a very small a in Sec. 4.1.
For baased methods, the ground truth heatmaps have already explicitly pointed out whether or not a pixel should be activated in the heatmap. Combined with its effective pixel-wise loss, the network learns faster than regression-basee methods; this is validated in the next section.
5.3 EXPERIMENTAL VERIFICATION
Idcalized Sample We start by considering the case of one sample for single joint and visualize the progression of (64, 48) heatmap it gets updated by Eq. 11. While V can be calculated mathematically by Eq. 10, we use autograd in Pytorch to obtain the gradients, which we verify to be equivalent. The four cases from Sec. 5.2 are initialized randomly, with a symmetric Gaussian (a = 2) centered at some upper left point, with a linear plane at the lower right quadrant, and symmetric Gaussian (a=2) centered ground truth, respectively. The progression of the differently initialized heatmaps can be observed in Fig. 3.
Mathematically, it clear from Eq. 2 that the resulting joint coordinate location from integral regres- sion can align with the ground truth joint even if the underlying heatmap does not follow localized heatmap model. This also illustrated the first three cases discussed in Sec. 5.2. However, from the thousands of training samples observed over many epochs, posit that during real-world training, the network can only consistently lower the loss if the heatmap Il is learned to represent P(J[Z). .) As such, the activation region & will be correctly localized over the corresponding seman- tie region in the image. Instead, what slows down convergence is the learning of ④ rather, the lack of direct guidance to yield a correct as per detection methods. For the same four cases, we visualize the updated heatmap of Eq. 11 from the gradients defined by the detection loss in Eq. 9.
8

---

Published as a conference paper at ICLR 2022
Left eye
Right ankle
Epoch
detection
detection
Left eye
Right ankle
Detection
Epoch
Regression
20
40
80
100
segremsion
togrosion
Epoch
(a) Comparisons of detection and regression in real
(b) Training Efficiency of Detection versus Regression on COCO validation set
training
Figure 4: (a) Comparisons of detection and regression in real training on an image selected from the COCO validation set at epoch and 10. Results of detection have already roughly stabilized while regression can only localize small extent in epoch 10 with some background pixels still activated. We enlarge the right ankle part of regression in the red hox for better visualization. More examples shown in Appendix A.8. (b) The regression method converges slower
detection method.
Unlike regression, each case leads consistently the same localized + centered on the ground truth joint coordinate, regardless of the initialization.
Real World Samples The four cases listed in Sec. 5.2 are not so clearly observable in real-world training as the output is based on the optimization results batch-wise training in a real network. To verify the results, compare the training progression for a detection and an integral regression network under the experimental settings described Sec. 4.1. Fig. (b) compares the training speed of both detection and integral regression and shows the slower convergence of integral regression.
We visualize for one sample from the MSCOCO validation set the heatmap at epochs and 10 to compare the training progression for detection and integral regression. Fig. 4 (a) shows the heatmaps of the "left eye" and "right ankle" joints. For regression, after one epoch, the activations are spread over quarter to half of the heatmap. This aligns roughly mix of Case and where activations are in the correct quadrant but not yet localized to small region of support. After 10 epochs of training, predictions for the left eye (which is actually occluded, making it "hard" sample) are localized with several activations of approximately the same value (all shown as red). For the right ankle (which is clearly present and is an "easy" sample), the network predicts heatmap values that have collapsed spatially and dominate at one or two pixel locations.
In contrast, the detection method presents a well-localized heatmap after only one epoch training. While there is some activation on the left ankle in epoch 1, this is unlikely to cause enrors for the argmax decoding. After 10 epochs, all activations are centered around the correct right ankle.
CONCLUSION
In this paper, we dive deeper into integral pose regression by comparing the differences in the heatmap activations and the gradients between detection and integral regression methods. Our theo- retical analysis paired with empirical verification shows that the shrinkage of heatmaps of the inte- gral regression method is the main cause for its lower performance compared to detection. Further- more, we investigate the gradients of integral regression by giving theoretical evidence, toy problem and real world illustrations to show that the learning of integral regression is slower to converge as result of only implicit supervisory signals. To mitigate the slow convergence and poor performance, we propose a simple spatial supervision on top of the existing framework.
This research project is supported by the Ministry of Education, Singapore, under its MOE Academic Research Fund Tier 2 (STEM RIE2025 M0F-122P2220--011,
Acknowledgements

---

Published as a conference paper at ICLR 2022
REFERENCES
COCO Leader Board.
Zhen-Hua Feng, Josef Kittler, Muhammad Awais, Patrik Huber, and Xiao-Jun Wu. Wing loss for robust facial landmark localisation with convolutional neural networks. CVPR, pp. 2235-2245, 2018.
Kerui Gu, Linlin Yang, and Angela Yao. Removing the bias of integral pose regression. In ICCV, pp. 11067-11076, 2021.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, pp. 770-778, 2016.
Junjie Huang, Zheng Zhu, Feng Guo, and Guan Huang. The devil in the details: Delving into unbiased data processing for human pose estimation. In CVPR, pp. 5700-5709, 2020.
Umar lqbal, Pavio Molchanov, Thomas Breuel Juergen Gall, and Jan Kautz. Hand pose estimation via latent 2.5 d heatmap regression. In ECCV, pp. 118-134, 2018.
Wenbo Li, Zhicheng Wang, Binyi Yin, Qixiang Peng. Yuming Du, Tianzi Xiao, Gang Yu, Hongtao Lu, Yichen Wei, and Jian Sun. Rethinking on multi-stage networks for human pose estimation. arXiv preprint arXiv: / 1901.0044, 2019.
Tsung Yi Lin, Michael Maire, Serge Belongie, James Hays, Pictro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV, pp. 740-755. Springer, 2014.
Alejandro Newell, Kaiyu Yang, and Jia Deng. Stacked hourglass networks for human pose estima- tion. In ECCV, pp. 483-499. Springer, 2016.
Aiden Nibali, Zhen He, Stuart Morgan, and Luke Prendergast. Numerical coordinate regression with convolutional neural networks. arXiv preprint arXiv: (.X1vv8801.0777: 2018.
Matteo Ruggero Ronchi and Pietro Perona. Benchmarking and error diagnosis multi-instance pose estimation. In ICCV, pp. 369-378, 2017.
Adrian Spurr, Umar Iqbal, Pavlo Molchanov, Otmar Hilliges, and Jan Kautz. Weakly supervised 3d hand pose estimation via biomechanical constraints. In ECCV, pp. 211-228, 2020.
Ke Sun, Bin Xiao, Dong Liu, and Jingdong Wang. Deep high-resolution representation learning for human pose estimation. In CVPR, pp. 5693-5703, 2019.
Xiao Sun, Bin Xiao, Fangyin Wei, Shuang Liang, and Yichen Wei. Integral human pose regression. In ECCV, pp. 529-5599-555 5459-555.440 2018.
Jonathan Tompson, Arjun Jain, Yann LeCun, and Christoph Bregler. Joint training of convolutional network and a graphical model for human pose estimation. NeurIPS, 27, 2014.
Bin Xiao, Haiping Wu, and Yichen Wei. Simple baselines for human pose estimation and tracking- In ECCV, pp. 466-481, 2018.
Linlin Yang, Shicheng Chen, and Angela Yao. Semihand: Semi-supervised hand pose estimation with consistency. In ICCV, pp- 11364-11377 2021.
Feng Zhang, Xiatian Zhu, Hanbin Dai, Mao Ye, and Ce Zhu. Distribution aware coordinate repre- sentation for human pose estimation. In CVPR, pp. 7093-7102, 2020.
Beier Zhu, Chunze Lin, Quan Wang, Renjie Liao, and Chen Qian. Fast and accurate: Structure coherence component for face aligmment. arXiv preprint arXiv:2006. 2020.
10

---

Published as a conference paper at ICLR 2022
A APPENDIX
A.1 EXTENSION OF FIG. 2 (B)
We provide an extended plot of Fig. 2 (b), which includes both detection and regression on easy, medium, and hard cases Fig. I.
A.2 PROOF FOR EQ. 7>.Q. SEC. 4.2
This subsection gives the detailed proof for Eq. 7>Eq. 8, which means
Σ
w - pll2 > || Mll2, where Σ peo(M,s)
w(p) = 1.
(I)
(J
In this way, this equation can be rewritten
w(p)]lJar DELL(
Σ
-ll2 2
Σ (E
w(p)|lJ 14l12-
(II)
As the distribution of $(u, s) is radially symmetric, a sufficient but unnecessary proof is that for any symmetric pair {pi (Ne a, By 6), (2 + a, My + b)}. it satisfies
Vxx - μz a)² + (y Hy 6)3 V Mz + a)² + (y My + 6)2 2 1=]= 22=
(v)
(III
According to Triangle Inequality in normed vector space, we arrive at
||u||2 ||||2 || v/12.
(IV)
When we set (x fzz a, pyy - b) and (x paz + a, U My + b), Eq. established.
A.3 PEARSON CHI-SQUARE VALUES ON DIFFERENT CASES
We report the detailed Pearson Chi-square values between heatmaps of both detection and regression in different scenarios and Gaussian templates with different values of Table. II, III. We set the threshold of A(s) 0.8 to show that the support region can represent the whole heatmaps. Therefore, for easy and medium cases, s 4; while for hard cases, 8-
1.0
Activation Sum A(s)
0.8
Regression-Easy
degression-Medlum
Regression-Hard
0.4
Regression-Overm Overall
Detection
Detection-Medium
0.2
Detection-Hard
Detection-Overall
10
Figure Activation Sum A(s) of both detection and regression method on easy, medium, hard, and overall cases.
11

---

Published as a conference paper at ICLR 2022
regression
delection
5
2
3
5
> 1000
4.76
4.12
3.95
>
1000
3.51
3.31
3.25
Table Results of the Pearson Chi-square test between distribution of heatmap H and Gaussian templates T with different window size and standard deviation ø in hard cases. = 5 is the best- matching template for both detection and regression. a stops at 5 as the Gaussian templates with size 8 become similar after.
regression
detection
0000
3.25
3.65
3.73
>1000
1.76
0.76
0.77
Table II: Results of the Pearson Chi-square test between distribution of heatmap H and Gaussian templates T with different window size s and standard deviation a medium cases. For detection, best matched should be between 3 and 4; while for regression, optimal in medium case should be between and 2.
A.4 DETAILS OF COMBINING PRIOR LOSS AND INTEGRAL LOSS
This subsection demonstrates the influence of weights of prior loss and integral loss, and the mag- nitudes of two gradients regarding the shared heatmaps during the training.
Let us assume up and w; are weights of the prior and integral losses, respectively. We consider the ratio up Wi XLg, where À = 10-2, to set the two to approximately equal magnitudes at the initial epoch. Table shows that the AP heavily depends on the value of a but for some fixed o (within each row), the ratio of the loss weightings has less impact and the results are stable within 2%, with a larger influence only at the extreme settings of 100 or 100.
Table IV (b) provides the ratios of the gradient magnitudes to help explain why the weightings have little impact. Table IV (b) gives the mean ratio 2e where 9p and g; are the gradient magnitudes the prior loss and integral loss, respectively, over the course of training. A column of 2/1/0/-1/-2 means 10F/10'/10/110010000 /10- ratio gradients over the epochs. The table shows that regardless of the weighting, the gradient ratio decreases over the course of training (i.e. dropoff with each column). Apart from an extremely high weighting of the prior, i.e. 100:1 in column 1, the prior influences the training only in the initial 10 epochs. After epoch 10, the gradient ratio is around the same scale or smaller.
A.5 DETAILS OF HEATMAP SHRINKAGE
To find out whether the added prior influences the spread of the heatmaps, we investigate the shrink- age change of heatmaps by separating the samples. We separate the samples based on easy/medium versus hard cases, where the former generally produce localized heatmaps while the latter have dis- persed heatmaps as per Fig. the alternative is to separate by thresholding the activation sum in fixed area (Eq. 6). We consider the activation sum A(1) > 0.34 of the integral regression baseline as localized, otherwise dispersed (threshold 0.34 selected based on Fig. I). We tally the activation sums for the localized and dispersed heatmaps, without and with the prior for different values of σ. Activation sums A(s)c/m- A(s) A(s), A(s) and Average Precision AP, APe/ms AP AP,. AP,, indicate the extent of localization and the performance of the two conditions, respectively. An in- crease activation sum indicates further heatmap localization; a decrease indicates dispersion. The "No prior" row is the baseline integral pose regression.
From Table IV (a), both separations present the same trend. For localized cases, forcing a to be degenerately small or too large (o 0.5 or 4) increases/decreases the activation sum over the baseline, thereby confirming the effect of the prior as intended. The change in AP follows, f.e. decreases when 0 becomes incompatible from being too small too large. For dispersed cases, the prior shrinks the activation sum as expected. Like the localized case, there an optimal range for which improvement in AP is better; degenerate will decrease the AP from the baseline.
12

---

Published as a conference paper at ICLR 2022
regression
detection
0.5
2
2.5
3.5
>+ 1000
3.24
3.53
3.75
3,89
> 1000
2.52
0.64
0.72
0.85
0.88
Table III: Results of Pearson Chi-square test between distribution of heatmap H and Gaussian tem- plates T with different window size and standard deviation in easy cases. Accordingly, for detection, a should be approximately 2.5; while for regression, optimal a should be between 0.5 and 1.
Up Ama
100:1
10:1
1:1
1:10
1:100
"ip Amn
100:1
10:1
1:1
1:10
1:100
0.5
63.6
64.8
65.1
65.7
66.4
epoch
2
1
I
67.6
68.1
68.5
68.3
67.8
epoch 10
2
0
+o
68.5
70.8
71.4
70.6
67.8
epoch
2
-1
+ 3
68.2
69.4
69.8
69.2
68.1
epoch 50
-1
-1
to 4
64.2
66.1
66.9
66.9
67.0
epoch 80
0
-1
-1
-1 -2
(a)
(b)
Table IV: (a) Performance of the integral model adding different priors with different weight ratios Wp Aw,. The performance depends largely on different dss, but less influenced with reasonable change. (b) Ratio of the gradient magnitudes of combined model (integral+o 2) with different weight ratios up : Aw; The gradient ratio decreases over the course of training. After epoch 10, the gradient ratio around the same scale or smaller.
model
A(2)a/m
APa/m
A(2)afm
AP:
A(2),
APμ
A(2),
APt
No prior
0.65
70.4
0.71
72.3
0.35
44.7
0.33
38.9
0.5
0.75
68.2
0.79
69.8
0.36
43.9
0.36
35.2
+
0.63
71.9
0.70
73.9
0.36
45.2
0.31
39.1
o 2
0.54
74.2
0.59
76.1
0.41
47.4
|
0.37
39.8
+
0.49
73.0
0.56
74.9
0.39
47.6
0.36
39.3
10 4
0.45
70.2
0.55
71.7
0.39
46.9
0.38
39.5
Table V: Localization extent and performance of divided localized samples versus dispersed sam- ples. The extent of localization largely depends on the added prior. With an optimal = 2 chosen, the previously localized heatmaps have been dispersed with improved performance.
A.6 DERIVATION OF EQ. 10
Eq. 10 is obtained by the chain rule. Therefore, we list the value of each term as follows:
tLi [s(J= J2) aj [s(J, Jz)
Shp [hh (1 hgg, q
(i+
(V)
aJ
Th,
Thg hhipha q q p
where q = (u, v) can be any pixel in Ω. Combining the terms, gradients at p can be calculated by
OLe = B(J - - (J, J,)0) hp)) Thp
Σ
B((J, J, (J,, - J1y (VI)
9Eo,
97p
Eq. 10 can obtained by rearranging the terms of Eq. VI.
A.7 DERIVATION OF EQ. 11
Let us suppose H,, g(w, IT) where H, is the heatmap of given image at the n-th iteration, g(w,) is the function of generating the heatmap. i.e. the backbone network, with weights at the n-th iteration. According Taylor expansion, arrive
13

---

Published as a conference paper at ICLR 2022
Left Iiee
High beee
Lect oekle
Right
Image
Lyf Right
akvakke
clow
Epadi
Rasdi
Faads
Ipach
Figure II: Visualization of the training performance of integral pose regression single image at epoch 0, 5, 10, 15, 20 (from top to down). Corner or edge pixels of the correct quadrant are largely activated in the epoch 0 (highlighted in the yellow boxes). Further training is focused narrowing down the activated region (shown in the red boxes).
Lol
Leli
Riglt
Lel.
Right
zye
sheskde aanaade
clase
cluo
lave
Upcch
Epeds
Pach
Epcda
13
20
Figure III: Visualization of the training performance of detection- based methods single image at epoch 0, 5, 10, 15, 20 (from top to down). One epoch has already enabled the network to roughly localize the joint compared with regression-based method (in the yellow boxes). Further training of detection-based methods makes the predictions more stable and precise (highlighted in red boxes).
g(w,+1) = ((w.) + 9 (Ww).
(Wn+1 w,) + 9 (Wa) 2!
(Wn+1 -W,,) w,,)?
(VII)
11
where Wa+1- WVV(w OL 9H9 (ww)) according to gradient descent, and (wx+1-W,,) is negligible compared with (Wa+1 w.) when i> 1. Therefore, Eq. VII can rewritten as
Hn+1 = g(w,+1) = g(w,) + (wa)(wa+1 w.) H,, a aL 0H 1g (w.).l
(VIII)
Therefore, for each pixel hp in H. the update rule is
n++ p
A
(IX)
where 7 represents allg' (w.))ll hy and V. denote heatmap and gradient value located at p in the heatmap the with iteration. B. denotes the original heatmap.
14

---

Published as a conference paper at ICLR 2022
A.8 REALWORLD VISUALIZATIONS
We show the visualization of the training performance of integral pose regression and detection- based methods training of integral pose regression is focused on narrowing down the activated region while the further training of detection-baser methods makes the predictions more stable and precise.
single image in Figs. Il and III, respectively. We can that the further
15


Reviews:
[{"review_id": "pBiIZsnc80S", "reviewer": "Reviewer_hXfD", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n1. This paper gives an interesting hypothesis on the cause of performance inconsistency in IPR methods. \n2. Some theoretical evidence and empirical results are provided to help understand and verify this hypothesis. \n\nCons:\n1. The authors claim that extremely localized heatmaps degrade performance (of IPR methods on easy samples). But this claim is not well supported by empirical results or theory.\n\n    (1) To verify the impact of sigma on IPR methods, the authors introduce a distribution prior to prevent the shrinkage of the heatmap. However, this method may also introduce some other confounding factors. For example, what are the weights of the prior loss term and the integral loss term? What are the magnitudes of the gradients produced by these two loss functions? Would the prior loss term dominate the training process thus Table 1 mainly reflects the impact of sigma on detection methods? These are not revealed. \n\n   (2) As shown by Fig 2(b), not all samples converge to the extremely localized heatmaps at the end of the training. After introducing an appropriate sigma prior, the overall performance improves as in Table 1. However, the respective impact of sigma on the previously localized heatmaps and previously dispersed heatmaps are not clearly shown. It might be better if the authors would show the change of shrinkage and performance separately on these two kinds of samples.\n\n   (3) Although theoretical evidence is provided to help understand the behavior of gradients of IPR methods, no theoretical evidence is provided to directly support the main claim: extremely localized heatmaps degrade performance.\n\n2. The novelty of the proposed heatmap prior loss is limited. It has already been explored in the initial IPR paper [1]. The authors in [1] combine the heatmap loss with the integral loss and show that this improves over the pure integral loss. What is more, the proposed heatmap prior loss does not substantially improve the gradient behavior of the IPR process. It is more likely that the prior loss dominates the backward update. It might be better if the authors would give more details.\n\nMinor Comments:\n\n1. Abbreviation was used before detailed explanation.  e.g.  Introduction Section :  EPE\n2. Typos:\n    (1) Subsection 3.3  hte --> the\n    (2) Subsection 4.2  E_{de} <= E_{re} -->  E_{de} >= E_{re}\n    (3) Subsection 5.1  location factor --> value factor\n3. Subsection 5.1: The detailed derivation of Eq10 is not shown in the Supplementary. \n\n[1] Sun et al. Integral human pose regression. ECCV 2018\n", "summary_of_the_paper": "This paper investigates the performance inconsistency of integral pose regression (IPR) methods on 'easy' pose samples and 'hard' pose samples (v.s. argmax methods). The authors find that the heatmap shrinkage caused by the IPR methods leads to the lower accuracy on 'easy' samples than argmax based methods. They give some toy examples and intuitively demonstrate how the shrinkage of heatmaps is caused by the gradient of the IPR method. They further propose a heatmap distribution prior loss to mitigate the shrinkage.", "summary_of_the_review": "Because the main claims are not well supported and the proposed heatmap prior loss lacks novelty, I lean towards reject.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "A12tgtVT5nw", "reviewer": "Reviewer_mXxk", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "- The paper considers a very interesting empirical setting. Keypoint Localization is a very popular task and many approaches rely on this. Getting more insight about each different method can be valuable to many people.\n\n- The paper combines some theoretical exploration along with empirical validation/demonstration in both toy settings, but also real settings. I find very useful this combination of theory/experiments and toy examples/real world settings. For example, the toy experiment shows different behavior than the real world training, so it is important to look at both of them at the same time. On the contrary, the theoretical study is a little basic, but it is enough to demonstrate the insights of the authors.\n\n- The insights and trade offs discussed are very useful for practitioners that use these methods. Some of these observations also exist in previous works, but it is helpful to have papers like that, that collect this type of observations (e.g., trade off for training speed, convergence, potential vanishing gradients, etc)\n\n- I really appreciate that when the authors are making specific assumptions (i.e., about the form of the heatmaps), they also provide the empirical demonstration of the fact.\n\n- In figure 4, the top red window (in third frame of bottom row) is not clear that it corresponds to the zoomed in version of the bottom window. I thought its goal was to indicate activations in locations other than the ankle. It would be good to indicate this to avoid confusion.\n\n- Is there any exploration to be done for cases where these two approaches are combined? E.g., training with heatmap supervision in the beginning to warm up the network, and with integral supervision later to get the best performance for hard examples too?", "summary_of_the_paper": "This paper is an in-depth study of the Integral Pose Regression method that is very common for Keypoint Localization. Integral Pose Regression has been adopted by many methods, and it is probably one of the two main paradigms for Keypoint Localization, along with Heatmap Regression. Many works have compared the two approaches, and in some cases (e.g., Gu et al, 2021) they provide a more detailed empirical comparison, identifying the strong and weak points of each approach. This paper has a similar goal, but combines theoretical investigation with empirical results to highlight insights about the two methods.", "summary_of_the_review": "Overall, I am positive about this paper. Obviously, the novelty is limited, but I believe that its value is high, particularly for practitioners. Since the analysis done here is more basic, I will give it a Weak Accept rating, but I think it is important to accept it to encourage similar efforts.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "3iKMWT8eOP1", "reviewer": "Reviewer_oYhA", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Strengths\n\n- The paper is well written and organized.\n- A theoretical study on how these two pose estimation methods, detection-based and integral regression-based, work is missing in the literature. This work has well studied this. \n- Comprehensive experiments are conducted to support the theoretical analysis.\n\n\n## Weaknesses\n- How to apply the conclusion in this work to further improve the pose estimation is missing. This is not a big issue to me. However, it would be great if the author can give such a guide according to the theoretical analysis. ", "summary_of_the_paper": "Heatmap detection-based method and integral regression-based method are widely used in the pose estimation task. The paper gave a detailed comparison and theoretical analysis between these two methods. A unified model is proposed to compare these two methods. Comprehensive experiments are conducted to support the theoretical analysis. ", "summary_of_the_review": "In the field of human pose estimation, the heatmap detection-based method and integral regression-based method are widely used. Previous works showed the performance and behavior differences between these two approaches. However, there are rarely works to theoretically study where the differences come from. This work proposed a unified model to analyze these two methods and is well supported by comprehensive experiments. It's great to see such a theoretical study on that. I'd like this paper, and I believe this work can help the research community in the field of human pose estimation to better understand the problem. Though the novelty of the paper is limited, I think we should encourage such efforts on how to theoretically analyze our methods.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}]

Author rebuttal:

Meta-review: All reviewers recognized the contribution of performing a theoretical study to investigate how the two technique lines (detection-based methods and integral regression-based methods) work for pose estimation. The study in this paper could be valuable for the researchers in the pose estimation domain, for the further research. The AC agrees with the reviewers and recommends accept for this paper.
Decision: Accept (Poster)