Explicit Box Detection Unifies End-to-End Multi-Person Pose Estimation
Authors: Jie Yang
Ailing Zeng
Shilong Liu
Feng Li
Ruimao Zhang
Lei Zhang
Venue: ICLR 2023 poster (2023)
Keywords: Multi-person Pose Estimation

Abstract: This paper presents a novel end-to-end framework with Explicit box Detection for multi-person Pose estimation, called ED-Pose, where it unifies the contextual learning between human-level (global) and keypoint-level (local) information. Different from previous one-stage methods, ED-Pose re-considers this task as two explicit box detection processes with a unified representation and regression supervision. First, we introduce a human detection decoder from encoded tokens to extract global features. It can provide a good initialization for the latter keypoint detection, making the training process converge fast. Second, to bring in contextual information near keypoints, we regard pose estimation as a keypoint box detection problem to learn both box positions and contents for each keypoint. A human-to-keypoint detection decoder adopts an interactive learning strategy between human and keypoint features to further enhance global and local feature aggregation. In general, ED-Pose is conceptually simple without post-processing and dense heatmap supervision. It demonstrates its effectiveness and efficiency compared with both two-stage and one-stage methods. Notably, explicit box detection boosts the pose estimation performance by 4.5 AP on COCO and 9.9 AP on CrowdPose. For the first time, as a fully end-to-end framework with a L1 regression loss, ED-Pose surpasses heatmap-based Top-down methods under the same backbone by 1.2 AP on COCO and achieves the state-of-the-art with 76.6 AP on CrowdPose without bells and whistles. Code is available at https://github.com/IDEA-Research/ED-Pose.

Published as a conference paper at ICLR 2023
EXPLICIT Box DETECTION UNIFIES END-TO-END MULTI-PERSON POSE ESTIMATION
Jie Yang!.2, Ailing
Liu', Feng Li", Ruimao Zhang"t, Lei Zhang International Digital Economy Academy (IDEA). Shenzhen Research Institute of Big Data, The Chinese University of Hong Kong, Shenzhen {zengai ling, liushilong, 1 feng, lei leizieine zhang) @ iidea. edu. cn jiccynng5eling zhangruimace cubk edu. on.
ABSTRACT
This paper presents a novel end-to-end framework with Explicit box Detection for multi-person Pose estimation, called ED-Pose, where it unifies the contex- tual learning between human-level (global) and keypoint-level (local) informa- tion. Different from previous one-stage methods, ED-Pose re-considers this task as two explicit box detection processes with unified representation and regres- sion supervision. First. we introduce a human detection decoder from encoded tokens to extract global features. It can provide a good initialization for the latter keypoint detection, making the training process converge fast. Second, to bring in contextual information near keypoints, point box detection problem to learn both box positions and contents for each keypoint. A Human-to-keypolee detection decoder adopts interactive learning strategy between human and keypoint features to further enhance global and local feature aggregation. In general, ED-Pose is conceptually simple without post- processing and dense heatmap supervision. It demonstrates its effectiveness and efficiency compared with both two-stage and one-stage methods. Notably, ex- plicit box detection boosts the pose estimation performance by 4.5 AP on COCO and 9.9 AP on CrowdPose. For the first time, fully end-to-end framework with L1 regression loss, ED-Pose surpasses heatmap-based Top-down meth- ods under the same backbone by 1.2 AP on COCO and achieves the state-of-the- art with 76.6 AP CrowdPose without bells and whistles, Code is available at https: www.tuhhhhhg..o com/ IDEA Research /DD Pose
regard pose estimation
key-
INTRODUCTION
Two-stage
Top-Down: Image
Global
Local
Global:
Where is the
Methodology
Target
"person" "?
Bottom-Up: Image
Local
Global
One-stage
Previous: Image
Locall
Local:
Where is the "hand"?
Ours:
Image
Global
Local
Perception of the multi-person pose estimation task
taxonomy estimation methods
Figure 1: Illustraioon of (a) the perception of the pose estimation task that usually captures global and local contexts concurrently; (b) taxonomy of existing estimators. ED-Pose (Ours) is novel one-stage method of learning both global and local relations in an end-to-end manner.
Multi-person human pose estimation has attracted much attention in the computer vision community for decades for its wide applications in areas of augmented reality (AR), virtual reality (VR), and interaction (HCI) Given image. it targets to localize the 2D keypoint positions for every person in the image. Although many methods have been developed (Xiao et al. 2018
"This work was done when Jie Yang was intern at IDEA.
"oorrsponniig author.

---

Published as a conference paper at ICLR 2023
Sun al. 2019 Cheng et al., 2020; Mao al. 2022 Shi et al. 2022). it remains challenging and intractable for situations with heavy occlusions, hard poses, and diverse body part scales.
Intuitively, as shown in Figure this task needs to focus on both global (human-level) and local (keypoint-level) dependencies, which concentrate on different levels of semantic granularity. Main- stream solutions normally two-stage methods, which divide the problem into two separate sub- problems (e.g., the global person detection and local keypoint regression). Such solutions include
Top-Down (TD) methods (Xiao al., 2018 Sun al., 2019; Li et al. 2021b Mao al. 2022) which are of high performance yet with a high inference cost and Bottom- Up (BU) solutions (Cao al., 2017, Newell et al., [017]] Cheng et al. 2020) which are fast in inference yet with relatively lower precision. However, all of these methods are non non-differentiable between their global and local stages due to hand-crafted operations, like Non-Maximum Suppression (NMS), Region of Interest (Rol) cropping, and keypoint grouping post-processing. Lately, Poseur (Mao et al. 2022) tries to directly apply top-down methods an end-to-end framework and finds that there will signifi- cant performance drop (about 8.7 AP on COCO), indicating the optimization conflicts between the learning of global and local relations.
Exploring a fully end-to-end trainable method to unify the two disassembled subproblems is at- tractive and important. Inspired by the success of recent end-to-end object detection methods, like DETR (Carion et al.] 22200) there is surge of related approaches that regard human pose estima- tion as a direct set prediction problem. They utilize a bipartite matching for one-to-one prediction with Transformers to avoid cumbersome post-processings (Li et al. [202Tb] Mao et al.. 2021a 2022
Stoffl et al. 2021] Shi et al. 2022). Recently, PETR (Shi et al. (022) proposes a fully end-to-end framework predict instance- aware poses without any post-processings and shows a favorable po- tential. Nevertheless, it directly uses a pose decoder with randomly initialized pose queries to query local features from images. The only local dependency makes keypoint matching across persons ambiguous and thus leading to inferior performance, especially for occlusions, complex poses, and diverse human scales in crowded scenes. Moreover, either two-stage methods or DETR-based es- timators suffer from slow training convergence and need more epochs (e.g., train a model above week) to achieve high precision. Additionally, the convergence speed of DETR-based methods is even slower than bottom-up methods (Cheng et al., 2020). We address the details in Sec 3
Based on the above observations, this work re-considers multi-person pose estimation as two Ex- plicit box Detection processes named ED-Pose. We realize each box detection by using decoder and cascade them to form an end-to-end framework, which makes the model fast in convergence, precise, and scalable. Specifically, to obtain global dependencies, the first process detects boxes for all persons via human box queries selected from the encoded image tokens. This simple step can provide good initialization for the latter keypoint detection to accelerate the training convergence. Then, to capture local contextual relations and reduce ambiguities in the feature alignment, we re- gard the following pose estimation task as keypoint box detection problem, where it learns both box positions and local contents for each keypoint. Such an approach can leverage contextual infor- mation near a keypoint by directly regressing the keypoint box position and learning local keypoint content queries without dense supervision (e.g.. heatmap). To further enhance the global- -lcall in- teractivity among external human-human, internal numan-keypolnn and internal syypoint-keypoiii we design interactive learning between human detection and keypoint detection.
Following the two Explicit box Detection processes, we can unify the global and local feature learn- ing using the consistent regression loss and the same box representation in an end-to-end frame- work. We summarize the related methods from the supervisions and representations. Compared with previous works, ED-Pose is more conceptually simple. Notably, we find that explicit global box detection will gain 4.5 AP on COCO and 9.9 AP on CrowdPose compared with a solution without such a scheme. In comparison to top-down methods, ED-Pose makes the human and key- point detection share the same encoders to avoid redundant costs from human detection and further boost performance by 1.2 AP on COCO and 9.1 AP on CrowdPose under the same ResNet-50 back- bone. Moreover, ED-Pose surpasses the previous end-to-end model PETR significantly by 2.8 AP on COCO and 5.0 AP on CrowdPose. In crowded scenes, ED-Pose achieves the ofate-of-the-art with 76.6 AP (by 4.2 AP improvement over the previous SOTA (Yuan et al. 2021)) without any bells and whistles (e.g., without multi-scale test and flipping). We hope this simple attempt at explicit box detection, simplification of losses, and no post-processing to unify the whole pipeline could bring in new perspectives to further one-stage framework designs.
2

---

Published as a conference paper at ICLR 2023
Table I: Comparisons of existing estimators from the losses and representations of the Human and Keypoints, Our proposed ED-Pose unifies both Human and Keypoint detection under the consistent LI regression loss and the box representation, making the end-to-end training simple yet effective.
Methodology
Human Loss
Keypoint Loss
Human Representation Keypoint Representution
Top-Down Bottom-Up
Regression
Heatmap
(x, g. m)
[ar, v) (a. M)
Two-Stage Methods
Heatmap
One Stage Methods
Previous
Regression Heatmap
[r,, y) (v, (,w)
Ours
Regression
Regression
2 RELATED WORK
One-stage Pertii Pose Estimation: With the development of anchor-free object detec- tors (Tian al. 2019b Huang et al. 2013). DirectPose (Tian al. 2019a) directly predicts instance- aware keypoints for all persons from an image. The direct end-to-end framework provides new perspective to avoid the above cumbersome issues met in two-stage methods. Generally speaking, these methods densely locate a set of pose candidates, which consist of joint positions from the same person. FCPose (Mao et al. 2021b) builds upon dynamic filters (Jia et al.] (0066) in compact keypoint heads to boost both accuracy and speed. Meanwhile, Inspose (Shi et al. [2021) designs aastacce-wware dynamic networks to adaptively adjust part of the network parameters for each in- stance. Nevertheless, these one-stage methods still need NMS to remove duplicates in the post- processing stage. To further remove such hand-crafted components, PETR (Shi et al. 2022) views pose estimation hierarchical set prediction problem and proposes the firs,fully end-to-end pose estimation framework with the advent of DETR (Carion et al. 2020).
For the first time, DETR (Carion et al. 2020) performs object detection in an end-to-end manner by using a set-based global loss that forces directly unique predictions via bipartite matching and a Transformer encoder-decoder architecture. simplifies object detection as a direct set prediction problem, dropping multiple hand-designed components and prior knowledge.
Detection Transformers:
Due to the effectiveness of DETR and varieties (e.g., Deformable DETR (Zhu et al.. 2020
their frameworks have been widely transferred in many complex tasks, such as Mask DINO (Li
al., (022b) for segmentation and PETR (Shi et al.] 2022) for pose estimation. Following the top-down methods, PRTR (Li et al. 2021b) and TFPose (Mao et al. [02[a) adopt Detection Trans- formers to estimate the cropped single-person images as a query-based regression task. To cap-
ture the underlying output distribution and further improve performance in the regression paradigm, Poseur (Mao et al., 2022) brings Residual
Estimation (RLE) (Li al.) (202 Ia) into the D top-down framework, achieving the state-of-the-arr performance of regression -based methods. For the one-stage manner, POET (Stoffl et al.] 2021) utilizes the property of DETR to directly regress the poses (instead of bounding boxes) of all person instances in an image. Recently, PETR (Shi et al. 2022) designs a fully end-to-end paradigm with hicrarchical attention decoders to capture both relations between poses and kinematic joints.
All the aforementioned methods take advantage of Detection Transformers and densely regress a set of poses (only local relations). However, they ignore the importance of introducing explicit box detection in pose estimation to model both global and local dependencies well.
RETHINKING ONE-STAGE MULTI-PERSON POSE ESTIMATION
The Necessities of One-stage Methods: For a long time, the two-stage paradigm dominates the mainstream methods for multi-person pose estimators. It can be generally divided into top-down methods and bottom-up methods. Top-Down (TD) methods (Xiao et al. 2018 Sun et al. 2019 Mao al.) 2022) disentangle the task into detecting and cropping each person with an object detector (e.g., Mask RCNN (He et al. (017)) in an image from the global (person-level) dependency and then conduct the single-person pose estimation via another model. They focus on local (keypoint- level) relation modeling and improving the accuracy of single-person pose estimation. However, these methods still suffer from 1). heavy reliance on the performance of the human detector, 2). redundantly computational costs for additional human detection and Rol operations, and 3). the separate training for the human detector and the corresponding pose estimator. Instead, Bottom- Up (BU) methods (Cao et al., 2017; Newell et al.) 2017;; Cheng et al. 2020 first detect all the keypoints in an instance-agnostic fashion. Next, they employ a heuristical grouping algorithm to associate the detected keypoints that belong to the same person, which improves efficiency. Even
3

---

Published as a conference paper at ICLR 2023
so, the complicated grouping scheme makes bottom-up methods hard to handle heavy occlusions and multiple-persee scales, resulting in inferior performance. More importantly, both of them suffer from non-differentiable optimization between the global and local features, which is not perceptive. Intuitively, one-stage methods could alleviate the above issues since all modules can be optimized in end-to-end manner, and balance effectiveness and efficiency. Interestingly, the recent DETR- based top-down method Poseur (Mao et al. (2022) tries to directly apply it to a one-stage framework and finds that it suffers from a significant performance drop. It might be the optimization conflicts between the global and local dependency learning using the shared encoder. Thus, how to design one-stage framework effectively is still challenging and questionable.
In terms of existing DETR- -based methods, most of them still adopt the top-down framework, and improve the second single-person pose estimation via regarding it as sequence prediction problem (Mao et al. 2021b] Shi et al. 202T). PETR is the first work to make the whole pipeline end-to-end without any post-processing However, existing methods still have some limitations. First, all of them only utilize local dependencies to regress keypoints, Directly regressing keypoints for each person via pose queries is semantically ambiguous the bottom-up strategies to find all keypoints from raw images instead of from cropped images. Second, the pose or keypoint queries proposed in the above methods are randomly initialized without utilizing previously extracted features, making the training phase slow and ineffective. Third, the keypoint representation as a point lacks contextual information when it queries from the encoded features, leading to feature misalignment. Last, the interactions among global-to-global global-to- local, and local-to-local are complex, especially in crowd scenes. The current models do not pay attention to handling these complicated relations. In this article, we try to tackle the above issues by using unified box representations and regression losses in a one-stage process.
The Bottlenecks of Existing One-stage Methods:
METHODOLOGY
4.1
OVERVIEW
Sona RF
Backbone
Th. Cx
ti 0
doo pa
Qa 0%
Human Detection Decrolrr
2% of Ch
Human-to Maman-to-KKyyooiie Keypoint Delection Decoder
PFN
+ BuxCllas
Eincdder
FFN
Keypoints
FFN
BaxClass
Keypoints
Course Human Query Selection
Fine Human
Hurmm--o-Keypoin Query Exponsion
Query Selection
Figure 2: The overview architecture of our ED-Pose, which contains a Human Detection Decoder and to-aantt--Kyooin Detection Decoder to detect human and keypoint boxes explicitly.
As illustrated in Figure 2 the proposed ED-Pose is a fully end-to-end multi-person pose estimation framework without any post-processing. Given an input image, we first utilize a backbone extract multi-scale features and obtain tokenized representations F. and then feed them into the Transformer encoder (e.g., deformable attention modules (Zhu et al. 20200) with Positional Embeddings (PE) to calculate the refined tokens F'. To improve the efficiency of the following decoding process, we conduct coarse human query selection from F' obtain the sparse human content queries Qf Then we utilize Qi to generate the corresponding position queries Q₁ via a Feed- Forward Network (FFN) and feed both Q3 and Q4 into the Human Detection Decoder to update into the correspond-
ing queries Q and Q respectively (see See 4.2). For each output human content query, we attach human box regression and class entropy supervision (I!, and E.) the I-th decoder layer. Next, we further perform fine human query selection to discard the redundant human queries, obtaining content queries Qi,, and position queries Q² We initialize keypoint queries based these re- tained high-quality human queries and concatenate the human and keypoint queries together to form human- keypoint queries. We name such scheme as -omaa-to--keyoin query expansion, and the obtained buman--keypon content queries Q K and position queries Q are fed into Human-to- Keypoint Detection Decoder. In this way, we realize the keypoint detection at an instance level and update these queries Q³' HH.K and Q QH, layer by layer (see 4.3). Also, the keypoint and human
4

---

Published as a conference paper at ICLR 2023
box regression loss and class loss (L), Lj and L') are added to the 1-th decoder layer. Finally, use several FFN heads to regress the keypoint positions in each human box.
Following the DETR (Carion al. 2020), employ set-based Hungarian loss that forces a unique prediction for each ground -hound-truth truth box and keypoint. Our overall loss functions contain classification Le, human box regression Lt, and keypoint regression loss Lą- Notably, L simply consists of the normal L1 loss and the constrained L1 loss named Object Keypoint Similarity (OKS)
Loss:
loss (Shi al. 2022 without any dense supervision (e.g., heatmap).
(x. AA)
dy,
Multi-scale Envoded Tokens
Deformable
Multi-scale Encoded Tokens
Bokken-to-HHumane Keypooonn Attention
Deformable
Hoken-to-Human Attention
Hlman-to-Hunaal Attention
Human-to Huuaa--o-Keyyoinng Keypoint
Interactive Attention
K
x2
Qh
Qu.
0
(a) Human Detection Decoder
(b) Hurmantt--Keynn Detection Decoder
(c) Interactive Leamiing
Figure 3: The detailed illustration of (a) Human Detection Decoder, (b) Human-to-Keypon Detec-
tion Decoder and (c) the detailed Interactive Learning in Human-to-Keypoint Detection Decoder.
4.2 HUMAN DETECTION DECODER
In Figure Bl (a), the proposed Human Detection Decoder aims to predict the candidate bounding box positions for each person in the image, as well as the corresponding content representation by leveraging image context from all encoded tokens. Given N input human content queries Q E NxD and human position queries Q E Nx4, where D is the channel dimension, the Human
Detection Decoder outputs N refined human box positions Ø and content representations Q
-oinse-to-Fine Human Query Selection: Unlike previous work (Shi et al. 2022) to randomly initialize the input queries, we adopt the query selection (QS) strategy for better initialization (Zhang scheme to progressively select high-quality human queries. In practice, the coarse human QS selects first from large number of refined tokens F' T where T is the number of the tokens (e.g., about 15K). Thus, the human content queries Q2 E NxD are initialized by the top-N refined tokens' features, ranked by their instance classification scores. Here N indicates the number of candidate human boxes (e.g.. 900). Then we use these selected human content queries to calculate their position queries Q E Nx by employing simple detection head. After passing through the Human Detection Decoder, we further conduct the fine human QS by only retaining M (e.g., 100) refined human content queries and their position queries according to the classification scores, denoted Q2, R MxD and Q E R Mx4
et al. 2022). Specifically, we propose coarse-to-fie
Human Box Detection: As illustrated in Figure 3 (a), we combine Qj with Q and feed them into Human-to-Human Attention (i.e. self-attention layer) to calculate the relations among hu- man queries, and output contextual enhanced ones. Motivated by the deformable DETR Zhu et al. 2020), these enhanced human content queries, together with their position queries further fed into Deformable Token-to- -oo-tttat-nn Attention (i.e. a cross-attention layer) to update human queries by conducting interaction with multi-scale tokens. Next, to adjust the human box positions, lever- age updated human queries to calculate the 4D offsets following the DAB-DETR (Liu et al. 2022), and add them back to previous human position queries. In this way, the Human Detection Decoder refine the human content and position queries progressively by stacking multiple aforementioned layers and output Q NxD and Q E Nx1.
5

---

Published as a conference paper at ICLR 2023
4.3 HUMAN-TO-KEYPON DETECTION DECODER
To unify both human and keypoint representations the box and facilitate follow-up interactive learning, we regard multi-person pose estimation as the multiple set keypoint box detection prob- lems. Based the selected high quality human queries QHz and Qúr, we initialize multiple sets of keypoint queries (i.e. including both content and position queries) through the Human- to- to-Keypoint Query Expansion process, and concatenate the human and keypoint queries together inputs of the Hluman-to-Keyponn Detection Decoder (Figure. 3 (b)) to calculate the precise keypoint positions for each person.
Human-to-Keypen Query Expansion: After obtaining Q and Q2+ the keypoint content query and positional query are initialized by the relevant human query instead of random initialization. In practice, we first initialize a set of learnable keypoint embeddings V. E IR IRIXXD where K is the total number of keypoints (e.g.. 17). To concretize these embeddings to multiple specific persons' keypoint queries, we broadcast the first dimension of Ve to the number of human queries M and then add it with Qi,, E TR. TRMx x obtain M set of keypoint content queries. For a specific set of keypoint position queries, we separate the initialization process into center coordinate initialization (e.g.. {x, y)) and box size initialization (e.g.. (w, h))). ). For the former, we adopt an FFN to regress all of the K positions" coordinates by employing the corresponding human content query. For the latter, the sizes of K boxes are conditioned by the width and height of the corresponding human box via dot- dot-muuliplying dynamic weights W R xxx2 After that, consider each human box and the corresponding set of keypoint boxes as a whole and generate the -eypnnt content queries QHI E IR (M (MM+M+K)D and position queries Q E IMM Mx) for further predictions.
The Interactive Learning between Human and Keypoint Detection: As illustrated in Figure (b) and (c), after generating numan-keypoin queries, we feed them into Human-to te-Kpppiit In- teractive Attention learn relations from internal human-keypoint, internal keypoint-keypoin and external human-human. Such an interactive learning method between global and local feature aggre- gations has been ingeniously introduced to ensure the effectiveness of keypoint feature extraction. In practice, we find that the fully connected interactive learning with external keypoint-key would cause a large disturbance, especially in crowded situations, similar to problems encountered in bottom-up methods. In contrast, external human-humai interactions effectively propagate global context across different candidates and distinguish their relations clearly. Then we take the enhanced numan-to-keypoe kens to obtain updated human-to-keypoint queries. By stacking multiple layers illustated in Figure (b), we finally obtain the refined numan-to-keypone queries, denoted Q QH.K and Q
content queries to conduct the interaction with multi-scale encoded to-
EXPERIMENTS
Due to the page limit, we leave the detailed experiment setup in A comparison results on COCO test-dev B. comparisons on human detection in C more qualitative results and analyses in D and more discussion for ED-Pose (in Sec.
5.1 RESULTS CROWDPOSE
We first verify the effectiveness of ED-Pose with other state-of-theeart methods the CrowdPose test set in Table 2 Compared with the top-down methods, we surpass the SimpleBaseline (Xiao) al. (218)) under the same backbone by 9.1 AP. we also show superiority on all previous methods combined with Swin-L (Liu et al.] 202T) backbone and outperform PETR by 1.5 AP under the same backbone. When we enlarge the scales of backbones, ED-Pose will achieve the state-of-the-arr 76.6 AP without multi-scale and flip tests.
5.2 RESULTS COCO
We further make comparisons with the state-of-the-art methods COCO val2017 and test- in Table. 3 and Table. [ respectively. In general, our ED-Pose outperforms all exist- ing bottom-up methods and one-stage methods under the same backbone without any tricks, even the dense heatmap- based top-down methods. The proposed method achieves 71.6 AP (by 2.8 AP improvement) with a 51.4% inference time reduction via the ResNet-50 backbone. Additionally, our best model with the Swin-L backbone achieves a 75.8 AP, showing a consistent performance improvement. The improvements in the test-dev are similar.

---

Published as a conference paper at ICLR 2023
Table 2: Comparisons with state-of-the-art methods on CrowdPose test dataset. TD, BU, OS mean top-down, bottom-up, and one-stage methods, respectively. We use "HM." and "R." for heatmap-base losses and regression losses. i denotes the flipping test. The model with is pre- trained on Objects365 (Shao et al. 2019a) with 5 feature scales. The underlined highlights the compared results. The best results are highlighted in bold.
Method
Loss
AP
AP
APm
AP
AРм
APи
Base, (ResNct-50)
HM.
60.8
81.4
65.7
71.4
61.2
51.2
HRNet (HRNet-w48) f
HM.
71.3
91.1
77.5
80.5
71.4
62.5
TransPosc-H
HM.
71.8
91.5
77.8
79.5
72.9
62.2
HRFormer-B
HM.
72.4
91.5
77.9
80.0
73.5
62.4
HrHRNet-w332f
HM.
65.9
86.4
70.6
73.3
66.5
57.9
DEKR (HrHRNet-W33)t
HM.
65.7
85.7
70.4
73.0
66.4
57.5
SWAHR (HrHIRNet-w32)
HM.
71.6
88.5
77.6
78.9
72.4
63.0
PETR (Swin-L)
R.+ R.+HM.
71.6
90.4
78.3
77.3
72.0
65.8
8 ED-Pose (ResNel-50)
R.
69.9
88.6
75.8
77.7
70.6
60.9
ED-Pose (Swin-L)
R.
73.1
90.5
79.8
80.5
73.8
63.8
ED-Pose (Swin-L**
76.6 1s.0
92.4
83.3
83.0
77.3
68.3
Table 3: Comparisons with state-of-the-art methods on COCO va val200 2017 7 dataset. i denotes the flip- ping test. I removes the prediction uncertainty estimation in Poseur as fair regression comparison. The underlined highlights the compared results. The inference time of all methods is tested on an A A100, except that the detector of top-down methods is tested by the MMdetection (i.e., 45ms).
Method
Backhone
AP
APso
AP7o
AРx
AP
Time [ms]
Sim.Base.
ResNet-50
HM.
70.4
88.6
78.3
67.1
77.2
45+86
HRNett
HRNet-w32
HM.
74.4
90.5
81.9
70.8
81.0
45+-111
Two-stage
PRTRI
ResNet-50
R.
68.2
88.2
75.2
63.2
76.2
45+85
Poscur
ResNet-50
RLE!
70.0
45482
Poseur
ResNet-50
RLE
74.2
89.8
81.3
71.1
80.1
45 82
HrHRRNet
HRN
HM.
67.1
86.2
73.0
61.5
76.1
322
DEKR
HRNet-w32
HM.
68.0
86.7
62.1
77.7
SWAHR
HRNct-w32
HM.
68.9
87.8
77.4
Mask R-CNN
ResNet-101
HM.
66.0
86.9
PRTRI
ResNet-101
HM.
64.8
85.1
70.2
60.4
73.8
Poseurt
ResNet-101
RLE
68.6
87.5
74.8
Poseurt
HRNet-w48
RLF
70.1
88.0
76.5
InsPose
ResNet-50
R.+HM.
65.2
87.2
71.3
60.6
72.2
78
PETR
ResNet-50
R.+HM.
68.8
87.5
76.3
62.7
77.7
105
g
PETR
Swin-L.
R.+HM
73.1
90.7
80.9
67.2
81.7
206
ED-Posee
ResNet-50
R.
71.6"a.s
89.6
78.1 1
65.9
79.8
51 51..1.4%
EDDPPsse
Swin-I.
R.
74.311.2
91.5
81.6
68.6
82.6
88 (37.333
Puse
Swin-L
R.
75.8 12.7
92.3
82.9
70.4
83.5
142.an.9%
5.2.1 COMPARISON OF EFFECTIVENESS
Comparison with one-stage methods:
Our method significantly outperforms all existing one-stage methods, especially in Table. such as DirecetPose (Tin et al. 20199)). FCPose (Mao et al. (0211)) Inspose (Shi al. 202I CenterNet (Zhou al. 2019). and PETR (Shi et al.) 2022). showing that ED-Pose with explicit box detection is an effective solution for the end-to-end framework.
Comparison with two-stage methods: For bottom-up methods, ED-Pose outperforms state-of-the- art methods by large margin, such as HigherHRNet (Cheng et al. 2020). DEKR (Geng et al.) 2021). and SWAHR (Luo al., 202T). Specifically. ED-Pose significantly surpasses the recently proposed SWAHR by 3.7 (P1.16 6 AP vv. 67.9 AP) even with a much smaller backbone (ResNet-50 vs. H32) For top-down methods, ED--Pse is 1.2 AP and 3.4 higher than SimpleBase- line (Xiao et al. 2018) (Sim.Base) and PRTR (Li et al. 2021b), respectively. To the best of our knowledge, this is the first time that a fully end-to-end framework can surpass top-down methods.
Comparison with end-to-end townn methods: Poseur (Mao et al., 2022 extends its framework to end-to-end human pose estimation with Mask-RCNN, Similarly, PRTR presents its end-to-end variant. However, compared to their two-stage paradigms, their end-to-end frameworks produce substantially inferior performance. Our fully end-to-end framework achieves much higher AP scores than theirs, indicating the effectiveness of our proposed methods.
7

---

Published as a conference paper at ICLR 2023
Qualitative results:
With explicit box detection, ED-Pose perform well wide range of poses, containing viewpoint change, occlusion, motion blur, and crowded scenes. Fig. demon- strates the results of the detected person and the corresponding keypoints. As can be observed, the human boxes are precise under many severe cases, thus they can provide effective global human in- formation for further keypoint detection. The local regions of the keypoint boxes are also reasonable for bringing abundant contextual information near the keypoint.
Figure 4: Qualitative results of ED-Pose COCO (the first row) and CrowdPose (the second row).
We present both explicitly detected person boxes and keypoint boxes to understand how they work.
5.2.2 COMPARISON OF EFFICIENCY
ELYPoow (SwiL_-Hag)
(R50-Rac)
AP on COCO var2017
(950
coco ve12017
74
Pioaur (P50-RLE)
PETR (Swihi-Rgg--IM)
HihetHPNcc (HRNGT HM!
Bim.Bose (R1G2 HMI
Rogi
Bis (R101-H10
Bass. (750--15)
Proeer RIe
(800 RLE))
PETR (960 RegeHiM)
58
HigherHRNeet (-RMer-w22--4
45
Epoch
Time (ms)
Figure 5: Comparisons of convergence speeds in the training stage (the left) and trade-offs be- tween inference time and performance (the right) cxisting mainstream methods. Our proposed one-stage method ED-Pose shows the superiority of efficiency compared with the Bottom-Up (BU) model HigherHRNet (Cheng et al., 2020). Top-Down (TD) models Sas.Base. (Xiao et al.) 2018 and DETR-based Poseur (Mao et 2022 the one-stage method PETR (Shi et al. 2022).
For the inference time, ED-Pose surpasses all the bottom-up and one-stage methods in both speed and accuracy fields from Table. and Notably, from the right of Fig. 5 we compare the inference time under different AP scores. ED-Pose achieves the best efficiency and effectiveness trade-offs. Under the competitive performance (about 74.2) with the DETR-Basse top-down method (Poseur with RLE loss ([.i et al. (021aa)) ED-Pose can boost inference speed by 30.7%. Moreover, the left of Fig. 5 shows the convergence speeds the training stage that are rarely discussed before. EP- Pose introducing explicit detection is faster than all methods and obtains better performance under the early epochs. Interestingly, all DETR-based methods are slower than even bottom-up methods.
and
5.3 ABLATION STUDY
Explicit human detection: We first analyze the effectiveness of explicit human detection on both the COCO va12017 and the CrowdPose test, using the ResNet-50 backbone. We remove all human detection losses in decoder layers to verify its effectiveness. Results in Table. clearly
8

---

Published as a conference paper at ICLR 2023
verify that explicit human detection supervision significantly improves the convergence speed of keypoint detection and precision, yielding +4.5 AP on COCO and +9.9 AP on CrowdPose. Table 4: Impact on explicit human detection (Human Det.) for convergence speed and precision on the COCO dataset (the left) and CrowdPose dataset (the right).
Human Det.
12c
24c
36c
48c
60c
Human Det.
12e
24c
36c
48c
60e
80e
41.1
56.6
61.0
64.7
67.1
13.1
20.5
31.1
42.6
51.3
60.0
60.5
67.5
69.7
70.8
71.6
37.1
54.8
62.4
66.4
68.2
69.9
ED-Pose reformulates pose estimation as the keypoint box de- tection, where the 2D center-point coordinate (z, y) of keypoint is extended to the 4D representation (ar y, h).. As the region of feature aggregation for keypoint depends on the width and height of the keypoint box. we explore the effect of (w, h)) under five initialization ways: 1). None discards width and height and keeps the original 2D coordinate representation ; 2). Min. denotes that the keypoint is initialized by 1% width and height of the corresponding human box; 3). Max. means that a keypoint is directly initialized by the width and height its corresponding human box; 4). FFN. is that apply an FFN network upon the human content query to regress 2D weight of each keypoint within the human box: 5). Ours means that we initialize learnable embeddings for each keypoint across the whole dataset and utilize it to weigh the width and height of the human box. As shown in Table. 5 the result without width and height representation is much lower than those of the other four settings, leading 13.3 AP drop compared with the best initialization way (Ours) since it will lose contextual information to query the local and global relations. Learnable width and height obtain better performance than the fixed sizes.
Keypoint detection representation:
The interactive learning between human and keypoint detection: Our human-to-keypoint de- tection decoder enhances the interactivity among external human-to-human, internal human-to- keypoint, and internal eeypoint-to-kepoiii
Thus, we explore the impact of different interactive learning strategies: 1) Full makes all human queries and keypoint queries interact one by one: 2) interaction; 3) w/o H-H discards the external
w/o H-K removes the internal human-to-keyne human-to-humen interaction; 4) Ours is our full interactive learning strategy. shown in Table 66) direct full connection for all queries could lead to 1.2 AP degradation. The reason is that external keypoint-keypott interaction a full connection strategy may cause large disturbance, especially in crowded situations. Besides, other interactive strategies slightly affect performance as well.
To explore the effect on the number M of selected queries in Sec. 4.2 we conduct three numbers with 50, 100, and 200, respectively. As shown in Table. 7 increasing selected human queries can improve performance, but meets a bottleneck for further improvement.
Hyperparameter tuning:
Table 5: Impact on the width and Table 6: Impact on interactive learn-
Table 7: Impact on the number of M lected queries.
weight initialization of the keypoint
ing between human and keypoint de- lection.
box.
hw
None
Min.
Max.
FFN.
Ours
Strategy
Full
who IIK
w/o I1-11
Ours
100
AP
58.3
70.8
70.8
71.2
71.6
AP
70.4
71.2
71.3
71.6
AP
70.9
71.6
71.6
AP
57.5
65.1
64.9
65.7
65.9
APM
64.6
65.6
66.0
65.9
AP
65.2
65.9
APE
60.0
79.2
79.3
79.2
79.8
APE
78.8
79.3
79.0
79.8
APL
79.2
79.8
79.6
CONCLUSION
In this work, we re-consider the multi-person pose estimation task as two explicit box detection processes. We unify the global person and local keypoint into the same box representation, and they can be optimized by the consistent regression loss in a fully end-to-end manner. Based on the novel methods, the proposed ED-Pose surpasses existing end-to-end methods by large margin and shows superiority over the simple attempt make the whole pipeline succinet and effective. We hope this work could inspire further one-stage designs.
two-stage methods on both COCO and CrowdPose. This is
ACKNOWLEDGEMENT
The work is partially supported in by the Young Scientists Fund of the National Natural Science Foundation of China under grant No. 62106154, by the Natural Science Foundation of Guangdong Province, China (General Program) under grant No. 20.22222155511154,, and by Shenzhen Science and Technology Program ZDSYS2021 TSSSS22220221111011 102111 1415025.

---

Published as a conference paper at ICLR 2023
REFERENCES
The Cao, Tomas Simon, Shih-En Wei, and Yaser Sheikh, Realtime multi-person 2d pose estimation using part affinity fields. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 7291-7299, 2017.
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European conference on computer vision, pp. 213-229. Springer, 2020.
Bowen Cheng, Bin Xiao, Jingdong Wang. Honghui Shi, Thomas S Huang. and Lei Zhang. Higherhr- net: Scale-aware representation learning for bottom-up human pose estimation. In Proceedings of the JEEE/CVE conference on computer vision and pallern recognition, pp. 5386-5395, 2020.
Zigang Geng, Ke Sun, Bin Xiao, Zhaoxiang Zhang, and Jingdong Wang. Bottom-up human pose estimation via disentangled keypoint regression. In Proceedings of the IEEE/CVF IEEE//CFFConferene Conference
Computer Vision and Pattern Recognition, pp. 14676-14686, 2021.
Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In Proceedings of the IEEE international conference computer vision, pp. 2961-2969, 2017.
Lichao Huang, Yi Yang, Yafeng Deng, and Yinan Yu. Densebox: Unifying landmark localization with end to end object detection. arXiv: Computer Vision and Pattern Recognition, 2015.
Xu Jia, Bert De Brabandere, Tinne Tuytelaars, and Lue Van Gool. Dynamic filter networks. neural information processing systems, 2016.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint
arXiv: 1e/.///226660 69800, 2014.
Feng Li, Hao Zhang, Shilong Liu, Jian Guo, Lionel M Ni, and Lei Zhang. Dn-detr: Accelerate detr training by introducing query denoising. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13619-13623 2022a.
Feng Li, Hao Zhang, Shilong Liu, Lei Zhang, Lionel M Ni, Hleug--Yeung Shum, al. Mask dino: Towards unified transformer- based framework for object detection and segmentation. arXiv preprint ar 2el.22206777777 2022b.
Jiefeng Li. Can Wang. Hao Zhu, Yihuan Mao, Hao-Shu Fang. and Cewu Lu. Crowdpose: Effi- cient crowded scenes pose estimation and a new benchmark. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10863-10872 2019.
Jiefeng Li, Siyuan Bian, Ailing Zeng, Can Wang, Bo Pang, Wentao Liu, and Cewu Lu. Human pose regression with residual log- log-likelihood estimation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 11025-11034 2021a.
Ke Li, Shijie Wang, Xiang Zhang, Yifan Xu, Weijian Xu, and Zhuowen Tu. Pose recognition with cascade transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1944-1953, 2021b.
Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740-755. Springer, 2014.
Tsung-Yi Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. Proceedings of the IEEE international conference on computer vision, pp- 2980-2988, 2017.
Shilong Liu, Feng Li, Hao Zhang, Xiao Yang, Xianbiao Qi, Hang Su, Jun Zhu, and Lei Zhang. Dab-detr: Dynamic anchor boxes are better queries for detr. arXiv preprint arXiv:2201. 12329, 2022.
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hicrarchical vision transformer using shifted windows. In Proceedings of the IEFF/CVF International Conference Computer Vision (ICCV), 2021.
10

---

Published as a conference paper at ICLR 2023
Iyy Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 2017.
Zhengxiong Luo, Zhicheng Wang, Yan Huang, Liang Wang, Tieniu Tan, and Erjin Zhou. Rethinking the heatmap regression for bottom-up human pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13264-13273, 2021.
Weian Mao, Yongtao Ge, Chunhua Shen, Zhi Tian, Xinlong Wang. and Zhibin Wang. Típose: Direct human pose estimation with transformers. arXiv preprint arXiv:2103. /53.0000, 2021a.
Weian Mao, Zhi Tian, Xinlong Wang, and Chunhua Shen. Fepose: Fully convolutional multi-person pose estimation with dynamic instance-aware convolutions. arXiv: Computer Vision and Pattern Recognition, 2021b
Weian Mao, Yongtao Ge, Chunhua Shen, Zhi Tian, Xinlong Wang, Zhibin Wang, and Anton van den Hengel. Poseur: Direct human pose regression with transformers. arXiv preprint arXiv:2201.0744 2022.
Alejandro Newell, Zhiao Huang, and Jia Deng. Associative embedding: End-to-end learning for joint detection and grouping. Advances in neural information processing systems, 30, 2017.
Joseph Redmon and Ali Farhadi. Yolov3:
An incremental improvement. arXiv preprint
arXiv:180..02667, 2018.
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015.
Hamid Rezatofighi, Nathan Tsoi, Young Gwak, Amir Sadeghian, Ian Reid, and Silvio Savarese. Generalized intersection over union: A metric and loss for bounding box regression. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 658-666, 2019.
Shuai Shao, Zeming Li, Tianyuan Zhang, Chao Peng, Gang Yu, Xiangyu Zhang, Jing Li, and Jian Sun. 0bjects365: A large-scale, high-quality dataset for object detection. international confer- ence on computer vision, 2019a.
Shuai Shao, Zeming Li. Tianyuan Zhang, Chao Peng, Gang Yu, Xiangyu Zhang, Jing Li, and Jian Sun. Objects365: A large-scale, high-quality dataset for object detection. In Proceedings of the IEEE/CVF international conference computer vision, pp. 8430 8439, 2019b.
Dahu Shi, Xing Wei, Xiaodong Yu, Wenming Tan, Ye Ren, and Shiliang Pu. Inspose: Instance- aware networks for single-stage multi-person pose estimation. acm nultimedia, 2021.
Dahu Shi, Xing Wei, Liangqi Li, Ye Ren, and Wenming Tan. End-to-end multi-person pose esti- mation with transformers. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1069-11077, 2022.
Lucas Stoffl, Maxime Vidal, and Alexander Mathis. End-to-end trainable mation with transformers. arXiv preprint arXiv:2103.12115, 2021.
pose esti-
Ke Sun, Bin Xiao, Dong Liu, and Jingdong Wang. Deep high-resolution representation learning for human pose estimation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5693-5703, 2019.
Zhi Tian, Hao Chen, and Chunhua Shen. Directpose: Direct end-to-end multi-person pose estima- tion. arXiv preprint 1XXx::91110075:: 2019a.
Zhi Tian, Chunhua Shen, Hao Chen, and Tong He. Feos: Fully convolutional one-stage object detection. international conference computer vision, 2019b.
Dongkai Wang and Shiliang Zhang. Contextual instance decoupling for robust multi-person person pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, pp. 11060- 1006-11068, 2022.
11

---

Published as a conference paper at ICLR 2023
Bin Xiao, Haiping Wu, and Yichen Wei. Simple baselines for human pose estimation and tracking. In Proceedings of the European conference on computer vision (ECCV). pp. 466-481, 2018.
Nan Xue, Tianfu Wu, Gui-Song Xia, and Liangpei Zhang. Learning local-global contextual adapta- tion for multi-person pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 13065-13074, 2022.
Yuhui Yuan, Rao Fu, Lang Huang, Weihong Lin, Chao Zhang, Xilin Chen, and Jingdong Wang- Hrformer: High-resolution vision transformer for dense predict. Advances in Neural Information Processing Systems, 34:7281-7293, 2021.
Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M Ni, and Heung-Yeung Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. arXiv 2022.
preprint arXiv:2203.08:
Xingyi Zhou, Dequan Wang, and Philipp Krähenbühl. Objects as points. arXiv preprint arXiv:1904.07850, 2019.
Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable transformers for end-to-end object detection, arXiv preprint 15X10:2010.04159, 2020.
12

---

Published as a conference paper at ICLR 2023
Appendix:
Explicit Box Detection Unifies End-to-Ena Multi-Person Pose Estimation
In this Appendix, we provide descriptions of detailed experimental setup (in Sec. A more compar- isons under COCO test-dev (in Sec. BB) results of human detection on both COCO and CrowdPose datasets (in Sec. C. more visualization show the effect of explicit box detection and compari- son with PETR (in Sec. D). more discussion for ED-Pose (in Sec. Eb. We also append our code to reproduce the results.
A EXPERIMENT SETUP
Dataset. Our experiments mainly conducted on the popular COCO2017 Keypoint Detection benchmark (Lin et al.] 2014), which contains about 250 person instances with 17 keypoints. We compare with other state-of-the-ard methods methods both the va va12c17 set and test-dev set. To verify the superiority of explicit detection, we also evaluate our approach on the CrowdedPose dataset (Li al. 2019) which is more challenging and includes many crowded and occlusion scenes. It consists of 20K images containing about 80K persons with 14 keypoints. For ablation studies, we report results on the COCO va12017 set. The OKS-based Average Precision (AP) is employed as the main evaluation metric on both datasets.
Implementation details. In the training stage, we augment input images by random crop, random flip, and random resize with the shorter sides in [480, 800) and the longer sides less or equal to 1333 following DETR (Carion et al.) 2020) and PETR (Shi et al., (0222.. To accelerate the early explicit human detection, we use human query denoising training strategy from DN-DETR (Li al. 2022a). We use the Adam (Kingma & Ba 2014 Loshchilov & Hutter, 2017) optimizer with weight decay of 1 10-4 and train our model on Nvidia A100 GPUs with batch size 16 for 60 epochs and 80 epochs on COCO and CrowdPose, respectively. The initial learning rate is 1 x 10 10-4 and is decayed at the 55th epoch and 75th epoch by factor of 0.1 on COCO and CrowdPose, respectively. The channel dimension D is set to 256. The number of layers in Human Detection Decoder and Human-to-Kan Keypooit Detection Decoder are and respectively. In the test stage, the input images are resized to have their shorter sides being 800 and their longer sides less or equal to 1333.
Loss Function. the overall loss function of ED-Pose can be formulated as:
L = Ln + La + Li
(1)
Ln = 1 - H 3(1 GIOU)
(2)
--= = ->a(1 - log (og()))) where Pt if y = 1, Pt = 1- 1-
(3)
LA = w WW - pp--1++0 Ex exp( - P. - P. /2/ 1 )(0; 0)
(4)
Σ 8(v, > 0)
where Ln is for human box regression that contains LI loss and GIOU Rezatofighi et al. 2019)
loss, Le is for human classification that focal loss (Lin et al. 201 7) with a = 0.25, 7 = 2, and LA is for keypoint regression that includes L1 loss and the constrained LI loss-OKS loss (Shi et al.) 2022 H H*| is the LI distance between the predicted human boxes and the ground-truth ones. y E 1 specifies the ground-truth class, and E [0, 1] is the ED-Pose's estimated probability for the class with label y = 1. P P-| is the L1 distance between predicted keypoints inside a human and the ground-truth ones. P. P. is the L1 distance between the i-th predicted keypoint and ground- truth one, v; is the visibility flag of the ground truth, s is the object scale, and k; is a per-keypoint constant that controls falloff.
The loss coefficients μ, B, X, w, 0 are 5, 2, 2, 10,
The Detailed Interactive Learning between Human Detection and Keypoint Detection. The relations from internal nuuman-keypolin internal syypont-keypoingg and external numan-huuman are learned from the self-attention mechanism. As shown in Figure. (c), Human-to-Ke Ko-ypoiin Interac-
13

---

Published as a conference paper at ICLR 2023
tive Attention can be computed as follow:
Qir,k softmax H( + PF(QH,, 1(Q + PF(QU, R + +M) f(Qrx))))
(5) where PE denotes positional encoding, f(-) is linear projection, QH E RRM+MMKKXDD and HH E IBMM+Mx))x4 are human-keypoint content queries and tumaan-keypoin position queries for M human candidates and the corresponding K' keypoints. D is 256 by default. In practice, to implement the three interactive learning processes in a simple way, we use an attention mask block the interactiveness between external keypoint-keypoint Thus, M can be formulated as:
VD
M
O M(i.j) = True
M(t,j) = -00 M(i.j) = False
(6)
where Mi is the location of the attention mask. We use the M to keep internal human-keypoin attention (True), internal syspointtkeypooii attention (True), and external
attention
(True) while avoiding external keypoint-keypoint attention (False).
Inference time. Comparison Methods: All of the methods in Table. 3 are tested on our A100 machine for fair comparison and the detector of top-down methods is tested by MMdetection. In tabel.88 [ the inference time with emphasized bold number is tested by our A100 machine, otherwise, it is from PETR's ' paper using a V100 GPU due to no available source code. 2) Testing Rules: We omit the time for data pre- pre-processing and only measure the time for model forwarding and data post- processing (i.e., grouping operation in bottom-up methods). For bottom-up methods and one-stage methods, we set the batch size to 1. For top-down methods, we set the batch size to for simulating the multi-person situation.
Unique Merit of ED-Pose for Human Detection Pre-training. Thanks to the explicit introduc- tion of human detection, ED-Pose has unique merit compared to other existing two-stage methods and one-stage methods, which can pre-train human detection to improve the detection performance to assist the subsequent pose estimation in fully end-to-end manner. Objects365 (Shao et al., 2019b) is a large-scale detection dataset with over 1.7M annotated images for training and 80, 000 annotated images for validation. We pre-train the human detection of ED-Pose (i.e., backbone, encoder. human detection decoder) on Objects365 for 26 epochs using 64 Nvidia A100 GPUs.
B COMPARISON OF COCO TEST-DEV.
From Table. g our method without dense heatmap loss can significantly outperform all existing one- stage methods, such as DirecetPose (Tian al.) 2019a), FCPose (Mao al. 2021b). Inspose (Shi et al. 2021). CenterNet (Zhou et al.) 2019), and PETR (Shi et al. (022)) To be specific, our proposed ED-Pose has 2.2 AP gains compared with the fully end-to-end method PETR (Shi et al. (0222) with a Re backbone or Swin-L backbone with about 50% inference time reduction.
C RESULTS OF HUMAN DETECTION
Due to the explicit person detection in our methods, we also report the human detection per- formance the COCO va 1017 and CrowdPose test in Figure. For the COCO dataset, ED-Pose with Swin-L backbone gains 7.7 APM and 6.7 APL improvement compared with FasN (Ren et al., 2015). For the CrowdPose dataset, most top-down methods directly use YOLOv3 (Redmon& & Farhadi 2018) pre-trained on COCO trainval to generate the detected human bounding boxes, making their performance even worse than recent bottom-up methods. ED- Pose can provide human detection results with nearly doubling performance on AP and AP, to serve future work. Moreover, Table 10 we study the impact of different decoders (i.e., human detection decoder and human-keypoint detection decoder) for human detection and keypoint detec- tion. First, we discover that the keypoint initialization provided by the human detection decoder has already achieved good results COCO and CrowdPose datasets, which proves the effectiveness of explicit human detection. Second, the keypoint detection introduced via the human- -eman-keypoint detec- tion decoder is also helpful improve large-scale human detection and obtain similar performance
14

---

Published as a conference paper at ICLR 2023
Table 8: Comparisons with state-of-the-art one-stage methods on COCO test-det dataset.
underlined highlights the compared results. Emphasized bold number in inference time is tested by our A100 machine.
Method
Backbone
Loss
AP
AP
AP75
AP
APr
Time [ms]
DirectPose
ResNet-50
R.
62.2
86.4
68.2
56.7
69.8
74
DirectPose
ResNet-101
R.
63.3
86.7
69.4
57.8
71.2
Non E-2-E
FCPuse
ResNet-50
R+HM.
64.3
87.3
71.0
61.6
70.5
68
FCPuse
ResNet-100
R+HM.
65.6
87.9
72.6
62.1
72.3
93
InsPose
ResNet-50
R+HM.
65.4
88.9
71.7
60.2
72.7
78
InsPose
ResNet-101
R+HM
66.3
89.2
73.0
61.2
73.9
100
CenterNet
Hourglass
R.
63.0
86.8
69.6
58.9
70.4
160
Fully E-2-E
PETR PETR
ResNet-50
R+HM.
67.6
89.8
75.3
61.6
76.0
105
Swin-I.
R+HM.
70.5
91.5
78.7
65.2
78.0
206
ED-Pose
ResNet-50
R.
69.81
90.2
77.2
64.3
77.4
57
551.4%
ED-Pose
Swin-L
R.
72.7
92.3
80.9
67.6
80.0
88
88..7.35
The inference time without Emphasized bold number from PETR paper because there is no public code replication.
on medium-scale human detection. Different from previous observations of severe optimization conflicts between human detection and keypoint detection, ED-Pose unifies the contextual learning between human-level and keypoint-level information and gains benefits from each other.
Table The Average Precision comparisons of the commonly used human detection methods (e.g., FaNter-RCNN and YOLOv3) with ours COCO (the left) and CrowdPose (the right). Notably, We only compare APas (medium object) and AP, (large object) here as small objects in the two datasets not labeled with keypoints. ED-Pose can provide better human detection to serve future work, especially for top-down and one-stage methods
Methods
AP
AP1
Methods
APM
AP1
Faster-RCNN
63.3
74.5
YOLOv3
37.4
46.5
ED-Pose (ResNet-50)
65.9
77.6
ED-Pose (ResNet-50)
58.0
ED-Pose (Swin-L)
69.7
80.2
ED-Pose (Swin-L)
61.8
78.4
ED-Pose (Swin-L*)
71.0
81.2
ED-Pose (Swin-L**)
66.8
82.2
Table 10: The performance comparisons of results from different decoders (i.e., human detection
decoder and human-keypoint detection decoder) on both COCO and CrowdPose datasets with the Swin-L as the backbone.
Method
Dataset
Decoder
APM M (Human)
AP,, (Human)
AP (Keypoint)
ED-Pose
COCO
Human
79.6
70.4
ED-Pose
COCO
Human Det Heman--eyooin Det.
80.1
74.3
ED-Pose ED-Pose
CrowdPose
Human Det.
61.7
76.9
68.0
CrowdPose
Human Det.+Human- Keepponn Det.
61.6
78.4
73.1
D QUALITATIVE RESULTS
Qualitative ablations for explicit human detection: In the main article, verify the effectiveness of explicit human detection for convergence speed and precision of the keypoint detection. Here, present the corresponding qualitative results on the CrowdPose test set shown in Fig. 6 In general, explicit human detection has several advantages: 1). It can provide global human-lever information, enabling the model be aware of flipping easily like the case in the first row. 2). It can give clear human identity to the crowded scene to avoid the keypoint mismatching across persons, such as the case the second row. 3) friendly for small human pose estimation as it can provide precise prior of human box position, e.g., the case in the third row.
Qualitative comparisons between PETR and ED-Pose: We present the visualization comparisons of ED-Pose and PETR (Shi et al., 2022) on the COCO dataset, as shown in Fig 7 From the first explicit detection
row, ED-Pose can pay attention to the flipping issue due to the introduction feature propagation. The second further refleets that
and the realization of Luman--kepooin explicit detection in ED-Pose can make the model aware of the human position when conducting pose estimation, rather than the chaotic and unconscious query full image like PETR. All
15

---

Published as a conference paper at ICLR 2023
possible detected keypoints from PETR have low scores, making it hard to distinguish the right and wrong results. The third row shows that the enhancement the global-local interactivity could relieve the estimation problem of the hard pose with heavy occlusions.
E MORE DISCUSSION
Discussion for Additional Related Works. (1) N. xue (Xue et al. 2022) focuses improving the performance of the heatmap-based bottom-up method by learning the local-global contextual adaptation without any explicit box detection.
(2) Different from the one-stage method, D. Wang (Wang & Zhang, 2022) decouples persons into multiple instance-aware feature maps to obtain the keypoints without any explicit box detection, and they also use heatmap-based supervision.
Different from their works, ED-Pose aims to utilize two explicit box detection processes with a unified box representation and light LI losses. It abandons all post-procsssings in fully end-to- end manner. In terms of performance, ED-Pose surpasses them by margin and even outperforms heatmap-based Top-down methods under the same backbone.
The More Detailed Advantage of ED-Pose. From the results, ED-Pose is efficient and effective even with the same backbone compared with previous works, indicating really helps with pose estimation. From the method with two box detection processes, we have admitted it is inspired by recent DETR-based methods. However, the motivation and usage have obvious differences due to the task differences between object detection and multi-person pose estimation. Multi-person pose estimation focuses on either human-level (global) or keypoint-level (local) information. ED- Pose re-considers this task two explicit box detection processes with unified representation and regression supervision. Moreover, we also compare with other DETR-based pose estimators, and ED-Pose shows its superiorities in both efficiency and performance. The first human box detection can extract global features and provide good initialization for the latter keypoint detection, making the training process converge fast. The second keypoint box detection can bring in local contextual information near keypoints. However, previous DETR-based pose estimation methods only directly regressed the keypoints' 2D coordinates, we argue that the keypoint representation as point lacks local contextual information. They directly regress local keypoints without global feature extraction, making the convergence slow and inaccurate estimation. In Figure. Gand Figure. 7 we demonstrate the qualitative comparisons of whether not we needed to use explicit detection and PETR with ours. Lastly, we conduct comprchensive experiments on how explicit detection works Table, and Table. [
16

---

Published as a conference paper at ICLR 2023
GT
AP-100 100
26.
Flip
Identity
Confusion
GT
65-65.0
AP-O
dentiss
Miss
(a) Ground Truth
(b) Explicit Detection
(c) w/o Explicit Detection
Figure 6: Visualization results of the ablation study for explicit human detection on the CrowdPose dataset. Notably, we ignore the keypoint box compare the keypoint localization the settings clearly. The white circle is the failure part with description of the type of failure.
AP-20.3
AP-75.4
Flip
GT
AP-O
AP 80.0
Unconscious
Localizaion
GT
AP=1
AP=1
Keypoin
Miss
(a) Ground Truth
(b) PETR
(c) ED-Pose
Figure 7: Visualization results of PETR and ED-Pose on COCO dataset. Both methods are based on ResNet-50 as the backbone. Notably, there are several candidates in the PETR's result at the second line as their classification scores are close and relatively low.
17


Reviews:
[{"review_id": "tuL-gSpSuWN", "reviewer": "Reviewer_Dxnp", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a method for multi-person pose estimation method, which integrates human detection and human pose estimation into an end-to-end framework. The first part of this framework is a human detector based on deformable DETR and DAB-DETR, and outputs human-instance features and estimated human bounding boxes.  The second part, i.e. keypoint detector, is fed with a human-instance query and a set of keypoint queries which are initialized with its corresponding human-instance query. By doing so, the queries of keypoints belonging to one human instance can interact with each other while each keypoint query can also interact with its human-instance query. The keypoint locations are regressed by the keypoint detector. Experiments are performed on multiple datasets. the proposed method significantly outperforms one-stage SOTAs. When compared with two-stage methods, the proposed method performs better than bottom-up SOTAs, while performs comparably or worse than top-down SOTAs but with better efficiency and more succinct pipeline.  Besides, the ablation study also verifies the effectiveness of some proposed algorithm components.", "summary_of_the_review": "The proposed one-stage end-to-end framework for multi-person pose estimation is succinct and efficient. The accuracy is also promising. The effective combinations of human detection and keypoint detection, global and local information are a novel extension of existing DETR architectures. ", "strength_and_weaknesses": "Strength:  \n1. Paper is written clearly.\n2. Global and local information are  reasonably combined by query initialization and human-to-keypoint query expansion.\n3. Human detection and keypoint detection are integrated into an end-to-end and succinct framework. each part of this framework can benefit each other during training.\n4. The proposed unified framework achieves promising accuracy and good efficiency.\n\nWeaknesses:\nThe whole model architecture can be considered as the variant of  DETR", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "rrSWfdxIma", "reviewer": "Reviewer_EVzH", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper presents an end-to-end approach for multi-person pose estimation. It unifies human detection and keypoint estimation in an one-stage framework. Based on transformer, human tokens and keypoint tokens are introduced and interacted to generate discriminate representations for human detection and keypoint estimation. The proposed aporoach achives state-of-the-art performance on COCO and CrowdPose datasets. Extensive experiments demonstrate the effectiveness of the the components in the proposed approach. ", "summary_of_the_review": "Overall, the paper is well written. The novelty and contributions of the paper are sufficient.", "strength_and_weaknesses": "Strengths:\n1. The proposed approach introduces human tokens to interact with keypoint tokens to resolve the ambiguous association between keypoints and humans in the transformer framework.The approach is reasonable and novel. Explicit human detection contributes significant performance improvement in the proposed framework. The interaction between human and keypoint tokens further improves the performance. \n\n2. The representation of (x, y, w, h) instead of (x, y) for keypoints provides more contextual information for keypoint estimation as demonstrated in Table 5.\n\n3. The proposed approach achieves state-of-the-art performance on COCO and CrowdPose datasets with competing inference speed as shown in Tables 2 and 3. The proposed approach also has faster convergence speed than competing methods as shown in Figure 5.  \n\nWeaknesses:\nSome implementation details are missing: \n(1) The detailed losses are not presented in neither the paper nor the supplementary material. \n(2) Some details of the network structures are not provided (e.g. how many human detection decoders and human-to-keypoint detection decoders are used?)\n ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "mm1pzu-xeU", "reviewer": "Reviewer_vkvL", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "  This paper proposes a large-scale Chinese cross-modal benchmark where one pre-training dataset and five fine-tuning datasets are involved. The new benchmark serves a significant contribution for the community. The paper proposes Global Contrastive Pre-Ranking and Fine-Grained Ranking to explore the matching relationship between image and text pairs. Besides, the paper employ distillation to overcome the problem of noisy labels.", "summary_of_the_review": "Taking both the contributions from both technique and dataset, I recommend a borderline.", "strength_and_weaknesses": "Strength: \n1. A new large-scale Chinese cross-modal benchmark which makes great contribution to the community.\n2. Different schemes are designed to learn better feature representations and overcome the problems of noisy labels.\n3. The paper is well written and easy to read.\nWeaknesses:\n1. There is no substantive modification of the ranking algorithm. These two losses are not novel somehow compared with previous contrastive loss.\n2. The distillation loss is not novel both on the target level and feature level. Some technologies such as \" adopt soft targets generated by momentum-updated encoders\" are common and there are no essential progresses over published approaches.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "RZylu_vgvXc", "reviewer": "Reviewer_uhwo", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a method of casting the problem of multi-person pose estimation as a two-level (human and keypoints) Explicit-box Detection problem. It is built on the DETR framework and its variants such as the Deformable DETR and DN-DETR.  The human detection component is a direct application of the existing DETR variant. The proposed human-to-keypoint decoder with the box-based keypoint formulation is the novel component. The proposed method is end-to-end trainable without resorting to post-processing, thanks to the DETR framework. It obtains better accuracy performance than the prior art on the COCO keypoint benchmark and the CrowdPose benchmark.  ", "summary_of_the_review": "Overall, this is a good paper on multi-person pose estimation. It leverages the strong DETR framework with a new human-to-keypoint component.  The reviewer would like to see the rebuttal on the aforementioned weaknesses. ", "strength_and_weaknesses": "+ The proposed method takes advantage of the fully end-to-end DETR framework for pose estimation in a straightforward yet effective way.\n+ The proposed human-to-keypoint decoder  with the box-based keypoint formulation seems to be effective. \n+ The proposed method obtains strong performance on two challenging datasets. \n\n- Some details of the box-based keypoint formulation are missing. How are those bounding boxes determined for different keypoints of different persons in training to compute the keypoint box regression loss, L_k^l? Is the same method used in both the COCO and CrowdPose datasets? If not, how is the dataset prior (e.g., crowd vs not-so crowd) leveraged?  And, how sensitive is the proposed method to the keypoint boxes?\n\n- The interactive learning part is not clearly presented. How is the human-to-human interaction computed, as well as the human-keypoint and keypoint-keypoint? \n\n- Two references that seem to be relevant are not discussed.  1) N. Xue et al Learning Local-Global Contextual Adaptation for Multi-Person Pose Estimation, CVPR22; 2) D. Wang and S. Zhang, Contextual Instance Decoupling for Robust Multi-Person Pose Estimation, CVPR22. \n\n- It may be interesting to test the COCO or CrowdPose trained model on the OCHuman dataset to see the transferability as done in 1) N. Xue's paper. It will also enable better understanding the impacts of the selection of the keypoint boxes. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "PvdwJou0g4u", "reviewer": "Reviewer_h7oS", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In summary, to deal with the task of multi-person pose estimation, this paper regards the task as two explicit box detection processes. This enables the model to be optimized in an end-to-end manner and achieves good result.", "summary_of_the_review": "Overall, my main concern w.r.t. this submission is on the proposed explicit box detection process (see the Strength And Weaknesses for more details). Specifically, I think there are some things that should be made clearer before the acceptance of this submission.", "strength_and_weaknesses": "Strength:\n1. The paper gives a clear summary and analysis of existing methods, which motivates their method well.\n2. Many experiments have been conducted and the results seem satisfactory.\n\nWeakness:\n1. My main concern with this paper is the proposed explicit box detection process. Specifically, I think a clearer definition of what is the so-called explicit box detection process should be given more precisely. In the current version, it seems that I can only guess what it means, which brings me reading difficulties. From my guess, does this explicit box detection process just mean to explicitly detect bounding boxes for both the human and the keypoints? I personally use this understanding to understand this paper and I am sorry if I misunderstand it. \n2. Then with such an understanding, I think the advantage of such a process should be mentioned in more detail. Only demonstrating the advantage of such a process through experiments seems to be not enough to me. Besides, I am not sure if this process has improved the results because it really helps with pose estimation, or because it is more relevant to the task of object detection and therefore fits DETR and other existing object detection methods better. Can the authors also explain this?\n3. I also have some other minor worries w.r.t. the experiments. (1) I am not sure whether the authors overclaim their improvement a bit as it seems a large proportion of their improvement on Swin-L (1.5 -> 5.0 on crowdpose and 1.2 -> 2.7 on coco) is because they use an extra dataset Objects365. (2) I am not sure if it is allowed to put the result on COCO-test dev in the appendix only, as I am not sure whether this should be considered as a new dataset or not.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster