Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a spatial temporal transformer deep network to tackle the egocentric pose estimation problem. The motivation of such a scheme is that it can help improve pose estimation when there are occlusion and large distortion.  When there is occlusion at a moment, hopefully the body part is visible at another time. The temporal pattern also helps disambiguate body poses.  The proposed method uses a resnet to extract local features. Combined with a learnable token map, they are sent to a transformer to aggregate space and time information. The transformer also uses a learnable spatial encoding. The output of the transformer than passes through a deconv layer to reconstruct a body joint heatmap. A fully connected network is then used to estimate the final 3D body pose.  Experiments show the proposed method give better results than different competing methods.
strength_and_weaknesses: Strength:
   + The proposed method uses spatial and temporal information to improve the results. The Transformer is a natural choice for such a scheme.
   + Different tricks may contribute to the overall improvement: the sequence input, the transformer, the learnable space encoding, the learnable token. 

Weakness:
   -  The paper never talks about how long the input sequence needs to be. The longer should be better but it also introduces long delay, which makes the proposed method not useful for real-time applications. 
  -   The writing is mostly clear. However, the details of the network are not completely included. This makes it hard to replicate the work. Most of the parameters used in the paper are not instantiated.
 -   The paper does not give procedure to train the network.
clarity,_quality,_novelty_and_reproducibility: The paper is most clear.  But, most of the parameters used in the paper are not instantiated. The paper does not give procedure to train the network. This makes it hard to replicate the work. Most of the component modules are well known. However, the combination of these components for the application of egocentric pose estimation is new. The overall approach is interesting. 
summary_of_the_review: This paper proposes a transformer based method which greatly improves the egopose estimation result. The proposed method is interesting. One concern is how this method can be used for real time applications which requires a short delay if a long input sequence is needed to estimate pose at each time instant.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper introduces a novel approach to the problem of human pose transfer, based on Vision Transformers (ViT). The authors propose a system with two encoding and one decoding Vision Transformers. The decoder consists of two branches, a warping branch that predicts the optical flow for warping the source image to the target pose, as well as a generation branch. Both branches possess a cross-attention module which has been shown to increase the quality of results. A mutual learning loss forces the two branches to be consistent with each other, in terms of intermediate feature map prediction. At the back end of the network, a fusion block combines the warped and generated images to form the final output. Finally, the authors provide both qualitative and quantitative comparisons against SOTA methods, along with an ablation study, which demonstrate the merits of their methodology.
strength_and_weaknesses: Strengths: The methodology is sound and the design choice of Vision Transformers is an interesting approach to the problem. The experiments, comparisons with SOTA and ablation studies are comprehensive and indicate a noteworthy improvement compared to previous works.

Weaknesses: The most profound shortcoming of the paper is the quality of the text. In my opinion, the Abstract, Introduction and Related Work sections require a significant revision, with the assistance of a native speaker if possible. I would strongly advise the authors to revisit these sections and improve their text. In addition, I would recommend trimming down the Abstract, as it is very long.
clarity,_quality,_novelty_and_reproducibility: Aside from the poor text quality in some parts of the paper, the methodology is clear and straightforward and the experimental results support the claims made by the authors. Moreover, the model appears to be reproducible, if the authors provide some more details of the discriminator network and the optimisation algorithm.
summary_of_the_review: The paper presents an interesting method for human pose transfer with Vision Transformers and shows promising results. My main concern has to do with the quality of the text, especially in the first two sections. Considering the above, I would suggest to accept the paper with some reservations, as the authors should revisit their text.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes Human MotionFormer, which utilizes convolutional layers and cross attentions for local and global matching. The network could achieve both warping and generation, posing the source image to the target pose.
strength_and_weaknesses: Strength: The two branches (generation+warping) seem reasonable and could benefit each other (ablation results).

Weaknesses:
1. The writing of this paper could be improved. The abstract and introduction section fails to address key challenges in the field and fall into trivial details.
2. The technical novelty is limited. The major contribution lies in the transformer decoder blocks (Sec. 3.2), and the techniques are not very novel in this task.
3. Lack of clarification. The target pose is used to guide the source image, but no discussion is provided on how the target pose is extracted, represented, and how its quality affects the final results. What if the target pose and source image have large structure variations? Is there a mechanism like [Chan et al. (2019);] to align them?



clarity,_quality,_novelty_and_reproducibility: The technical writing and presentation need significant improvement. Multiple grammar and formatting mistakes can be found in the manuscript. Besides, the presentation falls into trivial details and lacks an in-depth discussion of the problem. 
summary_of_the_review: This paper is not ready for publication at ICLR at this stage due to limited novelty and writing quality.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: A motion retargeting method based on vision transformer is proposed in this paper. This task is formulated into the pattern matching problem where global and local search are required in one human image. Considering the local perception of conv layer and global perception of cross attentions, this paper combines the advantage of both. The global and local perception are leveraged into both warping and generation branches of the decoder. This design brings higher quality warping and generation results, and finally produces higher quality motion retargeted human images. 
strength_and_weaknesses: + Global and local perception design within both warping and generation branches for high quality motion retargeting synthesis. A combination of global perception via cross-attentions (i.e., vision transformers) and local perception via conv layer (i.e., CNN) seems intuitive and demonstrated effective in practice. 
+ The design of mutual learning loss combines the advantage of cross-attention and convolutions to enhance the warping and generation completeness. This is illustrated important to enable two branches work with each other to produce correlated visual contents.
+ High quality results shown in the experiments and supplement video.

clarity,_quality,_novelty_and_reproducibility: The writing is clear, technical quality is good, the proposed method is somewhat novel and the code is provided as well.
summary_of_the_review: Overall, this is a nice framework combining vision transformers and conv layers for global and local matching to synthesize high quality motion retargeting results. Experiments have shown the effectiveness of the proposed method upon prior arts.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper proposes a simple but effective transformer-based model for motion synthesis, i.e. transferring motions from a target dynamic person to a source static person. The ViT based architecture leverages cross-attention in combination with convolution layers to build the correct global and local correspondence between the source person and a given pose of the target person. 

While acknowledging that the proposed model is interesting, intuitive, and effective in practice, and the experimental evaluations are comprehensive, the reviewers have raised concerns related to 1) presentation clarity (Reviewer Epkc), 2) limited technical novelty (Reviewer 6Q61), and 3) imperfections due to reflection (Reviewer 6Q61). (1) and (2) have been improved and clarified to some extend in the rebuttal. Among these, (3) did not have a substantial impact on the decision, but would be helpful to address in a subsequent revision. 
The review posted by mistake by Reviewer m6aS was not taken into account in the final recommendation. 

After further discussions with Senior AC, our final recommendation is acceptance. We urge the authors to further improve presentation clarity, i.e. to include clarifications of the rebuttal into the manuscript, highlighting the responses to the main concerns in the main paper. 

justification_for_why_not_higher_score: Paper with a wide spread of the scores and a strongly negative review.
Decision: Accept: poster