Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper presents a model, called PoET, for single-image pose estimation. They feed the image to a backbone object detector (Scaled-YOLO-v4), which produces multi-scale features and bounding boxes. An attention-decoder model subsequently processes each detected object, using an architecture based on Deformable DETR. The model then predicts rotation and translation, using geodesic loss and L2 distance. They evaluate the model on the YCB-V dataset, where their method gets better performance than many baselines, including T6d-direct and SilhoNet. Ablations in the supplementary material compare different design decisions, such as different model size and rotation representations. 
strengths_and_weaknesses: Strengths:

- The approach obtains good performance on YCB-V. It outperforms previous approaches that do not have 3D models available. It obtains similar performance to T6d-direction, without 
- The approach is a natural combination of object detection methods and transformers. It is arguably simpler than many of the baselines.

Weaknesses:
- The paper paper is not written very clearly. In particular, the paper does not very clearly explain its contribution, and the ablations are not well organized. 
- Only one qualitative result is shown, and it is in the supplementary material.
- I think that the ablations should be moved to the main paper, and presented more concisely. Given that the contribution is the architecture (and a combination of existing ideas), and the improvement over previous methods is not large, I think that these ablations are very important. It would also be helpful to see these expanded to cover some of the other key contributions, such as the design of the Deformable DETR module (which is one of the main novelties). 
- The approach is a straightforward application of existing methods (object detection followed DETR-like attention and regression). In particular, it's similar to T6D-Direct (which requires a 3D model for its symmetry-aware loss).

robotics_focus: Relevant but unlikely to deploy to hardware in near future
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: While this method gets good results with a simple method, the approach is quite incremental, and the paper is not very clearly written. Given that both the approach and the results are incremental, I would have expected there to be significantly more analysis. I think that this paper is an interesting step towards simpler, transformer-based pose regression, but it would benefit from additional revisions.
issues: As mentioned above, it would be helpful for the authors to provide more explanation for how the the technical approach is a significant advance over T6d-direct, provide additional ablations related to DETR, and describe future writing revisions.
review_title: null

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper proposes a system for object pose estimation from RGB images. The only input to the system is the RGB image, it does not use depth maps or 3D models. The system first uses an object detector to generate feature maps and bounding boxes, which are passed to a transformer module, along with auxiliary information such as box center coordinates. The output of the transformer is then passed to a translation head and a rotation head, which together output a pose for each object present. The method performs comparably to methods that use 3D models such as PoseCNN, and outperforms other RGB-only methods. The authors also demonstrate that their method can be used for camera localization given landmarks of known ground truth poses.
strengths_and_weaknesses: Strengths:
- The approach is RGB-only and does not require depth maps or 3D models of every object, so it is much more accessible as anyone with a regular camera could in theory use it

Weaknesses:
- While object pose estimation methods commonly struggle with accurate orientation prediction, the rotation error for PoET still seems a bit too high to truly be of use in robotic manipulation tasks
- The authors compare with methods that are several years old (PoseCNN and SilhoNet). Are they still the best methods to compare against?
- The method requires a large dataset of labeled images and is unlikely to generalize well to novel objects

robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: This paper proposes an interesting new method for object pose estimation using transformers with a simple and elegant architecture, and the authors show that it outperforms baselines. However, I have reservations about the baselines as well as the usefulness of the method for robotics given the high rotation errors.
issues: See weaknesses above
review_title: null

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper tackles the problem of 6D pose estimation object instances. The method is pretty standard. It is a transformer based architecture that takes in the features at different scales of the object detector as the input along side the positionally encoded information for each detection and predicts rotation and translation for each object. The method is evaluated on YCB-V dataset. It achieves decent (but not SOTA) on pose estimation. 

strengths_and_weaknesses: Weaknesses:
- Why both bbox coordinates and object centers are provided to transformer. What is the intuition behind having bbox info as object codes (as opposed to fixed learned object codes in other works) + providing center as the reference point? How would the results change if for example, object queries are fixed and learned and only reference points are provided?
- Backbone effect: the method takes as input the features of Yolo. I would like to see if the same results hold if the detector is changed or the method is tied to YOLO object detector.
- Loose connection to robotics application: I would like to see an experiment where the proposed method is used to estimate the pose of an object and then robot manipulate the object.

robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: I am more leaning toward the borderline and open to change based on authors' response to my comments.
issues: See weaknesses above.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper presents a framework for single-view, multi-object 6-D pose estimation. The method is based on an object detector backbone, which receives a single RGB image as input, and a modified version of a Deformable DETR Transformer, which is provided with multi-scale features extracted by the object detector, as well as the center coordinates and extent of the bounding-box predicted by the latter. The "object queries" returned by the Transformer are passed through two separate MLP to regress a translation and rotation vector for each detected object. The object detector is finetuned from a pretrained model, while the Transformer and MLPs are trained end-to-end with pose supervision  and receiving ground-truth bounding box and object class at training time. The approach is novel and achieves state-of-the-art results on the YCB-V and LM-O datasets for RGB-only methods. The method is also demonstrated for the localization of a camera against a set of landmarks on scenes from a dataset.
strengths_and_weaknesses: The ideas and the architecture proposed by the paper are mostly novel and provide interesting insights about the use of (Deformable DETR) Transformer networks for multi-object pose estimation. The experiments are thorough and extensive and the comparisons with previous work fair (e.g., all the baselines use synthetic data for augmentation during training, as the proposed method). The approach achieves state-of-the-art on the evaluated datasets for RGB-only methods, and runs at real-time rates. The supplementary material provides a large number of additional results and analyses.

However, a substantial improvement in the quality of the presentation is required. In particular:

- The quality of the visualizations is in my opinion not yet at the level of an international conference: Many writings in the figures and tables have an extremely small font size, and are hard to read even after zooming to 300%. Figure 1, as well as the line plots (Fig. 2, Suppl. Fig. 3. and 5) could be restyled to appear more professional, by, e.g.: making the font and its size uniform across the visual elements and w.r.t. the other figures (and ideally the text), making the colors more distinctive in the line plots for overlapping lines (so as to increase readability), and explaining the meaning of the legend and the other visual elements (e.g., the MLPs in Fig. 1).
- A large amount of typos and formatting issues are currently present (see "Issues -> Typos/formatting").
- Language is at times articulated or inaccurate (see "Issues -> Language suggestions" and partly "Issues -> Points to clarify")
- Occasionally, symbols are used, either in the equations or in Tables, without being introduced in the previous text or caption respectively.

Citations of related work sometimes lack preciseness (e.g., papers cited as examples of a class of method, but not actually matching the features mentioned in the text for these methods). A number of general statements made in the introduction and to provide context for the related work would need to be supported by references.

Finally, while for the "pose sensor" experiments results are shown that could be applied in a robotic scenario for camera localization, no actual deployment on hardware is demonstrated.
robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: Overall, I recommend accepting the paper, given that the proposed ideas are mostly novel, that the task investigated is of concrete relevance for robotics and that the results demonstrated, as well as the running speed of the algorithm, could make the method particularly suitable for real-world robotic applications. However, a non-negligible amount of editing should be performed to address the issues detailed in the "Issues" section.
issues: # Points to clarify
- L39: What does "_detailed_ data _base_" mean exactly? Also, this and the previous statements in the paragraph should be supported by references
- L40: "architecture": Is this referring only to network-based solutions? If so, it should be specified
- "pose sensor" (L52, 62, 299, 320): What exactly is meant by this? "Sensor" commonly refers to a piece of hardware
- L74: [7] does not assume 3D models
- L77: [9] does not perform 6D pose estimation, nor does it use a 3D model
- L91: What does "binning the translation and rotation" mean?
- L142: While represented with $\gamma$, the encoding from NeRF is never referred to in [28] as "gamma encoding"
- Eq. 2-3: The text should mention what R, $\lambda_t$ and $\lambda_{rot}$ are
- L176: Missing meaning of $n_{cls}$​. This part could be clearer in general, e.g., specify that one pose hypothesis is regressed for each class
- L193: How exactly is the match performed?
- Tab. 3: How can the large discrepancy on "scissors" be interpreted?
- L263: PoseCNN -> SilhoNet?
- Eq. 4: Why "W" in the top-left corner of translation only? Why "wo" subscripts in the RHS of translation and "wc" in the LHS?
- L294-295: How can this large discrepancy be interpreted?
- L308: Can this be supported with results?
- Suppl.
  - Sec. 2: Why does RP + Q perform much worse?
  - L67-69: If Jitter is defined in the text as having $\sigma=0.02$, how can it be evaluated for different sigmas in Fig. 1? Clarify that it is different from Tab. 1
  - L83: How can this be supported?
  - L131, Tab. 6: What does GT mean for YOLO/R-CNN? The GT bounding boxes do not come from a network
  - L138: Is Mask R-CNN or Faster R-CNN used?
  - Fig. 5: What does "out" mean?
  - L178: [13] -> [12]?

# Visualization
- Fig. 1: Unusual style: Very thin lines, not clear what the dots in the right part represent. Writings at the bottom are crucial for understanding, but barely visible. Almost all the text (e.g., $c_x, c_y$) is too small to be read without zoom
- Table 2-4, Fig. 2, Suppl.: Fig 3.: Numbers, text and lines are way too small to be read without zoom
- Suppl.
  - Tab. 1: Symbols in the column headers are not defined in the caption. At least a sentence mentioning that these are defined in the text should be inserted
  - Bold should be used in all Tables to highlight results, particularly in Tab. 4-5, in which Baseline does not always achieve the lowest error
- Inconsistent fonts across figures (cf., e.g., Fig. 1 and Suppl. Fig. 1)

# Typos/formatting
- Dashes in multi-word adjectives
  - state of the art -> state-of-the-art: L12, 60, 131, 223, 239. Suppl.: L71, 132, 140, 177, 181, 186, 204, Caption Tab. 6, 9
  - 6 DoF -> 6-DoF: L14
  - RGB only -> RGB-only: L60
  - pass through -> pass-through: L133
  - encoder refined -> encoder-refined: L150
  - ground truth -> ground-truth: L160, 192, 195, 235, 246, 273, 287, 297, Caption Tab. 1. Suppl.: L58, 59, 131, 135, 155, 170, 185, 197, Caption Fig. 1, 4
  - 6 dimensional -> 6-dimensional: L165
  - class specific/agnostic -> class-specific/agnostic: L172, 173. Suppl.: L19
  - real world -> real-world: L261
  - rotation symmetric -> rotation-symmetric: L303, 308
- L31: an -> a
- Missing commas after the following words
  - L36: "image"
  - L49: "framework", "Transformer)"
  - L88: "PoseCNN"
  - L101: Before "dubbed"
  - L111: "symmetries"
  - L123: "architecture"
  - L192: "disregarded"
  - L194: "inference"
  - L251: "translation"
  - L257: "intrinsics"
  - "e.g." -> ", e.g., ": L81, 164. Suppl.: L120
- L37: Missing apostrophe after "objects"
- L38: apart of -> apart from
- L43: What does "it" refer to?
- L46: to the -> into the
- L78: this -> these
- L98: takes [...] applies -> take [...] apply
- L120: section we, -> section, we
- L139: ";" at the end of paragraph -> ", as follows."?
- positional encoded -> positionally-encoded: L142, Suppl.: L109
- Exp. notation: $e$ -> $\textrm{e}$, else it is Euler's number: L169, Suppl.: Caption Fig 6
- L214: Remove comma after "though"
- Missing period at the end of caption: Table 1-3. Suppl.: Table 1-9
- L233: ROI -> ROIs
- L243: reduces -> reduce
- L254: "it" -> "them", or rephrase
- L285: hypothesis -> hypotheses
- Missing venue for ref. [5]
- Ref. [27] and [33]: Replace arXiv with ICLR 2021/2019 resp.
- Format of [32] is not uniform with other references (e.g., URL)
- Suppl.
  - L3: work -> works
  - L17: is -> are
  - Fig. 1: "AuC" -> "AUC" for consistency with text
  - L140: detector -> detectors
  - Caption Tab. 7: perfromance -> performance
  - Caption Fig. 4: Remove "as"
  - L199: PNP -> PnP

# Language suggestions
- L32-33: "Besides using 3D object models [...], they can be" -> "Besides being used for post processing, 3D models can be"
- L89: ". Moreover, [...] employ" -> ", which makes use of"
- L100: "as they" -> "who"
- L129: "attention-based" is redundant. One could rather specify "_vision_ transformer"
- Suppl.
  - L23: "layers," -> "layers;"
  - L216: "to further improve" -> "on further improving"
review_title: null

Meta-review: metareview: Dear Authors,

Thank you for submitting your manuscript to CoRL. I'm happy to inform you that your paper is acceptable for publication. We have completed the review of your manuscript and a summary is appended below. 
The reviewers have advised accepting your manuscript as a poster after improvement of the quality of the manuscript presentation based on the comments.
Please note it is crucial to incorporate all provided explanations from authors to convince the reviewers and recommended editing into the final manuscript.

Regards,
confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Poster)