Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper presents a method for computing Bird's Eye View (BEV) occupancy maps. Unlike methods that explicitly use geometric information or that compare all pairs of pixels, they use an encoder-decoder transformer model that contains a number of design decisions that allow it to be efficient and accurate. They fuse camera rays and what they term "semantic features" from a CNN. The rays incorporate geometric information by applying an MLP to the direction and camera center. Their model uses a fixed-size latent representation inspired by general-purpose architectures like the Perceiver. This avoids the quadratic penalty incurred by naive models. They evaluate on nuScenes vehicle segmentation, where they outperform several recent works. Through ablations, they find that the different aspects of the geometry and query embeddings improve results. 
strengths_and_weaknesses: Strengths:
- Their approach is simple. The fixed-size latent is a nice way of addressing the quadratic complexity incurred by prior work.
- The ablation experiments validate the different components of their approach.
- The approach outperforms previous work such as CVT and Lift-splat.
- They seem to have made an attempt to control for differences in resolution and other experimental details (although I'm not an expert on this benchmark and other reviewers may be better-suited to evaluating this). 

Weaknesses:
- The evaluation is a bit limited, covering only one task (vehicle segmentation). It would be helpful to expand this to other tasks considered in prior work (eg. the other comparisons in CVT [17]).
- I found the description of the latent representation (Section 3.2) to be sparse on details, although it is described more precisely in the supplemental material. It would be helpful to provide slightly more detail in the main text, given that this is one of the main sources of novelty.
- The approach is somewhat incremental. Most of the improvement comes from fairly well-known architectural changes.
- The conclusions in the attention study (Section 4.3) seem slightly overstated (eg. the claim "proving the network is able to retrieve the pixel relationships between views"). This study does not seem to have been conducted very systematically, and consists mostly of qualitative results. 

robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: Overall, I'm borderline on the paper but lean towards acceptance. While the approach is incremental, the architecture's design decisions seem reasonable, and this is an interesting application of a fixed-size latent space. The experiments suggest the model works well, though I will defer to other reviewers in assessing the soundness of the experimental design.
issues: If time permits, I would be interested in seeing how the model performs on other BEV prediction tasks.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: This paper a method for vehicle segmentation in BEV space for use in autonomous driving. The system takes in images from multiple cameras, feeds them through a CNN to extract features, and then combines the features with ray embeddings (a representation of the ray that captured each pixel). The ray embeddings encode the spatial relationship between the cameras. The fused representation goes into a transformer, and outputs a BEV map which is further refined using a CNN. The authors show the method outperforms baselines, and also conduct several interesting ablation studies justifying various design choices.
strengths_and_weaknesses: Strengths:
- The latent representation is less computationally expensive compared to methods that map densely between input image pixels and BEV pixels
- Does not require BEV projection, which can lead to compounding errors
- The model does not require a hardcoded BEV resolution or geometric assumptions
- Interesting analysis showing that the attention heads of the learned model cover many directions

Weaknesses:
- The performance of the system does not seem ready for use in such a safety critical application area

robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: This paper proposes a novel method for BEV vehicle segmentation that outperforms other methods with a simpler and less computationally intensive approach. Also, the use of ray embeddings in this system seem likely to be of interest for other systems tackling this problem.
issues: - In multiple places, need to change 32Go to 32Gb
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper proposes an efficient transformer-based encoder-decoder model for BEV vehicle semantic segmentation from camera-only inputs. It first encodes the semantic features from input images and camera intrinsic information into a set of latent variables with a system of cross-attention. These latent features are then processed with self-attention blocks and cross-attention with BEV queries to obtain the final semantic segmentation results. Experimental results demonstrate the efficacy and efficiency of the proposed method.
strengths_and_weaknesses: Strengths:
- The paper is well-written and the basic idea is easy to follow.
- Encoding the input images into a set of latent vectors can avoid the significantly increasing computational burden caused by high-resolution PV and BEV features.
- Related works are comprehensively analyzed and compared.
- The qualitative visualization in Sec. 4.3 is interesting.
- Experiments show state-of-the-art results compared to previous works.

Weaknesses:
- This method is general and should be able to be applied to other tasks, such as map segmentation (not only vehicles) and 3D detection. There are more competitive entries and more baselines that can be compared in those benchmarks. More experimental results on those tracks can distinguish this paper from others better and make the conclusion more convincing.
- Questions about the encoding of camera calibration parameters:

a. There is another related work required to be compared and discussed: PETR: Position Embedding Transformation for Multi-View 3D Object Detection. This work also encodes similar information from the input perspective and achieves improvements.

b. How is x formulated in Eq. (1)? Except for the pixel coordinates, the depth should also be considered to make it capable of being transformed to 3D space? And why here disentangle the rotation and translation when constructing such embeddings in Eq. (1) and (2)? It should be more common to use both to directly calibrate all these points of different camera views into a unified 3D space? Here may need more explanations.
robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: This paper proposes an effective framework for BEV vehicle semantic segmentation. It comprehensively analyzes previous works and improves several technical aspects over them. Experimental results can demonstrate its efficacy. Nevertheless, there are still questions in terms of the design details of camera calibration embeddings and more experiments on other relevant competitive benchmarks. I would vote for weak accept temporarily and would consider adjusting the score according to the authors' feedback on my questions.
issues: See weaknesses.
review_title: null

Meta-review: metareview: This paper presents an encoder-decoder transformer model for BEV vehicle semantic segmentation with camera-only inputs.

The technical quality is very good, and clarity and originality are good. 

The strengths are:
- Well-written and ideas easy to follow,
- The proposed approach is quite efficient in addressing common computational issues,
- The ablation study validates the different components,
- There is sufficient comparison with SOTA.


confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Poster)