Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: The paper targets navigation and obstacle avoidance using high-frequency event cameras. A new event VAE (eVAE) is proposed to encode inputs from event cameras. An unsupervised learning scheme for eVAE is also proposed. The encoder is first evaluated qualitatively under variations of the input data. The pre-trained eVAE is then used to learn RL navigation policies. Results favorably compare to alternative encoding methods.
main_review: Pros:
- Learning efficient visual representations for RL policies is an interesting and open problem
- Learning visiomotor policies for evet cameras should be interesting for many applications
- The paper is well-written and is easy to follow in general. 
- The proposed eVAE encoding is evaluated and compared to baselines under variations of observed scenes and used parameters, e.g. length of event sequences 
- The eVAE-based RL policy is tested for textures and shapes unseen during training and while changing the sparsity of event data 
- Results are convincing given the comparison to baselines.

Cons:
- The definition of terms in eq.(2) is missing. The exact form of the loss for learning eVAE is hence unclear. What are Q,z,x in (2)?
- The robot seems to be always moving forward. What will happen if gets to a trap with no free passage for moving forward?
- Results for RBP-xy are similar to RBP. Does this mean that "e" part in eVAE is not important? Is the contribution of eVAE validated in this case?   
- I have not found a commitment by the authors to release the experimental setup and environments. Such a release will be crucial for future comparisons. 
limitations_and_societal_impact: No specific issues other than standard for vision and robotics
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The author proposed an event based VAE for high speed robot navigation useful for reinforcement learning. They then compare to other image based RL approaches in AirSim and show they outperform other event based RL baselines. They demonstrate that the representation is useful for training visuomotor policies. They show that it reaches SOTA in event based reinforcement learning on the given tasks.
main_review: # Originality
The paper proposed an event based VAE and uses it for downstream reinforcement learning. The paper focuses on using high sampling rate data streams which is an important and not very often studied problem in RL (particularly due to the amount it slows down training in simulators like Airsim. 

# Quality
The paper provides a compelling argument that their technique performs better on high sampling rates particularly through the graphs in figure 4. They demonstrate that the paper reached SOTA on the obstacle avoidance task. I also appreciated the detailed discussion section explaining the benefits of the approach in regards to generalization. I also am glad that they examine the full event based representation and the simplified xy version and found different performance between the two. I don't see any issues with the methodology, but the RL problem is rather simplistic as the authors note. Especially given the small size of dataset and latent space, I am concerned about whether it would generalize to real world environments. However, this paper demonstrates that the method is more robust than RGB inputs, and therefore this is a promising first step.

# Clarity
I appreciate figure 3 for providing good qualitative visualization of the event based autoencoder, but I would recommend making the text larger in the final version. The impact of figure 4 is reduced due to the difficult of reading it. The paper text, itself is very well written and understandable. The supplemental material aided in the understanding of the paper as well so much so that parts of it (like an example of the environment should have been included in the main paper).

Would the authors clarify what they mean by: "Our RL problem also focuses on a simpler setting due to computational issues in event simulation" ? Is that saying that event based policies are computationally expensive? Or that the high framerate simulation required for training them is the main bottleneck? Specifying which computational issues are the constraints here, (speed, memory, gpu memory, physics simulation wall clock, graphics rendering, inference or training bottleneck in one of the network architectures etc.) is important for future authors who wish to build upon the approach. 

The main issue in clarity is that the simulation environment is not shown qualitatively in the main paper which made the problem setup difficult to visualize. 

# Significance
This representation could prove useful for drone in particular due to their high speed, but also is applicable to other high speed robotics. Additionally, the representation learned seems to have potential for improving sim2real transfer due to the robustness to shape and texture within simulation as shown in the appendix as well. They also demonstrate the importance of high frequency control in how all techniques struggle at a lower frequency of samples at 45hz.

I will admit that the paper does a good job proposing an interesting observation space that may generalize better for sim2real tasks as the paper demonstrates. With the changes to paper clarity and depending on the discussion of the computational bottlenecks, I would be willing to raise my score.

Edit: I would also like to apologize to the authors that my original review was truncated. I have edited my review.

Post Rebuttal:
Thank you for the authors detailed rebuttal. Ultimately, the rebuttal did not sufficiently address my concerns to raise my score beyond a 6.
limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The authors present an event-based variational auto-encoder (eVAE), which learns a latent space representation that is subsequently used for reinforcement learning (RL). The RL task consists of a simulated drone that has to avoid obstacles while flying forward in AirSim. Although the authors themselves do not mention this as a contribution in the paper, I think that the Event Context Network (ECN) they introduce is actually a major contribution of this article. In the ECN they combine an EventNet-like architecture with a Transformer-like position encoding (here temporal encoding). This ECN is a core part of the eVAE and works better than the original EventNet-way of encoding temporal order (as evidenced in the appendix). Finally, the authors show that with the eVAE, a learned policy transfers better to different environments.


main_review: The proposed ECN and eVAE will be a valuable contribution to the construction of event-based latency spaces for - eventually - use on embedded robots. In terms of originality, the proposed method is primarily a very skillful combination of techniques found in the literature, but that is fine in my opinion. 

The article is (apart from a few minor points) clear and well-written. However, I would advise the authors to include the new ECN as a contribution. This would also make the comparison with the original EventNet more important, which I would then suggest to also make part of the main article. I know that due to the page limits this may be challenging, but it could be added to, e.g., Fig 3 if the other panels there show 1 less example in order to make room for the EventNet comparison. On the topic of Figures, Fig 4 has super small legends, which are impossible to read. 

The main criticism I would have on the article is related to the simplicity of the dataset (also mentioned by the authors under limitations). A very small latent space is learned (8-dimensional) and the examples shown by the authors that illustrate the superiority of the method over other methods mostly show that given a few events, the latent space will interpret an image still like the most likely pose of a pole or gate. The real world is obviously more complex and it would be interesting to see the eVAE in action for a real-world event-camera dataset (potentially with a larger latent space). This is even more relevant since the authors simulate events based on image differencing. Is there a study in the literature on how good this model is compared to the real outputs of an event-based camera?

This criticism extends to the used simulator and simulated environments, which is far cry from real autonomous drone flight. However, I do think it is valid as a first step, given the goals of the article. Moreover, as the authors mention themselves, event-based camera inputs will generalize better to different settings (including possibly the real world) in comparison to RGB values. The experiments in simulation already testify to this.

Smaller remarks:
- "Event camera data was also used to power a closed-loop control scheme for a UAV in flight by tracking roll angles and angular velocities in [38]." --> these experiments were not in flight, but with a fixed bench setup.
- For Eq 1 I had to go back to the Transformer paper. i and d are not (well) explained in the text, as only D was mentioned.
- Figure 2: how does the MLP at the start compare to the one in PointNet / EventNet (PointNet's is much more extensive I have the idea)
- Figure 3c: "eVAE representations stay consistent for different lengths of event sequences." What lengths? (I see three panels, but no indication of what the lengths were)
- "We note that by encoding the arrangement of polarities, the latent space implicitly captures direction of motion, " --> This is only true if you know the fore- & background color, right? Else there is ambiguity...
- Figure 4b: at 100Hz the EIP works best. Is there any explanation for this?
- The main article should have a picture of the simulated environment for the obstacle avoidance.

POST-REBUTTAL:

After reading all reviews & rebuttals, and the responses to my own comments, my score has not changed. If the authors perform the modifications they mention in the rebuttal, I think this will be a valuable contribution to the field.





limitations_and_societal_impact: The authors have addressed both social impact and limitations of the work.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes a method for learning representations of event-based camera percepts which can be used for reinforcement learning. The method is able to digest irregular, streaming camera information using a VAE conditioned on a context vector computed from event data using a PointNet style network. The empirical evaluation shows that using this pretrained representation for PPO is more performant and generalizable than frame-based methods on a simulated UAV obstacle avoidance task. 
main_review: Originality: 

The proposed eVAE method appears to be a novel approach to representation learning from this modality. The empirical results, particularly the generalization results, are strong compared to existing methods (of which there are not too many, as this is a relatively new direction of research). 

Quality:

The experimental results provide reasonable support for the author’s claims, but are limited in scope by just one evaluation environment. The comparisons are appropriate and the proposed method achieves strong results. 

My concerns in this section are the following:

The results presented in Fig 4A do not have error bars, could they be trained with multiple random seeds and added?

When training on one scene and testing on others, it seems like generalization occurs because unseen states are mapped to the closest possible latent. However, it’s not clear that this would result in a policy which actually reacts to the new obstacle shape. For example, if the cones were upside-down and had a significantly different shape as the poles, but got mapped to poles through the latent space, would the UAV be able to avoid them? This seems unlikely if trained using only one scene, but maybe if it were trained using a greater number of (or more diverse) environments, this would work?

Clarity:

Overall, the paper is well written. The authors have provided and documented code to make this work as reproducible as possible.
It could help improve clarity to address the following questions:

In Fig 4a, does the eVAE pretraining lead to improved asymptotic performance compared to the EIP policy? If the EIP policy is allowed to train for the same amount of effective gradient steps as the VAE training + RL training, will it achieve the same performance (or better, since it can learn features with the RL objective directly)? 

It doesn’t appear that including the temporal or polarity information improves the performance of the learned policies. Do you have any explanation regarding why this information appears discardable for this task? Are there other tasks where it might not be? 

Additionally, it could help to add some justification for the timestamps of a particular event window to be normalized to 1. Wouldn’t this cause event windows of differing lengths (in seconds) to become encoded to the same vector?

While there is some runtime analysis in the appendix, It would be interesting to compare the computational effort of the eVAE-based policy to the EIP-based policy in wall-clock time.

A brief description about how the event image is formed from the events would be helpful for understanding the VAE training process. How is the polarity involved in the computation and how is the relative time scaling performed?

Nitpicks:

The dotted line in Fig 4a is unlabeled. The actual values of rewards in the environment have been omitted from the main text, so it is unclear how to interpret them.

Significance:  

As event based cameras could lead to the development of higher frequency control policies which are critical for certain types of tasks, the research question of how to best utilize this variable length asynchronous data stream is quite important.

Recommendation: Although the empirical evaluation could be more thorough in terms of tasks, environments, and more experimental trials, the eVAE method proposed is novel and appears effective, and seems like it will be quite impactful for future work using RL for control from event-based cameras. So, I think this is a good contribution and vote to accept.
limitations_and_societal_impact: The method is tested only in one domain (simulated quadrotor flying, using the poles data). While the authors explain in Section 6 that this work is an initial exploration, it could make the paper much stronger to be able to demonstrate it with another task or more visually diverse scenes.
review_title: null

Meta-review: This paper introduces a new type of variational auto-encoders that can handle streams of high-frequency event-based spatiotemporal data (t, x, y, p) in order to decode dense images from a small set of N events, then combines a pre-trained eVAE with traditional reinforcement learning (PPO) to learn visuomotor policies to control (and fly while avoiding obstacles) a quadcopter in the AirSim simulator.
Reviewers have praised the idea, the application of RL to event-based data streams, the writing of the paper and the extent of experiment and comparison to non-event-based VAE + RL. Reviewers had some questions about equations, about specifics of some ablations, about rendering time for event-based simulation, about showing the simulator, etc., that were all satisfactorily answered by the authors.
After careful consideration of the paper and given review scores of (6, 6, 7, 7, mean 6.5), it appears that the paper should be accepted. The authors are invited to carry out all the changes they promised to the reviewers, including open-sourcing their code.

Decision: Accept (Spotlight)