Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors have proposed a tailored network that extracts information from binary spikes with ST representation based on the differential of spike firing time and spatial information aggregation. In addition, their proposed network utilizes continuous motion clues through joint correlation decoding. 
strengths_and_weaknesses: All in all, this is a very well-structured paper. An extensive literature review has been performed to support the research objectives of the authors. I especially like the detailed outlining of the architecture of the network. 
The comparative results on real scenes with spike and flow as well as photo-realistic high-speed motion are adequately represented, and this demonstrates one of the strengths of this work.

questions: From a technical point of view, there isn't any serious concern regarding the content of this work. I would suggest the authors expand the joint decoding of the correlation section a bit more since it is an important part, but the current status of that section is not very readable to the reviewer. For example, I couldn't get the reasoning behind assigning 0.8 to the decay factor here.
limitations: Compared to previous works, the limitations and impact of the work are adequately outlined and addressed.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The authors propose Spike2Flow, a deep neural network that maps spikes from a spiking camera to optic flow estimates. The method encodes spikes with a differential of spike firing time (DSFT) transform. It achieves results competitive with (better than) the state of the art on a new dataset, real scenes with spikes and flow (RSSF).
strengths_and_weaknesses: Strengths:
* The network focuses on a new type of spiking camera, potentially allowing for very high-speed optic flow estimation.
* The encoding scheme may be useful for other work using spiking cameras.
* The performance of the algorithm seems competitive.

Weaknesses:
* The encoding scheme seems quite straightforward. A spiking camera accumulates illuminance, which means that a bright pixel will have a higher firing rate. The proposed DSFT basically makes a local estimate of this rate, encoding the time between two subsequent spikes. This seems quite a naive method that may be sensitive to noise. Nothing is mentioned about this. 
* As far as I understand, the network is not applied to actual spiking camera inputs, but to a simulation of  such a camera based on a normal (or high-speed) camera stream. The paper would be much strengthened by using actual spiking camera inputs. 
* RAFT and other sota methods are only applied on the modified  image inputs (AvgImg, Spike). The performance on normal images from the slowflow data set are not mentioned. 
questions: 1. Are you the first to apply the DSFT / what other encoding methods have been proposed? Can you make the relation with firing rate clearer? Can you show how sensitive this encoding is to noise?
2. Can you apply your trained network to spiking camera inputs? Does it generalize well to non-RSSF, non-"simulated" data?
3. The performance of Spike2Flow is always best. Can the performance of RAFT etc. be mentioned on normal RGB images for the dataset? This puts the performances more in context. For example, does RAFT lose a lot of performance when applying it to AvgImg? 
4. Is j-V in equation 7 correct?
5. The dataset RSSF is mentioned as a contribution, but if I understand correctly, it is an automatic adaptation of SlowFlow, which was made by others. Is this correct? Can it be clarified earlier in the text?
limitations: N/A
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The authors present an algorithm for optical flow from spiking camera data. The main contributions are:

1) the introduction of a spike based optical flow network, Spike2Flow which extracts features from binary spikes and estimates flow fields

2) a dataset for spike-based optical flow

3) Experimental results demonstrating good comparative results on various datasets and compared to other algorithms
strengths_and_weaknesses: Strengths:

S1: the paper is well written overall

S2: the paper deals with an interesting problem, namely using spike cameras for optical flow. Given that spike cameras have high temporal resolution, optical flow is a good application for these cameras

S3: the authors demonstrate good results compared to other similar algorithms

S4: the authors indicate that source code will be released upon publication

Weaknesses:

W1: the algorithm does not really seam terribly innovative. This made me a bit ambivalent in deciding the paper's final ranking, but given the good results, I think it should be of interest to the community that deals with spike and event cameras

W2: Eq 1: clarify it there is at most 1 spike per time intervale T=[T_pre, t]. Do all the spikes emerge from all pixels at the same time of is this asynchronous

W3: How much data is actually produces by this sensor per second? The potential issue I see here is that neuromorphic chips that process event data asynchronously typically have I/O limitations. It is not clear how suitable this algorithm would be for such hardware. Furthermore the use of floating point operations (see softmax in eq 4 for example) may limit the applicability of this algorithm in power efficient hardware. A comment on this would be good.

W4: line 209: typo: "sparsely sampling"
questions: See my comments above
limitations: N/A: not applicable
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper proposes a method, Spike2Flow, to estimate optical flow from videos recorded by spiking cameras. The input modality is rarely used and not properly introduced, so it hard for me to evaluate the merits of this work. Moreover, the technical contribution seems marginal.
strengths_and_weaknesses: Strengths:
1. The empirical performance seems good.

Weaknesses:
1. The presentation is quite unclear. Esp., as spiking cameras are an emerging device and rarely seen, most people are not familiar with its properties, advantages and specific challenges. The authors did not do well when introducing the background of this task.
2. The architecture presented in Fig. 1 seems very similar to RAFT. The Spatial Information Aggregation in Fig.2 seems very similar to CRAFT [a]. It gives me a feeling that the proposed method has rather limited novelty.

[a] CRAFT: cross-attentional flow transformers for robust optical flow. CVPR 2022.

questions: 1. Please give illustrations of the output (consecutive frames) of spiking cameras, so that readers will have an intuitive understanding what the videos look like, and what challenges come with this new modality.
2. Why spiking cameras are called with different names in different papers? The two cited references are [11; 19], but in [11] it's called a "spike camera" (instead of spiking), and in [19] it's called "vidar".

limitations: N/A
review_title: null

Meta-review: metareview: This work is focused on the estimation of optical flow from a neuromorphic camera that produces Poisson spiking at each pixel with a rate governed by overall intensity. The authors use local space-time aggregation of spike-time differentials to identify features that are then corresponded via a convGRU decoder. 

The reviewers found the application interesting, and noted the good performance of the method. There were however a number of concerns about innovation and novelty of the method. Specifically the aggregating spikes to operate on point process data is a standard approach and the assessment of the spiking source of the data was not analyzed. Regardless of the similarity to past methods, overall the reviewers felt that the strengths of the paper, specifically the combination of methods brought together to solve a unique problem, outweighed the weaknesses. Thus I recommend that this work be accepted. 
confidence: Less certain
Decision: Accept