Continuous Parametric Optical Flow
Authors: Jianqin Luo
Zhexiong Wan
yuxin mao
Bo Li
Yuchao Dai
Venue: NeurIPS 2023 poster (2023)
Keywords: optical flow
point trajectories
continuous motion
neural ordinary differential equation
TL;DR: A temporal-parametric model based on B-splines to utilize limited visible frames to fit point trajectory and output the parametric continuous optical flow

Abstract: In this paper, we present continuous parametric optical flow, a parametric representation of dense and continuous motion over arbitrary time interval. In contrast to existing discrete-time representations (i.e., flow in between consecutive frames), this new representation transforms the frame-to-frame pixel correspondences to dense continuous flow. In particular, we present a temporal-parametric model that employs B-splines to fit point trajectories using a limited number of frames. To further improve the stability and robustness of the trajectories, we also add an encoder with a neural ordinary differential equation (NODE) to represent features associated with specific times. We also contribute a synthetic dataset and introduce two evaluation perspectives to measure the accuracy and robustness of continuous flow estimation. Benefiting from the combination of explicit parametric modeling and implicit feature optimization, our model focuses on motion continuity and outperforms the flow-based and point-tracking approaches for fitting long-term and variable sequences.

Continuous Parametric Optical Flow
Jianqin Luo"
Zhexiong Wan"
Yuxin Mao
Bo Li
Yuchao Dait
Northwestern Polytechnical University, Xi' Xi'an, an. China
Shaanxi Key Laboratory of Information Acquisition and Processing
Abstract
In this paper, we present continuous parametric optical flow, a parametric repre- sentation of dense and continuous motion over arbitrary time interval. contrast existing discrete-time representations (i.e.flow in between consecutive frames), this new representation transforms the frame-to-frame pixel correspondences to dense continuous flow. particular, we present cemporal-paramette model that employs B-splines to fit point trajectories using a limited number of frames. To also add an
further improve the stability and robustness of the trajectories, encoder with a neural ordinary differential equation (NODE) to represent features associated with specific times. We also contribute synthetic dataset and introduce two evaluation perspectives to measure the accuracy and robustness of continuous flow estimation. Benefiting from the combination of explicit parametric modeling and implicit feature optimization, our model focuses on motion continuity and outperforms the flow-hased and point-tracking approaches for fitting long-term and variable sequences.
Introduction
Motion estimation, which characterizes the movement of targets over time, is crucial for video understanding tasks like keypoint tracking [1] or video interpolation [2]. also provides accurate -rommmmotoooooioonn
priors for 3D tasks such as non-rigid structure-from motion [3, 4] and simultaneous localization and mapping [5]. Fine-graiee Fine-grained motion understanding is necessary to achieve pixel-level correspondence. which is usually studied as an optical flow estimation problem. Some traditional methods [6, 7] based on variational optimization or recent deep methods focusing on the Tracking-Any-Ponn (TAP) task [8, 9] both attempt to estimate long-range flow for tracking points across multi-frames.
While these approaches have made significant progress in modeling correspondences, there still exist two obstacles to explaining and predicting practical motion, as shown in Fig. 1. First, the point trajectories generated by these approaches only reflect the possible locations certain moments instead of real motion trends because the frame-to-frame correspondence is temporally discrete and lacks enough inter-frame information describe continuous point motion. Second, optical flow-based methods could achieve dense but short-term point mapping while recent point trackers tend strengthen long-tent term tracking for sparse points. Neither can efficiently realize dense and long-term tracking simultaneously.
In this paper, aim estimate the dense and continuous pixel correspondence with the continuous flow that provides the moving trajectory from reference time to arbitrary target times within the whole
'These authors contributed equally to this work. "Corresponding author (daiyuchao @nwpu.edu.cn). *Project page: https://npucvr..iin wwtp:eewrwr..biiiiiiooeoooo ioCCFllo..////.......
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

(a) Discrete-time Optical Flow [12, 13]
(b) Point Tracking [9]
(c) Continuous Optical Flow (Ours)
Figure 1:
Comparison between continuous optical flow with spatially dense and temporally continuous attributes with discrete-ting time flow or pixel tracking.
video. We utilize a learnable parametric model to explicitly describe the continuous flow trajectory, which avoids some inconsistent and abrupt displacements compared to the implicit counterparts directly predicting the final result. Motivated by [10], we embed the continuous property into our end-to-end optimization pipeline to enhance continuity in feature space. By combining the powerful representational ability and explicit motion pattern continuous problems. To evaluate the continuous flow estimation, we introduce two evaluation perspectives, including continuous motion with invisible frames specific timescales and robustness validation at variable timescales. We also use Kubric [11] to create a synthetic dataset with dense and long trajectories for experiments.
method is more appropriate for solving
Our main contributions are summarized as:
We present continuous parametric optical flow, temporal parametric representation of dense and continuous motion over arbitrary time intervals.
We utilize Neural ODE and ConvGRU to aggregate the spatial-tempon information and enhance the continuity of features among target times.
We create a large-scale synthetic dataset with dense and long-term trajectories and propose a two-stage evaluation pipeline to measure the accuracy and robustness of continuous flow estimation.
2
Related Work
2.1 Optical Flow Estimation
Optical flow estimation is fundamental task in computer vision, and existing optimization-base methods [14, 15, 16, 17] have made explorations in challenging situations such as large motions [18, 19] and occlusions[20, 21]. While early learning baanidg-based methods take an encoder-decoder [22] or coarse-to-fine [23, 24] structure, recent methods, such as RAFT [12] and GMA [25], usually take an iterative update pipeline. These methods usually take procedures of supervised training simulated data [25, 13] or unsupervised training on real data [26, 27].
Unlike the above common two-frame input, some methods [26, 28] use multi-frames to aggregate additional flows to tackle the occlusion. MFSF [6] aims to establish the long-range correspondence of multi-frame flow by linear combination of trajectory basis. SlowFlow [29] links adjacent flows to longer trajectories by constraining brightness, smoothness, and visibility. Gehrig et [30] proposes to use event data to build motion trajectories for fast and non-line non-linear motion in short period.

---

2.2 Feature and Point Tracking
Among the self- supervieed feature tracking methods, TimeCycle [31] uses cycle consistency as the primary constraint for patch-level extraction and matching. CRW [32] designs special palindrome arrangement to introduce cycle loss. VFS [33] introduces contrastive learning to optimize frame-level matching. Instead of implicit exploration in unlabeled data, supervised methods [34, 35] rely on ground truth correspondence to learn a general feature representation. Although these methods perform well on general tasks like keypoint tracking semi-supervised video segmentation, pixel- level matching remains unsolved because image or patch embedding seems insufficient for specific point refinement.
For long-term point tracking, the key problem is how to match the temporll features of the long-range correspondences. PIPs [9] proposes to extract query point features for pixel tracking and occlusion estimation with iterative updates, motivated by Partiele Video [36]. TAP-Net [8] uses a flow-based method to assist annotators to track sparse points selected by random sampling and generate point trajectories with longer duration.
2.3 Parametric Motion Hypothesis
Rigid motion modeling was fashion for early work before various modern learning techniques. As for spatial modeling, some representative methods [37, 38] use robust parametric motion like affine transformation or based on paranttrrizaioonnn [39, 40] to estimate flow. Yang and Li [41] propose a piecewise, 8-Dof homography model to fuse multiple possible parametric motions. Besides, layered models [42, 43] also perform competitively with few overlapping motion layers that balance the need for smoothness and boundary separation.
For temporal consistency, the linear combination of motion basis [6, 44, 45] is common method for trajectory fitting in the time axis. These pre-defined bases like DCT [6] or Lagrange [44] can convert the continuous time domain into the discrete analysis. With different weight assignments for basis groups, this modeling approach is convenient to implement for deep paradigm [45]. However, limited by the sampling range, these models are not flexible for video clips with different frame numbers. Another alternative to perform temporally continuous curve fitting, including linear [46], quadratic [47], cubic [48] curve hypotheses with polynomial modeling.
In addition, complex motion trajectories can be represented by adaptive curves with a few control points like B-splines [49] or Bezier [30]. Although most of these parametric hypotheses were first introduced previous optimization methods, modern methods can still take a deep network as the process of motion parameterization by regressing set of coefficients or control points.
3 Parametric Modeling
3.1 Preliminaries of Continuous Flow
To describe the continuous flow that expresses the motion from the reference time to any target time, we define a triplet (1,, u, v) as a query of flow trajectory decoupled into two components along horizontal and vertical direction for corresponding pixel (Ir, y). t sampled from normalized time T' E [o, 1J. For brevity, the reference time is zero. Instead of directly fitting the pixel position, we choose to estimate the pixel displacements and the start point of flow trajectory (u, v) (0, 0). Note that although we can only use finite labels to evaluate our model, we introduce two evaluation perspectives to measure the performance under diverse situations.
Stage 1: Blind Estimation. Given a specific number of frames at sampled timestamps with a fixed timescale input, the model should predict the target flow trajectory unsampled times.
Stage 2: Temporal daptation. The pre-trained model on a fixed training range should adapt to diverse sequences with different timescales.
3

---

3.2 Parametric Curve for Continuous Flow Modeling
IN E Nx and basis functions (t, with degree k the point displacement trajectory F(t) is formulated as:
Given N control points {P; (Ir, 1))
N-1
F(t) Σ B.t()P. ==0
(I)
For any specific time the basis functions return group of weights to determine the contributions for the curve shape each control point. The parametric curve we employ is a cubic B-spline formed by few learnable control points and spline basis functions. In work, the coordinates of N control points serve as model outputs, whereas the normalized knots {4} are manually specified hyperparameters for local optimization in the temporal range. The detailed derivation of basis functions is referred to Cox-de Boor recursion [50, 51]:
B,,o(t) = { t 5ttt441 1 1-4
(2)
otherwise
144h+1 Bi+1,K-1(t).
(i,x(t)=
titb -t, Bix--(t) +
14++11 - t.+1
We assign each time to a single partition to ensure the sum of the weight coefficients of all control points is equal Knots can be repeated promote the curve from open close. In our experiment, a special shape like the clamp seems easier to reach convergence. Benefiting from the potential advantages parameter design, the cubic B-spline could achieve the combination of global relaxation and tin-ttunig on local segments.
The parametric curve is optimized independently for each pixel to build complete displacement flow field. As result, our model directly predicts a 3D tensor with the shape of 2N x H x W for control points in all trajectories.
4
Optimization
The optimization process consists of four components: multi-frame sampling, implicit continuity enhancement, multi-time correlation pyramids, and iterative control point decoder. According the order of the input video clip with L frames, the corresponding timestamps are normalized to [o, 1]. Our framework aims to regress of control points with their 2D coordinates for each pixel and to compute the final continuous flow trajectories by Eq, (1).
4.1 Multi-frame Sampling
For given video clip with L frames, the model input M frames are first sampled from the entire sequence as visible moments the continuous time domain while the remaining L - M frames are assumed to be invisible. We the ground-trute displacement fields sampled from all valid moments corresponding to the frames as the supervision continuous flow trajectories for model training.
During training, we first select the start frame with sufficient number of visible points from the given video clip. Then we randomly sample a subsequence clip with M frames with different intervals as model inputs to enhance the generation ability to different motion magnitudes. Notably, this selection process keeps the start and end frames and only randomly selects the intermediate M - 2 frames based on brightness differences.
4.2 Implicit Continuity Enhancement
As shown in Fig. we first extract M basic feature maps using a shared basic encoder for each of the input M frames. Similar to RAFT [12], we also feed the reference frame (i.e. the start frame with ( 1= 0) into the context encoder for extracting another context feature assist subsequent motion estimation. Unlike PIPs [9], we introduce ODE-ConvGRU proposed in [2] for spatio-temporal feature learning. This module consists of the Neural ODE for temporal continuity enhancement in feature space and the CoRv-GRU for spatio-temporal information aggregation. The Neural ODE formulated

---

Encoder
(0,0) 0)
ConvGRU
Iberativn Update
XL
Sulver
Transfomm
Ci
Oue Unit
Olll
ODE Salver
RCo)
DDR
ODF Uniil
Multi-time Correlation Pyramids
Basic Encoder
ODE Salver
Concatonation
Control Points
0DE
Figure 2: Pipeline of our model, combined with continuous temporal-spatial feature and explicit motion trajectory curves. ODE-ConvGRU enhances spatial features with continuous temporal information and generates powerful representations at arbitrary target moments.
by Eq. (4) includes a derivative estimator f as independent ODE unit and an ODE solver, in which the parameters involve the initial value (tu),, the current time f and input state hm (t). The former uses learnable fully-connected network f to fit the feature differentiation at time and the latter integrates to recover the hidden state h((t) at started from lo-
dltin (t) = (hinntt,
dt
htt) ODE(EEE(FFC f, (t))
(4)
h(t) hin(to) + ["' (t)
Jtn
The timestamps corresponding to the input M frames serve as source times (M9re)M to feed into the ODE-ConvGRU module and output features target times (1,95] for correlation construction. The Neural ODE models the features of the entire L-frame video clip including invisible frames, also outputs L target times.
Our ODE-ConvGRU module consists of three steps. First and foremost, we compute the initial hidden states ((ttr)) at source times trre by integrating the derivative of the previous hidden state at 4ome solved by network fo:
h(tere) = ODE(fe (,
(5)
where this process with h(4rr)) i.e., an zero feature map. Note that the computation process is backward, which means the original temporal order needs to be reversed.
Besides, given feature x extracted by the basic encoder, we update the initial hidden state T((gre) at source time tgro by aggregating them to generate refined hidden state h(tgre) with ConvGRU:
((grr)) = (( Xiçre)
Finally, the last output hidden state (0)) from ConvGRU aggregates valid information from all source moments, and is transformed into a novel initial value h(()) by small network g.. The derivative is computed by another ODE unit with network fo and evolves into the final hidden state outputs ho h(()9)) target time 1o
/ ODE(f6, (t/tt/ h())).
(7)
5

---

4.3 Multi-time Correlation Pyramid
To achieve fine- aeeegrained matching, we compute the spatial matching feature similarity between different target times. Due to the powerful spatio-temporal representation of ODE-ConvGRU, we can obtain an arbitrary number of feature pairs for continuous flow estimation. Thus we can perform the correlation operation between the reference state h(tb") and the rest of hidden states {(h(t) ) at target time construct L 1 cost volumes.
However, due to limited GPU memory, we still need to introduce sampling to reduce computational complexity while leveraging the continuous representation capability. We make trade-off by random sampling Q (Q & (L 1)) hidden states to construct Q multi-time correlations C' = {C₂] We then perform the average pooling operation and obtain the correlation pyramids C, {C;]² 2 (Z
is the pooling level) for each correlation separately, which covers matching similarities multiple
resolutions for subsequent local lookup.
4.4 Iterative Lookup and Update
Inspired by RAFT [12], we also adopt the lookup operation to search the local matching patches from multi-time correlations. Subsequently, we concatenate the patches with different moments and different pooling levels as single tensor and feed it into the decoder to regress the residual of control points and compute the continuous optical flow. This procedure local lookup and flow regression is iteratively executed by adjusting the local search starting point according to the updated control points.
For N control points (P.)*, the update unit outputs N translations (AP:) and updates iteratively:
p- P APP
(8)
where is the index of the control point and is the iteration number for lookup and update.
4.5 Supervision
As there are only discrete point displacement ground truths in the dataset, we can only use them to discretely supervise the predicted continuous optical flow of our model. Assuming that for each point there are Ngt displacement ground truths at different times, we calculate the point displacements on the predicted continuous trajectory with the corresponding times {1#P) ) according to Eq. (l).
The loss function measures the average displacement error at Ngt different times and all visible pixels with weighted sum of all iterations:
gt (F( (t 11
L
where the iteration times and weight y.. ((1SUP) is the point displacements at time 1 on the predicted continuous trajectory, Dye ((P)) is the the corresponding point displacement ground truth. Note that the starting point of the continuous trajectory is O, so we omit
Experiments
This section consists of four associated parts. We first use Kubric [11] to simulate a relatively dense and longer pixel-tracking dataset for our model training and as a basic benchmark for evaluation. Second, establish valid baselines, introduce two approaches [9, 12] to provide direct estimation for sampling moments and parametric assumption to simulate inter-frame motion. Third, we set error metrics for quantitative comparison and show the results synthetic scenes and real-world scenes. Finally, ablation experiments are implemented for module validity tests.
We train our model on four RTX 3090 GPUs with a batch size of 8. At the training stage, we use the resolution of x56x25c and randomly sample 20480 point trajectories as sparse supervisions for better performance. We train for 30,000 steps because more iterations may result in degradation on diverse timescales with learning rate of 2e-5 and a one-cycle schedule [52]. We repeat multiple times and report median results. We set the numbers of control points N 6 while the degree & = 3.

---

and the sampling number of frames M = 4. The number Q of correlation is 3, and the pooling level Z is 3. In the supervision process, the Ng and y = 0.8. For temporal augmentation, the length of video clips I. is variable but at least 8 frames.
5.1 Dataset
To maintain the same settings as [8, 9]. we utilize the Kubric simulator to generate two subsets: query-stride and query-first in our experiment. The video samples both the two subsets are with 24 frames at 256x256 resolution. Points trajectory is sampled from the whole spatial-temporal area in the query-stride version but from the single frame the query-first version. Query-stride is applied in the training phase for less structured constraints and query-first can be used for dense evaluation.
5.2 Metries
We use two metries to measure the accuracy at specific moments and the temporal trajectory smooth- ness of continuous flow estimation. From spatial perspective, the average displacement error (ADE) proposed in [9] is applied and divided into visible and occluded parts. To reflect the ability to fit the trajectory during blind time, an additional temporal average is introduced for sampling and non-sampling times. From a temporal perspective, the flow trajectory for determined point should be robust and smooth between any sequences. We use temporally averaged root-mean-squarc error (TRMSE) to measure trajectory smoothness, for every candidate pixel x in the spatial domain, and the RMSE along the temporal dimension is computed to measure the influence outliers. After that, the errors of all pixels averaged.
T-1 Σ
TRMSE N
Σ
1 T
Dgr(ta))1
(10)
where N. is the number of valid points, x E RN,x2 is the coordinates of valid points. T is the number times with ground-truth displacements for each point, to is the corresponding time.
5.3 Baseline
To generate the motion trajectory of each pixel, we choose flow estimation method RAFT [12] and the point-tracking method PIPs [9] as our baselines. RAFT predicts the optical flow between consecutive frames, and these flows are linked by bilinear interpolation as mentioned [9]. We use their pre-training model and fine-tune it on dataset. PIPs is recent pixel tracking algorithm with promising generalization performance. We retrain PIPs with 4 frames on our dataset and infer the corresponding movements of all pixels during whole input clips to ensure fairness of the comparison.
(a) Ground Truth
(b) RAFT
(c) PIPs
(d) Ours
Figure 3: The visualization of dense point trajectories on the Query-Firs First set.
7

---

Table 1: Evaluation synthetic datasets. Our model with cubic B-splines and ODE encoder outperforms baselines on sparse validation (Query-Stride set) and dense test (Query-Firs) set).
Query-Strice
Query- set
Method
Metric
Mark
10f
16f
24f
8f
10f
16f
24f
ADE_Vis
8.18
7.91
13.00
14,43
5.68
7.18
11.12
17.56
NS
7.13
6.08
10.21
11.31
4.94
5.42
8.47
13.91
ADE_Oce
9.62
12.16
27.73
24.95
6.26
8.60
14.77
24.86
RAFT
NS
8.79
9.37
21.98
18.06
5.60
6.43
10.72
19.04
AII
S
8.32
8.17
14.93
16.37
5.73
7.27
11.44
18.49
NS
7.39
6.37
12.37
13.06
5.00
5.51
8.75
14.72
TRMSE
8.50
7.82
15.81
16.66
5.86
6.77
10.35
16.85
ADE Vis
NS
S
3.97
4.66
4.39
3.40
6.92
8.77
10.13
7.92
3.12
2.97
3.26
4.12
5.97
7.81
10.27
14,23
ADF_Oce
7.13
7.71
21.54
19.97
4.85
7.28
12.89
22.34
PIPs
NS
6.43
5.93
16.83
15.16
4.23
4.03
9.31
16.35
ADE_AII
S
4.74
4.58
10.41
12.09
3.76
4.93
8.35
14.68
NS
4.13
3.61
8.68
9.86
3.25
3.76
6.42
11.76
TRMSE
4.93
4.49
11.27
13.02
3.90
4.72
7.76
13.70
S
3.81
3,88
8.08
8.22
2.88
3.72
6.26
12.08
ADE Vis
NS
3.38
3.11
6.47
6.60
2.51
2.86
4.83
9.73
ODE-6spline (Ours)
ADE_Occ
NS
S
5.65
6.24
8.09
6.35
22.72
T8.21
19.84
15.07
4.17
4,70
5.04
6.86
11.34
8.24
19.61
15.19
ADE_AII
4.00
4.13
9.89
10.61
2.96
3.88
6.71
13.05
NS
3.60
3.37
8.54
8.92
2.61
3.02
5.26
10.64
TRMSE
4.26
4.17
11.38
12.30
3.12
3.78
6.42
12.38
S, NS are respectively referred to as the sampling and non-sampling moments. ** is abbreviation of frames.
Since our goal is to predict continuous flow trajectories, not only corresponding to inputs, but also inter-frame trajectories. This allows us to set some of the whole frames in the dataset to be visible while others are hidden. The ground-truth displacements of the hidden frames can used for the continuity evaluation, including inter-frame motion. Therefore, assumptions for the baselines to connect discrete points into complete continuous trajectories. We experimented with multiple assumptions, but the linear one yielded more stable results.
introduce additional motion
5.4 Evaluation on Simulated Data
We use two subsets of our synthetic dataset for validation in sampling ranges with 8, 10, 16, and 24 frames with uniform sampling. As mentioned earlier for the two evaluation perspectives, we evaluated inter-frame motion accuracy and multi-frame motion robustness. respectively. The result of the quantitative comparison is found in Table 1. As shown in Fig. 3, benefited by global optimization, our temporal parametric model outperforms baselines at all moments for all variable temporal ranges.
From the accuracy perspective, benefiting from continuous explicit modeling, our algorithm remains high performance at both sampling and non-sampling moments. The rate of error accumulation with the extension of the inference range is also controlled validly, especially for visible points. From the temporal stability perspective, compared with widespread polynomial motion approximation between reference and target frames, our curves based on limited variable control points can adjust the local trajectory flexibly for overall smoothness. Besides, our model is capable of adapting to both sparse and dense inference, particularly on dense estimation of short distances, which can sustain more consistent predictions. To reduce the instability of parametric modeling a longer inference range, we expect to exert the powerful influence of implicit feature representation. By introducing the Neural ODE module with Conv-GRU update units, temporal features across multi-frames will be more convenient for motion separation and aggregation.

---

Table 2: Evaluation real-world datasets. Our model with cubic B-splines and ODE encoder outperforms baselines on real scenes.
Vid-DAVIS [8]
Vid-Kinetics [8]
Method
Metric
Mark
20r
24f
28f
32f
36f
48f
128f
250f
ADE_ Vis
S
16.84
19.72
19.42
23.37
15.97
19.00
30.19
35.93
NS
12.42
14.71
15.87
18.53
12.98
16.36
24.99
30.63
ADE Occ
S.
18.58
23.82
27.91
31.35
19.23
21.89
31.98
39.76
RAFT
NS
14.67
18.56
20.19
22.53
16.07
18.85
26.11
33.10
ADE_AII
NS
S
17.99
14.23
20.25
16.07
17.91
25.65
T9.68
16.74
14.90
20.70
17.73
31.47
25.16
37.81
31.49
TRMSE
16.91
18.95
21.28
23.81
16.62
24.51
27.50
34.23
ADE_ Vis
S
12.96
15.47
15.62
18.73
12.71
16.48
28.34 34
38.56
NS
10.39
11.39
12.60
15.27
10.49
13.30
22.31
31.08
ADE Occ
S 18.84
26.67
30.60
32.21
17.59
21.08
31.82
42.31
PIPs
NS
14,42
20.66
20.88
23.26
15.42
18.56
25.34
35.41
ADE_AII
S 15.18
17.98
22.06
23.31
13.70
17.79
31.76
40.59
NS
11.94
14.38
16.62
18.49
12.73
13.72
24.67
33,90
TRMSE
14.80
17.38
20.66
22.79
12.51
15.82
28.60
37.25
ADE_Vi Vis
S
11.37
16.80
16.16
18.99
12.09
15.02
25.25
31.18
NS
9.31
12.21
13.37
15.21
9.99
12.33
21.35
27.13
ODE-6splinc (Ours)
ADE Oce
NS
S
15.75
12.23
20.67
15.91
26.71
18.00
27.99
20.55
17.76
15.17
20.50
17.99
29.79
24.98
37.12
32.00
ADE_AI
S.
13.48
16.91
19.81
21.16
12.47
15.82
27.67
34,42
NS
10.79
13.30
15.07
16,68
10.47
13.10
22.80
29.65
TRMSE
12.96
15.76
18.12
20.27
12.06
14.91
25.44
32.78
5.5 Evaluation on Real Data
We choose the TAP-Vide Vid-DAVIS and TAP-Vid Vid-Kinetic datasets[8] to evaluate our model real data, with 20,24,28,32 frames for evaluation on DAVIS and 36,48, 06,48,112, non-sampling moments for Kinetics. For fairness of comparison, 128, frames including more need all methods complete dense flow estimation and use identical interpolation techniques to obtain the corresponding trajectory of sparse points. The evaluation result is shown Table 2 and the visualization result Fig. 4.
According to the test result, we find our powerful model can surpass the flow-based method and point-tracking algorithm PIPs in all test ranges for all time ranges. Although PIPs perform well in specific point tracking, the model needs heavy costs for large numbers of points. We only infer 3072 points simultaneously for 256x256 resolution in the test stage with the 4-frame model. For dense validation, we have to process in batch with longer inference time while a flow-based solution needs more link steps resulting in
time consumption.
However, our model still shows potential real scenes connected with our explicit continuous curve fitting. The motion trend is similar to ground truth even though there are slight trembles in the middle process. Thanks to large-scale training and implicit spatial-temporal continuous features, our work can keep relatively stable tracking which is helpful to reduce the drift in long-range prediction.
5.6 Ablation
We analyze two core components that contribute to our model on feature and trajectory continuity modeling. As for the feature encoder, the basic structure is from RAFT and no extra module is added for temporal representation. The ODE encoder will supplement an ODE unit with the combination of neural ODE function and Conv-GRU iterative part after basic feature extraction. As for the parametric curve, We set the hyper-parameter of the B-spline to reflect the number of control points which is designated 4 and 8. The result is shown in Table 3.
In the feature continuity aspect, ODE unit valid for our parametric curve and equipped with stronger ability to capture temporal-spatial connection. About the parametric curve selection, cubic B-spline with and control points are both weaker than control points because less or more partition from control points will generate more loosen tight constraints.

---

(a) Ground Truth
(b) RAFT [12]
(c) PIPs [9]
(d) Ours
Figure 4: The visualization of sparse point trajectories Vid-DAVIS.
Table 3: The ablation model components.
Module
Method
Metric
8f
10f
16f
24f
ADE_Ar All
4.41
3.97
9.35
9.68
ODE 4spline
TRMSE
4.83
4.57
12.09
12.65
Control Point
ADE_AI
3.94
3.91
9.06
9.13
ODE 8spline
TRMSE
4.40
4.47
11.66
12.14
ADE_ TRMSE
4.89
4.43
9.79
9.92
Neural-ODE
Basic 6pline
5,45
5.04
12.41
12.93
ADE All
3.77
3.62
8.80
8.86
All
ODE 6spline
TRMSE
4.26
4.17
11.38
12.03
6 Conclusion
We have presented continuous parametric optical flow, a parametric representation of dense and continuous motion, which describes real pixel motion trajectory and expects to promote frame-to- frame discrete flow into continuous time-to-time motion. We generate a synthetic dataset based on Kubric for our task and introduce evaluation rules for continuity. With multi-frame inputs, pipeline with the fusion of continuous features and the parametric curve proposed for flow estimation. Experimental results on synthetic and real scenes reveal that our method focuses more unseen inter-frame trajectory and there room for development for all moments.
Limitations and Broader Impacts. As parametric representation, the ability of our method to handle challenging complex motions will be limited by the choice of the degree of model and we have to admit the limitation in all parametric models, where a strong explicit constraint potentially leads to better performance on those more similar motion scenery with the training set. Our work is foundational research, for which
do not foresee it will bring negative social impacts. We also believe that the ethical concerns of our method in the training and evaluation process are minimal and there is no harm or bias to anyone. All weight parameters rely learning from valid synthetic data [11 ]] and real data utilized for evaluation comes from TAP-Vid [8] with clear statement for the faimess and unbiasedness of these benchmarks.
A Aeknowledgments. This research was supported in part by the National Natural Science Foundation of China (62271410, 62001394), the Fundamental Research Funds for the Central Universities, and the Innovation Foundation for Doctor Dissertation of Northwestern Polytechnical University (CX2023013).
10

---

References
[1] Sven Kreiss, Lorenzo Bertoni, and Alexandre Alahi. Openpifpaf: Composite fields for semantic keypoint detection and spatio-temporal association. IEEE Transactions on Intelligent Transportation Systems, 23)8): 13498-13511, 2021.
[2] Sunghyun Park, Kangyeol Kim, Junsoo Lee, Jacgul Choo, Joonseok Lec, Sookyung Kim, and Edward Choi. Vid-ode: Continuous-time video generation with neural ordinary differential equation. In AAAI Conference on Artificial Intelligence (AAAI), volume 35, pages 2412-2422, 2021.
[3] Vikramjit Sidhu, Edgar Tretschk, Vladislav Golyanik. Antonio Agudo, and Christian Theobalt. Neural dense non-rigid structure from motion with latent space constraints. In Computer Vision-ECCY 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XVI 16, pages 204-222. Springer, 2020.
[4] Hui Deng, Tong Zhang, Yuchao Dai, Jiawei Shi, Yiran Zhong, and Hongdong Li. Deep non-rigid fooee-foooooooonno A tosuenee-o-sequence translation perspective. arXiv preprint arXiv:2204.04770 044300 2022.
[5] Qiang Fu, Hongshan Yu, Xiaolong Wang, Zhengeng Yang. Yong Hc, Hong Zhang, and Ajmal Mian. Fast orb-slam without keypoint descriptors. IEEE Transactions on Imuge Procensing, 31:1433-1446 2021.
[6] Ravi Garg, Anastasios Roussos, and Lourdes Agapito. variational approach to vidco registration with subspace constraints. International Journal of Computer Vision (IJCV), 104:286-311 314, 2013.
[7] Wenbin Li, Darren Cosker, Matthew Brown. Drift robust non-rigid optical flow enhancement for long sequences. Journal of Intelligent & Fuzzy Systems, 31(5):2583 2595, 2016.
[8] Carl Doorsch, Ankush Gupta, Larisa Markceva, Adria Recasens Continente, Kucas Smaira, Yusuf Aytar, Joao Carreira, Andrew Zisserman, and Yi Yang. Tap-vid: A benchmark for tracking any point in video. In NeurlPS Datasets Track, 2022.
[9] Adam W Harley, Zhaoyuan Fang, Katerina Fragkiadaki. Particle video revisited: Tracking through occlusions using point trajectories. In European Conference on Computer Vision (ECCV), pages 59-75. Springer, 2022.
[10] Ricky TQ Chen. Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. Advances in Neural Information Processing Systems (NeurlPS), 31, 2018
[11] Klaus Greff, Francois Belletti, Lucas Beyer, Carl Doersch, Yilun Du, Daniel Duckworth, David J Fleet, Dan Gnanapragasam, Florian Golemo, Charles Hermman, et Kubric: A scalable dataset generator. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3749-3761, 2022.
[12] Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In European Conference on Computer Vision (ECCV), pages 402-419. Springer, 2020.
[13] Zhaoyang Huang, Xiaoyu Shi, Chao Zhang, Qiang Wang, Ka Chun Cheung, Hongwei Qin, Jifeng Dai, and Hongsheng Li. Flowformer: transformer architecture for optical flow. European Conference on Computer Vision (ECCV), pages 668 365. Springer, 2022.
[14] Berthold KP Horn and Brian G Schunck. Determining optical flow. Artificial intelligence, 17(1-3):185-003, 1981.
[15] Jerome Revaud, Philippe Weinzaepfel. Zaid Harchaoui, and Cordelia Schmid. Epicflow: Edge-preserving interpolation of correspondences for optical flow. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1164-1177 1172, 2015.
[16] Qifeng Chen and Vladlen Koltun. Full flow: Optical flow estimation by global optimization over regular grids. In IEEE Conference Computer Vision and Puttern Recognition (CVPR), pages 4706 -4114,7444 2016.
[17] Christian Bailer, Bertram Tactz, and Didier Stricker. Flow fields: Dense correspondence fields for highly accurate large displacement optical flow estimation. In IEEE International Conference on Computer Vision (ICCV), pages 4015-4023, 2015.
[18] Thomas Brox and Jitendra Malik. Large displacement optical flow: motion estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 33(3): 500-5133 2010.
descriptor matching in variational
11

---

[19] Philippe Weinzaepfel, Jerone Revaud, Zaid Harchaoui, and Cordelia Schmid. Deepflow: Large displace- ment optical flow with deep matching, In IEEE International Conference on Computer Vision (ICCV). pages 1385-1392, 2013.
[20] Marius Leordeanu, Andrei Zanfir, and Cristian Sminchisescu. Locally aftine sparse-to-dense matching for motion and occlusion estimation. In IEEE International Conference on Computer Vision (ICCV), pages 1721-1728, 2013.
[21] Yu Li, Dongbo Min, Minh N Do, and Jiangho Lu. Fast guided global interpolation for depth and motion. In European Conference on Computer Vision (ECCV), pages 717-733. Springer, 2016.
[22] Alexey Dosovitskiy, Philipp Fischer, Eddy Ilg, Philip Hausser, Caner Hazirbas, Vladimir Golkov, Patrick Van Der Smagt, Daniel Cremers, and Thomas Brox. Flownet: Learning optical flow with convolutional networks. In IEEE International Conference on Computer Vision (ICCV), pages 2758-2766, 2015.
[23] Deqing Sun, Xiaodong Yang, Ming-Yu Yu Liu, and Jan Kautz. Pwe-net: Cans for optical flow using pyramid, warping, and cost volume. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pages 8934-8941 8943, 2018.
[24) Jianyuan Wang, Yiran Zhong. Yuchao Dai, Kaihao Zhang, Pan Ji, and Hongdong Li. Displacement- invariant matching cost learning for accurate optical flow estimation. Advances in Neural Information Processing Systems (NeurlPS), 53:155200-5231, 2020.
[25] Shihao Jiang, Dylan Campbell, Yao Lu, Hongdong Li, and Richard Hartley. Learning to estimate hidden motions with global motion aggregation. In IEEE International Conference on Computer Vision (ICCV), pages 9772-9781, 2021.
[26] Joel Janai, Fatma Guney, Anurag Ranjan, Michael Black, and Andreas Geiger. Unsupervised learning of multi-frame optical flow with occlusions. In European Conference on Computer Vision (ECCV), pages 690-706, 2018
[27] Kunming Luo, Chuan Wang. Shuaicheng Liu, Haoqiang Fan, Jue Wang, and Jian Sun. Upflow: Upsampling pyramid for unsupervised optical flow learning. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1045-1054, 2021.
[28] Zhile Ren, Orazio Gallo, Deqing Sun, Ming-Hsuan Yang, Erik B Sudderth, and Jan Kautz. A fusion approach for multi-frame optical flow estimation. In IEEE Winter Conference on Applications of Computer Vision (WACV), pages
IEEE, 2019.
[29] Joel Janai, Fatma Guney, Jonas Wulff, Michael Black, and Andreas Geiger. Slow flow: Exploiting high-speed cameras for accurate and diverse optical flow reference data. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3597-3607, 2017.
[30] Mathias Gehrig, Manasi Muglikar, and Davide Scaramuzza. Dense continuous-time optical flow from events and frames. arXiv preprint 2203. 13674, 2022.
[31] Xiaolong Wang, Allan Jabri, and Alexei A Efros. Learning correspondence from the cycle-consistency of time. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2566-2576, 2019.
[32] Allan Jabri, Andrew Owens, and Alexei Efros. Space-time correspondence as contrastive random walk. Advances in Neural Information Processing Systems (NeurIPS), 33:1 33:19555-19560, 2020.
[33] Jiarui Xu and Xiaolong Wang. Rethinking supe-supe correspondence learning: A video feaee- similarity perspective. In IEEE International Conference on Computer Vision (ICCV), pages 10075-10085 2021.
[34] Wei Jiang, Eduard Trulls, Jan Hosang, Andrea Tagliasacchi, and Kwang Moo Yi. Cotr: Correspondence transformer for matching across images. In IEEF International Conference on Computer Vision (ICCV), pages 6207-6217, 2021.
2021.
[35] Olivia Wiles, Sebastien Ehrhardt, and Andrew Zisserman. Co-attention for conditioned image matching. In IEEE Conference Computer Vision and Pattern Recognition (CVPR), pages 15590-115929 2021.
[36] Peter Sand and Seth Teller. Particle video: Long-range motion estimation using point trajectories. Interna- tional Journal of Computer Vision (IJC),, 80:72-91. 2008
[37] Shanon X Ju. Michael J Black, and Allan D Jepson. Skin and bones: Multi-layer, locally affine, optical flow and regularization with transparency. In IEEE Conference on Computer Vision and Patiern Recognition (CVPR), pages 307-314, IEEE, 1996.
12

---

[38] Li Xu. Jianing Chen, and Jiaya Jia. A segmentation based variational model for accurate optical flow estimation. In European Conference Computer Vision (ECCV), pages 671-684. Springer, 2008.
[39] Tal Nir, Alfred M Bruckstein, and Ron Kimmel, Over-parameterizee variational optical flow. International Journal of Computer Vision (LJCV). 76:205 266600-216,, 2008.
[40] Michael Hornáček, Frederic Besse, Jan Kautz, Andrew Fitzgibbon, and Carsten Rother. Highly overparam- eterized optical flow using patchmatch belief propagation. In European Conference Computer Vision (ECCV), pages 220 234.. Springer, 2014.
[41] Jiaolong Yang and Hongdong Li. Dense, accurate optical flow estimation with piecewise parametric model. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1019-1027, 2015.
[42] Deqing Sun, Erik Sudderth, and Michael Black. Layered image motion with explicit occlusions, temporal consistency, and depth ordering. Advances Neural Information Processing Systems (Neur(PS), 23, 2010.
[43] Deqing Sun, Erik B Sudderth, and Michael Black. Layered segmentation and optical flow estimation over time. In IEEE Conference on Computer Vision and Puttern Recognition (CVPR), pages 1768-1775. IEEE, 2012.
[44] Chen Song. Qixing Huang, and Chandrajit Bajaj. E-cir: Event-enhanced continuous intensity recovery. In IEEE Conference on Compuder Vision and Pattern Recognition (CVPR), pages 7803-7812, 2022.
[45] Chaoyang Wang. Xueqian Li, Jhony Kaesemodel Pontes, and Simon Lucey. Neural prior for trajectory estimation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 6532-6542, 2022.
[46] Simone Meyer, Oliver Wang, Henning Zimmer, Max Grosse, and Alexander Sorkine-Hornung. Phase- based frame interpolation for video. In IEEE Conference Computer Vision and Puttern Recognition (CVPR), pages 1410 1418, 2015.
[47] Xiangyu Xu. Li Siyao, Wenxiu Sun, Qian Yin, and Ming- Hsuan Yang. Quadratic video interpolation. Advances in Neural Information Processing Systems (NeurIPS). 32, 2019.
[48] Zhixiang Chi, Rasoul Mohammadi Nasiri, Zheng Liu, Juwei Lu, Jin Tang, and Konstantinos N Plataniotis. All at once: Temporally adaptive multi-frame interpolation with advanced motion modeling. In European Conference on Computer Vision (ECCV), pages 107-123. Springer, 2020.
[49] Se Young Chun, Timothy G Reese, Jinsong Ouyang, Bastien Guerin, Ciprian Catana, Xuping Zhu, Nathaniel M Alpert, Georges El Fakhri. Mri-based nonrigid motion correction simultaneous pet/mni. Journal of Nuclear Medicine, 53(8): 12e1112849999 1291, 2012.
[50] Carl De Boor On calculating with b-splines. Journal of Approximation theory, 6(1):50-62, 62, 1972.
[51] Morris G Cox. Curve fitting with piecewise polynomials. IMA Journal of Applied Mathematics, 8(1): 36-52, 1971.
[52] Leslie N Smith and Nicholay Topin. Super-convergenic: Very fast training of neural networks using large learning rates. Artificial intelligence and machine learning for multi-domait operations applications. volume 11006, pages 369-386. SPIE, 2019.
13


Reviews:
[{"review_id": "BodnrodRtK", "reviewer": "Reviewer_7yvr", "summary": "A temporally continuous parametric optical flow method, based on B-splines, is presented. The proposed network takes as input L frames (and timestamps) and outputs a tensor of size 2NxHxW, i.e N control points for each pixel. In practice N=6. The network architecture relies on neural ODE, ConvGRU and multi-time correlation pyramid. The proposed method is evaluated against RAFT and PIPs and a synthetic dataset (which is a contribution of this paper) and two real datasets (Vid-DAVIS and Vid-Kinetics). An ablation study is also presented.", "questions": "Please answer to the above \"weaknesses\". I am currently recommending to reject the paper as I feel the contribution is rather limited for NeurIPS.", "limitations": ".", "rating": 3, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The idea of using B-splines to model a temporally continuous parametric optical flow, is simple and efficient.\n2. The proposed architecture is not trivial and well thought out.\n3. The method outperforms both RAFT and PIPs. ", "weaknesses": "1. The paper contains many typos, e.g. l.152 feed - > fed, l. 154 nerual, l. 251 will, etc.).\n2. Some results are strange, for instance, in Table 2, PIPs and ODE-6spline have the exact same TRMSE on Vid-DAVIS.\n3. In Table 1, PIPs outperforms ODE-6spline in terms of TRMSE at 16f on the Query-Stride set (11.27 vs 11.38) but it is not in bold.\n4. Using ODE-ConvGRU does not seem new (it is said it was proposed in [49]). ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Weakness** \n***\n*1.Typos*：\n- **Comment:** The paper contains many typos, e.g. l.152 feed - > fed, l.154 neural, l.251 will, etc.\n\n- **Response:** Thanks for pointing out the typos. We will fix all the typos and further proofread the manuscript.\n***\n*2.Data Error*：\n- **Comment:** Some results are strange, for instance, in Table 2, PIPs and ODE-6spline have the exact same TRMSE on Vid-DAVIS.\n- **Response:** Thanks for pointing out this issue. We are sorry that we unexpectedly copy erroneous TRMSE results from raw data, displaying exactly the same performance in PIPs and ODE-6spline. Here we show the corrected tables as below. \nAs the table reported, the TRMSE metric of ODE-6spline in fact outperform PIPs and RAFT in all inference range of the TAP-DAVIS benchmark and appear similar trend in other benchmarks.\n\nTable. Updated TRMSE metric of ODE-6spline in Vid-DAVIS benchmark\n\n|       Method       | Metric | Benchmark |    20f    |    24f    |    28f    |    32f    |\n|:------------------:|:------:|:---------:|:---------:|:---------:|:---------:|:---------:|\n| ODE-6spline (Ours) |  TRMSE | Vid-DAVIS | **12.96** | **15.76** | **18.12** | **20.27** |\n---\n*3.Omission Mark*：\n- **Comment:** In Table 1, PIPs outperforms ODE-6spline in terms of TRMSE at 16f on the Query-Stride set (11.27 vs 11.38) but it is not in bold.\n- **Response:** Thanks for pointing out the issue. We agree with the reviewer that PIPs slightly outperform ODE-6spline in terms of TRMSE metric at 16f on Query-Stride set in Table 1. We will update Table 1 correspondingly as suggested. Note that our method still shows comparable performance at 16f and better temporal smoothness in other inference scales.\n---\n*4.Contribution Clarification*：\n- **Comment:** Using ODE-ConvGRU does not seem new (it is said it was proposed in [49]).\n- **Response:** Thanks for the comments. Here would like to further clarify the main contributions of our work (as been stated in the main manuscript).\n\n     In the paper, we present *continuous parametric optical flow*, which is a parametric representation of dense and continuous motion over arbitrary time intervals. This contribution has been acknowledged by all the reviewers (Reviewer dH6J ``the concept of continuous parametric flow is interesting``, Reviewer sWYx ``seems novel``, Reviewer JXsp ``the idea of estimating continuous parametric flow is interesting``, and Reviewer TYqX ``the idea of estimating continuous optical flow from a neural network seems relatively novel``).Note that Reviewer 7yvr also marked our idea in developing continuous parametric optical flow as ``simple and efficient `` and ``the proposed architecture is not trivial and well thought out ``.\n\n     In solving the problem, we utilize ODE convolutional GRU (ODE-ConvGRU) to encode implicit continuous features and establish multiple cost volumes for refinement. This combination of neural optimization and neural ODES for optical flow computation also makes novelty as agreed by the reviewers.\n\n     In this paper, we employed the ODE-ConvGRU unit for our novel problem and solution framework while we have not claimed the architecture design of ODE-ConvGRU as our contribution. Note that, the usage strategy and objective of ODE-ConvGRU in this paper also differs from existing work such as Reference [49]. In our paper, the ODE-ConvGRU encoder will generate continuous spatial-temporal features defined at specific timestamps rather than discrete frames from a few visible inputs and provide targets for multi-time cost volume construction.\n"}, {"review_id": "uQUV6y5VLB", "reviewer": "Reviewer_dH6J", "summary": "This paper presents a parametric representation of dense and continuous pixel motion over arbitrary time intervals. The  ``continuous parametric flow`` concept is interesting. However, one of the core technique contributions is encoding the image with the ODE-ConvGRU, which is not closely related to the  ``continuous parametric flow`` and simply replaces CNNs used in PIPs and RAFT.", "questions": "See Weakness.", "limitations": "Limited by the chosen parametric model, the predicted point trajectory is challenging to express complex motions.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The concept ``continuous parametric flow`` is interesting.\n2. ODE-ConvGRU is a stronger image feature encoder compared with CNNs. \n3. The B-spline-based flow interpretation is novel.", "weaknesses": "1. The comparison is unfair. The proposed model is trained with $N_{gt}=8$ but the PIPs is only trained with 4-frames. Can you provide the performance of an 8-frame PIPs model and the officially released PIPs model?\n2. What does the ``implicit`` mean in the ``implicit feature representation``? For me, it's just a feature encoded by the ODE-ConvGRU instead of a CNN. I don't think it's necessary to introduce the ``implicit`` concept.\n3. What's the relationship between the continuous flow with ODE-ConvGRU?\n4. The two-stage evaluation is not novel, and should not be regarded as a contribution.\n5. Figure 2 should be improved. The symbols in the ``cost volume`` and ``correlation pyramid`` regions are not well aligned. The legend of the concatenation is weird.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Weakness:** \n***\n*1.Unfair Comparison:* \n- **Comment:** The comparison is unfair. The proposed model is trained with $ N_{gt} = 8 $ but the PIPs is only trained with 4-frames. Can you provide the performance of an 8-frame PIPs model and the officially released PIPs model? \n- **Response1:** Thanks for the comment. In terms of the training setting, we expect to utilize a few sampling frames as inputs and achieve trajectory estimation through continuous optical flow estimation. **All baselines including PIPs and our model use the same e.g. 4 sampling frames as inputs**. However, as the reviewer mentioned, the number of supervision moments $N_{gt}$ is a different concept from input frames. According to our assumption,**all moments in certain videos are valid for our continuous model.** Hence, the supervision can be randomly selected from these sampling or un-sampling timestamps with real spatial-temporal corresponding. But for the discrete point-tracking methods, the supervision is limited to selected from sampling moments, which causes **the supervision number is equal to inputs**. \n- **Response2:** As the reviewer mentioned, the official implementation of PIPs is an 8-frame inputs model, we will use this PIP model for comparison with our method with 8-frame inputs. **The performance on real-world benchmarks is shown in Table 1 in the attached PDF**. As the table reports, our 8-frame model still outperforms  PIPs with the same inputs in almost all inference ranges. Even though more inputs will provide more context information and improve accuracy and robustness at sampling moments for both methods, our proposed continuous parametric flow seems more stabel and smooth at non-sampling moments especially during the long-term blind time and maintain tracking  process through occlusion.\n***\n*2.Implicit Representation:* \n- **Comment:** What does the implicit mean in the implicit feature representation? For me, it's just a feature encoded by the ODE-ConvGRU instead of a CNN. I don't think it's necessary to introduce the implicit concept.\n- **Response:** Thanks for the comment. In our paper, the *implicit* concept is used as the counterpart to explicit parametric constraints. The implicit features generated by ODE-ConvGRU aim to aggregate continuous spatial-temporal information and this could be understood as a kind of *memory* for a continuous video sequence similar to the video compression work Nerv[1]. \n***\n*3.Relations between Continuous Flow and ODE-ConvGRU:* \n- **Comment:** What's the relationship between the continuous flow with ODE-ConvGRU?\n- **Response1:** Thanks for the comment. We employ the ODE-ConvGRU unit to generate temporally continuous features defined at arbitrary timestamps by utilizing a handful of image embedding at sampling moments. These features will be used with random sampling for the construction of multi-time cost volume pyramids, which eventually guide the regression of control points in an iterative manner. Through these control points, every flow trajectory will be represented with an explicit B-spline curve.\n- **Response2:** The key part is how to obtain spatial-temporal features at *all moments* by limited temporal-free embedding at sampling moments. The first step is to aggregate all sampling features (the abbreviation of spatial embedding at sampling moments) for a suitable initial value in solving an ordinary differential equation (ODE). ConvGRU is used for this purpose. Neural ODE aims to compute feature differential by independent neural network and finally obtain continuous features by adaptive built-in ODE solvers like Euler or Runge-Kutta method.\n***\n*4.Two-Stage Evaluation:*\n- **Comment:** The two-stage evaluation is not novel, and should not be regarded as a contribution.\n- **Response:** As stated in our contribution, the two-stage evaluation pipeline is designed for continuous flow estimation as there is no related work for this area. In fact, we hope continuous flow should equip with two attributes: one is the fitting ability at non-sampling or blind moments, and the other is for arbitrary time-to-time correspondence. Considering that the candidate moments for the continuous task are unlimited, we have to validate our method by the above two sub-tasks. We believe this also makes a contribution to the community.\n***\n*5.Present Improvement:*\n- **Comment:** Figure 2 should be improved. The symbols in the cost volume and correlation pyramid regions are not well aligned. The legend of the concatenation is weird.\n- **Response:** Thanks for the suggestion. We will modify the legends of Figure 2 and ensure the aesthetic of the whole figure. The revised Figure 2 is provided in the attached PDF.\n***\n**Limitations:** \n***\n*1.Parametric Model:*\n- **Comment:** Limited by the chosen parametric model, the predicted point trajectory is challenging to express complex motions.\n- **Response:** Thanks for the comment. We agree that all parametric models depend on motion prior in special scenes and appear limited to real-world complex motions. However, different from polynomial curves controlled by the only parameter *degree*, in this paper, we choose B-spline as our fitting model because the shape of this curve is additionally dependent on adjustable control points regressed by our end-to-end framework and partition knots. By adjusting certain control points, the local segment of the curve will be flexibly changed. Due to this partition optimization, B-spline enables higher flexibility in handling complex motions.\n***\n[1] Chen H, He B, Wang H, et al. Nerv: Neural representations for videos[J]. Advances in Neural Information Processing Systems, 2021, 34: 21557-21568."}, {"review_id": "hWTKHFu0Nd", "reviewer": "Reviewer_sWYx", "summary": "This paper suggests a new model to pixel wise compute temporally continuous optical flow by using B-splines. The input to the neural model are sequences of images, the output are N 2D control points for each pixel of the spline model. During training, the input is sampled from the dataset at varying time instants and by varying number of samples. The model architecture builds on known design patterns such as CNN feature encoding, iterative optimisation (search) by using a recurrent GRU network, by using correlation pyramid to compute motion features. New elements are the ODE modelling of the continuous trajectories and the overall decoding (regression) of the control points for each pixel.", "questions": "The model infers for all pixels in the reference image a 2NxWxH tensor with the control points of the pixel's B-splines. How do you infer the flow vector of all the pixels in the successive images (2nd, 3rd, ...) in the sequence?\n\nMight trajectories of different pixels in the reference (1st) image collapse to a single pixel in some image in the sequence?\n\n", "limitations": "A first assessment of the impact has not been given by the authors. Even fundamental work on motion analysis & tracking should project potential impacts on foreseen applications and uses or policies such as the UN SDGs. Focus on computer vision as a field and try to explain how your research might affect the field in the next years. Try then to explain how the interaction of computer vision with society might be affected by your research, e.g. a seamlessly tracking combined with generative models might in future it even harder to distinguish fake video from the real.\n\nGive real world examples why the B-spline model is limiting, why you need parametric motion and how you believe this dilemma could be overcome in future. The B-spline motion is smooth and introduces motion consistency which is in some cases a disadvantage, e.g. when it comes to abrupt motion cases.\n", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The paper introduces a model able to continuously capture motion in time. Applications like video editing could profit of such capabilities. The combination of neural optimisation and neural ODEs for optical flow computation seems novel to me.\n\n", "weaknesses": "Some insecurity in the argumentation, e.g. line 28: \"real motion follows ... principles\" - is there some motion of interest that does not follow physical principles?\n\nThe related work should be structured by the motion model, e.g. non-parametric motion, parametric motion. GMA is an attempt to overcome the limitations of non-parametric motion which is not clear by reading the text. The line of work called tracking (started with Lucas-Kanade) is non-parametric but sparse estimation of the apparent motion whereas particle video (Sand, Teller) tried to find a model in between dense optical flow and KLT.\n\nline 160: xhi is not shown in Figure 2\n\nline 166: computing correlation between features at reference time and target time is limited, e.g. think of in plane rotations.\n\nThere are typos in the text, e.g. line 254\n\nline 233, Eq. 10: I do not understand \"sampled point trajectories\". Why is N_s needed in the metrics?\n\nThe spline is also continuous in the image, not only in time, but the ground truth available in the datasets  is discretly sampled, so how do you match an estimate F(t_k) to F* in Eq. 10?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Weakness**\n***\n*1.Motion Claim:*\n- **Response:** Thanks for the comment. We agree that all the real-world motions follow classic physical principles. Thus, it is important to incorporate physical principles based on explicit constraints rather than directly using a neural network to regress point trajectories. Solely neural network based solutions may potentially cause abrupt changes that do not follow real-world physical principles.\n***\n*2.Related Work:*\n- **Response:** Thanks for the valuable suggestion. Firstly, in this paper, we mainly highlight continuous pixel motion and propose the novel concept *Continuous optical flow* to describe and utilize the combination of parametric curve and implicit continuous feature to achieve flow estimation. Hence, the parametric or non-parametric modelling is just one part of the complete framework. \n-   In terms of the structure of related work, we divide into three subsets according to the relations with the concept, vision application using recent techniques and parametric motion assumptions. Every part is connected with different aspect.\n-  Besides, we do not state that GMA (Optical flow estimation) intend to overcome limitations of non-parametric motion and only cite it as an example of iterative update pipeline. For the tracking part, we intend to state some long-term point trackers could achieve stable frame-to-frame mapping with multi-frame inputs (exactly the whole sequence) and inference whole trajectory at once, which is similar with the effect of our work. As for the Particle Video, we intend to cite it as the inspiration of PIPs.\n***\n*3.Figure Improvement:*\n- **Response:** Thanks for the comment. As suggested, we have updated Figure 2 as in the attached PDF.\n***\n*4.Correlation Limitation:*\n- **Response:** Thanks for the comment. For in-plane rotations, correlation between features at reference time and target time based on matching framework can describe well without the influence of perspective effect. Thus, we do not believe there is problem in computing correlation there.\n***\n*5.Typos:*\n- **Response:** Thanks for pointing out the typos. We will fix all the typos in the revised version.\n***\n*6.TRMSE Explanation:*\n- **Response:** Thanks for pointing out the issue. Eq.~(10) aims to measure the trajectory smoothness by using the TRMSE metric. $N_s$ is the number of all valid pixel trajectories (sampled point trajectories) in spatial dimension. For every pixel, the algorithm will infer a continuous trajectory. In this metric, we expect to obtain a spatially-mean value through this parameter.\n***\n*7.Contiguity Explanation:*\n- **Response:** Thanks for the comment. According to classic definition, optical flow is naturally dependent on 2D pixel coordinates. The parametric continuous curve in our work is only used for temporal representation. Hence, our model will provide an independent point trajectory for every pixel as mentioned in above TRMSE explanation. Considering this situation, we can directly sample corresponding ground truth at given moment for error measurement. In fact, the parametric model have been constructed when all control points are decided. Thus, for an arbitrary timestamp $t_k$, we could sample from this continuous curve and obtain the estimate $F(t_k)$.\n***\n**Questions**\n***\n*1.Trajectory Inference:*\n- **Response:** Thanks for the comment. As mentioned in our paper, the model generates all control points of B-splines for every pixel at once. According to the B-spline formulation, we only need a timestamp and could get corresponding flow vector relative to the reference time. Then, the flow vector of all the pixels in the successive images (2nd, 3rd, ...) in the sequence will be generated correspondingly. In our setting, the sequence of frames will be allocated a normalized timestamp calculated by division of frame index and sequence length. Besides, our model will directly infer long-range correspondences and not dependent on step-by-step iteration like classic optical flow limited to adjacent frames.\n\n*2.Pixel Independence:*\n- **Response:**  Thanks for the comment. In our paper, every single pixel's continuous flow is computed *independently*. Thus, whether multiple different trajectories collapse to a single pixel in some image does not matter any all. Interestingly, this will happen why occlusion is observed in 3D space.\n***\n**Limitations**\n***\n*1.Assessment of work:*\n- **Response:**  Thanks for the valuable comment.Through our method, stable continuous optical flow will provide realistic spatial-temporal corresponding priors for vision tasks such as non-rigid structure-from-motion, simultaneous localization and mapping, and video understanding of dynamic scenes. On the potential social impact, as the reviewer mentioned, continuous pixel motion trajectory could generate realistic dynamic change utilizing a few sequences and form a novel video. Hence, this technique can potentially make fake video with image generative framework and bring more challenges for official regulatory agencies. We will further elaborate the social impact as the reviewer suggested in details.\n***\n*2.B-spline Limitations:* \n- **Response:**  Thanks for the valuable comment. We agree that real-world  pixel motion could be rather complex. Consequently, for polynomial curves controlled by the only parameter *degree*, the flexibility will be limited. In this paper, we choose B-spline as our fitting model because the shape of this curve is additionally dependent on adjustable control points regressed by our end-to-end framework. By adjusting certain control points, the local segment of the curve will be be changed with higher flexibility. Even though flexible spline curves like B-spline could achieve improved ability in handling complex motions, we still have to admit the limitation in *all parametric models*, where a strong explicit constraint potentially leads to better performance on those more similar motion scenery with the training set.\n"}, {"review_id": "OMLsicI2Ut", "reviewer": "Reviewer_JXsp", "summary": "This paper proposed a continuous parametric optical flow estimation algorithm with B-spline temporal trajectory representation and ODE-ConvGRU based feature extraction. Experiment has been done on both synthetic and real-world dataset and the proposed continuous parametric method performs better than traditional flow-based and point-tracking based methods.", "questions": "1. How about the computational complexity / running speed of the proposed algorithm? how about the memory cost?", "limitations": "I donot see any potential negative societal impact", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The idea of estimating continuous parametric flow is interesting and the achieved experimental result is promising. The presentation is clear and easy to follow. The ablation study clearly shows the benefits of B-spline representation and ODE-ConvGRU feature extraction.", "weaknesses": "1. It is somewhat unclear how to specify the flexibility of the trajectories, as in Table 3, different number of control points may affect the performance significantly, and in real scenarios, different objects in a scene may have different degree of motion complexities, it is not clear whether a single choice of B-spline number can handle this?\n2. In table 2, the 'TRMSE' lines in 'PIPS' and 'ODE-6spline' under Vid-DAVIS dataset is exactly the same, this is problematic since their 'ADE_All' metrics are different, why?\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Weakness**\n***\n*1.Flexibility of Parametric Curve:*\n- **Comment:** It is somewhat unclear how to specify the flexibility of the trajectories, as in Table 3, different numbers of control points may affect the performance significantly, and in real scenarios, different objects in a scene may have different degrees of motion complexities, it is not clear whether a single choice of B-spline number can handle this? \n- **Response:** Thanks for the valuable comment. We agree that real-world  pixel motion could be rather complex. Consequently, for polynomial curves controlled by the only parameter *degree*, the flexibility will be limited.   In this paper, we choose B-spline as our fitting model because the shape of this curve is additionally dependent on adjustable control points regressed by our end-to-end framework. By adjusting certain control points, the local segment of the curve will be flexibly changed. Thus, for the splines, the degree only decides global complexity and the recovery of motion detail is achieved by locally optimization from multiple control points. Due to this partition optimization, **B-spline enables higher flexibility in handling complex motions**. Even though flexible spline curves like B-spline or Bezier curve could achieve improved ability in handling complex motions, we still have to admit the limitation in *all parametric models*, where a strong explicit constraint potentially leads to better performance on those more similar motion scenery with the training set.\n***\n*2.Data Error:*\n- **Comment:** In table 2, the 'TRMSE' lines in 'PIPS' and 'ODE-6spline' under the Vid-DAVIS dataset are exactly the same, this is problematic since their 'ADE\\_All' metrics are different, why?\n- **Response:** Thanks for pointing out this issue. We are sorry that we unexpectedly copy erroneous TRMSE results from raw data, displaying exactly the same performance in PIPs and ODE-6spline. Here we show the corrected tables as below. \nAs the table reported, the TRMSE metric of ODE-6spline in fact outperform PIPs and RAFT in all inference range of the TAP-DAVIS benchmark and appear similar trend in other benchmarks.\n\nTable. Updated TRMSE metric of ODE-6spline in Vid-DAVIS benchmark\n|       Method       | Metric | Benchmark |    20f    |    24f    |    28f    |    32f    |\n|:------------------:|:------:|:---------:|:---------:|:---------:|:---------:|:---------:|\n| ODE-6spline (Ours) |  TRMSE | Vid-DAVIS | **12.96** | **15.76** | **18.12** | **20.27** |\n\n***\n**Questions**\n***\n*1.Computational Complexity & Memory Cost:*\n- **Comment:** How about the computational complexity / running speed of the proposed algorithm? how about the memory cost?\n- **Response1:** Thanks for the valuable comment. As requested, we provide the running speed and computational complexity below. Due to the samples in all benchmarks being 256x256, we use Vid-DAVIS to test the average runtime in the 20-frame inference range and memory cost for our method and all baselines.\n- **Response2:** Our method performs **more efficiently with shorter inference time** to generate spatially-dense and temporally-continuous pixel trajectories rather than baselines because an independent parametric curve for every pixel could directly provide all corresponding at once instead of multi-step iteration along temporal sequences based on original optical flow or along spatial coordinates based on long-term point tracker with sparse query points. \n- **Response3:** For faster inference, our proposed algorithm relies on multi-frame inputs and multi-time cost volume, resulting in a controllable increase in memory compared to traditional two-frame flow and robustness for tracking in parallel instead of significant expansion in memroy caused by single point tracker.\n\nTable. Comparison of Runtime and Memory Cost on Vid-DAVIS \n| Method | Runtime/ms | Memory/MB |\n|:------:|:----------:|:---------:|\n|  Ours  |    274.6   |    2442   |\n|  RAFT  |    338.2   |    1902   |\n|  PIPs  |   9073.7   |   18042   |\n"}, {"review_id": "ymYVIMjkZd", "reviewer": "Reviewer_TYqX", "summary": "The submission 6736, entitled \"Continuous Parametric Optical Flow,\" presents a novel multi-frame optical flow strategy that expresses the flow continuously. This is in contrast to conventional flow strategies, which encode this displacement in a discretized manner. This result is made possible by regressing the B-spline parameters of the pixel displacement. The approach shares similarities with RAFT and PIPs, in that this motion is refined iteratively via a convolutional-GRU, and the network can accommodate a series of successive images as input (more than 2). An extensive series of experiments underline the approach's effectiveness against existing state-of-the-art in optical flow and keypoint tracking. Additionally, novel datasets specifically dedicated to the task at hand have been created.", "questions": "I have expressed most of my questions in the previous section of this review. Despite all the shortcomings mentioned earlier, I found this work new and interesting, and therefore, I would like to issue a relatively positive opinion regarding its acceptance. However, the reviewer would like to specify that he has not been working on such a topic lately and has low confidence.\n", "limitations": "See previous sections.", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The idea of estimating continuous optical flow from a neural network seems relatively novel. \n* The performance reported in the manuscript is very competitive.", "weaknesses": "* While the overall editorial quality of the paper is acceptable, it still contains a large number of typos, and certain parts of the paper would benefit from additional proofreading. The flow of the paper sometimes lacks smoothness and transitions.\n* The literature review is rather short and fails to position the paper with respect to other approaches. It gives the impression that the paper is poorly motivated. It would be helpful to underline more clearly the differences with other techniques and why this approach is substantially better.\n* Similar to the previous comment, it would be interesting to list what downstream applications could benefit from this technique (3D? Tracking? Stabilization?).\n* One of my main gripes regarding this manuscript is the lack of scalability. Due to its ability to input multiple frames, the network requires a significant amount of memory. This limits its applicability and raises the question of fairness when compared with less memory-intensive techniques. \n* Another important point to address is the lack of clear contribution. In the reviewer's opinion, the main novelty of this paper is the type of output as B-splines, while the rest of the work is strongly inspired by other techniques.\n* What are the scalability and clear limitations of the technique? How does it perform under very small or very large motions? It would be interesting to investigate such specific use cases. \n* In Table 3, \"Neural-ODE\" is written, but in the method section, it is written as \"Basic\" while all other techniques are using ODE.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "*1.Typos and Fluency:*\n- **Response:** Thanks for the valuable suggestion. We will fix all the typos and proofread the manuscript as suggested.\n***\n*2.Motivation & Related Work:*\n- **Response:** \n- Thanks for the suggestion. In the paper, first, as agreed by all the reviewers, our novel concept of *continuous optical flow* describes **dense and continuous pixel motion displacement** regarded as an extension of classic optical flow dependent on adjacent frames. Inspired by RAFT's iterative optimization, we use similar techniques to construct cost volume and set the proper decoder for our task but introduce a continuous encoder and fuse explicit parametric curve. Notably, the step-by-step method based on chained flow is inevitable to generate drift and fails through occlusion. **In summary, our flow can generate all continuous correspondence at once with better accuracy after occlusion.**\n-  Secondly, as for the closest vision task, generic point tracking also achieve long-term fine-grained matching for frame-to-frame mapping.  With large-scale synthetic training and multi-frame aggregation, these methods like PIPs achieve better generalization ability and robustness. However, there are two drawbacks in current techniques: the discrete correspondence far from continuous displacements inherently and sparse tracking hard to infer in parallel for all pixels at once. **In summary, our method could generate dense and continuous point tracking more efficently**.\n- Thirdly, as for the continuous modeling, we investigate other motion estimation methods. In the video interpolation field, there are some classic motion assumptions like linear, quadratic, and cubic assumptions. For the balance of flexibility and accuracy, we choose B-spline with multiple adjusted control points as our regression objective which is rare in previous papers. **Utilizing a parametric but more flexible representation is also more suitable for continuous motion modeling.**\n- To conclude, continuous parametric optical flow is proposed to describe *spatially-dense and temporally-continuous pixel motion* with continuous encoded features and flexible parametric curve. **The current techniques are greatly limited for this task.**\n***\n*3.Downstream Applications:*\n- **Response:** Thanks for the comment. Our method could provide spatial-temporal long-term correspondences for 3D vision tasks such as non-rigid structure-from-motion[1]  and simultaneous localization and mapping [2]. Another widespread downstream application is video analysis of dynamic scenes like arbitrary frame-rate video interpolation [3] and human keypoint tracking [4]. For the promotion of this field with more real motion data,[5] recently releases a novel large-scale dataset with high-quality annotation.\n***\n*4.Scalability:*\n- **Response:** Thanks for the comment. On the scalability issue of our method, we think that multi-frame inputs require more memory than iteratively-update methods based on classic optical flow. But our method could support longer frame inputs with a slight increase in memory because we randomly select a fixed number of features to guide the model. Compared to those methods with less memory, our method will show better performance on accuracy and run time.\n\nTable Comparison of Runtime and Memory Cost with Differnet Inputs\n| Input Frames | Runtime/ms | Memory/MB |\n|:------------:|:----------:|:---------:|\n|       4      |    274.6   |    2442   |\n|       8      |    280.3   |    2456   |\n|      12      |    304.3   |    2482   |\n|      16      |    307.4   |    2503   |\n\n***\n*5.Contribution:*\n- **Response:** Thanks for the comment. First, the novel concept of *continuous optical flow* is proposed to describe dense and continuous pixel motion with an explicit parametric curve B-spline. \nSecondly, The combination of neural optimization and neural ODEs (ODE-ConvGRU) for optical flow computation also makes novelty as agreed by the reviewers with different usage strategy and objective from existing video interpolation work. Thirdly, the field of continuous flow is almost blank, hence, the performance of algorithms needs a special evaluation system for nonsampling fitting and generalization. According to our investigation, it is also a valuable contribution to the training on an independent simulation dataset.\n***\n*6.Limitations:* \n- **Response:** Thanks for the constructive comments. In terms of the scalability of our method, as mentioned in the above response, we need multi-frames as inputs with a slight increase in memory but obtain better performance for occlusion and inference speed. We think the limitation in our work is about the parametric model because strong motion prior will potentially obstacle learning for out-of-distribution generalization and long-term prediction because of the complexity of real motion.  As requested by the reviewer, here we provide some special cases like small or large motions in Figure 1 in the attached PDF.\n***\n*7.Ablation:* \n- **Response:** Thanks for the comment. In ablation studies, the module ``Neural-ODE'' means we validate the function of this module. Hence,''Basic + 6spline'' denotes the setting that we use basic CNNs without the Neural-ODE module for evaluation while the full version with the Neural-ODE module is termed as ''All''. For other modules, the Neural-ODE encoder is used as the same setting to guarantee fair comparison.\n***\n\n[1] Sidhu V, Tretschk E, Golyanik V, et al. Neural dense non-rigid structure from motion with latent space constraints.\n\n[2] Fu Q, Yu H, Wang X, et al. Fast ORB-SLAM without keypoint descriptors.\n\n[3] Park S, Kim K, Lee J, et al. Vid-ode: Continuous-time video generation with neural ordinary differential equation.\n\n[4] Kreiss S, Bertoni L, Alahi A. Openpifpaf: Composite fields for semantic keypoint detection and spatio-temporal association.\n\n[5] Zheng Y, Harley A W, Shen B, et al. PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking."}]

Author rebuttal:
**Global Response**

In the attached PDF, we provide two figures and one table. Table 1 reports the comparison with PIPs with 8-frame inputs on real-world datasets. Figure 1 shows Special Cases of extremely Small Motion & Large Motion. Figure 2 illustrated the updated pipeline of our proposed framework.


Meta-review: Four reviewers recommend "borderline accept". From their comments it transpires that they all have a rather positive view of the paper,  clearly acknowledging that there is a novel and interesting contribution. On the contrary, one reviewer recommends rejection, but actually the strengths they list in their review appear to outweigh the listed weaknesses. The AC has read the paper and the authors' rebuttal, and concluded that the combination of a spline-based parametric flow model with the NODE idea is novel enough to warrant acceptance.
Decision: Accept (poster)