PPLNs: Parametric Piecewise Linear Networks for Event-Based Temporal Modeling and Beyond

Authors: chen  song
zhenxiao  liang
bo  sun
qixing  huang
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-13 23:58:04

Abstract: We present Parametric Piecewise Linear Networks (PPLNs) for temporal vision inference. Motivated by the neuromorphic principles that regulate biological neural behaviors, PPLNs are ideal for processing data captured by event cameras, which are built to simulate neural activities in the human retina. We discuss how to represent the membrane potential of an artificial neuron by a parametric piecewise linear function with learnable coefficients. This design echoes the idea of building deep models from learnable parametric functions recently popularized by Kolmogorov–Arnold Networks (KANs). Experiments demonstrate the state-of-the-art performance of PPLNs in event-based and image-based vision applications, including steering prediction, human pose estimation, and motion deblurring.

Main paper: 
 introduction: 
 Event cameras are neuromorphic sensors that summarize the evolving world as a stream of events. Each event describes the pixel coordinates, time, and polarity of an intensity change. Thanks to the simplicity of this data representation, event cameras enjoy multiple advantages over conventional cameras, including but not limited to fast data rate and high dynamic range (Gallego et al., 2020) . Over the past, researchers have developed event-based algorithms to solve various computer vision problems, such as motion deblurring (Pan et al., 2019 (Pan et al., , 2020;; Wang et al., 2020; Song et al., 2022 Song et al., , 2024)) , human pose estimation (Calabrese et al., 2019) , and autonomous driving (Binas et al., 2017; Hu et al., 2020) . A recent survey (Zheng et al., 2023) indicates a rapidly increasing community interest in event-based research, with motion deblurring being the most popular task. Extensive experiments demonstrate that by utilizing events as an auxiliary input, algorithms perceive fine motion details that are absent from conventional image captures, leading to substantial performance gain over methods that make inferences from conventional images alone.The success of event cameras demonstrates the power of imitating biological neuromorphic principles. The event camera comprises a rectangular array of dynamic vision sensors, each of which is analogous to a visual receptor neuron on the human retina dedicated to perceiving one specific location. The neuron experiences excitement and produces a spike when the environmental intensity varies significantly, corresponding to an event generated as a response to brightness changes. This paper presents Parametric Piecewise Linear Networks (PPLNs) to understand the bio-inspired event data with a bio-inspired deep learning model. As opposed to a generic network design, we believe and verify in this paper that it is highly beneficial to build a processing network that caters to the principles of the data source (event cameras). As illustrated by Figure 1 (Middle), the key idea is to explicitly approximate the membrane potential of a neuron as a piecewise linear mapping from time to electric voltage. Figure 1 (Right) presents the sketch of a PPLN node, whose internal mechanism The membrane potential, defined as the voltage difference between the interior and the exterior of the cell, regulates the neuron's behavior and can be approximately modeled by a piecewise linear function. ① When the neuron is at rest, the potential stays at a constant level V 0 . ② An external input is received by the dendrites, causing an instantaneous perturbation to the membrane potential. ③ The perturbation is not significant enough to excite the neuron, and the potential leaks over time exponentially (i.e., linearly in the logarithmic space). ④ Another external input happens. ⑤ The input fails to excite the neuron. ⑥ A third input causes the membrane potential to exceed the threshold voltage V th . The neuron becomes excited and generates a spike. ⑦ The excitement opens ion channels, and the ion flow causes a reset to the membrane potential. ⑧ After the excitement, the ion channels close again, and the potential continues to decay. ⑨ The neuron returns to the resting state, waiting for new inputs. (Right): A PPLN node. Given inputs {x i } k i=1 , we predict the linear coefficients Θ for the membrane potential function, including the slope, intercept, and endpoints of each line segment. The resulting parametric function ṼΘ is then used to evaluate the neuron output at the timestamp of interest y(x 1 , . . . , x k , t) = σ( ṼΘ (t)), where σ(•) is the integral normalization defined in Section 3.4. is explained thoroughly in Section 3. While the event camera imitates how a single layer of visual receptor neurons react to external intensity changes, PPLNs are motivated by observations of how layers of biological neurons communicate. Inspired by the Leaky Integrate-and-Fire model (Abbott, 1999) , we propose to use a piecewise linear parameterization to approximate its temporal evolution in the logarithmic space. The key difference between PPLNs and the bio-inspired Spiking Neural Networks (SNNs) (Eshraghian et al., 2021) in existing literature is that PPLNs are an alternative to general GPU-based temporal inference models, whereas SNNs aim at the deployment on hardware neuromorphic chips (Davies et al., 2018) . Instead of real-valued membrane potentials, SNNs also propagate binary spikes, leading to reduced energy consumption and training instabilities.PPLNs are conceptually similar to the emerging Kolmogorov-Arnold Networks (KANs) (Liu et al., 2024) . Both KANs and PPLNs leverage learnable parametric functions to build a deep network. Different from KANs, which use input-independent B-splines, PPLNs exploits input-dependent piecewise linear functions. Using piecewise linear functions allows our brain-inspired design to mimic biological neural principles and the event generation model. Predicting function coefficients at inference time allows the network to better handle input heterogeneity.We modify the network architecture in state-of-the-art event-based vision algorithms by replacing multi-layer perceptions and convolution operators with PPLN nodes and evaluate PPLNs on three applications: steering prediction, 3D human pose estimation, and motion deblurring. With fewer or a similar number of trainable parameters, PPLNs improve baselines by 30.8% in steering prediction, 11.1% in human pose estimation, and 5.6% in motion deblurring. To demonstrate the potential of PPLNs, we experiment on the conventional frame-based version of the same applications without the event input and observe consistent improvements. Additionally, we present a mathematical analysis of the convergence properties to showcase the robustness.In summary, we make the following contributions:• We propose Parametric Piecewise Linear Networks (PPLNs), mimicking biological principles by approximating membrane potentials as parametric mappings. • We show how to predict a set of parametric coefficients from the input of the PPLN node and evaluate the membrane potential at any timestamp of interest.• We present a mathematical analysis of the convergence properties of PPLNs.• We apply PPLNs to event-based and frame-based applications and achieve state-of-the-art results. 
related work: 
 Biological neurons. As shown in Figure 1 (Left), if the combined effect of small perturbations over time causes the membrane potential to exceed the threshold potential, the neuron will become excited.During the excitement, the neuron produces an output spike through the synapses (Lacinová, 2005; Maeda et al., 2009) . In Figure 1 (Middle), the Leaky Integrate-and-Fire model (Abbott, 1999) suggests that the membrane potential can be approximated by a piecewise linear function as a parametric mapping from time to the logarithm of the voltage difference.Spiking neural networks. Spiking Neural Networks (SNNs) are Recurrent Neural Networks (RNNs) that simulate biological neurons (Eshraghian et al., 2021) . Each SNN node carries an internal variable corresponding to the membrane potential. The output of the SNN node is a binary signal that is zero by default and becomes one if excited (Abbott, 1999) . The key advantage of SNNs is low power consumption, making them the ideal model to be deployed on hardware neuromorphic chips (Davies et al., 2018) . By contrast, PPLNs are designed to be an alternative to general GPU-based temporal inference models, even though SNNs and PPLNs are motivated by similar biological principles.Event cameras. Event cameras are neuromorphic devices that summarize the evolving environment as a stream of events (Lichtsteiner et al., 2008) . Each event is analogous to a significant intensity change that exceeds the hardware threshold and excites a biological light-sensing neuron. An event is represented as a 4-tuple (x, y, t, p), where (x, y) are the pixel coordinates, t is the timestamp, and p ∈ {-1, +1} is the polarity of the intensity change. Event cameras have a fast data rate, high dynamic range, low power consumption, and minimal motion blur compared to conventional cameras (Gallego et al., 2020) . A recent trend is to utilize hardware that simultaneously captures event and conventional streams, where the event stream is used as an auxiliary input to enhance regular computer vision algorithms. For example, while image-to-image deblurring is a well-studied problem in the research community (Richardson, 1972; Fish et al., 1995; Krishnan & Fergus, 2009; Joshi et al., 2009; Levin et al., 2007; Kim et al., 1998; Shan et al., 2008; Fergus et al., 2006; Xu et al., 2013; Xu & Jia, 2010; Perrone & Favaro, 2014; Babacan et al., 2012; Kupyn et al., 2018 Kupyn et al., , 2019) ) , several works in event-based vision demonstrate the possibility of converting a blurry image into a sharp video that explains the motion during the exposure interval (Pan et al., 2019 (Pan et al., , 2020;; Wang et al., 2020; Song et al., 2022 Song et al., , 2024)) .Learning activation functions. The Rectified Linear Unit (ReLU) (Nair & Hinton, 2010) represents a two-piece linear activation function, f (x) = x if x > 0, and f (x) = 0 if x ≤ 0 and has several variants. Our design is conceptually similar to the Piecewise Linear Unit (PWLU) (Zhou et al., 2021) , where the activation is defined as an n-piece linear function with learnable slopes and intercepts. Kolmogorov-Arnold Networks (KANs) have also received wide attention, which build a deep model by stacking layers of parametric B-spline activation functions (Liu et al., 2024) . In addition to the conceptual similarities, PPLNs are fundamentally different from PWLUs and KANs since PPLNs incorporate temporal modeling. Additionally, PPLNs allow discontinuities at segment endpoints and the learning of endpoint locations, neither of which is supported by PWLUs or KANs. Furthermore, the ReLUs are activation functions to be appended after prediction layers (e.g., linear and convolution layers), whereas the PPLNs are designed to replace the prediction layers in temporal learning models.Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: Discussed in Section A.8. Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
overview: 
 As shown in Figure 2 (a, b), a PPLN node implements the following mapping:f : R k × [0, 1] → R (1)where x ∈ R k is the k-dimensional non-temporal component of the input, and t ∈ [0, 1] is the normalized input scalar timestamp. The mapping f converts the input (x, t) to a scalar in R.The first step in the calculation of f is to predict the linear coefficients, Θ = {m, b, s}, using the trainable parameters, W m , W b , W s , and w V . Section 3.2 explains this process in detail. The predicted coefficients allow us to assemble the piecewise linear membrane potential function, as illustrated by the blue plot in the bottom-left corner of Figure 2 (a, b). To better handle the numerical instability, Section 3.3 discusses the procedure to smooth the boundaries of the predicted linear pieces. The smoothed function is then normalized by another predicted value V , as explained in Section 3.4. Finally, the output of the PPLN node is given as f (x, t) = σ( ṼΘ (t)) (i.e., the smoothed, and normalized potential function evaluated at the timestamp of interest). While it is straightforward to construct a fully-connected network by stacking PPLN nodes, Section 3.5 discusses how to support convolution operations.Let n be a hyper-parameter denoting the number of line segments in the piecewise linear modeling.The parametric coefficients Θ = {m, b, s} are given by:EQUATIONwhere W m , W b , and W s are n × k dimensional matrices containing trainable weights. m = (m 1 , . . . , m n ) T and b = (b 1 , . . . , b n ) T are the slopes and intercepts of n different line segments, respectively. s = (s 1 , . . . , s n ) T defines the temporal interval size for each line segment. Let t 0 = 0 and t i = t i-1 + s i . With the softmax function, the temporal space [0, 1] is divided into n non-overlapping intervals by s. We predict the aforementioned coefficients and approximate the membrane potential as:EQUATIONHere, the hyperbolic tangent function restricts the slope, m, preventing the exploding gradient problem commonly observed in temporal models (Pascanu et al., 2013; Bengio et al., 1994) .While we can build a network by stacking layers of the vanilla PPLN nodes described above, training presents a challenge for numerical stability. Equation (5) suggests that ∂ ṼΘ(t)∂t is an all-zero vector. In other words, gradient-based optimizers (Bottou et al., 1991; Kingma & Ba, 2014) cannot update the values of t i 's (the location of segment endpoints).To address this issue, we propose to smooth the segment boundaries. The key idea is to blend the linear pieces across adjacent intervals. Let (x, π i (x)) be the point on the i th predicted segment, that is, π i (x) := m i x + b i .Assuming t i-1 ≤ t < t i (i.e., t belongs to the i th predicted segment), the smoothed potential is:EQUATIONwhere the weights w (i) ← and w (i) → are defined through the temperature hyper-parameter T :w (i) l := 0 i = 1 1 + exp T (t -ti-1) -1 i = 2, . . . , n w (i) r := 1 + exp T (ti -t) -1 i = 1, . . . , n -1 0 i = n .Importantly,lim T →+∞ Ṽ T Θ (t) = ṼΘ(t).The difference between the smoothed potential, Ṽ T Θ (t), and the unsmoothed potential, ṼΘ (t), is that the smoothed gradients∂ Ṽ T Θ (t) ∂tdo not vanish. We present a theorem with proof in the appendix showing the local convergence properties of the piecewise linear model after smoothing. Assuming the underlying function to fit is indeed an n-piece piecewise linear function, the theorem states that the coefficients can be accurately learned from a set of noisy samples, provided that the noises are reasonably small, the coefficients are adequately initialized, and the temperature T is sufficiently large. For ease of discussion, the theorem uses segment endpoints t = {t i } instead of interval lengths s for the parameterization. Their relation is given in Section 3.2. Theorem 3.1. (Informal) Consider an underlying n-segment piecewise linear function parameterized by Θ ⋆ = {m ⋆ , b ⋆ , t ⋆ } as defined in (5). Let (τ j , v j ), j = 1, . . . , m be m point samples, where v j = ṼΘ ⋆ (τ j ) + ψ j in which ψ j is a small random noise.The L2 loss for the smoothed curve is defined by:EQUATION)Denote by Θ ⋆ T the weights at which the minimum of ( 7) is attained at temperature T . Then we show that starting from some initial Θ 0 close to Θ ⋆ , by applying vanilla gradient descent with appropriate temperature increase strategy and a learning rate η = O( 1 T ), Θ is guaranteed to converge to Θ ⋆ ∞ at a linear convergence rate. Specifically, the error of recovered segments is bounded by:EQUATIONwhere τ min , τ max are the smallest and largest values among τ j , for j = 1, . . . , m, respectively.In addition to the theorem above, the supplementary material uses ablation studies to discuss the practical implication of incorporating the smoothing operation.While the smoothing operator introduced above enriches temporal gradients, ∂ ṼΘ(t) ∂t , the integral normalization operator addresses the issue that ∂ ṼΘ (t) ∂m and ∂ ṼΘ(t) ∂b are both very sparse vectors with only one non-zero entry out of all n elements. From Equation (5), we have:EQUATIONLet V be a parameter that controls the mean of ṼΘ (t) when 0 ≤ t ≤ 1. The integral normalization operator σ(•) is defined as:EQUATIONAfter the normalization, ∂σ( ṼΘ(t)) ∂m and ∂σ( ṼΘ(t)) ∂b are both dense vectors containing rich gradient information in every element, encouraging a smooth and swift convergence. A side effect of the normalization is that the temporal derivative, ∂σ( ṼΘ(t)) ∂t , also becomes non-zero, allowing segment endpoints to be learned even without smoothing.Notably, the ground-truth parameter V is observable in certain applications. In motion deblurring, the task is to generate a sharp video from a blurry image. Mathematically, the mean of all the frames in the output must be equal to the input, restricting the temporal average 1 0 σ( ṼΘxy (t)) to be equal to the input pixel values. When it cannot be easily observed, V is regressed from the input x:EQUATIONwhere w V is a k-dimensional vector of trainable weights, and ⟨•, •⟩ stands for the inner product.Section 4.4 uses ablation studies to demonstrate the effectiveness of integral normalization. In the appendix, we additionally use a two-piece toy example to analyze the normalization.The above modeling naturally extends to convolutions, which are the fundamental building blocks of contemporary deep learning. In a convolution layer, each input pixel only affects the output in a small spatial neighborhood rather than across the entire grid. To support convolution, the trainable parameters, W m , W b , W s , and w V , become sparse matrices and vectors with non-zero entries only in locations within the spatial perceptive field. In practice, we predict the coefficients as:EQUATIONwhere, after reshaping, W m , W b , W s , and w V are kernels in the convolution operation. The channelwise softmax operator ensures the temporal interval sizes of each pixel add up to one, which is a requirement posed by the valid range of input timestamps (t ∈ [0, 1]). We refer interested readers to our code release for how the above design is implemented under PyTorch. 
evaluation: 
 Section 4.1 starts by showing how PPLNs outperform various methods in motion deblurring, the most popular event-based application as indicated by a recent survey (Zheng et al., 2023) . In Sections 4.2 and 4.3, we proceed with two other tasks where the goals are to predict the vehicle's steering angle from the dashcam footage and to estimate the 3D human pose from binocular 2D event camera captures. These are two "mainstream" applications in event-based vision, ranking immediately after deblurring, as reported by the survey. Finally, Section 4.4 and the appendix use ablation studies to demonstrate the importance of the integral normalization operator, the effect of changing the number of line segments in the parameterization n, as well as the practical implication of the smoothing operator.We emphasize event-based applications because event cameras and PPLNs are both designed to mimic biological neural principles. Due to the limited availability of high-quality data, event-based vision is an emerging field where modeling plays a more important role than data and the effects of PPLNs can be best demonstrated. However, we also present an evaluation on conventional frame-based tasks to demonstrate the generalizability. Meanwhile, we do not include Spiking Neural Network (SNN) baselines. While SNNs focus on the deployment onto neuromorphic chips, PPLNs are designed to be an alternative to general GPU-based temporal inference models.Task description. Event-enhanced motion deblurring is a popular research domain. Variants of the task include image-to-image and image-to-video deblurring. Our experiments focus on the highly challenging image-to-video problem, establishing a thorough competition against various state-ofthe-art approaches. Given a blurry image and its associated events during the exposure interval, our & Ba, 2014) to train the network for 50 epochs using the L1 loss. We set the learning rate to 10 -3 and reduce the rate by half after 20 and 40 epochs, respectively. The number of line segments is n = 3.Evaluation metric. We use the Mean Squared Error, the Peak Signal-to-Noise Ratio, and the Structural Similarity Index Measure.Results and discussions. As shown in Table 1 (Left), PPLN has a very strong performance in motion deblurring. PPLN improves DeblurSR (Song et al., 2024) , the state-of-the-art event-based motion deblurring model at the time of paper submission, by 5.6% in MSE, 0.372 dB in PSNR, and 4.9% in SSIM. Importantly, we underscore that the PPLN is a generic network architecture and does not require task-specific modeling, whereas the baseline approaches utilize techniques that cater to the deblurring problem, such as dictionary learning (eSL-Net) (Wang et al., 2020) , per-pixel polynomial approximation (E-CIR) (Song et al., 2022) , and implicit neural representation (DeblurSR) (Song et al., 2024) . The success of the PPLN reveals the strength of mimicking biological neural behaviors.In Table 1 (Left), we also use regular convolution layers to construct the U-Net. We increase the number of convolutional channels in the regular U-Net to approximately match the number of trainable parameters in the PPLN (173M versus 192M). The last two rows suggest that the PPLN improves the U-Net by 54.5% in MSE, 6.45 dB in PSNR, and 43.3% in SSIM. Qualitatively, as shown in Figure 3 , our method generates sharper and more realistic frames than the baseline approaches. The proposed PPLN offers vivid details around salient features. In the first row, our method reconstructs the dark patterns on the white background with sharp edges. In the second row, our method gives the best contrast between the white board and the black letters.Task description. The DAVIS Driving Dataset released in 2020 (DDD20) (Hu et al., 2020) As shown in Figure 2 (c), we build a PPLN upon the existing baseline, consisting of a convolutional head, a pooling layer, and a linear component. The convolutional head reduces the spatial dimension of the data and increases the number of channels. The pooling layer eliminates both spatial dimensions by averaging all the pixels on the reduced spatial grid. Finally, the linear component of the network maps the 64-dimensional feature to the scalar steering angle output. The network contains a total number of 463,425 (463K) parameters. The input frame and events contain 50 ms of historical data.Our model (Figure 2 (d) ) lowers the prediction frequency to every 500 ms. The network takes ten conventional frames and ten times as many events as input and predicts the steering angle at ten uniformly distributed timestamps. We replace the linear component with PPLN nodes and slightly increase the number of layers. To match the number of parameters in the original architecture, we shrink the convolutional head. This results in a network with 455,338 (455K) parameters. We utilize ADAM (Kingma & Ba, 2014) to train the network for 200 epochs using the L2 loss. The learning rate 10 -3 with a weight decay of 10 -4 . The number of line segments in the parameterization is n = 3.Evaluation metric. We use the Root Mean Square Error and the Explained VAriance. We train with five random seeds and report the mean and standard deviation.Results and discussion. As shown in Table 1 (Right), our approach outperforms the baseline model in RMSE, with a 17.0% improvement at night, 35.6% improvement during the day, and 30.8% improvement overall. Similar enhancement is observed in EVA, with 1.9% improvement at night, 23.6% improvement during the day, and 9.6% improvement overall.The input to our model contains ten times as much information as the baseline approach (Hu et al., 2020) . To investigate whether the performance gain is simply a result of enriched input information, we present an additional comparison where the baseline is modified to have the same input and output dimensions as our model (Figure 2 (e)). From the second and the third rows in Table 1 (Right), we observe that PPLN improves the modified baseline by 5.7% in RMSE and 1.0% in EVA.Task description. The dynamic Vision Sensor Human Pose (DHP19) dataset (Calabrese et al., 2019) is collected by inviting human subjects into a cubic space and using event cameras in the four ceiling corners to record various body movements, such as walking, jumping, and walking. DHP19 contains recordings of 17 human subjects performing 33 different body movements. In addition to the events, DHP19 includes the 3D coordinates of 13 body joints. The goal is to predict 3D joint coordinates.Calabrese et al. use two frontal cameras ("Cam 2" and "Cam 3") in their experiments. The overall pipeline has two stages. First, they utilize a deep network to predict the 2D joint coordinates from the events in each view. After that, they project the 2D predictions into 3D using the calibration matrices.The deep network used by Calabrese et al. is a fully convolutional network (Long et al., 2015) that contains 17 layers and 218,592 (219K) trainable parameters. The input events are represented as histograms (Zhu et al., 2019) , and the output 2D joint coordinates are represented as heatmaps. After collecting 25,000 events from all four views, the algorithm assembles event histograms from two frontal cameras and discards the events from the other two cameras. The model updates the 3D joint coordinates if the 2D prediction confidence exceeds a threshold τ = 0.3 in both views.In our experiment, the model makes an inference every 250,000 events. The network takes ten times as many events as input and predicts 2D joint coordinates at ten different timestamps. We modify the prediction network by introducing PPLN layers. The modified network contains 215,648 (216K) trainable parameters. We utilize RMSProp (Kingma & Ba, 2014) to train the network for 20 epochs using the L2 loss. The learning rate is 10 -3 in the first 10 epochs, 10 -4 from epochs 10 to 15, and 10 -5 from epochs 15 to 20. The number of line segments in the parameterization is n = 3.Evaluation metric. We use Mean Per Joint Position Error and report in 2D pixels and 3D millimeters. We refer readers to the appendix for the impact of line segment number n and smoothing on performance.To demonstrate the generalizability of PPLNs, we remove the event input from the steering angle prediction model. Table 2 (Right) shows that PSNNs can still outperform the baselines in the conventional frame-based only setting. Importantly, we observe a significant performance drop by taking out the event input from both baseline approaches. However, the frame-based PPLN has a surprisingly similar prediction quality to the dual-modal PPLN. This result demonstrates that by simulating the biological behaviors, our model can effectively overcome the imperfections in the input data. Note that among the three tasks discussed above, only steering prediction allows inference from conventional frames alone. Human pose estimation takes events as the single-modal input, and image-to-video deblurring requires events to address the motion ambiguity. 
conclusion and future work: 
 This paper presents Parametric Piecewise Linear Networks (PPLNs), a novel temporal learning architecture inspired by biological neural principles. The key idea is to represent the membrane potential as a parametric piecewise linear function with predictable coefficients. Experiments on various event-based vision applications, including steering prediction, human pose estimation, and motion deblurring, demonstrate that PPLNs outperform state-of-the-art models. In the future, we plan to use a recurrent prediction model to support a dynamic number of line segments. Another direction is to adopt more accurate modeling for the membrane potential function, including mechanisms such as the refractory period after each spike. 
acknowledgement: 
 We acknowledge the support from NSF Career IIS-2047677 and NSF HDR TRIPODS1934932.A Appendix / supplemental material A.1 Detailed results for task IIIWe report the 3D MPJPE in millimeters for all 33 different types of body movements evaluated on 5 testing subjects. As shown in Table 4 and Table 5 , simply increasing the input horizon leads to an overall performance degradation. One possible explanation is that the network becomes confused about the temporal relationship among the input events. The modified baseline is not a time series model and can only infer the temporal relation from the input stacking order (the first temporal event slice is in the first channel, the second temporal event slice is in the second channel, etc.). On the other hand, PPLN keeps track of the timestamp of interest in every model layer. This allows PPLN to outperform the baseline in almost all cases by a significant margin. We point out that in Table 4 and Table 5 , the asterisk for testing subject 2 in movement 29 indicates missing data in the DHP19 dataset (Calabrese et al., 2019) .  
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes] Justification: Discussed in Section A.7.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?Answer: [Yes] Justification: We have carefully read the NeurIPS Code of Ethics and confirm that our research conforms with the Code.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: Discussed in Section A.13.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: The paper poses no such risks. Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: Discussed in Section A.6. Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. 
 
 Appendix: Figure 4 presents seven additional examples for the motion deblurring results. We also encourage readers to watch the supplementary animations in the .gif format, which demonstrate the temporal smoothness of our results. Overall, the visual quality of PPLN reconstructions is on par with the state-of-the-art method (DeblurSR (Song et al., 2024) ). Quantitatively, PPLN outperforms DeblurSR by a small margin (see paper body). We point out that while all baseline approaches are specifically designed for motion deblurring, the proposed PPLN is a general framework that can be applied in various event-based vision tasks.Some properties of the proposed model are slightly different from the standard practice in existing research in computer vision and machine learning (i.e., Spiking Neural Networks, SNNs (Maass, 1997) ), even though both of them are inspired by biological neural principles (Lapicque, 1907) .First, existing SNNs focus on the interconnection between artificial neurons, with different layers communicating through binary signals. On the other hand, PPLN focuses on representing the membrane potentials, defined as the voltage differences between the interior and the exterior of the cell. The membrane potentials change with time according to the piecewise linear representation, whose parameters are predicted from the input.Second, the signal transmitted between SNN layers carries a one if the neuron is excited and a zero if the neuron is not excited. In contrast, our work models the membrane potential using a real value. When a spike occurs, the neuron becomes excited, and the real-valued membrane potential function experiences a discontinuous gap. Third, existing SNNs utilize the linear decay mechanism. In other words, the membrane potential function only decreases in the absence of excitement. This contrasts our parametric mechanism where the slope of each line segment can be either positive or negative. We point out that in biological neurons, the resting potential is not the "absolute" zero. When a biological neuron is at rest, there is naturally a voltage difference between the interior and the exterior of the cell. If the membrane potential exceeds the resting potential, the neuron will decrease the potential at an approximately linear rate. Similarly, if the membrane potential drops below the resting potential, the neuron will gradually increase the potential. The linear potential increment corresponds to a positive slope in the proposed mechanism.Fourth, we do not explicitly enforce any line segment to be flat. Instead, we expect the network to implicitly learn when a line segment should have a zero slope from the training dataset. This allows more flexibility in the design.In figure 5 , we plot a few randomly sampled piecewise linear functions predicted by the network. We observe uneven segment lengths and discontinuities (shown in orange) at boundaries. The piece-wise linear function has varying slopes, suggesting the model does not collapse to somewhere far away from the design. We also observe that with n = 3, the network has the capability to represent functions with less than 3 linear segments.Despite our best efforts, we fail to exactly reproduce the baseline performance reported by Hu et al.for steering angle prediction and Calabrese et al. for human pose estimation. According to Hu1 0 1 1 0 1 1 0 1 1 0 1 1 0 1 1 0 1 1 0 1 1 0 1Figure Compared to the relative improvement brought by PPLNs, the differences above are insignificant. We believe the inconsistency in steering angle prediction is the result of ambiguities in the data pruning procedure discussed in Section III-A of the original paper by Hu et al.. We provide step-by-step instructions for the data pruning procedure used when generating our reported figures in the code release. We believe training randomness and differences in optimizer hypermeters cause inconsistency in human pose estimation. Our code release includes the random seed, optimizer hypermeters, and the pre-trained weights for both the baseline model and the PPLN.In steering angle prediction, we use the DDD20 dataset (Hu et al., 2020) released under GNU Lesser General Public License v3.0. In human pose estimation, we use the DHP19 dataset (Calabrese et al., 2019) . The DHP19 dataset is released under the Creative Commons Attribution-ShareAlike 4.0 International License, and the dataset utility scripts are released under MIT License. In motion deblurring, we use the HQF dataset (Stoffregen et al., 2020) . At the time of paper writing, the HQF dataset is available for public download, but the licensing details are unclear. To the best of our knowledge, the datasets do not contain personally identifiable information or offensive content.We train the model for steering angle prediction using one NVIDIA TITAN V GPU. We train the model for human pose estimation using one NVIDIA Tesla V100 SXM2 GPU. We train the model for motion deblurring using one NVIDIA Tesla V100 SXM2 GPU.In this paper, we define the number of line segments in the piecewise linear approximation to the membrane potential function as a hyperparameter n. The choice of this hyperparameter affects the representation capacity of the PPLN node, as well as the run-time complexity and convergence rate.At the moment, we are unable to provide a theoretical guideline that allows developers to choose its value based on the input properties and output requirements. Hyperparameter tuning is needed to balance the prediction quality and computational cost.A.9 Proof of Theorem 3.1 Theorem 3.1 Consider an underlying n segment piecewise linear function parameterized by Θ ⋆ = {m ⋆ , b ⋆ , t ⋆ }. Let (τ j , v j ), j = 1, . . . , m be m point samples, where v j = ṼΘ ⋆ (τ j ) + ψ j , and ψ j is a small random noise.Recall that the ground-truth membrane potential was defined by:EQUATION)Without losing generality, we assume the following constraints:t ⋆ 0 = min 1≤j≤m τ j = 0 t ⋆ n = max 1≤j≤m τ j = 1.The following sampling assumption will be put in order to prevent uncontrollable error due to noise. Uniform Sampling Assumption: We assume there exists some constant c > 0 such that then i samples (τ i1 , v i1 ), • • • , (τ ini , v ini ) on the i th segment satisfy p<q |τ ip -τ iq | p<q (τ ip -τ iq ) 2 ≤ c/ max p,q |τ ip -τ iq |.Then under the objective:EQUATIONand by applying vanilla gradient descent, Θ ⋆ can be recovered up to an error:max 0≤t≤1 Ṽ T Θ∞ (t) -ṼΘ ⋆ (t) ≤ O max j |ψ j | (18)if the learning rate η = O h n , where parameter h > 0 measures the distance from the initialization to the ground truth:|t i (Θ 0 ) -t ⋆ i | ≤ h ≤ |t ⋆ i -t ⋆ i-1 | ∀i = 1, . . .Comment: The uniform sampling assumption is necessary. Imagine a scenario where a disproportionately large number of samples are clustered around (τ, v) = (0, 0), with only a single sample at (τ, v) = (1, 0). In such an extreme case, the linear regression model might yield a significantly steep slope due to noise, leading to substantial errors when predicting for τ = 1. This example highlights the potential pitfalls of non-uniform sampling in affecting the reliability of regression outcomes. In addition, it can be shown that a set of uniformly random samples would provide constant c with high probability.Proof. Let Θ r be the coefficients in the r th iteration of the optimization. We first consider the case where all samples (τ j , v j )'s are classified into the correct intervals (i.e.,t i-1 (Θ r ) ≤ τ j < t i (Θ r ) implies t ⋆ i-1 ≤ τ j ≤ t ⋆ i ).With a sufficiently large temperature T , the effect of smoothing becomes minimal, and the estimated potential is:Ṽ T Θ (t) = (m i t + b i ) 1 + o(exp(-T ))for all t i-1 ≤ t < t i .On the other hand, if t ⋆ i-1 ≤ τ j < t ⋆ i , the noisy sample is:v j = m ⋆ i τ j + b ⋆ i + ψ j .This gives the objective as:L T (Θ) = m j=1 Ṽ T Θ (τ j ) -v j 2 = n i=1 t * i-1 ≤τj <t ⋆ i (m i τ j + b i ) 1 + o(exp(-T )) -m ⋆ i τ j + b ⋆ i + ψ j 2 = n i=1 t * i-1 ≤τj <t ⋆ i (m i -m ⋆ i )τ j + (b i -b ⋆ i ) -ψ j + o(exp(-T )) 2 (19)To analyze Equation ( 19), we present a proposition to simplify notations. (au i + b + c i ) 2 (20)in which u i 's and c i 's are known constants with the constraint that u i 's are not all the same, satisfies that:a ≤ ζ(u) max i |c i |, b = - 1 k k i=1 c i - a k • k i=1 u i where ζ(u) = 2 i<j |u i -u j |/ i<j (u i -u j ) 2Proof of Proposition A.1. Let L(a, b) = i (au i + b + c i ) 2 and enforce ∂L ∂a = ∂L ∂b = 0, we have:a i u 2 i + b i u i + i c i u i = 0 (21)andEQUATION)This gives:a = i u 2 i -k -1 i u i 2 -1 • k -1 i u i i c i - i c i u i = i<j (u i -u j ) 2 -1 - i<j (u i -u j )(c i -c j ) ≤ i<j |u i -u j |(|c i | + |c j |) i<j (u i -u j ) 2 ≤ i<j |u i -u j | i<j (u i -u j ) 2 • 2 max i |c i |Therefore, a ≤ ζ(u) max i |c i | and the remaining part regarding b can be obtained directly from Eq. 22.The following corollary applies Proposition A.1 to the learning of the piecewise linear coefficients. Corollary A.2. The solution Θ ∞ of the optimization problem:EQUATIONsatisfies:EQUATIONProof of Corollary A.2. Comparing Equation ( 19) and Equation (20), by setting u i = τ j and c j = -ψ j + o(exp(-T )) we have:EQUATIONEQUATIONgiven that the sampling points τ j 's are fixed (i.e., ζ(u) is a constant). When T → ∞, we obtainm i = m ⋆ i + O max j (|ψ j |) and b i = b ⋆ i + O max j (|ψ j |) , which implies: ṼΘ∞ (τ ) -ṼΘ ⋆ (τ ) = m i (Θ ∞ ) -m ⋆ i τ + b i (Θ ∞ ) -b ⋆ i ≤ O max j (|ψ j |) since 0 ≤ τ ≤ 1.The rest of the proof shows that under our assumptions, the sample points will eventually be classified into correct intervals.In the following, we use {m i (T )}, {b i (T )}, and {t i (T )} to denote the optimal solution for the objective L T (Θ). Due to noisy samples and smoothing process, the optimal solution is different fromΘ ⋆ . Consider (τ j , v j ) with t * i-1 ≤ τ j < t * i . Since |t i (Θ 0 ) -t ⋆ i | ≤ h ≤ |t ⋆ i -t ⋆ i-1 |, τ j must fall into one of the intervals [t i-2 (Θ 0 ), t i-1 (Θ 0 )], [t i-1 (Θ 0 ), t i (Θ 0 )], and [t i (Θ 0 ), t i+1 (Θ 0 )].In the following, we discuss the stationary point at a specific temperature T . In practice, we can stay at the temperature T until the gradient vanishes and then we move on to a higher temperature.∂w (i) l ∂t i-1 = T • w (i) l 2 • exp T (τ -t i-1 ) ∂w (i) r ∂t i = -T • w (i) r 2 • exp T (t i -τ ) ,Setting ∂L T ∂ti = 0 we have:0 = T • ti≤τj <ti+1 ∆ j w (i+1) l 2 • exp(T (τ j -t i )) • (m i τ j + b i ) -(m i+1 τ j + b i+1 ) -T • ti-1≤τj <ti ∆ j w (i) r 2 • exp(T (t i -τ j )) • (m i+1 τ j + b i+1 ) -(m i τ j + b i )where ∆ j is defined by: ∆ j := Ṽ T Θ (τ j ) -v j Thus, we have:EQUATIONConsider the case where t i is not currently at the correct location. Without loss of generality, we assume t i is too far to the right. This implies that there exists τ j satisfying t i-1 ≤ t ⋆ i ≤ τ j < t i . Suppose:t i-1 ≤ τ j0 < τ j0+1 < • • • < τ j-1 < t ⋆ i and t ⋆ i ≤ τ j < t i ≤ τ j+1 < • • • < τ j1 < t i+1 .Since τ j+1 , . . . , τ j1 all fall inside the correct interval, we can fully recover the corresponding segment and have ∆ q = O(e -T ) for q = j + 1, . . . , j 1 . The left-hand side of Equation ( 27) is, therefore, O(exp(-T )), which enforces the right-hand side of the equation to keep getting smaller. Since (τ j , v j ) is not consistent with τ j0 , . . . τ j-1 and ∆'s are the errors of linear regression, we claim that ∆ r , r = j 0 , . . . , j -1 would never be close to zero. To make Equation (27) hold true, t i has to be decreased. This process will last until t i < τ j (i.e., the sample point (τ j , v j ) is excluded from the wrong interval [t i-1 , t i ]).In this way, we prove that eventually all sample points will be classified into correct intervals, thereby reducing to the base case that has been proved earlier. Recall that when we construct a PPLN node without integral normalization or smoothing, the output ṼΘ (t) is given as:ṼΘ (t) =        m 1 t + b 1 t 0 ≤ t < t 1 m 2 t + b 2 t 1 ≤ t < t 2 . . . m n t + b n t n-1 ≤ t ≤ t n (28) With m = (m 1 , . . . , m n ) T , b = (b 1 , . . . , b n ) T, and t = (t 0 , . . . , t n ) T , we have:EQUATION∂ ṼΘ (t k ) ∂b = (0, • • • , 0, 1, 0, • • • , 0) (30) ∂ ṼΘ (t k ) ∂t = 0 (31) at the input timestamp t 0 ≤ t k ≤ t n .We make two observations here. First, both ∂ ṼΘ(tk) ∂t is an all-zero vector. A gradient-based optimizer is therefore unable to adjust the segment lengths. This means the segment endpoints are hard-coded positions instead of trainable parameters.To illustrate the second observation, consider the toy example of a 2-segment linear signal, as shown in Figure 6 . The green real signal generates a set of orange training samples with measurement noises. Assuming no prior knowledge of the real signal besides that it has two segments, we initialize a 2-segment parametric model with zero slopes, zero intercepts, and equal segment lengths (Figure 6 (Left)). After training, the node converges to the model shown in Figure 6 (Middle). The second segment deviates from the real signal significantly because it attempts to accommodate the last several training samples in the first segment.As discussed in the body, we use the integral normalization operator σ(•) to address the above challenges:EQUATION)where V is a parameter that controls the mean of ṼΘ (t) when 0 ≤ t ≤ 1.This gives:EQUATIONwhere all three gradient vectors contain rich gradient information in every element, encouraging a smooth and swift convergence. Figure 6 (Right) illustrates how the model accurately approximates both linear segments in the toy example after incorporating integral normalization.A.11 Ablation studies: number of line segmentsAs shown in Table 6 , we fail to observe noticeable performance improvement when increasing the number of line segments n in the formulation. Therefore, we decide to use n = 3 in all the experiments discussed in the body of this paper for better efficiency.A.12 Ablation studies: smoothingAs shown in Table 7 , the smoothing operator has an insignificant impact on the prediction quality. However, we point out several facts related to the smoothing operator as a guideline for potential future applications. First, the smoothing operator allows us to have a simple model (Theorem 1) with analytical analyzable properties. Second, the smoothing operator does not introduce any additional trainable parameters. Third, when the size of the network is relatively small (i.e., steering prediction and human pose estimation), our results suggest that using PPLNs without smoothing is slightly better. Finally, when the size of the network is large (i.e., motion deblurring), smoothing introduces a small performance improvement.This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here. Regarding the environmental impact, our experiments are conducted on an internal GPU cluster, and the total emission is estimated to be 81.12 kgCO 2 eq, equivalent to 328 km driven by an average car. This emission estimation is conducted using the Machine Learning Impact calculator presented by Lacoste et al.. To mitigate repetitive labor and negative environmental impact in future research, we have released our open-source implementation together with trained network weights after the anonymous period.1. Claims Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? Answer: [Yes] Justification: The paper's main contribution is the proposal and evaluation of the parametric piecewise linear networks. In Section 4, our experiment emphasizes event-based vision applications because event cameras and PPLNs are both designed to mimic biological neural principles. Due to the limited availability of high-quality data, event-based vision is an emerging field where modeling plays a more important role than data and the effects of PPLNs can be best demonstrated. However, we also present an evaluation on conventional frame-based tasks to demonstrate the generalizability. Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 


Decision: The reviewers recognize the proposed approach as a novel and interesting approach to event-based vision processing, with a convincing demonstration of its performance, including a number of ablation studies. There were some discussions about inference time, hyperparameters and biological plausibility, which the authors could address for the most part during the discussion period. Overall, there is concensus that the paper makes a solid contribution and should be accepted.

Reviews: review: paper_summary: paper_summary: This paper introduces Parametric Piecewise Linear Networks (PPLNs), a novel approach to temporal modeling inspired by biological neural principles. PPLNs represent neuron membrane potentials as piecewise linear functions with learnable coefficients, aiming to allow for explicit temporal modeling. The authors evaluate PPLNs on three event-based vision tasks: motion deblurring, steering prediction, and human pose estimation, claiming significant improvements over baseline models and some state-of-the-art methods. The paper also attempts to show that PPLNs can generalize to conventional frame-based versions of these tasks.
strength_weakness: strength_weakness: 1.	The paper lacks a thorough analysis of the computational costs associated with PPLNs. Without this information, it's difficult to assess the practical viability of the approach, especially in comparison to existing methods.
2.	While the authors claim biological inspiration, the paper does not adequately explore how closely PPLNs actually mimic neuronal behavior. The connection to biological systems seems superficial and not well-substantiated.
3.	The paper's claims about PPLNs' applicability to conventional frame-based tasks are not adequately supported. The limited experiments in this domain do not provide convincing evidence of the method's broader applicability beyond event-based vision.
4.	The paper fails to provide a detailed analysis of how sensitive PPLNs are to hyperparameter choices, such as the number of line segments in the parameterization. This omission raises questions about the robustness and reproducibility of the results.
questions: questions: 1.	What is the exact computational overhead of PPLNs compared to traditional neural network layers?
2.	Can you provide a detailed sensitivity analysis for the key hyperparameters of PPLNs?
3.	Can you provide more evidence to support the biological plausibility of PPLNs beyond the initial inspiration?
limitations: limitations: The authors have addressed some limitations of their work, particularly in the supplementary material. However, a more comprehensive discussion of potential negative societal impacts and failure cases would be beneficial. The paper could also benefit from a more detailed analysis of the computational requirements and scalability of PPLNs.
paper_summary: paper_summary: 1. The paper presents Parametric Piecewise Linear Networks (PPLNs), a novel neural network architecture inspired by neuromorphic principles for temporal vision inference.

2. The innovative approach of PPLNs lies in modeling the membrane potential of artificial neurons as parametric piecewise linear functions with learnable coefficients, echoing the design of Kolmogorov-Arnold Networks (KANs) but with input-dependent adaptability.

3. Experimental results demonstrate PPLNs' state-of-the-art performance in event-based vision applications such as steering prediction, human pose estimation, and motion deblurring, outperforming existing methods with improved accuracy and efficiency.
strength_weakness: strength_weakness: 1. The authors did not compare with some of the latest event-based deblurring algorithms, such as REFID[1]. 

2. In addition, the authors should provide a direct comparative experiment using a KAN-based backbone to further demonstrate the superiority of the proposed solution.

[1] Sun, Lei, et al. "Event-based frame interpolation with ad-hoc deblurring." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.
questions: questions: 1. The piecewise linear nature of this paper is akin to an input-dependent activation function. An SNN using multi-layer LIF neurons with learnable parameters also seems to achieve a similar level of biological plausibility. Could the authors provide further explanation from this perspective?

2. Based on Figure 2d, I am still unclear whether the input t for PPLN is an external query (you can choose your desire timestamp t) or an inherent timestamp of the event. If it is the former, does this mean that all neurons query with the same input t? If it is the latter, how is this implemented? Given that there are many input events with numerous timestamps, how does the method handle this?

A good response to the Weaknesses and Questions will improve my initial rating.
limitations: limitations: The authors discussed the environmental impact of the training phase for their proposed solution.
paper_summary: paper_summary: This paper proposes Parametric Piecewise Linear Networks (PPLNs) for event-based temporal modeling, which emulate biological principles by representing membrane potentials as parametric mappings. The authors demonstrate how a straightforward modification enables standard multi-layer perceptrons and convolution operators to accommodate this parameterization of membrane potential. Experimental results showcase that the proposed PPLNs attain state-of-the-art performance in typical event-based vision tasks, including steering prediction, human pose estimation, and motion deblurring.
strength_weakness: strength_weakness: i) Could you give the time complexity of PPLNs, or the inference time on the three downstream tasks, such as steering prediction, human pose estimation, and motion deblurring?

ii) Which hyperparameters have the greatest influence on LLPNs? Please explain the analysis from an experimental perspective.
questions: questions: Please see the weakness and response each comment.
limitations: limitations: There is a lack of analysis of the computational complexity of PPLNs, including experimental tests on the inference time of three downstream event-based vision tasks.

Comments: title: Reviewer-Author Discussion Period
comment: Thank you very much for your reply and increasing the rating!
I appreciate the response, which has addressed my initial concerns. I will accordingly adjust my rating.
Thanks for rasing the score and the follow-up discussion! We agree that using different $t$'s in different layers is a biologically plausible design. In fact, $t$ does not even have to be part of the input. The previous layer can predict a timestamp for the next layer. We will make sure to discuss this in our revision.
Thank you for the enthusiastic reply. I understand that PPLN is a network with each neuron with a dynamic equation, which can be queried through t, and all neurons receive the same input t. This may sound a bit weird because the execution of neurons in different layers of a neural network is sequential rather than parallel. But I think it is still biologically plausible, as the dynamic equations of each layer are different, which in turn means that the 't' for each layer is different. Does the author have any further insights? 

Overall, after reading the opinions of other reviewers, I still think this paper presents some interesting insights. I am willing to raise my rating to increase the likelihood of this paper receiving more attention within the community.
Thanks for letting us know, and we are glad to hear the rebuttal has addressed your concerns! We will make sure to include the appendix as part of the camera-ready version. To enhance the chance of acceptance, may we ask if it is possible for you to increase the rating to weak accept (or above)?
The author has addressed my concerns, and I will maintain the original score. I hope the author will incorporate the  different  experimental tasks into the appendix of the camera-ready version.
Dear Reviewers,

Thank you for your constructive feedback and valuable suggestions! As a gentle reminder, there are just under three days remaining before the reviewer-author discussion period ends. We would like to seize this opportunity to address any questions or concerns you may still have regarding our submission. Furthermore, we encourage you to consider raising the rating if you deem it appropriate.

Many thanks in advance!

Sincerely,

Authors