Restoring Real-World Degraded Events Improves Deblurring Quality

Authors: yeqing  shen
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-02 05:03:39

Abstract: Due to its high speed and low latency, DVS is frequently employed in motion deblurring. Ideally, high-quality events would adeptly capture intricate motion information. However, real-world events are generally degraded, thereby introducing significant artifacts into the deblurred results. In response to this challenge, we model the degradation of events and propose RDNet to improve the quality of image deblurring. Specifically, we first analyze the mechanisms underlying degradation and simulate paired events based on that. These paired events are then fed into the first stage of the RDNet for training the restoration model. The events restored in this stage serve as a guide for the second-stage deblurring process. To better assess the deblurring performance of different methods on real-world degraded events, we present a new real-world dataset named DavisMCR. This dataset incorporates events with diverse degradation levels, collected by manipulating environmental brightness and target object contrast. Our experiments are conducted on synthetic datasets (GOPRO), real-world datasets (REBlur), and the proposed dataset (DavisMCR). The results demonstrate that RDNet outperforms classical event denoising methods in event restoration. Furthermore, RDNet exhibits better performance in deblurring tasks compared to state-of-the-art methods. DavisMCR are available at https://github.com/Yeeesir/DVS_RDNet.

Main paper: 
 introduction: 
 Dynamic Vision Sensor (DVS) rapidly captures changes in brightness in the environment, and its signals are referred to as events. Due to its high speed and low latency, it is often utilized for motion deblurring. Ideally, high-quality events can accurately record the changes in brightness in motion-blurred regions, guiding eventbased methods for better deblurring of motion regions. However, real-world events are degraded, introducing undesirable artifacts into the deblurred results and impacting the final image quality. For example, different pixels in DVS may experience varying degrees of threshold bias [28] , resulting in non-smooth black and white points in the deblurred results (Figure 1 (c1) and (d1)). Besides, DVS is constrained by hardware processing speed, leading to event loss and residual motion blur in the deblurred results (Figure 1 (c2) and (d2)). In addition, noise caused by various factors is present in real DVS circuits, and severe circuit noise would introduce undesirable artifacts into the deblurred results (Figure 1 (c3) and (d3)).The majority of current deblurring methods are exclusively trained on single-degradation events, posing challenges in handling real-world degraded events. They mostly focus on how to handle events for guiding image deblurring [15, 29, 31, 32, 46] . For example, BHA [22] proposes event double integral (EDI), which reconstructs the latent sharp image from the principle of DVS. [46] , [15] use CNN to simulate the process of EDI. [29] convert the event into attention map to guide image deblurring. [32] propose an event-enhanced sparse learning network. These studies concentrate on the design of model structures and multi-modal fusion, yet the effectiveness of deblurring is constrained by the degradation inherent in real-world events.To address this problem, we employ degradation modeling to guide the learning process of event restoration, which is, in turn, employed to improve deblurring quality with the help of RDNet. Specifically, we first model the primary degradation patterns influencing deblurring quality in DVS circuits, categorizing them into threshold bias, limited bandwidth, and circuit noise. Based on this degradation modeling, we construct pairs of undegraded events and degraded events to guide the learning of restoration. Subsequently, we use the proposed RDNet to improve the quality of deblurring. In the first stage, we train an event restoration model with paired event data, employing the rich texture information in blurry images to obtain reliable restored events. In the second stage, we train the deblurring model with paired images. We incorporate the brightness variation with high-temporal resolution from the restored events into the deblurring model, contributing to highquality deblurred results. In this way, our approach mitigates the potential artifacts introduced by degraded events.To better assess the deblurring performance of various methods on real-world degraded events, we introduce a real dataset named DavisMCR. Current datasets include only a limited number of event degradation patterns, posing challenges for a comprehensive evaluation of different methods across various degradation scenarios. By contrast, DavisMCR contains events with varying degrees of degradation collected by manipulating the level of environmental illumination and target objects contrast. Besides, DavisMCR comprises scenes with both simple and complex textures, providing a diverse set of scenarios for evaluating deblurring effects. We employ DavisMCR as a crucial benchmark to assess the performance of various methods in handling diverse degraded real-world events. Moreover, we reconfigure and simulate the event data within the GOPRO dataset. The test set employs simulation settings entirely distinct from the training set, generating a variety of randomly degraded events. We achieve more objective evaluation results on GOPRO by introducing a domain gap in the events data between the training and test sets.The main contributions of this paper are as follows:1. We characterize event degradation and create paired data to guide the learning process for event restoration. Subsequently, we propose RDNet, which improves the quality of deblurring by restoring real-world degraded events. 2. We introduce the DavisMCR dataset, which contains events with varying degrees of degradation by manipulating environmental illumination levels and contrasts of target objects. This dataset serves as a comprehensive evaluation platform for assessing the deblurring performance of various methods. 3. RDNet outperform the classic event denoising methods in the event restoration task. Moreover, RDNet exhibits better performance compared to current state-of-the-art methods on synthetic datasets like GOPRO, as well as real-world datasets such as REBlur and DavisMCR. 
related work 2.1 event simulator: 
 The existing DVS datasets are extremely scarce. To enhance the training of deep learning models, event simulators transform large image datasets into annotated event data. ESIM [24] is a classic event simulator widely employed in event-based research [12, 15, 18, 28, 29, 31, 36, 39, 46] for data preparation. ESIM proposes a paradigm of rendering continuous frames using a single image and camera trajectory. It simulates the events and employs an adaptive sampling strategy to mimic the signal characteristics of DVS under varying brightness conditions. However, ESIM only accounts for simple noise simulation caused by thresholding and does not simulate the complex degradation in actual DVS circuits. As a result, there exists a certain gap between the simulated events and real-world degraded events. Vid2E [7] explores variations in time sampling rates and circuit parameter augmentations based on ESIM. V2e [10] further simulates various characteristics of DVS circuits, including pixellevel Gaussian event threshold mismatch, finite intensity-dependent bandwidth, and intensity-dependent noise. With novel insights into sensor design and physics, DVS-Voltmeter [14] considers voltage variations in DVS circuits to account for the randomness caused by photon reception, as well as noise effects from temperature and parasitic photocurrent. It models the event generation process as a more comprehensive stochastic process. The aforementioned methods provide effective tools for the current research, in which we model event degradation and construct paired events on the foundation of the event simulator. These paired events serve as a guide for the learning of the event restoration.Image-only Deblurring. The mainstream image-only deblurring methods learn blurry features through deep learning models and then output clear images [2, 3, 5, 6, 9, 13, 20, 23, 25, 26, 30, [40] [41] [42] [43] 48] . For example, [5, 30, 42] employ the coarse-to-fine approaches, gradually restoring clear images at different resolutions in the pyramid. [2, 3] design backbone models from the perspective of network architecture. DeblurGAN [13] , based on conditional GAN, achieves optimization in both structural similarity metrics and visual appearance. SPAIR [23] utilizes the non-uniformity of severe degradation in the spatial domain and proposes a learning-based universal solution for recovering images subjected to spatially varying degradation. MPRNet [41] , through a multi-stage architecture, progressively learns the restoration function of the degraded input, achieving a balance between spatial details and high-level contextual information. Restormer [40] incorporates critical design in building blocks (multi-head attention and feedforward networks) to capture long-range pixel interactions, making it suitable for large images by applying a transformer. MSGD [25] introduces an imageconditioned Dynamic Programming Method, employing multiscale structure guidance as an implicit bias to guide image deblurring. MRLPFNet [6] proposes a learnable low-pass filter utilizing a selfattention mechanism to model low-frequency information, alongside an additional fully convolutional neural network employing standard residual learning to model high-frequency information. However, these image-only deblurring methods merely learn blur patterns from the images that lack motion information, and this results in insufficient generalizability of these methods. In scenarios with severe motion blur, image-only deblurring methods typically exhibit weaker performance compared to event-based deblurring.Event-based Deblurring. The events containing changes in brightness with high-temporal resolution provide guidance for deblurring images in dynamic scenes. Event data is a distinct modality from images and it features spatiotemporal sparsity. Current event-based deblurring methods focus on integrating these two modalities to achieve image deblurring [1, 4, 8, 11, 12, 19, 21, 22, 27, 29, 31, 32, 34, 35, 37, 38, [44] [45] [46] [47] . BHA [22] starts from the working principles of DVS, employing double integration on events to reconstruct the brightness of each pixel in the image. It elucidates the principles of event-based deblurring and demonstrates feasibility as shown in the experimental results. However, this approach introduces noise from events into the deblurred results, leading to severe artifacts. Following this line of reasoning, further studies attempt to improve the deblurring performance by optimizing the models. EVDI [46] simulates the double integral calculation through a convolutional neural network and uses the network to simulate the process of BHA. LEMD [11] introduces a sequential formulation for event-based motion deblurring and elucidates its optimization with a deep network. D2Nets [27] utilizes a bidirectional LSTM detector to identify the nearest sharp frame , upon which deblurring is subsequently performed. EBKE [19] proposes a novel approach for estimating blur kernels based on events, enabling the recovery of complex blur motions through the utilization of blur kernels. Because this method is not relying on machine learning methodologies, it operates independently of training data. HDN [44] adopts a recurrent encoderdecoder architecture to generate dense cyclic event representations, encoding the entirety of historical information, followed by deblurring processing. EvIntSR-Net [8] aligns the domains between event streams and intensity frames, learning to fuse latent frame sequences in a recurrently updated manner. DS-Deblur [38] proposes a dual-stream based event-image fusion framework for motion deblurring, adaptively aggregates the frame and event progressively at multiple levels. EIFNet [37] proposes an event-image fusion network that is grounded on modality-aware decomposition and recomposition techniques, facilitating enhanced integration of features from both event and image modalities. ERDNet [1] learns event-based motion deblurring with a residual learning approach. NEID [4] proposes a non-coaxial event-guided deblurring method that spatially aligns events to images while refining the image features from temporally dense event features. RED-Net [35] employs simultaneous estimation of optical flow and latent images, leveraging blur consistency and photometric consistency to enable self-supervision of the deblurring network using real-world data. SAN [47] proposes a scale-aware network capable of adapting to various spatial and temporal resolutions of motion blur. ESL-Net [32] uses the framework of sparse learning to restore low-quality images. [31] added modulated deformable convolutions to the model to make better use of dense motion information. UEVD [12] introduces a novel exposure time-based event selection method. It selectively utilizes event features by estimating cross-modal correlations between blurry frame characteristics and events. EFNet [29] converts events into attention maps to guide image deblurring. These studies provide valuable insights into model design for event-based deblurring. However, they lack the ability to handle the degradation in real-world events, resulting in a lack of robustness in practical applications. Therefore, we propose RDNet, which improves deblurring quality through event restoration. 
methodology 3.1 problem definition: 
 The principle of DVS can be expressed by Eq.1, where 𝐼 𝑡 represents the image at 𝑡 time, 𝑝 ∈ {-1, +1} indicates the polarity of the event signal, and 𝑐 indicates the threshold of DVS.EQUATIONWe define the event signal as 𝑒 𝑥 𝑦 (𝑡) = 𝑝 • 𝛿 (𝑡 -𝑡 0 ), which means that an event signal with polarity 𝑝 is generated at (𝑥, 𝑦) at the time of 𝑡 0 , and 𝛿 (•) is an impulse function. Blurry image 𝐵(𝑡 𝑓 ) is the integral of image 𝐼 (𝑡). The final deblurred result can be calculated by Eq.( 2). The detailed derivation of Eq.2 can be found in the supplementary material.EQUATIONIn order to facilitate learning event-based deblurring with a CNNbased network, we transform the original event data and pass the original event quadruple (𝑡, 𝑥, 𝑦, 𝑝) through Eq.3 into a tensor of 𝐻 × 𝑊 × 𝑁 𝑒 . In Eq.3, Blurry image and sharp image are defined as 3D tensors 𝐼 𝑏 and 𝐼 𝑠 , with dimensions 𝐻 × 𝑊 × 3.EQUATIONBased on the impact of degradation in real-world events on image deblurring, we categorize degradation patterns into threshold bias, limited bandwidth, and circuit noise. By employing degradation modeling, paired data consisting of degraded and undegraded events can be constructed to guide the learning process of event restoration. Threshold Bias. When there is a bias in the threshold, the number of triggered events in DVS deviates accordingly, and this leads to significant fluctuation in the event waveforms in motion regions (Figure 1 (c1) ). However, the deblurring model still performs deblurring based on the default standard thresholds during training, leading to outliers in the results, as depicted by the non-smooth black and white points in Figure 1 (d1). From Eq.1, it can be derived that the threshold determines the change ratio in brightness that triggers event signals for each pixel. As shown in Figure 2 (a), the black curve represents the log-domain brightness signal, and the blue and green dashed lines represent threshold1 and threshold2, respectively. These two thresholds with tiny difference triggers different numbers of event signals for the same black brightness curve (Green: 3 positive events, 2 negative events; Blue: 4 positive events, 2 negative events). This illustrates that small threshold bias results in significant discrepancies in events. Limited Bandwidth. While DVS is renowned for its high-speed recording of environmental changes, real devices are constrained by hardware processing speed, resulting in limited bandwidth. As depicted in Figure 1 (c2), when DVS bandwidth is limited, fewer events are triggered compared to undegraded conditions. The deblurring model is unable to accurately restore the changes in brightness in motion regions, resulting in residual motion blur in the Figure 1 (d2). When the ambient changes in brightness rapidly, the finite bandwidth leads to the loss of events. As illustrated in Figure 2 (b) , the black curve represents the log-domain brightness signal, and the blue dashed line represents the threshold. 𝑇 𝑠 represents the sampling period, which reflects the bandwidth. One event is generated within one sampling period at most. Consequently, only the blue arrows are sampled, while the gray arrows are not. Despite the fact that the actual brightness variation triggers 7 positive events and 6 negative events, only 2 positive events and 3 negative events are captured due to bandwidth limitations. Circuit Noise. The deblurring model demonstrates some denoising capability for spatially sparse background noise. However, in the presence of severe noise, distinguishing between noise signals and valid signals becomes a challenging task. As illustrated in Figure 1 (d3), severe circuit noise introduces undesirable artifacts into the deblurred results. In the photodetection process, the quantum properties of photons result in granular noise [10] , which is particularly severe under low-light conditions and is commonly modeled as a Poisson process. Leakage noise events caused by junction leakage and parasitic photocurrent during the reset switch transition of the DVS pixel lead to the noise. DVS sensors always exhibit some hot pixels that continuously trigger events at a high rate even in the absence of input. These hot pixels may arise from reset switches with exceptionally low thresholds or with very high dark currents [10] . Therefore, we introduce leakage noise and hot pixel noise into the noise modeling process. While there are other noise-contributing factors in real DVS circuits with effects similar to those on deblurring results, we do not offer further analysis here. Randomly generated event noise is shown in Figure 2 (c ). It can be observed that in motionless scenes, a substantial number of events are generated.The framework of RDNet is illustrated in Figure 2 (2, 3). We propose a two-stage pipeline to improve the quality of deblurring in the second stage through the restoration of degraded events in the first stage.In the first stage, we employ a model structure consisting of a dual-branch encoder and a single-branch event decoder. Initially, the degraded events 𝐸 𝑑 are fed into the event encoder, while the blurry image 𝐼 𝑏 is fed into the image encoder. Subsequently, the features generated by the image encoder are added to the features generated by the event encoder at the same scale, incorporating rich texture and color information from the image into the event branch to guide accurate restoration of events. Finally, the latent features generated by the encoder are fed into the event decoder to obtain the restored events 𝐸 𝑟 . Model training is supervised with undegraded events 𝐸 𝑢 , and the first term of the event restoration loss 𝐿 𝑒𝑟 is employed.In the second stage, we also employ a model structure with a dual-branch encoder and a single-branch image decoder. Initially, the restored events 𝐸 𝑟 are fed into the event encoder, and the blurry image 𝐼 𝑏 is fed into the image encoder. Subsequently, the features generated by the event encoder are added to the features generated by the image encoder at the same scale, integrating brightness variation with high-temporal resolution from the event branch into the image branch to guide deblurring. Finally, the latent features generated by the dual-branch encoder are fed into the image decoder to obtain the deblurred image. Model training is supervised with the sharp image, and the loss function employed is 𝐿 𝑑 . Additionally, the second term of the loss in the second stage is derived from 𝐿 𝑒𝑟 . Details on the structure of RDNet can be found in the supplementary. Loss Function. The loss 𝐿 𝑒𝑟 employed to supervise event restoration is computed according to Eq.4, where 𝛼 and 𝛽 are hyperparameters. The first term is the L1 norm between the restored events 𝐸 𝑟 and undegraded events 𝐸 𝑢 , directly utilized to supervise the training in the first stage. The second term is the L1 norm of the latent features of 𝐸 𝑟 and 𝐸 𝑢 , where 𝐹 (•) represents the event encoder used in the second stage. This term is employed during the second stage of training to finetune the restoration model. As events are sparsely distributed in space and time, 𝐿 𝑒𝑟 is computed only at the event response locations, which are the positions where the value of undegraded events 𝐸 𝑢 and degraded events 𝐸 𝑑 are non-zero. EQUATIONThe loss for supervising event-based deblurring, denoted as 𝐿 𝑑 , is calculated with the L1 norm between the deblurring image 𝐼 𝑏 and the sharp image, as described in Eq.5. This loss is applied during the training of the second stage.EQUATION 
davismcr dataset: 
 We propose a real-world dataset DavisMCR, comprising event data captured in varying environmental brightness and target object contrast conditions. Data collection is performed using the DAVIS 346c camera, and the ColorSpace CS-HDR-MFS lightbox is employed to set the ambient brightness. Figure 3 (a) represents the control group, a text scene with normal contrast captured at a brightness of lux=800. The event texture is clear, and the noise is low. To generate scenes with different contrast ratios in a real environment, we print versions of the same scene with varying transparencies. In scenes with lower contrast, the triggered event signals are more sparse, while scenes with higher contrast exhibit denser event signals. As shown in Figure3 (b), the events in the low-contrast text motion scene are weak, and the edges are not clear. To generate scenes with different brightness levels in a real environment, we capture targets with brightness ranging from lux=100 to lux=10000. Across different brightness levels, we keep the aperture size constant and adjust the exposure time of the APS image to avoid overexposure. Due to the asynchronous readout of DVS and APS circuits in DAVIS, the exposure time of APS does not affect the DVS signal. This implies that, under a constant aperture, a brighter environment allows more light to enter per unit time. In other words, in brighter scenes, the signalto-noise ratio of the induced current in the DVS circuit is higher. As seen in Figure 3 (c) , the text motion scene captured in a highlux environment features clear edges with low noise. To obtain events with different noise levels, we select scenes with different background brightness. Events of regions with bright backgrounds have less noise, and events of regions with dark backgrounds have more noise. As shown in Figure 3 (d) , events captured in the darkbackground text motion scene exhibit severe background noise. To comprehensively evaluate the deblurring results, we select multiple scenes as targets for capturing. The deblurred details on simple textures are easy to observe and discern as shown in Figure 3 (a) . By contrast, complex textures closely resemble real-world scenarios as depicted in Figure 3 (e) . More details about the DavisMCR dataset are shown in the supplementary material. 
experiment 5.1 dataset: 
 Our training set is the synthetic dataset GOPRO [17] , using v2e [10] for event simulation. Our test set includes the synthetic dataset GOPRO [17] , the real dataset REBlur [29] , and DavisMCR. The test set of GOPRO employs simulation settings entirely distinct from the training set, encompassing a variety of randomly degraded events. We, therefore, achieve more objective evaluation results on GOPRO by introducing a domain gap in the events data between the training and test sets. More details about the dataset can be seen in the supplementary materials.We crop the images of GOPRO data into 256×256 size, and the corresponding events are also cropped into the same size after being divided into channels. We set the parameters 𝑁 𝑒 as 10, indicating that events within the exposure time are divided into 10 channels. The rationale is that an inadequately small 𝑁 𝑒 can lead to the loss of temporal information, while an excessively large 𝑁 𝑒 would result in sparse events, increasing the difficulty of network training. Both 𝛼 and 𝛽 in 𝐿 𝑒𝑟 are set to 0.5. We use Adam with an initial learning rate of 2 × 10 -4 and a cosine learning rate strategy with a minimum learning rate of 10 -7 . The model is trained for 300k iterations with a batch size of 8. We use the same experimental setup for training in both stages. To facilitate subsequent comparisons, we introduce DeblurNet as a baseline model, which shares the same structure as the second-stage RDNet. To ensure a fair comparison, DeblurNet is designed with an equivalent number of parameters to the two-stage RDNet.As events are sparse and discontinuous both in space and time, and there is a lack of accurate ground-truth for events, our comparisons are conducted qualitatively on real datasets REBlur [29] and DavisMCR. We compare the event restoration results of the first stage of RDNet with two classic event denoising methods, SCF [16] and GEF [33] . As shown in Figure 4 , (a1) and (c1) are the input images, while (b1) and (d1) are the input original events. (b2-b4) and (d2-d4) show the event results restored by different methods, and (a2-a4) and (c2-c4) show the deblurred results using the corresponding restored events. To facilitate a more effective comparison of event restoration results, the restored events of SCF and GEF are fed into a DeblurNet to obtain deblurred results. This DeblurNet is trained with undegraded events.As shown in Figure 4 , comparing the input events (b1, d1) with the output of SCF (b2, d2), it can be observed that SCF can only denoise relatively discrete event noise in space, exhibiting weak event restoration capability. This leads to some residual motion blur in the deblurred results (a2, c2). Comparing the input events (b1, d1) with the output of GEF (b3, d3), it can be seen that the reconstruction of GEF relies on the image texture. When motion blur is severe, the restored events lose texture, leading to lots of residual motion blur in results (a3, c3). Additionally, GEF may introduce artifacts by erroneously restoring events in non-motion regions with strong textures. Comparing the input events (b1, d1) with the output of RDNet (b4, d4), it is evident that RDNet can effectively restore events at the motion regions and recover events in a smooth, highquality fashion. RDNet, based on the high-quality events restored in the first stage, achieves deblurred results with fewer residual motion blur and artifacts.Experimental results on the DavisMCR dataset can be found in the supplementary material.We quantitatively and qualitatively evaluate our RDNet and the other deblurring methods on GOPRO, REBlur, and DavisMCR. We compare our method with state-of-the-art image-only and eventbased deblurring methods. For image-only methods [2, 3, 5, 13, 23, 30, [40] [41] [42] , we use the official release model for results testing. For MSGD [25] and MRLPFNet [6] , since no reproducible models or parameters are public, only the numerical results on the GOPRO dataset are compared. For event-based methods [29, 32] , we retrain according to the official settings in order to offer a fair comparison. 5.4.1 Evaluation on GOPRO Dataset. We report deblurred results on GOPRO dataset in Table 1 . Compared to the best existing imageonly methods MRLPFNet [6] and event-based method EFNet [29] , our method achieves 3.32 dB and 1.72 dB improvement in PSNR and 0.012 and 0.007 improvement in SSIM, respectively. These image-only methods lack high temporal resolution information from events, making it challenging for them to accurately estimate the blur process and perform image deblurring. Existing eventbased methods, due to their lack of capability to recover degraded events, introduce artifacts in the deblurring results, leading to lower numerical outcomes. The two-stage RDNet effectively restores degraded events and combines high temporal resolution information from events to achieve accurate deblurring.We show qualitative results of deblurring on GOPRO in Figure 5 . As shown in Figure 5 (b1, b2), image-only methods MPRNet [41] , HINet [3] cannot accurately deblur the blurry image. Instead, they generate a lot of ghosts in deblurred results. Restormer [40] and NAFNet [2] achieve better visual effects as shown in Figure 5 (b3, Table 1 : Deblurring results on GOPRO. Note: As the results of image-only methods are unaffected by events, all results of these methods are directly adopted from previous studies. To ensure fairness in comparison, the results of all event-based methods are based on retrained models.Params(M) PSNR SSIMDeblurGAN [13] 4.30 28.70 0.858 SRN [30] 10.25 30.26 0.934 DMPHN [42] 7.23 31.20 0.940 SPAIR [23] -32.06 0.953 MIMO-UNet [5] 6.81 31.73 0.951 MPRNet [41] 20.13 32.66 0.959 HINet [3] 88.67 32.77 0.959 Restormer [40] 26.13 32.92 0.961 MSGD [25] 30.0 33.20 0.963 NAFNet [2] 67 ). Both methods restore clear edges and generate no additional artifacts. As shown in Figure 5 (e), the events in the flat area contain a lot of noise, and the events on the edge are not completely continuous due to the threshold bias. Event-based methods eSL-Net [32] , SAN [47] and EFNet [29] are also affected by degraded events as shown in Figure 5 (c1, c2, c3). There are some abnormal color pixels in the deblurred images, which make the edge blurry and the flat areas noisy. By contrast, as shown in Figure 5 (f), the event noise in the flat area is effectively suppressed, and the edges are clear and continuous. As a result, the output of RDNet in Figure 5 (c4) has sharp edges and few artifacts.Dataset. We report deblurred results on REBlur dataset in Table 2 . Compared to the best existing imageonly method Restormer [40] and event-based method EFNet [29] , our method achieves 1.92 dB and 0.83 dB improvement in PSNR and 0.021 and 0.006 improvement in SSIM, respectively. Notably, NAFNet [2] exhibits a significant degradation in performance, reflecting its poor generalization to data in different domains. We show qualitative results on REBlur in Figure 6 . As shown in Figure 6 (b1, b2, b3), HINet, MPRNet, and NAFNet show no significant deblurring effects when directly tested on the REBlur dataset. As depicted in Figure 6 (b4), Restormer is capable of partially restoring the shape of texture. However, considerable motion blur residues persist, and some details of the texture remain unclear. In Figure 6 (c1, c2, c2), event-based methods, ESL-Net, SAN and EFNet, exhibit evident deblurring effects in motion regions. These methods stably recover text, with only partial motion blur residues in dense texture areas. As shown in Figure 6 (c4), RDNet achieves a 7 (d ), text with different contrast produces events with different degradation conditions. The events on the top row are so sparse that the texture cannot be seen clearly. Besides, there are many discontinuous events at the edge of the motion region due to the threshold bias. As shown in the Figure 7 (c1-4), these sparse events lead to weak deblurring effects in the existing methods, and discontinuous events in the motion region cause unclear restored edges. By contrast, the result of RDNet has many details and clear edges as shown in the Figure 7 (c5). It is worth noting that the events in the green box above Figure 7 (e) are discarded, but those in Figure 7 (c5) still exhibit evident deblurring effects. This indicates that in areas with particularly sparse events, RDNet can achieve effective deblurring by relying solely on the information present in the image.In order to verify the effectiveness of the degraded event data and the proposed RDNet, we conduct comparative experiments on the GOPRO dataset. In the experiment, we adopt different data and different deblurring models. As shown in Table 3 -setting1, we use undegraded events to train DeblurNet and test with degraded events. Compared with setting-1, setting-2 replaces the training events with degraded events and obtains a PSNR improvement of 1.07dB, which illustrates the effectiveness of event degradation modeling. Setting-3 is the experimental setting of RDNet. Compared with setting-2, there is an improvement of 0.51 dB in PSNR. The RDNet is trained with both degraded and undegraded events in the first stage. During this supervised training, undegraded events provide additional information for RDNet, guiding the model to restore degraded events, thereby improving the quality of deblurring. Setting-4 is a set of ideal experiments. Both the training and test set are undegraded events. It finally reaches 37.96 dB in PSNR, which demonstrates the upper limit that event-based deblurring methods can achieve when events are not degraded.The above experiments demonstrate the effectiveness of the degraded event data and the RDNet. We also test the above model on the real dataset REBlur and DavisMCR, and the qualitative results are in the supplementary material. 
conclusion: 
 To reduce the artifact caused by real-world event degradation on deblurring results, we first characterize event degradation and create paired data to guide the learning process for event restoration. Subsequently, we use RDNet to improve the quality of deblurring by restoring degraded events. In addition, we propose a real dataset named DavisMCR, comprising event data captured under varying environmental brightness and target object contrast conditions. This dataset leverages real-world degraded events, which serves as a platform for assessing the deblurring performance of various methods. The results demonstrate that RDNet attains highquality deblurring performance on both synthetic and real datasets through event restoration. In the future, we plan to extend RD-Net to tasks such as event-based video frame interpolation and super-resolution. 
 
 Appendix:  


Decision: This paper received mixed scores in the first round of review: 1 Weak Reject, 2 Borderline Accept, 1 Weak Accept. After rebuttal, the reviewer with Weak Reject slightly raised the score to Borderline Reject, and the other two reviewers remain unchanged. Positive reviews praised the problem setup and the dataset prepared. Yet, negative reviews have concerns on the gap between real data and synthetic data, and experiment comparision. Given the tight condition, AC carefully examined all materials at hand, and believes that this paper has merits. Yet, considering the remaining issues, AC discussed this paper with SAC in depth, finally decided to accept it. The authors are required to enhance the paper as they said in the rebuttal.

Reviews: review: paper_summary: paper_summary: Two-stage pipeline:

The first stage is the restoration stage. It has a dual-branch encoder and a single-branch event decoder. The degraded events are fed into the event encoder while the blurry image is fed into the image encoder. Then, the features of the image encoder guide the event restoration process. 

The output is the restored events. 

The second Stage is the deblurring stage. It also has a dual-branch encoder and a single-branch image decoder. The restored events are used to guide the image deblurring task.

The output is the deblurred image.
strength_weakness: strength_weakness: (i) A great study is presented in the related work section where whatever work has been done till now is reviewed related to the event simulator and motion deblurring.

(ii) In Spite of the fact that the paper is using an event simulator that is generating the paired sets of {undegraded event, degraded event}, it has presented great detail on the three degradations that are introduced due to the DVS.

**These degradations are simulated by the event simulator.

(iii) The Two-Stage pipeline section is greatly explained and also a clear explanation of loss functions is there.

(iv) Results are well presented. Sufficient qualitative and quantitative analysis is provided.
limitations: limitations: **The title of the paper involves the term “Real-World Degraded Events”, but the entire model training is happening on simulated data. 
The CVPR 2021 paper “Motion Deblurring with Real Events” addressed the issue and proposed a semi-supervised model trained on real data.
(**Reference not provided in the paper)
Link: https://arxiv.org/pdf/2109.13695 

The paper we are reviewing evaluated all the other models by training them on the simulated data. However since the main motive of this paper is addressing the real-world degradations, it should present a study showing how the synthetically simulated degraded events are different from the real-world degraded events and the effect of this on the trained model. Since no proper explanation is presented, the practical application is questionable.

**The paper presented an understanding of the different degradations introduced by DVS but no analysis of how these are added to generate the paired set.

**Further, no mathematical analysis is presented in the entire paper.

**The explanation I found the paper is lacking is about the generation of complex degradations for the natural scene in the DAVISMCR dataset.
paper_summary: paper_summary: The paper addresses image deblurring from an event-based method，by modelling the degradation of events and propose RDNet to improve the quality of image deblurring. Furthermore，this paper presents a real-world dataset named DavisMCR.
strength_weakness: strength_weakness: 1. In this paper, DavisMCR dataset is proposed for evaluating the performance of different methods in dealing with real degradation events.

2. The experiments in the paper show that RDNet achieves good performance on both event recovery and deblurring tasks.
limitations: limitations: 1. The proposed method is too simple and the innovation is limited. Specifically, the conventional two-stage scheme and the encoder-decoder structure is adopted, and the innovation is only the introducing of events information.

2. More new events-based methods need to be added for comparison in the experimental section

3. The authors compared two event recovery methods, SCF and GEF, in Figure 4. Can further compare the differences with undegraded events in the REblur dataset? Additionally, could the deblurring results be compared with the visual quality of the ground truth images? Because visually it seems that the deblurring results obtained from events recovered using the SCF method also look good.

4. To facilitate a more effective comparison of event restoration results, the restored events of SCF and GEF are fed into a DeblurNet to obtain deblurred results. This DeblurNet is trained with undegraded events. However, in the ablation experiments it was illustrated that the RDNet was trained using both degraded and undegraded events. It was indicated that the results were better than using only degraded events or only undegraded events. Therefore, it would be advisable to feed the recovered events from SCF and GEF into the second stage of RDNet in order to compare the results of the deblurring, thus confirming that RDNet has a stronger event recovery capability. It is hoped that the author can provide more experimental details.

5. In terms of writing, it should be carefully improved. For example, should the b1,b2,b3 in Figure 1 described in lines 118 to 123 of the article be c1,c2,c3? The format of lines 330-334 is very non-standard.
paper_summary: paper_summary: This paper proposes a method, named RDNet, to improve the quality of image deblurring by restoring real-world degraded events captured by Dynamic Vision Sensors (DVS). The authors first analyze and model the degradation patterns in DVS events, including threshold bias, limited bandwidth, and circuit noise. They then construct paired degraded and undegraded events to train the first stage of RDNet for event restoration. The restored events are subsequently used to guide the second stage of RDNet for image deblurring. Additionally, the authors introduce a new real-world dataset, DavisMCR, containing events with varying degrees of degradation for comprehensive evaluation.
strength_weakness: strength_weakness: 1.The paper proposes a novel approach to improve image deblurring quality by addressing the degradation issue in real-world DVS events, which has been largely overlooked in previous works. The idea of restoring degraded events before deblurring is creative and well-motivated.
2.The authors provide a thorough analysis of the degradation mechanisms in DVS events and model them accordingly. The two-stage RDNet architecture, with the first stage dedicated to event restoration and the second stage for deblurring, is technically sound and well-designed.
3.The introduction of the DavisMCR dataset, containing real-world events with varying degradation levels, is a valuable contribution to the community. It provides a comprehensive testbed for evaluating deblurring performance under different degradation scenarios.
limitations: limitations: 1.Although the DavisMCR dataset is a valuable contribution, it would be beneficial to provide more details on the diversity of scenes and motion patterns it covers, as well as potential limitations or biases in the dataset.
2.The degraded and undegraded event pairs used for training the event restoration module were generated using v2e. A section comparing the degradation characteristics of real events and the simulated degradation from v2e should be added, analyzing the differences between them. While the simulated results from v2e are quite realistic, there may still be domain gaps between simulated and real events, and some analysis should be provided to avoid potential misunderstandings for the readers.
3.The degradation of events shown in the provided images is not apparent. Referring to the paper "arXiv2024 Seeing Motion at Nighttime with an Event Camera," events exhibit noticeable trailing artifacts in low-light conditions. Does the proposed method effectively correct the delayed or trailing events from event cameras in low-light scenarios?
4.More results on real-world events could be provided, along with no-reference perceptual quality metrics to demonstrate the superiority of the proposed method. Showing too many images and tables from the GOPRO dataset and REBlur is not very informative, as these datasets do not contain real events and the events are simulated from sharp frames. This provides additional information, but comparing against methods like Restormer is unfair.
paper_summary: paper_summary: Existing event-based image deblurring approaches focus on designing the network structures and multi-modality fusion units, neglecting the intrinsic degradation in real-world events. This paper designs a two-stage framework, where the first stage aims to restore the degraded events and the second stage incorporate the prior from the event restoration model into the image deblurring model.
strength_weakness: strength_weakness: 1.  The paper takes the degradations in the real-world events into consideration, thus attempting to firstly restoring the degraded events and then apply the event restoration model to guide the image deblurring. 

2. This paper develops a DavisMCR dataset, including extensive events with diverse degradation levels by manipulating environmental illumination and contrasts.
limitations: limitations: 1. The reviewer are confused whether the event branch in the second stage is fixed. If the image branch in the second stage is the pre-trained model from the first stage and whether it is updated in the second stage. The authors are encouraged to provide these details in Figure 2 and manuscript for clarity. 

2. This paper decomposes the event restoration and image deblurring into two stages. Why not formulate them into an end-to-end framework in a mutual learning manner? The reviewer expects the authors to discuss the motivation and purpose behind adopting a two-stage pipeline.