Learning Enriched Features via Selective State Spaces Model for Efficient Image Deblurring

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-29 12:31:31

Abstract: Image deblurring aims to restore a high-quality image from its corresponding blurred. The emergence of CNNs and Transformers has enabled significant progress. However, these methods often face the dilemma between eliminating long-range degradation perturbations and maintaining computational efficiency. While the selective state space model (SSM) shows promise in modeling long-range dependencies with linear complexity, it also encounters challenges such as local pixel forgetting and channel redundancy. To address this issue, we propose an efficient image deblurring network that leverages selective state spaces model to aggregate enriched and accurate features. Specifically, we introduce an aggregate local and global information block (ALGBlock) designed to effectively capture and integrate both local invariant properties and non-local information. The ALGBlock comprises two primary modules: a module for capturing local and global features (CLGF), and  a feature aggregation module (FA).  The CLGF module is composed of two branches: the global branch captures long-range dependency features via a selective state spaces model, while the local branch employs simplified channel attention to model local connectivity, thereby reducing local pixel forgetting and channel redundancy. In addition, we design a FA module to accentuate the local part by recalibrating the weight during the aggregation of the two branches for restoration. Experimental results demonstrate that the proposed method outperforms state-of-the-art approaches on widely used benchmarks.

Main paper: 
 introduction: 
 Image deblurring aims to recover a latent sharp image from its corrupted counterpart. Due to the ill-posedness of this inverse problem, many conventional approaches [12, 18] address this by explicitly incorporating various priors or hand-crafted features to constrain the solution space to natural images. Nonetheless, designing such priors proves challenging and lacks generalizability, which are impractical for real-world scenarios.Stimulated by the success of deep learning for high-level vision tasks, numerous data-driven methods have resorted CNN as a preferable choice and develop kinds of network architectural designs, including encoder-decoder architectures [4, 7, 9] , multistage networks [5, 47] , dual networks [2, 35, 39] , generative models [22, 23, 49] , and so on. While the convolution operation effectively models local connectivity, its intrinsic characteristics, such as limited local receptive fields and independence of input content, hinder the model's ability to eliminate long-range dependency features. To alleviate such limitations, various transformer variants [13, 21, 43, 45, 46] have been applied to image deblurring and have achieved better performance than the CNN-based methods as Figure 1 : Computational cost vs. PSNR of models on the Go-Pro dataset [34] . Our ALGNet achieve the SOTA performance while simultaneously reducing computational costs. they can better model the non-local information. However, image deblurring often deals with high-resolution images, and the attention mechanism in Transformers incurs quadratic time complexity, resulting in significant computational overhead. To alleviate computational costs, some methods [11, 46] opt to apply self-attention across channels instead of spatial dimensions. However, this approach fails to fully exploit the spatial information, which may affect the deblurring performance. While other methods [13, 45] utilize non-overlapping window-based self-attention for single image deblurring, the coarse splitting approach still falls short in fully exploring the information within each patch.State space models [15, 33, 40] , notably the enhanced version Mamba, have recently emerged as efficient frameworks due to their ability to capture long-range dependencies with linear complexity. However, Mamba's [15] recursive image sequence processing method tends to neglect local pixels, while the abundance of hidden states in the state space equation often results in channel redundancy, thereby impeding channel feature learning. Given the critical importance of local and channel features in image deblurring, directly applying the state space model often leads to poor performance. MambaIR [17] introduces the vision state space module, which utilizes a four-direction unfolding strategy to scan along four different directions for local enhancement, and incorporates channel attention to mitigate channel redundancy. Nevertheless, this four-directional scanning approach and the computation of state spaces result in increased computational overhead, potentially sacrificing the advantage of low computational resource utilization offered by the state spaces equation.Taking into account the above analyses, a natural question arises: Is it feasible to design a network that efficiently aggregates local and global features for image deblurring? To achieve this objective, we propose ALGNet, with several key components. Specifically, we present an aggregate local and global information block (ALGBlock) aimed at efficiently capturing and merging both local invariant properties and long-range dependencies. This ALGBlock consists of two key modules: a module dedicated to capturing local and global features (CLGF), and a feature aggregation module (FA). The CLGF module is further divided into two branches: the global branch, which utilizes a selective state spaces model to capture long-range dependency features with linear complexity, and the local branch, which incorporates simplified channel attention to effectively model local connectivity. This combination not only addresses issues like local pixel forgetting and channel redundancy but also empowers the network to capture more enriched and precise features. Additionally, given that image details are predominantly comprised of local features of images [6] , we design the FA module to underscores the significance of the local information in the restoration process by dynamically recalibrating the weights through a learnable factor during the aggregation of the CLGF two branches for restoration. Finally, we implement multiple scales for both input and output modes, aiming to alleviate training difficulty. As illustrated in Figure 1 , our ALGNet model achieves state-of-the-art performance while preserving computational efficiency compared to existing methods.The main contributions of this work are:( Due to the image deblurring ill-posed nature, many conventional approaches [12, 18] tackle this problem by relying on hand-crafted priors to constrain the set of plausible solutions. However, designing such priors is a challenging task and usually lead to complicated optimization problems. 
cnn-based methods.: 
 With the rapid advancement of deep learning, instead of manually designing image priors, lots of methods [2, 4, 22, 23, 35, 39, 47, 49] develop kinds of deep CNNs to solve image deblurring. To better explore the balance between spatial details and contextualized information, MPRNet [47] propose a cross-stage feature fusion to explore the features from different stages. MIRNet-V2 [48] introduces a multi-scale architecture to learn enriched features for image restoration. IRNeXt [9] rethink the convolutional network design and exploit an efficient and effective image restoration architecture based on CNNs. NAFNet [4] analyze the baseline modules and presents a simplified baseline network by either removing or replacing nonlinear activation functions. SFNet [10] and FSNet [8] design a multi-branch dynamic selective frequency module and a multibranch compact selective frequency module to dynamically select the most informative components for image restoration. Although these methods achieve better performance than the hand-crafted prior-based ones, the intrinsic properties of convolutional operations, such as local receptive fields, constrain the models' capability to efficiently eliminate long-range degradation perturbations.Due to the content-dependent global receptive field, the transformer architecture [44] has recently gained much popularity in image restoration [3, 13, 27, 43, 45, 46, 51] , demonstrating superior performance compared to previous CNN-based baselines. IPT [3] employs a Transformer-based multi-head multi-tail architecture, proposing a pre-trained model for image restoration tasks. However, image deblurring often deals with high-resolution images, and the attention mechanism in Transformers incurs quadratic time complexity, resulting in significant computational overhead. In order to reduce the computational cost, Uformer [45] , SwinIR [27] and U 2 former [13] computes self-attention based on a window. Nonetheless, the window-based approach still falls short in fully exploring the information within each patch. Restormer [46] and MRLPFNet [11] compute self-attention across channels rather than the spatial dimension, resulting in the linear complexity. However, this approach fails to fully exploit the spatial information. FFTformer [21] explores the property of the frequency domain to estimate the scaled dot-product attention, but need corresponding inverse Fourier transform, leading to additional computation overhead.State spaces models [15, 16, 32, 33, 40, 41] have recently emerged as efficient frameworks due to their ability to capture long-range dependencies with linear complexity. S4 [16] is the first structured SSM to model long-range dependency. S5 [41] propose the diagonal SSM approximation and computed recurrently with the parallel scan. Mega [32] introduced a simplification of S4 [16] to be realinstead of complex-valued, giving it an interpretation of being an exponential moving average. SGConv [14] and LongConv [26] focus on the convolutional representation of S4 and create global or long convolution kernels with different parameterizations. Mamba [15] propose a selective mechanism and hardware-aware parallel algorithm. Many vision tasks start to employ Mamba to tackle image classification [29, 52] , image segmentation [31] , and so on. However, the standard Mamba model still encounters issues with local pixel forgetting and channel redundancy when applied to image restoration tasks. To tackle this challenge, MambaIR [17] adopts a four-direction unfolding strategy to scan along four different directions and integrates channel attention. Nevertheless, this fourdirectional scanning approach results in increased computational overhead. In this work, we design an ALGBlock to efficiently capture and integrate both local invariant properties and long-range dependencies through a selective state spaces model with lower computational cost. As depicted in Figure 1 , our ALGNet outperforms MambaIR [17] while reducing computational costs by 96.1%. 
method: 
 In this section, we first outline the overall pipeline of our ALGNet. Subsequently, we delve into the details of the proposed ALGBlock, which includes the capturing local and global features module (CLGF) and the feature aggregation module (FA).The overall pipeline of our proposed ALGNet, shown in Figure 2 , adopts a single U-shaped architecture for image deblurring. Given a degraded image I ∈ R 𝐻 ×𝑊 ×3 , ALGNet initially applies a convolution to acquire shallow features F 0 ∈ R 𝐻 ×𝑊 ×𝐶 (𝐻,𝑊 , 𝐶 are the feature map height, width, and channel number, respectively). These shallow features undergo a four-scale encoder sub-network, progressively decreasing resolution while expanding channels. It's essential to note the use of multi-input and multi-output mechanisms for improved training. The low-resolution degraded images are incorporated into the main path through the Convs (consists of multiple convolutions and ReLU) and concatenation, followed by convolution to adjust channels. The in-depth features then enter a middle block, and the resulting deepest features feed into a four-scale decoder, gradually restoring features to the original size. During this process, the encoder features are concatenated with the decoder features to facilitate the reconstruction. Finally, we refine features to generate residual image X ∈ R 𝐻 ×𝑊 ×3 to which degraded image is added to obtain the restored image: Î = X + I. It's important to note that the three low-resolution results are solely used for training. We optimize the proposed network ALGNet with the following loss function: where 𝑖 denotes the index of input/output images at different scales, 𝐼 𝑖 denotes the target images and 𝐿 𝑐ℎ𝑎𝑟 is the Charbonnier loss:EQUATION𝐿 𝑐ℎ𝑎𝑟 = √︃ || Î𝑖 -𝐼 𝑖 || 2 + 𝜖 2 (2)with constant 𝜖 empirically set to 0.001 for all the experiments. 𝐿 𝑒𝑑𝑔𝑒 is the edge loss:𝐿 𝑒𝑑𝑔𝑒 = √︃ ||△ Î𝑖 -△𝐼 𝑖 || 2 + 𝜖 2 (3)where △ represents the Laplacian operator. 𝐿 𝑓 𝑟𝑒𝑞 denotes the frequency domains loss:EQUATION)where F represents fast Fourier transform, and the parameters 𝜆 and 𝛿 control the relative importance of loss terms, which are set to 0.1 and 0.05 as in [8, 47] , respectively.Transformer-based models [11, 13, 21, 45, 46] address the limitations of CNNs, such as a limited receptive field and lack of adaptability to input content. They excel in modeling non-local information, leading to high-quality image reconstruction, and have emerged as the dominant method for image deblurring. However, image deblurring commonly involves processing high-resolution images, and the attention mechanism in Transformers introduces quadratic time complexity, leading to considerable computational overhead. While it's possible to mitigate computational consumption by utilizing window-based attention [13, 45] or channel-wise attention [11, 46] , these methods inevitably lead to information loss.To address this challenge, we design the capturing local and global features module (CLGF) depicted in Figure 2 (b), aiming to capture long-range dependency features and model local connectivity with linear complexity. Specifically, given an input tensor 𝑋 𝑙 -1 , we initially process it through Layer Normalization (LN), Convolution, and Simple Gate (SG) to obtain spatial features 𝑋 𝑠 𝑙 -1 as follows:𝑋 𝑠 𝑙 -1 = 𝑆𝐺 (𝑓 𝑑𝑤𝑐 3×3 (𝑓 𝑐 1×1 (𝐿𝑁 (𝑋 𝑙 -1 )))) 𝑆𝐺 (𝑋 𝑓 0 ) = 𝑋 𝑓 1 ⊗ 𝑋 𝑓 2 (5)where 𝑓 𝑑𝑤𝑐 3×3 denotes the 3 × 3 depth-wise convolution, 𝑓 𝑐 1×1 represents 1 × 1 convolution. 𝑆𝐺 (•) is the simple gate, employed as a replacement for the nonlinear activation function. For a given input 𝑋 𝑓 0 ∈ R 𝐻 ×𝑊 ×𝐶 , SG initially splits it into two features In the global branch, depicted in Figure 2 (c), we opt for a statespace model (SSM) instead of Transformers to capture long-distance dependencies, ensuring linear complexity. Specifically, starting with an input feature 𝑋 𝑠 𝑙 -1 , we first reshape and normalize it using layer normalization (LN). Subsequently, it undergoes processing through two parallel branches. In the top branch, the feature channels are expanded by a linear layer, followed by activation through the SiLU function. In the bottom branch, the feature channels are expanded by a linear layer followed by the SiLU activation function, along with the selective state spaces model layer. The SSM inspired by the particular continuous system that maps a 1-dimensional function or sequence 𝑥 (𝑡) ∈ R → 𝑦 (𝑡) ∈ R through an implicit latent state ℎ(𝑡) ∈ R 𝑁 as follows:𝑋 𝑓 1 ∈ R 𝐻 ×𝑊 × 𝐶 2 and 𝑋 𝑓 2 ∈ R 𝐻 ×𝑊 × 𝐶 2 alongEQUATIONwhere A ∈ R 𝑁 ×𝑁 , B ∈ R 𝑁 ×1 , C ∈ R 1×𝑁 are four parameters, and 𝑁 is the state size. SSM first transform the continuous parameters A, B to discrete parameters A, B through fixed formulas A = 𝑒𝑥𝑝 (□A) and B = (□A) -1 𝑒𝑥𝑝 (□A -𝐼 ) • □B, where □ denotes the timescale parameter. After the discretization, the model can be computed as a linear recurrence way:ℎ 𝑡 = Aℎ 𝑡 -1 + B𝑥 𝑡 𝑦 𝑡 = Cℎ 𝑡 (7)or a global convolution way:EQUATION)where 𝑘 is the length of the input sequence, ⊛ denotes convolution operation, and K ∈ R 𝑘 is a structured convolution kernel. Selective SSM integrates a selection mechanism into SSM, making the parameters input-dependent. The selective SSM offers two key advantages. Firstly, it shares the same recursive form as Eq.7, enabling the model to capture long-range dependencies to aid in restoration. Secondly, the parallel scan algorithm enables SSM to leverage the advantages of parallel processing described in Eq.8, thereby facilitating efficient training.After that, features from the two branches are aggregated with the element-wise multiplication. Finally, the channel number is projected back and reshape to the original size. The total process can be defined as:𝐹 𝑡 = 𝑆𝑖𝐿𝑈 (𝐿𝑖𝑛𝑒𝑎𝑟 (𝑅𝑒𝑠ℎ𝑎𝑝𝑒 (𝐿𝑁 (𝑋 𝑠 𝑙 -1 ))), 𝐹 𝑏 = 𝑆𝑆𝑀 (𝑆𝑖𝐿𝑈 (𝑓 𝑐 1×1 (𝐿𝑖𝑛𝑒𝑎𝑟 (𝑅𝑒𝑠ℎ𝑎𝑝𝑒 (𝐿𝑁 (𝑋 𝑠 𝑙 -1 ))))) 𝐹 𝐺 = 𝑅𝑒𝑠ℎ𝑎𝑝𝑒 (𝐿𝑖𝑛𝑒𝑎𝑟 (𝐹 𝑡 ⊗ 𝐹 𝑏 )) (9)Despite the selective SSM's ability to capture long-range dependencies with linear computational complexity, it can result in issues like local pixel forgetting and channel redundancy, primarily due to the flattening strategy and an excessive number of hidden states. As shown in Figure 3 (a), when the 2D feature map is flattened into a 1D sequence, adjacent pixels (e.g., sequence number 1 and 11) become widely separated, leading to the issue of pixel forgetting. Additionally, as per [17] , we visualize the activation results for different channels in Figure 3 (b) and observe significant channel redundancy attributed to the larger number of hidden states in the selective SSM. To address the above challenges, we equip our CLGF with a local branch to model local connectivity and facilitate the expressive power of different channels. The local branch shown in Figure 2 (c), it is a simplified channel attention. Given the spatial features 𝑋 𝑠 𝑙 -1 , the local features 𝐹 𝐿 can be obtained by:EQUATION)where GAP is the global average pool. Noted that, in our CLGF module, we initially capture the spatial feature 𝑋 𝑠 𝑙 -1 , which aggregates neighboring information, before feeding it to the global branch. This approach effectively reduces the problem of local pixel forgetting.Given that image details primarily consist of local features, we design a feature aggregation (FA) module, depicted in Figure 2(b) , to highlight the significance of the local block in restoration. This is achieved by dynamically recalibrating the weights through a learnable factor during the aggregation of the two blocks for recovery. Specifically, given the global features 𝐹 𝐺 and local features 𝐹 𝐿 , the aggregate process can be defined as:EQUATIONwhere 𝑊 represents the learnable parameters, directly optimized by backpropagation and initialized as 1. It's worth noting that our design is exceptionally lightweight, as it does not introduce additional convolution layers. Finally, to capture richer and more accurate information, we refine the aggregated features 𝐹 𝐴 to obtain the output feature 𝑋 𝑙 of the ALGBlock as follows:EQUATION 
experiments: 
 We first describe the experimental details of the proposed ALGNet.Then we present both qualitative and quantitative comparisons between ALGNet and other state-of-the-art methods. Following that, we conduct ablation studies to validate the effectiveness of our approach. Finally, we assess the resource efficiency of ALGNet. Due to the page limit, additional results are provided in the supplementary material.4.1.1 Datasets. Image Motion Deblurring. Following recent methods [8, 47] , we train ALGNet using the GoPro dataset [34] , which includes 2,103 image pairs for training and 1,111 pairs for evaluation. To assess the generalizability of our approach, we directly apply the GoPro-trained model to the test images of the HIDE [37] and RealBlur [36] datasets. The HIDE dataset contains 2,025 images that collected for human-aware motion deblurring.Both the GoPro and HIDE datasets are synthetically generated, but the RealBlur dataset comprises image pairs captured under realworld conditions. This dataset includes two subsets: RealBlur-J, and RealBlur-R.Single-Image Defocus Deblurring. To evaluate the effectiveness of our method, we adopt the DPDD dataset [1] , following the methodology of recent approaches [8, 46] . This dataset comprises images from 500 indoor/outdoor scenes captured using a DSLR camera. Each scene consists of three defocused input images and a corresponding all-in-focus ground-truth image, labeled as the right view, left view, center view, and the all-in-focus ground truth. The DPDD dataset is partitioned into training, validation, and testing sets, comprising 350, 74, and 76 scenes, respectively. ALGNet is trained using the center view images as input, with loss values computed between outputs and corresponding ground-truth images.For various tasks, separate models are trained, and unless otherwise specified, the following parameters are utilized. The models are trained using the Adam optimizer [20] with parameters 𝛽 1 = 0.9 and 𝛽 2 = 0.999. The initial learning rate is set to 5 × 10 -4 and gradually reduced to 1 × 10 -7 using the cosine annealing strategy [30] . The batch size is chosen as 32, and patches of size 256 × 256 are extracted from training images. Data augmentation involves horizontal and vertical flips. We scale the network width by setting the number of channels to 32 and 64 for ALGNet and ALGNet-B, respectively.We present the performance of evaluated image deblurring approaches on the synthetic Go-Pro [34] and HIDE [37] datasets in Tables 1. Our ALGNet-B demonstrates a 0.43 dB improvement in performance over NAFNet-64 [4] on the GoPro [34] dataset. Compared with MambaIR [17] , which is also based on the state space model, our ALGNet demonstrates an improvement in performance by 0.28 dB, while ALGNet-B achieves a substantial improvement of 0.84 dB. Additionally, as depicted in Figure 1 , our ALGNet achieves even better performance through the scaling up of the model size, underscoring the scalability of ALGNet. Despite being trained solely on the GoPro [34] dataset, our network still achieves a significant gain of 0.19 dB PSNR over Restormer-Local [46] on the HIDE [37] dataset, demonstrating its generalization capability. Figure 4 illustrates that our model produces visually more pleasing results.We also evaluate our ALGNet on real-world images from the RealBlur dataset [36] under two experimental settings: (1) applying the GoPro-trained model directly on RealBlur, and ( 2) training and testing on RealBlur data. As shown in Table 2 , for setting 1, our ALGNet achieves performance gains of 0.16 dB on the RealBlur-R [46] and 0.13 dB on the RealBlur-J subset over DeBlurDiNAT-L [28] . Compared with MambaIR [17] , our gains are 0.37 dB and 0.30 dB on RealBlur-R and RealBlur-J, respectively. A similar trend is observed for setting 2, where our gains over MRLPFNet [11] are 0.24 dB on RealBlur-R. Although our ALGNet performs slightly inferiorly to MRLPNet in PSNR metric on the RealBlur-J dataset, our SSIM metric is higher. Moreover, for setting 1, our method outperforms MRLPNet, indicating superior generalization capability. Figure 5 presents visual comparisons of the evaluated approaches. Overall, the images restored by our model exhibit sharper details and are closer to the ground truth compared to those produced by other methods. 3 presents image fidelity scores of state-of-the-art defocus deblurring methods. ALGNet outperforms other state-of-the-art methods across all scene categories. Notably, in the combined scenes category, ALGNet exhibits a 0.15 dB improvement over the leading method IRNeXt [9] . In comparison to MambaIR [17] , which also relies on the state space model, our ALGNet showcases an improvement in performance by 0.48 dB in indoor scenes. The visual results in Figure 6 illustrate that our method recovers more details and visually aligns more closely with the ground truth compared to other algorithms.Here we present ablation experiments to verify the effectiveness and scalability of our method. Evaluation is performed on the GoPro dataset [34] , and the results are shown in Table . 4. The baseline is NAFNet [4] . We perform the break-down ablation by applying the proposed modules to the baseline successively, we can make the following observations:(1) When our CLGF module consists of only one local branch or global branch, the improvement in deblurring performance is not significant. There are two main reasons for this. Firstly, our baseline model is already capable of fully capturing local information, rendering the addition of a local branch ineffective in enhancing the model's representation ability. Secondly, since our global branch is based on the state-space model, although it can capture long-distance information, it often encounters issues such as local pixel loss and channel redundancy, as illustrated in Figure 3 . Therefore, when used alone, it fails to enhance performance. (2) When our CLGF module comprises both a local branch and a global branch, we observe a significant improvement in performance, up to 0.52 dB compared to the baseline. This indicates that CLGF has the ability to capture long-range dependency features and model local connectivity effectively. (3) The FA contributes a gain of 0.14 dB to our model.To further validate the effectiveness of our CLGF module, we apply ReLU and global average pooling operations on the output results of CLGF to obtain channel activation values (see Figure 7 ). It's evident that our CLGF successfully circumvents the issue of channel redundancy caused by an excessive number of hidden states in the state spaces model. Furthermore, to assess the advantage of our FA design, we compare it with other methods such as sum and concatenation. As shown in Table 5 , our FA consistently delivers superior results, indicating its effectiveness in emphasizing the importance of the local branch in restoration. Importantly, our design does not introduce any additional computational burden.Finally, we compare the feature maps before and after our AL-GBLock in Figure 8 . It is evident that the feature map from our local branch contains more detailed information compared to that from the global branch. Upon aggregation of the local and global branches of CLGF using FA, we observe a significant recovery of more details, particularly in the blurred license plate number present in the initial feature map.Table 6 : The evaluation of model computational complexity on the GoPro dataset [34] . The FLOPs are evaluated on image patches with the size of 256×256 pixels. The running time is evaluated on images with the size of 1280 × 720 pixels.Time(s) FLOPs(G) PSNR SSIM MPRNet [47] 1 We assess the model complexity of our proposed approach and state-of-the-art methods in terms of model running time and FLOPs. Table 6 and Figure 1 illustrate that our ALGNet model achieves SOTA performance while simultaneously reducing computational costs. Specifically, we achieve a 0.2 dB improvement over the previous best approach, FSNet [8] , with up to 84.7% cost reduction and nearly 1.5 times faster inference. Compared to MambaIR [17] , our ALGNet reduces computational costs by 96.1% and achieves 3.1 times faster inference. This underscores the efficiency of our method, demonstrating superior performance along with resource effectiveness. 
conclusion: 
 In  
 
 Appendix:  


Decision: In this paper, the authors propose an image deblurring network (ALGNet), which applies the selective state space model to aggregate the enriched features. The multiple ALGBlocks include a local and global feature capturing module and a feature aggregation module. The results show that the proposed ALGNet exhibits a relative better performance on various benchmarks than SOTA methods, while maintaining high computational efficiency. However, the reviewers still concern its weak novelty. 

After rebuttal, this paper receives 3 positive reviews and 1 negative review, so I recommend the acceptance of this paper.

Reviews: review: paper_summary: paper_summary: This paper propose a new network architecture called ALGNet for image deblurring, which aims to efficiently aggregate local and global features.ALGNet has designed a block that integrates local and global information (ALGBlock), which includes a module that captures local and global features (CLGF) and a feature aggregation module (FA). Experimental results show that ALGNet performs better than existing state-of-the-art methods while maintaining computational efficiency.
strength_weakness: strength_weakness: 1 The use of selective state spaces model (SSM) and the combination of local and global feature capturing modules (CLGF) is a novel approach that addresses the limitations of previous methods, such as local pixel forgetting and channel redundancy.

2 The FA module effectively highlights local features, which are crucial for image deblurring, without introducing significant computational overhead.

3 The paper presents extensive qualitative and quantitative evaluations on multiple datasets, including GoPro, HIDE, and RealBlur, demonstrating the generalizability and robustness of ALGNet.

4 ALGNet achieves significant improvements in performance while reducing computational costs and inference time, making it suitable for real-world applications.
limitations: limitations: 1 The selective state spaces model (SSM) used in the global branch tends to neglect local pixel information due to the flattening strategy and extensive distances between adjacent pixels when processed in a 1D sequence. This can result in a loss of local details that are crucial for effective image deblurring.

2 Although the SSM aims to model long-range dependencies with linear complexity, the four-directional scanning approach and the state spaces computations lead to increased computational overhead. This can potentially sacrifice the advantage of low computational resource utilization, which is one of the main benefits of using state space models.

3 The paper mentions that image deblurring often deals with high-resolution images, but the experimental results do not explicitly show the performance on such images. Including experiments on high-resolution images would help in understanding how well the model scales and performs on larger images.
paper_summary: paper_summary: This paper propose an UNet-strctured network with designed ALGBlocks for image deblurring. ALGBlock uses global branches with Mamba to realize linear-complexity long-range dependencies, and uses local branch to address pixel forgetting caused by Mamba. Subsequently it uses weighted summation for branch feature aggregation and finally uses refined blocks to obtain the output of block. Experimental results
show it outperforms existing approaches.
strength_weakness: strength_weakness: 1. The calirity is generally good.
2. The technical seems correct. The idea is clear.
3. The experiments contains Image Motion Deblurring and Single-Image Defocus Deblurring tasks, covering 4 datasets overall, which can show the generalization of the proposed method.
limitations: limitations: 1. Each subtitle in Related Works is still underconsidered. In my view, they should not be divided into 4 parts. They are 4 types of strategies the image deblurring adopts.
2. The novelty of overall method seems marginal. The overall architecture is UNet. The multi-input and multi-output is not a new strategy. Although the two-branch CLGF is issuse-driven, it is designed all based on previous works.
3. How to acquire the multiple inputs and outputs are not given in the text.
4. The writing logical of section 3.1 is a little confused. In line 404, the top branch (local branch) is mentioned first, but it is mentioned secondly in line 481 after introfucing the bottom branch (glocal branch). Also, there is a writing mistake in line 477 "as per [17]".
5. Although experimental results of several datasets prove the effectivenesss of proposed model, the improvement is minor.
6. It would be more convincing by showing the experimental result to prove that the local branch could solve the problem of pixel forgetting.
paper_summary: paper_summary: This paper introduces an efficient image deblurring network, ALGNet, which utilizes a selective state space model to aggregate enriched features. The network consists of multiple ALGBlocks, each containing a capturing local and global features module (CLGF) and a feature aggregation module (FA). Experimental results demonstrate that ALGNet outperforms state-of-the-art methods on multiple benchmarks while maintaining high computational efficiency.
strength_weakness: strength_weakness: 1. The innovation of this paper lies in the proposal of a novel ALGBlock, which integrates the selective state space model and channel attention mechanism. The model outperforms existing methods in multiple benchmark tests while maintaining high computational efficiency. 
2. The paper provides clear technical details to ensure the reproducibility of the experiments. The content is clearly expressed, and the figures effectively demonstrate the model structure and experimental results.
3. The ablation experiment design is comprehensive, effectively verifying the role of individual components.
limitations: limitations: 1. The novelty of the manuscript is somewhat week. The proposed method combines convolution and SSM. However, the idea of combining local processing and long-term dependency is commonly-seen in the transformer-based methods. Compared to these methods, the contribution of this method mainly lieds in the replacement of SSM with self-attention, which is somewhat week.
2. The related RNN-based works should be discussed. By viewing image as pixel/patch sequence, NLP architecutre can be used for image processing. Except for transformer, RNN has also been considered for image restoration, see [1,2]. The discussion about the advantages of SSM over RNN is necessary here. 
   [1] Learning Recursive Filters for Low-Level Vision via a Hybrid Neural Network,ECCV,2016
   [2] Deblurring Dynamic Scenes via Spatially Varying Recurrent Neural Networks, TPAMI, 2021
3. As I know, one of the advantages of SSM over RNN is that SSM allows parallel training. So, It would be better to highlight the advantage by comparing the training efficiency. However, relative experiments are lack in the manuscript.
paper_summary: paper_summary: This paper introduces the selective state space model, renowned for its long-range modeling within linear complexity, into the image deblurring task, further developing an aggregate local and global information block. Experimental results demonstrate that the proposed algorithm outperforms state-of-the-art approaches on various benchmarks.
strength_weakness: strength_weakness: 1. The proposed approach conduct the complementary global and local branch, implemented by a selective state space model and channel attention. 

2. The proposed ALGNet achieves the superior performance while preserving efficiency.
limitations: limitations: 1. MambaIR designs a Residue State Space Block by cascading VSSM and channel attention. It seem that the ALGNet reformulates this cascading structure into a parallel one. The authors are encouraged to discuss the specific improvements brought by this reformulation, excluding the performance gains. 

2. The authors claim that this combination of global and local brancher not only addresses issues like local pixel forgetting and channel redundancy but also empowers the network to capture more enriched and precise features. However, Figure 3 only illustrates the channel redundancy of the global branch. It does not adequately demonstrate how introducing the local branch alleviates this problem. Additional evidence is needed to support this claim.

3. The reviewer is curious whether cascading global and local branches can achieve comparable performance and diverse channel responses. This alternative structure should be explored and compared with the proposed parallel design.

4. The compared baselines in Tables 1, 2, and 4 are inconsistent. The authors are suggested to include the missing baselines in Tables 2 and 4. Specifically, Table 4 lacks comparisons with efficient algorithms like FFTformer [1]. Including such comparisons would provide a more comprehensive evaluation of the proposed method’s performance and efficiency. 

[1] Kong L, Dong J, Ge J, et al. Efficient frequency domain-based transformers for high-quality image deblurring[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023: 5886-5895.