Original: S ELECTIVE F REQUENCY N ETWORK FOR I MAGE R ESTORATION
Revised: S ELECTIVE F REQUENCY N ETWORK FOR I MAGE R ESTORATION

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Image restoration aims to reconstruct the latent sharp image from the corrupted observation.
Revised: Image restoration aims to reconstruct the latent sharp image from its corrupted counterpart.

Original: Besides dealing with this long-standing task in the spatial domain, a few approaches seek solutions in the frequency domain based on the large discrepancy between spectra of sharp/degraded image pairs.
Revised: Besides dealing with this long-standing task in the spatial domain, a few approaches seek solutions in the frequency domain in consideration of the large discrepancy between spectra of sharp/degraded image pairs.

Original: However, these works  utilizethe existing transformation tools, e.g. , Wavelet transform, to split the feature into several  parts, which is not flexible enough to select the most informative frequency component to recover.
Revised: However, these works commonly utilize transformation tools, e.g. , wavelet transform, to split features into several frequency parts, which is not flexible enough to select the most informative frequency component to recover.

Original: In this paper, we exploit a multi-branch and content-aware module to decompose the feature into separate frequency subbands dynamically and locally, and then accentuate the useful ones viathe channel-wise attention mechanism.
Revised: In this paper, we exploit a multi-branch and content-aware module to decompose features into separate frequency subbands dynamically and locally, and then accentuate the useful ones via channel-wise attention weights.

Original: In addition,aiming to cope with the large-scale degradation kernel, we propose an extremely simple decoupling and modulation module to enlarge the receptive field via global and window-based average pooling.
Revised: In addition, to handle large-scale degradation blurs, we propose an extremely simple decoupling and modulation module to enlarge the receptive field via global and window-based average pooling.

Original: Integrating two developed modules into a U-Net backbone, the proposed S elective F requency
Revised: Integrating two developed modules into a U-Net backbone, the proposed S elective F requency

Original: Net work (SFNet) performs favorably against state-of-the-art algorithms on five image restoration tasks, including single-image defocus deblurring, image dehazing, image motion deblurring, image desnowing, and image deraining.
Revised: Net work (SFNet) performs favorably against state-of-the-art algorithms on five image restoration tasks, including single-image defocus deblurring, image dehazing, image motion deblurring, image desnowing, and image deraining.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Image restoration aims to recover a high-quality image by removing degradations, e.g. , noise, blur, and snowflake.
Revised: Image restoration aims to recover a high-quality image by removing degradations, e.g. , noise, blur, and snowflake.

Original: In view of its important role in surveillance, self-driving techniques, and remote sensing, image restoration has gathered considerable attention from industry and academic communities.
Revised: In view of its important role in surveillance, self-driving techniques, and remote sensing, image restoration has gathered considerable attention from industrial and academic communities.

Original: However, due to its ill-posed property, many conventional approaches address this problem based on various assumption (Zhang et al., 2022; Yang et al., 2020b) or hand-crafted features (Karaali & Jung, 2017), which are incapable of generating faithful results in real-world scenarios.
Revised: However, due to its ill-posed property, many conventional approaches address this problem based on various assumptions (Zhang et al., 2022; Yang et al., 2020b) or hand-crafted features (Karaali & Jung, 2017), which are incapable of generating faithful results in real-world scenarios.

Original: Recently, deep neural networks have witnessed the rapid development of image restoration and obtained favorable performance compared to conventional methods.
Revised: Recently, deep neural networks have witnessed the rapid development of image restoration and obtained favorable performance compared to conventional methods.

Original: A flurry of convolutional neural networks (CNN) based methods have been developed for diverse image restoration tasks by inventing or borrowing advanced modules, including dilated convolution (Luo et al., 2022;
Revised: A flurry of convolutional neural networks (CNN) based methods have been developed for diverse image restoration tasks by inventing or borrowing advanced modules, including dilated convolution (Luo et al., 2022;

Original: Zou et al., 2021), U-Net (Ronneberger et al., 2015), residual learning (Zhang et al., 2017), multi-stage pipeline (Zhang et al., 2019b), and attention mechanisms (Liu et al., 2019).
Revised: Zou et al., 2021), U-Net (Ronneberger et al., 2015), residual learning (Zhang et al., 2017), multi-stage pipeline (Zhang et al., 2019b), and attention mechanisms (Liu et al., 2019).

Original: However, with convolution units, these methods have limited receptive fields, and thus they are not capable of capturing long-range dependencies.
Revised: However, with convolution units, these methods have limited receptive fields, and thus they are not capable of capturing long-range dependencies.

Original: This requirement is essential for restoration tasks, since a single pixel needs information from its surrounding region to be recovered.
Revised: This requirement is essential for restoration tasks, since a single pixel needs information from its surrounding region to be recovered.

Original: More recently, many frameworks tailor Transformer (Vaswani et al., 2017) fora variety of image restoration tasks, including motion deblurring (Tsai et al., 2022a), dehazing (Guo et al., 2022; Song et al., 2022) and desnowing (Chen et al., 2022b;c).
Revised: More recently, many researchers have tailored Transformer (Vaswani et al., 2017) for image restoration tasks, such as motion deblurring (Tsai et al., 2022), dehazing (Guo et al., 2022; Song et al., 2022) and desnowing (Chen et al., 2022b;c).

Original: Nonetheless, the above-mentioned methods  conduct restoration in the spatial domain, which do not sufficiently leveragethe frequency discrepancies betweenthe sharp/degraded image pairs.
Revised: Nonetheless, the above-mentioned methods mainly conduct restoration in the spatial domain, which do not sufficiently leverage frequency discrepancies between sharp/degraded image pairs.

Original: To this end, a few works utilize theexisting transformation tools, e.g., Wavelet transform or Fourier transform, to decompose the feature into various frequency components and then treatthe separate parts individually to reconstruct the corresponding feature (Selesnick et al., 2005; Yang & Fu, 2019; Zou et al., 2021; Mao et al., 2021).
Revised: To this end, a few works utilize the transformation tools, e.g., wavelet transform or Fourier transform, to decompose features into different frequency components and then treat separate parts individually to reconstruct the corresponding feature (Selesnick et al., 2005; Yang & Fu, 2019; Zou et al., 2021; Mao et al., 2021).

Original: Nevertheless, Wavelet transform decouples the feature map into different subbands in a fixed manner, and thus it is not capable of distinguishing the most informative or useless frequency components to enhance or suppress.
Revised: Nevertheless, wavelet transform decouples the feature map into different subbands in a fixed manner, and thus it is not capable of distinguishing the most informative or useless frequency components to enhance or suppress.

Original: In addition, these methods need corresponding inverse Fourier/Wavelet transform, leading tothe additional computation overhead.
Revised: In addition, these methods need corresponding inverse Fourier/wavelet transform, leading to additional computation overhead.

Original: To overcome the above drawbacks and select the most informative frequency component to reconstruct, we propose a novel decoupling and recalibration module for image restoration tasks, named Multi-branch Dynamic Selective Frequency module (MDSF).
Revised: To overcome the above drawbacks and select the most informative frequency component to reconstruct, we propose a novel decoupling and recalibration module for image restoration tasks, named Multi-branch Dynamic Selective Frequency module (MDSF).

Original: Specifically, we utilize the multi-branch learnable filters to generate high- and low-frequency maps dynamically and locally, and then leverage the channel-wise attention mechanism, modified from (Li et al., 2019), to emphasize or suppress the resulting frequency components.
Revised: Specifically, we utilize the multi-branch learnable filters to generate high- and low-frequency maps dynamically and locally, and then leverage the channel-wise attention mechanism, modified from (Li et al., 2019), to emphasize or suppress the resulting frequency components.

Original: Our module has two key advantages.
Revised: Our module has two key advantages.

Original: First, according to the input and task, the decoupling step dynamically generates the filter to decomposethe feature map.
Revised: Firstly, according to the input and task, the decoupling step dynamically generates filters to decompose feature maps.

Original: Second, our module does not introduceany extra inverse transformation.
Revised: Secondly, our module does not introduce extra inverse transform.

Original: Receptive field is another critical factor for image restoration tasks due to the various sizes of degradation kernels (Suin et al., 2020; Son et al., 2021).
Revised: Receptive field is another critical factor for image restoration tasks due to the various sizes of degradation blurs (Suin et al., 2020; Son et al., 2021).

Original: To complement the above dynamic module (MDSF) that processes features locally, we further propose a simple yet effective module, dubbed Multi-branch Compact Selective Frequency module (MCSF), to enhance the helpful frequency signals based onthe multiple and relatively global receptive fields.
Revised: To complement the above dynamic module, MDSF, that processes features locally, we further propose a simple yet effective module, dubbed Multibranch Compact Selective Frequency module (MCSF), to enhance the helpful frequency signals based on multiple and relatively global receptive fields.

Original: Specifically,based on the analyses, we utilizethe global and window-based average pooling  to attain disparate frequency maps, and then use learnable parameters to modulate the resulting maps without resorting to any convolution layers.
Revised: Specifically, we utilize global and windowbased average pooling techniques to attain disparate frequency maps, and then use learnable parameters to modulate the resulting maps without resorting to any convolution layers.

Original: Compared to MDSF, besides the enlarged receptive fields, MCSF is lightweight enough to be embeddedinto the backbone in multiple positions.
Revised: Compared to MDSF, besides the enlarged receptive fields, MCSF is lightweight enough to be embedded in multiple positions of the backbone.

Original: Our contributions  are summarized as follows:
Revised: The main contributions of this study are summarized as follows:

Original: • We propose a multi-branch dynamic selective frequency module (MDSF) that is capable of decouplingthe feature map into different frequency components dynamically via the theoretically proved filters, andthen selecting the most informative components to recover.
Revised: • We propose a multi-branch dynamic selective frequency module (MDSF) that is capable of decoupling feature maps into different frequency components dynamically via the theoretically proved filters, and selecting the most informative components to recover.

Original: • We design a multi-branch compact selective frequency module (MCSF) that performs frequency decoupling and recalibration based on multi-scale average pooling operations to pursue a large receptive field for the spatially-varying degradation kernels.
Revised: • We develop a multi-branch compact selective frequency module (MCSF) that performs frequency decoupling and recalibration using multi-scale average pooling operations to pursue a large receptive field for large-scale degradation blurs.

Original: • Incorporating MDSF and MCSF into a U-shaped backbone, a selective frequency network (SFNet)is formed for image restoration. SFNet achieves state- of-the-art results on five image restoration tasks: defocus/motion deblurring, dehazing, deraining, and desnowing.
Revised: • Incorporating MDSF and MCSF into a U-shaped backbone, the proposed selective frequency network (SFNet) achieves state-of-the-art results on five image restoration tasks, including image defocus/motion deblurring, dehazing, deraining, and desnowing.

Original:  R ELATED WORK
Revised:  R ELATED WORK

Original: Image Restoration.
Revised: Image Restoration.

Original: Prior to the deep learning era, a great number of methods have been proposed for image restoration problems based on various assumptions and hand-crafted features (Sezan &
Revised: Prior to the deep learning era, a great number of methods have been proposed for image restoration problems based on various assumptions and hand-crafted features (Sezan &

Original: Tekalp, 1990; Kundur & Hatzinakos, 1996; Calvetti et al., 1999).
Revised: Tekalp, 1990; Kundur & Hatzinakos, 1996; Calvetti et al., 1999).

Original: In recent years, with the rapid development of deep learning, a flurry of approaches have been investigated utilizing convolutional neural networks for image motion deblurring (Zamir et al., 2021; Yuan et al., 2020; Zhang et al., 2019b; Purohit et al., 2021), defocus deblurring (Ruan et al., 2022; Abuolaim & Brown, 2020; Son et al., 2021), desnowing (Chen et al., 2021c; 2020a), dehazing (Dong et al., 2020; Liu et al., 2019;
Revised: In recent years, with the rapid development of deep learning, a flurry of approaches have been investigated utilizing convolutional neural networks for image motion deblurring (Zamir et al., 2021; Yuan et al., 2020; Cui et al., 2023; Purohit et al., 2021), defocus deblurring (Ruan et al., 2022; Abuolaim & Brown, 2020; Son et al., 2021), desnowing (Chen et al., 2021c; 2020a), dehazing (Dong et al., 2020; Liu et al., 2019;

Original: Ren et al., 2016; Zhang et al., 2018), and deraining (Yang et al., 2020b; Wang et al., 2019).
Revised: Ren et al., 2016; Zhang et al., 2018), and deraining (Yang et al., 2020b; Wang et al., 2019).

Original: More recently, to capturethe long-range dependencies, many works have borrowedthe Transformermodel (Vaswani et al., 2017) from  natural language processing field into image restoration (Chen et al., 2021a; Liang et al., 2021; Zamir et al., 2022; Wang et al., 2022) and specific tasks such as image motion deblurring (Tsai et al., 2022a), dehazing (Song et al., 2022), and desnowing (Chen et al., 2022b).Additionally, a few works attempt to deal with image restoration using multi-layer perceptron (Tu et al., 2022) and adder neural networks (Song et al., 2020).
Revised: More recently, to capture long-range dependencies, many works have borrowed Transformer (Vaswani et al., 2017) from the natural language processing field into image restoration (Chen et al., 2021a; Liang et al., 2021; Zamir et al., 2022; Wang et al., 2022) and specific tasks such as image motion deblurring (Tsai et al., 2022), dehazing (Song et al., 2022), and desnowing (Chen et al., 2022b).

Original: In this paper, instead of exploiting the more advanced backbone for image restoration, we pay more attention to the frequency selection mechanism based onCNN, which is more efficient than Transformer type network.
Revised: In this study, instead of exploiting a more advanced backbone for image restoration, we pay more attention to the frequency selection mechanism based on efficient CNN.

Original: Frequency Based Image Restoration.
Revised: Frequency Based Image Restoration.

Original: Many algorithms have been developed to address various low-level vision problems from a frequency perspective.
Revised: Many algorithms have been developed to address various low-level vision problems from a frequency perspective.

Original: Specifically, Xu et al.
Revised: Specifically, Chen et al.

Original: (Chen et al., 2021c) propose a hierarchical desnowing network based on dual-tree complex wavelet representation (Selesnick et al., 2005).
Revised: (Chen et al., 2021c) propose a hierarchical desnowing network based on dual-tree complex wavelet representation (Selesnick et al., 2005).

Original: Yang et al.
Revised: Yang et al.

Original: (Yang & Fu, 2019) develop the Wavelet based U-Net to replace up-sampling and down-sampling.
Revised: (Yang & Fu, 2019) develop the wavelet based U-Net to replace up-sampling and down-sampling.

Original: Zou et al.
Revised: Zou et al.

Original: (Zou et al., 2021) utilize Wavelet transform based module to help recoverthe texture details.
Revised: (Zou et al., 2021) utilize wavelet transform based module to help recover texture details.

Original: Yang et al.
Revised: Yang et al.

Original: (Yang et al., 2020a) devise a Wavelet structure similarity loss function for training.
Revised: (Yang et al., 2020a) devise a wavelet structure similarity loss function for training.

Original: Mao et al.
Revised: Mao et al.

Original: (Mao et al., 2021) use Fourier transform tointegrate both high- and low-frequency residual information. Yoo et al. (Yoo et al., 2018) complete image restoration based on the estimation of the DCT coefficient distribution. In this work, we pursue a dynamic and efficient manner to select the useful frequency part at multiple receptive fields.
Revised: (Mao et al., 2021) use Fourier transform to

Original: C
Revised: C

Original: ResBlock ResBlock
Revised: ResBlock ResBlock

Original: ResBlock ResBlock
Revised: ResBlock ResBlock

Original: ResBlock ResBlock
Revised: ResBlock ResBlock

Original: Conv 3 ×
Revised: Conv 3 ×

Original: Conv 3 × Conv 3 ×
Revised: Conv 3 × Conv 3 ×

Original: Conv 1 × Conv 1 ×
Revised: Conv 1 × Conv 1 ×

Original: Conv 3 ×
Revised: Conv 3 ×

Original: C C
Revised: C C

Original: C
Revised: C

Original: Conv 3 × Conv 3 ×
Revised: Conv 3 × Conv 3 ×

Original: Shallow Layer
Revised: Shallow Layer

Original: Restored Image Degraded Image  ×  × × × × × Shallow{+(cid:3024)(cid:
 ) 𝐶×𝐻×𝑊+}
Revised: Restored Image Degraded Image 2𝐶× (cid:3009)(cid:2870) × (cid:3024)(cid:2870) 4𝐶× (cid:3009)(cid:2872) × {+(cid:3024)(cid:
 ) 𝐶×𝐻×𝑊+}

Original:  LayerG A P
Revised: Shallow Layer

Original: F C F C

Original: C o n c a t S o � m a x S p li t F C

Original: C o n v×
Revised: C o n v×

Original: C o n v× C o n v×
Revised: C o n v× C o n v×

Original: C o n v×* Convolu � on
Revised: C o n v×* Convolution

Original: C
Revised: C

Original: Element-wise Mul � plica � on Element-wise 
Revised: Element-wise Multiplication Element-wise Addition

Original: Addi � on Concatena � on
Revised: Concatenation

Original: Upsample Downsample× N × N × N × N × N × N (c) ResBlock (b) Shallow Layer (e) Modulator (d) Decoupler (a) 
Revised: Upsample Downsample (c) ResBlock (b) Shallow Layer (e) Modulator (d) Decoupler (a) Overall architecture

Original: Invert * S p li t
Revised: Invert * S p li t

Original: C o n c a t
Revised: C o n c a t

Original: G A P C o n v B N  *
Revised: G A P C o n v B N 𝑥 (cid:3036)(cid:3041) 𝑥 (cid:3035) 𝑥 (cid:3039) *

Original: C o n c a t
Revised: C o n c a t

Original:  C o n v×
Revised: G A P F C F C C o n c a t S o f t m a x S p li t F C S u m 𝑥 (cid:3035) 𝑥 (cid:3039)

Original: M C SFC
Revised: M C SF

Original:  o n v 3 ×
Revised: C o n v×

Original: M D SF
Revised: M D SF

Original: D e c o up l e r
Revised: D e c o up l e r

Revised: M o du l a t

Revised: o r

Original: M o du l a t o r (f) MCSF
Revised: C o n v× C o n v× C o n v× M C SF × N

Revised: the operation of subtracting the low-pass filter from the identity filter

Original: GAP
Revised: integrate both high- and low-frequency residual information. Yoo et al. (Yoo et al., 2018) complete image restoration based on the estimation of the DCT coefficient distribution. In this work, we pursue a dynamic and efficient manner to select the useful frequency part at multiple receptive fields.

Original:  M ETHODOLOGY
Revised:  M ETHODOLOGY

Original: We first describe the overall pipeline ofour SFNet (Fig. 1 (a)).
Revised: We first describe the overall architecture of SFNet (Fig. 1 (a)).

Original: Then we present the proposed modules, MDSF (Fig.
Revised: Then we present the proposed modules, MDSF (Fig.

Original: 1 (d,e)) and MCSF (Fig. 1 (f)).
Revised: 1 (d,e)) and MCSF.

Original: The loss function follows in the final part.
Revised: The loss functions follow in the final part.

Original: 3. O VERALL P IPELINE
Revised: 3. O VERALL A RCHITECTURE

Original: Our network applies thepopular encoder-decoder architecture to learn hierarchical representations.
Revised: Our network adopts the encoder-decoder architecture to learn hierarchical representations.

Original: Specifically, SFNet consists of three decoders and three encoders, each containing multiple ResBlocks (Fig.
Revised: Specifically, SFNet consists of a three-scale decoder and a three-scale encoder. Each scale is comprised of a ResBlock (Fig.

Original: 1 (c)).
Revised: 1 (c)).

Original: Our modules are inserted into the ResBlock successively where MDSF is only deployed in the last ResBlock of each en/decoder while MCSF exists in all blocks.
Revised: MDSF is only deployed in the last residual block of each ResBlock while MCSF exists in all blocks.

Original: Following  (Cho et al., 2021; Mao et al., 2021; Tu et al., 2022), multi-input and multi-output mechanisms are used to ease the difficulty of training.
Revised: Following previous methods (Cho et al., 2021; Mao et al., 2021; Tu et al., 2022), multi-input and multi-output mechanisms are used to ease training difficulty.

Original: To be specific, the images with reduced sizes are merged into the main path via the shallow layer (Fig.
Revised: Specifically, input images of reduced sizes are merged into the main path via the shallow layer (Fig.

Original: 1 (b))
Revised: 1 (b)),

Original: and the predicted images are produced by 3 ×convolutional layers.
Revised: and the predicted images are produced by 3 × 3 convolutional layers after each scale of decoder.

Original: Besides, we adoptthe feature-level and image-level skip connections to assist training.
Revised: In addition, we adopt feature-level and image-level skip connections to assist training.

Original: In Fig.
Revised: In Fig.

Original: 1 we only show the top-level image skip connection for clarity.
Revised: 1, we only show the top-level image skip connection for clarity.

Original: The up-sampling and down-sampling layers are implemented by transpose and stride convolutions, respectively.
Revised: The up-sampling and down-sampling layers are implemented by transposed and strided convolutions, respectively.

Original: 3. M ULTI - BRANCH DYNAMIC SELECTIVE FREQUENCY MODULE (MDSF)
Revised: 3. M ULTI - BRANCH D YNAMIC S ELECTIVE F REQUENCY M ODULE (MDSF)

Original: To select the informative frequency component to reconstruct, MDSF mainly consists of two elements: frequency decoupler (Fig. 1d) and modulator (Fig. 1e).
Revised: To select the informative frequency component to reconstruct, MDSF mainly contains two elements: frequency decoupler (Fig. 1 (d)) and modulator (Fig. 1 (e)).

Original: Decoupler is capable of decomposing the feature map into separate frequency parts dynamically based onthe learned filters, and  modulator utilizes channel-wise attention to accentuate the helpful frequency.
Revised: Decoupler decomposes features into separate frequency parts dynamically based on learned filters, and then modulator utilizes channelwise attention to accentuate the useful frequency.

Original: Besides, to provide various local receptive fields, MDSF contains multiple branches by splitting channel dimension, each with a different filter kernel size.
Revised: Additionally, to provide various local receptive fields, MDSF splits features among the channel dimension, and then applies different filter sizes to separate parts.

Original: For convenience, we only show  one-branch case in Fig.
Revised: We only show the one-branch case in Fig.

Original: 1 (d).
Revised: 1 (d) for simplicity.

Original: To dynamically decompose feature maps, we utilize the learnable and theoretically proven low-pass filter (refer to Appendix B for the proof) and the corresponding high-pass filter to generate low- and high-frequency maps.
Revised: To dynamically decompose feature maps, we utilize the learnable and theoretically proven low-pass filter (refer to Appendix B for the proof) and the corresponding high-pass filter to generate low- and high-frequency maps.

Original: The learned filters are shared across the group dimension to strike a balance between complexity and feature diversity.
Revised: The learned filters are shared across the group dimension to strike a balance between complexity and feature diversity.

Original: Specifically, given any feature map X ∈ R C × H × W , where C is the number of channels and H × W denotes the spatial dimension, we firstly leverage the filter-generating layer to produce the low-pass filter for each group of the input, formulated as
Revised: Specifically, given any feature map X ∈ R C × H × W , where C is the number of channels and H × W denotes the spatial dimension, we firstly leverage the filter-generating layer to produce the low-pass filter for each group of the input, formulated as

Original: [Equation]
Revised: [Equation]

Original: where F l ∈ R k 2 g × 1 × 1 , k 2 is the kernel size of low-pass filter; g denotes the number of groups;
Revised: where F l ∈ R k 2 g × 1 × 1 , k × k is the kernel size of low-pass filter; g denotes the number of groups;

Original: B , W ,  GAP are Batch Normalization (Ioffe & Szegedy, 2015), the parameters of convolution and global average pooling, respectively.
Revised: B , W , and GAP are Batch Normalization (Ioffe & Szegedy, 2015), the parameters of convolution and global average pooling, respectively.

Original: The group-based operation has fewer parameters and complexities than generating filters for each pixel.
Revised: The group-based operation has fewer parameters and lower complexity than generating filters for each pixel.

Original: The number of groups is discussed in Sec 4.3.
Revised: The number of groups is discussed in Sec 4.3.

Original: To attain the high-pass filter, we subtract the resulting low-pass filter from the identity kernel with central value as one and everywhere else  zero.
Revised: To attain the high-pass filter, we subtract the resulting low-pass filter from the identity kernel with central value as one and everywhere else as zero.

Original: Next, for each group featuremap X i ∈ R C i × H × W , where i is the group index and C i =
Revised: Next, for each group feature X i ∈ R C i × H × W , where i is the group index and C i =

Original: Cg , its low- and high-frequency components can be obtained by using the corresponding reshaped filter F L and F H ( ∈ R g × k × k ), which is expressed as:
Revised: Cg , its low- and high-frequency components can be obtained by using the corresponding reshaped filter F L and F H ( ∈ R g × k × k ), which is expressed as:

Original: [Equation]
Revised: [Equation]

Original: where c is the index of a channel; h and w denotethe spatial coordinates; and p, q ∈ {− 1 , 0 , 1 } .
Revised: where c is the index of a channel; h and w denote spatial coordinates; and p, q ∈ {− 1 , 0 , 1 } .

Original: After decoupling the feature map into different frequency components, we leverage the frequency modulator to emphasize the genuinely helpful part for reconstruction, as shown in Fig.
Revised: After decoupling the feature map into different frequency components, we leverage the frequency modulator to emphasize the genuinely useful part for reconstruction, as illustrated in Fig.

Original: 1 (e).
Revised: 1 (e).

Original: The modulator works among the channel dimension based on the modified SKNet (Li et al., 2020).
Revised: The modulator works among the channel dimension based on the modified SKNet (Li et al., 2020).

Original: Formally, given two frequency maps, X l and X h , we first generate the fused feature by,
Revised: Formally, given two frequency maps, X l and X h , we first generate the fused feature by,

Original: [Equation]
Revised: [Equation]

Original: where W fc is the parameters of a fully connected layer.
Revised: where W fc is the parameters of a fully connected layer.

Original: To attainthe channel-wise weights, we use two other fully connected layers followed by concatenation and Softmax function, formulated as:
Revised: To attain channel-wise weights, we use two other fully connected layers followed by concatenation and Softmax function, formulated as:

Original: [Equation]
Revised: [Equation]

Original: where W l and W h are channel-wise attention weights for two frequency parts; W l and W h are parameters of fully connected layers; [ · , · ] denotes concatenation; and c is the channel index ofthe concatenated features.
Revised: where W l and W h are channel-wise attention weights for two frequency parts; W l and W h are parameters of fully connected layers; [ · , · ] denotes concatenation; and c is the channel index of concatenated features.

Original: Compared to SKNet (Li et al., 2020), which performs Softmax on each channel as W 
Revised: Compared to SKNet (Li et al., 2020), which performs Softmax on each channel as W lc =

Original: [Equation]
Revised: e Wl ( Z )

Original:  , we consider all channels  to facilitate interactions between different channels of twoinput maps.
Revised: c e Wl ( Z ) c + e Wh ( Z ) c , we consider all channels into consideration to facilitate interactions between different channels of two maps.

Original: Then, the finalfrequency weights can be obtained by split operation.
Revised: Then, the final weights can be obtained by split operation.

Original: Based on the above one-branch case, the multiple branches with varied filter sizes can be express as:
Revised: Based on the above one-branch case, the multiple branches with varied filter sizes can be express as:

Original: [Equation]
Revised: [Equation]

Original: where D and M denote decoupler and modulator,  and X m is the equally split feature over channels.
Revised: where D and M denote decoupler and modulator, respectively, and X m is the equally split feature.

Original: 3. M ULTI - BRANCH COMPACT SELECTIVE FREQUENCY MODULE (MCSF)
Revised: 3. M ULTI - BRANCH C OMPACT S ELECTIVE F REQUENCY M ODULE (MCSF)

Original: Sincethe receptive field plays a critical role in image restoration, where degradation kernels always differ in size (Son et al., 2021; Mao et al., 2021), we develop MCSF to efficiently enlarge the receptive field of SFNet.
Revised: Since receptive field plays a critical role in image restoration, where degradation blurs always differ in size (Son et al., 2021; Mao et al., 2021), we develop MCSF to efficiently enlarge the receptive field of SFNet.

Original: MCSF has two branches with different receptive fields,  the global branch and window-based branch.
Revised: MCSF has two branches with different receptive fields, i.e., the global branch and window-based branch.

Original: Considering these branches share a similar paradigm, we only detail the window-based one, which is inspired from the idea of window-based attention (Liu et al., 2021).
Revised: Considering these branches share a similar paradigm, we only detail the window-based one, which is inspired by the idea of window-based attention (Liu et al., 2021).

Original: Specifically, given the split feature X ∈ R C 2 × H × W , it is partitioned into four windows, each with the size of 2 C × H 2 × W 2 .
Revised: Specifically, given the split feature X ∈ R C 2 × H × W , it is partitioned into four windows, each with the size of 2 C × H 2 × W 2 .

Original: To get the low-frequency part, a mean filter is applied to the resulting windows (refer to Appendix C for analyses of this option).
Revised: To get the low-frequency part, global average pooling is applied to the resulting windows (refer to Appendix C for analyses of this option).

Original: The corresponding high-frequency part can be obtained by subtracting the low-frequency map from the window feature.
Revised: The corresponding high-frequency part can be obtained by subtracting the low-frequency map from the partitioned feature.

Original: To select the useful frequency subbands, we rescale these two maps by learnable weights ( W h , W l in Fig. 1 (f)), which are directly optimized by back propagation.
Revised: To select the useful frequency subbands, we rescale these two maps by learnable weights, which are directly optimized by backpropagation.

Original: Finally, the updated frequency maps are reversed to the original resolution.
Revised: Finally, the updated frequency maps are reversed to the original resolution.

Original: The global branch has a similar operation, yet with a global receptive field.
Revised: The global branch has a similar pipeline, yet with a global receptive field.

Original: Compared to MDSF, besides the enlarged receptive field, MCSF does not accomplish frequency decoupling and modulating with the aid of convolution layers, resulting in fewer parameters and complexities (see Tab. 9 for details).
Revised: Compared to MDSF, besides the enlarged receptive field, MCSF does not accomplish frequency decoupling and modulating with the aid of convolution layers, resulting in fewer parameters and lower complexity (see Tab. 9 for details).

Original: Hence, MCSF can be embedded in multiple positions.
Revised: Hence, MCSF can be embedded in multiple positions.

Original: 3. L OSS F UNCTION
Revised: 3. L OSS F UNCTION

Original: To facilitate the frequency selection process, we adopt L 1 loss in the spatial and frequency domains:
Revised: To facilitate the frequency selection process, we adopt L 1 loss in both spatial and frequency domains:

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original:  where r= 3 means we use a degraded image plus two scaled-down ones for training; F denotes  Fourier transform; S r is the number oftotal elements for normalization; and ˆ X r , Y r arethe output and target images.
Revised: [Table] [Figure] where r denotes the index of input/output images of different resolutions; F represents fast Fourier transform; S r is the number of elements for normalization; and ˆ X r , Y r are output and target images, respectively.

Original: The  loss function is given by L =
Revised: The final loss function is given by L =

Original: L spatial + λL frequency , where λ is set as 0.1.
Revised: L spatial + λL frequency , where λ is set as 0.1.

Original: [Table]

Original: [Figure]

Original:  E XPERIMENTAL RESULTS 4. S ETTINGS
Revised:  E XPERIMENTAL RESULTS 4. S ETTINGS

Original: We evaluate the proposed SFNet on five restoration tasks: image motion/defocus deblurring, image deraining, image dehazing, and image desnowing.
Revised: We evaluate the proposed SFNet on five restoration tasks: image motion/defocus deblurring, image deraining, image dehazing, and image desnowing.

Original: More details of the used datasets and training settings for each task are provided in Appendix A. FLOPs is computed on the size of 256 × 256 .
Revised: More details of the used datasets and training settings for each task are provided in Appendix A. FLOPs are computed on patch size of 256 × 256 .

Original: We train separate models for different tasks.
Revised: We train separate models for different tasks.

Original: Unless mentioned otherwise, the following parameters are adopted.
Revised: Unless mentioned otherwise, the following parameters are adopted.

Original: The batch size is set as 4 with patch size of 256 × 256 .
Revised: The batch size is set as 4 with patch size of 256 × 256 .

Original: Each patch is randomly flipped horizontally for data augmentation.
Revised: Each patch is randomly flipped horizontally for data augmentation.

Original: The initial learning rate is 1 e − 4 and gradually reduced to 1 e −with the cosine annealing (Loshchilov & Hutter, 2016).
Revised: The initial learning rate is 1 e − 4 and gradually reduced to 1 e −with the cosine annealing (Loshchilov & Hutter, 2016).

Original: Adam ( β 1 = 0 .
Revised: Adam ( β 1 = 0 .

Original: 9 , β 2 = 0 .
Revised: 9 , β 2 = 0 .

Original: 999 ) is used for training .
Revised: 999 ) is used for training.

Original: There are 16 ResBlocks in each encoder and decoder.
Revised: N is set to 15 in Fig. 1 (c).

Original: MDSF has two branches with filter  sizes of 3 2 and 5 2 respectively, and the number of groups is 8.
Revised: MDSF has two branches with filter kernel sizes of 3 × 3 and 5 × 5 , respectively, and the number of groups is 8.

Original: We use PyTorch(Paszke et al., 2017) to implement our models on an NVIDIA Tesla V100 GPU and Intel Xeon Platinum 8255C CPU.
Revised: We use PyTorch to implement our models on an NVIDIA Tesla V100 GPU.

Original: 4. M AIN R ESULTS
Revised: 4. M AIN R ESULTS

Original: Single-image defocus deblurring results.
Revised: Single-image defocus deblurring results.

Original: Tab. 1 shows comparisons of defocus deblurring methods onthe DPDD (Abuolaim & Brown, 2020) dataset.
Revised: Tab. 1 shows comparisons of defocus deblurring methods on DPDD (Abuolaim & Brown, 2020).

Original: Our SFNet surpasses  state-of-the-art  onthe most metrics.
Revised: Our SFNet surpasses other state-of-the-art methods on most metrics.

Original: Particularly on the combined scene, SFNet obtains 0.25 dB PSNR improvement compared to the strong Transformer method Restormer (Zamir et al., 2022).
Revised: Particularly on the combined scene, SFNet obtains 0.25 dB PSNR improvement compared to the strong Transformer method Restormer (Zamir et al., 2022).

Original: In addition, our method provides a significant gain of 0.51 dB PSNR over the pre-trained network DRBNet (Ruan et al., 2022).
Revised: In addition, our method provides a significant gain of 0.51 dB PSNR over the pre-trained network DRBNet (Ruan et al., 2022).

Original: The visual results in Fig.
Revised: The visual results in Fig.

Original: 2 show that our method recovers more details than other algorithms.
Revised: 2 show that our method recovers more details than other algorithms.

Original: Due to space limitations, experimental results of  dual-pixel setting are provided in Appendix E.
Revised: Due to space limitations, experimental results of the dual-pixel setting are provided in Appendix E.

Revised: PSNRReference

Original: [Figure]
Revised: [Figure]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Original: Image dehazing results.
Revised: Image dehazing results.

Original: We perform dehazing experiments on the synthetic benchmark RESIDE (Li et al., 2018) and real hazy dataset Dense-Haze (Ancuti et al., 2019).
Revised: We perform dehazing experiments on the synthetic benchmark RESIDE (Li et al., 2018) and real hazy dataset Dense-Haze (Ancuti et al., 2019).

Original: For RESIDE, we train our models for the indoor and outdoor scenarios separately, and then evaluate on the corresponding
Revised: For RESIDE, we train our models for the indoor and outdoor scenarios separately

Original: [Table]
Revised: [Table]

Original: [Table] [Table] [Table] [Table]
Revised: and then evaluate on the corresponding

Original: SOTS-Indoor and SOTS-outdoor testsets.
Revised: SOTS-Indoor and SOTS-outdoor testsets.

Original: The quantitative results are shown in Tab. 2.
Revised: The quantitative results are shown in Tab. 2.

Original: As presented, our method obtains the highest scores on all metrics.
Revised: Our method obtains the highest scores on all metrics.

Original: Particularly on the outdoor scene, our network generates a substantial gain of 4.87 dB PSNR over DeHamer (Guo et al., 2022).
Revised: Particularly on the outdoor scene, our network generates a substantial gain of 4.87 dB PSNR over DeHamer (Guo et al., 2022).

Original: Compared to the recent works, DehazeFormer-L (Song et al., 2022) and PMNet (Ye et al., 2022), our method receives 0.19 dB and 2.83 dB higher PSNR on SOTS-Indoor testset, respectively.
Revised: Compared to the recent works, DehazeFormer-L (Song et al., 2022) and PMNet (Ye et al., 2022), our method receives 0. dB and 2.83 dB higher PSNR on SOTS-Indoor testset, respectively.

Original: Additionally, we validate the performance of our approach on the real hazy dataset Dense-Haze (Ancuti et al., 2019).
Revised: Additionally, we validate the performance of our approach on the real hazy dataset Dense-Haze (Ancuti et al., 2019).

Original: The results are shown in Tab. 4.
Revised: The results are shown in Tab. 4.

Original: As we can see, SFNet exhibits the superior ability of dealing with the real-world dehazing problem, receiving a gain of 0.84 dB over DeHamer (Guo et al., 2022).
Revised: As we can see, SFNet exhibits the superior ability of dealing with the realworld dehazing problem, receiving a gain of 0.84 dB over DeHamer (Guo et al., 2022).

Original: The dehazed results in Fig.
Revised: The dehazed results in Fig.

Original: 3 illustrate that SFNet is more effective in removing haze than other methods.
Revised: 3 illustrate that SFNet is more effective in removing haze than other methods.

Original: [Figure]

Original: Image motion deblurring results.
Revised: Image motion deblurring results.

Original: We evaluate our method on both the synthetic and real-world datasets.
Revised: We evaluate our method on both the synthetic and real-world datasets.

Original: The numerical comparisons on the synthetic GoPro (Nah et al., 2017) and HIDE (Shen et al., 2020) datasets are reported in Tab. 3.
Revised: The numerical comparisons on the synthetic GoPro (Nah et al., 2017) and HIDE (Shen et al., 2020) datasets are reported in Tab. 3.

Original: As can be seen, on the GoPro dataset SFNet shows 0. dB PSNR performance improvement over Restormer (Zamir et al., 2022) with ∼ 3× faster inference speed, shown in Tab. 8.
Revised: On GoPro, SFNet shows 0.35 dB PSNR performance improvement over Restormer (Zamir et al., 2022) with ∼ 3× faster inference speed (Tab. 8).

Original: Moreover, our method receives a performance gain of 0.19 dB compared to the recent algorithm Stripformer (Tsai et al., 2022b) with 34% fewer parameters and 26% fewer 
Revised: Moreover, our method receives a performance gain of 0.19 dB compared to the recent algorithm Stripformer (Tsai et al., 2022) with 34% fewer parameters and 26% fewer FLOPs. Notably, our SFNet

Original: FLOPs.
Revised: [Figure]

Original: Notably, our SFNet demonstrates stronger generalization capability to HIDE dataset than Stripformer on all metrics.
Revised: [Table] [Figure] demonstrates stronger generalization capability to HIDE dataset than Stripformer on all metrics.

Original: In addition to the synthetic datasets, we further evaluate the effectiveness[Table] [Figure] of our network on the real-world dataset.
Revised: In addition to the synthetic datasets, we further evaluate the effectiveness of our network on the realworld dataset.

Original: Tab. 5 shows quantitative comparisons on the newly proposed RSBlur (Jaesung Rim) dataset.
Revised: Tab. 5 shows quantitative comparisons on the newly proposed RSBlur (Jaesung Rim, 2022) dataset.

Original: SFNet sets new state-of-the-art on this dataset, providing a substantial gain of 0.37 dB PSNR over the previous best method Uformer-B (Wang et al., 2022).
Revised: SFNet sets new state-of-the-art on this dataset, providing a substantial gain of 0.37 dB PSNR over the previous best method Uformer-B (Wang et al., 2022).

Original: The qualitative comparisons on the GoPro (Nah et al., 2017) dataset are shown in Fig.
Revised: Fig.

Original: Our SFNet produces much sharper and visually pleasant result than competing algorithms.
Revised: 4 illustrates that SFNet produces more visually pleasant result than competing algorithms.

Original: Image desnowing results.
Revised: Image desnowing results.

Original: We compare our method on the CSD (Chen et al., 2021b) dataset with existing state-of-the-art methods (Chen et al., 2022a; Valanarasu et al., 2022; Chen et al., 2022c).
Revised: We compare our method on the CSD (Chen et al., 2021b) dataset with existing state-of-the-art methods (Chen et al., 2022a; Valanarasu et al., 2022; Chen et al., 2022c).

Original: As shown in Tab. 6, our framework yields a 0.15 dB PSNR improvement over theconcurrent Transformer model SnowFormer (Chen et al., 2022b).
Revised: As shown in Tab. 6, our framework yields a 4.66 dB PSNR improvement over the Transformer model MSP-Former (Chen et al., 2022c).

Original: The visual results in Fig.
Revised: The visual results in Fig.

Original: 5 show that our method is more effective in removing spatially varying snowflakes than competitors.
Revised: 5 show that our method is more effective in removing spatially varying snowflakes than competitors.

Original: More results on SRRS (Chen et al., 2020a) and Snow100K (Liu et al., 2018) are provided in Appendix E.
Revised: More results on SRRS (Chen et al., 2020a) and Snow100K (Liu et al., 2018) are provided in Appendix E.

Original: Image deraining results.
Revised: Image deraining results.

Original: Following recent works (Jiang et al., 2020; Purohit et al., 2021; Tu et al., 2022), we compare PSNR/SSIM scores on the Y channel in YCbCr color space.
Revised: Following recent works (Jiang et al., 2020; Purohit et al., 2021; Tu et al., 2022), we compare PSNR/SSIM scores on the Y channel in YCbCr color space.

Original: Tab.
Revised: Tab.

Original: 7 shows that our method achieves the best performance on the average PSNR category compared to competing approaches.
Revised: 7 shows that our method achieves the best performance on the average PSNR category compared to competing approaches.

Original: Moreover, on the Test100 dataset (Zhang et al., 2019a), the proposed SFNet obtains a performance boost of 0.30 dB PSNR over MLP model MAXIM-2S
Revised: Moreover, on the Test100 dataset (Zhang et al., 2019a), the proposed SFNet obtains a performance boost of 0.30 dB PSNR over MLP model MAXIM-2S

Original: (Tu et al., 2022).
Revised: (Tu et al., 2022).

Original: Visual results shown in Fig.
Revised: Visual results shown in Fig.

Original: 6 illustrate that our model recovers more fine details without artifacts.
Revised: 6 illustrate that our model recovers more fine details without artifacts.

Original: [Table]
Revised: [Table]

Original: Computational costs comparisons.
Revised: Computational costs comparisons.

Original: In
Revised: In

Original: Tab. 8, we evaluate the computational costs of five motion deblurring methods on the GoPro testset (Nah et al., 2017).
Revised: Tab. 8, we evaluate the computational costs of five motion deblurring methods on the GoPro testset (Nah et al., 2017).

Original: Evaluated on the full-resolution image, our method achievesthe fastest speed compared to competitors. Furthermore, SFNet achieves the comparable performance with fewer parameters.
Revised: Evaluated on the fullresolution image, our method achieves fastest speed than other state-of-the-art algorithms while achieving comparable performance with fewer parameters.

Original: [Figure]
Revised: [Figure]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original:  A BLATION S TUDIES
Revised: 4. A BLATION S TUDIES

Original: In this section
Revised: In this section, we first demonstrate the effectiveness of the proposed modules

Original: [Table]
Revised: [Table]

Original: we first show the effectiveness of the proposed modules, and then investigate the effects of different designs for each module.
Revised: and then investigate the effects of different designs for each module.

Original: Finally, we delve into the mechanism of MDCF to demonstrate its validity.
Revised: Finally, we delve into the mechanism of MDCF to demonstrate its validity.

Original: Following the recent method (Tu et al., 2022), all models are trained on the
Revised: Following the recent method (Tu et al., 2022), all models are trained on the

Original: GoPro (Nah et al., 2017) dataset for 1000 epochs [Table] and each decoder and encoder has 8 ResBlocks.
Revised: GoPro (Nah et al., 2017) dataset for 1000 epochs, and N is set to 7 in Fig.

Original: Influence of each module.
Revised: Influence of each module.

Original: Tab.
Revised: Tab.

Original: 9 shows that MDSF and MCSF yield a PSNR gain of 0.22 and 0.25 dB over the baseline model with few introduced computing burdens.
Revised: 9 shows that MDSF and MCSF yield performance gains of 0.22 and 0.25 dB over the baseline model with few introduced computing burdens.

Original: Deployed only in a single position in each en/decoder,
Revised: Deployed only in a single position in each scale,

Original: MDSF produces the similar performance with
Revised: MDSF produces the similar performance with

Original: MCSF, demonstrating the effectiveness of dynamic frequency selection mechanism.
Revised: MCSF, demonstrating the effectiveness of dynamic frequency selection mechanism.

Original: Furthermore, in Fig.
Revised: Furthermore, in Fig.

Original: 7, we plot the statistic differences between the ground truth and results of three methods on dehazing.
Revised: 7, we plot the statistic differences between the ground truth and results of three methods on dehazing.

Original: With the frequency selection mechanism, the statistics of our results are closer to that of ground truth.
Revised: With the frequency selection mechanism, the statistics of our results are closer to that of ground truth.

Original: Design choices for MCDF.
Revised: Design choices for MCDF.

Original: We study the influence of the number of MCDF in Tab. 10.
Revised: We study the influence of the number of MCDF in Tab. 10, where

Original: We increase the number of MCDF from back to front in each residual group, where 2 MCSF means  we employ  MCSF inthe last two ResBlocks in each en/decoder.
Revised: MCSF means that we employ the proposed MCSF in last two residual blocks of each ResBlock.

Original: Aswe can see from the table, the more MCSF leads to the consistently increasing performance from 31.22 to 31.45 dB PSNR.
Revised: As can be seen, using more MCSF leads to the consistently increasing performance from 31.22 to 31. dB PSNR while only introducing 0.01 M parameters and 0.04 G FLOPs.

Original: Due to its few introduced parameters and complexities, we insert MCSF in each ResBlock for frequency learning.
Revised: Due to its few introduced parameters and low complexity, we insert MCSF in each residual block for frequency learning.

Original: Design choices for MDSF.
Revised: Design choices for MDSF.

Original: Towards understanding the impact of the number of groups in MDSF, we test various configurations in Tab. 11.
Revised: To understand the impact of the number of groups in MDSF, we test various configurations in Tab. 11.

Original: Generally, the increasing number of groups leads to higher PSNR, demonstrating the effectiveness of the filter diversity.
Revised: Generally, the increasing number of groups leads to higher PSNR, demonstrating the effectiveness of the filter diversity.

Original: However, the accuracy saturates at group 16, which is  caused by overfitting probably.
Revised: However, the accuracy saturates at group 16, which is probably caused by overfitting.

Original: We finally pick 8 groups for better performance.
Revised: We finally pick 8 groups for better performance.

Original: [Table]
Revised: [Table]

Original: Alternatives for MDSF.
Revised: Alternatives for MDSF.

Original: To examine the advantage of our design, we compare our decoupler with a few alternatives in Tab.
Revised: To examine the advantage of our design, we compare our decoupler with several alternatives in Tab.

Original: We first substitute the learning-based and fixed frequency separation methods for our decoupler.
Revised: We first substitute the learning-based and fixed frequency separation methods for our decoupler.

Original: Weborrow the idea from FAN (Pang et al., 2020) to form Conv method (Tab. 12a), using stride convolution to generate different frequency parts with reduced resolution.
Revised: We form Conv method (Tab. 12a) by using strided convolution to generate different frequency parts with reduced resolution (Pang et al., 2020).

Original: The Octconv (Tab. 12b) version (Chen et al., 2020b) shares the similar idea with Conv , which utilizes down-sampling to reduce network redundancy.
Revised: The Octconv (Tab. 12b) version (Chen et al., 2020b) shares the similar idea with Conv , which utilizes down-sampling to reduce network redundancy.

Original: These variants only introduce extra low-frequency signal to the network that is harmful for image restorations.
Revised: These variants only introduce extra low-frequency signal to the network.

Original: We further utilizethe fixed separation methods to replace the proposed decoupler.
Revised: We further utilize fixed separation methods to replace the proposed decoupler.

Original: Gaussian (Tab. 12c) and Wavelet (Tab. 12d) produce the similar results, much lower than our MDSF.
Revised: Gaussian (Tab. 12c) and Wavelet (Tab. 12d) produce the similar results, much lower than our MDSF.

Original: Additionally, Wavelet needs more parameters to deal with its four branches.
Revised: Additionally, Wavelet needs more parameters to deal with its multiple branches.

Original: Since our filter kernel is generated by learning, we further compare our MDSF with two attention approaches to verify the validity of the proposed selection mechanism.
Revised: Since our filter kernel is generated by learning, we further compare our MDSF with two attention approaches to verify the validity of the proposed selection mechanism.

Original: Specifically, we utilize the widely used window-based self-attention (Wang et al., 2022)
Revised: Specifically, we utilize the widely used window-based self-attention (Wang et al., 2022)

Original: Tab. (12e) and dynamic convolution (Han et al., 2021; Wu et al., 2019)
Revised: (Tab. 12e) and dynamic convolution (Han et al., 2021; Wu et al., 2019)

Original: (Tab. 12f) to conduct comparisons.
Revised: (Tab. 12f) to conduct comparisons.

Original: As we can see from the table, our method has huge advantages over these methods, demonstrating the effectiveness of MDSF.
Revised: As we can see from the table, our method has huge advantages over these methods, demonstrating the effectiveness of MDSF.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: 4. Q UALITATIVE A NALYSES OF MDSF
Revised: 4. Q UALITATIVE A NALYSES OF MDSF

Original: We provide qualitative analyses of MDSF based on discrete Fourier transform.
Revised: We provide qualitative analyses of MDSF based on discrete Fourier transform.

Original: Results are  from thethe branch of 3 2 filter size.
Revised: Results are obtained from the branch of 3 × 3 filter.

Original: The input image is sampled fromthe deblurring dataset GoPro (Nah et al., 2017), shown in Fig.
Revised: The input image is sampled from GoPro (Nah et al., 2017), shown in Fig.

Original: The features are obtained from the last ResBlock in the last  decoder.
Revised: The features are obtained from the last residual block in the last ResBlock of decoder.

Original: We first verify the properties of the alleged low-/high-pass filters in MDSF.
Revised: We first verify the properties of the alleged low-/high-pass filters in MDSF.

Original: To this end, we iteratively apply the produced filters to the image.
Revised: To this end, we iteratively apply the produced filters to the image.

Original: The variance and corresponding spectral features ofthe intermediate images are provided in Fig.
Revised: The variance and corresponding spectral features of intermediate images are provided in Fig.

Original: Taking the low-pass filter as an example,along with the increasing of iteration times, the variance of the image decreases constantly, and the high-frequency signals in spectral features are reduced drastically.
Revised: Taking the low-pass filter as an example, with the increasing of iteration times, the variance of the image decreases constantly, and the high-frequency signals in spectral features are reduced drastically.

Original: The high-pass filter exhibits the opposite properties, demonstrating the effectiveness of our filters.
Revised: The high-pass filter exhibits the opposite properties. These results demonstrate the effectiveness of our filters.

Original: It is remarkable that the high-pass filter producesthe large variance with few iterations, hence it is more effective than the low-pass filter.
Revised: It is remarkable that the high-pass filter produces large variance with fewer iterations, hence it is more effective than the low-pass filter.

Original: As a result, it is easy for MDSF to introduce more high-frequency signals into the network for reconstruction.
Revised: As a result, it is easy for MDSF to introduce more high-frequency signals into the network for reconstruction.

Original: In MDSF, we generate different filters for each group to enhance the diversity of frequency features.
Revised: In MDSF, we generate different filters for each group to enhance the diversity of frequency features.

Original: To delve into this mechanism, we visualize the group-wise spectral features in Fig.
Revised: To delve into this mechanism, we visualize the group-wise spectral features in Fig.

Original: As expected, different groups focus on the learning of disparate low-/high-frequency signals, enriching the diversity of frequency representations for selection.
Revised: As expected, different groups focus on the learning of disparate low-/high-frequency signals, enriching the diversity of frequency representations for selection.

Original: We further compare the feature maps before and after our MDSF in Fig.
Revised: We further compare the feature maps before and after our MDSF in Fig.

Original: Using the attained filters, the decoupler of MDSF produces different frequency components.
Revised: Using the attained filters, the decoupler of MDSF produces different frequency components.

Original: The high-frequency feature contains much edge information.
Revised: The high-frequency feature contains much edge information.

Original: The resulting feature after modulator recovers more details of the number plate that is blurry in the initial feature.
Revised: The resulting feature after modulator recovers more details of the number plate that is blurry in the initial feature.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We present an image restoration framework, SFNet, which is built on frequency selection mechanism.
Revised: We present an image restoration framework, SFNet, which is built on frequency selection mechanism.

Original: We develop two key modules, MDSF and MCSF, to conduct frequency decomposition and recalibration with different receptive fields.
Revised: We develop two key modules, MDSF and MCSF, to conduct frequency decomposition and recalibration with different receptive fields.

Original: Specifically, our multi-branch dynamic selective frequency module (MDSF) builds a dynamic filter to decomposethe feature map into various frequency parts and utilizesthe modified channel attention to perform accentuation, thus effectively selecting the most informative frequency to recover.
Revised: Specifically, our multi-branch dynamic selective frequency module (MDSF) builds a dynamic filter to decompose feature maps into various frequency parts and utilizes channel attention to perform accentuation, thus effectively selecting the most informative frequency to recover.

Original: Furthermore, the proposed multi-branch compact selective frequency module (MCSF) introduces a simple yet effective manner to enlarge the receptive field and conduct frequency selection.
Revised: Furthermore, the proposed multi-branch compact selective frequency module (MCSF) introduces a simple yet effective manner to enlarge the receptive field and conduct frequency selection.

Original: With  designed modules, SFNet shows state-of-the-art results on five image restoration tasks, demonstrating the validity of our frequency selection mechanism.
Revised: With both designed modules, SFNet achieves state-of-the-art results on five image restoration tasks, demonstrating the validity of our frequency selection mechanism.

Original: [Figure]
Revised: [Figure]

Revised: A CKNOWLEDGEMENT

Revised: This work was supported by the National Natural Science Foundation of China under

Revised: Grant (62172409), Shenzhen Science and Technology Program (JCYJ20220818102012025,

Revised: JCYJ20220530145209022), and Beijing Nova Program (Z201100006820074)