Original: Enhanced Latent Space Blind Model for Real Image Denoising via Alternative Optimization
Revised: Enhanced Latent Space Blind Model for Real Image Denoising via Alternative Optimization

Original: Abstract
Revised: Abstract

Original: Motivated by the achievements in model-based methods and the advances in deep networks, we propose a novel enhanced latent space blind model based deep unfolding network, namely ScaoedNet, for complex real image denoising.
Revised: Motivated by the achievements in model-based methods and the advances in deep networks, we propose a novel enhanced latent space blind model based deep unfolding network, namely ScaoedNet, for complex real image denoising.

Original: It is derived by introducing latent space, noise information, and guidance constraint into the denoising cost function.
Revised: It is derived by introducing latent space, noise information, and guidance constraint into the denoising cost function.

Original: A self-correction alternative optimization algorithm is proposed to split the novel cost function into three alternative subproblems, i.e. , guidance representation (GR), degradation estimation (DE) and reconstruction (RE) subproblems.
Revised: A self-correction alternative optimization algorithm is proposed to split the novel cost function into three alternative subproblems, i.e. , guidance representation (GR), degradation estimation (DE) and reconstruction (RE) subproblems.

Original: Finally, we implement the optimization process by a deep unfolding network consisting of GR, DE and RE networks.
Revised: Finally, we implement the optimization process by a deep unfolding network consisting of GR, DE and RE networks.

Original: For higher performance of the DE network, a novel parameter-free noise feature adaptive enhancement (NFAE) layer is proposed.
Revised: For higher performance of the DE network, a novel parameter-free noise feature adaptive enhancement (NFAE) layer is proposed.

Original: To synchronously and dynamically realize internalexternal feature information mining in the RE network, a novel feature multimodulation attention (FM 2 A) module is proposed.
Revised: To synchronously and dynamically realize internalexternal feature information mining in the RE network, a novel feature multimodulation attention (FM 2 A) module is proposed.

Original: Our approach thereby leverages the advantages of deep learning, while also beneﬁting from the principled denoising provided by the classical model-based formulation.
Revised: Our approach thereby leverages the advantages of deep learning, while also beneﬁting from the principled denoising provided by the classical model-based formulation.

Original: To the best of our knowledge, our enhanced latent space blind model, optimization scheme, NFAE and FM 2 A have not been reported in the previous literature.
Revised: To the best of our knowledge, our enhanced latent space blind model, optimization scheme, NFAE and FM 2 A have not been reported in the previous literature.

Original: Experimental results show the promising performance of ScaoedNet on real image denoising.
Revised: Experimental results show the promising performance of ScaoedNet on real image denoising.

Original: Code is available at https://github.com/chaoren88/ScaoedNet .
Revised: Code is available at https://github.com/chaoren88/ScaoedNet .

Original:  Introduction
Revised:  Introduction

Original: Image denoising is important for various computer vision tasks, and numerous outstanding methods have been developed [13, 17, 46, 18, 36, 26, 49, 54, 27, 32, 20].
Revised: Image denoising is important for various computer vision tasks, and numerous outstanding methods have been developed [13, 17, 46, 18, 36, 26, 49, 54, 27, 32, 20].

Original: The ﬁltering-based methods are representative, such as the classic local low-pass ﬁltering methods: mean ﬁltering and median ﬁltering.
Revised: The ﬁltering-based methods are representative, such as the classic local low-pass ﬁltering methods: mean ﬁltering and median ﬁltering.

Original: In recent decades, many non-local ﬁltering methods [13, 14, 16, 25] have achieved a great success.
Revised: In recent decades, many non-local ﬁltering methods [13, 14, 16, 25] have achieved a great success.

Original: However, their results may suffer from blurring artifacts caused by their block-wise operations.
Revised: However, their results may suffer from blurring artifacts caused by their block-wise operations.

Original: The model-based methods largely depend on image priors, e.g., graph-based [34], sparsity [3, 30, 47], local smoothing [33, 45], and low-rank [17, 44, 46, 43, 55] priors.
Revised: The model-based methods largely depend on image priors, e.g., graph-based [34], sparsity [3, 30, 47], local smoothing [33, 45], and low-rank [17, 44, 46, 43, 55] priors.

Original: Pang et al .
Revised: Pang et al .

Original: [34] interpreted neighborhood graphs of pixel patches as discrete counterparts of Riemannian manifolds, providing insights into several fundamental aspects of graph Laplacian regularization for denoising.
Revised: [34] interpreted neighborhood graphs of pixel patches as discrete counterparts of Riemannian manifolds, providing insights into several fundamental aspects of graph Laplacian regularization for denoising.

Original: By introducing three weight matrices into the data and regularization terms of the sparse coding framework, Xu et al .
Revised: By introducing three weight matrices into the data and regularization terms of the sparse coding framework, Xu et al .

Original: [47] developed a denoising method to characterise the statistics of real noise and image priors.
Revised: [47] developed a denoising method to characterise the statistics of real noise and image priors.

Original: Gu et al .
Revised: Gu et al .

Original: [17] proposed an image restoration method by studying the weighted nuclear norm minimization problem.
Revised: [17] proposed an image restoration method by studying the weighted nuclear norm minimization problem.

Original: However, most of these methods focus on removing the additive white
Revised: However, most of these methods focus on removing the additive white

Original: Gaussian noise (AWGN), without taking the real noise and external guidance into consideration.
Revised: Gaussian noise (AWGN), without taking the real noise and external guidance into consideration.

Original: The learning-based methods are promising ways for addressing the image denoising problem.
Revised: The learning-based methods are promising ways for addressing the image denoising problem.

Original: Generally, it can be divided into traditional and deep network-based methods.
Revised: Generally, it can be divided into traditional and deep network-based methods.

Original: The deep network-based methods have superiority in denoising due to their excellent modeling ability.
Revised: The deep network-based methods have superiority in denoising due to their excellent modeling ability.

Original: For example, the noise map is introduced as the input for removing noise at different levels in [53].
Revised: For example, the noise map is introduced as the input for removing noise at different levels in [53].

Original: The studies [52, 39, 31] show that it is feasible to learn a single model for blind Gaussian denoising, but these blind models may be over-ﬁtted to AWGN and fail to handle real noise [19].
Revised: The studies [52, 39, 31] show that it is feasible to learn a single model for blind Gaussian denoising, but these blind models may be over-ﬁtted to AWGN and fail to handle real noise [19].

Original: Different from the previous empirically designed networks, the deep unfolding-based methods [11, 15, 24, 21] were proposed by implementing traditional methods via deep networks.
Revised: Different from the previous empirically designed networks, the deep unfolding-based methods [11, 15, 24, 21] were proposed by implementing traditional methods via deep networks.

Original: For instance, an advanced image denoising network was developed by solving a fractional optimal control problem in [21].
Revised: For instance, an advanced image denoising network was developed by solving a fractional optimal control problem in [21].

Original: The above methods generally aim at removing AWGN with regular distributions.
Revised: The above methods generally aim at removing AWGN with regular distributions.

Original: However, real image denoising is a more challenging task since the distribution of real noise is more complex.
Revised: However, real image denoising is a more challenging task since the distribution of real noise is more complex.

Original: For real image denoising, Guo et al .
Revised: For real image denoising, Guo et al .

Original: [19] adopted Poisson-Gaussian noise model and presented a blind denoising network.
Revised: [19] adopted Poisson-Gaussian noise model and presented a blind denoising network.

Original: A spatial-adaptive denoising network was proposed in [10] for efﬁcient blind noise removal.
Revised: A spatial-adaptive denoising network was proposed in [10] for efﬁcient blind noise removal.

Original: Byun et al .
Revised: Byun et al .

Original: [7] proposed a fast noise estimation network and an efﬁcient blind-spot network for Poisson-Gaussian noise removal.
Revised: [7] proposed a fast noise estimation network and an efﬁcient blind-spot network for Poisson-Gaussian noise removal.

Original: Anwar et al .
Revised: Anwar et al .

Original: [5] incorporated feature attention into denoising and developed a one-stage real image denoising network.
Revised: [5] incorporated feature attention into denoising and developed a one-stage real image denoising network.

Original: Cheng et al .
Revised: Cheng et al .

Original: [12] improved the image denoising performance by plugging the subspace attention module.
Revised: [12] improved the image denoising performance by plugging the subspace attention module.

Original: Liu et al .
Revised: Liu et al .

Original: [28] presented an invertible denoising network for real image denoising.
Revised: [28] presented an invertible denoising network for real image denoising.

Original: Based on the adaptive consistency prior,
Revised: Based on the adaptive consistency prior,

Original: Ren et al .
Revised: Ren et al .

Original: [36] proposed a novel model-based denoising method to inform the design of the network for both synthetic and real denoising.
Revised: [36] proposed a novel model-based denoising method to inform the design of the network for both synthetic and real denoising.

Original: Yue et al .
Revised: Yue et al .

Original: [49] proposed a uniﬁed framework, namely dual adversarial network, to simultaneously deal with the real noise removal and noise generation tasks.
Revised: [49] proposed a uniﬁed framework, namely dual adversarial network, to simultaneously deal with the real noise removal and noise generation tasks.

Original: In this paper, motivated by the advances in deep networks and relying on the rich body of the model-based methods, we propose a novel enhanced latent space (LS) blind model based deep unfolding network for real image denoising.
Revised: In this paper, motivated by the advances in deep networks and relying on the rich body of the model-based methods, we propose a novel enhanced latent space (LS) blind model based deep unfolding network for real image denoising.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 1, the whole architecture of ScaoedNet achieves the best promising visual performance when compared to other denoising methods.
Revised: 1, the whole architecture of ScaoedNet achieves the best promising visual performance when compared to other denoising methods.

Original: The main contributions of our method are as follows:
Revised: The main contributions of our method are as follows:

Original: • For the theoretical novelty, a novel enhanced model-based denoising cost function is proposed based on LS, noise information, and guidance constraint (GC).
Revised: • For the theoretical novelty, a novel enhanced model-based denoising cost function is proposed based on LS, noise information, and guidance constraint (GC).

Original: Then, a self-correction (SC) based alternative optimization algorithm is proposed to split the cost function into three alternative subproblems ( i
Revised: Then, a self-correction (SC) based alternative optimization algorithm is proposed to split the cost function into three alternative subproblems ( i

Original: e. , guidance representation (GR), degradation estimation (DE) and reconstruction (RE) subproblems).
Revised: e. , guidance representation (GR), degradation estimation (DE) and reconstruction (RE) subproblems).

Original: • To merge the power of the model-based framework with the recent advances in deep learning for real image denoising, a novel interpretable network (ScaoedNet) is designed via implementing the optimization process by a deep network, which consists of the GR, DE, and RE networks.
Revised: • To merge the power of the model-based framework with the recent advances in deep learning for real image denoising, a novel interpretable network (ScaoedNet) is designed via implementing the optimization process by a deep network, which consists of the GR, DE, and RE networks.

Original: • An effective DE network with a novel noise feature adaptive enhancement (NFAE) layer is proposed.
Revised: • An effective DE network with a novel noise feature adaptive enhancement (NFAE) layer is proposed.

Original: NFAE does not add extra parameters to the DE network, and leads to better noise estimation results.
Revised: NFAE does not add extra parameters to the DE network, and leads to better noise estimation results.

Original: • To enable internal-external feature mining, a novel feature multi-modulation attention residual block (FM 2 ARB) is proposed in the RE network, which is mainly based on feature self-modulation (FSM) and degradation external-modulation (DEM).
Revised: • To enable internal-external feature mining, a novel feature multi-modulation attention residual block (FM 2 ARB) is proposed in the RE network, which is mainly based on feature self-modulation (FSM) and degradation external-modulation (DEM).

Original: FM 2 ARB can dynamically fuse the encoded degradation representation and the image feature representation, leading to higher performance.
Revised: FM 2 ARB can dynamically fuse the encoded degradation representation and the image feature representation, leading to higher performance.

Original:  Proposed Model-based Denoising Method 2.1 Enhanced Latent Space Blind Model for Denoising
Revised:  Proposed Model-based Denoising Method 2.1 Enhanced Latent Space Blind Model for Denoising

Original: Let y ∈ R n · c be a noisy image with c channels and n pixels, x ∈ R n · c be its clean version, and ϕ ( · ) be the regularizer weighted by µ .
Revised: Let y ∈ R n · c be a noisy image with c channels and n pixels, x ∈ R n · c be its clean version, and ϕ ( · ) be the regularizer weighted by µ .

Original: The traditional model-based denoising method can be given by
Revised: The traditional model-based denoising method can be given by

Original: [Equation]
Revised: [Equation]

Original: (1) where H ( x , y ) is the data ﬁdelity which is usually set to (cid:107) y − x (cid:107) 22 .
Revised: (1) where H ( x , y ) is the data ﬁdelity which is usually set to (cid:107) y − x (cid:107) 22 .

Original: ϕ ( · ) can be any image regularizers, e.g. sparsity prior [3, 30, 47], local smoothing prior [33, 45], and low-rank
Revised: ϕ ( · ) can be any image regularizers, e.g. sparsity prior [3, 30, 47], local smoothing prior [33, 45], and low-rank

Original: prior [17, 44, 46, 43], etc.
Revised: prior [17, 44, 46, 43], etc.

Original: Analysis and Enhancement.
Revised: Analysis and Enhancement.

Original: We perform an in-depth analysis of Eq.
Revised: We perform an in-depth analysis of Eq.

Original: (1) in the following: 1) H ( x , y ) in Eq.
Revised: (1) in the following: 1) H ( x , y ) in Eq.

Original: (1) only utilizes the squared error distance (cid:107) y − x (cid:107) 22 between y and x , which is mainly derived from AWGN.
Revised: (1) only utilizes the squared error distance (cid:107) y − x (cid:107) 22 between y and x , which is mainly derived from AWGN.

Original: Inspired by the latest unfolding works [36, 6], we parameterize the metric (cid:107) y − x (cid:107) 22 by a pre-speciﬁed high-dimensional LS encoding function E : R n · c → R n · m ( m > c ), to obtain high-dimensional image embeddings z
Revised: Inspired by the latest unfolding works [36, 6], we parameterize the metric (cid:107) y − x (cid:107) 22 by a pre-speciﬁed high-dimensional LS encoding function E : R n · c → R n · m ( m > c ), to obtain high-dimensional image embeddings z

Original: = E ( x ) for better performance, instead of limiting the objective to the squared error in low-dimensional image space.
Revised: = E ( x ) for better performance, instead of limiting the objective to the squared error in low-dimensional image space.

Original: It allows us to directly model the image formation process in LS [36, 6], and to integrate LS priors into the prediction.
Revised: It allows us to directly model the image formation process in LS [36, 6], and to integrate LS priors into the prediction.

Original: 2) In real image denoising, noise is complex and unknown, and thus a blind model is needed.
Revised: 2) In real image denoising, noise is complex and unknown, and thus a blind model is needed.

Original: Since the noise information is largely ignored in Eq.
Revised: Since the noise information is largely ignored in Eq.

Original: (1), it is difﬁcult to well address the real noise removal problem.
Revised: (1), it is difﬁcult to well address the real noise removal problem.

Original: By introducing the real noise information and LS, we can learn a more general distance measure H ( z , u , y )
Revised: By introducing the real noise information and LS, we can learn a more general distance measure H ( z , u , y )

Original: ( u is the unknown real noise map), and further replace the regularizer by τφ ( z ) + ηψ ( u ) (regularizers for z and u weighted by τ and η respectively, where τ, η > 0 ).
Revised: ( u is the unknown real noise map), and further replace the regularizer by τφ ( z ) + ηψ ( u ) (regularizers for z and u weighted by τ and η respectively, where τ, η > 0 ).

Original: Compared with the regularizer based on general image statistical characteristics, better results can be obtained if certain speciﬁc information of the given image is exploited to guide the denoising process.
Revised: Compared with the regularizer based on general image statistical characteristics, better results can be obtained if certain speciﬁc information of the given image is exploited to guide the denoising process.

Original: Let g be certain sophisticated guidance information.
Revised: Let g be certain sophisticated guidance information.

Original: Then, we can improve φ ( z ) by a speciﬁc constraint G ( z , g ) for further performance enhancement, and we call it the GC scheme.
Revised: Then, we can improve φ ( z ) by a speciﬁc constraint G ( z , g ) for further performance enhancement, and we call it the GC scheme.

Original: Proposed Model-based Denoising Method.
Revised: Proposed Model-based Denoising Method.

Original: Before the introduction of our denoising framework, we ﬁrst introduce the real noise model.
Revised: Before the introduction of our denoising framework, we ﬁrst introduce the real noise model.

Original: Different from AWGN, real image noise is generally complex and signal-dependent.
Revised: Different from AWGN, real image noise is generally complex and signal-dependent.

Original: Let x r = p ( x ) ∈ R n be the irradiance image of RAW pixels, and p be the mapping function from RGB to RAW.
Revised: Let x r = p ( x ) ∈ R n be the irradiance image of RAW pixels, and p be the mapping function from RGB to RAW.

Original: According to [19], the following heteroscedastic Gaussian is a reasonable approximation for real noise distribution: n ( x r ) =
Revised: According to [19], the following heteroscedastic Gaussian is a reasonable approximation for real noise distribution: n ( x r ) =

Original: n s ( x r ) + n c ∼ N (0 , u 2 ( x r )) .
Revised: n s ( x r ) + n c ∼ N (0 , u 2 ( x r )) .

Original: Speciﬁcally, n c is a stationary noise component with variance u 2 c , and n s is a signal-dependent noise component with space-variant variance x r u 2 s .
Revised: Speciﬁcally, n c is a stationary noise component with variance u 2 c , and n s is a signal-dependent noise component with space-variant variance x r u 2 s .

Original: Then, y = q ( x r + n ( x r )) , where q is the mapping function from RAW to RGB.
Revised: Then, y = q ( x r + n ( x r )) , where q is the mapping function from RAW to RGB.

Original: Inspired by [19], the real image noise variance can be formulated by
Revised: Inspired by [19], the real image noise variance can be formulated by

Original: [Equation]
Revised: [Equation]

Original: where u s and u c are uniformly sampled from the ranges of [0, 0.16] and [0, 0.06].
Revised: where u s and u c are uniformly sampled from the ranges of [0, 0.16] and [0, 0.06].

Original: For more details, please refer to [19].
Revised: For more details, please refer to [19].

Original: Then, the noise map is deﬁned as u = (cid:112) u 2 ( x r ) ∈ R n , which is signal-dependent and space-variant with the same size of x r instead of a scalar.
Revised: Then, the noise map is deﬁned as u = (cid:112) u 2 ( x r ) ∈ R n , which is signal-dependent and space-variant with the same size of x r instead of a scalar.

Original: Next, according to the ‘Analysis and
Revised: Next, according to the ‘Analysis and

Original: Enhancement’ subsection, the proposed enhanced denoising method can be given as
Revised: Enhancement’ subsection, the proposed enhanced denoising method can be given as

Original: [Equation]
Revised: [Equation]

Original: where D : R n · m → R n · c is the LS decoding function that maps the latent embeddings ˆ z to ˆ x .
Revised: where D : R n · m → R n · c is the LS decoding function that maps the latent embeddings ˆ z to ˆ x .

Original: 2.2 Self-Correction (SC) Alternative Optimization
Revised: 2.2 Self-Correction (SC) Alternative Optimization

Original: According to the widely used traditional alternative optimization algorithm [8, 9, 38, 29] in image restoration, Eq.
Revised: According to the widely used traditional alternative optimization algorithm [8, 9, 38, 29] in image restoration, Eq.

Original: (3) can be solved by splitting it into two alternative subproblems with respect to z (reconstruction (RE) subproblem) and u (degradation estimation (DE) subproblem):
Revised: (3) can be solved by splitting it into two alternative subproblems with respect to z (reconstruction (RE) subproblem) and u (degradation estimation (DE) subproblem):

Original: [Equation]
Revised: [Equation]

Original: where i is the iteration number.
Revised: where i is the iteration number.

Original: However, the last estimates of z and u are not used in Eq.
Revised: However, the last estimates of z and u are not used in Eq.

Original: For higher performance, it is better to use the last estimates for SC when estimating z and u .
Revised: For higher performance, it is better to use the last estimates for SC when estimating z and u .

Original: In addition, since the estimated noise map is available during the iterations, it can be reasonable information to guide the reconstruction ( i.e. , g can be obtained according to u ( i ) ).
Revised: In addition, since the estimated noise map is available during the iterations, it can be reasonable information to guide the reconstruction ( i.e. , g can be obtained according to u ( i ) ).

Original: Thus, instead of using Eq.
Revised: Thus, instead of using Eq.

Original: (4), we use the SC scheme (introducing u ( i ) and z ( i ) ) and specify the form of g , and Eq.
Revised: (4), we use the SC scheme (introducing u ( i ) and z ( i ) ) and specify the form of g , and Eq.

Original: (3) becomes:
Revised: (3) becomes:

Original: [Equation]
Revised: [Equation]

Original: where G ( · ) is the guidance information generator, (cid:101) G ( · ) becomes the joint constraint of GC and SC for z , and (cid:101) ψ ( · ) becomes the joint constraint of noise information and SC for u .
Revised: where G ( · ) is the guidance information generator, (cid:101) G ( · ) becomes the joint constraint of GC and SC for z , and (cid:101) ψ ( · ) becomes the joint constraint of noise information and SC for u .

Original: The two functions
Revised: The two functions

Original: Z ( · ) and U ( · ) are the optimization solving processes for the z and u subproblems in Eq.
Revised: Z ( · ) and U ( · ) are the optimization solving processes for the z and u subproblems in Eq.

Original: Finally, x can be obtained by applying the LS decoding function D to z .
Revised: Finally, x can be obtained by applying the LS decoding function D to z .

Original: Note that u (0) can be directly estimated by any reasonable DE estimator U ini ( i.e. , u (0)
Revised: Note that u (0) can be directly estimated by any reasonable DE estimator U ini ( i.e. , u (0)

Original: =
Revised: =

Original: U ini ( y ) ), and z (0) can be simply set to E ( y ) .
Revised: U ini ( y ) ), and z (0) can be simply set to E ( y ) .

Original: The overall algorithm is called SC alternative optimization based enhanced denoising (SCAOED), and is described in Algorithm 1 of the ‘Supplementary Material’.
Revised: The overall algorithm is called SC alternative optimization based enhanced denoising (SCAOED), and is described in Algorithm 1 of the ‘Supplementary Material’.

Original:  SCAOED Driven Denoising Network 3.1 Implicitly Implement SCAOED Algorithm via Deep Network
Revised:  SCAOED Driven Denoising Network 3.1 Implicitly Implement SCAOED Algorithm via Deep Network

Original: It is very challenging to manually design the optimal operators in SCAOED, e.g. E, D, G, U ini , U, Z .
Revised: It is very challenging to manually design the optimal operators in SCAOED, e.g. E, D, G, U ini , U, Z .

Original: Therefore, the deep unfolding scheme is applied to implicitly implement these operators via deep network, leading to ScaoedNet.
Revised: Therefore, the deep unfolding scheme is applied to implicitly implement these operators via deep network, leading to ScaoedNet.

Original: Speciﬁcally, we parameterize these operators by introducing learnable parameters
Revised: Speciﬁcally, we parameterize these operators by introducing learnable parameters

Original: Θ = { Θ E , Θ D , Θ G , Θ U ini , Θ U , Θ Z } , which are learned in a discriminative manner.
Revised: Θ = { Θ E , Θ D , Θ G , Θ U ini , Θ U , Θ Z } , which are learned in a discriminative manner.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 2, ScaoedNet is an iterative real denoising framework in LS.
Revised: 2, ScaoedNet is an iterative real denoising framework in LS.

Original: Overall, LS encoding module, LS decoding module, G -module, initial DE network ( u ini -Net), DE network ( u -Net), and RE network ( z -Net) are used to implement E , D , G , U ini , U , and Z , respectively .
Revised: Overall, LS encoding module, LS decoding module, G -module, initial DE network ( u ini -Net), DE network ( u -Net), and RE network ( z -Net) are used to implement E , D , G , U ini , U , and Z , respectively .

Original: Speciﬁcally, in order to reduce the computational complexity while increasing the receptive ﬁeld, the
Revised: Speciﬁcally, in order to reduce the computational complexity while increasing the receptive ﬁeld, the

Original: LS encoding module (the learned version of E ) is implemented by a pixel-shufﬂe layer with factor 1/2 (PS × 1 / 2 ) followed by a 3 × 3 Conv layer, which can project the original low-dimensional image space input y ∈ R c
Revised: LS encoding module (the learned version of E ) is implemented by a pixel-shufﬂe layer with factor 1/2 (PS × 1 / 2 ) followed by a 3 × 3 Conv layer, which can project the original low-dimensional image space input y ∈ R c

Original: × H × W
Revised: × H × W

Original: ( H · W = n ) to the high-dimensional embedding space by E ( .
Revised: ( H · W = n ) to the high-dimensional embedding space by E ( .

Original: | Θ E ) ∈
Revised: | Θ E ) ∈

Original: R m × H/ 2 × W/ 2 ( m is set to 64) as initial latent image embeddings z (0) , i.e. z (0)
Revised: R m × H/ 2 × W/ 2 ( m is set to 64) as initial latent image embeddings z (0) , i.e. z (0)

Original: =
Revised: =

Original: E ( y | Θ E ) .
Revised: E ( y | Θ E ) .

Original: In the meanwhile, u (0) ∈ R 1 × H × W is estimated by u ini -Net (the learned version of U ini ) to implicitly implement U ini ( y ) .
Revised: In the meanwhile, u (0) ∈ R 1 × H × W is estimated by u ini -Net (the learned version of U ini ) to implicitly implement U ini ( y ) .

Original: Then, GR is obtained by inputting u (0) into the G -module (the learned version of
Revised: Then, GR is obtained by inputting u (0) into the G -module (the learned version of

Original: G ).
Revised: G ).

Original: Next, y , g (0) , z (0) , and u (0) are fed into z -Net
Revised: Next, y , g (0) , z (0) , and u (0) are fed into z -Net

Original: (the learned version of Z ) to get z (1) .
Revised: (the learned version of Z ) to get z (1) .

Original: After that, y , u (0) , and z (1) are fed into the next u -Net
Revised: After that, y , u (0) , and z (1) are fed into the next u -Net

Original: (the learned version of U ) to obtain u (1) .
Revised: (the learned version of U ) to obtain u (1) .

Original: Then, g , u and z are alternatively updated by the following G -modules, u -Nets, and z -Nets, whcih implicitly implement Eq.
Revised: Then, g , u and z are alternatively updated by the following G -modules, u -Nets, and z -Nets, whcih implicitly implement Eq.

Original: Finally, z ( K ) is reconstructed by the LS decoding module (the learned version of D ) to obtain the ﬁnal image space estimate for x , i.e. x = D
Revised: Finally, z ( K ) is reconstructed by the LS decoding module (the learned version of D ) to obtain the ﬁnal image space estimate for x , i.e. x = D

Original: ( z ( K ) | Θ D ) , where the LS decoding module is implemented by ‘Conv-PS × 2 ’ layers.
Revised: ( z ( K ) | Θ D ) , where the LS decoding module is implemented by ‘Conv-PS × 2 ’ layers.

Original: Note that the parameters of all u -Nets, G -modules, and z -Nets are respectively shared.
Revised: Note that the parameters of all u -Nets, G -modules, and z -Nets are respectively shared.

Original: For the reconstruction loss, we adopt the L 1 loss between the output and the ground-truth image.
Revised: For the reconstruction loss, we adopt the L 1 loss between the output and the ground-truth image.

Original: In addition, we adopt structural similarity (SSIM) loss [51] L S to pay attention to image structures.
Revised: In addition, we adopt structural similarity (SSIM) loss [51] L S to pay attention to image structures.

Original: For the noise map estimation loss, we adopt the L 1 loss between the estimated noise map and the ground-truth version.
Revised: For the noise map estimation loss, we adopt the L 1 loss between the estimated noise map and the ground-truth version.

Original: Note that the initial DE network is pre-trained only with the noise map estimation loss.
Revised: Note that the initial DE network is pre-trained only with the noise map estimation loss.

Original: Let { x g , y g } Ng
Revised: Let { x g , y g } Ng

Original: =1 be N clean-noisy training pairs, ⊗ be the element-wise product, and η, γ be two positive constants.
Revised: =1 be N clean-noisy training pairs, ⊗ be the element-wise product, and η, γ be two positive constants.

Original: The total loss function can be given by
Revised: The total loss function can be given by

Original: [Equation]
Revised: [Equation]

Revised: where ˆ u ( i ) g is the estimated noise map of the i th DE network for y g , and u g is the ground-truth version.

Original: where κ i,g = ν i · α g .
Revised: κ i,g = ν i · α g .

Original: ν i is the weight for the i th DE network ( ν i
Revised: ν i is the weight for the i th DE network ( ν i

Original: -s compose the geometric sequence with a common ratio ι and sum 1, i.e. , ν i =
Revised: -s compose the geometric sequence with a common ratio ι and sum 1, i.e. , ν i =

Original: ( ι − 1) ι i − 1 / ( ι K − 1 − 1) ).
Revised: ( ι − 1) ι i − 1 / ( ι K − 1 − 1) ).

Original: α g is the g th element of the indicator vector α for the noise constraint.
Revised: α g is the g th element of the indicator vector α for the noise constraint.

Original: For a synthetic training pair with known noise map, α g = 1 .
Revised: For a synthetic training pair with known noise map, α g = 1 .

Original: For a real training pair without known noise map, α g = 0 .
Revised: For a real training pair without known noise map, α g = 0 .

Original: 3.2 Initial Degradation Estimation (DE) Network ( u ini -Net) and DE Network ( u -Net)
Revised: 3.2 Initial Degradation Estimation (DE) Network ( u ini -Net) and DE Network ( u -Net)

Original: Basic Architecture of u ini -Net and u -Net.
Revised: Basic Architecture of u ini -Net and u -Net.

Original: In u ini -Net, the input y is ﬁrst processed by a noise feature enhancement module Q consisting of a PS × 1 / 2 layer, four ‘Conv-ReLU’ layers, and an
Revised: In u ini -Net, the input y is ﬁrst processed by a noise feature enhancement module Q consisting of a PS × 1 / 2 layer, four ‘Conv-ReLU’ layers, and an

Original: NFAE layer, to get the enhanced noise feature map.
Revised: NFAE layer, to get the enhanced noise feature map.

Original: Next, the feature map is projected to the noise map by a Conv layer and a PS × 2 layer.
Revised: Next, the feature map is projected to the noise map by a Conv layer and a PS × 2 layer.

Original: The procedure of u ini -Net can be written as:
Revised: The procedure of u ini -Net can be written as:

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: where U  ( .
Revised: where U ini ( .

Original: | Θ U ini ) is the initial DE estimator parameterized by Θ U ini .
Revised: | Θ U ini ) is the initial DE estimator parameterized by Θ U ini .

Original: The u -Net implicitly implements DE subproblem via a deep network similar to u ini -Net.
Revised: The u -Net implicitly implements DE subproblem via a deep network similar to u ini -Net.

Original: u ( i ) is ﬁrst downsampled by PS × 1 / 2 followed by a Conv layer to obtain the feature map u ( i ) p .
Revised: u ( i ) is ﬁrst downsampled by PS × 1 / 2 followed by a Conv layer to obtain the feature map u ( i ) p .

Original: In addition, y is input into the feature extraction module Q with shared parameters of Q in u ini -Net to implicitly enhance the non-linear ﬁtting ability of u -Net.
Revised: In addition, y is input into the feature extraction module Q with shared parameters of Q in u ini -Net to implicitly enhance the non-linear ﬁtting ability of u -Net.

Original: Then, Q ( y ) is processed by a Conv layer, and further fused with u ( i ) p and z ( i +1) by a Concat layer and a 1 × 1 Conv layer to obtain the fused feature map.
Revised: Then, Q ( y ) is processed by a Conv layer, and further fused with u ( i ) p and z ( i +1) by a Concat layer and a 1 × 1 Conv layer to obtain the fused feature map.

Original: Next, four ‘Conv-ReLU’ layers followed by an NFAE layer are used.
Revised: Next, four ‘Conv-ReLU’ layers followed by an NFAE layer are used.

Original: After that, the enhanced feature map is projected to the noise residual map by a Conv layer and a PS × 2 layer.
Revised: After that, the enhanced feature map is projected to the noise residual map by a Conv layer and a PS × 2 layer.

Original: Finally, a skip connection from u ( i ) to the noise residual map is added.
Revised: Finally, a skip connection from u ( i ) to the noise residual map is added.

Original: The procedure of u -Net can be written as:
Revised: The procedure of u -Net can be written as:

Original: [Equation]
Revised: [Equation]

Original: where U ( .
Revised: where U ( .

Original: | Θ U ) is the optimization solving process of the u subproblem parameterized by Θ U .
Revised: | Θ U ) is the optimization solving process of the u subproblem parameterized by Θ U .

Original: Noise Feature Adaptive Enhancement (NFAE) Layer.
Revised: Noise Feature Adaptive Enhancement (NFAE) Layer.

Original: The feature of real image noise should have the following properties: a salient voxel with strong degradation should be much different from the voxels around it.
Revised: The feature of real image noise should have the following properties: a salient voxel with strong degradation should be much different from the voxels around it.

Original: The difference can be easily measured by the variance, and a salient voxel with a larger variance in and between channels of E ( i +1) in ( E ( i +1) in is the input of NFAE) should have a larger weight than a non-salient voxel.
Revised: The difference can be easily measured by the variance, and a salient voxel with a larger variance in and between channels of E ( i +1) in ( E ( i +1) in is the input of NFAE) should have a larger weight than a non-salient voxel.

Original: Let v thw be the variance for the hw th voxel within the t th channel of
Revised: Let v thw be the variance for the hw th voxel within the t th channel of

Original: E ( i +1) in , Z hw be the mean of v thw -s at the location hw , and Z t be the mean of v thw -s within the t th channel.
Revised: E ( i +1) in , Z hw be the mean of v thw -s at the location hw , and Z t be the mean of v thw -s within the t th channel.

Original: We can calculate the weight without adding parameters to u ini -Net and u -Net by
Revised: We can calculate the weight without adding parameters to u ini -Net and u -Net by

Original: [Equation]
Revised: [Equation]

Original: t + δ )) , (9) where S ( · , · ) is the product of the Sigmoid results of its two inputs, i.e. , S ( a, b ) =
Revised: where S ( · , · ) is the product of the Sigmoid results of its two inputs, i.e. , S ( a, b ) =

Original: Sigmoid ( a ) ·
Revised: Sigmoid ( a ) ·

Original: Sigmoid ( b ) = 1 /
Revised: Sigmoid ( b ) = 1 /

Original: [(1 + e − a ) · (1 + e − b )] , where e is the Euler’s number.
Revised: [(1 + e − a ) · (1 + e − b )] , where e is the Euler’s number.

Original: δ ( 10 − 16 ) is a small positive factor for numeral stability.
Revised: δ ( 10 − 16 ) is a small positive factor for numeral stability.

Original: Thus, the output of NFAE can be obtained by v ⊗ E ( i +1) in .
Revised: Thus, the output of NFAE can be obtained by v ⊗ E ( i +1) in .

Original: 3.3 Guidance Representation Module ( G -Module) and Reconstruction Network ( z -Net)
Revised: 3.3 Guidance Representation Module ( G -Module) and Reconstruction Network ( z -Net)

Original: Since the estimated noise map is available during the iterations, it can be reasonable information to guide the reconstruction.
Revised: Since the estimated noise map is available during the iterations, it can be reasonable information to guide the reconstruction.

Original: Therefore, g ( i ) in Eq.
Revised: Therefore, g ( i ) in Eq.

Original: (5) is simply implemented by the encoded degradation representation obtained by the G -Module in Fig.
Revised: (5) is simply implemented by the encoded degradation representation obtained by the G -Module in Fig.

Original: The procedure of G -Module can be written as:
Revised: The procedure of G -Module can be written as:

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: where G ( .
Revised: where G ( .

Original: | Θ G ) is the guidance information generator parameterized by Θ G .
Revised: | Θ G ) is the guidance information generator parameterized by Θ G .

Original: z -Net is designed to restore the latent z , which implicitly implement the u subproblem via a deep network.
Revised: z -Net is designed to restore the latent z , which implicitly implement the z subproblem via a deep network.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 2, with the degradation representation g ( i ) , we further propose the
Revised: 2, with the degradation representation g ( i ) , we further propose the

Original: FM 2 A module shown in Fig.
Revised: FM 2 A module shown in Fig.

Original: 3 to adjust the feature map according to the internal and external information.
Revised: 3 to adjust the feature map according to the internal and external information.

Original: In detail, we adopt the FSM module to capture spatial and channel information of image features themselves, and the DEM module to dynamically recalibrate the image features according to the guidance information.
Revised: In detail, we adopt the FSM module to capture spatial and channel information of image features themselves, and the DEM module to dynamically recalibrate the image features according to the guidance information.

Original: By applying FM 2 A to a residual unit, FM 2 ARB is proposed to comprehensively modulate the residual features.
Revised: By applying FM 2 A to a residual unit, FM 2 ARB is proposed to comprehensively modulate the residual features.

Original: After that, an skip connection from the Conv layer after z ( i ) to the recalibrated feature by DEM is added.
Revised: After that, an skip connection from the Conv layer after z ( i ) to the recalibrated feature by DEM is added.

Original: Finally, the feature map is adjusted by a
Revised: Finally, the feature map is adjusted by a

Original: Conv layer, and then added with E ( y ) .
Revised: Conv layer, and then added with E ( y ) .

Original: The procedure of u -Net can be written as:
Revised: The procedure of u -Net can be written as:

Original: [Equation]
Revised: [Equation]

Original: where Z ( . | Θ Z ) is the optimization solving process of the z subproblem parameterized by Θ Z .
Revised: where Z ( . | Θ Z ) is the optimization solving process of the z subproblem parameterized by Θ Z .

Original: 3.4 Feature Multiple-Modulation Attention
Revised: 3.4 Feature Multiple-Modulation Attention

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 3, a novel FM 2 A module consisting of FSM and DEM modules is proposed.
Revised: 3, a novel FM 2 A module consisting of FSM and DEM modules is proposed.

Original: Feature Self-Modulation (FSM).
Revised: Feature Self-Modulation (FSM).

Original: Attention mechanism is very important in image restoration tasks.
Revised: Attention mechanism is very important in image restoration tasks.

Original: Although self-attention mechanism [42] has been widely used recently, because of its high parameter number and complexity, the spatial/channel attention mechanism is ﬁnally adopted in our work for a balance between performance and complexity.
Revised: Although self-attention mechanism [42] has been widely used recently, because of its high parameter number and complexity, the spatial/channel attention mechanism is ﬁnally adopted in our work for a balance between performance and complexity.

Original: Different from the traditional spatial attention (SA) and channel attention (CA) mechanisms [50], we introduce complementary feature information into the proposed FSM module to enhance the recalibration ability in and between channels of the feature map.
Revised: Different from the traditional spatial attention (SA) and channel attention (CA) mechanisms [50], we introduce complementary feature information into the proposed FSM module to enhance the recalibration ability in and between channels of the feature map.

Original: It consists of complementary information guided (CIG) SA ( i.e. , CIG-SA) and CIG-CA.
Revised: It consists of complementary information guided (CIG) SA ( i.e. , CIG-SA) and CIG-CA.

Original: For
Revised: For

Original: CIG-SA, CIG spatial feature is ﬁrst achieved by channel-wise modulation, and then used to calculate the spatial weights.
Revised: CIG-SA, CIG spatial feature is ﬁrst achieved by channel-wise modulation, and then used to calculate the spatial weights.

Original: For CIG-CA, CIG channel feature is ﬁrst achieved by space-wise modulation, and then used to calculate the channel weights.
Revised: For CIG-CA, CIG channel feature is ﬁrst achieved by space-wise modulation, and then used to calculate the channel weights.

Original: After both the spatial and channel weights are obtained, they are respectively used to recalibrate the input feature map, and then their results are further fused.
Revised: After both the spatial and channel weights are obtained, they are respectively used to recalibrate the input feature map, and then their results are further fused.

Original: Note that by removing the CIG scheme in FSM, it becomes a spatial-channel attention (SCA) based on traditional SA and CA.
Revised: Note that by removing the CIG scheme in FSM, it becomes a spatial-channel attention (SCA) based on traditional SA and CA.

Original: For more details about CIG-SA and CIG-CA, please refer to the ‘Supplementary Material’.
Revised: For more details about CIG-SA and CIG-CA, please refer to the ‘Supplementary Material’.

Original: Degradation External-Modulation (DEM).
Revised: Degradation External-Modulation (DEM).

Original: Because of the complexity of real degradation, it is a great challenge to estimate the clean image directly by the usual deep network.
Revised: Because of the complexity of real degradation, it is a great challenge to estimate the clean image directly by the usual deep network.

Original: As analyzed in section 2, it is promising to exploit the guidance information (degradation representation)
Revised: As analyzed in section 2, it is promising to exploit the guidance information (degradation representation)

Original: g to guide the reconstruction.
Revised: g to guide the reconstruction.

Original: Therefore, to exploit g in the implementation of GC, the feature map processed by the FSM module is further recalibrated by the tensor generated by the DEM weight tensor generator.
Revised: Therefore, to exploit g in the implementation of GC, the feature map processed by the FSM module is further recalibrated by the tensor generated by the DEM weight tensor generator.

Original: By applying both FSM and DEM, the FM 2 A module can synchronously realize external GC and the internal important information mining. It can be written as follows:
Revised: For the input feature map a , by applying both FSM and DEM, the FM 2 A module can synchronously realize external GC and the internal important information mining to get the output feature map b :

Original: [Equation]
Revised: [Equation]

Original: (12) where C attdem is the DEM weight tensor generator, and C fsm( a ) is theoutput of the FSM module.
Revised: (12) where C attdem is the DEM weight tensor generator, and C fsm is the FSM operator.

Original:  Experiments 4.1 Training and Testing
Revised:  Experiments 4.1 Training and Testing

Original: To train ScaoedNet, Div2K [41] is adopted to generate the synthetic clean-noisy image pairs according to the noise model in subsection 2.1, and the real clean-noisy image pairs from SIDD Medium Dataset [2] and RENOIR Dataset [4] are also used for training.
Revised: To train ScaoedNet, Div2K [41] is adopted to generate the synthetic clean-noisy image pairs according to the noise model in subsection 2.1, and the real clean-noisy image pairs from SIDD Medium Dataset [2] and RENOIR Dataset [4] are also used for training.

Original: Random rotations of 90 ◦ , 180 ◦ , 270 ◦ , and horizontal ﬂipping are performed for each training sample to achieve data augmentation.
Revised: Random rotations of 90 ◦ , 180 ◦ , 270 ◦ , and horizontal ﬂipping are performed for each training sample to achieve data augmentation.

Original: We empirically set K = 5 , η = 0 .
Revised: We empirically set K = 5 , η = 0 .

Original: 5 , γ = 0 .
Revised: 5 , γ = 0 .

Original: 25 , ι = 4 , and T = 16 .
Revised: 25 , ι = 4 , and T = 16 .

Original: Network details ( e.g. , channel number, kernel size) are provided in Figs.2-3, and the default number m is set to 64.
Revised: Network details ( e.g. , channel number, kernel size) are provided in Figs.2-3, and the default number m is set to 64.

Original: Adam [23] is used as the optimizer for the network model with default settings.
Revised: Adam [23] is used as the optimizer for the network model with default settings.

Original: During the training, we initially set the patch size to 64 × 64 , the mini-batch size to 64, and the learning rate to 10 − 4 .
Revised: During the training, we initially set the patch size to 64 × 64 , the mini-batch size to 64, and the learning rate to 10 − 4 .

Original: For ﬁne-tuning, we set the patch size to 256 × 256 , the mini-batch size to 4, and the learning rate to 10 − 5 .
Revised: For ﬁne-tuning, we set the patch size to 256 × 256 , the mini-batch size to 4, and the learning rate to 10 − 5 .

Original: We use
Revised: We use

Original: PyTorch to implement ScaoedNet and train it with an Nvidia GeForce RTX 3090 GPU.
Revised: PyTorch to implement ScaoedNet and train it with an Nvidia GeForce RTX 3090 GPU.

Original: Furthermore,
Revised: Furthermore,

Original: [Table] [Table]
Revised: [Table] [Table]

Original: PSNR and SSIM metrics are utilized for performance evaluation on the following real noisy datasets:
Revised: PSNR and SSIM metrics are utilized for performance evaluation on the following real noisy datasets:

Original: DnD benchmark [35], SIDD benchmark [2], and SIDD validation dataset [2].
Revised: DnD benchmark [35], SIDD benchmark [2], and SIDD validation dataset [2].

Original: The near noise-free images of DnD and SIDD benchmarks are not publicly available.
Revised: The near noise-free images of DnD and SIDD benchmarks are not publicly available.

Original: However, we can submit the denoised image to the DnD and SIDD online servers to obtain the PSNR/SSIM results.
Revised: However, we can submit the denoised image to the DnD and SIDD online servers to obtain the PSNR/SSIM results.

Original: Due to the limited space, please refer to the ‘Supplementary Material’ for more experimental results.
Revised: Due to the limited space, please refer to the ‘Supplementary Material’ for more experimental results.

Original: 4.2 Real-World Denoising on DnD
Revised: 4.2 Real-World Denoising on DnD

Original: DnD consists of 50 pairs of real clean-noisy scenes, where 20 smaller images of size 512 ×are extracted from each scene.
Revised: DnD consists of 50 pairs of real clean-noisy scenes, where 20 smaller images of size 512 ×are extracted from each scene.

Original: However, the corresponding ground-truth images are not available for users.
Revised: However, the corresponding ground-truth images are not available for users.

Original: In this subsection, we evaluate the performance of ScaoedNet on the DnD benchmark.
Revised: In this subsection, we evaluate the performance of ScaoedNet on the DnD benchmark.

Original: Speciﬁcally, 12 competing methods are tested, including CBM3D [14], TNRD [11], DnCNN [52],
Revised: Speciﬁcally, 12 competing methods are tested, including CBM3D [14], TNRD [11], DnCNN [52],

Original: FFDNet [53], DCDicL
Revised: FFDNet [53], DCDicL

Original: [57], CBDNet [19], VDN [48], RIDNet [5], AINDNet [22], InvDN [27],
Revised: [57], CBDNet [19], VDN [48], RIDNet [5], AINDNet [22], InvDN [27],

Original: DANet [49], and DeamNet [36].
Revised: DANet [49], and DeamNet [36].

Original: Note that AINDNet(TF) is tested in the experiment since it can get the best overall performance on both DnD and SIDD among all AINDNet models according to [22, 36].
Revised: Note that AINDNet(TF) is tested in the experiment since it can get the best overall performance on both DnD and SIDD among all AINDNet models according to [22, 36].

Original: DCDicL
Revised: DCDicL

Original: [57] gets lower results because it is designed for AWGN.
Revised: [57] gets lower results because it is designed for AWGN.

Original: To better show the denoising performance of ScaoedNet, the results of the self-ensemble [40] version denoted by the super script † are also reported.
Revised: To better show the denoising performance of ScaoedNet, the results of the self-ensemble [40] version denoted by the super script † are also reported.

Original: Table 1 and Fig.
Revised: Table 1 and Fig.

Original: 4 show the objective and visual results.
Revised: 4 show the objective and visual results.

Original: Due to the limited space, please enlarge the ﬁgures on the screen for better comparison.
Revised: Due to the limited space, please enlarge the ﬁgures on the screen for better comparison.

Original: According to the results, ScaoedNet signiﬁcantly outperforms these competing methods in PSNR/SSIM results.
Revised: According to the results, ScaoedNet signiﬁcantly outperforms these competing methods in PSNR/SSIM results.

Original: In addition, these methods prone to generate either over-smooth results or distorted results with remaining noise.
Revised: In addition, these methods prone to generate either over-smooth results or distorted results with remaining noise.

Original: On the contrary, ScaoedNet can provide a clean output image with robust noise removal, effective artifacts suppression, and excellent image edges preservation.
Revised: On the contrary, ScaoedNet can provide a clean output image with robust noise removal, effective artifacts suppression, and excellent image edges preservation.

Original: 4.3 Real-World Denoising on SIDD
Revised: 4.3 Real-World Denoising on SIDD

Original: We further evaluate the generalization and performance of ScaoedNet on SIDD captured by ﬁve representative smartphones.
Revised: We further evaluate the generalization and performance of ScaoedNet on SIDD captured by ﬁve representative smartphones.

Original: Only 320 clean-noisy image pairs (SIDD Medium Dataset) are provided for training.
Revised: Only 320 clean-noisy image pairs (SIDD Medium Dataset) are provided for training.

Original: For testing, SIDD benchmark only with the noisy images is provided.
Revised: For testing, SIDD benchmark only with the noisy images is provided.

Original: In addition, SIDD validation consists of 1280 clean-noisy image pairs can also be used.
Revised: In addition, SIDD validation consists of 1280 clean-noisy image pairs can also be used.

Original: In this subsection, extensive experiments are carried out on SIDD validation and SIDD benchmark to evaluate the superiority of
Revised: In this subsection, extensive experiments are carried out on SIDD validation and SIDD benchmark to evaluate the superiority of

Original: [Table] [Table]
Revised: [Table] [Table]

Original: ScaoedNet.
Revised: ScaoedNet.

Original: Table 2 and Fig.
Revised: Table 2 and Fig.

Original: 5 illustrate the quantitative and visual comparisons between the previous methods and ScaoedNet.
Revised: 5 illustrate the quantitative and visual comparisons between the previous methods and ScaoedNet.

Original: We can notice that ScaoedNet achieves the best results compared to other competing methods in terms of both the objective evaluation and the visual performance.
Revised: We can notice that ScaoedNet achieves the best results compared to other competing methods in terms of both the objective evaluation and the visual performance.

Original: 4.4 Inﬂuence of Parameters K and T
Revised: 4.4 Inﬂuence of Parameters K and T

Original: Since the iteration stage number K and the FM 2 ARB number T are crucial for denoising performance, the PSNR/SSIM evolutions versus K and T are analyzed.
Revised: Since the iteration stage number K and the FM 2 ARB number T are crucial for denoising performance, the PSNR/SSIM evolutions versus K and T are analyzed.

Original: Speciﬁcally, we train six ScaoedNet variants by setting K = 1 , 2 , 3 , 4 , 5 , 6 and ﬁve ScaoedNet variants by setting T = 4 , 8 , 12 , 16 , 20 .
Revised: Speciﬁcally, we train six ScaoedNet variants by setting K = 1 , 2 , 3 , 4 , 5 , 6 and ﬁve ScaoedNet variants by setting T = 4 , 8 , 12 , 16 , 20 .

Original: The PSNR/SSIM results on SIDD validation dataset are provided in Tables 3 and 4.
Revised: The PSNR/SSIM results on SIDD validation dataset are provided in Tables 3 and 4.

Original: According to the results, when K = 1 , ScaoedNet can already achieve promising denoising performance with 39.24dB/0.9170.
Revised: According to the results, when K = 1 , ScaoedNet can already achieve promising denoising performance with 39.24dB/0.9170.

Original: When K reaches 5, PSNR/SSIM further converge to 39.48dB/0.9186, which indicates that G -Module, u -Net, and z -Net have learned to cooperate with each other in this case.
Revised: When K reaches 5, PSNR/SSIM further converge to 39.48dB/0.9186, which indicates that G -Module, u -Net, and z -Net have learned to cooperate with each other in this case.

Original: For
Revised: For

Original: T , we can see that T = 16 is signiﬁcantly better than T = 4 , 8 , 12 , and there is little PSNR/SSIM improvements when T is larger than 16.
Revised: T , we can see that T = 16 is signiﬁcantly better than T = 4 , 8 , 12 , and there is little PSNR/SSIM improvements when T is larger than 16.

Original: These suggest that K = 5 and T = 16 are good choices.
Revised: These suggest that K = 5 and T = 16 are good choices.

Original: 4.5 Ablation Study
Revised: 4.5 Ablation Study

Original: Ablation on LS.
Revised: Ablation on LS.

Original: By removing the LS encoding and decoding modules from ScaoedNet, we can obtain the image space denoising network.
Revised: By removing the LS encoding and decoding modules from ScaoedNet, we can obtain the image space denoising network.

Original: Because the Shufﬂe operator with factor 1/2 is used, the output of each RE network in the image space version have only 12 channels for color images and 4 channels for gray images.
Revised: Because the Shufﬂe operator with factor 1/2 is used, the output of each RE network in the image space version have only 12 channels for color images and 4 channels for gray images.

Original: We can see from Fig.
Revised: We can see from Fig.

Original: 6 that the PSNR/SSIM results of the version without LS on SIDD validation dataset are only 39.19dB/0.9163, and the performance reaches to 39.48dB/0.9186 by adding LS.
Revised: 6 that the PSNR/SSIM results of the version without LS on SIDD validation dataset are only 39.19dB/0.9163, and the performance reaches to 39.48dB/0.9186 by adding LS.

Original: Therefore, the LS scheme is essential for the denoising performance.
Revised: Therefore, the LS scheme is essential for the denoising performance.

Original: These reasons make the LS scheme essential for the performance.
Revised: These reasons make the LS scheme essential for the performance.

Original: In Sec.
Revised: In Sec.

Original: 4.5 ‘Ablation Study’, ‘w/o LS’ corresponds to the network implementation without LS, and the performance decreases signiﬁcantly in this case, which veriﬁes our analyses.
Revised: 4.5 ‘Ablation Study’, ‘w/o LS’ corresponds to the network implementation without LS, and the performance decreases signiﬁcantly in this case, which veriﬁes our analyses.

Original: In addition, by visualizing the features after E , we ﬁnd it can well separate the image signal related features and noise related components, which also makes it easier to denoise complex real noise.
Revised: In addition, by visualizing the features after E , we ﬁnd it can well separate the image signal related features and noise related components, which also makes it easier to denoise complex real noise.

Original: Ablation on SC.
Revised: Ablation on SC.

Original: In our method, the SC scheme is proposed for enhancing the optimization results.
Revised: In our method, the SC scheme is proposed for enhancing the optimization results.

Original: The ablation study on SC is conducted and the performance comparisons are shown in Fig.
Revised: The ablation study on SC is conducted and the performance comparisons are shown in Fig.

Original: Note that the version without SC means u ( i ) and z ( i ) are respectively removed when estimating u ( i +1) and z ( i +1) .
Revised: Note that the version without SC means u ( i ) and z ( i ) are respectively removed when estimating u ( i +1) and z ( i +1) .

Original: We can see that the PSNR/SSIM results decrease from 39.48dB/0.9186 to 39.22dB/0.
Revised: We can see that the PSNR/SSIM results decrease from 39.48dB/0.9186 to 39.22dB/0.

Original: without SC.
Revised: without SC.

Original: This observation indicates that the SC scheme is an effective scheme in ScaoedNet.
Revised: This observation indicates that the SC scheme is an effective scheme in ScaoedNet.

Original: Ablation on GC.
Revised: Ablation on GC.

Original: For further performance enhancement, the GC scheme is proposed.
Revised: For further performance enhancement, the GC scheme is proposed.

Original: Note that the version without GC means the branches related to g ( i ) -s are removed.
Revised: Note that the version without GC means the branches related to g ( i ) -s are removed.

Original: In other words, G -Module and DEMs are removed.
Revised: In other words, G -Module and DEMs are removed.

Original: We can conclude from Fig.
Revised: We can conclude from Fig.

Original: 6 that the PSNR/SSIM results decrease 0.13dB/0.0015 without GC.
Revised: 6 that the PSNR/SSIM results decrease 0.13dB/0.0015 without GC.

Original: This observation indicates the superiority of GC in ScaoedNet.
Revised: This observation indicates the superiority of GC in ScaoedNet.

Original: Ablation on Attention.
Revised: Ablation on Attention.

Original: The proposed attention module can synchronously realize external-internal important information mining.
Revised: The proposed attention module can synchronously realize external-internal important information mining.

Original: Fig.
Revised: Fig.

Original: 6 shows that, by removing FM 2 A, the PSNR/SSIM will decrease
Revised: 6 shows that, by removing FM 2 A, the PSNR/SSIM will decrease

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: from 39.48dB/0.9186 to 39.28dB/0.9164.
Revised: from 39.48dB/0.9186 to 39.28dB/0.9164.

Original: Consequently, the effectiveness of FM 2 A is veriﬁed.
Revised: Consequently, the effectiveness of FM 2 A is veriﬁed.

Original: More details of FSM and DEM are provided in ‘Supplementary Material’.
Revised: More details of FSM and DEM are provided in ‘Supplementary Material’.

Original: Ablation on SSIM Loss.
Revised: Ablation on SSIM Loss.

Original: To verify the inﬂuence of the SSIM loss in our method, we remove it from the loss function in Eq. 6.
Revised: To verify the inﬂuence of the SSIM loss in our method, we remove it from the loss function in Eq. 6.

Original: We can see from Fig.
Revised: We can see from Fig.

Original: 6 that PSNR/SSIM of the one without SSIM loss will decrease slightly to 39.46dB/0.9179.
Revised: 6 that PSNR/SSIM of the one without SSIM loss will decrease slightly to 39.46dB/0.9179.

Original: We should note that, even without the SSIM loss, our method can also achieve higher performance than other comparisons baselines.
Revised: We should note that, even without the SSIM loss, our method can also achieve higher performance than other comparisons baselines.

Original: 4.6 Study on NFAE layer
Revised: 4.6 Study on NFAE layer

Original: The proposed parameter-free NFAE layer is an important component in ScaoedNet.
Revised: The proposed parameter-free NFAE layer is an important component in ScaoedNet.

Original: To show the effectiveness of NFAE, we use BSD68 [37] and Kodak24 [1] to synthesize the noisy images according to the noise model in subsection 2.1.
Revised: To show the effectiveness of NFAE, we use BSD68 [37] and Kodak24 [1] to synthesize the noisy images according to the noise model in subsection 2.1.

Original: Then, we build ‘noisy image and noise map’ pairs as the testing dataset.
Revised: Then, we build ‘noisy image and noise map’ pairs as the testing dataset.

Original: u ini -Net with NFAE, u ini -Net without NFAE, and the noise estimation networks from
Revised: u ini -Net with NFAE, u ini -Net without NFAE, and the noise estimation networks from

Original: CBDNet [19], AINDNet [22], and PRIDNet [56] are used for comparisons.
Revised: CBDNet [19], AINDNet [22], and PRIDNet [56] are used for comparisons.

Original: The PSNR result and the L 1 distance (cid:80) Vg =1 (cid:107)
Revised: The PSNR result and the L 1 distance (cid:80) Vg =1 (cid:107)

Original: ˆ u g − u g (cid:107) 11 /
Revised: ˆ u g − u g (cid:107) 11 /

Original: ( H g W g V )
Revised: ( H g W g V )

Original: ( V is the number of the test images, and H g , W g are the image height and width of the g th test image) between the estimated ˆ u g and the ground-truth noise map u g are reported in Table 5.
Revised: ( V is the number of the test images, and H g , W g are the image height and width of the g th test image) between the estimated ˆ u g and the ground-truth noise map u g are reported in Table 5.

Original: Note that during the calculations of the L 1 distance and PSNR, u is scaled to lie in the range of [0 255].
Revised: Note that during the calculations of the L 1 distance and PSNR, u is scaled to lie in the range of [0 255].

Original: The results show that the NFAE layer is essential for the estimation performance, and the network with NFAE can obtain more accurate estimates than the noise estimation networks from CBDNet [19], AINDNet [22], and PRIDNet [56].
Revised: The results show that the NFAE layer is essential for the estimation performance, and the network with NFAE can obtain more accurate estimates than the noise estimation networks from CBDNet [19], AINDNet [22], and PRIDNet [56].

Original: 4.7 Parameter Number
Revised: 4.7 Parameter Number

Original: In this subsection, we report the network parameter numbers vs .
Revised: In this subsection, we report the network parameter numbers vs .

Original: the average PSNRs/SSIMs of the competing methods in Fig.
Revised: the average PSNRs/SSIMs of the competing methods in Fig.

Original: We can see that ScaoedNet has a moderate parameter number, which is signiﬁcantly lower than CBDNet [14], VDN [48], AINDNet [22], and DANet [49].
Revised: We can see that ScaoedNet has a moderate parameter number, which is signiﬁcantly lower than CBDNet [14], VDN [48], AINDNet [22], and DANet [49].

Original: Furthermore, although other methods have smaller parameter numbers than ScaoedNet, their objective performances are much lower.
Revised: Furthermore, although other methods have smaller parameter numbers than ScaoedNet, their objective performances are much lower.

Original: In summary, our ScaoedNet is effective with a moderate network parameter number.
Revised: In summary, our ScaoedNet is effective with a moderate network parameter number.

Original:  Conclusion
Revised:  Conclusion

Original: In this paper, we propose a new denoising network for real noise images.
Revised: In this paper, we propose a new denoising network for real noise images.

Original: Although many denoising networks have been proposed, most of them are limited to AWGN and ignore the valuable achievements of the classic denoising methods.
Revised: Although many denoising networks have been proposed, most of them are limited to AWGN and ignore the valuable achievements of the classic denoising methods.

Original: In contrast, we ﬁrst propose a novel model-based denoising cost function, and then solve it by the proposed SC alternative optimization algorithm.
Revised: In contrast, we ﬁrst propose a novel model-based denoising cost function, and then solve it by the proposed SC alternative optimization algorithm.

Original: After that, a deep unfolding denoising network is proposed according to the optimization process.
Revised: After that, a deep unfolding denoising network is proposed according to the optimization process.

Original: Our denoising network combines some valuable achievements of the classical denoising methods.
Revised: Our denoising network combines some valuable achievements of the classical denoising methods.

Original: Extensive experiments verify that our method achieves excellent performance on real image denoising.
Revised: Extensive experiments verify that our method achieves excellent performance on real image denoising.

Original: Broader Impact
Revised: Broader Impact

Original: As an important computer vision task, image denoising has largely beneﬁted the society in various areas and has no negative impact yet.
Revised: As an important computer vision task, image denoising has largely beneﬁted the society in various areas and has no negative impact yet.

Original: The proposed method could further improve the performance especially in the challenging real-world cases where the noise information is unknown.
Revised: The proposed method could further improve the performance especially in the challenging real-world cases where the noise information is unknown.

Original: This work has no negative impact on the ethical and societal aspects.
Revised: This work has no negative impact on the ethical and societal aspects.

Original: Acknowledgements
Revised: Acknowledgements

Original: This work was supported by the National Natural Science Foundation of China under Grant 62171304, and in part by the Key Research and Development Program of Sichuan Province under Grant 2022YFS0098.
Revised: This work was supported by the National Natural Science Foundation of China under Grant 62171304, and in part by the Key Research and Development Program of Sichuan Province under Grant 2022YFS0098.

Original: Checklist
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?

Original: [Yes] All claims are supported by extensive empirical experiments (b) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: [Yes] All claims are supported by extensive empirical experiments (b) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] (c) Did you discuss any potential negative societal impacts of your work?
Revised: [Yes] (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] see section ‘ Broader Impact ’ (d) Did you describe the limitations of your work?
Revised: [Yes] see section ‘ Broader Impact ’ (d) Did you describe the limitations of your work?

Original: [Yes] see section ‘S15.
Revised: [Yes] see section ‘S15.

Original: Limitations and Future Works’ in Supplementary Material 2.
Revised: Limitations and Future Works’ in Supplementary Material 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A] (b) Did you include complete proofs of all theoretical results?
Revised: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 3.
Revised: [N/A] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] See subsection ‘4.1 Training and Testing’ in main paper (c)
Revised: [Yes] See subsection ‘4.1 Training and Testing’ in main paper (c)

Original: Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [N/A] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: [N/A] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] See subsection ‘4.1 Training and Testing’ in main paper 4.
Revised: [Yes] See subsection ‘4.1 Training and Testing’ in main paper 4.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?
Revised: [Yes] (b) Did you mention the license of the assets?

Original: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes] See the beginning of Supplementary Material (d) Did you discuss whether
Revised: [Yes] See the beginning of Supplementary Material (d) Did you discuss whether

Original: and how consent was obtained from people whose data you’re using/curating?
Revised: and how consent was obtained from people whose data you’re using/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?

Original: [N/A] 5.
Revised: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]