Original: Panchromatic and Multispectral Image Fusion via Alternating Reverse Filtering Network
Revised: Panchromatic and Multispectral Image Fusion via Alternating Reverse Filtering Network

Original: Anonymous Author(s)

Original: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Panchromatic (PAN) and multi-spectral (MS) image fusion, named Pan-sharpening,refers to super-resolve the low-resolution (LR) multi-spectral (MS) images in thespatial domain to generate the expected high-resolution (HR) MS images, conditioning on the corresponding high-resolution PAN images.
Revised: Panchromatic (PAN) and multi-spectral (MS) image fusion, named Pan-sharpening, refers to super-resolve the low-resolution (LR) multi-spectral (MS) images in the spatial domain to generate the expected high-resolution (HR) MS images, conditioning on the corresponding high-resolution PAN images.

Original: In this paper, we presenta simple yet effective alternating reverse filtering network for pan-sharpening.
Revised: In this paper, we present a simple yet effective alternating reverse filtering network for pan-sharpening.

Original: Inspired by the classical reverse filtering that reverses images to the status beforefiltering, we formulate pan-sharpening as an alternately iterative reverse filteringprocess, which fuses LR MS and HR MS in an interpretable manner.
Revised: Inspired by the classical reverse filtering that reverses images to the status before filtering, we formulate pan-sharpening as an alternately iterative reverse filtering process, which fuses LR MS and HR MS in an interpretable manner.

Original: Different fromexisting model-driven methods that require well-designed priors and degradationassumptions, the reverse filtering process avoids the dependency on pre-definedexact priors.
Revised: Different from existing model-driven methods that require well-designed priors and degradation assumptions, the reverse filtering process avoids the dependency on pre-defined exact priors.

Original: To guarantee the stability and convergence of the iterative processvia contraction mapping on a metric space, we develop the learnable multi-scale
Revised: To guarantee the stability and convergence of the iterative process via contraction mapping on a metric space, we develop the learnable multi-scale

Original: Gaussian kernel module, instead of using specific filters.
Revised: Gaussian kernel module, instead of using specific filters.

Original: We demonstrate the theoretical feasibility of such formulations.
Revised: We demonstrate the theoretical feasibility of such formulations.

Original: Extensive experiments on diverse scenes tothoroughly verify the performance of our method, significantly outperforming thestate of the arts.
Revised: Extensive experiments on diverse scenes to thoroughly verify the performance of our method, significantly outperforming the state of the arts.

Original: The code will be released.

Original:  Introduction
Revised:  Introduction

Original: Multispectral images are widely used in various fields such as resource monitoring [1], environmental protection [2, 3] and ecological monitoring [4].
Revised: Multispectral images are widely used in various fields such as resource monitoring [1], environmental protection [2, 3] and ecological monitoring [4].

Original: However, due to the hardware limitations ofmultispectral sensors, multispectral images usually lack high spatial resolution [5, 6].
Revised: However, due to the hardware limitations of multispectral sensors, multispectral images usually lack high spatial resolution [5, 6].

Original: In contrast,high-resolution panchromatic (PAN) images that contain rich spatial details of the same scene areeasy to obtain.
Revised: In contrast, high-resolution panchromatic (PAN) images that contain rich spatial details of the same scene are easy to obtain.

Original: Therefore, pan-sharpening, generating high-resolution multispectral images by fusingpanchromatic images with low-resolution multispectral images, has become an important issue in thefield of remote sensing [7, 8].
Revised: Therefore, pan-sharpening, generating high-resolution multispectral images by fusing panchromatic images with low-resolution multispectral images, has become an important issue in the field of remote sensing [7, 8].

Original: Many efforts have been made to solve the pan-sharpening problem, which can be generally dividedinto two large groups: traditional pan-sharpening methods and deep learning-based methods [9,10].
Revised: Many efforts have been made to solve the pan-sharpening problem, which can be generally divided into two large groups: traditional pan-sharpening methods and deep learning-based methods [9, 10].

Original: Traditional pan-sharpening methods usually require strict assumptions of multispectral imagedegradation via prior knowledge.
Revised: Traditional pan-sharpening methods usually require strict assumptions of multispectral image degradation via prior knowledge.

Original: Otherwise, the inexact assumptions may cause system model error.
Revised: Otherwise, the inexact assumptions may cause system model error.

Original: For example, both the assumptions established by component substitution methods [11, 12] andmulti-resolution analysis methods [13, 14] focus on the relationship between PAN and HR MS, whichis destined to make these methods prone to spectral distortion.
Revised: For example, both the assumptions established by component substitution methods [11, 12] and multi-resolution analysis methods [13, 14] focus on the relationship between PAN and HR MS, which is destined to make these methods prone to spectral distortion.

Original: Unlike the above-mentioned traditionalapproaches, variational optimization approaches consider the relationship among HR MS, LR MSand PAN, and then construct the energy function based on well-designed priors [15, 16].
Revised: Unlike the above-mentioned traditional approaches, variational optimization approaches consider the relationship among HR MS, LR MS and PAN, and then construct the energy function based on well-designed priors [15, 16].

Original: However,the methods of this kind inflict high computational burden, restricting their practical applications.
Revised: However, the methods of this kind inflict high computational burden, restricting their practical applications.

Original: Most recently, deep learning exhibited outstanding performance in the field of remote sensing images[17, 18].
Revised: Most recently, deep learning exhibited outstanding performance in the field of remote sensing images [17, 18].

Original: Undoubtedly, the deep learning-based (DL) methods become a newly developed category to solve pan-sharpening [19, 20].
Revised: Undoubtedly, the deep learning-based (DL) methods become a newly developed category to solve pan-sharpening [19–22].

Original: Unfortunately, the pan-sharpening networks commonly lack theinterpretability in theirs designs, which limits their performance.
Revised: Unfortunately, the pan-sharpening networks commonly lack the interpretability in theirs designs, which limits their performance.

Original: To solve this issue, the modelbased deep learning methods build the network by unrolling the specific optimization algorithm[21, 22].
Revised: To solve this issue, the modelbased deep learning methods build the network by unrolling the specific optimization algorithm [23, 24].

Original: However, the optimization algorithms of model-based deep learning methods still requirewell-designed priors or assumptions.
Revised: However, the optimization algorithms of model-based deep learning methods still require well-designed priors or assumptions.

Original: Additionally, the convergence of the optimization algorithms isnot taken into account in the design of the unrolling networks.
Revised: Additionally, the convergence of the optimization algorithms is not taken into account in the design of the unrolling networks.

Original: To address these problems, we propose a novel pan-sharpening approach called alternating reversefiltering network , which combines classical reverse filtering [23] and deep learning.
Revised: To address these problems, we propose a novel pan-sharpening approach called alternating reverse filtering network , which combines classical reverse filtering [25] and deep learning.

Original: Unlike previousmethods, we formulate pan-sharpening as a reverse filtering process, thus avoiding the dependency onpre-defined priors or assumption.
Revised: Unlike previous methods, we formulate pan-sharpening as a reverse filtering process, thus avoiding the dependency on pre-defined priors or assumption.

Original: In addition, we tailor the classical reverse filtering in an alternatingiteration manner for the pan-sharpening problem.
Revised: In addition, we tailor the classical reverse filtering in an alternating iteration manner for the pan-sharpening problem.

Original: The process of alternating iterations is unrolledinto a network.
Revised: The process of alternating iterations is unrolled into a network.

Original: We demonstrate such formulations is theoretical feasibility.
Revised: We demonstrate such formulations is theoretical feasibility.

Original: To guarantee the stabilityand convergence of the iterative process on the basis of contraction mapping in a metric space, weintroduce the learnable multi-scale Gaussian kernel module in the network.
Revised: To guarantee the stability and convergence of the iterative process on the basis of contraction mapping in a metric space, we introduce the learnable multi-scale Gaussian kernel module in the network.

Original: Such a key issue iscommonly neglected in previous unrolling-based deep learning methods.
Revised: Such a key issue is commonly neglected in previous unrolling-based deep learning methods.

Original: The main contributions of this paper can be summarized as follows: 1) We introduce a new perspectivefor pan-sharpening by formulating it as a reverse filtering process.
Revised: The main contributions of this paper can be summarized as follows: 1) We introduce a new perspective for pan-sharpening by formulating it as a reverse filtering process.

Original: To the best of our knowledge, thisis the first effort to solve multispectral image fusion problem using the method of this kind. 2)
Revised: To the best of our knowledge, this is the first effort to solve multispectral image fusion problem using the method of this kind. 2)

Original: Incontrast to existing model-driven methods, our iterative network can obtain HR MS without the needfor pre-defined exact priors or assumptions.
Revised: In contrast to existing model-driven methods, our iterative network can obtain HR MS without the need for pre-defined exact priors or assumptions.

Original: 3) Instead of using specific filters in reverse filtering, weconstrain their formulation in a more compact learnable multi-scale Gaussian kernel module, whichguarantees the stability and convergence of the iterative process.
Revised: 3) Instead of using specific filters in reverse filtering, we constrain their formulation in a more compact learnable multi-scale Gaussian kernel module, which guarantees the stability and convergence of the iterative process.

Original: In addition, extensive experimentalresults on simulated and real-world scenes show that the proposed network significantly outperformsthe state of the arts.
Revised: In addition, extensive experimental results on simulated and real-world scenes show that the proposed network significantly outperforms the state of the arts.

Original:  Related Work
Revised:  Related Work

Original: Pan-sharpening.
Revised: Pan-sharpening.

Original: Traditional pan-sharpening methods are classified into three types: componentsubstitution (CS)-, multi-resolution analysis (MRA)-, and variational optimization (VO)-basedmethods [24, 16, 25–27].
Revised: Traditional pan-sharpening methods are classified into three types: component substitution (CS)-, multi-resolution analysis (MRA)-, and variational optimization (VO)-based methods [26, 16, 27–29].

Original: The main idea of CS-based methods is to separate spatial and spectralinformation of the MS image in a suitable space and further fuse them with the PAN image.
Revised: The main idea of CS-based methods is to separate spatial and spectral information of the MS image in a suitable space and further fuse them with the PAN image.

Original: Therepresentative CS-based methods include intensity hue-saturation (IHS) fusion [28], the principalcomponent analysis (PCA) methods [11, 29], Brovey transforms [30], and Gram-Schmidt (GS)orthogonalization method [31].
Revised: The representative CS-based methods include intensity hue-saturation (IHS) fusion [30], the principal component analysis (PCA) methods [11, 31], Brovey transforms [32], and Gram-Schmidt (GS) orthogonalization method [33].

Original: MRA-based methods decompose MS and PAN images into multiscale space via decimated wavelet transform (DWT) [32], high-pass filter fusion (HPF) [33], indusionmethod [13] and atrous wavelet transform (ATWT) [34].
Revised: MRA-based methods decompose MS and PAN images into multiscale space via decimated wavelet transform (DWT) [34], high-pass filter fusion (HPF) [35], indusion method [13] and atrous wavelet transform (ATWT) [36].

Original: Then, the decomposed version PAN imagesare injected into the corresponding MS images for information fusion.
Revised: Then, the decomposed version PAN images are injected into the corresponding MS images for information fusion.

Original: VO-based methods regardthe pan-sharpening tasks as an ill-posed problem by minimizing a loss function, including dynamicgradient sparsity property (SIRF) [35], local gradient constraint (LGC) [15], group low-rank constraintfor texture similarity (ADMM) 
Revised: VO-based methods regard the pan-sharpening tasks as an ill-posed problem by minimizing a loss function, including dynamic gradient sparsity property (SIRF) [37], local gradient constraint (LGC) [15], group low-rank constraint for texture similarity (ADMM) [16].

Original: However, the performance of these methods is limited due tothe shallow non-linear expression in these models.
Revised: However, the performance of these methods is limited due to the shallow non-linear expression in these models.

Original: Since then, deep learning-based pan-sharpeningalgorithms have dominated this field [9, 36, 37].
Revised: Since then, deep learning-based pan-sharpening algorithms have dominated this field [9, 38, 39].

Original: Masi et al.
Revised: Masi et al.

Original: [9] are the first to use CNN to dealwith the issue of pan-sharpening.
Revised: [9] are the first to use CNN to deal with the issue of pan-sharpening.

Original: Although the structure is simple, the effect is much better than thetraditional methods.
Revised: Although the structure is simple, the effect is much better than the traditional methods.

Original: Then, Yang et al.
Revised: Then, Yang et al.

Original: [36] designed a deeper CNN by relying on resblock in [38].
Revised: [38] designed a deeper CNN by relying on resblock in [40].

Original: Meanwhile, Yuan et al.
Revised: Meanwhile, Yuan et al.

Original: [39] introduced multi-scale module into the basic CNN architecture.
Revised: [41] introduced multi-scale module into the basic CNN architecture.

Original: Unrolling-based deep learning method.
Revised: Unrolling-based deep learning method.

Original: In recent years, many researchers [40–45] attempt tocombine domain knowledge with deep neural networks to propose deep unrolling networks which takeadvantages of the model-based methods’ interpretability and learning-based methods’ strong mappingability.
Revised: In recent years, many researchers [42–47] attempt to combine domain knowledge with deep neural networks to propose deep unrolling networks which take advantages of the model-based methods’ interpretability and learning-based methods’ strong mapping ability.

Original: Specifically, the deep unrolling network firstly unrolls certain optimization algorithms [46–48, 21, 49–55] and utilizes deep neural network to parameterize the unrolling model, then minimizesthe loss function and optimizes the parameters in an end-to-end manner.
Revised: Specifically, the deep unrolling network firstly unrolls certain optimization algorithms [48– 50, 23, 51–57] and utilizes deep neural network to parameterize the unrolling model, then minimizes the loss function and optimizes the parameters in an end-to-end manner.

Original: For example, Xu et al.[22] developed two separate priors of PAN and MS modality to design the unrolling structure forpan-sharpening.
Revised: For example, Xu et al. [24] developed two separate priors of PAN and MS modality to design the unrolling structure for [Figure] pan-sharpening.

Original: The model-driven methods have better interpretability and clearer physical meaning.
Revised: The model-driven methods have better interpretability and clearer physical meaning.

Original: Cao et al.
Revised: Cao et al.

Original: [44] unrolled an alternate optimization algorithm into CNN.
Revised: [46] unrolled an alternate optimization algorithm into CNN.

Original: However, the optimizationmodels of model-based deeplearning methods also require well-designed priors and exact degradationassumptions.
Revised: However, the optimization of model-based deep methods also require well-designed priors and exact degradation assumptions.

Original: Additionally, the convergence of the optimization algorithm is not taken into account inthe design of the unrolling network.

Original: [Figure]

Original:  Proposed Method
Revised:  Proposed Method

Original: In this section, we provide a detailed introduction to our proposed alternating reverse filtering networkfor pan-sharpening.
Revised: In this section, we provide a detailed introduction to our proposed alternating reverse filtering network for pan-sharpening.

Original: For convenience, we first define some notations.
Revised: For convenience, we first define some notations.

Original: Concretely, L ∈ R w × h × Bdenotes the low-resolution (LR) multispectral image, H ∈ R W × H × B represents the correspondinghigh-resolution (HR) multispectral image, and P ∈ R W × H × b is the PAN image.
Revised: Concretely, L ∈ R w × h × B denotes the low-resolution (LR) multispectral image, H ∈ R W × H × B represents the corresponding high-resolution (HR) multispectral image, and P ∈ R W × H × b is the PAN image.

Original: 3.1 Problem Formulation
Revised: 3.1 Problem Formulation

Original: For multispectral image restoration, the degradation model is commonly formulated as
Revised: For multispectral image restoration, the degradation model is commonly formulated as

Original: [Equation]
Revised: [Equation]

Original: where ∗ , k , ↓ s and ϵ denote the convolution operation, blurring kernel, down-sampling operator andthe measurement noise, respectively.
Revised: where ∗ , k , ↓ s and ϵ denote the convolution operation, blurring kernel, down-sampling operator and the measurement noise, respectively.

Original: To restore high quality H from L , high-resolution PAN image
Revised: To restore high quality H from L , high-resolution PAN image

Original: P is introduced to help enhance structural detail.
Revised: P is introduced to help enhance structural detail.

Original: In CS-based models, the intensity component I ,a generalized IHS concept [56], is usually replaced by P but the discrepancy between P and I cancause spectral distortion in the fused image.
Revised: In CS-based models, the intensity component I , a generalized IHS concept [58], is usually replaced by P but the discrepancy between P and I can cause spectral distortion in the fused image.

Original: The VO-based models solve the problem by explicitlyconstructing many well-designed priors among H , L and P .
Revised: The VO-based models solve the problem by explicitly constructing many well-designed priors among H , L and P .

Original: However, hand-crafted priors don’twork well in practical complex situations.
Revised: However, hand-crafted priors don’t work well in practical complex situations.

Original: Inspired by classical reverse filtering [23], we proposealternating reverse filtering method to estimate H by the more general multispectral image priors:L = f ( H ) , (2) P [Equation]
Revised: Inspired by classical reverse filtering [25], we propose alternating reverse filtering method to estimate H by the more general multispectral image priors:

Original: g

Original: [Equation]
Revised: [Equation]

Original: (3) where f ( · ) and g ( · ) denote the degradation processes, and H
Revised: where f ( · ) and g ( · ) denote the degradation processes, and H

Original: I is the intensity component of H .
Revised: I is the intensity component of H .

Original: 3.2 Model Optimization
Revised: 3.2 Model Optimization

Original: Definition 3.2 Suppose ( H , d ) is a metric space and T : H → H is a mapping function.
Revised: Definition 3.2 Suppose ( H , d ) is a metric space and T : H → H is a mapping function.

Original: For allx, y ∈ H , if there exists a constant c ∈ [0 , 1) that makes the following formula
Revised: For all x, y ∈ H , if there exists a constant c ∈ [0 , 1) that makes the following formula

Original: [Equation]
Revised: [Equation]

Original: mapping T : H → H is called Contraction Mapping.
Revised: mapping T : H → H is called Contraction Mapping [59].

Original: In the image space, the metric space ( H , d ) can be expressed as
Revised: In the image space, the metric space ( H , d ) can be expressed as

Original: [Equation]
Revised: [Equation]

Original: where w × h is the number of image pixels and d ( x, y ) is the Euclidean distance.
Revised: where w × h is the number of image pixels and d ( x, y ) is the Euclidean distance.

Original: Theorem 3.2 A variable x ∗ is a fixed point for a given function Φ if Φ( x ∗ ) = x ∗ .
Revised: Theorem 3.2 A variable x ∗ is a fixed point for a given function Φ if Φ( x ∗ ) = x ∗ .

Original: When mappingΦ : H → H is a contraction mapping, Φ admits a unique fixed-point x ∗ in H .
Revised: When mapping Φ : H → H is a contraction mapping, Φ admits a unique fixed-point x ∗ in H .

Original: Further, x ∗ can befound in the following way.
Revised: Further, x ∗ can be found in the following way.

Original: Let the initial guess be x 0 and define a sequence { x n } as x n = Φ( x n − 1 ) .
Revised: Let the initial guess be x 0 and define a sequence { x n } as x n = Φ( x n − 1 ) .

Original: When the iterative process converges, lim n →∞ x n = x ∗ .
Revised: When the iterative process converges, lim n →∞ x n = x ∗ .

Original: Reverse Filtering Without loss of generality, the degenerate processes f ( · ) and g ( · ) can be considered as broadly defined filters F ( · ) that smooth images, texture removal or other properties.
Revised: Reverse Filtering Without loss of generality, the degenerate processes f ( · ) and g ( · ) can be considered as broadly defined filters F ( · ) that smooth images, texture removal or other properties.

Original: Filteringprocess can be described as
Revised: Filtering process can be described as

Original: [Equation]
Revised: [Equation]

Original: where x and y are the input image and the filtering result.
Revised: where x and y are the input image and the filtering result.

Original: When F ( · ) is unknown, it’s difficult toapply well-designed image priors to obtain the x .
Revised: When F ( · ) is unknown, it’s difficult to apply well-designed image priors to obtain the x .

Original: However, reverse filtering can estimate x withoutneeding to compute F − 1 ( · ) and update restored images according to the filtering effect as[Equation]
Revised: However, reverse filtering can estimate x without needing to compute F − 1 ( · ) and update restored images according to the filtering effect as

Original: [Equation]
Revised: [Equation]

Original: where x k is the current estimate of x in the k -th iteration.
Revised: where x k is the current estimate of x in the k -th iteration.

Original: The iteration starts from x 0 = y and x kgets closer and closer to x with the increasing k .
Revised: The iteration starts from x 0 = y and x k gets closer and closer to x with the increasing k .

Original: We make auxiliary function φ ( · ) as
Revised: We make auxiliary function φ ( · ) as

Original: [Equation]
Revised: [Equation]

Original: Therefore, the above iterative process can be regarded as a fixed point iteration
Revised: Therefore, the above iterative process can be regarded as a fixed point iteration

Original: [Equation]
Revised: [Equation]

Original: With the above analysis, we take the filtering function F ( · ) in Equation 8 as f ( · ) and g ( · ) in Equation2 to obtain two reverse filtering:
Revised: With the above analysis, we take the filtering function F ( · ) in Equation 8 as f ( · ) and g ( · ) in Equation 2 to obtain two reverse filtering:

Original: [Equation]
Revised: [Equation]

Original: and hence our alternating reverse filtering method can be written by following
Revised: and hence our alternating reverse filtering method can be written by following

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where ˆL is the upsampled LR multispectral image L and ˜H kI is an approximate estimate of H I .
Revised: where ˆL is the upsampled LR multispectral image L and ˜H kI is an approximate estimate of H I .

Original: Equation 11 starts with initial state H 

Revised: Equation 11 starts with initial state H

Original: = ˆL .
Revised: = ˆL .

Original: Note that the calculated H k + 12 is used as input in the nextiteration, the intensity component H k
Revised: Note that the calculated H k + 12 is used as input in the next iteration, the intensity component H k

Original: + 

Revised: +

Original: I is then sent into another iteration ˜H k +1{++
+} I= ˜H
Revised: I is then sent into another iteration ˜H k {++
+} I

Original: [Equation]
Revised: [Equation]

Original: + P − g ( ˜H kI )to enhance the structural details with the help of P image.
Revised: to enhance the structural details with the help of P image.

Original: After that, the enhanced intensitycomponent replaces the original component H k + 

Revised: After that, the enhanced intensity component replaces the original component H k +

Original: I ← ˜H k
Revised: I ← ˜H k

Original: +1 I and the new H k +1 is used as input inthe next iteration.
Revised: +1 I and the new H k +1 is used as input in the next iteration.

Original: 3.3 Alternating Reverse Filtering Network
Revised: 3.3 Alternating Reverse Filtering Network

Original: The end-to-end model we construct for GMIR, named as ARFNet (Alternating Reverse Filtering
Revised: The end-to-end model we construct for GMIR, named as ARFNet (Alternating Reverse Filtering

Original: Network), is based on two fixed point iterations in Equation 11 with multi-scale Gaussian convolutionmodule acting as filters.
Revised: Network), is based on two fixed point iterations in Equation 11 with multi-scale Gaussian convolution module acting as filters.

Original: See Figure 1 for the overview of the proposed method.
Revised: See Figure 1 for the overview of the proposed method.

Original: If reverse filteringsatisfies the sufficient condition for definition 3.2, ARFNet will converge lim k →∞ H k
Revised: If reverse filtering satisfies the sufficient condition for definition 3.2, ARFNet will converge lim k →∞ H k

Original: = H ∗ andfinally reaches f ( H ∗ )
Revised: = H ∗ and finally reaches f ( H ∗ )

Original: ≈ ˆL .
Revised: ≈ ˆL .

Original: Take the φ 1 ( · ) for instance, the sufficient condition that theorem 3.2holds is that φ 1 ( H ) forms a contraction mapping
Revised: Take the φ 1 ( · ) for instance, the sufficient condition that theorem 3. holds is that φ 1 ( H ) forms a contraction mapping

Original: [Equation]
Revised: [Equation]

Original: For linear filters, the condition is further simplified as
Revised: For linear filters, the condition is further simplified as

Original: [Equation]
Revised: [Equation]

Original: In our case, the degenerate processes f (· ) and g ( · ) are implemented by multi-scale Gaussian convolution modules.
Revised: In our case, the degenerate processes f (

Original: Multi-scale Gaussian Convolution Module Given an input image x , the output y of Gaussianconvolution [57] can be expressed as[Equation]
Revised: Multi-scale Gaussian Convolution Module Given an input image x , the output y of Gaussian convolution [60] can be expressed as

Original: [Equation]
Revised: [Equation]

Original: where σ k denotes the variance of 2D Gaussian kernels.
Revised: where σ k denotes the variance of 2D Gaussian kernels.

Original: In particular, we define σ 1 = 0 and g ( σ 1 ) =
Revised: In particular, we define σ 1 = 0 and g ( σ 1 ) =

Original: δwhich is Dirac delta.
Revised: δ which is Dirac delta.

Original: Therefore,
Revised: Therefore,

Original: [Equation]
Revised: [Equation]

Original: where (cid:99) H denotes Fourier transform, ⊙ denotes point-wised product and 1 denotes all-one matrix.
Revised: where (cid:99) H denotes Fourier transform, ⊙ denotes point-wised product and 1 denotes all-one matrix.

Original: Thus, the inequality holds true when g ( σ k ) is a normalized Gaussian kernel that means these filterscan be strictly reversed using fixed-point iteration.
Revised: Thus, the inequality holds true when g ( σ k ) is a normalized Gaussian kernel that means these filters can be strictly reversed using fixed-point iteration.

Original: In our model, normalized Gaussian kernel is usedfor initialization and its parameters can be further learned from the data in an end-to-end manner.
Revised: In our model, normalized Gaussian kernel is used for initialization and its parameters can be further learned from the data in an end-to-end manner.

Original: Although the learned convolution kernels may not completely satisfy the condition, fixed pointiteration can be split into two independent sequences:
Revised: Although the learned convolution kernels may not completely satisfy the condition, fixed point iteration can be split into two independent sequences:

Original: [Equation]
Revised: [Equation]

Original: 16) where { H kc } is guaranteed to converge to the unique solution and { H ko } could oscillate.
Revised: where { H kc } is guaranteed to converge to the unique solution and { H ko } could oscillate.

Original: Fortunately,after the first few epochs, the learned convolution kernels are close to the kernels we initialize, whichmakes { H kc } the majority and dominates the convergence of the whole process.
Revised: Fortunately, after the first few epochs, the learned convolution kernels are close to the kernels we initialize, which makes { H kc } the majority and dominates the convergence of the whole process.

Original: In the implementationof the algorithm, we adopt multi-scale Gaussian convolution module to obtain better filtering effect.
Revised: In the implementation of the algorithm, we adopt multi-scale Gaussian convolution module to obtain better filtering effect.

Original: Compared to Gaussian convolution, the multi-scale Gaussian convolution module integrates Gaussiankernels with different kernel sizes, expressed as
Revised: Compared to Gaussian convolution, the multi-scale Gaussian convolution module integrates Gaussian kernels with different kernel sizes, expressed as

Original: [Equation]
Revised: [Equation]

Original: (17) where γ k is the learnable mixing coefficient and k denotes
Revised: [Equation] where γ k is the learnable mixing coefficient and k denotes

Original: different kernel sizes 1 × 1 , 3 × 3 , 5 ×5 , · · · , M × M .
Revised: different kernel sizes 1 × 1 , 3 × 3 , 5 × 5 , · · · , M × M .

Original: The multi-scale Gaussian convolution is a series simple group convolutional layersdefined by initialized 2D Gaussian kernels.
Revised: The multi-scale Gaussian convolution is a series simple group convolutional layers defined by initialized 2D Gaussian kernels.

Original: Clearly, the form of weighted summation still conformsto the above analysis about sufficient condition.
Revised: Clearly, the form of weighted summation still conforms to the above analysis about sufficient condition.

Original: Taken together, the forward process of alternating reverse filtering network can be described as
Revised: Taken together, the forward process of alternating reverse filtering network can be described as

Original: Algorithm 1.
Revised: Algorithm 1.

Original: Algorithm 1 Proposed algorithm.
Revised: Algorithm 1 Proposed algorithm

Original:  Loss Function We utilize two loss functions, there are the reconstruction loss L r and the structureloss L s , as following
Revised: [Equation] Loss Function We utilize two loss functions, there are the reconstruction loss L r and the structure loss L s , as following

Original: [Equation]
Revised: [Equation]

Original: (18) where λ is the hyperparameter which determines the balance between the overall performance andthe structure texture details.
Revised: where λ is the hyperparameter which determines the balance between the overall performance and the structure texture details.

Original: Specifically, reconstruction loss L r is a common pixel-wise L2 loss andstructure loss L s is based on structural similarity (SSIM).
Revised: Specifically, reconstruction loss L r is a common pixel-wise L2 loss and structure loss L s is based on structural similarity (SSIM).

Original: Thus, the corresponding losses are definedas follows:
Revised: Thus, the corresponding losses are defined as follows:

Original: [Equation]
Revised: [Equation]

Original: [Equation] [Equation] where H , H I , H K and ˜H KI are the ground truth, intensity component of H , the output of alternatingreverse filtering network φ 1 ( · ) and φ 2 ( · ) respectively.
Revised: 20) where H , H I , H K and ˜H KI are the ground truth, intensity component of H , the output of alternating reverse filtering network φ 1 ( · ) and φ 2 ( · ) respectively.

Original: [Table]
Revised: [Table]

Original:  Experiments
Revised:  Experiments

Original: In this section, the datasets and experimental settings are firstly described.
Revised: In this section, the datasets and experimental settings are firstly described.

Original: Then, we evaluate theeffectiveness of our proposed alternating reverse filtering network (ARFNet) on simulated and realworld full-resolution scenes.
Revised: Then, we evaluate the effectiveness of our proposed alternating reverse filtering network (ARFNet) on simulated and realworld full-resolution scenes.

Original: Additionally, we conduct an ablation study to gain insight into therespective effect of different parameter configurations.
Revised: Additionally, we conduct an ablation study to gain insight into the respective effect of different parameter configurations.

Original: More experimental results are included in thesupplemental material.
Revised: More experimental results are included in the supplemental material.

Original: 4.1 Datasets and Experimental Settings
Revised: 4.1 Datasets and Experimental Settings

Original: In order to verify the effectiveness of our models for GMIR, multispectral and panchromatic imagesobtained on three commercial satellites that widely are used, including WorldViewII (WV2), WorldViewIII (WV3), and GaoFen2 (GF2).
Revised: In order to verify the effectiveness of our models for GMIR, multispectral and panchromatic images obtained on three commercial satellites that widely are used, including WorldViewII (WV2), WorldViewIII (WV3), and GaoFen2 (GF2).

Original: Each database contains thousands of image pairs, and they aredivided into training, validation and testing sets that follow the prior works to generate the trainingset by employing the Wald protocol tool [58].
Revised: Each database contains thousands of image pairs, and they are divided into training, validation and testing sets that follow the prior works to generate the training set by employing the Wald protocol tool [61].

Original: In the training set, each training pair contains oneguided PAN image with the size of 128 × 128 , one LR MS patch with the size of 32 × 32 , and oneground truth HR MS patch with the size of 128 × 128 .
Revised: In the training set, each training pair contains one guided PAN image with the size of 128 × 128 , one LR MS patch with the size of 32 × 32 , and one ground truth HR MS patch with the size of 128 × 128 .

Original: Models are implemented via PyTorch and one NVIDIA GTX 3090 GPU is used for training.
Revised: Models are implemented via PyTorch and one NVIDIA GTX 3090 GPU is used for training.

Original: In theexperiments, the SGD algorithm with a momentum equals to 0.9 is adopted to train the models andthe minibatch size is set to 4.
Revised: In the experiments, the SGD algorithm with a momentum equals to 0.9 is adopted to train the models and the minibatch size is set to 4.

Original: The initial learning rate is set to 1 × 10 − 2 .
Revised: The initial learning rate is set to 1 × 10 − 2 .

Original: When reaching 1000 and1500 epochs, the learning rate is decayed by multiplying 0.5, and training ends after 2000 epochs.
Revised: When reaching 1000 and 1500 epochs, the learning rate is decayed by multiplying 0.5, and training ends after 2000 epochs.

Original: Through all experiments, We set the hyperparameter λ in loss function 18 to 0.1, the number K ofalternate iteration to 5 and the maximum size M of Gaussian kernels to 17. 
Revised: Through all experiments, We set the hyperparameter λ in loss function 18 to 0.1, the number K of alternate iteration to 5 and the maximum size M of Gaussian kernels to 17. The sigma of the Gaussian function is set to one fourth of the kernel size.

Original: 4.2 Comparison with SOTAs
Revised: 4.2 Comparison with SOTAs

Original: We conduct several experiments on the benchmark datasets compared with several representativeguided multispectral image restoration methods: five promising traditional methods, includingsmoothing filter-based intensity modulation ((SFIM) [59], Brovey [30], GS [31], intensity huesaturation fusion (IHS) [60], and PCA guided filter (GFPCA) [61]; five commonly-recognizedstate-of-the-art deep-learning based methods, including PNN [9], PANNET [36], multiscale andmultidepth network (MSDCNN) [39], super-resolution-guided progressive network (SRPPNN) [62],and deep gradient projection network (GPPNN) [63].
Revised: We conduct several experiments on the benchmark datasets compared with several representative guided multispectral image restoration methods: five promising traditional methods, including smoothing filter-based intensity modulation ((SFIM) [62], Brovey [32], GS [33], intensity huesaturation fusion (IHS) [63], and PCA guided filter (GFPCA) [64]; five commonly-recognized state-of-the-art deep-learning based methods, including PNN [9], PANNET [38], multiscale and multidepth network (MSDCNN) [41], super-resolution-guided progressive network (SRPPNN) [65], and deep gradient projection network (GPPNN) [66].

Original: In our experiments, we select the widely-used image quality assessment (IQA) metrics for evaluationsuch as the peak signal-to-noise ratio (PSNR), the structural similarity (SSIM), the relative dimensionless global error in synthesis (ERGAS) [64], the correlation coefficient (SCC), the four-band extension
Revised: In our experiments, we select the widely-used image quality assessment (IQA) metrics for evaluation such as the peak signal-to-noise ratio (PSNR), the structural similarity (SSIM), the relative dimension-

Original: [Figure]
Revised: [Figure]

Original: [Figure]

Original: [Table]
Revised: [Table]

Original:  of Q, the spectral angle mapper (SAM) [65], the spectral distortion index D λ , the spatial distortionindex D S , the quality without reference (QNR) [66].
Revised: less global error in synthesis (ERGAS) [67], the correlation coefficient (SCC), the four-band extension of Q, the spectral angle mapper (SAM) [68], the spectral distortion index D λ , the spatial distortion index D S , the quality without reference (QNR) [69].

Original: The last three indicators are non-referencemetrics.
Revised: The last three indicators are non-reference metrics.

Original: Results on Simulated Scene To quantitatively compare the fused multispectral images withthe paired reference ground truth images offered on the simulated datasets, we conduct repeatedexperiments on three datasets.
Revised: Results on Simulated Scene To quantitatively compare the fused multispectral images with the paired reference ground truth images offered on the simulated datasets, we conduct repeated experiments on three datasets.

Original: The average performance of representative GMIR methods is tabled in
Revised: The average performance of representative GMIR methods is tabled in

Original: Table 1.
Revised: Table 1.

Original: The higher values of PSNR and SSIM, the more similar structure between fused multispectralimages and ground truth images.
Revised: The higher values of PSNR and SSIM, the more similar structure between fused multispectral images and ground truth images.

Original: ERGAS takes into account the relative errors of all channels.
Revised: ERGAS takes into account the relative errors of all channels.

Original: SAM,
Revised: SAM,

Original: Q and SCC focus on measuring spectral distortion.
Revised: Q and SCC focus on measuring spectral distortion.

Original: More experimental metric results are included inthe supplemental material.
Revised: More experimental metric results are included in the supplemental material.

Original: The qualitative comparison of the visual results over the representativesample from the WorldView-III dataset is in Figure 2.
Revised: The qualitative comparison of the visual results over the representative sample from the WorldView-III dataset is in Figure 2.

Original: To highlight the differences in detail, we showthe error map between fused image and ground truth image in the last row.
Revised: To highlight the differences in detail, we show the error map between fused image and ground truth image in the last row.

Original: Similarly, more qualitativecomparisons are shown in the supplemental material.
Revised: Similarly, more qualitative comparisons are shown in the supplemental material.

Original: Results on Real-world Full-resolution Scene To assess the generalization performance of modelsin real-world scene, we apply models trained on the GF2 dataset to additional 200 paired GaoFen2satellite images which are constructed using the full-resolution setting as the real scene.
Revised: Results on Real-world Full-resolution Scene To assess the generalization performance of models in real-world scene, we apply models trained on the GF2 dataset to additional 200 paired GaoFen satellite images which are constructed using the full-resolution setting as the real scene.

Original: Lackingavailable ground-truth at full-resolution scene, we employ three widely-used non-reference metrics forassessing the performance: D λ , D S and QNR.
Revised: Lacking available ground-truth at full-resolution scene, we employ three widely-used non-reference metrics for assessing the performance: D λ , D S and QNR.

Original: The quantitative and qualitative results are summarizedin Table 2 and Figure 3 that clearly demonstrate the higher generalization capacity of the proposedalternating reverse filtering network. 
Revised: The quantitative and qualitative results are summarized in Table 2 and Figure 3 that clearly demonstrate the higher generalization capacity of the proposed alternating reverse filtering network. [Figure] [Table] [Figure]

Revised: [Table]

Revised: [Table]

Original: 4.3 Ablation Study
Revised: 4.3 Ablation Study

Original: Ablation studies are implemented on the WordView-II dataset to explore the effect of differentparameters and components on the performance of models.
Revised: Ablation studies are implemented on the WordView-II dataset to explore the effect of different parameters and components on the performance of models.

Original: We use the ARPNet in subsection 4.1 as

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure] [Table] [Table] the baseline for comparison by changing parameters and components, and all comparison models aretrained in the same way.
Revised: We use the ARFNet in subsection 4.1 as the baseline for comparison by changing parameters and components, and all comparison models are trained in the same way.

Original: Firstly, to balance the overall performance and the structure details, we fixthe other components and change only the value of the hyperparameter λ .
Revised: Firstly, to balance the overall performance and the structure details, we fix the other components and change only the value of the hyperparameter λ .

Original: The results in Table 3 showthat the larger the value of λ is within a certain range, the higher the structure similarity betweenthe fused image and ground truth will be.
Revised: The results in Table 3 show that the larger the value of λ is within a certain range, the higher the structure similarity between the fused image and ground truth will be.

Original: Note that if the reverse filtering network φ 2 ( · ) lack the supervision L s that the performance of the entire network will be a significant decrease.
Revised: Note that if the reverse filtering network φ 2 ( · ) lack the supervision L s that the performance of the entire network will be a significant decrease.

Original: Furthermore, we compare the results of different initialization methods.
Revised: Furthermore, we compare the results of different initialization methods.

Original: In Table 4, (I) representsthe kernels  are randomly initialized by Kaiming [67] and (II)  represents the kernels  are initializedby Gaussian kernels.
Revised: In Table 4, (I) represents the kernels that are randomly initialized by Kaiming [70], (II) and (III) represents the kernels that are initialized by Gaussian kernels, but the kernels in (III) are fixed.

Original: From Table 4 and Figure 4, one could see that: 1) The kernels learned by therandomly initialized network cannot satisfy the sufficient condition, which leads to poor performance.
Revised: From Table 4 and Figure 4, one could see that: 1) The kernels learned by the randomly initialized network cannot satisfy the sufficient condition, which leads to poor performance.

Original: The learned kernels initialized by Gaussian kernels are 
Revised: The learned kernels initialized by Gaussian kernels are close to the kernels we initialize, that makes { H kc } the majority and dominates the convergence of the whole process.

Original:  close to the  kernels we initialize
Revised: Although the learned kernels are close to the initialized Gaussian kernels, a large number of multi-scale Gaussian kernels can still bring performance improvements.

Original: , that makes

Original: { H kc } the majority and dominates the convergence of the whole process

Original: As can be seen from Table 5, the ablation studies about maximum kernel size M show that themulti-scale Gaussian convolution module can bring better performance improvement.
Revised: As can be seen from Table 5, the ablation studies about maximum kernel size M show that the multi-scale Gaussian convolution module can bring better performance improvement.

Original: To explore the impact of the number of iterations K on the performance, we experiment with varyingnumbers of K .
Revised: To explore the impact of the number of iterations K on the performance, we experiment with varying numbers of K .

Original: Table 6 shows the results of different K from 1 to 7.
Revised: Table 6 shows the results of different K from 1 to 7.

Original: It can be seen that the PSNRperformance increases as the number of stages increases. We choose K = 5 in our implementationto balance the performanceand computational complexity. 4.4 Limitations and Discussions First, we evaluate the effectiveness of the proposed framework over panchromatic and multispectralimage fusion and we will extend the framework to other multispectral fusion tasks, such as RGB andmultispectral image fusion.
Revised: It can be seen that the PSNR performance

Original: Second, although we develop the multi-scale Gaussian kernel module

Original: 3 ×7 ×13 ×17 ×3 ×7 ×13 ×17 ×
Revised: 3 ×7 ×13 ×17 ×3 ×7 ×13 ×17 ×

Original: Initialized Gaussian kernels
Revised: Initialized Gaussian kernels

Original: Learned kernels (initialized by Gaussian kernels) 3 ×7 ×13 ×17 ×

Revised: Learned kernels (initialized by Gaussian kernels) 3 ×7 ×13 ×17 ×

Original: ×7 ×13 ×17 × 
Revised: ×7 ×13 ×17 × increases as the number of stages increases. We choose K = 5 in our implementation to balance the performance and computational complexity.

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: First, we evaluate the effectiveness of the proposed framework over panchromatic and multispectral image fusion and we will extend the framework to other multispectral fusion tasks, such as RGB and multispectral image fusion.

Original:  to ensure the convergence of the alternating reverse filtering, there is still a large room to explore alearnable filter function that can strictly satisfy sufficient conditions.
Revised: Second, although we develop the multi-scale Gaussian kernel module to ensure the convergence of the alternating reverse filtering, there is still a large room to explore a learnable filter function that can strictly satisfy sufficient conditions.

Original:  Conclusion
Revised:  Conclusion

Original: In this paper, we presented a simple yet effective alternating reverse filtering network for pansharpening.
Revised: In this paper, we presented a simple yet effective alternating reverse filtering network for pansharpening.

Original: The proposed approach formulates pan-sharpening as a reverse filtering process andcombines classical reverse filtering and deep learning.
Revised: The proposed approach formulates pan-sharpening as a reverse filtering process and combines classical reverse filtering and deep learning.

Original: The classical reverse filtering is unrolled toa network without pre-defined exact priors in an alternating iteration manner.
Revised: The classical reverse filtering is unrolled to a network without pre-defined exact priors in an alternating iteration manner.

Original: Besides, multi-scale
Revised: Besides, multi-scale

Original: Gaussian kernel module is developed to ensure the convergence of the iterative process.
Revised: Gaussian kernel module is developed to ensure the convergence of the iterative process.

Original: Furthermore,the ablation studies verified the effectiveness of the multi-scale Gaussian kernel module.
Revised: Furthermore, the ablation studies verified the effectiveness of the multi-scale Gaussian kernel module.

Original: Extensiveexperimental results on simulated and real-world scenes show that the proposed network significantlyoutperforms the state of the arts.
Revised: Extensive experimental results on simulated and real-world scenes show that the proposed network significantly outperforms the state of the arts.

Original: In the future, we will study the extension to other image fusionproblems such as RGB and multispectral image fusion.
Revised: In the future, we will study the extension to other image fusion problems such as RGB and multispectral image fusion.

Original: Broader Impact
Revised: Broader Impact

Original: This research aims to address the problem of panchromatic and multispectral image fusion, which isa key pre-processing technology overcoming the constraints of hardware before using high-resolutionmultispectral image.
Revised: This research aims to address the problem of panchromatic and multispectral image fusion, which is a key pre-processing technology overcoming the constraints of hardware before using high-resolution multispectral image.

Original: The fused multispectral images are needed as a reference in the field of resourcemonitoring, such as land use planning, ocean development, and urban management, and in the fieldof ecological and environmental protection areas, such as pollution monitoring, vegetation biologyand precision agriculture research.
Revised: The fused multispectral images are needed as a reference in the field of resource monitoring, such as land use planning, ocean development, and urban management, and in the field of ecological and environmental protection areas, such as pollution monitoring, vegetation biology and precision agriculture research.

Original: Despite the many benefits of fused multispectral images, negativeconsequences can still occur in several special environments.
Revised: Despite the many benefits of fused multispectral images, negative consequences can still occur in several special environments.

Original: When there is a case of algorithmfailure, artifacts generated on the fusion image may affect subsequent use and lead to misjudgments. 
Revised: When there is a case of algorithm failure, artifacts generated on the fusion image may affect subsequent use and lead to misjudgments. [Table] Acknowledgements

Revised: This work was supported in part by the National Natural Science Foundation of China under Grant 42271350.

Revised: We gratefully acknowledge the support of MindSpore, CANN and Ascend AI Processor used for this research.

Revised: Checklist

Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Revised: [Yes] (b) Did you describe the limitations of your work?

Revised: [Yes] , see Section 4.4.

Revised: (c) Did you discuss any potential negative societal impacts of your work?

Revised: [Yes] , see Section 5.

Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Revised: [Yes] 2.

Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Revised: [Yes] , See Section 3.2 and Section 3.3.

Revised: (b) Did you include complete proofs of all theoretical results?

Revised: [Yes] , See Section 3.2 and Section 3.3.

Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Revised: [Yes] , Instructions about experimental settings are in Section 4.1, and the URL of the project will be released after the paper is accepted.

Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Revised: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Revised: [No] , the effect of random seed could almost be negligible since we set the same initiation seed during experiments.

Revised: Reproducibility can be guaranteed.

Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Revised: [Yes] 4.

Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Revised: [Yes] (b) Did you mention the license of the assets?

Revised: [Yes] (c) Did you include any new assets either in the supplemental material or as a URL?

Revised: [Yes] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Revised: [Yes] , the data creators claim that they allow and encourage the data for scientific research.

Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Revised: [N/A] , We train and test the framework using the datasets following the prior works.

Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount

Revised: spent on participant compensation?

Revised: [N/A]