Fourier-enhanced Implicit Neural Fusion Network for Multispectral and Hyperspectral Image Fusion

Authors: yu-jie  liang
zihan  cao
shangqi  deng
hong-xia  dou
liang-jian  deng
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-12 16:18:42

Abstract: Recently, implicit neural representations (INR) have made significant strides in various vision-related domains, providing a novel solution for Multispectral and Hyperspectral Image Fusion (MHIF) tasks. However, INR is prone to losing high-frequency information and is confined to the lack of global perceptual capabilities. To address these issues, this paper introduces a Fourier-enhanced Implicit Neural Fusion Network (FeINFN) specifically designed for MHIF task, targeting the following phenomena: The Fourier amplitudes of the HR-HSI latent code and LR-HSI are remarkably similar; however, their phases exhibit different patterns. In FeINFN, we innovatively propose a spatial and frequency implicit fusion function (Spa-Fre IFF), helping INR capture high-frequency information and expanding the receptive field. Besides, a new decoder employing a complex Gabor wavelet activation function, called Spatial-Frequency Interactive Decoder (SFID), is invented to enhance the interaction of INR features. Especially, we further theoretically prove that the Gabor wavelet activation possesses a time-frequency tightness property that favors learning the optimal bandwidths in the decoder. Experiments on two benchmark MHIF datasets verify the state-of-the-art (SOTA) performance of the proposed method, both visually and quantitatively. Also, ablation studies demonstrate the mentioned contributions.  The code can be available at https://github.com/294coder/Efficient-MIF.

Main paper: 
 introduction: 
 Hyperspectral imaging captures scenes across contiguous spectral bands, offering intricate details compared to traditional single or limited-band images, and improving computer vision application accuracy, such as target recognition, classification [47] , tracking, and segmentation [12, 38, 39, 37, 44, 48, 49] . However, practical optical sensors face challenges in balancing spatial resolution and spectral precision. Images with over 100 bands often exhibit lower spatial resolution, while those with fewer † Equal contribution. bands display higher spatial resolution. Efforts for MHIF are underway to fuse high spatial-resolution multispectral images (HR-MSI) with low spatial-resolution hyperspectral images (LR-HSI) to finally obtain high spatial-resolution hyperspectral images (HR-HSI). Actually, MHIF technology could fuse hyperspectral images with multispectral images, extracting information not detectable by HR-MSI to enhance richness and precision. Recent MHIF literature explores model-based approaches [8, 9, 45] and deep learning methods [17, 10, 3, 54] . While model-based methods leverage image priors, challenges persist in obtaining high-fidelity, low-distortion HR-HSI due to the lack of large-scale training datasets. Among deep-learning approaches, CNN-based networks for HR-MSI and LR-HSI tend to be limited and lack interpretability for MHIF tasks and Transformer frameworks [15, 7] address the small receptive field of CNN but bring greater computational overhead.In recent years, implicit representations of 3D scenes have garnered significant attention from researchers. For instance, Neural Radiance Field [43] models 3D static scenes by mapping coordinates to signals through a neural network. Inspired by this, researchers have revisited image representation for 2D tasks. Recent studies [5, 20, 34, 4] have achieved arbitrary-scale super-resolution (SR) by replacing commonly used upsampling layers with local implicit image functions. Though these methods demonstrate superior performance in 2D tasks, they still have some drawbacks. Firstly, INR calculates the RGB values of a queried coordinate based on the relative distances to the surrounding four pixels, treating it as a local operation in space that lacks consideration for global information. Additionally, the MLP-ReLU structure used in traditional INR inherent high-frequency information bias [29] which is challenging to be eliminated during training.To address these issues, we propose implicit fusion functions tailored for the MHIF task as a novel fusion paradigm. We first employ encoders to extract prior information from LR-HSI and HR-MSI, which is then fed into the implicit fusion functions in the form of latent codes. Unlike traditional INR, we transform latent codes into the Fourier domain and simultaneously perform spatial and frequency fusion in a unified network. This approach not only rectifies the high-frequency insensitivity induced by the MLP but also effectively extends the receptive field, encompassing a more comprehensive scope of global information. To integrate spatial and frequency domain representations efficiently, we design a decoder with time-frequency tightness, mapping features on both domains to pixel space. The contributions of this work are three folds:• We define a novel fusion framework based on INR, which innovatively extracts information from the spatial and Fourier domains, effectively enhances the representation ability of high-frequency information, and expands the receptive field.• We propose a new decoder employing a Gabor wavelet activation function to enhance the interaction of INR features. Furthermore, we theoretically prove that the complex Gabor wavelet activation possesses a time-frequency tightness property, which facilitates the decoder in learning the optimal bandwidths.• The proposed network reaches state-of-the-art (SOTA) performance on the MHIF task across two widely used hyperspectral datasets at various fusion ratios. Fig. 1 provides a fair comparison with other SOTA methods. for example, UltraSR [46] incorporates residual networks, merging spatial coordinates and depth encoding. DIINN [26] utilizes a dual-interactive implicit neural network to decouple content and position features, improving decoding capabilities. JIIF [36] proposes joint implicit image functions for multimodal learning, extracting priors from guided images. Regarding activation functions in the MLP, SIREN [34] recommends utilizing periodic activation functions for continuous INR to fit complex signals. On the other hand, WIRE [32] further employs continuous complex Gabor wavelet activation functions to activate non-linearity, focusing more on spatial frequencies. However, there is limited research dedicated to designing INR architectures specifically for the MHIF task. The unique characteristics of hyperspectral images pose challenges for INR networks, in their insensitivity to high-frequency information.Latent Enhancement by Fourier Transform Fourier transform is a commonly used timefrequency analysis technique in signal processing, which converts signals from the time domain to the frequency domain. The Fourier domain has global statistical properties, and in recent years, many works use the Fourier transform to enhance the representation ability of neural networks. For example, FDA [53] proposes exchanging amplitude and phase components in Fourier space between images to enhance and adjust frequency information. FFC [6] introduces a novel convolution module that internally fuses cross-scale information to capture global features in Fourier space. Similarly, GFNet [30] uses 2D discrete Fourier transform to extract features, implements learnable global filtering, and replaces the self-attention layer in Transformer. UHDFour [21] embeds Fourier transform into the image enhancement network to model global information. Together, these studies demonstrate the utility of frequency domain information in improving performance on visual tasks. We exploit the architecture of FeINFN to transform latent codes into the frequency domain, implicitly integrating representations of amplitude and phase components, and enhancing high-frequency injection.Motivation [29] finds that most neural networks exhibit a phenomenon of spectral bias through Fourier analysis. This includes neural networks such as MLP, which tend to learn low-frequency information during the early stages of training and are insensitive to high-frequency information. Moreover, we found this issue occurs in the MHIF task according to an experimental analysis as shown in Fig. 2 (a), where HR-HSI and LR-HSI were concatenated with HR-MSI and fed into a trained encoder to obtain latent codes. These codes were transformed into the frequency domain to visualize the amplitude and phase. It can be observed that the amplitudes from HR-HSI and LR-HSI are very similar, while the phases differ significantly. The phase of HR-HSI should naturally contain more texture than LR-HSI, a hypothesis validated by the visualized phase maps. Based on this finding, we transformed the latent codes into the Fourier domain to separately process amplitude and phase, to enhance the global learning of high-frequency information in the images. 
limitations: 
 Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: The paper discusses the limitations in Appendix A.1, such as assumptions made in the model and potential areas where the results may not be generalized. Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
methodology: 
 In this section, we first present the preliminary of INR and then provide the proposed framework tailored for MHIF task. Subsequently, we elaborate on the implementation details of the composited modules of the proposed FeINFN.Figure 3 : The flowchart of the FeINFN framework which is composed of a spectral encoder E χ , a spatial encoder E ψ , MHIF task-designed spatial and Fourier domains implicit fusion functions, and a pixel space mapping decoder. Please note that I LR is the LR-HSI, I HR is the HR-MSI, I LR up is the bicubic interpolation LR-HSI, and X HR is the HR normalized 2D coordinate map. z spe , z spa , z hp , δx correspond to individual pixel units, A and P represents amplitude and phase, respectively.Neural Radiance Fields [43] is represented by integral construction scenes. The value of a pixel in a certain viewing angle image is regarded as the integral of the characteristics of the sampling point from the proximal end to the far end of the ray. During actual training, the integral needs to be discretized. Extended to 2D image representation [5] , it is sampled pixel by pixel from the vicinity of the query target. Taking the low-resolution (LR) image I ∈ R h×w×3 upsampling to the high-resolution (HR) image Î ∈ R H×W ×3 as an example, the process of generating the RGB values of the target coordinates x q ∈ R 2 can be regarded as interpolation form, expressed as:EQUATIONwhere v q,i ∈ R 4×4×3 is the interpolation pixel of i interpolated by q's surrounding pixels N q ∈ R 4 and w q,i ∈ R signifies the interpolation weight. In the implicit representation of local image features, the weights w q,i = S i /S, where S i represents the area formed by q and i in the diagonal region and S denotes the total area enclosed by the set N q . The interpolation value v q,i is effectively generated by a basis function:EQUATIONwhere ϕ θ is typically an MLP, z i is the latent code generated by an encoder for the coordinates x i , and x q -x i represents the relative coordinates. From the above equations, it can be inferred that the interpolation features can be represented by a set of local feature vectors in the LR domain. Typically, interpolation-based methods [28, 18] achieve upsampling by querying x q -x i in the arbitrary SR task. See more details in [5] .In this work, we propose the FeINFN, which adopts a novel framework for simultaneously performing neural implicit representation in both the spatial and frequency domains to execute the MHIF task. Fig. 3 provides an overview of the proposed framework, designed to fuse LR-HSI I LR ∈ R h×w×S and HR-MSI I HR ∈ R H×W ×s to generate HR-HSI I ∈ R H×W ×S based on a upsampling scale r.Initially, the LR-HSI is fed into encoder E χ to extract spectral features Z spe ∈ R h×w×C . Simultaneously, the concatenated bicubic interpolation LR-HSI I LR up ∈ R H×W ×S and I HR , are fed into encoder E ψ to extract spatial features Z spa ∈ R H×W ×C . Additionally, the pixel's central position is represented as the coordinate point. The coordinate map is normalized into a two-dimensional grid [-1, 1] × [-1, 1], obtaining a HR normalized 2D coordinate map X HR ∈ R H×W ×2 . The extracted Z spe and Z spa , along with the 2D coordinates of I HR , are forwarded to Spatial-Frequency Implicit Fusion Function (Spa-Fre IFF), outputting spatial domain features E s ∈ R H×W ×S and frequency domain features E f ∈ R H×W ×S . The E s and E f as inputs to a pixel space mapping decoder which generates the residual image I HR r ∈ R H×W ×S . Finally, the residual image I HR r is combined with the bicubicly upsampled image I LR up via element-wise addition, yielding the ultimate fusion image I.Analogous to local implicit representation functions [5, 20, 34, 4] , the initial step involves extracting latent code representations. For the MHIF task, we address the challenges of both upsampling and fusion simultaneously, employing implicit neural representations as the solution.The INR encoders try to extract spatial and spectral latent codes Z spa ∈ R H×W ×C , Z spe ∈ R h×w×C : one is extracted from I LR , serving as the carrier for spectral information; the other is encoded from the concatenation of I LR up and I HR , aiding in spatial information during the fusion process. This process can be denoted as:EQUATIONwhere E χ is the spectral encoder parameterized by χ, E ψ is the spatial encoder parameterized by ψ, and Cat(I LR up , I HR ) denotes the concatenation along the channel dimension. In practice, we utilize EDSR [23] as INR encoder networks.To address the mentioned issues 2, we propose Spatial-Frequency Implicit Fusion Function, dubbed Spa-Fre IFF which is a dual-branch fusion function and utilized for computing the fusion feature of Z spe and Z spa in the spatial and frequency domains, respectively. Given a queried HR coordinate x q ∈ X HR of a pixel unit q, Spa-Fre IFF estimates spatial feature vector ε s ∈ R 1×1×S (ε s ∈ E s ) and frequency feature vector ε f ∈ R 1×1×S (ε f ∈ E f ) as follows:EQUATIONwhere z spe ∈ R 1×1×C represents the spectral latent code vector corresponding to x q , and z spa ∈ R 4×4×C is the spatial latent code vector. δx denotes the set of local relative coordinates, expressed by the following formula:EQUATIONwhere x q,i refers to the coordinates most proximate to the query coordinate x q , representing the four corner pixels closest to q in the HR space.The Spatial Implicit Fusion Function aims to leverage the powerful representation capabilities of INR to achieve implicit fusion in the spatial domain, as shown in Fig. 3 (see branch "Spatial Domain"). Specifically, we employ high-pass operators H to filter the spectral latent codes, as a complement to the high-frequency information on the spectrum:EQUATIONwhere z hp ∈ R 1×1×C represents the high-frequency latent code of I LR . Also, we suggest frequency encoding for relative positional coordinates as follows:EQUATIONwhere L is a hyperparameter, in practice, we set L to 10. Additionally, leveraging the graph attention mechanism [36] , we parameterize the solution for interpolation weights w q,i ∈ R 1×S , and the implicit fusion function simultaneously outputs fusion interpolation values v q,i ∈ R 4×4×S and interpolation weights w q,i . The implicit fusion function is specifically expressed as:EQUATIONwhere ϕ θ is an MLP parameterized by θ. The weights used for interpolation need to pass through a softmax function, obtaining normalized weights w q,i . The spatial implicit fusion interpolation, as shown in Eq. ( 1), yields the fused spatial feature ε s ∈ R 1×1×S and can be described as follows:EQUATIONFrequency Implicit Fusion Function From Fig. 2 (a), we observed characteristics in the frequency features between LR-HSI and HR-HSI. Hence, we design a frequency implicit fusion function to express global features continuously in the Fourier domain. Notably, directly applying static kernel convolution in the frequency domain would only enhance a specific frequency range, which is inappropriate for the fusion task. However, by learning feature content to generate weights, INR can be seen as a dynamic interpolation method in continuous space, adaptively enhancing information in the frequency domain without overly altering the frequency distribution. Therefore, introducing INR into the Fourier domain is reasonable. Since amplitude and phase exhibit different forms, as shown in Fig. 2 (a), we handle them separately.With the considerations mentioned above, as illustrated in Fig. 3 (see branch "Fourier Domain"), we initially employ FFT to transform latent codes z spe and z spa from the spatial domain to the frequency domain, obtaining f spe ∈ R 1×1×C and f spa ∈ R 4×4×C . After the transformation, we further obtain amplitude components A(f spe ) and A(f spa ), as well as phase components P(f spe ) and P(f spa ).For the amplitude, as shown in Fig. 2 (b), the amplitude distribution of LR-HSI and HR-HSI are very similar, and the non-point-wise convolution (e.g. Conv 3 × 3) causes an issue of spectrum leakage, confusing channel information. In contrast, point-wise convolution does not span multiple locations in the frequency domain and has no overlap allowing it to capture information across channels effectively. Thus the fusion function for amplitude components is more suitable when applying point-wise convolution:EQUATIONwhere w A q,i ∈ R 1×S and v A q,i ∈ R 4×4×S are the weights and interpolated values for the corresponding amplitude component, and ϕ A α is a simple network composed of two layers of point convolutions parameterized by α. Similar to operations in the spatial domain, implicit fusion interpolation is performed after obtaining interpolated values v A q,i and the normalized weights w A q,i :EQUATIONwhere A ′ f ∈ R 1×1×S is the integrated amplitude component. For the phase, which encapsulates information such as texture details, LR-HSI and HR-HSI often have different phase information. It is known that point convolutions fail to capture sufficient spatial representations. Therefore, we use a 3 × 3 convolution to learn phase information. Additionally, small changes in the frequency domain may result in significant variations in the spatial domain. We still consider using the form of INR interpolation for phase learning. The handling of the phase components P(f spe ) and P(f spa ) are formally similar to Eqs. ( 10) and ( 11):EQUATIONThe simple network ϕ P β consists of two layers of 3 × 3 convolutions parameterized by β. P ′ f ∈ R 1×1×S represents the integrated phase component. Finally, IFFT is applied to map the frequency features A ′ f and P ′ f back to the image space, obtaining the frequency domain feature ε f ∈ E f . Since in frequency space, one frequency point may correspond to multiple pixels at different positions in the spatial domain, the receptive field of INR in the frequency domain is enlarged in the spatial domain. After obtaining the spatial feature map and frequency domain feature map, it is essential to consider how to integrate them seamlessly. Firstly, our decoder needs to have dual input and interactive capabilities. Secondly, it is necessary to focus on representing images in the spatial-frequency domain. With this in mind, we introduce the complex Gabor wavelet activation function with good time-frequency tightness and propose the Spatial-Frequency Interactive Decoder (SFID). Specifically, SFID consists of three layers, taking spatial and frequency domain features as inputs. The outputs I HR r and I HR up contribute to the final fused image I. The decoding process is illustrated in Fig. 4 . The complex Gabor wavelet function is defined as:EQUATION)where ω 0 is the center frequency in the frequency domain, υ 0 is a constant that is considered as the standard deviation of the Gaussian function, and x is a vector in the time (or spatial) domain. In what follows, we provide a theorem below that this Gabor wavelet activation has time-frequency tightness [1] , which is helpful for the decoder's information interaction. Theorem 1. The complex Gabor wavelet activation in Eq. ( 13) has the time-frequency tightness property (more preliminary can be found in [1] ). Moreover, from the perspective of signal spectrum analysis, this activation helps the decoder learn the optimal bandwidths. The Fourier transform of the Gaussian term e -|v0x| 2 remains a Gaussian function, and its bandwidth in the frequency domain is influenced by υ 0 . Due to the characteristics of the Gaussian function in the frequency domain, G(x) is mainly concentrated around ω = ω 0 . Combining the narrow-bandwidth properties in both time and frequency domains, we can apply the uncertainty principle to demonstrate the timefrequency tightness of the complex Gabor function:EQUATION)where υ 0 is the time-domain bandwidth, and |ω 0 | is the frequency-domain bandwidth. In practical training, we provide an initial set of bandwidths and allow the network to learn the optimal bandwidths, which concludes the proof.As depicted in Fig. 5 , the frequency response of the decoder with Gabor wavelet activation closely approximates the optimal bandwidth. Moreover, the decoder with Gabor activation achieves consistency with GT in frequency, demonstrating rapid frequency alignment. Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? 
experiments: 
 Datasets To evaluate the efficacy of our model, we conducted experiments using the CAVE and Harvard datasets. The CAVE dataset comprises 32 Hyperspectral Images (HSIs) with 31 spectral bands spanning from 400 nm to 700 nm at 10 nm intervals. We randomly selected 20 images for training and used the remaining 11 for testing. The Harvard dataset consists of 77 HSIs depicting indoor and outdoor scenes, covering the spectral range from 420 nm to 720 nm. We standardized the data by cropping the upper left sections of 20 Harvard images, with 10 for training and the rest for testing. The simulation of data can be found in Appendix.We implement the proposed method FeINFN with Pytorch [27] on a workstation with an Intel I9 CPU and two 3090 GPUs. The optimizer is chosen as AdamW [19] and we use a Cosine anneal learning rate scheduler. The base channel number of the encoder is 128, that of the proposed implicit fusion function is 32 and in the decoder, the channel number is 31.In this section, we evaluate the effectiveness of FeINFN on the CAVE dataset and compare it with five traditional methods and some state-of-the-art deep learning-based approaches. As shown in Tab. 1 on the left, our method achieves optimal performance in the tasks of ×4 in all metrics. In the ×4 experiment, compared to currently leading methods such as DSPNet [35] , 3DT-Net [25] , and MogDCN [10] , our approach demonstrates improvements in PSNR by 1.29dB/1.09dB/0.84dB, respectively. Notably, our method exhibits even more pronounced superiority in the ×8 experiment, showcasing good generalization across various resolutions. To illustrate the advantages of our method, we provide visual comparisons in Fig. 6 , including close-ups and error maps to highlight specific details. Our fusion results closely match the ground truth, achieving the best quality. In comparing error maps, the darker colors indicate closer proximity to the original image. In contrast to other excellent methods, the error maps of FeINFN distinctly exhibit superior restoration effects on details.Results on Harvard Dataset In Tab. 1, the right columns present the comparison results of our FeINFN with other methods on the Harvard dataset at scale factors 4. Our method performs exceptionally well, with only SAM being slightly surpassed by 3DT-Net [25] . FeINFN exhibits significant gains in PSNR/ERGAS metrics compared to the current state-of-the-art [16] , with improvements of 0.77dB/0.09, respectively. The results with a scale factor of 8 can be found in Appendix. As depicted in Fig. 1 , our model outperforms others, highlighting the crucial role of FeINFN's continuous representation capability in high-scale factor scenarios. To better visualize the performance gap, Fig. 6 illustrates the fused images and error maps, confirming that our FeINFN maintains high fidelity in recovering the texture details of the images. Upsampling Methods Implicit image representation can be seen as an advanced interpolation algorithm, offering additional spatial information and parameterized weight generation. In this section, we compare INR with other upsampling methods. We replace INR with pixel-shuffle [33] and traditional CNN interpolation methods, presenting a comparative analysis. As seen in Tab. 2, our approach outperforms other methods in MHIF tasks.To assess the dual-domain model's efficacy, we performed model reduction, preserving spatial and Fourier domains independently. As shown in Tab. 3, FeINFN excels by using both spatial and Fourier domains concurrently, underscoring the positive impact of Fourier domain integration on overall network performance.Spectral deviation occurs during training, where the network tends to prioritize low-frequency information, capturing high-frequency details only in later stages. To validate our resolution of this issue, we remove the "Fourier Domain" from Spa-Fre IFF, or retain it, and the corresponding training data is illustrated in Fig. 7 . Our FeINFN, which incorporates Fourier domain fusion, leads to faster PSNR convergence and overall higher efficiency. The visual comparison of high-frequency details in "chart and stuffed toy" from the cave dataset at 80k iterations further supports the significant improvement achieved with our results.Decoder with Different Nonlinear In this section, we evaluate the impact of different activation functions in SFID, aiming to match SFIFF. Our dual-input decoder incorporates a complex Gabor wavelet activation function to facilitate the fusion of spatial and frequency domain features.Through experiments, we replaced the Gabor wavelet activation with other activations, presenting the results in Tab. 4. The findings distinctly demonstrate the enhanced fusion quality achieved with the complex Gabor wavelet activation. This emphasizes the critical role of wavelet activation in promoting robust and reliable learning in SFID. 
conclusion: 
 Inspired by the distinct behaviors of LR-HSI and HR-HSI in the Fourier domain, we introduce a novel Fourier-enhanced Implicit Neural Fusion Network (FeINFN) based on INR. Through Fourier transformation, latent features are converted into the frequency domain, allowing the modeling of frequency components to enrich high-frequency information in images. Additionally, we propose a spatial-frequency decoding module, achieving a unified representation of both spatial and frequency domains using a time-frequency-tight activation function. Thanks to the unique design of our network, it outperforms state-of-the-art methods in MHIF with appealing efficiency. We desire that our work will inspire future research on frequency fusion-based MHIF methods.Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [Yes] Justification: The datasets are publicly accessible, and the code will be released soon.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
acknowledgement: 
 This work is supported by the National Natural Science Foundation of China under Grants 12271083, 12171072 and Natural Science Foundation of Sichuan Province under Grants 2023NSFSC1341.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: In Sec. 4, we present the datasets and implementation details, and the simulation details of the datasets can be found in Appendix A.6.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [No] Justification: The paper does not involve error bars or experiments concerning statistical significance.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: The models compared in the paper have either received author permission or are publicly available. The datasets are public, and original papers are cited meticulously.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: The models compared in the paper have either received author permission or are publicly available. The datasets are public, and original papers are cited meticulously.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [NA]Justification: The paper does not release new assets.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA]Justification: The paper does not involve crowdsourcing or research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing or research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?Answer: [NA]Justification: The paper does not involve crowdsourcing or research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing or research with human subjects.• Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: This supplementary material provides additional insights into the background, methodologies, and experimental details outlined in our paper. It includes limitations and broader impact, experiments compute resources, details on CNNs in MHIF, 2D Fourier transform, an elucidation of the global receptive field of convolution operations within the Fourier Domain , a description of data simulation, and quality metrics. Furthermore, we present a comprehensive comparison of all methods applied to the CAVE and Harvard datasets with a scale factor of 8. This includes an ablation study, affirming the efficacy of incorporating the Fourier domain on the CAVE(×8). The provided information aims to enhance the reader's understanding of the intricacies involved in our research and its practical applications.Limitations This study has certain limitations that should be acknowledged. One primary limitation is the unavailability of ground truth (GT) data in real-world settings for the task of multispectral and hyperspectral image fusion. Due to this constraint, all datasets used in our experiments are simulated. The detailed steps for data simulation can be found in Appendix A.6. This reliance on simulated data may affect the generalizability of our results to real-world scenarios. Consequently, while our methods show promising performance in experiments, their effectiveness in practical applications remains to be fully validated. However, this limitation is a challenge faced by the entire field, not unique to our work. Surveys [40, 31] in the field of multispectral and hyperspectral image fusion highlight this common issue and discuss the need for improved data simulation methods and benchmarks.Broader Impact This research addresses the task of multispectral and hyperspectral image fusion, which is crucial for enhancing the spatial resolution of hyperspectral images while preserving their spectral fidelity. The resulting high-resolution hyperspectral images (HR-HSI) are invaluable for various applications, such as resource monitoring, environmental management, and urban planning. In the environmental domain, fused images aid in pollution tracking, vegetation analysis, and precision agriculture, contributing to sustainable practices and environmental protection. These fused images facilitate more accurate and detailed analysis in these fields, potentially leading to better-informed decisions and more effective resource management. Despite these benefits, there are potential negative consequences to consider. Image fusion is a low-level task that significantly impacts subsequent image-processing steps. If the fusion process fails, resulting in distorted HR-HSI, it may adversely affect follow-up tasks and analyses, leading to incorrect conclusions or misguided decisions. Thus, ensuring the robustness and accuracy of the image fusion algorithm is critical to mitigating these risks.Our experiments were conducted on a workstation equipped with an Intel 12th Gen i7-12700K processor, two NVIDIA RTX 3090 GPUs, and 128GB of memory. This setup provided sufficient computational power to handle the intensive tasks involved in multispectral and hyperspectral image fusion.In recent years, CNN-based methods have exhibited significant success in the domain of multispectral and hyperspectral image fusion (MHIF). Their efficacy lies in their adeptness to extract high-level features from input data through end-to-end learning. SSRNet [52] leverages three distinct convolutional modules, w.r.t, a fusion module, a spatial edge module, and a spectral edge module, excelling in image reconstruction by associating a spatial-spectral loss function, contributing to robust learning outcomes. Similarly, ResTFNet [24] adopts residual structures and a two-stream fusion network, drawing inspiration from the extensive application of ResNet [13] in super-resolution image processing. In contrast, the MHF network [50] incorporates a well-explored linear mapping that connects HR-HSI to HR-MSI and LR-HSI, facilitating ease of interpretation. MoG-DCN [10] employs a dedicated subnet to approximate the decomposition matrix and conducts hyperspectral image super-resolution using DCN-based image regularization, leveraging prior knowledge of HSI. For the simultaneous extraction of spatial and spectral information and the acquisition of high-quality details, HSRNet [16] integrates spatial and channel attention modules, enhancing the fusion performance. However, due to the model scaling and limited convolutional receptive field, the CNN-based models still struggle to obtain satisfactory results for the MHIF task.Fourier transform is commonly employed in digital signal processing [2] , aiming to convert signals from the time domain to the frequency domain. Through this domain transformation, previously imperceptible features often become observable. For two-dimensional images, the Fourier transform converts the signal from the spatial domain to the frequency domain, enabling the transformation of images into spectrograms in the frequency domain. Given a single-channel image X ∈ R H×W , the Fourier transform translates it into the Fourier space as the complex component Y ∈ C H×W . This process can be represented as follows:EQUATIONwhere (h, w) denotes the coordinates of x in the spatial space, and (u, v) represent the coordinates of Y in the Fourier space. The Fourier space is spanned by complex orthogonal basis functions, and each complex frequency component can be expressed as amplitude A(Y(u, v)) and phase P(Y(u, v)) components:EQUATIONEQUATIONwhere ℜ(Y) and ℑ(Y) respectively represent the real and imaginary parts. For multi-channel images, in the utilization of the Fourier transform, we perform individual Fourier calculations for each channel. Additionally, the Fourier transform is a reversible transformation, enabling bidirectional conversion between the original signal and the transformed signal, we denote F -1 as the Fourier inverse transform. The proposed architecture takes LR-HSI and HR-MSI pairs (I LR , I HR ) as input, with the training ground-truth (GT) being HR-HSI. However, due to the unavailability of HR-HSI as a reference, a simulation phase is necessary. In our experiments using the CAVE dataset, we cropped 20 training images, generating 3920 overlapping patches of size 64 × 64 × 31. These patches serve as HR-HSI (ground truth) patches. To simulate appropriate LR-HSIs, we applied a 3 × 3 Gaussian blur kernel (a) (b) (c) (d) (e) (f) (g) (h) (i) (j) (k)(a) (b) (c) (d) (e) (f) (g) (h) (i) (j)We compare our method with other methods using different image quality metrics to validate the image fusion capability of our model, including the Spectral Angle Mapper (SAM) [51] , the Erreur Relative Globale Adimensionnelle de Synthèse (ERGAS) [41] , the Peak Signal-to-Noise Ratio (PSNR) [14] , and the Structural SIMilarity (SSIM) [42] .PSNR evaluates the spatial quality of each band in the reconstructed HR-HSI. It is calculated as follows:EQUATIONHere, I i ∈ R H×W and I i ∈ R H×W represent the i-th band of I ∈ R H×W ×B and I ∈ R H×W ×B , respectively. The PSNR function is defined as:EQUATIONwhere MSE (Mean Square Error) between I i and I i , and max(•) is the maximum value of I i .SAM measures the spectral distortion of each hyperspectral pixel in the reconstructed HR-HSI. It is given by:EQUATION)where cos -1 is the arccosine function, I i ∈ R B×1 and I i ∈ R B×1 are the spectra of the i-th pixel of I and I, respectively, || • || 2 is the ℓ 2 norm, and • T denotes the transpose.ERGAS measures the global statistical quality of the reconstructed HR-HSI, taking into account the ratio of the ground sample distances between HR-MSI and LR-HSI. It is formulated as:EQUATION)where c is the scaling factor, and µ 2 I i is the square of the mean value of I i . SSIM is used to assess the structural differences between GT and the reconstructed HR-HS, incorporating both luminance and structural contrast functions. The SSIM function is defined as:EQUATION)where, B is the number of bands, and I and I are sets containing I i and I i for i = 1 to B, respectively. µ I i and µ I i represent the mean values of I i and I i , while σ 2 I i and σ 2 I i denote their variances. The term σ I i I i indicates the covariance between I i and I i . Constants C 1 and C 2 are fixed values.Higher PSNR values indicate better performance, while lower SAM and ERGAS values signify higher quality of the reconstructed HR-HSI. SSIM values range from -1 to 1, with values closer to 1 indicating better quality. Ideally, PSNR should be infinite, SAM and ERGAS should be zero, and SSIM should be one.To evaluate FeINFN's performance, we compare it with MHIF methods on the CAVE and Harvard datasets. The bicubic-interpolated result of the upsampled LR-HSI in Tab. 1 serves as our baseline. Various model-based techniques, including the CSTF-FUS [22] , LTTR [9] , LTMR [8] , and IR-TenSR [45] approaches, are considered. Additionally, we compare our approach with various deep learning methods, such as SSRNet [52] , ResTFNet [24] , HSRNet [16] , MoGDCN [10] , Fusformer [15] , and DHIF [17] , PSRT [7] , 3DT-Net [25] , DSPNet [35] , MIMO-SST [11] . We compare our method with other methods using different image quality metrics to validate the image fusion capability of our model, including SAM [51] , ERGAS [41] , PSNR [14] , and SSIM [42] .Due to space constraints in the main text, we present a more detailed comparison of our FeINFN with other methods on four metrics for the CAVE dataset and the Harvard dataset in the supplementary material. As shown in Tab. 5, FeINFN demonstrates the best overall performance. While it ranks second in SAM on the CAVE dataset, it maintains the optimal results for other metrics.In the main text, we described our ablation experiments to investigate the effectiveness of incorporating the Fourier domain. Additionally, we were interested in assessing its efficacy at larger scale factors. Therefore, we applied pruning to the model on the CAVE (×8) dataset to validate its effectiveness. The experimental results, as shown in Tab. 6 , indicate that the model performs best when incorporating the Fourier domain operation, aligning with our hypothesis. This provides robust evidence supporting the enhancement of network performance through INR in the Fourier domain. NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 


Decision: The paper proposes a Fourier-enhanced implicit neural fusion network specifically designed for the multispectral and hyperspectral image fusion task. The paper identifies the unique characteristics of the amplitude and phase of the latent codes in the high-frequency and low- frequency components, and proposes to enhance high-frequency details and expand the receptive field from the Fourier domain.  Extensive experiments are theoretical analysis are provided in the submission and the rebuttal for validation. After the rebuttal, all the reviewers give positive recommendation for this submission.

Reviews: review: paper_summary: paper_summary: This paper proposes a quite interesting hyperspectral and multispectral image fusion framework via implicit representation. Moreover, the authors introduce the Fourier transformation to decouple the amplitude and phase domain.
strength_weakness: strength_weakness: 1.	The proposed Fourier decomposition seems to only result in very limited performance improvement. The results from table 3 and figure 7 seem not consistent.
2.	Actually, such implicit representation is more plausible to work in a self-regularized overfitting manner than current train-test setting. Authors can make further design or give a unsupervised version.
3.	Authors should also compare the computational cost of different methods.
4.	Authors should also evaluate the performance with different SR ratios.
questions: questions: Improvement of Fourier design.
Unsupervised training manner. 
Computational cost. 
Different SR ratios.
As the weakness Sec.
limitations: limitations: Please refer to weakness sec.
paper_summary: paper_summary: The paper proposes a novel Fourier-enhanced Implicit Neural Fusion Network (FeINFN) specifically designed for the Multispectral and Hyperspectral Image Fusion task. The paper identifies the unique characteristics of the amplitude and phase of the latent codes in both HRHSI and LRHSI, and proposes to enhance high-frequency details and expand the receptive field from the Fourier domain. Additionally, the paper introduces a new decoder to achieve better interaction between spatial domain features and frequency domain features.
strength_weakness: strength_weakness: 1.	What does 'current neural network-based methods are insensitive to high-frequency information' mean, and how does the proposed model demonstrate sensitivity to high-frequency information?
2.	As mentioned in Section 3.4, the Spatial Implicit Fusion Function utilizes frequency encoding in the spatial domain. Does the Frequency Implicit Fusion Function also utilize frequency encoding in the frequency domain?
3.	Should the vertical axis scales of (c), (d), and (e) in Figure 5 be consistent? In my understanding, at least (c) and (d) should be consistent to demonstrate that the ground truth (GT) frequencies and the frequencies captured using Gabor activation are similar.
4. This paper proposes a new fusion architecture based on INR. It would be even better if there were discussions and analyses regarding network complexity, such as comparisons in spatial complexity with Transformer methods mentioned in section I.

5. There are some editing errors: the double quotes are displayed incorrectly in lines 307 and 310 as “Fourier Domain" and “chart and stuffed toy".

6. In line 226, “I” should correspond to elsewhere in the article.
questions: questions: See weaknesses
limitations: limitations: N/A
paper_summary: paper_summary: The paper introduces a novel Fourier-enhanced Implicit Neural Fusion Network .The core innovation lies in the integration of Fourier transformations within an Implicit Neural Representation framework to address the loss of high-frequency information—a common limitation in existing INR approaches. The effectiveness of FeINFN is demonstrated through experiments on two benchmark MHIF datasets, showing superior performance in terms of PSNR and other metrics compared to state-of-the-art methods.
strength_weakness: strength_weakness: 1. Some sections of the paper are overly technical and dense, which might be challenging for readers unfamiliar with the specific domain of Fourier transforms and neural networks.

2. In Table 3, the description of S and F in the last row is incorrect and should be indicated with two check marks. This seems to be an editing error.

3. In the ablation experiments section of the paper, tables 2, 3, and 4 do not provide the corresponding parameter quantities. If available, could you please provide them? Ablation experiments need to compare models under similar parameter conditions.
questions: questions: 1. Network Complexity: The introduction of Spa-Fre IFF and SFID seems to increase the model complexity. Could the authors comment on the computational efficiency of their proposed method compared to the baselines?

2. In the explanation of the compound Gabor wavelet activation function in the article, Theorem 1 mentions optimal bandwidth. What does this refer to, and how does it affect the activation of features?

3. The abstract of the paper claims that the proposed model achieved state-of-the-art performance on two datasets. For the Harvard x4 experiments, how can the fact that the SAM and SSIM metrics did not reach SOTA be explained?

4. “frequency point may correspond to multiple pixels at different positions in the spatial domain”， Why does transforming into the Fourier domain amplify the frequency domain? I'm a bit unclear on this point.
limitations: limitations: The authors have provided reasonable Limitations and Broader Impact.
paper_summary: paper_summary: The paper provides a novel INR-based fusion framework tailored for the MHIF task, effectively capturing high-frequency details and global information through innovative architectural components. The key contributions are transforming the latent features into the frequency domain to enhance high-frequency information and introducing a complex Gabor wavelet activation function to strengthen feature representation and fusion.  Experiments on two MHIF datasets demonstrate the state-of-the-art performance achieved by the proposed method, outperforming existing approaches both visually and quantitatively.
strength_weakness: strength_weakness: - Complexity: Some parts of the methodology, particularly the theoretical proofs and implementation details, are complex and may be challenging for readers to fully grasp without additional explanation.
- Quality Metrics: The paper does not seem to provide the meaning and calculation methods of the four metrics (PSNR, SAM, ERGAS, SSIM).
- Experimental Section: Besides the two benchmark MHIF datasets used in the experiments, how does the proposed method perform in other image fusion tasks or datasets? It would be better if there were experiments on other datasets, but it's not essential.
questions: questions: 1. How does Fourier transformation within the INR framework specifically contribute to observed performance improvements? What are the practical benefits of frequency-domain features?
2. After reviewing the proof of Theorem 1, I don't understand why finding the optimal bandwidth leads to better representation. Could you provide further explanation, especially regarding Fig. 5.
3. For the PSNR curve in Figure 7, on which dataset was this experiment conducted? Was it on the training set, validation set, or test set? The paper does not explicitly specify this.
4. Regarding Fig. 6, were the errors normalized? The more the error maps lean towards red, the greater the difference between the pixel values and the ground truth. Is my understanding correct? The differences between the close-up images seem very small to the naked eye, and I can't discern any differences between the images in the first and third rows.
limitations: limitations: The authors have included a discussion on the limitations of their work in Section A.1, covering the data simulation, and potential societal impacts.

Comments: title: Comments on the rebuttal
Supplement about the complete version of  tables in rebuttal
comment: The response addressed most of my concerns. Thus, I will raise the score to accept.
The response addressed all my comments well. Therefore, I'll raise the score to ACCEPT.
**A3.**  Tab 2,3 updates as shown below：
|Methods|PSNR ($\uparrow$)|SAM ($\downarrow$) |ERGAS ($\downarrow$) |SSIM ($\uparrow$)|Params|
|:-:|:-:|:-:|:-:|:-:|:-:|
|Bilinear|52.23$\pm$4.40|1.92$\pm$0.60|1.03$\pm$0.86|0.997$\pm$0.0021|3.119M|
|Bicubic|52.22$\pm$4.31|1.95$\pm$0.61|1.02$\pm$0.82|0.997$\pm$0.0021|3.119M|
|Pixel Shuffle|52.26$\pm$4.37|**1.90$\pm$0.59**|1.02$\pm$0.85|0.997$\pm$0.0022|3.057M|
|Our|**52.47$\pm$4.10**|1.91$\pm$0.59|**0.98$\pm$0.74**|**0.998$\pm$0.0015**|3.165M|

|$\mathcal{S}$/$\mathcal{F}$|PSNR ($\uparrow$)|SAM ($\downarrow$)|ERGAS ($\downarrow$)|SSIM ($\uparrow$)|Params|
|:-:|:-:|:-:|:-:|:-:|:-:|
|$\checkmark$/✘|52.11$\pm$4.22|1.95$\pm$0.59|1.04$\pm$0.82|0.998$\pm$0.0017|2.869M|
|✘/$\checkmark$|47.86$\pm$3.42|3.49$\pm$1.30|1.67$\pm$1.13|0.995$\pm$0.0020|2.940M|
|$\checkmark$/$\checkmark$|**52.47$\pm$4.10**|**1.91$\pm$0.59**|**0.98$\pm$0.74**|**0.998$\pm$0.0015**|3.165M|



**A6.**  Detailed results on the Chikusei dataset and Pavia Center dataset are as follows:
| |Chikusei $\times 4$| | | | |Pavia $\times 4$| | | | |
|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|
|**Methods**|**PSNR ($\uparrow$)**|**SAM ($\downarrow$)**|**ERGAS ($\downarrow$)**|**SSIM ($\uparrow$)**|**Params** |**PSNR ($\uparrow$)**|**SAM ($\downarrow$)**|**ERGAS ($\downarrow$)**|**SSIM ($\uparrow$)**|**Params**|
|Bicubic|33.35|4.00|7.65|0.815|-|26.65|7.07|8.46|0.614|-|
|CSTF-FUS|35.40|5.40|7.88|0.844|-|30.93|11.08|5.74|0.791|-|
|LTTR|37.86|3.68|6.27|0.917|-|31.15|6.58|5.50|0.801|-|
|LTMR|41.21|2.98|4.84|0.950|-|32.33|6.35|5.10|0.820|-|
|IR-TenSR|36.00|5.12|7.86|0.868|-|30.87|6.81|5.82|0.783|-|
|ResTFNet|42.33|2.48|3.93|0.950|2.471M|33.56|4.99|4.16|0.885|2.448M|
|SSRNet|42.36|2.35|3.92|0.951|0.446M|33.20|5.36|4.36|0.876|0.282M|
|HSRNet|42.01|2.33|3.95|0.947|0.633M|32.17|5.60|4.60|0.867|2.061M|
|MogDCN|42.21|2.27|3.76|0.936|6.840M|33.84|4.61|4.07|0.889|7.202M|
|Fusformer|43.37|2.03|3.49|0.959|0.504M|35.31|4.33|3.37|0.924|0.539M|
|DHIF|43.69|1.94|3.33|0.960|22.462M|35.30|4.36|3.35|0.924|38.785M|
|PSRT|43.48|2.01|3.47|0.961|0.303M|34.86|4.47|3.54|0.916|0.288M|
|3DT-Net|43.53|2.03|3.46|**0.963**|3.464M|35.10|4.44|3.35|**0.927**|3.482M|
|DSPNet|43.55|2.03|3.44|0.960|6.138M|35.47|4.26|**3.30**|**0.927**|6.115M|
|BDT|43.25|2.09|3.44|0.955|3.263M|34.55|4.66|3.70|0.904|3.056M|
|MIMO-SST|43.36|2.09|3.48|0.958|4.983M|35.37|4.48|3.34|0.922|5.227M|
|**Proposed**|**43.88**|**1.91**|**3.30**|**0.963**|3.488M|**35.51**|**4.15**|**3.30**|**0.927**|3.399M|
Thanks for the responses. My concerns are resolved. I'll raise the score to WA.