Spatial-Spectral Binarized Neural Network for Panchromatic and Multi-spectral Images Fusion

## Abstract

Abstract Remote sensing pansharpening aims to reconstruct spatial-spectral properties during the fusion of panchromatic (PAN) images and low-resolution multi-spectral (LR-MS) images, finally generating the high-resolution multi-spectral (HR-MS) images. Although deep learning-based models have achieved excellent performance, they often come with high computational complexity, which hinder their applications on resource-limited devices. In this paper, we explore the feasibility of applying the binary neural network (BNN) to pan-sharpening. Nevertheless, there are two main issues with binarizing pan-sharpening models: ( i ) the binarization will cause serious spectral distortion due to the inconsistent spectral distribution of the PAN/LR-MS images; ( ii ) the common binary convolution kernel is difficult to adapt to the multi-scale and anisotropic spatial features of remote sensing objects, resulting in serious degradation of contours.
To address the above issues, we design the customized spatial-spectral binarized convolution (S2B-Conv), which is composed of the Spectral-Redistribution Mechanism (SRM) and Gabor Spatial Feature Amplifier (GSFA). Specifically, SRM employs an affine transformation, generating its scaling and bias parameters through a dynamic learning process. GSFA, which randomly selects different frequencies and angles within a preset range, enables to better handle multi-scale and-directional spatial features.
A series of S2B-Conv form a brand-new binary network for pan-sharpening, dubbed as S2BNet. Extensive quantitative and qualitative experiments have shown our high-efficiency binarized pan-sharpening method can attain a promising performance. The code link is https://github.com/Ritayiyi/S2BNet .

## 1 Introduction

With the increasing demand of earth observation and monitoring, existing optical satellites (e.g., GaoFen-2, QuickBird) can simultaneously record bundled low-resolution multispectral (LR-MS) and high-resolution panchromatic (PAN) images from the same scene. Due to physical limitations of existing satellite sensors, the recorded LR-MS image usually includes rich spectral property but relatively sparse spatial property, while their corresponding PAN image contains abundant spatial property but sparse spectral property. Since the remote sensing image with rich spatial-spectral properties, i.e., high-resolution multispectral (HR-MS) image, are crucial for practical applications Li et al. (2022); Han et al. (2024); Asokan & Anitha (2019), pansharpening technique, which could obtain the HR-MS image by reconstructing spatial-spectral properties during the fusion of the recorded PAN and LR-MS images, has been widely explored Xing et al. (2023).

Existing state-of-the-art (SOTA) pan-sharpening methods are based on deep learning. Convolutional neural network (CNN) and Transformer Wu et al. (2022); Tan et al. (2024); Li et al. (2023a) have been employed as powerful models to reconstruct spatial-spectral properties for target HR-MS images. For example, SRPPNN Cai & Huang (2020) performs the pan-sharpening learning using a deeper CNN network architecture implemented residual learning. Zhou et al. propose a series of Transformer-based algorithms push the performance boundary again Zhou et al. (2022b). Although superior performance is achieved, these CNN-/Transformer-based methods require powerful hardwares with abundant computing and memory resources, such as costing 439.38M in HyperTransformer Bandara & Patel (2022b). This motivates us to reduce the memory and computational burden of pan-sharpening methods while preserving the performance as much as possible so that the algorithms can be deployed on resource-limited devices.

To improve the efficiency of deep neural networks, many network compression techniques are proposed, including network quantization Qin et al. (2020a), parameter pruning Wang & Fu (2022) and knowledge distillation Wang et al. (2022). Among these approaches, Binarized neural network (BNN) stands out as an extreme case of network quantization, which binarizes both weights and activations into only $1$-bit. In particular, the foundation of BNNs lies in their pure logical computations, primarily XNOR and bit-count operations. This makes them highly energy-efficient, particularly for embedded devices Zhang et al. (2024). However, directly applying model binarization for pan-sharpening algorithms may face several challenges. (i) The density and distribution of the spectra within the LR-MS image are different, similarly, the spectral density and distribution between LR-MS and PAN images are also different. During the fusion of LR-MS and PAN images, equally binarizing the activations of different spectral channels may lead to severe distortion of spectral features. (ii) Remote sensing scenes are inherently multi-scale and anisotropic. Directly using the common binary convolutions, such as those used in BNNs, has two drawbacks: *(1)* the receptive field is limited, making it difficult to understand the global structure, resulting in blurred or disjointed contours; and *(2)* the convolution kernel is isotropic, which means it learns an ”averaged” feature response during training, which can easily produce jagged effects or blurred edges.

Bearing the above challenges in mind, we propose a novel binarized method, namely Spatial-Spectral Binarized Neural Network (S2BNet) for efficient remote sensing pan-sharpening. Different from the recent binarized multi-spectral fusion network Hou et al. (2025), our work does not include complex computations like diffusion operation and attention mechanism that are difficult to implement on edge devices. Firstly, we design the basic unit, spatial-spectral binarized convolution (S2B-Conv), used in model binarization. S2B-Conv can adapt inconsistent spectral distribution and diverse spatial features before binarizing the activation. Secondly, the Spectral-Redistribution Mechanism (SRM) in S2B-Conv is employed to generate scaling and bias parameters in a adaptive learning manner, adjusting the density and distribution of spectral bands. Thirdly, we propose the Gabor Spatial Feature Amplifier (GSFA) in S2B-Conv, which captures multi-scale and-directional spatial features by randomly selecting different frequencies and angles within a preset range. Finally, we derive our S2BNet by using the proposed spatial-spectral binarized convolution to binarize the base model. As shown in Tab. [1](https://arxiv.org/html/2509.23321v2#S4.T1), S2BNet outperforms SOTA BNNs by large margins, over $2.49$ dB.
In a nutshell, our contributions can be summarized as follows:

- •
We propose a novel BNN-based algorithm S2BNet, which is composed of several S2B-Conv units, for remote sensing pan-sharpening. In particular, the S2B-Conv could adapt complex spatial-spectral features before binarizing the model.
- •
The Spectral-Redistribution Mechanism (SRM) in S2B-Conv is introduced to adjust the distribution in spectral dimension through adaptive learning process.
- •
The Gabor Spatial Feature Amplifier (GSFA) in S2B-Conv is introduced to learn multi-scale and multi-directional spatial features by randomly selecting different frequencies and angles.
- •
Experiments on multiple remote sensing benchmark datasets show that the proposed S2BNet outperforms state-of-the-art binary neural networks.

## 2 Related Work

### 2.1 Pan-sharpening

The classic pan-sharpening methods mainly consist of $3$ categories: CS-based Gillespie et al. (1987), MRA-based Nunez et al. (1999), and VO-based methods Fasbender et al. (2008). Deep learning-based methods have significantly improved the performance of pan-sharpening tasks in recent times. As a groundbreaking study, PNN Masi et al. (2016) first introduced CNN-based methods into pan-sharpening tasks. PANNet Yang et al. (2017) leverages residual connections and high-frequency filtering techniques within a CNN framework. Moreover, HyperTransformer Bandara & Patel (2022a) was one of the early attempts to introduce Transformer-based methods into this field. PanFormer Zhou et al. (2022a) utilizes the Transformer framework to
improve spatial resolution while maintaining spectral integrity.

Although deep learning models have achieved remarkable progress in pan-sharpening over traditional methods, they typically require powerful hardware with considerable memory and computational capacity, that hinders their deployment on resource-limited satellites. In this study, we explore the potential of using binarized neural networks for pan-sharpening to make the model more lightweight.

### 2.2 Binarized Neural Network

Binary Neural Networks (BNNs) have emerged as a promising solution for deploying deep learning models on resource-limited devices Zhang et al. (2024). By leveraging binary representations for weights and activations, BNNs achieve substantial reductions in memory footprint and computational requirements Hubara et al. (2016b).

The pioneering method Hubara et al. (2016a) employs the sign function to derive binarized activations and weights, and it utilizes the straight-through estimator Bengio et al. (2013) for optimizing network parameters. Nevertheless, BNNs encounter precision loss attributable to quantization error and gradient error Zhang et al. (2024). To tackle the quantization error, XNOR-Net Rastegari et al. (2016) incorporates scaling factors for both weights and activations. To mitigate the gradient error, Liu et al. Liu et al. (2018) propose employing a piecewise quadratic function.

Recently, exploration of binary networks for low-level vision applications has drawn great attention. Xin et al. Xin et al. (2020) propose a novel binarization method with a bit accumulation mechanism to approximate full-precision convolution for image super-resolution task. Work Zhang et al. (2024) investigates the use of BNNs for low-light raw video enhancement, facing the challenges of efficiently fusing temporal information and bridging the performance gap with full-precision models.

The inherent resource constraints of remote sensing satellites make model compression a necessity. However, there is hardly any research devoted to light-weighting of remote sensing.

Figure: Figure 1: The overall frame of our method. Our model primarily consists of the novel convolution (S2B-Conv), which incorporates two core modules: the Spectral-Redistribution Mechanism (denoted as ”Redistribution”) and the Gabor Spatial Feature Amplifier (denoted as ”Gabor”). The specific implementation of Gabor is detailed in Algorithm 1.
Refer to caption: x1.png

## 3 Method

In this section, we present the overall architecture of our network for pan-sharpening. Then, we elaborate on the details of the spatial-spectral binarized convolution, which mainly consists of the Spectral-Redistribution Mechanism and Gabor Spatial Feature Amplifier.

### 3.1 Overall Architecture

Binary networks have achieved significant success in the fields of hyperspectral image reconstruction and low-light raw video enhancement. However, few studies have attempted to apply them to the task of pan-sharpening. Building on the existing research in low-level vision tasks and considering the characteristics of pan-sharpening, we design a network that focuses on both spatial and spectral attributes, as the reconstruction quality of these attributes greatly affects the outcome of pan-sharpening.

The general architecture of our S2BNet network is shown in Fig. [1](https://arxiv.org/html/2509.23321v2#S2.F1). Inspired by Cai et al. (2023), we use a U-shaped structure for the base model. First, we upsample the LR-MS images and concatenate them with the Pan images, and feed the output into the full-precision convolutional layer to generate a shallow feature $X_{s}\in\mathbb{R}^{H\times W\times C}$ and subsequently into the S2B-Conv block. Here, $H$, $W$, and $C$ denote the height, width, and the number of channels. Then it undergoes two encoders, a bottleneck, and two decoders. To alleviate the information loss during processing, skip connections between the encoders and the decoders are employed. After that, the S2B-Conv block maps the features to generate the deep feature $X_{d}\in\mathbb{R}^{H\times W\times C}$. Finally, the sum of $X_{s}$ and $X_{d}$ is fed into another full-precision convolutional layer to produce the reconstructed HR-MS images. Specifically, each encoder consists of a basic block followed by a B-Down block, while each decoder is composed of a B-Up block, a Fusion Decrease block, and a basic block. Furthermore, the basic block serves as the bottleneck. It is worth noting that the entire network only employs two full-precision convolutions, while all other convolutions are binary convolutions.

As illustrated in Fig. [1](https://arxiv.org/html/2509.23321v2#S2.F1), the Fusion Increase and Fusion Decrease modules both use binarized convolutional layers with a kernel size of 1 to aggregate the feature maps and modify the channels. The upsample module employs bilinear interpolation followed by binarized convolutional layers with a kernel size of 3 to upscale the feature maps and halve the channels, while the downsample module uses binarized convolutional layers with a kernel size of 3 to downscale the feature maps and double the channels. Moreover, the structure of the basic block is also presented in Fig. [1](https://arxiv.org/html/2509.23321v2#S2.F1). It consists of Layer Norm, Fusion Increase block, S2B-Conv block, Fusion Decrease block, and ReLU activation function. The S2B-Conv block will be thoroughly discussed in subsequent chapters.

### 3.2 spatial-spectral binarized convolution

As shown in the Fig. [1](https://arxiv.org/html/2509.23321v2#S2.F1), we first adjust the spectral distribution of the full-precision activations $X_{f}\in\mathbb{R}^{H\times W\times C}$ using the Spectral-Redistribution Mechanism to get $X_{r}\in\mathbb{R}^{H\times W\times C}$ (which will be introduced in Section [3.2.1](https://arxiv.org/html/2509.23321v2#S3.SS2.SSS1)), and generate weights $W_{f}$ using the Gabor Spatial Feature Amplifier (which will be introduced in Section [3.2.2](https://arxiv.org/html/2509.23321v2#S3.SS2.SSS2)). Then, referring to Zhang et al. (2024); Cai et al. (2023), we binarize both $X_{r}$ and $W_{f}$, and fuse these two features using an XOR operation, as detailed subsequently.

The activation $X_{r}$ is binarized by a sign function in 1-bit values, producing $X_{b}\in\mathbb{R}^{H\times W\times C}$, where each element has a value of +1 or -1. The binarization procedure can be formulated as:

$$ $X_{b}=\text{Sign}(X_{r})=\begin{cases}+1,&X_{r}>0\\ -1,&X_{r}\leq 0.\end{cases}$ (1) $$

Since the Sign function is non-differentiable, we use a scalable hyperbolic tangent function to approximate the Sign function in the backpropagation as:

$$ $x_{b}=\text{Tanh}(\alpha x_{r})=\frac{e^{\alpha x_{r}}-e^{-\alpha x_{r}}}{e^{\alpha x_{r}}+e^{-\alpha x_{r}}},$ (2) $$

where $\alpha\in\mathbb{R}^{+}$is a learnable parameter that adaptively adjusts the distance between Tanh and Sign.

For weight $W_{f}$, we also utilize the sign function to generate the binarized weight $W_{b}$. And we adopt a piecewise linear function, Clip, during the backpropagation as:

$$ $W_{b}=\text{Clip}(W_{f})=\begin{cases}+1,&W_{f}\geq 1\\ x,&-1<W_{f}<1\\ -1,&W_{f}\leq-1\end{cases}$ (3) $$

Subsequently, the computationally intensive floating-point matrix multiplication operations in full-precision convolution can be substituted with pure logical XNOR and bit-count operations as:

$$ $X_{b}\otimes W_{b}=\text{bitcount}(\text{XNOR}(X_{b},W_{b})).$ (4) $$

Following Zhang et al. (2024), we multiply the mean absolute value of the 32-bit weight value $W_{f}$ to narrow down the difference between binarized and full-precision weights, i.e.,

$$ $S_{i}=\frac{\|W_{f}^{i}\|_{1}}{C\times K\times K},\quad i=1\ldots C_{\text{out}},$ (5) $$

$$ $Y=(X_{b}\otimes W_{b})\odot S,$ (6) $$

where $S\in\mathbb{R}^{C_{\text{out}}}$ is the scaling factor and $Y\in\mathbb{R}^{H\times W\times C}$ denotes the output activation rescaled with the factors from weights.

To mitigate the loss of information from binarization, we introduce a residual connection that sums $X_{f}$ and $Y$. However, due to the significant difference in their value ranges, combining them directly using an identity mapping might obscure the information in $Y$. To resolve this issue, we first apply the $Y$ to RPReLU Liu et al. (2018), to adjust its value range before the summation. This step can be represented as:

$$ $X_{o}=X_{f}+\text{RPReLU}(Y),$ (7) $$

where $X_{o}\in\mathbb{R}^{H\times W\times C}$ is the output feature and RPReLU is formulated as:

$$ $\text{RPReLU}(Y)=\left\{\begin{array}[]{ll}Y_{i}-\gamma_{i}+\zeta_{i},&Y_{i}>\zeta_{i}\\ \beta_{i}(Y_{i}-\gamma_{i})+\zeta_{i},&Y_{i}\leq\zeta_{i},\end{array}\right.$ (8) $$

where $\gamma_{i},\zeta_{i},\beta_{i}\in\mathbb{R}^{C_{\text{out}}}$ are learnable parameters.

#### 3.2.1 The Spectral-Redistribution Mechanism

Upon analyzing the LR-MS images, we observe that the distribution and density of spectral information differ significantly across various bands Cai et al. (2023). The spectral information of Pan images and LR-MS images also has significant differences. This inconsistency in spectral information can negatively impact the reconstruction of spectral features, leading to potential inaccuracies in the final output.

Existing studies attempt to adjust the distribution of spectral information using affine transformations. However, they predominantly utilize a randomly initialized approach for scaling and biasing, which does not adapt to the input data. These methods lack the ability to dynamically adjust parameters based on input variations, potentially leading to suboptimal performance across diverse datasets.

To address this limitation, we introduce a Spectral-Redistribution Mechanism that generates scaling and bias parameters through a data-driven mechanism. This approach not only enhances adaptability to varying input characteristics but also enhances feature reconstruction accuracy by optimizing the redistribution process through learning global features.

As depicted in Fig. [1](https://arxiv.org/html/2509.23321v2#S2.F1), first, the input feature $X_{f}$ undergoes global average pooling and is then flattened to obtain the global feature $X_{g}\in\mathbb{R}^{H\times W\times C}$. Subsequently, two fully connected layers along with a ReLU activation function are employed to generate the scaling factor $k$ and the bias $b$. Specifically, the number of channels of feature $X_{g}$ is first mapped from $C$ to $C/r$, and after passing through the ReLU, it is then mapped from $C/r$ to $2C$. Following this, the resulting feature is split along the channel dimension into two separate features, each with $C$ channels. Here $C$ and $r$ denote the number of channels and scaling factor, respectively. After that, we employ the sigmoid activation function to constrain the scaling factor $k$ within the range of 0 to 1, obtaining $k^{\prime}$, and utilize the tanh function to limit the bias $b$ between -1 and 1, resulting in $b^{\prime}$. Ultimately, the redistributed features are acquired through an affine transformation. The specific formulas are as follows:

$$ $k^{\prime}=\sigma(k)=\frac{1}{1+e^{-k}},$ (9) $$

$$ $b^{\prime}=\tanh(b)=\frac{e^{b}-e^{-b}}{e^{b}+e^{-b}},$ (10) $$

$$ $X_{r}=X_{f}\cdot k^{\prime}+b^{\prime}.$ (11) $$

#### 3.2.2 Gabor Spatial Feature Amplifier

The spectral characteristics of parking lots and open spaces, as well as bare land and leveled construction sites, are very similar and need to be distinguished by spatial features. However, their spatial shapes and sizes are also similar, and the resolution of the LR-MS images is too low. Therefore, a method that can extract spatial textures more accurately is needed.

To address this challenge, we propose the Gabor Spatial Feature Amplifier, which initializes the weights of convolutional layers using Gabor kernels. Gabor kernels, by randomly selecting different frequencies and angles within a preset range, enable subsequent convolutional layers to better handle multi-scale and multi-directional features, which is conducive to the learning of spatial features.

Although Gabor kernels are widely used in image processing to simulate the perception of edges and textures by the human visual system, existing research has not yet explored the application of Gabor kernels in binary convolutional networks.

First, we define two lists: one containing different frequency values and another containing different angle values. For each output channel, we randomly select a frequency and an angle from these lists to generate the Gabor kernel. The Gabor kernel is generated based on the following formula:

$$ $g(x,y;\lambda,\theta,\psi,\sigma,\gamma)=\exp\left(-\frac{x^{\prime 2}+\gamma^{2}y^{\prime 2}}{2\sigma^{2}}\right)\exp\left(i\left(2\pi\frac{x^{\prime}}{\lambda}+\psi\right)\right),$ (12) $$

where

$$ $\begin{cases}x^{\prime}=x\cos\theta+y\sin\theta\\ y^{\prime}=-x\sin\theta+y\cos\theta.\end{cases}$ (13) $$

However, here we only use the real part, that is,

$$ $g(x,y;\lambda,\theta,\psi,\sigma,\gamma)=\exp\left(-\frac{x^{\prime 2}+\gamma^{2}y^{\prime 2}}{2\sigma^{2}}\right)\cos\left(2\pi\frac{x^{\prime}}{\lambda}+\psi\right).$ (14) $$

The parameters of the Gabor function each play a distinct role in shaping the characteristics of the filter. The parameter $\lambda$ signifies the wavelength of the sinusoidal component within the Gabor function. $\theta$ indicates the orientation or direction of the filter. The phase offset of the sinusoidal function is denoted by $\psi$. The parameter $\sigma$ represents the standard deviation of the Gaussian envelope, and $\gamma$ is the aspect ratio that influences the ellipticity of the Gaussian function. Since each output channel selects different parameters, the $C_{out}$ output channels will have $C_{out}$ different Gabor kernels, thus enabling the learning of spatial features at different directions and scales. Because the generated Gabor kernels are intended to replace the weights in the convolutional layer, we need to normalize them so that their mean and variance are similar to the default weights of the convolutional layer. This ensures that the Gabor kernels are compatible with the initialization scheme typically used in convolutional neural networks. For the detailed algorithm, please refer to Algorithm. [1](https://arxiv.org/html/2509.23321v2#alg1).

Figure: Algorithm 1 Gabor Spatial Feature Amplifier

### 3.3 Loss Function

We use $L_{1}$ loss as the reconstruction loss:

$$ $\mathcal{L}_{rec}=\|G-H\|_{1},$ (15) $$

where $G$ is the ground truth image, and $H$ denotes the reconstructed image.

## 4 Experiments

In this section, we evaluate our S2BNet on three pan-sharpening datasets. We also conduct extensive experiments to analyze our proposed model.

### 4.1 Experimental Settings

Datasets. For the pan-sharpening, three $4$-band datasets acquired by WorldView-2, GaoFen-2 and QuickBird sensors are adopted for experimental analysis. Due to the unavailability of ground-truth (GT) images for training, following Wald’s protocol Wald et al. (1997). We employ downsampling operations to produce corresponding dataset for each satellite sensor, as shown in the Appendix, we present the detailed information of traning dataset and testing dataset in the experiment.

Training Details. Moreover, the proposed network is built with PyTorch and trained on four NVIDIA RTX $A5000$ GPUs. Training is conducted with a batch size of $16$, and the Adam optimizer is adopted, setting $\beta_{1}=0.9$ and $\beta_{2}=0.999$ for its momentum coefficients.The initial learning rate is set to $1.5\times 10^{-3}$ and is multiplied by $0.85$ every $100$ epochs. The whole training procedure runs for $1500$ epochs before stopping.

Metrics. Following previous studies on pan-sharpening, five image quality assessment metrics Yang et al. (2023) are employed for evaluation on reduced-resolution images, including spectral angle mapper (SAM), dimensionless global error in synthesis (ERGAS), the structural similarity (SSIM), the peak signal-to-noise ratio (PSNR), and the Q-index. Since there are not GT images available for the full-resolution dataset, we use three non-reference metrics to evaluate the performance of the model: Spectral Distortion Index ($D_{\lambda}$), Spatial Distortion Index ($D_{s}$), and No-Reference Quality (QNR) Alparone et al. (2008).

Efficiency Evaluation. Following previous BNN work Cai et al. (2023), the FLOPs per second for BNN is determined by $\mathrm{Flops}^{\mathrm{b}}=\mathrm{Flops}^{\mathbf{f}}/64$. The overall FLOPs is $\mathrm{Flops}^{\mathrm{b}}+\mathrm{Flops}^{\mathrm{f}}$. The parameters of BNN are calculated as $\mathrm{Params}^{\mathrm{b}}=\mathrm{Params}^{\mathrm{f}}/32$. The total number of parameters is $\mathrm{Params}^{\mathrm{b}}+\mathrm{Params}^{\mathrm{f}}$.

### 4.2 Compare with State-of-the-arts

Comparison Methods. We compare our method with various full precision pan-sharpening networks including DMFNet Yuan et al. (2019), PANFormer Zhou et al. (2022a), MutInf Zhou et al. (2022c), FAMENet Xuanhua et al. (2024), CANNet Duan et al. (2024), UGCC Zeng et al. (2025) and MSCSCFormer Ye et al. (2024). We also compare our spatial-spectral binarized convolution with other binarization methods, including BNN Hubara et al. (2016a), LCRBNN Shang et al. (2022), BiSRNet Cai et al. (2023), BBCU Xia et al. (2022), FABNet Jiang et al. (2023), IRNet Qin et al. (2020b) and E2FIF Song et al. (2023).

**Table 1: Quantitative comparison of our S2BNet with our full-precision and binary methods on the GaoFen-2 dataset.**
|  |  |  |  | Reduced-Resolution | Full-Resolution |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Category | Method | Params (K) | Flops (G) | PSNR$\uparrow$ | SSIM$\uparrow$ | $Q_{4}\uparrow$ | SAM$\downarrow$ | ERGAS$\downarrow$ | $D_{\lambda}\downarrow$ | $D_{s}\downarrow$ | QNR$\uparrow$ |
|  | DMFNet  Yuan et al. (2019) | 1631.924 | 145.811 | 43.1223 | 0.9734 | 0.8557 | 0.0322 | 1.5660 | 0.0678 | 0.1134 | 0.8265 |
|  | PANFormer Zhou et al. (2022a) | 1530.300 | 12.002 | 44.8501 | 0.9805 | 0.8865 | 0.0271 | 1.3337 | 0.0670 | 0.1806 | 0.7639 |
|  | MutInf Zhou et al. (2022c) | 185.496 | 9.986 | 44.8306 | 0.9800 | 0.8836 | 0.0277 | 1.3394 | 0.0755 | 0.1762 | 0.7613 |
|  | FAMENet Xuanhua et al. (2024) | 1244.228 | 39.272 | 45.6617 | 0.9837 | 0.8966 | 0.0248 | 1.2142 | 0.0697 | 0.1800 | 0.7622 |
|  | CANNet Duan et al. (2024) | 785.118 | 3.237 | 45.2222 | 0.9816 | 0.8869 | 0.0267 | 1.2748 | 0.0654 | 0.1969 | 0.7498 |
|  | UGCC  Zeng et al. (2025) | 233.233 | 307.495 | 40.5620 | 0.9586 | 0.7803 | 0.0448 | 2.1717 | 0.0893 | 0.0608 | 0.8564 |
| Full | MSCSCFormer Ye et al. (2024) | 1950.686 | 582.435 | 42.6871 | 0.9696 | 0.8350 | 0.0360 | 1.7149 | 0.0692 | 0.0973 | 0.8403 |
|  | BNN  Hubara et al. (2016a) | 17.760 | 1.938 | 34.4931 | 0.8802 | 0.4701 | 0.0860 | 4.1576 | 0.0985 | 0.0871 | 0.8243 |
|  | LCRBNN  Shang et al. (2022) | 17.760 | 1.938 | 32.2058 | 0.8165 | 0.2852 | 0.0952 | 5.4921 | 0.1043 | 0.1621 | 0.7503 |
|  | BiSRNet  Cai et al. (2023) | 26.459 | 0.664 | 43.3446 | 0.9727 | 0.8424 | 0.0327 | 1.5748 | 0.0669 | 0.0781 | 0.8604 |
|  | BBCU  Xia et al. (2022) | 23.520 | 0.664 | 43.0233 | 0.9712 | 0.8361 | 0.0335 | 1.6449 | 0.0676 | 0.0900 | 0.8484 |
|  | FABNet  Jiang et al. (2023) | 59.296 | 0.664 | 43.0144 | 0.9709 | 0.8370 | 0.0342 | 1.6477 | 0.0676 | 0.0867 | 0.8515 |
|  | IRNet  Qin et al. (2020b) | 17.760 | 1.938 | 42.8422 | 0.9714 | 0.8394 | 0.0338 | 1.6636 | 0.0692 | 0.1095 | 0.8285 |
|  | E2FIF  Song et al. (2023) | 20.640 | 0.664 | 41.0767 | 0.9544 | 0.7992 | 0.0402 | 2.0694 | 0.0740 | 0.0909 | 0.8422 |
| Binary | S2BNet | 75.407 | 0.920 | 43.5619 | 0.9745 | 0.8523 | 0.0317 | 1.5384 | 0.0655 | 0.0680 | 0.8714 |

Quantitative Comparison. As summarized in Tab. [1](https://arxiv.org/html/2509.23321v2#S4.T1), our method delivers substantial performance improvements over other binary techniques across all metrics, showcasing superior spectral and spatial fidelity. Lower SAM values demonstrate its effective spectral preservation, while higher PSNR scores confirm its robust spatial detail retention. More importantly, our model surpasses the majority of full-precision baselines, demonstrating its promising expressiveness while benefiting from the efficiency of model binarization.

We additionally assess our model’s generalization capability in real-world scenarios by testing it on full-resolution data. As shown in Tab. [1](https://arxiv.org/html/2509.23321v2#S4.T1), our model consistently achieves superior performance across all three datasets, yielding the optimal scores on most evaluation metrics. The quantitative results on the WorldView-2 and QuickBird examples are shown in Appendix.

Figure: Figure 2: Visual comparison between our model and other binary methods on GF-2 example. The top two lines represent the reconstructed results and corresponding MAE maps of the reduced-resolution example, and the last line represents the reconstructed results of the full-resolution example.
Refer to caption: x2.png

Visual Comparison. The qualitative results on the GaoFen-2 dataset are illustrated in Fig. [2](https://arxiv.org/html/2509.23321v2#S4.F2). It can be observed that our model produces image with clear textures and visually pleasing spectra, and the corresponding MAE map exhibits minimal bright spots, indicating a high resemblance to the ground truth. In addition, the image generated by our S2BNet exhibits reduced aberrations and artifacts in full-resolution example. The visualization results on the WorldView-2 example are shown in Appendix.

Performance-Efficiency Comparison. The performance-efficiency comparison between our method and other full-precision and binary benchmarks is also presented in Tab. [1](https://arxiv.org/html/2509.23321v2#S4.T1). Compared with full-precision methods, our approach not only surpasses most methods in performance but also minimizes computational overhead. Compared to other binary models, our S2BNet achieves the highest PSNR score, with a marginally lower FLOPs than IRNet.

### 4.3 Ablation Study

Effect of the Spectral Redistribution Mechanism. To verify the effect of Spectral-Redistribution Mechanism (SRM) in the S2B-Conv, Config (II) in Tab. [3](https://arxiv.org/html/2509.23321v2#S4.T3) is obtained by removing Spectral Redistribution Mechanism, and uses random initial scaling and bias values. It could be reported in Tab. [3](https://arxiv.org/html/2509.23321v2#S4.T3) that the model achieves $0.31$ dB improvement when we exploit SRM.

Effect of the Gabor Spatial Feature Amplifier. As shown in Tab. [3](https://arxiv.org/html/2509.23321v2#S4.T3), the Config (I) is denoted, which replaces the Gabor Spatial Feature Amplifier (GSFA) with traditional convolutional weights. Our GSFA dramatically surpasses the common convolutional weights by $0.46$ dB.

Binarizing Different Parts. We binarize one part of the S2BNet while keeping the other parts full-precision to study the binarization benefit. The results are reported in Tab. [3](https://arxiv.org/html/2509.23321v2#S4.T3). The base model (full precision) yields 42.2759 dB in PSNR while costing 13.884 G OPs and 349.695 K Params. We can find that (i) Binarizing the bottleneck reduces the Params the most (100.329K) with the smallest performance drop (only 0.16 dB). (ii) Binarizing the decoder achieves the largest OPs reduction (6.177G) while the performance degrades by a moderate margin (0.57 dB).

**Table 2: Comparison of different configurations.**
| Config | GSFA | SRM | PSNR$\uparrow$ | SSIM$\uparrow$ | SAM$\downarrow$ | QNR$\uparrow$ |
| --- | --- | --- | --- | --- | --- | --- |
| (I) | ✕ | ✓ | 43.107 | 0.972 | 0.033 | 0.851 |
| (II) | ✓ | ✕ | 43.250 | 0.972 | 0.033 | 0.859 |
| Ours | ✓ | ✓ | 43.562 | 0.975 | 0.031 | 0.871 |

## 5 Conclusion

In this paper, we propose a novel BNN-based method, S2BNet, for pan-sharpening. S2BNet is the compact and easy-to-deploy base model with simple computation operations. Specifically, we customize the basic unit S2B-Conv for model binarization. S2B-Conv utilizes the Spectral-Redistribution Mechanism, which can adaptively adjust the density and distribution of spectral features. Besides, S2B-Conv employs the Gabor Spatial Feature Amplifier to capture multi-scale and multi-directional spatial features. Comprehensive quantitative and qualitative experiments demonstrate that our S2BNet significantly outperforms SOTA BNNs and even achieves comparable performance with full-precision pan-sharpening algorithms.

## 6 Reproducibility Statement

To ensure the reproducibility of our research, we will make the core code publicly available in the supplementary materials.

## References

- Alparone et al. (2008)
Luciano Alparone, Bruno Aiazzi, Stefano Baronti, Andrea Garzelli, Filippo Nencini, and Massimo Selva.
Multispectral and panchromatic data fusion assessment without reference.
*Photogrammetric Engineering & Remote Sensing*, 74(2):193–200, 2008.
- Asokan & Anitha (2019)
Anju Asokan and J. Anitha.
Change detection techniques for remote sensing applications: a survey.
*Earth Science Informatics*, pp. 143–160, Jun 2019.
- Bandara & Patel (2022a)
Wele Gedara Chaminda Bandara and Vishal M. Patel.
Hypertransformer: A textural and spectral feature fusion transformer for pansharpening.
In *IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022*, pp. 1757–1767. IEEE, 2022a.
- Bandara & Patel (2022b)
Wele Gedara Chaminda Bandara and Vishal M Patel.
Hypertransformer: A textural and spectral feature fusion transformer for pansharpening.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pp. 1767–1777, 2022b.
- Bengio et al. (2013)
Yoshua Bengio, Nicholas Léonard, and Aaron Courville.
Estimating or propagating gradients through stochastic neurons for conditional computation.
*arXiv preprint arXiv:1308.3432*, 2013.
- Cai & Huang (2020)
Jiajun Cai and Bo Huang.
Super-resolution-guided progressive pansharpening based on a deep convolutional neural network.
*IEEE Transactions on Geoscience and Remote Sensing*, 59(6):5206–5220, 2020.
- Cai et al. (2023)
Yuanhao Cai, Yuxin Zheng, Jing Lin, Haoqian Wang, Xin Yuan, and Yulun Zhang.
Binarized spectral compressive imaging, 2023.
- Duan et al. (2024)
Yule Duan, Xiao Wu, Haoyu Deng, and Liang-Jian Deng.
Content-adaptive non-local convolution for remote sensing pansharpening.
In *2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, pp. 27738–27747, 2024.
- Fasbender et al. (2008)
D. Fasbender, J. Radoux, and P. Bogaert.
Bayesian data fusion for adaptable image pansharpening.
*IEEE Transactions on Geoscience and Remote Sensing*, pp. 1847–1857, Jun 2008.
- Gillespie et al. (1987)
Alan R Gillespie, Anne B Kahle, and Richard E Walker.
Color enhancement of highly correlated images. ii. channel ratio and “chromaticity” transformation techniques.
*Remote Sensing of Environment*, pp. 343–365, Jul 1987.
- Han et al. (2024)
Zhu Han, Shuyi Xu, Lianru Gao, Zhi Li, and Bing Zhang.
Gretnet: Gaussian retentive network for hyperspectral image classification.
*IEEE Geoscience and Remote Sensing Letters*, 2024.
- Hou et al. (2025)
Junming Hou, Xiaoyu Chen, Ran Ran, Xiaofeng Cong, Xinyang Liu, Jian Wei You, and Liang-Jian Deng.
Binarized neural network for multi-spectral image fusion.
In *Proceedings of the Computer Vision and Pattern Recognition Conference*, pp. 2236–2245, 2025.
- Hubara et al. (2016a)
Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio.
Binarized neural networks.
In *Proceedings of the 30th International Conference on Neural Information Processing Systems*, NIPS’16, pp. 4114–4122, Red Hook, NY, USA, 2016a. Curran Associates Inc.
- Hubara et al. (2016b)
Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio.
Binarized neural networks.
*Advances in neural information processing systems*, 29, 2016b.
- Jiang et al. (2023)
Xinrui Jiang, Nannan Wang, Jingwei Xin, Keyu Li, Xi Yang, Jie Li, Xiaoyu Wang, and Xinbo Gao.
Fabnet: Frequency-aware binarized network for single image super-resolution.
*IEEE Transactions on Image Processing*, 32:6234–6247, 2023.
- Li et al. (2022)
Jiaxin Li, Ke Zheng, Jing Yao, Lianru Gao, and Danfeng Hong.
Deep unsupervised blind hyperspectral and multispectral data fusion.
*IEEE Geoscience and Remote Sensing Letters*, 19:1–5, 2022.
- Li et al. (2023a)
Mingsong Li, Yikun Liu, Tao Xiao, Yuwen Huang, and Gongping Yang.
Local-global transformer enhanced unfolding network for pan-sharpening.
In *Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI 2023, 19th-25th August 2023, Macao, SAR, China*, pp. 1071–1079. ijcai.org, 2023a.
- Li et al. (2023b)
Mingsong Li, Yikun Liu, Tao Xiao, Yuwen Huang, and Gongping Yang.
Local-global transformer enhanced unfolding network for pan-sharpening.
*arXiv preprint arXiv:2304.14612*, 2023b.
- Liu et al. (2018)
Zechun Liu, Baoyuan Wu, Wenhan Luo, Xin Yang, Wei Liu, and Kwang-Ting Cheng.
Bi-real net: Enhancing the performance of 1-bit cnns with improved representational capability and advanced training algorithm.
In *Proceedings of the European conference on computer vision (ECCV)*, pp. 722–737, 2018.
- Ma et al. (2025)
Mengting Ma, Yizhen Jiang, Mengjiao Zhao, Xiaowen Ma, Wei Zhang, and Siyang Song.
Deep spatial–spectral fusion transformer for remote sensing pansharpening.
*Information Fusion*, 118:102980, 2025.
- Masi et al. (2016)
Giuseppe Masi, Davide Cozzolino, Luisa Verdoliva, and Giuseppe Scarpa.
Pansharpening by convolutional neural networks.
*Remote Sensing*, 2016.
doi: 10.3390/rs8070594.
URL [http://dx.doi.org/10.3390/rs8070594](http://dx.doi.org/10.3390/rs8070594).
- Nunez et al. (1999)
J. Nunez, X. Otazu, O. Fors, A. Prades, V. Pala, and R. Arbiol.
Multiresolution-based image fusion with additive wavelet decomposition.
*IEEE Transactions on Geoscience and Remote Sensing*, pp. 1204–1211, May 1999.
- Qin et al. (2020a)
Haotong Qin, Ruihao Gong, Xianglong Liu, Xiao Bai, Jingkuan Song, and Nicu Sebe.
Binary neural networks: A survey.
*Pattern Recognition*, 105:107281, 2020a.
- Qin et al. (2020b)
Haotong Qin, Ruihao Gong, Xianglong Liu, Mingzhu Shen, Ziran Wei, Fengwei Yu, and Jingkuan Song.
Forward and backward information retention for accurate binary neural networks.
In *2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, pp. 2247–2256, 2020b.
- Rastegari et al. (2016)
Mohammad Rastegari, Vicente Ordonez, Joseph Redmon, and Ali Farhadi.
Xnor-net: Imagenet classification using binary convolutional neural networks.
In *European conference on computer vision*, pp. 525–542. Springer, 2016.
- Shang et al. (2022)
Yuzhang Shang, Dan Xu, Bin Duan, Ziliang Zong, Liqiang Nie, and Yan Yan.
Lipschitz continuity retained binary neural network.
In *European conference on computer vision*, pp. 603–619. Springer, 2022.
- Song et al. (2023)
Chongxing Song, Zhiqiang Lang, Wei Wei, and Lei Zhang.
E2fif: Push the limit of binarized deep imagery super-resolution using end-to-end full-precision information flow.
*IEEE Transactions on Image Processing*, 32:5379–5393, 2023.
- Tan et al. (2024)
Jiangtong Tan, Jie Huang, Naishan Zheng, Man Zhou, Keyu Yan, Danfeng Hong, and Feng Zhao.
Revisiting spatial-frequency information integration from a hierarchical perspective for panchromatic and multi-spectral image fusion.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pp. 25922–25931, 2024.
- Wald et al. (1997)
Lucien Wald, Thierry Ranchin, and Marc Mangolini.
Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images.
*Photogrammetric engineering and remote sensing*, 63(6):691–699, 1997.
- Wang & Fu (2022)
Huan Wang and Yun Fu.
Trainability preserving neural pruning.
*arXiv preprint arXiv:2207.12534*, 2022.
- Wang et al. (2022)
Huan Wang, Suhas Lohit, Michael N Jones, and Yun Fu.
What makes a” good” data augmentation in knowledge distillation-a statistical perspective.
*Advances in Neural Information Processing Systems*, 35:13456–13469, 2022.
- Wu et al. (2022)
Xiao Wu, Ting-Zhu Huang, Liang-Jian Deng, and Tian-Jing Zhang.
Dynamic cross feature fusion for remote sensing pansharpening.
In *2021 IEEE/CVF International Conference on Computer Vision (ICCV)*, Mar 2022.
- Xia et al. (2022)
Bin Xia, Yulun Zhang, Yitong Wang, Yapeng Tian, Yang Wenming, Radu Timofte, and Luc Gool.
Basic binary convolution unit for binarized image restoration network.
10 2022.
doi: 10.48550/arXiv.2210.00405.
- Xin et al. (2020)
Jingwei Xin, Nannan Wang, Xinrui Jiang, Jie Li, Heng Huang, and Xinbo Gao.
Binarized neural network for single image super resolution.
In *European conference on computer vision*, pp. 91–107. Springer, 2020.
- Xing et al. (2023)
Yinghui Xing, Yan Zhang, Houjun He, Xiuwei Zhang, and Yanning Zhang.
Pansharpening via frequency-aware fusion network with explicit similarity constraints.
*IEEE Transactions on Geoscience and Remote Sensing*, pp. 1–1, Jan 2023.
- Xuanhua et al. (2024)
He Xuanhua, Yan Keyu, Li Rui, Xie Chengjun, Zhang Jie, and Zhou Man.
Frequency-adaptive pan-sharpening with mixture of experts.
In *2024 Conference on Innovative Applications of Artificial Intelligence (AAAI)*, 2024.
- Yang et al. (2023)
Gang Yang, Xiangyong Cao, Wenzhe Xiao, Man Zhou, Aiping Liu, Xun chen, and Deyu Meng.
Panflownet: A flow-based deep network for pan-sharpening.
May 2023.
- Yang et al. (2017)
Junfeng Yang, Xueyang Fu, Yuwen Hu, Yue Huang, Xinghao Ding, and John Paisley.
Pannet: A deep network architecture for pan-sharpening.
In *2017 IEEE International Conference on Computer Vision (ICCV)*, Oct 2017.
doi: 10.1109/iccv.2017.193.
URL [http://dx.doi.org/10.1109/iccv.2017.193](http://dx.doi.org/10.1109/iccv.2017.193).
- Ye et al. (2024)
Yongxu Ye, Tingting Wang, Faming Fang, and Guixu Zhang.
Mscscformer: Multiscale convolutional sparse coding-based transformer for pansharpening.
*IEEE Transactions on Geoscience and Remote Sensing*, 62:1–12, 2024.
- Yuan et al. (2019)
Jianzhong Yuan, Wujie Zhou, and Ting Luo.
Dmfnet: Deep multi-modal fusion network for rgb-d indoor scene segmentation.
*IEEE Access*, 7:169350–169358, 2019.
doi: 10.1109/ACCESS.2019.2955101.
- Zeng et al. (2025)
Haoying Zeng, Xiaoyuan Yang, Kangqing Shen, Yixiao Li, Jin Jiang, and Fangyi Li.
Cross-modal contrastive pansharpening via uncertainty guidance.
*IEEE Transactions on Geoscience and Remote Sensing*, 63:1–14, 2025.
- Zhang et al. (2024)
Gengchen Zhang, Yulun Zhang, Xin Yuan, and Ying Fu.
Binarized low-light raw video enhancement.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pp. 25753–25762, 2024.
- Zhou et al. (2022a)
Huanyu Zhou, Qingjie Liu, and Yunhong Wang.
Panformer: A transformer based model for pan-sharpening.
In *2022 IEEE International Conference on Multimedia and Expo (ICME)*, pp. 1–6. IEEE, 2022a.
- Zhou et al. (2022b)
Man Zhou, Jie Huang, Yanchi Fang, Xueyang Fu, and Aiping Liu.
Pan-sharpening with customized transformer and invertible neural network.
In *Proceedings of the AAAI conference on artificial intelligence*, volume 36, pp. 3553–3561, 2022b.
- Zhou et al. (2022c)
Man Zhou, Jie Huang, Keyu Yan, Hu Yu, Xueyang Fu, Aiping Liu, Xian Wei, and Feng Zhao.
Spatial-frequency domain information integration for pan-sharpening.
In *European conference on computer vision*, pp. 274–291. Springer, 2022c.

## Appendix A The use of LLMS

We utilized GPT and Kimi to correct grammatical errors and translate some sentences.

## Appendix B Appendix

In this Appendix, we provide more experimental details of our S2BNet as follows.

Pan-sharpening Datasets. We conduct experiments using the widely recognized WorldView-2, QuickBird and GaoFen-2 datasets Li et al. (2023b). The WorldView-2 dataset consists of instances acquired by the sensor aboard the WorldView-2 satellite. This sensor captures data which covers wavelengths from $0.4$ to $1$ $\mathrm{\mu m}$, with a spatial resolution of $1.24$ $\mathrm{m}$. The QuickBird dataset consists of instances acquired by the sensor aboard the QuickBird satellite. This sensor captures data across four spectral bands, covering wavelengths from $0.45$ to $0.9$ $\mathrm{\mu m}$, with a spatial resolution of $2.4$ $\mathrm{m}$. The images in the GaoFen-2 dataset are collected by the sensor onboard the GaoFen-2 satellite, which records data across four spectral bands within the wavelength range of $0.45-0.89$ $\mathrm{\mu m}$. Additionally, this sensor provides a spatial resolution of $3.2m$. As shown in Tab. [4](https://arxiv.org/html/2509.23321v2#A2.T4), we present the detailed information of training dataset and the testing dataset in the experiment.

**Table 4: The detailed dataset information (GaoFen-2, WorldView-2 and QuickBird), followed by Ma et al. (2025).**
| Sensor | bit depth | #Images | Scale | Training | Testing |
| --- | --- | --- | --- | --- | --- |
| WordView-2 | 11 | 1 | Reduced | #Patches: 1012 | #Patches: 145 |
| PAN: 128 $\times$ 128 $\times$ 1 | PAN: 128 $\times$ 128 $\times$ 1 |  |  |  |  |
| LR-MS: 32 $\times$ 32 $\times$ 4 | LR-MS: 32 $\times$ 32 $\times$ 4 |  |  |  |  |
| Output: 128 $\times$ 128 $\times$ 4 | Output: 128 $\times$ 128 $\times$ 4 |  |  |  |  |
| Full | - | #Patches: 120 |  |  |  |
| - | PAN: 128 $\times$ 128 $\times$ 1 |  |  |  |  |
| - | LR-MS: 32 $\times$ 32 $\times$ 4 |  |  |  |  |
| - | Output: 128 $\times$ 128 $\times$ 4 |  |  |  |  |
| QuickBird | 11 | 3 | Reduced | #Patches: 1024 | #Patches: 128 |
| PAN: 128 $\times$ 128 $\times$ 1 | PAN: 128 $\times$ 128 $\times$ 1 |  |  |  |  |
| LR-MS: 32 $\times$ 32 $\times$ 4 | LR-MS: 32 $\times$ 32 $\times$ 4 |  |  |  |  |
| Output: 128 $\times$ 128 $\times$ 4 | Output: 128 $\times$ 128 $\times$ 4 |  |  |  |  |
| Full | - | #Patches: 128 |  |  |  |
| - | PAN: 128 $\times$ 128 $\times$ 1 |  |  |  |  |
| - | LR-MS: 32 $\times$ 32 $\times$ 4 |  |  |  |  |
| - | Output: 128 $\times$ 128 $\times$ 4 |  |  |  |  |
| GaoFen-2 | 11 | 1 | Reduced | #Patches: 1036 | #Patches: 136 |
| PAN: 128 $\times$ 128 $\times$ 1 | PAN: 128 $\times$ 128 $\times$ 1 |  |  |  |  |
| LR-MS: 32 $\times$ 32 $\times$ 4 | LR-MS: 32 $\times$ 32 $\times$ 4 |  |  |  |  |
| Output: 128 $\times$ 128 $\times$ 4 | Output: 128 $\times$ 128 $\times$ 4 |  |  |  |  |
| Full | - | #Patches: 120 |  |  |  |
| - | PAN: 128 $\times$ 128 $\times$ 1 |  |  |  |  |
| - | LR-MS: 32 $\times$ 32 $\times$ 4 |  |  |  |  |
| - | Output: 128 $\times$ 128 $\times$ 4 |  |  |  |  |

More experimental results. To show the effectiveness of the proposed method, we also present the
visualize the quantitative results and visual results of
the proposed method on other pan-sharpening datasets in Tab. [5](https://arxiv.org/html/2509.23321v2#A2.T5), Tab. [6](https://arxiv.org/html/2509.23321v2#A2.T6), and Fig. [3](https://arxiv.org/html/2509.23321v2#A2.F3).

**Table 5: Quantitative comparison of our S2BNet with binary methods on the QuickBird dataset.**
| Methods | Params (K) | FLOPs (G) | Reduced | Full |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| PSNR$\uparrow$ | SSIM$\uparrow$ | $Q_{4}\uparrow$ | SAM$\downarrow$ | ERGAS$\downarrow$ | QNR$\uparrow$ |  |  |  |
| BNN | 17.760 | 1.938 | 23.4593 | 0.7868 | 0.4876 | 0.2729 | 7.3432 | 0.6854 |
| LCRBNN | 17.760 | 1.938 | 23.1537 | 0.7257 | 0.2900 | 0.2712 | 7.7950 | 0.5757 |
| BiSRNet | 26.459 | 0.664 | 30.6142 | 0.9252 | 0.8172 | 0.0602 | 3.6351 | 0.8468 |
| BBCU | 23.520 | 0.664 | 31.1226 | 0.9340 | 0.8398 | 0.0846 | 3.7487 | 0.8519 |
| FABNet | 59.296 | 0.664 | 32.3291 | 0.9338 | 0.8361 | 0.0625 | 3.0356 | 0.8555 |
| S2BNet | 75.407 | 0.920 | 32.7623 | 0.9340 | 0.8411 | 0.0675 | 2.9236 | 0.8495 |

**Table 6: Quantitative comparison of our S2BNet with our full-precision and binary methods on the WorldView-2 dataset.**
|  |  |  |  | Reduced-Resolution | Full-Resolution |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Category | Method | Params (K) | FLOPs (G) | PSNR$\uparrow$ | SSIM$\uparrow$ | $Q_{4}\uparrow$ | SAM$\downarrow$ | ERGAS$\downarrow$ | $D_{\lambda}\downarrow$ | $D_{s}\downarrow$ | QNR$\uparrow$ |
|  | DMFNet  Yuan et al. (2019) | 1631.924 | 145.811 | 41.0268 | 0.9716 | 0.8184 | 0.0252 | 1.0855 | 0.0644 | 0.0863 | 0.8557 |
|  | PanFormer  Zhou et al. (2022a) | 1530.300 | 12.002 | 41.3495 | 0.9731 | 0.8237 | 0.0242 | 1.0621 | 0.0628 | 0.0844 | 0.8590 |
|  | MutInf  Zhou et al. (2022c) | 185.496 | 9.986 | 41.9527 | 0.9760 | 0.8258 | 0.0227 | 1.0152 | 0.0622 | 0.0794 | 0.8643 |
|  | FameNet  Xuanhua et al. (2024) | 1244.228 | 39.272 | 42.0278 | 0.9768 | 0.8332 | 0.0222 | 0.9936 | 0.0624 | 0.0753 | 0.8678 |
|  | CANNet  Duan et al. (2024) | 785.118 | 3.237 | 41.5868 | 0.9737 | 0.8256 | 0.0237 | 1.0517 | 0.0612 | 0.0767 | 0.8679 |
|  | UGCC  Zeng et al. (2025) | 233.233 | 307.495 | 35.8744 | 0.9296 | 0.6827 | 0.0460 | 1.9253 | 0.0887 | 0.0872 | 0.8327 |
| Full | MSCSCFormer  Ye et al. (2024) | 1950.686 | 582.435 | 40.5338 | 0.9691 | 0.7963 | 0.0270 | 1.2016 | 0.0654 | 0.0772 | 0.8634 |
|  | BNN  Hubara et al. (2016a) | 17.760 | 1.938 | 32.9122 | 0.8706 | 0.4595 | 0.0679 | 3.1063 | 0.0970 | 0.1244 | 0.7916 |
|  | LCRBNN  Shang et al. (2022) | 17.760 | 1.938 | 29.3907 | 0.7781 | 0.1877 | 0.0886 | 4.9267 | 0.0968 | 0.2997 | 0.6323 |
|  | BiSRNet  Cai et al. (2023) | 26.459 | 0.664 | 40.5263 | 0.9680 | 0.8016 | 0.0270 | 1.1755 | 0.0631 | 0.0756 | 0.8670 |
|  | BBCU  Xia et al. (2022) | 23.520 | 0.664 | 40.4211 | 0.9675 | 0.7957 | 0.0272 | 1.1904 | 0.0627 | 0.0758 | 0.8671 |
|  | FABNet  Jiang et al. (2023) | 59.296 | 0.664 | 40.0005 | 0.9647 | 0.7898 | 0.0287 | 1.2297 | 0.0624 | 0.0748 | 0.8684 |
|  | IRNet  Qin et al. (2020b) | 17.760 | 1.938 | 39.2731 | 0.9615 | 0.7736 | 0.0311 | 1.3434 | 0.0687 | 0.0801 | 0.8577 |
|  | E2FIF  Song et al. (2023) | 20.640 | 0.664 | 37.1290 | 0.9407 | 0.7273 | 0.0412 | 1.6997 | 0.0741 | 0.0833 | 0.8497 |
| Binary | S2BNet | 75.407 | 0.920 | 40.6059 | 0.9684 | 0.8035 | 0.0267 | 1.1742 | 0.0619 | 0.0760 | 0.8677 |

Figure: Figure 3: Visual comparison between our model and other binary methods on WV-2 example. The top two lines represent the reconstructed results and corresponding MAE maps of the reduced-resolution example, and the last line represents the reconstructed results of the full-resolution example.
Refer to caption: x3.png

## Peer Reviews

confidence: 5
contribution: 3
initial_rating: 4
presentation: 2
questions: * How does the adaptive SRM perform on datasets with greater spectral diversity?
* How does S2BNet’s binarization affect color fidelity and edge preservation in visually demanding applications? In ablation studies where numerical differences are subtle, visual comparisons would be more informative.
* Why is binarization not applied in conjunction with linear layers? What trade-offs exist between binarization and module design? Could partial binarization offer a better balance?
* Have the SRM and GSFA modules been tested in other network architectures? Without such validation, the effectiveness of these modules lacks statistical support and may be difficult to substantiate.
In summary, the effectiveness of this method remains unclear.
soundness: 2
strengths: 1.	Substantially reduces model size and FLOPs, making it suitable for deployment on resource-limited satellite or embedded platforms without significant performance degradation.
2.	Enhances the recovery of both spectral and spatial information.
summary: This paper introduces S2BNet, a Spatial–Spectral Binarized Neural Network tailored for remote sensing pan-sharpening. The network incorporates two novel components: a Spectral Redistribution Mechanism (SRM) and a Gabor Spatial Feature Amplifier (GSFA). Evaluated on four-band datasets—GaoFen-2, WorldView-2, and QuickBird—S2BNet achieves performance that is competitive with or superior to both full-precision and other binarized models, while maintaining significantly higher computational efficiency.
weaknesses: * Limited parameter analysis is provided.
* Generalization to datasets with more spectral bands remains unexplored.
* No runtime comparison is included. While “efficiency” is emphasized, inference time benchmarks on actual hardware are missing.
* Citation formatting should be corrected.
* The related work section should more systematically review prior binarized methods.
* Metrics and methods are tested with inconsistent presentation across tables. For example, the QNR is not comprehensive for full-resolution cases.
* The reference for metrics is missing. 
* The simulated process is missing.

---

confidence: 5
contribution: 2
initial_rating: 6
presentation: 2
questions: How does S2B-Conv differ mathematically from existing spectral recalibration (SE, FiLM) and orientation-sensitive convolutions (GaborNet, DCTNet)?

Can the authors provide quantitative analyses on spectral distortion reduction introduced by SRM, beyond PSNR/SSIM metrics?

What is the real computational saving in milliseconds or watts on embedded hardware compared to full-precision models?

How robust is S2BNet under cross-sensor transfer (e.g., training on GF-2, testing on WV-2)?

Figures should be enlarged and restructured for readability—particularly the architecture and ablation visualizations.
soundness: 2
strengths: Motivation relevance: The work targets a meaningful problem—deploying deep pan-sharpening models on resource-limited devices such as satellites—where efficiency is crucial.

Architecture clarity: The proposed S2B-Conv structure (with SRM and GSFA) is conceptually simple and well integrated into a binarized U-Net-like framework.

Empirical completeness: The paper provides quantitative and qualitative experiments, including ablation on both SRM and GSFA and comparisons with state-of-the-art (SOTA) binary and full-precision models.

Energy efficiency: The use of bitwise operations (XNOR, bit-count) is suitable for low-power environments, aligning with practical needs in embedded EO applications.
summary: This paper proposes a Spatial-Spectral Binarized Neural Network (S2BNet) for efficient panchromatic and multispectral image fusion (pan-sharpening). The method introduces a custom Spatial-Spectral Binarized Convolution (S2B-Conv) layer, consisting of two modules:
(1) Spectral-Redistribution Mechanism (SRM) — dynamically adjusts spectral distribution through data-driven affine scaling and bias;
(2) Gabor Spatial Feature Amplifier (GSFA) — captures multi-scale, multi-directional spatial patterns using Gabor kernels.
S2BNet is designed to achieve comparable reconstruction quality to full-precision networks while being lightweight enough for resource-limited satellite platforms. Experiments on GaoFen-2, WorldView-2, and QuickBird datasets are used to demonstrate that S2BNet outperforms other binary networks and approaches full-precision models in PSNR, SSIM, and QNR metrics
weaknesses: Limited originality / incremental contribution.
The technical innovation is minimal. Both SRM and GSFA are straightforward combinations of existing ideas: adaptive scaling (used in SE, FiLM) and Gabor filters (widely applied in image enhancement and texture analysis). Their integration into BNNs does not represent a fundamental advance in pan-sharpening or network design. The approach is essentially a routine adaptation of BiSRNet (Cai et al., 2023) with Gabor initialization.

Lack of theoretical or methodological depth.
There is no formal justification for why SRM or GSFA mitigates spectral distortion or anisotropy introduced by binarization. The discussion is qualitative and lacks rigorous analysis of spectral error propagation or frequency response behavior.

Overstated performance claims.
While numerical improvements over other BNNs (~0.3–0.5 dB PSNR) are reported, these are minor and may not be statistically significant. Results still lag behind top full-precision models (e.g., FAMENet, CANNet). The claim that S2BNet “outperforms most full-precision baselines” is exaggerated.

Insufficient novelty compared with prior lightweight or quantized models.
The paper overlooks related quantization-aware pan-sharpening or efficient Transformer works (e.g., HyperTransformer, LitePNN). The positioning of S2BNet as a novel direction in efficient fusion is weak.

Poor figure readability.
Figures (e.g., Fig. 1, 2, 3) are too small, and the text is illegible. Important architectural components and qualitative comparisons are difficult to interpret, limiting reproducibility.

Limited generalization and scalability.
All experiments are conducted on moderate-resolution datasets; there is no evidence that S2BNet scales to very large images or complex multimodal datasets. The efficiency discussion also lacks actual inference latency or hardware deployment results.

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: see the Weaknesses
soundness: 2
strengths: 1. First BNN for PAN-MS fusion; SRM & GSFA are novel in the binary context.
2. multi-sensor, multi-metric evaluation; ablation of each submodule
3. Overall pipeline easy to follow; equations complete. The presentation is good.
summary: This paper introduces S2BNet, a binary neural network tailored for panchromatic–multispectral pansharpening. The core innovation is the S2B-Conv that contains two novel sub-modules:
Spectral-Redistribution Mechanism – learnable affine re-scaling of each spectral band to combat spectral distortion caused by heterogeneous PAN/LR-MS distributions;
Gabor Spatial Feature Amplifier – randomly-selected Gabor kernels that enrich binary filters with multi-scale / anisotropic cues before binarization.
Stacking S2B-Conv units into a light U-Net yields S2BNet, which is trained end-to-end with pure L1 loss. Extensive experiments on WorldView-2, GaoFen-2 and QuickBird show S2BNet outperforming every published BNN (+2.5 dB PSNR gap) and many full-precision models, while keeping ≈ 1/32 parameters and ≈ 1/60 FLOPs of the latter.
weaknesses: 1.This paper is not the first to apply binarized neural networks to pansharpening.
The paper titled "Binarized Neural Network for Multi-spectral Image Fusion" holds that distinction.
2.The authors claim that their method surpasses other BNNs by more than 2 dB in PSNR; however, Table 1 shows that it is only about 0.2 dB higher than BiSRNet.
3.Likewise, Table 1 shows that the proposed method has three times the parameters of BiSRNet and significantly higher FLOPs. Is the reported performance gain entirely due to this larger model capacity rather than to the novel SRM/GSFA components?
4.From Figure 2 it is completely impossible to tell which method produces higher visual-quality fused images, and the method names in the figure are somewhat blurry.
5.The motivation behind the proposed core modules—the Spectral-Redistribution Mechanism and the Gabor Spatial Feature Amplifier—is unclear. There is only some textual explanation, but no other intuitive or quantifiable evidence is provided.

---

confidence: 3
contribution: 2
initial_rating: 4
presentation: 3
questions: Robustness and generalization:
- Add cross-sensor generalization (train on GF-2, test on WV-2/QB) and cross-resolution tests. These would strengthen claims that S2BNet is not sensor-specific.
- Report multiple seeds (mean±std) and significance tests for main results and ablations to contextualize ~0.3–0.5 dB gains.

Spectral fidelity and full-resolution analysis
- Provide per-band SAM and errors on spectral indices (e.g., NDVI) to directly assess spectral preservation.
- For full-resolution, include qualitative failure cases (e.g., thin structures, haze, strong PAN–MS mismatch).
soundness: 3
strengths: - Practical and targeted adaptation of BNNs to pansharpening, addressing two concrete pain points (spectral mismatch and anisotropic/multi-scale spatial textures) via SRM and GSFA.
- Extensive experiments on three common satellites (GF-2, WV-2, QB), with strong baselines including recent CNN/Transformer and binary methods; both quantitative and qualitative results; ablations showing each component’s effect and binarization placement trade-offs.
- Architecture and modules are explained clearly; equations for SRM and Gabor filters help reproducibility.
summary: The paper tackles remote sensing pansharpening—fusing a high-resolution panchromatic (PAN) image with a low-resolution multispectral (LR-MS) image to produce a high-resolution multispectral (HR-MS) image—under tight compute and memory constraints. The authors propose S2BNet, a largely binarized U-shaped network for efficient pan-sharpening. Its core is a Spatial-Spectral Binarized Convolution (S2B-Conv) that combines with Spectral-Redistribution Mechanism (SRM) and Gabor Spatial Feature Amplifier (GSFA).
S2BNet places these modules within a U-shaped architecture (dual encoders, bottleneck, dual decoders with skip connections), using binary convolutions for most layers and a small number of full-precision layers. Experiments on GaoFen-2, WorldView-2, and QuickBird (4-band) with standard Wald protocol and metrics (e.g., PSNR, SSIM, SAM, ERGAS, Q-index, QNR and its components) show that S2BNet outperforms other binary approaches and is competitive with some full-precision baselines. Ablation studies indicate both SRM and GSFA contribute measurable gains, and binarizing different parts trades accuracy for parameter/OPs reduction. The authors claim code will be released.
weaknesses: Limited novelty: SRM is essentially a bounded affine calibration and GSFA leverages classical Gabor filtering with randomized parameterization. While these yield practical gains, the conceptual advance is incremental relative to recent pansharpening models introducing stronger spatial–spectral mechanisms, such as transformer-based fusion [1], content-adaptive non-local convolution [2], adaptive kernel-shape learning [3], and frequency-domain mixture-of-experts [4].                    

Generalization and robustness: No cross-sensor generalization (train on one sensor, test on another). In full-resolution evaluation, per-band spectral fidelity (e.g., band-wise SAM) and spectral index errors (e.g., NDVI) are not reported. There is no sensitivity analysis for GSFA parameter ranges nor an exploration of learnable Gabor parameters, despite recent emphasis on adaptivity in spatial kernels and frequency components [2][3][4].

[1] Zhou, H., Liu, Q., & Wang, Y. (2022). PanFormer: A transformer based model for pan-sharpening. 2022 IEEE International Conference on Multimedia and Expo (ICME). arXiv:2203.02916

[2] Duan, Y., Wu, X., Deng, H., & Deng, L.-J. (2024). Content-adaptive non-local convolution for remote sensing pansharpening. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 27738–27747.

[3] Wang, X., Zheng, Z., Shao, J., Duan, Y., & Deng, L.-J. (2025). Adaptive rectangular convolution for remote sensing pansharpening. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 17872–17881.

[4] He, X., Yan, K., Li, R., Xie, C., Zhang, J., & Zhou, M. (2024). Frequency-adaptive pan-sharpening with mixture of experts. Proceedings of the AAAI Conference on Artificial Intelligence, 38(3), 2121–2129. https://doi.org/10.1609/aaai.v38i3.27984


Decision: Reject