HetSSNet: Spatial-Spectral Heterogeneous Graph Learning Network for Panchromatic and Multispectral Images Fusion
Authors: Mengting Ma
Yizhen Jiang
Mengjiao Zhao
Jiaxin Li
Wei Zhang
Venue: ICML 2025 poster (2025)
Keywords: Image Fusion
heterogeneous graph
remote sensing pansharpening
Primary area: applications->computer_vision

Abstract: Remote sensing pansharpening aims to reconstruct spatial-spectral properties during the fusion of panchromatic (PAN) images and low-
resolution multi-spectral (LR-MS) images, finally generating the high-resolution multi-spectral (HR-
MS) images. In the mainstream modeling strategies, i.e., CNN and Transformer, the input images are treated as the equal-sized grid of pixels in the Euclidean space. They have limitations in facing remote sensing images with irregular ground objects. Graph is the more flexible structure, however, there are two major challenges when modeling spatial-spectral properties with graph: 1) constructing the customized graph structure for spatial-spectral relationship priors; 2) learning the unified spatial-spectral representation through the graph. To address these challenges, we propose the spatial-spectral heterogeneous graph learning network, named HetSSNet.
Specifically, HetSSNet initially constructs the heterogeneous graph structure for pansharpening, which explicitly describes pansharpening-specific relationships. Subsequently, the basic relationship pattern generation module is designed to extract the multiple relationship patterns from the heterogeneous graph. Finally, relationship pattern aggregation module is exploited to collaboratively
learn unified spatial-spectral representation across different relationships among nodes with adaptive importance learning from local and global perspectives. Extensive experiments demonstrate the significant superiority and generalization of HetSSNet.

HetSSNet: Spatial-Spectral Heterogeneous Graph Learning Network for Panchromatic and Multispectrall Images Fusion
Mengting Ma" Yizhen Jiang" 2 Mengjiao Zhao? Jiaxin Li Wei Zhang?
Abstract
SSNet.
Remote sensing pansharpening aims to recon- struct spatial-spectral properties during the fu- sion of panchromatic (PAN) images and low- resolution multi-spectral (LR-MS) images, finally generating the high-resolution multi-spectral (HR- MS) images. In the mainstream modeling strate- gies, i.c., CNN and Transformer, the input im- ages are treated as the equal-sized grid of pixels in the Euclidean space. They have limitations in facing remote sensing images with irregular ground objects. Graph is the more flexible struc- ture. however, there are two major challenges when modeling spatial-spectral properties with graph: 1) constructing the customized graph struc- ture for spatial-specttre relationship priors; 2) learning the unified spatial-spectee representa- tion through the graph. To address these chal- lenges, we propose the spatial-spectral heteroge- neous graph learning network, named HetSSNet. Specifically, HetSSNet initially constructs the het- erogeneous graph structure for pansharpening. which explicitly describes pansharpening-speee relationships. Subsequently, the basic relationship pattern generation module is designed to extract the multiple relationship patterns from the hetero- geneous graph. Finally, relationship pattern ag- gregation module is exploited to collaboratively learn unified spatial-spectra representation across different relationships among nodes with adaptive importance learning from local and global per- spectives. Extensive experiments demonstrate the significant superiority and generalization of Het-
1. Introduction
With the increasing demand of earth observation and mon- itoring, existing optical satellites (e.g., GaoFen-2, Quick- Bird) can simultancously record bundled low-resolution multispectral (LR-MS) and high-resolution panchromatic (PAN) images from the same scene (Ma et al., 2025b;a), Due to physical limitations of existing satellite sensors, the recorded LR-MS image usually includes rich spectral prop- erty but relatively sparse spatial property, while their cor- responding PAN image contains abundant spatial property but sparse spectral property. Since the remote sensing im- age with rich spatial-spectral properties, i.e., high-resolution multispectral (HR-MS) image, are crucial for practical appli- cations (Li et al., 2022; Han et al., 2024; Asokan & Anitha, 2019; Cheng & Han, 2016; Li et al., 2022), pansharpen- ing technique, which could obtain the HR-MS image by reconstructing -patial-spectaa properties during the fusion of the recorded PAN and LR-MS images, has been widely explored (Xing al., 2023).
Conventional pansharpening methods include component substitution (CS) approaches (Carper et al., 1990), multi- resolution analysis (MRA) methods (Khan et al., 2008), and variational optimization (VO) methods (Deng et al., 2019). These methods only have the ability of shallow non-linear representation; therefore, it is difficult achieve good per- formance of spatial and spectral properties reconstruction. Recently, pansharpening has witnessed significant progress with the development of deep learning. However, it is taken for granted that the mainstream learning based models pro- pose to process the input LR-MS and PAN images as a regu- lar grid of pixels in the Euclidean space, i.e., treat all pixels of LR-MS and PAN images features a fairly rigid manner. For example, in the CNN-based methods (e.g.. DCFNet (Wu et al., 2022) and MDCUN (Yang et al., 2022)), each pixels of PAN and LR-MS images is rigidly designated to commu- nicate with its nearest neighbors; in the Transformer-basee model (e.g., HyperTransfromm CTINN (Zhou et al., 2022b)), all pixels of LR-MS and PAN images are assigned equal-sized attention grids for atten-
Equal contribution 'School of Software Technology, Zhejiang University, Hangzhou, China 'School of Computer Science and Technology, Zhejiang University, Hangzhou, China 'School of Computer Science and Technology (National Exemplary Software School), Chongqing University of Posts and Telecommunications, Chongqing. China. Correspondence to: Wei Zhang <estzhang- wei@zju.edin wei@zju.edu.cn>..
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
(Bandara & Patel, 2022) and

---

HetSSNet: Spatial-Spectral Heterogeneous Graph Learning Network for Panchromatic and Multispectral Images Fusion
tion operations, trying to model all potential relationships between PAN and LR-MS images.
we propose the relationship pattern aggregation mod- ule from the local and global perspectives, so that our model adaptively learn unified representation for spatial-spectral properties reconstruction.
Remote sensing images consist of relatively stable ground objects, such as oceans, forests, buildings, and streets, etc. (Duan et al., 2024). Since these ground objects are usu- ally not quadrate whose shape is irregular, the commonly- used grid structures in the modeling architecture like CNN and Transformer are redundant and inflexible to process them. In contrast, the graph structure is the more flexible topology (Han et al., 2022) to represent complex relation- ships (i.e., graph edges) between different attributes (i.e., graph nodes). However, there are two challenges to model spatial--pectral relationships with graph: (i) designing the customized graph structure in non-Fuclidean space, which describes pansharpening-spen
Extensive experimental results on three datasets show that our HetSSNet not only achieves favorable perfor- mance against previous SOTA approaches, but also generalizes well in real-world full-resolution scenes.
2. Related Work
2.1. Learning-based pansharpening method
In recent years, deep learning have dominated the remote sensing pansharpening community. These techniques lever- age the powerful feature learning and nonlinear fitting ca- pabilities inherent in neural networks, significantly outper- forming traditional approaches. The focus of pansharpening is to model the spatial-spectral relationship during the fusion of PAN and LR-MS images. Recently, CNN-based models like PanNet (Yang et al., 2017), SRPPNN (Cai & Huang, 2020; Li et al., 2024; 2023a;b)and DCFNet (Wu et al., 2022) treat PAN and LR-MS images as a regular grid of pixels. In Transfromer-basse models like syperTransfrome (Bandara & Patel, 2022) and CTINN (Zhou et al., 2022a), all pixels of feature maps extracted from PAN and LR-MS images are as- signed to equal-sized attention grids for attention operations, trying to model all potential relationships between PAN and LR-MS images. Since the CNN and Transformer rigidly model relationships in Euclidean space, it is sub-optimal for remote sensing images with irregular objects. Graph is the more flexible modeling structure. In this paper, we construct the first spatial-spectren heterogeneous graph for pansharp- ening, which models the spatial-spectral relationships in -un-Eucide space.
relationships priors, i.c., spatial relationship of PAN image, the intra-spectra rela- tionship of LR-MS image, and the spectral relationship between LR-MS and PAN images, as analyzed in Sec. B of Appendix; (ii) modeling spatial and spectral relationships with one graph structure, which facilitates maintaining the balance of reconstructed spatial-spectral properties.
To address the aforementioned challenges, we propose the spatial-spectral heterogeneous graph learning network (HetSSNet) for remote sensing pansharpening that learns unified spatial-spectral relationships with one customized graph. Specifically, the spatial-spectral heterogeneous graph is first constructed for pansharpening, named HetSS- Graph, which consists of multiple heterogeneous nodes and edges, allowing the provided ppatial-spectri) relationship priors to be clearly described in the MetSS-Graphe Then, we design the basic relationship pattern generation module to extract the multiple spatial-spectra relationship patterns between nodes in HetSS-Graph. From the local and global perspec- tives, we design the spatial-spectral relationship aggrega- tion module, which can aggregate the local information with adaptive importance learning of all basic relationship patterns, as well as learn global relevant information ac- cording to the similarity of global-wise relationship pat- terns between graph nodes. HetSSNet presents the first exploration in learning unified representations from multi- ple pansharpening- properties reconstruction.
2.2. Graph Representation Learning
GCN eases the assumption of prior conditions, which takes the research object as the node and the correlation or simi- larity between objects the edge. It can deal with complex paired interactions and integrate global spatial data, make full use of the internal relations between objects, and mine invisible relations between objects. In recent years, the graph convolution theory has developed rapidly. It has not only been widely applied to various high-level vision tasks, such as action recognition (Zhao et al., 2019) and semantic segmentation (Li et al., 2020), (Qi et al., 2017), but also started to be used to solve low-level vision tasks, such as image inpainting (Wadhwa et al., 2021), image deraining (Fu et al., 2021), and image denoising (Valsesia et al., 2019), Furthermore, dual GCNs (Zhang et al., 2019) with different mapping strategies become popular. Bandara et al. (Bandara et al., 2022) proposed spatial and interaction space graph
-ew.sarrreennnsspeiifffie specific relationships for spatial-spectral
We construct the first spatial-spectral heterogeneous graph structure, for remote sensing pansharpening. To the best of our knowledge, it's the first attempt for the potential of modeling pansharpening-spelee spatial- spectral relationships in non-euclidean space.
We customize basic relationship generation module, for extracting multiple base spatial-spectral relation- ship patterns.
Based on the extracted base relationship patterns,
2

---

HetSSNet: Spatial-Spectral Heterogencous Graph Learning Network for Panchromatic and Multispectral Images Fusion
Heterogeneous Graph Construction
Basic Relationship Pattern Generation
Relationship Pattem Aggregation
Weighed Concar
B
PNpatches
1000
LR-MS
Bitwise AND Operation
Normalize (B B!)
Rows Sum
Rows Sum
Rows Sum
GCN
patches
A
A
A
PAN
0-0 0
0
1 0- 0 0
O 0
0
0
1
0
0
NCE
LR-MS
0 0 0
0
0
GCN
Pooling
PAN
[4)
62
Gr
Aa
Weighed Sum
Local
HR-MS
spectral relationship between LR-MS and PAN intra spectral relationship of LR-MS spatial relationship of PAN
Figure 1. The overview of HetSSNet. Our HetSSNet consists of three components: spatial-spectral heterogeneous graph construction, basic relationship pattern generation and relationship pattern aggregation. According the provided relationship priors, construct the spatial-spectral heterogencous graph structure in non- -on-Euciddean space. Based on the constructed graph, we generate a series of basic spatial-spectral relationship pattern matrices, and finally aggregate these basic relationship patterns from the local and global perspectives.
reasoning to extract roads from acrial images. As far as we know, GCN is currently used for very little hyperspectral imagery. Qin et al. (Qin et al., 2018) and Wan et al. (Wan al., 2019) have related work, but it is limited to the task of hyperspectral image classification (Hong et al., 2020; 2021; Kang et al., 2020). Recently, GPCNet (Yan et al., 2022) uses two independent GCN to directly model the spatial and spectral relationships LR-MS and PAN image features' combination. In fact, it only uses GCN to model global relationships Euclidean space. Our method designs graph structures in non-European space for pansharpening, and uses GCN to learn the graph node representations from both local and global perspectives.
between same node pairs, the graph is called as Attributed multiplex heterogeneous graph.
Definition 2 (Basic Relationship Pattern). The basic relation- ship pattern between two node types O, and Oj in the het-
erogeneous graph is defined as O; [11]6//r]&&--<(/imm))
O which describes the complex relationships between two nodes, where [-] denotes optional, and at least one relation r; exists. Notice that for heterogeneous graph with R relationships, it can generate up to 2 R] 1 basic relation- ship patterns. Taking the our constructed HetSS -rapph in Fig. as an example, it has three types of edges, so it could generate up to 7 basic relationship patterns.
Based on the above definitions, we formally defined our studied problem for multiplex spatial-spectral relationships learning over the constructed heterogeneous graph.
3. Method
3.1. Problem Definition
Definition I (Antributed Multiplex Heterogeneous Graph). Given the defined graph 9 = (V, Fr, U), where V is set of nodes, E is set of observed edges. With the consideration of node and edge heterogeneity, we define the node type and edge type mapping functions as dz V + O and wh: E + R. where O and R denote the set of all node types and the set of all edge types, respectively. Each node v E V belongs to a particular node type, and each edge e E F. is categorized into a specific edge type. if the U is the matrix that consists of node attribute features for all nodes, where each row is the associated node feature vector of node vi, and | + R > 2, and there exist different types of edges
3.2. Overall Architecture
As illustrated in Fig. 1, we present the details of our HetSS- Net with the overall architecture. Particularly, our HetSSNet consists of three key components: (i) spatial-spectral het- erogeneous graph construction module (ii) Basic relation- ship pattern generation module, (iii) Relationship pattern aggregation module. Spatial-spectral heterogeneous graph structure is first constructed which explicitly describes the provided relationships priors for pansharpen- ing. Basic relationship pattern generation module is used extract all basic spatial-spectral relationship patterns from
3

---

HetSSNet: Spatial-Spectral Heterogencous Graph Learning Network for Panchromatic and Multispectral Images Fusion
the constructed heterogeneous graph, so to make full use of the multiple spatial-spectral relationships in the hetero- gencous graph. Relationship pattern aggregation module is designed to collaboratively learn node representations across different spatial-spectral relationships among nodes with adaptive importance learning from the local and global perspectives, finally fuse them to obtain the final output representation.
3.4. Basic Relationship Pattern Generation Module
To make full use of the complex spatial-spectral relation- ships between nodes in the constructed heterogeneous graph, we first design a basic relationship pattern generation mod- ule that could directly extract all the basic spatial-spectral relationship pattern matrices from the constructed HetSS- Graph.
We first decouple the heterogeneous graph according to the type of edges. Let {A, R 1,2,3] de- note the basic adjacency matrices, where is the number of all nodes in the graph. Then, each adjacency matrix and a corresponding logical variable (i.e., 0 or 1) are oper- ated with the XNOR to generate 3 intermediate matrices {Á, E |55xx5n|r | = 1,2,3]. Here, a logical variable of 1 is taken if the relation represented by the adjacency matrix is preserved in the basic spatial-spectral relationship pat- ter,, and 0 otherwise. Finally, the intermediate matrices bitwise AND operated to obtain the basic spatial-spectral relationship pattern matrices. That is, these relations corre- sponding to logical variable value of 1 are retained in the final basic spatial- relationship pattern. Notice that if zero matrix is obtained, then this basic spatial- relationship pattern does not exist in the graph. By adjusting the logical variables, all the basic spatial-spectral relation- ship pattern matrices {A, R5ax60/1 1, ..... 7} can be obtained.
3.3. Spatiall-peettral Heterogeneous Graph Construction
The spatial-spectral heterogeneous graph structure (HetSS- Graph) is denoted as the attributed multiplex heterogeneous graph, which consists of multiple heterogeneous nodes and edges, explicitly describing pansharpening -ppccii relation- ship priors, i.e., spatial relationship of PAN image, intra- spectra relationship of LR-MS image and spectral relation- ship between PAN and LR-MS images, as shown in Fig. 1.
Node in HraSS-Graph 1) The first type. Given a PAN image P E # we divide it into N overlapping patches. By transforming each patch of PAN image into a feature vector x; E R,, we have Xp = X2, -. .,x]] where d is the feature dimension and i = ..,,,...NN N. These features could be viewed as a set of unordered nodes which are denoted as Vp = (V1, .., V. 2) The second type. Given a LR-MS image L Nxxxx we also divide it into N overlapping patches. For the LR-MS image patch No R x N X4 we transform each spectral band into feature vector y E TRd b 1, 2, 3, 4, and we have Y; = {y] 2 .. where d is the feature dimension and i = 1,2, .2....... N. These features can be viewed as the second type of nodes which are denoted as VL = [v.. v?, .....
3.5. Relationship Pattern Aggregation Module
To automatically capture local information and global rele- vant information across different complex relationship pat- tern between nodes, we design the relationship pattern ag- gregation module from the local and global perspective.
Edge in MetsS-Graph 1) The first type. For the first type of node Vi, we use the k-nearest neighbor algorithm to find k nodes that have the most similar features to that node, and connect edges between node v; and these neighboring nodes set Net (v:). and add the first type of edge directed from vj to v; for all N (v,). The first type of edge establishes the spatial relationship of PAN image. 2) The second type. For the second type of node v/. we find its k nearest neighbors N (vf) and add the second type of edge directed from v to va for all N (v:). The second type of edge establishes the intra-spectra relationship of LR-MS image. 3) The third type. Given the first type of node v, and the second type of node vl. we add the third type of edge directed from vp for all N (v?) to v; for all A (v:). The third type of edge establishes the spectral relationship between LR-MS image and PAN image. In particular, the each added edge is measured by cosine similarity.
Local-wise Aggregation. The overall process of local-wise aggregation is shown in the Fig. 1. After obtaining basic spatial-spectral relationship patterns, the local-wise aggrega- tion aims to aggregate features from the complex structures among nodes from the local perspective by differentiating each basic spatial-spectral relationship pattern with different importance.
Specifically, the local-wise aggregation first uses a set of learnable weight parameters a, to aggregate basic spatial- spectral relationship patterns
Å ALooal
Σ C,,Ar,
(I)
where N is the number of obtained basic spatial-spectral relationship patterns. Then we feed the aggregated matrix A₁ into the graph convolution network (GCN). Following MHGCN (Yu et al., 2022b), our convolution also adopts the idea of simplifying GCN, that is, no nonlinear activation function is used. The single-layer GCN can effectively learn
4

---

HetSSNet: Spatial-Spectral Heterogencous Graph Learning Network for Panchromatic and Multispectral Images Fusion
the node representation that contains interaction information in all basic spatial-spectral relationship patterns. The I-layer node representation could been denoted as:
tern similarities:
H'Globat AClobal U-
W Global
W Global
Locat = Locat U
WT WLocal
W Locat
Local
(2)
The last layer output Global is denoted as the global node representation Glocal-
where U is the node attribute matrix, where each row is the associated node feature vector of Vp and VL. WT Local the learnable weights of inh layer, i 1.2 ..... I. We finally fuse outputs of all layers to obtain the local-wise node representation as:
The Final Output. Contrastive learning has demonstrated its superiority in various graph learning tasks. Inspired by that, we propose to use contrastive learning to empower the representation learning ability of the model that maximizes the agreement of node representations learned from a local perspective and a global perspective.
Local 7 E
1 H Local
exp (s( Locat,, HGLobal,))
Edt
Σ EEV
log
Ejev (xp(s(HLocat)) HGtoobat, ///T)
Global-wise Aggregation. The purpose of global-wise aggregation is to aggregate the features between nodes from the global perspective based on the similarity of global spatial-spectral relationship patterns between nodes. The overall process of global-wise aggregation is shown in the Fig. 1.
(8)
where H is the local bocallwwse/global-wiis node representa- tion of the i-th node, s(,,,) denotes the cosine similarity function, and r is the tunable temperature hyperparameter to adjust the scale for softmax. This contrastive learning allows the local spatial-spectral relationship pattern and global spatial-spectral relationship pattern to collaboratively supervise each other, which enhances the node representa- tion learning. Eventually, we use the outputs of both the local-wise aggregation and the global-wist aggregation to obtain the final node representation H E Rnnd through the average pooling operation for downstream tasks as:
Specifically, we first generate the matrix to represent the global spatial-spectral relationship pattern of nodes based on the obtained basic spatial-spectral relationship patterns. In particular, we first add the rows to get a column vector for each basic spatial-spectrac relationship pattern matrix. Here, each column vector describes the number of corre- sponding basic spatial-spectral relationship patterns of all nodes relative to all other nodes. Because different basic spatial--pectral relationship patterns may have different con- tributions to the similarity when calculating the similarity of global spatial-spectral relationship patterns, we use a set of learnable weights 3; concatenate pattern-specifie col- umn vectors to obtain the global spatial-spectral relationship pattern matrix B E RaxN as:
H 1 (HLocal + HHGlobat).
(9)
2
3.6. Optimization
We adopt the L1 loss function, which ensures that the net- work output H is as close as possible to the corresponding ground truth image GT.
L
H - GT
(10)
Bii())
Σ Am(s.))
(4) Finally, we integrate L1 loss with our contrastive learning loss to optimize our model jointly:
B = Concat (Bi... ... ......B)--.. Ag.
(5)
-= L1 + +
(11)
where 7 is the hyperparameter for tuning the importance of contrastive learning.
where Bnn is the column vector corresponding to the m-th basic spatial-spectral relationship pattern, Ag Diag($1,3.. ... ,8)) is the learnable diagonal matrix. Then we multiply the global spatial-spectral relationship pattern matrix by its transpose and normalize it to obtain the global spatial-spectreg relationship pattern similarity matrix:
4. Experiments
4.1. Dataset and Benchmark
1) Dataset: We conduct experiments using the widely recog- nized WorldView-3, QuickBird and GaoFen-2 datasets (Ma et al., 2024). The WorldView-3 dataset consists of instances acquired by the sensor aboard the WorldView-3 satellite. This sensor captures data which covers wavelengths from 0.4 to μa,, with spatial resolution of 1.24 m. The Quick- Bird dataset consists of instances acquired by the sensor
A normalize (B ).). -B-B
(6) rela-
Intuitively, the more similar the global tionship pattern of two nodes is, the greater their weight in the similarity matrix is. Next, we input the global spatial- spectral relationship pattern similarity matrix into the GCN for information aggregation, so as to obtain the node repre- sentation from the global spatial-spectral relationship pat-
5

---

HetSSNet: Spatial-Spectral Heterogencous Graph Learning Network for Panchromatic and Multispectral Images Fusion
Dalasets
GaoFen-22
QuickBird
WorldView-3
assess the generalization ability of our method, we test it on the corresponding full-resolution real-world scene. Since there are not GT images available for the full-resolution dataset, we use three non-reference metries to evaluate the performance of the model: Spectral Distortion Index (Dx), Spatial Distortion Index (D,), and No-Reference Quality (QNR) (Alparone et al., 2008). Dx concerns spectral shifts between LR-MS and fused HR-MS images, while D, con- siders spatial disparities between PAN and HR-MS images. QNR is composite metric measuring spatial and spectral deviations. In these metrics, ideal values for SAM, ERGAS, D., and Dx are O, while higher values indicate better model performance for the remaining metrics.
Raw image
5
3
Bit depth
11 35725
10
10
Training set
12119
11856
Testing set
356
3639
LR MS image size
332
332
32 32
x x 4
PAN image size
128
128 x
128 x 128 x 1
128 x 128 x 1
Target HR-MS image size
128
128 x
128 x 128 x 4
128% x 128 x
Table 1. The detailed dataset information (GaoFen-2. WorldView- 3 and QuickBird).
aboard the QuickBird satellite. This sensor captures data across four spectral bands, covering wavelengths from 0.45 to 0.9 ym, with a spatial resolution 2.4 m. The images in the GaoFen-2 dataset are collected by the sensor onboard the 2aoFen-2 satellite, which records data across four spectral bands within the wavelength range of 0.45 - 0.89 jam. Ad- ditionally, this sensor provides a spatial resolution of 3.2m. The data generation process adheres strictly to Wald's proto- col (Wald et al., 1997), with comprehensive details provided in (Deng et al., 2022). As shown in Tab. 1, we present the detailed information of traning dataset and testing dataset in the experiment.
4.3. Training Details
During the training of our networks on the Wordview-3, QuickBird, and GaoFen-2 datasets, the number of iterations is set to 30000, 28000, and 28000, respectively. The Adam optimizer with B1 = 0.9 and 82 0.999 is employed for the optimization, and the batch size is set as 4. Our proposed method is trained for 30000 iterations. The initial learning rate is set to 1 x 10-1, and decays by 0.85 after every 3000 iterations. Additionally, all the experiments are conducted in the PyTorch framework with four NVIDIA RTX A6000 GPUs.
2) Benchmark: We compare our method with the follow- ing two groups of methods: (1) Learning-based methods: SRPPNN (Cai & Huang. 2020), DCFNet (Wu et al., 2022), CTINN (Zhou et al., 2022b), Hyperfomer (Bandara & Patel, 2022), SFIINet (Zhou et al., 2022c), BiMPan (Hou et al.,
4.4. Experiment Results
In Tab. 2, we report the average evaluation metrics of our proposed method and other selected algorithms on the three datasets, i.e., WorldView-3, Quick and GaoFen-2. The best and second-best results are highlighted in boldface and underlined, respectively. It is evident that our method out- performs previous methods in almost all metrics across all three datasets. In particular, all the evaluation metrics of the learning- based methods significantly outperform the tra- ditional methods on the three adopted datasets, which is consistent with the subjective analysis mentioned above and aligns with our understanding. This is because traditional methods are often based on hand-crafted designs, limiting properties for
2023), MDCUN (Yang al., 2022), MSDDN (He et al., 2023), LGTEUN (Li et al., 2023c), FAMENet (Xuanhua et al., 2024), GPCNet (Yan et al., 2022). FusionMamba (Peng et al., 2024), WINet (Tan et al., 2024) and HFIN (Zhang et al., 2024). (2) Traditional methods: SFIM (Liu, 2002), GS (Sandhu et al., 2021) and BROVEY (Gillespie et al., 2003). All comparison methods are re-trained on the adopted datasets, without directly using the experi- menial details in the original articles. A description of all comparison methods could be shown in the appendix.
4.2. Evaluation Metrics
the capability to reconstruct target HR-MS images. The experimental results also demon- strate that HetSS- excels in spatial texture reconstruc- tion and spectral fidelity, as evidenced by its minimum SAM value and maximum SSIM and PSNR values. Specifically, CNN and Transfromer methods (e.g., DCFNet and Hyper- Transformer) generally perform poorly on the QuickBird dataset, which is because the rigid modeling architecture limits their ability to handle complex and irregular objects in remote sensing images; GPCNet directly use GCNs to model spatial/spectral relationship between PAN and LR- MS images features, breaking the balance between spectral property recovery and spatial property reconstruction. Our HetSSNet designs the graph structure in the Non-euclidean
Following previous studies on pansharpening, five image quality assessment metrics (Yang et al., 2023) are employed for evaluation on reduccd-resolltion images, including spec- tral angle mapper (SAM), dimensionless global error in synthesis (ERGAS), the structural similarity (SSIM), the peak signal-to-nowl noise ratio (PSNR), and the spatial corre- lation coefficient (SCC) (Zhou et al., 1998). Specifically, PSNR measures the ratio between peak signal and noise in the image, reflecting the distortion level of the fusion result. SSIM and SCC effectively measure the spatial similarity of the results. SAM and ERGAS are used to measure angular and dynamic range differences between the fused image and the ground truth image, respectively. Additionally, to further
6

---

HetSSNet: Spatial-Spectral Heterogencous Graph Learning Network for Panchromatic and Multispectral Images Fusion
WorldView-3 SSIM SAM 4 ERGAS 4 SCC1
QuickBird
GaoFen-2
PSNR1
PSNR SSIM SAM, ERGAS J. SCC1
PSNR
SSIM
SAM. ERGAS $ SCC1
SFIM
30.982
0.749
0.076
6.497
0.821
33.463
0.818
0.036
3.071
0.792
37.840
0.917
0.042
3.836
0.806
BROVEY
30.857
0.761
0.086
5.506
0.819
30.109
0.837
0.077
2.473
0.739
36.046
0.902
0.036
2.261
0.835
GS
35.143
0.807
0.078
5.409
0.825
32.454
0.767
0.033
2.198
0.751
38.328
0.930
0.037
2.826
0.852
SRPPNN
38.642
0.887
0.066
5.244
0.904
33.966
0.886
0.082
2.840
0.875
40.918
0.961
0.036
2.683
0.911
DCFNet
34,429
0.862
0.098
6.373
0,568
34.661
0.912
0.072
2.573
0.848
47.055
0.988
0.027
1.270
0.912
CTINN
39.497
0.889
0.063
5.064
0.962
35.705
0.898
0.038
2.422
0.922
46.875
0.987
0,023
1.302
0,960
SFINNtt
39.026
0.885
0.064
5.248
0.643
34,459
0.893
0.041
2.811
0.912
47.195
0.988
0.022
1.267
0,980
Hyperformer
34.121
0.843
0.098
6.356
0.867
34.374
0.869
0.072
2.555
0.914
45.499
0,981
0.034
1.584
0.959
MDCUN
39.929
0.892
0.056
4.864
0.970
36.070
0.891
0.036
2.377
0.910
48.668
0.991
0.018
1.068
0.986
BiMPan
38.955
0.879
0.069
5.253
0.925
37.162
0.919
0.032
2.045
0.931
45,901
0.984
0.026
1.465
0,974
LGTTUNN
40.124
0.895
0.055
4,791
0.974
35.911
0.924
0.045
2.170
0.950
48.283
0.992
0.022
1.126
0,979
MSDDN
35.127
0.840
0.085
6.232
0.766
34.751
0.872
0.038
2.538
0.935
39.902
0,942
0.039
2.883
0,886
FAMENet
37.219
0.883
0,071
5.737
0.902
37.218
0.929
0.030
1.996
0.953
42.983
0,981
0.017
1.013
0.978
GPCNet
38.821
0.883
0.062
5.242
0.909
33,972
0.889
0.076
2.835
0.889
40.926
0.967
0.032
2.671
0.923
FusionMambu
38.727
0.872
0.069
5,437
0.934
34.021
0.881
0.047
2.931
0.904
46.242
0,976
0.027
1.278
0,972
WINet
39.781
0.887
0.056
4,769
0.969
37.209
0.927
0.032
2.212
0.931
49 405
0,984
0.016
0,968
0.977
HFIN
32.126
0.837
0.083
6.627
0.837
33.616
0.811
0.050
3.174
0.869
41.938
0.966
0.039
2.534
0.894
HetSSNet
|40.623
0.905
0.054
4.741
0.972
37.228
0.931
0.029
2.022
0.951
49.541
0.994
0.015
0.955
0.990
Table 2. The average quantitative results of reduced-resolution scenes over three datasets. Best results are highlighted in boldface, the second best one underlined.
space, and learns a unified representation of spatial-spectral properties in the graph structure, achieving competitive re- sults in all indicators.
Model
PSNR
SAM 4
SSIM 1
QNR
D 4
(a)
48.212
0.035
0.976
0.841
0.137
(b)
48.923
0.023
0.981
0.852
0.121
The subjective evaluation results are shown in Fig. 2 for the WorldView-3 dataset, using Sed-Greee-BBee
(c)
48.536
0.027
0.972
0.843
0.139
bands com- position for presentation. The first two rows present the pan-sharpened images, while the third and fourth rows dis- play the mean square error (MSE) maps between the pan- sharpened images and the HR-MS images. As evident from the results, our HetSSNet is capable of restoring more re- alistic textures than existing pansharpening methods and markedly retain the spectral property of ground objects. The traditional methods, i.e., SFIM, BROVEY, and GS struggle to represent details satisfactorily. leading to blurred building edges and spectral distortion. Similarly, the learning-bass based methods face challenges in generating sharp and clear tex- tures as well. For instance, SRPPNN and MSDDN produce results with notable color inconsistency. Although MDCUN and FAMENet produce visually more promising results, their eтror values are still higher than ours. The HetSS- Net generates results that are closer to the target HR-MS images with higher perceptual quality and lower reconstruc- tion errors. The visualization results on the QuickBird and 2 scenes are shown in Sec. D of Appendix.
(d)
49.221
0.019
0.991
0,857
0.112
Ours
49.541
0.015
0.994
0.860
0.107
Table 3. Ablation study of base relationship pattern extraction and relationship pattern aggregation.
4.5. Ablation Studies
To evaluate the effectiveness of each component of our model, we further conduct the ablation studics on the GaoFen-2 dataset.
Basic relationship pattern generation module. As shown in Tab. 3, in Model (a), we replace the basic relationship pattern generation module with the meta-path sampling strat- egy (Fu et al., 2020). In Model (b), we replace the basic relationship pattern generation module with the decoupled adjacency matrices (Yu et al., 2022a). Compared Model (a) and Model (b), the performance of our HetSSNet drops significantly on the GaoFen-2 dataset. This also demon- strates the contribution of the proposed relationship pattern generation module to the model performance improvement.
To demonstrate the generalization capability of the proposed method, conduct quantitative evaluations of our pre- trained model on 20 unseen full-resolution (real-worid) im- ages, the corresponding results are presented in Sec. D of Appendix.
Relationship pattern aggregation module. As shown in Tab. 3, in Model (c), we remove the local-wise relation- ship pattern aggregation module, that is, only global-wist -ise relationship pattern aggregation module is kept. In Model
7

---

HetSSNet: Spatial-Spectral Heterogencous Graph Learning Network for Panchromatic and Multispectral Images Fusion
30
25
SFIM
BROVEY
GS
SRPPNN
DCFNet
CTINN
ARFNet
Hyperformer
20
SFIINet
MDCUN
LGTEUN
MSDDN
FAMENet
GPCNet
Ours
GT
10
SFIM
BROVEY
GS
SRPPNN
DCFNet
CTINN
ARFNet
Hyperformer
SFIINet
MDCUN
LGTEUN
MSDDN
FAMENet
GPCNet
Ours
GT
Figure 2. Qualitative results of reduced-resolltion scene on the WorldView-3 dataset. Top group: the fused results. Bottom group: the error between fused results reference.
(d), only keep the local-wise relationship pattern ag- gregation module. Model (c) performs the worst and is significantly worse than Model (d). Local-wise aggrega- tion to aggregate local feature information, which is the intrinsic attribute of node representation and thus plays a decisive role in graph learning. Global-wise aggregation can pass feature information between nodes based on the sim- ilarity of spatial-spectral relationship patterns, effectively supplementing global relevant information to facilitate node representation. Model (c) and Model (d) are worse than the our method, which verifies that both play an important role in node representation learning of the HetSSNet for spatial-spectral properties reconstruction.
Layer
PSNR f
SAM 1
SSIM ^
QNR
D, 4
1=1 1
47.683
0.041
0.967
0.836
0.145
= 2
49.541
0.015
0.994
0.860
0.107
= 3
49.227
0.018
0.991
0.856
0.112
= 4
49.104
0.024
0.986
0.851
0.119
Table 4. Ablation study of aggregation layer.
PSNR
SAM 4
SSIM
QNR 1
D₂ 4
48.559
0.036
0.981
0.842
0.129
0.01
49.541
0.015
0.994
0.860
0.107
0.5
49.103
0.018
0.988
0.852
0.112
The number aggregation layer. As shown in Tab. 4, firstly, the performance of HetSSNet increases with the increasing number of layers. When the number of layers reaches to 3, the model has significant performance drop on several metrics. It should be emphasized that the purpose of this work is not to solve the swo-shhg problem of GCN, but to aggregate the relevant global information more effectively. Thus, even when the number of layers set small, e.g., 2, model also learn global spatial-spectral relationship to improve model performance.
48.629
0.028
0.987
0.845
0.125
Table 5. Ablation study hyperparameter 7 shown Eq. 11.
nate the node representation learning the two aggregation modules HetSSNet for improving graph node representa- tion performance. When 7 is set too large (e.g., 7 = 1), will affect the importance of the L1 loss in the model learn- ing, and thus damaging the reconstruction performance.
Effect of hyperparameter 7. From Tab. 5, we can find that our WettS-Graph achieves the best performance on almost all datasets when y falls near 0.03. The model performance improved on all datasets when y increases from 0 to 0.03, indicating that contrastive learning can effectively coordi-
5. Conclusion
This paper presents HetSSNet, a novel spatial-spectral heterogencous graph learning network dedicated to learn- ing unified spatial-spectral representation for pansharpen-
8

---

HetSSNet: Spatial-Spectral Heterogencous Graph Learning Network for Panchromatic and Multispectral Images Fusion
ing. We first construct the first spatial-spectral hetero- geneous graph structure to explicitly deseribe different pansharpening-speciff relationship priors. Based on the constructed heterogeneous graph structure, we propose the base relationship pattern generation module to extract mul- tiple spatial-spectral relationship patterns. Furthermore, we propose an relationship pattern aggregation module to learn local-wise and global-wise information among nodes Ex- perimental results show that our HetSSNet outperforms existing SOTA methods and especially excels in reconstruct- ing spectall properties. In the future, we will focus on evaluating our method under more challenging satellite image scenarios, e.g., there are fast-moving objects in the scenarios.
Carper, W., Lillesand, T., Kiefer, R., al. The of snienstty-hue-sturaan transformations for merging spot panchromatic and multispectral image data. Photogram- metric Engineering and remote sensing, 56(4))459-467, 1990.
Cheng, G. and Han, J. A survey on object detection in optical remote sensing images. ISPRS Journal of Pho- togrammetry and Remote Sensing, pp. 11-28, Jul 2016.
Deng, L.-J., Feng, M., and Tai, X.-C. The fusion of panchro- matic and multispectral remote sensing images via tensor- based sparse modeling and hyper-laplacian prior. Infor- mation Fusion, 52:76-89, 09776669,200 2019.
Deng, L.J.,, Vivone, G., Paoletti, M. E., Scarpa, G., He, J., Zhang, Y., Chanussot, J., and Plaza, A. Machine learning in pansharpening: A benchmark, from shallow to deep networks. IEEE Geoscience and Remote Sensing Magazine, 10(3):279-311 315, 2022.
Acknowledgements
This work was in part supported by the Science and Technol- ogy Innovation 2025 Major Project of Ningbo City (Grant No. 2023Z130).
Duan. Y., Wu, X., Deng, H., and Deng, L.-J. Content- adaptive non-local convolution for remote sensing pan- sharpening, In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 27738- 27747, 2024.
Impact Statement
This paper presents work whose goal is advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Fu, X., Zhang, J., Meng. Z.,, and King, I. MAGNN: metapath aggregated graph neural network for heterogeneous graph embedding. In Huang, Y., King, I., Liu, T., and van Steen, M. (eds.), WWW '20: The Web Conference 2020, Taipei, Taiwan, April 20-24, 2020, pp. / /IWCC2, 2020.
References
Alparone. L., Aiazzi, B., Baronti, S., Garzelli, A., Nencini, F., and Selva, M. Multispectral and panchromatic data fusion assessment without reference. Photogrammetric Engineering Remote Sensing, 74(2):193-2)) 200,900-22202200022 2008
ACM
Fu, X., Qi. Q. Zha, Z.-.,, Zhu, Y., and Ding, X. Rain streak removal via dual graph convolutional network. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 35, pp. 1352 1360, 2021.
Asokan, A. and Anitha, J. Change detection techniques for remote sensing applications: survey. Earth Science Informatics, pp. 143-160, Jun 2019.
Gillespie, A. R., Kahle, A. B., and Walker, R. E. Color enhancement of highly correlated images. ii channel ratio and "chromaticity" transformation techniques. Remote Sensing of Environment, pp. 343-365, Aug 2003.
Bandara, W. G. C. and Patel, V. M. Hypertransformer: A textural and spectral feature fusion transformer for pan- sharpening. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1767- 1777, 2022.
Han, K., Wang, Y., Guo, J., Tang, Y., and Wu, E. Vision gnn: An image is worth graph of nodes. Advances in neural information processing systems, 35:8291-8300 2022.
Bandara, W. G. C., Valanarasu, J. M. J., and Patel, V. M. Spin road mapper: Extracting roads from aerial images via spatial and interaction space graph reasoning for au- tonomous driving. In 2022 International Conference on Robotics and Automation (ICRA), pp. 343-350. IEFF, 2022.
Han, Z., Xu, S., Gao, L., Li, Z., and Zhang, B. Gretnet: Gaussian retentive network for hyperspectral image clas- sification. IEEE Geoscience and Remote Sensing Letters, 2024.
Cai, J. and Huang, B. Super-resolluion-guiie progressive pansharpening based on a deep convolutional neural net- work. IEEE Transactions on Geoscience and Remote Sensing, (9(6))2006-52)) 5220, 2020.
He, X., Yan, K., Zhang, J., L., Xie, C., Zhou, M., and Hong, D. Multi-scale dual-domain guidance network for san-sharpening. IEEE Transactions on Geoscience and Remote Sensing, 2023.

---

HetSSNet: Spatial-Spectral Hetcrogencous Graph Learning Network for Panchromatic and Multispectral Images Fusion
Hong, D., Gao, L.,, Yao, J., Zhang, B., Plaza, A., and Chanus- sot, J. Graph convolutional networks for hyperspectral image classification. IEEE Transactions Geoscience and Remote Sensing, 59(7):5966-55) 5978, 2020.
Li, X., Yang, Y., Zhao, Q. Shen, T., Lin, Z., and Liu, H. Spatial pyramid based graph reasoning for semantic seg- mentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8950- 8959, 2020.
Hong, D., Gao, L., Wu, X., Yao, J., and Zhang, B. Re- visiting graph convolutional networks with mini-batch sampling for hyperspectral image classification. In 2021 1th Workshop on Hyperspectral Imaging and Signal Pro- cessing: Evolution in Remote Sensing (WHISPERS), pp. 1-5, 2021.
Liu, J. G. Smoothing filter-based intensity modulation: A spectral preserve image fusion technique for improving spatial details. International Journal of Remote Sensing, pp. 3461-3472, Jul 2002.
Ma, M., Hu, C., Zhang, H., Ma, X., Feng, T., and Zhang, W. Crocfun: Cross-modal conditional fusion network for pansharpening. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing 2024.
Hou, J., Cao, Q. Ran, R., Liu, C., Li, J., and Deng, L.- Bidomain modeling paradigm for pansharpening. In Proceedings of the st ACM International Conference on Multimedia, pp. 347-357, 2023.
(ICASSP), Pp.
Kang, J., Bermandez-Belrann, R., Hong. D., Chanussot, J., and Plaza, A. Graph relation network: Modeling relations between scenes for multilabel remote-sensing image clas- sification and retrieval. IEEE Transactions on Geoscience and Remote Sensing, 59(5):3355 4369, 2020.
Ma, X., Lian, R.. Wu, Z. Guan, R., Hong, T., Zhao, M., Ma, M., Nie, J., Du, Z., Song, S., et al. A novel scene cou- pling semantic mask network for remote sensing image segmentation. ISPRS Journal of Photogrummelry and Remote Sensing. 221:44-63, 2025a.
Ma. X., Lian, R., Wu, Z., Guo, H., Yang, F. Ma, M., Wu, S., Du, Z., Zhang, W., and Song, S. Logcan++: Adaptive local-glober class-aware network for semantic segmen- tation of remote sensing images. IEEE Transactions on Geoscience and Remote Sensing, 2025b.
Khan, M. M., Chanussot, J., Condat, L., and Montanvert, A. Indusion: Fusion of multispectral and panchromatic images using the induction scaling technique. IEEE Geo- science and Remote Sensing Letters, 102, 2008
Li, J., Zheng, K., Yao, J., Gao, L., and Hong, D. Deep unsupervised blind hyperspectral and multispectral data fusion. IEEE Geoscience and Remote Sensing Letters, 19:1-5, 2022.
Peng. S., Zhu, X., Deng, H., Deng, L.-J,, and Lei, Z. Fu- sionmamba: Efficient remote sensing image fusion with state space model. IEEE Transactions on Geoscience and Remote Sensing. 2024.
Li, J., Zheng. K., Li. Z., Gao, L., and Jia, X. X-shaped inter- active autoencoders with cross-modality mutual learning for unsupervised hyperspectral image super-resolution. IEEE Transactions on Geoscience and Remote Sensing, 61:1-17, 2023a.
Qi, X., Liao, R., Jia. J., Fidler, S., and Urtasun, R. 3d graph neural networks for rgbd semantic segmentation. In Proceedings of the IEEE international conference on computer vision, pp. 5199-5208, 2017.
Qin, A., Shang, 7.,, Tian, J., Wang, Y., Zhang, T., and Tang, Y. Y. Spectral-spatial graph convolutional networks for semisupervised hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters, 16(2):241-245. 2018.
Li, J., Zheng, K., Liu, W., Z., Yu, H., and Ni, L. Model- guided coarse-to-fin toneee fusion network for unsupervised hyperspectral image super-resolution. IEEE Geoscience and Remote Sensing Letters, 20:1-5, 2023b.
Li, J.. Zheng, K., Gao, L., Ni, L., Huang, M., and Chanussot, J. Model-informed multistage unsupervised network for hyperspectral image super-resolution. IEEE Transactions on Geoscience and Remote Sensing, 2024.
Sandhu, K. S., Patil, S., Pumphrey, M. O., and Carter, A. Multi-trait machine and deep learning models for genomic selection using spectral information in wheat breeding program. Apr 2021.
Li, M., Liu, Y., Xiao, T., Huang, Y., and Yang, G. Local- global transformer enhanced unfolding network for pan- sharpening. In Proceedings of the Thirty-Secone Interna- tional Joint Conference on Artificial Intelligence, IJCAI 2023, 19th-25th August 2023, Macao, SAR, China, pp- 1071-1079. ijeai.org, 2023c.
Tan, J., Huang, J., Zheng, N., Zhou, M., Yan, K., Hong, D., and Zhao, F. Revisiting spatial-frequency information integration from a hierarchical perspective for panchro- matic and multi-spectral image fusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2024.
10

---

HetSSNet: Spatial-Spectral Heterogencous Graph Learning Network for Panchromatic and Multispectral Images Fusion
Valsesia, D., Fracastoro, G., and Magli, E. Image denoising with graph-convolutional neural networks. In 20/9 IEFE international conference image processing (ICIP), pp. IEEE, 2019.
In Zhang, A. and Rangwala, H. (eds.), KDD '22: The 28th ACM SIGKDD Conference Knowledge Discovery and Data Mining, Washington, DC, USA, August 14 18, 2022, pp.
ACM, 2022a.
Wadhwa, G., Dhall, A., Murala, S., and Tariq, U. Hyperreal- istic image inpainting with hypergraphs. In Proceedings of the IEEE/CVF winter conference applications of computer vision, pp. 3912-39211 2021.
Yu, P., Fu, C., Yu, Y., Huang, C., Zhao, Z., and Dong, J. Multiplex heterogeneous graph convolutional network. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 2377-2381 2022b.
Wald, L., Ranchin, T., and Mangolini, M. Fusion of satel- lite images of different spatial resolutions: Assessing the quality of resulting images. Photogrammetric engineer- ing and remote sensing, 63(6):691-69) 699, 1997.
Zhang, J.. He, X., Yan, K., Cao. K., Li, R., Xie, C., Zhou, M., and Hong, D. Pan-sharpening with wavelct-enhanced high-frequener information. IEEE Transactions on Geo- science and Remote Sensing, 62:1 2024.
Wan, S., Gong. C., Zhong, P., Du, B., Zhang, L.,, and Yang, Multiscale dynamic graph convolutional network for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 58(5)33162-3177
Zhang, L., X., Arnab, A., Yang, K., Tong, Y., and Torr, P. H. Dual graph convolutional network for semantic segmentation. arXiv preprint arXiv: / 1909.06/21, 2019.
2019.
Zhao, L., Peng, X., Tian, Y., Kapadia, M., and Metaxas, D. N. Semantic graph convolutional networks for 3d human pose regression. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 3425-3435, 2019.
Wu, X., Huang, T.-Z., Deng, L.-J,, and Zhang, T.-J. Dy- namic cross feature fusion for remote sensing pansharp- ening. In 2021 IEEE/CVF International Conference Computer Vision (ICCV), Mar 2022.
Xing. Y.. Zhang, Y., He, H., Zhang, X., and Zhang, Y. Pansharpening via frequency-awan fusion network with explicit similarity constraints. IEEE Transactions Geoscience and Remote Sensing, pp. 1-1, Jan 2023.
Zhou, H., Liu. Q., and Wang. Y. Panformer: transformer based model for pan-sharpening. In 2022 IEEE Interna- tional Conference on Multimedia and Expo (ICME), pp. 1-6, 2022a.
Xuanhua, H., Keyu, Y., Rui, L., Chengjun, X., Jie, Z., and Man, Z. Frequency-adaptive aan-sharpening with mixture of experts. In 2024 Conference on Innovative Applica- lions of Artificial Intelligence (AAAI), 2024.
Zhou, J., Civco, D. L., and Silander, J. A. A wavelet transform method to merge landsat and spot panchro- matic data. International Journal of Remote Sensing, pp.
743-757, Jan 1998.
Yan, K., Zhou, M., Liu, L., Xie, C., and Hong, D. When pan- sharpening meets graph convolution network and knowl- edge distillation. IEEE Transactions on Geoscience and Remote Sensing, 60:1-15 2022.
Zhou, M., Huang, J., Fang. Y., Fu, X., and Liu, A. Pan- sharpening with customized transformer and invertible neural network. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pp. 3553 3361,, 2022b.
Yang. G., Zhou, M., Yan, K., Liu, A., Fu, X., and Wang, F. cemmoryaaugmnted deep conditional unfolding network for pan-sharpening. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 1788-1797 2022.
Zhou, M., Huang, J., Yan, K., Yu, H., Fu, X., Liu, A., Wei, X., and Zhao, F. Spatial-frequency domain information integration for pan-sharpening. In European Conference on Computer Vision, pp. 274-291, 2022c.
Yang, G., Cao, X., Xiao, W., Zhou, M., Liu, A., chen, X., and Meng, D. Panflownet: A flow-based deep network for pan-sharpening. May 2023.
Yang, J., Fu, X., Hu, Y., Huang, Y., Ding, X., and Paisley, Pannet: deep network architecture for pan-sharpening. In Proceedings of the IEEE international conference on computer vision, pp. 5449-5457 2017.
Yu, P., Fu, C., Yu, Y., Huang, C., Zhao, Z., and Dong, J. Multiplex heterogeneous graph convolutional network.

---

HetSSNet: Spatial-Spectral Heterogencous Graph Learning Network for Panchromatic and Multispectral Images Fusion
A. Appendix Outline
In this supplementary material, we provide more details our HetSSNet as follows:
Sec. B conducts detailed analysis and reveals the complex relationship priors for pansharpening, as illustrated in Fig. 3 and Fig. 4,
Sec. C describes more details about each benchmark.
Sec. D reports more quantitative and visual results, i.e., visual results on dellucddrresluttin scene, quantitative results full-resolution dataset, quantitative results on eight-band WorldView-3 dataset and visual results full-resolution scene.
Sec. E reports complexity analysis of our HetSSNet.
B. Pansharpening-ppecffic relationship priors
In this section, we reveal a spatial property-related relationship prior (Prior 1) and a spectral property-related relationship prior (Prior 2), as illustrated in Fig. and Fig. 4. This analysis is performed on the widely-used GaoFen-2 dataset, which have 38645 sets of PAN, LR-MS and corresponding HR-MS (GT) images.
Prior 1: In order reconstruct spatial property for target HR-MS image, spatial relationship of PAN need be modeled.
Analysis 1: Fig. 3 compares the spatial distribution histograms between PAN image and GT image in terms of each of their spectral band, as well as LR-MS image and their GT images in terms of spectral band. As shown in each sub-graph of Fig. 3, the correlation coefficient measures the correlation between the two histograms, which obtained by the EMD distance. From the first and second rows, we could obtain that the spatial distribution of PAN is more similar to the spatial distribution of GT image's each spectral band. Here, the spectral band defined as band,, where
2, 3, For example, the correlation coefficient between the PAN image and GT image's band, is 0.423, while the correlation coefficient between LR-MS's band, and GT's band; is 0.023. Similar results can be found for the remaining sub-graph of Fig. 3. Compared by LR-MS image, the spatial distribution of PAN image is much more similar to the spatial distribution of its target HR-MS (GT) image. Thus, we obtain the conclusion that only modeling spatial relationship of PAN image could reconstruct required spatial property for the target HR-MS image.
Correlation coefficient 0.423
Correlation coefficient: 0.707
Correlation coefficient 0.691
Correlation coefficient 0.828
PAN vs GT
300
100
Correlation coefficient: 0.023
Correlation coefficient: 0.343
Correlation coefficient: 0.309
Correlation coefficient: 0.614
LR-MS vs GT
700
200
100
Band,
Band
Band
Band
The distribution comparison between PAN (blue lines) / LR-MS (blue lines) and GT (red lines).
Figure 3. The spatial distribution comparison between LR-MS/PAN and GT (target HR-MS) in terms of cach spectral band. The band illustrated in the figure refers to the spectral band. The correlation coefficient represents the correlation between the two distribution curves, and the larger the value, the greater the correlation.
Prior 2: In order to reconstruct spectral property for target HR-MS image, intra-spectra relationship of LR-MS, as well as the spectral relationship between PAN and LR-MSS images, needs be modeled.
12

---

HetSSNet: Spatial-Spectral Heterogencous Graph Learning Network for Panchromatic and Multispectral Images Fusion
Analysis 2: Here, we investigate the correlation between the LR-MS' spectral bands and the correlation between the GT's spectral bands. Fig. 4 visualizes their correlation and compares the distribution histograms between spectral bands (band, vs. bandg, band2 vs. band3 and band3 vs. band4) of the paired LR-MS and GT images. It can be observed that the distribution correlation between the LR-MS's spectral bands and distribution correlation between the GT's spectral bands is largely different. For example, the correlation coefficient between the GT"s band, and GT's band, is 0.641, while the correlation coefficient between the LR-MS's spectral bands (band3 vs. band,) is 0.263. Similar results can be found for the remaining sub-graph of Fig. 4. The above analysis shows that sub-optimal that model only the LR-MS's spectral relationship. Thus, reconstruct spectral property for target HR-MS image, the spectral relationship of LR-MS, as well the spectral relationship between PAN and LR-MS images, needs to be modeled.
Correlation coefficient: 0.071
Correlation coefficient 0.167
Correlation coefficient: 0.263
390
LR-MS
(00)
200
Correlation coefficient:
Correlation coefficient: 0.581
Correlation coefficient: 0.641
Band,
Band2
Band,
303
200
100
200
Band, Vs. Band
Band, Band
Band, vs. Band,
Band,
The distribution comparison between spectral bands of LR-MS/GT.
Figure 4. The distribution comparison between LR-MS's spectral bands, and the pixel distribution comparison between GT's spectral bands. The band illustrated the figure denotes the spectral hand. The correlation coefficient represents the correlation between the two distribution curves, and the larger the value, the greater the correlation.
C. The benchmarks
SRPPNN proposes pansharpening algorithm based on deep convolutional neural network. It extracts intrinsic spatial details from multispectral images through super-resolution process and combines progressive pansharpening and high-pass residual module to enhance the spatial resolution of the images.
DCFNet is a dynamic cross-feature fusion network based on convolutional neural networks. It employs three parallel branches (high, medium, and low resolutions) for feature fusion to generate high-resolution multispectral images with enhanced spatial resolution and complete spectral information.
CTINN builds the Transformer-CNN dual-branch network, which uses CNN and Transformer to simultaneously extract the local and global features of PAN and LR-MS images' combination.
Hyperformer is based on transformer architecture and adopts the stacked multi-head attention to model the spatial dependencies between LR-MS and PAN image features.
SFIINet uses cascaded spatial and channel attention mechanisms to fuse local-global features encoded by two CNN-based branches.
BiMPan is based on CNN, and it introduces bidomain modeling approach for pansharpening, combining local feature extraction in the spatial domain with global detail reconstruction in the Fourier domain to enhance the spatial resolution of multispectral images.
MDCUN is an interpretable deep neural network that formulates the pan-sharpening problem as a variational model minimization task. It employs a CNN-based iterative algorithm to construct the model, searching for similarities between long-range patches. Additionally, combines PAN images with each band of MS images to selectively provide high-
13

---

HetSSNet: Spatial-Spectral Heterogencous Graph Learning Network for Panchromatic and Multispectral Images Fusion
frequency details.
MSDDN is method based on CNN that enhances the effect of pansharpening by integrating multi-scale information from both the spatial domain and the frequency domain. Specifically, it employs spatial guidance uub-network to learn local spatial information and a frequency guidance sub-network to learn global frequeney-donee domann information.
LGTEUN is an interpretable deep unfolding network that alternately completes data updates each iterative stage through a CNN-based data module and Transformer-based prior module.
FAMENet is implemented based on CNN, where the Adaptive Frequeney Separation Prediction Module utilizes the discrete cosine transform achieve frequeney separation by predicting frequency mask. The Sub- Sub-Frequency Learning Expert Module is responsible for reconstructing high-frequency and low-frequency information. The Expert Mixture Module dynamically weights the high-ffreuuency and low-frequency information described above.
GPCNet is implemented based on graph convolutional networks and enhances the model's representation capability by introducing asynchronous knowledge distillation. Specifically, it employs the spatial GCN module to capture global spatial information in the image, the spectral band GCN module to capture the global correlation of spectral features and enhance spectral fidelity, and the atrous spatial pyramid module to learn multi-scale feature information.
30
SFIM
BROVEY
GS
SRPPNN
DCFNet
CTINN
ARFNet
Hyperformer
20
SFIINet
MDCUN
LGTEUN
MSDDN
FAMENet
GPCNet
Ours
GT
15
SFIM
BROVEY
GS
SRPPNN
DCFNet
CTINN
ARFNet
Hyperformer
5
SFIINet
MDCUN
LGTEUN
MSDDN
FAMENet
GPCNet
Ours
GT
Figure 5. Qualitative results of reduced-resolutte the QuickBird dataset. Top group: the fused results. Bottom group: the error between fused results and reference.
D. More quantitative and visualization results
As shown in Fig. 5 and Fig. 6, we present the visualization results of the QuickBird and GaoFen-2 datasets.
We present the results on the eight-band Vorld dataset, and the results prove that our method has obtained promising results, as shown Tab. 7. To demonstrate the generalization capability of the proposed method, we conduct quantitative evaluations of our pre-trained model on 20 unseen full-resolution (real-world) images for each of the GaoFen-2. World 3, and QuickBird datasets. Tab. presents the results, and the best and second-best results highlighted in boldface and underlined, respectively. We could see that the deep learning-based techniques still perform favorably against their traditional counterparts. In addition, our proposed framework achieves the optimal outcomes for all indexes, confirming its superior generalization capability compared to both traditional and deep learning-based pansharpening methods.
Additionally, we conduct qualitative analysis on the real-world scene of GaoFen-2 dataset. The visual comparison of several methods on representative full-resolution test scene has been illustrated Fig. 7. From the figure, it is evident
14

---

HetSSNet: Spatial-Spectral Heterogencous Graph Learning Network for Panchromatic and Multispectral Images Fusion
30
25
SFIM
BROVEY
GS
SRPPNN
DCFNet
CTINN
ARFNet
Hyperformer
20
SFIINet
MDCUN
LGTEUN
MSDDN
FAMENet
GPCNet
Ours
GT
15
SFIM
BROVEY
GS
SRPPNN
DCFNet
CTINN
ARFNet
Hyperformer
5
SFIINet
MDCUN
LGTEUN
MSDDN
FAMENet
GPCNet
Ours
GT
Figure 6. Qualitative results of reduced-resolution scene the GaoFen-2 dataset. Top group: the fused results. Bottom group: the error between fused results reference.
that traditional methods either suffer severe blurring, such as SFIM and GS, or exhibit pronounced spectral distortions, as seen with BROVEY. Even though deep learning has brought some spatial detail improvements to pansharpening models, modeling spectral properties has been challenge for these models, exemplified by ARFNet, HyperTransformer, and MSDDN. Compared with the these CNN and Transformer- relationship based non-Euclidean graph structure, which has advantages in reconstructing the spatial-spectral properties of complex objects. Overall, our method demonstrates outstanding performance in both quantitative and qualitative experiments on the full-resolution dataset, indicating its strong generalizability.
methods, method models the spatial-spectral
SFIM
BROVEY
GS
SRPPNN
DCFNet
CTINN
ARFNet
Hyperformer
SFIINet
MDCUN
LGTEUN
MSDDN
FAMENet
GPCNet
Ours
PAN
Figure Visual comparison on the GaoFen-2 full-resolution scene, some details magnified for better comparison.
E. Complexity Analysis
The time complexity of aggregating all basic spatial-spectral relationship patterns is O(Nn²), and the time complexity of graph convolution is O dl (1- so the total time complexity of local-wise aggregation is O(n dl) + (I- The global-wise aggregation first calculates the global-wise spatial-spectre relationship pattern similarity matrix, and the time complexity is N). The time complexity of the graph convolution also 2 +mmdnnd+ddd and thus the overall time complexity O O(n ( + dl) + nand nd 1)). Lastly, the time complexity of contrastive
15

---

HetSSNet: Spatial-Spectral Heterogencous Graph Learning Network for Panchromatic and Multispectral Images Fusion
GaoFen-2
WorldView-3 Dat
QuickBird D&4
Method
Dx4
D&t
QNRt
DxL
QNR1
Dx+
QNR1
SFIM
0.082
0.159
0.821
0.137
0.421
0,683
0.219
0.496
0.672
BROVEY
0.138
0.261
0.739
0.136
0.417
0.689
0.226
0.516
0.651
GS
0.074
0.246
0.703
0.141
0.407
0.676
0.231
0.526
0.632
SRPPNN
0.077
0.116
0.817
0.128
0.319
0.757
0.159
0.362
0.700
DCFNet
0.073
0.115
0.839
0.092
0.277
0.774
0.198
0.332
0.723
CTINN
0.072
0.114
0.834
0.072
0.114
0.834
0.133
0.338
0.732
ARFNet
0.081
0.129
0.817
0.230
0.372
0.709
0.210
0.397
0.652
SFIINet
0.068
0.112
0.847
0.094
0.271
0.706
0.141
0.337
0.732
Hyperformer
0.075
0.119
0.823
0.222
0.357
0.775
0.201
0.420
0,653
MDCUN
0.065
0.110
0.850
0.088
0.257
0.783
0.138
0.339
0.730
LGTEUN
0.063
0.121
0.841
0.091
0.254
0.785
0.137
0.336
0.737
MSDDN
0.069
0.112
0.858
0.149
0.353
0.710
0,149
0.353
0,710
FAMENet
0.071
0.153
0.821
0,094
0,259
0.779
0.141
0.339
0,730
GPCNet
0.072
0.113
0.820
0.116
0.312
0.759
0.144
0.338
0.726
Ours
0.061
0.107
0.860
0.085
0.252
0.792
0.130
0.328
0.742
Table The average results on the real-world full-resolution scenes from GaoFen-2, WorldView-3, and QuickBird datasets. The best results highlighted in boldface, the second best one underlined.
learning is d). Therefore, the total time complexity of our HetSSNet is O(NN dl)n² 2 + (m dl 1))n).
Method
PSNR t
SSIM f
SCC
SAM
ERGAS
SFIM
21.415
0.542
0.721
0.115
8.855
BROVEY
22.506
0.547
0.733
0.571
8.233
GS
28.417
0.693
0.812
0.102
6.799
SRPPNN
30.304
0.918
0.956
0.078
3.188
DCFNet
30.624
0.924
0.958
0.072
3.092
CTINN
31.856
0.952
0.960
0.066
2.742
SFIINet
30.597
0.924
0.956
0.074
3,080
Hyperformer
29.943
0.913
0.945
0.081
3,426
MDCUN
31.299
0.953
0.956
0.066
2.930
BiMPan
30.419
0.941
0.952
0.069
2.912
LGTEUN
32.219
0.955
0.950
0.061
2.629
MSDDN
30.850
0.926
0.955
0.073
2.995
FAMENet
30.990
0.929
0.954
0.070
2.953
GPCNet
30.543
0,922
0.955
0.077
3.105
Ours
32.589
0.958
0.963
0.059
2.524
Table 7. Comparison of different methods eight-band WorldView-3 datasct
The input of local- wise spatial-spectral relationship pattern aggregation includes matrices composed of various basic spatial- spectral relationship patterns and attribute matrix Raxm, and the weight vector and matrices contain O1-N, W! Locat Rmmdd and Wal Local Rdxd and thus the space complexity of local-wise spatial-spectral relationship pattern aggregation is O(n2N + nm md + df 1) + N). Similarly, the input of the global-wise spatial-spectral relationship pattern aggregation includes matrix B E R=*N and adjacency matrix AGtobal R=*n and the weight vector and matrices contain $1.,, Wilobal E mmd and wl Wo E RRd and thus the space complexity of global-wise spatial-spectral relationship pattern aggregation O(NN + n² +md + dl 1) + N). Therefore, the total space complexity of HetSSNet ONN?² + (m + Nyn + md + dl(l - 1) NN.
16


Reviews:
[{"review_id": "YeqrOOpIrc", "reviewer": "Reviewer_UFR7", "summary": "Remote sensing pansharpening involves fusing panchromatic (PAN) images with low-resolution multi-spectral (LR-MS) images to produce high-resolution multi-spectral (HRMS) images. Traditional methods like CNN and Transformer treat images as grids of pixels in Euclidean space, which struggle with irregular ground objects in remote sensing. Graphs offer a more flexible structure but face challenges in modeling spatial-spectral properties: 1) creating a customized graph structure for spatial-spectral relationships, and 2) learning a unified spatial-spectral representation. To overcome these, the authors propose HetSSNet, a spatial-spectral heterogeneous graph learning network. HetSSNet constructs a heterogeneous graph tailored for pansharpening, extracts multiple relationship patterns, and aggregates these patterns to learn a unified spatial-spectral representation from both local and global perspectives. Extensive experiments show that HetSSNet outperforms existing methods and generalizes well.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful and encouraging review. The point-to-point answer is provided below.\n\n**A1.** The convolution kernel of CNN assumes that local features are spatially invariant, but the morphology of objects (such as rivers and roads) may change nonlinearly with the terrain. For example, the curvature of mountain roads is different from that of plains. It is difficult for a fixed-size convolution kernel to adaptively adjust the receptive field and may perform redundant calculations on irrelevant areas (such as the background). Transformer captures long-range dependencies through self-attention, but the correlation of objects in remote sensing images often has local-global heterogeneity (such as regular local structures of buildings, but random distribution of vegetation). Global calculations introduce noise from irrelevant pixels and are computationally expensive. Mamba and Diffusion also have the problem of rigid modeling. Specifically, Mamba's default scanning path (such as from left to right, from top to bottom) is not suitable for anisotropic objects (such as diagonal roads or radial urban layouts). Diffusion is due to the denoising network (usually U-Net) still relying on the local inductive bias of convolution.\n\n**A2.** We use two-layer GCN. In fact, the over-smoothing problem limits the GCN model from stacking more aggregation layers. The purpose of this work is not to solve the over-smoothing problem of GCN, but to use GCN with a suitable number of layers to learn the designed spatial-spectral heterogeneous graph. Ablation experiments in the manuscript have shown that even a small number of GCN layers can achieve promising performance (Tab. 4).\n\n**A3.** GPCNet uses the full-graph modeling strategy, in which all nodes and edges are of the same type and all nodes have relationships. Pansharpening involves the fusion of PAN and LR-MS, which have heterogeneous characteristics. Therefore, the full-graph structure is sub-optimal for pansharpening. The designed spatial-spectral heterogeneous graph can explicitly model the heterogeneity of PAN and LR-MS, flexibly fuse spatial-spectral relationships, and adapt to local and global modeling requirements.  In addition, we will add the analysis of recent heterogeneous graph learning strategies in the final version (please refer to **A4**).\n\n**A4.**  Meta-path sampling requires pre-setting of fixed paths and cannot flexibly adapt to the spectral-spatial coupling of different regions; it requires sampling all possible paths, which has high computational complexity (especially for long paths); the path's semantics are single, making it difficult to integrate the multi-objective constraints of spectral fidelity and spatial enhancement. However, we use the base relationship pattern strategy to model multiple edge types at the same time and dynamically weight the relationships between different relationships. The ablation model (Modal (a) in Tab.3) verifies this in the manuscript.\n\n**A5.**  The experimental results of the eight-band worldview-3 dataset are shown in the table below, which shows that our method could achieve promising results on the eight-band dataset. We will add these experimental results in the final revision.\n\n| Methods    | PSNR ↑ | SSIM ↑ | SCC ↑ | SAM ↓ | ERGAS ↓ |\n|------------|--------|--------|-------|-------|---------|\n| SFIM       | 21.415 | 0.542  | 0.721 | 0.115 | 8.855   |\n| BROVEY     | 22.506 | 0.547  | 0.733 | 0.571 | 8.233   |\n| GS | 28.417 | 0.693  | 0.812 | 0.102 | 6.799  |\n| SRPPNN| 30.304 | 0.918  | 0.956 | 0.078 | 3.188   |\n| DCFNet     | 30.624 | 0.924  | 0.958 | 0.072 | 3.092   |\n| CTINN      | 31.856 | 0.952  | 0.960 | 0.066 | 2.742   |\n| SFIINet    | 30.597 | 0.924  | 0.956 | 0.074 | 3.080   |\n| Hyperformer | 29.943 | 0.913 | 0.945 | 0.081 | 3.426 |\n| MDCUN      | 31.299 | 0.953  | 0.956 | 0.066 | 2.930   |\n| BiMPan      | 30.419 | 0.941  | 0.952 | 0.069 | 2.912  |\n| LGTEUN     | 32.219 | 0.955  | 0.950 | 0.061 | 2.629   |\n| MSDDN      | 30.850 | 0.926  | 0.955 | 0.073 | 2.995   |\n| FAMENet    | 30.990 | 0.929  | 0.954 | 0.070 | 2.953   |\n| GPCNet      | 30.543 | 0.922  | 0.955 | 0.077 | 3.105   |\n| **Ours**   | **32.589** | **0.958** | **0.963** | **0.059** | **2.524** |\n\n**A6.**  Thanks for your suggestion, we will add sources of the comparison methods in the Tab.2 and Tab.6.\n\n**A7.**  Thanks for your suggestion, we will remove these abbreviations and recalibrate correspond description in the final version.\n\n**A8.** Thank you for pointing it out, we will correct it.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "In order to solve the rigid modeling of CNN/Transformer, the author introduces the heterogeneous graph to establish complex spatial-spectral relationships, this approach is interesting and novel. Experiments on three pansharpening datasets demonstrate consistent improvements over CNN-based/Transformer-based methods, proving that the proposed method has the potential to break through the limitation of the current modeling framework.", "theoretical_claims": "The spatial-spectral relationship priors for the pan-sharpening task are verified and analyzed on a large amount of pansharpening data.", "experimental_designs_or_analyses": "The experimental design is reasonable and provides competitive results on reduced-and full-resolution datasets.", "relation_to_broader_scientific_literature": "Recent learning-based methods often rely on the Euclidean space assumption, which may not fully capture the complex geometric structure of remote sensing data. The non-Euclidean framework proposed in this paper draws on the progress of graph, which have shown potential in dealing with irregular data. This contribution extends the application of non-Euclidean methods to the field of pansharpening, addressing the urgent need for more flexible and adaptive spatial-spectral modeling.", "essential_references_not_discussed": "It is recommended that the author should discuss the GCN-based panchromatic sharpening method, GPCNet, published in TGRS2024.", "other_strengths_and_weaknesses": "Strengths ：\n\n1. The paper analyzes the two spatial-spectral relationship priors in the pansharpening, providing a good basis for subsequent modeling.\n\n2. The paper uses heterogeneous graph to construct complex and diverse spatial-spectral relationships between LR-MS and PAN images, this is an interesting solution. In addition, to learn unified spatial-spectral representation, the proposed method extracted base relationship pattern and learn them from the local and global perspective.\n\n3. Comprehensive experiments on WorldView-3, GaoFen-2 and QuickBird datasets demonstrate consistent improvements over state-of-the-art pansharpening methods, validated by ablation studies analyzing individual HetSSNet components.\n\n4. Unlike the other CNN/Transformer-based methods, the proposed method attempts to design a new pansharpening modeling framework in non-Euclidean space. This study shows a commendable beginning.\n\n5. The article is well written and the figures are clear.\n\nWeakness:\n\n1. Why do CNN and Transformer structures lead to redundancy and also do Mamba or Diffusion structures suffer from the above drawbacks? Please provide a more in-depth explanation of the following point:‘’Since these ground objects are usually not quadrate whose shape is irregular, the commonly used grid structures in the modeling architecture like CNN and Transformer are redundant and inflexible to process them “.\n\n2. In the Local-wise Aggregation, the authors used only a single-layer GCN to learn node representations with interaction information. The motivation for this approach is not clearly explained.\n\n3. The author should discuss the GCN-based pansharpening method (i.e., GPCNet[1]) in the introduction section to make it clearer how it differs from this proposed method. In addition, it is recommended to add the analysis of recent heterogeneous graph learning strategies (e.g., meta-path sampling) in the related work section.\n\n4. This paper uses the basic relationship modeling strategy. Meta-path sampling is a commonly used heterogeneous graph modeling mechanism. Why not use the mete-path sampling mechanism? What are the advantages of the learning strategy used in this paper?\n\n5. The author should add the experimental results of the eight-band pansharpening dataset.", "other_comments_or_suggestions": "1. The table should be expanded to include the sources or origins (provenance) of the different methods used in the comparison.\n\n2. Avoid using abbreviations like \"i.e.\" in the abstract and conclusion sections.\n\n3. In section 3.6, it contains a typo (\"Optimization \" instead of \" optimization \").", "questions_for_authors": "In the section on Pansharpening-specific relationship priors, the comparison of spectral band distributions is currently limited to neighboring spectra, such as band1 and band2. Could the analysis be extended to explore distribution comparisons across broader spectral ranges, for example, between band1 and band3, or band1 and band4?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RvqDaS8c4q", "reviewer": "Reviewer_XsAf", "summary": "In this paper, authors propose a spatial-spectral heterogeneous graph learning network, termed as HetSSNet. Specifically, it segments each band of the pansharpening data into non-overlapping patches. By leveraging graph structures, it captures intrinsic relationships among these patches. Furthermore, it employs contrastive learning to enrich feature representations and enhance the overall performance of pan-sharpening.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful and encouraging review. The point-to-point answer is provided below.\n\n**A1.** The non-Euclidean characteristics of the method are reflected in two aspects: the data structure and relationship modeling. **(a) Spatial-spectral heterogeneous graph construction.** We construct the spatial-spectral heterogeneous graph with two types of nodes (PAN and LR-MS nodes) and heterogeneous edges modeling complex spatial-spectral relationships . The graph topology is dynamically constructed via an adaptive $k$-nearest neighbor algorithm (Line 190-206). Thus, the defined spatial-spectral heterogeneous graph inherently represents a non-Euclidean data structure. **(b) Non-Euclidean relationship modeling.** We design relationship pattern aggregation module based on the non-Euclidean learning operation (i.e., GCN) (Sec. 3.5). It explicitly respect the graph topology, avoiding the limitations of Euclidean assumptions.\n\n**A2.** The convolution operation is applied to the PAN image patches and obtain the corresponding feature vector.\n\n**A3.** The experimental setting (c) and (d) could verify the role of local features and global features in contrastive learning. In fact, Setting (c) and (d) individually makes Eq.8 degenerate into self-contrast learning with only global/local features. The results show that our cross-view local-global contrastive learning is more effective. We add two sets of ablation experiments: visual ablation experiment (https://anonymous.4open.science/r/kk3/exp.png), and the ablation experiment of the temperature coefficient $\\tau$ in Eq.8 as follows.\n\n|$\\tau$|PSNR↑|SAM↓|SSIM↑|QNR↑|$D_{s}$↓|\n|--|--|--|--|--|--|\n|0.01|48.577|0.025|0.985|0.849|0.118|\n| **0.1** |**49.541** |**0.015**|**0.994**|**0.860**|**0.107**|\n|0.5|49.311|0.019|0.990|0.856|0.109|\n|1.0|48.011|0.029|0.976|0.840|0.125|\n\n**A4.** We present the inference time, number of parameters, and FLOPs of our method. We will add them in the final version.\n\n| Metric |Time (s/img) | Params (M) | FLOPs (G) | \n|--------------|-------|-------|------|\n| HetSSNet| 0.0327|0.167| 1.927|  \n\n**A5.** The final node representation $\\mathbf{H}$ is first mapped to the target dimension through the fully-connected layer, and then refined using the convolutional operation to generate the final HR-MS image. We will add the corresponding description after Eq.9.\n\n**A6.** We add three comparison methods published in 2024, i.e., FusionMamba [1], WINet [2] and HFIN [3], as shown in tables below, these comparison methods will be updated in the final version. In addition, we will update the references in the Sec.1 and Sec.2.1.\n\n**WV-3:**  \n| Method|PSNR↑|SSIM↑|SAM↓|ERGAS↓|SCC↑|$D_{\\lambda}$|$D_{s}$↓|QNR↑|\n|-|-|-|-|-|-|-|-|-|\n|FusionMamba|38.727|0.872|0.069|5.437|0.934|0.097|0.261|0.752|\n|WINet|39.781|0.887|0.056 |4.769|0.969|0.132|0.331| 0.740|\n|HFIN|32.126|0.837|0.083|6.627|0.837|0.142|0.341|0.723|\n|**Ours**|40.623|0.905|0.054|4.741|0.972|0.085|0.252|0.792|\n\n**QB:** \n|Method|PSNR↑|SSIM↑|SAM↓|ERGAS↓|SCC↑|$D_{\\lambda}$↓|$D_{s}$↓|QNR↑|\n|-|-|-|-|-|-|-|-|-|\n| FusionMamba|34.021|0.881|0.047|2.931|0.904|0.152|0.381|0.697|\n|WINet|37.209|0.927|0.032|2.212|0.931|0.142|0.341|0.714|\n|HFIN|33.616|0.811|0.050|3.174|0.869|0.211|0.376|0.662 |\n|**Ours**|37.228|0.931|0.029|2.022|0.951|0.130|0.328|0.742|\n\n**GF-2:**  \n|Method|PSNR↑|SSIM↑|SAM↓|ERGAS↓|SCC↑|$D_{\\lambda}$↓|$D_{s}$↓|QNR↑|\n|-|-|-|-|-|-|-|-|-|\n|FusionMamba|46.242|0.976| 0.027|1.278|0.972|0.074|0.121|0.756|\n|WINet|49.405| 0.984 |0.016| 0.968|0.977|0.060|0.108|0.855|\n|HFIN|41.938|0.966 |0.039|2.534|0.894|0.136|0.158|0.722|\n|**Ours**|49.541|0.994|0.015| 0.955|0.990|0.061|0.107|0.860|\n\n**A7.** The matrix $\\mathrm{U}$ is constructed by stacking the feature vectors of all nodes in the spatial-spectral heterogeneous \ngraph, where each row is the associated node feature vector of node $v_i$ (Line 150-152). For example, the corresponding feature vectors of PAN images/ LR-MS images patches are used to form the matrix $\\mathrm{U}$ (Line 214).\n\n**A8.**  \"Non-European\" is a typo, it should be \"Non-Euclidean\". We will correct it in Line 135.\n\n**A9.** During the implementation for the pansharpening models, the LR-MS needs to be upsampled to the size of PAN before fusion [2]. In addition, the dimension definition of LR-MS follows the published pansharpening work [2], thus, in method section, it is reasonable to define it as the same length and width as the PAN image. \n\n**A10.**  Thanks for your suggestion, we will correct them.\n\n**A11.**  It is a typo, it should be \"$\\tilde{\\mathbf{A}}_{\\boldsymbol{Local}} $ \" generated from Eq.1. We will correct it in Line 205.\n\n[1] Peng S, et al. Fusionmamba: Efficient remote sensing image fusion with state space model. TGRS, 2024.\n\n[2] Tan J, et al. Revisiting Spatial-Frequency Information Integration from a Hierarchical Perspective for Panchromatic and Multi-Spectral Image Fusion. CVPR, 2024.\n\n[3] Zhang J, et al. Pan-Sharpening With Wavelet-Enhanced High-Frequency Information. TGRS, 2024.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "No, there is no theoretical proof in the article.", "experimental_designs_or_analyses": "Yes, the experiment complies with the general settings.", "relation_to_broader_scientific_literature": "The main contribution of the paper is the introduction of the graph neural network structure in the field of pan-sharpening. This provides insights into improving the network structure in related fields. However, there are not many related articles in recent years, and it may be widely explored by the community in the future.", "essential_references_not_discussed": "No, all related pan-sharpening methods based on graph neural networks have been discussed.", "other_strengths_and_weaknesses": "### Strengths:\n[+] Based on statistical analysis, the authors constructed a graph neural network incorporating a spatial-spectral heterogeneous graph construction module, a basic relationship pattern generation module, and a relationship pattern aggregation module to enhance the performance of pansharpening.\n\n[+] The authors theoretically analyzed the computational complexity of the proposed method.\n\n[+] The authors utilize graph structures to capture long-range dependencies within images, which is a relatively novel approach in the field of pansharpening.\n\n### Weaknesses:\n[-] The authors claim that the method is non-Euclidean but abandon this concept in the implementation, making the theoretical and intuitive justification unconvincing.\n\n[-] In Section 3.3, the authors mention, \"We divide the image into N overlapping patches and transform each patch of the PAN image into a feature vector.\" However, the details of this transformation are unclear.\n\n[-] The ablation experiments in the paper are insufficient. Specifically, in experimental settings (c) and (d), the contrastive loss cannot be applied due to the absence of positive and negative sample pairs. After combining the two modules, the authors do not adequately discuss the role of the contrastive loss, both qualitatively and quantitatively.\n\n[-] The running efficiency, number of parameters, and floating-point operations per second (FLOPs) of the method are not reported.\n\n[-] The relationship between the final node representation $H$ and the network output is not clearly explained.\n\n[-] The article employs comparative methods that are somewhat outdated, focusing primarily on approaches developed before 2023. Additionally, the references need to be updated to include more recent research and developments in the field.", "other_comments_or_suggestions": "[-] In Section 3.1, the symbol \"U\" in the defined graph $G(V,E,U)$ is not denoted.\n\n[-] In Section 2.2, the description \"Non-European\" appears inappropriate or unclear.\n\n[-] In line 176 and line 182, the width and height of LRMS image should not be assumed to be equal to those of PAN image.\n\n[-] The distinguishability of the symbols should be improved for clarity.\n\n[-] In line 205, $\\tilde{A}_l$ lacks a clear definition.", "questions_for_authors": "Please refer to the weakness.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "w2H6kUZyZc", "reviewer": "Reviewer_LVzz", "summary": "For pansharpening, recent modeling frameworks are relatively rigid and have limitations when dealing with irregular ground objects in remote sensing images. To address this issue, this paper proposes a spatial-spectral heterogeneous graph learning network named HetSSNet. It constructs a heterogeneous graph structure that explicitly describes the specific relationships between PAN and MS images. A basic relationship pattern generation module is designed to extract multiple fundamental spatial-spectral relationship patterns from the constructed heterogeneous graph. To automatically capture local information and global relevant information across different complex relationship pattern between nodes, it designs the relationship pattern aggregation module from the local and global perspective. The experimental results demonstrate that the model achieves satisfactory performance on multiple datasets.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful and encouraging review. The point-to-point answer is provided below.\n\n**A1.** The convolution kernel of CNN slides on a regular grid and cannot adapt to the curved, broken, radial and other remote sensing image features. The core of Transformer is global self-attention, and fixed-size patch splitting (such as 16$\\times$16) will break the structure of irregular features (such as rivers and roads). In fact, the rigidity of CNN/Transformer comes from its regular modeling assumption of Euclidean space, which is inconsistent with the non-Euclidean data characteristics of remote sensing image features (such as mountains, rivers, etc.).\n\n**A2.** We give two examples of the base relationship pattern. For example, $\\mathbf{P}\\xrightarrow{spectral}\\mathbf{L}$ is a basic relationship pattern between PAN and LR-MS nodes, which represents that there is only spectral interaction between the PAN node and the LR-MS node . $\\mathbf{P}\\xrightarrow{spatial/spectral}\\mathbf{L}$ is also a basic relationship pattern, where there are spatial interaction and cross-modal spectral interaction between the PAN node and the LR-MS node.\n\n**A3.**   We will add related work of heterogeneous graph learning strategies in Sec. 2.2. For example, GATNE [1] learns the representation of nodes in each specific relationship, respectively, and then aggregate them to obtain the final node representation. DualHGNN [2] introduces two hypergraphs into the representation learning of multiplex heterogeneous networks, and learns node embedding using spectral hypergraph convolutional networks. FAME [3] realizes message passing in multiplex heterogeneous networks by automatically capturing useful relation-aware meta-paths.\n\n[1] CEN Y, et al. Representation Learning for Attributed Multiplex Heterogeneous Network. SIGKDD, 2019. \n\n[2] XUE H, et al. Multiplex Bipartite Network Embedding using Dual Hypergraph Convolutional Networks. arXiv, 2021.\n\n[3] LIU Z, et al. Fast Attributed Multiplex Heterogeneous Network Embedding. CIKM, 2020.", "claims_and_evidence": "The methods and experimental design of the paper are clearly presented and well-structured.", "methods_and_evaluation_criteria": "The datasets used are classic in the field of pansharpening, and the full-resolution and reduced-resolution experiments are also commonly conducted in this field.", "theoretical_claims": "The authors could further explain why the method based on CNN/Transformer is rigid and why the method based on heterogeneous graphs is more flexible.", "experimental_designs_or_analyses": "The authors conduct ablation experiments to validate the rationality of the Basic Relationship Pattern Generation module and the Relationship Pattern Aggregation module.", "relation_to_broader_scientific_literature": "The paper provides examples and introduces CNN-based and Transformer-based methods, such as DCFNet and CTINN, pointing out that their modeling is overly rigid. To address this issue, the paper introduces its innovative approach.", "essential_references_not_discussed": "I have not seen any similar spatial-spectral heterogeneous graph structure for remote sensing pansharpening. The paper is therefore innovative in this regard.", "other_strengths_and_weaknesses": "Strengths:\n1. The article is well-structured and logically coherent, progressing systematically from problem statement to method design and experimental validation.\n2. The authors analyze pansharpening data in the appendix section, and obtain the priors of task-related spatial-spectral relationships. The paper proposes a novel spatial-spectral heterogeneous graph to model these relationships in a more flexible way.\n3. The experimental design is rigorous, utilizing three widely recognized datasets and incorporating both comparative and ablation studies for validation. \n4. The proposed method extracts multiple basic relationship matrices based on edge types and aggregates these basic relationships from local and global perspectives to obtain a unified spatial-spectral relationship representation.\n\nWeaknesses:\n1. The authors says that recent CNN/Transformer is rigid in modeling relationships in pan-sharpening. What does this \"rigidity\" mean in remote sensing image? Further discussion is recommended.\n2. In definition 2, the authors give the definition of the basic relationship pattern. It is recommended that the authors give the example to illustrate the basic relationship pattern in the constructed spatial-spectral heterogeneous graph.\n3. The related work needs improvement. Personally, in the current version, it should express what are the current mechanisms for learning heterogeneous graph.", "other_comments_or_suggestions": "Please refer to the Weaknesses.", "questions_for_authors": "Please refer to the Weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QswPkgKV9b", "reviewer": "Reviewer_bvrf", "summary": "This paper proposes a spatial-spectral heterogeneous graph learning network for panchromatic and multispectral images fusion. Specifically, the authors explore the pansharpening-specific relationships in the heterogeneous graph structure. Then, they extract the multiple relationship patterns by the designed basic relationship pattern aggregation module. Finally, the unified spatial-spectral representation across different relationships is learned from local and global perspectives.\n\n## update after rebuttal\nMy concerns have been addressed and would like to recommend accept.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful and encouraging review. The point-to-point answer is provided below.\n\n**A1.** We analyze the spatial relationship of a large number of PAN/LR-MS/target HR-MS (GT) image pairs in the Appendix. As can be seen from Fig. 3 in the Appendix, the spatial histogram of each spectral band of LR-MS is extremely different from that of each spectral band of GT. In contrast, the spatial histogram of PAN is very close to that of each spectral band of GT. Therefore, we can obtain spatial property-related relationship prior, i.e., modeling only the spatial relationship of PAN images can reconstruct the required spatial property of target HR-MS images.\n\n**A2.** The designed spatial-spectral heterogeneous graph is regarded as the attributed multiplex heterogeneous graph. Existing methods do not explicitly model multiplex heterogeneous structures, but rather regard them as a linear superposition of relations. For example, MHGCN realizes message passing in multiplex heterogeneous networks by automatically capturing useful relation-aware meta-paths. It is only regarded as a simple superposition of relations, while ignoring the learning of semantic information. But in fact, when learning the spatial-spectral heterogeneous graph, we should not simply regard it as a linear superposition of simple individuals, but should focus on its own high-level semantic information. The proposed method can express rich high-level semantic information by extracting basic spatial-spectral relationship patterns and multi-level relationship aggregation. The ablation experiments in the manuscript (Tab.3) also demonstrate the effectiveness of our learning mechanism. \n\n**A3.**  In this paper, we use the contrastive learning loss (InfoNCE loss [1]) to balance the local-wise and global-wise aggregation branch, as denoted in Eq.8. In Eq.8, the positive sample pairs are the numerator, i.e., $\\exp\\mathrm{(}\\boldsymbol{s}(\\mathbf{H} _{\\boldsymbol{Local},i},\\mathbf{H} _{\\boldsymbol{Global},i}))$, which enforces the local features to be similar to their corresponding global features; The negative sample pairs are denominator, i.e., $\\sum _{j\\in \\mathrm{V}}{\\exp}(\\boldsymbol{s}(\\mathbf{H} _{\\boldsymbol{Local},i},\\mathbf{H} _{\\boldsymbol{Global},j}))$, which contains the global features of all samples ($j\\ne i$). Negative sample pairs force the model to learn discriminative local features while requiring the global features to be sufficiently discriminative. The temperature coefficient $\\tau$ in Eq.8 controls the weight of local-wise and global-wise branches.  In the early stages of training, the model may focus more on local details. As training progresses, global features gradually dominate, guiding local features to learn semantic consistency.\n\n**A4.**  The experimental results of eight-band WorldView-3 dataset are shown in the table as below. We will add these results in final version.\n\n| Methods    | PSNR ↑ | SSIM ↑ | SCC ↑ | SAM ↓ | ERGAS ↓ |\n|------------|--------|--------|-------|-------|---------|\n| SFIM       | 21.415 | 0.542  | 0.721 | 0.115 | 8.855   |\n| BROVEY     | 22.506 | 0.547  | 0.733 | 0.571 | 8.233   |\n| GS | 28.417 | 0.693  | 0.812 | 0.102 | 6.799  |\n| SRPPNN| 30.304 | 0.918  | 0.956 | 0.078 | 3.188   |\n| DCFNet     | 30.624 | 0.924  | 0.958 | 0.072 | 3.092   |\n| CTINN      | 31.856 | 0.952  | 0.960 | 0.066 | 2.742   |\n| SFIINet    | 30.597 | 0.924  | 0.956 | 0.074 | 3.080   |\n| Hyperformer | 29.943 | 0.913 | 0.945 | 0.081 | 3.426 |\n| MDCUN      | 31.299 | 0.953  | 0.956 | 0.066 | 2.930   |\n| BiMPan      | 30.419 | 0.941  | 0.952 | 0.069 | 2.912  |\n| LGTEUN     | 32.219 | 0.955  | 0.950 | 0.061 | 2.629   |\n| MSDDN      | 30.850 | 0.926  | 0.955 | 0.073 | 2.995   |\n| FAMENet    | 30.990 | 0.929  | 0.954 | 0.070 | 2.953   |\n| GPCNet      | 30.543 | 0.922  | 0.955 | 0.077 | 3.105   |\n| **Ours**   | **32.589** | **0.958** | **0.963** | **0.059** | **2.524** |\n\n**A5.** We have calculated the space complexity of our proposed model in the manuscript (Line 844-851). As shown in the table below, we present computational cost of our method and all comparison methods.\n\n| Methods   | SRPPNN | DCFNet | CTINN | SFIINet | Hyperformer | MDCUN | BiMPan | LGTEUN | MSDDN |FAMENet | GPCNet | Ours |\n|---|--------|--------|-------|-------|---------|--------|--------|--------|---------|-------|---------|---------|\n|Params|1.711| 2.905 | 0.039 | 0.085 | 43.938 |0.098| 0.396  | 0.202| 0.011 | 0.091| 0.087| 0.167|\n\n**A6.**  Thank you for pointing it out. We will carefully correct the descriptions in the Line 35 and 200. \n\n[1] Chen T,  et al. A simple framework for contrastive learning of visual representations. ICML, 2020.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "Yes. The arguments and derivations about the heterogeneous graph architecture related to spatial-spectral relationships appear logically sound.", "experimental_designs_or_analyses": "Yes. The experimental designs including baseline comparisons and ablation studies are coherent. No major flaws or inconsistencies are found in the experimental setup or analyses.", "relation_to_broader_scientific_literature": "The previous method [1] only uses GCN to model global relationships in Euclidean space. The authors pioneer spatial-spectral heterogeneous graph structure in non-European space for remote sensing pansharpening.  \n[1] When pansharpening meets graph convolution network and knowledge distillation. IEEE Transactions on Geoscience and Remote Sensing.", "essential_references_not_discussed": "None", "other_strengths_and_weaknesses": "Strengths\n1. The authors design the first spatial-spectral heterogeneous graph structure for remote sensing pansharpening.\n2. Building on the spatial-spectral heterogeneous graph structure, the authors further develop both local and global relationship pattern aggregation mechanisms.\n3. The paper is well-organized and clearly written.\nWeaknesses:\n1. In the discussion of the graph construction challenge at line 60, the authors highlight the spectral relationship between LR-MS and PAN images. Why do they not also consider the spatial relationship between these images?\n2. What are the advantages of the heterogeneous graph learning mechanism designed by the author compared with the methods used in previous heterogeneous graph learning networks (e.g., MHGCN)?\n3. The roles of local- and global-level aggregations warrant further analysis, and the authors should clarify how these two aspects are balanced.\n4. Pansharpening datasets also have some eight-band remote sensing images. Can the authors provide experimental comparison of eight-band datasets?\n5. Graph structure typically demand substantial computational resources when dealing with large-scale graphs or high-dimensional node features. Therefore, the authors should compare the computational cost of their proposed approach with that of existing methods.", "other_comments_or_suggestions": "There are some inaccurate expressions in the manuscript—particularly around lines 35 and 200. The authors should carefully review and correct these sections.", "questions_for_authors": "Please refer to the Weakness.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: After the author-reviewer discussion stage, all reviewers are positive about this submission. AC agrees with the reviewers and recommends acceptance. Given that the authors provided substantial additional supporting materials in the rebuttal, which helped resolve the reviewers' concerns, it is important that these materials are properly incorporated into the camera ready version of the paper.
Decision: Accept (poster)