Title: Satformer: Accurate and Robust Traffic Data Estimation for Satellite Networks
Venue: NeurIPS 2024
Accepted: yes
Keywords: Satellite Networks; traffic measurement; Tensor Completion; Neural Network
Abstract: The operations and maintenance of satellite networks heavily depend on traffic measurements. Due to the large-scale and highly dynamic nature of satellite networks, global measurement encounters significant challenges in terms of complexity and overhead. Estimating global network traffic data from partial traffic measurements is a promising solution. However, the majority of current estimation methods concentrate on low-rank linear decomposition, which is unable to accurately estimate. The reason lies in its inability to capture the intricate nonlinear spatio-temporal relationship found in large-scale, highly dynamic traffic data. This paper proposes Satformer, an accurate and robust method for estimating traffic data in satellite networks. In Satformer, we innovatively incorporate an adaptive sparse spatio-temporal attention mechanism. In the mechanism, more attention is paid to specific local regions of the input tensor to improve the model's sensitivity on details and patterns. This method enhances its capability to capture nonlinear spatio-temporal relationships. Experiments on small, medium, and large-scale satellite networks datasets demonstrate that Satformer outperforms mathematical and neural baseline methods notably. It provides substantial improvements in reducing errors and maintaining robustness, especially for larger networks. The approach shows promise for deployment in actual systems.

Satformer: Accurate and Robust Traffic Data Estimation for Satellite Networks

Abstract

The operations and maintenance of satellite networks heavily depend on traffic measurements. Due to the large-scale and highly dynamic nature of satellite networks, global measurement encounters significant challenges in terms of complexity and overhead. Estimating global network traffic data from partial traffic measurements is a promising solution. However, the majority of current estimation methods concentrate on low-rank linear decomposition, which is unable to accurately estimate. The reason lies in its inability to capture the intricate nonlinear spatio-temporal relationship found in large-scale, highly dynamic traffic data. This paper proposes Satformer, an accurate and robust method for estimating traffic data in satellite networks. In Satformer, we innovatively incorporate an adaptive sparse spatiotemporal attention mechanism. In the mechanism, more attention is paid to specific local regions of the input tensor to improve the model's sensitivity on details and patterns. This method enhances its capability to capture nonlinear spatio-temporal relationships. Experiments on small, medium, and large-scale satellite networks datasets demonstrate that Satformer outperforms mathematical and neural baseline methods notably. It provides substantial improvements in reducing errors and maintaining robustness, especially for larger networks. The approach shows promise for deployment in actual systems.

1 Introduction

As a potential complement to terrestrial networks, satellite networks are envisioned to provide broadband connectivity with seamless coverage and in a cost-effective manner. Internet service and content providers are interested in satellite networks due to their wide international coverage and lower entry costs in rural and underdeveloped areas [1].

Traffic engineering [2, 3, 4] and topology engineering [5] of satellite networks, such as access control, routing and congestion control, are key to achieve efficient control of satellite networks, which rely on real-time perception of global traffic data [6]. Timely and accurate traffic measurements beyond basic metrics are undoubtedly beneficial for such applications.

However, it is troublesome and costly to collect massive traffic data by measuring all transmission pairs directly [7], since traffic data is naturally distributed throughout the entire network. In order to support the network operation of the emerging mega-constellations, there is an urgent need to

explore cost-effective traffic measurement methods. Traffic data estimation is a feasible approach for large-scale satellite networks, where the global traffic data can be estimated according to partial traffic sampling and measurement [8].

Due to inherently dynamic natures of spatial distances and orbital positions in satellite networks, traffic volumes and patterns vary over time [9]. Emerging mega-constellations networks typically involve numerous satellites, so the dynamic traffic data between satellite pairs can be represented as high-dimensional matrices or tensors. This complexity makes it difficult to capture the complicated relationships within the data [10]. Furthermore, the instability of inter-satellite and satellite-ground links often leads to the loss of traffic data during transmission. It should also be noted that not all satellite pairs have constant communication demands. As a result, the measured traffic data is often sparse and incomplete, making traffic data estimation complex [1]. Therefore, the primary challenge in accurately and robustly estimating satellite network traffic data lies in effectively capturing the complex and nonlinear spatio-temporal correlations while maintaining robustness for varying sequence lengths [11].

Indeed, most efforts in traffic data estimation focus solely on low-rank linear decomposition, which cannot effectively capture the nonlinear spatio-temporal correlations among large-scale and dynamic traffic data, leading to inaccurate estimations. Therefore, developing a novel approach is crucial for enhancing traffic estimation performance to effectively extract and utilize the complex and nonlinear spatio-temporal correlations among inter-satellite traffic data.

For large-scale and highly dynamic satellite network traffic data, we propose Satformer, a new neural network architecture designed for accurate and robust traffic estimation. Satformer systematically constructs encoder-decoder components with stacked spatio-temporal modules to effectively capture complex spatio-temporal correlations in traffic data. Within each module, an adaptive sparse spatiotemporal attention mechanism (ASSIT) is adopted to extract key features from numerous sparse inputs by focusing on specific local regions. This enables Satformer to capture nuanced traffic patterns essential for accurate estimation. This is particularly useful in satellite networks where traffic may be concentrated in certain areas due to regional demand or satellite coverage. Additionally, ASSIT is more robust to sparsity as it can identify and focus on areas with higher data density, which may contain more informative traffic features, rather than being overwhelmed by overall sparsity. Simultaneously, we utilize a graph embedding module to effectively process non-Euclidean data through a Graph Convolutional Network (GCN). These components in spatio-temporal module enhance Satformer's ability to capture and exploit the nonlinear and complex information present in traffic data. Furthermore, a transfer module is incorporated to disseminate global context information throughout the model.

Our contributions are as follows:

We designed ASSIT, which adopts a multi-head self-attention structure. It can learn the correlation representation of traffic data at different spatial and temporal scales. We added a sparsity threshold to the attention matrix to efficiently process a large number of sparse inputs. By dynamically adjusting the threshold value, ASSIT adapts to the sparsity levels of various datasets, thereby enhancing the model's inference efficiency. Additionally, ASSIT allows the model to dynamically allocate computational resources to regions of interest, making the model operate more efficiently and enhancing its scalability.

To process non-Euclidean structured data, we introduce a graph embedding module within each module via GCN. Since the graph embedding module learns the relationship between nodes and neighbors adequately, it can extract the local and global information of nodes from non-Euclidean structured data. It improves the ability of the model to extract nonlinear spatio-temporal correlation.

We add a transfer module to the Satformer framework, which can blend and reshape the traffic representation learned by the previous modules, conveying a global temporal and spatio perspective, while also helping to strengthen the generalization ability of the model on different types of datasets

This paper is structured as follows. Section 2 surveys relevant research. Section 3 explains our proposed Satformer methodology. Section 4 presents experimental verification and comparisons. Section 5 makes a discussion and concludes this paper.

2 Related Works

We provide a review of the existing work on network traffic estimation. Existing traffic data estimation methods can be mainly divided into matrix completion based, tensor completion based and neural network based methods.

Matrix Completion (MC) methods have found widespread application in the estimation of traffic data. Some algorithms, such as the convex relaxation method based on minimum nuclear norm approximation [12] and matrix factorization-based methods [13], leverage the linear spatiotemporal characteristics of traffic data to infer missing values. However, these methods are often too simplistic, which can lead to inaccurate estimations when applied to large-scale traffic data.

As an extension of matrix completion, the goal of tensor completion aims to reconstruct low-rank tensors based on sparse observations of their entries. Several studies have adopted tensor completion, including recent works [14, 15, 16].To achieve higher accuracy in traffic data estimation, these works propose the use of tensor completion methods, which can more comprehensively capture spatio-temporal features in traffic data, effectively. A typical work of such a method is LTC [17], which leverages the strong local correlation of the data to identify and complete each subtensor with low rank. However, many traffic estimation algorithms based on tensor completion rely on CANDECOMP/PARAFAC (CP) or Tucker decompositions, commonly using inner products as interaction functions. This approach can often reduce estimation performance to some extent due to its limited ability to capture both linear and nonlinear correlations in traffic data.

In recent years, deep learning methods have shown notable advancements in traffic network analysis. Notably, research such as NTF [18] and [19] have explored the application of deep learning models, including Recurrent Neural Networks (RNNs), to achieve adaptive grouping and prediction of traffic tensors within large-scale networks. Noteworthy among these efforts is CoSTCo [20], which incorporates two convolutional layers to extract features from stacked embeddings, enhancing awareness of network dynamics through the acquisition of complex spatio-temporal features. Recent studies [21, 22] employ meta-learning and other algorithms, alongside attention mechanisms, to dynamically adapt to rapid changes in traffic patterns within the network. However, current deep learning models may focus more on global features, while neglecting the local and hidden spatiotemporal correlations in traffic data, which may lead to suboptimal estimation effects

3 Estimation Model: Satformer

3.1 System Model & Problem Definition

In satellite networks, inter-satellite traffic data can be modeled as a time-space matrix, which reflects the data volume to be transmitted between all node-node pairs over satellite networks. For the problem statement of traffic estimation over satellite networks, we introduce the following symbols: N : Number of satellites, T : Discrete time steps, we define the inter-satellite traffic matrix Y ∈ R I × J × T , where Y ijt represents the data transmission from satellite i to satellite j at time step t . The t -th layer of this matrix represents a discrete time step.

Considering the influence of spatio distance and transmission delay in satellite networks, we can adjust the inter-satellite traffic by introducing a weight matrix. Let W ∈ R N × N be the weight matrix representing spatial distance and transmission delay, where W ij denotes the weight from satellite i to satellite j . Thus, the adjusted inter-satellite traffic data matrix can be represented as ˙ X = Y glyph[circledot]W , where glyph[circledot] denotes element-wise multiplication. Taking into account these factors, the mathematical modeling of inter-satellite traffic data can be expressed as follows:

where i, j = 1 , 2 , . . . , N , and t = 1 , 2 , . . . , T . This model considers the spatio distance and transmission delay between satellites, allowing the traffic data matrix to more accurately reflect the actual communication scenarios in the satellite networks. In the process of sampling and recovering inter-satellite traffic data, we begin by introducing the sampling matrix S , the sampled data X , and the nonlinear estimation function F . The sampling process can be expressed using mathematical notation: X = ˙ X glyph[circledot] S .

This process retains elements in the inter-satellite traffic matrix ˙ X where the corresponding positions in the sampling matrix S are 1, while setting other positions to zero, resulting in the sampled data matrix X . To recover complete traffic data from the sampled data, we introduce a nonlinear estimation function F . This function involves a complex nonlinear mapping to better estimate actual traffic data.

where X represents sampled data, and ˜ X is the recovered data obtained through the non-linear estimation function F .

3.2 Satformer Overview

Spatio-TemporalModule

Spatio-TemporalModule

Output Tensor Y

MLP

Embedding

Module

Graph

Satformer

Block

Transfer

Module

Embedding

Module

Graph

Satformer

Block

Output

LN

Time Step

Time Step

ASSIT

Block

Feature Matrix

Attention Scores

Feature Matrix

Attention Scores

LN

(a)Framework of Satformer

Encoder

Decoder

1(b)Satformer Block

Dynamic

Graphs

Region Split

Concat&Reshape

Links with unknown traffic

For each region

LEOMega-Constellation

with N satellites

Links with known traffic

(o）M

(t)M

Destination

Input Tensor X

Unknown traffic data

X

Known traffic data

H(0)

H(1)

(c)Traffic Data Generate

(e)ASSIT Block

We design Satformer, a tensor completion model designed for the accurate and robust estimation of global traffic data in satellite networks. As illustrated in Fig. 1, Satformer is structured as an encoderdecoder architecture, with both components featuring multiple spatio-temporal modules. Residual connections interlink these modules to prevent neural network degradation. Each spatio-temporal module comprises a Graph Embedding Module and a Satformer Block. The key of Satformer to improve the estimation accuracy is that it can extract features efficiently and accurately from a large number of sparse satellite network traffic data. This is achieved through adaptive sparse spatiotemporal attention inside each Satformer block, facilitating the estimation of traffic data. A transfer module facilitates the seamless transmission of features from the encoder to the decoder. The encoder encodes the input traffic information, while the decoder is tasked with estimating the missing traffic data. The subsequent section provides a detailed description of each module.

3.3 Spatio-Temporal Module

Satformer utilizes spatio-temporal modules to extract spatio-temporal features from input tensors; this module primarily consists of graph embedding components and Satformer blocks.

Graph Embedding : The Spatio-Temporal module serves the goal of extracting spatio-temporal features from input tensor. Considering the inherent high sparsity of observed traffic data in realworld, it becomes imperative to represent tensors as low-dimensional vectors. Through the learning of embedded representations for nodes, the model inherently captures both structural and semantic information of nodes within the graph. This capability enables the model to comprehend relationships between nodes more effectively, facilitating the extraction of meaningful features from the X ∈ R I × J × T . Each Origin-Destination (OD) pair corresponds to an origin node, a destination node, and the traffic of the OD pair. To address the non-Euclidean nature of the data, particularly the spatio relationships within each OD pair, we employ Graph Embedding through Graph convolutional neural network (GCN), which has been widely used in many works [23, 24]. This approach allows the model to effectively handle non-Euclidean data, enhancing its capacity to capture and utilize the structural information present in the tensor X ∈ R I × J × T .

In Satformer, each Spatio-Temporal module contains a GCN model. A GCN model contains two layers of convolutional layer, the feature propagation rule can be stated as follows:

where, H ( l ) signifies the node embedding matrix for layer l , A represents the adjacency matrix and I represents the self-connections matrix of A . ˜ A ∈ R I × J represents the adjacency matrix with self-connections. ˜ D ∈ R I × I denotes the degree matrix, which is a diagonal matrix with each element on the diagonal representing the sum of the corresponding row in ˜ A . The weight matrix for layer l is denoted as W ( l ) ∈ R I × M , and H ( l +1) ∈ R I × J × M represents the node embedding matrix for layer l + 1 . W (0) ∈ R K × L denotes the weight matrix from the input layer to the hidden layer, and W (1) ∈ R K × L denotes the weight matrix from the hidden layer to the output layer. Both σ ( · ) and ReLU are activation functions employed in the model. Z ∈ R I × M × K represents the output embedding tensor.

Satformer Block : As shown in Fig. 1 (b), in each Satformer block, we use a layer normalization at the beginning to normalize the input embedding tensor. We then apply an ASSIT mechanism and a 2-layer MLP module for sparse spatio-temporal feature modeling and per-location embedding, respectively.

In the domain of communication network tensor completion, the spatio-temporal relationships among traffic data are complex, and it is necessary to model these relationships effectively. Traditional attention mechanisms, with their intensive nature, may encounter challenges related to high computational complexity and difficulties in capturing global relationships in such intricate scenarios. Several works proposed different sparse attention mechanism to mitigate such issue either relay on static patterns or skip computations in specific regions. As shown in Fig. 1(e), in this work, we explore an adaptive, sparse spatio-temproal mechanism. Detailed descriptions are as follows:

Given an input embedding feature tensor Z ∈ R I × M × K . First we divide the tensor slice into several local regions Z div , each of which has a size of D × D . In our module, Q , K and V respectively represent query, key, and value, which are used to calculate attention weights and generate the final output[25]. Then we calculate Q , K and V tensor with linear projections for each region:

where W q , W k and W v are projection weights for query Q , key K and value V respectively. We then consider introducing a local attention mechanism when calculating the attention score α to make the model pay more attention to each local region in the input tensor. This improvement is designed to sharpen the model's attention specifically on local regions within the input sequence. The goal is to augment the expressiveness and robustness of the model by enabling it to capture and leverage more nuanced details and patterns present in localized segments of the input data. The implementation involves incorporating a position-related weight when calculating attention scores. The local attention in each region is operationalized through the use of a two-dimensional mask matrix Ψ ∈ R D × D , wherein, elements inside a defined center window H are retained, while elements in other positions are set to zero. The size of the center window H is a hyperparameter of the model, and its optimal value is determined through experiments on different datasets. The calculation of attention score α i for each region can be denoted as follows:

where glyph[circledot] is an element-wise product and C is scaling factor. And the final output α t are computed as:

To regulate the sparsity of the attention scores and channel the model's focus onto specific portions of the input, an adaptive sparse regularization term is introduced. This involves applying L1 regularization to each element of the attention score matrix. The utilization of ReLU operations ensures that the

attention scores remain non-negative. Thus the sparse mask can be denoted as: ReLU(1 -W r α i ) . Finally, we apply the weighted Value to the attention score, while introducing additional learnable parameters to allow the model to adaptively learn the weighted sum of each position, resulting in the final output α t , as shown in Eq. 9. Where, W r is the weighted matrix of L1 regularization, W s is the scaling matrix, both W r and W s are trainable parameters.

3.4 Transfer Module

(a)AttentionWeight

a

Linear

ScaledDot

Procudt

Attention

Attention

Attention

Attention

Weight

Weight

Weight

Weight

W=

(b)AttentionScore

W

Attention

Attention

Attention

Attention

Score

Score

Score

Score

Linear

ao

a

a-1

a

1

1

softmax

1+e

1+e

1+e"

1+e"

d,

a=sofimax(p

K

The conventional information transfer between the encoder and decoder typically relies on the output of the last layer of encoder. However, this approach may fall short in adequately conveying global context information, particularly when dealing with input tensors spanning a large number of time slices. The accumulation of errors over time can become a challenge. Consequently, it is necessary to add a module between encoder and decoder to effectively transfer the information. Satformer incorporates a self-attention-based transfer module between the encoder and the decoder. This module leverages Self-Attention, enabling the seamless transfer of globally contextual information learned in the encoder to the decoder. This augmentation empowers the decoder to more comprehensively consider information from the entire input sequence when generating output for each time slice, thus enhancing the estimation accuracy of missing values. Moreover, the transfer module enables the model to integrate spatio-temporal information in a more fine-grained manner, improving its adaptability to patterns across different temporal and spatio scales. The mathematical description of the Transfer Module is as follows:

Suppose the encoder outputs an eigenvector e t for each time step t input x t , where t = 1 , 2 , . . . , T , then the output sequence of the encoder is E = e 1 , e 2 , . . . , e T . The goal of Transfer Module is to convert the output of the encoder E to a new set of feature vector D = d 1 , d 2 , . . . , d T , where each d t is a feature enhanced representation corresponding to time step t . This process is achieved through the following self-attention mechanisms:

Calculate Query, Key, and Value: The query vector Q = EW Q represents the query of future time points against past time points. The key vector K = EW K represents the encoding of a past point in time. The value vector V = EW V represents the specific characteristics of past time points. W Q , W K , and W V are learnable weight matrices.

Calculate attention weight: Calculate the attention weight α t,t -1 of each time step t and consider the effect that past attention scores exert on the present. C t is scaling factor at time step t , i belongs to 1 to t -1 , p and q are parameters to control the effort of past time attention scores.

Generate transformation feature vector: Based on weights, attention to each time step t to generate a new feature vector d t .

This process enables the Transfer Module to accurately measure the relationship between each future time point and all past time points, and to generate a new set of features that represent valuable information for future predictions.

3.5 Loss Function

During the training stage, the primary objective is to minimize the discrepancy between the actual and predicted traffic data. To achieve this, the loss function employed by Satformer is the mean square error (MSE), as expressed in Eq. 12. Additionally, to curtail the growth of model weights and mitigate the risk of overfitting, a penalty term is incorporated into the loss function.

where ¯ A denotes the set of observed traffic data, χ ijk and ˆ χ ijk are the truth and estimated traffic data respectively, θ represents all trainable parameters in Satformer, λ is weight decay coefficient.

4 Experiments

4.1 Experimental Settings

Datasets. To assess the performance of Satformer, we employ it on three real-world satellite networks: Iridium, Telesat, and Starlink, thereby evaluating its capabilities across varying network scales: smallscale, medium-scale, and large-scale environments. Given the ongoing construction and utilization of many satellite networks, acquiring actual traffic data proves to be challenging. Thus, we generate corresponding traffic datasets using real satellite parameters and ground station coordinates. Similar methods have been used in many previous studies, and the specific details of this process are explained in the Appendix A. The traffic data collection interval was 1 second for all three datasets.

Iridium [26] : The Iridium constellation comprising a total of 66 satellites uniformly distributed across 6 orbital planes. For our experimentation, we focus on the initial six periods, encompassing 36,000 time slices.

Telesat [27] : It collects traffic data from the Telesat constellation which has a total of 298 satellites distributed in 26 orbital planes. We select the first five periods about 31500 time slots in our experiment.

Starlink [28] : The traffic data recording originates from the Starlink constellation, comprising 1584 Low Earth Orbit (LEO) satellites evenly dispersed across 72 orbital planes. The first six periods about 32400 time intervals in our experiment.

For all three datasets, we divided the original dataset into a training set and a test set in an 8:2 ratio using the time slice partitioning method. We then used the training set for model training and the validation set for model validation and tuning. Subsequently, we constructed the test set by randomly masking portions of the training and validation sets that were not used for training. This approach ensures that the model is trained and validated on distinct segments of the data, which can help prevent overfitting and improve the model's ability to generalize to new, unseen data.

Baselines. For comparative analysis against our Satformer model, we select the following baseline models: three mathematical tensor completion models, namely HaLRTC, LATC and LETC, and four state-of-the-art neural network-based tensor completion models, CoSTCo, DAIN, SPIN and STCAGCN.

HaLRTC [29] : A prototypical high-accuracy low-rank tensor completion algorithm utilizes the Alternating Direction Method of Multipliers (ADMMs) to attain precise outcomes, effectively managing dependencies among various constraints.

LATC [30] : It introduces a novel regularization term, integrating temporal variation, into a third-order tensor completion model.

LETC [31] : a Laplacian enhanced low-rank tensor completion framework for large-scale traffic speed kriging.

CoSTCo [20] : An innovative Convolutional Neural Network (CNN)-based model developed for tensor completion to overcome the limitations associated with traditional low-rank tensor factorization approaches.

CDSA [32] : A novel cross-dimensional self-attention approach for imputing missing values in multivariate, geo-tagged time series data.

DAIN [33] : This method explicitly crafted to enhance the accuracy of neural tensor completion methods when predicting missing values within sparse, multi-dimensional datasets.

SPIN [34] : An attention-based architecture using spatiotemporal graphs and autoregressive models for effectively reconstructing missing data in sparse, multivariate time series.

SAITS [35] : a self-attention-based method for multivariate time series imputation that uses joint-optimization and diagonally-masked self-attention blocks.

STCAGCN [36] : A graph-based deep learning method for traffic volume estimation by utilizing a graph attention-based speed pattern-adaptive adjacency matrix and a customized temporal attention mechanism.

Evaluation Metrics. Two widely employed metrics are applied to evaluate the estimation performance of Satformer. The calculation equations for these metrics are presented as follows:

Normalized Mean Absolute Error (NMAE) :

Normalized Root Mean Squared Error (NRMSE) :

where χ ijk and ˜ χ ijk represent the truth value and estimated value, ¯ A denotes the set of unobserved traffic data. For both two metrics, the smaller they get to 0, the better the estimation performance of the model.

4.2 Performance Comparison with Baselines

Compare Satformer with mathematical baselines. Table 1 provides a summary of the experimental results for our Satformer and the mathematical tensor completion baselines, HaLRTC, LATC and LETC. Performance evaluations, measured by NMAE and NRMSE, are conducted across three datasets with sampling ratios ranging from 2% to 10%. Our Satformer consistently outperforms the mathematical tensor completion algorithms, achieving significant improvements. Notably, even at the minimal 2% sampling ratio, Satformer maintains proficient performance, with NMAE values recorded as 0.098, 0.1017, and 0.1402 for the Iridium, Telesat, and Starlink datasets, respectively. In comparison, the leading mathematical models exhibit higher NMAE values of 0.2782, 0.2723, and 0.3784 under the same 2% sampling ratio. The observed performance enhancement in Satformer quantifies at 84.38%, 86.43%, and 106.77% for the respective datasets. Similar trends are also observed in NRMSE. These results indicate that mathematical models based on Alternating Direction Method of Multipliers or reliant on strong assumptions struggle to capture the complex spatiotemporal characteristics. In contrast, neural network-based models such as Satformer demonstrate formidable nonlinear representation capabilities, enabling effective extraction of spatio-temporal features from traffic data.

Compare Satformer with neural network-based baselines. Our Satformer outperforms the neural network-based baselines (CoSTCo, DAIN, SPIN, and STCAGCN) across all datasets, achieving the best estimation performance, as shown in Table 1. Notably, even with a 2% traffic data sampling rate, Satformer demonstrates significant improvements compared to the best-performing neural network-based baselines. On the Iridium dataset (66 satellites), Satformer improves NMAE and NRMSE by 8.57% and 8.95%, respectively. As the size of the dataset increases, performance

improvements continue and escalate. On the Telesat dataset (298 satellites), Satformer achieves improvements of 27.63% in NMAE and 18.05% in NRMSE. For the Starlink dataset (1584 satellites), Satformer exhibits even more substantial improvements, with NMAE and NRMSE increasing by 38.66% and 32.32%, respectively. These results highlight Satformer's effectiveness in handling large-scale datasets, suggesting potential deployment in real-world satellite networks. The limitations

of CoSTCo are evident due to its exclusive reliance on two-dimensional convolution for spatial feature extraction without explicitly modeling temporal features. DAIN falls short by not explicitly modeling interactions between entities, which limits information utilization, despite its combination of information for data augmentation. SPIN's ability to handle sparsity or irregularly sampled data might be limited, which could affect the accuracy of traffic estimation in satellite networks where data is often incomplete. STCAGCN captures time-asynchronous correlations may not fully account for the complex temporal dynamics in satellite network, leading to less accurate estimations. The architecture of STCAGCN cannot ensure the information learned at earlier stages is preserved and utilized in later stages. Although CDSA also utilizes the self-attention mechanism, its dimension-wise processing may limit its ability to capture complex interactions. RNN-based models of SAITS are generally inferior to Transformer architectures in terms of handling long-distance dependencies and efficiency. In contrast, Satformer excels by explicitly incorporating both spatial and temporal features within each module. The graph embedding captures nonlinear information, the Satformer module integrates the ASSIT, and the transfer module seamlessly transmits global contextual information. This comprehensive design enables Satformer to deliver exceptional performance in inter-satellite traffic data estimation, effectively addressing the challenges of large-scale, sparsely populated datasets.

5 Conclusion and Discussion

This paper proposes Satformer, a novel traffic data estimation algorithm for large-scale satellite networks, aiming at fast and accurate estimating global traffic matrix from partial sampling in a cost-effective manner. Motivated by this, we design a region-aware sparse spatio-temporal attention mechanism to concentrate on specific local regions of the input tensor, where the input tensor is embedded in a graph convolutional neural network. Thus, spatio-temporal features from the traffic matrix are effectively extracted with computational efficiency and robustness.

Extensive experiments with datasets of varying scales-small, medium, and large have shown that Satformer has significant advantages on both accuracy and efficiency for traffic estimation compared with baselines, particularly in larger networks. Moreover, we analyze the robustness of Satformer under different conditions and further verify the role of each module through ablation studies. The results demonstrate the potential of Satformer for deployment in actual systems.

Despite Satformer is effective adopted for traffic estimation, deep learning models for traffic estimation remain mostly black boxes. It is quite important to understand the reasons behind inferences in the satellite networking domain. In addition, although Satformer is cost-effective, it is necessary to further reduce its computational complexity, considering the limited computational resources of existing satellites.

Future works should prioritize enhancing computational efficiency. It is also important to explore interpretability and decision basis of our deep learning model for traffic estimation. For example, explanation techniques, such as a local interpretable model-agnostic explanation (LIME) [37], are able to make a visual analysis of the model and analyze the internal working mechanism from specific examples. Additional explanatory tools, such as feature importance analysis, will help users in understanding the model's workings.

Acknowledgments and Disclosure of Funding

We thank Xiaoshan Yu for the inspiration we got from chatting with him, and Xingyu Liu, our seven anonymous reviewers, and audience at ICML and this conference for helpful comments and suggestions. This work was supported in part by the National Natural Science Foundation of China under Grant 61934002 and 62102302, the Fundamental Research Funds for the Central Universities under Grant XJSJ23088, the National Key Laboratory of Advanced Communication Networks under Grant FFX22641X007, BAX24641X002, and also supported by The Youth Innovation Team of Shaanxi Universities.

References

Yu Zhang, Yanmin Gong, Lei Fan, Yu Wang, Zhu Han, and Yuanxiong Guo. Quantum-assisted joint caching and power allocation for integrated satellite-terrestrial networks. Dec 2023.

Hossein Akhlaghpasand and Vahid Shah-Mansouri. Traffic offloading probability for integrated leo satellite-terrestrial networks. IEEE Communications Letters , 2023.

Lei Lei, Anyue Wang, Eva Lagunas, Xin Hu, Zhengquan Zhang, Zhiqiang Wei, and Symeon Chatzinotas. Spatial-temporal resource optimization for uneven-traffic leo satellite systems: Beam pattern selection and user scheduling. IEEE Journal on Selected Areas in Communications , 2024.

Binquan Guo, Zheng Chang, Zhu Han, Wanting Yang, and Zehui Xiong. Network slicing strategy for real-time applications in large-scale satellite networks with heterogeneous transceivers. IEEE Wireless Commun. Lett. , May 2024.

Zhuangzhuang Ma, Yongli Zhao, Wei Wang, and Jie Zhang. Demonstration of highly dynamic satellite optical networks supporting rapid reconfiguration. In 2021 17th International Conference on the Design of Reliable Communication Networks (DRCN) , pages 1-3. IEEE, 2021.

Hossein Akhlaghpasand and Vahid Shah-Mansouri. Traffic offloading probability for integrated leo satellite-terrestrial networks. Jul 2023.

Kun Xie, Lele Wang, Xin Wang, Gaogang Xie, Guangxing Zhang, Dongliang Xie, and Jigang Wen. Sequential and adaptive sampling for matrix completion in network monitoring systems. In 2015 IEEE Conference on Computer Communications (INFOCOM) , Apr 2015.

Joao Marco C. Silva, Paulo Carvalho, and Solange Rito Lima. Computational weight of network traffic sampling techniques. In 2014 IEEE Symposium on Computers and Communications (ISCC) , Jun 2014.

Haitao Xu, Shuying Han, Xuhui Li, and Zhu Han. Anomaly traffic detection based on communication-efficient federated learning in space-air-ground integration network. IEEE Transactions on Wireless Communications , 22(12):9346-9360, 2023.

Lei Lei, Anyue Wang, Eva Lagunas, Xin Hu, Zhengquan Zhang, Zhiqiang Wei, and Symeon Chatzinotas. Spatial-temporal resource optimization for uneven-traffic leo satellite systems: Beam pattern selection and user scheduling. IEEE Journal on Selected Areas in Communications , 2024.

Zhuangzhuang Ma, Yongli Zhao, Wei Wang, and Jie Zhang. Demonstration of highly dynamic satellite optical networks supporting rapid reconfiguration. In 2021 17th International Conference on the Design of Reliable Communication Networks (DRCN) , Apr 2021.

Jian-Feng Cai, EmmanuelJ. Candès, and Zuowei Shen. A singular value thresholding algorithm for matrix completion. arXiv: Optimization and Control,arXiv: Optimization and Control , Oct 2008.

Yehuda Koren, Robert Bell, and Chris Volinsky. Matrix factorization techniques for recommender systems. Computer , page 30-37, Aug 2009.

Kun Xie, Lele Wang, Xin Wang, Gaogang Xie, Jigang Wen, Guangxing Zhang, Jiannong Cao, and Dafang Zhang. Accurate recovery of internet traffic data: A sequential tensor completion approach. IEEE/ACM Transactions on Networking , 26(2):793-806, 2018.

Qianqian Wang, Lei Chen, Qin Wang, Hongbo Zhu, and Xianbin Wang. Anomaly-aware network traffic estimation via outlier-robust tensor completion. IEEE Transactions on Network and Service Management , 17(4):2677-2689, 2020.

Dingde Jiang, Wenjuan Wang, Lei Shi, and Houbing Song. A compressive sensing-based approach to end-to-end network traffic reconstruction. IEEE Transactions on Network Science and Engineering , 7(1):507-519, 2020.

Kun Xie, Can Peng, Xin Wang, Gaogang Xie, Jigang Wen, Jiannong Cao, Dafang Zhang, and Zheng Qin. Accurate recovery of internet traffic data under variable rate measurements. IEEE/ACM transactions on networking , 26(3):1137-1150, 2018.

Xian Wu, Baoxu Shi, Yuxiao Dong, Chao Huang, and Nitesh V Chawla. Neural tensor factorization for temporal interaction learning. In Proceedings of the Twelfth ACM international conference on web search and data mining , pages 537-545, 2019.

Yudian Ouyang, Kun Xie, Xin Wang, Jigang Wen, and Guangxing Zhang. Lightweight trilinear pooling based tensor completion for network traffic monitoring. In IEEE INFOCOM 2022-IEEE Conference on Computer Communications , pages 2128-2137. IEEE, 2022.

Hanpeng Liu, Yaguang Li, Michael Tsang, and Yan Liu. Costco: A neural tensor completion model for sparse tensors. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , pages 324-334, 2019.

Yuhui Li, Wei Liang, Kun Xie, Dafang Zhang, Songyou Xie, and KuanChing Li. Lightnestle: quick and accurate neural sequential tensor completion via meta learning. In IEEE INFOCOM 2023-IEEE Conference on Computer Communications , pages 1-10. IEEE, 2023.

Jingyi Cai, Shutian Song, Haipeng Zhang, Ruiliang Song, Bo Zhang, and Xiang Zheng. Satellite network traffic prediction based on lstm and gan. In 2023 IEEE 3rd International Conference on Information Technology, Big Data and Artificial Intelligence (ICIBA) , volume 3, pages 175-178. IEEE, 2023.

Yu Sha, Shuiping Gou, Bo Liu, Johannes Faber, Ningtao Liu, Stefan Schramm, Horst Stoecker, Thomas Steckenreiter, Domagoj Vnucec, Nadine Wetzstein, et al. Hierarchical knowledge guided fault intensity diagnosis of complex industrial systems. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , pages 5657-5668, 2024.

Liang Qin, Huaxi Gu, Wenting Wei, Zhe Xiao, Zexu Lin, Lu Liu, and Ning Wang. Spatiotemporal communication network traffic prediction method based on graph neural network. Information Sciences , page 121003, 2024.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, AidanN. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Neural Information Processing Systems,Neural Information Processing Systems , Jun 2017.

Zaher M Kassas, Sharbel Kozhaya, Haitham Kanj, Joe Saroufim, Samer W Hayek, Mohammad Neinavaie, Nadim Khairallah, and Joe Khalife. Navigation with multi-constellation leo satellite signals of opportunity: Starlink, oneweb, orbcomm, and iridium. In 2023 IEEE/ION Position, Location and Navigation Symposium (PLANS) , pages 338-343. IEEE, 2023.

Nils Pachler, Inigo del Portillo, Edward F Crawley, and Bruce G Cameron. An updated comparison of four low earth orbit satellite constellation systems to provide global broadband. In 2021 IEEE international conference on communications workshops (ICC workshops) , pages 1-7. IEEE, 2021.

Sami Ma, Yi Ching Chou, Haoyuan Zhao, Long Chen, Xiaoqiang Ma, and Jiangchuan Liu. Network characteristics of leo satellite constellations: A starlink-based measurement from end users. In IEEE INFOCOM 2023-IEEE Conference on Computer Communications , pages 1-10. IEEE, 2023.

Ji Liu, Przemyslaw Musialski, Peter Wonka, and Jieping Ye. Tensor completion for estimating missing values in visual data. IEEE transactions on pattern analysis and machine intelligence , 35(1):208-220, 2012.

Xinyu Chen, Mengying Lei, Nicolas Saunier, and Lijun Sun. Low-rank autoregressive tensor completion for spatiotemporal traffic data imputation. IEEE Transactions on Intelligent Transportation Systems , 23(8):12301-12310, 2022.

Tong Nie, Guoyang Qin, Yunpeng Wang, and Jian Sun. Correlating sparse sensing for large-scale traffic speed estimation: A laplacian-enhanced low-rank tensor kriging approach. Transportation research part C: emerging technologies , 152:104190, 2023.

Jiawei Ma, Zheng Shou, Alireza Zareian, Hassan Mansour, Anthony Vetro, and Shih-Fu Chang. Cdsa: cross-dimensional self-attention for multivariate, geo-tagged time series imputation. arXiv preprint arXiv:1905.09904 , 2019.

Sejoon Oh, Sungchul Kim, Ryan A Rossi, and Srijan Kumar. Influence-guided data augmentation for neural tensor completion. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management , pages 1386-1395, 2021.

Ivan Marisca, Andrea Cini, and Cesare Alippi. Learning to reconstruct missing data from spatiotemporal graphs with sparse observations. Advances in Neural Information Processing Systems , 35:32069-32082, 2022.

Wenjie Du, David Côté, and Yan Liu. Saits: Self-attention-based imputation for time series. Expert Systems with Applications , 219:119619, 2023.

Tong Nie, Guoyang Qin, Yunpeng Wang, and Jian Sun. Towards better traffic volume estimation: Jointly addressing the underdetermination and nonequilibrium problems with correlationadaptive gnns. Transportation Research Part C: Emerging Technologies , 157:104402, 2023.

Aditya Bhattacharya. Applied Machine Learning Explainability Techniques: Make ML models explainable and trustworthy for practical applications using LIME, SHAP, and more . Packt Publishing Ltd, 2022.

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014.

Rob J Hyndman and George Athanasopoulos. Forecasting: principles and practice . OTexts, 2018.

Yingyue Bi, Yingcong Lu, Zhen Long, Ce Zhu, and Yipeng Liu. Tensor decompositions: computations, applications, and challenges. Tensors for Data Processing , pages 1-30, 2022.

Chuishi Meng, Xiuwen Yi, Lu Su, Jing Gao, and Yu Zheng. City-wide traffic volume inference with loop detector data and taxi trajectories. In Proceedings of the 25th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems , SIGSPATIAL '17, New York, NY, USA, 2017. Association for Computing Machinery.

A Traffic Generator

To the best of our knowledge, there is no publicly available inter-satellite traffic data or literature describing the traffic distribution for any existing satellite networks. This includes long-established networks like Iridium, as well as those currently under development, such as Starlink. The characteristics of traffic load borne by satellite networks are intricate. Satellites predominantly communicate with terrestrial terminals via ground stations. Spatially, the distribution of ground stations is uneven due to factors such as topography, economic considerations, and geopolitical influences. In regions with extreme environmental conditions or economic underdevelopment, such as oceans, deserts, and polar areas, the received traffic is significantly lower compared to more favorable environments, contributing to an uneven spatial distribution of traffic in the satellite network. Furthermore, the global distribution of earth stations spans various time zones, resulting in non-stationary traffic generation at different times. This temporal variability leads to significant traffic variations among stations. Additionally, to ensure link quality between satellites and ground stations and to mitigate the impact of frequent satellite handoffs, ground stations must consider multiple factors, including elevation angle, service time, and signal strength, when selecting communication satellites. This complexity adds to the challenges associated with managing traffic in satellite networks.

Space

The coordinates of more than 1000 stations in SOD

Visibility analysis of ground stations and LEO satellites

Access selection between ground stations and LEO satellites

Generate traffic matrix sequence between ground

stations under distributation of gravity

Traffic load of each ground station in different zones

Time

Sequence

of LEO

inter-

satellite

traffic

matrices

(a)

(b)

normalized network trafficvalue

0.8

0.6

0.4

0.2

60

0:00

1:30

3:00

4:306:007:30

9:0010:3012:0013:30

15:00

16:30

18:00

Time UTC(1hour)

The pivotal aspect in assessing the effectiveness of the proposed scheme is constructing a coherent satellite network traffic model that accurately represents the traffic characteristics of the satellite network. The devised traffic generation method, as illustrated in the accompanying figure, takes into account the spatial distribution of ground stations, temporal characteristics, and satellite-ground access. Although this approach primarily focuses on inter-satellite communication and excludes satellite-ground communication, it offers comprehensive consideration of three key factors: spatial distribution of ground stations, temporal dynamics, and satellite-ground access. The culmination of these considerations results in the generation of a sequence of inter-satellite traffic matrices.

spatio distribution of ground stations. Due to limitations imposed by antenna size, equipment volume, and quality, current user communications with satellites predominantly occur through ground

(a)

(b)

(c)

stations. The spatial distribution of traffic load in a satellite network is intricately linked to the global geographical distribution of earth stations. In this work, ground stations are strategically positioned based on the coordinates provided by the Standard Object Database (SOD) within the Satellite Tool Kit (STK). The SOD database contains geographical location information for 1,016 Earth stations worldwide, primarily situated in islands, mountainous areas, rural locales, and other remote regions effectively served by satellites. In contrast to alternative assumptions regarding user distribution, leveraging the SOD ensures a more accurate representation of user distribution and density, which is crucial for this work. Consequently, this facilitates an effective depiction of the spatial distribution of traffic load within the satellite network.

temporal dynamics. The temporal variations in traffic load within non-geostationary orbit satellite networks predominantly arise from two factors: the diurnal fluctuations induced by regional local times and the geographic variations in daily traffic patterns influenced by global time zones. In the foreseeable future, the satellite optical network is expected to handle a traffic load comparable to that of the ground network, either matching, proportionally scaling, or exhibiting similar patterns. To ensure accuracy and effectiveness in generating traffic scenarios, this paper employs the four-month average daily traffic change trend from the GEANT network to characterize daily traffic variations. Fig.4b depicts the normalized cumulative traffic load over a 24-hour period, with the highest peak value normalized to 1. Notably, the flow intensity peaks around 12 noon, gradually diminishes, and then experiences a subsequent rise around 5 AM the following day. In addressing geographical variations, for the sake of model simplicity, the local time of a ground station within its respective time zone is incremented by one hour for every 15 degrees of longitude eastward from Greenwich Mean Time. This adjustment is contingent on the specific time zone associated with each ground station.

satellite-ground access. The capability of a ground station to establish satellite-ground communication links with multiple satellites within a given time window is influenced by factors such as satellite density and the coverage area of an individual satellite. Different satellite-ground access methods introduce varying effects on the characteristics of traffic load. Utilizing visibility analysis outcomes obtained through the Satellite Tool Kit (STK) and considering conditions for establishing satellite-ground links, the service time offered by all satellites visible to a ground station in each time window is computed. The satellite offering the longest service time is then selected for access, allowing for the flexibility to choose alternative or custom-designed satellite-ground access methods as needed. The ground station selects the destination node, following a uniform distribution. The traffic density at the ground station is jointly determined by the local time within the time zone and the spatial distribution of ground stations across each time zone.

Assuming the network time is based on GMT (Greenwich Mean Time) and considering the generation interval ∆ t ( s ) of the traffic matrix, the computation process for the traffic matrix sequence { F t ter | t ∈ N ∗ } between ground stations is as follows:

For any time t , the total traffic D t sent by all ground stations in the whole network is calculated as Eq.15.

where, offerload is the network traffic load, B is the maximum bandwidth of the inter-satellite link, and n ter is the total number of ground stations.

The latitude and longitude coordinates of any ground station i is ( x i , y i ) , then the local time t m ( h ) of the time zone m of any ground station i can be calculated by Eq.16:

Combined with the normalized cumulative traffic load in 24 hours in Fig.4b, the traffic intensity weight of the time zone m where the ground station i is located can be calculated:

where, w total is the total traffic load and w t m is the load at time t corresponding to time zone m .

The traffic ( f t m ) z that a ground station i located in time zone m needs to send at time t can be calculated by Eq.18 as follows.

where, n m denotes the number of ground stations in time zone m , and 23 ∑ m =0 n m = n ter .

Since the destination node is selected by the ground station according to uniform distribution, the traffic sent from the ground station i to the ground station j at time t can be calculated by Eq.19:

where, F ( i, j ) denotes the traffic from the source ground station i to the destination ground station j , and U (0 . 1 , 1) represents the uniform distribution from 0.1 to 1.

By traversing all the ground stations in the network at each time interval, the traffic matrix sequence { F t ter | t ∈ N ∗ } between the ground stations can be calculated. According to the inter-satellite visibility analysis results provided by STK, as shown in Fig.5, combined with the satellite-to-ground access method described previously, the inter-satellite traffic matrix sequence { F t sat t ∈ N ∗ } can be obtained.

B Theoretical Analysis

Lemma 1. Convergence of GCN Layer-wise Propagation.

Statement : For a multi-layer GCN with layer-wise propagation defined as H ( l +1) = σ ( D -1 2 ˜ AD -1 2 H ( l ) W ( l ) ) , the node embeddings H ( l ) converge as l increases, under mild conditions on the activation function σ and weight matrices W ( l ) . In our work, Lemma 1 supports the use of two-layer convolutional layers in GCN to effectively propagate features, thereby validating the capability of GCN to capture and utilize the structural information present in the input tensor.

Proof. Consider a GCN with L layers. The layer-wise propagation is given by:

where H ( l ) ∈ R N × d l is the node feature matrix at layer l , D is the degree matrix, ˜ A = A + I is the adjacency matrix with self-loops, and W ( l ) ∈ R d l × d l +1 is the weight matrix.

Assume the activation function σ is Lipschitz continuous with Lipschitz constant L σ , i.e., for all x, y ∈ R ,

Assume the weight matrices W ( l ) are bounded, i.e., there exists a constant M such that ‖ W ( l ) ‖ ≤ M for all Define the propagation operator Φ : R N × d l → R N × d l +1 as:

To show Φ is a contraction, consider two node feature matrices H 1 , H 2 ∈ R N × d l . We need to show that:

for some 0 ≤ k < 1 .

Compute the difference:

Using the Lipschitz continuity of σ :

Apply the sub-multiplicative property of norms:

Since D and ˜ A are derived from the graph structure and ‖ W ‖ is bounded by M :

where λ max is the largest eigenvalue of the normalized adjacency matrix.

Combining these, we get:

For Φ to be a contraction, we need:

If L σ λ max M < 1 , then Φ is a contraction mapping.By the Banach fixed-point theorem, every contraction mapping on a complete metric space has a unique fixed point.Therefore, the node embeddings H ( l ) will converge to a fixed point as l increases.

Lemma 2. Stability of Attention Mechanism with Masking.

Statement : The attention mechanism with a mask matrix focusing on the center region of the input data remains stable and does not degrade performance, provided the mask matrix is appropriately designed. Lemma 2 supports the introduction of a local attention mechanism in the adaptive sparse spatio-temporal attention mechanism, enabling the model to better capture and utilize details and patterns in localized regions of the input tensor.

Proof. The mask matrix M is designed such that M ij = 0 for elements outside the central region and M ij = 1 within the central region.This implies that only the attention scores corresponding to the central region are retained, effectively reducing the complexity of the attention mechanism by filtering out less relevant data.Mathematically, M acts as a sparsity-inducing regularizer:

By focusing on the central region, M ensures that the attention mechanism does not overfit to peripheral noise, enhancing generalization.

The central region often contains the most informative parts of the data, as observed in empirical studies (e.g., Fig.9(a)).

By applying M , the attention mechanism A ′ ( X ) prioritizes the computation of attention scores within this region, thus capturing critical local patterns:

This prioritization ensures that the most relevant features are emphasized, leading to improved prediction accuracy.

To show that the deviation introduced by M is bounded, consider the difference between the original and masked attention mechanisms:

Since M ij ∈ { 0 , 1 } , it acts as a binary mask, thus the modification is limited to setting some attention scores to zero while retaining the rest:

Given that softmax is a Lipschitz continuous function with constant 1 , the Frobenius norm ‖ ∆ A ‖ F is bounded by the norm of the difference in the inputs:

Since M zeros out peripheral entries, the difference is confined to the less relevant regions, ensuring that the overall deviation remains controlled.

C Implemenation Details

Satformer and the neural network-based baselines are implemented in PyTorch, while the mathematical baselines are implemented using Numpy. We evaluated Satformer against the baselines on a server equipped with an NVIDIA RTX 2080Ti GPU, 128 GB DDR4 RAM, and an Intel Xeon Silver 4208 CPU, running the Ubuntu 18.04 operating system. All models are trained for a range of 50 to 300 epochs with the first 5 epochs designated for warmup, and early stopping is adopted during the training process. The Adam optimizer [38] is used to optimize our model. A grid search strategy is applied to determine the best learning rate, epochs, and batch size. Based on the results of the grid search strategy, the optimal hyperparameters for the Satformer model and the neural network baselines are presented in Table 2, with the best weight decay determined to be 0.00001. We run each model 10 times with the same parameters and record the mean results in Table 1.

D Model Parameter Selection

Impact of module number. The module number indicates how many spatio-temporal modules should be contained in Satformer. It significantly impacts the computational efficiency and accuracy of Satformer. We adjust the number of modules from 1 to 20 and record the NMAE and NRMSE for each dataset, set the sample ratio to 2% and maintain the other hyperparameters constant. As shown in Fig.6, we observe that the error of Satformer continually decreases as the number of modules increases, up to a point (10 for all three datasets), after which it begins to increase. The reason is that increasing the number of modules enhances Satformer's ability to extract spatio-temporal features,

NMAE

NRMSE

0.13

0.18

0.25

0.12

0.16-

Error

Error

0.11

0.14

0.10-

?

0.12Q

0.15

0.10

0.09

5

10

15

20

5

10

15

20

5

10

15

20

Block Number

Block Number

Block Number

0.20

0.25 -

0.18

0.25

0.16

0.14

0.15

G

0.12

0.15

P

0.10

0.10-

0.10

20

40

60

20

40

60

20

40

60

CentreWindowSize

Centre WindowSize

Centre WindowSize

(a) Iridium

(b)Telesat

(c)Starlink

while an excessive number of modules can lead to model overfitting and increased complexity. Consequently, we select a module number of 10 for all three datasets.

Impact of centre window size. The centre window size indicates how many elements should be retained in a mask matrix. It also significantly influences the accuracy of Satformer. It is another crucial hyperparameter of Satformer. The results in Fig. 6 show that, for Iridium and Telesat, the estimation performance of Satformer continues to improve until it starts to decrease at a certain point (16 for Iridium and 32 for Telesat). Conversely, for Starlink, the estimation performance continues to increase. It can be observed that the window size is proportional to the dataset's scale. For small and medium-scale datasets, the model may extract useful information with a small center window size, thereby improving computational efficiency. However, for large-scale datasets, a larger center window size enables the model to extract more features while maintaining high accuracy. Accordingly, we set the center window size to 16 for Iridium, 32 for Telesat, and 64 for Starlink.

E Robustness Analysis

HaLRTC

LATC

￥LETC

中

CoSTCo

DAIN

4SPIN

STCAGCN

→—Satformer

0.8

0.8

0.8

NMAE

0.6

NMAE

0.6

NMAE

0.6

0.4

0.4

0.4

0.2

0.2

0.2

500

1000

1500

500

1000

1500

500

1000

1500

Time Slices

Time Slices

Time Slices

0.9

0.8

0.8

0.8

NRMSE

0.6

NRMSE

0.6

NRMSE

0.7

0.6

0.4

0.5

0.4

0.4

0.2

1.

0.2

0.3

500

1000

1500

500

1000

1500

500

1000

1500

Time Slices

Time Slices

Time Slices

(a)Iridium

(b)Telesat

(c) Starlink

To thoroughly evaluate the robustness and performance of Satformer, we conducted a comprehensive comparison of its Normalized Absolute Error (NMAE) and Normalized Root Mean Squared Error (NRMSE) metrics against all six baseline models across three diverse datasets. These datasets encompass varying numbers of time slices, ranging from 100 to 1500, providing a broad spectrum of temporal scales for analysis. As the number of time slices of the output increases, the accuracy will inevitably decrease, mainly because the complexity of the data increases exponentially with each additional dimension, making it harder to model and predict accurately [39]. In addition, In the process of tensor completion, especially over many time slices, there can be a loss of information that is critical for making accurate predictions. Each step in the prediction process may introduce slight inaccuracies, which accumulate over time [40]. The results from our experiments unequivocally demonstrate the superior and consistent performance of Satformer across all datasets, irrespective of the number of time slices involved. Notably, as the number of input time slices increases, Satformer consistently outperforms other models in terms of reliability, maintaining consistently low NMAE and NRMSE indicators. Importantly, even with the escalation of the temporal dimension and the expansion of dataset sizes, Satformer exhibits remarkable stability in its results. These findings underscore the robustness and scalability of Satformer, rendering it not only valuable in theoretical contexts but also highly applicable in real-world engineering scenarios.

F Ablation Study

Satformer-GE

中

Satformer-TM

Satformer-AT

中

Satformer

0.20

0.250

0.16

0.18

0.225

G

0.14

G

NMAE

0.16

0.200-

NMAE

0.14

0.175

0.12

0.10

中

0.150

中

0.10

中

中

中

中

0.08

日

0.125

2%

4%

6%

8%

10%

2%

4%

6%8%

10%

2%

4%

6%

8%

10%

Sample Ratio

Sample Ratio

Sample Ratio

0.45

0.35

0.25

0.40

NRMSE

NRMSE

0.30

NRMSE

0.20

0.35-

0.25

0.15

0.20

0.30

0.10

中

中

中

中

中

中

2%

4%

6%8%

10%

2%

4%

6%

8%

10%

2%

4%

6%

8%

10%

SampleRatio

SampleRatio

SampleRatio

(a)Iridium

(b)Telesat

(c) Starlink

To assess the impact of our graph embedding module (GE), ASSIT module, and transfer module on Satformer's performance, we conducted ablation experiments, systematically removing these key components to create various model variants. Specifically, we evaluated the following variants: 1) Satformer-GE omits the graph embedding module; 2) Satformer-AT removes the adaptive sparse spatio-temporal attention mechanism module; 3) Satformer-TM does not incorporate the transfer module.

Experimental results consistently demonstrate the superior performance of the complete model as compared to the variant models from which crucial components have been removed, across three datasets that have varying sampling rates. Notably, the absence of the graph embedding module hinders the effective capture of topological structure information, underscoring its pivotal role in augmenting overall model performance. The significance of the ASSIT module becomes evident in addressing sparsity issues within spatio-temporal data; when it is absent, the model experiences a noticeable decline in performance, validating its effectiveness in uncovering temporal and spatial dependencies. Furthermore, the transfer module plays a crucial role in facilitating the conversion of information between different feature spaces, thereby enhancing the model's capability in feature representation.

G Runtime Analysis

Table 3 presents a comparison of the training and inference times of Satformer with various baseline models. Although SPIN incorporates a sparse attention mechanism, it can be computationally intensive. This may lead to longer processing times. Notably, Satformer demonstrates the fastest training and inference times, which makes it particularly well-suited for real-world deployment. The significant reduction in these times is primarily attributed to the adaptive spatio-temporal attention mechanism, which introduces strategic sparsity in the sampling of input tensors and markedly reduces the number of parameters, offering a substantial time-saving advantage.

H Virtual Attention

The adaptive sparse spatio-temporal attention mechanism allows Satformer to focus on specific local regions of the input tensor, which is particularly beneficial for handling the large-scale and highly dynamic nature of satellite networks. We visualize the attention mechanism in Satformer to verify whether the functionality is achieved, as shown in Fig.9.

Fig.9(a) presents the initial traffic matrices for Iridium, Telesat, and Starlink satellite constellations. Each matrix's dimension is determined by the number of satellite nodes, denoted as N , with each point representing the volume of traffic between respective node pairs. Fig.9(b) illustrates the traffic matrices after applying a 10% sample rate to the three datasets, demonstrating a clear reduction in data density. Fig.9(c) displays the attention map, which are derived from the Satformer module using the attention scores, α s , as defined by Eq.9. The attention map's dimensions are R ( D × D × heads), which is then averaged across the head dimension and scaled to N × N in the D × D dimensions for visualization purposes. This scaling process is designed to be intuitive without altering the inherent relationships within the tensors. Finally, Fig.9(d) represents the estimated traffic matrices, which are the reconstructed traffic data from the sampled data.

Upon examining the initial traffic matrix for Iridium constellation, a notable volume of traffic is evident within the red-boxed area. After sampling, only sparse data points remain, yet the attention map successfully captures the significance of this high-traffic area, as indicated by the high attention scores. The estimated traffic matrix aligns well with the actual traffic in this region. Similar observations can be made for the Telesat and Starlink datasets, where the attention mechanism effectively identifies and emphasizes critical traffic areas, leading to accurate estimations within the reconstructed traffic matrices.

I General Tensor Completation Tasks

Although Satformer was developed to address traffic data estimation in satellite networks, the core strengths of its methodology endow it with the potential to be applied to other tensor completion

Original matrix of Iridium

Sample matrix of Iridium

Attentionmap of Iridium

Estimate matrix of Iridium

1.0

1

1

1.0

1.0

Destination

20

30

10

0.6

0.8

Destination

0.6

0.8

20

10

国

2:

1

.3

!'L

1

:

0.6

0.8

10

20

0.8

0.6

40

0.4

40

0.4

II

0.4

0.4

50

50

10

20

Source

30

40

50

60

0.2

0.0

50

09

10

20

Source

30

40

05

60

0.2

0.0

50

0

10

20

...

Source

.

30

40

1.1.**.

1

50

09

I

0.0

0.2

50

60

10

20

Source

30

40

50

60

0.2

0.0

OriginalmatrixofTelesat

Sample matrixof Telesat

1.0

AttentionmapofTelesat

1.0

1.0

50

0.8

50

0.8

50

0.8

50

0.8

100

100

100

100

0.6

150

200

200

200

0.4

200

0.4

250

0.2

250

0.2

250

0.2

250

0.2

50

100

Source

150

200

250

0'0

50

100

Source

150

200

250

0.0

50

100

Source

150

250

0.0

50

100

Source

150

200

250

0.0

Original matrixof Starlink

Sample matrix of Starlink

AttentionmapofStarlink

EstimatematrixofStarlink

k.0

L.U

1.0

1.0

200

200

200

200

400

0.8

400

0.8

400

0.8

400

0.8

600

600

0.6

800

800

800

1000

1000

1000

1000

0.4

1200

0.2

1200

0.2

1200

0.2

1200

0.2

1400

1400 -

1400

1400

200

400

600 800 1000 1200 1400

0.0

0

200 400 600 800 1000 1200 1400

0.0

0

400

600 800 1000 1200 1400

0.0

200

400

600800 1000 1200 1400

0.0

Source

Source

Source

Source

(a) Original matrices

(b) Sample matrices

(c)Attention maps

(d) Estimate matrices

tasks requiring the handling of large-scale, sparse, and complex spatio-temporal characteristics. This includes, but is not limited to, social network analysis, environmental monitoring, bioinformatics, and other domains that necessitate the reconstruction and analysis of multidimensional data.

We use the Foursquare [33] and PeMS-Bay [41] tensor dataset to evaluate Satformer's performance on general tensor completion tasks. The Foursquare dataset is a point-of-interest tensor defined by user, location, and timestamp. Utilizing this dataset allows researchers and developers to gain insights into the dynamics of geographic social networks and to develop innovative applications and services

based on these insights. The PeMS-Bay data collect this traffic volume data from 325 loop sensors in the San Francisco bay area, ranging from January to March 2018 with a 5-min time interval.

The test results on two general datasets demonstrate the applicability of the Satformer method to other general tensor completion tasks. These positive results indicate that Satformer can serve as a powerful tool across various fields that involve complex spatio-temporal data, including social network analysis, traffic flow forecasting, environmental monitoring, and others. Naturally, additional adjustments and optimizations may be necessary for different application scenarios to achieve optimal performance.

J Broader Impacts

Satformer is primarily used in emerging large-scale satellite communication networks. Currently, optimizing satellite network traffic engineering [6], [10] or topology engineering [11] relies on real-time collection of global traffic data. However, due to the limitations of satellite networks, real-time collection of comprehensive traffic status information incurs significant costs and overhead, making it nearly impossible to achieve, which further hinders the deployment of these solutions. Satformer addresses this issue by recovering global traffic data from a low sample rate (2%), which significantly reducing costs and overhead, thereby enabling the deployment of these solutions in real satellite networks. Additionally, Satformer helps improve network administrators' awareness of network states, optimizing network operations and maintenance. Beyond satellite communication networks, Satformer has the potential to be extended to other scenarios, such as transportation networks, the Internet of Things (IoT), and image processing. We have validated this in Appendix I. However, Satformer also presents interpretability risks. We recommend that researchers focus on interpretability when using Satformer, enhancing the transparency of application decisions related to traffic scheduling, routing, and congestion control.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We make clearly states about contributions and assumptions in abstract and Introduction.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We have discussed the limitions of this work in section 5 .

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We provide the theoretical results in Appendix B.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We have described the method clearly and fully in section 3. In addition, we provide the ways to construct the datasets in Appendix A.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We have provided the code and data of this paper in supplemental material. Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We have presented all training and test details in section 4 and Appendix C, D, E, F, G, H and I.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We have stated the appropriate information about the statistical significance of the experiments in section 4.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We have provided the sufficient information on computer resources in Appendix C.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research conducted in this paper conform, in every respect, with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We have discussed both potential societal impacts in Appendix J.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: This paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: The existing assets used in this paper are the license and terms of use explicitly mentioned and properly respected.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: The code of this paper is well structured.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
comments: SUMMARY:
This paper proposed a satellite network traffic data estimation method which is called Satfomer.  The proposed method uses the adaptive sparse spatio-temporal attention mechanism to capture nonlinear spatio-temporal relationships. The proposed method is assessed on different satellite network datasets and compared to different existing methods.

STRENGTHS:
1. The writing and organization of the paper are clear and easy to follow.
2. Using adaptive sparse spatio-temporal attention mechanism to capture nonlinear spatio-temporal relationships within satellite network traffic data seems feasible. 
3. The proposed method is assessed on the satellite network traffic datasets of different scales and compared with existing methods.

WEAKNESSES:
1. The overall idea of the proposed is based on the attention mechanism, which is not novel in the field of spatio-temporal forecasting.
2. The motivation of this work is unclear, what is the main difference between the satellite network traffic data and other spatio-temporal data, e.g., traffic flow/speed data?
3. NMAE and NRMSE are used as the metrics to gauge the prediction performance of the methods in this work, what about MSE, MAE, and MAPE? 
4. Some important baselines are missing, e.g., GraphWavenet, AGCRN, and GMAN, etc.

QUESTIONS:
See Weaknesses.

LIMITATIONS:
The novelty of the proposed method is limited.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2024-07-07

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces Satformer, a novel method for accurate and robust traffic data estimation in satellite networks. It addresses the challenges of large-scale and dynamic nature of satellite networks by proposing an adaptive sparse spatio-temporal attention mechanism that focuses on specific local regions of the input tensor, enhancing the model's sensitivity to details and patterns. Satformer incorporates a graph convolutional network for processing non-Euclidean data and a transfer module for disseminating global context information, demonstrating superior performance over mathematical and neural baseline methods, especially for larger networks. Extensive experiments on simulated satellite network datasets, including Iridium, Telesat, and Starlink, show Satformer's significant improvements in reducing errors and maintaining robustness. The paper concludes that Satformer shows promise for deployment in actual systems and potential application in other tensor completion tasks requiring handling of complex spatio-temporal data.\n\nSTRENGTHS:\n1. The paper introduces Satformer, a neural network architecture that incorporates an adaptive sparse spatio-temporal attention mechanism (ASSIT) for estimating traffic data from partial measurements. ASSIT focuses on specific local regions of the input tensor to improve the model's sensitivity to details and patterns, enhancing its capability to capture nonlinear spatio-temporal relationships. The integration of a graph convolutional network (GCN) for processing non-Euclidean data and a transfer module for disseminating global context information are innovative aspects of the model.\n2. The paper provides a detailed explanation of the system model, the proposed Satformer architecture, and the components within it, including the graph embedding module and the transfer module. The experiments are rigorous, conducted on datasets of varying scales, and compared against several baseline methods, demonstrating Satformer's superior performance.\n3. The paper is well-structured, with a clear abstract, introduction, methodology, experimental setup, results, discussion, and conclusion. The figures and tables are used effectively to illustrate the model's architecture and the results of the experiments.\n4. The work addresses a critical need for cost-effective traffic measurement methods in satellite networks, which is essential for network monitoring, routing, and performance diagnosis.\n\nWEAKNESSES:\n1. The paper does not present any theoretical results or proofs to support its claims, focusing instead on empirical results.\n2. The paper primarily focuses on the application of Satformer in satellite networks. While it mentions potential applications in other areas, it does not provide extensive testing or results in those domains, which may limit the perceived generalizability of the model.\n3. The paper may rely on certain assumptions that are not fully discussed or tested under various conditions. For example, the model's performance might vary under different network configurations or traffic patterns that were not part of the evaluation.\n\nQUESTIONS:\n1. What is the unique challenges of traffic data estimation for satellite networks, compared with other network types?\n2. The research motivation is not convincing when there is few relevant literatures.\n\nLIMITATIONS:\n1. Only simulated datasets are used and it is difficult to collect real-world datasets to validate the proposed method.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

comments: SUMMARY:
This paper introduces novel network designed for satellite traffic data estimation. Proposed approach includes several novelties and the experimental results demonstrate that the propsoed methods outperformed several alternative mehtematical and neural baseline methods.

STRENGTHS:
The paper proposes a new method for traffic data estimation in satellite networks. The problem is well described and the proposed method integrates several new compoenents that are railored to solve this spatio-temporal matrix completion problem.
The paper is well written but some points require further elaboration.

WEAKNESSES:
No major weakness but on the write up and method description can be improved:

QUESTIONS:
* The significance of solving this traffic prediction problem should be made clearer in the introduction. 
* You need a figure for the transformer module (Section 3.4). 
*  Please provide references for Eq. (5) and (4) or describe the rational behind them. 
* In Page 5, why do you need the mask matrix? Why the focus on the center region? How is this not impacting performance?
* In Page 5, "the implementation involves incorporating a position-related weight when calculating attention scores". What is the mathematical symbol for this weight matrix in the equations?
* Page 6, C_{t} is not in equation (10) ?
* Add a figure dsecribing the transfer module.

LIMITATIONS:
Limitations were discussed.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2024-07-13

comments: SUMMARY:
This paper presents to make use of the transformer for spatio-temporal data imputation, and the application is for satellite networks.

STRENGTHS:
- The paper is well-written and the model descriptions are clear
- It is interesting to see the spatio-temporal imputation can be used in satellite networks

WEAKNESSES:
-  The spatial characteristics of the data in satellite networks are not presented. Many other spatial-temporal data are also high-dimensional and non-linear. Many existing approaches have been developed for those data, such as SPIN, CDSA, SAITS. The authors only compared with SPIN.
- The authors explained that their model outperforms SPIN because “SPIN’s ability to handle sparsity or irregularly sampled data might be limited”. However, SPIN is designed for sparse data, and hence the intuition on why the proposed method is better is not provided.
- Applying L1 to the attention score matrix cannot make the output to be sparse. 
- There are quite a few approaches to obtain graph embedding (or just a node embedding), why the authors choose this specific GCN for embedding?

QUESTIONS:
Please see weaknesses

LIMITATIONS:
Please see weaknesses
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2024-07-13