# Multi-Satellite Task Parallelism via Priority-Aware Decomposition and Dynamic Resource Mapping
## 4 Methodology
### Resource Mapping and Allocation Strategy Based on Game Theory and Reinforcement Learning  
To tackle the heterogeneity and dynamic nature of satellite resources in multi-satellite cooperative computing, a resource mapping and allocation strategy based on game theory and reinforcement learning is proposed. We model the resource allocation problem as a resource competition game among satellite nodes. The optimal allocation is then derived by solving the Nash equilibrium of the game. Furthermore, a deep reinforcement learning algorithm is integrated to dynamically adjust resource allocation in response to the changing environment, ensuring adaptive and efficient resource mapping. In a multi-satellite cooperative system, each satellite node competes for limited computational, communication, and storage resources to maximize its utility. This competition can be naturally modeled as a non-cooperative game where each satellite \(s_{i}\) selects a resource allocation strategy \(\mathbf{r}_{i}=[r_{i}^{b},r_{i}^{c},r_{i}^{p}]\). The utility function for each satellite is defined as  
\[U_{i}(\mathbf{r}_{i},\mathbf{r}_{-i})=\frac{f_{i}(\mathbf{r}_{i})}{g(\mathbf{ r}_{i},\mathbf{r}_{-i})}, \tag{28}\]  
where \(f_{i}(\mathbf{r}_{i})\) represents the benefit derived from the allocated resources, and \(g(\mathbf{r}_{i},\mathbf{r}_{-i})\) represents the cost influenced by other satellites' resource strategies \(\mathbf{r}_{-i}\). By modeling the allocation as a game, the Nash equilibrium condition ensures that no satellite can improve its utility by unilaterally changing its resource strategy, providing a theoretically sound framework for optimal resource distribution. To find the optimal resource allocation, we define the Nash equilibrium of this game. A strategy profile \((\mathbf{r}_{1}^{*},\mathbf{r}_{2}^{*},\ldots,\mathbf{r}_{n}^{*})\) is a Nash equilibrium if  
\[U_{i}(\mathbf{r}_{i}^{*},\mathbf{r}_{-i}^{*})\geq U_{i}(\mathbf{r}_{i}, \mathbf{r}_{-i}^{*}),\quad\forall i,\forall\mathbf{r}_{i} \tag{29}\]  
We use the best response dynamics to iteratively adjust each satellite's resource strategy toward equilibrium. However, due to the dynamic nature of the environment, solving for Nash equilibrium in real time is computationally expensive. Considering the dynamic and complex nature of resource competition, we introduce DRL to adaptively adjust resource allocation. Each satellite node is modeled as an agent in a Markov decision process (MDP) where the state \(s_{t}\) captures the current resource usage and task demand, the action \(a_{t}\) represents the selected resource allocation strategy, and the reward \(r_{t}\) is derived from the utility function \(U_{i}\). The objective of the agent is to learn an optimal policy \(\pi^{*}(s_{t})\) that maximizes its expected cumulative reward. The learning process employs DQN to approximate the optimal action-value function:  
\[Q^{*}(s_{t},a_{t})=\max_{\pi}\mathbb{E}\left[\sum_{k=0}^{\infty}\gamma^{k}r_{t +k}\mid s_{t},a_{t}\right]. \tag{30}\]  
The proposed resource allocation algorithm combines game theory and DRL to achieve an optimal and adaptive resource distribution. Algorithm 2 outlines the iterative learning and allocation process. This algorithm begins by initializing the Q-network and the replay buffer (Lines 2-3). During each iteration, every satellite first observes its current state (Line 6) and selects a resource allocation action according to the \(e\)-greedy policy derived from the Q-network (Line 7). The chosen action is then executed, yielding a reward and a subsequent state (Line 8), and the transition tuple is stored in the replay buffer (Line 9). A minibatch is subsequently sampled from the replay buffer (Line 11), and for each transition in the minibatch, a target Q-value is computed using the Bellman update (Line 13). The Q-network parameters are then optimized by minimizing the loss function (Line 14). Finally,for each satellite, the optimal resource allocation is obtained by selecting the action that maximizes the Q-network's output (Line 16).