# Toward Low Time Fragmentation of Equipment: A Double-DQN Based TT&C Task Planning Approach
## 5 Low Time Fragmentation-Oriented TT&C Task-Planning Algorithm
### Vertex Attribute Extraction and Parametric Design of \(Q\) Function  
Define the vertex attribute \(x_{v}=(d_{v},\beta_{v},\alpha_{v},\omega_{v})\) of the task-planning conflict graph, where \(d_{v}\) is the degree of vertex \(v\) in the current task-planning conflict graph \(CG_{\overline{TS}}\), and \(\beta_{v}\) is the number of vertices in \(CG_{\overline{TS}}\) that correspond to the same task as vertex \(v\) in \(CG_{\overline{TS}}\), namely,  
\[\beta_{v}=\left|\left\{v^{\alpha}_{i}|v^{\alpha}_{i}\in V_{\overline{TS}} \right\}\right| \tag{3}\]  
Figure 5: Directed graph of visible windows. where the operator \(|\cdot|\) is the number of elements in the set. \(\varpi_{v}\) represents the newly added time fragments after the feasible scheme represented by vertex \(v\) is added to the current local solution. \(\varpi_{v}\) is the revenue obtained after the task feasible scheme corresponding to vertex \(v\) is completed. In order to simplify the symbolic expression, \(v\) is used in the following text to express vertex \(v_{i}^{x}\) in the task-planning conflict graph. Graph structure embedding is carried out recursively. In each step of the recursive process, information is embedded for each vertex of the graph, so that it contains a larger range of graph structure and vertex attribute information. By embedding different degrees of information on each vertex, the computation can be effectively reduced and the algorithm efficiency can be improved. In particular, in the recursive process, a q-dimensional vector \(\mu_{v}^{(k)}\) is used to represent the embedding vector of the vertex \(v\) obtained after the \(k\)-th iteration. The initial value of the embedding vector of vertex \(v\) is set to 0. \[\mu_{v}^{(k+1)}\leftarrow\mathit{relu}(\theta_{1}x_{v}+\theta_{2}\sum_{u\in N _{\overline{TS}}(v)}\mu_{u}^{(k)}) \tag{4}\]  
It can be seen from Equation (4) that the updating process of graph structure embedding is based on the adjacency relationship of vertices in the graph, where \(N_{\overline{TS}}(v)\) is the vertices adjacent to the vertex \(v\) in the \(CG_{\overline{TS}}\) graph. \(\theta_{1}\) and \(\theta_{2}\) are the model parameters, and \(\mathit{relu}()\) is the rectified linear unit. Based on Equation (4), one can see that the embedding update process is carried out based on the graph topology. A new round of embedding sweeping across the vertexes will start only after the embedding update for all nodes from the previous round has finished. The above process goes through a total of K iterations, and each vertex obtains an embedding vector \(\mu_{v}^{(K)}\). After the \(K\)-step graph embedding information is completed, the embedding vector is defined as the evaluation function \(\mathrm{Q}(IS,v;\Theta)\). \(\sum_{u\in IS}\mu_{u}\) and \(\mu_{v}\) are used to represent the graph \(CG\) and vertex \(v\), respectively, and \(\mathrm{Q}(IS,v;\Theta)\) is defined as follows:  
\[\mathrm{Q}(IS,v;\Theta)=\theta_{3}^{\overline{T}}\mathit{relu}[\theta_{4}\sum _{u\in IS}\mu_{u},\theta_{5}\mu_{v}] \tag{5}\]  
where \(\theta_{3}^{\overline{T}}\), \(\theta_{4}\), and \(\theta_{5}\) are the parameters to be trained for the model, and \([..]\) is the concatenation operator. The parameters \(\mu_{u}\) and \(\mu_{v}\) are calculated by Equation (4). They are affected by the parameters \(\theta_{1}\) and \(\theta_{2}\) in the embedding process of the graph structure. So the evaluation function \(\mathrm{Q}(IS,v;\Theta)\) is determined by five parameters \(\{\theta_{i}\}_{i=1}^{5}\). Algorithm 1 shows the specific calculation steps of the evaluation function \(\mathrm{Q}(IS,v;\Theta)\), where \(\sum_{u\in IS}\mu_{u}^{k}\) needs to be recalculated according to the task-planning the subgraph of task planning conflict graph at that time in each iteration. The parameters \(\{\theta_{i}\}_{i=1}^{5}\) are obtained through deep Q-learning training.