# Deep Reinforcement Learning for the Agile Earth Observation Satellite Scheduling Problem
## 4 Experiments
### Experimental Setting for AEOSSP
#### 4.2.2. GDNN Structure Optimization  
A network with a structure that is too complex may affect the calculation speed and convergence, and a network with a structure that is too simple may be unable to characterize the problem well. To improve the efficiency of the GDNN, we analyze the GDNN structure.  
Due to the limited availability of servers for large-scale training, we only considered three parameters: the embedding layer dimension \(F_{3}\), the number of hidden layers \(n_{hid}\), and the number of embedding layers \(n_{em}\).  
For the embedding layer dimension \(F_{3}\) and the number of hidden layers \(n_{hid}\), experiments are designed to test networks of \(128\times 4\), \(64\times 3\) and \(32\times 2\) (\(F_{3}\times n_{hid}\)) corresponding to high, medium, and low dimensions. We train networks of different dimensions using instances with 100 tasks and training episodes of \(10,000\) and test them with 50 instances with tasks scaled 40, 60, 80, and 100. Results are shown in Table 6. In addition, the number of embedding layers is tested with 3, 4, 5 and 6. The networks are trained by instances with  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Method** & **Parameter** \\ \hline \multirow{6}{*}{GRILS} & Maximum iteration number with no improvement: 300 \\  & StartGreed = 1, GreedRange = 0.2 \\  & GreedDecrease = 0.02 \\  & Place of start removal \(S_{d}=1\) \\  & Number of observations removed each time \(R_{d}=1\) \\ \hline \multirow{6}{*}{SDE} & Population size \(Pop=50\) \\  & Iteration number \(N_{\text{iter}}=200\) \\  & Initial crossover probability \(Cr=0.5\) \\  & Initial mutation probability \(Mp=0.2\) \\  & Encoding method: real-valued encoding \\  & Offspring selection: championship \\  & Elite solution retention ratio: 0.1 \\ \hline \multirow{6}{*}{SGA} & Population size \(Pop=50\) \\  & Iteration number \(N_{\text{iter}}=200\) \\  & Initial crossover probability \(Cr=0.8\) \\ \cline{1-1}  & Initial mutation probability \(Mp=0.2\) \\ \cline{1-1}  & Encoding method: integer encoding \\ \cline{1-1}  & Offspring selection: roulette \\ \cline{1-1}  & Elite solution retention ratio: 0.1 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Parameters setting of high-quality competitors.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Parameter** & **Description** & **Value** \\ \hline \multirow{2}{*}{\(c\)} & Trim function parameter & 0.1 \\  & Mean square error factor & 0.5 \\  & Entropy factor & 0.001 \\  & Batch size & 32 \\  & Parameter update step & 1024 \\  & Parameter update optimization times & 3 \\  & Training episode generation & 50,000 \\  & Learning rate & 0.0005 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Main parameters of the GDNN training.  
a task scale of 40 and training episodes of 10,000. We test them with 50 instances with a task scale of 40. The test result is shown in Table 7.  
From Table 6, at the current cost of training, the low-dimensional network is the fastest but least profitable. The solution time of the high-dimensional network is longer, but the improvement in solution quality is small compared to the medium-dimensional network. For the number of embedding layers, results in Table 7 show that the network with four embedding layers can obtain the best ASP. The networks with five and six embedding layers have longer scheduling times but no improvement in scheduling profits. Therefore, the network parameters are finally set to \(F_{3}=64\), \(n_{hid}=3\) and \(n_{em}=4\) in this study, and thus, the GDNN-64x3 network with four embedding layers is considered the best network for the AEOSSP.