## 2 Methods
### Experiment 1  
The parameters of the HRL algorithm are designed as follows:  
The coordinates of agents and tasks are designed to arrange in the task area randomly. The learning rate is 0.4, and the attenuation value is 0.6 in the HRL algorithm. It is worth noting that the coordinates of each agent change with their movement. In Figure 5, the numbers of optimization iterations for the HRL-based first and second layers are shown.  
To demonstrate the proposed approach's effectiveness, the RL algorithm is chosen as a representative of the traditional human-robot function allocation algorithm, and the same computation is performed with the same number of agents to compare the task assignment efficiency of the two algorithms.  
The reward function of the RL algorithm is as follows:  
\[R_{m}(s,a)=\begin{bmatrix}\alpha&\beta&\theta\end{bmatrix}\begin{bmatrix}TF(a, t_{m})\\ EF(a,t_{m})\\ DF(a,t_{m})\end{bmatrix} \tag{22}\]  
In the Q-learning algorithm, the state representing the astronaut and the robot is denoted by \(s\), while the agent representing the human-robot collaborative combination is denoted by \(a\). There are three coefficients defined as \(\alpha\), \(\beta\), and \(\theta\).  
\[\left\{\begin{aligned} S^{1}&=[H,R]\\ A&=h_{m}^{i}\cup r_{n}^{j}\end{aligned},\;i\epsilon[0,I],je[0,J ]\right. \tag{23}\]  
In the above equation, the variables \(I\) and \(J\) denote the total count of astronauts and robots, respectively. The variable \(h_{m}^{i}\) represents the \(m\)th astronaut belonging to the \(i\)th type, whereas \(r_{n}^{j}\) denotes the \(n\)th lunar robot belonging to the \(j\)th type. Here, is the formula for updating:  
\[Q(s,a)=Q(s,a)+\delta Q(s,a) \tag{24}\]  
\[Q(s,a)=R(s,a)+\theta\underset{A}{\max}Q\Big{(}(s)^{\prime},a\Big{)}-Q(s,a) \tag{25}\]  
In this scenario, \(s\) and \((s)^{\prime}\) denote the current state and the following state, respectively. \(a\) represents the agent involved, while \(Q(s,a)\) is referred to as the \(Q\) value. Meanwhile, \(\delta\) and \(\theta\) stand for the learning rate and attenuation value of RL, respectively.  
The DRL and HDRL algorithms are constructed by combining RL and HRL algorithms with a three-layer neural network with the number of neurons in the three layers being 10, 15 and 10, respectively.  
Table 3 shows the setting of the agents and tasks in experiment 1. The experiment involves a total of 24 agents, comprising of four astronauts inside the capsule (\(H_{1}^{in}\)-\(H_{4}^{in}\)), four astronauts outside the capsule (\(H_{1}^{out}\)-\(H_{4}^{out}\)), four small exploration robots (\(Re_{1}\)-\(Re_{4}\)), four large rescue robots (\(Rr_{1}\)-\(Rr_{4}\)), four material transport robots (\(Rt_{1}\)-\(Rt_{4}\)), and four lunar rovers (\(Rl_{1}\)-\(Rl_{4}\)). There are four types of tasks set up in the experiment, which include ground exploration (point E), robot rescue (point R), material transport (point T), and lunar surface samples (point S).  
The proposed HRL and RL, DRL, and HDRL algorithms are used to compute the optimal human-robot function allocation scheme for the number of agents and tasks set in Table 3, respectively, and their task allocation efficiency is compared by the number of iterations of the algorithms.