## 3 Proposed Algorithm
\begin{tabular}{l l} \hline \hline  & **Algorithm 1** TD3-based task partitioning ratio decision algorithm \\ \hline
1 & Initialize critic networks \(Q_{\theta_{1}},Q_{\theta_{2}}\) and actor network \(\pi_{\phi}\) \\  & with random parameter \(\theta_{1},\ \theta_{2},\ \phi\) \\
3 & Initialize target networks \(\theta_{1}^{\prime}\leftarrow\theta_{1},\ \theta_{2}^{\prime}\leftarrow\theta_{2},\ \phi^{\prime}\leftarrow\phi\) \\
4 & Initialize replay buffer \(\mathcal{B}\) \\
5 & Initialize HSR environment \\
6 & **for \(c=1\) to \(E\)do** \\
7 & **for \(t=1\) to \(T\)do** \\
8 & **for \(n=1\) to \(N\)do** \\
9 & Calculate MECS load probability \(P_{load}\) \\
10 & Select a set of serving MECSs \((\mathcal{K})\) using \(P_{load}\) \\
11 & Select action with exploration noise \(a\leftarrow\pi_{\phi}(s)+\epsilon,\ \epsilon\ \sim\mathcal{N}(0,\sigma)\), \\  & and observe reward \(r\) and new state \(s^{\prime}\) \\
12 & Store transition tuple \((s,\ a,\ r,\ s^{\prime})\)in \(\mathcal{B}\) \\
13 & Sample mini \(-\) batch of \(B\) transitions \((s,\ a,\ r,\ s^{\prime})\) from \(\mathcal{B}\) \\
14 & \(\widetilde{a}\leftarrow\pi_{\phi^{\prime}}(s^{\prime})+\epsilon,\epsilon\ \sim clip( \mathcal{N}(0,\sigma),-c,c)\) \\
15 & \(y\gets r+\gamma\text{min}_{i=1,2}\mathcal{Q}_{\theta_{i}^{\prime}}(s^{ \prime},\widetilde{a})\) \\
16 & Update critics \(\theta_{i}\gets argmin_{\theta_{i}}B^{-1}\sum\bigl{(}y-Q_{\theta_{i}}(s,a )\bigr{)}^{2}\) \\
17 & **if \(t\) mod \(d\) then** \\
18 & Update \(\phi\) by deterministic policy gradient: \\
19 & \(\
abla_{\phi}J(\phi)=B^{-1}\sum\
abla_{a}Q_{\theta_{1}}(s,a)\left|{}_{a=\pi_{ \phi}(s)}\
abla_{\phi}\pi_{\phi}(s)\right.\) \\
20 & Update target network parameters: \\
21 & \(\theta_{i-1,2}^{\prime}\gets\tau\theta_{i}+(1-\tau)\theta_{i}^{\prime}\) \\
22 & \(\phi^{\prime}\leftarrow\tau\phi+(1-\tau)\phi^{\prime}\) \\
23 & **endif** \\
24 & **end for** \\
25 & **end for** \\
26 & **end for** \\ \hline \hline \end{tabular}   
First, regarding the Actor network activation function of lines 11 and 14 of Algorithm 1, the existing TD3 limited the network output to the Tanh (Hyperbolic Tangent) function, so the action range was set to \([-1,1]\). However, the problem of this paper is a ratio determination problem, so the action range should be 0 to 1 and the total sum of the actions should be 1. Therefore, the Dirichlet distribution was used instead of the Tanh activation function. First, the Dirichlet distribution is a probability distribution that generates a multi-dimensional probability vector. The result sampled from this distribution is a probability vector, and each element has a value between 0 and 1, and the sum of the vectors is always 1. The input parameter is used in the Dirichlet distribution to sample the output value, which is the action. Since this sampling is random, various output values can be obtained even with the same input parameter. This change contributes to solving the problem in a differentiated way from the basic structure of TD3 by reflecting uncertainty and dynamic task requirements in a MECS-based HSR environment. Deep reinforcement learning (DRL) is a methodology in which an agent learns an optimal action policy while interacting with the environment, and it is based on the Markov Decision Process (MDP). The MDP generally refers to the process of probabilistically controlling states and actions at discrete time steps, which provides the mathematical basis for reinforcement learning. MDP formally defines the dynamic characteristics of the environment and the decision-making process of the agent. The agent can select the optimal action using it. The main components of MDP are \(S\), which represents a state set, and \(A\), which represents an action set. \(r\) represents a reward function, and \(P\) represents the state transition probability which is unknown in our proposed problem. It is usually expressed in the form of a tuple such as (\(S\), \(A\), \(P\), \(r\)).