## 3 Proposed Algorithm
In this paper, the MDP that determines the optimal task partitioning ratio that maximizes the throughput of tasks and minimizes the delay time in an HSR environment with an MECS. MDP modeling is critical for performance optimization in dynamic environments. In the proposed environment, the number of HSTs and the density of users aboard the HST vary over time, resulting in fluctuations in the amount of tasks generated. Consequently, the load on the MECS also changes dynamically. The algorithm addresses these changes by adjusting the task partitioning ratio in real time, which directly impacts task throughput and latency. Task latency is influenced by waiting time, which is, in turn, affected by the MEC server load. When MECSs become overloaded, tasks may exceed the latency constraints due to prolonged waiting times, making it difficult to ensure QoS. To address this, the proposed algorithm determines the task partitioning ratio based on the dynamically changing MEC load and the size of the generated tasks. The optimal task partitioning ratio calculated by the algorithm allocates fewer tasks to heavily loaded MECSs and more tasks to lightly loaded MECSs. This approach improves overall system efficiency by maximizing task throughput and maintaining QoS through reduced waiting times. Additionally, the algorithm minimizes delay time by distributing tasks effectively across MEC servers, achieving load balancing and optimizing resource utilization. Based on this proposed environment, we define MDP as follows:  
* Agent: All HSRs are agents. \(N=\{1,\ldots,\,n\}\), which observes the state of the HSR environment with MECSs, chooses actions according to the policy, receives rewards, and updates the environment. By repeating this, they learn the optimal action strategy to maximize the cumulative reward. * State: The state \(s_{t}\) consists of the information (\(W_{n}(t)\)) of the task generated in HST \(n\) and the information (\(M(t)\)) of the currently considered serving MECSs \(s(t)=\{W_{n}(t),M(t)|\forall n\in N\}\). \(W_{n}(t)\) consists of the data size, workload, and delay constraint of the task, and \(M(t)\) represents the load states of \(k\) serving MECSs considered by HST \(n\). This provides environmental information for task partitioning ratio decisions. * Action: \(a_{t}\) defines as the task partitioning ratio to be processed among the \(K\) serving MECSs considered by HST \(a_{t}=\big{\{}x_{1}^{n}(t),\ldots,x_{k}^{n}(t),\ldots,x_{K}^{n}(t)\big{\}}\). * Reward: In Equation (15), \(r(s_{t},a_{t})\) is defined based on the task throughput \(T(t)\) and the delay \(L(t)\). The reward function is designed to minimize delay \(L(t)\) while maximizing task throughput \(T(t)\). Here, task throughput refers to the resources of a task that succeeds within the delay constraint, while delay represents the time consumed when processing a task. These two objectives are conflicting, as one seeks to minimize delay and the other seeks to maximize the throughput. To address this, the reward function penalizes task execution delay (\(L(t)\)) by applying it as a penalty term, while task throughput (\(T(t)\)), whose goal is to maximize, is applied as a reward term. In addition, the weights in the reward function that constitutes this multi-objective sum to 1 and represent values between 0 and 1. This indicates importance. If greater emphasis on the task throughput is desired, the weight assigned to the task throughput can be increased accordingly:  
\[r(s_{t},a_{t})= \begin{cases}\hskip 28.452756 pt-L(t)\hskip 28.452756 pt,&L_{max}<L(t)\\ \omega\cdot T(t)-(1-\omega)\cdot L(t),&L_{max}\geq L(t)\end{cases}  \tag{15}\]  
Figure 3 illustrates a simplified flowchart for determining the task partitioning ratio in the proposed system. The flowchart depicted in Figure 3 represents the TD3 learning process at time slot \(t\). Based on the MDP defined above, it demonstrates the operation of the TD3 algorithm within the HSR environment. In this setup, multiple agents represent HSTs, which operate independently without sharing a common environment.