## 3 Problem Description
### Edge Workflow Computing Platform Architecture  
The edge workflow computing platform is primarily used to address collaborative work scenarios involving multiple services in edge environments. In an industrial environment, it is common practice to \"embed\" the execution flow of hardware or software into their respective code. This approach, on the one hand, leads to high coupling between individual hardware or software, and on the other hand, results in significant idle time for hardware or software. On the Edge Workflow Computing Platform, a variety of services are running, such as hardware control services, data storage services, and AI services. Compared to \"embedding\" process flows into these services, the interaction between these services is defined through workflows. In this scenario, the same service can simultaneously serve different workflows, significantly increasing service utilization. However, at the same time, the system's complexity also increases, requiring the design of additional modules to control the execution of different workflows to ensure that tasks within the workflows are correctly processed by the corresponding service. For this purpose, a workflow design and execution platform has been developed to address this issue. The structure of the entire system is illustrated in Figure 1. Users design and submit workflows through the user service. The Management Service is responsible for providing real-time feedback on execution information to users and receiving workflows from users to be handed over to the Orchestrator. The Orchestrator is the designated location for running the proposed method and is responsible for deciding which tasks should be added \begin{table}
\begin{tabular}{p{113.8 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline
**Work** & **Problem** & **Method** & **Objective** \\ \hline \multirow{3}{*}{[PERSON] et al. [27]} & \multirow{3}{*}{TSP} & Neural networks generate result (not DRL); Pointer network (RNN-based); & Minimize total tour length \\ \hline \multirow{3}{*}{[PERSON] et al. [28]} & \multirow{3}{*}{TSP} & Neural networks generate result; Pointer network (RNN-based); Actor-Critic; & \\ \hline \multirow{3}{*}{[PERSON] et al. [30]} & \multirow{3}{*}{VRP} & Neural networks generate result; Pointer network (CNN and RNN); Actor-Critic & \\ \hline \multirow{3}{*}{[PERSON] et al. [31]} & \multirow{3}{*}{TSP with variations} & Neural networks generate result; Pointer network (GNN and LSTM); Policy gradient & \\ \hline \multirow{3}{*}{[PERSON] et al. [32]} & \multirow{3}{*}{TSP, VRP and routing problem} & Neural networks generate result; Transformer; Policy gradient & \\ \hline [PERSON] et al. [33] & VRP & Neural networks search result; Policy gradient & Minimize total tour length \\ \hline [PERSON] et al. [34] & TSP and CVRP & Neural networks search result; Policy gradient & \\ \hline \end{tabular}
\end{table} 
Table 2: Related work on using deep reinforcement learning to address other problems. to the task queue. The task queue holds all scheduled and pending tasks for execution. The Edge Server runs the corresponding services that handle sub-tasks. The Resource Manager is responsible for monitoring the resource usage in edge servers and providing feedback to the Orchestrator. Services can concurrently process sub-tasks when there are sufficient computational resources available. However, adding too many sub-tasks to the task queue results in a large number of tasks that services have to handle simultaneously. This can lead to resource competition among services, causing delayed completion times for multiple sub-tasks. Therefore, the scheduling objective is to balance the execution of sub-tasks across different workflows, minimizing overall response time while avoiding resource competition, and optimizing resource utilization. The Orchestrator parses all the workflows and extracts sub-tasks as an input sequence, denoted as \(V=\{v_{1},v_{2},\ldots,v_{|V|}\}\), which is fed into the neural network.