## 4 Industrial Federated Deep Deterministic Policy Gradient  
In this section, we mainly elaborate and introduce the algorithm we invented to solve the optimization problem (16). In the proposed multiedge-terminal collaborative computing, multiple variables need to be considered, such as offloading decisions, communication resource allocation, and computing resource allocation. The proposed optimization is complex and nonconvex, and traditional optimization algorithms have difficulty in directly obtaining analytical solutions. One effective approach is to use DRL. However, traditional DRL is prone to difficulties in convergence or local convergence problems when dealing with complex nonconvex problems. In addition, in the environment of multiedge-terminal Industrial Internet intelligent factories, terminal devices are different. When using the traditional DDPG algorithm to optimize a single terminal device and sum the results, it may lead to suboptimal energy optimization due to differences in convergence accuracy and speed among multiple terminal devices. FL is advantageous in reducing convergence problems, improving algorithm performance, and ensuring data privacy, making it more suitable for the industrial internet scenario. In response to the above problems, we introduce the Industrial Federated Deep Deterministic Policy Gradient (IF-DDPG) algorithm, which is designed for application in the environment of Industrial Internet intelligent factories. This algorithm is built upon the foundations of both DDPG and FL. The IF-DDPG algorithm combined with FL involves multiple DDPG algorithms, and each DDPG algorithm corresponds to a terminal device. The IF-DDPG algorithm enhances the accuracy of both individual DDPG algorithms and the overall accuracy when multiple DDPG algorithms are combined. This is achieved by exchanging model parameters among the DDPG algorithms, allowing for simultaneous optimization of energy consumption for each terminal device and the entire system. Furthermore, factories have high requirements for data privacy. Since FL does not require offloading local data, data privacy can also be protected by the DDPG algorithm combined with FL [47]. The IF-DDPG algorithm consists of the DDPG and the FL. The DDPG algorithm is run on each edge server, and then federated averaring is applied the neural network parameters from the DDPG algorithm of multiple edge servers to implement the IF-DDPG algorithm. The agent is the main entity in IF-DDPG. The environment refers to the entity that the agent interacts with during the learning process. The mathematical foundation and modeling tool for the algorithm is the MDP [48]. The MDP usually consists of state \(s\), action \(a\), and reward \(r\). At each epoch, the environment has a state \(s\). It is the summary of the current environment. The state is the basis for algorithmic decisions. Action \(a\) is the decision made by the agent due to the present state. The reward \(r\) refers to a value that the environment returns to the agent after the agent performs an action. It is usually assumed that the reward is a function of the current state \(s\), current action \(a\), and next epoch state \(s^{\prime}\). Denote the reward function as \(r(s,a,s^{\prime})\). The algorithm provides specific definitions for states, actions, and rewards, as outlined below:  
1. State Space: The state space is denoted by a total of \(M\) mobile smart Industrial Internet of Things sensors/terminal devices and \(N\) small base stations equipped with edge servers.