# Research on the Dynamic Characteristics Analysis and Power Control Method of Heat Pipe Reactors
## 1 Introduction
The research on the start-up condition found that the isothermal characteristics of the heat pipe caused the reverse heat transfer between the heat pipe and the core structure, which could flatten the temperature distribution in the reactor core. For the structure, design, and safety analysis of HPRs, the high-fidelity simulation method of the neutronics-thermal elasticity multi-physics coupling model mentioned above is necessary. However, its cost is the complexity of the calculation process and a high computational workload. When studying control methods for the HPR, the model not only needs to accurately reflect changes in parameters during the dynamic process but also requires a small amount of calculation to adapt to the rapid simulation verification of the control method. In the aspect of reactor control, ensuring energy balance is one of the most important control objectives of nuclear reactor installation. The proportion-integration--differentiation (PID) algorithm has become the most widely used controller due to its simple principle and easy implementation. However, before the PID controller is put into use, it must go through a tedious parameter-setting process, and once the operating state of the control object changes significantly, a PID controller with the same parameters may not achieve the same control effect as before. Therefore, optimization algorithms such as the genetic algorithm and particle swarm optimization algorithm [23,24] and intelligent algorithms such as fuzzy theory [25] are often used to improve the traditional PID control algorithm to achieve better control effects. In addition, control methods based on deep reinforcement learning (DRL) are also applied to reactor control. [PERSON] et al. used the deep deterministic policy gradient (DDPG) strategy to train the RL controller for the power control of a boiling water reactor [26]. In the simulation of continuously introducing disturbances, the reactor power has obvious oscillation under the control of the method, but after using the RL method, the reactor system can overcome various disturbances and ensure that the power regulation process is rapid and stable. In fact, the control methods based on reinforcement learning regard the reactor system as a \"black box\", and the agent is trained in the virtual environment and learns how to obtain higher rewards driven by specific learning strategies such as deep Q-network (DQN), DDPG, and TD3 [27]. In this work, in response to the requirements of speed and accuracy of the simulation model for research on HPR power control methods, a lightweight dynamic model of the HPR from core to heat pipe exchanger is established. Taking the MegaPower HPR as example, the arrangement of the components in the core is summarized, and the lumped parameter method is used to make the single heat-generating component of the core equivalent to a cylindrical shape. Meanwhile, the simplified thermal resistance network method is used to analyze the heat transfer process of the heat pipe, and the core parameters of the heat pipe heat exchanger are designed. Secondly, the dynamic characteristics of HPRs with reactivity disturbance and flow disturbance are analyzed separately. Finally, the TD3 algorithm, with excellent performance in continuous control, is selected as the core algorithm of HPR power control, and the application of the reinforcement learning control method in HPR power regulation is explored.