# Research on the Dynamic Characteristics Analysis and Power Control Method of Heat Pipe Reactors
## 4 Research on Power Control Method of HPR
### TD3 Algorithm
For example, under reactivity disturbance, the RL controller makes the nuclear power peak less than 0.3%, while the nuclear power peak reaches 2% with the PID controller. During the linear change in load power, the actual nuclear power is basically consistent with the load power under the regulation of the RL controller. In the subsequent power reduction and power upturn, the overshoot from PID controlling is higher than 17% and the settle time is greater than 70 s, while the overshoot from the RL controlling is less than 7% and the settle time is less than 6 s, which indicates that the RL controller can effectively improve the dynamic performance of nuclear power. At the same time, the highest steady-state error of nuclear power under RL control is only 0.57%, which shows good steady-state performance. To verify the control stability of the RL controller in the full power range, the control effect of the RL controller and the PID controller is compared under significant load

Figure 14: The response curves of the relative power (**a**) and the control drums’ speed (**b**) under compound condition. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Characteristics** & **PID Controller** & **RL Controller** \\ \hline Max rise time & 10.25 s & 2.16 s \\ Max overshoot & 17.27\% & 6.07\% \\ Max settle time & 73.96 s & 5.57 s \\ Max steady-state error & 0.04\% & 0.57\% \\ \hline \hline \end{tabular}
\end{table} 
Table 5: A comparison of control performance between RL control and PID control at compound condition. power fluctuations. In the simulation, the load changes are 100%FP-70%FP-40%FP-20%FP-40%FP-70%FP-70%FP-100%FP, and the PID parameters are the same as above. The response of nuclear power is shown in Figure 15. Meanwhile, the response characteristics of nuclear power at each step are calculated, as shown in Table 6. Under different power conditions of the HPR, the operation characteristics of the HPR change. Therefore, the control performance of a PID controller tuned under the high-power condition gradually deteriorates during the power reduction process. During the power decrease, the overshoot and settle time of the nuclear power gradually become larger, and even a slight oscillation occurs at the step from 40%FP to 20%FP. These show that PID control has a small application range in the power control of HPRs, and only one setting of PID parameters cannot successfully complete the power control task within the full power range. For the RL controller, the dynamic performance is better in the whole power adjustment task, the overshoot is less than 7.38%, and the settle time is less than 51.70 s. Although the control performance of the RL controller is slightly reduced under the condition of low power, the control performance of the RL controller is obviously better than that of the PID control. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Load Power Step** & **Characteristics** & **PID** & **RL** \\ \hline  & \(OS\) & 14.56\% & 2.74\% \\
100\%FP–70\%FP & \(ST\) & 77.27 s & 14.99 s \\  & \(SSE\) & 0.01\% & 0.67\% \\  & \(OS\) & 17.55\% & 3.73\% \\
70\%FP–40\%FP & \(ST\) & 86.26 s & 31.05 s \\  & \(SSE\) & 0.55\% & 1.19\% \\  & \(OS\) & 23.31\% & 4.51\% \\
40\%FP–20\%FP & \(ST\) & 192.69 s & 51.70 s \\  & \(SSE\) & 0.52\% & 2.37\% \\  & \(OS\) & 33.84\% & 7.38\% \\
20\%FP–40\%FP & \(ST\) & 80.95 s & 18.33 s \\  & \(SSE\) & 0.07\% & 1.20\% \\  & \(OS\) & 23.04\% & 3.71\% \\
40\%FP–70\%FP & \(ST\) & 76.93 s & 11.08 s \\  & \(SSE\) & 0.08\% & 0.69\% \\  & \(OS\) & 17.263 & 2.98\% \\
70\%FP–100\%FP & \(ST\) & 65.12 s & 6.25 s \\  & \(SSE\) & 0.09\% & 0.48\% \\ \hline \hline \end{tabular}
\end{table} 
Table 6: The control performance of RL control and PID control under significant load power fluctuations. Figure 15: The response of nuclear power under significant load power fluctuations.