## 4 Simulation Results
### Performance Evaluation
#### 4.3.1 A Comparison of the Experiments Performed Under Active Power Change  
The results of four controllers in active power and frequency change are drawn as follows. The results of the four controllers' active power and frequency changes are illustrated in Figures 14 and 15. Figure 14 shows that the TD3 model had the fastest response speed and the smallest turbulence compared to the other two MMC-VSG methods. The fixed VSG model reached a peak of around 360 kW, while the DDPG method reached 310 kW, and the TD3 method reached 290 kW, with the transient time decreasing in the same sequence. All three VSG control methods had a steady active power of around 280 kW, in contrast with the vector MMC method, which showed a relatively shorter transient state with output power remaining after oscillation. In the experiment, voltage changes were observed in response to a sudden increase in the grid-side load. The excitation controller in the MMC-VSG system adjusted its power output to maintain the grid voltage and frequency stability, resulting in optimal stability and response times. However, when the grid-side load increased rapidly, the system's output power remained unchanged, and the vector control method exhibited only a short transient process. In this scenario, the MMC system was unable to independently adjust its power output based on the change in grid-side load, leading to a lack of fast power response due to its lack of inertia and damping characteristics as well as its inability to participate in power-grid regulation. The performance of the four control methods was further analyzed by comparing their frequency change under grid-connected operating conditions as shown in Figure 15. All three VSG control methods showed similar patterns, ending at around 49.85 Hz, while the vector MMC ended at 49.79 Hz. The vector MMC showed a larger frequency deviation due to the PLL's poor ability to track the grid frequency after a drop. During the dynamic response period, the TD3 model showed the smallest frequency fluctuation and dynamic response time among the three VSG methods. Hence, it can be concluded that the TD3 technique has stronger resistance to disturbance, better dynamic performance, and a quicker response time compared to the DDPG-based algorithm and fixed VSG control. The stability of the training environment was improved, and the overestimation error was reduced through the use of a truncated end naturalized Q-network critic, which minimized the overestimation bias of a single agent's critic network in the population. The algorithm's performance was also improved, indicating that the recommended method has superior demonstration and robustness against uncertainty in conditions. The simulation results for the output current and voltage are shown in Figure 16. It can be observed that a sudden increase of 200 kW in the grid active power causes the MMC system's output voltage to decrease significantly, with a decrease in amplitude to around 230 V. This is due to the use of a fixed number control with the vector control, causing the current to increase significantly from 70 to around 90 A in order to maintain constant active power. Both the current and voltage have a very short transient time. The other three sub-figures demonstrate that the VSG control can become involved in the regulation of both the voltage and frequency of the AC grid. The voltage and current vary in response to changes in power, with TD3-VSG having the smallest distortion of about 140 A in current compared to the largest distortion of about 100 A for the fixed VSG. The dynamic response time for all three methods is less than 1 s. Compared to the vector control, the dynamic response time during the fluctuation process is significantly longer for the VSG system, and the steady-state amplitude will also change. As a result, under the influence of a rapid load increase, the system's current varies and becomes larger than it was prior to the load increase after stability. The voltage amplitude remains largely unchanged during this time.