## 2 Description of MMC-VSG Structure and Modeling
### MMC-VSG Topology and Control Strategy
#### 2.2.2 VSG Control Strategy
Thus, according to Equation (12), the reactive power control diagram is shown in \[J\omega_{0}\frac{\mathrm{d}\omega}{dt}=P_{ref}-P_{e}-D_{P}(\omega-\omega_{0}) \tag{11}\]2  
From the previous narration, the VSG output active power can be derived from Figure 3,  
\[J\omega_{0}\frac{\mathrm{d}\omega}{dt}=P_{ref}-P_{e}-D_{P}(\omega-\omega_{0}) \tag{11}\]3  
where  
\[J\omega_{0}\frac{\mathrm{d}\omega}{dt}=P_{ref}-P_{e}-D_{P}(\omega-\omega_{0}) \tag{11}\]4  
\[J\omega_{0}\frac{\mathrm{d}\omega}{dt}=P_{ref}-P_{e}-D_{P}(\omega-\omega_{0}) \tag{11}\]5  
\[J\omega_{0}\frac{\mathrm{d}\omega}{dt}=P_{ref}-P_{e}-D_{P}(\omega-\omega_{0}) \tag{11}\]6 Reactive power control diagram for VSG. \[J\omega_{0}\frac{\mathrm{d}\omega}{dt}=P_{ref}-P_{e}-D_{P}(\omega-\omega_{0}) \tag{11}\]7 Active power control diagram for VSG. Thus, the output power \(P_{\text{out}}\) can be written as  
\[ \begin{split} P_{\text{out}}\,=\lim_{s\to 0}(P_{e})=\lim_{s\to 0} \Bigl{(}G_{r}(s)P_{ref}+G_{f}(s)(\omega_{0}-\omega)\Bigr{)}\\ =\lim_{s\to 0}\biggl{(}\frac{k_{P}}{J\omega_{0}s^{2}+Dps+k_{P}}P_{ ref}+\frac{k_{P}(J\omega_{0}s+D_{P})}{J\omega_{0}s^{2}+Dps+k_{P}}(\omega_{0}- \omega)\biggr{)}\end{split}  \tag{23}\]  
The output power for the steady state is  
\[P_{\text{out}}\,=P_{\text{ref}}+D_{P}(\omega_{0}-\omega) \tag{24}\]  
Equation (10) indicates that the steady-state output consists of two parts: active power \(P_{ref}\) and steady-state active power deviation \(D_{P}(\omega_{0}-\omega)\). Therefore, in order to reduce the steady-state error, \(D_{p}\) should be minimized. Here, the closed-loop transfer function of the small signal can be summarized as Equation (9), which can be derived as the small signal model of the MMC-VSG shown in Figure 5.  
The damping coefficient and natural oscillation angular frequency are as follows  
\[\xi=\frac{D_{P}}{2}\sqrt{\frac{1}{k_{P}J\omega_{0}}} \tag{25}\]  
\[\omega=\sqrt{\frac{k_{P}}{J\omega_{0}}} \tag{26}\]  
Equation (14) shows that the system's natural oscillation angular frequency is independent of the value of the damping coefficient \(D\) and falls off as the inertia \(J\) rises. The system damping is proportional to the damping coefficient \(D\) and decreases with the uplift of the inertia \(J\). Figures 6 and 7 show the step response of the MMC-VSG small-signal model under different \(D\) and \(J\) parameters. Through the analysis of Figures 6 and 7, it is not difficult to determine that the inertia determines the reaction and system adjustment speed of the dynamic response process, and the \(J\) increases with the reaction and system adjustment time. The damping coefficient determines the speed of the oscillation attenuation. The larger \(D\) is, the faster the attenuation speed is. Furthermore, the system switches from under-damping to overdamping when \(J\) becomes larger or when \(D\) becomes smaller. It can be concluded that the active power steady-state characteristic during the VSG operation is related to \(D\), instead of \(J\), in accordance with the aforementioned contents and automatic control principle. While the dynamic feature of VSG is connected to \(J\) and \(D\), an increase in \(D\) may lead to growth in the steady-state inaccuracy of the VSG grid connection. The system damping increases with \(D\) and reduces the power overshoot, and the system regulation time increases with slower reaction times. Additionally, the system damping decreases as \(J\) increases, increasing the overshoot in the dynamic process and degrading stability. Consequently, there are inconsistencies in parameters \(J\) and \(D\) that prevent them from meeting the demands of a small steady-state error, a large dynamic damping, and a quick system adjustment speed requirement. Figure 5: Small signal model of the MMC-VSG. With the purpose of improving the dynamic features to meet the above properties of the MMC-VSG system without damaging the steady-state characteristics, this article discusses incorporating a VSG control based on the TD3 algorithm.