## 2 Control Formulation and Design
### Proposed Adaptive Droop Control Design  
The fixed droop-based inverter control curtails PV power based on a constant factor for all scenarios, leading to unnecessary PV power reduction [23]. The aim is to minimize the power curtailment (whenever possible), restricting the voltage within the boundary condition of overvoltage. For this, a supplementary controller based on ADP is proposed to adjust droop coefficient while decreasing PV power curtailment. Figure 3 shows the control strategy of ADP assisted droop-based power curtailment that adapts the droop value to minimize power curtailment, such that \(V(t)<V_{cri2}\) (1.058 p.u. of 240 V), and \(0<P_{inv}\leq P_{MPPT}\). The input signals to ADP controller are voltage difference, power curtailment, and the time-delayed signals of voltage difference and power curtailment, which provides information on both objectives, i.e., the overvoltage prevention and the power curtailment minimization [31]. ADP is a machine learning-based approach to optimize the performance of dynamic systems over time [32; 34]. Figure 4 shows the ADP controller with two neural network structures: the action network and the critic network. The action network generates the control action and the critic network optimizes the weights of the action network, based on a \"reward\" signal. In this case, the action network has a 6-6-1 structure with \(\mathrm{i}x_{1}(t),x_{2}(t),x_{3}(t),x_{4}(t),x_{5}(t)\), and \(x_{6}(t)\) as input signals. \(x_{1}(t)\) is the p.u. value of the positive voltage difference between \(V(t)\) and \(V_{cri1}\) (1.050 p.u. of 240 V). \(x_{2}(t)\) and \(x_{3}(t)\) are the p.u. values of one-time step and two-time step delayed values of the voltage difference, respectively. Likewise, \(x_{4}(t)\) is the p.u. value of the power curtailment. \(x_{5}(t)\) and \(x_{6}(t)\) are the p.u. values of one-time step and two-time step delayed signals of the power curtailment, respectively. The six hidden nodes are denoted as \(H_{1}\) to \(H_{6}\), and the output node is denoted as \(u(t)\). The output of the action network is the droop value, denoted as \(m_{ADP}(t)\) which adjusts the constant droop coefficient \(m_{c}\) to compute the total droop coefficient \(m_{T}(t)\) for the proposed combined controller. The critic network has a 7-6-1 structure with seven  
Figure 2: Linear, quadratic, and exponential droop for APC method. inputs: \(x_{1}(t)\), \(x_{2}(t)\), \(x_{3}(t)\), \(x_{4}(t)\), \(x_{5}(t)\), \(x_{6}(t)\), and \(u(t)\). Similar to the action network, it also has six hidden nodes \(H_{1}\) to \(H_{6}\) and an output node \(J(t)\). The critic network is formulated in such manner that optimal cost function \(J^{*}(x(t))\) satisfies Bellman principle of optimality, such that  
\[J^{*}(x(t))=\min_{u(t)}\left\{J^{*}(x(t+1))+r(x(t))-U_{c}\right\} \tag{5}\]  
where \(r(x(t))\) is the immediate cost incurred by \(u(t)\) at time \(t\), and \(U_{c}\) is a heuristic term used to balance. The output of critic network updates the weight of action network, as the  
Figure 4: Structure of ADP controller with two neural network structures: action and critic network. Figure 3: Adaptive dynamic programming (ADP) controller for adaptive droop control with droop-based APC technique. value of \(U_{c}\) is set 0 for our design. The reward signal is defined in a binary format with \"0\" representing success, or \" \(-1\)\" representing failure. Here, the reward signal is defined as  
\[r(t)=-c(a_{1}x_{1}^{2}+a_{2}x_{2}^{2}+a_{3}x_{3}^{2}+a_{4}x_{4}^{2}+a_{5}x_{5}^{ 2}+a_{6}x_{6}^{2}) \tag{6}\]  
where \(c,a_{1},a_{2},a_{3},a_{4},a_{5}\), and \(a_{6}\) are constants. The reward signal is devised to restrain the voltage from exceeding \(V_{cr12}\) at the same time minimizing the power curtailment. The value of \(c\) in the reward signal design is represented as \"1\" for positive value of the voltage difference and the power curtailment. This would ensure a negative reward would be applied in the ADP algorithm until the minimum curtailment is achieved or the voltage crosses the limit.