## 2 Performance Indicators of System Response
### Augmented Lagrangian Method  
The penalty methods' main drawback is the difficulty of determining the optimal penalty coefficient for different problems and the fact that an exact solution to the constrained problem can only be obtained with infinite penalties [29]. The family of augmented Lagrangian methods circumvents this limitation by adding additional penalty-like components to the unconstrained objective function, which are adapted during optimization to determine their optimal parameter values. The objective function with added terms is called the augmented Lagrangian (AL). The solution of the constrained problem is obtained by sequentially solving the unconstrained problem with the augmented Lagrangian as the objective function. After each iteration, the penalty factors of the AL are updated, and the solutions obtained are used as initial guess for the next iteration. Alternatively, the augmented Lagrangian can be minimized only once with modifications of the penalty factors being performed online during the minimization. The second approach is more suitable for evolutionary swarm algorithms for global searches [11].  
For a problem defined as  
\[\min\mathcal{F}(x) \tag{24}\]  
\[\mathcal{G}(x)\leq 0 \tag{25}\]  
the AL function is given by the Powell-Hestenes-Rockafellar (PHR) formula [30]:  
\[\mathcal{L}(x,\lambda,\rho)=\mathcal{F}(x)+\frac{\rho}{2}\sum_{i=1}^{m}\biggl{[} max\biggl{(}0,\mathcal{G}_{i}(x)+\frac{\lambda_{i}}{\rho}\biggr{)}\biggr{]}^{2} \tag{26}\]  
where \(\mathcal{G}(x)\) represents the inequality constraint function, \(m\) is the number of constraints, \(\rho\) is the penalty factor. \(\lambda_{i}\) variables represent the Lagrange multipliers for each constraint.  
To find the solution of the equivalent constrained problem, the Lagrangian is minimized by an unconstrained or box-constrained optimization algorithm such as PSO or ABC. The augmented Lagrangian method can be outlined in the following steps:  
1. The initial values of the Lagrange multipliers are set to 0, and the penalty parameter is set to [31] \[\rho_{0}=max\biggl{[}\rho_{min},min\biggl{(}\rho_{max},\frac{2|\mathcal{F}(x_{ 0})|}{||\mathcal{G}(x_{0})||^{2}}\biggr{)}\biggr{]}\] (27) where \(\rho_{min}=10^{-6}\) and \(\rho_{max}=10\) are the minimum and maximum values of the penalty factor, respectively, and \(x_{0}\) is the initial guess of the optimization problem.
2. Minimize \(\mathcal{L}(x,\lambda,\rho)\) using an unconstrained or box-constrained optimization algorithm.
3. While the augmented Lagrangian is minimized, the penalty factor \(\rho\) is updated every N iterations: \[\rho_{k+1}=\begin{cases}\gamma\rho_{k}&when\quad ICM_{k+1}>\frac{ICM_{b}}{2}\\ \rho_{k}&otherwise\end{cases}\] (28) where \(\gamma=10\) is the penalty increase factor; ICM is the infeasibility/complementarity measure defined as [30] \[ICM=\biggl{\|}max\biggl{(}\mathcal{G}_{i}(x),-\frac{\lambda_{i}}{\rho}\biggr{)} \biggr{\|}_{\infty}\] (29)4. While the augmented Lagrangian is minimized, the Lagrange multipliers are updated every N iterations: \[\lambda_{i,k+1}=max[\lambda_{min},min(\lambda_{max},\lambda_{i,k}+\rho_{k+1}\mathcal{ G}_{i}(x_{0}))]\] (30) where \(\lambda_{min}=-10^{20}\) and \(\lambda_{max}=10^{20}\) are the minimum and maximum values of the Lagrange multipliers, respectively.
5. If the termination criteria of the optimization algorithm are met, return the results.