# Generalized Optimality Criteria Method for
## 2 Generalized Optimality Criteria Method
### Generalized Optimality Criteria Method
Instead, the Lagrange multipliers are updated at every optimization step so that when the optimization is converged, the necessary condition in Equation (10) is satisfied. [PERSON] et al. (2016) proposed several methods of updating the Lagrange multiplier. Table 1 summarizes the three updating schemes. In the table, \(p_{0}\) is the initial value of an update parameter, and \(\alpha\) is an acceleration parameter used to modify the update parameter. [PERSON] et al. (2016) suggested \(p_{0}=0.5\) and \(\alpha=1.0\). In this article, the updating algorithm of the Lagrange multiplier is composed of two steps: (a) initial estimate and (b) update during iteration. First, the initial values of Lagrange multipliers are estimated using the inverse form in Table 1. However, it would be computationally complicated to calculate the inverse of the constraint gradient matrix, and it would be unnecessary as it provides the initial estimate. Instead, the uncoupled version of the inverse form is used for the initial estimate as  
\[\lambda_{i}=-\frac{\
abla_{\textbf{x}}f^{\intercal}\
abla_{\textbf{x}}g_{i}}{ \
abla_{\textbf{x}}g_{i}^{\intercal}\
abla_{\textbf{x}}g_{i}} \tag{11}\]  
The above estimate works well in most cases except when either the objective gradient or constraint gradients are zero or too small. In the optimization problem formulation, the objective function and constraints are normalized to the initial value and/or constraint limit. Therefore, the initial value of \(\lambda_{i}=1\) is a good starting point when \(\
abla_{\textbf{x}}f+\lambda\
abla_{\textbf{x}}g\approx 0\). The basic concept of GOCM is when the constraint is violated, the Lagrange multiplier needs to increase, while it is decreased when the constraint is inactive. Therefore, a simple updating formula would be \(\lambda_{i}^{k+1}=\lambda_{i}^{k}(1+g_{i}^{k})\). In this simple formula, the Lagrange multiplier increases when the constraint is positive (i.e., the constraint is violated) or it decreases when the constraint is negative (i.e., the constraint is inactive). This simple algorithm can cause some problems when the constraint violation is not recovered quickly, or the constraint is inactive for many steps. For example, if the constraint is violated a lot and stays violated for many steps, then the Lagrange multipliers can increase too fast. Therefore, it would be necessary to include the effect of change in constraint. In the article, the following updating rule is proposed for the Lagrange multiplier:  
\[\lambda_{i}^{k+1}=\lambda_{i}^{k}[1+p_{0}(g_{i}^{k}+\Delta g_{i}^{k})] \tag{12}\]  
where \(p_{0}\) is the update parameter similar to those used in Table 1. The update parameter can have a positive value when both the sign of \(g_{i}^{k}\) and \(\Delta g_{i}^{k}\) are the same. A more complex scheme can also be used, as shown in the Matlab code in Section 3. The updating algorithm in Equation (12) is applied (a) when the current constraint is violated and the constraint increases from the previous step or (b) when the current constraint is inactive and the constraint continues to decrease. In order to make the updating process stable, the maximum change of the Lagrange multipliers is used. Moreover, the \begin{table}
\begin{tabular}{c c} \hline \hline
**Updating Methods** & **Updating Formulas** \\ \hline Linear form & \(\lambda_{i}^{k+1}=\lambda_{i}^{k}(1+\alpha^{k}p_{0}g_{i})\) \\ Exponential form & \(\lambda_{i}^{k+1}=\lambda_{i}^{k}(g_{i})^{\alpha^{k}p_{0}}\) \\ Inverse form & \(\mathbf{\lambda}=[\
abla_{\textbf{x}}\textbf{g}^{\intercal}\
abla_{\textbf{x}} \textbf{g}]^{-1}\{\
abla_{\textbf{x}}\textbf{g}^{\intercal}\
abla_{\textbf{x}}f\}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Lagrange multiplier update formulas. Lagrange multipliers are limited to change within the lower- and upper-bounds. Since all constraints are normalized by their bounds, the magnitudes of the Lagrange multipliers are in a similar order of magnitude.