## 9 Extremum Seeking Control  
Extremum Seeking Control (ESC) is utilized for the design of certain controllers, offering the significant advantage of enabling optimization based on a cost function without requiring explicit knowledge of the system model. In this approach, the system is treated as a black box, and its performance is optimized solely through evaluations of the cost function. This methodology is particularly advantageous for systems characterized by complex or poorly understood dynamics. In the present work, ESC is employed to optimize the weighting matrices \(Q\) and \(R\) of a discrete Linear Quadratic Regulator (LQR), with the aim of minimizing a specified cost function to enhance both the transient and steady-state performance of the controlled system.  
\[J(\theta)=\frac{1}{T}\int_{0}^{T}e^{2}(t,\theta)dt \tag{58}\]where \(\theta=\left[\begin{array}{ccccc}\theta_{1}&\theta_{2}&\ldots&\theta_{Nq+n+p} \end{array}\right]=\left[\begin{array}{ccccc}R_{1}&\ldots&R_{p}&Q_{1}&Q_{2}& \ldots&Q_{Nq+n}\end{array}\right]\), \(e=y_{r}-y(k)\), and \(T\) is the simulation time. Remember that \(Q\in\Re^{(Nq+n)\times(Nq+n))}\) and \(R\in\Re^{(p\times p)}\) are diagonal matrices:  
\[Q=\left[\begin{array}{ccccc}Q_{1}&0&\ldots&0\\ 0&Q_{2}&\ddots&0\\ \vdots&\ddots&\ddots&\vdots\\ 0&0&\ldots&Q_{Nq+n}\end{array}\right]\text{y }R=\left[\begin{array}{ccccc}R_{1}&0&\ldots&0\\ 0&R_{2}&\ddots&0\\ \vdots&\ddots&\ddots&\vdots\\ 0&0&\ldots&R_{p}\end{array}\right]\]  
Furthermore, as explained in [31], it is necessary to know the values of \(h\), \(\omega\), \(\alpha\), and \(\gamma\):  
\[\omega =\left[\begin{array}{ccccc}\omega_{1}&\omega_{2}&\ldots& \omega_{Nq+n+p}\end{array}\right] \tag{59}\] \[\alpha =\left[\begin{array}{ccccc}\alpha_{1}&\alpha_{2}&\ldots& \alpha_{Nq+n+p}\end{array}\right]\] \[\gamma =\left[\begin{array}{ccccc}\gamma_{1}&\gamma_{2}&\ldots& \gamma_{Nq+n+p}\end{array}\right]\]  
where \(0<h<1\), \(\omega_{i}=a^{i}\pi\) and \(0<a<1\). The values for \(\alpha_{i}\) and \(\gamma_{i}\) are appropriately selected, as they, together with the cost function, influence the stability and convergence of the method. These values are used in the equations presented in [32]:  
\[\begin{array}{rl}\zeta(k)&=-h\zeta(k-1)+J(\theta(k-1))\\ \widehat{\theta}_{i}(k+1)&=\widehat{\theta}_{i}(k)-\gamma_{i}\alpha_{i}cos( \omega_{i}(k))[J(\theta(k))-(1+h)\zeta(k)]\\ \theta_{i}(k+1)&=\widehat{\theta}_{i}(k+1)+\alpha_{i}cos(\omega_{i}(k+1)) \end{array} \tag{60}\]  
The steps to implement Extremum Seeking Control are as follows:  
1. Define a cost function \(J(\theta)\).
2. Specify the initial parameters \(\theta\).
3. Select the optimizer parameters \(h\), \(\omega\), \(\alpha\) and \(\gamma\).
4. Solve the optimizer equations.
5. Obtain the state feedback matrix \(K\) using the \(Q\) and \(R\) matrices provided by the optimizer.
6. Return to step 4 until the optimal values \(Q\) and \(R\) are found.
7. Verify whether the obtained result corresponds to a global minimum; if not, return to step 3.  
It is worth noting that another advantage of this method is that it eliminates the need for manual tuning of the \(Q\) and \(R\) matrices, ensuring that they are appropriately selected by optimizing the cost function.