# A Bilevel Optimization Approach for Tuning a Neuro-Fuzzy Controller
## 3 Optimization  
In general terms, optimization can be defined as the process of obtaining the best selection between a set of items available, according to some criteria. In a mathematical form, an optimization problem consists of finding a set of values \(\vec{x}\), as follows:  
\[\min/\max\quad f_{m}(\vec{x}) \tag{16}\]  
subject to:  
\[g_{i}(\vec{x})\leq 0,i=1,...,n_{i} \tag{17}\]  
\[h_{j}(\vec{x})=0,j=1,...,n_{j} \tag{18}\]  
\[x_{k}^{L}\leq x_{k}\leq x_{k}^{UI},k=1,...,n_{k} \tag{19}\]  
where \(f_{m}(\vec{x})\) represents the \(m\)th objective function to minimize/maximize, \(g_{i}(\vec{x})\) is the \(i\)th inequality constraint, \(h_{j}(\vec{x})\) is the \(j\)th equality constraint, and \(x_{k}^{L}\), \(x_{k}^{UI}\) delimit the search space of the \(x_{k}\) element in \(\vec{x}\). The optimization problem is considered mono-objective if it only has one objective function.  
There are different categories of optimization problems, according to specific characteristics of the objective functions, types of constraints, or the size and shape of the search space. One of these categories is the bilevel optimization problems, where for its solution, the optimization cases are divided into two sequential problems.  
In this work, a bilevel optimization is implemented to solve a special problem. The solution set \(\vec{x}\) in the first problem includes continuous variables as well as at least one integer variable, categorizing it among mixed optimization problems as described in [26]. Also, it can be considered a noisy optimization problem, applying the concepts in [27], where this type of optimization problem is described as one where the objective function is affected by a noise function, as shown in (20), where \(N(0,1)\) is the normal distribution with a mean of 0 and a variance of 1.  
\[\min\quad f(\vec{x})+N(0,1) \tag{20}\]  
For this work, the optimization function is denoted by (21), where the set \(\vec{w}\) corresponds to the weights of the NFC, whose size can vary in number depending on its architecture. It prevents the direct implementation of a metaheuristic, so in the first level of optimization, these weights are treated as noise elements. Different initialization values  
Figure 2: Feedback implementation of the neuro-fuzzy controller [13].  
can lead to varying results, despite the learning component of the network. A uniform normal distribution is used to generate \(\vec{w}\) in each evaluation of the function at the first level, considering that the gradient descent method mitigates the variance in the results. This consideration is used in diverse developments where the architecture is tuned without taking into account the initial value of the weights.  
\[\min\quad f(\vec{x},\vec{w}) \tag{21}\]  
On the other hand, the second optimization problem has an order constraint, and this means that any subset of \(\vec{x}\) must satisfy the following: (22),  
\[x_{k+1}\leq x_{k+2}\leq...\leq x_{k+n} \tag{22}\]  
In general, these definitions will be used to model an optimization problem that, when solved, results in the tuning of an NFC to perform the task of trajectory tracking in the transesterification process.