# Control Synthesis as Machine Learning Control by Symbolic Regression Methods
## 3 Small Variations of the Basic Solution  
Searching for an optimal solution in the space of codes is complicated by the fact that this space does not have a metric measure. For such search spaces it is impossible to use evolutionary algorithms with arithmetic operations. Evolutionary algorithms are the search engine in all symbolic regression methods. Most of known evolutionary algorithms include arithmetic operations to transform possible solutions and produce evolution. Therefore, genetic algorithm is a main searching algorithm on the space of codes, that doesn't use arithmetic operations in its steps. Studies of this problem have led to the formulation of the principle of small variations of the basic solution. According to this principle, the search for the mathematical expression can be started in the neighbourhood of one given possible solution. This solution is coded by a symbolic regression method and it is called a basic solution. The essence of the principle is that for a code of function, many possible small variations are determined. A small variation is such a minor change in the code that leads to the appearance of a new possible solution. According to the principle, others possible solutions are coded as sets of small variations of this basic solution. To obtain any other possible solution it is necessary to apply a vector of small variations to the basic solution. Genetic operations are applied to the vector of variations. After some generations the basic solution is changed on the best current solution. Such an approach is very convenient for search of control systems, because there are many specialists in control that can create good control system intuitively or on the basis of their experience. This control system can be considered as a basic solution. The code of small variation is an integer vector with three or four components depending on the method of symbolic regression. These components include information about the location of the element to be changed and its new value. For example, to record a small variation in Cartesian genetic programming it is enough to use an integer vector with three components  
\[\mathbf{w}=[w_{1}\ w_{2}\ w_{3}]^{T}, \tag{18}\]where \(w_{1}\) is a number of column in the code, \(w_{2}\) is a number of line in the column \(w_{1}\), \(w_{3}\) is a new value of element. The vector of small variation in the network operator method consists of four elements  
\[\mathbf{w}=[w_{1}\ w_{2}\ w_{3}\ w_{4}]^{T}, \tag{19}\]  
where \(w_{1}\) is a type of variation, \(w_{2}\) is the line number, \(w_{3}\) is the column number of the network operator matrix, and \(w_{4}\) is a new value of an element. Possible solutions are coded as sets of small variation vectors  
\[\mathrm{W}^{i}=\{\mathbf{w}^{i,1},\ldots,\mathbf{w}^{i,d}\} \tag{20}\]  
where \(d\) is a length or depth of variations. Genetic algorithm performs evolution on the sets of small variation vectors. For example, to perform the operation of crossover two possible solutions are selected  
\[ \begin{array}{rcl}\mathrm{W}^{a}&=&\{\mathbf{w}^{a,1},\ldots,\mathbf{w}^{a, d}\}\\ \mathrm{W}^{\beta}&=&\{\mathbf{w}^{\beta,1},\ldots,\mathbf{w}^{\beta,d}\} \end{array}  \tag{21}\]  
A point of crossover is determined randomly \(k_{c}\in\{1,\ldots,d\}\). New possible solutions are received by exchanging tails of the selected possible solutions starting from the crossover point  
\[ \begin{array}{rcl}\mathrm{W}^{\gamma}&=&\{\mathbf{w}^{a,1},\ldots,\mathbf{ w}^{a,k_{c}}\ \mathbf{w}^{\beta,k_{c}+1},\ldots,\mathbf{w}^{\beta,d}\}\\ \mathrm{W}^{\delta}&=&\{\mathbf{w}^{\beta,1},\ldots,\mathbf{w}^{\beta,k_{c}},\mathbf{w}^{a,k_{c}+1},\ldots,\mathbf{w}^{a,d}\}\end{array}  \tag{22}\]  
To receive the symbolic regression code of possible solution \(\mathrm{G}_{i}\) and evaluate the performance of this solution, the set of small variations is applied to the basic solution  
\[\mathrm{G}_{i}=\mathbf{w}^{i,d}\circ\ldots\circ\mathbf{w}^{i,1}\circ\mathrm{G }_{0}, \tag{23}\]  
where \(\mathrm{G}_{0}\) is a code of the basic solution.