# Universal Approach to Solution of Optimization Problems by Symbolic Regression
## 4 Small Variations for Symbolic Regression Methods
### Variational Genetic Programming  
The GP code for a mathematical expression is a computational tree. Arguments of a mathematical expression are located on the branches of the tree. The computational tree for the mathematical expression (38) is shown in Figure 2.  
In GP computational tree functions are placed in the nodes. The functions from the sets (40) and (41) were used. The number of arguments in the leaves of the tree must match the number of times these arguments are used in the expression. For example, the parameter \(q_{2}\) appears in a mathematical expression twice, so it appears twice in the leaves of the tree.  
To store the computational tree in the computer memory a vector of two components is used  
\[\mathbf{s}_{GP}=[s_{1}\;s_{2}]^{T}, \tag{47}\]  
where \(s_{1}\) is a number of function arguments, \(s_{2}\) is the function index.  
Arguments of a mathematical expression are represented as functions without arguments  
\[\text{F}_{0}=\{f_{0,1}=x_{1},f_{0,2}=x_{2},f_{0,3}=q_{1},f_{0,4}=q_{2}\}. \tag{48}\]  
Figure 2: GP computational tree for mathematical expression (38).  
of elements from the set of arguments. The result of the calculation of the function call is immediately added to the set of arguments so that it can be used in subsequent calls.  
Consider an example of coding mathematical expression (38) by CGP. We will use the set of arguments (39), combine all functions from (40) and (41) into one set, and exclude identity function \(f_{1,1}(z)=z\)  
\[\begin{array}{rcl}\text{F}&=&\{f_{1}(z)=-z,f_{2}(z)=\sin(z),f_{3}(z)=\cos(z),\\ &&f_{4}(z)=\exp(z),f_{5}(z_{1},z_{2})=z_{1}+z_{2},f_{6}(z_{1},z_{2})=z_{1}z_{2} )\}.\end{array} \tag{58}\]  
Since in (58) there are only functions with one and two arguments, it is sufficient to use a vector of three elements to encode a function call. For the function with one argument, the third element is not used.  
The CGP for a mathematical expression (38) is as follows  
\[\text{S}=\left(\begin{bmatrix}6\\ 3\\ 1\end{bmatrix},\begin{bmatrix}5\\ 5\\ 4\end{bmatrix},\begin{bmatrix}3\\ 6\\ 1\end{bmatrix},\begin{bmatrix}2\\ 2\end{bmatrix},\begin{bmatrix}6\\ 4\\ 2\end{bmatrix},\begin{bmatrix}1\\ 3\end{bmatrix},\begin{bmatrix}4\\ 10\\ 4\end{bmatrix},\begin{bmatrix}5\\ 8\\ 11\end{bmatrix}\right). \tag{59}\]  
Small variation of code in CGP is a change of one element of the code [28]. To present a small variation, it is enough to use an integer vector of three elements  
\[\mathbf{w}=[w_{1}\ w_{2}\ w_{3}]^{T}, \tag{60}\]  
where \(w_{1}\) is an index of column in the code, \(w_{2}\) is an index of row in the column \(w_{1}\), and \(w_{3}\) is a new value of the element.  
If we vary the first element of the vector of an elementary function, then its new value is determined by the index of the element from the set of elementary functions (58). If we vary some other element, then its value must be less than the sum of the number of elements in the set of arguments (39) and the index \(w_{1}\) of the varied call vector.  
Let us define some variations of the CGP code of (59).  
\[\begin{array}{rcl}\mathbf{w}^{1}&=&[2\ 1\ 3]^{T},\\ \mathbf{w}^{2}&=&[3\ 2\ 5]^{T},\\ \mathbf{w}^{3}&=&[6\ 2\ 8]^{T},\\ \mathbf{w}^{3}&=&[6\ 1\ 6]^{T}.\end{array} \tag{61}\]  
Having performed small variations (61), we obtain the following CGP code  
\[\mathbf{w}^{4}\circ\mathbf{w}^{3}\circ\mathbf{w}^{2}\circ\mathbf{w}^{1}\circ \text{S}=\left(\begin{bmatrix}6\\ 3\\ 1\end{bmatrix},\begin{bmatrix}3\\ 5\\ 4\end{bmatrix},\begin{bmatrix}3\\ 5\\ 1\end{bmatrix},\begin{bmatrix}2\\ 2\end{bmatrix},\begin{bmatrix}6\\ 4\\ 2\end{bmatrix},\begin{bmatrix}6\\ 8\\ 3\end{bmatrix},\begin{bmatrix}4\\ 10\\ 4\end{bmatrix},\begin{bmatrix}5\\ 8\\ 11\end{bmatrix}\right). \tag{62}\]  
that corresponds to mathematical expression  
\[y=\exp(q_{1}\sin(\cos(q_{1}x_{1})))+\sin(\cos(q_{1}x_{1})). \tag{63}\]  
The disadvantage of CGP is that some calls of function in the final mathematical expression may not be used.