# Universal Approach to Solution of Optimization Problems by Symbolic Regression
## 4 Small Variations for Symbolic Regression Methods
### Network Operator Method  
The network operator method encodes mathematical expressions in the form of directed graphs [5; 27]. For coding, the method uses functions with one and two arguments. On the graph, functions with one argument are associated with the edges, functions with two arguments are associated with the nodes, and the arguments of the encoded mathematical expression are associated with the source nodes of the graph. Functions with two arguments must be commutative, associative, and have a unit element. An integer matrix of the network operator is used to store the code in the computer memory. Let us consider an example of coding in the network operator method. Let a mathematical expression be given as  
\[y=\sin(\cos(q_{1}x_{1}+q_{2}))+\exp(-q_{2}x_{2}), \tag{38}\]  
where \(q_{1}\), \(q_{2}\) are parameters, \(x_{1}\), \(x_{2}\) are variables. The parameters and the variables are arguments of the mathematical expression. To encode a mathematical expression, it is sufficient to use the following sets of arguments and elementary functions  
The set of arguments  
\[\mathrm{F}_{0}=\{x_{1},x_{2},q_{1},q_{2}\}. \tag{39}\]  
The set of function with one argument  
\[\mathrm{F}_{1} = \{f_{1,1}(z)=z,f_{1,2}(z)=-z,f_{1,3}(z)=\sin(z), \tag{40}\] \[f_{1,A}(z)=\cos(z),f_{1,5}(z)=\exp(z)\}.\]  
The set of function with two arguments  
\[\mathrm{F}_{2}=\{f_{2,1}(z_{1},z_{2})=z_{1}+z_{2},f_{2,2}(z_{1},z_{2})=z_{1}z_ {2}\}. \tag{41}\]  
The graph of the network operator for (38) is given in Figure 1. In the source nodes of the graph, there are arguments of the mathematical expression. The remaining nodes contain indices of functions with two arguments. Next to the edges there are indices of functions with one argument. The indices correspond to the second index of elements in the sets of elementary functions (40) and (41). The nodes are enumerated according to topological sorting in their upper parts. The number of the node from which the edge exits is less than the number of the node where the edge enters. Such enumeration is always possible for graphs without loops, and it allows one to obtain an upper triangular matrix of the network operator. For nodes 8 and 9, the second argument is not specified. We use a unit element, zero for the addition function, as the second argument. The network operator matrix for (38) is the following  
\[\boldsymbol{\Psi}= \begin{bmatrix}0&0&0&0&1&0&0&0&0&0\\ 0&0&0&0&0&1&0&0&0&0&0\\ 0&0&0&0&1&0&0&0&0&0\\ 0&0&0&0&0&1&1&0&0&0\\ 0&0&0&0&2&0&1&0&0&0\\ 0&0&0&0&0&2&0&2&0&0\\ 0&0&0&0&0&0&1&0&4&0\\ 0&0&0&0&0&0&0&1&0&5\\ 0&0&0&0&0&0&0&0&1&3\\ 0&0&0&0&0&0&0&0&1\end{bmatrix} . \tag{42}\]  
The numbers of rows and columns in the network operator matrix correspond to the node number in the graph. Edges exiting the node are located in a row, edges entering a node are located in a column. The diagonal elements contain indices of functions with two arguments. The rest nonzero elements are the indices of functions with one argument. Let us introduce small variations for the code of the network operator: 1--replacement of a nonzero off-diagonal element, 2--replacement of a nonzero diagonal element, 3--replacement of a zero off-diagonal element, 4--zeroing of an off-diagonal nonzero element. Small variation 4 is performed only if at least one off-diagonal nonzero element remains in the given row and in the given column. To present a small variation use a vector of four components  
\[\mathbf{w}=[w_{1}\;w_{2}\;w_{3}\;w_{4}]^{T}, \tag{43}\]  
where \(w_{1}\) is an index of small variation, \(w_{2}\) is an index of row, \(w_{3}\) is an index of column, \(w_{4}\) is a new value of element. Suppose that we have a set of four vectors of variations  
\[ \begin{array}{l}\mathbf{w}^{1}=[1\;7\;9\;2]^{T},\\ \mathbf{w}^{2}=[4\;6\;8\;1]^{T},\\ \mathbf{w}^{3}=[2\;6\;6\;1]^{T},\\ \mathbf{w}^{4}=[3\;5\;9\;3]^{T}.\end{array}  \tag{44}\]  
Figure 1: The network operator graph for mathematical expression (38).