# Universal Approach to Solution of Optimization Problems by Symbolic Regression
## 3 Principle of Small Variations of Basic Solution  
The principle of small variations is universal and can be applied to any symbolic regression method. To apply the principle, small variations are initially determined for the code of symbolic regression method. Small variation is coded by an integer vector of small dimension. This code contains the information required to perform the small variation. Then, one possible solution, let us name it a basic solution, is encoded by the method of symbolic regression. The basic solution is set by the researcher, and it is the closest to the optimal solution of the problem. All other possible solutions are determined by ordered sets of vectors of small variations. The search for the optimal solution is performed by the GA, which is called variational genetic algorithm (VarGA), that searches for the optimal solution on the space of ordered sets of vectors of small variations. During the search and after a given number of generations, the basic solution is replaced by the best possible solution found by this moment. Let us consider application of the principle of small variations in detail. In general, the elements of the code space of the non-numerical optimization problem can be written in the form of ordered sets of integer vectors  
\[\Xi=\{S_{1},S_{2},\dots\}, \tag{16}\]  
where \(S_{i}\) is a coded element. Each element of (16) consists of a given number of integer vectors  
\[S_{k}=(s^{k,1},s^{k,2},\dots,s^{k,n_{k}}). \tag{17}\]  
Here, \(s^{k,j}\) is an integer vector, \(j=\overline{1,n_{k}}\), \(n_{k}\) is a length of one element code,  
\[s^{k,j}=[s_{1}^{k,j}\dots s_{r}^{k,j}]^{T}, \tag{18}\]  
where \(r\) is a length of code vector. An element of code can be a vector, a matrix or an ordered set of matrices. In general, these constructions of code can always be presented as an ordered set of integer vectorsif you do not use special mathematical operations for them. For example, if a code is an integer matrix  
\[\mathbf{A}=[a_{i,j}],\ \ i=\overline{1,m},\ \ j=\overline{1,n}, \tag{19}\]  
then  
\[\mathrm{S}=(\mathbf{s}^{1},\ldots,\mathbf{s}^{m}), \tag{20}\]  
where  
\[\mathbf{s}^{i}=[s^{i}_{1}=a_{i,1}\ldots s^{i}_{n}=a_{i,n}]^{T},\ \ i=\overline{1,m}. \tag{21}\]  
Hence, the code of the symbolic regression method is an ordered set of integer vectors. A set can have a different number of vectors, but all integer vectors have the same number of elements. Let us introduce an elementary variation of the code of a non-numerical element. Definition 3: An elementary variation of the code is the replacement of one element of an integer vector. Replacement of one element does not always result in a new valid code. There are certain coding rules that should not be violated. Let us define a small variation. Definition 4: A small variation is the required minimum set of elementary variations that are necessary to obtain valid code of the element from another valid code. In some symbolic regression methods, a small variation consists of one elementary variation, and in other methods, several elementary variations are needed. For a given set of valid codes, let us define a finite set of small variations  
\[\Omega(\Xi)=\{\delta_{1}(\mathrm{S}),\ldots,\delta_{M}(\mathrm{S})\}. \tag{22}\]  
Completeness is the main property of a set of small variations for a set of valid codes with bounded length. Definition 5: A set of small variations is complete if for any two valid codes \(\forall\mathrm{S}^{j}\), \(\mathrm{S}^{j}\in\Xi\) it is always possible to find a finite number of small variations to obtain a valid code \(\mathrm{S}^{j}\in\Xi\) from valid code \(\mathrm{S}^{j}\in\Xi\). \[\mathrm{S}_{j}=\delta_{k_{1}}(\ldots(\delta_{k_{d}}(\mathrm{S}_{i})\ldots). \tag{23}\]  
Any small variation is a mapping function of set of valid element codes into itself  
\[\delta_{k}(\mathrm{S}):\Xi\rightarrow\Xi,\ \ \delta_{k}(\mathrm{S})\in\Omega. \tag{24}\]  
Let us define the distance between two valid codes.