# Use RBF as a Sampling Method in Multistart Global Optimization Method
## 2 Method Description
### RBF Networks  
An RBF neural network typically is expressed as a function:  
\[y(\overrightarrow{x})=\sum_{i=1}^{k}w_{i}\phi(\|x-c_{i}\|) \tag{3}\]  
where the vector \(\overrightarrow{x}\) stands for the input vector of the network and the vector \(\overrightarrow{w}\) is called weight vector with \(k\) elements. Typically, the function \(\phi(x)\) is the so-called Gaussian function defined as:  
\[\phi(x)=\exp\left(-\frac{(x-c)^{2}}{\sigma^{2}}\right) \tag{4}\]  
where the value \(\phi(x)\) depends mainly on the distance between \(x\) and \(x\). The vector \(\overrightarrow{c}\) is called centroid and the vector \(\overrightarrow{\sigma}=(\sigma_{1},\sigma_{2},\ldots,\sigma_{k})\) is considered as the variance vector. A typical plot of this function is shown in Figure 3.  
The network of Equation (3) can be used to approximate functions \(f(x),\ x\in S\subset R^{n}\) by minimizing the error:  
\[E\big{(}y\big{(}\overrightarrow{x}\big{)}\big{)}=\sum_{i=1}^{M}(y(x_{i})-f(x _{i}))^{2} \tag{5}\]  
where the variable \(M\) denotes the number of training samples provided for the function \(f(x)\). The RBF network is shown graphically in Figure 4. During a training procedure, the parameters of the RBF network are adapted in order to minimize the error of Equation (5). The RBF network us trained using a two-phase methodology:  
Figure 1: Progress of Multistart for the Hansen function. Figure 2: Plot of the used termination rule for the EXP8 test function. During the first phase the \(k\) centers of and the associated variances are calculated through K-Means algorithm [66]. 2. During the second phase, the weight vector \(\overrightarrow{w}=(w_{1},w_{2},\ldots,w_{k})\) is calculated by solving a linear system of equations with the following procedure: 1. **Set**\(W=w_{kj}\), the matrix for the \(k\) weights 2. **Set**\(\Phi=\phi_{j}(x_{i})\) 3. **Set**\(T=\{t_{i}=f(x_{i}),i=1,..,M\}\). **(d)** The system to be solved is defined as: \[\Phi^{T}\Big{(}T-\Phi W^{T}\Big{)}=0\] (6) The solution is: \[W^{T}=\Big{(}\Phi^{T}\Phi\Big{)}^{-1}\Phi^{T}T=\Phi^{\dagger}T\] (7) The matrix \(\Phi^{\dagger}=\big{(}\Phi^{T}\Phi\big{)}^{-1}\Phi^{T}\) is the so - called pseudo-inverse of \(\Phi\), with the property \[\Phi^{\dagger}\Phi=I\] (8)  
In the proposed technique, the previously defined network constructs an approximation of the objective function \(f(x)\) and subsequently the method Multistart takes samples from the approximation of the objective function. The process starts by taking some samples from the actual \(f(x)\) function. These samples are then used to train an RBF neural network. After training, many samples are taken from the neural network function and the best ones will be used in the global optimization method. The overall sampling procedure is shown in Algorithm 2. The overall algorithm is graphically represented in Figure 5 in the form of a flowchart. \[\phi(x)=\exp\left(-\frac{(x-c)^{2}}{\sigma^{2}}\right) \tag{4}\]0 Typical plot of the Gaussian function. ```
1. Initialization step. (a) Set \(N\), the number of required samples. (b) Set \(\mathrm{ISAMPLES}\), the initial samples that will be drawn from the function \(f(x)\). (c) Set \(\mathrm{FR}\) a real number, with \(\mathrm{FR}\ >N\). For example, \(\mathrm{FR}=10\times N\) (d) Set \(\mathrm{IS}=\emptyset\) (e) Set \(\mathrm{FS}=\emptyset\). This is the final outcome of the algorithm. (f) Set \(k\), the number of weights for the RBF network,
```
2. Initial Sampling step. (a) For \(i=1,\ldots,\mathrm{ISAMPLES}\)do i. Take a sample \(s_{i}=(x_{i},f(x_{i})),x_{i}\in S\subset R^{n}\) ii. \(\mathrm{IS}=\mathrm{IS}\cup s_{i}\) (b) End For
3. Training step. (a) Construct an RBF network \(y(x)\) with \(k\) weights. (b) Train \(y(x)\) using the set \(\mathrm{IS}\) by minimizing the train error of Equation (5). (a) Find **sampling** step. (a) For \(i=1,\ldots,\mathrm{FR}\)do i. Take a sample \(s_{i}=(x_{i},y(x_{i}))\) ii. \(\mathrm{FS}=\mathrm{FS}\cup s_{i}\) (b) End For (c) Sort \(\mathrm{FS}\) according to the function values.