# Pattern Classification Based on RBF Networks with Self-Constructing Clustering and Hybrid Learning
## 5 Parameter Refinement Phase
### Centers and Deviations  
First, we describe how to refine the centers and deviations associated with the basis functions in the hidden layer. To make the refinement possible, we express \(\hat{y}_{f}\) in Equation (5) in terms of the hyperbolic tangent sigmoid function which is analytic, i.e.,  
\[\hat{y}_{f}\approx a_{f}=g\left(s_{f}\right)=\frac{e^{\beta s_{f}}-e^{-\beta s _{f}}}{e^{\beta s_{f}}+e^{-\beta s_{f}}} \tag{22}\]  
for \(1\leq f\leq m\), where \(\beta\) is a slope-controlling constant and \(s_{f}\) is defined in Equation (4). Note that \(a_{f}\) provides a good approximation to the sign function and it is differentiable at all points. Let us denote the initial values of the centers and deviations, shown in Equation (9), as \(c_{k,j}(0)\) and \(v_{k,j}(0)\), \(1\leq k\leq n\), \(1\leq j\leq J\), and the initial values of the weights and biases, obtained in Equation (21), as \(w_{f,i}(0)\) and \(b_{f}(0)\), \(1\leq f\leq m\), \(1\leq i\leq J\). As in [56], the mean squared error for the training set \(X\) can be estimated as  
\[ \begin{split}\hat{E}(k)&=\parallel\textbf{y}(k)- \hat{\textbf{y}}(k)\parallel^{2}\\ &\approx\parallel\textbf{y}(k)-\textbf{a}(k)\parallel^{2}=\sum_{l =1}^{m}[y_{l}(k)-a_{l}(k)]^{2}\end{split}  \tag{23}\]  
where \((x(k),y(k))\) can be any pattern in the training dataset \(X\). By steepest descent backpropagation, the centers and deviations of the basis functions in the hidden layer are updated as  
\[c_{i,j}(k+1)=c_{i,j}(k)-\eta\frac{\partial\hat{E}(k)}{\partial c_{i,j}},1\leq i \leq n,1\leq j\leq J \tag{24}\]  
for \(1\leq i\leq n\), \(1\leq j\leq J\). In matrix form this becomes:  
\[c_{j}(k+1)=c_{j}(k)+4\eta h_{j}(k)\mathbf{D}_{1,j}(k)\mathbf{q}_{1,j}(k)\mathbf{ w}_{j}(k)^{\mathrm{T}}\hat{\mathbf{G}}(\mathbf{s}(k))\mathbf{e}(k) \tag{36}\]  
\[v_{j}(k+1)=v_{j}(k)+4\eta h_{j}(k)\mathbf{D}_{2,j}(k)\mathbf{q}_{2,j}(k) \mathbf{w}_{j}(k)^{\mathrm{T}}\hat{\mathbf{G}}(\mathbf{s}(k))\mathbf{e}(k) \tag{37}\]  
for \(1\leq j\leq J\), where  
\[\mathbf{q}_{1,j}(k)=\left[ \begin{array}{ccccc}x_{1}(k)-c_{1,j}(k)&x_{2}(k)- c_{2,j}(k)&\cdots&x_{n}(k)-c_{n,j}(k)\end{array} \right]^{\mathrm{T}} \tag{38}\]  
\[\mathbf{q}_{2,j}(k)=\left[ \begin{array}{ccccc}\left[x_{1}(k)-c_{1,j}(k) \right]^{2}&\left[x_{2}(k)-c_{2,j}(k)\right]^{2}&\cdots&\left[x_{n}(k)-c_{n,j} (k)\right]^{2}\end{array} \right]^{\mathrm{T}} \tag{39}\]  
\[\mathbf{D}_{1,j}(k)=\left[ \begin{array}{ccccc}\frac{1}{\left[\alpha v_{1,j} (k)\right]^{2}}&0&0&\cdots&0&0\\ 0&\frac{1}{\left[\alpha v_{2,j}(k)\right]^{2}}&0&\cdots&0&0\\ \vdots&\vdots&\vdots&\ddots&\vdots&\vdots\\ 0&0&0&\cdots&0&\frac{1}{\left[\alpha v_{n,j}(k)\right]^{2}}\end{array} \right] \tag{40}\]  
\[\mathbf{D}_{2,j}(k)=\left[ \begin{array}{ccccc}\frac{1}{\left[\alpha^{2}v_{ 1,j}^{3}(k)\right]^{2}}&0&0&\cdots&0&0\\ 0&\frac{1}{\left[\alpha^{2}v_{2,j}^{2}(k)\right]^{2}}&0&\cdots&0&0\\ \vdots&\vdots&\vdots&\ddots&\vdots&\vdots\\ 0&0&0&\cdots&0&\frac{1}{\left[\alpha^{2}v_{n,j}^{3}(k)\right]^{2}}\end{array} \right] \tag{41}\]  
\[\mathbf{w}_{j}(k)=\left[ \begin{array}{ccccc}w_{1,j}(k)&w_{2,j}(k)&\cdots&w_ {m,j}(k)\end{array} \right]^{\mathrm{T}} \tag{42}\]  
\[\hat{\mathbf{G}}(\mathbf{s}(k))=\left[ \begin{array}{ccccc}\dot{g}(s_{1}(k) )&0&0&\cdots&0&0\\ 0&\dot{g}(s_{2}(k))&0&\cdots&0&0\\ \vdots&\vdots&\vdots&\ddots&\vdots&\vdots\\ 0&0&0&\cdots&0&\dot{g}(s_{m}(k))\end{array} \right] \tag{43}\]  
\[\mathbf{e}(k)=\left[ \begin{array}{ccccc}y_{1}(k)-a_{1}(k)&y_{2}(k)-a_{2}(k)& \cdots&y_{m}(k)-a_{m}(k)\end{array} \right]^{\mathrm{T}} \tag{44}\]  
Note that the algorithm described above is the stochastic steepest descent backpropagation which involves incremental training. However, we perform batch training in which the complete gradient is computed, after all inputs are applied to the network, before the centers and deviations are updated.