# Pattern Classification Based on RBF Networks with Self-Constructing Clustering and Hybrid Learning
## 3 Proposed Approach  
Let \(X=\left\{\left(\mathbf{x}^{(i)},\mathbf{y}^{(i)}\right)\middle|1\leq i\leq N\right\}\) be a finite set of \(N\) training patterns, where \(\mathbf{x}^{(i)}\in\mathbb{R}^{n}\) is an input vector with \(n\) feature values, i.e., \(\mathbf{x}^{(i)}=\left[x_{1}^{(i)}x_{2}^{(i)}\cdots x_{n}^{(i)}\right]^{T}\), and \(\mathbf{y}^{(i)}\) is the corresponding category vector with \(m\) components, i.e., \(\mathbf{y}^{(i)}=\left[y_{1}^{(i)}y_{2}^{(i)}\cdots y_{m}^{(i)}\right]^{T}\), of pattern \(i\), defined as  
\[y_{f}^{(i)}=\left\{ \begin{array}{l}+1,\text{if }\mathbf{x}^{(i)}\text{ belong to category }f;\\ -1,\text{if }\mathbf{x}^{(i)}\text{ does not belong to category }f\end{array} \right. \tag{1}\]  
for \(1\leq f\leq m\). Note that \(m\) denotes the number of categories associated with the training dataset \(X\). For convenience, the categories are labelled as \(1\), \(2\), \(\cdots\), \(m\) respectively. For single-label classification, one of the elements in \(\mathbf{y}^{(i)}\) is \(+1\) and the other \(m\)\(-\)\(1\) elements are \(-1\). For a multi-label classification, two or more components in \(\mathbf{y}^{(i)}\) can be \(+1\). Pattern classification is concerned with constructing a predicting model from \(X\), and for any input vector \(\mathbf{x}\) the model can respond with output \(\overset{\wedge}{\mathbf{y}}\) showing which categories \(\mathbf{x}\) belongs to. In this work, we construct RBF networks as predicting models for classification. The data for training and testing the models will be collected from the openly accessible websites. The number of hidden nodes will be determined by clustering on the training data. The parameters of an RBF network will be learned by applying a hybrid learning algorithm. The performance of the constructed models will be analyzed by five-fold cross validation. Experiments for comparing performance with other methods will be conducted. Our RBF network architecture, shown in Figure 1, is a two-layer network consisting of one hidden layer and one output layer. There are \(n\) input nodes, each node receiving and feeding forward one feature value of the input vector to the hidden layer. The hidden layer has \(J\) nodes, each with a basis function as its activation function. Each input node is fully connected to every node in the hidden layer. The output layer has \(m\) nodes, each corresponding to one distinct category. The hidden and output layers are also fully connected, i.e., each node in the hidden layer is connected to each node in the output layer. The weight between node \(j\), \(1\leq j\leq J\), of the hidden layer and node \(f\), \(1\leq f\leq m\), of the output layer is denoted by \(w_{fj}\) for \(1\leq j\leq J\) and \(1\leq f\leq m\). Figure 1: Architecture of the proposed network model. When an input vector \(\mathbf{x}=\left[x_{1}x_{2}\cdots x_{n}\right]^{T}\) is presented to the input nodes of the network, the network produces the output vector \(\overset{\wedge}{\mathbf{y}}=\left[\hat{y}_{1}\hat{y}_{2}\cdots\hat{y}_{m} \right]^{T}\) at the output layer. The network operates as follows. * Input node \(\boldsymbol{i}\), \(\boldsymbol{1}\leq\boldsymbol{i}\leq\boldsymbol{n}\), takes \(\boldsymbol{x_{i}}\) and dispatches it to every node in the hidden layer. * Hidden layer. Hidden node \(\boldsymbol{j}\), \(\boldsymbol{1}\leq\boldsymbol{j}\leq\boldsymbol{J}\), is responsible for computing the value of its designated basis function as \[B_{j}(\mathbf{x})=\prod_{k=1}^{n}\exp\left[-\left(\frac{x_{k}-c_{k,j}}{ \alpha v_{k,j}}\right)^{2}\right]\] (2) where \(c_{k,j}\) and \(v_{k,j}\) denote the center and deviation, respectively, of the \(k\)th dimension. Note that \(\alpha\) is a constant for controlling the scope of the function. Let \[h_{j}=B_{j}(\mathbf{x})\] (3) which is forwarded to the output layer. * Output layer.