# Pattern Classification Based on RBF Networks with Self-Constructing Clustering and Hybrid Learning
## 6 Experimental Results
### Discussions  
Our proposed approach includes two phases, network setup and parameter refinement. In the network setup phase, the iterative self-constructing clustering algorithm is used to produce \(J\) clusters. Assume that \(P_{1}\) iterations is performed. In each iteration, each \(n\)-dimensional instance is compared against each of the \(J\) existing clusters. Therefore, the complexity of this phase is \(O(nN|P_{1})\). In the parameter refinement phase, a hybrid learning, integrating the steepest descent and least squares methods, is applied to refine the parameters associated with the network. Let \(P_{2}\) be the number of iterations performed in this phase. The complexity of the least squares method is \(O(N|J^{2})\), while the complexity of steepest descent is \(O(nN|J)\). Therefore, the complexity of the parameter refinement phase is \(O(nN|P_{2}+N)^{2}P_{2})\). As a result, the total complexity of our approach is \(O(nN|P_{1}+nN|P_{2}+N)^{2}P_{2})\). We feel that it may not be appropriate to compare the performance from the speed viewpoint among different methods. The programs of the other existing methods are taken from different websites. They might be implemented in different languages or with different optimizing mechanisms. For support-vector based classifiers, e.g., LinearSVC and SVC-rbf, the quadratic programming solver takes a complexity between \(O(nN^{2})\) and \(O(nN^{3})\)[11; 59]. For NN with \(k\) hidden layers each having \(h\) neurons, the time complexity of backpropagation is \(O(nNmh^{k}P)\) where \(P\) is the number of iterations [60]. For KNN, the brute-force implementation takes a complexity of \(O(nN^{2})\). The computational time of each method is usually of the order of seconds for data sets of small or moderate size. High-dimensional datasets may cause data sparsity and computational intractability, leading to low forecasting accuracy of classification algorithms. Two strategies, correlation analysis and dimensionality reduction, can be employed to reduce the dimensionality of a dataset. By correlation analysis, the correlation, e.g., Pearson product-moment correlation coefficient [65], for each pair of variables, is calculated. Variables are sorted out according to their relevance to the target to be predicted. Four levels of correlation, uncorrelated, weakly correlated, moderately correlated, and strongly correlated, are defined [66]. It was shown in [67] that by ignoring weakly correlated or uncorrelated variables, forecasting accuracy can be improved. Principal component analysis (PCA) [68] is a popular method for dimensionality reduction. PCA reduces the dimensionality of a dataset while retaining the variation present in the dataset up to the maximum extent. This is done by transforming the original variables to a new set of variables known as the principal components. Dimensionality reduction is done by choosing a small number of principal components. Some strategies were proposed to determine an appropriate number of principal components. One common strategy chooses \(g\) principal components so that \(g\) is as small as possible and the cumulative energy is above a certain threshold. To address the scalability concern about \(N\), two methods--sampling [69] and clustering [70]--can be adopted. We may take a random sample, with size \(N_{s}\), from the \(N\) instances, such that \(N_{s}\) is smaller than \(N\). We may apply clustering to reduce the training set size. The \(N\) instances are grouped into clusters and the obtained clusters are used as the representatives for training the model. Another approach through exploiting the parallel or distributed facility of the computing system has long been adopted to address the scalability problem [71]. The \(N\) instances are divided and dispatched to different CPUs or computing nodes which can run simultaneously to improve the training efficiency of the classification task.