A statistical, self-organizing learning system with validation
In this paper, a new learning system called statistical, self-organizing learning system (SSOLS), which combines functional-link neural networks, statistical hypothesis testing and self-organization of number of enhancement nodes, is introduced. Its structure consists of two stages, a mapping stage and a learning stage. The input training vectors are initially mapped to the enhancement vectors in the mapping stage by multiplying with a random matrix, followed by pointwise nonlinear transformation. Starting with only one enhancement node, the enhancement layer incrementally adds an extra node in each iteration. In the learning stage, both the input vectors and the enhancement vectors are fed into a least squares learning module to obtain the estimated output vectors. This is made possible by choosing the output layer linear. The optimum dimension of the enhancement layer is determined by testing against a separate validation set. In this way, the number of enhancement nodes is also learned automatically. Two inclusions to the mapping stage, the t-test and the gradient descent update algorithm, further reduce the number of enhancement nodes required, resulting in a more compact network. The method is simple, fast to compute, and very suitable for many real-world applications.
self-organization validation hypothesis testing least squares learning t-test functional-link networks