Using fuzzy clustering with ellipsoidal units in neural networks for robust fault classification

Abstract The limitations of standard backpropagation networks on issues such as the robustness of fault classification and speed of network training have been pointed in recent investigations. Kavuri and Venkatasubramanian demonstrated that a network using ellipsoidal activation functions is a better choice for fault classification problems and that it has certain advantages over the standard backpropagation and radial basis function approaches. Ellipsoidal activation functions generate ellipsoids in the space of the network inputs. Since a mixture of ellipsoids form asymmetric bounded decision regions, they are better suited for fault diagnosis problems where the fault classes are bounded and arbitrarily shaped. Furthermore, ellipsoids indirectly use a distance measure, thus avoiding unintuitive and arbitrary classification of unspecified regions in the measurement space. In this paper, we address issues concerning the generation of ellipsoids, such as the number, location and size of these ellipsoids. The backpropagation algorithm does not guarantee the proper determination of the ellipsoids as it runs into local minima problems, attributable mainly to poor choices of initial network weights. To avoid this and related problems, we have developed a two-stage algorithm. In the first stage, a fuzzy clustering algorithm is used to determine the number of hidden nodes and the initial estimates for the hidden layer weights. The algorithm is demonstrated to determine a minimal number of hidden nodes. The algorithm also avoids having to determine the number of hidden nodes a priori . In the second stage, network weights are tuned to minimize RMS error in classification. Backpropagation is used in this stage for fine-tuning of ellipsoids initialized by the fuzzy clustering algorithm. The performance of the proposed approach on typical classification problems is presented. Ellipsoidal activation functions generate ellipsoids in the space of network inputs and are similar to Gaussian units. Since the ellipsoids form bounded decision regions, they are better suited for fault diagnosis problems where the fault classes, as specified through the training patterns, are bounded. Furthermore, they indirectly use a distance measure and thus avoid unintuitive and arbirtrary classification of unspecified regions in the measurement space. A two-stage algorithm of fuzzy clustering followed by weights tuning has been proposed to generate the ellipsoids for any given problem. The fuzzy clustering algorithm has been used as a means to: (a) avoid local minima traps by proposing a set of initial weights; and (b) determine a minimal number of hidden nodes. Through illustrations, it has been demonstrated that the clustering algorithm utilizes only a few cluster centers and leaves the rest unutilized. Use of fuzzy membership helps reduce gravity effects resulting in the utilization of very few clusters. This solves the problem of having to use a trial-and-error approach to determine the number of hidden nodes. The cluster data are then used to set up the initial ellipsoids and hence the network structure. This initial network is already quite close to the optimal structure and configuration. This network is then trained with a modified backpropagation algorithm to fine-tune the weights. Fine-tuning greatly enhances the classification results and has been shown to be desirable. Performance of this approach was evaluated for the figure and ground problem. This problem was considered because it is a difficult classification problem with the classes sharing an adjacent nonlinear boundary. For the figure and ground problem, it was seen that the tuning of the ellipsoids required only 400 epochs while greatly improving the classification results. Network training time was significantly reduced as a result of the clustering and the network decomposition that results from not sharing the hidden nodes.