## 4 Neuro-Fuzzy Knowledge-Based System
### Simulation and Testing of Neuro-Fuzzy Knowledge-Based System  
Firstly, we evaluated the clustering performances of the SOMs by changing the data used for their training. An initial architecture was considered (SOM initial architecture). For this architecture, in the training we used the profiles obtained that were presented in Section 3.2. The SOM architecture was trained using weight and bias learning rules with batch updates. Weights and biases updates occurred at the end of an entire pass through the input data. Training stopped when the maximum number of epochs (repetitions) was reached. This number was changed to visualize the performances of the SOMs in this stage. When the training stopped, the SOM with the best parameters (topology and distance function) was selected as SOM1 (see Figure 2). In our case, the hexagonal topology and the link distance function had the best performances in clustering. On the other hand, for training we used the profiles obtained with the help of FCM clustering, an algorithm presented in Section 4.1. The SOM architecture, after training, became SOM2 (see Figure 2). The proposed neuro-fuzzy tool provided the possibility to evaluate and visualize the progress of the SOM network's performances for neighbor connections, neighbor distances, sample hits, and SOM weight positions. To evaluate these performances, in the testing phase, SOM1 and SOM2 received a new profile that was not considered in the parametrization or learning phases (Figure 7b). Figure 6: Neuro-fuzzy knowledge-based system graphical interface: (**a**) FCM panel (**b**) SOM panel. Figure 7a shows the SOM layer, with neurons as gray-blue patches, and their direct neighbor relations with red lines. The SOM had a two-dimensional topology, and the relationships among the cluster's centers can be visualized in the weight distance matrix (called U- matrix) (Figure 7b). The blue hexagons represent the neurons; the red lines connect neighboring neurons. The colors in the regions containing the red lines indicate the distances between neurons. The darker colors represent larger distances, and the lighter colors represent smaller distances. A band of dark segments crosses from the lower-right region to the upper-center region. The relative number of vectors for each neuron is shown via the size of a color patch (Figure 7c). It plots an SOM layer, with each neuron showing the number of input vectors classified. Figure 7d plots the input data along the weights. The input vectors are represented as green dots, and show how the SOM classified the input space by showing blue-grey dots for each neuron's weight vector, and connecting neighboring neurons with red lines. Since SOM training did not need a target output to be specified, unlike many other types of neural networks, as a metric of SOM performances is considered the winning node (best matching unit--BMU), which is calculated and displayed in green color in Figure 8.  
Figure 8: Best matching unit of SOM in the testing phase. Figure 7: SOM training clustering performances. Secondly, we compared the performances in clustering of SOM1, SOM2, and FIS. The data flow, described in the scenario presented above, was the same. In addition, we could visualize the progress of the FIS with the FCM clustering process as an objective function. At each iteration, the FCM algorithm minimized this function to find the best location for the clusters, and its values were returned in graphical format. The progress of FIS errors, in the training and testing phase, are presented in Figure 9a,b. The research allowed us to identify the strengths and the weaknesses of each clustering method used. Thus, the FCM and SOM tended to do well only for a small number of clusters and/or a fixed number of samples. Increasing the number of samples from 100 to 300 usually improved the error rate a little. The learning algorithms implemented in the SOMs proved to be poor for improving the error rate, from an RMSE = 0.65 to an RMSE = 0.50 when additional samples were used.