## 4 Parsimonious Network Based on Fuzzy Inference System (PANFIS)
### Rule Growing Mechanism of PANFIS  
PANFIS uses the concept of datum significance (DS), which estimates the statistical contribution of data streams. A data point offering a high statistical contribution leads to a new rule. This concept is inspired by the neuron growing mechanism of GGAP-RBF [12] and SAFIS [6]. The key difference, however, can be found in the fact that PANFIS extends the statistical contribution concept to the framework of the multivariate Gaussian function. Suppose that a hypothetical rule (\(C+1\)) is created using the newest data point, the DS method can be formulated as follows:  
\[D_{C+1}=|e_{n}|\frac{\left(V_{C+1}\right)^{n}}{\sum\limits_{i=1}^{C+1}\left( V_{i}\right)^{n}} \tag{4}\]  
where \(e_{n}=|T_{n}-y_{n}|\) denotes the system error. \(T_{n},y_{n}\) refer to the target value and the predictive output at the \(n\)-th or current training observation, respectively. \(V_{i}\) denotes the volume of the \(i\)-th rule, which can be simply calculated as the determinant of the non-diagonal covariance matrix \(\det(\Sigma_{i})\). It is seen from (4) that a datum is considered to significantly contribute to the training process provided that it causes a high system error. On the other hand, the volume of a rule portrays its zone of influence in the input space and a cluster with a small influence zone potentially contributes little during its lifespan. Note that (4) contains the implicit distance information due to the initialization of the covariance matrix. This facet addresses the novelty of the hypothetical rule because the hypothetical rule occupying a remote region possesses a high coverage span. The hypothetical rule is confirmed as a new rule if it induces high statistical contribution \(D_{C+1}\geq g_{1}\), where \(g_{1}\) denotes a conflict threshold. The higher the values of the conflict threshold, the lower the number of fuzzy rules added; while the lower the values of the conflict threshold, the higher the number of fuzzy rules generated during the training process. If a new fuzzy rule is added, the center of the multivariate Gaussian function is set as the data sample of interest, while the diagonal element of inverse covariance matrix is set according to the \(\epsilon\)-completeness principle, as follows:  
\[C_{C+1}=X_{N} \tag{5}\]  
\[diag(\Sigma_{C+1}{}^{-1})=\frac{\max(|C_{i}-C_{i-1}|,|C_{i}-C_{i+1}|)}{\sqrt{ \ln(\frac{1}{\epsilon})}} \tag{6}\]  
where \(\epsilon\) is usually set at 0.6. The \(\epsilon\)-completeness principle borrows the seminal work of DFNN and GDFNN in [52] and [53]. It is said that there does not exist any data point with a membership degree less than \(\epsilon\) if this setting is implemented and has been mathematically confirmed. It is worth noting that the covariance matrix of the multivariate Gaussian function is vital to the success of PANFIS. Too large values lead to averaging, while too small values lead to overfitting. Another situation may occur during the training process where a data sample induces minor conflict. That is, the rule growing condition is not satisfied \(D_{C+1}<g_{1}\). This triggers the so-called rule premise adaptation phase since these samples remain important to refine the network structure. The original PANFIS utilizes the evolving self-organizing map (ESOM) to update the winning rule in this situation. The underlying bottleneck of the ESOM for PANFIS rule premise adaptation is seen in the reinversion requirement, which sometimes causes instability if the covariance matrix is not in the full-rank condition. To correct this shortcoming, the direct update scheme of GENEFIS [5], and GEN-SMART-EFS [4] is adopted.