## 3 Fuzzy Information System with Tagaki-Sugeno Reasoning
### Takagi-Sugeno Reasoning with Optimization  
Consequently, for each rule premises, configuration of different _sigma_offset_ are considered. Additionally, in terms of first-order Takagi-Sugeno systems, two parameters defining the corresponding linear function for each fuzzy set in the rule premises should be calculated. The assumed varieties of type-2 fuzzy sets along with the _sigma_offset_ value and the linear function parameters define an optimization problem. A finite set of membership functions is considered, generated with respect to the _medium_ membership function. The assumed numbers of membership functions are: [3; 5; 7; 9; 11]. Considering the membership function configurations and the _sigma_offset_, the grid search algorithm in a k-fold cross-validation manner for these hyperparameters optimization was applied. The linear function parameters for each attribute, were optimized using the Covariance Matrix Adaptation Evolutionary Strategy (CMA-ES) [40], i.e., as each cross-validation step defines a new training set, the induced rules also differ. Therefore, at each cross-validation step, we applied CMA-ES for linear function parameters optimization. Additionally, in the conducted experiments as possible data pre-processing, a dimension reduction procedure using the well-known principal component analysis method (PCA) and the Random Oversampling (ROS) [41; 42; 43] to handle with class imbalance were applied. The oversampling was important in terms of rule induction, as the inconsistency elimination step ([22], Section 2.2.3, algorithm 1) strongly favors classes with large presence of data. The complete set of hyperparameters to be optimized is given in Table 5. On the other hand, Table 6 presents the hyperparameter values used in the CMA-ES procedure itself. The number of folds for each dataset are given in Table 7. \begin{table}
\begin{tabular}{l l} \hline \hline
**Hyperparameter** & **Value** \\ \hline Initialization & Random \\ \hline Maximum evaluations of fitness function & 20,000 \\ \hline Number of restarts & 1 \\ \hline Population size increase ratio after restart & 2 \\ \hline Standard deviation in each coordinate & 0.7 \\ \hline Population size & 20 \\ \hline Stagnation tolerance & 100 evaluations \\ \hline Parameters values range & [\(-400,400\)] \\ \hline \hline \end{tabular}
\end{table} 
Table 6: CMA-ES hyperparameters. \begin{table}
\begin{tabular}{l l} \hline \hline
**Hyperparameter** & **Value** \\ \hline Number of Gaussian functions & 3, 5, 7, 9, 11 \\ \hline Whether the std applied to Gaussians are the same & Yes/No \\ \hline If the mean value of the medium membership & \\ function is derived directly from mean value of the & Yes/No \\ corresponding feature & \\ \hline Sigma\_offset & [0.5, 0.9] with step of 0.05 \\ \hline Use PCA & Yes/No \\ \hline Use ROS & Yes/No \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Tuned hyperparameters of sets of membership functions. The experimental procedure pipeline in this research was defined as follows:  
1. Split a considered dataset into k-folds. 2. For each set of hyperparameters: 1. For each cross-validation step: 1. Set one fold as held-out for validation, use the rest for training. 2. Induce the knowledge base using the training set. 3. Infer the crisp value for each sample in the training set with the Takagi-Sugeno model. 4. Classify the sample with the threshold function set at 0. Treat the sample as negative if the crisp value is lower than 0. 5. Calculate the F1 metric. 6. Optimize the parameters of linear functions with CMA-ES, defining the F1 metric as the fitness function. 7. Evaluate the model on the validation set when the fitness function is optimized or the maximum number of evaluations passes. Otherwise return to (iii). 2. Choose a set of hyperparameters which maximize the mean value of the F1 metric over the validation sets. In order to evaluate the model for each dataset, we conduct the following procedure:  
1. Choose the best hyperparameters set for a dataset. 2. Optimize the fitness function on the training set. 3.