# A Hybrid Approach for Modular Neural Network Design Using Intercriteria Analysis and Intuitionistic Fuzzy Logic

Authors: Sotir Sotirov; Evdokia Sotirova; Vassia Atanassova; Krassimir Atanassov; Oscar Castillo; Patricia Melin; Todor Petkov; Stanimir Surchev
Journal: Complexity
 (2018)

---

## Abstract

Intercriteria analysis (ICA) is a new method, which is based on the concepts of index matrices and intuitionistic fuzzy sets, aiming at detection of possible correlations between pairs of criteria, expressed as coefficients of the positive and negative consonance between each pair of criteria. Here, the proposed method is applied to study the behavior of one type of neural networks, the modular neural networks (MNN), that combine several simple neural models for simplifying a solution to a complex problem. They are a tool that can be used for object recognition and identification. Usually the inputs of the MNN can be fed with independent data. However, there are certain limits when we may use MNN, and the number of the neurons is one of the major parameters during the implementation of the MNN. On the other hand, a high number of neurons can slow down the learning process, which is not desired. In this paper, we propose a method for removing part of the inputs and, hence, the neurons, which in addition leads to a decrease of the error between the desired goal value and the real value obtained on the output of the MNN. In the research work reported here the authors have applied the ICA method to the data from real datasets with measurements of crude oil probes, glass, and iris plant. The method can also be used to assess the independence of data with good results.


---

## Body

## 1. Introduction

One of the open and important questions in biology is the ability of biological systems to adapt to new environments, a concept termed evolvability [1]. A typical feature of evolvability is the fact that many biological systems have modularity; especially many biological processes and structures can be modeled as networks, such as metabolic pathways, gene regulation, protein interactions, and brains [1–5]. This feature has motivated important concepts in intelligent systems, such as modular neural network and evolutionary computation.Neural networks are considered modular if they are comprised of highly connected clusters of nodes that are connected to nodes in other clusters [4, 6, 7]. Despite importance and continuous research in this area, there is no agreement on why modular biological systems can evolve [4, 8, 9]. There is evidence that modular systems look more adaptable in nature [10] than the monolithic networks [11, 12]. Consequently, there are many papers dedicated to this problem, for example, the work in [12].In this paper, we introduce a hybrid combination between the intercriteria analysis (ICA, see [13–18]) method and modular neural network models. The ICA employs the apparatus of the intuitionistic fuzzy sets (IFS) for detecting possible correlations between pairs of criteria. Introduced in [19], IFSs are one of the extensions of Zadeh’s fuzzy sets [20]. In contrast to fuzzy sets, IFS [21–24] have two degrees: of membership (validity, etc.,  μA) and of nonmembership (nonvalidity, etc., νA), so that for each element x of the universe, over which an IFS A is defined, the following inequality is valid: 0≤μA(x)+νA(x)≤1. In this case, a pair a,b, where a,b,a+b∈[0,1], is called an intuitionistic fuzzy pair (IF pair). The ICA method produces the so-called positive and negative consonance coefficients between the different criteria used for evaluation of different objects.The main contribution of the paper is the proposed hybrid approach combining intuitionistic fuzzy logic (through the ICA method) with modular neural networks for designing a powerful neural model for classification. The neural network model is tested with benchmark problems and a real world case to show the advantages of the proposed approach. As regards existing works that could be considered similar to this one, we can mention that intuitionistic fuzzy logic has not been considered in conjunction with modular neural networks previously, so it can be considered an original contribution to the area of computational intelligence that combines the advantages of the two methods. For the purpose of testing the proposed method for preprocessing the information going into MNNs, we use data from the LUKOIL Neftochim Burgas AD from the measurements of a set of crude oil probes (objects, in terms of ICA) against a set of technological properties (criteria, in terms of ICA), which precedes and conditions the process of production of petrochemical products from the crude oil [25], dataset for iris plant [26], and glass types [27].The remainder of the paper is organized as follows. In Section2 some short remarks about the intercriteria analysis method are given, which is based on intuitionistic fuzzy logic. Section 3 describes basic concepts about modular neural networks. Section 4 describes the simulations and a discussion of the results. Finally, Section 5 offers the conclusions and outlines future work in this area.


## 2. Short Remarks on the Index Matrices and Intercriteria Analysis Method

As we mentioned above, the ICA method [13, 14] is based on two main concepts: intuitionistic fuzzy sets and index matrices. A brief description is offered below for completeness. Index matrices allow summarizing the criteria relevant to a particular decision making problem.LetI be a fixed set of indices and let R be the set of the real numbers. An index matrix (IM) with sets of indices K and L(K,L⊂I) is defined by (see [13])(1)
K
,
L
,
a
k
i
,
l
j
≡
l
1
l
2
⋯
l
n
k
1
a
k
1
,
l
1
a
k
1
,
l
2
⋯
a
k
1
,
l
n
k
2
a
k
2
,
l
1
a
k
2
,
l
2
⋯
a
k
2
,
l
n
⋮
⋮
⋮
⋱
⋮
k
m
a
k
m
,
l
1
a
k
m
,
l
2
⋯
a
k
m
,
l
n
,where K={k1,k2,…,km}, L={l1,l2,…,ln}, for 1≤i≤m, and 1≤j≤n:aki,lj∈R.For any two IMs, a series of relations, operations, and operators have been defined. The theory behind the IMs is described in a more detailed fashion in [13].Here, following the description of the ICA approach, given by [14], we will start with the IM called M with index sets with m rows {O1,…,Om} and n columns {C1,…,Cn}, where for every p, q(1≤p≤m,1≤q≤n), Op is an evaluated object, Cq is an evaluation criterion, and eOp,Cq is the evaluation of the pth object against the qth criterion, defined as a real number that is comparable according to relation R with all the remaining elements of the IM M.(2)
M
=
C
1
⋯
C
k
⋯
C
l
⋯
C
n
O
1
e
O
1
,
C
1
⋯
e
O
1
,
C
k
⋯
e
O
1
,
C
l
⋯
e
O
1
,
C
n
⋮
⋮
⋱
⋮
⋱
⋮
⋱
⋮
O
i
e
O
i
,
C
1
⋯
e
O
i
,
C
k
⋯
e
O
i
,
C
l
⋯
e
O
i
,
C
n
⋮
⋮
⋱
⋮
⋱
⋮
⋱
⋮
O
j
e
O
j
,
C
1
⋯
e
O
j
,
C
k
⋯
e
O
j
,
C
l
⋯
e
O
j
,
C
n
⋮
⋮
⋱
⋮
⋱
⋮
⋱
⋮
O
m
e
O
m
,
C
1
⋯
e
O
m
,
C
j
⋯
e
O
m
,
C
l
⋯
e
O
m
,
C
n
,From the requirement for comparability above, it follows that for eachi, j, k the relation R(eOi,Ck, eOj,Ck) holds. The relation R has a dual relation R-, which is true in the cases when the relation R is false, and vice versa. For instance, if R is “greater,” the dual relation R- is “less.”For the requirements of the proposed method, pairwise comparisons between every two different criteria are made along all evaluated objects. During the comparison, a counter is maintained for the number of times when the relationR holds, as well as another counter for the dual relation.LetSk,lμ be the number of cases in which the relations R(eOi,Ck,eOj,Ck) and R(eOi,Cl,eOj,Cl) are simultaneously satisfied. Let also Sk,lν be the number of cases in which the relations R(eOi,Ck, eOj,Ck) and the dual R-=(eOi,Cl,eOj,Cl) are simultaneously satisfied. As the total number of pairwise comparisons between the objects is given by m(m–1)/2, it can be verified that the following inequalities hold:(3)0≤Sk,lμ+Sk,lν≤mm-12.For everyk, l, such that 1≤k≤l≤n and for m≥2 two numbers are defined:(4)μCk,Cl=2Sk,lμmm-1,νCk,Cl=2Sk,lνmm-1.The pair constructed from these two numbers plays the role of the intuitionistic fuzzy evaluation of the relations that can be established between any two criteriaCk and Cl. In this way, the IM M that relates evaluated objects with evaluating criteria can be transformed to another IM M∗ that gives the relations detected among the criteria, where stronger correlation exists where the first component μCk,Cl is higher while the second component νCk,Cl is lower.(5)
M
∗
=
C
1
⋯
C
n
C
1
μ
C
1
,
C
1
,
ν
C
1
,
C
1
⋯
μ
C
1
,
C
n
,
ν
C
1
,
C
n
⋯
⋯
⋯
⋯
C
n
μ
C
n
,
C
1
,
ν
C
n
,
C
1
⋯
μ
C
n
,
C
n
,
ν
C
n
,
C
n
.From practical considerations, it has been more flexible to work with two IMsMμ and Mν, rather than with the IM M∗ of IF pairs. IM Mμ contains as elements the first components of the IFPs of M∗, while Mν contains the second components of the IFPs of M∗. Once the intercriteria pairs have been calculated, for example, using the software described in [28], the question arises about defining the thresholds against which the membership and the nonmembership parts are evaluated [29, 30].As has been discussed in some publications on ICA, for example, in [31, 32], the ICA results are very close to those obtained with the correlation analyses of Spearman, Pearson, and Kendall. It is worth noting the so far empirically observed fact that when in the data there are mistakes (e.g., shift of the decimal separator) these three correlation analyses give a larger deviation of the value than ICA; that is, ICA is less sensitive, so the use of them together can be used as a way of detecting errors in the input data.


## 3. Modular Neural Networks

Modular neural networks [33, 34] are one of the models that can be used for object recognition, classification, and identification (see Figure 1). A modular neural network can be viewed as a set of monolithic neural networks [35–37] that deal with a part of a problem, and then their individual outputs are combined by an integration unit to form a global solution to the complete problem. The main idea is that a complex problem can be divided into simpler subproblems that can be solved by simpler neural networks and then the total solution will be a combination of the outputs of the simple monolithic neural networks.Figure 1
The structure of the MNN.In the proposed hybrid approach each of the MNN Modules takes as its input the result of the application of ICA method over three datasets (dataset for production of petrochemical products from the crude oil [25], iris plants [26], and glass types [27]). Every module is a two-layer Multilayer Perceptron and the output of the second layer of the ANN is ai2 (for i∈{1,…,n}), where n is the maximal number of modules.The outputO is calculated according to the following equation, which basically performs a weighted integration of the module outputs:(6)O=∑i=1nai2giki∑i=1ngi,where ai2 is output of the module i∈[1,2,…,n]; gi is average deviation of the output values of the module i; ki is coefficient of existence of module i. The coefficient of existence of every module shows the presence/absence of the respective module, with respect to the need of that module in the particular case.For illustration purposes and simplifying calculations, we can choose to reduce the structure of the MNN and use 2 modules from the structure from Figure2 (k1=1,k2=1,k3=0,…,kn=0).Figure 2
The structure of the MNN with 2 modules.The first module considers all independent inputs (weak dissonance, dissonance, and strong dissonance). The second module takes the inputs that have strong negative consonance, negative consonance, and weak negative consonance, and weak positive consonance, positive consonance, and strong positive consonance.In the second module, we can reduce some of the inputs if they have very strong positive consonance. In this case, we can remove one of the inputs. In the case that they have strong negative consonance, this means that we can also remove some of the inputs. Other configurations for the modular neural network are possible, depending on how the inputs are selected.


## 4. Discussion of Results

For verifying the accuracy of the proposed method the data of the real dataset with measurements of crude oil probes, glass, and iris plants were used.


### 4.1. Testing with Data for Crude Oil Probes

For the learning process of MNN, we set the following parameters: performance (MSE) = 0.00001; validation check = 15. The dataset was divided into three different parts: training (from 1 to 100); validation (from 101 to 120), and testing (from 121 to 140). For the purpose of learning algorithms, one of the variants of the backpropagation algorithm is used, namely, Levenberg-Marquardt. As a target for the model, the values from the databases were used.In the MNN, the data for 140 crude oil probes, measured against 8 physical properties (“criteria” in the ICA terminology) were used (for the input data, see [34]). The eight criteria are as follows: (1) density at 15°C g/cm3; (2) 10% (v/v) ASTM D86 distillation, °C; (3) 50%  (v/v) ASTM D86 distillation, °C; (4) 90%  (v/v) ASTM D86 distillation, °C; (5) refractive index at 20°C; (6) H2 content, % (m/m); (7) aniline point, °C; (8) molecular weight g/mol. Using the ICA approach, we are seeking for correlations between these eight criteria on the basis of the 140 crude oil probes (“objects” in ICA). Using the ICA software [28] applied to the whole 140 × 8 matrix with measurements, we obtain 28 IF pairs, giving the pairwise correlations between the eight criteria. For easier processing of the result of ICA application, the output is given in the form of two IMs, containing the membership and the nonmembership parts of the IF correlations discovered between each pair of criteria (Tables 1 and 2), which in Table 3 are sorted in descending order according to the membership part of the intercriteria pairs, from strong positive consonance (i.e., pair 1-5 with degree 0.956;0.042), through dissonance, to negative consonance (i.e., pair 5-6 with degree 0.134;0.866).Table 1
Membership parts of the intuitionistic fuzzy pairs, giving the intercriteria correlations between measurements of crude oil probes.
μ 1 2 3 4 5 6 7 8 1 1.000 0.699 0.770 0.658 0.956 0.176 0.446 0.703 2 0.699 1.000 0.787 0.597 0.676 0.408 0.640 0.775 3 0.770 0.787 1.000 0.777 0.728 0.394 0.665 0.921 4 0.658 0.597 0.777 1.000 0.627 0.468 0.674 0.771 5 0.956 0.676 0.728 0.627 1.000 0.134 0.404 0.661 6 0.176 0.408 0.394 0.468 0.134 1.000 0.730 0.473 7 0.446 0.640 0.665 0.674 0.404 0.730 1.000 0.743 8 0.703 0.775 0.921 0.771 0.661 0.473 0.743 1.000Table 2
Nonmembership parts of the intuitionistic fuzzy pairs, giving the intercriteria correlations between measurements of crude oil probes.
ν 1 2 3 4 5 6 7 8 1 0.000 0.288 0.217 0.326 0.042 0.822 0.552 0.295 2 0.288 0.000 0.204 0.391 0.312 0.580 0.348 0.213 3 0.217 0.204 0.000 0.212 0.261 0.595 0.325 0.068 4 0.326 0.391 0.212 0.000 0.359 0.518 0.312 0.215 5 0.042 0.312 0.261 0.359 0.000 0.866 0.596 0.339 6 0.822 0.580 0.595 0.518 0.866 0.000 0.270 0.527 7 0.552 0.348 0.325 0.312 0.596 0.270 0.000 0.257 8 0.295 0.213 0.068 0.215 0.339 0.527 0.257 0.000Table 3
Relations among the criteria with specific values.
IF pair Criteria Type of intercriteria relation 0.956 ; 0.042 1-5 Strong positive consonance 0.921 ; 0.068 3-8 Positive consonance 0.787;0.204 2-3 Weak positive consonance 0.777 ; 0.212 3-4 Weak positive consonance 0.771 ; 0.215 4-8 Weak positive consonance 0.770 ; 0.217 1-3 Weak positive consonance 0.743 ; 0.257 7-8 Weak dissonance 0.730 ; 0.270 6-7 Weak dissonance 0.728 ; 0.261 3-5 Weak dissonance 0.703 ; 0.295 1-8 Weak dissonance 0.699 ; 0.288 1-2 Weak dissonance 0.676 ; 0.312 2-5 Weak dissonance 0.674 ; 0.312 4-7 Weak dissonance 0.665 ; 0.325 3-7 Dissonance 0.661 ; 0.339 5-8 Dissonance 0.658 ; 0.326 1-4 Dissonance 0.640 ; 0.348 2-7 Dissonance 0.627 ; 0.359 4-5 Dissonance 0.597 ; 0.391 2-4 Dissonance 0.473 ; 0.527 6-8 Strong dissonance 0.468 ; 0.518 4-6 Strong dissonance 0.446 ; 0.552 1-7 Strong dissonance 0.408 ; 0.580 2-6 Dissonance 0.404 ; 0.596 5-7 Dissonance 0.394 ; 0.595 3-4 Dissonance 0.394 ; 0.594 3-6 Dissonance 0.176 ; 0.822 1-6 Weak negative consonance 0.134 ; 0.866 5-6 Negative consonanceOn this basis, we separate these 28 intercriteria pairs in two groups, whereModule 1 of the MNN takes as input the dissonant pairs, andModule 2 takes as input the consonant pairs, either positive or negative. In this way, we aim to reduce the number of input parameters of the MNN, yet keeping high enough level of precision.In Table4 we are presenting the values from 22 neural network simulations and the identifiers of criteria that we provide at the input on each module. These 22 simulations are all the possible combinations of criteria, whereModule 2 obtains as input at least one criterion, starting with all of the consonant criteria (1, 3, 5, 6, and 8 in row 1) down to at least one of the consonant criteria (1, 3, 5, 6, or 8, in rows 18–22). Based on the values from Table 3, we offer the use of the relations of positive consonance and negative consonance, since both affect in the same way the inputs of the neural network. The parameters g1, g2, and g3 are the average deviations of the output values ofModule 1,Module 2, and whole neural network, respectively. A detailed discussion of the results follows in Section 4.4.Table 4
Results from the simulations.
Number Description Number of the inputs g 1 g 2 g 3 Number of weight coefficients 1 Module 1—2, 4, and 7Module 2—1, 3, 5, 6, and 8 8 2.2971 2.1991 2.2330 315 2 Module 1—2, 4, and 7Module 2—3, 5, 6, and 8 (without 1) 7 2.2139 2.1650 2.1562 280 3 Module 1—2, 4, and 7Module 2—1, 5, 6, and 8 (without 3) 7 2.2023 2.1498 2.1672 280 4 Module 1—2, 4, and 7Module 2—1, 3, 6, and 8 (without 5) 7 2.2032 2.1559 2.1564 280 5 Module 1—2, 4, and 7Module 2—1, 3, 5, and 8 (without 6) 7 2.1352 2.2455 2.1593 280 6 Module 1—2, 4, and 7Module 2—1, 3, 5, and 6 (without 8) 7 2.1703 2.1578 2.1638 280 7 Module 1—2, 4, and 7Module 2—5, 6, and 8 (without 1 and 3) 6 2.1844 2.1196 2.1457 245 8 Module 1—2, 4, and 7Module 2—1, 3, and 6 (without 5 and 8) 6 2.1881 2.1500 2.1572 245 9 Module 1—2, 4, and 7Module 2—1, 6, and 8 (without 3 and 5) 6 2.2057 2.2465 2.1572 245 10 Module 1—2, 4, and 7Module 2—3, 5, and 6 (without 1 and 8) 6 2.1715 2.1531 2.1437 245 11 Module 1—2, 4, and 7Module 2—3, 6, and 8 (without 1 and 5) 6 2.1679 2.1536 2.1370 245 12 Module 1—2, 4, and 7Module 2—1, 3, and 5 (without 6 and 8) 6 2.2181 2.1415 2.1398 245 13 Module 1—2, 4, and 7Module 2—1, 3, and 8 (without 5 and 6) 6 2.1883 2.1624 2.1551 245 14 Module 1—2, 4, and 7Module 2—3 and 6 (without 1, 5, and 8) 5 2.1478 2.1885 2.1485 210 15 Module 1—2, 4, and 7Module 2—3 and 5 (without 1, 6, and 8) 5 2.1542 2.1210 2.0914 210 16 Module 1—2, 4, and 7Module 2—1 and 3 (without 5, 6, and 8) 5 2.1436 2.2551 2.1580 210 17 Module 1—2, 4, and 7Module 2—3 and 8 (without 1, 5, and 6) 5 2.1777 2.2327 2.1981 210 18 Module 1—2, 4, and 7Module 2—1 (without 3, 5, 6, and 8) 4 2.2006 7.0510 5.6472 210 19 Module 1—2, 4, and 7Module 2—3 (without 1, 5, 6, and 8) 4 2.1554 6.6392 5.2857 175 20 Module 1—2, 4, and 7Module 2—5 (without 1, 3, 6, and 8) 4 2.1856 6.8357 5.4230 175 21 Module 1—2, 4, and 7Module 2—6 (without 1, 3, 5, and 8) 4 2.1873 3.6355 2.7732 175 22 Module 1—2, 4, and 7Module 2—8 (without 1, 3, 5, and 8) 4 2.2109 6.1718 4.9165 175


### 4.2. Testing with Data for Iris Plants

As another test case for the proposed method, we also consider the dataset [26] that contains four parameters (criteria): (1) sepal length in cm, (2) sepal width in cm, (3) petal length in cm, and (4) petal width in cm. There are 3 classes of 50 instances each [26], where each class refers to a type of iris plant. The targets areIris setosa, 1,Iris versicolor, 2, andIris virginica, 3.The ICA method was applied to the 150 × 4 matrix in the same fashion as previously (see Tables5, 6, and 7), and the results are given in Table 5, sorted in descending order according to the membership part of the intercriteria pairs.Table 5
Relations among the criteria.
IF pair Criteria Type of intercriteria relation 0.843;0.538 3-4 Weak positive consonance 0.819;0.504 1-3 Weak positive consonance 0.764;0.486 1-4 Weak positive consonance 0.416;0.139 1-2 Dissonance 0.366;0.121 2-4 Dissonance 0.364;0.070 2-3 DissonanceTable 6
Membership parts of the intuitionistic fuzzy pairs, giving the intercriteria correlations between measurements of iris plant.
μ 1 2 3 4 1 1.000 0.416 0.819 0.764 2 0.416 1.000 0.364 0.366 3 0.819 0.364 1.000 0.843 4 0.764 0.366 0.843 1.000Table 7
Nonmembership parts of the intuitionistic fuzzy pairs, giving the intercriteria correlations between measurements of iris plant.
ν 1 2 3 4 1 0.000 0.486 0.121 0.139 2 0.486 0.000 0.538 0.504 3 0.121 0.538 0.000 0.070 4 0.139 0.504 0.070 0.000In the neural network inputs were assumed with the following parameters: parameter 2 (in the first neural network input) and parameters 1, 3, and 4 in consonance (in the second neural network input). In Table8 the values from 7 neural network simulations are presented and a number of parameters that we defined for each module are described.Table 8
Results from the simulations.
Number Description Number of the inputs g 1 g 2 g 3 Number of weight coefficients 1 Module 1—2Module 2—1, 3, 4 4 0.5725 0.0084 0.5726 175 2 Module 1—2Module 2—3, 4 (without 1) 3 0.5725 0.0321 0.5749 140 3 Module 1—2Module 2—1, 4 (without 3) 3 0.5725 0.6667 1.2898 140 4 Module 1—2Module 2—1, 3 (without 4) 3 0.5725 0.0527 0.5793 140 5 Module 1—2Module 2—4 (without 1, 3) 2 0.5725 0.0631 0.5846 105 6 Module 1—2Module 2—3 (without 1, 4) 2 0.5725 0.0807 0.5907 105 7 Module 1—2Module 2—1 (without 3, 4) 2 0.5725 0.6667 1.2898 105


### 4.3. Testing with Data for Glass

Finally, as a third case for testing the proposed approach we consider the benchmark glass data. In this case, in the input of this modular neural network, we use the experimental data from [27] for obtaining the type of glass. We work with data for 214 building and vehicle window glasses, measured against 9 criteria: (1) RI: refractive index; (2) Na: sodium (unit measurement: weight percent in corresponding oxide); (3) Mg: magnesium; (4) Al: aluminium; (5) Si: silicon; (6) K: potassium; (7) Ca: calcium; (8) Ba: barium; (9) Fe: iron. In the output (as a target) we give information about the glass type.As was done in the previous sections, the input parameters (criteria) are divided in two groups: parameters in dissonance (parameters 2, 3, 4, 5, 6, and 9) and parameters in consonance (see Table9), with the results sorted in descending order according to the membership part of the intercriteria pairs.Table 9
Relations among the criteria.
IF pair Criteria Type of intercriteria relation 0.760 ; 0.234 1-7 Weak positive consonance 0.564;0.423 4-5 Dissonance 0.547 ; 0.409 4-6 Strong dissonance 0.532 ; 0.455 2-4 Strong dissonance 0.532 ; 0.404 3-6 Strong dissonance 0.527 ; 0.425 1-3 Strong dissonance 0.513 ; 0.481 1-2 Strong dissonance 0.497 ; 0.493 2-7 Strong dissonance 0.488 ; 0.470 5-6 Strong dissonance 0.441 ; 0.509 2-3 Strong dissonance 0.419 ; 0.571 5-7 Dissonance 0.410 ; 0.090 8-9 Dissonance 0.384 ; 0.604 4-7 Dissonance 0.383 ; 0.607 2-6 Dissonance 0.371 ; 0.579 3-7 Dissonance 0.368 ; 0.594 1-6 Dissonance 0.353 ; 0.596 3-5 Dissonance 0.318 ; 0.672 1-4 Weak dissonance 0.316 ; 0.237 6-9 Weak dissonance 0.314 ; 0.645 6-7 Weak dissonance 0.311 ; 0.236 3-9 Weak dissonance 0.304 ; 0.240 7-9 Weak dissonance 0.300 ; 0.693 1-5 Weak dissonance 0.299 ; 0.245 1-9 Weak dissonance 0.288 ; 0.659 3-4 Weak dissonance 0.272 ; 0.057 4-8 Weak dissonance 0.262 ; 0.697 2-6 Weak dissonance 0.257 ; 0.070 2-8 Weak dissonance 0.253 ; 0.291 4-9 Weak dissonance 0.253 ; 0.291 5-9 Weak dissonance 0.210 ; 0.334 2-9 Weak negative consonance 0.203 ; 0.123 5-8 Weak negative consonance 0.162 ; 0.164 7-8 Weak negative consonance 0.123 ; 0.202 1-8 Negative consonance 0.113 ; 0.213 6-8 Negative consonance 0.059 ; 0.249 3-8 Negative consonanceIn Table10, the values from 7 neural network simulations are presented with the respective identifiers of the criteria that are feeding the inputs of each module.Table 10
Results from the simulations.
N: Description Number of the inputs g 1 g 2 g 3 Number of weight coefficients 1 Module 1—2, 3, 4, 5, 6, 9Module 2—1, 7, 8 9 0.7658 0.8796 1.6721 350 2 Module 1—2, 3, 4, 5, 6, 9Module 2—7, 8 (without 1) 8 0.7658 0.8696 1.6584 315 3 Module 1—2, 3, 4, 5, 6, 9Module 2—1, 8 (without 7) 8 0.7658 0.9277 1.7492 315 4 Module 1—2, 3, 4, 5, 6, 9 Module 2—1, 7 (without 8) 8 0.7658 1.5826 2.8488 315 5 Module 1—2, 3, 4, 5, 6, 9Module 2—8 (without 1, 7) 7 0.7658 0.8756 1.6626 280 6 Module 1—2, 3, 4, 5, 6, 9Module 2—7 (without 1, 8) 7 0.7658 1.7190 3.0898 280 7 Module 1—2, 3, 4, 5, 6, 9Module 2—1 (without 7, 8) 7 0.7658 1.6378 2.9400 280The information is divided into two groups: the first part is the group of independent evaluation criteria (see Tables11 and 12): criteria 2, 4, and 7 (see Table 4 for the crude oil probes data), criterion 2 (see Table 5 for the iris plant data), and criteria 2, 3, 4, 5, 6, and 9 (see Table 8 for the glass types data). These data were used on the inputs of the first module of the neural networks. The other criteria (dependent parameters) were considered on the inputs of the second module of the neural networks.Table 11
Membership parts of the intuitionistic fuzzy pairs, giving the intercriteria correlations of glass.
μ 1 2 3 4 5 6 7 8 9 1 1.000 0.513 0.527 0.318 0.300 0.368 0.760 0.123 0.299 2 0.513 1.000 0.441 0.532 0.383 0.262 0.497 0.257 0.210 3 0.527 0.441 1.000 0.288 0.353 0.532 0.371 0.059 0.311 4 0.318 0.532 0.288 1.000 0.564 0.547 0.384 0.272 0.253 5 0.300 0.383 0.353 0.564 1.000 0.488 0.419 0.203 0.253 6 0.368 0.262 0.532 0.547 0.488 1.000 0.314 0.113 0.316 7 0.760 0.497 0.371 0.384 0.419 0.314 1.000 0.162 0.304 8 0.123 0.257 0.059 0.272 0.203 0.113 0.162 1.000 0.410 9 0.299 0.210 0.311 0.253 0.253 0.316 0.304 0.410 1.000Table 12
Nonmembership parts of the intuitionistic fuzzy pairs, giving the intercriteria correlations.
ν 1 2 3 4 5 6 7 8 9 1 0.000 0.481 0.425 0.672 0.693 0.594 0.234 0.202 0.245 2 0.481 0.000 0.509 0.455 0.607 0.697 0.493 0.070 0.334 3 0.425 0.509 0.000 0.659 0.596 0.404 0.579 0.249 0.236 4 0.672 0.455 0.659 0.000 0.423 0.409 0.604 0.057 0.291 5 0.693 0.607 0.596 0.423 0.000 0.470 0.571 0.123 0.291 6 0.594 0.697 0.404 0.409 0.470 0.000 0.645 0.213 0.237 7 0.234 0.493 0.579 0.604 0.571 0.645 0.000 0.164 0.240 8 0.202 0.070 0.249 0.057 0.123 0.213 0.164 0.000 0.090 9 0.245 0.334 0.236 0.291 0.291 0.237 0.240 0.090 0.000The inputs on the first module are not removable, because the ICA approach tests the independence of the criteria. The results from applying the ICA method over tree types of data show that most of the parameters are selected in a proper way (criteria in dissonance). For some of the parameters that were provided on the inputs of the second module, an approach for reasonable elimination of some of these criteria can be adopted.


### 4.4. Discussion

In the first step we use neural networks without any removal of information and without removing of the inputs. The corresponding results can be found on the first rows in Tables4, 8, and 10.On the second step the data that have the highest and lowest consonance coefficients (strong positive consonance, positive consonance, strong negative consonance, and negative consonance) were removed. For example, in Table4 we use parameters 2, 4, and 7 inModule 1 and parameters 3, 5, 6, and 8 (without 1) inModule 2. In the study, parameter 1 is removed due to the high value of membership coefficient μ=0.956 (between parameters 1 and 5). After simulating the neural network an average deviation of the MNN g3=2.1562 (g1=2.2139, g2=2.1650) is obtained.The removed parameters do not have substantial influence on the result, because of decreasing the number of the weight coefficients, and along with this the error of the output values also decreases.In the same way the second group of parameters (from the pairs) were successively removed. This process continues until finishing the pairs in the highest and lowest consonance.The next step is to remove two parameters, for example, in row 5 (Table8) the situation with all parameters onModule 1, and parameter 4 (without 1 and 3) inModule 2 was presented. In the study, parameters 1 and 3 are removed due to the high value of membership coefficient μ=0.81852 (between parameters 1 and 3). After simulation of the neural network, an average deviation of the MNN g3=0.5846 (g1=0.5725, g2=0.0631) is obtained. The removed parameters 1 and 3 do not have substantial influence on the result, because of decreasing the number of the weight coefficients, and along with this the error of the output values also decreases.The process can continue iteratively in the same way. In summary, a new approach for building modular neural networks based on ICA, which uses intuitionistic fuzzy logic, has been proposed and tested with three benchmark databases. The simulation results are good, in this way verifying the advantages of the hybrid method.In Table13 comparative analysis of the results of the three different datasets is made. The table includes a short description of each dataset, the initial number of inputs (number of the inputs before reduction), the number of inputs after reduction, and average deviation of the output values of the neural networks (g3), as well as minimal value of average deviation of the output values of the neural networks (g3, min), the number of weight coefficients without reduction and with reduction, the percentage ratio between the average deviation of the output values of the neural networks and the minimal value of average deviation of the output values of the neural networks, respectively, and the number of weight coefficients without and with reduction.Table 13
Results from the simulations.
Number Description Number of the inputs before reduction Number of the inputs after reduction g 3 g 3 min Number of weight coefficientswithout reduction Number of weight coefficientswith reduction g 3 % Number of weight coefficients% 1 Data with parameters of crude oil 8 5 2.2330 2.0914 315 210 93,65 66,67 2 Data of iris plant parameters 4 2 0.5726 0.5446 175 105 95,11 60 3 Data of glass parameters 9 7 1.6721 1.6626 350 280 98,2 80From the data in the table, it is seen that reducing the number of input parameters and, respectively, the number of inputs of the MNN, the value of average deviation of the output values of the MNN decreases (93,65%–98,2%). On this basis, we further observe decrease of the number of weight coefficients (60%–80%).


## 4.1. Testing with Data for Crude Oil Probes

For the learning process of MNN, we set the following parameters: performance (MSE) = 0.00001; validation check = 15. The dataset was divided into three different parts: training (from 1 to 100); validation (from 101 to 120), and testing (from 121 to 140). For the purpose of learning algorithms, one of the variants of the backpropagation algorithm is used, namely, Levenberg-Marquardt. As a target for the model, the values from the databases were used.In the MNN, the data for 140 crude oil probes, measured against 8 physical properties (“criteria” in the ICA terminology) were used (for the input data, see [34]). The eight criteria are as follows: (1) density at 15°C g/cm3; (2) 10% (v/v) ASTM D86 distillation, °C; (3) 50%  (v/v) ASTM D86 distillation, °C; (4) 90%  (v/v) ASTM D86 distillation, °C; (5) refractive index at 20°C; (6) H2 content, % (m/m); (7) aniline point, °C; (8) molecular weight g/mol. Using the ICA approach, we are seeking for correlations between these eight criteria on the basis of the 140 crude oil probes (“objects” in ICA). Using the ICA software [28] applied to the whole 140 × 8 matrix with measurements, we obtain 28 IF pairs, giving the pairwise correlations between the eight criteria. For easier processing of the result of ICA application, the output is given in the form of two IMs, containing the membership and the nonmembership parts of the IF correlations discovered between each pair of criteria (Tables 1 and 2), which in Table 3 are sorted in descending order according to the membership part of the intercriteria pairs, from strong positive consonance (i.e., pair 1-5 with degree 0.956;0.042), through dissonance, to negative consonance (i.e., pair 5-6 with degree 0.134;0.866).Table 1
Membership parts of the intuitionistic fuzzy pairs, giving the intercriteria correlations between measurements of crude oil probes.
μ 1 2 3 4 5 6 7 8 1 1.000 0.699 0.770 0.658 0.956 0.176 0.446 0.703 2 0.699 1.000 0.787 0.597 0.676 0.408 0.640 0.775 3 0.770 0.787 1.000 0.777 0.728 0.394 0.665 0.921 4 0.658 0.597 0.777 1.000 0.627 0.468 0.674 0.771 5 0.956 0.676 0.728 0.627 1.000 0.134 0.404 0.661 6 0.176 0.408 0.394 0.468 0.134 1.000 0.730 0.473 7 0.446 0.640 0.665 0.674 0.404 0.730 1.000 0.743 8 0.703 0.775 0.921 0.771 0.661 0.473 0.743 1.000Table 2
Nonmembership parts of the intuitionistic fuzzy pairs, giving the intercriteria correlations between measurements of crude oil probes.
ν 1 2 3 4 5 6 7 8 1 0.000 0.288 0.217 0.326 0.042 0.822 0.552 0.295 2 0.288 0.000 0.204 0.391 0.312 0.580 0.348 0.213 3 0.217 0.204 0.000 0.212 0.261 0.595 0.325 0.068 4 0.326 0.391 0.212 0.000 0.359 0.518 0.312 0.215 5 0.042 0.312 0.261 0.359 0.000 0.866 0.596 0.339 6 0.822 0.580 0.595 0.518 0.866 0.000 0.270 0.527 7 0.552 0.348 0.325 0.312 0.596 0.270 0.000 0.257 8 0.295 0.213 0.068 0.215 0.339 0.527 0.257 0.000Table 3
Relations among the criteria with specific values.
IF pair Criteria Type of intercriteria relation 0.956 ; 0.042 1-5 Strong positive consonance 0.921 ; 0.068 3-8 Positive consonance 0.787;0.204 2-3 Weak positive consonance 0.777 ; 0.212 3-4 Weak positive consonance 0.771 ; 0.215 4-8 Weak positive consonance 0.770 ; 0.217 1-3 Weak positive consonance 0.743 ; 0.257 7-8 Weak dissonance 0.730 ; 0.270 6-7 Weak dissonance 0.728 ; 0.261 3-5 Weak dissonance 0.703 ; 0.295 1-8 Weak dissonance 0.699 ; 0.288 1-2 Weak dissonance 0.676 ; 0.312 2-5 Weak dissonance 0.674 ; 0.312 4-7 Weak dissonance 0.665 ; 0.325 3-7 Dissonance 0.661 ; 0.339 5-8 Dissonance 0.658 ; 0.326 1-4 Dissonance 0.640 ; 0.348 2-7 Dissonance 0.627 ; 0.359 4-5 Dissonance 0.597 ; 0.391 2-4 Dissonance 0.473 ; 0.527 6-8 Strong dissonance 0.468 ; 0.518 4-6 Strong dissonance 0.446 ; 0.552 1-7 Strong dissonance 0.408 ; 0.580 2-6 Dissonance 0.404 ; 0.596 5-7 Dissonance 0.394 ; 0.595 3-4 Dissonance 0.394 ; 0.594 3-6 Dissonance 0.176 ; 0.822 1-6 Weak negative consonance 0.134 ; 0.866 5-6 Negative consonanceOn this basis, we separate these 28 intercriteria pairs in two groups, whereModule 1 of the MNN takes as input the dissonant pairs, andModule 2 takes as input the consonant pairs, either positive or negative. In this way, we aim to reduce the number of input parameters of the MNN, yet keeping high enough level of precision.In Table4 we are presenting the values from 22 neural network simulations and the identifiers of criteria that we provide at the input on each module. These 22 simulations are all the possible combinations of criteria, whereModule 2 obtains as input at least one criterion, starting with all of the consonant criteria (1, 3, 5, 6, and 8 in row 1) down to at least one of the consonant criteria (1, 3, 5, 6, or 8, in rows 18–22). Based on the values from Table 3, we offer the use of the relations of positive consonance and negative consonance, since both affect in the same way the inputs of the neural network. The parameters g1, g2, and g3 are the average deviations of the output values ofModule 1,Module 2, and whole neural network, respectively. A detailed discussion of the results follows in Section 4.4.Table 4
Results from the simulations.
Number Description Number of the inputs g 1 g 2 g 3 Number of weight coefficients 1 Module 1—2, 4, and 7Module 2—1, 3, 5, 6, and 8 8 2.2971 2.1991 2.2330 315 2 Module 1—2, 4, and 7Module 2—3, 5, 6, and 8 (without 1) 7 2.2139 2.1650 2.1562 280 3 Module 1—2, 4, and 7Module 2—1, 5, 6, and 8 (without 3) 7 2.2023 2.1498 2.1672 280 4 Module 1—2, 4, and 7Module 2—1, 3, 6, and 8 (without 5) 7 2.2032 2.1559 2.1564 280 5 Module 1—2, 4, and 7Module 2—1, 3, 5, and 8 (without 6) 7 2.1352 2.2455 2.1593 280 6 Module 1—2, 4, and 7Module 2—1, 3, 5, and 6 (without 8) 7 2.1703 2.1578 2.1638 280 7 Module 1—2, 4, and 7Module 2—5, 6, and 8 (without 1 and 3) 6 2.1844 2.1196 2.1457 245 8 Module 1—2, 4, and 7Module 2—1, 3, and 6 (without 5 and 8) 6 2.1881 2.1500 2.1572 245 9 Module 1—2, 4, and 7Module 2—1, 6, and 8 (without 3 and 5) 6 2.2057 2.2465 2.1572 245 10 Module 1—2, 4, and 7Module 2—3, 5, and 6 (without 1 and 8) 6 2.1715 2.1531 2.1437 245 11 Module 1—2, 4, and 7Module 2—3, 6, and 8 (without 1 and 5) 6 2.1679 2.1536 2.1370 245 12 Module 1—2, 4, and 7Module 2—1, 3, and 5 (without 6 and 8) 6 2.2181 2.1415 2.1398 245 13 Module 1—2, 4, and 7Module 2—1, 3, and 8 (without 5 and 6) 6 2.1883 2.1624 2.1551 245 14 Module 1—2, 4, and 7Module 2—3 and 6 (without 1, 5, and 8) 5 2.1478 2.1885 2.1485 210 15 Module 1—2, 4, and 7Module 2—3 and 5 (without 1, 6, and 8) 5 2.1542 2.1210 2.0914 210 16 Module 1—2, 4, and 7Module 2—1 and 3 (without 5, 6, and 8) 5 2.1436 2.2551 2.1580 210 17 Module 1—2, 4, and 7Module 2—3 and 8 (without 1, 5, and 6) 5 2.1777 2.2327 2.1981 210 18 Module 1—2, 4, and 7Module 2—1 (without 3, 5, 6, and 8) 4 2.2006 7.0510 5.6472 210 19 Module 1—2, 4, and 7Module 2—3 (without 1, 5, 6, and 8) 4 2.1554 6.6392 5.2857 175 20 Module 1—2, 4, and 7Module 2—5 (without 1, 3, 6, and 8) 4 2.1856 6.8357 5.4230 175 21 Module 1—2, 4, and 7Module 2—6 (without 1, 3, 5, and 8) 4 2.1873 3.6355 2.7732 175 22 Module 1—2, 4, and 7Module 2—8 (without 1, 3, 5, and 8) 4 2.2109 6.1718 4.9165 175


## 4.2. Testing with Data for Iris Plants

As another test case for the proposed method, we also consider the dataset [26] that contains four parameters (criteria): (1) sepal length in cm, (2) sepal width in cm, (3) petal length in cm, and (4) petal width in cm. There are 3 classes of 50 instances each [26], where each class refers to a type of iris plant. The targets areIris setosa, 1,Iris versicolor, 2, andIris virginica, 3.The ICA method was applied to the 150 × 4 matrix in the same fashion as previously (see Tables5, 6, and 7), and the results are given in Table 5, sorted in descending order according to the membership part of the intercriteria pairs.Table 5
Relations among the criteria.
IF pair Criteria Type of intercriteria relation 0.843;0.538 3-4 Weak positive consonance 0.819;0.504 1-3 Weak positive consonance 0.764;0.486 1-4 Weak positive consonance 0.416;0.139 1-2 Dissonance 0.366;0.121 2-4 Dissonance 0.364;0.070 2-3 DissonanceTable 6
Membership parts of the intuitionistic fuzzy pairs, giving the intercriteria correlations between measurements of iris plant.
μ 1 2 3 4 1 1.000 0.416 0.819 0.764 2 0.416 1.000 0.364 0.366 3 0.819 0.364 1.000 0.843 4 0.764 0.366 0.843 1.000Table 7
Nonmembership parts of the intuitionistic fuzzy pairs, giving the intercriteria correlations between measurements of iris plant.
ν 1 2 3 4 1 0.000 0.486 0.121 0.139 2 0.486 0.000 0.538 0.504 3 0.121 0.538 0.000 0.070 4 0.139 0.504 0.070 0.000In the neural network inputs were assumed with the following parameters: parameter 2 (in the first neural network input) and parameters 1, 3, and 4 in consonance (in the second neural network input). In Table8 the values from 7 neural network simulations are presented and a number of parameters that we defined for each module are described.Table 8
Results from the simulations.
Number Description Number of the inputs g 1 g 2 g 3 Number of weight coefficients 1 Module 1—2Module 2—1, 3, 4 4 0.5725 0.0084 0.5726 175 2 Module 1—2Module 2—3, 4 (without 1) 3 0.5725 0.0321 0.5749 140 3 Module 1—2Module 2—1, 4 (without 3) 3 0.5725 0.6667 1.2898 140 4 Module 1—2Module 2—1, 3 (without 4) 3 0.5725 0.0527 0.5793 140 5 Module 1—2Module 2—4 (without 1, 3) 2 0.5725 0.0631 0.5846 105 6 Module 1—2Module 2—3 (without 1, 4) 2 0.5725 0.0807 0.5907 105 7 Module 1—2Module 2—1 (without 3, 4) 2 0.5725 0.6667 1.2898 105


## 4.3. Testing with Data for Glass

Finally, as a third case for testing the proposed approach we consider the benchmark glass data. In this case, in the input of this modular neural network, we use the experimental data from [27] for obtaining the type of glass. We work with data for 214 building and vehicle window glasses, measured against 9 criteria: (1) RI: refractive index; (2) Na: sodium (unit measurement: weight percent in corresponding oxide); (3) Mg: magnesium; (4) Al: aluminium; (5) Si: silicon; (6) K: potassium; (7) Ca: calcium; (8) Ba: barium; (9) Fe: iron. In the output (as a target) we give information about the glass type.As was done in the previous sections, the input parameters (criteria) are divided in two groups: parameters in dissonance (parameters 2, 3, 4, 5, 6, and 9) and parameters in consonance (see Table9), with the results sorted in descending order according to the membership part of the intercriteria pairs.Table 9
Relations among the criteria.
IF pair Criteria Type of intercriteria relation 0.760 ; 0.234 1-7 Weak positive consonance 0.564;0.423 4-5 Dissonance 0.547 ; 0.409 4-6 Strong dissonance 0.532 ; 0.455 2-4 Strong dissonance 0.532 ; 0.404 3-6 Strong dissonance 0.527 ; 0.425 1-3 Strong dissonance 0.513 ; 0.481 1-2 Strong dissonance 0.497 ; 0.493 2-7 Strong dissonance 0.488 ; 0.470 5-6 Strong dissonance 0.441 ; 0.509 2-3 Strong dissonance 0.419 ; 0.571 5-7 Dissonance 0.410 ; 0.090 8-9 Dissonance 0.384 ; 0.604 4-7 Dissonance 0.383 ; 0.607 2-6 Dissonance 0.371 ; 0.579 3-7 Dissonance 0.368 ; 0.594 1-6 Dissonance 0.353 ; 0.596 3-5 Dissonance 0.318 ; 0.672 1-4 Weak dissonance 0.316 ; 0.237 6-9 Weak dissonance 0.314 ; 0.645 6-7 Weak dissonance 0.311 ; 0.236 3-9 Weak dissonance 0.304 ; 0.240 7-9 Weak dissonance 0.300 ; 0.693 1-5 Weak dissonance 0.299 ; 0.245 1-9 Weak dissonance 0.288 ; 0.659 3-4 Weak dissonance 0.272 ; 0.057 4-8 Weak dissonance 0.262 ; 0.697 2-6 Weak dissonance 0.257 ; 0.070 2-8 Weak dissonance 0.253 ; 0.291 4-9 Weak dissonance 0.253 ; 0.291 5-9 Weak dissonance 0.210 ; 0.334 2-9 Weak negative consonance 0.203 ; 0.123 5-8 Weak negative consonance 0.162 ; 0.164 7-8 Weak negative consonance 0.123 ; 0.202 1-8 Negative consonance 0.113 ; 0.213 6-8 Negative consonance 0.059 ; 0.249 3-8 Negative consonanceIn Table10, the values from 7 neural network simulations are presented with the respective identifiers of the criteria that are feeding the inputs of each module.Table 10
Results from the simulations.
N: Description Number of the inputs g 1 g 2 g 3 Number of weight coefficients 1 Module 1—2, 3, 4, 5, 6, 9Module 2—1, 7, 8 9 0.7658 0.8796 1.6721 350 2 Module 1—2, 3, 4, 5, 6, 9Module 2—7, 8 (without 1) 8 0.7658 0.8696 1.6584 315 3 Module 1—2, 3, 4, 5, 6, 9Module 2—1, 8 (without 7) 8 0.7658 0.9277 1.7492 315 4 Module 1—2, 3, 4, 5, 6, 9 Module 2—1, 7 (without 8) 8 0.7658 1.5826 2.8488 315 5 Module 1—2, 3, 4, 5, 6, 9Module 2—8 (without 1, 7) 7 0.7658 0.8756 1.6626 280 6 Module 1—2, 3, 4, 5, 6, 9Module 2—7 (without 1, 8) 7 0.7658 1.7190 3.0898 280 7 Module 1—2, 3, 4, 5, 6, 9Module 2—1 (without 7, 8) 7 0.7658 1.6378 2.9400 280The information is divided into two groups: the first part is the group of independent evaluation criteria (see Tables11 and 12): criteria 2, 4, and 7 (see Table 4 for the crude oil probes data), criterion 2 (see Table 5 for the iris plant data), and criteria 2, 3, 4, 5, 6, and 9 (see Table 8 for the glass types data). These data were used on the inputs of the first module of the neural networks. The other criteria (dependent parameters) were considered on the inputs of the second module of the neural networks.Table 11
Membership parts of the intuitionistic fuzzy pairs, giving the intercriteria correlations of glass.
μ 1 2 3 4 5 6 7 8 9 1 1.000 0.513 0.527 0.318 0.300 0.368 0.760 0.123 0.299 2 0.513 1.000 0.441 0.532 0.383 0.262 0.497 0.257 0.210 3 0.527 0.441 1.000 0.288 0.353 0.532 0.371 0.059 0.311 4 0.318 0.532 0.288 1.000 0.564 0.547 0.384 0.272 0.253 5 0.300 0.383 0.353 0.564 1.000 0.488 0.419 0.203 0.253 6 0.368 0.262 0.532 0.547 0.488 1.000 0.314 0.113 0.316 7 0.760 0.497 0.371 0.384 0.419 0.314 1.000 0.162 0.304 8 0.123 0.257 0.059 0.272 0.203 0.113 0.162 1.000 0.410 9 0.299 0.210 0.311 0.253 0.253 0.316 0.304 0.410 1.000Table 12
Nonmembership parts of the intuitionistic fuzzy pairs, giving the intercriteria correlations.
ν 1 2 3 4 5 6 7 8 9 1 0.000 0.481 0.425 0.672 0.693 0.594 0.234 0.202 0.245 2 0.481 0.000 0.509 0.455 0.607 0.697 0.493 0.070 0.334 3 0.425 0.509 0.000 0.659 0.596 0.404 0.579 0.249 0.236 4 0.672 0.455 0.659 0.000 0.423 0.409 0.604 0.057 0.291 5 0.693 0.607 0.596 0.423 0.000 0.470 0.571 0.123 0.291 6 0.594 0.697 0.404 0.409 0.470 0.000 0.645 0.213 0.237 7 0.234 0.493 0.579 0.604 0.571 0.645 0.000 0.164 0.240 8 0.202 0.070 0.249 0.057 0.123 0.213 0.164 0.000 0.090 9 0.245 0.334 0.236 0.291 0.291 0.237 0.240 0.090 0.000The inputs on the first module are not removable, because the ICA approach tests the independence of the criteria. The results from applying the ICA method over tree types of data show that most of the parameters are selected in a proper way (criteria in dissonance). For some of the parameters that were provided on the inputs of the second module, an approach for reasonable elimination of some of these criteria can be adopted.


## 4.4. Discussion

In the first step we use neural networks without any removal of information and without removing of the inputs. The corresponding results can be found on the first rows in Tables4, 8, and 10.On the second step the data that have the highest and lowest consonance coefficients (strong positive consonance, positive consonance, strong negative consonance, and negative consonance) were removed. For example, in Table4 we use parameters 2, 4, and 7 inModule 1 and parameters 3, 5, 6, and 8 (without 1) inModule 2. In the study, parameter 1 is removed due to the high value of membership coefficient μ=0.956 (between parameters 1 and 5). After simulating the neural network an average deviation of the MNN g3=2.1562 (g1=2.2139, g2=2.1650) is obtained.The removed parameters do not have substantial influence on the result, because of decreasing the number of the weight coefficients, and along with this the error of the output values also decreases.In the same way the second group of parameters (from the pairs) were successively removed. This process continues until finishing the pairs in the highest and lowest consonance.The next step is to remove two parameters, for example, in row 5 (Table8) the situation with all parameters onModule 1, and parameter 4 (without 1 and 3) inModule 2 was presented. In the study, parameters 1 and 3 are removed due to the high value of membership coefficient μ=0.81852 (between parameters 1 and 3). After simulation of the neural network, an average deviation of the MNN g3=0.5846 (g1=0.5725, g2=0.0631) is obtained. The removed parameters 1 and 3 do not have substantial influence on the result, because of decreasing the number of the weight coefficients, and along with this the error of the output values also decreases.The process can continue iteratively in the same way. In summary, a new approach for building modular neural networks based on ICA, which uses intuitionistic fuzzy logic, has been proposed and tested with three benchmark databases. The simulation results are good, in this way verifying the advantages of the hybrid method.In Table13 comparative analysis of the results of the three different datasets is made. The table includes a short description of each dataset, the initial number of inputs (number of the inputs before reduction), the number of inputs after reduction, and average deviation of the output values of the neural networks (g3), as well as minimal value of average deviation of the output values of the neural networks (g3, min), the number of weight coefficients without reduction and with reduction, the percentage ratio between the average deviation of the output values of the neural networks and the minimal value of average deviation of the output values of the neural networks, respectively, and the number of weight coefficients without and with reduction.Table 13
Results from the simulations.
Number Description Number of the inputs before reduction Number of the inputs after reduction g 3 g 3 min Number of weight coefficientswithout reduction Number of weight coefficientswith reduction g 3 % Number of weight coefficients% 1 Data with parameters of crude oil 8 5 2.2330 2.0914 315 210 93,65 66,67 2 Data of iris plant parameters 4 2 0.5726 0.5446 175 105 95,11 60 3 Data of glass parameters 9 7 1.6721 1.6626 350 280 98,2 80From the data in the table, it is seen that reducing the number of input parameters and, respectively, the number of inputs of the MNN, the value of average deviation of the output values of the MNN decreases (93,65%–98,2%). On this basis, we further observe decrease of the number of weight coefficients (60%–80%).


## 5. Conclusions

The number of the neurons is one of the major parameters that need to be defined during the realization of the MNN. Of course, the higher number of neurons in a neural network has the effect of slowing down the learning process. Here, we use the intuitionistic fuzzy logic based method of intercriteria analysis to reduce the number of input parameters of the modular neural network. This leads to reduction of the weight matrices and thus allows implementation of the neural network in limited hardware, saving time and resources in training. The method can also be used to assess the independence of the criteria against which data are measured. Three different real datasets with measurements of glass types, crude oil probes, and iris plant specimens were used to verify the accuracy and efficiency of the proposed approach. As future work, we envision using the proposed approach with other types of complex problems or more real life situations, like in the design of intelligent controllers for robotic systems or in the design of pattern recognition systems for human identification based on biometric measures. We believe that for real world situations some pertinent modifications or improvements to our model could be needed, but the essence of the solution to the problems is already provided in the proposed approach. As another leg of future investigation in the proposed direction, we consider a three-module MNN, which will be fed with the following inputs: one with the intercriteria pairs exhibiting positive consonance, another with the intercriteria pairs exhibiting negative consonance, and the third one with the pairs exhibiting dissonance.



---

*Source: 3927951-2018-04-19.xml*