# Distributed Fuzzy Cognitive Maps for Feature Selection in Big Data Classification
## 3 Materials and Methods
### Distributed Fuzzy Cognitive Maps  
A wide range of applications are available for FCMs. They provide a modeling technique that is useful to represent highly complex systems, and they can be also used to model uncertainties and improve accuracy in various application problems. A fuzzy cognitive map is a signed weighted digraph that includes fuzzy causal relationships between its nodes. An FCM consists of three components: concepts, state vector, and weight matrix. As part of the construction of an FCM, the state vector and weight matrix must be initialized, followed by training of the FCM model. The state vector is composed of all the values of all the concepts in the system. Each positive value of the state vector depicts the inclusion of a particular feature. The weight matrix should be initialized to define the semantic properties of the dataset. A correlation matrix is used to initialize the weight matrix of the FCM. Within the weight matrix, each element represents the correlation coefficient between different features of the dataset. Each element is populated with the Pearson correlation coefficient value. To compute the correlation coefficient, the following formula is used:  
\[r=\frac{\sum_{i}(x_{i}-\overline{x})(y_{i}-\overline{y})}{\sqrt{\sum_{i}(x_{i} -\overline{x})^{2}}\sqrt{\sum_{i}(y_{i}-\overline{y})^{2}}} \tag{1}\]  
where \(r\) is the correlation coefficient, \(x_{i}\) is a sample's \(x\)-variable values, \(\overline{x}\) is their mean, \(y_{i}\) is a sample's \(y\)-variable values, and \(\overline{y}\) is their mean. Regarding the values of \(r\), \(r=1\) indicates a complete positive correlation, while \(r=-1\) means a perfect negative correlation. The weight matrix and the state vector are stored in a resilient distributed dataset. FCM learning is applied on the initial state vector using the following formula:  
\[A_{i}^{(k+1)}=f\Bigg{(}A_{i}^{k}+\sum_{j\
eq i,j=1}^{N}A_{j}^{k}.W_{ij}\Bigg{)} \tag{2}\]  
where \(W_{ij}\) is the weight of the link between concepts \(C_{i}\) and \(C_{j}\), and \(A_{i}^{(k+1)}\) indicates the value of concept \(C_{i}\) at step \(k+1\).The sigmoid function is utilized as the threshold function\(f\) (\(x\)):  
\[f(x)=\frac{1}{1+e^{-\lambda x}} \tag{3}\]  
Iteratively, the state vector values are computed and the weight is modified until epsilon, a residual value that yields the smallest error difference between succeeding concepts, is attained. The equation used in each iteration step to update the weight matrix is given in Equation (4). \[W_{ij}(k)=W_{ij}(k+1)+\Big{(}\eta_{k}*\ A_{i}^{(k-1)}-\Big{(}A_{j}^{(k-1)}*\ W_{ij}^{(k-1)}\Big{)}\Big{)} \tag{4}\]  
Figure 1: Proposed model workflow. where \(Wji\) (\(k\)) represents the revised weight after the kth iteration. \(\eta_{k}\) represents the value of the learning parameter in the kth iteration. Concept values fall within the interval [0,1], whereas weight values fall within the interval [-1,1]. A parallel learning process is proposed for FCM, as depicted in Figure 2. The parallelize function is given the weight matrix of the Spark's Resilient Distributed Datasets (RDD) as input. Through the parallelize function, the weight matrix RDD is divided into sets of causal relations, and multiple new RDDs are created that contain subsets of weight matrix values for each node in the distributed system. As well as the weight matrix, the FCM learning requires the state vector. This requires the state vector to be available in all nodes where the weight matrix has been distributed. Therefore, we use the broadcast function on the state vector RDD, and the state vector is duplicated across all nodes. All distributed nodes cache the state vector. As the weight matrix RDD contains multiple rows and columns, it occupies very large amounts of space. However, the state vector RDD is a one-dimensional vector, so replicating it across the nodes will not impact memory capacity. FCM learning is applied at each node using Equation (2), and a partial result is generated.