# A Knowledge-Guided Competitive Co-Evolutionary Algorithm for Feature Selection
## 3 Knowledge-Guided Evolutionary Algorithm
### Feature Grouping Based on Correlation  
Datasets in real applications include various features. The correlation among different features is diverse. Some are related, and some are not. Some have positive correlation, while others have negative. Additionally, there might be numerous high-dimensional relationships between features that are difficult to identify, which could significantly impact the outcome. For instance, in finance, stock prices and market indices might show positive correlations, but interest rates and stock prices might have a negative correlation. Therefore, we aim to use correlation to group features. This strategy aims to improve the efficiency of subsequent evolutionary searches while maximizing the diversity of the population. Ideally, our expected feature grouping outcome should achieve balance among feature groups, i.e., avoid overly large feature groups and a multitude of isolated features. Through experimentation, we propose a significantly effective and simple threshold-based grouping method. The pseudo-code of the described algorithm is shown in Algorithm 1. Initially, the algorithm conducts a detailed analysis of the correlation among features. By calculating the Pearson correlation coefficient between features, we obtain a correlation matrix where each element of the matrix represents the correlation strength between a pair of features. After obtaining the correlation matrix, we traverse each feature and check its correlation with the features in the existing groups based on a preset threshold determined through experimentation. If a feature's absolute correlation with all features in a certain group exceeds the threshold, then the feature is added to the corresponding group. If a feature cannot be placed into any existing group, a new feature group is created. This grouping mechanism ensures high correlation within groups while maintaining relative independence between groups. ```
0: correlation_matrix of size \(n\times n\), threshold
0: List of feature groups
1: Initialize \(feature\_groups\) to an empty list
2:\(num\_features\leftarrow\) number of columns in \(correlation\_matrix\)
3:for\(i=0\) to \(num\_features-1\)do
4:\(added\leftarrow\) False
5:for



This feature grouping mechanism based on correlation analysis acts as knowledge and provides guidance for the subsequent knowledge-guided evolutionary process. Its main objective is to reduce the dimensions of the search space by focusing on combinations of features with high correlation, capturing possible higher-dimensional feature relationships and thereby improving the search efficiency of the evolutionary algorithm. In this manner, the KGEA can effectively balance exploration and exploitation in evolutionary searches, promoting the diversity and quality of solutions. ### Knowledge-Guided Crossover and Mutation

The unique aspect of the KGEA lies in its utilization of predetermined groupings of features to direct the genetic operators of crossover and mutation. This approach contrasts sharply with traditional methods where each gene (or feature) is considered independently. In the realm of evolutionary optimization, particularly for feature selection in high-dimensional datasets, the techniques of knowledge-guided crossover and mutation play a pivotal role. By intelligently grouping features based on their correlations--as delineated in prior sections--this method not only simplifies the search space but also ensures a focused exploration that significantly enhances efficiency. In the KGEA, features are encoded using a binary system. As shown in Figure 1, when the encoding position \(x_{1}\) for Featurnel is 1, it indicates that we have selected this feature. This encoding is crucial as it directly influences the genetic operations that follow. Figure 2 provides an example of knowledge-guided crossover and mutation assuming we have six features \(\{F_{1},F_{2},F_{3},F_{4},F_{5},F_{6}\}\) with corresponding chromosome encoding \(\{x_{1},x_{2},x_{3},x_{4},x_{5},x_{6}\}\).