# Guided Particle Swarm Optimization for Feature Selection: Application to Cancer Genome Data
## 3 Approach  
An overview of the proposed approach is shown in Figure 1. The two proposed algorithms have two goals in mind: (1) expansion of the search space using three filter-based methods to generate the particles and (2) using a fitness factor that weights accuracy and size of a feature set, including knowledge transfer, in order to prevent premature convergence. The figure as well as Algorithm 1 describe the approach. The Guided Particle Swarm Optimization (PSO) algorithm for feature selection begins by initializing a swarm of particles, where each particle represents a potential subset of features. The goal of the algorithm is to identify the best feature subset that maximizes a fitness function, which likely measures the subset's ability to accurately classify or predict outcomes based on the dataset provided. Each particle in the swarm is assigned a random position (corresponding to a feature subset) and velocity, while the global best position, denoted as \(gbest\), is initialized with an extremely high score, ensuring that any better solution will replace it. The initial positions of the particles are informed by three feature selection methods: Gini index, ANOVA, and Relief. Particles are divided equally among the positions computed by these methods, and their feature subsets are initialized using the knee point method, which balances the number of features with their relevance. The algorithm then enters an iterative process, where it repeatedly evaluates and updates the particles over a series of iterations. During each iteration, the fitness of each particle's feature subset is calculated. Fitness is calculated according to Equation (3):  
\[fitness=\gamma\cdot CER+(1-\gamma)\cdot(SFS/TFS) \tag{1}\]  
where \(\gamma\) is the weighting factor, \(CER\) is the classification error rate, \(SFS\) is the selected feature set, and \(TFS\) is the total feature set. If a particle's current subset yields a better fitness score than its previously best-known score, the particle updates its best-known position. Similarly, if a particle's fitness score is better than the global best score across the entire swarm, the global best position is updated. The following velocity and position update equations are used:  
\[v_{i}^{(t+1)}=\omega v_{i}^{(t)}+c_{1}r_{1}(pbest_{i}-x_{i}^{(t)})+c_{2}r_{2} (gbest-x_{i}^{(t)}) \tag{2}\]  
\[s(v_{i}^{(t+1)})=\frac{1}{1+e^{-v_{i}^{(t+1)}}} \tag{3}\]  
Figure 1: Overview of proposed guided feature selection PSO. \[x_{i}^{(t+1)}= \begin{cases}1,&\text{if }r<s(v_{i}^{(t+1)})\\ 0,&\text{otherwise}\end{cases}  \tag{4}\]  
where \(v_{i}^{(t+1)}\) is the updated velocity of particle \(i\) at time step \(t+1\); \(\omega\) is the inertia weight; \(v_{i}^{(t)}\) is the previous velocity of the particle; \(c_{1}\) and \(c_{2}\) are the cognitive and social coefficients; \(r_{1},r_{2}\sim U(0,1)\) are random numbers sampled from a uniform distribution; \(\textit{pbest}_{i}\) is the personal best position of particle \(i\); \(\textit{gbest}\) is the global best position; \(x_{i}^{(t)}\) is the current position of particle \(i\); \(s(v_{i}^{(t+1)})\) is the sigmoid function applied to the velocity; \(r\sim U(0,1)\) is a random number sampled from a uniform distribution; and \(x_{i}^{(t+1)}\) is the updated binary position of the particle. After evaluating all particles, the swarm is sorted based on their best scores. A subset of elite particles--those with the best performance--is identified, and knowledge from these elite particles is transferred to others in the swarm to guide the search towards more promising areas of the solution space; this is what we refer to as the knowledge transfer mechanism. Finally, each particle's velocity and position are updated, allowing the swarm to explore new subsets of features while being influenced by both individual and collective experiences.