# Using Feature Selection with Machine Learning for Generation of Insurance Insights
## 2 Related Works
### Feature Selection Techniques
PEFS formulates the problem of feature selection as a Pareto-based optimization problem with two objectives. It employs four different feature selection algorithms and then combines their results using two aggregation methods. It evaluates these results using a bi-objective optimization. The features are ranked according to their crowding distance in the bi-objective space. In multi-objectives optimization literature, the crowding distance of a solution estimates the density of surrounding solutions [46]. PEFS utilises a combination of redundancy and relevancy feature selection techniques to give a higher ranking to the features which have the higher relevancy score and the lowest redundancy score. Here, we discuss the candidate feature selection algorithms for use in our work. LS and Spec algorithms are feature selection techniques for continuous datasets. Thus, each nominal feature is replaced by an ordinal feature. Each category is transformed by an integer that represents its order of appearance within this column. We have chosen these methods because they are widely used in the literature because they are typically scalable and applicable to high-dimensional data [42]. The Greedy Feature Selection Algorithm (GFSA) [39] is based on selecting the set of features that has the minimum association amongst them. This algorithm searches for a diverse and minimally dependent set of features. It formulates the feature selection problem as an optimization problem. GFSA handles categorical as well as numerical features. The GFSA algorithm follows a step-by-step procedure to find the desired dimension that minimizes the multiple associations among them. It starts with choosing the least dependent pair of features. Then it increases the solution by one feature at each iteration until it reaches the desired number of features. The GFSA algorithm assumes that the incoming feature should have the minimum pair-wise association with one feature in the solution set. It then computes the multiple association for all candidate sets and selects the set with the least multiple association [39]. The Laplacian Score (LS) [40] is a filter based feature selection algorithm. The LS feature selection is designed only for continuous data. The Eigensystem of the Laplacian matrix is employed to measure the relevance of each feature. The Laplacian matrix is computed from the similarities among objects. The features are ranked according to their Laplacian scores which reflect locality preserving power of a single feature. The LS score relies on the geometric structure of objects. At the beginning, the LS algorithm builds a nearest neighbour graph \(G\). Each node represents a feature. Two nodes are IFF and one node is among the \(k\) nearest neighbours of the other node. It then computes a Laplacian score for each feature based on the \(k\) nearest neighbour graph, which is used a measure its power of locality preserving. The LS looks for those features that respect this graph structure [40]. The spectral feature selection method, Spec, [41] relies on the spectral graph theory. The Spec algorithm utilizes the intrinsic properties to construct a similarity graph, \(G\), to represent the dataset. The relevance of features is measured by their consistency with the structure of the similarity graph among objects. Moreover, a similarity score for each feature is computed. This score should be consistent with the graph structure. The Spec algorithm computes a consistency score for each feature to measure the coherence between this feature and the nontrivial eigenvectors of the Laplacian matrix. The Unsupervised Spectral Feature Selection Method (USFSM), is an unsupervised feature selection technique for mixed data. It is based on building a similarity matrix between objects using a kernel function. The kernel function makes use of distance functions for numeric and nominal features. The structural information of the data is contained in the spectrum of the Laplacian matrix through the separation of its elements.