# Using Feature Selection with Machine Learning for Generation of Insurance Insights
## 2 Related Works
### Feature Selection Techniques  
In this section, we provide an overview of feature selection approaches. We then give a more detailed description for the specific feature selection methods used in our study. Feature selection techniques based on the selection strategy can be classified into three types: wrapper, filter and embedded [32]. Wrapper methods employ a predefined supervised or unsupervised learning algorithm to find the best subsets of features that improve the classification accuracy or clustering quality of corresponding learning algorithms [33,34]. Therefore, they are dependent on the learning algorithm. Wrapper feature selection techniques often get high performance measures but they consume high computational time to run. Moreover, the group of selected features for a certain dataset may be changed if another learning algorithm is used. They are limited to being employed in conjunction with a specified learning algorithm [35]. Filter feature selection techniques are independent of the machine learning algorithm used. They use general characteristics of features such as variance, consistency, correlation, and information to pick features without involving any machine learning algorithms [36]. Despite the fact that filter based feature selection methods do not often outperform wrapper-based methods, they are widely used in the literature because they are typically fast, scalable, and applicable to high-dimensional data. Furthermore, they are usually performed before classification, and thus they they are not biased towards learning algorithms [4]. Embedded techniques combine characteristics of both filter and wrapper methods. They usually employ feature selection in conjunction with predefined learning process similar to wrapper methods but they do not iterate the learning algorithm, which is in contrast to original wrapper methods. Therefore, they are more efficient than wrapper methods but they do not typically outperform them [37,38]. In this article, we choose filter based feature selection methods to demonstrate and investigate the potential for feature selection on insurance datasets, due to their wide use in the literature and independence of the learning algorithm used. Structured insurance datasets are typically mixed containing a mix of numeric and categorical data so techniques suited to mixed data will be considered. Filter based feature selection methods make use of some intrinsic properties to compute a score for each feature. These methods evaluate the relevance of the features either individually or jointly. They usually require the identification of the target number of selected features or the desired percentage of total number of features such as 0.2, 0.5, and 0.8. There are two methodologies for selecting features: ranking based methods or selection based methods. The selection based features selection is an iterative method wherein the algorithm starts with the best performing feature in the result set. Next, it adds another feature that gives the best performance in combination with the features in the result set until the desired number of features is reached. However, ranking based methods compute a relevance score for each feature, ranking all features based on this score. Next, they choose the top percentage of features that have the best scores. To demonstrate the potential for feature selection, we need to use several methods to remove reliance on any single set of results. We select the following four feature selection techniques that are suitable for mixed datasets: the Greedy Feature Selection Algorithm (GFSA) [39], the Laplacian Score (LS) [40], the spectral algorithm (Spec) [41], and the Unsupervised Spectral Feature Selection Method (USFSM) [42]. We will explain each of these below. Before moving on to explain the four techniques that we have used, we highlight some new approaches that stand out as interesting.