# Deciphering Autonomous Vehicle Regulations with Machine Learning
## 3 Methodology
### Machine Learning
#### 3.3.2 Recursive Feature Elimination (RFE)  
For small datasets, the process of feature selection becomes critical yet challenging. Selecting the right features (variables) that have significant predictive power is crucial because a small dataset magnifies the impact of each feature. However, with limited data, it can be difficult to discern which features are truly relevant and which are coincidentally correlated with the target variable. Recursive feature elimination with cross-validation (RFECV) is a method in the ML workflow that identifies the most expressive features for a predictive model. RFECV begins by using cross-validation to train a model with the full set of features to obtain their level of importance either through coefficients or feature importance scores, and then iteratively eliminates the least prominent features. Cross-validation accounts for the variability in the dataset and reduces the risk of overfitting, \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **Theory of Operations** & **Advantages** & **Disadvantages** \\ \hline SVM & \begin{tabular}{c} Analyzes data for classification by \\ maximizing the margin between \\ decision boundaries and data points in \\ high-dimensional feature space. \\ \end{tabular} & \begin{tabular}{c} Effective in high-dimensional \\ spaces, versatile with different \\ kernel functions, and provides \\ a unique solution since the \\ optimization problem \\ is convex. \\ \end{tabular} & \begin{tabular}{c} Not suitable for large datasets, \\ less effective on noisier \\ datasets with overlapping \\ classes, and requires a good \\ kernel choice. \\ \end{tabular} \\ \hline LR & \begin{tabular}{c} Measures the relationship between a \\ categorical dependent variable and \\ independent variables by estimating \\ probabilities using a logistic function. \\ \end{tabular} & \begin{tabular}{c} Simple and efficient for binary \\ classification problems. \\ Outputs probabilities that \\ are interpretable. \\ \end{tabular} & \begin{tabular}{c} Assumes linearity between the \\ dependent variable and \\ independent variables. Can \\ struggle with complex \\ relationships and is not flexible \\ enough to capture more \\ complex patterns. \\ \end{tabular} \\ \hline ANN & \begin{tabular}{c} Modeled loosely after the human brain \\ to recognize patterns by interpreting \\ sensory data through machine \\ perception, labeling, and clustering. \\ \end{tabular} & \begin{tabular}{c} Highly flexible and capable of \\ learning nonlinear \\ relationships. Performs well on \\ a large variety of tasks and is \\ scalable to large datasets. \\ \end{tabular} & \begin{tabular}{c} Not appropriate for small \\ datasets. Computationally \\ intensive, prone to overfitting, \\ and lacks interpretability. \\ \end{tabular} \\ \hline Con. & \begin{tabular}{c} The constant (Con.) model always \\ predicts the same output regardless of \\ the input. \\ \end{tabular} & \
\begin{tabular}{c} Provides a baseline for model \\ performance comparison. \\ \end{tabular} &
\begin{tabular}{c} Offers no real insight into \\ data patterns. \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table} 
Table 2: _Cont._leading to a more robust feature selection. Cross-validation iteratively divides the dataset into a specified number of subsets called folds, trains the model on the union of all folds except one, and validates the predictive performance on the held-out fold. The cross-validation ends after involving each fold in the validation at least once. RFECV keeps track of the model's performance based on the average AUC score for each set of features and reports the final selection that maximizes its predictive performance. The methods used to determine feature importance scores vary, depending on the model type. For example, linear models assign coefficients to the features based on the sensitivity of the target variable to changes in each feature, with all other features remaining constant.