## 5 Empirical Study
### Machine Learning Concepts and Techniques  
This section provides a brief overview of the machine learning techniques applied to implement our empirical study. 1. _Regression in Machine Learning_: Regression is a fundamental concept in machine learning used to predict continuous numerical values based on attributes [74]. Linear Regression (LR) is one of the most widely used regression techniques. It captures the linear relationship between data points in the data [75], particularly between independent and dependent dimensions. This empirical study employed Linear Regression to predict effort in hours on the dataset. 2. _Extra Trees Regressor_: Extra Trees are ensemble learning models based on a collection of decision tree models for regression tasks [76]. The extra tree regressor aggregates tree predictions by averaging them to generate a final prediction. This Regressor is robust to overfitting and performs well with high-dimensional data. Capturing the non-linear relationship between data features makes it a powerful model for complex regression tasks. 3. _AdaBoost Regressor_: AdaBoost is a boosting regressor that sequentially combines more than one learner to generate a strong predictive model [77]. AdaBoost can capture non-linear relationships between data points, which demonstrates a powerful regressor against complex and high-dimensional data. 4. _Random Forest Regressor_: Random Forest is an ensemble learning method constructed using multiple decision trees combined by outputting the average prediction from all individual trees [78]. This approach helps avoid overfitting and results in a more robust model. The Random Forest model can be used for both classification and regression (i.e., supervised learning tasks). Each tree in the forest is trained using a random subset of data, which excels in handling large datasets and can capture non-linear relationships between features and target variables. 5. _Support Vector Machine_: Support Vector Machine (SVM) can handle Linear and Non-Linear Regression relationships between independent and dependent targets. SVM can be used for regression or classification tasks. It starts by representing the data points around a line called the hyperplane, and the points nearest to the line are called support vectors. The SVM aims to maximize the margin between the vector points through kernel functions to capture the relationships between features. 6. _Decision Tree_: A Decision tree is a supervised learning algorithm that can be used for regression and classification. The algorithm generates a tree structure in which nodes represent decisions and the leaves represent the prediction output [79]. The model approach works recursively to split data based on a root node to minimize the variance or error within each subset. The information Gain and Gini Index can be used to determine the optimal node splitter as the root. The model can handle numerical and categorical data to capture the complex relationships between the data features. 7. _K-Nearest Neighbors_: K-Nearest Neighbors (KNN) is a supervised model that can handle forecasting and prediction tasks [80]. Here, K represents several values. The default value is five and represents a specific number of neighbors, which allows the model to measure the distance between the data point and other data points and determine the neighbors. The model then starts by taking the average prediction value for the neighbors. Neighbors are determined based on a distance metric, typically Euclidean Distance. KNN can handle non-linear relationships between data features, particularly when data are high-dimensional. * _Gradient Boosting_: Gradient Boosting is an ensemble-based model generated by sequentially combining decision trees [81]. Unlike Random Forest, which is generated based on a bagging ensemble, gradient boosting focuses on reducing the bias by optimizing the model performance through the gradient descent approach.