## 4 The Proposed Model
### Estimation Models  
Once the features are extracted from the structured data, SP estimation models are constructed using these extracted features. Given that it is a regression problem, the GBT models are deemed suitable for the dataset. Three powerful GBT models were applied:  
1. **Catboost**: Known for efficiently handling categorical features, CatBoost excels in automatic categorical feature conversion and provides robust handling without the need for manual preprocessing [121]. Given the dataset's inclusion of categorical variables, CatBoost is a fitting choice.
2. **Xgboost**: Another GBT model with improved speed and performance, XGBoost incorporates regularization techniques to prevent overfitting [122]. Its flexibility in defining custom optimization objectives makes it a valuable addition to SP estimation models.
3. **LightGBM**: Utilizing a histogram-based learning approach, LightGBM proved effective in handling large datasets. Its capability to handle categorical features, coupled with outstanding parallelization, significantly improved performance and reduced memory usage [123]. These models were implemented and trained in Python 3.8 using the CatBoost-Regressor 1.2.5 ([[https://catboost.ai/](https://catboost.ai/)]([https://catboost.ai/](https://catboost.ai/)), accessed on 18 August 2024), XGBoressor 2.1.1 ([[https://xgboost.readthedocs.io/en/stable/](https://xgboost.readthedocs.io/en/stable/)]([https://xgboost.readthedocs.io/en/stable/](https://xgboost.readthedocs.io/en/stable/)), accessed on 18 August 2024), and LGBMRegressor 4.5.0 ([[https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.LGBMRegressor.html](https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.LGBMRegressor.html)]([https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.LGBMRegressor.html](https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.LGBMRegressor.html)), accessed on 18 August 2024) libraries, respectively.  
For the optimization of these GBT models, various techniques are employed, including **manual tuning**[124] and **automated hyperparameter tunings**[124]. For automated hyperparameter tuning, operations such as **random search**[124] and **Bayesian optimization**[124] were implemented using libraries such as RandomizedSearchCV 1.5.1 ([[https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.RandomizedSearchCV.html](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.RandomizedSearchCV.html)]([https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.RandomizedSearchCV.html](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.RandomizedSearchCV.html)) accessed on 18 August 2024), optuna 3.6.1 ([[https://optuna.org/](https://optuna.org/)]([https://optuna.org/](https://optuna.org/)) accessed on 18 August 2024), and hyperopt 0.2.7 ([[https://pypi.org/project/hyperopt/](https://pypi.org/project/hyperopt/)]([https://pypi.org/project/hyperopt/](https://pypi.org/project/hyperopt/)) accessed on 18 August 2024). In all these approaches, the root mean square error (RMSE) was utilized as the objective function to assess and guide the optimization of the GBT models.  
After evaluating each model's performance and optimizing models using different hyperparameter tuning techniques with a focus on their SA in estimating SP values, the model that demonstrates the best performance is selected as the primary model. After the completion of experiments, the **SBERT-LGBMTC** model with manual tuning is selected as the primary model.  
This comprehensive model ensures a thorough exploration of the capabilities of each model, taking into account both manual and automated hyperparameter tuning strategies to enhance the accuracy of SP estimation in the context of Scrum software projects.  
Figure 1 illustrates the flowchart of our proposed model. A detailed diagram of the proposed model is provided in Appendix A.