## 3 Background
### Estimation Model  
**GBTs:** GBTs represent a powerful ensemble learning technique that has gained prominence in various ML applications. GBTs are particularly effective in regression and classification tasks, leveraging the strength of DTs to create a robust predictive model [116]. GBTs are an ensemble of DTs in which each tree is constructed sequentially in order to correct the errors of the preceding ones. The model starts with a single, relatively simple tree and subsequently adds more trees, each focusing on the mistakes made by the ensemble so far [116]. The combination of these weak learners results in a strong predictive model. In each iteration, a new tree is added to the ensemble, and its predictions are combined with the predictions of the existing trees [116]. The key innovation lies in adjusting the weights of the observations, giving more emphasis to the instances where the model has performed poorly. This iterative process continues until a predefined number of trees is reached or until further tree additions cease to improve performance.  
Recent studies have reinforced the preeminence of tree-based models like GBTs in handling structured data. Extensive benchmarks on various datasets demonstrate that tree-based models, including GBTs, continue to be the state-of-the-art for medium-sized data, outperforming standard and novel DL methods, even without considering their superior speed [117].  
Some specific strengths of GBTs make them a suitable choice to be used in SP estimation models. GBTs are versatile and can handle both regression and classification tasks effectively. GBTs inherently capture non-linear relationships in the data, making them suitable for complex, real-world problems. The sequential construction of trees allows GBTs to adapt and learn from errors, leading to robust and accurate predictions. GBTs provide a measure of feature importance, allowing users to identify which variables contribute most significantly to the model's predictive power [118; 119].  
The other reason why we chose GBTs as SP estimation models is that GBTs exhibit exceptional performance with structured data, making them a popular choice for text-based regression and estimation problems. Their ability to handle a mix of numerical and categorical features, capture non-linear relationships, and effectively manage large datasets aligns well with the characteristics of structured data. GBTs provide:  
* **Categorical Feature Handling**: They can naturally handle categorical features without the need for one-hot encoding, simplifying the preprocessing steps for structured data and outperforming other algorithms [117];
* **Robust Performance**: They often outperform other algorithms when faced with structured data due to their adaptability and ability to capture intricate patterns [117];
* **Predictive Accuracy**: They excel at achieving high predictive accuracy, making them particularly valuable in scenarios in which precise predictions are crucial [117]; and
* **Interpretability**: While not as inherently interpretable as simpler models, GBTs provide insights into feature importance, aiding in understanding the underlying patterns in tabular datasets [117].  
In conclusion, GBTs represent a formidable ML tool, especially when dealing with structured data, like in the problem of SP estimation based on user story or issue texts.