## 2 Materials and Methods
### Hybrid (GP-XGBoost)  
Combining GP and XGBoost creates a powerful hybrid methodology that integrates the interpretability of GP with the robust predictive capabilities of XGBoost. This combination is particularly valuable for solving complex problems where understanding the underlying relationships is as important as achieving high predictive accuracy. Below is a comprehensive outline of how these methods can be combined. In this research, method 2 was employed as it showed better performance in prediction. Method 1. Sequential Hybrid Approach  
In this approach, GP is used as a preprocessing step to create features or refine the input data for XGBoost, which then focuses on predictive modeling. Step 1: Feature Engineering with GP  
1. Symbolic Regression: GP is employed to explore relationships between input variables (e.g., the LL, the PI, and \(e_{0}\)) and the target variable (\(C_{c}\)). The output is symbolic equations that describe the following relationships: \[C_{c}=\frac{LL}{PI+e_{0}}\] (18) \[C_{c}\approx LL\text{.log}(PI)\] (19)
2. Feature Transformation: The symbolic equations are converted into new features, such as the following: \[f_{1}=\frac{LL}{PI+e_{0}}\] (20) \[f_{2}=LL\text{.log}(PI)\] (21)
3. Feature Selection: The symbolic features are evaluated based on their importance and correlation with the target variable, and only the most relevant features are retained for the next step. Step 2: Predictive Modeling with XGBoost  
1. Dataset Augmentation: The refined features from GP are added to the original dataset, enriching the input space for XGBoost. 2. Training XGBoost: XGBoost is trained on the augmented dataset. The combination of raw and GP-derived features allows XGBoost to capture complex patterns and residual errors that are not fully explained by GP. 3. Evaluation: The performance of the model is assessed using metrics such as the MSE and \(R^{2}\). By using GP's symbolic equations, this approach improves XGBoost's ability to generalize complex nonlinear relationships. Method 2. Integrated Hybrid Approach  
This approach iteratively combines GP and XGBoost, with feedback loops allowing both methods to influence each other. Step 1: Initial Training with XGBoost  
1. XGBoost is trained in the raw features to establish a baseline model. 2. The feature importance from XGBoost is extracted to identify which features contribute most to the predictions. Step 2: Feedback to GP for Feature Discovery  
1. GP uses the ranked features from XGBoost to focus on the most influential variables. 2. It generates symbolic relationships, such as Equations (11) and (12). 3. GP-derived features are added to the original dataset. Step 3: Iterative Refinement  
1. The new dataset, enriched by GP, is fed back into XGBoost for retraining. 2. XGBoost's predictions and feature importances are analyzed, and further refinement of features is performed by GP. 3. This feedback loop continues until convergence or performance improvement stagnates. Method 3. Parallel Hybrid Approach  
In this approach, GP and XGBoost work independently, and their outputs are combined for the final prediction. Step 1: Independent Training  
1. GP is trained independently to create symbolic models for predicting \(C_{c}\), generating outputs (\({C_{c}}^{GP}\)). 2. XGBoost is trained independently on the same dataset to produce output (\({C_{c}}^{XGBoost}\)). Step 2: Weighted Combination of Outputs  
1. The predictions from both models are combined using a weighted averaging approach:  
\[C_{c}=\omega_{1}\cdot{C_{c}}^{GP}+\omega_{1}\cdot{C_{c}}^{XGBoost} \tag{22}\]  
where \(w_{1}\) and \(w_{2}\) are the weights determined through cross-validation. 2. Alternatively, a stacking ensemble can be used, where the outputs of GP and XGBoost serve as inputs to a meta-model that learns how to combine them optimally. Step 3: Final Model Evaluation  
The combined predictions are evaluated using standard performance metrics, ensuring that the complementary strengths of GP (interpretability) and XGBoost (accuracy) are fully utilized.