## 3 Methodology
### Machine Learning Models  
The five machine learning regression models (XGBoost (2.1.1), CatBoost (1.2.5), LightGBM (4.5.0), AdaBoost (1.4.2), and Histogram-Based Gradient Boosting (1.4.2)) were selected for this study due to their proven performance in regression tasks, their ability to handle complex non-linear relationships, and their diverse approaches to gradient boosting. XGBoost was chosen for its regularization techniques and efficiency in handling large datasets. CatBoost was included due to its ability to effectively handle categorical features without extensive preprocessing. LightGBM was selected for its speed and efficiency, particularly with high-dimensional data. AdaBoost was chosen as a representative boosting algorithm with a different approach to weighting and combining weak learners. Finally, Histogram-Based Gradient Boosting was included for its computational efficiency and ability to handle large datasets with reduced memory consumption. This combination of models allows for a comprehensive comparison of different gradient boosting techniques and their suitability for predicting PV system efficiency. While Deep Neural Networks (DNNs) have shown promise in regression tasks, particularly those involving non-categorical parameters, we opted to focus on gradient boosting methods in this study for several reasons. First, gradient boosting methods are known for their ability to achieve high performance with relatively small datasets, which is relevant given the size of our dataset (_n_ = 468). Second, gradient boosting methods often require less hyperparameter tuning than DNNs, making them more efficient to implement and optimize. Third, the interpretability of gradient boosting models, particularly when combined with XAI techniques like SHAP and LIME, aligns with our goal of understanding the factors influencing PV system efficiency. While future work could explore the application of DNNs to this problem, we believe that the gradient boosting methods provide a strong balance of performance, efficiency, and interpretability for the scope of this study. * Extreme Gradient Boosting (XGBoost)  
XGBoost is a sophisticated version of gradient-boosting techniques, enhanced for computational efficiency and model efficacy [20]. It employs a gradient boosting framework that constructs additive models in a sequential manner, facilitating the optimization of any differentiable loss functions. XGBoost employs regularization methods (L1 and L2) to mitigate overfitting, hence improving the model's generalization capabilities. XGBoost utilizes second-order Taylor series expansions of the loss function for enhanced accuracy and computational efficiency. It facilitates parallel processing, manages missing values internally, and incorporates sparsity-aware algorithms, rendering it exceptionally efficient for large-scale and high-dimensional data. XGBoost's characteristics render it especially adept at modeling the intricate non-linear interactions inherent in photovoltaic system efficiency prediction [20]. Figure 1 depicts XGBoost, which exemplifies the gradient boosting framework, wherein each successive tree models the residuals of preceding trees to optimize the objective function. Likewise, CatBoost, LightGBM, AdaBoost, and Histogram-Based Gradient Boosting adhere to boosting concepts, although they enhance training using distinct methodologies. CatBoost adeptly manages categorical data, LightGBM employs leaf-wise tree development, AdaBoost allocates increased weights to misclassified instances, and Histogram-Based Gradient Boosting enhances training speed by utilizing binned feature values. In the figure, functions \(f_{1}(X,\theta_{1})\), \(f_{2}(X,\theta_{2})\),..., \(f_{k}(X,\theta_{k})\) represent each subsequent tree that models the residuals from previous iterations, aiming to reduce errors in the dataset. Each tree splits the data based on the residuals calculated by the objective function to minimize overall error.