# Building Energy Efficiency: Using Ensemble Learning to Forecast the Load on Heating, Ventilation, and Air Conditioning Systems

Nihad Aghbalou, Abderafi Charki, Hanae Errousso, Youssef Filali, El Arbi Abdellaoui Alaoui

► To cite this version:

Nihad Aghbalou, Abderafi Charki, Hanae Errousso, Youssef Filali, El Arbi Abdellaoui Alaoui. Building Energy Efficiency: Using Ensemble Learning to Forecast the Load on Heating, Ventilation, and Air Conditioning Systems. Journal of Engineering, 2025, 2025 (1), 10.1155/je/4764874. hal-05325072

HAL Id: hal-05325072

https://univ-angers.hal.science/hal-05325072v1

Submitted on 21 Oct 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

Research Article

# Building Energy Efficiency: Using Ensemble Learning to Forecast the Load on Heating, Ventilation, and Air Conditioning Systems

Nihad Aghbalou,<fnref n="1" /> Abderafi Charki ID,<fnref n="2" /> Hanae Errousso,<fnref n="1" /> Youssef Filali,<fnref n="3" /> and El Arbi Abdellaoui Alaoui<fnref n="4" />

Correspondence should be addressed to Abderafi Charki; abderafi.charki@univ-angers.fr

Received 16 November 2023; Revised 11 January 2024; Accepted 1 September 2025

Academic Editor: Giovanni Angiulli

Copyright © 2025 Nihad Aghbalou et al. Journal of Engineering published by John Wiley & Sons Ltd. This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

Currently, many countries are prioritizing energy efficiency in buildings as a crucial component of the global energy transition to decrease dependence on foreign energy and mitigate environmental impacts caused by wasteful energy consumption. Efficient management of energy consumption in buildings helps improve occupant comfort and optimize the use of various consumer products. Previous work in load forecasting has often used slack feature selection methods, limiting their ability to identify the most relevant variables and thus affecting their performance and generalization. Furthermore, few studies have explored the combined potential of hyperparameter tuning and exhaustive variable selection, particularly with ensemble models recognized for their robustness. While complex approaches such as deep neural networks have been used, their tradeoffs between accuracy, overfitting, interpretability, and computational cost remain poorly evaluated. Here, the aim is to develop an innovative approach combining exhaustive feature selection and fine-tuned hyperparameter optimization to significantly improve the accuracy of load forecasts in future smart urban environments while ensuring interpretability and computational efficiency. Multiple ensemble machine learning approaches, each with distinct model architectures, training methodologies, and performance characteristics, are experimented, including linear regression, decision tree regressor, gradient boosting machine, AdaBoost, XGBoost, light gradient boosting, and CatBoost, as well as a dense neural network with eight key building features: relative compactness, surface area, wall area, roof area, overall height, orientation, glazing area, and glazing area distribution. Findings indicate that XGBoost performs best for heating load predictions, closely followed by gradient boosting machine. For cooling load forecasts, the order is reversed. The dense neural network ranks third in cooling load prediction accuracy. The study also reveals that XGBoost and gradient boosting machine maintain high accuracy and robustness even with reduced input features, providing insights into the most influential building parameters for HVAC (heating, ventilation, and air conditioning) load prediction. The models achieved an improvement of up to 31% in MSE (mean squared error) for XGBoost and 26.2% for GBM (gradient boosting machine), demonstrating their robustness and efficiency. In summary, the proposed methodology is an innovative hybrid framework that combines exhaustive feature selection with an ensemble learning method. It proves to be practical and relatively straightforward to implement, particularly for initial load estimations during the early stages, aiding sustainable building design.

**Keywords:** boosting algorithms; building energy efficiency; ensemble learning; feature selection; HVAC load prediction; mixed data analysis

<footnote n="1">¹EIGSI, Casablanca, Morocco</footnote>

<footnote n="2">²LARIS, Polytech, University of Angers, Angers, France</footnote>

<footnote n="3">³Faculty of Medicine and Pharmacy, Mohammed V University, Rabat, Morocco</footnote>

<footnote n="4">⁴Image Laboratory, Department of Sciences, Moulay Ismail University, Meknes, Morocco</footnote>
# 1. Introduction

The sharp rise in global energy consumption has brought concerns about resource depletion, supply problems, and major environmental impacts (including ozone layer damage, global warming, and climate change). Residential and commercial buildings in developed nations consume between 20% and 40% of all energy, more than other major businesses, including the transportation sector [1]. Moreover, the rising trend in energy consumption will persist due to the expanding demand for comfort and building services. In most buildings, heating, ventilation, and air conditioning (HVAC) systems use 20% of the energy [2]. For a variety of building types, HVAC systems are required, including residential residences, offices, factories, hospitals, schools, and more. These systems include components like heating units (e.g., furnaces, boilers, electric heaters, and heat pumps) and systems that distribute heat, such as forced air (ductwork, radiant heating, or baseboard heaters), cooling equipment (e.g., air conditioners), and ventilation systems. Ventilation systems may include fans and air exchange mechanisms. Common air conditioning systems include central air conditioners, window units, split systems, and evaporative coolers. Modern HVAC systems also frequently use advanced control systems to regulate and optimize the indoor environment and cutting-edge technologies to control systems, maximize energy efficiency and lower operating expenses [3]. The cost of running and maintaining such equipment might be high. The operation of this equipment can be scheduled and sequenced with the use of load forecasting, ensuring that it operates effectively when necessary and idle during periods of low load. To maintain a balance between supply and demand in HVAC systems, particularly in real-time scenarios, precise load forecasting is essential. Moreover, HVAC systems perform at their peak efficiency with accurate load forecasting. When the heating pad (HL) or cooling load (CL) can be predicted with precision, the HVAC system may be adjusted to deliver just the necessary amount of heating or cooling, preventing excessive or inefficient energy consumption. Hence, cost reductions and increased energy efficiency are the immediate results of this [4]. Other practical significance of accurate load forecasting including planning maintenance activities, grid stability, integration with renewable energy and resilience, and emergency preparedness in emergency situations such as extreme weather events. For instance, Figure 1, adopted from [5] displays an HVAC system's energy balance. Thermal load in the zones ($\dot{Q}^{O/A}$), active heating/cooling input ($\dot{Q}^{H/C}$), and heat gain/loss via air exchange with the surrounding environment ($\dot{Q}^{O/A}$) are the three categories of thermal energy flows that have been recognized.

The system needs to regulate energy flows to achieve constant indoor conditions through a specific relationship:

$$Q^{Load} + Q^{H/C} + Q^{O/A} = 0, \quad (1)$$

where $\dot{Q}^{O/A} = \dot{m}^{O/A}(h^{O/A} - h^{E/A})$, $\dot{m}^{O/A}$ is the mass flow rate of outside air, $h^{E/A}$ is the specific enthalpy of exhaust air

<figure><img src="image_3.png" /><figcaption>FIGURE 1: HVAC system energy balance.</figcaption></figure>

(return or expelled air), and $h^{O/A}$ is the specific enthalpy of outside air.

Heating and cooling systems must operate continuously to counterbalance both internal heat changes and outdoor air effects for maintaining stable indoor conditions.

The process depends on exact predictions of zone loads together with ventilation needs. The ability to anticipate demand enables HVAC systems to function efficiently by providing the required energy amounts at specific points in time. The predictive strategies minimize energy usage while lowering operational costs and decreasing the environmental footprint of buildings.

The integration of advanced load forecasting in HVAC management creates immediate efficiency benefits and serves sustainability targets at broader levels. These approaches optimize energy use to support climate change mitigation efforts through smart building technologies that align with global environmental goals.

Numerous significant factors, including weather, installed technology and equipment, occupant behavior, and building physics (dimensions, material conductivity, coefficient form, length-to-width ratio, window-to-floor ratio, window-to-wall ratio, etc.), influence a building's energy consumption. Since the HL and CL consume the most energy, all influencing elements must be considered in cost and scientific analysis in order to estimate these loads and create accurate models of energy consumption. Early predictions of HL and CL are especially helpful in selecting the appropriate HVAC system to ensure a comfortable interior air quality while creating sustainable structures with a greater emphasis on energy efficiency. Building energy consumption forecasts have been the subject of numerous research studies recently. The literature categorizes forecasting methods for building energy consumption into two main categories: both data-driven and physical methods [6]. The physical-based models create equations that can be used to estimate a building's energy consumption by utilizing theoretical presumptions, equations, the physical characteristics of the building, and thermodynamic principles. This method entails simulating building energy use with algorithms [7]. The physical model's ability to anticipate energy usage effectively depends on several elements, including solar radiation,
interior load, HVAC system, and physical characteristics. These qualities can be difficult to achieve, and inadequate simulation methods may not be able to completely consider the complex environmental factors that affect energy use, which would result in subpar model performance [7]. For instance, Curto et al. [8] used a TERMUS BIM software and a thermal transmittance calculation to study the structure of the “Ex Institute of Zoology and Comparative Anatomy” in Palermo in all its components. The authors found that the process of redeveloping structures that are a part of cultural heritage can be replicated and modified to consider the various architectural features, building materials, legal restrictions, and time-related restrictions. Furthermore, the obtained values result in significant energy savings of 35% on overall consumption, an 8% reduction in consumption with a single intervention, and a 20% reduction by developing a new centralized system to replace the individual splits, replacing the light fixtures with LED lights, and installing photovoltaic panels on the building's highest roofs. The new fixtures have thermal break glass and are charged with argon gas. However, data-driven methods, also called data-centric models, are produced by processing and examining large datasets of building energy consumption and characteristics in order to find patterns, linkages, and insights that can be drawn straight from the datasets [9]. Accurately estimating the energy consumption of buildings is challenging due to the complicated and nonlinear relationship between the influencing components and the energy performance of buildings.

Building energy load prediction studies through previous review papers demonstrates machine learning methods deliver improved accuracy together with nonlinear relationship modeling capabilities compared to traditional physical model approaches. Feature selection methods bring essential benefits to building models by minimizing their complexity and computational overhead and preventing overfitting, which proves vital for real-world applications. Research gaps persist according to multiple studies that examine hybrid models which combine multiple algorithms to maximize their strengths while also revealing limited deep learning applications for automatic feature extraction and insufficient assessment of model robustness in noisy or incomplete data scenarios. Most previous research investigates individual algorithms instead of conducting multiple model comparative studies, which restricts understanding about optimal model choice across different scenarios. The purpose of this research is to present a workable data-driven methodology for preliminary load forecasting during the architectural design phase of residential buildings. The dataset from building information modeling consisting of 768 different attributes is considered to analyze the HL and CL of buildings while taking eight affecting factors into account, such as relative compactness (RC), area of the surface, walls, roof, height overall, orientation, area of the glazing, and distribution of the glazing area. Exhaustive feature selection techniques have been carried out to reduce data dimensionality. The idea is to eliminate redundancy, given that certain features might be highly correlated with each other, that is, they contain essentially the same information. Feature selection techniques can

identify these correlations and thereby minimize the number of features and filter out any noise or measurement errors by focusing on major directions of variation. Consequently, machine learning algorithms benefit from coherent, homogeneous data. Another important benefit of dealing with excess dimensionality is to decrease model complexity, minimizing training time and preventing overfitting when training data is limited. In a similar vein, it might even reduce data storage requirements and computing costs associated with training and using models. To the best of our knowledge, deep learning algorithms are less explored with this issue. However, most authors have resorted to either conventional or hybrid approaches. Deep learning, notably dense neural network, offers several significant advantages, making it a powerful and versatile approach for machine learning. Its major strength lies in its ability to automatically extract relevant features from raw data. This means that deep neural networks can effectively learn to represent input data, capturing complex relationships between input features and the target variable without needing to explicitly specify any form of these relationships. As a result, they are suitable for modeling continuous variable prediction tasks where relationships are nonlinear. They are also capable of handling interactions between input features. In regression problems, it is crucial to consider these interactions to obtain accurate predictions. Moreover, deep neural networks are trainable for different data distributions. They are not limited by strict assumptions about data patterns. The remainder of the document is arranged as follows. A summary of the most pertinent studies and recent publications on the subject is given in Section 2. Section 3 provides information on the materials and explains the fundamental ideas that underpin baseline models. Results from the conversations and framework utilized are presented in Section 4. Results of the paper are summed up in Section 5.

## 2. Literature Review

There has been continuing research for some time now on how to handle energy-saving issues and create a safe and comfortable indoor environment. In addition, future predictions indicate that the need for efficient HVAC systems is expected to rise. To enhance HVAC system performance and design, energy modeling and model predictive control (MPC) play an imperative role in the efficient design and operation of HVAC systems at every decision step [10]. Its use has recently become more potent because the majority of modern buildings are outfitted with intricate heating systems and/or on-site intermittent systems, such as renewable energy sources and/or grid connections [11]. Additionally, the effectiveness of the MPC for developing applications has been attributed to recent trends in the inexpensive cost of pertinent hardware components (e.g., controllers, communication infrastructure, and sensors), software, and the ease of algorithm implementation [12]. Kim et al. [10] provided a full evaluation of building energy modeling approaches and state-of-the-art updates for MPC HVAC applications linked to the modeling methodologies and their practical applications. Authors in [13] analyzed historical energy consumption profiles of different buildings, small-
and medium-sized, residential, industrial, and service buildings powered by renewable energy sources. Findings of this study revealed that structures of residential buildings demonstrate the greatest volatility in electricity. This is due to the random nature of the process; even if the actions repeat daily, resident habits, employment status, and age still play a role. A variety of uncontrollable factors, such as the climate and even a person's health, affect how energy is consumed. The day being examined, which may be a movable day like a Saturday or Sunday, is also important in defining the type of the profile. Cai et al. employed the SVR supervised machine learning algorithm in a recent study [14] to forecast and predict the energy consumption of buildings. Six population-based algorithms were used to improve the parameters (slime mold algorithm, AEO, sparrow search algorithm, gray wolf optimizer, artificial bee colony, and arithmetic optimization algorithm (AOA)). Consequently, for both heating and cooling assessments, the SVR-artificial ecosystem-based optimization hybrid model performed the best out of the six choices that were considered. Using data from a sizable Saudi Arabian commercial building, heap optimization based generalized intelligent neural fuzzy control (HO-GINFC) was introduced in [2] to anticipate the cooling demand for air conditioning systems based on cold thermal storage. In comparison to traditional techniques like mean impact value combined with improved gray wolf optimizer-based support vector regression (SVR), Elman neural network, deep reinforcement learning, and gradient boosting decision trees (DTs), the suggested study showed an improved performance on the forecasting and control capabilities of the cold thermal storage-based air conditioning systems. To anticipate the HVAC system, Huang and Li [15] enhanced the ant colony optimization and wavelet neural network (I-ACO-WNN) model. To maximize WNN learning rates, the self-adaptive mutation operator was used to prevent an early local convergence of ants. The results of the Spearman technique, which show that a building's total height and roof area (RA) have the most effects on load, were used to train the model. In order to create an ensemble prediction model known as WGPRLSB, Pachauri and Ahn [16] examined three techniques: marine predator optimization (MPO) algorithm, least squared boosted (LSB) regression trees, and Gaussian process regression (GPR). Additionally, a sensitivity analysis was performed to determine the degree to which the input variables of the suggested predictive model influence the model's predicted output. It was discovered that the most significant input variables for the energy consumption forecasts in the case of HL are RA, surface area (SA), and glazing area. For HL, these variables' $S_A$ levels of dependence were 0.9353, 0.9279, and 0.9020. On the other hand, the critical input variables in the CL case are SA, RA, wall area (WA), and glazing area, with $S_A$ equal to 0.9554, 0.9572, 0.9297, and 0.9356, respectively.

To solve the problem of time series prediction with a small dataset, Kim et al. [17] presented a building energy load forecasting approach based on long short-term memory (LSTM) and transfer learning (TL) strategies. Practical applications continue to raise questions given the variety of building types and weather patterns, even though the

developed LSTM-TL model can achieve better performance than the prediction model which only uses LSTM under different weather conditions with relatively insufficient datasets, according to the results of their practical applications. Wang [18] introduced a model based on MABC-BP to better address China's residential structures' needs for energy conservation. To maximize the energy-saving benefit, this model optimizes residential building energy conservation from multiple perspectives. Bian et al. [19] selected three metaheuristic optimization algorithms including Aquila optimizer (AO), AOA, and parallel swarm optimization algorithm (PPSO) and combined them with to produce hybrid models such as categorical boosting (Cat-Boost)-AO, CatBoost-AOA, CatBoost-PPSO, adaptive boosting (AdaBoost)-AO, AdaBoost-AOA, and AdaBoost-PPSO to improve power management and resource efficiency in HVAC systems. Results revealed that the model CatBoost-AO has outperformed other models based on $R^2$, Nash–Sutcliffe efficiency coefficient, the test score of Jensen–Shannon divergence, root mean squared error (RMSE), and the variance account for. Mohan and Pachauri [20] used a stacked ensemble model combining extreme gradient boosting (XGBoost), DT, and random forest (RF) algorithms using Bayesian optimization for hyperparameter values. Compared to SVR, K-nearest neighbors (KNNs), artificial neural networks (ANNs), ridge regression, and Lasso regression, their model reached the lowest RMSE of 0.484 and 0.948 for HL and CL, respectively. Zaki et al. [21] conducted four models including KNN regression, extra tree regressor (ETR), voting hybrid regression (VHR), and multilayer perceptron regression (MLPR) to forecast the office buildings' CLs. These two models (ETR and VHR) were found to be very successful in representing the complex relationships between variables such as the building location and size, as well as the external environmental conditions and their effects on the building energy consumption.

This study's primary goal is to predict HVAC requirements using a well-established ensemble learning approach. We conducted a thorough evaluation of several prominent machine learning algorithms to create a dependable system for estimating heating and CLs in future smart urban environments. The algorithms examined include the following:

* Linear regression (LR)

* Decision tree regressor (DTR)

* Gradient boosting machine (GBM)

* Adaptive boosting (AdaBoost)

* XGBoost

* Light gradient boosting machine (LightGBM)

* Categorical boosting (CatBoost)

* Dense neural network

These fundamental models have shown promise across various fields. For example, Sabri et al. [22] demonstrated their effectiveness in healthcare applications. In the realm
of urban planning, Errousso et al. [23, 24] applied similar techniques to smart parking systems and parking availability forecasting. Additionally, these methods have been successfully employed in energy-related research, as evidenced by Baseer et al. [25].

# 3. Methods

## 3.1. Ensemble Learning Methods

3.1.1. LR. Basically, the main idea behind ensemble learning is to increase machine learning performance by combining multiple models, also known as base learners, instead of using a single model [26]. Since its inception in the early 19th century, LR has been in use. It is one of the most reputable and well-established statistical modeling techniques that forecasts a continuous outcome variable using one or more input features [27]. Assuming a linear relationship between the predictors and the target variable, they may be stated as follows:

$$y_i = b_0 + b_1 x_{i1} + \dots + b_p x_{ip} + \epsilon_i; i = 1, 2, \dots, n, \quad (2)$$

where y is the dependent variable, x is the independent variable, b is the intercept, n is the number of observations, b₀ is the y-intercept, b₁, ..., bₚ are the regression coefficients, and εᵢ = yᵢ - yᵢ is the residual error reflecting the difference between the observed (yᵢ) and fitted (yᵢ) values.

The estimated coefficients bᵢ, i ∈ 0, 1, 2, ..., p, are the values maximizing the likelihood function or minimizing the sum of squared errors for the sample:

$$b \in \operatorname{argmin}_{(b \in \mathbb{R}^p)} \sum_{(i=1)}^{n} (y_i - y_i)^2. \quad (3)$$

In addition to the previously mentioned hyperparameters, LR can also involve feature scaling and feature selection techniques. LR's training process involves finding the best-fitting linear equation by minimizing a loss function (e.g., mean squared error (MSE)) using an optimization algorithm like gradient descent.

3.1.2. Decision Regression Trees (DRTs). DRTs are machine learning models that use a structured network of decision and leaf nodes. These trees are typically built using methods like recursive partitioning. The tree-building process continues until certain criteria are met, such as reaching a specified maximum depth or having a minimum number of data points in each leaf. One challenge with regression trees is their susceptibility to overfitting, especially if allowed to grow too complex. This is because each model in the tree can be quite powerful on its own. The tree construction process involves repeatedly dividing the data based on features to predict a continuous target variable in the leaf nodes. It begins with a single root node containing all data points. Each nonterminal node splits into two child nodes, which then become parent nodes for further splits unless they are leaf nodes. When choosing how to split nodes, the algorithm selects a feature and split point that minimizes the squared error between observed

values and the mean within each resulting node. This approach aims to create nodes with similar data points (high homogeneity) or, equivalently, to minimize the impurity within nodes. The mathematical representation of this optimization process can be expressed as follows:

$$\min_{x_k,s} \left[ \sum_{C(x_k,s) \in \text{Leaves}(T)} \sum_{i \in C(x_k,s)} (y_i - y_C)^2 \right], \quad (4)$$

where y_C is the prediction for leaf C of tree T and is equal to the sample mean of the dependent variable in this cell. The tree building process goes on until it is impossible to continue. It is stopped when the minimum value of the splitting rule, maximum number of leaves, or minimum number of data points in the terminal nodes is reached. DRT constructs a tree by repeatedly splitting data based on feature conditions that maximize information gain (or minimize impurity). This continues until stopping criteria are met. In this study, the stopping criterion was the parameter max depth and tuned to 10 leaves maximum.

3.1.3. GBM. GBM combines multiple DTs to create a robust predictive model. It uses a gradient-boosting framework to optimize the ensemble. GBM hyperparameters include the number of trees, tree depth, learning rate, subsample ratio, and regularization terms.

$$y = \sum_{(i=0)}^{K} y_i = \sum_{(i=0)}^{K} y_{i-1} + \gamma_i \cdot M_i, \quad (5)$$

where K is the number of iterations, M₀ is the initial guess, Mᵢ is the function increment, and γᵢ is a constant chosen at each step to minimize the following equation:

$$\gamma_i = \operatorname{argmin}_y \sum_{j=1}^{n} \frac{1}{2} (y_j - y_{i-1}(x_j) - \gamma \cdot M_i(x_j))^2. \quad (6)$$

This learning system employs an iterative approach to improve prediction accuracy. It sequentially builds and integrates new models to refine its estimates of the target variable. The core concept driving this algorithm is the creation of additional base learners that align closely with the negative gradient of the error function. This error function is calculated based on the performance of the entire model ensemble. For measuring prediction errors, the algorithm utilizes the standard squared difference between predicted and actual values.

$$y_i - y_i = \frac{-1}{2} \frac{\partial(y_i - y_i)^2}{\partial y_i}; i = 1, 2, \dots, n. \quad (7)$$

GBM builds DTs sequentially, focusing on correcting errors made by previous trees by assigning more weight to misclassified data points.

3.1.4. AdaBoost. AdaBoost is one boosting technique that reduces the chance of underfitting by combining several
weak base learners with the Bayesian strategy. From this idea, the iterative AdaBoost was created [28]. The AdaBoost regressor iteratively reweights training samples depending on the prediction error to generate weak regressors with high bias error and low variance error. Every data sample has the same weight at first, but in the following iteration round, the weights of the samples are changed; each data sample is given a weight that indicates how important it is to be chosen as a training sample. This is how the subsequent regressor identifies cases where the previous phase's predictions proved to be inaccurate. Stated differently, its adaptability stems from either increasing the weights of the incorrectly categorized samples or decreasing the weights of the correctly classified ones. The misclassified samples, which may be close to the classification boundary, are highlighted by the new classifier, and finally reduce the error. The weighted majority vote is used to make the final prediction. By combining the strengths of the strong and weak base classifiers, a model with lower low variance and bias errors is produced. The classifier generated by the selection process, upon algorithmic termination at iteration I, is as follows:

$$H(x) = \text{sign} \left( \sum_{t=1}^{T} \alpha_t h_t(x) \right), \quad (8)$$

where $h_t(x)$ is the output of the weak classifier t for input x, t is the weight assigned to the classifier calculated as follows:

$$\alpha_t = 0.5 * \ln \left( \frac{(1|E)}{E} \right), \quad (9)$$

and E is the error rate.

### 3.1.5. XGBoost.
Large-scale machine learning problems can be effectively handled using the well-known enhanced implementation of an optimized gradient boosting tree-based technique called XGBoost [29]. Its outstanding performance and reasonably priced time and memory complexities have led to its widespread application in numerous research domains. The fundamental principle of this approach is to train using a sequential tree-building technique with parallel implementations. The tree is then developed to its maximum depth and backwards pruned until the regularized loss function rises below a previously set threshold. The following is the expression for XGBoost's objective function:

$$\mathcal{L}^t = \sum_{i=1}^{n} l(y_i, y_i^{(t-1)} + f_t(x_i)) + \Omega(f_t), \quad (10)$$

where $l(y_i, \hat{y}_i)$ is the loss function that represents the error between the predicted and actual values, $\Omega(f_t)$ is the regularization term, and $f_t$ is an independent function such as the following:

$$y_i = \sum_{k=1}^{K} f_k(x_i), \quad (11)$$

and K is the number of DTs.

### 3.1.6. LightGBM.
Tree-based learning techniques are used by powerful computational algorithms known as LightGBM [30]. They were developed to speed up model training, use less memory, and improve predictability. The approaches grow vertically, that is, like leaves, in contrast to other algorithmic trees that develop horizontally. LightGBM selects leaves with notable reductions in growth. Despite increasing leaves, they can decrease losses more than level-based tactics. When trees grow along the direction of their leaves, the complexity of the model can also increase, perhaps resulting in excessive data modifications when using smaller datasets [31]. Limiting the depth of its tree structure is another way that LightGBM can prevent overfitting. Splits in LightGBM are computed considering both weak and strong learners. To distinguish between small and large gradient values, this method assesses gradients.

LightGBM model iteratively minimizes the regularized objective function as follows:

$$\mathcal{L}^{(t)} = \sum_{i=1}^{n} \left[ L(y_i, F_{t-1}(x_i)) + g_i f_t(x_i) + \frac{1}{2} h_i f_t^2(x_i) \right] + \Omega(f_t), \quad (12)$$

where $g_i = \partial_{F_{t-1}} L(y_i, F_{t-1}(x_i))$ and $h_i = \partial_{F_{t-1}}^2 L(y_i, F_{t-1}(x_i))$ are first- and second-order gradients. $f_t(x)$ is the prediction from the tth weak learner (tree). $F(x)$ the final prediction after adding up all boosted trees. Each tree partitions the data into L leaves, and the optimal weight $w_l$ for leaf l is computed as follows:

$$w_l = - \frac{G_l}{H_l + \lambda}, \quad (13)$$

where $G_l$ is sum of gradients in leaf l and $H_l$ is sum of Hesians in leaf l and $\lambda$ the regularization parameter on leaf weights.

The optimal split is determined by maximizing the gain criterion:

$$\mathcal{G}_{\text{split}} = \frac{1}{2} \left[ \frac{G_L^2}{H_L + \lambda} + \frac{G_R^2}{H_R + \lambda} - \frac{(G_L + G_R)^2}{H_L + H_R + \lambda} \right] - \gamma, \quad (14)$$

where $G_L = \sum_{i \in I_L} g_i$, $H_L = \sum_{i \in I_L} h_i$ (similarly for $G_R, H_R$) are aggregated statistics for left/right nodes, with $\lambda$ as $L_2$ regularization and $\gamma$ as split complexity penalty. The final model combines T weak learners with shrinkage rate:

$$F(x) = F_0 + \eta \sum_{t=1}^{T} f_t(x). \quad (15)$$

### 3.1.7. CatBoost.
The drawbacks of existing DT techniques are addressed by CatBoost, which typically calls for data
<table><caption>TABLE 1: Comparison of the used machine learning methods.</caption><thead><tr><td>Method</td><td>Pros</td><td>Cons</td></tr></thead><tbody><tr><td>Linear regression [26, 27]</td><td>Simple, interpretable, computationally efficient, suitable for linear relationships</td><td>Limited in capturing nonlinearities, performance drops with complex data patterns, sensitive to multicollinearity</td></tr><tr><td>Decision regression trees [33]</td><td>Easy to interpret, handles nonlinear relationships, requires minimal data preprocessing</td><td>Prone to overfitting, especially with deep trees, can be unstable with small variations in data, less generalizable</td></tr><tr><td>Gradient boosting machine [34]</td><td>Very good accuracy, capable of modeling complex relationships, flexible (customizable loss function)</td><td>Slow training, prone to overfitting, requires precise hyperparameter tuning</td></tr><tr><td>AdaBoost [28, 35]</td><td>Reduced bias and variance, simple to implement, good performance on clean data</td><td>Very sensitive to noisy data and outliers, less efficient on complex data</td></tr><tr><td>LightGBM [30, 31]</td><td>Very fast training, handles large datasets efficiently, native support for categorical variables</td><td>Less efficient on small datasets, may be biased toward variables with few distinct values</td></tr><tr><td>XGBoost [29]</td><td>Very efficient and fast (memory and CPU optimization), built in regularization against overfitting</td><td>More complex to understand and tune, may be too resource intensive for very large ensembles</td></tr><tr><td>CatBoost [35]</td><td>Automatically handles categorical variables, less preprocessing required, less susceptible to overfitting</td><td>Relatively new, less extensive documentation, slower than LightGBM in some cases</td></tr></tbody></table>

pretreatment to execute one-hot encodings, transform categorical string variables into numerical values, and so forth. This approach can employ a combination of category and non-categorical explanatory variables directly, without the need for preprocessing [32]. This approach can employ a combination of category and noncategorical explanatory variables directly, without the need for pre-processing. Pre-processing is what the algorithm takes care of. CatBoost encodes category entities using ordered encoding. Target statistics of all rows prior to a data point are taken into consideration by ordered encoding in order to compute a value and substitute the categorical item. Another characteristic that sets the CatBoost algorithm apart is its utilization of symmetrical trees. This indicates that at every depth level, every decision node employs the same criterion. Using the same features, CatBoost divides the training sample into left and right divisions, producing a tree with precisely 2k leaves and a depth of k. During training, a set of DTs is constructed one after the other. Every additional tree is created at a lower loss than the previous tree. The initial parameters control the number of trees to prevent overfitting. If overfitting occurs, CatBoost may terminate training before the parameters specify. The algorithm model can be defined as follows:

$$F(x) = \sum_{t=0}^{T} a_t h_t(x, w_t) = \sum_{t=0}^{T} f_t(x, w_t), \quad (16)$$

where $F(\cdot)$ is the output of the entire DT; $x$ is the input of the sample; $w$ is the parameter of the whole DT; $\alpha_t$ is the weight of the $t$th tree; $T$ is the number of trees; $h_t(\cdot)$ is the output of the $t$th DT; $w_t$ is the parameter of the $t$th DT; and $f_t(\cdot)$ is the

weighted output of the $t$th DT. The parameters of the optimal model are expressed as follows:

$$(a_t, w_t) = \arg \min_{a, w} \sum_{i=1}^{n} L(y_i, F(x_i, w)), \quad t = 1, 2, \dots, T, \quad (17)$$

where $L(\cdot)$ is the loss function; $y_i$ is the actual output of sample $i$; $x_i$ is the input of sample $i$; and $N$ is the number of samples.

It should be noted that machine learning algorithms have limitations as they have advantages. For example, ensemble models require more computational time, especially when the data size increases compared to LR or DTs. In addition to the number of parameters to be adjusted, which increases computational complexity and hinders the interpretability and deployment of model decisions, the latter remain sensitive to noisy or incomplete data. Hereinafter, Table 1 presents a comparison of the different methods.

### 3.2. Dense Neural Network.
A dense neural network, also known as a fully connected neural network or feed-forward neural network, is a fundamental architecture in deep learning. It consists of layers of interconnected neurons (or nodes), where each neuron in one layer is linked to every neuron in the subsequent layer. This interconnected structure allows dense neural networks to model complex relationships and learns intricate patterns in data [36, 37]. In a dense neural network, each neuron processes its input and produces an output using an activation function. The output of a neuron is typically denoted as 'a' for activation. The input to a neuron 'i' in layer 'j' is represented as 'z,' which is a weighted sum of the outputs from the previous layer, followed by an activation function 'σ':
<table><caption>TABLE 2: Description of the features in the experimental dataset.</caption><thead><tr><th>Variable</th><th>Parameters</th><th>Min</th><th>Max</th><th>Unit</th><th>Values</th></tr></thead><tbody><tr><td>X1</td><td>Relative compactness</td><td>0.62</td><td>0.98</td><td>N/A</td><td>12</td></tr><tr><td>X2</td><td>Surface area</td><td>514.5</td><td>808.5</td><td></td><td></td></tr><tr><td>X3</td><td>Wall area</td><td>416.5</td><td>318.50</td><td>m<sup>2</sup></td><td>12</td></tr><tr><td>X4</td><td>Roof area</td><td>110.25</td><td>220.5</td><td>m<sup>2</sup></td><td>7</td></tr><tr><td>X5</td><td>Overall height</td><td>3.50</td><td>7</td><td>m<sup>2</sup></td><td>4</td></tr><tr><td>X6</td><td>Orientation</td><td>2</td><td>5</td><td>m<sup>2</sup></td><td>2</td></tr><tr><td>X7</td><td>Glazing area</td><td>0</td><td>0.4</td><td>N/A</td><td>4</td></tr><tr><td>X8</td><td>Glazing area distribution</td><td>0</td><td>5</td><td>m<sup>2</sup></td><td>4</td></tr><tr><td>HL (Y1)</td><td>Heating load</td><td>6.01</td><td>43.1</td><td>kW</td><td>568</td></tr><tr><td>CL (Y2)</td><td>Cooling load</td><td>10.9</td><td>48.03</td><td>kW</td><td>636</td></tr></tbody></table>

$$z_i^{(j)} = \sum_{k=1}^{n^{(j-1)}} w_{ik}^{(j)} a_k^{(j-1)} + b_i^{(j)}, \quad (18)$$

where

$z_i^{(j)}$ is the input to neuron 'i' in layer 'j', $w_{ik}^{(j)}$ is the weight of the connection between neuron 'i' in layer 'j' and neuron 'k' in layer 'j-1', $a_k^{(j-1)}$ is the output of neuron 'k' in layer 'j-1', $b_i^{(j)}$ is the bias term associated with neuron 'i' in layer 'j', and 'n' represents the number of neurons in a layer.

The process of computing the output of the network for a given input is called forward propagation. It involves computing the activations 'a' for each neuron in each layer, starting from the input layer and moving forward through the hidden layers to the output layer. This can be represented as follows:

$$a_i^{(j-1)} = \sigma(z_i^{(j)}) \quad (19)$$

where $a_i^{(j-1)}$ is the output of neuron 'i' in layer 'j' after applying the activation function and $\sigma$ is the activation function.

The final output of the dense neural network is typically produced in the output layer. For a classification task with 'C' classes, the output layer often employs a softmax activation function to compute the class probabilities:

$$a_i^{(\text{output})} = \frac{e^{z_i^{(\text{output})}}}{\sum_{k=1}^{C} e^{z_k^{(\text{output})}}}, \quad (20)$$

where $a_i^{(\text{output})}$ is the probability that the input belongs to class 'i' and $z_i^{(\text{output})}$ is the input to neuron 'i' in the output layer.

To train the network, a loss function is defined to measure the error between the predicted output and the actual target values. Common loss functions include MSE for regression tasks and categorical cross-entropy for classification tasks. Further, the neural network is trained using backpropagation, which involves computing the gradients of the loss with respect to the network's weights and biases.

### 3.3. Random Search Optimization.
Random search is an optimization technique commonly used in machine learning and deep learning to refine hyperparameters, especially to define neural network architecture. This algorithm works according to a systematic but simple sequence of steps [38]. The process begins by defining the hyperparameter search space, which contains the range of feasible values or acceptable ranges for each hyperparameter to be refined. It goes on to determine the number of random hyperparameter combinations or samples to be evaluated. This parameter intrinsically affects the balance between exploration and exploitation of the search space. A larger number of samples enables more exhaustive exploration but requires greater computing resources. Next, hyperparameter values are randomly sampled from the search space defined for each sample. These sample combinations encompass a diverse set of hyperparameters. For each sampled hyperparameter combination, the model is trained with its values. Basically, a designated subset of the data is used to train this model. Subsequently, the model's performance is evaluated using a separate validation dataset. Throughout the iterative approach, the performance of each sampled configuration is carefully monitored. Once all specified samples have been evaluated, the configuration that gives the best performance according to the selected evaluation metric is identified. This configuration represents the optimal set of hyperparameters for the model in the predefined search space.

### 3.4. Exhaustive Feature Selection.
Wrapping methods correspond to feature selection techniques that involve selecting subsets of features and evaluating them against the performance criterion of a machine learning model. Their main characteristic consists of exploiting the model's predictive performance as feedback to decide which features to include or exclude in the selected subset. Belonging to this family, and its particularity lies in exploring all possible combinations systematically to find the one with the highest goodness value [39]. It follows a top-down methodology, commencing with an empty feature subset and progressively working towards the solution. The feature set with the greatest goodness value is deemed the most optimal [40]. Despite its capacity to guarantee the best possible solution, the
<figure><img src="image_4.png" /><figcaption>FIGURE 2: (a–h) Probability distribution estimate of input and output variables.</figcaption></figure>

exhaustive search algorithm is hampered by its exponential algorithmic complexity. Consequently, it is only practical to employ exhaustive search when dealing with datasets characterized by a limited number of features, and this is our case [41].

## 4. Experimental Data and Implementation Details

### 4.1. *Dataset Description.*
The energy efficiency dataset used in this work was created by Angeliki Xifara and analyzed
by the Oxford Centre for Industrial and Applied Mathematics at the University of Oxford [42]. Eight input parameters are included in the dataset shown in Table 2, which are used to predict two real-valued outputs, HL and CL, for 12 residential buildings with the same volume. Although the volume (771.75 m³) and materials of each construction are identical, there are differences in their RC, orientation, SA, WA, RA, overall height (OH), and glazing area distribution (GAD). The Oxford Centre for Industrial and Applied Mathematics at the University of Oxford University processed the 768-vector dataset under study. Several correction operations were carried out to ensure there were no missing data points and to eliminate any noise or inconsistent values that could skew the results:

* Delete rows with odd values and remove unnecessary columns.

* Remove duplicates from data to avoid overrepresentation of certain observations.

* Standardize, if features follow an approximate normal distribution, or normalize, if not, data to scale feature values.

* Apply mathematical transformations (such as logarithms and square roots) to stabilize variance.

## 4.2. Feature Selection in Data-Driven Methods.
Figure 2 shows the estimated probability distributions for the input and output variables. It is observed that none of the variables is distributed regularly. To identify and test the strength of a relationship between the input and output variables, the results of the statistical analysis using the Spearman rank correlation are depicted in Figure 3. This figure illustrates the significant link between the input variables X4 and X5 and the output variables HL (Y1) and CL (Y2). Specifically, substantial positive monotonic correlations are indicated by correlation coefficients of 0.89 and 0.9 between HL, CL, and input variable X5. In contrast, HL, CL, and input variable X4 have a significant negative monotonic connection with a value of -0.86. A weak correlation with X6 is noted. In addition, some coupling and correlation relationships are present among the input variables. For example, X3 is weakly correlated with X1, X2, X4, and so forth. Furthermore, there is a high association between X1 and X2, X4 and X5, X2 and X4, and X5.

## 4.3. Implementation Details.
The dataset was divided into two parts, a training set and a test set using a random allocation, with 80% of the data assigned to the training set and the remaining 20% allocated to the test set. The training set was used for model development and training purposes, while the test set served as the means for the final evaluation of the models. To optimize the hyperparameters of the machine learning models, cross-validation has been carried out [43]. The K-fold cross-validation technique divides the training set into K sets that are mutually exclusive and as equal in size as possible [44]. During this process, the model is trained on K-1 of these folds and validated on the

<figure><img src="image_5.png" /><figcaption>FIGURE 3: The correlation matrix between input and output variables obtained using the Spearman rank relations.</figcaption></figure>

remaining onefold. This procedure is repeated K times, with each fold and its complementary set alternating between validation and training roles. Ultimately, the performance of the cross-validated model is determined by averaging the performance metrics across the K validation sets [45]. The proposed methodology is summarized in Figure 4. The methodology of the proposed solution consists of five main stages: data preprocessing, boosting model training, deep learning model training, model performance testing, and feature selection. Each stage comprises several sub-steps, the key ones being as follows:

* Preprocess the dataset and divide it into two distinct subsets: 80% for training and 20% for testing

* Optimize the boosting algorithm's hyperparameters by randomly splitting the training set into k folds of equal size using cross-validation

* Train, on the training set, the selected boosting algorithms

* Optimize dense neural network architecture, including the number of layers and the number of neurons in each layer, with random search

* Evaluate all learning methods by making predictions on the test set and computing several error metrics

* Reduce data dimensionality by retaining only the most informative features, thanks to exhaustive feature selection

* Train the final model only with the selected features

# 5. Results and Discussion

## 5.1. Model Performance and Hyperparameter Tuning.
The main emphasis of this section is on the experimental results obtained from data processing, model building, and model evaluation. To address the time-consuming manual tuning
<figure><img src="image_6.png" /><figcaption>FIGURE 4: The proposed methodology adapted from [46, 47].</figcaption></figure>

of hyperparameters and long search times in large parameter spaces, the optimal set of hyperparameters that yield the best model performance was found using a K-fold cross-validation technique. The average size of errors between predicted and actual values is measured using the mean absolute error (MAE); a lower MAE denotes better model performance because lower prediction errors are desirable. The performance of the chosen baseline models in the prediction and comparative trials was assessed using relevant performance indicators, including MAE, MSE, $R^2$, EXV, RAP, MPD, and MGD. The graphical representations of the confusion matrices corresponding to the outcomes of the classical procedures LR and DRT are shown in Figure 5. The first two columns clearly indicate that the combination of fit intercept = False (i.e., excluding the intercept term in the calculations) and copy X = True (i.e., copying the input data before converting it for internal use) yields the lowest MAE value. The DRT's performance when training models to predict HL and CL is displayed in the final column. In order to create the HL predictor, the darkest hue correlates to the lowest MAE value, which occurs when the maximum tree depth is 30 and the minimum number of samples required to split an internal node is 5. Nevertheless, lower MAE values are observed for the CL predictor when the maximum tree depth is 70 and the minimum number of samples required to split an internal node is 5. Figure 6 illustrates the corresponding MAE values across the space defined by the learning rate and the number of estimators, highlighting the robustness of AdaBoost's loss function. For predicting HL and CL, AdaBoost performs well when using an exponential loss function, 150 estimators, and a learning rate of 0.1 in both models.

The effectiveness of splitting in the model was evaluated using Fridman MSE. Figures 7a, 7b, 7d, and 7e demonstrate how the GBM model's performance changes when adjusting

early stopping estimators and learning rate. Figure 7c,f reveal a notable dependence on the XGBoost input data. For the HL model, optimal training was achieved with 10 parallel trees and a maximum depth of 30. In contrast, the CL model performed best with a maximum depth of 20 and 5 parallel trees. Figure 8 presents the key hyperparameters for LightGBM models, including the learning rate, maximum tree depth, and booster type. The analysis showed that a learning rate of 0.2 and a maximum depth of 15 yielded the best results. The traditional gradient boosting DT booster produced the lowest MAE. For the CatBoost algorithm (Figure 8c,f), a lower MAE was associated with greater depth (30) and a reduced learning rate (0.1).

## 5.2. Performance Metrics Assessment.
Model performance was assessed using multiple metrics: MAE, MSE, $R^2$, EXV, RAP, MPD, and MGD. The results are presented in Figures 9 and 10. After thorough evaluation, XGBoost emerged as the top performer for HL prediction, closely followed by GBM, where the test errors are: MAE = 0.56, MSE = 0.65, $R^2$ = 0.98, EXV = 0.987, RAP = 0.020, MPD = 0.03, and MGD = 0.0009. For CL prediction, GBM slightly outperformed XGBoost, with test errors of MAE = 0.60, MSE = 1.13, $R^2$ = 0.992, EXV = 0.992, RAP = 0.022, MPD = 0.02, and MGD = 0.0008.

The DsNN model ranked third for CL forecasting and fourth for HL. Overall, XGBoost and GBM demonstrated superior performance among the tested models, although both exhibited sensitivity to the dataset during hyperparameter tuning.

## 5.3. Influence of Feature Selection on Predictive Accuracy.
The analysis of adapted F-test and mutual information statistics is presented in Figure 11. The data reveal that features X6 and X8 consistently show the lowest values across both
<figure><img src="image_7.png" /><figcaption>FIGURE 5: (a-c) Heat map showing the different hyperparameter tuning during 10-fold CV of LR1, LR2, and DT for HL. (d-f) Heat map showing the different hyperparameter tuning during 10-fold CV of LR1, LR2, and DT for CL.</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>FIGURE 6: (a-c) Heat map showing the different hyperparameter tuning during 10-fold CV of AdaBoost for HL. (d-f) Heat map showing the different hyperparameter tuning during 10-fold CV of AdaBoost for CL.</figcaption></figure>

statistical measures. Conversely, the set X5, X3, and X2 exhibit the highest F values, while X1, X2, X3, and X4 demonstrate the highest mutual information scores. This analysis highlights significant variability and shared information among features and variables, which may influence model

accuracy. Given these findings, implementing feature selection could be beneficial when developing predictive models to reduce input dimensionality. An exhaustive feature space search was conducted using XGBoost for HL and GBM for CL, as illustrated in Figures 12 and 13. This comprehensive
<figure><img src="image_9.png" /><figcaption>FIGURE 7: Heat map showing the different hyperparameter tuning during fivefold CV of (a, b) GBM and (c) of XGBoost for HL. Heat map showing the different hyperparameter tuning during fivefold CV of (d, e) GBM and (f) of XGBoost for CL.</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>FIGURE 8: Heat map showing the different hyperparameter tuning during fivefold CV of (a, b) LGBM and (c) of CatBoost for HL. Heat map showing the different hyperparameter tuning during fivefold CV of (d, e) LGBM and (f) of Catboost for CL.</figcaption></figure>

approach ensures that all possible feature combinations are evaluated to identify the most informative subsets for each predictive model. The results indicate that the optimal feature set for predicting HL is X1, X3, X6, X7, and X8, while X1, X6, X7, and X8 prove most effective for CL prediction,

which is consistent with physical intuition. Table 3 shows the performance improvements achieved by applying these optimized feature sets to XGBoost and GBM models for HL and CL predictions, respectively. This feature selection process demonstrates notable enhancements in model
<figure><img src="image_11.png" /><figcaption>FIGURE 9: Statistical validation metrics of the models for HL.</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>FIGURE 10: Statistical validation metrics of the models for CL.</figcaption></figure>

performance metrics, underscoring the importance of thoughtful feature selection in machine learning model development.

**5.4. Benchmarking Against Existing Models.** To analyze the performance of the applied models, results are compared with those of related studies available in the literature study-

ing the same dataset (see Tables 4 and 5). As far as the HL prediction, XGBoost achieved the highest R² value (99.78%), outperforming the second-ranked work by 0.05%. Admittedly, this difference is almost negligible, but our model is very simple in contrast to the model proposed in [16], and therefore faster to train and deploy. It also requires less computing power. This is particularly
<figure><img src="image_13.png" /><figcaption>FIGURE 11: Bar plot of the *F* values for the dataset features for (a) HL and (b) CL. Bar plot of the mutual index (MI) for the dataset features for (c) HL and (d) CL.</figcaption></figure>

<figure><img src="image_14.png" /><figcaption>FIGURE 12: Exhaustive feature selection HL.</figcaption></figure>

<figure><img src="image_15.png" /><figcaption>FIGURE 13: Exhaustive feature selection CL.</figcaption></figure>

important in real-time applications or when resources are limited. Considering MAE and MSE values, it is clear that XGBoost performs best of all the tested models. It is followed by WGPRLSB [16] with a performance drop of 18% in MAE and 47% in MSE. Ultimately, XGBoost is ranked first without affecting results interoperability, that is, it is easy to understand how predictions are generated and explain those forecasts clearly. Analyzing results obtained for CL, GBM is the most efficient model based on MAE, 8.3% better than the proposed approach in [16], 25% better than Tran et al. [52] proposal and 124% better than Xu et al. [51] solution. Measured against $R^2$, GBM obtains an excellent rating, beating the methods proposed in [51, 52] and [15], in that order, where score variations range from 1.24% to 7.05%. However, it is ranked second, ahead of [16] with a performance improvement of 0.12%. Looking at MSE, GBM ranks joint-top with [16] for a 9.3% performance difference, pursued by [51, 52] in third place, with scores fluctuating between 50% and 110% lower. It is essential to stress that this comparative study does not imply that previous studies were ineffective, as each was conducted in a specific setting (e.g., with different training and test samples). By reducing dimensions, model rankings change slightly. Our two new models, trained and tested in lower-dimensional spaces, significantly outperform all others in terms of both accuracy and computational efficiency. Another key finding concerns overall neural network performance. Despite
<table><caption>TABLE 3: Output metric after feature selection.</caption><thead><tr><th>Variable</th><th>Best subset</th><th>MAE</th><th>MSE</th><th>R<sup>2</sup></th><th>EXV</th><th>MAPE</th><th>MPD</th><th>MGD</th></tr></thead><tbody><tr><td>HL (Y1)</td><td>{X1, X3, X6, X7, X8}</td><td>0.278</td><td>0.145</td><td>0.998</td><td>0.998</td><td>0.0114</td><td>0.005</td><td>0.0002</td></tr><tr><td>CL (Y2)</td><td>{X1, X6, X7, X8}</td><td>0.49</td><td>0.48</td><td>0.99</td><td>0.994</td><td>0.0195</td><td>0.016</td><td>0.0006</td></tr></tbody></table>

<table><caption>TABLE 4: Comparison of the proposed model to the models existing in the literature for HL.</caption><thead><tr><th>Method</th><th>MAE</th><th>MSE</th><th>R<sup>2</sup></th><th>Reference</th><th>Rank</th></tr></thead><tbody><tr><td>XGBoost</td><td>0.33</td><td>0.21</td><td>0.9978</td><td>—</td><td>2</td></tr><tr><td>EFS-XGBoost</td><td>0.278</td><td>0.145</td><td>0.998</td><td>—</td><td>1</td></tr><tr><td>TLBO-MLP</td><td>1.8630</td><td>6.6013</td><td>0.9370</td><td>[48]</td><td>7</td></tr><tr><td>SSA</td><td>1.9178</td><td>7.5774</td><td>0.9248</td><td>[49]</td><td>8</td></tr><tr><td>I-ACO-WNN</td><td>0.8472</td><td>3.4779</td><td>0.9714</td><td>[15]</td><td>5</td></tr><tr><td>VSA</td><td>1.8556</td><td>6.7236</td><td>0.9524</td><td>[50]</td><td>6</td></tr><tr><td>BBO</td><td>2.3500</td><td>12.0409</td><td>0.9440</td><td>[51]</td><td>9</td></tr><tr><td>ENMIM</td><td>0.6100</td><td>0.4225</td><td>0.9800</td><td>[52]</td><td>4</td></tr><tr><td>WGPRLSB</td><td>0.3947</td><td>0.3190</td><td>0.9973</td><td>[16]</td><td>3</td></tr></tbody></table>

<table><caption>TABLE 5: Comparison of the proposed model to the models existing in the literature for CL.</caption><thead><tr><th>Method</th><th>MAE</th><th>MSE</th><th>R<sup>2</sup></th><th>Reference</th><th>Rank</th></tr></thead><tbody><tr><td>GBM</td><td>0.56</td><td>0.65</td><td>0.992</td><td>—</td><td>2</td></tr><tr><td>EFS-GBM</td><td>0.49</td><td>0.48</td><td>0.99</td><td>—</td><td>1</td></tr><tr><td>TLBO-MLP</td><td>2.1360</td><td>9.7481</td><td>0.8997</td><td>[48]</td><td>7</td></tr><tr><td>SSA</td><td>2.2652</td><td>10.1633</td><td>0.8884</td><td>[49]</td><td>9</td></tr><tr><td>I-ACO-WNN</td><td>2.1830</td><td>9.9042</td><td>0.8929</td><td>[15]</td><td>8</td></tr><tr><td>VSA</td><td>1.8124</td><td>6.7304</td><td>0.9227</td><td>[50]</td><td>6</td></tr><tr><td>BBO</td><td>1.2700</td><td>3.7249</td><td>0.9500</td><td>[51]</td><td>5</td></tr><tr><td>ENMIM</td><td>0.6142</td><td>0.9604</td><td>0.9800</td><td>[52]</td><td>4</td></tr><tr><td>WGPRLSB</td><td>0.61427</td><td>0.5985</td><td>0.9935</td><td>[16]</td><td>3</td></tr></tbody></table>

their complexity, these models consistently rank just below the top-performing model in both scenarios. This is due to their strong adaptability to training data, but it also increases their susceptibility to overfitting. Such algorithms may perform well on training data, but generalize poorly on new data. Apart from overfitting, small amounts of data can cause instability in deep learning models, meaning that they deliver different results on each training run, due to their sensitivity to limited training data. Larger datasets help to stabilize models and reduce overlearning. A simple model, meanwhile, is less likely to overfit and may perform better on unseen data.

### 5.5. Discussion.
The performance evaluation of the different models applied in this work for predicting HVAC loads in residential buildings indicates that the XGBoost and GBM models outperform other models in terms of accuracy and robustness. This is due to several key aspects. First, the boosting strategy, which weightily combines the prediction of several weak models trained sequentially. Second, the

study showed that XGBoost was able to accurately capture complex and nonlinear relationships between variables that better predict the HL, while GBM was able to accurately capture variables that better predict the CL.

This can be explained by several factors. The HL represents a stable dynamic influenced by less variable characteristics such as compactness, insulation quality, and thermal inertia of the building, which allowed XGBoost to effectively capture it thanks to its regularization strategy for advanced overfitting management. In contrast, the CL is strongly influenced by rapid peaks of sunshine. Based on gradient descent optimization and iterative learning, GBM progressively refines the modeling of the CL, while capturing the thermal fluctuations induced by the climate. Finally, the application of exhaustive feature search allowed maintaining or even improving the predictive performance of these two models, which suggests their robustness with respect to the introduction of irrelevant features in the dataset. This comprehensive approach demonstrates not only the technical capabilities of XGBoost and GBM, but also their suitability to the theoretical and practical aspects of building energy load forecasting research.

## 6. Conclusion

Energy use in buildings plays a crucial role in both economic assessments and academic research. Developing precise models for energy consumption is critical, as inaccurate predictions can have significant consequences. Underestimation may lead to system inadequacies that negatively affect social and economic welfare; overestimation can result in resource inefficiencies and excess capacity. The primary function of HVAC systems is to create a pleasant indoor environment by regulating temperature, ensuring good air quality, and providing adequate ventilation. This study proposes a practical method for initial energy load forecasting during the early stages of designing residential buildings. The approach is intended to be useful for architects and planners in the preliminary phases of their projects. Significant work has gone into modeling and forecasting building cooling and HCs using deep neural networks and supervised ensemble machine learning. Eight input criteria comprise the data used in this study: RC, orientation, SA, WA, RA, OH, and GAD. Out of the eight models, the XGBoost and the GBM were determined to be the most accurate in terms of correlation and error parameters for predicting HL and CL, respectively. XGBoost was shown to be the best performer for HL prediction with MAE = 0.56, MSE = 0.65, R² = 0.98, EXV = 0.987, RAP = 0.020, MPD = 0.03, and MGD = 0.0009. While for CL, GBM slightly outperformed XGBoost, with test errors of MAE = 0.60, MSE = 1.13, R² = 0.992, EXV = 0.992, RAP = 0.022, MPD = 0.02, and
MGD = 0.0008. Indeed when these models were paired with exhaustive feature selection, the results were superior in terms of error metrics and correlation. Exhaustive feature selection identified RC, WA, glazing area, orientation, and glazing distribution as key predictors for heat load using XGBoost and led to the following improvements: MAE decreased by 15.8% (from 0.33 to 0.278), indicating better average prediction accuracy, MSE improved by 31.0% (from 0.21 to 0.145), reflecting a substantial reduction in larger errors, and $R^2$ saw a marginal increase (0.02%), maintaining near-perfect explanatory power. However, the feature selection method enhanced GBM's CL predictive performance substantially and identified RC, orientation, glazing area, and GAD as key features. Particularly, the MAE decreased by 12.5% (from 0.56 to 0.49), the MSE showed a significant reduction of 26.2% (from 0.65 to 0.48), and $R^2$ remained consistently high (0.992–0.99) with minimal 0.002% difference. Moreover, results revealed that lightweight models are sufficient for this forecasting task, without the need for heavyweight architectures such as dense neural networks. However, there are still obstacles to be addressed, such as generalization problems and hyperparameter adjustment. Therefore, in upcoming work, an effort will be made to create a TL-based approach that can be applied to a different kind of dataset and integrated with a real-world scenario. Incorporating these models with architectural engineering software such as Autodesk Revit, or similar BIM tools could provide real-time feedback to clients on energy load estimation based on proposed building dimensions and features to determine the heating and cooling needs of each building configuration, ensuring adequate sizing of the HVAC system during the early-stage planning. Additionally, the proposed models can assist in compliance with energy efficiency standards for buildings such as ISO 50001, LEED (Leadership in Energy and Environmental Design), or BREEAM (Building Research Establishment Environmental Assessment Method) by providing accurate load estimation for certification purposes.

## Data Availability Statement

The data is available on reasonable request and with permission from the relevant institutions.

## Conflicts of Interest

The authors declare no conflicts of interest.

## Funding

No funding was received for this manuscript.

## References

[1] S. Mostafavi, H. Doddi, K. Kalyanam, and D. Schwartz, “Non-linear Moving Horizon Estimation and Model Predictive Control for Buildings With Unknown HVAC Dynamics,” *IFAC-PapersOnLine* 55, no. 41 (2022): 71–76, https://doi.org/10.1016/j.ifacol.2023.01.105.

[2] K. Irshad and S. Algarni, “Novel Optimized Hybrid Neuro-fuzzy Approach for Analysis of Cold Thermal Storage System-Assisted Air Conditioning System Performance,” *Journal of Building Engineering* 65 (2023): 105729, https://doi.org/10.1016/j.jobe.2022.105729.

[3] G. Serale, M. Fiorentini, A. Capozzoli, D. Bernardini, and A. Bemporad, “Model Predictive Control (MPC) for Enhancing Building and HVAC System Energy Efficiency: Problem Formulation, Applications and Opportunities,” *Energies* 11, no. 3 (2018): 631, https://doi.org/10.3390/en11030631.

[4] A. S. Abdelrazik, B. Shboul, M. Elwardany, R. N. Zohny, and A. Osaka, “The Recent Advancements in the Building Integrated Photovoltaic/Thermal (BIPV/T) Systems: An Updated Review,” *Renewable and Sustainable Energy Reviews* 170 (2022): 112988, https://doi.org/10.1016/j.rser.2022.112988.

[5] Y. Zhang, J. A. Wright, and V. I. Hanby, “Energy Aspects of HVAC System Configurations—Problem Definition and Test Cases,” *HVAC Research* 12, no. sup3 (2006): 871–888, https://doi.org/10.1080/10789669.2006.10391214.

[6] Y. Chen, M. Guo, Z. Chen, Z. Chen, and Y. Ji, “Physical Energy and Data-Driven Models in Building Energy Prediction: A Review,” *Energy Reports* 8 (2022): 2656–2671, https://doi.org/10.1016/j.egyr.2022.01.162.

[7] L. Wang, D. Xie, L. Zhou, and Z. Zhang, “Application of the Hybrid Neural Network Model for Energy Consumption Prediction of Office Buildings,” *Journal of Building Engineering* 72 (2023): 106503, https://doi.org/10.1016/j.jobe.2023.106503.

[8] D. Curto, V. Franzitta, A. Guercio, M. Mantegna, and D. Milone, “Energy Efficiency in Historic Architecture: The “Ex Institute of Zoology and Comparative Anatomy” in Palermo,” *Applied Sciences* 13, no. 15 (2023): 8882, https://doi.org/10.3390/app13158882.

[9] A. A. Al-Shargabi, A. Almhafdy, D. M. Ibrahim, M. Alghieth, and F. Chiclana, “Buildings’ Energy Consumption Prediction Models Based on Buildings’ Characteristics: Research Trends, Taxonomy, and Performance Measures,” *Journal of Building Engineering* 54 (2022): 104577, https://doi.org/10.1016/j.jobe.2022.104577.

[10] D. Kim, J. Lee, S. Do, P. J. Mago, K. H. Lee, and H. Cho, “Energy Modeling and Model Predictive Control for HVAC in Buildings: A Review of Current Research Trends,” *Energies* 15, no. 19 (2022): 7231, https://doi.org/10.3390/en15197231.

[11] N. S. Raman, K. Devaprasad, B. Chen, H. A. Ingley, and P. Barooah, “Model Predictive Control for Energy-Efficient HVAC Operation With Humidity and Latent Heat Considerations,” *Applied Energy* 279 (2020): 115765, https://doi.org/10.1016/j.apenergy.2020.115765.

[12] K. M. Al-Obaidi, M. Hossain, N. A. Alduais, H. S. Al-Duais, H. Omrany, and A. Ghaffarianhoseini, “A Review of Using IoT for Energy Efficient Buildings and Cities: A Built Environment Perspective,” *Energies* 15, no. 16 (2022): 5991, https://doi.org/10.3390/en15165991.

[13] M. Kampik, M. Fice, A. Pilśniak, K. Bodzek, and A. Piaskowy, “An Analysis of Energy Consumption in Small- and Medium-Sized Buildings,” *Energies* 16, no. 3 (2023): 1536, https://doi.org/10.3390/en16031536.

[14] W. Cai, X. Wen, C. Li, J. Shao, and J. Xu, “Predicting the Energy Consumption in Buildings Using the Optimized Support Vector Regression Model,” *Energy* 273 (2023): 127188, https://doi.org/10.1016/j.energy.2023.127188.

[15] Y. Huang and C. Li, “Accurate Heating, Ventilation and Air Conditioning System Load Prediction for Residential
Buildings Using Improved Ant Colony Optimization and Wavelet Neural Network," *Journal of Building Engineering* 35 (2021): 101972, https://doi.org/10.1016/j.jobe.2020.101972.

[16] N. Pachauri and C. W. Ahn, "Weighted Aggregated Ensemble Model for Energy Demand Management of Buildings," *Energy* 263 (2023): 125853, https://doi.org/10.1016/j.energy.2022.125853.

[17] D. Kim, Y. Lee, K. Chin, P. J. Mago, H. Cho, and J. Zhang, "Implementation of a Long Short-Term Memory Transfer Learning (LSTM-TL)-Based Data-Driven Model for Building Energy Demand Forecasting," *Sustainability* 15, no. 3 (2023): 2340, https://doi.org/10.3390/su15032340.

[18] N. Wang, "Multi-Objective Energy Saving Optimization of Residential Buildings Based on MABC-BP," *Energy Reports* 9 (2023): 922-929, https://doi.org/10.1016/j.egyr.2023.04.182.

[19] J. Bian, J. Wang, and Q. Yece, "A Novel Study on Power Consumption of an HVAC System Using CatBoost and AdaBoost Algorithms Combined With the Metaheuristic Algorithms," *Energy* 302 (2024): 131841, https://doi.org/10.1016/j.energy.2024.131841.

[20] R. Mohan and N. Pachauri, "An Ensemble Model for the Energy Consumption Prediction of Residential Buildings," *Energy* 314 (2025): 134255, https://doi.org/10.1016/j.energy.2024.134255.

[21] A. M. Zaki, M. E. Zayed, M. H. Bargal, et al., "Environmental and Energy Performance Analyses of HVAC Systems in Office Buildings Using Boosted Ensembled Regression Trees: Machine Learning Strategy for Energy Saving of Air Conditioning and Lighting Facilities," *Process Safety and Environmental Protection* 198 (2025): 107214, https://doi.org/10.1016/j.psep.2025.107214.

[22] M. A. Sabri, Y. Filali, H. El Khoukhi, and A. Aarab, "Skin Cancer Diagnosis Using an Improved Ensemble Machine Learning Model," in *Proceedings of the 2020 International Conference on Intelligent Systems and Computer Vision (ISCV) (IEEE, 2020)*, 1-5, https://doi.org/10.1109/ISCV49265.2020.9204324.

[23] H. Errousso, J. El Ouadi, S. Benhadou, H. Medromi, and N. Malhene, "Improving Delivery Conditions by Dynamically Managing the Urban Parking System: Parking Availability Prediction," in *Proceedings of the 2020 IEEE 13th International Colloquium of Logistics and Supply Chain Management (LOGISTIQUA) (IEEE, 2020)*, 1-6, https://doi.org/10.1109/LOGISTIQUA49782.2020.9353890.

[24] H. Errousso, N. Malhene, S. Benhadou, and H. Medromi, "Predicting Car Park Availability for a Better Delivery Bay Management," *Procedia Computer Science* 170 (2020): 203-210, https://doi.org/10.1016/j.procs.2020.03.026.

[25] M. Abdul Baseer, A. Almunif, I. Alsaduni, and N. Tazeen, "Electrical Power Generation Forecasting From Renewable Energy Systems Using Artificial Intelligence Techniques," *Energies* 16, no. 18 (2023): 6414, https://doi.org/10.3390/en16186414.

[26] G. Kunapuli, *Ensemble Methods for Machine Learning* (Simon and Schuster, 2023).

[27] R. J. Casson and L. D. Farmer, "Understanding and Checking the Assumptions of Linear Regression: A Primer for Medical Researchers," *Clinical & Experimental Ophthalmology* 42, no. 6 (2014): 590-596, https://doi.org/10.1111/ceo.12358.

[28] D. C. Feng, Z. T. Liu, X. D. Wang, et al., "Machine Learning-Based Compressive Strength Prediction for Concrete: An Adaptive Boosting Approach," *Construction and Building Materials* 230 (2020): 117000, https://doi.org/10.1016/j.conbuildmat.2019.117000.

[29] T. Chen, T. He, M. Benesty, et al., "XGBoost: Extreme Gradient Boosting," *R package Version* 0.4-2 1, no. 4 (2015): 1-4.

[30] J. Guo, S. Yun, Y. Meng, et al., "Prediction of Heating and Cooling Loads Based on Light Gradient Boosting Machine Algorithms," *Building and Environment* 236 (2023): 110252, https://doi.org/10.1016/j.buildenv.2023.110252.

[31] X. Ma, J. Sha, D. Wang, Y. Yu, Q. Yang, and X. Niu, "Study on a Prediction of P2P Network Loan Default Based on the Machine Learning LightGBM and XGBoost Algorithms According to Different High Dimensional Data Cleaning," *Electronic Commerce Research and Applications* 31 (2018): 24-39, https://doi.org/10.1016/j.elerap.2018.08.002.

[32] L. Prokhorenkova, G. Gusev, A. Vorobev, A. V. Dorogush, and A. Gulin, "CatBoost: Unbiased Boosting With Categorical Features," in *NIPS'18: Proceedings of the 32nd International Conference on Neural Information Processing Systems* (Curran Associates Inc., 2018), https://doi.org/10.5555/3327757.3327770.

[33] B. De Ville, "Decision Trees," *Wiley Interdisciplinary Reviews: Computational Statistics* 5, no. 6 (2013): 448-455, https://doi.org/10.1002/wics.1278.

[34] A. Natekin and A. Knoll, "Gradient Boosting Machines, a Tutorial," *Frontiers in Neurorobotics* 7 (2013): 21, https://doi.org/10.3389/fnbot.2013.00021.

[35] M. Alizamir, M. Wang, R. M. A. Ikram, et al., "An Interpretable XGBoost-SHAP Machine Learning Model for Reliable Prediction of Mechanical Properties in Waste Foundry Sand-Based Eco-Friendly Concrete," *Results in Engineering* 25 (2025): 104307, https://doi.org/10.1016/j.rineng.2025.104307.

[36] R. M. Cichy and D. Kaiser, "Deep Neural Networks as Scientific Models," *Trends in Cognitive Sciences* 23, no. 4 (2019): 305-317, https://doi.org/10.1016/j.tics.2019.01.009.

[37] R. Miikkulainen, J. Liang, E. Meyerson, et al., "Evolving Deep Neural Networks," in *Artificial Intelligence in the Age of Neural Networks and Brain Computing* (Academic Press, 2024), 269-287, https://doi.org/10.1016/B978-0-12-815480-9.00015-3.

[38] Z. B. Zabinsky, *Random Search Algorithms* (Department of Industrial and Systems Engineering, University of Washington, 2009).

[39] S. Nersisyan, V. Novosad, A. Galatenko, et al., "ExhauFS: Exhaustive Search-Based Feature Selection for Classification and Survival Regression," *PeerJ* 10 (2022): e13200, https://doi.org/10.7717/peerj.13200.

[40] A. Jović, K. Brkić, and N. Bogunović, "A Review of Feature Selection Methods With Applications," in *Proceedings of the 2015 38th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO) (IEEE, 2015)*, 1200-1205, https://doi.org/10.1109/MIPRO.2015.7160458.

[41] G. Chandrashekar and F. Sahin, "A Survey on Feature Selection Methods," *Computers & Electrical Engineering* 40, no. 1 (2014): 16-28, https://doi.org/10.1016/j.compeleceng.2013.11.024.

[42] A. Tsanas and A. Xifara, "Accurate Quantitative Estimation of Energy Performance of Residential Buildings Using Statistical Machine Learning Tools," *Energy and Buildings* 49 (2012): 560-567, https://doi.org/10.1016/j.enbuild.2012.03.003.

[43] H. Errousso, E. A. Abdellaoui Alaoui, S. Benhadou, and H. Medromi, "Exploring How Independent Variables
Influence Parking Occupancy Prediction: Toward a Model
Results Explanation With SHAP Values," *Progress in Artificial
Intelligence* 11, no. 4 (2022): 367–396, https://doi.org/10.1007/
s13748-022-00291-5.

[44] T. G. Wakjira, A. Abushanab, U. Ebead, and W. Alnahhal, “FAI: Fast, Accurate, and Intelligent Approach and Prediction Tool for Flexural Capacity of FRP-RC Beams Based on Super-Learner Machine Learning Model,” *Materials Today Communications* 33 (2022): 104461, https://doi.org/10.1016/j.mtcomm.2022.104461.

[45] T. G. Wakjira, M. Ibrahim, U. Ebead, and M. S. Alam, “Explainable Machine Learning Model and Reliability Analysis for Flexural Capacity Prediction of RC Beams Strengthened in Flexure With FRCM,” *Engineering Structures* 255 (2022): 113903, https://doi.org/10.1016/j.engstruct.2022.113903.

[46] N. Aghbalou, A. Charki, H. Errousso, and Y. Filali, “Ensemble Learning Method for Forecasting HVAC System Demand,” in *Proceedings of the International Conference on Advanced Materials for Sustainable Energy and Engineering* (Springer, 2024), 1–9, https://doi.org/10.1007/978-3-031-57022-3_1.

[47] E. M. Elkhattabi, M. Boutahir, K. Termentzidis, K. Nakamura, and A. Rahmani, eds., *Advanced Materials for Sustainable Energy and Engineering: Selected Proceedings of the 2023 International Conference on Advanced Materials for Sustainable Energy and Engineering (ICAMSEE)* (Springer Nature, 2024).

[48] G. Zhou, H. Moayedi, and L. K. Foong, “Teaching–Learning–Based Metaheuristic Scheme for Modifying Neural Computing in Appraising Energy Performance of Building,” *Engineering with Computers* 37, no. 4 (2021): 3037–3048, https://doi.org/10.1007/s00366-020-00981-5.

[49] Z. Guo, H. Moayedi, L. K. Foong, and M. Bahiraei, “Optimal Modification of Heating, Ventilation, and Air Conditioning System Performances in Residential Buildings Using the Integration of Metaheuristic Optimization and Neural Computing,” *Energy and Buildings* 214 (2020): 109866, https://doi.org/10.1016/j.enbuild.2020.109866.

[50] D. Wu, L. K. Foong, and Z. Lyu, “Two Neural-Metaheuristic Techniques Based on Vortex Search and Backtracking Search Algorithms for Predicting the Heating Load of Residential Buildings,” *Engineering with Computers* 38, no. 1 (2022): 647–660, https://doi.org/10.1007/s00366-020-01074-z.

[51] Y. Xu, F. Li, and A. Asgari, “Prediction and Optimization of Heating and Cooling Loads in a Residential Building Based on Multi-Layer Perceptron Neural Network and Different Optimization Algorithms,” *Energy* 240 (2022): 122692, https://doi.org/10.1016/j.energy.2021.122692.

[52] D. H. Tran, D. L. Luong, and J. S. Chou, “Nature-Inspired Metaheuristic Ensemble Model for Forecasting Energy Consumption in Residential Buildings,” *Energy* 191 (2020): 116552, https://doi.org/10.1016/j.energy.2019.116552.