Hybrid and ensemble-based time series transfer learning for building energy consumption prediction
Creators: name: Banda, Paul
Publisher: name: RMIT University
Year: 2024
Subjects: classificationCode: 330299
subject: Building not elsewhere classified
classificationCode: 400599
subject: Civil engineering not elsewhere classified

description: Energy demand is ever-growing as the global population grows and technology advances. Accurate prediction of the energy demand of large buildings is essential for property managers to operate these facilities more efficiently and greener. The current data-driven methods are tailored for specific buildings and rely heavily on the availability of quality and adequate training data. However, newly constructed buildings or buildings with newly installed energy monitoring systems do not have sufficient data for training good-performing models. The current prediction methods perform poorly under limited labelled data availability, thus undermining their practical application in the real world. Therefore, this research proposes a framework to improve building energy consumption prediction in limited data scenarios.
 
Cross-building prediction (transfer learning) is a relatively new and novel solution proposed to solve the limited data availability problem in building energy prediction studies. It involves using a pre-trained model to expedite the training of another model learning a similar task. The pretrained model is called the source model, and the model to be trained is called the target model. The source model is trained on a data-rich building to facilitate the prediction of a target building with limited data. However, the full potentials of transfer learning in building energy predictions are yet to be comprehensively and systematically examined. In response to this research gap, a cross-building transfer learning strategy for short-term building prediction at different prediction horizons is proposed to solve the challenge of limited data. A hybrid deep learning model is trained on a data-rich building, and the model’s weights are transferred to help predict a building with limited training data. The proposed model-based transfer learning methodology is an encoder-decoder hybrid deep learning model. It comprises a 2D convolutional-long short-term memory module and a bidirectional recurrent neural network (RNN) module to learn robust and efficient embeddings of different high-dimensional multivariate building energy use profiles. The proposed model also utilises a fully connected layer before the final prediction layer. Experiments have investigated transfer learning capabilities involving different building energy use cases using different implementation strategies. The experimental results consolidate the understanding of the proposed model's operations and develop guidelines for transfer learning model selection and design by practitioners.

The proposed model is tested under a strict four-tier model validation system to be deemed reliable. In this four-tier validation system, the proposed model is expected to outperform (i) the other competitive transfer learning-based models, (ii) new encoder-decoder models built from scratch, (iii) pretrained models developed at source building, and (iv) state-of-the-art machine learning models. Outperforming other transfer learning models means the proposed model is a competitive transfer learning model and can be named among known building energy and time series transfer learning models (Weber et al. 2021), (Pinto et al., 2022). Dominating performance against the new models is crucial as it justifies transfer learning. Outperforming the pretrained models proves that the target dataset used to fine-tune the proposed model was relevant. Superior performance over the machine learning models highlights the competitiveness of the proposed hybrid model against the state-of-the-art models. Three independent datasets outside building energy studies were used as testbeds to evaluate transfer learning capabilities. The proposed transfer learning model demonstrated improved prediction accuracy across all tested datasets under limited data. A comparison of transfer learning models on similar datasets and learning scenarios is still lacking in transfer learning studies (Weber et al., 2021), and the contributions from this study directly address this literature gap.

This research study introduced a novel transfer learning application involving one building with different data distributions. Most recorded transfer learning studies for building energy prediction involve transferring knowledge between buildings, commonly called cross-building transfer learning. However, in this research work, we demonstrate a new application of transfer learning involving one building, a phenomenon we intuitively termed intra-building transfer learning. The latter addresses the challenge of limited data that arises when a sudden shift in a building’s energy consumption profile is experienced, for example, when a new energy alternative is introduced that alters the current consumption profiles. The experiments showed that transfer learning provided encouraging results and is a potential solution when faced with sparse data owing to a drastic shift in the building energy consumption profile.

Ensemble models have produced encouraging results in individual building energy consumption prediction, with numerous studies reporting improved accuracy. A few studies have adopted homogenous ensemble transfer learning techniques in transfer learning and reported positive results. Our study proposes a novel generalised heterogeneous transfer learning ensemble framework to improve target building accuracy for limited data settings.
The ensemble model leverages the advantages and diversity of the base models by combining their weighted predictions before making a prediction. The ensemble framework was tested using numerous transfer learning schemes in various sectors. The results from extensive experimentation demonstrate the effectiveness of the proposed ensemble framework across the tested sectors at target data prediction.

A comparison of transfer learning schemes was introduced for the first time in this study. Different transfer learning schemes' results are analysed to verify the similarity between the source and target buildings. The compared transfer learning schemes have different source buildings but the same target building for a fair comparison. The study found that transfer learning schemes with similar buildings usually have lower error scores than those with dissimilar buildings. For example, transfer learning scores between leisure centres had lower error margins than transferring between office and leisure centres. The results from this section of experiments were compared to the more established dynamic time warping (DTW) method, which measures similarities between time series. The results from comparing transfer learning schemes matched most DTW scores from different experiments, which further validates our transfer learning results.

This study is one of the early contributors to this relatively novel transfer subject in building energy studies. The authors are confident the research outputs will go a long way in addressing fundamental questions that will advance transfer learning in the time series domain.