Comprehensive Sensitivity Analysis Framework for Transfer Learning Performance Assessment for Time Series Forecasting: Basic Concepts and Selected Case Studies

Authors:
raw_author_name: Witesyavwirwa Vianney Kambale
raw_affiliation_string: Faculty of Information and Communication Technology, Tshwane University of Technology, Private Bag x680, Pretoria 0001, South Africa; Institute for Smart Systems Technologies, Universitaet Klagenfurt, 9020 Klagenfurt, Austria
raw_author_name: Mohamed Salem
raw_affiliation_string: Institute for Smart Systems Technologies, Universitaet Klagenfurt, 9020 Klagenfurt, Austria
raw_author_name: Taha Benarbia
raw_affiliation_string: Institute of Maintenance and Industrial Security, University of Oran 2, Oran 31000, Algeria
raw_author_name: Fadi Al Al Machot
raw_affiliation_string: Faculty of Science and Technology, Norwegian University of Life Sciences (NMBU), 1430 Ås, Norway
raw_author_name: Kyandoghere Kyamakya
raw_affiliation_string: Faculté Polytechnique, Université de Kinshasa, P.O. Box 127, Kinshasa XI, Democratic Republic of the Congo; Institute for Smart Systems Technologies, Universitaet Klagenfurt, 9020 Klagenfurt, Austria

Abstract: Recently, transfer learning has gained popularity in the machine learning community. Transfer Learning (TL) has emerged as a promising paradigm that leverages knowledge learned from one or more related domains to improve prediction accuracy in a target domain with limited data. However, for time series forecasting (TSF) applications, transfer learning is relatively new. This paper addresses the need for empirical studies as identified in recent reviews advocating the need for practical guidelines for Transfer Learning approaches and method designs for time series forecasting. The main contribution of this paper is the suggestion of a comprehensive framework for Transfer Learning Sensitivity Analysis (SA) for time series forecasting. We achieve this by identifying various parameters seen from various angles of transfer learning applied to time series, aiming to uncover factors and insights that influence the performance of transfer learning in time series forecasting. Undoubtedly, symmetry appears to be a core aspect in the consideration of these factors and insights. A further contribution is the introduction of four TL performance metrics encompassed in our framework. These TL performance metrics provide insight into the extent of the transferability between the source and the target domains. Analyzing whether the benefits of transferred knowledge are equally or unequally accessible and applicable across different domains or tasks speaks to the requirement of symmetry or asymmetry in transfer learning. Moreover, these TL performance metrics inform on the possibility of the occurrence of negative transfers and also provide insight into the possible vulnerability of the network to catastrophic forgetting. Finally, we discuss a sensitivity analysis of an Ensemble TL technique use case (with Multilayer Perceptron models) as a proof of concept to validate the suggested framework. While the results from the experiments offer empirical insights into various parameters that impact the transfer learning gain, they also raise the question of network dimensioning requirements when designing, specifically, a neural network for transfer learning.