Loss meta-learning for forecasting

Meta-learning of loss functions for supervised learning has been used to date for classification tasks , or as a way to enable few-shot learning . In this paper , we show how a fairly simple loss meta-learning approach can substantially improve regression results . Specifically , we target forecasting of time series and explore case studies grounded on real-world data , and show that meta-learned losses can benefit the quality of the prediction both in cases that are apparently naive , and in practical scenarios where the performance metric is complex , time-correlated , non-differentiable , or not known a priori . 1 INTRODUCTION . Loss functions drive the training process of supervised machine learning models . In the vast majority of cases , loss functions are designed to be generic enough to work well with a wide range of application scenarios . In regression problems , including forecasting tasks , Mean Absolute Error ( MAE ) , Mean Square Error ( MSE ) , or Mean squared logarithmic error ( MSLE ) are common choices for expressing the loss . In this paper , we question the assumption that a fixed and generic loss function is in fact the best choice in regression –and more specifically forecasting– problems , and investigate how losses learned from experience benefit the model performance in this type of task . Therefore , our work contributes to recent efforts in meta-learning , and specifically to those aimed at so-called learning to teach ( Wu et al. , 2018 ) . In this context , several proposals have been set forth for automatically learning the parameters , components , or shape of loss functions for neural network training . However , as extensively discussed in Section 2 , prior studies have focused on parametrizable losses for classification , and little attention has been paid to the learning to teach paradigm in the context of regression tasks . Part of the reason comes from the fact that loss meta-learning has been generally considered an inefficient approach for single-task regression ( Sung et al. , 2017 ) , under the assumption that losses such as MAE or MSE can already optimally drive regressor training in that case . Our study challenges such an assumption and shows that , in practical cases , a simple yet aptly designed meta-learning model can in fact improve the training of regression models with respect to legacy and presumably optimal losses . As detailed in Section 3 , our proposed model , named MetaLoss , builds on a joint co-training of the main regressor network and of the loss-learning network , and takes advantage of controlled noise during training to implement the exploration of the correct loss function behavior for rarely observed samples . We apply MetaLoss to the specific problem of time series forecasting in Sections 4 and 5 , and demonstrate how meta-learning of loss functions can help forecasting tasks under both apparently naive and complex performance metrics . In the case of naive metrics such as MAE or MSE , we disclose how the automated tailoring of the loss to different magnitudes ( and not only errors ) of the prediction can provide substantial gains on the accuracy . In the case of complex metrics , we prove that MetaLoss can successfully learn differentiable approximations of time-correlated and non-differentiable performance measures observed in two real-world applications , driving the regressor network towards forecasts that improve solutions currently considered in practice . Overall , by unveiling for the first time the advantages of meta-learning of loss functions for forecasting tasks , our study paves the road for the adoption of this paradigm in a previously unexplored machine learning domain . 2 RELATED WORK . Meta-learning , also referred to as learning-to-learn , overcomes the limitations of fixed learning-based models , and allows automatically tuning different aspects of the learning algorithm to the target task ( Hospedales et al. , 2021 ) . Meta-learning has been successfully applied to , e.g. , distillation ( Wang et al. , 2020 ) , augmentation ( Cubuk et al. , 2019 ) or batching ( Fan et al. , 2018 ) of training data , initialization ( Finn et al. , 2017 ) or optimization ( Andrychowicz et al. , 2016 ) of the model parameters , tuning ( Micaelli & Storkey , 2020 ) of its hyper-parameters , and discovery ( Liu et al. , 2019 ) of the actual architecture , possibly as a composition of modules ( Alet et al. , 2019 ) . Our focus is on metalearning of loss functions , which aims at learning the loss to be used to train the actual model . The problem can be seen as an instance of a hierarchical optimization , where a meta-model is optimized under a constraint represented by the main model optimization ( Franceschi et al. , 2018 ) . We stress that this is semantically different from meta-learning optimization schedules in iterative and alternate optimization processes ( Xu et al. , 2019 ) . Specifically , we distinguish two main approaches to loss meta-learning , discussed next . The first approach consists in having a teacher network infer the most suitable configuration of a predefined , parametrizable loss function . In this direction , several studies have investigated the use of decision networks to select among a set of predefined ( family of ) loss functions ( Liu & Lai , 2020 ; Denevi et al. , 2018 ) , while others have focused on multi-part loss functions , where the goal is setting ( Huang et al. , 2019 ; Zhao et al. , 2019 ) and possibly dynamically updating ( Heydari et al. , 2019 ) the function weights based on ( live ) performance metrics . In a similar way , models have been proposed to compose a loss function from primitive mathematical operators ( Li et al. , 2021 ) , or to express the performance metric as a function of a ( reduced ) set of simple surrogates ( Grabocka et al. , 2019 ; Jiang et al. , 2020 ) . Also related to the same concept are strategies such as training a network to correct the optimization trajectory produced by a fixed loss ( Huang et al. , 2021 ) , or introducing general loss functions that contain hyper-parameters to be learned during training along with the neural network parameters ( Barron , 2019 ) . In all these cases , the loss — independently of whether it is expressed as a tunable function , set of primitives , or surrogates — must be designed or selected manually , which is very challenging or even not possible when the performance metric of interest is especially complex , non-differentiable or even not known a priori . With respect to these works , we seek instead a solution that can learn a clean-slate loss . The second approach to loss meta-learning is more appropriate for clean-slate losses , and thus the one we also adopt in our work . The basic concept is representing the loss function itself via a ( typically lightweight ) neural network , which is fed with relevant input ( e.g. , main model output , labels , or features ) and produces a fit loss to be used to train the main model . In an early work , it was proposed the use of a teacher network to dynamically train parameters of a loss function that adapts to the learning stage of the main model ( Wu et al. , 2018 ) ; yet , this approach still relies on a generic known loss function to be parametrized by the teacher , hence suffers from the same limitations of the studies listed above . Closer to our methodology , the seminal idea of a trainable task-parametrized loss generator was introduced for reinforcement and supervised learning by the meta-critic model , where an action-value function neural network learns to criticise the actions in a specified task ( Sung et al. , 2017 ; Zhou et al. , 2020 ) . However , the meta-critic model is only applied to supervised learning problems as a tool for pre-training that allows for the few-shot learning of new tasks ( e.g. , by generalizing to unseen value ranges in the same domain ) . Indeed , the authors make it explicit that , in the case of a single task , modeling the loss via a dedicated neural network creates an indirection that makes learning more inefficient ( Sung et al. , 2017 ) . We show instead that loss meta-learning can in fact improve over presumably optimal losses also in simple single forecasting tasks . As such , our model is in fact complementary to the meta-critic one , as MetaLoss could be integrated with the meta-critic helper block to automatically model more tasks . It is worth mentioning that in the vast majority of previous works , and in other studies ( Shu et al. , 2020 ; Jiang et al. , 2020 ; Gao et al. , 2021 ; Jiang et al. , 2020 ; Marchetti et al. , 2021 ) that also include recent proposals to employ genetic programming tools to learn loss functions ( Gonzalez & Miikkulainen , 2020 ; 2021 ) , the aim of the loss meta-learning strategy is on discrete-space classification tasks or ranking problems . Little attention has been instead paid until now to loss meta-learning for regression . By investigating meta-learning solutions that target loss functions for forecasting tasks , our work shed new light on the advantages that this emerging paradigm can bring to a class of machine learning problems where loss meta-learning has been overlooked to date . 3 A MODEL FOR REGRESSION LOSS META-LEARNING . The proposed approach , named MetaLoss , consists of a loss-function-agnostic regressor that performs twofold learning . First , it must learn to forecast the value of action that minimizes a certain loss function . Second , it must also learn which is the said loss function according to the measures obtained from the environment . This concept has obvious applications in many practical regression problems where we can measure the performance resulting from the actions taken by the system , but the expression of the objective can not be characterized manually because of its complexity , or it is directly not known a priori ; in these scenarios , we can not apply a particular or well-defined loss function during the training phase . However , we will show that the approach can also benefit simpler cases , where the goal is a pure prediction of a time series , and where MAE , MSE , or variants of the same are generally considered to be very effective losses . In the following , we first provide a formal definition of the problem , and then present the high-level concept of MetaLoss as well as the detailed design of its architecture and operation . 3.1 PROBLEM FORMULATION . Let us denote the space of system state variables as S. We also denote the input space of the predictor as X and the output space as Y , such that the predictor can be modeled as fWp : X → Y , and the decision for time t + 1 taken at time t by the predictor can be written as1 ŷt+1 = fWp ( xt ) , where xt ∈ X includes the past observations of the system state , and Wp represents the parameters of the predictor . Let us denote the performance cost of the predictor ’ s decision taken at time t , measured at time t+ 1 , asMt+1 = fM ( yt+1 , ŷt+1 , vt+1 ) , where vt+1 denotes the current observations ( at the time of the measurement t+ 1 ) of the system variables that may impactMt+1 , yt+1 denotes the actual value that ŷt+1 attempts to predict , and fM ( · ) represents the a priori unknown expression of the objective . Note that , even if this expression is not known , the performance is assumed to be measurable , and samples of fM ( · ) can be obtained by observing the outcome of the predictor ’ s output on the system . Since the relation fM ( · ) is uncharted at first , the model also has to learn it . For that , we define a second optimization function to describe the loss-learning task . Such loss-learning task takes as inputs ( i ) the predictor ’ s decision ŷt+1 and ( ii ) the current observations vt+1 , and it casts a performance cost estimate M̃t+1 = fW ` ( yt+1 , ŷt+1 , vt+1 ) , where W ` represents the parameters of the loss-learning process . Consequently , MetaLoss is composed of two optimization problems . First , the loss-learning task aims at correctly characterizing fM ( yt+1 , ŷt+1 , vt+1 ) through the estimated fW ` ( yt+1 , ŷt+1 , vt+1 ) . This is done by computing a legacy loss function ( e.g. , MAE or MSE ) . For example , if we consider the MSE L2 ( a , b ) = ||a− b||2 , it follows that the loss-learning optimizer objective is min W ` L2 ( fW ` ( yt+1 , ŷt+1 , vt+1 ) , fM ( yt+1 , ŷt+1 , vt+1 ) ) . ( 1 ) In turn , the predictor ’ s objective is to minimize the performance cost of its decisions based on the predicted performance fW ` ( yt+1 , ŷt+1 , vt+1 ) , i.e. , to solve the following optimization : min Wp fW ` ( yt+1 , fWp ( xt ) , vt+1 ) . ( 2 )

Summary: This paper proposes a loss learning framework for the timeseries forecasting regression problem. They introduce two learning blocks, one for producing the next step prediction and the other for learning the loss function. The experimental results show that the proposed method slightly increase the performance compared to MAE.