Loss meta-learning for forecasting

Meta-learning of loss functions for supervised learning has been used to date for classification tasks , or as a way to enable few-shot learning . In this paper , we show how a fairly simple loss meta-learning approach can substantially improve regression results . Specifically , we target forecasting of time series and explore case studies grounded on real-world data , and show that meta-learned losses can benefit the quality of the prediction both in cases that are apparently naive , and in practical scenarios where the performance metric is complex , time-correlated , non-differentiable , or not known a priori . 1 INTRODUCTION . Loss functions drive the training process of supervised machine learning models . In the vast majority of cases , loss functions are designed to be generic enough to work well with a wide range of application scenarios . In regression problems , including forecasting tasks , Mean Absolute Error ( MAE ) , Mean Square Error ( MSE ) , or Mean squared logarithmic error ( MSLE ) are common choices for expressing the loss . In this paper , we question the assumption that a fixed and generic loss function is in fact the best choice in regression –and more specifically forecasting– problems , and investigate how losses learned from experience benefit the model performance in this type of task . Therefore , our work contributes to recent efforts in meta-learning , and specifically to those aimed at so-called learning to teach ( Wu et al. , 2018 ) . In this context , several proposals have been set forth for automatically learning the parameters , components , or shape of loss functions for neural network training . However , as extensively discussed in Section 2 , prior studies have focused on parametrizable losses for classification , and little attention has been paid to the learning to teach paradigm in the context of regression tasks . Part of the reason comes from the fact that loss meta-learning has been generally considered an inefficient approach for single-task regression ( Sung et al. , 2017 ) , under the assumption that losses such as MAE or MSE can already optimally drive regressor training in that case . Our study challenges such an assumption and shows that , in practical cases , a simple yet aptly designed meta-learning model can in fact improve the training of regression models with respect to legacy and presumably optimal losses . As detailed in Section 3 , our proposed model , named MetaLoss , builds on a joint co-training of the main regressor network and of the loss-learning network , and takes advantage of controlled noise during training to implement the exploration of the correct loss function behavior for rarely observed samples . We apply MetaLoss to the specific problem of time series forecasting in Sections 4 and 5 , and demonstrate how meta-learning of loss functions can help forecasting tasks under both apparently naive and complex performance metrics . In the case of naive metrics such as MAE or MSE , we disclose how the automated tailoring of the loss to different magnitudes ( and not only errors ) of the prediction can provide substantial gains on the accuracy . In the case of complex metrics , we prove that MetaLoss can successfully learn differentiable approximations of time-correlated and non-differentiable performance measures observed in two real-world applications , driving the regressor network towards forecasts that improve solutions currently considered in practice . Overall , by unveiling for the first time the advantages of meta-learning of loss functions for forecasting tasks , our study paves the road for the adoption of this paradigm in a previously unexplored machine learning domain . 2 RELATED WORK . Meta-learning , also referred to as learning-to-learn , overcomes the limitations of fixed learning-based models , and allows automatically tuning different aspects of the learning algorithm to the target task ( Hospedales et al. , 2021 ) . Meta-learning has been successfully applied to , e.g. , distillation ( Wang et al. , 2020 ) , augmentation ( Cubuk et al. , 2019 ) or batching ( Fan et al. , 2018 ) of training data , initialization ( Finn et al. , 2017 ) or optimization ( Andrychowicz et al. , 2016 ) of the model parameters , tuning ( Micaelli & Storkey , 2020 ) of its hyper-parameters , and discovery ( Liu et al. , 2019 ) of the actual architecture , possibly as a composition of modules ( Alet et al. , 2019 ) . Our focus is on metalearning of loss functions , which aims at learning the loss to be used to train the actual model . The problem can be seen as an instance of a hierarchical optimization , where a meta-model is optimized under a constraint represented by the main model optimization ( Franceschi et al. , 2018 ) . We stress that this is semantically different from meta-learning optimization schedules in iterative and alternate optimization processes ( Xu et al. , 2019 ) . Specifically , we distinguish two main approaches to loss meta-learning , discussed next . The first approach consists in having a teacher network infer the most suitable configuration of a predefined , parametrizable loss function . In this direction , several studies have investigated the use of decision networks to select among a set of predefined ( family of ) loss functions ( Liu & Lai , 2020 ; Denevi et al. , 2018 ) , while others have focused on multi-part loss functions , where the goal is setting ( Huang et al. , 2019 ; Zhao et al. , 2019 ) and possibly dynamically updating ( Heydari et al. , 2019 ) the function weights based on ( live ) performance metrics . In a similar way , models have been proposed to compose a loss function from primitive mathematical operators ( Li et al. , 2021 ) , or to express the performance metric as a function of a ( reduced ) set of simple surrogates ( Grabocka et al. , 2019 ; Jiang et al. , 2020 ) . Also related to the same concept are strategies such as training a network to correct the optimization trajectory produced by a fixed loss ( Huang et al. , 2021 ) , or introducing general loss functions that contain hyper-parameters to be learned during training along with the neural network parameters ( Barron , 2019 ) . In all these cases , the loss — independently of whether it is expressed as a tunable function , set of primitives , or surrogates — must be designed or selected manually , which is very challenging or even not possible when the performance metric of interest is especially complex , non-differentiable or even not known a priori . With respect to these works , we seek instead a solution that can learn a clean-slate loss . The second approach to loss meta-learning is more appropriate for clean-slate losses , and thus the one we also adopt in our work . The basic concept is representing the loss function itself via a ( typically lightweight ) neural network , which is fed with relevant input ( e.g. , main model output , labels , or features ) and produces a fit loss to be used to train the main model . In an early work , it was proposed the use of a teacher network to dynamically train parameters of a loss function that adapts to the learning stage of the main model ( Wu et al. , 2018 ) ; yet , this approach still relies on a generic known loss function to be parametrized by the teacher , hence suffers from the same limitations of the studies listed above . Closer to our methodology , the seminal idea of a trainable task-parametrized loss generator was introduced for reinforcement and supervised learning by the meta-critic model , where an action-value function neural network learns to criticise the actions in a specified task ( Sung et al. , 2017 ; Zhou et al. , 2020 ) . However , the meta-critic model is only applied to supervised learning problems as a tool for pre-training that allows for the few-shot learning of new tasks ( e.g. , by generalizing to unseen value ranges in the same domain ) . Indeed , the authors make it explicit that , in the case of a single task , modeling the loss via a dedicated neural network creates an indirection that makes learning more inefficient ( Sung et al. , 2017 ) . We show instead that loss meta-learning can in fact improve over presumably optimal losses also in simple single forecasting tasks . As such , our model is in fact complementary to the meta-critic one , as MetaLoss could be integrated with the meta-critic helper block to automatically model more tasks . It is worth mentioning that in the vast majority of previous works , and in other studies ( Shu et al. , 2020 ; Jiang et al. , 2020 ; Gao et al. , 2021 ; Jiang et al. , 2020 ; Marchetti et al. , 2021 ) that also include recent proposals to employ genetic programming tools to learn loss functions ( Gonzalez & Miikkulainen , 2020 ; 2021 ) , the aim of the loss meta-learning strategy is on discrete-space classification tasks or ranking problems . Little attention has been instead paid until now to loss meta-learning for regression . By investigating meta-learning solutions that target loss functions for forecasting tasks , our work shed new light on the advantages that this emerging paradigm can bring to a class of machine learning problems where loss meta-learning has been overlooked to date . 3 A MODEL FOR REGRESSION LOSS META-LEARNING . The proposed approach , named MetaLoss , consists of a loss-function-agnostic regressor that performs twofold learning . First , it must learn to forecast the value of action that minimizes a certain loss function . Second , it must also learn which is the said loss function according to the measures obtained from the environment . This concept has obvious applications in many practical regression problems where we can measure the performance resulting from the actions taken by the system , but the expression of the objective can not be characterized manually because of its complexity , or it is directly not known a priori ; in these scenarios , we can not apply a particular or well-defined loss function during the training phase . However , we will show that the approach can also benefit simpler cases , where the goal is a pure prediction of a time series , and where MAE , MSE , or variants of the same are generally considered to be very effective losses . In the following , we first provide a formal definition of the problem , and then present the high-level concept of MetaLoss as well as the detailed design of its architecture and operation . 3.1 PROBLEM FORMULATION . Let us denote the space of system state variables as S. We also denote the input space of the predictor as X and the output space as Y , such that the predictor can be modeled as fWp : X → Y , and the decision for time t + 1 taken at time t by the predictor can be written as1 ŷt+1 = fWp ( xt ) , where xt ∈ X includes the past observations of the system state , and Wp represents the parameters of the predictor . Let us denote the performance cost of the predictor ’ s decision taken at time t , measured at time t+ 1 , asMt+1 = fM ( yt+1 , ŷt+1 , vt+1 ) , where vt+1 denotes the current observations ( at the time of the measurement t+ 1 ) of the system variables that may impactMt+1 , yt+1 denotes the actual value that ŷt+1 attempts to predict , and fM ( · ) represents the a priori unknown expression of the objective . Note that , even if this expression is not known , the performance is assumed to be measurable , and samples of fM ( · ) can be obtained by observing the outcome of the predictor ’ s output on the system . Since the relation fM ( · ) is uncharted at first , the model also has to learn it . For that , we define a second optimization function to describe the loss-learning task . Such loss-learning task takes as inputs ( i ) the predictor ’ s decision ŷt+1 and ( ii ) the current observations vt+1 , and it casts a performance cost estimate M̃t+1 = fW ` ( yt+1 , ŷt+1 , vt+1 ) , where W ` represents the parameters of the loss-learning process . Consequently , MetaLoss is composed of two optimization problems . First , the loss-learning task aims at correctly characterizing fM ( yt+1 , ŷt+1 , vt+1 ) through the estimated fW ` ( yt+1 , ŷt+1 , vt+1 ) . This is done by computing a legacy loss function ( e.g. , MAE or MSE ) . For example , if we consider the MSE L2 ( a , b ) = ||a− b||2 , it follows that the loss-learning optimizer objective is min W ` L2 ( fW ` ( yt+1 , ŷt+1 , vt+1 ) , fM ( yt+1 , ŷt+1 , vt+1 ) ) . ( 1 ) In turn , the predictor ’ s objective is to minimize the performance cost of its decisions based on the predicted performance fW ` ( yt+1 , ŷt+1 , vt+1 ) , i.e. , to solve the following optimization : min Wp fW ` ( yt+1 , fWp ( xt ) , vt+1 ) . ( 2 )

Summary: This work proposes meta-learning of loss functions for regression tasks. This approach has powerful applications for cases when the performance metrics are non-differentiable. Such cases often arise in tasks where the cost of misprediction is asymmetric and ripe with conditional clauses. The authors have proposed a simple yet powerful two-step process, wherein two neural networks are involved such that (a) a loss-learning NN learns an approximate differentiable loss function that mimics the non-differentiable performance metric, and (b) predictor NN that aims to perform well on the loss function defined by this loss-learning NN. The authors specify two crucial approaches to make this proposal work - (a) loss exploration, and (b) co-training of two neural networks. Their proposal is well supported by the experiments on time-series prediction tasks.