Backward Compatibility During Data Updates by Weight Interpolation (2024)

Abstract: Backward compatibility of model predictions is a desired property when updating a machine learning driven application. It allows to seamlessly improve the underlying model without introducing regression bugs. In classification tasks these bugs occur in the form of negative flips. This means an instance that was correctly classified by the old model is now classified incorrectly by the updated model. This has direct negative impact on the user experience of such systems e.g. a frequently used voice assistant query is suddenly misclassified.A common reason to update the model is when new training data becomes available and needs to be incorporated. Simply retraining the model with the updated data introduces the unwanted negative flips. We study the problem of regression during data updates and propose Backward Compatible Weight Interpolation (BCWI). This method interpolates between the weights of the old and new model and we show in extensive experiments that it reduces negative flips without sacrificing the improved accuracy of the new model. BCWI is straight forward to implement and does not increase inference cost. We also explore the use of importance weighting during interpolation and averaging the weights of multiple new models in order to further reduce negative flips.

## introduction in conventional software development it is an established routine to identify and fix regression bugs before deploying a new version. regression bugs describe defects in already existing features and are particularly sensible for end users because accustomed workflows are affected. in machine learning driven applications however the main focus usually lies on improving the underlying model and regression is rarely measured, let alone actively old data finetune old data new data finetune ## data update scenarios the data that is available to train a given classification model changes over time. this can be due to several reasons. more labeled data for the existing classes is obtained by annotating instances from the initial source or from observed queries. data for new classes is added to support additional downstream features or classes are split up to allow for more fine-grained classification. the retraining of an existing model on the evolved data basis is called data update. in this work, we focus on two isolated data update scenarios that cover two common use cases, namely adding i.i.d. data and adding new classes. we simulate the two scenarios in order to study the prevalence and mitigation of regression during data updates. add_data scenario in the add_data (ad) scenario, the amount of available data is increased by adding new instances for the current set of classes. this is the most basic type of data update and aims at improving the classification performance of the derived model. the additional data is usually obtained by annotating more instances from the initial data source or from the observed model queries. figure 2: results for the ad and ac scenario evaluated on our test sets of massive @xcite , banking77 @xcite and ag news @xcite . the gray horizontal bar is spanned by the 95% confidence interval of the target model and indicates the level of accuracy a model should reach. baselines are prior weight decay @xcite , mixout @xcite , ewc @xcite , distillation @xcite , bitfit @xcite and ia 3 @xcite . identical markers belong to the same method evaluated with different trade-off parameters. markers to the right of the target model are cut off. for bcwi this is α in 0.1 steps where 0.0 is equivalent to the new model and 1.0 is equivalent to the old model. the trade-off parameters for the baselines are listed in appendix a. the ideal case for a new model is to have zero negative flips while maintaining the target accuracy. bcwi consistently produces a model that is closer to the ideal case than any of the baseline methods and is more stable across different trade-off parameters. while in the latter case the distribution can shift over time, we assume i.i.d. data for this scenario. add_classes scenario in the add_classes (ac) scenario, we study data updates that consists of adding new classes and corresponding instances to the existing data. this is necessary when the text classification based system supports new features. for example, a virtual assistant is extended with a food delivery feature, a news classification model covers emerging topics or medical reports are classified according to new diseases codes. ## experiments we evaluate our proposed bcwi method on the above described data update scenarios, each constructed for three different datasets. 1 we choose roberta @xcite as a pretrained model because it is widely used and a representative encoder-only transformer model. the old model and target model are trained by finetuning roberta base on the old and updated data respectively (see equation 2 and 3 ). the new model is trained by finetuning the old model on the updated data (see equation 5 ). the empirical fisher information matrix used by fisherbcwi and ewc is calculated on the training and development instances. the classification performance baselines we compare bcwi to distillation training where the old model is used as the teacher @xcite . besides distillation, the method additionally applies higher weight to negative flip instances in the training set. this is a strong baseline and produces state-of-the-art results for reducing regression in architecture updates. we also compare our proposed approach to methods that are used to avoid catastrophic forgetting in continual learning. feed-forward networks. the proper model weights are frozen. all baselines are trained by finetuning the old model according to equation 5. ## analysis method properties in this section we discuss the training and inference resources required by bcwi and the utilized baselines listed in the calculations necessary to evaluate the regularization terms amount to 1.1 -1.9× longer training time. ewc additionally keeps the fisher information matrix in memory, which is pre-calculated before training. the pre-calculation takes the time of roughly one epoch of training. this is also necessary for the fisherbcwi method. in order to tune the regularization strength of ewc, prior wd and mixout, the model needs to be retrained entirely. on the other hand, the α-value of bcwi is tuned after the training is completed, by interpolating the converged model weights. this property of bcwi is a big advantage because it saves training resources and allows to quickly adjust to e.g. user complaints about too many regression errors in an updated model. output ensemble of old and new model the weight interpolation between old and new model can be seen as an ensemble in weight space. but in contrast to output ensembles, it does not increase inference cost. in an output ensemble, the input needs to be passed through both models and the final prediction is a combination of the two output probability distributions. this renders output en- ## conclusion we studied the problem of regression during data updates in text classification. retraining a model with a larger amount of training data increases accuracy but also introduces negative flips. we propose bcwi which describes the interpolation between the weights of the old model and the weight of the new model. we empirically show on three datasets and two update scenarios that bcwi models significantly reduce negative flips while not sacrificing accuracy. we compare bcwi to strong continual learning methods and achieve similar or better results, while not increasing training or inference cost. another big advantage of bcwi is that the trade-off parameter α can be tuned without retrain-ing the model. this saves additional training cost and only requires to store the weights of the old and new model. we extend bcwi by using the fisher information matrix as importance factor in weight interpolation and show that it further reduces negative flips. using multiple new models as in proposed soupbcwi also reduces regression without increasing the inference cost. in principle bcwi is architecture and task agnostic with the possibility to explore effectiveness in applications such as image classification or natural language generation left for future work.