In industrial machine learning pipelines, data often arrive in parts. Particularly in the case of deep neural networks, it may be too expensive to train the model from scratch each time, so one would rather use a previously learned model and the new data to improve performance. However, deep neural networks are prone to getting stuck in a suboptimal solution when trained on only new data as compared to the full dataset. Our work focuses on a continuous learning setup where the task is always the same and new parts of data arrive sequentially. We apply a Bayesian approach to update the posterior approximation with each new piece of data and find this method to outperform the traditional approach in our experiments. Recent work has shown promise in incremental learning; for example, a set of reinforcement learning problems have been successively solved by a single model with a help of weight consolidation (Kirkpatrick et al., 2016) or Bayesian inference (Nguyen et al., 2017) . In this work we focus on a specific incremental learning setting -we consider a single fixed task when independent data portions arrive sequentially. We formulate a Bayesian method for incremental learning and use recent advances in approximate Bayesian inference (Kingma & Welling, 2013; Kingma et al., 2015; Louizos & Welling, 2017) to obtain a scalable learning algorithm. We demonstrate the performance of our method on MNIST and CIFAR-10 is improved relative to a naive fine-tuning approach and can be applied to a conventional (non-Bayesian) pre-trained DNN. . In an incremental learning setting, this dataset is divided into T parts D = {D 1 , . . . , D T }, which arrive sequentially during training. The goal is to build an efficient algorithm that takes a model, trained on the first t−1 units of data D 1 , . . . , D t−1 , and retrain it on a new unit of data D t without access to D 1 , . . . , D t−1 and without forgetting dependencies. The most naive deep learning approach for incremental learning is to apply the Stochastic Gradient Descent (SGD) updates with the same loss function on the new data parts, to fine-tune the model. However, in that case, the model is likely to converge to a local optima on a new data unit without saving the information learned from the previous parts of the data. The Bayesian framework is a powerful tool for working with probabilistic models. It allows to estimate the posterior distribution p(w | D 1 , . . . , D t ) over the weights w of the model. We can use the Bayes rule to sequentially update the posterior distribution in the incremental learning setting: Unfortunately, in most cases the posterior distribution p(w | D 1 , . . . , D t ) is intractable, so we can use stochastic variational inference (Hoffman et al., 2012) to approximate it. In the next section we present a scalable method for incremental learning, and study different variational approximations of the posterior distribution. We apply variational inference to approximate p(w | D 1 , . . . , D t ) ≈ q(w | φ t ) ∈ Q with access only to D t and the previous approximation q(w | φ t−1 ). To train our model we follow Kingma & Welling (2013) and use the reparameterization trick to obtain an unbiased differentiable minibatch-based Monte Carlo estimator of the variational lower bound The prior distribution p t (w) is the posterior approximation q(w | φ t−1 ) from the previous step. Unfortunately, this approximation is not exact and as a result, the incremental procedure becomes biased. The quality of the incremental learning algorithm depends strongly on the posterior approximation q, with more expressive families having a lower approximation gap, but poorer stability. We investigate, how different approximations behave in a Bayesian incremental learning algorithm.
In contrast to Bayesian techniques REF , these methods do not allow to obtain a posterior distribution over parameters of the model, and in most cases, they require to store convolutional weights of pre-trained models and careful tuning of hyperparameters.
Bayesian Incremental Learning for Deep Neural Networks
ArXiv
ArXiv
Mathematics Computer Science