Tackling Online One-Class Incremental Learning by Removing Negative Contrasts
Abstract
Recent work studies the supervised online continual learning setting where a learner receives a stream of data whose class distribution changes over time.
Distinct from other continual learning settings the learner is presented new samples only once and must distinguish between all seen classes.
A number of successful methods in this setting focus on storing and replaying a subset of samples alongside incoming data in a computationally efficient manner.
One recent proposal ER-AML achieved strong performance in this setting by applying an asymmetric loss based on contrastive learning to the incoming data and replayed data.
However, a key ingredient of the proposed method is avoiding contrasts between incoming data and stored data, which makes it impractical for the setting where only one new class is introduced in each phase of the stream.
In this work we adapt a recently proposed approach ( BYOL ) from self-supervised learning to the supervised learning setting, unlocking the constraint on contrasts.
We then show that supplementing this with additional regularization on class prototypes yields a new method that achieves strong performance in the one-class incremental learning setting and is competitive with the top performing methods in the multi-class incremental setting.
Introduction
Continual learning is a paradigm that aims to allow deep learning algorithms which will have the ability to learn online from a non-stationary and never-ending stream of data.
Such systems must become capable of acquiring new knowledge, while avoiding catastrophic forgetting of previously seen data, a problem commonly known and suffered by gradient-based neural networks [11].
A number of common continual learning scenarios exist in the classification setting, each with their own set of related but also different challenges.
These are often characterized along a number of axes.
The first is which information is available to the learner at training and test time.
In our setting we focus on the single-head setting where a learner is unaware at test time which task the data belongs to.
In other words, when new classes are presented the learner must learn to distinguish them from all previously observed classes.
Another common distinction is the online and offline setting.
In the offline setting the learner receives the full set of data for each task and can perform unlimited training on this before moving to the next.
On the other hand in the online setting the learner receives one or a small number of samples from a stream and must process and/or store these samples under a computational and memory budget.
In this work we focus on the latter.
An extreme continual classification scenario involves the learner observing individual classes at each changing point in the data stream.
Until now this scenario has been studied to a limited degree in the offline setting [13, 10], but has not been considered in the online setting.
This is due to the inherent challenging nature of the online setting which has only recently started obtaining results competitive with iid baselines [5, 4].
Indeed, in [10] they use an expensive regularization approach that is impractical under the constraints of online continual learning.
Furthermore, the method proposed by [10] uses a pre-trained model, performing an initial stage of offline iid pretraining on half of the classes in the dataset.
Where as, our aim is to tackle the problem without any assumption of pre-trained models.
In the online continual learning setting the best performing methods rely on various forms of rehearsal, where old samples are stored in a finite memory and reused at later points in the stream.
A common strong baseline is experience replay[8, 2].
Recently, Caccia et al.
[4] showed that in online continual learning settings, after each distribution shift (task boundaries), the model observes a significant drift in the representation of previously learned classes.
They hypothesize that this is fundamentally due to: (i) new class samples representations lying close to older classes and (ii) the loss structure of the standard cross entropy applied on a mix of seen and unseen classes.
To mitigate this, they proposed a method to allow fine-grained control over which samples will be pushed away from other samples given an incoming batch.
The main downside of methods proposed by Caccia et al.
[4], is the critical need for negative samples in the incoming batch to learn the representation of incoming data while avoiding catastrophic forgetting.
The result is the inability of their methods to be applied to the more challenging setting where the model observes one class at a time in an online data stream.
In this work, we apply recent ideas from [12], which break the dependence on negative samples in self-supervised contrastive learning to the supervised continual learning setting.
This allows us to maintain an asymmetric loss structure between replay samples and incoming data as in [4], while providing learning on new incoming class data without the need to contrast to old classes or their representations.
Augmenting this approach with a regularization term that constrains class prototypes, further yields a new method which far exceeds performance of strong online CL benchmarks in the one class incremental setting and is competitive with the top performing methods in the multi-class incremental setting.
Methods
Similar to [4], given a model f θ ( x ) representing a neural network architecture with parameters θ , we want to minimize the classification loss L on the newly arriving data batch while not negatively impacting previous learning of other classes, and have the ability to be applied to one-class incremental settings.
We opt for a specific loss structure on the incoming batch that would enable the model to learn the representation of each class independently and in isolation from all other classes, either in the incoming batch or in buffered samples.
This removes the need for negative samples and enables the model to learn useful representations, even in one-class incremental settings.
We first present our supervised modification of BYOL and apply it in the asymmetric setting of [4], then propose our new method.
2.1 Supervised BYOL (SupBYOL)
In order to remove the need for negative samples to learn the representation of incoming batch, we apply a supervised modification of BYOL [12] on the incoming data.
BYOL uses a twin architecture with online and target networks.
The online network is comprised of three stages: an encoder f θ , a projector g θ , and a predictor q θ .
The target network has the same architecture but with different parameters ξ , and is the exponential moving average of the online network: ξ ← τξ + (1 − τ ) θ .
We use the following loss, a modification of BYOL loss [12], on the incoming data X in .
[Equation]
where sim ( a, b ) =
a T b τ ∥ a ∥∥ b ∥ , and z θ = g θ
( f θ ( x i )) and z ξ = g ξ ( f ξ ( x i )) are the projections outputted by online and target networks respectively.
We denote the incoming N data points by X in , data replayed from the buffer by X bf , and the set of positive samples with respect to x i by P .
Following [4], for the rehearsal step, we apply a modified cross-entropy objective as per [14].
[Equation]
[Figure]
where Y all is the set of all observed classes, and z i =
g θ
( f θ ( x i )) .
2.2 CCP: Continual Contrast of Class Prototypes
Although BYOL can learn the representation of each class independently from other classes, in case of supervised learning, specially where we have multiple classes in the incoming data, it is tricky to evade collapsed representations.
In some cases, we observed representation collapse between new classes in the incoming data or old classes in the memory buffer, which results in forgetting of the corresponding classes (drop in accuracy).
Also, due to the twin architecture design, BYOL has a larger compute footprint than ER-AML, which makes it less suitable for online continual learning.
Inspired by [6], we use randomly initialized prototypes as class cluster representatives to reduce intra-class variance while enforcing inter-class variance.
We represent each observed class y ∈ Y by a prototype c y in prototypes memory C .
The network parameters θ and class prototypes C are jointly optimized to project an instance x i ∈ X in of class y close to its corresponding cluster prototype c y as well as other samples of class y in the batch, i.e. positive samples P ( x i ) .
In order to evade collapsed representations, we use a contrast term between class prototypes which enforces inter-class variance.
So, as the prototypes act like a representative for the whole corresponding class samples, we can enforce inter-class variance without any need for negative samples, and also provide stable training without BYOL’s optimization tricks such as twin architecture and predictor head.
We formulate the objective as follows:
[Equation]
where sim ( a, b ) =
a T b τ ∥ a ∥∥ b ∥ , z i =
g θ
( f θ ( x i )) , and y ( x ) denotes the class label of x .
As in the case of SupBYOL, we combine this with the same L 2 term applied to the buffered samples.
In order to avoid contrast between new and old classes introduced by the last term, we do not directly update old class prototypes using gradients.
In other words, for the incoming data, we perform stochastic optimization to minimize L 1 + L 2 with respect to θ and c ∈ C in , where C in is the class labels of the incoming data.
However, to update the old class prototypes, we follow [9] and use the replayed samples to obtain a momentum update after each training step to stabilize the model against drastic changes in the representation of learned classes: c y ← αc y + (1 − α )
¯ c y where y ∈ Y bf denotes the label of old classes stored in the buffer and ¯ c y is the updated prototype for class y .
Experiments
Our method enables the model to learn the representation of each class in isolation from other classes without the need to have more than one class in the incoming data.
Following [7, 4], we use a reduced
[Table] [Table]
[Table]
Resnet-18 with batch size and the rehearsal batch size of 10 samples.
All of the models are trained in single head setting, so the task id is not revealed to the model at test time.
Similar to [4], we find data augmentation to be useful in most of the settings, specially in simple datasets like CIFAR-10 with a small buffer size where the model might overfit on the buffer samples.
Our data augmentation pipline consists of a simple random crop followed by random horizontal flip.
We use SGD for optimization with a learning rate of 0.1 as in [1].
We now present the experiments on 10 and 100 task settings for
Split CIFAR-10 and Split CIFAR100.
Split CIFAR-10 typically partitions the dataset into 5 disjoint tasks containing two classes each ( [1, 16]).
In this work we also consider partitioning into 10 disjoint sets (1 class each).
When applying the 10 class split we will indicate S = 1 , while the case of 2 class tasks will be denoted S = 2 .
We show in Table 1 results for split CIFAR10 on S = 1 and for S = 2 in Table 2 showing the overall accuracy and forgetting at the end of the task sequence for a variety of memory settings.
Observe that in the one class setting ER-ACE and ER-AML cannot be applied as they require other classes in the incoming data.
SupBOYL and CCP greatly outperform the existing methods in this setting, with CCP obtaining top performance in all categories.
For the multi-class setting of S = 2 , SupBYOL performs poorly, but CCP greatly improves upon SupBYOL and achieves performance close to ER-AML and
ER-ACE in both accuracy and forgetting categories.
Split CIFAR-100 Consists of 100 classes typically split into 20 tasks, each containing a disjoint set of 5 labels ( S = 5 ).
In our one class incremental work we will also consider the case of splitting into 100 distinct task switches ( S = 1 ).
All CIFAR experiments process 32 × 32 images.
In Figure 1, we show the results on S = 1 and S = 5 .
In the S = 1 setting especially as the number of classes grows, we observe increasing margins over existing methods for both SupBYOL and CCP.
On the other hand, in the S = 5 setting, SupBYOL does not perform well as in the CIFAR-10, while
CCP matches the performance of ER-AML and is close to the performance of ER-ACE (which nearly matches the i.i.d performance).
Conclusion
Our major contribution is a new method to handle online one-class incremental learning without the need for negative contrasts.
We demonstrated that this method can outperform strong baselines, and is also applicable and highly competitive in traditional online continual learning settings.
Furthermore we have shown that recent advances in self-supervised learning without contrasts can be adapted to supervised settings, particularly in continual classification.
Future work can consider if our approach can be applied in the offline one class incremental setting
Acknowledgements
This work is partially funded by NSERC DG “Towards Continual and Compositional Learning in the
Visual World”.
We would like to thank Concordia University and Mila for the provided computational resources and research environments.
We also would like to acknowledge the support from Compute Canada.
Lastly, we thank Amir Sarfi for helpful discussions and suggestions.
A Appendix
In this section we provide some extra anytime evaluation results for various memory sizes on the considered datasets , i e. Split CIFAR-10 and Split CIFAR-100, in both one-class(left plots) and multi-class(right plots) incremental learning.
The results complement the ones presented in the main text
[Figure] [Figure]