Online Continual Learning Without the Storage Constraint
Authors: name: Ameya Prabhu, name: Zhipeng Cai, name: Puneet Dokania, name: Philip Torr, name: Vladlen Koltun, name: Ozan Sener

Online continual learning (OCL) research has primarily focused on mitigating
catastrophic forgetting with fixed and limited storage allocation throughout
the agent's lifetime. However, the growing affordability of data storage
highlights a broad range of applications that do not adhere to these
assumptions. In these cases, the primary concern lies in managing computational
expenditures rather than storage. In this paper, we target such settings,
investigating the online continual learning problem by relaxing storage
constraints and emphasizing fixed, limited economical budget. We provide a
simple algorithm that can compactly store and utilize the entirety of the
incoming data stream under tiny computational budgets using a kNN classifier
and universal pre-trained feature extractors. Our algorithm provides a
consistency property attractive to continual learning: It will never forget
past seen data. We set a new state of the art on two large-scale OCL datasets:
Continual LOCalization (CLOC), which has 39M images over 712 classes, and
Continual Google Landmarks V2 (CGLM), which has 580K images over 10,788 classes
-- beating methods under far higher computational budgets than ours in terms of
both reducing catastrophic forgetting of past data and quickly adapting to
rapidly changing data streams. We provide code to reproduce our results at
https://github.com/drimpossible/ACM.