Paper:
We introduce and study the problem of Online Continual Compression, where one attempts to learn to compress and store a representative dataset from a non i.i.d data stream, while only observing each sample once. This problem is highly relevant for downstream online continual learning tasks, as well as standard learning methods under resource constrained data collection. To address this we propose a new architecture which Stacks Quantization Modules (SQM), consisting of a series of discrete autoencoders, each equipped with their own memory. Every added module is trained to reconstruct the latent space of the previous module using fewer bits, allowing the learned representation to become more compact as training progresses. This modularity has several advantages: 1) moderate compressions are quickly available early in training, which is crucial for remembering the early tasks, 2) as more data needs to be stored, earlier data becomes more compressed, freeing memory, 3) unlike previous methods, our approach does not require pretraining, even on challenging datasets. We show several potential applications of this method. We first replace the episodic memory used in Experience Replay with SQM, leading to significant gains on standard continual learning benchmarks using a fixed memory budget. We then apply our method to online compression of larger images like those from Imagenet, and show that it is also effective with other modalities, such as LiDAR data.

Review:
This work contributes to introducing a problem called Online Continual Compression. This problem requires to avoid catastrophic forgetting and learn in an online way. Generative methods should be one of the popular ways to do continual learning. This work’s model can be categorized into this clue since it also aims to save samples from old tasks by learning a generative model. In this way, the generator plays a similar role Experience Replay (ER) (here is called Generative Replay). The main core of this work should be the stacked quantization modules (SQM) which can be regarded as a hierarchical variant of the VQ-VAE model. In their SQM, hidden encodings z_q^i will be encoded and its input is z_q^{i-1} which is from previous layer. 
This works covers related works very well. However, there are some questions I am really concerned:
1)	About the studied problem “Online Continual Compression”, what’s the difference between “online” and “continual”? In continual learning, tasks will be learned sequentially, right? If so, continual learning should run in an online learning way. 
2)	The motivation of the hierarchy in this work is unclear. What I mean is that the hierarchical model should be expected to capture higher-level semantic features. But in this work, the index outputs z_q^{i-1} is encoded by its subsequent layer. It seems a bit weird since the z_q^{i-1} is not an image and its elements are index values. So what is the higher-level semantic information? By the way, it seems that there is an error in the model figure 1. The last MSE from Block 1 should be connected to the block before decoder 1 in Block 1, rather than the reconstructed one from decoder 1. Therefore, I strongly suggest authors give more insights and clarify the motivation of hierarchy. Writings in the METHODOLOGY part is unclear. More details about the SQM model should be described in a mathematical way.  
3)	Another question about the details of generative replay. How do you do the replay? Details about this can’t be found in this work? In Alg.1, what is the \theta? Is the \theta_{ae} at line 14 of Alg.1 wrong? It should be \theta_{gen}, right? 
4)	You use the data-stream technique reservoir sampling to add and update the memory buffer (alg. 4). Will it lead to some information loss? Can we just update memory without reservoir sampling? Please give more insights about this.
5)	How to find the distortion threshold d_th in Alg.2? 
6)  the part of ablation studies is good. But I suggest authors should consider a baseline with the same proposed framework but using a single-layer VQVAE with the same memory capacity as the hierarchical models.