Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: I am not familiar with the generative model and continual learning. Thus, I can only give my review based on the authors writing and other reviewers' comments. 
- The paper proposes a new problem setup as "online continual compression".
- The paper gives a combination of many existing techniques to address the new problem. (I agree with Review #4)

I think the presentation and the organization of this paper should be improved in order to properly place their contributions in the literature. 
- Since the authors try to promote a new problem set up with a solution containing little technical breakthroughs, I suggest the authors put more space on motivating the application and showing its impotence. Currently, their presentation focuses too much on methodology parts. 
- Thus, it is better to organize the paper as an application from LiDAR and convince reviewers why their method is good for such an application (Review #1 also thinks the presentation is poor).
- If the authors insist on keeping their paper as a  methodology one, at least one more experiment (not the synthetic one on ImageNet) from real applications are needed (same as Review #2).

Overall, I think the paper is not ready for being published. 
review_title: Official Blind Review #5

final_score: rating: 3: Weak Reject
review_content: This work contributes to introducing a problem called Online Continual Compression. This problem requires to avoid catastrophic forgetting and learn in an online way. Generative methods should be one of the popular ways to do continual learning. This work’s model can be categorized into this clue since it also aims to save samples from old tasks by learning a generative model. In this way, the generator plays a similar role Experience Replay (ER) (here is called Generative Replay). The main core of this work should be the stacked quantization modules (SQM) which can be regarded as a hierarchical variant of the VQ-VAE model. In their SQM, hidden encodings z_q^i will be encoded and its input is z_q^{i-1} which is from previous layer. 

This works covers related works very well. However, there are some questions I am really concerned:
1)	About the studied problem “Online Continual Compression”, what’s the difference between “online” and “continual”? In continual learning, tasks will be learned sequentially, right? If so, continual learning should run in an online learning way. 
2)	The motivation of the hierarchy in this work is unclear. What I mean is that the hierarchical model should be expected to capture higher-level semantic features. But in this work, the index outputs z_q^{i-1} is encoded by its subsequent layer. It seems a bit weird since the z_q^{i-1} is not an image and its elements are index values. So what is the higher-level semantic information? By the way, it seems that there is an error in the model figure 1. The last MSE from Block 1 should be connected to the block before decoder 1 in Block 1, rather than the reconstructed one from decoder 1. Therefore, I strongly suggest authors give more insights and clarify the motivation of hierarchy. Writings in the METHODOLOGY part is unclear. More details about the SQM model should be described in a mathematical way.  
3)	Another question about the details of generative replay. How do you do the replay? Details about this can’t be found in this work? In Alg.1, what is the \theta? Is the \theta_{ae} at line 14 of Alg.1 wrong? It should be \theta_{gen}, right? 
4)	You use the data-stream technique reservoir sampling to add and update the memory buffer (alg. 4). Will it lead to some information loss? Can we just update memory without reservoir sampling? Please give more insights about this.
5)	How to find the distortion threshold d_th in Alg.2? 
6)  the part of ablation studies is good. But I suggest authors should consider a baseline with the same proposed framework but using a single-layer VQVAE with the same memory capacity as the hierarchical models. 

review_title: Official Blind Review #4

final_score: rating: 6: Weak Accept
review_content: The study tackled the problem of limited storage for ever-growing data for a long-term learning scenario. The authors proposed to stack Quantization Modules while separating them during training to obtain an online compression system that has multiple resolutions, different memory horizons, and reduced catastrophic forgetting. They also proposed a modified reservoir sampling to accommodate this architecture. 

The idea is very simple yet interesting, the paper is a good read, and the results seem promising. The ablation study is well designed but not discussed enough. Additionally, the experiments cannot support the idea well since it is on a very special setting, the LiDAR experiment is missing quantitative evaluation, and different tasks (such as text classification, or visual tracking with only one labeled sample) might introduce different difficulties in this online learning setting. I recommend a weak accept for this paper to encourage the idea. 

Therefore, I would recommend the authors to explore other tasks and see if their idea applies to different domains and tasks. Also, a quantitative evaluation for the LiDAR experiment with enough details and some explanation of the inner dynamics of the system during learning seems essential. 

The paper could enjoy a pass of proofreading and typesetting (especially please pay attention to the correct use of \cite{} and \citep{}). Algorithm 1 is not mentioned in the body of the manuscript.
review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: This paper presented a Stacked Quantization Modules (SQM) for the problem of Online Continual Compression, based on the VQ-VAE framework by van den Oord et al. (2017). Experiments were conducted on online continual image classification benchmarks to show the effectiveness of the proposed SQM. In general, the novelty of the paper is a little bit limited and the writing of the paper is not very easy to follow.

- The SQM was constructed by stacking the known VQ-VAE. It is unclear why the stacking works for online continual compression. How many stacks should be used? What are the yellow rectangle parts in Figure 1?

- What are the relationship between Alg 1-4 ? More explanations or discussions are necessary.

- In Section 3.1, "The high level training of the online learned compression is described in Alg. 4.". It is very confused. I can't see the related content in Alg.4.

- In Section 4.1, "In short, we apply Algorithm 4, with an additional online classifier being updated at line 13." I don't understand it. I cannot see line 13 in Algorithm 4, because there is only 10 lines in Algorithm 4.

- In Section 3.3, BITS(.) needs definition.

- In Section 4.1, "Here we consider the more challenging shared-head setting, where the model is not informed of the task (and thereby the subset of classes) at test time. This is in contrast to other (less realistic) CL classification scenarios where the task, and therefore subset of classes, is provided explicitly to the learner Farquhar & Gal (2018)." It is very difficult to understand what the above experimental settings are.

- For Figure 3, the textures or lines of the bottom reconstructed one are not so smoothed or straight as the top one.
review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: This paper focuses on the problem of continual learning with limited memory storage. Specifically, the training data is arrived sequentially (might not be i.i.d.) for a model to exploit and there is not enough storage capacity to keep all the data without compression. This problem is important in many real-world applications with massive amount of data collected. The authors propose an approach named Stacked Quantization Modules to compress the data so that they can be stored efficiently. Each module is an auto-encoder with quantized latent representations. Several aspects including the communication between these stacked modules, and which level will a specific sample be compressed at, are taken into account in the algorithm design. In the experiments, the authors show some quantitative evaluations on CIFAR10 and ImageNet that the proposed method surpass several baseline methods. A qualitative visualization of LiDAR data reconstruction is also demonstrated. Overall I think the paper is tackling an interesting problem with an effective and novel solution. 

I have a few concerns that I wish the authors could help to clarify. First, in the VQ-VAE, each image is quantized to be H*W*D, where each D-dimensional vector is represented by the index of the nearest neighbor in the embedding table of each module. I checked the paper but could not find a place that discuss how this embedding table comes from. It is pre-defined with some pattern or is it learnt somehow? 

What is the latent space size of each module when trained on CIFAR10 and ImageNet? 

The experiments on ImageNet only select 100 classes out of the 1000 classes. Would this method extends to large-scale datasets? How would the form of the tasks (in case of number of classes per task) affect the results? 

There seems to be some typos. For example, the end of the first paragraph of Sec. 4.1 mentioned "line 13" of Alg. 4, which is not referred correctly as Alg. 4 only has 10 lines. 
review_title: Official Blind Review #3

Meta-review: The paper proposes a new problem setup as "online continual compression". The proposed idea is a combination of existing techniques and very simple, though interesting. Parts of the algorithm are not clear, and the hierarchy is not well-motivated. Experimental results seem promising but not convincing enough, since it is on a very special setting, the LiDAR experiment is missing quantitative evaluation, and different tasks might introduce different difficulties in this online learning setting. The ablation study is well designed but not discussed enough.
Decision: Reject