Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This paper proposes to use hypernetwork to prevent catastrophic forgetting. In deep learning, the information of the samples are converted to parameters during the training process, however, future training process could interfere with the information from the previous tasks. One of the method to prevent forgetting is to use reheasal, which retrains the network with previous data. The mechanism of this work is to store the previous samples as a trained point in the parameter space, so that a set of points in the original space is stored and thus rehearsed as one point in the parameter space, this saves both the memory and computation.

I give a weak accept of this paper due to the following reasons:
Pros:
- The idea of converting a set of data points to one point and rehearse at a meta level is a smart and novel idea.
- It shows significant improvement compared to baseline methods, especially for split CIFAR experiments.
- The Appendix contains a fair amount of details and additional experiments on generative models.

Cons:
- This works assumes a task incremental setting, during training process task is received one by one, within each task we could assume i.i.d shuffling of the data. During testing, the task boundary is optional. Although this setting has been taken by many other works in this field, it is also criticised that availability of task boundary is an unrealistic setting. A more realistic setting would be to continually learn with a continuous non-stationary stream of data, which indicates there's no split of train / test phase. Thus a general continual learning method should not require task boundary, which would be problematic for this work as it depends on task conditioning.
- For the rehearsal objective in 2, L2 penalty is used. This could be a problem as minimizing the L2 distance in the parameter space does not necessarily minimize the task loss.

Questions I have that needs clarification:
- Chunking: Are the chunking parameters shared/updated across tasks?
review_title: Official Blind Review #2

final_score: rating: 8: Accept
review_content: Review of “Continual learning with hypernetworks”

This paper investigates the use of conditioning “Hypernetworks” (networks where weights are compressed via an embedding) for continual learning. They use “chunked” version of the hypernetwork (used in Ha2017, Pawlowski2017) to learn task-specific embeddings to generate (or map) tasks to weights-space of the target network.

There is a list of noteworthy contributions of this work:

1) They demonstrate that their approach achieves SOTA on various (well-chosen) standard CL benchmarks (notably P-MNIST for CL, Split MNIST) and also does reasonably well on Split CIFAR-10/100 benchmark. The authors have also spent some effort to replicate previous work so that their results can be compared (and more importantly analyzed) fairly to the literature, and I want to see more of this in current ML papers. (one note is that the results for CIFAR-10/100 is in the Appendix, but I think if the paper gets accepted, let's bring it back to the main text and use 9 pages, since the results for CIFAR10/100 are substantial).

2) In addition to demonstrating good results on standard CL benchmarks, they also conduct analysis of task-conditioned hypernetworks with experiments involving long task sequences to show that they have very large capacities to retain large memories. They provide a treatment (also with visualization) into the structure of low-dim task embeddings to show potential for transfer learning.

3) The authors will release code to reproduce all experiments, which I think is important to push the field forward. Future work can not only reproduce this work, but also the cited works.

The work seems to be well written, and the motivation of using hypernetworks as a natural solution to avoid catastrophic loss is clearly described. Overall, I think this work is worthy of acceptance and should encourage more investigation into hypernetworks for CL and transfer learning going forward in the community.

review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: Paper 1872
Paper proposes a method for CL. The method is based on hypernetworks. These networks are a metamodel, which produce the parameters (from a task-conditioned embedding) which will be used in the main network. Preventing forgetting in the main network is now, replaced by preventing forgetting in the hypernetwork. This is done by imposing a regularization on the hypernetwork outcome, imposing that the generated weights should be similar for previous tasks (similar to Li & Hoiem who impose this on the network outputs). In addition, the paper proposes chunking, which refers to using an additional set of chunk, embeddings which are shared for all tasks, which allow compressing the hypernetwork. Furthermore, they propose an extension that allows for image replay (this is not an easy extension and an impressive contribution on itself, but maybe confusing for the current paper).

CONCLUSION
Overall, I like the idea of the paper and it is well explained. However, I found that the experiments of the paper where not well designed to verify the main contribution (hypernetworks), nor where they compared to the most relevant methods. I am borderline with this paper, and recommend borderline accept (borderline not being an option).

QUESTIONS
1. I think the motivation of why hypernetworks are expected to have less forgetting (than addressing forgetting directly in a network) should be discussed early in the paper. 

2.I do not understand why the training is performed in two steps. First computing a candidate Delta THETA_H and then o ptimizing Eq 2. Why not directly optimizing Eq 2, replacing the second factor with|| f_h(e^t, THETA^*_h)-f_h(e^t, THETA_h) ||. This is how this regularization is normally applied (e.g. Li & Hoiem). If the authors insist in using Eq 2, I would like to see it compared with the proposed version. 

3. The experiments should show that hypernets better address CL then addressing this directly in the network (and preferably provide reasons for this). Comparison with the closest methods like HAT and PackNet should be included. Especially, HAT is interesting since it is also based on an embedding. 

4. Also, more experiments on CIFAR would be welcome. The MNIST variations already provide very high accuracies. For CIFAR-10/100 groups of 20 classes are added in 5 steps ? Scenario CL3 would be interesting for CIFAR as well. 

5. I would like to see more analysis and results for the chunking. (As said before the replay is also a nice addition, but it seems an add-on of the main-text, shrinking the space to analyze the main contributions of the paper in the experiments.)
I guess HNET+ENT for CL1 scenario does not use ENT and is just HNET?

review_title: Official Blind Review #1

Meta-review: This paper proposes to use hypernetwork to prevent catastrophic forgetting. Overall, the paper is well-written, well-motivated, and the idea is novel. Experimentally, the proposed approach achieves SOTA on various (well-chosen) standard CL benchmarks (notably P-MNIST for CL, Split MNIST) and also does reasonably well on Split CIFAR-10/100 benchmark. The authors are suggested to investigate alternative penalties in the rehearsal objective, and also add comparison with methods like HAT and PackNet.
Decision: Accept (Spotlight)