Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper explores the usage of EBMs in continual learning for classification. Although the application of EBMs in continual learning is novel, the general idea is a special case of the usage of EBMs for structured prediction, which has been widely studied. For instance, multi-class classification can be considered as a special version of multi-label classification, which has been studied in Belanger and McCallum (2016) and a set of follow-up works. The main difference here is that multi-class classification is a simpler problem, and all possible classes can be enumerated in O(N), but in multi-label classification, more complicated inference such as gradient-descent based approaches must be used.
The contrastive training can be seen as a special case of margin-based training (Belanger and McCallum, 2016; Rooshenas et al. 2019), where the margin is infinity.
I believe the works in using EBMs for structured prediction must be cited here as they are closely related.

The authors also explored the effect of ML training on interference with past data and showed that using single sample ML approximation can significantly alleviate the catastrophic forgetting problem.  
I believe that this is an interesting observation. 

Typo: "current bath" in Section 5.1.4

Belanger and McCallum (2016), Structured Prediction Energy Networks.
Gygli et al. (2017), Deep Value Networks Learn to Evaluate and Iteratively Refine Structured Outputs.
Rooshenas et al. (2019), Search-Guided, Lightly-supervised Training of Structured Prediction Energy Networks
review_title: Maximum likelihood training results in catastrophic forgetting

final_score: rating: 5: Marginally below acceptance threshold
review_content: ==========================

before revision

==========================

Review: Motivated by the effectiveness and naturalness of energy-based models, this paper proposes to use energy-based learning framework for continual learning. Empirical studies are performed to validate the proposed strategy on several continual learning benchmarks.

Strength: 
+ This paper applies EBMs to the task of continual learning, which is interesting and relevant to ICLR conference. 
+ The paper is well written and easy to follow. 
+ The paper is technically sound, since the formulation of the EBMs are well derived by other prior works.  


Concerns: 
+ The contribution of the paper is insufficient for publication. The energy-based learning framework for discriminative purpose has been developed for a long time, even though recently researchers in the field machine learning are enthusiastic about developing energy-based models for data generation. 
+ The underlying theory of the proposed method is developed by other papers, the only contribution of this paper is to apply the EBM to the continual learning, which is quite straightforward. 
+ Missing key reference about EBM for discriminative learning. The core of this paper is mainly based on the finding of the transition between discriminative EBM and generative EBM, which is originally presented in reference [a]. The current paper misses to discuss and cite this paper. 
+ missing relevant reference about generative EBMs in related work. Even though this paper is not directly related to EBMs for data generation, but it DID discuss the development of it in its paper. The current related work about EBMs for generative purpose is incomplete in the sense that it skipped some pioneering works and important application with EBMs. For examples,  [1] is the first paper to use ConvNet-parameterized EBMs with Langevin for image generation.  Training EBMs with assisting networks can be found in [2] and [3].  Also, writing a section of comprehensive related works about energy-based learning is not necessary but encouraging.  

references
+ [1] A Theory of Generative ConvNet (ICML 2016)
+ [2] Cooperative learning of descriptor and generator networks. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI 2018).
+ [3] Divergence triangle for joint training of generator model, energy-based model, and inference model. (CVPR 2019)   

===================================

After a revision

===================================
Thank you for your efforts to revise the paper. The revised parts about related work look good to me. I agree on that citing all those EBM application papers is not necessary. But doing so can provide a comprehensive and complete development of  the DeepNet-EBM. Again, this is not required and it will not affect the rating.   

I also acknowledge the existing contributions in the current paper and admit that such a direction is promising, but I still feel that the current paper doesn't fully explore this area with more solid experiments. Thus, the whole contribution is quite marginal. By taking into account all these concerns, I will change my rating from 4 to 5.    
  

review_title: Insufficient novelty + relevant reference missing

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary: This work shows that energy-based models (EBMs)  are a promising model class for continual learning problems. According to the experiments, EBMs outperform the baseline methods by several continual learning benchmarks.
 
+ves: 
1. It is interesting to see that energy-based models are introduced for the classification continual learning problems. In the paper, the authors show that EBMs achieve a significant improvement on four standard CL benchmarks. It is really surprising that the improvements are so large. 
 
2. Some analysis, for example,  “energy landscape” and “Interference with past data.” seems interesting. It shows benefit of energy-based model for continual learning. Both of them indicate that y EBMs suffer less from catastrophic forgetting.
 
 
Concerns
1. It seems like the gradient computation of equation (5) is not corresponding to equation (4). 

2. It is nice to see the improvement of EBMS. In this work, a difference architecture is usually in EBMS. The architecture is different from the baseline. It is nice that EMBS has a more flexible architecture to score the input x and output y. However, the improvement of your work is from architecture or from the training. It is better to do a clear claim. If the benefit is from the architecture, do you treat it as a black box? Or it is from the training objective?  It is possible to show some learned structure in your formulation. The architecture of EBM seems large. 


Questions during the rebuttal period: 
Please address and clarify the cons above:
 
1. The experiment setting detail is unclear. The training details of the proposed method and baseline are unclear.  And the baseline detail is a little confusing to me. In section 5.1.2, it says “ All the baselines and EBMs are based on the same model architecture”. It seems the architecture of EBMs is different.

2. Some related work on energy-based model: multiple label classification[1] , sequence labeling [2] and machine translation [3]
[1] Structured Prediction Energy Networks, ICML 2016
[2] Benchmarking Approximate Inference Methods for Neural Structured Prediction. NAACL 2019
[3] ENGINE: Energy-Based Inference Networks for Non-Autoregressive Machine Translation
ACL 2020

review_title: Review 3

final_score: rating: 4: Ok but not good enough - rejection
review_content: The writing of the paper needs improvements. I am still unsure how the paper learns each new task. You talked about batches but never talked about how each new task is learned specifically. It creates doubts in my mind. E.g., does each batch contain some examples from old tasks? Is this training like for multi-task learning?

You wrote “In the continual learning setting, we assume classes in Y_B are uniformly distributed in every new batch.” Is Y_B fixed for each task or each batch? 

You need a negative class in each batch. This means that your algorithm cannot work in the scenario where a task has one class only. Most of existing techniques can handle this case although they use 2 or more classes in a task in their experiments. I think this is a serious limitation. It is only suitable for Task-IL, which is an easier problem to solve. 

I think the claim that existing techniques need to fix the number of classes beforehand is not correct. I know hat most of them fix the number in their code or experiments, but I don’t see why they cannot use a large number or dynamically add new class heads when needed, say, using cross-entropy as the loss function. 

The experiments are not well described, and baselines are old. Many newer baselines should be included, e.g., 

Learning a Unified Classifier Incrementally via Rebalancing. CVPR 2019. 
Overcoming catastrophic forgetting for continual learning via model adaptation. ICLR, 2019. 
Random path selection for continual learning. NeurIPS 2019
Continuous learning of context-dependent processing in neural networks. Nature Machine Intelligence, 2019
Large scale incremental learning. CVPR 2019

For each dataset, you used one setting for tasks only, e.g., CIFAR100, 10 tasks. More than one setting should be tried to show the generality of the approach. 

review_title: This paper proposes an energy-based model for continual learning, which seems to be new. However, I have several concerns about the paper, which are described in the detailed comments. 

Meta-review: There were opinions on both sides of this paper from the reviewers.  Reviewers were excited by the novel application of energy-based models (EBMs) to continual learning and the resulting performance gains, but were concerned by the more direct application of EBMs (which has been explored in other work, and here adapted to the continual learning setting, so its contribution is marginal) and with the depth of the evaluation, which they thought could be pushed farther. Overall, the reviewers agreed that this paper could benefit from another round of revisions to strengthen its contribution, incorporating many of the excellent points made by the authors in their responses.
Decision: Reject