Paper: Amphibian: A Meta-Learner for Rehearsal-Free Fast Online Continual Learning (ICLR-2024)

Reviewer comment (Experiments):
I have concerns about the fairness of the comparable online setting. In both La-MAML and the code provided in your appendix, you have the hyper-parameter 'self.glances', which allows your online training batch to be optimized multiple times. While it's understandable that the early CL work La-MAML adopts this 'single-pass' setting due to the lack of clear definitions for online and offline CL settings, if you're adopting the online CL setting, you need to clearly highlight the differences between your experimental setting and the standard online CL setting where each example can only be seen once. Furthermore, you should provide results of other comparable methods under this setting or set your hyperparameter 'self.glances' to 1 for a fair comparison.

Author response:
As per La-MAML we used the online (single epoch) learning setup where there is a glance hyperparameter. To have a fair comparison, in Table 1 we have reported results for all the other baselines in the same setup where multiple glances are allowed. Table 6 in the Appendix lists all the hyperparameters for these baselines, which contains the glance values as well.

Reviewer comment (Experiments):
La-MAML, as the most important baseline, also learns the learning rate through bi-level optimization, similar to your learned diagonal scaled matrix. Despite the results provided in Table 3, I'm still unclear if the learned diagonal scaled matrix truly outperforms the learned learning rate for each parameter of La-MAML. The differences between La-MAML and the proposed Amphibian are: (a) La-MAML uses samples from the memory buffer, while Amphibian does not; (b) Both La-MAML and Amphibian apply the ReLU operation on the learned learning rate or the diagonal scaled matrix, but La-MAML only applies this ReLU operation during the outer loop, while Amphibian uses it in both the inner and outer loops. Existing research [1] shows that using the ReLU operation on the learning rate during both inner-loop and outer-loop can effectively improve performance. So it is unclear if your performance gains lie in this different operation. In Table 3, you only show the ablation study on the first point. Therefore, it doesn't convince me that the learned diagonal scaled matrix is truly superior to the learning rate learned by La-MAML.  
Reference: [1] Learning where to learn: Gradient sparsity in meta and continual learning. NeurIPS, 2021

Author response:
In their implementation, La-MAML used ReLU operation in both inner and outer loops (Please see official La-MAML implementation [1] lines 55 and 99). In Table 1 we showed results for La-MAML (with rehearsal) and in Table 3 La-MAML (without rehearsal). Comparing these results we see that La-MAML with replay performs significantly better in terms of accuracy and forgetting mitigation. Since our method does not use any rehearsal and outperforms La-MAML (with rehearsal), learned diagonal matrices (unique component of our method) play a major role in preventing catastrophic forgetting.

Reviewer comment (Theory):
In my view, the learned diagonal scaled matrix is equivalent to learning the important weights for the current task. However, like EWC, it learns the important weights (i.e., the Fisher information matrix) for each task and suppresses the model’s updates in these directions. I'm still unsure how the timely learned diagonal matrix can prevent catastrophic forgetting of previous tasks. I believe the authors need to provide more explanations. Is the proposed method, Amphibian, only applicable in the relatively simple Task-IL setting? Providing the Class-IL online CL performance could be much more convincing.

Author response:
The proposed method can also work in Class-IL online setup and the results are given already in the manuscript (in Figure 4, section 6.1).

Reviewer comment (Novelty):
Could you please provide a more detailed explanation of how the proposed method differs from La-MAML?

Author response:
The main differences between our method (Amphibian) and La-MAML are: (1) La-MAML uses a replay buffer for storing old data, whereas Amphibian does not use any replay buffer. Thus La-MAML is a rehearsal-based method whereas Amphibian is a rehearsal-free method. (2) LA-MAML learns per parameter learning rate using data from both current and past tasks, whereas Amphibian learns layer-wise diagonal scale matrix using only the current batch data. Thus, the number of extra learnable parameters (memory overhead) in La-MAML is very high compared to Amphibian (Please see Figure 4(c)). (c) Amphibian minimizes a meta-objective that encourages alignments of gradients among given data samples along selected basis directions in the gradient space (Eq 7), La-MAML optimizes a different meta-objective.

Reviewer comment (Theory):
Could you explain the rationale behind constraining the matrix to a scale matrix?

Author response:
We adopted a low-dimensional gradient space representation in each layer of the network (please see section 3 for details). For each basis describing this space, we wanted to assign a scale by which incoming gradients will be modified during CL training. For this purpose, we introduced a diagonal scale matrix (in Eq. 4,5).