Paper: Balancing Stability and Plasticity in Continual Learning: the readout-decomposition of activation change (RDAC) framework (ICLR-2024)

Reviewer comment (Writing):
I would think the clarity part in gradient decomposition could be better elaborated.

Author response:
Dear reviewer,

Thank you for your comments. Please check the comment - https://openreview.net/forum?id=9vkgAaCI3F&noteId=rcFk2d3LLJ - for a summary of the changes to the paper.

We have taken your suggestion into account and reworded that section, Section 5, to make it clear what parts refer to the gradient decomposition algorithm and what refers to the causal analysis between the stability-plasticity trade-off and the activation changes in the readout range and null space.

We hope that our responses and revisions have clarified our contributions and we want to thank you for your helpful insights.

Thank you, The Authors

Reviewer comment (Experiments):
So the explanatory power of the framework for nonlinear networks is unclear. The analysis relies on precisely computing the readout null space, which may be difficult in nonlinear models where the spaces are less well-defined. It's also unclear if the insights will fully translate when there are multiple nonlinear layers. The analysis of activation changes is quite limited for the nonlinear network experiments. They only approximate the null spaces for complicated models, analyze a single layer rather than the whole network, and observe overall trends without an in-depth study like was done for the linear case.

Author response:
Re: the point about "analyze a single layer rather than the whole network", the RDAC framework was built around the observation that readout misalignment constitutes a major problem in continual learning. This implies that we can get a lot of insights by just looking at the readout layer. As shown in our experiments, this turns out to be quite informative (Figures 2 & 3). 
Re: the point about "observe overall trends without an in-depth study like was done for the linear case", as we mentioned in Section 5 and Appendix A.3, it is hard to analytically derive a solution to an analog of Eq. 1 (Eq. 2 for a 3-hidden layer network) in deep non-linear networks. Moreover, the framework serves as a diagnostic tool for continual learning algorithms and the hope is that the analysis shown in Figure. 2 and outlined in Section 3 can be used generally (it works for a larger dataset and architecture as shown in Appendix A.2).

Reviewer comment (Experiments):
The continual learning scenarios are relatively simple (Split CIFAR and MNIST). Needs more exploration on complicated dataset benchmarks and continual learning scenarios.

Author response:
We now include results on continual learning on TinyImagenet with a slim ResNet that qualitatively shows the same results as in Figure. 2, solidifying the generality of our framework.

Reviewer comment (Experiments):
No comparison against latest SOTA.

Author response:
Our aim was to analyze continual learning algorithms from the two major classes - regularization and rehearsal, to assess our framework. All the algorithms considered are still competitive benchmarks in the field. Future work, especially when it comes to designing algorithms based on this framework, will indeed account for the SOTA algorithms.

Reviewer comment (Evaluation):
To the best of my understanding, the RDAC framework _interprets_ projections onto the null and range spaces and any changes thereof as representing stability and plasticity in the networks. This seems to be a logical leap that is not verified -- the experiments in section 4 and 5 simply analyse the changes of gradient projections for different networks in these spaces. While I find the connection between the various network hyperparameters and the changes in these spaces interesting, it is not clear whether they truly represent a stability / plasticity tradeoff without experiments also showing how changes in these subspaces are correlated with performance on the continual learning tasks.

Author response:
Re: Weakness, Point 1 (+ Questions, Q1):
In the updated paper, as described in Section 3, the predicted correlations between the stability-plasticity trade-off and the activation changes in the readout range and null space are complex. We laid out the different scenarios (Cases). In the results, Section 4.2., we link the behavior of various continual learning algorithms to those Cases, thereby establishing the predicted correlations. Indeed, this does not establish a causal link. However, in Section 5, we analytically derive such a causal link for a one-hidden layer linear network, which provides support to our account. Such a causal link is hard to establish for deep non-linear networks as we do not yet have a method for disentangling the projections of the readout range and null space in the upstream layers of the network, as mentioned in Section 5. In sum, we outlined a framework based on previous findings mentioned in the introduction and took the first steps in ascertaining its causal validity.

Reviewer comment (Theory):
The linear approximation in the RDAC framework is useful and makes it easy to analyse / apply to a variety of networks. However, given that it is non-trivial to derive a similar framework for non-linear activations or readouts, it is difficult to see how insights from the current linear framework can be extrapolated to the nonlinear setting. While the paper presents results on nonlinear networks in section 4, it is not clear whether these insights will extrapolate to other nonlinear networks.

Author response:
Re: Weakness, Point 2:
Indeed deriving an analytical solution for deep networks is hard, as now shown in Appendix A.3. However, the idea that the range is related to stability and that the null space is related to plasticity comes from considerations of activation changes at the readout, as explained in Section 3. Although we do not have an analytical solution for the deep non-linear case, we think that our intuitions are valuable in characterizing the link between the stability-plasticity trade-off and the activation changes in the readout range and null space

Reviewer comment (Experiments):
There are no experiments showing how tuning the regularisation strength for SI, EWC and LwF, or memory strength / replay-buffer size for GEM to achieve a particular balance of stability and plasticity affects network performance in these tasks. This also makes it hard to judge whether the insights derived from the RDAC framework on nonlinear tasks really hold, can be extrapolated to other networks / methods and whether they are useful in developing methods for continual learning.

Author response:
Re: Weakness, point "Furthermore, there are no experiments showing how tuning the regularisation strength ...":
Figure 2 shows the relationship between the regularization strength of each algorithm and its influence on the stability-plasticity trade-off and the readout-decomposed activation changes. To show that these results are generalizable, in Appendix A.2, we show that the same pattern holds for a larger dataset (TinyImagenet) and network (slim ResNet), giving us confidence that the framework is indeed generalizable.

Reviewer comment (Theory):
Related to point 2, while it is non-trivial to derive results for nonlinear projections, it would be good to have at least some intuition on how incorrect / applicable insights from a linear approximation would be to nonlinear networks, and whether any future endeavour to derive results for nonlinear networks could correct them.

Author response:
Re: Weakness, Point 2:
Indeed deriving an analytical solution for deep networks is hard, as now shown in Appendix A.3. However, the idea that the range is related to stability and that the null space is related to plasticity comes from considerations of activation changes at the readout, as explained in Section 3. Although we do not have an analytical solution for the deep non-linear case, we think that our intuitions are valuable in characterizing the link between the stability-plasticity trade-off and the activation changes in the readout range and null space

Reviewer comment (Evaluation):
Do the changes in the range / null space truly correlate with plasticity / stability in training the networks?

Author response:
Re: Weakness, Point 1 (+ Questions, Q1):
In the updated paper, as described in Section 3, the predicted correlations between the stability-plasticity trade-off and the activation changes in the readout range and null space are complex. We laid out the different scenarios (Cases). In the results, Section 4.2., we link the behavior of various continual learning algorithms to those Cases, thereby establishing the predicted correlations. Indeed, this does not establish a causal link. However, in Section 5, we analytically derive such a causal link for a one-hidden layer linear network, which provides support to our account. Such a causal link is hard to establish for deep non-linear networks as we do not yet have a method for disentangling the projections of the readout range and null space in the upstream layers of the network, as mentioned in Section 5. In sum, we outlined a framework based on previous findings mentioned in the introduction and took the first steps in ascertaining its causal validity.

Reviewer comment (Theory):
How do we interpret insights from a linear approximation of gradient projections from a nonlinear network?

Author response:
Re: Questions, Q2 (+ Q3):
None of our experiments engage in a linear approximation of gradient projections from a nonlinear network. In the linear network, we do project the gradients during learning. However, that projection is derived from the actual projection our framework wants which is related to the activations, see: Eq. 1. In the non-linear case, solving Eq. 1 does not involve simply projecting the gradients. As mentioned at the end of Section 5, "The non-linear case is analytically more complex to analyze, as the simple decomposition of the task mapping into the old mapping and the change in that mapping, as seen in Eq. 1, is not possible."

Reviewer comment (Theory):
How bad are these linear approximations, and how can we correct them?

Author response:
Re: Questions, Q2 (+ Q3):
None of our experiments engage in a linear approximation of gradient projections from a nonlinear network. In the linear network, we do project the gradients during learning. However, that projection is derived from the actual projection our framework wants which is related to the activations, see: Eq. 1. In the non-linear case, solving Eq. 1 does not involve simply projecting the gradients. As mentioned at the end of Section 5, "The non-linear case is analytically more complex to analyze, as the simple decomposition of the task mapping into the old mapping and the change in that mapping, as seen in Eq. 1, is not possible."