Paper: Online Weight Approximation for Continual Learning (ICLR-2024)

Reviewer comment (Novelty):
It is unclear the advantage of the proposed method compared to other related methods. In the introduction and the method sections, the paper should state the advantages of the proposed method compared to others. This would make the positions of the paper more clear.

Author response:
The advantage of the proposed method consists in the possibility to train a
single model on several tasks while retaining the capability to predict on
previous tasks by means of a simple weight adaptation. The memory required
for this adaptation is limited, being only N times the number of parameter in
the network, where N is the number of coefficients required to approximate
the trajectory of the weights across the task.  As we show in the
experiments, the proposed approach is particularly efficient in scenarios
with many tasks with respect to a simple experience replay method, which
would require much more memory to obtain similar results.

Reviewer comment (Experiments):
Your first set of experiments (section 4.1) should have some knowledge transfer. References (3), (4) and (5) can transfer knowledge across tasks. Your method by nature cannot perform knowledge transfer.

Author response:
However, we do not understand the comment, “Your method by nature cannot
perform knowledge transfer”. The manner in which the coefficients
approximating the weight trajectory among different tasks are adjusted,
distinctly retains a memory of the previous tasks. Conversely, newer tasks
modify the weight approximation even at the outset of the task sequence. We
would greatly appreciate it if you could provide further clarification on
this remark.

Reviewer comment (Writing):
Is w^i representing the set of weights for task i?

Author response:
Regarding the data used to train Eq. (7), the update of the coefficient is
solely based on the value of the $i$-th weight of the network at a specific
training epoch, as outlined in Algorithm 1. The notation $w_i^k$ denotes the
value of the $i$-th weight of the network at the $k$-th update step in our
method. If we employ an update policy that adjusts the coefficients only once
per task, then indeed $w_i^k$ represents the value of the $i$-th parameter of
the network at the conclusion of the $k$-th task.

Reviewer comment (Experiments):
I also think that it is not fair to reduce the number of replay samples of a rehearsal based methods such as Experience Replay, as you are model (parameters) is using very limited memory. This type of comparison is unfair to a rehearsal based methods. In that case, I would suggest the authors to not to use any memory at all and simply compare with regularization based methods such as EWC[1], MAS[2] etc.

Author response:
Regarding the choice on the number of replay samples, we agree that increasing the number of samples would have increased the performance of the compared replay method. However, we think that employing a number of samples equivalent to the memory required by our method would be the most fair setting.

Reviewer comment (Reproducibility):
It is mentioned in the paper that memory budget in your method is constant. Could you explain that? Particularly, I wonder the number of Euler steps $S$ in the coefficients $\tilde{C}_i^S$ was fixed or not. If not, how does it evolve properly, without proportional increasing?

Author response:
We thank the reviewer for their specific questions, helping us to enhance the
presentation of our work.

Our method relies on the concept of approximating the trajectory of weights
across different tasks. We achieve this by introducing a technique to update
the coefficients of a specific polynomial expansion for each weight in the
neural network (NN). Once we determine the order of the approximation,
denoted as $N$ (representing the number of coefficients to retain in the
expansion presented in Eq. 2), irrespective of the Euler's step performed,
only $N$ values need to be stored in memory for each weight of the
network. The elegance of the method lies in the fact that we simply
*update* these coefficients rather than adding new ones. The update is
performed in a manner that preserves the memory of the information observed.

Reviewer comment (Experiments):
Which replay method did you compare with specifically? Replay methods have had a long history since the emergence of continual learning research, and their performances varied quite a lot. But the performance in your experiment is not so good, which is hard to believe as a state-of-the-art. It would be valuable to discuss how OWA compares to other methods.

Author response:
We thank the reviewer for their specific questions, helping us to enhance the
presentation of our work.

Regarding the replay strategy, we employed a straightforward experience
replay method, determining the size of the buffer as outlined in the paper.