Paper: Mitigating Interference in the Knowledge Continuum through Attention-Guided Incremental Learning (ICLR-2024)

Reviewer comment (Theory):
A single lightweight task-specific vector may not be sufficient to adequately represent and distinguish the crucial information among multiple tasks.

Author response:
We appreciate the reviewer's perspective, but respectfully disagree regarding the sufficiency of a single lightweight task-specific vector. For every continual learning approach, mitigating catastrophic forgetting is a balancing act between several factors like model capacity, parameter growth, buffer size and so on. While we acknowledge that a solitary lightweight task-specific vector may not completely eliminate catastrophic forgetting, our innovative attention mechanism has demonstrated significant effectiveness, leading to performance improvements across various continual learning scenarios.

AGILE proposes a novel mechanism to integrate task attention and task specificity with minimal computation and memory overhead so that the model is scalable to longer task sequences. We believe AGILE proves to be a valuable step in the quest of zero forgetting in Class-IL by tackling the problem of task interference.

Reviewer comment (Novelty):
This method is less innovative and mainly focuses on solving the task interference problem. How to weigh the importance of solving the interference problem or solving the forgetting problem in continual learning?

Author response:
We respectfully disagree with the reviewer's assertion that addressing the task interference problem does not contribute to resolving the forgetting issue in continual learning. In the context of continual learning, task interference and forgetting are not mutually exclusive challenges. Inter-task separation is a significant hurdle in Class-IL learning, particularly in realistic scenarios where task-id is unavailable during inference, leading to catastrophic forgetting [1].

The sequential learning of tasks with limited buffer data tends to blur the boundaries between classes distributed across different tasks. Therefore, minimizing task interference becomes crucial for the model to accurately predict the correct class within the current task, free from the influence of classes from other tasks. This approach aids in mitigating the impact of forgetting in continual learning scenarios.

Reviewer comment (Theory):
The innovation in this paper is that the task-attention module is used to solve the task-id prediction problem, and within-task prediction problem how can it be solved efficiently?

Author response:
AGILE takes a distinctive approach by not explicitly focusing on task-id prediction. Instead, the task projection vectors play a crucial role in transforming feature vectors with task-specific information. This transformation empowers task-specific classifiers to enhance within-task class predictions. Unlike traditional methods that rely on explicit task-id prediction, AGILE leverages these task projection vectors to navigate and capture essential task-specific nuances, contributing to improved model performance within each task.

Furthermore, our novel attention mechanism is a key component designed to mitigate inter-task interference. By incorporating attention mechanisms, AGILE minimizes the impact of information from one task bleeding into another, thus reducing interference between tasks. What sets AGILE apart is its ability to achieve this with minimal memory and computational overhead, making it scalable and efficient for handling longer sequences of tasks in continual learning scenarios. This dual strategy of leveraging task projection vectors and employing a novel attention mechanism positions AGILE as a robust solution for addressing both within-task class prediction and inter-task interference in continual learning.

In section 4.1, we discuss how AGILE facilitates learning a good task-prediction and within-task prediction. We also show how task projection vectors are distributed along the principal components using PCA in Figure 3 (middle). AGILE entails a shared task-attention module and as many lightweight, learnable task projection vectors as the number of tasks. As each task projection vector learns the task-specific transformation, they project samples belonging to the corresponding task differently, resulting in less interference and improved WP and TP in CL.

Reviewer comment (Experiments):
Can this method be used in other continual learning scenarios, such as Task-free scenario?

Author response:
AGILE is well suited for Class-IL and Task -IL continual learning scenarios. AGILE in its current formulation cannot be applied to general continual learning scenario where the task boundaries are blurry in training and inference. We will include it as a limitation in our final revision.

Reviewer comment (Writing):
In section 3.4 only the extension of the classifiers was carried out, what exactly does the network extension refer to?

Author response:
We regret any confusion caused in the manuscript. Network expansion is intended to show how the model expands when a new task is encountered. AGILE adds a new task projection vector and extends the classification head to accommodate the new classes added.

Reviewer comment (Theory):
Using EMA is a critical point in the proposal, and the authors do not mention it too much. EMA can also be used to reduce weight modification, meaning that it can mitigate forgetting with a favorable beta. The authors present it to increase generalization.

Author response:
Following CLS-ER[1], we employ EMA as it functions as a self-ensemble of the intermediate model states that leads to better internal representations. Figure 3 in CLS-ER [1] shows that learning with EMA results in convergence to flatter minima, indicating better generalization [2]. 
While EMA helps in better generalization, the inter task inference problem still prevails and hinders the performance of the model. On the other hand, AGILE effectively reduces inter task interference with our novel task attention mechanism. As can be seen in Table 1, we outperform CLS-ER by significant margins across all datasets in Class-IL and Task-IL settings.

Reviewer comment (Experiments):
Experiments showing evidence that EMA increases generalization could help mitigate the doubts.

Author response:
Following CLS-ER[1], we employ EMA as it functions as a self-ensemble of the intermediate model states that leads to better internal representations. Figure 3 in CLS-ER [1] shows that learning with EMA results in convergence to flatter minima, indicating better generalization [2]. 
While EMA helps in better generalization, the inter task inference problem still prevails and hinders the performance of the model. On the other hand, AGILE effectively reduces inter task interference with our novel task attention mechanism. As can be seen in Table 1, we outperform CLS-ER by significant margins across all datasets in Class-IL and Task-IL settings.

Reviewer comment (Experiments):
Did you have an analysis of the beta value?

Author response:
We have updated the manuscript with the Hyperparameter Tuning results in the Appendix (section D.5.1). The performance of our model reduces as the $\beta$ value is increased. Higher the $\beta$, higher the restriction on the model to preserve old knowledge, thus limiting its ability to learn new information.

Reviewer comment (Presentation):
It is challenging to understand where there are linear layers and where there is soft attention in the proposed methods. The image does not help.

Author response:
Section 3.3 describes the different components in the task attention module in detail. As mentioned there, both feature encoder and feature selector in task attention module is composed of a linear layer followed by Sigmoid activation.

Reviewer comment (Writing):
It could be helpful to decrease the amount of terms, names or losses used in the explanation.

Author response:
We regret any confusion caused due to the explanations provided. We have taken utmost care to include all the components and losses in the model so that we are fully transparent. Figure 1, reflects notations used in Algorithm 1 and Proposed Method Section for proper correspondence. Furthermore, we provide the code to the reviewers to ensure complete transparency.

Reviewer comment (Presentation):
From the Figure, one can assume that there is one Task-Attention Module for each task. However, the Task-Attention Module is shared, no?

Author response:
AGILE uses a single task attention module that is shared across multiple tasks, however it requires as many forward passes through the module as the number of tasks (mentioned in Section 3.4). In Figure 1, we show separate task projection vector and task classifiers for each task whereas the same attention module is shared for each forward pass.

Reviewer comment (Writing):
Didn’t find Definition 1 and 2.

Author response:
Owing to the space limitations, we have provided Definitions 1 and 2 in the Theoretical Insights section in the Appendix.

Reviewer comment (Evaluation):
How much overhead in terms of time is added when adding a Task-Attention Module?

Author response:
Below we provide the relative time taken to train on Seq-CIFAR100 for 5 tasks with a buffer size of 200. As can be seen, even with as many forward passes through the attention module as the number of tasks, the additional computational overhead incurred in AGILE is very minimal.

Method|ER|DER++|AGILE
-|-|-|-
Time|1x|1.5x|1.7x

Reviewer comment (Novelty):
There are some similarities to BiC that you can find interesting.

Author response:
BiC attempts to correct the recency bias in continual learning by adapting the output logits for recent classes with two bias parameters. However, BiC does not entail any task specific learning to reduce task interference in continual learning. Task interference is a significant problem in Class IL that lead to catastrophic forgetting. AGILE, on the other hand, tackles the problem of task interference through the novel task attention mechanism.

Reviewer comment (Experiments):
The reviewer does not remember if BiC works in class or task-incremental, but there have been extensions that work in class-incremental settings.

Author response:
BiC attempts to correct the recency bias in continual learning by adapting the output logits for recent classes with two bias parameters. However, BiC does not entail any task specific learning to reduce task interference in continual learning. Task interference is a significant problem in Class IL that lead to catastrophic forgetting. AGILE, on the other hand, tackles the problem of task interference through the novel task attention mechanism.

Reviewer comment (Experiments):
Do you know how your proposal scales with the memory size? The reviewer has seen methods that scale well (such as DER), but others could be better (like iCarl).

Author response:
Increasing the memory size largely reduces the catastrophic forgetting problem as you have more exemplars from old classes to closely represent old data distribution. This enables the model to learn the decision boundaries better. We expect the margins between rehearsal based approaches to be minimal with large memory size.
We studied AGILE in a much harder scenario of relatively low memory size regimes where the catastrophic forgetting is exacerbated by task interference. For example, in Seq-TinyImageNet with a buffer size of 200, there are hardly 2 examples per class in the buffer for the model to preserve learnt information. Our novel task attention incorporates task specific learning into the model with task projection vectors and outperforms all other baselines by significant margin.

Reviewer comment (Experiments):
Have you tried this approach with a fixed pre-trained model?

Author response:
We have not studied this approach with a fixed pre-trained model. We followed the same settings in which the baselines have been studied by training the model from scratch for different datasets.

Reviewer comment (Experiments):
I feel like the paper is written in a rush. The experiment setup is not mentioned in the main paper. It's not clear how many tasks are used in the sequential data (e.g., Seq-CIFAR100), and what architecture is used. I couldn't find where I can find the information in the main text.

Author response:
Owing to space limitations in the paper, we relocated details regarding the experimental setup and datasets to Appendix D. The decision was based on the assumption that, given the adherence to standard model architecture setups similar to other baselines, these details could be appropriately placed in the appendix. However, in the final revision, we will include a reference to these specifics in the Experimental Results section (Section 4) for better accessibility.

Reviewer comment (Evaluation):
It's not clear why the shared task-attention module improves WP and TP when this module itself also suffers from forgetting.

Author response:
As outlined in the paper, the performance of Class Incremental Learning (Class IL) can be understood as a synergy between TP (task prediction) and WP (within-class prediction). Given the sequential nature of learning in Continual Learning (CL), the model tends to excel in distinguishing between classes within the same task compared to those across different tasks. In AGILE, we emulate this characteristic by employing task projection vectors that transform backbone features into task-specific latent spaces, implicitly encouraging TP. Subsequently, these transformed features undergo processing by task-specific classifiers, addressing WP.

While it is acknowledged that the task-attention module in AGILE is susceptible to forgetting to some extent, our strategy involves capturing task-specific information within the task projection vectors, while task-agnostic information is preserved in the shared task attention module. As shared, generic information is less prone to forgetting, AGILE exhibits improved performance compared to other approaches.

Reviewer comment (Reproducibility):
How does the model make the final class prediction? Does it first predict the task-id using the attention module and make a within-task prediction?

Author response:
AGILE does not rely on explicit task-id prediction. Instead, it employs task-specific projection vectors for the transformation of feature vectors associated with each task. These transformed vectors are subsequently processed through task-specific classifiers, facilitating within-class predictions. The outputs from these classifiers are then concatenated to form the final output vector, encompassing all classes.

Reviewer comment (Theory):
What's the purpose of using the task projection vectors and why is it used to compute both z_s and z_tp?

Author response:
The task projection vector plays a pivotal role in capturing task-specific information. In the encoder module of the shared task attention, features from the backbone are projected into a common latent space. This projection is further transformed using a task projection vector, directing the features to their respective task-specific latent spaces. Subsequently, these transformed features undergo processing through the t_s and t_tp modules to generate the decoded feature vector and facilitate auxiliary task classification learning.

Reviewer comment (Novelty):
Novelty concern. I would like to point out that the idea of leveraging trainable parameters to store task information has been investigated in previous works [*] [**]. L2P has shown its effectiveness in continual learning areas in recent years.

Author response:
While the idea of leveraging trainable parameters has been investigated, we believe the novelty comes from how the idea is implemented. Transformer architectures allow us to somewhat intuitively extend the input tokens with special tokens (say class token) that capture specific information. Learning to prompt and DualPrompt are both transformer based architectures that use special prompt tokens with pre-trained models to then learn task specific information.

However such an extension is not straightforward for CNN architecture. CNN does not have any attention mechanisms or tokens. AGILE leverages parameter isolation to bring in task specificity with little computational or memory overhead. Our novelty comes in emulating such task attention through a shared under complete autoencoder and task projection vectors. These projection vectors then capture task specific information through Our auxiliary task classification loss and pair wise discrepancy loss. Even in CNN architectures, several approaches have tried to capture task specific information through parameter isolation. We compare against such approaches in Figure 2.

In this study, we have explored the integration of our method into the widely used CNN architecture, specifically ResNet18. We have examined various baseline methods within the CNN framework and conducted a comparative analysis, demonstrating that AGILE achieves superior performance due to its innovative task attention mechanism. While AGILE's task attention mechanism utilizes simple undercomplete autoencoders and tensors, devoid of architecture-specific layers like convolutions, we concur with the reviewer's suggestion that AGILE could potentially be generalized to transformers. However, we acknowledge that extending AGILE to different architectures, such as transformers, is a non-trivial task requiring extensive experiments and hyperparameter tuning. Such endeavors go beyond the scope of this rebuttal period. Recognizing the potential benefits of extending AGILE to transformers for enhanced generalizability and comparison with methods such as L2P, DualPrompt, we propose to address this in future work and will update our limitations section accordingly.

Reviewer comment (Experiments):
Lack of a comprehensive comparison. There are many works using prompting (learnable parameters) in continual learning and achieving SOTA performance. I suggest the author conduct a comprehensive comparison with these works.

Author response:
We respectfully disagree with the reviewer's assertion that our comparison with other state-of-the-art (SOTA) approaches is not comprehensive. It is crucial to emphasize that our current focus is confined to CNN architecture. Given the well-established exploration of continual learning within CNN architectures, we have conducted a thorough comparison with various popular and recent rehearsal approaches, as detailed in Table 1 under consistent experimental settings. Additionally, our evaluation extends to the comparison with several methods specifically designed to capture task-specific information, as illustrated in Figure 2.

Reviewer comment (Experiments):
Could the author conduct a comprehensive comparison with CL works using prompting (learnable parameters)?

Author response:
The current scope of our work is primarily centered on CNN architecture. While we have extensively compared our approach with various state-of-the-art rehearsal methods and those focusing on task-specific information capture within the CNN framework, the specific comparison with continual learning works using prompting and learnable parameters falls outside the scope of our study. We acknowledge the importance of this avenue for future exploration and welcome it as a potential direction for further research.

Reviewer comment (Novelty):
The idea of using task-attention or task embedding vector is not quite novel. For example, DyTox [1] also has a task attention module, L2P [2] leverages task-specific prompts.

Author response:
Transformer architecture provides a rather straightforward way to extend self attention mechanism with learnable parameters to capture any specific information. However such attention learning is not intuitive in other architectures, like CNNs.Our novelty is in emulating the task attention mechanism with a bottleneck architecture and encouraging task specific learning with task projection vectors by transforming the encoded latent space output in CNNs. Our auxiliary task classification loss and pair wise discrepancy loss enables the projection vectors to capture task specific information. 

For the scope of this work, we have studied the integration of our method into the widely popular CNN architecture (ResNet18). Several baseline methods are all studied in the CNN architecture, We compare AGILE against these methods under the same experimental setting and show that AGILE can achieve better performance through our novel task attention mechanism.

Reviewer comment (Experiments):
Following the first one, I think the paper misses several recent competitive methods to compare against. For example, I understand both DyTox and L2P are based on transformers. However, if the proposed method AGILE is generalizable enough, it should be compatible with transformer architectures as well, making comparison with more advance methods like DyTox, L2P possible.

Author response:
Since AGILE uses simple under complete auto-encoders and tensors( no architecture specific layer like convolutions ) for our task attention mechanism, we agree with the reviewer that AGILE can be generalized to transformers. However, extending it to other architectures like transformers may not be trivial and it warrants quite some experiments and hyperparameter tuning which is beyond the scope of this rebuttal period. We acknowledge that extending AGILE to transformers could in fact strengthen the method for its generalizability. We leave that as a future work and will update our limitation section to reflect the same.

Reviewer comment (Presentation):
The contents in middle and right subfigures in figure 3 seems missing?

Author response:
We see that the PDF is not loading fully in browser tabs. We request the reviewer to use a PDF viewer application. We will update the PDF in the meantime for proper loading in browser windows.