$SMoA: Soft Mixture-of-Adapters for Sample-Adaptive Knowledge Transfer in Lifelong Sequence Generation$
Authors: JunBeom Lee


Reviewer paper summaries: The core content of this paper revolves around a novel method for Lifelong Sequence Generation (LSG), named Soft Mixture-of-Adapters (SMoA). LSG is a type of lifelong machine learning approach aimed at enabling language models to continuously learn a series of sequence generation tasks. The main objective of this method is to retain knowledge from previously learned tasks to prevent forgetting and to achieve selective knowledge transfer when learning new tasks.

In a lifelong learning environment, the model needs to continuously acquire new tasks while retaining the knowledge from prior tasks and be able to selectively transfer this knowledge to facilitate learning new tasks. Existing methods effectively retain knowledge by training specific adapters for each task, but face challenges in maintaining the integrity of retained knowledge during inter-task knowledge transfer.

The authors propose a novel method that allows the language model to adaptively transfer knowledge to new tasks by soft-mixing the outputs of task-specific adapters. Inspired by the Soft Mixture-of-Experts, this method trains an SMoA router to enable the language model to blend appropriate knowledge based on the input sample.

The authors conducted extensive experiments on two different LSG benchmark datasets, demonstrating the performance advantages of the SMoA method compared to existing methods.

The main contributions of this paper are:

- Proposing SMoA, a new LSG method capable of achieving both effective knowledge retention and sample-adaptive knowledge transfer.
- To the best of the authors' knowledge, they are the first to explore a method for addressing LSG without accessing task IDs, by selecting and transferring knowledge at the sample level.
- Demonstrating the effectiveness of this method compared to existing methods through extensive experiments on two different LSG benchmark datasets.
Summary of strengths: Relative advantages of the SMoA method:

1. SMoA can adaptively select and transfer knowledge based on the characteristics of each input sample, rather than relying on a fixed task ID, which makes the model more flexible and adaptable.
2. By training task-specific adapters for each task and freezing the original Transformer parameters during new task learning, SMoA can effectively retain knowledge from previous tasks.
3. SMoA utilizes soft-mixed adapter outputs, allowing the model to share and transfer knowledge across different tasks, potentially improving efficiency in learning new tasks.
4. SMoA does not require a task ID during the inference phase, increasing convenience and applicability in practical applications.
5. SMoA manages model size by evaluating and removing replaceable task-specific adapters, thus avoiding unlimited growth of the model as tasks increase.


Advantages of the experiments:

1. SMoA was compared against the following baseline methods: Finetune, LAMOL, AdapterCL, ACM, and DMEA.
2. The average LSG performance of SMoA was analyzed across INTENT, DST, NLG (SER), and NLG (BLEU) tasks, and performance change graphs as the number of sequentially learned tasks increased were plotted.
Summary of weaknesses: 1. Like other lifelong learning methods, SMoA relies on pseudo-data replay techniques to handle unknown task samples, which might limit the model's performance in some practical applications.

2. Although SMoA attempts to manage model size, as the number of learning tasks increases, the model size might still become large, necessitating further research into more effective model compression or pruning methods.

3. As tasks increase, SMoA might retain a large number of adapters, potentially increasing model complexity. Further optimization is needed to reduce unnecessary adapters.

4. While SMoA transfers knowledge at the sample level, its generalization ability, especially between completely different tasks or domains, requires further research and validation.

5. The training and inference of SMoA may require substantial computational resources, particularly when dealing with large-scale datasets or complex tasks.
Comments, suggestions and typos: 1. For line 097: Firstly, adding an adapter for each task significantly increases the number of parameters and computation. Additionally, the iterative selection process for each new task further increases the computational load. Could the authors provide charts or tables regarding the computational resources required? Moreover, I have a concern about this selection method: if task k could better assist task k+n, this cannot be verified under the current setup.

2. For line 287: First, is there a Router for each Transformer block, or is there only one Router globally? Second, are the weights generated by the Router scalar or vector for each task? My current understanding of the formula is that there is only one scalar weight for each type of task k, which implies that adapters of the same task but different layers have the same function and importance. This is almost impossible according to many studies on Transformer layers.

3. For line 320: Since all types of data are converted into QA templates, what will happen if we directly gather all samples together into a training set to see the performance? Since all inputs are in QA form, it seems like the same task (perhaps just different domains in Q and A).

4. In Table 4, only the average LSG performance is listed. I believe that since LSG focuses on the forgetting ability regarding old tasks when learning new tasks, it would be better to list the performance on each specific task rather than just an overall average performance.

5. Still regarding Table 4, it appears that in many settings, the SMoA method proposed by the authors does not show significant improvement and in some cases, even performs worse than previous methods.

6. Concerning Figures 5 and 6, the same issue persists: the SMoA method proposed by the authors does not show significant differences compared to previous methods.

7. For the ablation study, I believe it is incomplete. SMoA has many intermediate steps and modules, which should be subjected to more detailed experiments. Additionally, pseudo-data replay has shown extremely important significance in the ablation study, raising concerns about the generalizability and practicality of this method.

8. Overall, the SMoA method, which increases parameters (many adapters) and uses data augmentation (pseudo-data replay), does not show significant improvement. Furthermore, the experiments lack detailed results (only average LSG scores are provided without detailed results and analysis for each previous task), and the ablation study lacks components and further explanations.
Overall assessment scores: 2