Paper: Dynamic Adapter Merging for Continual Video Question-Answering Learning (ICLR-2024)

Reviewer comment (Experiments):
Why do you set up N adapters for each domain instead of one?

Author response:
We regret any confusion. Given that adapter design is not the primary focus of our work, we adhere to recent literature  [1,2,3] in incorporating multiple adapters (N) into the pretrained model for multimodal applications. These adapters are typically inserted after the self-attention and feed-forward layers in Transformer-based architectures. The primary objective of employing multiple adapters is to augment the model's capacity with minimal impact on computational cost and parameters. This design choice allows for the dynamic merging of expert models for each test instance with negligible overhead, as detailed in Section C3.3 of the rebuttal.

Reviewer comment (Experiments):
Why do you choose to insert domain-specific adapters after the self-attention and feed-forward layers, respectively? What are the considerations?

Author response:
We regret any confusion. Given that adapter design is not the primary focus of our work, we adhere to recent literature  [1,2,3] in incorporating multiple adapters (N) into the pretrained model for multimodal applications. These adapters are typically inserted after the self-attention and feed-forward layers in Transformer-based architectures. The primary objective of employing multiple adapters is to augment the model's capacity with minimal impact on computational cost and parameters. This design choice allows for the dynamic merging of expert models for each test instance with negligible overhead, as detailed in Section C3.3 of the rebuttal.

Reviewer comment (Reproducibility):
What is the exact structure of the adapter?

Author response:
Following previous works [1,2,3], the adapters in our approach consist of a downsampling and an upsampling linear layer, along with a residual connection. The linear layers are set with an 8x downsample scale to intermediate hidden size and the upsampler maps back to the original dimensionality [1]. 

[1]: Sung, Yi-Lin et al. "Vl-adapter: Parameter-efficient transfer learning for vision-and-language tasks." CVPR 2022.  
[2]: Houlsby, Neil, et al. "Parameter-efficient transfer learning for NLP." ICML 2019.  
[3]: Yang, Antoine, et al. "Zero-shot video question answering via frozen bidirectional language models." NeurIPS 2022.

Reviewer comment (Experiments):
The definition of domain regarding VideoQA is not clear. The authors simply treat different datasets as different domains. This is certainly problematic and prevents detailed model analysis. For example, regarding the question type, all datasets define similar questions except for LSMDC with fill-in-blank setting. Regarding the video type, there are instructional videos (iVQA), social videos (MSVD, MSRVTT, TGIF), movie videos (LSMDC) and activity videos. Regarding video length, all videos are short (3~15s) except for ActivityNet (3 mins). It would be better to experiment with more clarified domains instead of datasets.

Author response:
Our definition of the domain diverges from the conventional definition in the continual learning community. Please refer to the **“Definition of domains” of G1 in General Response** for more detail.

We also report the results on 7 diverse VidQA domains (separated by video type) in **“DIL on more diverse domains” of G2 in General Response**.

Reviewer comment (Evaluation):
While the ‘dynamic merging’ design mitigates the problem of catastrophic forgetting and improves the overall performance as well, it necessitates all the learned adapters for inference. This resembles more on model ensemble versus continual learning a ‘single’ model. It is necessary to show the size of the adapters and analyze the efficiency.

Author response:
The adapters introduced in each domain contribute merely **2.5%** of the pretrained model's parameters (CLIP-L/14 + DeBerTa-V2-XLarge), totaling **30M** parameters. With 10 domains, this results in only a 25% increase in parameters, a reasonable augmentation given DAM's robust performance.

In terms of computational cost, merging adapter parameters incurs just **0.09 GFLOPs** (30M *(2k-1), k=2 in our case), notably lower than the **162 GFLOPs** required by CLIP-L/14 for a single image processing. We appreciate the reviewer's suggestion and will integrate this analysis into the revision.

Reviewer comment (Evaluation):
The authors obtain the upper-bound results by individually finetuning on target datasets. My concern is that this ‘upper-bound’ may not be the actual upper-bound for incremental-learning because of data augmentation. Moreover, the gap between DAM' results and this upper-bound results is too small to show that there is need for future efforts as a novel task setting. The authors need to find a more convincing upper-bound or just mention the current one as a reference.

Author response:
We conducted both individual finetuning (Ind-FT) and multi-task finetuning (MLT-FT), the latter involving joint training on all datasets. As indicated in the table, MLT-FT yields a notable 1.8% enhancement in MSVD-QA, while maintaining a comparable average accuracy to Ind-FT (**52.5% vs. 52.6%**).

In the realm of continual learning, multi-task finetuning is conventionally considered an upper bound. We apologize for the confusion and will replace the upper bound with MLT-FT in the revised manuscript. 

|Method|iVQA|MSVD|MSR-VTT|LSMDC|ActivityNet|TGIF|Avg.|
|-|-|-|-|-|-|-|-|
|Ind-FT|39.8|54.8|46.7|63.0|42.4|68.0|52.5|
|MLT-FT|39.7|56.6|46.7|62.9|42.2|67.8|52.6|

Reviewer comment (Experiments):
According to the task setting, providing more analyses /comparisons on an OOD setting (aside from Fig.3(b)) would make the experiments more sound.

Author response:
We further experiment on 7 diverse domains as described in G2 of the General Response. We continually train the proposed DAM on the first 3 domains and evaluate the trained model on the other 3 domains. From the table, DAM outperforms S-Prompts consistently in all the domains, attributed to the larger capacities of adapters and the robustness of dynamic merging.

||In Distribution|||Out-of-Distribution|||
|-|-|-|-|-|-|-|
|Method|MSVD|iVQA|LSMDC|ActivityNet|AGQA|Env-QA|TrafficQA|
|Upper-Bound|56.6|39.7|63.0|42.2|63.4|32.3|67.8|
|DAM|54.4|39.4|63.0|20.3|25.4|5.6|18.9|
|S-Prompts|47.7|34.0|57.4|18.7|22.9|5.5|15.1|

Reviewer comment (Evaluation):
Why is the performance on ActivityNet not as good as on other datasets?

Author response:
We thank the reviewer for pointing out the room for future work. There are still unaddressed issues that lead to a relatively large accuracy drop on some specific datasets, i.e. MSRVTT (**-4.5%**) and ActivityNet (**-6.1%**). Other future work includes experimenting with **many more domains (or datasets) like 100**, and extending our method to extremely large models (i.e. > **10B** parameters).

Regarding the ActivityNet dataset, we analyze the performance drop might be due to the high similarity of its question type with the iVQA dataset, while the videos are quite different. This inconsistency may lead to confusion within our proposed model.

Reviewer comment (Writing):
In Sec. 3.2, what specific model does ‘f’ refer to?

Author response:
We refer to the original pretrained model as “f”, i.e. the model used for zero-shot inference (FrozenBiLM in our case) except that we extract the hidden states from the 4th last layer as the output. We thank the reviewer’s suggestion and will add a detailed description in the supplementary materials.

Reviewer comment (Experiments):
Limited Dataset Diversity: The article's experimental use of six datasets with relatively small differences between them, especially MSVD and MSR-VTT, raises concerns about the method's domain adaptation and continual learning capabilities. The use of internet-sourced videos in the datasets does not fully explore the potential challenges posed by more diverse datasets, such as those collected in virtual environments (e.g., Env-QA[1]), traffic scenarios (e.g., TrafficQA[2]), or indoor human activities (e.g., AGQA[3]). What’s more, the out-of-date issue proposed in Figure 1 hasn’t been evaluated, also.

Author response:
> **C2.1 Limited Dataset Diversity: The article uses 6 datasets with relatively small differences between them, especially MSVD and MSR-VTT.**

We thank the reviewer for the valuable suggestion. We believe the confusion is caused by the question of **whether domains should be orthogonal**.

We elaborate on our point in “**Dataset Diversity” of  G1 in the General Response** above, please refer to that section for more detail. We also report the results with more diverse datasets in **G2 in the General Response**.  

---

> **C2.2 The out-of-date issue proposed in Fig. 1 hasn’t been evaluated.**

We further evaluate the CIL scenario to mimic the evaluation of the out-of-date issue. The old model may not be able to answer questions in new tasks as they never see the classes in the new tasks before, which is similar to the example in Fig. 1 that a VidModel trained in 2021 may struggle with questions about the 2023 movie “Barbie”. Our proposed DAM method archives significant improvements compared to the second-best approach on CIL settings.

We refer the reviewer to **G2 (Generalization) in the General Response** for more details.

---

> **C2.3 The use of internet-sourced videos in the datasets does not fully explore the potential challenges posed by more diverse datasets, such as those collected in virtual environments (Env-QA), traffic scenarios (TrafficQA), or indoor human activities (AGQA).**

Thanks for the insightful suggestion. We further experiment with **7 diverse video datasets** including all the datasets that the reviewer mentioned. Results show that our DAM method achieves even less forgetting on more diverse datasets. 

Please refer to “**DIL on more diverse domains” of G2 in General Response** with more detail.

Reviewer comment (Experiments):
While the article demonstrates the effectiveness of the adapter and router, their simple design might not generalize well to more challenging datasets. The reviewer has doubts about their applicability in more complex scenarios.

Author response:
> **C2.4 Their simple design might not generalize well to more challenging datasets. The reviewer has doubts about this.**

In **G2 Generalization of the General Response**, we showcase DAM’s generalization ability by experimenting on DIL with more diverse domains, CIL, and TIL scenarios. In Sec 4.5 and Tab. 4 in the main paper, we show DAM can be applied to the visual (image)-QA task using a different large model BLIP2.

All the experimental results support the conclusion that DAM can generalize well to challenging datasets, scenarios, and even more tasks and models. This is because we do not need complex design on top of pretrained large models as they already provide enough generalizability.

Reviewer comment (Experiments):
The article does not provide a fair comparison with backbone models under few-shot learning setting. A direct comparison between in-context learning using FrozenBiLM and the proposed approach could offer a more comprehensive evaluation.

Author response:
> **C2.5 The article does not provide a fair comparison with few-shot learning. A direct comparison between in-context learning using FrozenBiLM and the proposed approach could offer a more comprehensive evaluation.**

We appreciate that the reviewer pointed out another interesting direction (in-context learning, ICL) to address the continual learning problem. We further experiment with the one-shot in-context learning using FrozenBiLM and report the results below. The proposed DAM outperforms one-shot FrozenBiLM by **29.3%** in average accuracy. The inferior performance of one-shot ICL is because LLM with at least 6.7B parameters **begin to have** in-context learning ability on multimodal tasks (Koh et al. 2023).

We will add this comparison and mention that ICL with LLM could be a potential direction for continual learning in the revised manuscript.

|Method|iVQA|MSVD|MSR-VTT|LSMDC|ActivityNet|TGIF|Avg.|
|-|-|-|-|-|-|-|-|
|Zero-shot FrozenBiLM|26.8|33.0|15.0|51.5|25.5|41.9|32.3|
|One-shot ICL FrozenBiLM|17.9|22.5|9.7|34.5|17.8|23.1|20.9|
|DAM(Ours)|39.1|53.6|42.2|63.0|36.3|66.8|50.2|

Koh, Jing Yu et al. "Grounding Language Models to Images for Multimodal Inputs and Outputs." (2023).

Reviewer comment (Experiments):
The article does not provide sufficient evidence of severe catastrophic forgetting in current large models.

Author response:
> **C2.6 The article does not provide sufficient evidence of severe catastrophic forgetting in current large models.**

As shown below, sequentially finetuning (Seq-FT) the large model (FrozenBiLM) without any continual learning technique leads to a 12.7% accuracy drop, which we believe is sufficient to show severe catastrophic forgetting. Thanks for the suggestion and we will add this baseline in Table 1 in the revision.

|Method|iVQA|MSVD|MSR-VTT|LSMDC|ActivityNet|TGIF|Avg.|
|-|-|-|-|-|-|-|-|
|UpperBound|39.8|54.8|46.7|63.0|42.4|68.0|52.5|
|Seq-FT|28.4|36.0|23.7|52.1|31.2|67.6|39.8|

Reviewer comment (Novelty):
It is worth discussing whether there are unique challenges related to continual learning in the domain of VideoQA.

Author response:
> **C2.7 It is worth discussing whether there are unique challenges related to continual learning in the domain of VideoQA.**

As mentioned on Page 2 (below Fig. 1) in the main draft, in contrast to unimodal tasks like image classification, VidQA involves both video and text inputs, necessitating the model to jointly reason on both modalities. Within VidQA domains, question types may exhibit significant similarity (e.g., when, how, where), while the corresponding answers can vary considerably. The juxtaposition of a shared input space (video+question) and a diverse output space adds complexity to continual VidQA. Consequently, router-based continual learning methods may face challenges in accurately predicting domain identities, leading to inferior answer prediction accuracies.

Our proposed method, DAM, effectively mitigates the issue of inaccurate routers by dynamically merging domain adapters (refer to Sec. 4.4). We appreciate the suggestion and will incorporate this discussion in the revision.