Paper: Towards Robust Multi-Modal Reasoning via Model Selection (ICLR-2024)

Reviewer comment (Experiments):
The significance of the problem is not well illustrated. While the paper has shown the existence of model selection problem, I am not aware of how important this problem is. There can be lots of problems in multi-modal reasoning, but some may not be of much value. Specifically, is there a huge gap between an oracle model selection and a random selection? Is there a naïve solution that can approach the oracle performance? The authors are suggested to add these preliminary experiments to illustrate the significance of the problem.

Author response:
The significance of the problem we introduced to the community and studied in this manuscript can be identified by the performance gap between oracle and our current SOTA method, as illustrated in Section 4.2. In detail, 

1. In our constructed MS-GQA, every sample has at least one model selection choice that would output the correct answer. Therefore, the theoretical upper bound for the entire test set is 100% SER (Successful Execution Rate, where a successful execution in our context indicates that the agent could output the correct answer by executing the selected models);
2. As shown in Table 1, a model selector employing random model selection can only achieve approximately 56% SER, while a rigid model selector like VisProg also achieves only 59%, which is not very satisfactory;
3. Our method (M3) achieves a SER of 68.7%, significantly surpassing other baselines but still maintaining a gap from the oracle model selector.

In summary, a good, robust model selector can significantly improve SER compared to simple strategies (~56% to 68.7%), but there is still considerable room for improvement compared to the Oracle model selector. This indicates the value of the current research direction.

Reviewer comment (Experiments):
Lack of ablation study on model inputs. The paper claims that other model selection methods do not take subtask dependency into account. However, the ablation study does not show the effect of using subtask dependency as input. More broadly, because the framework uses various inputs, including multi-modal inputs, node embedding, subtask dependency, a more extensive ablation can be done by removing each component successively. This will show the importance of each component.

Author response:
As discussed in Section 3.2, M3 unifies the key components that emerged in the unexamined scenario of multi-step multi-modal reasoning procedure, namely multi-modal inputs, model (node) embedding, and subtask dependency (see Figure 3).

Below we explain why removing any specific component would make our M3 framework ineffective:

1. **Multi-modal inputs and model embeddings are irremovable.** Multi-modal inputs and model (node) embeddings are fundamental and basic inputs for model selection, which essentially involves finding the most suitable model for each sample;
2. **Subtask dependency is also irremovable.** The representation of subtask dependency through edges in the computation graph is an indispensable component of the computation graph;

In our initial manuscript, we extensively explored the impact of the replaceable components in M3. Specifically, in Appendix D, the ablation study encompasses experiments that span the selection of multi-modal feature extractors to the determination of the objective function.

Besides, we have separately studied the effect of subtask dependency in the conventional model selection methods (namely NCF and MetaGL). It again reveals the uniqueness of our methodology by taking all components as a complete unit.

- Section 4.4 elucidates the advantage of M3 in leveraging subtask dependency information compared to the NCF++ and MetaGL++.

Reviewer comment (Writing):
Presentation should be improved. I find some of the notations are used without introducing first, which hinders a smooth reading. Especially in Section 3.2: Page 4, bottom line, $\phi \circ \psi$ is used but they have not been introduced. Page 5, Section 3.2.1, Para 1, $\psi := [\psi_1, \psi_2]$ is used without explanation.

Author response:
Thanks for bringing up the concerns about our notations. We've made necessary adjustments in the latest version of the manuscript (in blue) to address those issues.

Reviewer comment (Evaluation):
Why choose SER as the metric rather than accuracy? From Figure 1, I thought a wrong model selection will mainly cause the system to give false answer. But the main metric adopted is to measure the successful execution rate. I think there is a difference between successful execution and final accuracy.

Author response:
We find accuracy to be too general and prefer using SER as a metric, as it attributes each execution result solely to model selection. This allows us to focus on the impact of model selection methods on the overall system.

It is important to note that, by default in MS-GQA, cases where model selection leads to execution interruptions are not considered in our manuscript. These samples were removed during preprocessing, and we assume the agent will provide an answer for every model selection choice, which may result in either a successful or unsuccessful execution.

Reviewer comment (Experiments):
What is the difficulty level in tab2? How do the authors define the difficulty level?

Author response:
In the caption of Table 2, we provided an explanation for the concept of “difficulty level”, which is further detailed as follows:  

1. In our experiments in Section 4.3, we categorized all questions into five levels (1, 2, 3, 4, 5). 
2. The determination of question difficulty was based on the number of candidate models capable of solving each question. 
3. After sorting all questions by difficulty, each level contains the corresponding 20% of samples, with lower level numbers representing questions with the least difficulty among all samples. For example, level 1 corresponds to the 20% of samples with the lowest difficulty in the entire set.

Reviewer comment (Theory):
The method uses a heuristic process to perform selection which the capacity is relying on the pre-trained models themselves.

Author response:
We are uncertain about the concept of “**heuristic process”** and “**pre-trained models”**  mentioned by the reviewer. It would be very appreciated if the reviewer could elaborate on it a bit.  A more detailed explanation would greatly assist us in addressing any potential misunderstandings and further refining our work accordingly.

Besides, we would like to clarify that our study isolates the effect of multi-step reasoning quality—-caused by the LLMs—-and focuses on the unexplored model selection scenario for the given reasoning paths. We aim to utilize model selection for each sub-task to further improve the robustness of the multi-modal agent system.

Reviewer comment (Experiments):
How about the generalization capacity for the zero-shot tasks?

Author response:
As highlighted in [here](https://openreview.net/forum?id=KTf4DGAzus&noteId=wuDiUQJVxC), creating a novel model selection benchmark is a non-trivial task. Because constrained by the resources, we paid our efforts only on collecting an MS-GQA dataset and haven’t tested the generalization capacity for other zero-shot tasks. We acknowledge the importance of assessing generalization across diverse datasets and plan to incorporate widely recognized datasets in our future research to thoroughly investigate M3's performance in handling zero-shot tasks.

Reviewer comment (Novelty):
The primary concern is that model selection is a small part of multi-modal reasoning. It remains to be seen whether it is important for the entire task and how it can benefit real-world applications. The selection method proposed in this paper involves complex proxy training and may need to be more universally applicable or scalable for different reasoning tasks.

Author response:
> **W1.1:** *The primary concern ... can benefit real-world applications.*
> 
We summarize and elucidate the value of exploring model selection in multi-modal reasoning scenarios from four specific perspectives:

1. In Section 2.2, we demonstrate the proven value of model selection in other domains (e.g. outlier detection, time-series prediction, and graph learning);
2. Table 1 provides experimental evidence, showing that model selection methods successful in other domains do not perform well in multi-modal reasoning scenarios under the agent. Additionally, our analysis in Section 4.3 further indicates that effectively leveraging subtask dependency, unique to multi-step reasoning, significantly improves the successful execution rate of the overall system;
3. Existing methods still have a significant gap from the oracle model selector, see Appendix E.1 for more details;
4. A newly added Appendix E.4 and our joint reply discusses the necessity for a reliable model selector in current multi-modal agents. This enhances the overall robustness of the agent system, facilitating their application in real-world domains like finance, education, robotics, and embodied intelligence.

> **W1.2:** *The selection method proposed in this paper involves complex proxy training*
> 
Certainly, proxy training introduces additional overhead, but:
1. As indicated in our analysis in Section 4.3, training-free methods exhibit a noticeable performance gap compared to training-based approaches.
2. Still in Section 4.3, the SOTA model selection methods in other domains, though being training-based, fall short of addressing the multi-modal reasoning scenarios.
3. As an initial attempt in the field, our proposed framework (M3) incorporates all necessary components required to model the dependent reasoning process for model selection. Our simple design takes less than an hour of training. Section 4.5 further clarifies the efficiency of our method during test-time.
4. As our next step, we are working on reducing the overhead of proxy training from the aspect of ability assignment and in-context learning. This exploration is beyond the scope of our current manuscript.

> **W1.3:** *may need to be more universally applicable or scalable for different reasoning tasks*
> 
> **W4.2:** *It would be more convincing if experiments were conducted on more reasoning tasks, as done in VisProg and HuggingGPT.*
> 
As the very initial exploration of the community, we established a benchmark and introduced MS-GQA, a significant dataset derived from GQA.

- We assert MS-GQA's representativeness for model selection for multi-modal reasoning' given GQA's wide recognition in the Visual Question Answering community [3,4,5] and its use in testing multi-modal agents like VisProg [1] and ViperGPT [2].
- Additionally, to validate our framework on diverse test data, we separately divided MS-GQA into five sub-datasets with different question structures and five sub-datasets with varying difficulty levels. (see Sec 4.3)
- Furthermore, creating a novel model selection dataset is a non-trivial task involving several steps:
    1. finding a publicly accessible multi-modal reasoning dataset covering various reasoning types; 
    2. identifying the dataset's subtask types (e.g., object detection, segmentation, and visual question answering); 
    3. deploying popular candidate models for different subtask types; 
    4. utilizing existing multi-modal agents to execute the dataset with diverse model selection choices; 
    5. post-processing the results, discarding or revising samples with unsuccessful executions due to engineering problems or errors in the LLM-generated task execution graph. 
    
    This entire process is laborious and time-consuming, not to mention the significant costs associated with invoking LLM (GPT-3.5-turbo).
    

While we recognize the limitation of conducting experiments solely on MS-GQA, it is important to note that: 1) MS-GQA serves as a representative dataset, 2) our method consistently performs well across various test scenarios, highlighting its robustness, and 3) developing a new model selection dataset presents substantial challenges. Due to these factors, we cannot provide experimental results for different multi-modal reasoning tasks at the moment. 

We are planning to include other widely recognized datasets in future research to strengthen the persuasiveness of our findings, considering our resource limitations.

[1] Gupta, Tanmay, and Aniruddha Kembhavi. "Visual programming: Compositional visual reasoning without training." *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2023.

[2] Surís, Dídac, Sachit Menon, and Carl Vondrick. "Vipergpt: Visual inference via python execution for reasoning." *arXiv preprint arXiv:2303.08128* (2023).

[3] Li, Junnan, et al. "Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models." *arXiv preprint arXiv:2301.12597* (2023).

[4] Tiong, Anthony Meng Huat, et al. "Plug-and-play vqa: Zero-shot vqa by conjoining large pretrained models with zero training." *arXiv preprint arXiv:2210.08773* (2022).

[5] Tan, Hao, and Mohit Bansal. "Lxmert: Learning cross-modality encoder representations from transformers." *arXiv preprint arXiv:1908.07490* (2019).

Reviewer comment (Reproducibility):
Lack of reproducibility: The paper must include crucial details, such as the LLM used. The constructed MS-GQA dataset is not yet open-sourced, and the paper fails to provide even a single example of the dataset. Furthermore, the paper does not demonstrate how the proposed methods can improve various reasoning tasks and whether they can be applied to open-source models like LLaMA.

Author response:
> **W2.4:** *whether they can be applied to open-source models like LLaMA*
> 
> **W2.1:** *The paper must include crucial details, such as the LLM used*
> 
In a multi-modal agent scenario, the LLM's role is to decompose GQA questions into AI subtasks and determine their execution order. In our case, we utilize the LLM to construct the MS-GQA dataset, and our method design and experimental results are irrelevant to the LLM. Note that,
- Our study isolates the effect of multi-step reasoning quality—-caused by the LLMs—-and focuses on the unexplored model selection scenario for the given reasoning paths.
- As explained in our Section 3, M3 relies on the generated reasoning path from a LLM, and performs the model selection along the path per subtask node.
- We have elaborated the LLM details in the Appendix A (of the original submission), where we use **GPT-3.5 turbo** as our default LLM.

We agree with the reviewer that the significance of model selection would be more pronounced when the quality of the reasoning path cannot be guaranteed, e.g., with open-source models. We leave this part for future work.

> **W2.2:** *The constructed MS-GQA dataset is not yet open-sourced, and the paper fails to provide even a single example of the dataset*
> 
We have open-sourced our constructed dataset and source code in the supplementary materials.

Statistical information about MS-GQA can be found in Sec 4.1 and Appendix A. Additionally, we have updated the introduction of dataset files and provided examples of samples in Appendix A.2 and A.3. The dataset files are also available in the supplementary materials.

We also commit to promptly releasing the code and dataset as indicated in our abstract.

> **W2.3:** *the paper does not demonstrate how the proposed methods can improve various reasoning tasks*
>
To showcase the robustness of our method in handling different reasoning tasks:
1. We categorized the existing MS-GQA dataset into five subgroups/tasks (Query, Choose, Compare, Verify, and Logical) based on question structures, each with distinct reasoning characteristics (see Appendix A.3 for examples).
2. We also determined the question difficulty by considering the number of candidate models capable of solving each question. Questions solvable by any model selection choices were deemed *the easiest*, while those with limited viable choices were considered *challenging*. Then, questions were categorized by their difficulty levels.

In both scenarios, the results in Tables 1 and 2, along with the analysis in Section 4.3, demonstrate the robust performance of our method across various reasoning tasks, surpassing other baselines.

Reviewer comment (Experiments):
The implementation of the baselines is weak: The original HuggingGPT paper dynamically selected models for tasks through in-context task-model assignment, yet this paper describes it as only using "external metrics" and implements it as "choosing the most recently published one for each subtask", which is misleading and causes unfair comparisons.

Author response:
> **W3:** The implementation of the baselines is weak: The original HuggingGPT paper dynamically selected models for tasks through in-context task-model assignment, yet this paper describes it as only using "external metrics" and implements it as "choosing the most recently published one for each subtask", which is misleading and causes unfair comparisons.
> 

Thank you for highlighting the issue in our HuggingGPT baseline implementation. We agree with the reviewer that the original HuggingGPT uses in-context learning for model assignment, rather than directly selecting models based on “external metrics”. We have made revisions in blue in the new version.

Moreover, we conducted experiments on the "in-context task-model assignment" model selection strategy of HuggingGPT, and displayed the partial results in the table below: 
- The selected model consistently remained the same despite changes in question descriptions or structures.
- The consistency indicates that leveraging the in-context learning capability of LLM currently falls short of achieving genuine and effective dynamic model selection.
|  | model 1: vilt-b32-finetuned-vqa | model 2: git-base-textvqa | model 3: blip-vqa-base  | model 4: blip2-opt-2.7b | model 5: blip2-flan-t5-xl | model 6: instructblip-vicuna-7b | model 7: instructblip-flan-t5-xl |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Query | 100% | 0% | 0% | 0% | 0% | 0% | 0% |
| Choose | 100% | 0% | 0% | 0% | 0% | 0% | 0% |
| Compare | 100% | 0% | 0% | 0% | 0% | 0% | 0% |
| Logical | 100% | 0% | 0% | 0% | 0% | 0% | 0% |
| Verify | 100% | 0% | 0% | 0% | 0% | 0% | 0% |

The corresponding experimental details and observations are outlined below:

- Settings. We randomly selected 100 questions from GQA, and constructed prompts following HuggingGPT's description. 100 questions cover 5 different question structures, and each question structure typically has distinct reasoning characteristics. The corresponding experimental code is included in the supplementary materials.
- Observations. All questions, irrespective of task type and question itself, are assigned to the first model (vilt-b32-finetuned-vqa) with five 100% values in the first model column.
- Comment. The experiment above was conducted only in a one-step scenario, where one model is selected for a single task. Though simple, we believe this case is sufficient to state the fact that “in-context task-model assignment” may not be particularly effective in a multi-step setting.

Reviewer comment (Experiments):
The experiments could be more convincing: This paper only reports results on a newly created MS-GQA dataset. Even though it's compared to simple baselines that do not require training, like directly using the latest or best models (as shown in Table 2), the proposed M^3 method does not show consistent improvements and may even significantly degrade performance. It would be more convincing if experiments were conducted on more reasoning tasks, as done in VisProg and HuggingGPT.

Author response:
> **W4.1:** *Even though it's compared to simple baselines that do not require training, like directly using the latest or best models (as shown in Table 2), the proposed M^3 method does not show consistent improvements and may even significantly degrade performance.*
> 

Training-free methods may perform well on one sub-test set (If their choice happens to align with the preference) but poorly on others, while the M3 framework consistently performs well across all sub-test sets, as mentioned in our paper Section 4.3:

- First of all, preferences for models vary across different sub-test sets and subtask types.
- The training-free baseline tends to fixate on a specific model for each subtask type, meaning some training-free methods may perform well on one sub-test set (e.g., VisProg performs best in the Compare sub-test set) but poorly on others (e.g., VisProg performs worst in the Query and Choose sub-test sets).
- Tables 1 and 2 demonstrate that M3 performs best on many sub-test sets and still exhibits strong performance on the remaining sets, showing its robustness.

The experimental results in Section 4.4 also demonstrate a notable improvement in all training-based methods with an increase in the number of samples. Therefore, although M3 may **occasionally** perform less effectively than training-free methods on the current scale of the MS-GQA dataset, we have reason to believe that M3 will exhibit better performance on larger datasets in the future.

> **W4.2:** *It would be more convincing if experiments were conducted on more reasoning tasks, as done in VisProg and HuggingGPT.*
> 
As the very initial exploration of the community, we established a benchmark and introduced MS-GQA, a significant dataset derived from GQA.

- We assert MS-GQA's representativeness for model selection for multi-modal reasoning' given GQA's wide recognition in the Visual Question Answering community [3,4,5] and its use in testing multi-modal agents like VisProg [1] and ViperGPT [2].
- Additionally, to validate our framework on diverse test data, we separately divided MS-GQA into five sub-datasets with different question structures and five sub-datasets with varying difficulty levels. (see Sec 4.3)
- Furthermore, creating a novel model selection dataset is a non-trivial task involving several steps:
    1. finding a publicly accessible multi-modal reasoning dataset covering various reasoning types; 
    2. identifying the dataset's subtask types (e.g., object detection, segmentation, and visual question answering); 
    3. deploying popular candidate models for different subtask types; 
    4. utilizing existing multi-modal agents to execute the dataset with diverse model selection choices; 
    5. post-processing the results, discarding or revising samples with unsuccessful executions due to engineering problems or errors in the LLM-generated task execution graph. 
    
    This entire process is laborious and time-consuming, not to mention the significant costs associated with invoking LLM (GPT-3.5-turbo).
    

While we recognize the limitation of conducting experiments solely on MS-GQA, it is important to note that: 1) MS-GQA serves as a representative dataset, 2) our method consistently performs well across various test scenarios, highlighting its robustness, and 3) developing a new model selection dataset presents substantial challenges. Due to these factors, we cannot provide experimental results for different multi-modal reasoning tasks at the moment. 

We are planning to include other widely recognized datasets in future research to strengthen the persuasiveness of our findings, considering our resource limitations.

[1] Gupta, Tanmay, and Aniruddha Kembhavi. "Visual programming: Compositional visual reasoning without training." *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2023.

[2] Surís, Dídac, Sachit Menon, and Carl Vondrick. "Vipergpt: Visual inference via python execution for reasoning." *arXiv preprint arXiv:2303.08128* (2023).

[3] Li, Junnan, et al. "Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models." *arXiv preprint arXiv:2301.12597* (2023).

[4] Tiong, Anthony Meng Huat, et al. "Plug-and-play vqa: Zero-shot vqa by conjoining large pretrained models with zero training." *arXiv preprint arXiv:2210.08773* (2022).

[5] Tan, Hao, and Mohit Bansal. "Lxmert: Learning cross-modality encoder representations from transformers." *arXiv preprint arXiv:1908.07490* (2019).