MMDT: Decoding the Trustworthiness and Safety of Multimodal Foundation Models

Authors: Chejian Xu, Jiawei Zhang, Zhaorun Chen, Chulin Xie, Mintong Kang, Yujin Potter, Zhun Wang, Zhuowen Yuan, Alexander Xiong, Zidi Xiong, Chenhui Zhang, Lingzhi Yuan, Yi Zeng, Peiyang Xu, Chengquan Guo, Andy Zhou, Jeffrey Ziwei Tan, Xuandong Zhao, Francesco Pinto, Zhen Xiang, Yu Gai, Zinan Lin, Dan Hendrycks, Bo Li, Dawn Song
Primary area: datasets and benchmarks
Keywords: Trustworthiness, multimodal foundation models, safety, hallucination, OOD generalization, bias/fairness, privacy, adversarial robustness

Abstract:
Multimodal foundation models (MMFMs) play a crucial role in various applications, including autonomous driving, healthcare, and virtual assistants. However, several studies have revealed vulnerabilities in these models, such as generating unsafe content by text-to-image models. Existing benchmarks on multimodal models either predominantly assess the helpfulness of these models, or only focus on limited perspectives such as fairness and privacy. In this paper, we present the first unified platform, MMDT (Multimodal DecodingTrust), designed to provide a comprehensive safety and trustworthiness evaluation for MMFMs. Our platform assesses models from multiple perspectives, including safety, hallucination, fairness/bias, privacy, adversarial robustness, and out-of-distribution (OOD) generalization. We have designed various evaluation scenarios and red teaming algorithms under different tasks for each perspective to generate challenging data, forming a high-quality benchmark. We evaluate a range of multimodal models using MMDT, and our findings reveal a series of vulnerabilities and areas for improvement across these perspectives. This work introduces the first comprehensive and unique safety and trustworthiness evaluation platform for MMFMs, paving the way for developing safer and more reliable MMFMs and systems. Our platform and benchmark are available at https://mmdecodingtrust.github.io/.

Decision: accept (poster)
Reviewer scores: 8, 8, 6, 6

Reviews and discussion:
<thread>
creation date:
1729991616431

thread_type:
official_review

<thread_object>
creator: reviewer_mkar

<content>
summary:
The paper introduces Multimodal DecodingTrust (MMDT) to provide a comprehensive evaluation of safety and trustworthiness across multiple dimensions: safety, hallucination, fairness, privacy, adversarial robustness, and out-of-distribution generalization. MMDT includes diverse evaluation scenarios and red teaming algorithms, creating challenging data for each evaluation task to rigorously assess these perspectives.

strengths:
The paper provides a comprehensive evaluation framework to evaluate Multimodal Foundation Models (MMFMs).

The evaluation using MMDT reveals specific vulnerabilities and areas of improvement in multimodal models, providing actionable insights for researchers and developers.

weaknesses:
As a benchmark paper, I did not find many unique issues of MMFM compared to the general LLM or foundation models. I appreciate the large workload for benchmarking, but this is more like an experimental report than a solid research paper. For example, there are limited analytical components or theoretical insights.

questions:
Can you emphasize some unique discoveries from your evaluation process? For example, something that is not straightforward, or something happens in MMFM but not in pure LLM?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730394365020

thread_type:
official_review

<thread_object>
creator: reviewer_sjeb

<content>
summary:
This paper introduces MMDT, a unified platform for evaluating the safety and trustworthiness of MMFMs. Unlike existing benchmarks, MMDT assesses models across multiple dimensions, including safety, hallucination, fairness/bias, privacy, adversarial robustness, and OOD generalization. The platform includes diverse evaluation scenarios and red teaming algorithms to generate challenging data. Evaluations conducted using MMDT reveal key vulnerabilities and areas for improvement in multimodal models, establishing MMDT as a foundational tool for advancing safer and more reliable MMFMs.

strengths:
Strengths:

+ This work offers MMDT, a unified platform for evaluating the safety and trustworthiness of MMFMs. 
+ Comprehensive and thorough assessment.
+ Revealed the security risk of the MMFMs.

weaknesses:
Weaknesses:

- No obvious weaknesses.

questions:
Questions:

I am pleased to see that the authors have provided a comprehensive and in-depth benchmark evaluation of MMFMs in terms of safety, hallucination, fairness, and privacy, particularly including mainstream commercial models such as GPT-4 in the evaluation. Additionally, the availability of detailed code, data, and case examples is valuable, allowing readers to conduct further research and extensions. However, I have a few additional questions I would like to discuss with the authors.


- Q1: Although the authors provide a comprehensive and rigorous assessment of MMFMs, especially through the design of red team models for each evaluation objective, it would be beneficial if the authors could also briefly discuss blue team design. Would the authors consider adding a discussion on future blue team approaches in the manuscript?

- Q2: Have the authors examined the security threats of multimodal inputs to MMFMs? For instance, [R1] presents a multimodal red team model for MMFMs; addressing this would expand the assessment's scope.

[R1] Liu Y, Cai C, Zhang X, et al. Arondight: Red Teaming Large Vision Language Models with Auto-generated Multi-modal Jailbreak Prompts[C]//Proceedings of the 32nd ACM International Conference on Multimedia. 2024: 3578-3586.

- Q3: Could the authors elaborate on the differences and connections between the red team model introduced in this paper and existing work? Highlighting these distinctions would improve clarity.

- Q4: For dynamically evolving MMFMs, how does the proposed red team model adapt to ongoing changes? It would be helpful if the authors could address whether this adaptability has been considered in the current model.

- Q5: Finally, have the authors accounted for the diversity of test cases? Given that diversity is a crucial indicator of a red team model's effectiveness, an overview of this aspect would strengthen the evaluation.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730576223395

thread_type:
official_review

<thread_object>
creator: reviewer_dzze

<content>
summary:
This paper presents MMDT (Multimodal DecodingTrust), the first unified platform dedicated to comprehensively evaluating the safety and trustworthiness of multimodal foundation models (MMFMs). MMDT provides a thorough assessment of both text-to-image (T2I) and image-to-text (I2T) models across key dimensions, including safety, hallucination, fairness, privacy, adversarial robustness, and out-of-distribution generalization. Through diverse evaluation scenarios and red teaming algorithms, MMDT establishes a high-quality benchmark that uncovers significant vulnerabilities within MMFMs.

strengths:
1. The evaluation dimensions in this paper are comprehensive, covering essential aspects of trustworthiness, such as safety, hallucination, fairness, privacy, adversarial robustness, and out-of-distribution robustness.
2. The paper provides a thorough evaluation of multimodal foundation models from both T2I (text-to-image) and I2T (image-to-text) perspectives, resulting in a more unified assessment.
3. The writing is well-structured and logically clear.

weaknesses:
1. The formatting of the paper appears to have some issues—could it be too wide?
2. The paper lacks a description of the design of the evaluation platform. As a trustworthiness-focused evaluation platform, this section is essential to highlight its unique contributions.
3. Please provide more detail on the criteria used to select models for evaluation. Appendix A.2 offers only a brief overview, but I would like to understand the rationale behind these choices. Furthermore, the selection of evaluated models seems limited, particularly for I2T models, as classic models like the Claude series are notably missing.
4. We notice that some data in the dataset are generated using models like GPT-4, and that GPT-4o is also used as an evaluation model. Could this approach inherently favor the GPT-4 series in the evaluation?

questions:
Please refer to the weakness. If questions are addressed, I will raise the rating.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730716403860

thread_type:
official_review

<thread_object>
creator: reviewer_akat

<content>
summary:
This paper introduces MMDT (Multimodal DecodingTrust), the first unified platform for comprehensive safety and trustworthiness evaluation of multimodal foundation models (MMFMs). Unlike existing benchmarks that primarily assess helpfulness or focus on limited aspects such as fairness and privacy, MMDT evaluates MMFMs across multiple dimensions: safety, hallucination, fairness/bias, privacy, adversarial robustness, and out-of-distribution generalization. The platform includes diverse evaluation scenarios and red teaming algorithms to create a challenging benchmark. Through testing various multimodal models, MMDT identifies vulnerabilities and areas for improvement, contributing to the development of safer and more reliable MMFMs and systems.

strengths:
1. MMDT provides a thorough evaluation across multiple dimensions of trustworthiness—safety, hallucination, fairness, privacy, adversarial robustness, and out-of-distribution (OOD) generalization. This comprehensive approach is unique and valuable for identifying vulnerabilities in multimodal foundation models (MMFMs).'
2. The study includes a range of scenarios tailored to different tasks, such as object recognition, counting, spatial reasoning, and attribute recognition. The use of red teaming algorithms for generating challenging data is innovative and enhances the robustness of the benchmark​.
3. By testing MMFMs in applications like autonomous driving and healthcare, the paper addresses real-world scenarios where safety and reliability are critical. This relevance adds practical value to the findings.
4. I like the very detailed appendix, which provide comprehensive information about the trustworthiness of MMFMs.

weaknesses:
1. Although the paper identifies vulnerabilities and areas for improvement, it lacks a detailed discussion or guidance on specific mitigation techniques for enhancing model trustworthiness. Including concrete strategies for addressing the identified issues could improve the paper’s utility.
2. While MMDT identifies vulnerabilities across multiple dimensions, it may lack sufficient guidance on interpreting and addressing these vulnerabilities for practitioners, particularly those unfamiliar with the nuances of each trustworthiness perspective.
3. The study highlights general issues with MMFMs, but not propose in-depth comparisons between models and answer what make specific model has  strength and weaknesse.​

questions:
Thank you for submitting your work to ICLR! Overall, I like to read this paper as it produces very comprehensive evaluation in terms of the trustworthiness of the MMFMs. While reading this work, I have the following questions and I wish the author can resolve them:

1. In Table 1, as jailbreaking harmful instructions are optimizing adversarial prompts to help bypass safety filters, why the jailbreaking has lower HGR compared with the some of Vanilla case? I thought the jailbreaking is optimizing adversarial prompts, so that it should achieve higher success rate with higher HGR and BR, but it seems that the HGR is not optimized. Can the author explain why they end with this result, and why not optimizing HGR when crafting the adversarial prompts?

2. It seems like GPT-4v achieves best performance in defending against generating harmful text output with different image manipulation, can the author discuss the possible reason behind this?

3. You extend beyond social stereotypes to include decision-making and “overkill fairness.” Could you clarify the criteria used to define and measure overkill fairness? For example, how do you balance fairness goals with historical accuracy, and do you find certain models more prone to overkill fairness in specific contexts?

4. In your OOD evaluation, you find that DALL·E 3 and GPT-4o outperform other models, especially in tasks involving spatial reasoning and attribute recognition. Could you specify the types of OOD transformations that most challenge MMFMs, and whether these models employ specific techniques that enhance their resilience to such transformations?

5. The privacy benchmark assesses object-level memorization, focusing on CLIP embedding similarity to detect privacy leaks. Given this method, could you discuss any limitations or potential improvements? For instance, could certain model fine-tuning approaches mitigate memorization without significantly impacting performance?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732268346968

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response

comment:
We sincerely thank the reviewers for their constructive feedback and thoughtful questions about our paper. We greatly appreciate the reviewers’ recognition of our contributions, including the comprehensive evaluation framework, innovative construction of red teaming algorithms, and practical relevance for improving the trustworthiness of MMFMs. Below, we outline the key changes we have made in response to the reviewers’ comments and summarize how their feedback has helped us strengthen the paper. The paper is updated and the revision is highlighted in blue.

1. Additional model evaluation. We have added the evaluation of additional models such as Claude 3.5 Sonnet, further enhancing the comprehensiveness of our benchmark.

1. Model Selection Criteria. We have clarified the criteria for selecting models for evaluation in Appendix A.2, ensuring transparency about the choices made.

1. Model Comparisons. We have added a thorough discussion in the main paper and Appendix C.1 to highlight the strengths and weaknesses of different models, including observations on conservativeness, alignment, architecture, and training data diversity.

1. Interpretation of Vulnerabilities. To provide clearer guidance on interpreting vulnerabilities, we now include detailed explanations for each trustworthiness perspective in Appendix C.2, outlining the causes and implications of the observed issues in MMFMs.

1. Mitigation Strategies for Enhancing Trustworthiness. We have expanded the discussion on mitigation strategies for each trustworthiness perspective (safety, hallucination, fairness, privacy, adversarial robustness, and out-of-distribution robustness) in the main paper with more details in Appendix C.3, as suggested. These strategies provide actionable insights for practitioners to address the vulnerabilities identified by MMDT.

1. Evaluation Platform Design. To address requests for more detail, we now include a description of the platform’s modular design in the main paper, along with more details in Appendix B, along with a figure illustrating its architecture. This highlights the platform's scalability, extensibility, and ability to conduct rigorous multimodal evaluations.

1. Unique Findings in MMFMs. We have emphasized discoveries unique to MMFMs compared to LLMs, such as cross-modal vulnerabilities, attribute amplification, and modality-specific privacy risks, addressing concerns about limited analytical insights.

In summary, we have taken significant steps to address the reviewers' questions by expanding our evaluations, improving clarity, enhancing the structure of our paper, and adding discussions on practical mitigation strategies. We also integrated these improvements in our revisions and we believe they will further strengthen the contribution and impact of our work.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper introduces MMDT (Multimodal DecodingTrust), the first unified platform for comprehensive safety and trustworthiness evaluation of multimodal foundation models (MMFMs). Unlike existing benchmarks that primarily assess helpfulness or focus on limited aspects such as fairness and privacy, MMDT evaluates MMFMs across multiple dimensions: safety, hallucination, fairness/bias, privacy, adversarial robustness, and out-of-distribution generalization. The platform includes diverse evaluation scenarios and red teaming algorithms to create a challenging benchmark. Through testing various multimodal models, MMDT identifies vulnerabilities and areas for improvement, contributing to the development of safer and more reliable MMFMs and systems. The reviewers agree on the work: (1) MMDT provides a thorough evaluation across multiple dimensions of trustworthiness. This comprehensive approach is unique and valuable for identifying vulnerabilities in MMDT. (2) The study covers a range of scenarios and tasks including the real-world scenarios where safety and reliability are critical. (3)  The writing is well-structured and logically clear with a detailed appendix.  

The reviewers also have some concerns including: (1) lacking a detailed discussion or guidance on specific mitigation techniques for enhancing model trustworthiness. (2) Lacking in-depth comparisons between models and answers to what makes specific models have strengths and weaknesses.​ (3) Lacking a description of the design of the evaluation platform. (4) Lacking the explanation of the use of GPT-4 in dataset generation and evaluation inherently favors GPT-4 series models. (5) The lack of analytical components or theoretical insights for the MMFMs vs. LLMs. Most of the issues are properly addressed in the rebuttal and revised version. 

According to the reviewers' comments and rebuttal discussion, we agree this work is important for the community and should be definitely accepted.