Rethinking Token Reduction in MLLMs: Towards a Unified Paradigm for Training-Free Acceleration
Abstract
To accelerate the inference of heavy Multimodal Large Language Models (MLLMs), this study rethinks the current landscape of training-free token reduction research. We regret to find that the critical components of existing methods are tightly intertwined, with their interconnections and effects remaining unclear for comparison, transfer, and expansion. Therefore, we propose a unified ''filter-correlate-compress'' paradigm that decomposes the token reduction into three distinct stages within a pipeline, maintaining consistent design objectives and elements while allowing for unique implementations. We additionally demystify the popular works and subsume them into our paradigm to showcase its universality. Finally, we offer a suite of methods grounded in the paradigm, striking a balance between speed and accuracy throughout different phases of the inference. Experimental results across 10 benchmarks indicate that our methods can achieve up to an 82.4% reduction in FLOPs with a minimal impact on performance, simultaneously surpassing state-of-the-art training-free methods. Our project page is at https://ficoco-accelerate.github.io/.
Community
Project page: https://ficoco-accelerate.github.io/
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Rethinking Token Reduction for State Space Models (2024)
- ZipVL: Efficient Large Vision-Language Models with Dynamic Token Sparsification and KV Cache Compression (2024)
- γ-MoD: Exploring Mixture-of-Depth Adaptation for Multimodal Large Language Models (2024)
- Anchor Attention, Small Cache: Code Generation with Large Language Models (2024)
- ADEM-VL: Adaptive and Embedded Fusion for Efficient Vision-Language Tuning (2024)
- To Preserve or To Compress: An In-Depth Study of Connector Selection in Multimodal Large Language Models (2024)
- TokenSelect: Efficient Long-Context Inference and Length Extrapolation for LLMs via Dynamic Token-Level KV Cache Selection (2024)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper