Simignore: Enhancing Multimodal Large Language Models Complex Reason via Similarity Computation
Authors: FanshuoZeng, Xiaofeng Zhang, Zheng Hui


Reviewer paper summaries: This paper introduces the Simignore algorithm, which aims to improve the efficiency and reasoning capabilities of multimodal large language models (MLLM) by computing and leveraging the similarity between image and text embeddings. Simignore selectively ignores irrelevant image tokens, thereby enhancing model performance in complex reasoning tasks. The effectiveness of the algorithm is demonstrated through experiments on the ScienceQA dataset.
Summary of strengths: The paper identifies that selectively ignoring certain image tokens can lead to improved performance on the ScienceQA benchmark, indicating a potential reduction in computational load while maintaining or enhancing accuracy in complex reasoning tasks.
Summary of weaknesses: The primary concern is the limited experimental validation, as the study relies solely on the ScienceQA benchmark. This raises questions about the generalizability of the findings and whether they can be substantiated across diverse datasets or real-world applications.
Comments, suggestions and typos: **Typo Corrections:**
- Line 167: Change "output.Wei" to "output. Wei".
- Line 171: Change "Kojima et al. Kojima (Kojima et al., 2022)" to "Kojima et al. (Kojima et al., 2022)".
- Line 221: Change "1," to "1.".
- Line 503: Change "’spy’" to "`spy'".
- Additionally, there are frequent spacing errors throughout the paper that need to be corrected to meet publication standards.
Overall assessment scores: 1