A Prompt-Based Framework for Multi-Hop Visual Question Answering with Reasoning Path Generation
Authors: qiyuan Feng, Zhenyun Deng, Xiaofeng Zhu


Reviewer paper summaries: This paper proposes a new prompt-based methodology for enhancing reasoning capabilities in multi-hop visual question answering (VQA) tasks. The proposed technique focuses on the identification and resolution of intermediate "unknown hops," or missing steps in the logical progression required to answer a visual question.
Summary of strengths: - this paper proposes the introduction of an intermediate unknowns detection and alignment module designed to pinpoint and align intermediate unknown hops within multi-hop visual question answering (MVQA) tasks. By extracting these missing steps and their corresponding answers, we lay the groundwork for a more robust reasoning process.

 - propose a novel approach to generate reasoning paths specifically tailored for prompt-based MVQA. This method aims to mitigate the risk of exploiting reasoning shortcuts, ensuring that the model engages in a more comprehensive and accurate reasoning process.
Summary of weaknesses: - The reasons for employing Stanza for entity phrase extraction and GroundingDINO for object detection remain unexplained in the paper.
- The paper asserts in lines 233-235 that "a large number of entities unrelated to the image may be extracted, which may affect the efficiency and accuracy of subsequent entity alignment," yet not to provide any empirical evidence or demonstrations to substantiate this claim.
-  The paper doesn't introduce the baselines employed in their study and provide a rationale for selecting those specific methods.

- The evaluation metrics chosen for this study warrant further consideration. It would be insightful to examine the performance of the proposed methods using metrics like CIDEr, which assesses the similarity between generated responses and reference captions, particularly in terms of n-gram overlap and precision. This could provide a more comprehensive understanding of the model's ability to generate accurate and informative responses

- The experimental results would be more comprehensive if they included a comparison with adaptive methods such as Chain of Spot [1]. This would provide a better understanding of the proposed method's relative strengths and weaknesses compared to other approaches in addressing the challenges of multi-hop visual question answering.
-  The lack of clarity in the paper's structure, particularly in Section 4, creates confusion regarding the selection of methods for the main results and the ablation study. A more transparent explanation of the criteria used to distinguish between these two sets of methods would greatly improve the paper's coherence and comprehensibility.


[1]. Liu, Z., Dong, Y., Rao, Y., Zhou, J., & Lu, J. (2024). Chain-of-Spot: Interactive Reasoning Improves Large Vision-Language Models. arXiv preprint arXiv:2403.12966.
Comments, suggestions and typos: - The authors should ensure the clarify of paper, some in appendix should be put in experiment section and reversely to assist readers easier to follow
- in lines 233-235 that "a large number of entities unrelated to the image may be extracted, which may affect the efficiency and accuracy of subsequent entity alignment,", the authors should declare some claims or experiments for demonstrate this. I'm really curious about this. 
- The author should declare reasons why they choose baselines.
Overall assessment scores: 2.5