Chain of Evidences and Evidence to Generate: Prompting for Context Grounded and Retrieval Augmented Reasoning
Authors: Md Rizwan Parvez


Reviewer paper summaries: The paper addresses the limitations of existing prompting frameworks for context-aware and retrieval-augmented reasoning in large language models (LLMs). It highlights issues such as hallucination and the overemphasis on grounding, which can hinder the model's ability to leverage common sense and world knowledge. To overcome these challenges, the authors introduce two novel prompting methods: Chain of Evidences (COE) and Evidence to Generate (E2G), designed to enhance the model's reasoning capabilities by focusing on contextually grounded evidence. The results demonstrate substantial improvements in accuracy and robustness across various tasks, suggesting that these methods can effectively reduce unsupported claims and improve logical reasoning in LLMs.
Summary of strengths: 1. The introduction of Chain of Evidences (COE) and Evidence to Generate (E2G) strategies advances context-aware reasoning in LLMs by addressing limitations of existing prompting techniques.

2. The paper demonstrates state-of-the-art performance on benchmarks like LogiQA and DROP, showcasing the practical effectiveness and robustness of the proposed methods.

3. The paper is well-structured and includes a comprehensive review of related literature, effectively contextualizing the contributions within the broader research landscape.

4. The authors mention the exact prompts used in experiments, thereby helping in reproducibility.
Summary of weaknesses: While the proposed methods show strong performance on specific benchmarks, there is a concern about their generalizability across a wider range of tasks and domains. The paper could benefit from additional experiments to demonstrate robustness in diverse contexts.
Comments, suggestions and typos: See weekness
Overall assessment scores: 3.5