Title: A Chain-of-Thought Is as Strong as Its Weakest Link: A Benchmark for Verifiers of Reasoning Chains
Benchmark: REVEAL (Reasoning Verification Evaluation)
Summary: This paper introduces REVEAL, a benchmark dataset created to evaluate automatic methods for verifying step-by-step reasoning chains, specifically Chain-of-Thought (CoT) answers from language models in open-domain QA. REVEAL provides fine-grained annotations for each reasoning step, assessing its relevance, type (attribution or logic), factual correctness against evidence (attribution), and logical consistency with previous steps. The benchmark aims to support research in improving the reliability and correctness of LLM reasoning.
Contribution: The main contribution is the REVEAL dataset, the first benchmark for detailed, step-level evaluation of CoT reasoning verifiers. It includes a comprehensive annotation schema covering relevance, step type, attribution, and logic, applied to CoT answers from multiple LLMs across diverse QA datasets. The dataset also features annotator justifications for each label. The paper provides baseline results for several verifiers, highlighting current challenges, especially in verifying logical correctness.
Phenomenon type: General Capability (A broadly useful ability, which could be relevant to multiple applications)
Target phenomenon: Correctness verification of reasoning chains (Chain-of-Thought); Assessing attribution and logical validity of reasoning steps.
Phenomenon defined by authors: Yes
Phenomenon definition: The correctness of a reasoning step within a Chain-of-Thought answer is defined along multiple dimensions: relevance to the question, the step's type (introducing external facts vs. logical inference vs. both), its attribution status relative to provided evidence (fully supported, partially supported, contradicted, unsupported), and its logical consistency with preceding steps (correct vs. incorrect).
Definition scope: Comprehensive
Definition integrity: Composite phenomenon
Definition integrity detail: Yes
Phenomenon contested: Widely-agreed
Purpose notes: The work is motivated by the need for reliable methods to evaluate and improve the correctness of LLM-generated reasoning chains, as incorrect reasoning can undermine the utility of CoT prompting despite potentially correct final answers. The dataset isolates verification from evidence retrieval.
Task definition: Step-level Reasoning Verification: Given a question, a CoT answer, a specific step within that CoT, preceding steps, and potentially external evidence passages, classify the step based on its relevance, type (attribution/logic/both), attribution correctness relative to evidence (if applicable), and logical correctness relative to preceding steps (if applicable).
Task item definition: An instance comprises a question (from StrategyQA, MuSiQue, Sports Understanding, or Fermi), a CoT answer generated by an LLM (Flan-PaLM, GPT-3, or Flan-UL2), and step-level annotations. For each step, these annotations include relevance, step type, logical correctness label, and (for attribution steps) attribution labels relative to up to three retrieved Wikipedia evidence paragraphs. Each label comes with free-text justifications from 5 annotators.
Task definition detail: Verification is performed at the step level. Attribution uses Wikipedia as the knowledge source. A distinction is made between steps requiring factual attribution, logical inference, or both. Ambiguous/low-agreement cases are separated into REVEAL-Open.
Task source: Real task examples (e.g. GitHub issues), Author-crafted task examples (e.g. hand-written examples, manual transformation of existing data into questions), Crowd-sourced task examples (e.g. Prolific-created tasks), LLM-generated task examples (e.g. Filtered from responses to a prompt)
Task source detail: Questions are from StrategyQA, MuSiQue, Sports Understanding, Fermi. CoT answers generated by Flan-PaLM-540B, GPT-3, Flan-UL2-20B. Evidence retrieved from a 2021 Wikipedia snapshot using GTR/BM25 after decontextualization. Step-level verification annotations collected from 13 human annotators (5 per item) using a custom two-task protocol.
Task ecology: Representative task (e.g. answering medical licensing exam questions)
Task ecology detail: Assesses fundamental aspects (factual grounding, logical flow) required for reliable reasoning, applicable to various domains where complex problem-solving is needed.
Dataset size: REVEAL-Eval (main evaluation set): 1002 CoT answers / 3360 steps.
Dataset size notes: REVEAL-Open (low agreement set): 224 CoT answers / 847 steps. No training/validation splits defined.
Dataset metadata available: Yes
Dataset metadata detail: Source QA Dataset, CoT Generating LLM, Question, Full CoT Answer, Step Index, Step Text, Relevance Label, Step Type Label, Logical Correctness Label, Evidence Passages (for attribution steps), Attribution Label (per step-evidence pair), Annotator Justifications (free text).
Sampling method: Convenience sample (creators found a set of tasks that was readily accessible), Targeted items (creators defined a task space and chose tasks within it strategically)
Split reviewed: Test
Response format: Multiple choice
Response format detail: Automatic verifiers output class labels for relevance, step type, attribution correctness, and logical correctness for each step.
Metric definition: Exact Match (accuracy, F1, precision, recall)
Metric definition detail: Macro F1 score is used for evaluating the step-level classification tasks (attribution 2-class, attribution 3-class, logic, type) and the CoT-level correctness task. Per-class F1 scores are also provided.
Metric aggregation: Simple Mean
Metric statistics:  Macro F1 score, per-class F1 score
Metric subscores: Yes
Metric subscores detail: Performance is reported per task (Attribution 2/3-class, Logic, Type, CoT-level). Full CoT correctness breakdown by source dataset and generating model is shown. Analysis of unsupported steps and disagreement categories in REVEAL-Open is provided. Per-class F1 scores are available.
Metric access: Outputs alone
Authorship: Mix (multiple authors from industry and academia)
Benchmark availability: Yes
Task face validity: Yes
Metric face validity: Yes
Results interpreted by authors: Yes
Results compared: Yes
Results comparison explanation: Yes
Results realism: No
Human baseline: No
Authors discuss validity: Yes
Author validity detail: A two-task annotation protocol was designed to reduce cognitive load. 5 annotators provided labels and justifications for each item. 3 pilot rounds refined the process. Inter-annotator agreement measured (Krippendorff's alpha 0.49 for attribution, 0.46 for logic). Low-agreement (ambiguous/difficult) cases were identified and separated into REVEAL-Open, with analysis of disagreement reasons. Reasons for unsupported attribution labels were also analyzed.
Taxonomy: Reasoning / Logical
New taxonomy: General Capability / Logical Reasoning
Procedural notes: Source questions from StrategyQA, MuSiQue, Sports Understanding, Fermi. CoT answers generated by Flan-PaLM-540B, GPT-3 (text-davinci-003), Flan-UL2-20B. Evidence from 2021 Wikipedia via GTR/BM25. Decontextualization applied before retrieval. 13 annotators involved. Data contamination mitigation practices used. Detailed prompts provided in appendix.
Other notes: A key distinction is the fine-grained, step-level verification, compared to evaluating only the final answer or full chain correctness. The dataset highlights that verifiers find logical correctness harder to assess than attribution, while CoT generators often struggle more with attribution. The inclusion of free-text justifications is a valuable resource for future work.