ICLR 2025

Published as a conference paper at ICLR 2025
UNVEILING THE MAGIC OF CODE REASONING
THROUGH REFLECTIVE HYPOTHESIS
DECOMPOSITION AND AMENDMENT
Yuze Zhao1, Tianyun Ji1,∗, Wenjun Feng1,∗, Zhenya Huang1,2,†, Qi Liu1,2,
Zhiding Liu1, Yixiao Ma1, Kai Zhang1, Enhong Chen1
1State Key Laboratory of Cognitive Intelligence,
University of Science and Technology of China
2Institute of Artificial Intelligence, Hefei Comprehensive National Science Center
yuzezhao@mail.ustc.edu.cn huangzhy@ustc.edu.cn
ABSTRACT
The reasoning abilities are one of the most enigmatic and captivating aspects of
large language models (LLMs). Numerous studies are dedicated to exploring and
expanding the boundaries of this reasoning capability. However, tasks that em-
body both reasoning and recall characteristics are often overlooked. In this paper,
we introduce such a novel task, code reasoning, to provide a new perspective for
the reasoning abilities of LLMs. We summarize three meta-benchmarks based
on established forms of logical reasoning, and instantiate these into eight specific
benchmark tasks. Our testing on these benchmarks reveals that LLMs continue
to struggle with identifying satisfactory reasoning pathways. Additionally, we
present a new pathway exploration pipeline inspired by human intricate problem-
solving methods. This Reflective Hypothesis Decomposition and Amendment
(RHDA) pipeline consists of the following iterative steps: (1) Proposing potential
hypotheses based on observations and decomposing them; (2) Utilizing tools to
validate hypotheses and reflection outcomes; (3) Revising hypothesis in light of
observations. Our approach effectively mitigates logical chain collapses arising
from forgetting or hallucination issues in multi-step reasoning, resulting in per-
formance gains of up to 3×. Finally, we expand this pipeline by applying it to
simulate complex household tasks in real-world scenarios, specifically in Virtu-
alHome, enhancing the handling of failure cases. We release our co...