A Balanced Neuro-Symbolic Approach for Commonsense Abductive Logic
Authors: Joseph Cotnareanu
Didier Chételat
Yingxue Zhang
Mark Coates
Venue: ICLR 2026 Poster (2026)
Keywords: Reasoning
Logic
Artificial Intelligence
Large Language Models
Abduction
Primary area: foundation or frontier models, including LLMs

Abstract: Although Large Language Models (LLMs) have demonstrated impressive formal reasoning abilities, they often break down when problems require complex proof planning. One promising approach for improving LLM reasoning abilities involves translating problems into formal logic and using a logic solver. Although off-the-shelf logic solvers are in principle substantially more efficient than LLMs at logical reasoning, they assume that all relevant facts are provided in a question and are unable to deal with missing commonsense relations.  In this work, we propose a novel method that uses feedback from the logic solver to augment a logic problem with commonsense relations provided by the LLM, in an iterative manner. This involves a search procedure through potential commonsense assumptions to maximize the chance of finding useful facts while keeping cost tractable. On a collection of pure-logical reasoning datasets, from which some commonsense information has been removed, our method consistently achieves considerable improvements over existing techniques, demonstrating the value in balancing neural and symbolic elements when working in human contexts.

Published as a conference paper at ICLR 2026
A BALANCED NEURO-SYMBOLIC APPROACH FOR COM- MONSENSE ABDUCTIVE LOGIC
Joseph Cotnareanu 1,3,4, Didier Chetelat', Yingxue Zhang', Mark Coates 1.3,4
McGill University Huawei Noah's Ark Lab
3 International Laboratory on Learning Systems (ILLS) 4 MILA
(joseph. cotnaaaa mark. coates @mail 1al.1.mmgili picggill... ca, didier.chetellt, yingxue. zhang) ) @huawei.
ABSTRACT
Although Large Language Models (LLMs) have demonstrated impressive formal reasoning abilities, they often break down when problems require complex proof planning. One promising approach for improving LLM reasoning abilities involves translating problems into formal logic and using a logic solver. Although off- the-shelf logic solvers are in principle substantially more efficient than LLMs at logical reasoning, they assume that all relevant facts are provided in a question and are unable to deal with missing commonsense relations. In this work, we propose novel method that uses feedback from the logic solver to augment a logic problem with commonsense relations provided by the LLM, in an iterative manner. This involves a search procedure through potential commonsense assumptions maximize the chance of finding useful facts while keeping cost tractable. On a collection of pure-logical reasoning datasets, from which some commonsense information has been removed, our method consistently achieves considerable improvements over existing techniques, demonstrating the value in balancing neural and symbolic elements when working human contexts.
INTRODUCTION
I Large Language Models (LLMs) have demonstrated impressive abilities to reason formally, often via thain-of- reasoning (Wei et al., 2022). While the state of the art modern -.L.M- systems show impressive reasoning capabilities, it unclear whether this comes from the LLM itself, or sophisticated post-learning refinement algorithms. At the same time, open-sourced LLMs still demonstrate inability to naturally scale to problems that require complex proof planning (Saparov and He 2023) Dziri et al.) 2023). Such problems are exactly the type on which symbolic logical solvers excel: such solvers have long history and were for long time considered key component of any path to artificial intelligence (Nilsson 1991). Nevertheless, they are greatly restricted by their need for problems be stated in symbolic language and for every relevant fact to be provided as input. These constraints have ultimately limited them to highly specialized applications, and they have never had the broad impact that was hoped for (Crevier) 1993).
These complimentary strengths of neural and symbolic methods have motivated revival of interest in neuro- symbolic methods, where an LLM incorporates a logic solver to improve its reasoning abilities (Ye et al., 2023 Lee and Hwang, 2024: Lyu et al., 2023) Olausson et al.) 2023). In these approaches, the LLM translates problems formulated
natural language into symbolic language, addressing one of the key deficiencies of a purely symbolic approach. Nonetheless, these hybrid systems remain impractical because they are ultimately purely deductive: that is, every relevant fact must be provided input. This means that the symbolic solvers are often unable to reach conclusion simply because obvious, commonsense assumptions are left unstated, and is often difficult to predict which should be included until one is presented with failed reasoning chain.
A logic solver would return "unknown" for the target query as, formally speaking, neither its truth nor its falsehood is implied by the premises. A human, however, would easily solve this problem by supplying the additional commonsense fact that white
For example, consider the problem in Figure I

---

Published as a conference paper at ICLR 2026
Context: Some animals tough winter out. They do not leave. They do not hide. They must survive. Sometimes nature helps them out. Some animals grow thicker coats in the winter. Other animals change color. The arctic fox is brown in the summer. His coat turns white in the winter.
3x mough_out(s winter)
3x -hide(x) A -leave(x) A survive(x)
3x helps(nature,
grow_coatta winter)
3x tttange_colorxx, winter)
brown(fox, summer)
turns white ox, winter)
The arctic fox's coat turns white in the winter because white absorbs the sun and is warmer.
Question:
absorbs(whitte,sue sun) warmer(coat) turns_ lurns_white(fos winter)
Figure I: An example from children's 's comprehension exercise booklet in human language. Right: the same problem translated to first-order-loiic.
Left: the problem phrased
surfaces reflect light (turns_ve si__wwhtte(ff (urrs_wwhtte(ffe missing information is usually themed abductive reasoning, and a key mark of human intelligence.
winter) reflects(sfr sum)). This ability supply
The limitation of current neuro-symbolic LLM systems deductive reasoning means that they have mostly been so far of theoretical interest, since they tend to break down when confronted with more complex problems where enumerating every possible background fact is realistic. However, besides their translation skills, LLMs possess also another striking ability: their training on prodigious amounts of internet data has made them very adept recognizing commonsense statements, the point where they have been regarded as potential universal databases (Petroni et al. 2019). In way. LLMs seem to have internalized most commonsense knowledge.
This realization has led some works to use an LLM itself to supply missing but relevant clauses when reasoning. Notably, Toroghi et al. (2024) proposed method that operates an ex- haustive search
Symbolic (logical)
LLM Tres
Logic of Thought
Linguistic (neural)
heavily restrained set of rules the symbolic space, whereas Liu al. (2024) proposed method that uses I.LMM prompting to produce new rules which might be deducible from the given logical context. While these methods lie on opposite ends of the symbolic-liguusticc reasoning spectrum (Fig- ure 2). they both limit themselves to searching over such restricted space of possible common- sense that they cannot solve practical problems.
Semantic Parsers
ARGOS
COT
Figure 2: Symbolic-Linguistic Spectrum depicting the positioning of LLM-Tree (Tooghhi et al. 2024 Logic-of-Thour Thought (COT) (Wei et al. approach.
(Liu al., 2024), and Chain-of-
2022 relative to
In this work, we seek to improve AI reasoning abilities by using an LLM to provide relevant unstated commonsense clauses to a logic solver, but unlike previous works, without imposing significant constraints on the shape content of such clauses. Furthermore, and most importantly, our method ARGOS (Abductive Reasoning with Generalization Over Symbolics) can abduce propositions not previously instantiated in the input problem. To compensate for the far more general search space, we guide the search using feedback from the logic solver in the form of the SAT problem backbone, another novel contribution. The resulting system strikes balance between linguistic and symbolic approaches, allowing us to use both their strengths while minimizing their weaknesses to achieve true abductive reasoning.
The contributions of this paper are as follows.
We propose novel framing of the commonsense logical reasoning problem founded upon classical logical principles and an aim towards more realistic use-cases.
'Taken from ht tps: :ttp::::wwwwe /wwww hwt.:wereeddaee wweadingworksheeeeeee.oeo.e s. com/ httm///okssheeteee readii/eeannnnggna www.r/egggnnnnnneeeeeeeeeeeeee iomofiict: on mn-passages/// wi intertii me) We selected a choice from multiple choice question 3 and re-phrased as True/False question, according to the logic-problem framing.
2

---

Published as a conference paper at ICLR 2026
We introduce novel algorithm that (i) searches over larger spaces of commonsense facts; and (2) uses logic solver feedback in the form of the backbone graph to increase practicality and efficiency.
We demonstrate empirically on multiple benchmarks and large language models that our method improves substantially over existing symbolic and neural methods on abductive reasoning problems where background information is missing.
RELATED WORK
Previous LLM-related logical reasoning methods combine symbolic and neural approaches, but usually rely much more on one or the other. Appendix G Provides an extended review.
Neural Methods Wei et al. (0222) were the first to present a framework for LLM-based reasoning, showing that providing examples of rationales for answers to questions can induce the LLM to do the same, leading to improved accuracy. Kojima et al. (2022) showed that this can be induced without any few-shot examples by prepending the sentence "Let's ss think step by step" before generating an answer. This is known as "Chain of Thought" (COT). Following this, Wang et al. (2023) proposed self-consistency (SC), using COT multiple times and taking the mode as the prediction. However, Saparov and He ) observed that COT and SC suffer from challenges in proof planning
rationale steps lend to be factual but of low value. This motivated guidance of the LLM at step-level.
Yao et al. (2023) proposed Tree of Thoughts (TOT), which explores hand-crafted trees using an LLM to solve reasoning tasks. TOT is poorly suited logical reasoning settings as logic problems have highly variable tree-structures. Kazemi et al. (2023) and Lee and Hwang (2024) proposed more logic-focused methods, with reverse reasoning, starting at the answer and ending at the problem. These back-chaining methods, however, underperform symbolic approaches.
Symbolic Methods Acknowledging that LLMs are poor proof-planners, a series of methods, including F-COT (Lyu et al., 2023) and SAT-LM (Ye et al. 2023), proposed offload the reasoning burden from the LLM to more specialized tools. In these works, the LLM converts the text to symbolic logic, and solver is then employed. Logic-IM (Pan al. 2023) extended this to include self-refinement step. While these methods perform well on simple datasets, they fail to account for ambiguity and the exclusion of common knowledge. Addressing this, [iu et al. (2024) and Wang et al.) (002)) proposed algorithms that produce new clauses via logical deduction and then add the logic back to the text for an LLM to solve. While this might help the LLM, it does not add information to the problem, because any added relations are already deducible. Instead of producing clauses via deduction, [Toroghi et al. (2024) proposed a method that exhaustively searches for new single- proposition modus- the question, and repeated until the problem is solvable by classical logic, diminishing robustness. This search space is highly restricted and leaves out nearly all necessary information for some logic problems.
clauses. However, the search is conducted only over the propositions from
3 BACKGROUND
Propositional logic is a logical system built around propositions, which are statements of fact such as "It is sunny" or "I need an umbrella" which can be true or false. Propositions are often denoted by single letter variables such as A or B, called a propositional variable, which can be tied together by logical connectives (such as A, V or +) form further compound propositions.
A deductive propositional logic problem is composed of a set of propositional variables V, a set of propositions (each represented by a propositional variable in V) and compound propositions (built by using logical connectives to connect propositions by their representative propositional variables in V) called the premises P = (Pi,... , Pk) ). and a proposition or compound proposition Q also built from those variables, called the query. The premises are given to be true (- PP), and the goal of the problem is to determine whether they imply the query, P I Q, or its negation, P - -Q. Such problems are usually solved by translating them into two Boolean Satisfiability (SAT) problems, one for Q and one for QQ Let L(V) = {A| A E V} {-A| A E V} denote the set of all so-called literals of the problem. The backbone of the problem is the collection of all those literals which are implied by the premises, (ackbone(P)) = (L E CIP P + L}. In effect, they are values for the propositions represented by the variables in the problem which can be inferred from the premises. In
3

---

Published as a conference paper at ICLR 2026
A)
The Framework
B) The Logic Module
C) The Augmentation Module
SAT Solver
in
in
Antecedent Selection
Text
Logic
solvable?
SC
Scores
Generate
B)
yes
Logic
Solution
conf confident idence >
C) Problem Augmentation
S scores > f
Solution
Not Solvable
Generated
Solution
Clause
start
stop
neural step
symbolic step
Figure 3: ARGOS glance. See Section 4.1 and Appendix F for details. (A) Given propositional
logic problem, we iteratively augment the problem with new propositions until it is solvable. (B) We attempt solve the problem both with logic solver, and with self-consistency (Wang et al. 2023) (C) we fail, we attempt to add additional commonsense propositions by combining literals from the backbone antecedents, and generating right-hand-side using an LLM. We test the proposition for commonsense and relevance using this same LLM, and add it to the pool if it passes the tests.
an abductive commonsense propositional logic problem the premises P entail neither the query Q nor its negation -Q: the problem is underdetermined. Instead, must augment the premises with additional commonsense propositions C, which represent background facts or knowledge left unstated in the problem, until either (P AC) + Q (P A AC) + -Q. Thus, the goal of an abductive problem is to not only find the truth-value of Q, but also a corresponding set of commonsense propositions C to complete the problem. We assume that P C. 1, that is that the premises P not contradictory with commonsense (i.e. that / and C' are consistent). One can show that, under this assumption, the answer the problem will not depend the choice of commonsense C: details are provided in Appendix/A
In practice, the problems we encounter real life are often stated in terms of first-order logic. First-order logic is logical system that extends propositional logic to entities and their predicates. n-ary predicate a symbol of a relation, such as MotherOf, that takes as arguments n terms such as and to become formula MotherO (x, y). and becomes true or false when constants, such as Alice and Bob, are used grounding for its arguments. Predicates can be connected by logical connectors, and can also be quantified over a discrete or abstract set of entities with V and 3, to form compound propositions such as Va'yy (odtyyMooeerO) (z, y) + -Male(x)..
First-order logic formulas over finite set of entities can always be converted into equivalent propositional logic formulas, a process known as grounding, by instantiating propositional variables for every predicate F(x) and entity A. and expanding Vr Fxx) into the compound proposition (F(A) A F(B) A...) and Er F(x) into (F(A) V F(B) V..).. Given two propositional literals, will declare them related in first-order logic if they have entity in common. For example, MotherOf (Alice, Bob) and -Male (Alice) are related because both involve the entity Alice.
3.1 PROBLEM STATEMENT
We are given an abductive propositional logic problem in both textual and logical form, as defined in Section 3, and we are also provided with large language model and SAT solver. As described, the task to determine whether the target query is true or false given the premises and some additional commonsense propositions which must be found. Four annotated examples are provided, intended for fho-shot prompting. In particular, the task is inference-only and no training phase is involved. We evaluate performance based on the number of correctly answered questions test dataset.
4

---

Published as a conference paper at ICLR 2026
4 METHODOLOGY
We now describe our novel algorithm to tackle the problem described in Section 3.1 This algorithm described by the diagram in Figure 3 and formally as Algorithm in Appendix D
4.1 ALGORITHM
We start the algorithm by initializing our set of commonsense propositions as empty, C = 0. As shown in module B of Figure 3 we first try to solve the problem using the SAT Solver (sat_solve) to test whether either (P A C) Q or (P A C. H -Q. If it reaches one of these conclusions, our job is finished; if not, we at least obtain from our call the backbone B = {L E C C|P+ L}.
attempt to solve the problem using the LLM (11m solve) by k--hoot consiseny (we use k = 5 in our experiments). We ask the LLM whether the query is true or false, providing it the premises and the commonsense found so far. Details can be found in Appendix F.1 If the fraction of votes pass a certain threshold 7, we also conclude either (P PCC F Q or (P CJ F -Q respectively, and the algorithm is finished. This parameter 7 (initialized at 1 in our experiments) is reduced by a fixed amount 7 + a at every iteration (a = 0.1 in our experiments). Thus, the maximum cost of our algorithm, in terms of number of COTs required, is bounded at cost. < k=2-0.5 since when 7 0.5, the fraction of votes is guaranteed to pass the threshold since the vote is over binary classes. For details on empirical cost, see Appendix B
Next, still in module B of Figure
a
If neither solving method succeeds in establishing Q or -Q, we try to add new commonsense
proposition to our pool C, as illustrated in module C of Figure 3 In practice, we define a proposition to be commonsense if it seems true to a large language model without any context. To guarantee that the added proposition will grow the problem's 's backbone, we search for commonsense propositions of the form L L2 + Lighht where L1 and L2 are literals in the backbone B, and Liight is a new literal suggested by the LLM. Note that Ln and L2 may be the same literal, in which case we in effect have a formula of the form I, - Lrightt thereby allowing both single and two- literal antecedents. In addition, by adding 0 to the set of backbone literals, we can also have @ + Lright. allowing O-literal antecedents. This search routine [find_new site__coomooseeeee described in Algorithm 2 in the Appendix. In detail, we start by iterating over pairs of literals in the backbone. We iterate by prioritizing the literals that share the most entities with others in the backbone, (C)res(L)
is
#(L' E B L' has an entity in common with L), so that we take highly- scored literals first. This gives a measure of relevance of the literal to the problem. To understand the rationale behind this choice, consider and only one is known about Jane. If asked to guess about whom the problem is about, the natural guess would be John, since while problems often include extraneous information, it is rare that the majority of the problem is extraneously included. Next, for a given pair of literals L1, L2, we prompt the LLM (1 (1lm_ limm__nnerttte e) to generate a right-hand-side literal Lright for L2 L.2 - Lright In doing so, the LLM might introduce new variables not previously involved in the problem. Details can be found in Appendix F22 We choose this forward-chaining approach rather than a goal-oriented nackwards-chaining for simplicity, since LLMs are much easier to prompt for forward-chaining (COT) than backwards chaining (recursive algorithms such as LAMBADA).
example in which six relations are known about John
Finally, for each generated Liigtt use the LLM (1im_commonse wow.onsenseeeeecooecooo.oosseee. _etpeeeeeeee_eeeeeeeeeeeeeee_eee to score whether L1 A Le + Lright is likely to be commonsense. Second, we use the LLM again [elm_relevance___mmmmm score) to score whether L2 A L2 + Liigh is likely to be relevant to our current context, Each procedure returns a probability between 0 and 1. Details can be found in Appendices F3 and F.4 respectively, and human evaluation in F.5
the LLM _ii__sccrree____ twice to evaluate it. First,
We stop the search at the first new proposition Ln L2 >> Lright whose commonsense and relevance scores are both above given threshold r (we use 7 = 0.3 in our experiments). When this happens, we update the commonsense set C with this new proposition, and restart the process. If not, running new iterations will not change anything and we fall back on our best guess, namely the self- colfscoonist estimate. In addition, if after multiple iterations the self-consistency threshold reaches zero, we also exit with the self-consistency estimate.
5

---

Published as a conference paper at ICLR 2026
Some animals tough winter out. They do not leave. They do not hide. They must survive. Sometimes nature helps them out. Some animals grow thicker coats in the winter. Other animals change color. The arctic fox is brown in the summer. His coat turns white in the winter. Query: The arctic fox's coat is white in the winter because white absorbs the sun and is warmer.
Legend
Query
New
: out(z, winter))
Backbone
brownl foz, summer) berns moaas_shhieeeee fon, winserr)
brown! foz, summer) turne_ wfrrua_kiitffoz, winter)
query: : (bhorba(white, sun) A warmer(cout) A turne_whste((ou winter)
Ziz : taugh__nnt( winter)
Backbone
brown! fox, sunamer) turns_ (iiassssseeee_eee fox, winter)
brown fox, aumaerr) whitel foz, winter) reflects (foz, sun)
p
query: : absorba whitee, sun) warmer((cout) A Murns_wwhteeffoe winter)
whitelJ winter) reflects (foz, sun)
3x : tough_.utii (2, winter) brownl faz, summer)
Backbone
brown!l oor, summer)) turna whitel fozz winter)) re reflectal ( fox, sun) "aasaobaa fox, sum)
serns__chiteffff foz, winter)
query: absorda (white, swn) warmer(cooat) turra_ whote winter)
vurns_wwitee[ fozz, winter) - reefla (foz, sum)
reflectol (fon, foz, -absarbel fooe, sun)
Ta tanugh_out(1,,, winter) brown! for, summerr)
Backbone
brown! for aumwerr) turna whitel ( foz, winter) re refecctsl ( forr, sun) -ubaorbaa farrrafff,, for, sun)
parns_whited I ooz, winter)
query: : abeorba whitte, sun) A warmer(coat) A surnes_ winter)
burns whiteefor,, winter) reflecte/fox, suns)
-absorbal while, sun)
reflectsl ffoz sun) - vabaorba(foz, sun) -absorba foz, sun)
whitelfoz, winter) - abeorrbbbttee sun)
Query is FALSE
Figure 4: Overview of ARGOS with the winter fox example. We iteratively add to the logic problem and query logic solver to look for conflicts within the backbone compared to the query. Eventually, we find that absor bs( bscorrsssettee,, sun) is False, contradicting the query.
4.2 EXAMPLE
Consider again the winter fox problem from the introduction section. Let us describe in Figure 4 hypothetical run of our ARGOS algorithm to illustrate how it could solve the problem. To simplify the illustration, let use only the SAT solver, and not self-consistency. We start with the premises (in black) and the query (in purple) on the top seft-hand-side.
We first the logic solver, which fails to reach any conclusion, but returns initial back- bone. The algorithm chooses the antecedents L, = L1 this backbone, generating a new proposition turns_white( It is commonsensical and relevant to the question, so we add it to the question. We call the SAT solver again, which adds reflects(for, sun) to the backbone. Next, the algorithm se- lects the antecedent L1 L2 reflects(foz: sun) from the new backbone and generates re reflects(fon sun) - -absorbs(foor sun), which is similarly commonsensical and relevant. The SAT solver is called again and adds -absorbs(for) sun) to the backbone. Finally, in the third itera- tion ARGOS picks L, -absorbs(sser sun) and L2 turrtsssuo carrssshhittefoss winter) from the back- bone and generates -absorbs (foz, swn) A turns [arrtssteee_______ (fote winler) -absorbs( (while, sun), which is a logical conclusion deems consistent with commonsense and relevant to the question. At this point, we call the SAT solver again, which concludes that mabsorbs( babsorbs(whhite, (whête, sun) is true and therefore that the query must be false, returning P A C F -Q as conclusion.
jurns_whüte( for, winder) from winter) > re reflects(ser fox, sun).

---

Published as a conference paper at ICLR 2026
(a)
(b)
35
flipped correct (n=112)
0.8
SC
flipped incorrect (n- 35)
ARGOS
Propartion Correct
25
0.6
Problems
20
0.4
15
0.2
0
0.0
3-4
5-6
7-8
9-10
2
10
Argos Iterations
ARGOS Iterations
Figure 5: (a) The number of CLUTRR problems for which ARGOS flips SC predictions correctly and incorrectly. (b) SC and ARGOS accuracies on CLUTRR subsets, partitioned by the number of ARGOS iterations each datapoint receives.
5 EXPERIMENTS
Models We employ Llama3-8B (L8B), Llama3-70B (L700), and Mistral 7B (M7B) as LLMs. Our method is dependent access logit-level outputs, closed-source models excluded. 2
Benchmarks Unfortunately, there are few natural language reasoning datasets that are strongly logically-structured and commonsense-abduee commonsense-basee logic problems, data transformations to introduce the need for abductions are typically achievable. For list of common datasets which have proven unsuitable for our setting, and corresponding explanation, see Appendix [1 For our experiments, we use abductive versions of ProntoQA (Saparov and He 2023). CLUTRR (Sinha et al. 2019), and FOLIO (Han et al. 2024). CLUTRR is not originally True/False, but is multiple-choice. We modify it to be True/False output by making the question randomly either ask the correct or an incorrect choice is True. While these datasets are better described with first-order logic, render them propositional by unrolling their quantified formulas over all instantiated terms. While strongly logical and therefore obvious choices, these datasets are representative of real-world application generalizability of our method. To test our method's generalizability as well as its broader applicability to real use-cases, we also include some datasets that are not strictly logical. CosmosQA (Huang al. 2019) and QUAIL (Rogers et al.) (2200) are reading comprehension MCQA datasets. Reading comprehension key for general summarization and interactive QA tasks, which are certainly common LLM use-case in practice. ESNLI (Camburu et al. 2018) is short-form natural-anggaage-infeeeen Each these datasets requíres some form reasoning, but structure of both the text and the necessary reasoning is generally fuzzy, requiring subjective interpretation. For the MCQA datasets, we process them into True/False questions similarly to how it was done for CLUTRR. We note that ProntoQA, CosmosQA and ESNLI performances already saturated by self-consistency. Despite this, the results are valuable as they demonstrate that on these apparently simple tasks ARGOS is able compare with purely neural methods, avoiding the performance collapse that more symbolic methods encounter. For few-shot examples, we randomly remove four problems from each dataset and annotate them with COTs, using the same four examples and annotations for each method. For more dataset details, please see Appendix H
However, given dataset of classical
dataset.
Evaluation We compare against COT Wei et al., 2022), Self-Consistency Wang et al. 2023 SAT-LM (Ye et al. 2023). Logic-of-Thoughtt (Liu al.) 2024) and LLM-Tres Toroghi et al. 2024). For a fair comparison with 20-shot self-connistency and LOT. we set ARGOS's s hyperparameters such that it makes more than 20 COT calls per problem average. Details provided in Appendix abduction-modified evaluation sets and report results in Table
We report accuracy on the abduction modified
"Experiments are cach conducted on I or 2 NVIDIA Tesla V100 GPUs, depending on the LLM's GPU memory requirement. As logic solver, Cadical (Birre
2024

---

Published as a conference paper at ICLR 2026
5.1 RESULTS AND DISCUSSION
As can be seen in Table 1 ARGOS provides significant performance improvements over existing methods (up +13%). Of the datasets, FOLIO is the most representative of human-generated logical reasoning problems. ARGOS outperforms the baselines for FOLIO, improving performance by 3-10%. For more structured problems (CLUTRR), the symbolic components of ARGOS become more reliable, and we see more consistent gains of 6-8%. On QUAIL, highly ambiguous dataset that also formatted in strange ways due it being constructed by scraping forums and wikis, ARGOS improves compared to self-consistency by up to 13%, demonstrating its ability to adapt to even non-logical contexts. On ProntoQA, ESNLI and CosmosQA, despite the very competitive neural baseline performances, ARGOS performs comparably. Symbolic baselines (SAT-LM, LoT-20, LLM-Tres) see large performance gaps, at times being reduced to guessing. SAT-LM, despite the fact that some datasets strongly logically structured and that filter out mis translanddd problems, still can not answer problems. Even in the best case, it is impossible for purely symbolic methods to handle realistic reasoning scenarios. LLM-TTres, despite having abductive capabilities, is restricted in its abduction space that it it almost never capable of identifying the necessary rules to solve CLUTRR or ESNLI problems.
Binary classification accuracy (True/False) of all methods on the datasets, using the chosen language models. Bolded text indicates that the method has the best performance, and that its performance is better than the next-best-perforee berf-perfon method in a statistically significant way (p-value < 0.005 according to a Wilcoxon pair-wise rank test). Small-font numbers to the right indicate the bounds of the 95% confidence interval, derived via bootstrap approach.
Table 1:
FOLIO
CLUTRR
PQA
M7B
L8B
L70B
M7B
L8B
L70B
M7B
L8B
L77B
SC20
66%
66.4
71%
71.7
77%
77.7
59%
59.3
69%
69.5
69%
69.4
97%
97.2
95%
95.6
93%
94.1
75.9
68.8
92.4
COT
66%
68%
72%
72.5
59%
68%
66%
66.3
82%
82.9
90%
91.2
93%
94.1
65.5
67.2
71.8
58.8
67.8
65.6
81.7
89.6
92.4
SAT-LM
43.2 43%
43%
43.2
43.2 43%
50.4 50%
50%
50.4
50%
50.4
50.3 50%
50%
50.3
50%
50.3
42.8
42.8
42.8
49.9
49.9
49.9
49.8
49.8
49.8
LoT-20
57%
57.3
69%
69.5
70%
70.4
71%
71.6
70%
70.2
69%
69.3
88%
88.4
97%
98.2
95%
95.7
68.7
96.3
94.3
LLM-Tres
66%
63%
63.2 63%
51%
51 %
51.6
53%
80%
83%
83.8
76%
65.9
62.4
62.4
50.8
50.8
52.8
79.2
82.3
75.2
ARGOS
70%
70.6
81 %
81.8
80 %
80.5
78 %
78.4
76 %
76.3 75.8
78 %
78.2
98 %
98.7
98.2 97%
97%
98.1
69.8
80.0
78,8
77.7
77.7
97.9
96.3
96.2
CosmosQA
ESNLI
QUAIL
M7B
L8B
L70B
M7B
L8B
L70B
M7B
L8B
L70B
SC20
84%
81%
90%
97
96 %
99%
70%
68%
69.1
75%
82.9
79.7
89.9
97.1
96.3
99.2
67.9
65.6
72.4
COT
81%
81.4
76%
77.5
88%
88.7
96%
95.8
88%
86.9
99%
98.9
71%
71.9
65%
66.2
75%
75.6
SAT-LM
37.5 35%
35%
37.5
35%
37.5
49%
50.0
49%
50.0
49%
50.0
53%
$5.0
53%
55.0
53%
55.0
34.8
34.8
34.8
47.7
47.7
47.7
51.6
51.6
51.6
LoT-20
77% 76.8
75%
85%
85.7
71%
77.5 76%
75%
76.1
62%
56%
$7.1
72%
74.2
84.3
70.4
75.7
74.4
59.9
53.8
69.9
LLM-Tres
73% 72.1
72%
72.7
71 5%
71.5
51 %
52.5
51 %
52.5
51% %
$2.5
63% 65.9
60% 62.1
58%
60.1
69.8
50.8
57.1
ARGOS
84.3 84%
83 %
84.0
90%
90.7
95% 95.6
96%
96.2
98%
98.0
82 %
83.4
82 %
83.4
80%
$1.8
82.7
82.6
89.4
94.9
95.5
97.4
80.6
80.7
78.5
How useful are the scoring and backbone-tracking elements? In Table 2 we test the importance of two elements of ARGOS: (i) score thresholding and (ii) backbone computation. The ablation of each element in isolation results in decrease in performance. In addition, the ablation of both results in a larger performance drop than even the sum the two single ablations' decreases. The fully-ablated method, however, still shows strong performance relative the next strongest
RQ1:
8

---

Published as a conference paper at ICLR 2026
baseline (SC-20), highlighting the strength of the general concept behind the method. For further ablations, see Appendix
RQ2: How often are ARGOS' added clauses useful or harmful? An important criterion when adding clauses is that they do not corrupt the logic of the problem, undesirably changing the outcome of the logic. can be shown (see Appendix A that so long as clauses are commonsensical, their addition will not corrupt the problem. However, it is possible that our method adds non- commonsensical clauses, since the commonsense scoring is not perfectly reliable. Given CLUTRR's strict structure, since we know the full knowledge base from which it was constructed, we can re-construct the full problems and test if ARGOS' added clauses corrupt the logical arithmetic such that a different answer is found for the logic problem. We find that on CLUTRR, ARGOS never corrupts a problem. It is then not surprising that ARGOS sees significant performance gains: added information should in principle never negatively affect wholly rational reasoner's solution and so performance should only improve. It is also, of course. important that the added clauses contribute to the (correct) solution of the problem. In order to identify what information is important to the solution of the problem, we add the full relational reasoning rules to the SAT problem representing each CLUTRR example. We then extract the proof, taking all variables mentioned in the proof as important to solving the problem. We can then measure the number of problems for which at least one new variable is added, which is important to the proof. We find that ARGOS, on Llama 8B, identifies important new variables for 65% of the CLUTRR questions we test on.
RQ3: Does ARGOS attribute more compute to harder problems? How does this affect the solution of harder problems? In Figure (b). we examine the proportion of CLUTRR problems that are solved correctly by SC and ARGOS, over subsets of the dataset grouped by the number of ARGO iterations before termination. The error bars are 5/95% confidence intervals. As the number of ARGOS iterations increases, the problems become harder for SC to solve (indicated by lower proportion of correct solutions by SC), This tells us that ARGOS' method of evaluating solvability is working as intended; harder problems are being assigned more computation. Another interpretation of this result is that problems which have more missing information, or for which the missing information is more difficult to infer, are attributed more ARGOS iterations (in order for ARGOS find the necessary information). This is supported by the fact that the decrease in proportion seen in SC is not present for ARGOS: if SC's performance is dropping due to missing information, then ARGOS successfully recovering the necessary missing information.
This ability to address the obstacles which cause SC performance to drop contribute to large number of answers being flipped from incorrect (when solved by self-consistency) to correct (when solved by ARGOS). Changes to the answers caused by new information are more often than not in the right direction. On CLUTRR L70B, we find 112 correct and 35 incorrect flips. Figure 5 (a) shows the number of correct and incorrect flips ARGOS achieves. As the number of ARGOS iterations increases, both the correct and incorrect flip counts increase, but the correct flip counts increase much faster. For closer look at confidence-score vs. iteration behavior, Appendix K
provideess an example of a question from CLUTRR that is misclassified by self-consistency but flipped to correct by ARGOS. The COT seems confused, displaying its characteristic inability to plan out a proof: in steps 1-3 it provides disjoint pieces of information that neither follow from each other nor move towards the target conclusion. This confusion eventually leads an incorrect step: "Shantel is Laura's 's aunt", resulting to an incorrect conclusion. ARGOS, after 3 iterations, provides several pieces of key information which would require least one additional reasoning step to find, halving the necessary chain-length. For some examples in which ARGOS fails, see Appendix ] For results, evaluated by a human, on ARGOS and COT faithfulness on FOLIO, see Appendix
5.2 IMPACT OF IMPERFECT LOGICAL TRANSLATION
Here, we test if the assumption of perfect logical translation make in our experimental procedure is justified via empirical result. In our experiments, we assumed that we started from a propositional logic formulation. Some datasets came with an official formulation, while for the others we translated from text using Claude Opus 4, filtering to remove failed translations. This was done to fairly evaluate the methods on abductive reasoning, regardless of the quality of translation. In general, logical translation is kept as separate module to the proof planning/execution module in logical reasoning

---

Published as a conference paper at ICLR 2026
Context:
Antonio is the father of Laura.
Table 2: Ablations. We ablate ele- ments of ARGOS: (i) the score thresh- olding, taking the first clause sampled each iteration (ARGOS No T), (ii) the backbone-tracking, generating prompts by randomly selecting two variables (ARGOS No BB).
Pedro is the son of Harold. of Harold.
Antonio is the
Shantel is the mother of Pedro.
Question: Is Shantel Laura's grandmother?
Label True
COT: Let's think step by step. Shantel Pedro's mother. Antonio Harold's Antonio Laura's father. Shantel Laura's aunt. Therefore, the answer to the question is No
enong
FOLIO L8B
SC-20
71.7 71% 70.1
ARGOS No T
79% 78.5
79.8
ARGOS:
Here Antonio the brother of Pedro.
additional facts
ARGOS No BB
79%
78.4
79.4
Harold the husband of Shantel. Let's think step by step.
Shantel the mother of Antonio.
ARGOS
ARGOS
Both
76%
75.2% 77.2%
Antonio Shantel's Laura Antonio's daughter.
ARGOS
81 %
81.8
Shantel Laura's grandmother.
80.0
Therefore, the answer the question is Yes
Figure 6: COT vs ARGOS on a CLUTRR problem.
systems. Also, the translation task intrinsically simpler for LLMs, since linguistic rather than cogmitive. LLMs have already demonstrated strong abilities at logic translation (Yang et al. 2024). and are expected continue improving faster than at reasoning. To validate this claim in our context, we re-tested ARGOS with Llama 8B on FOLIO using a translator, but including failed translations. Performance only decreased marginally, from 80% to 78%, still outperforming the next best method (SC at 71%). On QUAIL, ARGOS performance dropped from 82% to 73%, which while large relative to the drop on FOLIO still keeps ARGOS as the best performing method on QUAIL.
CONCLUSION
have presented a method for addressing realistic natural-language logic problems, where "realistic" entails a need for abduction and commonsense. Whether neural symbolic, we demonstrate empirically that existing methods struggle in this setting. The method we present addresses this weakness by (a) balancing neural and symbolic elements and allowing them speak to each-other; and (b) avoiding the commonplace design choice of heavily restricting the abduction-clauss search space. On both general and highly structured logic problems, a balanced neuro-symbolic approach, outperforming all existing work meaningfully.
method demonstrates the power of
Limitations A limitation of our work is that it is currently restricted to problems which strictly True or False, eliminating cases where logic might be used to select an option from list of choices, or cases where the correct answer is "Maybe". In our experimental work, we addressed the consequences this had on dataset selection by converting datasets to be True/False. The method could however be extended to multiple-choic questions by asking each question as an individual True/False question, combined with a decision heuristic for when no/multiple choices are determined True. Another limitation is that we restrict ARGOS to generating rules with up to two literals in the antecedent. While mate-literal propositional formulas can often be decomposed into smaller ones, this may not always be the case and an ideal method would allow for large-antecedent generation. Thirdly, while our goal was to develop methods for open-source LLMs, the method would be more easily applicable if it did not require logit-level access. A potential future direction might be to convert the scoring system from logit-based one verbalized score. Additionally, most benchmarks employed were modified to our setting, making the evaluated tasks at times artificial. Also. ARGOS sometimes depends upon self-consistency for problem solution. So, there times when unfaithful or hallucinatory COT' will impact ARGOSs final prediction. Finally, this work focuses on forward chaining. A future direction may be backwards-chaining approaches abductive reasoning.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
In the supplementary material, we provide our full code which was used to implement and benchmark our method as well the baselines. The code also includes data processing steps. We took great care to include in the Appendix, as well, detailed descriptions of our algorithm and our prompts. While our human modification of FOLIO text is not provided, the process for generating it is described carefully in the Appendix.
REFERENCES
Armin Biere, Tobias Faller, Katalin Fazekas, Mathias Fleury, Nils Froleyks, and Florian Pollitt. 2024.
CaDiCal 2.0. In Proc. Int. Conf. on Computer Aided Verification. Springer, 133-152.
Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. 2018. e-SNLI: Natural Language Inference with Natural Language Explanations. In Proc. Conf. Neural Informa- tion Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa- Bianchi, and R. Garnett (Eds.), Vol. 31. Curran Associates, Inc.
Peter Clark, Oyvind Tafjord, and Kyle Richardson. 2021. Transformers soft reasoners over language. In Proc. Int. Joint Conf. on Artificial Intelligence. 3882-3890.
Daniel Crevier. 1993. Al: The Tumultuous Search for Artificial Intelligence. Basic Books, Chapter 6: The Tree of Knowledge.
Nouha Dziri, Ximing Lu, Melanic Sclar, Xiang (Lorraine) Li, Liwei Jiang, Bill Yuchen Lin, Sean Welleck, Peter West, Chandra Bhagavatula, Ronan Le Bras, Jena Hwang, Soumya Sanyal, Xi- ang Ren, Allyson Ettinger, Zaid Harchaoui, and Yejin Choi. 2023. Faith and Fate: Limits of Transformers on Compositionality, In Proc. Conf. Neural Information Processing Systems, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.), Vol. 36. Curran Associates, Inc., 70293-70332.
Simeng Han, Hailey Schoelkopf, Yilun Zhao, Zhenting Qi, Martin Riddell, Wenfei Zhou, James Coady, David Peng. Yujie Qiao, Luke Benson, Lucy Sun, Alexander Wardle-Solano, Hannah Szabó, Ekaterina Zubova, Matthew Burtell, Jonathan Fan, Yixin Liu, Brian Wong, Malcolm Sailor, Ansong Ni, Linyong Nan, Jungo Kasai, Tao Yu, Rui Zhang, Alexander Fabbri, Wojeiech Maciej Kryscinski, Semih Yavuz, Ye Liu, Xi Victoria Lin, Shafiq Joty, Yingbo Zhou, Caiming Xiong, Rex Ying, Arman Cohan, and Dragomir Radev. 2024. FOLIO: Natural Language Reasoning with First-Order Logic. In Proc Conf. on Empirical Methods in Natural Language Processing. 22017-22031.
Lifu Huang, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2019. Cosmos QA Machine Reading Comprehension with Contextual Commonsense Reasoning. In Proc. Conf. Empirical Methods in Natural Language Processing and Int. Joint Conf. on Natural Language Processing (EMNLP-IJCNIP), Hong Kong, China, 2391-2401.
Mchran Kazemi, Najoung Kim, Deepti Bhatia, Xin Xu, and Deepak Ramachandran. 2023. LAM- BADA: Backward Chaining for Automated Reasoning in Natural Language. In Proc. Conf. Associ- ation for Computational Linguistics. 6547-6568.
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. In Proc. Conf. Neural Informations Processing Systems. 22199-22213.
Jinu Lee and Wonseok Hwang. 2024. SymBa: Symbolic Backward Chaining for Structured Natural Language ReasoningSymBa: Symbolic Backward Chaining for Structured Natural Language Reasoning. arXiv:2402. 12806 [cs.CL]
Jian Liu, Leyang Cui, Hanmeng Liu, Dandan Huang, Yile Wang, and Yue Zhang. 2021. LogiQA: a challenge dataset for machine reading comprehension with logical reasoning. In Proc. Int. Joint Conf. on Artificial Intelligence. 3622-3628.
11

---

Published as a conference paper at ICLR 2026
Tongxuan Liu, Wenjiang Xu, Weizhe Huang, Yuting Zeng, Jiaxing Wang, Xingyu Wang. Hailong Yang, and Jing Li. 2024. ofgic-of-thought Injecting logic into contexts for full reasoning in large language models. In arXiv preprint ar arXiv:2409.177 17539.
Qing Lyu, Shreya Havaldar, Adam Stein, Li Zhang, Delip Rao, Eric Wong, Marianna Apidianaki. and Chris Callison-Burche 2023. Faithful Chain-of-Thought Reasoning. In Int. Joint Conf. Natu- ral Language Processing and Conf. Asia-Pacific Chapter of the Association for Computational Linguistics.
Nils J. Nilsson. 1991. Logic and artificial intelligence. Artificial Intelligence 47, (1991), 31-56.
Theo Olausson, Alex Gu, Ben Lipkin, Cedegao Zhang, Armando Solar-Lezama, Joshua Tenenbaum, and Roger Levy. 2023. LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Provers. In Proc. Conf. on Empirical Methods in Natural Language Processing. 5153-5176.
Liangming Pan, Alon Albalak, Xinyi Wang. and William Wang. 2023. Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning. In Findings of the Association for Computational Linguistics, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). 3806-3824.
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. Language Models as Knowledge Bases?. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (Eds.). Association for Computational Linguistics, Hong Kong, China, 2463-2473.
Aske Plaat, Annie Wong. Suzan Verberne, Joost Broekens, Niki van Stein, and Thomas Back. 2024. Reasoning with large language models, a survey. In arXiv preprint arXiv:240..11511.
Anna Rogers, Olga Kovaleva, Matthew Downey, and Anna Rumshisky. 2020. Getting closer to Al complete question answering: A set of prerequisite real tasks. In Proc. AAAI Conf. Artificial Intelligence, Vol. 34. 8722-8731.
Abulhair Saparov and He He. 2023. Language Models Are Greedy Reasoners: Systematic Formal Analysis of Chain-of-Thought. In Proc. Int. Conf. Learning Representations.
Koustuv Sinha, Shagun Sodhani, Jin Dong, Joelle Pineau, and William I., Hamilton, 2019. CLUTRR: Diagnostic Benchmark for Inductive Reasoning from Text. In Proc. Conf. Empirical Methods of Natural Language Processing.
Oyvind Tafjord, Bhavana Dalvi, and Peter Clark. 2021. ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Language. In Proc. Conf. Association for Computational Linguistics: ACI-IICNIP.. 3621-3634.
Jidong Tian, Yitian Li, Wenging Chen, Liqiang Xiao, Hao He, and Yaohui Jin. 2021. Diagnosing the first-order logical reasoning ability through LoI.cNLI. In Proc. Conf. Empirical Methods in Natural Language Processing. 3738-3747.
Armin Toroghi, Willis Guo, Ali Pesaranghader, and Scott Sanner. 2024. Verifiable, Debuggable, and Repairable Commonsense Logical Reasoning via LLM-based Theory Resolution. In Proc. Conf. on Empirical Methods in Natural Language Processing. 6634-6652.
Siyuan Wang, Wanjun Zhong, Duyu Tang, Zhongyu Wei, Zhihao Fan, Daxin Jiang, Ming Zhou, and Nan Duan. 2022. Logic-Driven Context Extension and Data Augmentation for Logical Reasoning of Text. In Proc. Conf. Association for Computational Linguistics, Smaranda Muresan, Preslay Nakov, and Aline Villavicencio (Eds.). 1619-1629.
Xuezhi Wang. Jason Wei, Dale Schuurmans, Quoe V Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-Consistency Improves Chain of Thought Reasoning in Language Models. In Proc. Int. Conf. on Learning Representations.
12

---

Published as a conference paper at ICLR 2026
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc Le, Denny Zhou, et al. 2022. Ohain-of-though prompting elicits reasoning in large language models. In Proc. Conf. Neural Information Processing Systems. 24824-24837,
Jundong Xu, Hao Fei, Liangming Pan, Qian Liu, Mong-Li Lee, and Wynne Hsu. 2024. Faithful Logical Reasoning via Symbolic Chain-of-Thought. In Proc. Conf. Association for Computational Linguisties.. 13365.
Shangzi Xue, Zhenya Huang, Jiayu Liu, Xin Lin, Yuting Ning, Binbin Jin, Xin Li, and Qi Liu. 2024. Decompose, analyze and rethink: Solving intricate problems with human-like reasoning cycle. In Proc. Conf. Neural Information Prucessing Systems. 357-385.
Yuan Yang, Siheng Xiong, Ali Payani, Ehsan Shareghi, and Faramarz Fekri. 2024. Harnessing the Power of Large Language Models for Natural Language First-Order Logic Translation. In Anmual Meeting of the Association for Computational Linguistics 2024. Association for Computational Linguisties (ACL), 6942-6959.
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. In Proc. Conf. Neural Information Processing Systems. 11809-11822.
Xi Ye, Qiaochu Chen, Isil Dillig, and Greg Durrett. 2023, SatLM: Satisfiabilit- Language Models Using Declarative Prompting. In Proc. Conf. Neural Information Processing Systems.
Weihao Yu, Zihang Jiang, Yanfei Dong, and Jiashi Feng. 2020. ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning. In Proc. Int. Conf. on Learning Representations.
13

---

Published as a conference paper at ICLR 2026
A ABDUCTIVE LOGIC PROBLEMS ARE WELL-DEFINED
In this section we prove that the solution of an abductive propositional logic problem, given in Section does not depend on the choice of commonsense set C.
Proposition 1. Let P be a set of premises, Q a query proposition, and C1, C2 subsets from com- monsense set C of additional propositions such that (P C1) L1 and (P A C2) H L2 for literals L2 E {Q, [Q] IfP is consistent with C (P A C of 1) then L1 ++ L2.
Proof. Let's say L1 J/ Ly: without loss of generality we can take (P (PAC) and F
So. A Ci A C2) + (Q -Q) L. But C1, C2 C, so therefore (P A C) + [C| C, U C2)), so P P ++ + LA UC2)], so P PACH- This contradicts our assumption
that P PACYL.
B COST DISCUSSION
While in theory COT generation is meant to be done until answer is found, in practice it is necessary that an upper-limit on number of tokens generated is enforced. This is in case (a) the LLM continues generating past its answer, (b) the LLM goes off-track and never answers the question. In any case, this means that each COT generation will be, at worst-case porst-case-assumpae assumption, equal cost. In addition, the various method -peccifi LLM generations that are employed require small token-ling limits relative to COT, and so the number of COT calls made dominates the total number of tokens generated by any method. Additionally, as problems get harder and more logically complex, necessary COT generation length increases, making this even more true. So, we can say that cost for each method scales in proportion to the number of COTs generated. For example, Colf-Consistency takes two hours longer to run than AROGS on FOLIO with Llama 8B, despite requiring more total LLM calls (where we include scoring and geera-gennraiton calls in our count). This is because the average number of COT- calls is lower for ARGOS than SC, and the scoring and literal generation calls are much shorter than COT calls. In our implementation, we generate at most 25 tokens for ftteralggeneratonn, generation. Given this, budgeting method costs in terms of COT calls is well-justified. For SC and COT, the cost evaluation is trivial: COT always makes 1 COT call and SC makes fixed number of COT calls, specified as parammerr.ettr.. Similarly, LOT makes some small generative calls followed by SC, so its number of COT calls is fixable. LLM- Tres makes no COT calls, and neither does SAT-LM. ARGOS' cost varies according to the entry, but its syper-parameter (number of COT calls per-iteration and thresholddanneelingg constants) can set such that its average number (or worst-case) number of calls is less than budget. A summary of method cost in terms of COT calls is provided in Table 3 In Figure 7 we show a histogram of the individual problem costs incurred by ARGOS with Llama 8B, expressed in terms of the number of COTs required per problem. For most problems, only 10 COTs are required. This allows ARGOS to exceed the on-average cap for problems requiring more ablation deeper search. If compose new dataset of only the hardest. problems, for example the CLUTRR problems for which ARGOS takes 8-10 ARGOS steps (the final bar in Figure 5(b).. Testing ARGOS, limited at 20 COTs per-problem (strictly, not average), we see that its performance drops from 65% to 54% on these problems, whereas SC performs at 40S with these problems. This indicates clear and steep cost cost-performanc tradeoff for ARGOS, but even with strict cost limitations ARGOS outperforms SC.
token maximum for scoring, and 300 tokens maximum for COT
14

---

Published as a conference paper at ICLR 2026
Table 3: Average number of COT calls required by each method.
Cost (Avg # COT)
COT
1
SC
20
LOT
20
SAT-LM
0
LLM-Tres
0
ARGOS
18.4
CLUTRR cost histogram
600
500
count
300
200
100
10
15
20
25
30
35
COTs
Figure Histogram of individual problem costs ARGOS-Llama 8B on CLUTRR
C MANUAL HUMAN FAITHFULNES: EVALUATION
For COT, we manually checked the chains which result in the correct final answer, generated with Llama 8B. For ARGOS, if ARGOS made the correct final prediction using the SC output, will check one of the chains from the final SC call. If ARGOS made the final prediction using the symbolic solver, the reasoning is considered correct if none of the added clauses are incorrect (the reasoning itself must be beyond reproach as it is executed by solver). Additionally, for ARGOS, we evaluate the usefulness of the added information in solving the problem, and whether the added information new to the problem. We find that Llama 8B generates faithful COT reasoning processes when it gets the answer right 72% of the time. We find that ARGOS-L8B generates faithful reasoning process when it gets the answer right 85% of the time, showing that in general ARGOS-L8B is more faithful than pure-COr COT based methods (i.e. COT, SC). Three potential explanations for this result, order from least to most in terms of their strength in justifying our method, are that (1) The logical content of ARGOS's s augmented prompt, regardless of content, incites a more logical structure in the LLM generation. (2) ARGOS successfully extracts the key elements of the problem, stabilizing the LLM and making it less likely to become unfaithful or to hallucinate. (3) ARGOS adds new information which allows
solve otherwise difficult problems. Empirically, we find that ARGOS-L8B adds at least one piece of information necessary for the generating the faithful proof 85% of the time. This seems lend credibility to explanation (2). Additionally, ARGOS adds new information the text-problem 72% of the time, which lends some credibility to explanation (3), which would strongly justify the extend of our method, whose goal is explicitly stated as searching for new and crucial commonsense. Most interestingly, information added by ARGOS is faithful, useful and novel. This is perhaps not surprising, since to achieve the contrary, Nonetheless, an extremely satisfying finding, as it shows that when we are able to avoid typical symbolic robustness issues such symbol mismatch, ARGOS maintains the rigour characteristic of symbolic methods.
find that 100% of the time in which we solve symbolically, the
would have to either add exactly contradictory information to the true proof.
15

---

Published as a conference paper at ICLR 2026
D ALGORITHM DESCRIPTION
In this section we provide a detailed description of our ARGOS algorithm. The main procedure is summarized Algorithm which uses the _inn__new_commeeeee comeonsnneeeee subroutine in Algorithm
Algorithm ARGOS
Require: premises P, query Q, SC sample-count k; scoring threshold T E (0, 1].
self-consistency threshold y E (0, 1] and decay a (0,
1: commonsense set C
0
2: while y 0 do
3:
// Attempt solving with the SAT solver
4: sat_ _ot_conclusion, backbone B t sat sat_solve(P) ACF Q, -Q)
5: if sat woc.clunnnoonnnc is (P A C) + Q (PAC) then return C, sat_conclusion
6:
7: end if
8: // Else attempt solving with the LLM (k-shot self-consistency)
9: Ilm conclusion, 1lm confidence 1 11m solve AC+ Q, -Q)
10:
if Ilm confidence then return C, Ilm_conclusion end if
12:
13:
Else find a new commonsense proposition to add to the pool C' find serdd_ew_weee wtr__ooooonsssee(P.....
14:
15:
if C' is not None then
16:
/1 New commonsense has been found, try again with enlarged C and smaller - (C) {C} 7 7-
17:
18:
19:
else
We failed, return best guess
21: return C, IIm_conclusion
22:
end
23:
end while
24: // ran out of time, return best guess
25: return C, IIm_conclusion
Algorithm 2 Findd_eww_commonee
Require: premises P, commonsense C, backbone B backbone A C), scoring threshold т 10.1]
1: for LI E B from highest to lowest scores(L1) do
2: for L2 E from highest to lowest scoreg L2) do
3:
for Light in 11m generatteeeeea
Le +?) do commonsense scrrrensessesccrreeeeee f- 11m wwwmonsense_ (coreeL A L2 + Light) relevance_scone score 1 1m_reell commonsense__eee scoreeecreee > T and relevance scoreeee_sereeeeeee > then // We found
4:
5:
L1 T-2 Tright)
6:
7:
relevant commonsense clause
8:
return L1 A L2 + Leight
end if 10: end for
9:
end for
12:
end for
// We failed find a new relevant commonsense clause
14: return None
16

---

Published as a conference paper at ICLR 2026
E ABLATIONS
FOLIO 8B
CLUTRR SB
ARGOS-SS ARGOS-Symbolit ARGOS
59%
72%
81%
76%
SC
71%
69%
LLM-Tres
63%
51%
Table 4: Ablating the SC-solver on ARGOS. ARGOS-Symbolic denotes the ablated version of ARGOS.
In Table 4 ablate the SC-solver, solving problems with only symbolic solver. We impose a threshold of 100 proposed rules, since the previous bounding system was in terms of SC confi- dence.We find that removing the SC option only somewhat hurts ARGOS performance on CLUTRR, but has a catastrophic effect on FOLIO. This is not at all surprising, since CLUTRR is far simpler and more logically structured than FOLIO. Interestingly, we find that LLM-Tre: performs slightly better than ARGOS-Syymond
on FOLIO. On investigation, we found that propositions with single-litens antecedent were generally enough to solve FOLIO problems, and so LLM-Tres' methodological restriction to such rules benefits it, whereas in CLUTRR where nearly all necessary propositions have two literals the antecedent, LLM-Trres reduced guessing. This massive performance gap which we see ARGOS in the more linguistic and more logical datasets illustrates the need for balanced, ncuro-symbolic approaches.
FOLIO 8B
No Commonsense 79%
No Contextual
80%
Full AROGS 81%
Table 5: Ablating individual score thresholds
In T we ablate separately the commonsense and contextualness scoring components. We find that both necessary. This makes sense, since while they may have some overlap in terms of the rules which they serve to reject (for example. gibberish output would be neither commonsensical nor contextually relevant), is easily conceivable that the sets of proposed propositions which they reject correctly is not totally overlapping.
F ALGORITHM LLM DETAILS
In this section we provide details about the parts of the algorithm that involve interactions with the large language model.
F.1 SOLVING WITH 5- S-ROUND
This subroutine aims to establish whether the premises and commonsense entails the query, i.e. (P F Q or (P A C)= + --Q. This routine always returns answer, but can make mistakes. We few-shot prompt the LLM times with the prompt Table
Table 6: COT prompt
Here are some facts and rules: [premises P]
Here is some additional info we found: [commonsense C)
True or false: [query Q1?
Answer:
Each call returns an answer a,,..., a5 E [True, False), with a certain confidence CI...., O5 (0, 1]. The algorithm returns the most common answer (True False), and total confidence score given by
17

---

Published as a conference paper at ICLR 2026
the sum of confidences of the most common answer a", divided by 5:
5
Σ c a
F.2 GENERATING NEW COMMONSENSE LITERALS LRIGHT (LIM_GENERATE)
This subroutine aims find plausible hight-hand-iiee literal Light for a proposition L1ALz Lightt The new literal might potentially involve new variables not previously seen in the problem. We use a slightly different prompt for CLUTRR and the others, because of CLUTRR's more distinct structure. Anecdotally, we found that CLUTRR's 's more consistent linguistic structure allowed for prompt formats which were very straightforward. For more linguistically complex datasets, however, more robust prompt formatting required. For example, asking if rule seems contradictory was more robust in situations where rule was ambiguous without context (i.c. student(Rina) implics like_ (ik__coffeeRRina), Mom(Hannah, Sam) and Sibling(Sam,Mann) implies Mom(Hanna, Mary)).
Table 7: Clause generation prompt for all datasets but CLUTRR. is entity appearing in Lı or L2-
Fill in the blank with known predi cate: [T A 12] implies les ([e]).
Known predicates are: [all predicates appearing in the premises and the commonsense C)
Answer:
Table 8: Clause generation prompt for CLUTRR. and are entities appearing in L1 or L2.
If [L A L2] then ([e1], [e2]). . Fill in the blank.
Answer:
Thus, for example, in FOLIO if Lı = drinksCoffeel (Rin)) and L22 = Loves(Marg) Sam), then we would make three calls, one for (afssCefffeeeenno A Loves( [M ary, Sam) F(Rina), (innssoofeee(Rima) A Loves(Mars Sam) + F(Mary) and drinksCoffee Rina) A Loves Loves(Mary (Mary, Sam) > F(Sam) respectively. With such calls, the method might return the set (productive (Rina), hasF Feelings M Mary), (LLove((Sam) for example.
F.3 SCORING PROPOSITIONS L1 A L2 + LRIGHT FOR COMMONSENSE [LLN_COMKONSENNE_
This procedure uses the LLM to score how much our new proposition L1 A L2 + Leight likely to be commonsense. In detail, we ask the LLM whether, without any context, the rule seems contradictory (FOLIO/ProntoQA) "Yes" and "No" tokens following the prompt, and we return as commonsense score P[No] = exp(logit except for CLUTRR where we return P Yes) exp (ogglogity... (/ (exp(((ogit)) + exp(logit since the question is inverted.
true (CLUTRR). We record the logits of the
Table 9: Commonsense scoring prompt for FOLIO/ProntoQA.
Does the following rule seem contradictory?
Rule: [I, A L2 Leight]
Answer:
F.4 SCORING PROPOSITIONS L, A L2 - LRIGHT FOR CONTEXT-RELEVANCE (LIMLEELRVANCE_SCOM)
This procedure uses the LLM to score how much our new proposition L, A L2 - Liihtt is likely to be relevant to the context. This helps eliminate propositions, like "The sky is blue". that are
18

---

Published as a conference paper at ICLR 2026
Table 10: Commonsense scoring prompt for CLUTRR.
Does the following rule seem true?
Rule: [L1 A Le + ] Answer:
true and commonsense but unlikely to help prove our query. In this case, we use the same prompt for all datasets. We record the logits of the tokens "Yes" and "No" following the text, and return P[Yes] (/ ((xpp(loixx
exp(logit as relevance score.
Table 11: Context scoring prompt for all datasets.
Here are some facts and rules: [premises P and commonsense C] Does the following new rule seem contextually relevant to the facts and rules? [L A L2 [right] Answer:
F.5 MANUAL EVALUATION OF SCORING MODULE
We have conducted human review of 50 ARGOS- proppose rules and their corresponding scores, for Llama 8B on FOLIO. We found that, by classifying with the decision rule of thresholding at 0.3 for commonsense ness and contextual ness separately, the commonsense binary classification correct 76% of the time, and the contextualness classification accuracy is correct 91 % of the time. We find that the scores are not general well-calibrated, but this not needed for the thresholding that we do.
G IN-DEPTH LITERATURE REVIEW
Since 2022, when Wei et al. (022) found that models had the capacity solve (at the time) difficult reasoning problems by simply prompting the model to output detailed rationale (a "chain of thought" (COT)) before making decision, there has been significant interest in leveraging/improving LLMs' capacity for reasoning.
Semantic Logic Parsers More recently, series of methods were proposed which effectively by- passed the reasoning process by prompting LLMs to translate the given input to symbolic language (parsing the logic) and then using external, programmatic solvers to solve the problem in the symbolic space. F-COT, proposed by Lyu et al. (2023) and SAT-LM, proposed by Ye et al. (2023) are two contemporaneous works which prompt the LLM translate the given problem into its corresponding symbolic language, to be solved by the appropriate solver. Logic-LM, proposed by Pan et al. (2023), also published around the same time, includes a self-refinement step in order to catch mistranslations and to re-translate them, but this module provided only minor improvements.
The
strategy proved extremely effective, converting reasoning tasks into the more linguistic translation task. Since the algorithmic tools never make mistakes, if the translation is accurate, then the solution will be too. These methods, however, rely upon the strong assumption which goes widely unacknowledged that all necessary information is provided at input. By motivation, this assumption implies a well-informed, precise, and careful end-user. This critical assumption greatly injures the applicability and generalizaaility of these symbolic methods. In order address this, ARGOS aims specifically to provide missing information by exploring the logical space and leveraging logical tools. Xu et al. (2024), argue that using external solvers is not relevant to the LLM's actual reasoning capacity and present method which still parses the text into symbolic language, uses the I.LM as the symbolic solver by inputting the symbolic expressions directly to the LLM. In fact, this was an alternative presented in SAT-LM, and was shown to be weaker than directly using an external solver in cases where the problems are fully defined (which is the case in the setting chosen by both works).
19

---

Published as a conference paper at ICLR 2026
Deductive Logical Algorithms Given the findings discussed above that LLMs were efficient
logic-parsers, the door was opened to more agentic algorithms which operate in the logic-space (as opposed to the textual space). Algorithms were proposed by Kazemi et al. (2023) and Lee and Hwang (2024) which attempted to reason back wards through the problem in logic space, using the LLM after each deduction step to choose the next deduction. These methods proved perform worse than the previous parsing methods as they relied on the same assumptions of completeness in the input, but also required the LLM to greedily trace a reasoning path, opening another avenue for errors. This step is unnecessary since logic tools are able to search exhaustively over the solution space at very low cost.
Clause-generative Algorithms (Abduction) In response to the literature's 's inability to address cases where the completeness assumption fails, some work was proposed which explicitly aims to generate logical clauses in the textual space. The very similar algorithms proposed by Liu al. (2024) and Wang et al. (2022) first translate the text to logic and then produce some new clauses which are deducible via elassical logic given the logical translation of the input. By then translating the newly deduced logic back to text, the textual representation of the logic is now richer from a linguistic point of view (although no new information was added the underlying logic), and then COT is used to solve the augmented problem in text-space. In the logical sense, this method is not truly abductive as no new logical information is produced.
Instead of producing clauses via deduction on the input, Toroghi et al. (2024) proposed a method which leverages LLMs' knowledge of commonsense to supply missing clauses during the reasoning process. Given the logical translation of the input text, the space of all 2-variable clauses which are possible to construct using the variables given in the question is explored exhaustively, with each rule being given probability of being true by the LLM. Reasoning paths are formed with the various rules.
This method suffers from rigidity regarding the search space: due to the exhaustive search the space must be restricted to only 2-variable rules constructible from the input. Often the missing information from logic problems may include variables which are not named in the problem and so unseen in the abduction input. In other cases, the necessary rules might be of different sizes. These cases easily seen even in the commonly used datasets for bogic-reasoning evaluation. For example, CLUTRR Sinha et al. (2019), which is dataset of reasoning problems related to family relationships, requires 3-variable rules of the form mom( A, B) A sister( B, C) mom A, C). This algorithm is by construction incapable of producing such information. The evaluation procedures presented by Toroghi et al. (2024) use simple and rigidly structured data such that only simple 2-variable rules constructed from existing variables are required.
Since the algorithm output is dependent on the logical proof, if the necessary clauses cannot be found then a full proof will never be found, and the algorithm is forced to provide either no output or a guess. In contrast to these works, ARGOS aims to introduce logical clauses which provide new information about potentially unseen variables and which can be of varying sizes. In order to accommodate the very large search space, we introduce several innovative methods to dynamically select sub-spaces and to efficiently search them. Our proposed method aims to build its solution from both the language-reasoning space and the logic space in order to leverage the exactness of logical tools while remaining robust to failures to find the necessary logical clauses.
Generalist Reasoners Despite much research, Chain-of-Thouug (COT), proposed by Wei et al. (2022). remains one of the most generally applicable and robust reasoning approaches Plaat et al. (0044) Thus, much research has been done on how to augment the COT process (the previously discussed work by Liu et al. (2024) could be viewed as part of this category). While the focus of our paper is logical algorithms, there are some generalist methods which cannot be ignored. A simple but highly effective approach known as self-consistency (SC), proposed by Wang et al. (2023), prompts via COT several times and takes the mode of the outputs as the prediction. This method benefits from adding further robustness by smoothing potential outlier errors which might be present in a few chains, and is easily scalable according to computational budget by choosing the appropriate number of samples to take. However, the marginal return decreases as the number of sampled chains increases, as is shown in the original paper Wang et al. (202)). COT and SC's poor performance in proof planning (Saparov and He, 2023) motivated Tree of Thoughts (TOT), proposed by Yao et al.) (2023), in which a reasoning tree is hand-crafted for a problem and then the LLM is prompted to
20

---

Published as a conference paper at ICLR 2026
explore this tree for likely paths. This method demands very specific tree topology and exploration designs for each task and so is more of a general framework than an explicit method applicable to logical reasoning. The tree structure of logical reasoning problems even in the same dataset are highly varied. Given this, TOT is poorly suited to logical reasoning settings. There are many works which go beyond TOT in terms of exploring potential chains. Most recently, Xue et al. (2024) proposed recursive method which also conducts tree-like search, but allows for dynamically sertiically-ssuuuturee trees.
H
USED DATASETS
The following is a description of the datasets which we have used in our experiments.
ProntoQA is dataset which comes in three types of ground- ings over the same symbolic structure, which is string of single-variable modus ponens operations. One of these types is hand-crafted grounding designed be true according to commonsense. This dataset is, at this point, fairly easy for language models to handle. However, it remains common dataset in logical reasoning, and comes with the convenience that a simple random removal of inference rules included in the problem will build abduction cases, as all rules are commonsense. There are 59 problems in the dataset.
ProntoQA (Saparov and He, 2023
CLUTRR is dataset of family-relational reasoning problems in which some family relations (i.e. "Sam is the mother of John") are given in the form of simple stories (i.e. "John went with his mom Sam to the mall"). Traditionally, the task in CLUTRR is to deduce the relationship between two people, given the context. In order to structure the problem as true/false classical logic problem, however, we restructure the task to determine if a given relationship between two people is true or not. Practically, we construct the labels by taking the ground-truth relationship as the query 50% of the time (so the new label is "True"), and taking a random other relationship between the given two people ("False") the other 50% of the time, in order to balance the dataset. While the task is naturally abductive in that the input contexts do not include abstract or even grounded relational inference rules (i.e. "if A is the mother of B then B is the child of A"), most symbolic methods rely upon a practitioner to hand-craft a knowledge base of relational rules which are appended each problem in the dataset. Simply by forbidding this provision, the task becomes truly abductive for the reasoning model. There are 1000 problems in the dataset.
CLUTRR (Sinha et al. 2019)
FOLIO (Han et al., 2024) FOLIO is dataset of logic problems which were hand-crafted by "expert annotators". The dataset is the most diverse of the three we use, both linguistically and structurally. While not perfectly commonsensical, it is generally based in commonsense simply because the annotators were humans who exist in and tend to operate within real-world contexts. Given this couuud--cmmoonsssiclllttty random removal of rules to introduce commonsense abduction not an option, non comeommmeeeeee may be removed from the context. Thus, we engaged human annotators to replace randomly selected phrases from problems with semantically equivalent expressions, so that each replacement would require minimum of one new rule, indicating that the replaced phrase is implied by its replacement. For example, "NBA Player" - "Pro Basketball Player". Annotators were instructed reject problems where no replacement could easily be found, and some annotations failed to impact the problem due to either weak replacements or non-interaction with the solve-path of the problem. These problems were discarded, leaving us an abduction- of FOLIO with 108 True-or-Falss problems.
ESNLI (Camburu et al. 2018 ESNLI is a dataset of premise-conclusion pairs, stemming from the human-explanation NLI field. A machine is asked to explain how the premise yields or contradicts the conclusion. We adopted this dataset by inverting the task, so that given the premise and conclusion the machine must determine whether the conclusion is entailed (True) contradicted (False). The task naturally ensures that abduction is necessary, as we leave out the human explanation with which the pairs are annotated. The dataset, on inspection, is mostly common-sensical.
CosmosQA (Huang et al. 2019) CosmosQA is an MCQA dataset designed to test machine reading comprehension. We adapt our setting by constructing problems for which the answer "False" by taking questions for which "None of the Above" is the correct answer and randomly selecting
21

---

Published as a conference paper at ICLR 2026
another of the answer choices to be the query (for problems for which the answer is "True", the process is obvious). The dataset is mostly commonsensical.
QUAIL (Rogers al. 2020) QUAIL is MCQA dataset also designed test machine reading comprehension. It is derived primarily by scraping wiki and forum pages on online, and so often contains artifacts such as timestamps CSS formatting quirks. We adopt from MCQA to True/False in an identical fashion to CosmosQA. Becasue of its provenance, questions are often extremely vague: the larger context the webpage from which the problem comes is not included but is often key to answering the problem. On manual solution of QUAIL problems, the intuitive approach is often start by inferring the original context of the scraping, making the dataset of high value for abductive settings and for robust evaluation of commonsense flexibility.
I UNUSED DATASETS
While there are many logical-reasoonnn--eelaed datasets available, many are unsuitable because they either not truly aggicall-sstrcctuee are commonsensical. Here, will list some of the most commonly used datasets for logic-adjacent applications and explain their weaknesses/umsuitabiliy for our setting.
LogiQA While LogiQA (Liu et al. 2021) is generally commonsensical and logically themed, its questions do not in fact impose an immediate logical problem. fact, many of the questions are in fact meta-logical, in that they ask questions about the underlying logic of the text. For example: "Which of the following makes the same logical mistake above". These questions could indeed have a formal-logical re- re-framing, but this would require far more logical aptitude than is currently held by language models, and hand-translating LogiQA questions to logic problems too time-consumptive.
RECLOR RECLOR Yu et al. 2020) suffers the same weakness LogiQA. Questions are meta- logical or ask for subjective qualifications regarding some described commonsense logic. Again, this type of question both fails to evaluate true logical reasoning in real-world contexts, and proves problematic for careful evaluation given the inconsistency the task in that different questions ask different things of the reasoner.
Soft Reasoner
The Soft Reasoner dataset (Clark et al. 2021 is strictly logical, but is plainly non-commonsensical by construction. The logical problems are constructed without considera- tion of commonsense or real-world contexts. The dataset is constructed by building clauses from variables/predicates which are randomly selected from a selected bag of words.
LogicNLI LogicNLI (Tian et al. 2021) suffer from the Soft Reasoner dataset's 's weakness to an even greater degree while the problems are also randomly generated and not comply with commonsense, they also often do not comply with grammar. For example, phrases such "Quinlan does not entire" appear frequently. While this may suit the authors' aims of producing arbitrary text as stand-in for symbolic logic, it is not amenable to the evaluation of real-world logical reasoning in human contexts.
ProofWriter ProofWriter (Tafjord al. 2021) again suffers from the same weakness which all semi-random auto-generatte datasets suffer: non-commonsensenss. By picking clauses randomly by sampling bags of predicates, guarantees can be made the realism of the data. Examination of the dataset will show that the "facts" described in problem contexts range from unlikely to non-sensical the very first problem includes a rule "All red things are rough". Within the real world this is obviously not true, as we can find examples of red things which are not rough. Of course, it was not the dataset creators' aim to build a commonsensical dataset and this is of little surprise.
J CONFUSING PROBLEMS FOR ARGOS
Here' we give some examples of problems which confused ARGOS. Note that all of the prob- lems provided are examples in which the given premises are at least somewhat contradictory with commonsense, breaking the setting.
22

---

Published as a conference paper at ICLR 2026
Table 12:
Confusing since Fido typically specifically dog's name (and not cat's). If ask the LLM if Fido is a dog, with no context, it will say yes.
All tigers are cats. No cats are dogs. All Bengal tigers are tigers. All huskies are dogs. Fide is either Bengal tiger or a cat.
True or False: Fido is a husky animal.
Table 13: Confusing since Detroit City is probably not horse according to commonsense.
Detroit City is a horse. Some horses are racehorses. If horse falls in race, it poses risks to its rider. Detroit City fell in a race. A horse is a racehorse if it is in a race. True or False: Detroit City has been in multiple races.
Table 14: Confusing because (1) edible can refer to beverages when the contextual distinction is between safe and unsafe for consumption, but when the distinction between eating and drinking; (2) Coke is not apple juice.
All drinks on the counter are edible. All juices on the counter are drinks. Orange juice is type of juice. Everything on the counter is either orange juice or apple juice. All apple juices on the counter are sweet. The coke is on the counter and if the coke is apple juice, then the coke is a drink. If the cake is not apple juice, then the coke is not edible.
True or False: The coke is edible and sweet.
K SOLVABILITY PROGRESSION ILLUSTRATION
In this section we provide figures for CLUTRR, CosmosQA and QUAIL. illustrating how ARGOS' SC-solvability measure progresses per-question, over a 100-question clipping from each dataset. Positive % values indicate positive classification, and negative the like.
23

---

Published as a conference paper at ICLR 2026
Confidence progression
100%
50%
Question ID
0%
-50%
-100%
terminated
SC
1
2
5
6
9
10
Number of Iterations
Figure 8: CosmosQA: This dataset, being mostly solved by vanilla SC, sees little fluctuation and exits from ARGOS early. With that said, we still see flip from low-confidence negative to high-confidence positive.
24

---

Published as a conference paper at ICLR 2026
Confidence progression
100%
50%
Question ID
0%
-50%
-100%
terminated
SC 1 2
3
4 5 6
8
9 10
Number of Iterations
We see significant amount of confidence fluctuation and flipping. indicating that meaningful elements within the logic of the problem being modified by ARGOS in order to affect the answer. This is not surprising, since our construction of generated propositions as taking purely backbone variables as antecedents ensures that added ARGOS propositions will be effectual.
Figure 9: CLUTRR:
25

---

Published as a conference paper at ICLR 2026
Confidence progression
100%
50%
Question ID
0%
-50%
-100%
terminated
SC 1
2
4
5
6
8
9
10
Number of Iterations
Figure 10: QUAIL: despite ARGOS's strong performance QUAIL relative SC, we find that in fact few ARGOS iterations are necessary. While QUAIL is made complicated for language models by its irregular form and often disjoint nature, its logical structure (while very ambiguous) simple, meaning that we can solve QUAIL problems with only small number of well-chosen propositions.
26


Reviews:
[{"review_id": "YJNKRbYdgg", "reviewer": "Reviewer_7mDQ", "summary": "This paper addresses the problem of reasoning with missing commonsense information. It proposes a method of iteratively providing with an LLM the missing information in the form of L1 \\land L2 \\imply L3, where L1, L2, and L3 are all literals, and L1 and L2 are already deducible from the current premises. The paper experiments with 3 logical datasets and 3 less logical datasets. Experimental results demonstrate that the proposed method outperforms existing neural and neural-symbolic methods.", "questions": "Does the paper deal with propositional or first-order reasoning? Which SAT solver is used?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1.\tThe paper addresses the important problem of reasoning with missing commonsense information.\n\n2.\tThe paper proposes a simple but potentially effective method to abduce and reasoning with the missing information. In contrast to neural-symbolic methods based on auto-formalization, the method resorts to more involved interaction of neural and symbolic methods. \n\n3.\tExperimental results demonstrate the viability of the proposed method.", "weaknesses": "1.\tThe paper states that it is dealing with abductive propositional logic problems (sec 4. Problem statement). But I believe the reasoning problem is first-order. Especially, the used dataset FOLIO is a typical dataset for natural language reasoning with first-order logic. The paper does not specify which SAT solver it is using.\n\n2.\tSome use of logical notions in the paper is improper. For example, Line 141: “Propositional logic is a logical system that involves propositions about variables”. This is not a proper introduction of propositional logic. Line 91: “contain variables not previously mentioned in the input problem“. I had difficulty understanding this in the beginning. But later, I understand it actually means “contain propositions”. \n\n3.\tMany logical notations used in the paper are problematic. I only give some examples here. The logic formula in Line 187 is incorrectly written. The formula in Line 214 is confusing. Proposition 1 in Appendix A is not well-stated. \n\n4.\tFrom Sec 6.2, it seems that the work of the paper is founded on perfect logical translation. On the one hand, auto-formalization is still a challenging topic. On the other hand, when the paper presents the performance of SAT-LM, I assume it is based on auto-formalization, then the comparison might be unfair.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Q4n8Xo1xn2", "reviewer": "Reviewer_Kqo3", "summary": "The paper presents ARGOS, a neuro-symbolic framework designed to solve logic problems that require abductive reasoning—the ability to infer missing commonsense information. It addresses a well-known gap in existing systems: while symbolic solvers are rigorous, they are brittle and require a complete set of premises, whereas Large Language Models (LLMs) possess vast commonsense knowledge but often fail at complex proof planning.", "questions": "Minor suggestion on paper structure: Section 4 (\"PROBLEM STATEMENT\") is very concise and could be integrated into the end of Section 3 (\"BACKGROUND\") to improve the paper's narrative flow.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- Clarity: The paper is exceptionally well-written and easy to follow.\n- Quality: The experiment is executed to a high standard, both methodologically and empirically.", "weaknesses": "- Durability of the Problem Statement Against Frontier Models: The paper's motivation hinges on the inability of LLMs to perform abductive reasoning. I find that SOTA thinking models like Gemini-2.5-pro can solve the paper's motivating \"winter fox\" example directly via chain-of-thought. This raises the question of whether the proposed method addresses a fundamental limitation or a capability gap in a specific class of models that may soon be obsolete.\n\n- Worst-Case Complexity: The paper reports an average cost of 18.4 COT calls (Table 3), which is reasonable. However, the worst-case cost is unbounded in theory and in practice determined by the number of iterations allowed. For very hard problems requiring many abduction steps, the cost could become prohibitive, as each iteration involves multiple LLM calls (generation, commonsense scoring, relevance scoring) and solver calls. A discussion of the distribution of costs, not just the average, and the performance/cost trade-off would be valuable.\n\n- Inability to Express More Complex Rules: Real-world commonsense often takes more complex forms.  The current llm_generate prompt structure seems hard-coded for the two-antecedent form. The paper would be more complete if it acknowledged this limitation and discussed potential extensions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "15ODVBpXAm", "reviewer": "Reviewer_mBHM", "summary": "The paper proposes a neuro-symbolic framework called ARGOS to improve commonsense reasoning. This framework addresses the inability of logic solvers to handle missing commonsense facts by using an LLM to iteratively provide new commonsense propositions. An interesting contribution is the use of feedback from the symbolic SAT solver itself to guide the LLM's search for relevant facts. This allows ARGOS to search a larger space of potential facts, including new variables not present in the original problem. The framework also uses the LLM to score the generated facts for commonsense and relevance before adding them . The authors show that this approach improves performance on three abductive reasoning datasets.", "questions": "- Since ARGOS depends on the LLM's ability to reliably score commonsense and relevance, did you do any human analysis to verify that the LLM-generated scores are calibrated & accurate?\n- How often does ARGOS introduced an unseen variable that is important for solving the problem?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The proposed method provides an intuitive framework for combining the strengths of symbolic solvers and LLMs.\n- The use of the SAT solver's backbone to guide the generation of new commonsense facts is novel.\n- The empirical results are strong and show consistent improvements over existing neural and symbolic baselines on 3 datasets.\n- The ablation studies show the value of the two main contributions, ie, backbone-guided search and score-based thresholding.", "weaknesses": "- The tasks/datasets used are not practically relevant and lack real-world applicability. In addition, the paper relies on modified versions of existing datasets (ProntoQA, CLUTRR, FOLIO) to create an abductive setting, which means the evaluation is on a somewhat artificial task.\n- The method requires logit-level access to score generated clauses for commonsense and relevance, which may not always be accessible for closed-source models.\n- The main experiments assume a perfect logical translation from text, as failed translations were filtered out. But this ignores the issue of imperfect translation, which could be a bottleneck for this method and neuro-symbolic systems in general.\n- The method relies on an LLM itself to score its own generated clauses for commonsense and relevance. The reliability of this LLM-as-a-judge component is not validated against human-annotated scores.\n- There is a lack of examples accompanying the error analysis in the paper, showing failure cases of ARGOS.\n- The paper does not report the latency of all approaches compared in the main results. This is important because it would seem to me that ARGOS likely takes much higher computation time.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "w5v8ndk3UN", "reviewer": "Reviewer_L1Mk", "summary": "This paper proposes a new neuro-symbolic approach for enhancing the logical commonsense reasoning capabilities of LLMs and suggests a new way for LLM’s integration with logic solvers. In the approach proposed in this paper, the LLM iteratively provides unstated commonsense clauses to a logic solver, which is guided by feedback from the solver in the form of the SAT problem backbone. This approach allows the system to perform abductive reasoning, filling in missing background facts while keeping the search tractable. Overall, this work aims to contribute to leveraging the benefits of existing neural and symbolic methods  to tackle commonsense logical reasoning problems.", "questions": "1- In Line 228, the expression rankB(L) = #{L′ ∈ B | L′ has an entity in common with L} is written. Is this intended to be score(L)? Why would the rank of each literal be the number of existing literals that share the same variable? Regardless, the rationale for this approach is unclear to me. The only explanation provided is “which gives a measure of relevance of the literal to the problem” which is vague. I understand space limitations in the main paper, but I strongly suggest you explain the rationale near algorithm 2 in the appendix.\n\n2- The methodology proposed in this work for leveraging the LLM’s commonsense knowledge is restricted to generating literals that can be deduced from the existing literals in the backbone. While this approach is in agreement with the way existing logical reasoning datasets are formed, I don’t think it is general enough for all practical applications. For example, a commonsense rule can be generated using only one literal from the backbone (e.g., ∀x Car(x) → Vehicle(x)), so why are pairs of literals necessary in the proposed approach?\n\n3- Aside from being limited, I think the commonsense rule generation process in this work is also inefficient. Every pair of literals is presented to the LLM, and the scoring mechanism explained in appendix D4 is used to filter irrelevant ones. Some works cited in the paper such as LAMBADA and LLM TRes take a goal-driven approach to only generate rules that can contribute to solving the problem. Why isn’t a similar approach taken in ARGOS?\n\n\n4- In appendix D3, why are the scoring propositions approaches different across datasets? I suggest a clarifying sentence to explain the rationale. I also appreciate the running example in section 5.2 which facilitates understanding.\n\n5- In line 202, it’s stated that: “Four annotated examples are provided, intended for few-shot prompting.“\nHow are these few-shot examples chosen? Do they differ per-dataset? Are they chosen in a way that there is no risk of revealing the answer to the LLM?\n\n6- What is the rationale for reducing γ at each iteration? By reducing γ, you are making the method more lenient, accepting answers even if there is less consistency in LLM generations. Doesn’t this approach reduce the rigor of reasoning as the algorithm proceeds?\n\n7- Self-consistency is a key component of ARGOS and in fact one of the ways by which ARGOS generates its answers. As I mentioned in my earlier comments, using the LLM to generate the final answer might be sub-optimal, potentially generating hallucinated answers. Figure 5 nicely provides insight about how ARGOS improves accuracy of self-consistency responses, but I think the paper’s analysis also requires reporting the accuracy of reasons for all methods, at least for one dataset. Also, I think a study in which the self-consistency module of ARGOS is ablated should be provided, at least for one dataset.\n\n8- Regarding RQ1, two mechanisms are used in ARGOS for scoring in the thresholding calculation. Are they both necessary? An additional ablation would be helpful. I also appreciate the honest discussion of limitations in RQ2.\n\n9- I find the discussion in section 6.2 is questionable. Logical translation is a core part of ARGOS, not an orthogonal one and assuming having a correct propositional translation for real applications is a very big assumption to make. I think experiments on at least one dataset is needed without filtering the failures to shed light on how critical this step is to the framework. Using a more powerful LLM than the ones used for generating the commonsense rules is acceptable if it’s a major bottleneck, but having an experiment using the same LLM that ARGOS uses for reasoning is also quite beneficial. A proposed method isn’t required to beat all baselines on all tasks, but the reader must know the strengths and limitations of the proposed method.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1- The paper is easy to follow and is well-presented (modulo some issues that I point out in the weaknesses). The worked example presented in Section 5.2 and the methodology overview in Figure 3 faciliate understanding of the work.\n\n2- The proposed methodology is novel and insightful. I think the general idea of the work in providing a new paradigm of interaction between an LLM and a symbolic solver is interesting. Existing approaches either initiate reasoning from the LLM and delegate theorem proving to a solver, mimic inference rules using an LLM, or propose methodologies to leverage the LLM to undergo a rigorous reasoning process while leveraging its commonsense. This paper introduces interactions between the LLM and the solver which I find novel and interesting.\n\n3- The topic of focus, commonsense logical reasoning of LLMs is a quite an important topic with numerous practical applications. I think the idea of proposing novel frameworks for LLM interaction with formal reasoners can be be impactful by reducing reasoning errors of LLMs, but the paper's evaluation needs to be strengthened to validate this effect more properly.", "weaknesses": "1- There are several statements in Section 3 which I think are vague, inaccurate, or wrong:\n\n- Line 141: “Propositional logic is a logical system that involves propositions about variables.”\n\n   Propositional logic does *not* involve “propositions about variables.” It is a logic of propositions themselves, and variables are symbols for propositions, not objects that propositions are “about.”\n\n - “A proposition, such as A → ¬B, is some statement about literals”\n\n     Propositions aren’t “about literals”. They’re built from literals (or propositional variables) using logical connectives.\n\n- “We assume that ¬(P ∧ C → ⊥), that is that the premises P are not contradictory with commonsense.”\n\n    The correct way to show consistency is “P∧C⊬⊥”.\n\n-  “A predicate is a function, such as MotherOf (x, y)”\n\n    A predicate represents a property or relation that can be true or false depending on its arguments. Whereas FOL functions point to a particular object in the domain as their output. For example, MotherOf (x, y) is a predicate which can be true or false, but MotherOf (x) is a function that returns a specific object y, i.e., MotherOf (x) = y.\n\n- “∀(x, y)MotherOf (x, y) → ¬Male(x)”\n\n    This is a very unusual syntax. In standard FOL, you either write ∀x ∀y (MotherOf(x, y) → ¬Male(x)) or ∀x ∀y [MotherOf(x, y) → ¬Male(x)].\n\n- Line 190: “First-order logic problems…”\n\n   I encourage the authors to use the conventional terms “grounding” or “instantiation”.\n\n- Line 214: we first try to solve the problem using the SAT Solver (sat_solve) to test whether (P ∧ C) = P ⊢ Q or ¬Q\n\n   what does (P ∧ C) = P ⊢ Q or ¬Q mean? I think you’re just trying to show whether (P ∧ C) ⊢ Q or (P ∧ C) ⊢ ¬Q.\n\n2- The use of self-consistency as one of the ways ARGOS can come up with the final answer is questionable. At the end of the day, self-consistency is relying on the LLM to do the reasoning, but the reason why people leverage or combine symbolic theorem proving with the LLM reasoning is because LLMs alone may make errors in their reasoning or generate hallucinated answers.\nThe experiments section only reports accuracy of the final answers, whereas in LLM reasoning works such as [1], the correctness of the reasoning process is also critical. Specifically, I think this metric can be insightful to see whether the correctness of reasoning process for answers provided by self-consistency mechanism of ARGOS is also improved or not. This can be a useful complement to the results in Figure 5.\n\n3- There are some typos in the text such as line 81.\n\n[1] Kazemi, Mehran, et al. \"LAMBADA: Backward Chaining for Automated Reasoning in Natural Language.\" Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)