From Heuristic to Analytic: Cognitively Motivated Strategies for Coherent Physical Commonsense Reasoning
Authors: Zheyuan Zhang
Shane Storks
Fengyuan Hu
Sungryull Sohn
Moontae Lee
Honglak Lee
Joyce Chai
Venue: EMNLP 2023 Main (2023)
Keywords: commonsense reasoning
physical commonsense
language model analysis
cognitive modeling
TL;DR: Similar to humans, PLMs can produce better analytic physical commonsense rationalizations when conditioned on higher-level heuristic decisions in both in-context learning and fine-tuning.

Abstract: Pre-trained language models (PLMs) have shown impressive performance in various language tasks. However, they are prone to spurious correlations, and often generate illusory information. In real-world applications, PLMs should justify decisions with formalized, coherent reasoning chains, but this challenge remains under-explored. Cognitive psychology theorizes that humans are capable of utilizing fast and intuitive *heuristic* thinking to make decisions based on past experience, then rationalizing the decisions through slower and deliberative *analytic* reasoning. We incorporate these interlinked dual processes in fine-tuning and in-context learning with PLMs, applying them to two language understanding tasks that require coherent physical commonsense reasoning. We show that our proposed Heuristic-Analytic Reasoning (HAR) strategies drastically improve the coherence of rationalizations for model decisions, yielding state-of-the-art results on Tiered Reasoning for Intuitive Physics (TRIP). We also find that this improved coherence is a direct result of more faithful attention to relevant language context in each step of reasoning. Our findings suggest that human-like reasoning strategies can effectively improve the coherence and reliability of PLM reasoning.

From Heuristic to Analytic:
Cognitively Motivated Strategies for
Coherent Physical Commonsense Reasoning
Zheyuan Zhang
Shane Storks' Fengyuan Hu'
Sungryull Sohn?
Moontae Lee
Honglak Lee
1.2
Joyce Chai
University of Michigan
LG Al Research
University of Illinois at Chicago
(zheyuan, sstorks, hufy, chai chmmich@ edu
(srsohn, moontae. lee, (ongla)ellgeen grggreeaarg......ommmmomoooo ai
Abstract
with PLMs creates demand for more reliable, logical, and transparent reasoning strategies com- patible with differentiable architectures like PLMs (Marcus, 2020; LeCun, 2022).
Pre-trained language models (PLMs) have shown impressive performance in various lan- guage tasks. However, they prone spu- rious correlations, and often generate illusory information. worldworld applications, PLMS should justify decisions with formalized, co- herent reasoning chains, but this challenge re- mains exper-explored. Cognitive psychology theorizes that humans are capable of utilizing fast and intuitive heuristic thinking to make decisions based on past experience, then ratio- nalizing the decisions through slower and de- liberative analytic reasoning. We incorporate these interlinked dual processes in fine-tuning and in-context learning with PLMs, applying them to two language understanding tasks that require coherent physical commonsense rea- soning. We show that our proposed Heuristic- Analytic Reasoning (HAR) strategies drasti- cally improve the coherence of rationalizations for model decisions, yielding state-of -fee----tth-trttt art re- sults on Tiered Reasoning for Intuitive Physies (TRIP). We also find that this improved coher- ence a direct result more faithful attention relevant language context in each step of rea- soning, Our findings suggest that human-like reasoning strategies can effectively improve coherence and reliability of PLM reasoning.
Meanwhile, in theories of cognitive psychol- ogy, drawing conclusions in reasoning problems and coherently rationalizing them have long been thought to come from dual processes of human cognition (Wason and Evans, 1974; Evans and Wa- son, 1976; Evans, 1984, 2003; Tsujii and Watan- abe, 2009; Evans, 2011): fast, associative heuristic thinking based on experience, and slower, delibera- tive analytic thinking, which requires more work- ing memory. Specifically, prior work theorizes that heuristic processes enable us to extract the most relevant information from the context and provide quick intuition for decisions, which can then inform analytic processes that operate this information to perform inference and rationalize when needed (Evans, 1984, 2010; Kahneman, 2011).
In this work, inspired by the synergy between these dual processes in humans, we propose anal- ogous hcuristic- reasoning (HAR) strate- gies for PLMs, which bootstrap lower-level (ana- lytic) rationalization from higher-level (heuristic) decision-making. Using physical commonsense an example, we implemented HAR for both fine- tuning and in-context learning. Through evaluation on two benchmarks. Ticred Reasoning for Intuitive Physics (TRIP: Storks et al., 2021) and a novel re- framing of ProPara (Dalvi et al., 2018) requiring multiple levels of reasoning, we find that HAR con- sistently enables more coherent reasoning, achiev- ing state-of of-the-atteeeee- art results on coherence metrics in TRIP. More importantly, our analysis shows this improved coherence in in-context learning is en- abled by more faithful and transparent rationaliza- tion through stronger attention to relevant language context reasoning. This demonstrates that simi- lar to humans, PLMs' reasoning is strengthened by linking heuristic and analytic processes, and such
1 Introduction
Pre-trained language models (PLMs) have recently shown strong performance on an increasing vari- ety of challenging reasoning problems (Radford et al., 2018, 2019; Devlin et al., 2019; Brown et al., 2020; OpenAL, 2023). While their capabilities are impressive, their tendency to make artificial mis- takes in reasoning, including exploiting spurious correlations (Schwartz al., 2017; Gururangan al., 2018) and hallucinating factual information (Dziri et al., 2022; Ji et al., 2023) in order to reach conclusions, makes them difficult to rely on in prac- tice. This possibility of incoherence in reasoning
Authors contributed equally to this work.

---

strategies may be applied in other reasoning tasks for more trustworthy secision-making
full demonstrations at inference time before ask- ing it to solve lask. A number of works found applications in-context learning PLMs for complex reasoning tasks (Dalvi et al., 2021; Tafjord et al., 2021; Spiliopoulou et al., 2022). Among these works, significant improvements came from inserting generating free-text rea- soning chains in prompts to support task predic- tions (Wei et al., 2022; Kojima et al., 2022). These findings sparked the exploration of many differ- ent in in-context learning and sequential prompting approaches to strengthen reasoning in PLMs and tackle various tasks (Kazemi et al., 2023; Nye et al., 2022; Zelikman et al.. 2022: Wu et al.. 2022: Yao et al., 2023; Long, 2023; Wang et al., 2023). These methods usually rely on an assumption that by de- composing high-level task into many low-level sub-tasks, the model can solve the low-Ievel sub- tasks casily, which helps it achieve better perfor- mance on the high-level task. However, in complex cases like commonsense reasoning. even lower- level sub-tasks are hard to solve due to the require- ment of retrieving and incorporating knowledge beyond the text. As such, our heuristic-analye analytic in-context learning approach instead uses higher- level decisions to refine the generation of low-level commonsense knowledge from PLMs.
2
Related Work
Dual process theory in AI. Dual process theo- ries of cognitive psychology have recently attracted interest in areas of Al. Anthony et al. (2017) apply them to augment reinforcement learning al- gorithms with deliberative planning of policies through tree search. Ganapini et al. (2022) combine them for more efficient navigation with Al agents that evolve from slow to fast decision Decision-making while navigating. Similarly inspired by dual process the- ories, Hua and Zhang (2022) apply logical reason- ing over representation learning for more accurate commonsense knowledge base completion. Lee et al. (2023) use dual-process inspired associative selection combined with evidence generation to perform question answering on scientific articles. Booch et al. (2021) propose additional research questions and directions around the upplication of dual process theories of human cognition in Al. Unlike these past works, apply dual process theories of human cognition in coherent physical commonsense reasoning with PLMs, both through fine-tuning and in-context learning.
Reasoning with PLMs. Prior work has studied fine-tuning PLMSs on various reasoning tasks, in- cluding mathematical reasoning (Geva et al., 2020), temporal reasoning (Zhou al., 2021), and com- monsense reasoning (Rajani et al., 2019; Sap et al..
Coherent Commonsense Reasoning
We study the problem of coherent commonsense reasoning in natural language understanding. Here, coherent reasoning refers to the ability of lan- guage understanding system to justify a high-level decision on a reasoning task with valid low-level evidence from or beyond the language context that supports the decision (Storks and Chai, 2021; Storks et al., 2021). We specifically focus on the domain of physical commonsense:: humans' intu- itive set of rules and concepts that explain how the world physically works. Since humans build their physical commonsense intuition from a young age (Bliss, 2008; Lake et al., 2017), this knowledge may be considered obvious to most humans (Forbes and Choi, 2017), creating a challenging problem for PL.Ms which cannot directly interact with the phys- ical world, and may not encounter much of this knowledge in pre-training. While humans rarely communicate about physical commonsense in lan- guage or carefully rationalize their intuitions in terms of low-level physical states, is essential for AI systems to do so for safety, transparency, and
2019), mostly through data-driven approaches to strengthen PLMs' reasoning abilitics. Recent work has attempted to build in more explicit coherent reasoning structures (Storks et al., 2021; Ma et al., 2022; Richardson et al., 2022; Li et al., 2023), but these approaches do not take advantage of how higher-level tasks may inform lower-level tasks, rather they reason from the bottom up over various representations of the world state, or jointly opti- mize all reasoning steps without dependency. Un- like these works, we take inspiration from human reasoning to explore the role of sequential heuristic and analytic processes in fine-tuning and in-context learning, bootstrapping low-level commonsense ra- tionalization from higher-level decisions.
With the introduction of GPT-3 (Brown al., 2020), in-context learning became a common way to apply PLMs new tasks without in-domain gradient-based training, where one prompts the PLM with helpful task-specific knowledge or even

---

Story A:
Story A:
1. Mary went to the fridge.
1. Air brought in through the mouth.
2. Mary took out a bowl from the fridge.
2. Passes through the lungs.
3. The bowl had cucumber and a donut in it.
3. And into the bronchial tissue.
4. Mary put the cucumber on the counter.
4. The carbon dioxide removed.
5. Mary ate the donut.
5. The lungs bring the oxygen to the rest of the body.
Story 8:
Story 8:
1. Mary went to the fridge.
1. Carbon dioxide enters the leaves through the stomates by diffusion.
2. Mary took out a bowl from the fridge.
2. Water is transported to the leaves in the xylem.
3. The bowl had cucumber and a donut in it.
3. Energy harvested through light reaction is stored by forming ATP,
Mary tossed the donut the trash.
4. Carbon dioxide and energy from ATP are used to create sugar.
Mary ate the donut.
5. Oxygen exits the leaves through the stomata by diffusion....
Plousible story: A Conflicting sentences: (4, 5)
V accurate
Carbon diaxide conversion story: B
consistent veriftable
Carbon dioxide conversion sentence: Carbon dioxide conversion sugar sugar
States: inedible(donut) > edible/donut)
Figure I: TRIP (left; Storks et al., 2021) and proposed Tiered-ProPara (right) tasks for coherent physical common- sense reasoning. Each task requires multiple levels of reasoning from surface-level story and sentence selection and commonsense physical state prediction. While accuracy only evaluates the ability to perform the highest-level task, consistency and verifiability are used to evaluate lower-level steps and judge the coherence of reasoning.
trustworthiness..
20 physical attributes justify the identified con- flict, and verifiability measures how often system can produce a fully correct reasoning chain includ- ing these physical states, conflicting sentences, and story plausibility. Consisteney and verifiability both require an accurate story plausibility predic- tion as a prerequisite, but additionally evaluate the coherence of PLMs' reasoning to support the pre- diction. To demonstrate deeper understanding, AI system should not just achieve high accuracy on TRIP, but comparably high consistency and veri- fiability. If consistency and verifiability are lower than accuracy, this indicates incoherent reasoning, as the PLM is making correct predictions on the end task without valid underlying support.
In the context of coherence, we define physi- cal commonsense reasoning (PCR) as the appli- cation of this complex background knowledge to understand language describing physical situations and procedures. PCR may include inferring low- level, concrete physical states of participants in text (Dalvi al., 2018; Zellers et al., 2021), or reasoning over them to perform higher-level tasks, e-g., answering questions about procedures (Bisk et al., 2020; Storks et al., 2021). We focus on two benchmarks that include both types PCR tasks in multi-step chains enabling evaluation of coherence: Tiered Reasoning for Intuitive Physics (TRIP: Storks et al., 2021) and a new reframing of ProPara (Dalvi et al., 2018) called Tiered-ProPare examples of which are shown in Figure 1.
Two recent state-of-the the-tr-ttttttttt approaches for TRIP Coaleseing Global and Local Information (CGLI; Ma et al., 2022) and Breakpoint Trans- former (Richardson et al., 2022). While they achieve up to near-perfect testing accuracy, con- sistency and verifiability still lag behind, reaching respective maximum of 77.3% and 32.4%. There- fore, coherent PCR remains a difficult problem.
3.1 TRIP
Tiered Reasoning for Intuitive Physies (TRIP) is benchmark for measuring coherence of PCR, where high-level predictions over procedural text must be justified with low-level physical commonsense evidence, i... physical states of entities (Storks et al., 2021). As shown in Figure 1, given two stories (A and B), system predicts which story is more plausible; a system's 's ability to perform this end task can be measured by traditional clas- sification accuracy. It then identifies which two sentences are conflicting the implausible story, and consistency measures how often a system can support an accurate plausibility prediction with cor- rect conflicting sentences. Lastly, the system pre- dicts precondition and effect physical states over
3.2 Tiered-PrPPara
ProPara a dataset of texts about scientific pro- cesses annotated with the dynamic existence and location of entities throughout the processes (Dalvi et al., 2018). While ProPara originally focused on the low-level task of predicting the states of entities before and after each sentence of passages, we pro-
VVerifiability requires that at least one non-default physical state label is predicted within cach conflicting sentence, and all such predicted states are correct.

---

Story
Story
4 HAR for PLM Fine-Tuning
Air is brought...
1. Carbon dioxide..
corbon...
diovide..
Fine-tuning PLMs is one popular approach for adapting them to downstream tasks, applied in cent work toward coherent commonsense reason- ing (Ma et al., 2022; Richardson et al., 2022), and suitable for applications with compute or privacy restrictions. As shown in Figure 2, we can build explicit heuristic-analytic reasoning structure into PLM fine-tuning for our target tasks by deleting parts of the language context that are longer relevant the model makes predictions for each step of reasoning. both during training and infer- ence. While our approach provides just one ex- ample, this reasoning trick can apply to PLM fine- tuning for any multi-step reasoning problem where the most relevant language context to support rea- soning changes with each step.
The lungs...
Orygen exits..
Story:
PLM
Story Classifier
Story-A
Story 1. Carbon dioside...
4. Carbon closide..
Su-Tho lungsvu
5. Dxygen exits..
Converston Sontence:
PLM
Sentence Classifier
Story-Sh
1. L:Corbon-diee diovidew
Carbon dioside..
G--The thugfeeee
S-Gaggen-eie
Converted Entily:
PLM
Classifier
Entity
sugar
Figure 2: Heuristic-amalytic reasoning for fine-tuning PLMs, where the language context iteratively refined using classification predictions during training and in- ference. In Tiered-ProPaee after the PLM is used to classify which story contains conversion, the other story is deleted from the model inputs. After classi- fying which sentence describes the conversion, other sentences deleted. Lastly, the resulting entity after the conversion is identified.
4.1 Fine-Tuning Experiments
Next, we introduce experiments with HAR in fine-tuning PLMs.
Implementation.
To implement HAR in fine- tuning, we use the recent state-of-the-art Coalesc- ing Global and Local Information (CGLI) model for TRIP and ProPara (Ma et al., 2022) as a back- bone for HAR in fine-tuning. We apply two tricks to better focus the model on relevant information. First, while the original CGLI model takes one text as input, Focused CGLI (FCGLI) takes two texts, enabling the model consider both together. Ad- ditionally, filtered down training annotations in TRIP for low-level state prediction tasks only include the most relevant physical states, i..., those causing conflicts. 2 This allows the model to fo- cus on learning the most important physical states from the high-dimensional hypothesis space. De- velopment and testing data remain unchanged for faireess. FCGLI with heuristic-anall analytic reasoning (FCGLI-HAR) applies the above iterative deletion strategy to the input context of FCGLI.
pose Tiered-ProPatt novel reframing of the task which requires multiple levels of reasoning. As shown Figure 1, in this version of the task, a sys- tem is presented with two passages from ProPara with shared entities, and asked in which story is a particular type of entity, e.g., carbon dioxide, con- verted into another entity. In addition to this, the system must identify the sentence in that story in which the conversion occurs, and what the entity is converted into. Similarly to TRIP, we use these two lower-level tasks evaluate consistency and verifiability of system predictions on the end task of choosing a story. More details on how we generate this data are given in Appendix B.
Motivated by dual process theories for human cognition, we propose heuristic-analytic reason- ing (HAR) strategies for these tasks for fine-tuning (Section 4) and in-context learning (Section 5) with PLMs. Each strategy allows the PLM to make easier high-level decisions first (e.g.. story and sen- tence selection), then use them condition low- level rationalization (e.g., physical state prediction), the essence of combined heuristic and analytic pro- cesses in human reasoning.
Baselines. To measure the advantage of inform- ing low-level reasoning steps with higeer-level steps through sequential heuristic and analytic pro- cesses in FCGLI-HAR, we use FCGLI, where all reasoning steps are performed jointly without de- pendency between them, as baseline. To con- textualize TRIP results with past work, we also include RoBERTa (Liu al., 2019) results from
Mooee fine-tuning details provided in Appendix C.

---

Accuracy
0.8
Verifibillity
0.5
0.8
PCGLI
0.6
003
POGLI-HAR
0.4
0.0
Epach
Epach
Epoch
Accuracy
0.99
1,0
G 110
Consistency 0.4
Verifiabiltty
0.5
6.6
0.0
0.7
0.6
0.3
1
2
4
7
2
4
10
2
10
Epach
Epach
Epoch
Figure 3: Validation metrics for unstructured FCGLI baseline and FCGLI with annristic-analliic reasoning (FCGLI- HAR) through epochs of training TRIP (top) and Tirred-ProPara (bottom).
TRIP
most difficult to learn. However, FCGLI-HAR verged 1-2 training epochs faster than FCGLI on both datasets, suggesting that HAR may enable
Approach
Accuracy
Consistency
Verifiability
RoBERTa
72.9
19.1
9.1
CGLI
94.1
77.3
28.0
Breakpoint
80.6
53.8
32.4
more efficient learning of coherent reasoning.
FCGLI
93.7
66.2
33.8
HAR for PLM In-Context Learning
ECGLI-HAR
94.3
75.4
41.1
Tiered- ProPara Consistency
Tiered-ProPara
While HAR can benefit domain-specific applica- tions when integrated into PLM fine-tuning, it re- quires expensive training on in-domain data that may sacrifice generalizability. To alleviate the limi- tations fine-tuning, we also integrated HAR into in-contexi learning, taking advantage of emergent abilities of large PLMs to perform coherent reason- ing through free-form language generation. Prompting techniques like chain-of-thought (CoT; Wei al., 2022) can be useful in in-context learning to demonstrate valid reasoning strategies to reach conclusions. However, CoT has tradition- ally been used to improve performance on complex high-level tasks by breaking them down into sim- pler low-level steps and reasoning from the bottom up. Mcanwhile, the most complex sub-task and bottleneck in our problem is low-level physical state prediction (Storks et al., 2021), which is im- possible to further break down, as descriptions of actions directly invoke world state based on phys- ical commonsense rules how the world works. Since it would thus be difficult to use traditional CoT to generate a useful explanation to support physical state prediction, we apply HAR through reverse CoT method where tiered tasks are demon- strated and predicted in top-down sequence from high-level decisions low-level rationalization. In TRIP, as shown in Figure 4, PLMs are condi- tioned to first predict plausibility, a relatively easy heuristic process. To further refine the relevant con-
Approach
Accuracy
Verifiability
FOGLI
94.5
56.7
36.2
ECGLI-HAR
95.1
83.6
57.4
Table I: TRIP and Tiered-ProPam results for baselines (introduced in Section 4.1), and heuristic-analytic rea- soning with fine-tuned PL.Ms (FCGLI-HAR).
Storks et al. (2021), CGLI (Ma al., 2022), and Breakpoint Transformer (Richardson et al., 2022).
Results. Results are listed in Table 1.3 On TRIP, FCGLI-HAR exceeds or achieves compurable per- formance to baselines on all three metrics, pushing verifiability up to 41.1 1.1 % and setting new state- of-the-art result on the most difficult sub-task of TRIP. On Tiered-Prov ProPara, FCGLI-HAR exceeds the FCGLI baseline all metrics, with consis- tency and verifiability reaching respective 83.6% and 57.4%, thus beginning close the gap between accuracy and these coherence metrics. This shows that HAR is indeed a promising strategy to improve coherence and dependability of reasoning in PLMs.
4.2 Learning Curves for FCGLI-HAR
We plotted the validation metrics through fine- tuning FCGLI and FCGLI-HAR Figure 3. Unsur- prisingly, we see that consistency and verifiability increase slower than accuracy through training, sug- gesting that these lower-level objectives are indeed
*PLMs are first conditioned with 4 consistent demonstra- tions of the ICL-HAR strategy from the training set. More details in Appendix D.
"Staristical significance testing of finc-tuning performance improvements in Appendix A.I.

---

entity after conversion, which requires common- sense understanding of how entities change as a result of various actions and processes.
Story A:
1. Mary went to the fridge.
Mary took out a bowl from the fridge.
3. The bowl had cucumber and donut in it.
5.1
In-Context Learning Experiments
Mary put cucumber on the counter.
5. Mary ate the donut.
Next, we introduce our experiments on in-context learning with HAR (ICL-HAR).
Story
1. Mary went to the fridge.
Mary took bowl from the fridge.
Implementation. We apply ICL-HAR with In- structGPT (Brown et al., 2020; Ouyang et al.. 2022) and LLaMA-65B" (Touvron et al., 2023) using greedy decoding. Since LLaMA is limited to context length of 2048 tokens, while prompts for TRIP include over 3000 tokens to familiarize the model with the physical state classification label space.' we apply it filtered version of TRIP which only includes instances where anno- tated states involve only the top-6 most frequent physical precondition-effet pairs.
3. The bowl had a cucumber and donut in it.
4. Mary tossed the donut the trash.
5. Mary ate the donut.
PLM
"Story is more plausible."
Heuristic Decisions
"In Story A, sentences 4 and conflict with each other."
"For sentence After Mary tossed the donut in the trash the donut now inedible."
Analytic Rationalization
Unstructured baseline. We propose more tra- ditional, unstructured in-context learning (ICL-U) bascline to help measure the advantage of HAR strategies applied in in-context learning. Instead of prompting PLMs to predict all 3 reasoning steps in sequence, we extract each step of the task through separate but comparable prompts. 8 The PLM is provided both input stories for each task, and given 4 task-specific demonstrations comparable to ICL- HAR. We then combine the 3 extracted predictions on each testing example into one reasoning chain to calculate evaluation metrics them.
"For sentence Before Mary ate the donut... the donut was edible."
Figure 4: Heuristic-analytic reasoning (HAR) for in- context learning with pre-trained language models (PLMs). HAR uses chain-of- -hain-of-tthough prompting to bootstrap low-level analytic rationalization (e.g.. physi- cal state prediction) from high-level heuristic decision- making (e.g., implausible story and conflicting sentence selection), focusing the PLM's attention to the most rel- evant context in each reasoning step.
Traditional chain-of- th-huutt baseline. Despite the anticipated limitations discussed above, we aug- ment the ICL-U baseline with traditional CoT for comparison, creating an additional in-context learn- ing with CoT (ICL-CoT) baseline. Specifically, we prompt InstructGPT with "Iet's think step by step about (sub-task) (Kojima et al., 2022) to generate free-text explanation for each separate reasoning step. We then append these explanations to their re- spective prompts before eliciting predictions from
text, the PLM then predicts conflicting sentences, another heuristic judgement that need not be di- rectly based on low-level commonsense knowledge, but may still benefit from being conditioned on the higher-level implausible story prediction. Lastly, the PLM rationalizes these decisions with low-level physical states, an analytic process requiring the in- tegration of external background knowledge about actions and objects. Instead of breaking down phys- ical state prediction further (the typical purpose of CoT), we hypothesize that conditioning this sub- task with higher-level heuristic processes in HAR helps focus the model the correct context and reason more coherently. We similarly apply HAR in Tiered-ProPaan by prompting the PLM to first select story and sentence in which a particular entity is converted to another, and finally rational- izing these decisions with the name of the resulting
SSpecifically, we use text-davinci-0e2 version through an Azure OpenAl deployment.
"Specifically, we use a HuggingFace (Wolf et al., 2020) compatible version available at ht httpps///hut //uugg fnce co/ decapoda cecasadda-eseerrchee wwwerreeeeeeeeeeee waarrrhhhllaaaaeeeeeee 65b--5----- hf at the time of writing.
"See Appendix D.I for more information.
"Example prompts provided in Appendix
"Based on preliminary experiments with both PLMs. we expected InstructGPT to generate more reasonable explana- tions on the in-context demonstration examples, so we used its explanations in prompting both InstructGPT and LLaMA.

---

InstruciGPT
the models. Unlike HAR, which enforces top- down chain-of-thought, this traditional application of CoT allows the PLM to attempt to break down each step before making a prediction for it.
TRIP
Tiered-ProPara
Approach
Acc.
Cony.
Ver.
Acc.
Cons.
Ver
ICL-U
70.9
40.7
54,9
17.4
5.2
ICI.CoT
75.0
40.7
10.8
50.7
19.2
7.5
Results. As shown in Table 2, we observe sharp performance improvements 10 from ICL-HAR over the baselines, particularly in the coherence metrics of consistency and verifiability, where our proposed strategy primarily comes into play. Meanwhile, compared to both baselines, HAR improves In- structGPT consistency TRIP from 40.7% up to 47.9%, and verifiability from maximum of 10.8% up 23.9 33,9% over a 100% improvement on the lat- ter. LLaMA sees similar improvements, especially verifiability. On Tiered-ProPare InstructGPT's consistency improves from 19.2% to 31.5%, and verifiability improves from 7.5% to 20.7%, nearly a 200% improvement on the latter. LLaMA again sees similar improvements. These results demon- strate that compared common approaches for in-context learning with PLMs, human-inspired heuristic-analltic reasoning can significantly im- prove coherence and reduce hallucination.
ICL-HAR
72.6
47.9
23.9
54.9
31,5
20.7
L.LMA
TRIP Cons.
Tiered-ProPara
Approach
Acc.
Ver.
Acc.
Cons.
Ver
ICL-U ICL-CoT
70.4
42.3
14.8
51.2
3,8
1.4
74.6
42.3
19.7
57.3
9.4
4.2
ICL-HAR
55.6
44.4
35.2
41.8
17.8
13.1
Table Accuracy, consistency, and verifiability per- centages for in-context learning with heuristic-analytic reasoning (ICL-HAR) PLMs, compared an unstruc- tured in-context learning (ICL-U) baseline that tackles reasoning steps through separate focused prompts.
tion 3.1, it is not idcal for accuracy to far exceed consistency and verifiability, each of which require end-task predictions to be accurate as a prerequisite, as this indicates incoherent reasoning with insuffi- cient support for the end-task prediction. Therefore, drops in accuracy with HAR are necessarily problematic. Nonetheless, we believe it occurs due to the smaller complexity of LLaMA (65B), making it more sensitive to long prompts and gener- ations. quick fix could be to sequentially prompt the PLM multiple times with shorter prompts for each reasoning step, where the prompt for each step is informed by the previous step's 's prediction. We explore this approach in Appendix A.2, and indced find the accuracy does not drop with HAR.
As expected, traditional chain-of- siint-oftthoughh in the ICL-CoT baseline brought only marginal improve- ments in most cases, especially for verifiability, as physical state prediction (the bottleneck in coherent physical commonsense) cannot be further decom- posed. Instead, we saw that the free-form explana- tions generated by InstructGPT for physical state prediction typically repeated specific sentences and actions from the story, which introduced no new information. ICL-HAR reveals possible new use case for ofain-of-thhought-syy prompting: refin- ing PLM attention the most relevant language context. We investigate this further in Section 5.2. Interestingly, HAR not only brought vast im- provements on verifiability metrics, bul also some improvements in consistency. In other words, the mid-level sentence selection tasks benefited slightly from being conditioned on higher-level story se- lection tasks, despite being considered heuristic processes that simply refine the context. This may suggest that rather than belonging to separate dual processes, these consecutive steps of reasoning may fall along a spectrum from heuristic to analytic pro- cesses a recursive manner in PLMs.
5.2 Faithful Attention in ICL-HAR
To explore possible reasons for why HAR strength- ens PLMs' reasoning, we last compare and ex- amine models' attention weights when generating language in our in-context learning experiments, where the model has access to the entire input lan- guage context throughout inference. Earlier in this section, we hypothesized that ICL-HAR enables the model focus in on key parts of the context to make decisions and rationalize them more faith- fully, similar to its role in human reasoning. For example, as shown in Figure 4, after a PLM identi- fies the implausible story in TRIP, it should attend more to that story when identifying conflicting sen- tences. After identifying these sentences, it should attend more those sentences when generating physical commonsense evidence.
We also observed that LLaMA's accuracy de- creased with ICL-HAR. As discussed in Sec-
"Statisticaal significance testing of in-context learning per- formance improvements in Appendix A.I.
In order to validate this hypothesis, we ag-

---

gregate and normalize transformer self-attention weights for each story or sentence within the in- put prompt," 11 then use the ground truth reasoning chains for TRIP and Tiered-ProPutte to evaluate the faithfulness of them. To our knowledge, prior work has not studied attention weights in this way for in-context learning with PLMs, so we hope to yield new insights on the nature of reasoning with them. We next introduce the evaluation criteria used for attention weights, then the results of the analysis.
language context produce each level of the rea- soning chain. We expect that higher ratios indicate more faithful rationalizations from the model.
A Attentional precision and recall. Beyond the faithfulness of attention, we would like to under- stand how faithful attention relates to coherent rea- soning, i.c., the PLM's predicted sentence(s) and physical state(s) to rationalize which story chose. For each of these reasoning steps, there are four possible combinations of faithfulness of model at- tention and correctness of its predictions:
5.2.1 Attention Evaluation Criteria
We propose two kinds of measures to capture the faithfulness of attention and its relationship with coherence in TRIP and ratios and attentional precision and recall.
1. Attends to the correct context, and generates a correct prediction (true positive)
attentional
2. Attends to the correct context, but generates incorrect prediction (false positive)
Attentional ratios. To evaluate the faithfulness of PLMs attention, we can compare the attention weights for the correct segments of language con- text (i.e., stories or sentences) versus others through attentional ratio. In both TRIP and ProPara, the model must first identify one of two stories contain- ing some physical phenomenon before identifying which sentence(s) in that story contain it (sentence selection step). In TRIP, in cases where it correctly identifies the implausible story, we can calculate the attentional ratio for sentence selection by tak- ing the ratio of the mean attention weight of the implausible story (i.e., where the PLM must at- tend to identify confficting sentences) to that of the plausible story. Similarly, in Piered-rroPara when the model correctly identifies which story contains an entity conversion, we calculate the ratio of the mean attention weight for the story containing an entity conversion to that of the story that does not.
3. Attends to the incorrect context, and generates an incorrect prediction (true negative)
4. Attends to the incorvect context, but generates an correct prediction (false negative)
We can calculate the precision and recall of atten- tion to measure how the correctness of attended lan- guage context correlates with correctness of these model predictions (and thus coherence of reason- ing). Given a set of evaluation examples, we define attentional precision the number of true pos- itives divided by all positives, representing how often the PLM is correct given faithful attention. We define attentional recall as the number of true positives divided by the sum of true positives and false negatives, representing how often the PLM attends faithfully given a correct prediction. To- gether, these metrics can provide an impression of the connection between faithful attention and coher- ent reasoning under different prompting strategies.
When the model correctly identifies which sen- tence(s) contain a phenomenon (i.e., a plausibility conflict or entity conversion), the model must lastly generate physical commonsense knowledge for those sentences to rationalize its decisions (phys- ical state prediction step). In TRIP, we calculate the attentional ratio for physical state prediction by taking the ratio of the mean attention weight for conflicting sentences (i.e., sentences from which physical states must be predicted) to that of all other sentences. In Tiered-ProPaic we similarly calcu- late the ratio between the mean attention weights for the conversion sentence and other sentences. Together, these ratios can provide a sense of how strongly the PLM is attending to the relevant
5.2.2 Attention Analysis Results
To understand why HAR improves coherence so significantly, we compare PLM self-attention dis- tributions as a reasoning chain is generated in the in-context learning setting. As this analysis re- quires access to PLM internals, we can only use open-source models like LLaMA here. Following findings from prior work that the middle layers of wansformer-basee language representations con- tain the most transferable semantic information (Peters et al., 2018: Tenney et al., 2019), we ex- tract self attention weights from the center-moss 20 layers of the transformer backbone in LLaMA.
Atteation faithfulness classified by threshold. More details and supporting examples listed in Appendix E,2,
Il More details in Appendix E.I.

---

Sentence Selection Step
versions) and predicting states of entities. While observed ratios show that ICL-HAR has more faith- ful attention to relevant parts of the context, the high values of attentional precision (up to 80%) and recall (up to 98%) show that faithful attention and coherent reasoning go hand-in in-hand; faithful attention in ICL-HAR is likely to bring coherent reasoning, and vice-versa. This demonstrates that HAR enables more trustworthy reasoning in PLMs.
TRIP
Tiered-ProPara Prec.
Approach
Ratio
Prec.
Rec.
Ratio
Rec.
ICL-U
0.96
42.6
39.6
0.90
14.8
30.6
ICL-HAR
1.07
75.2
48.7
1.80
51.1
58.2
Physical State Prediction Step
TRIP
Tiered-ProPara Prec.
Approach
Ratio
Prec.
Rec.
Ratio
Rec.
ICI-UU
1.23
43.0
35.4
1.21
14.6
25.9
ICL-HAR
1.95
79.8
98.2
2.20
72.1
83.3
Lastly, we present example visualizations of self- attention patterns (Yang and Zhang, 2018) for ICL.- HAR compared to the ICL-U baseline. In Figure 5, see that in the sentence selection step in Tiered- ProPara, LLaMA had higher average attention on the story containing a conversion of energy under ICL-HAR than the baseline. Similarly, in Figure 6, we see that under ICL-HAR, LLaMA paid more attention to the conflicting sentences in the physical state prediction step in TRIP, whereas the baseline had high attention on irrelevant sentences. This shows how HAR can help focus PLMs the most relevant language context at each step of reasoning. More visualizations are provided in Appendix A.3.
Table 3: Attentional ratio, average precision (%). and av- erage recall (%) for LLaMA baseline and HAR strategy, during different physical commonsense reasoning steps. Precision and recall averaged across several attention thresholds, as outlined in Appendix E.2.
Story A: 11.0%
1. Cool heatec, in the booler.
2. The water tonk over the
heated.
3. Chentes steam.
4. The staem is fonmuded to the piston.
5. Piston uses the
eneggy
6. The paston couses the crankshaftt to move
Story B. 59.02
1. Plates
Earth's
slowly post other.
2. the ploles move, they exert a greal force
3. When the force is lange enough. she crest breskes)
relessed 5. The enesgy moves through Earth
1. The
of woves
6. We icel the ourthquokk
Story A: 15.9%
1. beated the
Conclusion and Future Work
2. The water toak over the boller is heated,
3. Cheates steam.
4. The stasom finnuded the piskon.
In this work, we took inspiration from the syn- ergy between heuristic and analytic processes in hu- man reasoming to explore how high-level decision- making tasks in commonsense reasoning can con- dition and drive lower-level rationalization tasks supporting them in PLMs. We proposed two gen- eral strategies to integrate heuristic-analyiie rea- soning (HAR) into PLM fine-tuning and in- in-context learning, and found that HAR sharply improved rea- soning coherence, outperforming competitive base- lines on two benchmark tasks. In fine-tuning, we saw that HAR enabled PLMs to learn to reason not only more coherently, but also faster. Meanwhile, in in-context learning, we found that improvements were enabled by more faithful attention to the lan- guage context within each step of reasoning, shed- ding light on the nature of incoherence in language generation. While this human-insping approach shows promising strides toward more trustworthy reasoning from PLMs, future work should continue to dive deeper into cognitively motivated strategies to further strengthen coherent reasoning in Al sys- tems and improve human-machine alignment." 13
5. Pistco uses the steom as meegy:
6. The piston causes the conbosialt to move
Stocy 83.7%
Plates
Earth's crust move slawly past each other
2. As the plates move, they exert a great fares)
3. When the force is lange enough, the ened. limakke
The strees relessed as energyy
5. The enesgy moves through the Earth in the form oE waves
We icel the carthquoke
Figure 5: Attention visualization Tiered-Prol's in selecting which sentence energy is converted in, baseline ICL.UU (top) vs. ICL-HAR (bottom). Attention averaged across stories and reflected by the intensity of color.
be
5.08
Tous bound
Ton geoled hard bolted 66
Tom pentod had botted 7.18
Tom aloud the with hntte 5.63
2. Tun aboud
with audi 2.38 4. Tous waaked the laite the stale a.st
Tom wasled the indi the sinti 9.42
5. Yom ate tos cream Ser draim 8.42
5. Tim ais ioe crears for drownt 18.44 Story is
Story B
Tom pouned glass of milk: 10.
1. Tom gooned a glos willi 7-28
Too pertod i hand botted egz) 25.43
2 Toos perkod hood botted egg Rlby
waaled oge with kndoo 2.38
Tous alood ope annis 2:45
25.28
Tous weaked liste stali 20.0x
12.55
Figure 6: Sentence- wise attention visualization on TRIP in state change prediction for baseline ICL-U (left) vs. ICL-HAR (right). Attention averaged across sentences.
As shown in Table 3, ICL-HAR sharply ex- ceeds the unstructured ICL-U baseline across all attentional metrics, both when selecting sentences (i.e., confficting sentences or sentences with con-
I3 Source code to reproduce results at https: thub. com/sled- hem//seedgroo//Herriieeee proup/Heuristic-Anaee -eaatiiiea...nnng..

---

Acknowledgements
tant and under-explored task, evidenced by cur- rent PLMs' tendency to perform incoherent reason- ing and hallucinate, making them difficult to use for reasoning in practice. Providing detailed evi- dence for conclusions in reasoning is especially im- portant in physical commonsense, as agents must have deep understanding of the possible implica- tions of actions on the enviromment around them in order to safely collaborate with humans. Fur- ther, despite our focused scope, our work proposes attention-
This work was supported by LG Al Research, We would like thank the anonymous reviewers for their valuable comments and suggestions.
Limitations
Cascading errors in HAR. In all presented ap- proaches, model outputs may be influenced by de- cisions on higher-level tasks, whether explicitly through modifying language context remove in- formation based on previous decisions, or more indirectly from in-context learning. As the model cannot backtrack from incorrect decisions, this can cause cascading errors when the model makes a mistake. For example, the model predicts the incorrect conflicting sentences, there is no way for it to recover in lower-level tasks. While our goal was to demonstrate that heuristic decisions can be used to improve performance analytic rational- izations in PLMs, more robust, non-greedy decod- ing strategies may help alleviate this problem. For example, a beam search or more recent work tree search for reasoning in in-contexi learning could be applied (Yao et al., 2023; Long, 2023) to gener- ate multiple candidate reasoning chains and choose one based on language model likelihoods. In fact, such strategies could strengthen all coherent rea- soning approaches studied in this work, including those from prior work. As this effort is thus orthog- onal to our goal and would increase the number of variables in our study, we chose not to explore these approaches here and avoid distracting from our key message. Furthermore, this requires the ability to systematically extract and compare all intermediate reasoning steps. which is possible due to the dense annotations in TRIP and ProPara, but may not be possible in all applications.
coherence metrics upon existing strategies for evaluating reasoning in PLMs. This enables future work to dive deeper into this prob- lem across domains, and better characterize the do- mains where various strategies for applying PLMs are helpful. Findings from such efforts may some- day lead to the integration of such reasoning strate- gies into LM pre-training from the ground up improve their reliability.
Ethics Statement
This works deals with applications of large pre- trained language models (PLMs), which are prone to bias in their training data. Before deploying PLMs for real-world reasoning applications that in- terface with users, efforts must be taken mitigate such bias. Bender et al. (2021) provide an overview of such possible dangers of PLMs and recommend some solutions. On the other hand, by emphasiz- ing consistent and verifiable reasoning, this work brings potential positive impact toward trustwor- thiness and transparency in interacting with these systems, especially for embodied AI applications where coherent physical commonsense reasoning necessary for safety of human users.
References
Thomas Anthony, Zheng Tian, and David Barber. 2017. Thinking fast and slow with deep learning and tree search. Advances in Neural Information Processing Systems, 30.
Generalizability of approach. A possible limi- tation of this work is that our approach is suited to coherent reasoning tasks where a high-level deci- sion needs be supported by lower-level reasoning which depends on small segments of the language context containing key information. We argue that problems requiring souristic-analytic reasoning are actually quite prevalent in the real world; for exam- ple, when debugging code, an experienced devel- oper will localize bug to small segment of the code before closely tracing through it solve the problem.
Emily M. Bender, Timnit Gebru, Angelina McMillan- Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language mod- els be too big? In Proceedings of the 2021 ACM Conference Fairness, Accountability, and Trans- parency, FAceT '21, page 610-623, New York, NY, USA. Association for Computing Machinery.
Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. 2020. PIQA: Reasoning about Physical Commonsense in Natural Language. Proceedings of the Thirth AAAI Conference
Furthermore, coherent reasoning is an impor-

---

on Artificial Inteligence (AAAI-20), New York, NY, USA. AAAI Press.
Jonathan St BT Evans. 2010. Intuition and reasoning: A proccss perspective. Psychological Inquiry. 21(4):313-32)
Joan Bliss. 2008.
Commonsense reasoning about the physical world. Studies in Science Education, 44(2): 123-155.
Jonathan St.B.T. Evans. 2003. In minds: dual-
process accounts of reasoning. Trends in Cognitive Sciences, 7(10):554-43 459.
Grady Booch, Francesco Fabiano, Lior Horesh, Ki- ran Kate, Jonathan Lenchner, Nick Linck, Andreas Loreggia, Keerthiram Murgesan, Nicholas Mattei, Francesca Rossi, and Biplay Srivastava. 2021. Think- ing fast and slow in ai. Proceedings of the AAAI Conference Artificial Intelligence, 35(17):15042- 15046.
Jonathan St.B.T. Evans. 2011, Dual-process theories of reasoning: Contemporary issues and developmental applications, Developmental Review, 31(2):86-102 Special Issue: Dual-Process Theories of Cognitive Development.
Maxwell Forbes and Yejin Choi. 2017. Verb Physics: Relative Physical Knowledge of Actions and Objects. In Proceedings of the 55th Anmual Meeting of the As- sociation for Computational Linguistics (ACL 2017), Vancouver, BC, Canada. Association for Computa- tional Linguisties.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neclakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in Neural Information Processing Systems, 33: 1877
Marianna B. Ganapini, Murray Campbell, Francesco Fabiano, Lior Horesh, Jon Lenchner, Andrea Loreg- gia, Nicholas Mattei, Taher Rahgooy, Francesca Rossi, Biplav Srivastava, and Brent Venable. 2022. Combining fast and slow thinking for human-like and efficient navigation in constrained environments.
Bhavana Dalvi, Lifu Huang, Niket Tandon, Wen-tau Yih, and Peter Clark. 2018. Tracking state changes in procedural text: challenge dataset and models for process paragraph comprehension. In Proceedings of the 2018 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long Pu- pers), pages 1604, New Orleans, Louisiana. Association for Computational Linguistics.
Mor Geva, Ankit Gupta, and Jonathan Berant. 2020. Injecting numerical reasoning skills into language models. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 946-958, Online. Association for Computa- tional Linguistics.
Bhavana Dalvi, Peter Jansen, Oyvind Tafjord, Zhengnan Xie, Hannah Smith, Leighnnnaa Pipatanangkura, and Peter Clark. 2021. Explaining answers with entail- ment trees. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Process- ing, pages 7358-7370, Online and Punta Cana, Do- minican Republic. Association for Computational Linguistics.
Suchin Gururangan, Swabha Swayamdipta, Omer Levy. Roy Schwartz, Samuel Bowman, and Noah A. Smith. 2018. Annotation artifacts in natural language infer- ence data. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume 2 (Short Papers), pages 107-112, 12, New Orleans, Louisiana. Association for Computa- tional I. Linguistics.
Jacob Devlin, Ming-Wei Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume I (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Wenyue Hua and Yongfeng Zhang. 2022, System + system 2 = better world: Neural-symbunt chain of logic reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 601-612, Abu Dhabi, United Arab Emirates. Associ- ation for Computational Linguistics.
Nouha Dziri, Hannah Rashkin, Tal Linzen, and David Reitter. 2022. Evaluating attribution in dialogue sys- tems: The BEGIN benchmark. Transactions of the Association for Computational Linguistics, 10:1066- 1083.
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang. Andrea Madotto, and Pascale Fung. 2023. Survey of halluci- nation in natural language generation. ACM Comput. Surv, 55(12).
J St BT Evans and Peter C. Wason. 1976. Rationalization in reasoning task. British Journal of Psychology. 4844):4794486
Daniel Kahneman. 2011. Thinking, Fast and Slow, chap- ter 21. MacMillan, New York.
Mehran Kazemi, Najoung Kim, Deepti Bhatia, Xin Xu, and Deepak Ramachandran. 2023. LAMBADA: Backward chaining for automated reasoning nat- ural language. In Proceedings of the 61st Annual
Jonathan St B. T. Evans. 1984, Heuristic and analytic processes in reasoning*. British Journal of Psychol- ogy, 75(4):451-44)

---

Meeting of the Association for Computational Lin- guistics (Volume 1: Loong Papers), pages 6547-6566 6568, Toronto, Canada. Association for Computational Lin- guistics.
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Car- roll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instruc- tions with human feedback.
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yu- taka Matsuo, and Yusuke Twasawa. 2022. Large lan- guage models are zero-shot reasoners. In Advances in Neural Information Processing Systems.
Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the 2014 Confer- ence on Empirical Methods in Natural Language Pro- cessing (EMNLP), pages 1532- 1343, Doha, Qatar. Association for Computational Linguisties.
Brenden M Lake, Tomer D Ullman, Joshua B Tenen- baum, and Samuel J Gershman. 2017. Building ma- chines that learn and think like people. Behavioral and Brain Sciences, 40:e253.
Matthew E. Peters, Mark Neumann, Luke Zettlemoyer, and Wen-tau Yih. 2018. Dissecting contextual word embeddings: Architecture and representation. In Pro- ceedings of the 2018 Conference on Empirical Meth- ods in Natural Language Processing, pages 1499- 1509, Brussels, Belgium. Association for Computa- tional Linguistics.
Yann LeCun. 2022. A path towards autonomous chine intelligence version 0.9. 2, 2022-06-27. Open Review, 62.
Yoonjoo Lee, Kyungjae Lee, Sunghyun Park, Dasol Hwang, Jachyeon Kim, Anthony Moi, Hong-in Lee, and Moontae Lee. 2023. Qasa: Advanced question answering on scientific articles. In Proceedings of the 39th International Conference on Machine Learning.
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, al. 2018. Improving language under- standing by generative pre-training. OpenAl.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, llya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAl blog, 188):9.
Belinda Z. Li, Maxwell Nye, and Jacob Andreas. 2023. I Language modeling with latent situations. In Find- ings of the Association for Computational Linguistics: ACL 2023, pages 12571,, Toronto, Canada, As- sociation for Computational Linguistics.
Nazneen Fatema Rajani, Bryan McCann, Caiming Xiong, and Richard Socher. 2019. Explain your- self! leveraging language models for commonsense reasoning. In Proceedings of the 57th Annual Meet- ing of the Association for Computational Linguistics, pages 4932-4942, Florence, Italy. Association for Computational Linguistics.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretrain- ing Approach.
Jieyi Long. 2023. Large language model guided tree-of- thought.
Kyle Richardson, Ronen Tamari, Oren Sultan, Dafna Shahaf, Reut Tsarfaty, and Ashish Sabharwal. 2022. Breakpoint transformers for modeling and tracking intermediate beliefs. In Proceedings of the 2022 Con- ference on Empirical Methods in Natural Language Processing, pages 9703-9719, Abu Dhabi, United Arab Emirates. Association for Computational Lin- guistics.
Kaixin Ma, Filip Ilievski, Jonathan Francis, Firie Nyberg, and Alessandro Oltramari. 2022. Coalescing global and local information for procedural text understand- ing. In Proceedings of the 29th International Con- ference on Computational Linguistics. pages 1534- 1545, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. 2019. Social IQa: Com- monsense reasoning about social interactions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joins Conference on Natural Lan- guage Processing (EMNLP- IJCNLP), pages 4463- 4473, Hong Kong, China. Association for Computa- tional Linguistics.
Gary Marcus. 2020. The next decade in Al: Four steps towards robust artificial intelligence.
Quinn McNemar. 1947. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2):153-157
Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, Charles Sutton, and Augustus Odena. 2022. Show your work: Scratchpads for intermediate computation with language models. In International Conference Learning Representations 2022.
Roy Schwartz, Maarten Sap, Ioannis Konstas, Leila Zilles, Yejin Choi, and Noah A. Smith. 2017. The effect of different writing tasks linguistic style: A case study of the ROC story cloze task. In Pro- ceedings of the 2 Conference on Computational Natural Language Learning (CoNLL 2017). pages 15-25, Vancouver, Canada. Association for Compu- tational Linguistics.
OpenAl. 2023. GPT-4 technical report.

---

Evangelia Spiliopoulou, Artidoro Pagnoni, Yonatan Bisk, and Eduard Hovy. 2022. EvEntS ReaLM: Event reasoning of entity states via language models. In Proceedings of the 2022 Conference on Empiri- cal Methods in Natural Language Processing, pages 1982-1997, Abu Dhabi, United Arab Emirates. Asso- ciation for Computational Linguistics.
Thomas Wolf. Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier- rie Cistac, Tim Rault, Remi Louf, Morgan Funtow- icz, Joe Davison, Sam Shleifer. Patrick von Platen, Clara Ma. Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush, 2020. Trans- formers: State-of -hee-----hhheeeee art natural language processing. In Proceedings of the 2020 Conference Empirical Methods Natural Language Processing: System Demonstrations, pages 38 38-45, Online, Association for Computational Linguistics.
Shane Storks and Joyce Chai. 2021. Beyond the tip of the iceberg: Assessing coherence of text classifiers. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3169-3177, Punta Cana, Dominican Republic. Association for Compu- tational Linguistics.
Tongshuang Wu, Ellen Jiang, Aaron Donsbach, Jeff Gray, Alejandra Molina, Michael Terry, and Carrie Cai. 2022. Promptchainer: Chaining large language model prompts through visual programming. In CHI Conference on Human Factors in Computing Systems Extended Abstracts, pages 1-10.
Shane Storks, Qiaozi Gao, Yichi Zhang, and Joyce Chai. 2021. Tiered reasoning for intuitive physics: Toward verifiable commonsense language understanding. In Findings of the Association for Computational Lin- guistics: EMNLP 2021, pages 4902-4918, 18,, Punta Cana. Dominican Republic. Association for Compu- tational Linguistics.
Jie Yang and Yue Zhang. 2018. NCRF++ An open- source neural sequence labeling toolkit. In Proceed- ings of the 56th Annual Meeting of the Association for Computational Linguistics.
Oyvind Tafjord, Bhavana Dalvi, and Peter Clark. 2021. ProofWriter: Generating implications, proofs, and abductive statements over natural language. In Find- ings of the Association for Computational Linguis- fics: ACL-IICNLP 1021, pages 6621-3634, Online. Association for Computational Linguistics.
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao. and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models.
lan Tenney, Patrick Xia, Berlin Chen, Alex Wang. Adam Poliak, R Thomas McCoy, Najoung Kim, Ben- jamin Van Durme, Sam Bowman, Dipanjan Das, and Bllie Pavlick. 2019. What do you learn from con- text? probing for sentence structure in contextualized word representations. In International Conference on Learning Representations.
Eric Zelikman, Jesse Mu, Noah D. Goodman, and Yuhuai Tony Wu. 2022. Star: Self-taught reasoner bootstrapping reasoning with reasoning. Advances in Neural Information Processing Systems.
Rowan Zellers. Ari Holtzman, Matthew Peters. Roozbeh Mottaghi, Aniruddha Kembhavi, Ali Farhadi, and Yejin Choi. 2021. PIGLeT: Language grounding through neuro-symbolic interaction in a 3D world. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Lan- guage Processing (Volume 1: Long Pupers), pages 2050-2050, Online. Association for Computational Linguistics.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet. Marie-Ann Lachaux. Timothée Lacroix, Baptiste Rozière, Naman Goyal, Fric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. LLaMA: Open and efficient foundation language models.
Takeo Tsujii and Shigeru Watanabe. 2009. Neural cor- relates of dual-task effect belicf-bias syllogistic reasoning: near-infrared spectroscopy study, Brain research, 1287:118-121
Ben Zhou, Kyle Richardson, Qiang Ning. Tushar Khot, Ashish Sabharwal, and Dan Roth. 2021. Temporal reasoning on implicit events from distant supervision. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 1361-1371, Online. Association for Computa- tional Linguistics.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations.
Peter C. Wason and St BT Evans. 1974. Dual processes in reasoning? Cognition, 3(2):141-154
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoc V Le, and Denny Zhou. 2022. Chain of thought prompt- ing elicits reasoning in large language models. In Advances in Newral Information Processing Systems.

---

A Supplementary Results
Story
1. Air brought through the mouth.
2. Passes through the lungs.
Here, we include several supplementary results that could not fit in the main body of our paper, but shed more light on PLM performance under HAR- inspired strategies.
And
branchial carbon dionde removed.
5. The lungs bring the uregen the rest of the body. Story 6:
Earbon diocede enters leaves through the stomatees diffusion.
Energy transported to
the sylem.
through light reaction
by forming ATP,
Carbon dioviee and energy from ATP are used create sugar
A.1 Statistical Significance Testing
5. Oeggen eoits the leoves through the stamats by diffusion.
6. The
water the water exits through the stomata...
While the performance gain in consistency and verifiability from our HAR strategies over base- line approaches was quite large in most cases, we performed McNemar's s tests to measure the statis- tical significance of these gains (McNemar, 1947). We found that in the TRIP and Tirred-ProPara re- sults presented in Table 1, the differences in consis- tency and verifiability between FCGLI-HAR and the FCGLI baseline were statistically significant 0.05). This was also true for the differences in consistency and verifiability on both tasks be- tween ICL-HAR and both ICL-U and ICL-CoT in Table 2. except for LLaMA on TRIP, where the consistency gain from ICL-HAR over either base- line was not significant. As expected. HAR indeed brings about statistically significant performance gains in verifiability of physical commonsense rea- soning, while sometimes also significantly improv- ing consistency.
PLM
"Carbon dioxide converted story B:"
Story:
1. Carbon diovide enters the kaves through the stomates by diffusion.
transported to
sylem.
Energy harvested through light reaction stored by forning ATP:
Carbon discate energy
ATP are usrd create sugar
5. Oesgen exis the leoves through the stomats by diffusion.
6. The plant reuvs the water the water eehs through the stomata...
PLM
"aarbon dioside is converted in sentence 4."
Carbon diowide and energy from ATP are used to create sugar
PLM
CCrrbon dioside converted to sugar."
Figure 7: Heuristic-analylee reasoning with prompt chaining for in-context learning with PLMs on phys- ical commonsense rationalization (PCICL-HAR), On Tiered-ProPara, the PLM will first decide which story conversion of an entity occurs in, then this will be used to refine the language prompt before asking the PLM which sentence the conversion occurs in. Lastly, the chosen sentence will be used to predict the resulting entity after conversion.
A.2 HAR for Multi-Prompt In-Context Learning
On top of the chain-of-thought (CoT) implemen- tation of HAR for in-context learning presented in the paper, we experimented with a more strict form which chained multiple prompts together, one for each step of the reasoning tasks. This adds explicit structure that may make it more dependable. As shown in Figure 7, each successive prompt is gener- ated based on the previous higher-level prediction. In TRIP and Tiered- ProPara, PLMs must first select one of two stories in which some phenomenon oc- curs (i.e., implausibility or conversion of entities). Then, the chosen story is used in a separate prompt to that PLM and predict the sentence(s) where the phenomenon occurs. Lastly, given predicted sen- tence(s), PLMs should predict the specific states underlying the phenomenon. While similar to the approach presented in the paper, this approach en- ables us to completely remove irrelevant informa- tion from the context each step of reasoning and rationalization, so that the model can focus only on the correct parts of the language context. While this restriction may help PLMs ignore irrelevant con-
text, this structure limits the general applicabilty of this approach compared to the more flexible chain- of thought approach presented in the paper.
As shown in Tables 4 and 5, prompt chaining for in-context learning with HAR (PCICL-HAR) also brings performance improvements over the ICR- U baseline both TRIP and ProPara. In Tiered- ProPara, PCICL-HAR slightly exceeds HAR with chain-of-thought (ICL-HAR) in consistency when applied to both InstructGPT and LLaMA (up to 36.2% and 21.6% respectively), and in verifiability with LLaMA (up to 17.4%). This suggests that in some cases, more explicitly structured HAR can be beneficial.
Tiered-ProPara,
A.3 Attention Weight Visualizations
We present additional visualizations of self- attention patterns under in-context learning with heuristic-analytic reasoning (ICL-HAR), compared
As mentioned in Section 5.1, LLaMA is evaluated on a subset of TRIP, so in-context learning results on different PLMs are not directly comparable.

---

InstructGPT
in a later sentence. Other stories have implicit
Approach
Accuracy
Consistency
Verifiability
ICL-U
70.9
40.7
7.1
For example, if Mary put the cucumber on a plate and toxsed the donut in the trash, then in a later sentence, Mary ate the dona, the effect of the former sentence (i.e. donut is inedible) conflicts with the precondition of the latter sentence (i.e. donut is edible).
ICL-CoT
75.0
40.7
10.8
ICL-HAR
72.6
47.9
23,9
PCICL-HAR
70.4
39.6
12.8
LLaMA
Approach
Accuracy
Consistency
Verifiability
A: 12
ICL-U
70.4
42.3
14.8
1. The book was on the tabble
ICI-CoT
74.6
42.3
19.7
2. Sazah took the book the copier:
ICL-HAR
55.6
44.4
35.2
3. copied fow pages
stapler koop them logatharr
4. Sarth could not find ber botebook. to the knyboord)
PCICL-HAR
70.4
40.8
28.2
Sazsh put
Story B: 48.9%
1. The book
the table,
Table 4: TRIP results of heuristic-analytic reasoning (HAR) strategies in in-context learning with PLMs, in- cluding PCICL-HAR, the prompt-chaining alternative to ICL-HAR."
2. Sarsh
book cxquior
a. Sazah copled few pages and used the stapler keeop them together,
4, Sazah put the peges inside notebook.
5. Sarah put the notebook next to the keyboard.
Story A: 61.7%
I. The book
the table
InstructGPT
2. Sarah took book the copter
3. Sarth eapied low and used the stapler koop them logathor could
Approach
Accuracy
Consistency
Verifiability
het
5. Sarsh put the notebook nest to the knybooeed]
ICL-U
54.9
17.4
5.2
Story B: 38.33
ICL-CoT
50.7
19.2
7.5
book was on table
ICL-HAR
54.9
31.5
20.7
2. Sarah took the book to the copter
Sazsh copied few papes used stapler keep logether
PCICL-HAR
54.5
36.2
18.8
4.
Sarch put the pages inside her notebook
5. Sarah put the notebook next to the keyboard.
LLaMA
Approach
Accuracy
Consistency
Verifiability
Figure 8: Story-wits wise attention visualization on TRIP in sentence-of-convene detection, ICL.U (top) vs. ICL. HAR (bottom).
ICL-U
51.2
3,8
1.4
ICL-CoT
57.3
9.4
4.2
ICL-HAR
41.8
17.8
13.1
PCICL-HAR 51.2
21.6
17.4
Story A:
1. Plants have roots, 10, 8%
youw 3. Roots have fibers that are attached to them. B.
2.
Table S: Tiered-ProPaan ProPara results couristic-anllyee reasoning (HAR) strategies in in-context learning with PLMs, including PCICL-HAR, the prompt-chainigg alternative to ICL-HAR.
5. They attract water, 6.72
They sack up water, 1%
6. They absorb the water, 12.
1. The air cold, 6.0%
2. Water in the 1.4%
3. water lorms lhn ieo cryslals, 9.9%
our unstructured in-context learning (ICL-U) baseline, From Figure and 8, we found that when our HAR method was applied, in second-level sen- tence selection tasks, the model had higher average attention on the story of interest compared with the baseline. We suspect that this increase in average attention helped increase the performance. Simi- larly, from Figure 6 and 9, we found that after HAR method was applied, the model paid higher average attention to the sentence of interest in the third-level physical state prediction tasks.
4. The ice crystals collide with cach other, 4.3%
ice crystale stick
3.0%
6. The ice crystals get bigger as more them stick together. 9.4X
7. The ies cryslals gel. loo losvy to bu the air. 3.8% crystals
snowflokes. 9.4%
9. The snow flaess bxll to the ground as snow: 8.
What hapgemasl water?
Story A:
1. Plants lave roots. 2.58
roots grow 1.6%
3. Roots have fibers that are attached to shem. 3.48
They
2.8%
5. They suck up water. 3.2%
G. They abworb the water. 11.4%
Story
1. The sir is cold, 4.4g
2. Water in the
3. The water forue thny ice crystals 26
4. The ios erystals ovilide with ouch other 9.98
crystals
other. 4.9%
A.4 Implicit vs. Explicit Conflicts on TRIP
6. The ics crystals gei bigger as more of them stick together: 6.5%
7. lee erystals get too heavy he the 4.0%
8. The ics crystals become snowflakes. 8.6%
Plausibility conflicts between physical states in TRIP can take two forms. First, explicit conflicts like the one shown in Figure 4 exhibit direct dis- agreements in physical effect states of a particular entity after one sentence and its precondition states
9. The snow flakes fall to the ground as snom: 6.48
What happened
Figure Sentence- wise attention visualization on Tiered-Pro ProPara in entity conversion prediction, ICL-U (top) vs. ICL-HAR (bottom).

---

Explicis Conflicts
B Tiered-Pro Generation Details
Approach
Accuracy
Consistency
Verifiability
Tiered- Prred-a was generated by a simple match- ing process over the original dataset. We first re-split the dataset to distribute more stories into the testing and development sets from training set. Then, we applied pairwise matching process select two stories. Specifically, the selected entity appears in both stories, but only converts to another entity in one of them. Also, the entity being converted must disappear after the conversion, while the entity converted must appear only after the conversion. The selected entity is pro- vided as a known fact before the inference. There- fore, similar to TRIP, system needs to predict which story has the conversion of the known en- tity given two stories, measured by accuracy. And Consistency is used to measure how often system predicting the correct story and the sentence that has the conversion. Additionally, Verifiability mea- sures how often a system identifies the fully correct reasoning chain, from story and sentence predic- tion to physical state prediction of understanding what entity has converted to. After conversion of the dataset, there are 496 training instances, 206 development instances, and 213 testing instances.
ICL-U
71.7
42.2
10.0
ICL-CoT
73.9
41.7
16.7
ICL-HAR
72.2
53,3
36.1
PCICL-IIAR
70.6
47.2
17.8
FCGLI
98.3
81.1
56.7
FCGLI-HAR
98.7
89.4
68.9
enforce that
Implicit Conflicts Consistency
Approach
Accuracy
Verifiability
ICL-U
70.2
39.2
ICL-CoT
76.0
39.8
4.7
ICL-HAR
73.1
42.1
11.1
PCICL-HAR
70.2
31.6
7.6
FOGLI
88.7
50.2
9.3
FOGLI-HAR
89.5
60.3
11.3
Table 6: TRIP results of heuristic-analyee reasoning (HAR) strategies in in-context learning with Instruct- GPT (top) and fine-tuning FCGLI (bottom) for explicit and implicit plausibility conflicts.
conflicts where no such disagreement exists, though the story may violate commonsense expectations. 16
While our evaluation on LLaMA focused on ex- plicit conflicts to ensure only the top-6 most com- mon physical attributes could be used to rationalize plausibility conflicts, our evaluation on Instruct- GPT included both. As the connection between physical states and plausibility conflicts is unclear in implicit conflicts and our HAR strategies intended to strengthen lower-level rationalization support higher-level tasks, we may expect them to be especially beneficial for explicit conflicts. Ta- ble 6 includes results for both types of conflicts on the fine-tuning and in-context learning approaches introduced so far. We see that implicit conflicts are more difficult to rationalize across the board, even with traditional chain-of-thought as applied in the ICL-CoT baseline. However, HAR indeed has much more significant impact on TRIP examples with explicit conflicts, increasing verifiability from 10.0% to 36.11 16,1% compared an increase from 4.1% to 1 11.3% on implicit conflicts. This suggests that models' heuristic predictions of which story is plausible can indeed help improve performance on the analytic predictions of conflicting sentences and physical states, the latter of which has large search space.
C PLM Fine-Tuning Details
In Section 4, we proposed two fine-tuning ap- proaches powered by CGLI (Ma et al., 2022), which predicts reasoning steps through task- specific layers. These layers are linear projec- tions for the tiered tasks of selecting stories, sen- tences, and physical states. We preserve the entity- aware and timestep-aware encodings in CGLI. Specifically, given an entity E, we concatenate it with a story pair S to create prompt C [CLS) E [SEP) S [SEP]. Following CGLI, C' is then mapped with the embedding layer of the lan- guage model and summed with the timestep embed- ding. Finally, it is encoded by the language model to create a latent representation for task-specific classifications. We jointly optimize three cross- entropy losses for the story selection step, sentence selection step, and physical state prediction step (Lotory + Lsenttnce + L.state)-
In TRIP, in order to make proposed Focused CGLI (FCGLI) models focused on predicting ex- plicit conflicts (defined in Appendix A.4) in stories. we do not optimize the physical state prediction loss for implicit conflicts. As such, the physical state information for each task instance consists of
"ooor example, in one sentence, we may see that Tom put the soup in the microwave, which indirectly implics the soup should be heated up, then in later sentence, Tom ate the cold soup.

---

an entity, an attribute, effect state, and precondi- tion state. The physical state loss is calculated by averaging four losses.
states (e.g. dirty, clean, unpowered, powered) and --shot example for each one." 17 This process is called familiarization, and essential to fully spec- ify the expected outputs for the reasoning task and enable systematic evaluation, as the model will be more likely to predict physical states within this demonstrated space of labels. As Tiered-ProPara's low-level state space consists of entities mentioned in the given text, this is only necessary for TRIP.
We selected the model for the test set by using the model with the highest validation verifiability for each task. We used consistent set of hyper- parameters across tasks: learning rate of 5e-6, maximum of 10 training epochs, and batch size of (the maximum that could fit in GPU memory). The weight decay set to be 0.01 for all parameters except for bias and aayerNoom..eiiht, and we use a warmup scheduler following CGLL. We report the performance by averaging three random runs. All experiments were performed on single NVIDIA GeForce RTX 3090 Ti graphics card (24GB).
Filtering TRIP to shorten prompts. Since LLaMA is limited to context length of 2048 to- kens, we create filtered version of TRIP which only includes instances where annotated states in- volve only the top-6 most frequent physical at- tributes. The familiarization process mentioned above contains ~ 70% tokens in the full prompt (~ 3800 tokens). Therefore, we first perform a sta- tistical analysis on the dataset and select 6 highest- frequency physical states for effect and precondi- tion, namely (no longer existent, existent), (bro- ken, functional), (in pieces, whole), (turned off. turned on), (inedible, edible), (unpowered, pow- ered). After that, the prompt length is reduced to 1600 tokens. As familiarization of physical states used to familiarize the model with the clas- sification space, we filtered the dataset to stories that only contain explicit conflicts between these high-frequency physical states.
PLM Prompt Details
Here, we include some extra details about prompt- ing PLMs.
D.1 Automatic Exemplar Generation
We automatized exemplar generation for smoother and consistent experiments on TRIP and Tiered- ProPara.
D.1.1 TRIP
The lowest level task in TRIP dataset is annotated with symbolic states to represent precondition and effect states over 20 physical states. Therefore, we convert symbolic physical states to natural lan- guage. For stories with explicit conflict, we it- erate through all combinations of entities and 20 attributes for the 2 conflicting sentences in order to find the conflicting physical states. For some con- flicts that have different entity name for the same entity, we used the following algorithm to first find all possible conflicting entity pairs, and then iter- ate them for the conflicting entity pair which re- sults in maximum cosine similarity (argmax) be- tween GLoVe Ve embedding vectors (50-dimensional) (Pennington et al., 2014) and more specifically, the argmaz function is defined by:
D.1.2 Tiered-ProPara
The automatic exemplar generation process for Ticred- Ticred-Propara is similar to the TRIP, but more straightforward. There's no familiarization stage for Tiered- Prooprr. After two-story prompt, we asked the model with a question: What happened to [converted entiry]? The answer prompt in the demonstration is composed by filling ground truth labels (story, sentence, and the entity converted to) to the template.
D.2 Full Prompt Examples
From our in-context learning experiments, we in- clude full example prompts used with InstructGPT for the in-context learning with heuristic-analyeee reasoning (ICL-HAR) and unstructured in-context learning (ICL-U) strategies. Figures 10 and 11 show examples for ICL-HAR in TRIP and ProPara,
Ve GloVe (loVee(etityy) "(ooVe(entstyy)) G(oVeeenttyy)))
Ve VGooVeeentity Ve
arg max cEcundidates
Ve
Physical state familiarization. To prime PLMs for the highly-dimensions physical state classifi- cation step of TRIP, all in-context learning experi- ments are prepended with a list of possible physical
For example, After Mary sliced the apple, what is the state of the apple? The apple is now in pieces. and Before Tom opened the door, what was the state of the door? The door was closed may he used to familiarize the model with the physical state classification space.

---

respectively. Figures 12, 13, and 14 show exam- ples for the ICL-U in TRIP, while Figures 15, 16, and 17 show examples for the ICL-U baseline in ProPara. For the ICL-CoT baseline, we simply ap- pend "Lett' think step by step..." and zero-shot CoT generated by InstructGPT into the ICL-U prompting demonstrations before final predictions are made for each sub-task. We provide the full zero-shot CoT prompts we used for each sub-task in TRIP and
E.2 Attentional Precision and Recall Details
Attentional precision and recall are calculated by converting normalized attention weights into binary measures of whether attention is correct. To do this, we check whether the average attention weight for the relevant segment of language context (i.e., the appropriate story sentence(s)) exceeds thresh- old. We calculate the average precision and recall over a set of candidate thresholds centered around 0.1 (0.08 0.12) with an interval 0.005 because on average, there are about 10 sentences on in a pair stories, and all sentences' attention are normalized to a sum of 1.
below:
TRIP, Story Selection: Let's think step by step about which story is more plausible.
TRIP, Sentence Selection: Let's think step by step about which sentences are conflicting in one story.
Given this binary measure, we can then clas- sify each PLM output into four combinations over whether its attention is faithful, and whether its intermediate predictions are coherent (i.e., consis- tent or verifiable); true positive, false positive, true negative, and false negative. We provide some ex- amples here assuming a static threshold 0.09. the story-level prediction example from Figure 5, we calculate the average sentence-wis attention in the story containing conversion (i.e., story B), and compare it to the threshold. These val- ues 0.590/6 0.098 and 0.837/6 0.140 for ICL-U and ICL.HAR, respectively. Both val- ues of ICL-U and ICL-HAR exceeded the thresh- old, but ICL-U didn't 't correctly identify the sen- tence that contains a conversion, while ICL-HAR did. Therefore, we classify the example from ICL-U as false negative and ICL-HAR as true positive. Similarly, in the physical state detec- tion example from Figure age sentence-wise attention on the two conflicting sentences, which are sentences and in story A. These values are (0.09 + 0.086) = 0.088 and (0.213 + 0.154)/2 = 0.184 for ICL-U and ICL-HAR, resp. Because ICL-HAR exceeded the threshold but ICL-U didn't, and ICL-HAR actu- ally generated the correct response while ICL-U didn't, 't, classify the example from ICL-U as true negative and ICL-HAR true positive.
TRIP, Physical State Prediction: Let's think step by step about which physical states are conflicting in two sentences in one story.
Tiered-Propara, Story Selection: Let's think step by step about which story [entity] were converted in.
Tiered-Proper Sentence Selection: Let's think step by step about which sentence [en- tity] were converted in one story.
Tiered-Propara, Physical State Prediction: Let's think step by step about what [entity] were converted to in one sentence in one story.
E Attention Analysis Details
calculate the aver-
E.1
Self-Attention Weight Extraction
To enable our attention analysis for soft HAR in in-context learning, we used the output_att _attettions flag in the Transformers (Wolf et al., 2020) library to extract the raw attentions computed during infer- ence with LLaMA. An attention mask is applied to the attentions remove the attentions associated with the demonstration prompt and special char- acters, only keeping the attentions associated with the tokens from the test prompt. We then summed up the attentions across each sentence in the test prompt, averaged across a subset of the generated tokens, and normalized by dividing the sum of the attentions. Here, computed the token subset's average attention and we used it to measure the importance of a sen- tence to the model's is reasoning outcomes. We then used these normalized weights to calculate the eval- uation criteria proposed in Section 5.2.
sentence of the story prompt,

---

TRIP ICL-HAR
Familiarization Prompts- Physical States (6 of 80):
Physical state options: powered, edible, whole
Before Tom turned on the microwave, what was the state of the microwave? The microwave was powered.
Before Tom ale the cereal, what was the state of the cereal? The cereal was edible.
Before Tom cut the banana into slices, what was the state of the banana? The banana was whole.
Physical state options: unpowered. inedible, in pieces...
After Tom unplugged the mierowave, what is the state of the microwave? The microwave is now unpowered.
After Tom threw the cereal in the dustbin, what is the state of the cereal? The cereal is now inedible.
After Tom mashed the hanana, what is the state of the banana? The banana is now in pieces.
Demonstration Prompts Physical States (1 of 4):
Story A:
Mary the fridge.
2. Mary took bowl from the fridge.
3. The bowl had cucumber and donut in it.
Mary put the cucumber plate and tossed the donut in trash. Mary ate the donut.
Story B:
Mary went to fridge.
Mary look out bowl from fridge.
The bowl a cucumber a donut in it.
Mary put the cucumber on the counter.
Mary ate the donut.
Story is more plausible.
In Story A, sentences and 5 conflict with each other. For sentence 4:
After Mary put the cucumber on plate and tossed the donut the trash, what is the state the donut?
The donut is now inedible.
For sentence
Before Mary ate the donut, what was the state of the donut? The donut was edible.
Test Prompt:
Story A:
1. Ann opened the toolbox and realized that somcone took away all of her tools.
2. Ann took out an axe, pair of scissors, rope and few other things.
3. Ann put these items in box and placed them in the trunk of her car.
4. Ann fastened her bicycle to the bicycle rack on her car.
5. Ann went inside the house get the rest of her gear to put in the car.
Story B:
1. Ann opened the toolbox get few things for her camping trip.
2. Ann took out
pair of seissors, rope and few other things.
3. Ann put these items in box and placed them in the trunk of her car.
4. Ann fastened her bicycle the bicycle rack her
5. Ann inside the house get the rest of her gear put in the car.
LLM Sample Output:
Story B plausible.
In Story A, sentences 1 and 2 conflict with each other. For sentence
After Ann opened the toolbox and realized that someone took away all her tools, what the of the tools? The longer existent.
tools
For sentence
Before took an axe, a pair of scissors, rope and few other things, what were state of the things? The things existent.
Figure 10: Sample prompt used for in-context learning with heuristic-anale analytic reasoning (ICL-HAR) TRIP. Some redundant sections of the prompt are omitted (indicated with "..." and in section headings).

---

Tiered-ProPara ICL-HAR
Demonstration Prompts (1 of 4):
Story A:
1. Wind and water erode rocks.
2. The wind and water pick up sediment that came off the rocks.
3. Sediment carried
by the wind. water, air, ice.
4. Sediment deposited the same spot many times.
5. The sediment piles top of other sediment for long time.
6. The sediment the bottom compressed by the weight of sediment.
The sediment becomes sedimentary rock is pushed together by the heavy weight.
Story B:
Waves hit coast.
2. Waves have energy that pushes the and sediment.
The sediment can help erode land like sandpaper.
Water washes the sediment back.
Some sediment is as sand.
What happened sediment? Sediment is converted story A.
In story A, sediment is converted sentence
After the sediment becomes sedimentary rock as it pushed together by the heavy weight, sediment is converted to sedimentary rock.
Test Prompt:
Story A:
1. Coal is heated in the boiler.
2. The water tank over the boiler is heated.
3. Creates steam.
4. The steam funneled to the piston.
5. Piston uses the steam as energy.
6. The piston causes the crankshaft to move. Story B:
1. Plants obtain water through the soil they are growing in.
2. The plants roots absorb the water from the soil.
3. Transport the water to the parts of the plant where the water is needed.
4. The plant uses the water as part of the photosynthesis process.
5. The plant creates food, called glucose, for itself.
6. The plant uses water to circulate the glucose around the plant to deliver the glucose to where it's needed. What happened water?
LLM Sample Output:
Water converted in story
In story A, water is converted in sentence
After creates steam, water is converted steam.
Figure 11: Sample prompt used for in-context learning with heuristic-analytic reasoning (ICL-HAR) on Tiered- ProPara. Some redundant sections of the prompt are omitted (indicated with "..." and in section headings).

---

TRIP ICL-U Baseline Plausible Story Selection
Demonstration Prompts (1 of 4):
Story A:
1. Mary went to the fridge.
2. Mary tnok out bowl from fridge.
3. The bowl had a cucumber and a donut in it.
4. Mary put the cucumber on a plate and tossed the donut in the trash.
5. Mary ate the donut.
Story B:
1. Mary went to the fridge.
2. Mary took out bowl from the fridge.
3. The bowl had a cucumber and donut in it.
4. Mary put the cucumber on the counter.
5. Mary ate the donut.
Story B is more plausible.
Test Prompt:
Story A:
1. Ann opened the toolbox and realized that someone took away all of her tools.
2. Ann took out an axe, pair of scissors, rope and few other things.
3. Ann put these items in box and placed them in the trunk of her car.
4. Ann fastened her bicycle the bicycle rack her
5. Ann inside the house get the rest of her gear put in the car. Story B:
Ann opened the toolbox get few things for her camping trip.
2. Ann took an axe, pair of scissors, rope and few other things.
3. Ann put these items in box and placed them the trunk of her Ann fastened her bicycle the bicycle rack her
5. Ann inside the house get the rest of her gear put the
LLM Sample Output:
Story B more plausible.
Figure 12: Sample prompt used for plausible story selection step of unstructured in-context learning (ICL-U)
baseline on TRIP. Some redundant sections of the prompt are omitted (indicated with "..." and section headings).

---

TRIP ICL-U Baseline Conflicting Sentence Selection
Demonstration Prompts (1 of 4):
Story A:
1. Mary went to the fridge.
2. Mary tnok out
from fridge. 3. The bowl had a cucumber and a donut in it.
4. Mary put the cucumber on a plate and tossed the donut in the trash
5. Mary ate the donut.
Story B:
1. Mary went to the fridge.
2. Mary took out bowl from the fridge.
3. The bowl had a cucumber and donut in it.
4. Mary put the cucumber on the counter.
5. Mary atc the donut.
Sentences 4 and 5 conflict with each other in story A.
Test Prompt:
Story A:
1. Ann opened the toolbox and realized that someone took away all of her tools.
2. Ann took out an axe, pair of scissors, rope and few other things.
3. Ann put these items in box and placed them in the trunk of her car.
4. Ann fastened her bicycle the bicycle rack her
5. Ann inside the house get the rest of her gear put in the car.
Story B:
Ann opened the toolbox get few things for her camping trip.
2. Ann took an axe, pair of scissors, rope and few other things.
3. Ann put these items in box and placed them the trunk of her Ann fastened her bicycle the bicycle rack her
5. Ann inside the house get the rest of her gear put the
LLM Sample Output:
Sentences I and conflict with each other story
Figure 13:
Sample prompt used for conflicting sentence selection step of unstructured in-context learning (ICL-U)
baseline on TRIP. Some redundant sections of the prompt are omitted (indicated with ".."" and section headings).

---

TRIP-ICL-U Baseline Physical State Prediction
Familiarization Prompts- Physical States (6 of 80):
Physical state options: powered, edible, whole
Tom turned on the microwave. Before, what was the state of the microwave? The microwave was powered. Tom ate the cereal. Before, what was the state of the cereal? The cereal was edible.
Tom cut the banana into slices. Before, what was the state of the banana? The banana was whole.
Physical state options: unpowered, inedible, in pieces
Tom unplugged the microwave. After, what is the state of the microwave? The microwave now unpowered. Tom threw the cereal in the dustbin. After, what is the state of the cereal?? The cercal is now inedible.
Tom mashed the banana. After, what is the state of the banana? The banana is now in pieces.
Demonstration Prompts (1 of 4):
Story A:
1. Mary went to the fridge.
2. Mary took out bowl from the fridge.
3. The bowl had cucumber and donut in it.
4. Mary put the cucumber on plate and tossed the donut in the trash.
5. Mary ate the donut.
Story B:
1. Mary went to the fridge.
2. Mary took
bowl from the fridge. 3. The bowl had cucumber and donut in it.
4. Mary put the cucumber the counter.
5. Mary ate the donut.
After, what the state of the donut? The donut is now inedible. Before, what the state of the donut? The donut edible.
Test Prompt:
Story A:
Ann opened the toolbox and realized someone took away all of tools.
Ann took out an axe, a pair of scissors, rope and other things. Ann put these items box placed them trunk her car.
Ann fastened
bicycle the bicycle rack Ann went inside the house to get the rest of gear to put car.
car.
Story B:
Ann opened the toolbox get a few things for her camping trip.
2. Ann took out an axe, a pair of scissors, rope and few other things.
3. Ann put these items in box and placed them in the trunk of her car. Ann fastened her bicycle the bicycle rack on her car.
5. Ann went inside the house to get the rest of her gear to put in the car.
LLM Sample Output:
After, what are the state of the tools? The tools are now no longer existent.
Before, what were the state of the things? The things were existent.
Figure 14: Sample prompt used for physical state prediction step of unstructured in-context learning (ICL-U)
baseline on TRIP. Some redundant sections of the prompt are omitted (indicated with ".." and section headings).

---

ICL-U Baseline Conversion Story Selection
Demonstration Prompts (1 of 4):
Story A:
1. Wind and water erode rocks.
2. The wind and water pick up sediment that came off the rocks.
3. Sediment carried Sediment deposited the spot many times.
by the wind. water, air, ice.
5. The sediment piles on top of other sediment for long time.
6. The sediment the hottom compressed by the weight of sediment.
The sediment becomes sedimentary rock as is pushed together by heavy weight.
Story
Waves coast.
Waves have energy that pushes water sediment.
The sediment can help erode land like sandpaper.
Water washes the sediment back.
Some sediment is as sand.
What happened sediment? Sediment is converted story A.
Test Prompt:
Story A:
1. Coal is heated in the boiler.
2. The water tank over the boiler is heated.
3. Creates steam.
4. The steam funneled to the piston.
5. Piston uses the steam as energy.
6. The piston causes the crankshaft to move.
Story B:
1. Plants obtain water through the soil they are growing in.
2. The plants roots absorb the water from the soil.
3. Transport the water to the parts of the plant where the water is needed.
4. The plant uses the water as part of the photosynthesis process.
5. The plant creates food, called glucose. for itself.
6. The plant uses water to circulate the glucose around the plant to deliver the glucose to where it's needed.
What happened water?
LLM Sample Output:
Water converied story
Figure 15: Sample prompt used for conversion story selection step of unstructured in-context learning (ICL.-U) baseline on Tiered-ProPara. Some redundant sections the prompt are omitted (indicated with "..." and section headings).

---

ICL-U Baseline Conversion Sentence Selection
Demonstration Prompts (1 of 4):
Story A:
1. Wind and water erode rocks.
2. The wind and water pick up sediment that came off the rocks.
3. Sediment is carried to area by the wind, water, air, ice.
4. Sediment is deposited
the same spot many times.
5. The sediment piles top of other sediment for long time.
6. The sediment near the bottom is compressed by the weight of newer sediment.
7. The sediment becomes sedimentary rock as is pushed together by the heavy weight.
Story B:
1. Waves hit the coast.
2. Waves have energy that pushes the water and sediment.
3. The sediment can help erode land like sandpaper.
4. Water washes the sediment back.
5. Some sediment is left as sand.
What happened sediment? Sediment converted sentence 7 story A.
Test Prompt:
Story A:
Coal heated the boiler.
The water tank over the boiler is heated.
Creates steam.
The steam funneled
piston.
Piston uses steam as energy.
6. The piston causes the crankshaft to move. Story B:
Plants obtain water through the soil they are growing in.
2. The plants roots absorb the water from the soil.
3. Transport water to the parts the plant where water needed.
The plant uses the water as part of the photosynthesis process.
5. The plant creates food, called glucose, for itself.
6. The plant uses water to circulate the glucose around the plant to deliver the glucose to where it's needed.
What happened to water?
LLM Sample Output: Water is converted in sentence 3 in story A.
Figure 16: Sample prompt used for conversion sentence selection step of unstructured in-context learning (ICL-U) baseline on Piered-ProPara. Some redundant sections of the prompt are omitted (indicated with "..." and in section headings).

---

ICL-U Baseline Physical State Prediction
Demonstration Prompts (1 of 4):
Story A:
1. Wind and water erode rocks.
2. The wind and water pick up sediment that came off the rocks.
3. Sediment carried Sediment deposited the spot many times.
by the wind. water, air, ice.
5. The sediment piles on top of other sediment for long time.
6. The sediment the hottom compressed by the weight of sediment.
The sediment becomes sedimentary rock as is pushed together by heavy weight.
Story
Waves coast.
Waves have energy that pushes water sediment.
The sediment can help erode land like sandpaper.
Water washes the sediment back.
Some sediment is as sand.
What happened sediment?
Sediment is converted to sedimentary rock.
Test Prompt:
Story A:
1. Coal is heated in the boiler.
2. The water tank over the boiler is heated.
3. Creates steam.
4. The steam funneled to the piston.
5. Piston uses the steam as energy.
6. The piston causes the crankshaft to move.
Story B:
1. Plants obtain water through the soil they are growing in.
2. The plants roots absorb the water from the soil.
3. Transport the water to the parts of the plant where the water is needed.
4. The plant uses the water as part of the photosynthesis process.
5. The plant creates food, called glucose. for itself.
6. The plant uses water to circulate the glucose around the plant to deliver the glucose to where it's needed.
What happened water?
LLM Sample Output:
Water converied sleam.
Figure 17: Sample prompt used for conversion entity prediction step of unstructured in-context learning (ICL--U) baseline on Tiered-ProPara. Some redundant sections the prompt are omitted (indicated with "..." and section headings).


Reviews:
[{"review_id": "mNmuIEe4E5", "reviewer": "Reviewer_KHzS", "summary": "", "questions": "1. #305 - #307, authors include previously published RoBERTa results from Storks et al. (2021), CGLI (Ma et al., 2022), and BT (Richardson et al., 2022, are these results reported under the same settings?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful and constructive feedback, and for recognizing the novelty of our work in using insight from cognitive psychology to improve coherence and trustworthiness of reasoning in LLMs, and in proposing a reframed ProPara dataset. We respond to concerns about the work below.\n\n**W1: Regarding Cascading Errors**\n\nYes, as discussed in our Limitations section, the cascading errors in PLMs’ reasoning is a possible limitation. It is indeed possible that various non-greedy generation or reasoning strategies may be applied on top of HAR to mitigate this limitation, such as beam search or [tree-of-thought](https://arxiv.org/abs/2305.10601) to generate multiple candidate rationalizations and choose one based on LLM likelihoods. Such methods to reduce cascading error are orthogonal to our efforts, as they could theoretically be applied to any of the comparison approaches used in this work (not just HAR). As our goal was to demonstrate that heuristic decisions can be used to improve performance on analytic rationalizations in PLMs, we chose not to explore these approaches here as they would increase the moving parts in our study, and possibly distract from this key message.\n\nNonetheless, we show that even with possible cascading error, the model still outperforms all baselines by a great margin. The gradient-based fine-tuning model achieved state-of-the-art results on TRIP despite this effect. We’ll discuss this issue further in our revised version.\n\n**W2: Gains from Breaking Down the Complex Task**\n\nThis is an important point which was also at the front of our minds when designing experiments! This possibility motivated our unstructured baselines which still broke the complex tasks down into smaller parts (e.g., by prompting LLMs with smaller separate prompts for each sub-task), but did not condition rationalization steps with higher-level heuristic decisions as we did in HAR methods. By using this as a comparison, we demonstrated that a large amount of performance gains indeed come from connecting heuristic and analytic reasoning steps as inspired by dual process theories in cognitive psychology. Furthermore, we empirically verify that this happens due to refocusing of model attention on the most relevant information during reasoning, which resembles what is theorized for human reasoning.\n\nTesting on broader tasks would have been ideal, but there are not many available resources for this type of evaluation of coherent reasoning. While this approach can theoretically be applied to any language problems with higher-level decisions supported by more complex rationalizations, there are few datasets for this (partly why we created Tiered-ProPara in this work). Despite this, it is worth noting that these efforts in physical commonsense can have a broad impact, as coherent reasoning will become especially important with the development of embodied AI, as a human user will need to trust that an embodied agent deeply understands the physical actions they communicate with the agent about, and how they will impact the world around them.\n\nWe also want to add here that traditional chain-of-thought (CoT) has this motivation of breaking down complex tasks to improve performance, but method (HAR) differs from traditional Chain-of-Thought (CoT). Traditional CoT breaks down a hard problem into smaller subproblems to improve complex reasoning, such as for multi-step math problems. In physical commonsense, low-level atomic physical states are the most difficult for AI models to predict, and they are difficult to further decompose. \n\nNonetheless, we attempted to integrate CoT into our framework by augmenting the ICL-U prompts for each of the 3 sub-tasks (i.e., story selection, sentence selection, and state prediction) with free-text explanations. These explanations are generated by the LLM by prompting it with “Think step by step” after the sub-task context for each in-context demonstration and the testing example being solved, then they’re appended to the prompt. This gave us a few-shot CoT method that applies to our task. The results below show that applying traditional CoT (ICL-CoT) doesn’t improve performance much. Further, we observed that the generated reasoning explanations tend to iterate through sentences in stories to do step by step reasoning upon them, which doesn’t add useful information to the problem or simplify it- the model still needs to make predictions on lower-level tasks from a large context with a lot of irrelevant information. Therefore, we proposed HAR, a method that can use high-level (heuristic) predictions to focus on the correct context for low-level (analytic) predictions.\n\n*TRIP (InstructGPT)*\n| Method | Accuracy | Consistency | Verifiability |\n|--|--|--|--|\n| ICL-U | 70.9 | 40.7 | 7.1 |\n| ICL-CoT | 70.7 | 39.6 | 8.3 |\n| ICL-HAR | **72.6** | **47.9** | **23.9** |\n\n*TRIP (LLaMA)*\n| Method | Accuracy | Consistency | Verifiability |\n|--|--|--|--|\n| ICL-U | **70.4** | 42.3 | 14.8 |\n| ICL-CoT | 62.0 | 43.7 | 14.1 |\n| ICL-HAR | 55.6 | **44.4** | **35.2** |\n\n*Tiered-Propara (InstructGPT)*\n| Method | Accuracy | Consistency | Verifiability |\n|--|--|--|--|\n| ICL-U | **54.9** | 17.4 | 5.2 |\n| ICL-CoT | 52.1 | 23.9 | 12.2 |\n| ICL-HAR | **54.9** | **31.5** | **20.7** |\n\n*Tiered-Propara (LLaMA)*\n| Method | Accuracy | Consistency | Verifiability |\n|--|--|--|--|\n| ICL-U | **51.2** | 3.8 | 1.4 |\n| ICL-CoT | 48.8 | 4.7 | 1.9 |\n| ICL-HAR | 41.8 | **17.8** | **13.1** |\n\nWe will add this result into the revised version.\n\n**W3: Accuracy Drop in ICL-HAR with LLaMA**\n\nGood observation! Although ICL-U is more accurate than ICL-HAR in some cases, we would highlight that the focus of these tasks and our work is to improve the *coherence* of reasoning, which accuracy does not capture (only measures performance on the end task of story selection). Instead, our goal is to optimize **consistency** and **verifiability**, each of which already require end-task predictions to be accurate as a prerequisite, and measure the correctness of the sentences and physical states identified by the model to rationalize them. In fact, it is not ideal for accuracy to far exceed these metrics, as this indicates the model is making a correct prediction on the end task without valid underlying support (incoherent reasoning). Therefore, a drop in accuracy is not problematic in these tasks. We will better clarify the significance of each evaluation metric in our revision.\n\nThat said, this drop only happens with LLaMA-65B, and we think this is because it’s much smaller than InstructGPT (175B), so it is more sensitive to long prompts and generations. A quick fix could be to sequentially prompt the LLM multiple times with shorter prompts for each sub-task. We presented results for an approach like this in Appendix A.2, and they don’t exhibit such a drop in accuracy.\n\nRegarding the sample size possibly not being big enough, we do agree that TRIP and Tiered Propara datasets are small datasets (densely annotated data for physical commonsense is scarce and expensive to collect). However, both datasets have been used extensively in empirical research in prior work, and the test sets are sufficiently large for statistically significant results (TRIP has 351 examples and Tiered-ProPara has 214 examples after our augmentation). While the differences between most results are already quite large, we additionally ran a McNemar test and found statistically significant (p < 0.05) differences in consistency and verifiability for our ICL-HAR and ICL-U results on TRIP. As *p*-values were many orders of magnitude less than 0.05, we expect other results to show similar significance, and will complete the remaining analysis for our revised paper.\n\n**Q1: Settings for Past Work Comparisons**\n\nWe made substantial efforts to ensure that all results were comparable to those from past work, including verifying the same testing examples were used and using a slight modification of CGLI as a baseline (FCGLI) and largely mirroring the training processes of these past works (Storks et al., 2021; Ma et al., 2022), including using the same RoBERTa backbone and total number of training epochs. For BT (Richardson et al., 2022), we reported the results from their paper, but there are some differences: they use T5 as the PLM, and some hyperparameters may be in slightly different ranges (e.g., number of training epochs). Several published papers from this line of work on physical commonsense also report these numbers from prior work.\n\n**Presentation Improvements: Abbreviations**\n\nThank you for letting us know that the abbreviations caused some difficulty in understanding. We will remove unnecessary abbreviations in the revised version of the paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Novelty: Drawing the idea from cognitive psychology, this paper explores strategies to improve the coherence and trustworthiness of reasoning. The key ideas are using human-inspired heuristics and then analytic reasoning steps to improve the coherence of PLMs, testing this on physical commonsense tasks, and showing it strengthens rationalization by attention.\n2. The authors innovatively propose Tiered-ProPara tasks for coherent physical commonsense reasoning, which leaves room for further exploration.\n3. The paper is generally well-written, and most parts of the paper are easy to follow.", "reasons_to_reject": "1. The proposed heuristic-analytic reasoning strategies rely on making sequential predictions, which could cause cascading errors if the model makes a mistake on an early heuristic prediction.\n2. Some of the gains may come from simply breaking down a complex task rather than the proposed psychology-inspired heuristic-analytic duality. Testing on broader tasks could help isolate the benefits.\n3. Although the authors claim that \"verifiability from 7.1% up to 23.9%, over a 200% improvement\" for InstructGPT and \"5.2% to 20.7%, nearly a 300% improvement\" for LLaMA, it seems that it will hamper the accuracy scores, and need further discussion. Also, the sample size seems to be not big enough, and more extensive evaluations could help validate the results and trends.", "questions_to_authors": "1. #305 - #307, authors include previously published RoBERTa results from Storks et al. (2021), CGLI (Ma et al., 2022), and BT (Richardson et al., 2022, are these results reported under the same settings?", "paper_topic_and_main_contributions": "The paper explores strategies to improve the coherence and trustworthiness of reasoning by large pre-trained language models (PLMs) on physical commonsense reasoning tasks. It takes inspiration from dual-process theories in cognitive psychology, which posit humans use fast, intuitive heuristic thinking and slower, logical analytic thinking. Experiments on two physical commonsense benchmarks, TRIP, and Tiered-ProPara, show HAR improves coherence metrics like consistency and verifiability.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "1. From my perspective, I think that there're too many abbreviations in the papers that might cause some difficulty in understanding."}, "extra_text": {}}, {"review_id": "uyLRxrDGK8", "reviewer": "Reviewer_nRZC", "summary": "", "questions": "A. I am unclear about line 390 to line 393 even after looking at the appendix. How are the 3 extracted predictions on each testing exampled combined? How is the metrics calculated?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the thoughtful comments and suggestions! We appreciate your recognition of our models’ performance and the effectiveness of our analysis, and respond to your review below.\n\n**W1: Generalizability**\n\nThis is a good point! While the datasets we studied highlight the benefit of heuristic-analytic reasoning due to critical information being localized in small parts of text, this is not a requirement for HAR to be applied, especially for the in-context learning implementation which is soft and can still take the entire context into account as more heuristic information is introduced. Although not the focus of our work, it would be interesting for future work to study attention patterns in these sorts of tasks where critical information can be more widespread.\n\nFurthermore, the type of heuristic-analytic reasoning we study in this work can actually be quite abundant in our real world. For example, when we are trying to identify a bug in our code, we often utilize a heuristic intuition on where the bug may be located based on sparse information. Then, we will use analytic reasoning to find out the bug (e.g., trace code with debugger, add print statements, etc.) and fix it.\n\nWe will discuss this issue of generalizability in more detail in our revision.\n\n**W2: Relevance to Cognitive Processes**\n\nThis work specifically focuses on how chaining together these dual processes may support coherent reasoning in LLMs. HAR is motivated by the dual process theory which utilizes intuition to remove irrelevant information and process low-level rationalizations. In the paper “Heuristic and analytic processes in reasoning” (Evans, 1984), Evans highlighted “a distinction between heuristic processes which select items of task information as ‘relevant’, and analytic processes which operate on the selected items to generate inferences or judgements.” Works cited in Section 1 review a breadth of cognitive psychology research that supports this theory.\n\nSimilar to Evans’ description, HAR first makes an intuitive prediction on a high-level task which acts as a heuristic for rationalizing a low-level task. Taking the TRIP dataset as an example, HAR judges whether a story is plausible or not in the first step, identifies the conflicting sentences in the second step, and then rationalizes the physical states that cause the conflict in the third step. Our attention analysis experiments in Section 5.2 verify that this occurs due to the model’s stronger focus on relevant information during reasoning, which resembles the above synergy of heuristic and analytic processes in human cognitive psychology. \n\nFurthermore, it may be worth noting that our method (HAR) differs from traditional Chain-of-Thought (CoT) used for step-by-step reasoning. Traditional CoT breaks down a hard problem into smaller subproblems to improve complex reasoning, such as for multi-step math problems. In physical commonsense, low-level atomic physical states are the most difficult for AI models to predict, and they are difficult to further decompose.\n\nNonetheless, we did attempt to integrate CoT into our framework by augmenting the ICL-U prompts for each of the 3 sub-tasks (i.e., story selection, sentence selection, and state prediction) with free-text explanations. These explanations are generated by the LLM by prompting it with “Think step by step” after the sub-task context for each in-context demonstration and the testing example being solved, then they’re appended to the prompt. This gave us a few-shot CoT method that applies to our task. The results below show that applying traditional CoT (ICL-CoT) doesn’t improve performance much. Further, we observed that the generated reasoning explanations tend to iterate through sentences in stories to do step by step reasoning upon them, which doesn’t add useful information to the problem or simplify it- the model still needs to make predictions on lower-level tasks from a large context with a lot of irrelevant information. Therefore, we proposed HAR, a method that can use high-level (heuristic) predictions to focus on the correct context for low-level (analytic) predictions.\n\n*TRIP (InstructGPT)*\n| Method | Accuracy | Consistency | Verifiability |\n|--|--|--|--|\n| ICL-U | 70.9 | 40.7 | 7.1 |\n| ICL-CoT | 70.7 | 39.6 | 8.3 |\n| ICL-HAR | **72.6** | **47.9** | **23.9** |\n\n*TRIP (LLaMA)*\n| Method | Accuracy | Consistency | Verifiability |\n|--|--|--|--|\n| ICL-U | **70.4** | 42.3 | 14.8 |\n| ICL-CoT | 62.0 | 43.7 | 14.1 |\n| ICL-HAR | 55.6 | **44.4** | **35.2** |\n\n*Tiered-Propara (InstructGPT)*\n| Method | Accuracy | Consistency | Verifiability |\n|--|--|--|--|\n| ICL-U | **54.9** | 17.4 | 5.2 |\n| ICL-CoT | 52.1 | 23.9 | 12.2 |\n| ICL-HAR | **54.9** | **31.5** | **20.7** |\n\n*Tiered-Propara (LLaMA)*\n| Method | Accuracy | Consistency | Verifiability |\n|--|--|--|--|\n| ICL-U | **51.2** | 3.8 | 1.4 |\n| ICL-CoT | 48.8 | 4.7 | 1.9 |\n| ICL-HAR | 41.8 | **17.8** | **13.1** |\n\nWe will add this result into the revised version.\n\n**W3: Accuracy Drop in ICL-HAR with LLaMA**\n\nGood observation! Although ICL-U is more accurate than ICL-HAR in some cases, we would highlight that the focus of these tasks and our work is to improve the *coherence* of reasoning, which accuracy does not capture (only measures performance on the end task of story selection). Instead, our goal is to optimize **consistency** and **verifiability**, each of which already require end-task predictions to be accurate as a prerequisite, and measure the correctness of the sentences and physical states identified by the model to rationalize them. In fact, it is not ideal for accuracy to far exceed these metrics, as this indicates the model is making a correct prediction on the end task without valid underlying support (incoherent reasoning). Therefore, a drop in accuracy is not problematic in these tasks. We will better clarify the significance of each evaluation metric in our revision.\n\nThat said, this drop only happens with LLaMA-65B, and we think this is because it’s much smaller than InstructGPT (175B), so it is more sensitive to long prompts and generations. A quick fix could be to sequentially prompt the LLM multiple times with shorter prompts for each sub-task. We presented results for an approach like this in Appendix A.2, and they don’t exhibit such a drop in accuracy.\n\n**QA: Metric Calculation**\n\nSorry for glossing over the details here. As shown in Figures 12-14 and 15-17, we prompt LLMs separately for 3 tiered tasks of story, sentence, and low-level physical state classification. Taking TRIP as an example (Figures 12-14), we use regular expressions to extract the LLM’s predicted plausible story, conflicting sentences, and underlying physical state labels automatically from generated text. These 3 predictions make up a complete reasoning chain on the TRIP problem, as introduced in [Storks et al. (2021)](https://aclanthology.org/2021.findings-emnlp.422.pdf). Once we have extracted such reasoning chains from each example in the evaluation dataset, we calculate the metrics. Accuracy is taken as the proportion of chains such that the chosen plausible story is correct. Consistency is taken as the proportion of chains such that the predicted pair of conflicting sentences is additionally correct. Lastly, verifiability is taken as the proportion of chains such that the underlying physical states within those correct conflicting sentences are also correct (at least one non-default state is predicted in each sentence, and all such predicted states are correct). As such, it is always the case that accuracy >= consistency >= verifiability, and each of these metrics evaluates lower-level pieces of evidence in the reasoning chain to target the coherence of reasoning.\n\nWe will clarify this in our revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The principled framework significantly improves the reasoning consistency on physical commonsense reasoning benchmarks.\n\n2. The attention weights analysis effectively explains why the framework helps: the guided procedure successfully shifts model attention to the key aspects of the long discourse. ", "reasons_to_reject": "1. Generalizability: both fine-tuning and in-context learning strategies seem to be tailored for shifting model attention to a smaller chunk of key information, which is confirmed by the attention weight analysis. This makes me to worry to what extent this method could be generalized to other datasets where the information is not presented in contrastive pairs, and where the conflicting information is not restricted to one or two sentences but widely spread in the entire passage. For example, the task of identifying conflicting sentences might have over-simplified the reasoning task by taking the short-cut to ignore lots information, which just happens to be trivial in these specific datasets. \n\n2. The developed strategies, while interesting, might just be marginally relevant to the cognitive process of heuristic / analytical dual passes of human reasoning. The heuristic reasoning process is more related to the information being utilized and the amount of attention paid to more fine-grained details. For instance, in online language comprehension, comprehenders might ignore fine-grained syntactic structured and rely on the semantic meaning of the words and their prior knowledge to interpret \"the hearty meal was devouring...\" as \"the hearty meal was devoured...\" (Kim and Osterhout, 2005). However, the heuristic process is less concerned with gratuity of final decision, as implied by the HAR model. The HAR framework breaks the reasoning tasks into multiple sub-tasks, where the gratuity of the decision gradually becomes finer-grained. This might be better characteristics as step-by-step chain of reasoning rather than heuristic decision-making. \n\n3. The ICL-HAR, while improving consistency and verifiability, has greatly impedes the accuracy scores (dropping from 70.4 to 55.6 on TRIP).  This should be discussed or at least acknowledged in the main text in more detail. \n", "questions_to_authors": "A. I am unclear about line 390 to line 393 even after looking at the appendix. How are the 3 extracted predictions on each testing exampled combined? How is the metrics calculated?", "paper_topic_and_main_contributions": "This paper proposes one fine-tuning and one in-context learning frameworks to improve the reasoning consistency and verifiability on physical commonsense reasoning benchmarks (TRIP and ProPara). The heuristic-analytical reasoning (HAR) framework is inspired by human cognitive processes of reasoning, where a heuristic greedy decision is firstly arrived based on simple associative features before engaging in an analytical and rational process. The framework is implemented by breaking the reasoning task into multiple sub-steps (e.g. identify story plausibility -> detect conflicting sentence -> focus on physical state). The HAR significantly improves the coherence of model decisions, and that the improvement is attributed to a faithful attention to relevant language context. ", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "9WWl35vkwP", "reviewer": "Reviewer_CnKW", "summary": "", "questions": "QA. Why is HAR more suitable to improve performance on physics tasks than other prompting methods (such as CoT)? Given its resemblance to CoT prompting, I would like to see some similar techniques (e.g., those mentioned in Section 2) as baselines in Section 5.\n\nQB. In Table 3, the ICL-U method exhibits superior accuracy compared to ICL-HAR in both TRIP (70.4/55.6) and Tiered-ProPara (51.2/41.8). Can you provide an explanation for this discrepancy in the paper?\n\nQC. How do you deal with the problem mentioned in the limitations that model outputs may be influenced by decisions/errors on higher-level tasks? Will it affect the effectiveness of the model?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your thoughtful comments and feedback! We appreciate your recognition of the valuable insights our work can bring for AI-based reasoning, and respond to your concerns below.\n\n**W1: Cognitive Motivations of HAR**\n\nSorry, perhaps there is a misunderstanding here. HAR is motivated by the dual process theory which utilizes intuition to remove irrelevant information and process low-level rationalizations. In the paper “Heuristic and analytic processes in reasoning” (Evans, 1984), Evans highlighted “a distinction between heuristic processes which select items of task information as ‘relevant’, and analytic processes which operate on the selected items to generate inferences or judgements.” Works cited in Section 1 review a breadth of cognitive psychology research that supports this theory.\n\nSimilar to Evans’ description, HAR first makes an intuitive prediction on a high-level task which acts as a heuristic for rationalizing a low-level task. Taking the TRIP dataset as an example, HAR judges whether a story is plausible or not in the first step, identifies the conflicting sentences in the second step, and then rationalizes the physical states that cause the conflict in the third step. Our attention analysis experiments in Section 5.2 verify that this occurs due to the model’s stronger focus on relevant information during reasoning, which resembles the above synergy of heuristic and analytic processes in human cognitive psychology. Therefore, we think our motivation is appropriate and verified.\n\nFurthermore, it is worth noting that HAR differs from traditional chain-of-thought in prompting LLMs (CoT). We discuss this in more detail in our response to your QA.\n\n**W2: Cascading Errors**\n\nYes, as discussed in our Limitations section, the cascading errors in PLMs’ reasoning is a possible limitation. It is indeed possible that various non-greedy generation or reasoning strategies may be applied on top of HAR to mitigate this limitation, such as beam search or [tree-of-thought](https://arxiv.org/abs/2305.10601) to generate multiple candidate rationalizations and choose one based on LLM likelihoods. Such methods to reduce cascading error are orthogonal to our efforts, as they could theoretically be applied to any of the comparison approaches used in this work (not just HAR). As our goal was to demonstrate that heuristic decisions can be used to improve performance on analytic rationalizations in PLMs, we chose not to explore these approaches here as they would increase the moving parts in our study, and possibly distract from this key message.\n\nNonetheless, we show that even with possible cascading error, the model still outperforms all baselines by a great margin. The gradient-based fine-tuning model achieved state-of-the-art results on TRIP despite this effect. We’ll discuss this issue further in our revised version.\n\nAnd while these approaches may seem simple, they lead to novel and valuable insights. HAR enables performance improvements by refocusing LLMs to use the correct context to generate more coherent reasoning chains, a phenomenon which (to our knowledge) hasn’t yet been studied in in-context learning with LLMs. This finding aligns with dual process theories from cognitive psychology research (Wason, 1974; Evans 1976; Evans 1984), providing hope and motivation for future work to further explore cognitively-motivated strategies to enable more coherent reasoning in LLMs, and mitigate timely issues like hallucination which limit their practical usefulness.\n\n**W3: Analysis of Tasks**\n\nGood point! We agree that dual process theory is a common way of thinking for humans and is not limited to any specific task. We choose physical commonsense reasoning as an example because it involves high-level conclusions (e.g. plausibility of a story or other decisions about texts describing actions) and low-level predictions (e.g. physical states of objects) and there are datasets that contain annotations for different levels. Additionally, our paper addresses one of the specific challenges in physical scenarios (improve coherence and low-level physical state predictions), which has a possibly broad impact in embodied AI, where human users will communicate with agents about actions, and expect that agents have a coherent understanding of them in terms of how they will impact the surrounding environment.\n\nTo the best of our knowledge, there are few available datasets for other scenarios that can be used in such a tiered reasoning setup (we have searched extensively). We acknowledge the importance of creating such datasets and encourage future works to further explore in this direction to better characterize the space of tasks where heuristic-analytic reasoning can be beneficial. We will add more discussion about this in our revision.\n\n**QA: Comparison to Chain-of-Thought**\n\nWe provide some explanation above as to why we focused on physical commonsense. As for the resemblance to CoT prompting, there’s an important distinction between our method (HAR) and traditional Chain-of-Thought (CoT). Traditional CoT breaks down a hard problem into smaller subproblems to improve complex reasoning, such as for multi-step math problems. In physical commonsense, low-level atomic physical states are the most difficult for AI models to predict, and they are difficult to further decompose. As such, **we argue that for these kind of problems, the key for solving them is not decomposing them, but rather helping the model make predictions over the correct context (e.g., pay more attention to the correct context and remove irrelevant information).** HAR achieves this by conditioning rationalization predictions with information from higher-level heuristic predictions. We will clarify this in the revised version.\n\nNonetheless, we attempted to integrate CoT into our framework by augmenting the ICL-U prompts for each of the 3 sub-tasks (i.e., story selection, sentence selection, and state prediction) with free-text explanations. These explanations are generated by the LLM by prompting it with “Think step by step” after the sub-task context for each in-context demonstration and the testing example being solved, then they’re appended to the prompt. This gave us a few-shot CoT method that applies to our task. The results below show that applying traditional CoT (ICL-CoT) doesn’t improve performance much. Further, we observed that the generated reasoning explanations tend to iterate through sentences in stories to do step by step reasoning upon them, which doesn’t add useful information to the problem or simplify it- the model still needs to make predictions on lower-level tasks from a large context with a lot of irrelevant information. Therefore, we proposed HAR, a method that can use high-level (heuristic) predictions to focus on the correct context for low-level (analytic) predictions.\n\n*TRIP (InstructGPT)*\n| Method | Accuracy | Consistency | Verifiability |\n|--|--|--|--|\n| ICL-U | 70.9 | 40.7 | 7.1 |\n| ICL-CoT | 70.7 | 39.6 | 8.3 |\n| ICL-HAR | **72.6** | **47.9** | **23.9** |\n\n*TRIP (LLaMA)*\n| Method | Accuracy | Consistency | Verifiability |\n|--|--|--|--|\n| ICL-U | **70.4** | 42.3 | 14.8 |\n| ICL-CoT | 62.0 | 43.7 | 14.1 |\n| ICL-HAR | 55.6 | **44.4** | **35.2** |\n\n*Tiered-Propara (InstructGPT)*\n| Method | Accuracy | Consistency | Verifiability |\n|--|--|--|--|\n| ICL-U | **54.9** | 17.4 | 5.2 |\n| ICL-CoT | 52.1 | 23.9 | 12.2 |\n| ICL-HAR | **54.9** | **31.5** | **20.7** |\n\n*Tiered-Propara (LLaMA)*\n| Method | Accuracy | Consistency | Verifiability |\n|--|--|--|--|\n| ICL-U | **51.2** | 3.8 | 1.4 |\n| ICL-CoT | 48.8 | 4.7 | 1.9 |\n| ICL-HAR | 41.8 | **17.8** | **13.1** |\n\nWe will add this result into the revised version.\n\n**QB: Accuracy Drop in ICL-HAR with LLaMA**\n\nGood observation! Although ICL-U is more accurate than ICL-HAR in some cases, we would highlight that the focus of these tasks and our work is to improve the *coherence* of reasoning, which accuracy does not capture (only measures performance on the end task of story selection). Instead, our goal is to optimize **consistency** and **verifiability**, each of which already require end-task predictions to be accurate as a prerequisite, and measure the correctness of the sentences and physical states identified by the model to rationalize them. In fact, it is not ideal for accuracy to far exceed these metrics, as this indicates the model is making a correct prediction on the end task without valid underlying support (incoherent reasoning). Therefore, a drop in accuracy is not problematic in these tasks. We will better clarify the significance of each evaluation metric in our revision.\n\nThat said, this drop only happens with LLaMA-65B, and we think this is because it’s much smaller than InstructGPT (175B), so it is more sensitive to long prompts and generations. A quick fix could be to sequentially prompt the LLM multiple times with shorter prompts for each sub-task. We presented results for an approach like this in Appendix A.2, and they don’t exhibit such a drop in accuracy.\n\n**QC: Cascading Errors**\n\nYes, as discussed in our Limitations section, the cascading errors in PLMs’ reasoning is a possible limitation. It is indeed possible that various non-greedy generation or reasoning strategies may be applied on top of HAR to mitigate this limitation, such as beam search or [tree-of-thought](https://arxiv.org/abs/2305.10601) to generate multiple candidate rationalizations and choose one based on LLM likelihoods. Such methods to reduce cascading error are orthogonal to our efforts, as they could theoretically be applied to any of the comparison approaches used in this work (not just HAR). As our goal was to demonstrate that heuristic decisions can be used to improve performance on analytic rationalizations in PLMs, we chose not to explore these approaches here as they would increase the moving parts in our study, and possibly distract from this key message.\n\nNonetheless, we show that even with possible cascading error, the model still outperforms all baselines by a great margin. The gradient-based fine-tuning model achieved state-of-the-art results on TRIP despite this effect. We’ll discuss this issue further in our revised version.\n\n**Typos: Model Context Length**\n\nThank you for pointing out this typo! We should use “latter” instead of “former” to refer to LLaMA-65B model. InstructGPT is indeed having a context length of 4096 tokens, but LLaMA-65B only has a context length of 2048 tokens so we created a filtered version of TRIP to make the prompt shorter for the familiarization. We will fix this in our revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Introducing heuristic-analytic reasoning to PLMs for coherent commonsense reasoning, inspired by human cognition, offers valuable insights for enhancing AI reasoning.\n2. Demonstrating significant performance improvements with HAR on TRIP and Tiered-ProPara benchmarks highlights its effectiveness in enhancing reasoning coherence, for both fine-tuning and in context learning using HAR.\n3. Coherent reasoning is an important research question of PLMs, this paper fills the void and uses consistency and verifiability to evaluate lower-level steps.", "reasons_to_reject": "1. The HAR method in this article shares some similarities with step-by-step thinking (a chain-of-thought with fixed inference steps), but it significantly differs from the dual pathway in cognitive science, which involves combining intuition and rationality. As a result, the motivation for this article may not be appropriate. \n2. As mentioned in the limitation section, this approach potentially suffers from cascading errors in PLMs' reasoning, propagating from higher-level to lower-level tasks. Now that there are works on reflective/ iterative reasoning, the novelty of this work seems a little limited.\n3. I would like to see a more in-depth analysis of this task. This work mainly focuses on physical commonsense reasoning task. However, dual process theory is a common way of thinking for humans and is not limited to specific task scenarios. So it would be better to clearly point out and address the specific challenges of physical scenarios.", "questions_to_authors": "QA. Why is HAR more suitable to improve performance on physics tasks than other prompting methods (such as CoT)? Given its resemblance to CoT prompting, I would like to see some similar techniques (e.g., those mentioned in Section 2) as baselines in Section 5.\n\nQB. In Table 3, the ICL-U method exhibits superior accuracy compared to ICL-HAR in both TRIP (70.4/55.6) and Tiered-ProPara (51.2/41.8). Can you provide an explanation for this discrepancy in the paper?\n\nQC. How do you deal with the problem mentioned in the limitations that model outputs may be influenced by decisions/errors on higher-level tasks? Will it affect the effectiveness of the model?", "paper_topic_and_main_contributions": "This paper introduces the concept of heuristic-analytic reasoning (HAR) in pre-trained language models (PLMs) for coherent commonsense reasoning. The authors propose two strategies to incorporate HAR into PLM fine-tuning and in-context learning. They evaluate the effectiveness of HAR on two benchmark tasks, Tiered Reasoning for Intuitive Physics (TRIP) and Tiered-ProPara, and compare it to baseline approaches. The results demonstrate that HAR leads to substantial enhancements in reasoning coherence, surpassing competitive baselines. Additionally, the approach enables PLMs to focus more faithfully on relevant language context during reasoning.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "- In Line 374, the authors mentioned that former PLM is limited to a context length of 2048 tokens, which is not true for InstructGPT (4096 tokens)."}, "extra_text": {}}, {"review_id": "irwJuOJtsU", "reviewer": "Reviewer_5psW", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the thoughtful comments and feedback! We appreciate your recognition of our thorough experiments and analysis, including our effort to reframe the ProPara dataset. We respond to your concerns below.\n\n**W1: Innovativeness of HAR**\n\nWhile these approaches may seem simple, they lead to novel and valuable insights. HAR enables performance improvements by refocusing LLMs to use the correct context to generate more coherent reasoning chains, a phenomenon which (to our knowledge) hasn’t yet been studied in in-context learning with LLMs. This finding aligns with dual process theories from cognitive psychology research (Wason, 1974; Evans 1976; Evans 1984), providing hope and motivation for future work to further explore cognitively-motivated strategies to enable more coherent reasoning in LLMs, and mitigate timely issues like hallucination which limit their practical usefulness. We will better emphasize this in our revision.\n\n**W2: Missing Paper “Language Modeling with Latent Situations”**\n\nWhile our paper works on some of the same problems as this paper which recently appeared in Findings of ACL 2023, our approach is quite different. Like this work, we propose strategies for applying LLMs to downstream reasoning tasks in both in-context learning and fine-tuning settings. Their work shows that learning (low-level) world state representations improves coherent reasoning in LLMs. Our work, somewhat complementary, shows that supporting the (analytic) prediction of these low-level physical states is improved by conditioning it with higher-level (heuristic) predictions about texts, and this is enabled by a shift in attentional patterns when solving tasks.\n\nWe would have liked to include this paper in our results, but it does not report the same evaluation metrics on TRIP as we do. However, we will certainly add this paper to our Related Work section now that it is published. Thank you for pointing it out!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The authors demonstrate that PLMs' reasoning capabilities can benefit from incorporating complete cognitive processes containing both heuristic and analytic thinking. Experiments on fine-tuning and in-context learning with PLMs show that passing information from high-level to low-level tasks helps PLMs achieve better performance on low-level tasks.\n2. The authors refashion the Propara dataset into a Tiered-Propara dataset and provide ample experimental results.\n3. For in-context learning, the authors delve into the internals of LLMs. Utilizing the open-source LLaMA-65B, the authors visualize the Faithful Attention within the LLM, clearly and effectively displaying how HAR aids LLMs in focusing on the correct content.", "reasons_to_reject": "1. The innovativeness of HAR is limited to some extent. On the one hand, HAR for PLM fine-tuning can be considered as constructing a pipeline of subtasks, so its performance gains are unsurprising. On the other hand, HAR for PLM in-context learning is a kind of \"reverse\" chain-of-thought method tailored for physical commonsense reasoning, obtaining the result first and then analyzing the process.\n2. The idea behind HAR is similar to the paper \"Language Modeling with Latent Situations\" published in December 2022, both utilizing intermediate state inference to enhance reasoning capabilities, and applying both fine-tuning and prompting. The authors should include a comparison to that work and analyze the unique aspects of HAR in more depth.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper proposes heuristic-analytic reasoning (HAR) strategies to improve coherent physical commonsense reasoning in pre-trained language models (PLMs). It takes inspiration from dual process theories in cognitive psychology, where humans use quick heuristic thinking to make decisions, then slower analytic thinking to rationalize them. The paper implements HAR for fine-tuning and in-context learning with PLMs. In fine-tuning, models make high-level plausibility judgments on stories first, then these decisions are used to mask irrelevant information and focus attention when making lower-level judgments about conflicting sentences and physical states. In in-context learning, PLMs are prompted to generate reasoning chains from high-level story selection to low-level physical state prediction. Experiments on TRIP and a novel Tiered-ProPara dataset show HAR achieves promising performance compared to baselines. Analysis reveals that improved coherence is enabled by more faithful attention to relevant context when generating each step of reasoning under HAR. The key contribution of this paper is demonstrating that human-inspired heuristic and analytic reasoning strategies can mitigate incoherence and improve trustworthiness in PLMs for physical commonsense reasoning.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper takes inspiration from dual process theories in cognitive psychology, where humans use quick heuristic thinking to make decisions, then slower analytic thinking to rationalize them. While they use a simple approach (like a reverse CoT), it leads to valuable insights on their new benchmark "Tiered Propara". To understand these empirical gains, they visualize the Faithful Attention within the LLM, clearly and effectively displaying how their approach aids LLMs in focusing on the correct content. 

Limitations:
While the simple approach presented in this paper works for the two benchmarks, fine-tuning and in-context learning strategies seem to be tailored for datasets where key information is localized, allowing for shifting model attention to a smaller chunk of key information. Thus, it remains to be empirically verified if this approach generalizes to other datasets where the key information is not localized. Further, errors in can be cascaded in this reverse CoT reasoning and this issue must be discussed in more detail.

Overall, the strengths outweigh the limitations. It would be valuable to include the reviewer's suggestions and rebuttal content in the camera ready.
Decision: Accept-Main