Language Agents Mirror Human Causal Reasoning Biases. How Can We Help Them Think Like Scientists?
Authors: Anthony GX-Chen
Dongyan Lin
Mandana Samiei
Doina Precup
Blake Aaron Richards
Rob Fergus
Kenneth Marino
Venue: COLM 2025 (2025)
Keywords: agents
decision making
exploration
cognitive science
causal inference
TL;DR: Language model agents exhibit human-like reasoning biases, leading them to arrive at incorrect conclusions of causal relationships

Abstract: Language model (LM) agents are increasingly used as autonomous decision-makers which need to actively gather information to guide their decisions. A crucial cognitive skill for such agents is the efficient exploration and understanding of the causal structure of the world—key to robust, scientifically grounded reasoning. Yet, it remains unclear whether LMs possess this capability or exhibit systematic biases leading to erroneous conclusions. In this work, we examine LMs’ ability to explore and infer causal relationships, using the well-established Blicket Test paradigm from developmental psychology. We find that LMs reliably infer the common, intuitive disjunctive causal relationships but systematically struggle with the unusual, yet equally (or sometimes even more) evidenced conjunctive ones. This “disjunctive bias” persists across model families, sizes, and prompting strategies, and performance further declines as task complexity increases. Interestingly, an analogous bias appears in human adults, suggesting that LMs may have inherited deep-seated reasoning heuristics from their training data. To this end, we quantify similarities between LMs and humans, finding that LMs exhibit adult-like inference profiles (but not child-like). Finally, we propose a test-time sampling method which explicitly samples and eliminates hypotheses about causal relationships from the LM. This scalable approach significantly reduces the disjunctive bias and moves LMs closer to the goal of scientific, causally rigorous reasoning.

Published as a conference paper at COLM 2025
Language Agents Mirror Human Causal Reasoning Biases. How Can We Help Them Think Like Scientists?
Anthony GX-Chen , Dongyan Lin 23,t Mandana Samiei 233.t, Doina Precup 233,4
Blake Aaron Richards 2,34, Rob Fergus Kenneth Marino
NNw York University, USA
McGill University, Canada
Mila Quebec Artificial Intelligence Institute, Canada
CIFAR Learning in Machines and Brains Program, Canada
'he University of Utah, USA
Equal contribution, ordered alphabetically
Correspondence: anthony. gx. chen@nyu. edu
Abstract
Language model (LM) agents are increasingly used autonomous decision- makers which need to actively gather information to guide their decisions. A crucial cognitive skill for such agents is the efficient exploration and understanding of the causal structure of the world-key to robust, scientif- ically grounded reasoning. Yet, it remains unclear whether LMs possess this capability or exhibit systematic biases leading to erroneous conclu- sions. In this work, we examine LMs' ability explore and infer causal relationships, using the well-established Blicket Test paradigm from devel- opmental psychology. We find that LMs reliably infer the common, intuitive disjunctive causal relationships but systematically struggle with the un- usual, yet equally (or sometimes even more) evidenced conjunctive ones. This "disjunctive bias" persists across model families, sizes, and prompting strategies, and performance further declines as task complexity increases. Interestingly, an analogous bias appears in human adults, suggesting that LMs may have inherited deep-seated reasoning heuristics from their train- ing data. To this end, quantify similarities between LMs and humans, finding that LMS exhibit adult-like inference profiles (but not child-like). Finally, we propose P test-time sampling method which explicitly samples and eliminates hypotheses about causal relationships from the LM. This scalable approach significantly reduces the disjunctive bias and moves LMs closer to the goal of scientific, causally rigorous reasoning.
Introduction
Language models (LM) have achieved remarkable recent advances, driving progress in natural language processing, human-computer interaction, and robotics. Building on these models, LM "agents"-whice observe and act in an environment over time-are rapidly gaining prominence. Such "agents" offer the promise of fully autonomous intelligent decision making, and exploratory works have already applied these agents challenging settings such as designing new antibody fragments (Swanson et al., 2024), and taking over the full experimental and paper writing loop of machine learning conference papers (Lu et al., 2024).
central cognitive ability of any intelligent agent is an ability to discover causal relation- ships in its environment (Gopnik et al., 2004b; Blaisdell et al., 2006). Despite their success, unclear LM agents naturally possesses this ability. Additionally, human data fun- damentally shape LM agents (Gao et al., 2020; Raffel et al., 2020). LMs pre-trained to imitate human-text, fine-tuned on human-derivee signal including supervised instructions (Ouyang et al. 2022; Wang et al. 2022) and/ or preference-based reward models (Christiano

---

Published as a conference paper at COLM 2025
You room. see machine the center of this room.
There also objects scattered around the Vou sesve them: object 0 is on the floor, object 1 is on the floor, object 2 on the floor.
Exploration Stage
The mochine hums softly in front of you, seemingly waiting. The light on the machine is currently off. You wonder if there relationship between the objects and machine.
put object machine
You object on the machine. The light the machine is currently off.
put object machine
"Disjunctive" Condition:
You object on the machine. The light the machine is now
any blickets the machine turns on lig hig
Q&A
Based the information you have gathered, answer fallowing question: object blicket?
"Conjunctive" Condition:
all blickets must be the machine to turn on light
Figure The Blicket Test.
et al., 2017; Bai et al., 2022). Even techniques like chain-of-thought prompting (Wei et al., 2022) still rely on human-documented reasoning. Yet, decades of research in psychology show that humans can be systematically irrational many contexts (Kahneman, 2011). A more specific question, then, is whether LMs trained to mimic human behaviour inherit human-like biases and heuristics when reasoning about causal relationships?
To answer the above questions, has long been used to study how individuals throughout development infer and discover causal relationships (Gopnik & Sobel, 2000a; Gopnik et al., 2004a; Sobel et al., 2004a; Gopnik et al., 2001; Sobel & Kirkham, 2007; Lucas et al., 2014a; Walker & Gopnik, 2014; Bonawitz et al., 2010). These works have shed light on how human adult causal inferences can deviate from purely rational norms, but infants and toddlers reason with less bias like "scientists in the crib".
adopt the "Blicket Test" from cognitive science, which
In this paper, we adapt the Blicket Test into text-based sequential decision making game to evaluate the LM's causal reasoning capabilities. Critically, we assume the LM is "agentic" and is capable of active learning: it must first take actions to discover how its world works, then reflect on its own accumulated past experiences (which, in our set-up, is always stored and given back to the LM as prompts) to infer correct causal relationship in its world.
The contributions are as follows:
1. We conduct rigorous experiments of LM agents' performance in our text-based Blicket Test to study their ability to explore and reason about causal relationships.
2. We show they explore poorly, act inefficiently to narrow down hypotheses, and can fail infer correct causal relationships even when provided with perfect exploration data.
3. We directly compare LM behaviour to human developmental data, and find that LMs exhibit reasoning biases similar to adults (but not children).
4. We propose test-time procedure that add this bias by explicitly constructing flatter prior, and prompting the LM to eliminate hypotheses under this new prior. This significantly improves their performance.
2 Experimental Set-Up
2.1 The Blicket Test
The Blicket Test is an experimental paradigm involving N objects and a Blicket-detecting machine (Gopnik & Sobel, 2000a). A subset of the objects are "Blickets", which activate the
2

---

Published as a conference paper at COLM 2025
machine following some unobserved rule. There are two possible rules. The "disjunctive" rule describes an OR relationship, where the machine turns on when any Blicket objects are placed on it. The "conjunctive" rule describes an AND relationship, where the machine only turns on when all Blicket objects are placed on it. These structures are ubiquitous in real-world reasoning settings: a burglar alarm may be triggered by an intruder or the wind, while a heart attack might require both high blood cholesterol and genetic susceptibility (Lucas et al., 2014b). Importantly, the structural causal model for these two rules is the same: If the Blicket-detecting machine is the child node, then its parent nodes are the Blickets that can turn it on, regardless of whether the rule is conjunctive or disjunctive. An agent trying to solve the Blicket Test needs to interact with the environment to figure out which objects turn on the machine, and the rule governing the machine (see detailed discussion in Appendix A).
We convert the Blicket Test to text-based game (Figure 1). First, the agent enters an exploration phase. In each step, the agent can place single object on or off the machine (via specifying "> put [object id] [on/off] the machine"), "), and observe the state of the machine (whether the light turned on off). The agent also can perform auxiliary actions including terminating the episode before the pre-specified maximum number of steps each trial (via "> exit") if it believes it has collected enough information. After the exploration stage, the agent enters the Q&A phase. It is provided with the full observation history and asked whether each of the N objects are Blickets. The agent answers "> True False" to each question.
The goal of the agent is to correctly answer all of the questions. To do this, the agent must collect sufficiently informative observations during the exploration phase in order to resolve the uncertainty about Blicket identities and the underlying rule.
2.2 Information Gain
One way to measure optimal behaviour in the Blicket Test is through each action's informa- tiveness (Kosoy et al., 2022b). Concretely, the space of hypothesis F is the set of functions mapping from the objects' states (on off the machine) X E X' to the machine's state (light
is on or off), Y y. F X + y. The agent's goal is to discover the correct F E F, via maximizing information gain (Bernardo, 1979; Rainforth al., 2024):
(ifoGain(x.y) := H[p(F)] (F(((Fxx))))
(1)
This describes the reduction in (Shannon) entropy from the prior over the hypothesis space, p(F), to the posterior after observing new data, y)] In the Blicket Test, the space of hypothesis is discrete, consisting of all combinations of items being Blickets with the number of rules.? 2 Further, if we assume the the distribution p(F) is always uniform over all non-zero hypotheses, then maximizing information gain correspond to eliminating the most number of hypotheses. When all but one hypothesis remains, p(F) has zero entropy and no further information gain is possible.
In practice, an agent does not know the outcome y a priori. It can instead maximize expected information gain,
G(x) == E E((y)) [InfoGain(x, y)] = IE ((F))(((Fx [log((F(x.)) p(F)
(2)
As a baseline, we will construct an "Oracle InfoGain" agent which explicitly computes and maximizes this quantity based one-step information maximization as good approxima- tion of the upper-bound for how well we can explore (more details in Appendix B.1).
2.3 Models and Baselines
We evaluate a number of LMs (Appendix B.2), over a range of carefully designed system messages (Appendix B.4), with multiple prompting methods (Appendix B.5). All agents are
Thhre can be more than two rules, but we used the two most commonly used rules in the existing literature within the scope of this paper. 2For N items and 2 rules (conjunctive/ / disjunctive), there are total 2+1 hypotheses.
3

---

Published as a conference paper at COLM 2025
allowed up to 32 steps in given trial. We evaluate in easier environment containing 4 objects (2 are Blickets), and a harder environment containing 8 objects (2 are Blickets). An example full interaction trace is provided in Appendix B.6.
Baseline Models We compare the LM agents against non-LM baseline agents. One is the InfoGain Oracle, which explicitly calculates the expected information gain for each action, and takes the action with the max information gain. This "Oracle" reasons about how to do exploration optimally and provides an approximate upper-bound for performance in this environment. The second agent is simply a random agent which takes random actions (to put objects on or off the machine), and also randomly guesses the answers during the Q&A phase, providing a lower-bound for performance.
Having now established the task and experimental set-up, we report the ability of LMs to explore and infer causal relationships below.
3
LM Agents Exhibit Cognitive Biases
3.1 LM agents all perform poorly in the Blicket Test
Best Model Accuracy 14 objects)
Best Model Accuracy (8 objects)
Model Accuracies (8 Objects)
0.0
0.0
dajunctiow
contunetivve
dsjunctiv
conunddive
Objects
Objects
Dajurctune Accurcy
beselie
got-fo
despsembehal
irfoGa n enads dospuost chat
cst
InfoCain
gemms3:277
bacedine modion
deeponeic meppone
defuut
g2h-la
gemrmw7:275
mact
got-lommi
reflcion
(a) Best models over prompt variants. Error bar denote stan- dard error of mean multiple random seeds. Details in Appendix B.3.
(b) All models disj/ / conj accuracies. Shape shows prompt techniques.
Figure 2: Quiz accuracy of various models during the peeetionnand-answeriii phase of the Blicket Test. The accuracy measures the proportion of trials where the model correctly identifies all Blickets
First, we directly evaluate the ability LM agents to successful identify all Blickets. Figure 2a shows the result of the best model within each model class over all system messages and prompting methods. Across the board, all LMs struggle when the number of objects increase from 4 to 8. Interestingly, LMs also systematically struggle when going from the disjunctive ("OR") rule to the conjunctive (" ("AND") rule. This is not due to conjunctive rules being harder: the InfoGain Oracle can perfectly resolve the entire hypothesis space every time and achieves an Q&A accuracy of 1. This hints at the LMs having a disjunctive bias: a preference for a disjunctive interpretation of the world over a conjunctive one. Models perform worse in the 8 object, conjunctive case. This effect is further observed in Figure 2b: across various models and prompting techniques, the LMS systematically skew toward lower conjunctive accuracy. We report both the 4 and objects results in Figure 11.
The results here outline a fundamental inability of LMs to causally explore and discover its environment, even though we are in setting where optimal solution is tractably computable. We study this further. We first investigate general factors that correlate with Blicket Test success in Section 3.2, study LM's (in)ability to do efficient exploration in Section 3.3, and evaluate their (in)ability to infer causal relationship unbiasedly from data in Section 3.4.
4

---

Published as a conference paper at COLM 2025
Gan Accuaacy
Valiiaions Acceaacy
Rule: disjunctive, Objects:
Rule: conjunctive, Objects:
Spearman's p-0.76 2-8.0e-14
Spearman' p=0.007
Remaining
20+
101
Telmetmeto Saln
Humber Unbus Stites Walked
Stass Aocuracy
Rassorss Longth Accusacy
Spicamma's pw-0.35 p-0.004
Spearmanr p=30.30 g=0.013"
disjunctive, Objects:
Rule: confunctive, Objects:
Poporhetts Remanin
202
10*-
20
20
Sumber Adtem Sdbre Di
Figure 3: Correlation analysis of fac- tors contributing to model performance. Each point unique model prompt + environment rule combination. The Spearman's 's rank correlation along with the p-value is reported.
100
20
Stops
Stops
gammm11:12
gpt-to
gol-fo-min gommor. 99.27
docppeck chat
Info soacc
baserine random
qwe
Figure 4: Hypothesis elimination efficiency.
3.2 Which factors contribute to success?
We begin by studying common factors which may contribute to agent performance by analyzing correlations in the objects setting. For each factor, we compute the best logistic fit, and conduct statistical hypothesis testing using Spearman's rank correlation to determine whether the factor is (rank) correlated with final performance. Spearman's 's instead of Pearson's correlation is used as the data do not necessarily obey linear relationships. We report the result and p-values in Figure 3.
Information Gain (Equation 1) is computed between the initial and final p(F) (assuming uniform distribution over consistent hypothesis). This shows the strongest correlation with performance accuracy, leading credence to information gain being measure of optimality in this task.
Unique State Visitation is commonly used metric in reinforcement learning as proxy for good exploration (Bellemare et al., 2016). While this correlated with final performance (p < 0.05), it shows weaker correlation compared to information gain.
Number of steps taken a proxy for when the LM believes it has gathered enough information, and therefore exits the trial early. Interestingly, this is strongly (negatively) correlated with performance, hinting at the LMs having a good notion of when it has explored well.
Response length measures the average number of output tokens during the Q&A phase (including reasoning tokens for reasoning models). This is proxy for the amount of "reasoning" " the LM performs. We find that longer reasoning correlate with lower performance (p < 0.05).
As information gain-an an exploration metric-shows the strongest correlation with final performance, we now turn to an in-depth study of each agent's ability to efficiently explore.
3.3 All LM agents do not explore efficiently
To study exploration efficiency, we quantify the number of hypotheses consistent with observed data as function of actions taken. An agent that efficiently explores should rapidly eliminate hypotheses down to just one. This equivalent to maximizing information gain as defined in Section 2.2.
5

---

Published as a conference paper at COLM 2025
We observe in Figure that the InfoGain Oracle efficiently reduces the number of hypothesis down to one. LM agents, on the other hand, perform worse. Similar to Section 3.1, they struggle when the number of objects are increased, and with conjunctive ("AND") rules. This is likely not due to the conjunctive setting being inherently more difficult, as the InfoGain Oracle resolves both in similar number of actions (we díscuss this further in Appendix C.1). This provides further evidence for the disjunctive bias (here specifically for exploration). We also observe variability between the LM agents in their exploration efficiencies, with GPT-4o performing the best, while other frontier models such as Gemma3 and -ressee at times exploring worse than simply taking random actions. We additionally plot the agents' performance as [0,1] progress, and progress normalized by the baseling ine random performance to account for potential differences environment complexity. This is detailed in Appendix C.1, and show similar trend as Figure 4.
3.4
Most LM agents show reasoning bias when inferring causal relationships
Disjunctive Rule
Conjunctive Rule
1.0
Inference Accuracy
0.8
0.6
0.2
0.0
Ilm (gpt-4o)
oracle
random
object count
Iim tapt- 4o)
oracle
random
object count
Exploration
Data Source
Exploration Data Source
gpt-40
gpt-44-miini
desprooakrmaaannr
deepseek-chot
InfoGain
explore
Figure 5: Evaluating LM's ability to infer causal relationship when the same exploration data given as context in the 8 objects setting. Error bar denote standard error of mean.
Finally, investigate the LM's stand-alone ability to infer causal relationships by providing all LMs with standardized set of exploration data, including exploration trajectories generated by (i) an LM (GPT-4o), (ii) the InfoGain Oracle (which perfectly resolve the hypothesis space), (iii) random exploration, and (iv) count-based exploration agent that place objects onfoff the machine inversely proportional how often they've ve been perturbed before.
The results are in Figure 5 (additional data in Appendix C.3). Of particular interest is the conjunctive (" ("AND") setting: we observe that all LMs improve in Q&A performance when provided with oracle exploration data as supposed exploration plays causal role in their poor Q&A performance. However, for all but the deepseek-reasone model, the LMs still do not achieve near perfect accuracies despite the oracle exploration data. Further, disjunctive bias is still present for models such GPT-4o, which still performs worse in the conjunctive setting than the disjunctive one even with oracle data present in both.
LM-exploration, suggesting that bad
LM's cognitive biases appear similar to human adults
The systematic disjunctive bias observed in LM's causal reasoning raises a question: where do these cognitive biases originate? We postulate that LMs, trained on vast corpora of internet text predominantly generated by adult humans, naturally internalize cognitive biases characteristic of adult human reasoning. To this end, we further evaluate the LM agents in Blicket Test experiments where human data is available (Lucas et al., 2014b; Gopnik et al., 2017; Kosoy et al., 2022b) in order to provide direct comparisons of LM behaviours to that of humans. We conduct two sets of experiments which (i) compare the LM agents' preference when performing inference based on ambiguous information to human children and adults, and (ii) compare LM agents' exploration characteristics to children performing

---

Published as a conference paper at COLM 2025
Corjunctive Evidence
Disjunctive Evidence
Ambigueus Evidence
10.85
HM
oduls HH
aduls
ph14
0hi4
Inna3
1:270-
gemmm22:1
opt-to
got-4-min
soo-to-misi
spi-4o-
g21-1-
sp--fo
seepaeckrrssooe
cespreelecte:
deetuee-chal
despueele-the
more dij
more con)
more disj-
H moore con)
more disge
more con)
LI
Human
Figure 6: The number between 0 and (x-axis) is the proportion of responses that favour conjunctive inferences (about whether an ambiguous key object is Blicket), when conjunc- tive, disjunctive, or ambiguous hints about the rule governing the machine were presented.
the same task. Unlike the experiments in Section 3, all experiments in this section will be conducted with objects and Blickets to be consistent with the psychology literature.
4.1 Most LM agents prefer disjunctive answers given uncertain data (like adults)
We replicate the Blicket Test experiment from Lucas et al. (2014b) and Gopnik et al. (2017), which investigates children's and adults' preference for disjunctive versus conjunctive inference when ambiguous evidence is presented. In short, the agents are first presented with exploration data hinting at the Blicket-detecting machine following disjunctive ("OR") or conjunctive ("AND") rule. Then, the agent is provided with additional data with new objects, but the same machine. The new data is by design ambiguous, thus the agent must combine their previous belief about the machine with the new objects' data to identify which (new) objects are Blickets. In particular, for one new object, identifying it as a Blicket means that the agent believes that the machine obeys the conjunctive rule. We provide more details for this set-up in Appendix B.8.
We report the LM's tendency to identify the key object Blicket in Figure 6, along with that of (i) human children (4-year-olds) and (ii) human adults. The human data is replicated from Gopnik et al. (2017). We observe that general, LMs show high "adult-like" bias toward thinking the machine is disjunctive, even when the evidence suggests it's conjunctive. 4-year-olds, on the other hand, correctly infer that the machine is conjunctive when the evidence suggests so.
4.2
LM agents spend less time exploring disjunctive settings (unlike children)
Previous psychology research on the Blicket Test has predominantly focused on causal nhere participants are passively presented with observations. However, Kosoy et al. (2022b) took first step towards examining how children generate data through active causal exploration. Their results revealed that children's exploration is characterized by an intrinsic, unbiased curiosity; the underlying causal rule does not significantly impact children's exploratory behaviour (Figure 7, second and fourth panels from the left; additional data in Appendix C.4).
Here, we analyzed the exploration trajectories generated by LMs in 3-object setting, using system prompts as similar to the psychology experimental scripts as possible, to allow direct com camisonoon& to the psychology literature. We measured the number of unique object combinations attempted (same as unique states visited) as well as the number of steps taken before exiting in each trial, and compared the results to children data from Table 1 of Kosoy et al. (2022b). Our analysis showed that, unlike children, LMs explored differently for the different causal structures (Figure 7, first and third panels from the left): LMs generally attempted less unique combinations of objects and spent less time exploring in the disjunctive condition, suggesting that their exploration is influenced by their biases.
7

---

Published as a conference paper at COLM 2025
# Combinations
Combinations
# Actions
Actions
25
ns
F5
20
15
10
gpt-40
spt-4o-mini
deepseek-
deepseel-
Children
gpt-40
gpt upt-4o-mini
deepseaf-
despseak
Children
reasoner
chat
reasoner
chatt
Conjunctive
Disjanctive
Figure 7: LM exploration more affected by the causal rule than children. Left panels: average number of unique combinations tried per trial by LMs children. Right panels: exploration extensiveness per trial, measured as the average number of steps taken before exiting the trial in LMs, or the number of times children pressed the "check" button. Error bar denote standard deviation. Two-sample t-test: p < 0.05, < 0.01, *** < 0.001, ns: not significant.
Making LMs more scientific through inference time hypothesis sampling
Construct flattened prior via rejections
Belief update via hypothesis elimination
Example Blicket hypotheses
Repeat: "GLven the initial obracvation, geserate hypotheses aa python functions. bo net gunezate [..]"
hypotheeta passible"
Xpe
typerhests 1iabjects) babl: return dbjocts
typarhests Piahjects) dbjectslll
Run each function to evaluate pOan-1 x/xx))))
cef hypotmente 300b ects) =s bonl: abjects objerns(?]
P(F)
(F)
q0' ((
Figure 8: Hypothesis sampling agent first samples (without replacement) from the LM prior to construct new prior (F), then prompt the LM to maximize expected information gain under (F)).
One way we can interpret the LM's deep rooted bias for disjunctive hypotheses that the LM has skewed prior pFF). One way to fix this is to "flatten out" this prior. While is highly non-trivial to do this via weight updates, we can do this during inference time by explicitly representing uniform belief over a discrete set of hypothesis. Specifically, we construct new prior, (F),, which is "flatter" than the LM prior. We do this by sampling discrete hypotheses from p(F) and rejecting identical samples. We give each accepted samples uniform weight, which gives us (F). It is trivial to show that (F) is uniform over its support and that its entropy monotonically increases with each additional unique sample (Remark 1). After constructing the new prior (FF), we maximize expected information gain under this prior. Since the prior assigns uniform probability over each hypothesis, maximizing expected information gain corresponds to eliminating hypotheses. We therefore prompt the LM to "take actions to eliminate as many hypotheses as possible" from (F), update the hypotheses in ((F) based on new observations, and repeat. This procedure is outlined Figure and in more details in Appendix C.5.1.
We show the results from this procedure in Figure 9, With enough unique samples, expect (F) to provide a less skewed prior (as compared to p(F)), and maximizing expected information gain under (F) to better resolve the disjunctive conjunctive differences.
8

---

Published as a conference paper at COLM 2025
Q&A Performance
Num Hypotheses Remaining
default lim
1.0
10²
hypothesis sampl = 16)
Proprtion All Correc
0.8
hypothesis sampl (n 32)
hypothesis sampl 64)
0.4
hypothesis sampl 96)
conjunctive
disjunctive
infeGain oracle
0.2
baseline LM, disjunctive conjunctive
10
30
16
Steps
N Initial Unique Hypotheses
(a) Hypothesis elimination in conjunctive setting. Disjunctive results in Figure 15.
(b) Q&A performance function of initial unique sampled hypotheses.
Figure 9: Hypothesis sampling agent in the 8 objects environment.
Indeed, we see that with more initial unique samples, the LM agents both explore better, and do better in Q&A, showing no decrease in performance in conjunctive vs. disjunctive set- tings. Additionally, the hypothesis sampling agent no longer shows (disjunctively) biased exploration pattern initially observed in Section 4.2. We report this result in Appendix C.5.4.
Conclusion
Our study reveals that language model agents exhibit systematic biases in causal reasoning, particularly a "disjunctive bias" that resembles the reasoning patterns of human adults rather than children. Our test-time hypothesis sampling method significantly reduces this bias, advancing LMs toward more scientifically rigorous causal reasoning crucial capability for autonomous decision-making systems. Future work should explore whether our hypothesis sampling approach generalizes to more complex causal structures beyond Blicket tests, and to what extent it can be integrated with other reasoning techniques. Finally, additional human data, specifically detailing exploration patterns, can further shed light on the priors adults and children bring discovering causal relationship.
Source of human-like bias We speculate that the "disjunctive bias" exhibited by LM agents may stem from the way they are trained. LMs are pre-trained on vast corpora of human-generated text, fine-tuned using human instructions, and/or aligned through human preferences. Through language, any systematic tendencies and cognitive heuristics in human reasoning may be baked into the models' internal priors. Future work should investigate whether a causal link exists between human-generate training texts and an LM's disjunctive bias.
Construct validity of the Blicket task for LMs The Blicket task has an objectively correct answer and an inforrmaiinn-thooetiica testbed for causal reasoning LMs. Nevertheless, observing similar behaviours between LMs and humans does not imply identical internal cognitive mechanisms; rather, it suggests that the two may have overlapping capabilities. Drawing strong conclusions about inter- nal mechanisms of cognitive capabilities would require models with human-like sensory modalities, embodiment, and developmental learning trajectories-aa direction that out of reach with current LM systems (Frank, 2023). In its absence, we approximate the core structure of the task text-based format. As with any evaluations, these are approximate measures of some aspects of the overall capabilities of an intelligent system. The goal is that such benchmarks weahherrks-wwether -hethher in math, coding, or causal inference-would shed light on where the current system is from being "generally intelligent".
optimal exploration strategy, making it a useful

---

Published as a conference paper at COLM 2025
Acknowledgement
This work is supported by ONR MURI #N00014-22-112777, ONR #N00014-21-1-2778, and the National Science Foundation under NSF Award 1922658. Additionally, we acknowledge sup- port by NSERC (Discovery Grant: RGPIN-2020-055055; Discovery Accelerator Supplement: RGPAS-2020-00031; Arthur B. McDonald Fellowship: 566355-2022) and CIFAR (Canada AT Chair; Learning in Machine and Brains Fellowship). This research was enabled in part by support provided by (Calcul Québec) (https//wwww.calcllluubbbl Research Alliance of Canada material support of NVIDIA in the form of computational resources. AGC is supported by the Natural Sciences and Engineering Research Council of Canada (NSERC), PGSD3- 559278-2021. DL is supported by the Natural Sciences and Engineering Research Council of Canada (NSERC) M.S. was supported by Fonds de recherche du Québec - Nature et Technologies (FRQNT) #2023-2024-B2X-336333 and UNIQUE PhD Excellence Scholarship. We are grateful to Alison Gopnik, Eunice Yiu, and Fei Dai for their generous and insightful feedback on earlier results, which were invaluable in shaping the direction of this project. We are also thankful for insightful discussions with Shannon Yasuda, Solim LeGris, and Jeff Guo which have made this paper better in many ways.
/nn) and the Digital The authors acknowledge the
and #CGS-MSFSS-5881588 1588--2233, Additionally,
References
Yuntao Bai, Andy Jones, Kamile Ndousse, Amanda Askell, Anna Chen, al. Training helpful and harmless assistant with reinforcement learning from human feedback, 2022.
https://arxivv https://arxx..orr/bb//2222 org//bs//204. 05862.
Marc Bellemare, Sriram Srinivasan, Georg Ostrovski, Tom Schaul, David Saxton, and Remi Munos. Unifying count-based exploration and intrinsic motivation. Advances in neural information processing systems, 29, 2016.
José M Bernardo. Expected information as expected utility. the Annals of Statistics, pp. 686-690, 1979.
Aaron P Blaisdell, Kosuke Sawa, Kenneth Leising, and Michael R Waldmann. Causal reasoning in rats. Science, 11666111 2006.
Daniil A. Boiko, Robert MacKnight, and Gabe Gomes. Emergent autonomous scientific research capabilities of large language models. arXiv preprint axXiv:2304.05332, 2023. https://arxiv. wttpaaawwiivvorrarrraaaaaaaaae/e wttg/bs/////...../ 05332.
Elizabeth B Bonawitz et al. Children balance theories and evidence in exploration, expllaa tion, and learning. Cognitive Psychology, 2012.
Elizabeth Baraff Bonawitz, David Ferranti, Rebecca Saxe, Alison Gopnik, Andrew N. Melt- zoff, and James Woodward. Just do it? investigating the gap between prediction and action in toddlers' causal inferences. Cognition, 115(1):104-17,2 2010 2010.
Tom Brown et al. Language models are few-shot learners. Advances in Neural Information Processing Systems, 2020.
Sébastien Bubeck al. Sparks of artificial general intelligence: Early experiments with gpt-4. arxiv preprint ar drXiv:2303.12711 2023.
Paul E. Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences, 2017. https://arxtr www.ii.wgg/aaaaaaeeeeeeeooooeeee org/abs/ 1706. 03741.
Ishita Dasgupta, Andrew K Lampinen, Stephanie CY Chan, Hannah R Sheahan, Antonia Creswell, Dharshan Kumaran, James McClelland, and Felix Hill. Language models show human-like content effects reasoning tasks. arXio preprint Xix: 07051, 2022.
10

---

Published as a conference paper at COLM 2025
Michael C Frank. Bridging the data gap between children and large language models. Trends in Cognitive Sciences, (711)):900-992 2023.
Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, et al. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint ar Xit:2101.00027 2020.
Alison Gopnik and David M. Sobel. Detecting blickets: How young children use information about novel causal powers in categorization and induction. Child Development, 71(5):1200- 1222, 2000a.
Alison Gopnik and David M Sobel. Detecting blickets: How young children use information about novel causal powers in categorization and induction. Child Development, 2000b.
Alison Gopnik, David M. Sobel, Laura E. Schulz, and Clark Glymour. Causal learning mechanisms in very young children: Two- three-, and four-year-olds infer causal relations from patterns of variation and covariation. Developmental Psychology, 37(5):620-66 629, 2001.
Alison Gopnik, Clark Glymour, David M. Sobel, Laura Schulz, Tamar Kushnir, and David Danks. A theory of causal learning in children: causal maps and bayes nets. Psychological Review, 111 2004a.
Alison Gopnik, Clark Glymour, David M Sobel, Laura E Schulz, Tamar Kushnir, and David Danks. A theory of causal learning in children: Causal maps and bayes nets. Psychological Review, 111(1):3-32, 2004b.
Alison Gopnik, Shaun O'Grady, Christopher G Lucas, Thomas L. Griffiths, Adrienne Wente, Sophie Bridgers, Rosie Aboody, Hoki Fung, and Ronald E Dahl. Changes in cognitive flexibility and hypothesis search across human life history from childhood to adolescence to adulthood. Proceedings of the National Academy of Sciences, 114(300:7892-7889, 2017.
Thomas Griffiths and Joshua Tenenbaum. Theory-based causal induction. In The psychology of learning and motivation, volume 51, pp. 59-108. Elsevier, 2009.
Thilo Hagendorff. Machine psychology: Investigating emergent capabilities and behavior in large language models using psychological methods. arXiv preprint ar arXv::233 13988, 1, 2023.
Daniel Kahneman. Thinking, fast and slow. macmillan, 2011.
Eliza Kosoy, David M Chan, Adrian Liu, Jasmine Collins, Bryanna Kaufmann, Sandy Han Huang, Jessica Hamrick, John Canny, Nan Rosemary Ke, and Alison Gopnik. To- wards understanding how machines can learn causal overhypotheses. arXiv preprint ar arXit:2206.088353 2022a.
Eliza Kosoy, Adrian Liu, Jasmine L. Collins, David Chan, Jessica B. Hamrick, Nan Rosemary Ke, Sandy Huang, Bryanna Kaufmann, John Canny, and Alison Gopnik. Learning causal overhypotheses through exploration in children and computational models. In Proceedings of the First Conference on Causal Learning and Reasoning, pp. 390-406. PMLR, 2022b.
Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, et al. Agentbench: Evaluating llms as agents. arviv preprint XrXi:22308.03663 033888 2023.
Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha. The ai scientist: Towards fully automated open-ended scientific discovery. arXio preprint ar Xiw:2448800629, 2024.
Christopher G. Lucas, Sophie Bridgers, Thomas L. Griffiths, and Alison Gopnik. When children are better (or at least more middem) learners than adults: Developmental differences in learning the forms of causal relationships. Cognition, 131(2))284-2)) 299, 2014a. 4a.
Christopher G Lucas et al. Children's causal inferences from conflicting testimony and observations. Developmental Psychology, 2014b.
11

---

Published as a conference paper at COLM 2025
Ollama. Ollama: Get up and running with Ilama 3.3, deepseek-rl, phi-4, gemma 3, and other large language models. https://githun www/cooooooooooo ollama/ollama, 2025. Accessed: 2025-03-28.
OpenAl. Gpt-4 technical report. Technical report, arXiv preprint arXiv:2303.08774, 2023,
Ian Osband, Charles Blundell, Alexander Pritzel, and Benjamin Van Roy. Deep exploration via bootstrapped dqn. In Advances in Neural Information Processing Systems, pp. 4026-4034, 2016. URL https: / /ttp/vawivivvaa///////ae//a.aa http://aaaii.oorar////aaoe wrwg//b////..eeeeere 04621.
Xi Ouyang, Ye Wu, and (etc.). Training language models to follow instructions with human feedback, 2022. httpps///aaxxi..rr/ab..ss//// 02151
Top Pirriakulkij, Cassidy Langenfeld, Tuan Anh Le, and Kevin Ellis. Doing experiments and revising rules with natural language and probabilistic reasoning. Advances in Neural Information Processing Systems, 37:53102-53137, 2024.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1-67,20 2020. http://jmlr.. org/ wwwpg//app/////////// 200 htm..
Tom Rainforth, Adam Foster, Desi Ivanova, and Freddie Bickford Smith. Modern bayesian experimental design. Statistical Science, 39(1):100-114, 2024.
Noah Shinn et al. Reflexion: autonomous agent with dynamic memory and self-reflection. arviv preprint ar arXiv:2303.11366 2023.
David M. Sobel and Natasha Z. Kirkham. Bayes nets and babies: Infants' developing representations of causal knowledge. Developmental Science, 10(3):298-33) 306, 2007.
David M. Sobel, Joshua B. Tenenbaum, and Alison Gopnik. Children's 's causal inferences from indirect evidence: Backwards blocking and bayesian reasoning in preschoolers. Cognitive Science, 28(3):303-333 2004a.
David M Sobel et al. Children's 's causal inferences from indirect evidence: Backwards blocking and bayesian reasoning in preschoolers. Cognitive Science, 2004b.
Kaya Stechly, Karthik Valmeekam, Atharva Gundawar, Vardhan Palod, and Subbarao Kamb- hampati. Beyond semantics: The unreasonable effectiveness of reasonless intermediate tokens. Xrv preprint arXiv:2505.13775 2025.
Kyle Swanson, Wesley Wu, Nash L Bulaong, John E Pak, and James Zou. The virtual lab: Ai agents design new sars-cov-2 nanobodies with experimental validation. bioRxiv, pp. 2024-11, 2024.
Joshua B Tenenbaum, Thomas L Griffiths, and Sourabh Niyogi. Intuitive theories as gram- mars for causal inference. Causal learning: Psychology, philosophy, and computation, 301:321, 2007.
Caren M. Walker and Alison Gopnik. Toddlers infer higher-order relational principles in causal learning. Psychological Science, 25(1):161-169, 2014.
Guanzhi Wang et al. Voyager: An open-ended embodied agent with large language models. arXiv preprint ar XrXi:2235..1621, 2023a.
Ruocheng Wang, Eric Zelikman, Gabriel Poesia, Yewen Pu, Nick Haber, and Noah D Goodman. Hypothesis search: Inductive reasoning with language models. arXiv preprint wwXiv:2309.05666 2023b.
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, et al. Self- instruct: Aligning language model with self generated instructions, 2022. https://arxi org/abs/2212. 10560.
Jason Wei et al. Chain-of thouugghtt prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 2022.
12

---

Published as a conference paper at COLM 2025
Shunyu Yao et al. React: Synergizing reasoning and acting in language models. arxio preprint ar Xii::2210.03629, 2022.
13

---

Published as a conference paper at COLM 2025
Appendix
A
Related Works
The Blicket-detecting Machine The Blicket detector task has a long history in develop- ment psychology extending back to seminal studies in causal reasoning and categorization (Gopnik & Sobel, 2000a; Gopnik et al., 2004a; Sobel et al., 2004a; Gopnik et al., 2001; Sobel & Kirkham, 2007; Lucas et al., 2014a; Walker & Gopnik, 2014; Bonawitz et al., 2010). The core premise of the Blicket detector is machine that activates under specific conditions, challenging an agent (in the original setting a child) to discover which objects (called "Blick- ets") trigger the machine and according to what rule (also known as "over-hypothesis"). A disjunctive rule will activate the machine when any Blicket is present, whereas a conjunctive rule requires multiple Blickets to activate the machine. By exploring object combinations and/ /or observing machine activation, agents must generate and test hypotheses about causal relationships. While simple, it reveals profound insights into how intelligent sys- tems-whetiee humans or di-discover and reason about causal structures, In our study, we translate this experimental paradígm into text-based interaction, probing LM's capacity to explore, infer, and reason about causal relationships.
Reasoning and Exploration in LMs Recent advancements in LMs have significantly expanded their applications in autonomous decision-making settings Brown et al. (2020); OpenAl (2023); Bubeck et al. (2023). Particularly, LMs have been successfully applied in tasks requiring exploration and reasoning, such as interactive decision-making Yao et al. (2022); Shinn et al. (2023); Wang et al. (2023a) and scientific discovery Boiko et al. (2023). For example, Yao et al. (2022) proposed ReAct, enabling LMs to interleave reasoning and acting, thus improving decision-making capabilities Yao et al. (2022). More recent frameworks like Reflexion (Shinn et al., 2023) and Voyager (Wang et al., 2023a) further leverage iterative exploration and reflection to boost autonomous exploration performance. However, existing literature also points to significant limitations in the exploratory capabilities of LMs. Osband et al. (2016) highlight that effective exploration remains a fundamental challenge even in traditional reinforcement learning agents (Osband et al., 2016). Recent work by Liu et al. (2023) further underscores the insufficient exploratory behaviours of LMs, suggesting that their tendency to imitate uuman-generated text leads to suboptimal macisinn strategies Liu et al. (2023). Our work complements these findings by systematically studying whether LMs exhibit biases specifically in causal inference contexts, thus bridging a gap in current understanding of LM exploration and inference.
A closely related work to our hypothesis sampling agent is Piriyakulkij et al. (2024), which similarly study Blicket-like environment and samples hypotheses from an LMM Our work is complementary to theirs, though we focus much more on investigating the LM's ability (and failure) to do causal exploration and reasoning, in a task where the LM needs to perform more actions, and evaluate over greater range of models. Another related work is Wang al. (2023b) which similarly generates code as hypotheses, although focusing on the inductive reasoning ARC task rather than agentic causal discovery tasks.
Biases in Human and Machine Reasoning Psychological research extensively documents systematic biases in human causal reasoning Tenenbaum et al. (2007); Griffiths & Tenenbaum (2009); Lucas et al. (2014b). Among these, the Blicket paradigm introduced by Gopnik and Sobel (2000) Gopnik & Sobel (2000b) has emerged as a canonical method for investigating causal inference in humans, particularly children. Crucially, research using Blicket tests shows robust disjunctive bias, where humans, especially adults, systematically favour simpler disjunctive causal hypotheses over conjunctive ones, even when evidence supports the latter equally or more strongly Lucas et al. (2014b); Bonawitz et al. (2012); Sobel et al. (2004b). These biases likely arise from human reliance cognitive heuristics or simplicity priors during causal reasoning
Previous studies indicate that LMs internalize human-like cognitive biases from their training data, including reasoning shortcuts or heuristics (Hagendorff, 2023; Dasgupta et al., 2022). Our study directly addresses whether these biases also manifest in LMS
14

---

Published as a conference paper at COLM 2025
performing causal reasoning tasks, using the Blicket paradigm to rigorously quantify such tendencies.
B
Experimental Details
B.1 The Oracle InfoGain Agent
For objects' states x E X, machine state y E V, and the set all hypotheses f F, the InfoGain agent maximizes expected information gain G(x) at each step. It iterates through the following:
1. Generating set of candidate next observations x to reach,
2. For each candidate, compute G(x),
3. Go to that maximizes G(x).
The set of candidate next observations are all results of single actions. We represent it as binary vector. E.g. if the current state is 10, 1, II, one candidate next state could be [0, 0, 11, another is [0, 1, 0}, and so on. For N objects, there are N such candidates at each step.
Concretely, have the space of hypotheses f E F which is all possible items being the Blicket not Blicket, and the two rules (disj conj). There are 2 hypotheses total for N objects. Algorithmically we represent them as boolean masks (for which objects are Blicket) and numpy operations. E.g. conjunctive rule where the last two objects are Blicket is:
def func (x)
mask np. array ([O, 1, 11, dtype=bool) return np. all (x [mask])
For each candidate x, we compute:
G(x) H(F) - Ep(yl)) [H(F[x,y)
(3)
Below we show all components of G(x) are computable quantities. First, H(F) is the prior entropy:
H(F) - P(f
fEr
where p(f) is uniformly distributed over all f's consistent with all observations so far. The second term is,
Br(k [H(F)x,y)] = [p(y))) Ep(fly,x) log
log p(fly
(5)
Note the summation over y amounts iterating over all values of y, which in our case is binary E {0, }} (machine off or on).
Finally, we can write using Bayes,
p(fly,x) p(ylfx))((f) P(y(f,))))(()
(6)
where p(f) is from the prior defined above. p(ylf,x) 1f f(x) y, and 0 otherwise.
B.2 Language Model Access
We select mix of models to span several key axes-reas reasonngg capability, capacity, openness, and architectural family-to explore how each factor may influence hypothesis elimination and exploration behaviour. The GPT family represents some of the highest-performe state-of-the-art LMs available. Within this family, we include both gpt-4o and 4o- 40---iii... enabling comparison of model size effects within similar architectures. The DeepSeek
15

---

Published as a conference paper at COLM 2025
family offer complementary point of comparison: it includes state-of-the-art LMs in their own right, allowing us to compare reasoning model vs. non reasoning model, along with full access to intermediate reasoning content. We also study few small-to-medium sized models, including QwQ (medium sized reasoning model from Qwen), and the gemma3 series of models (< 30B).
We accessed the GPT models and DeepSeek models using the OpenAl and DeepSeek APIs, respectively. Specifically, we gpt-40-2024-08-00 and As DeepSeek does not provide reproducible access specifications that we are aware of, we report we access the DeepSeek-V3 (deepseek-chall and DeepSeek-RI (deepseek-reasones models over the course of March 23 to March 28, 2025. Otherwise, we use Ollama (Ollama, 2025).
B.3 Statistical Validation
We evaluate all LMs in the "Blicket Test" text environment detailed in Section 2.1. For experiments in Section 3.1-3.3 and Section 5, we run the agent in the game with either objects, where of them are Blickets (randomly assigned at the start of each episode). All objects start off on the floor or the machine with 0.9 or 0.1 independent probability. The agent is allowed up to 32 actions in the environment before the game transitions to the Q&A phase where the agent is asked binary questions about the identity of the Blickets.
For the main results (Figure 2), we use minimum of 16 independent trials per combination of model prompt variant system prompt. Specifically for Figure 2a, we plot the best system message and prompt for each model, averaging over random seeds to compute the 95 standard error of mean.
B.4 System Message
We carefully design three different system message variants based on how information presented human participants historically in Blicket Test experiments (Lucas al., 2014a). The first is human default, which is based on a common-sense explanation of the task. The second is human conjunctive, which makes use of the description "Blickets have blicketness". As Lucas et al. (2014a) notes, this induces more conjunctive semantics. Finally, the variant math definitin tion provides rigorous math definition describe Blickets which maximally avoid ambiguity.
The general system message prompt template follows the following format:
[## INITIAL MESSAGE ##]
Your goals are:
Identify which objects blickets.
Infer the underlying rule for how the machine turns on.
Here are the available commands:
look:
describe the current
put ....
put an object on the machine the floor take off ....: exit:
take an object off the machine
exit the game
[## TIPS ##1
You have #HORIZON# steps to complete the task. You can also exit the task early if you think you understand the relationship between the objects and the machine. After the task is done, you will be asked which objects are blickets, and the rule for turning on the machine.
You will be prompted at each turn to choose actions.
16

---

Published as a conference paper at COLM 2025
B.4.1 Human Default Variant
Initial Message:
You are an intelligent, curious agent. You are playing a game where you are in room with different objects, and machine. Some of these objects are blickets. You can't tell which object blicket just by looking at it. Blickets make the machine turn on, following some hidden rule.
Tips:
Tips:
All objects can be either on the machine or on the floor.
You should think about how to efficiently explore the relationship between the objects and the machine.
B.4.2 Human Conjunctive Variant
Initial Message:
You are an intelligent, curious agent. You are playing game where you are in a room with different objects, and a machine. Some of these objects are blickets. You can't tell which object is a blicket just by looking at it, but they have blicketness inside of them. Blicketness makes the machine turn on, following
hidden rule.
Tips:
Tips:
All objects be either the machine the floor. should think about efficiently explore relationship between the objects and the machine.
B.4.3 Math Definition Variant
Initial Message:
You are an intelligent, curious agent. You are playing a game where you are in room with different objects, and machine. Some of these objects are blickets. You can's tell which object a blicket just by looking it, but they have blicketness inside them. Blicketness makes the machine turn on, following some hidden rule.
Tips:
Tips:
All objects can be either on the machine or on the floor
You should think about how to efficiently explore the relationship
between the objects and the machine.
blicket is defined as object whose state not independent the state of the machine (in other words, the object 's state distribution and the machine ss state distribution have nonzero mutual information)
B.5 Prompting Techniques
For non-reasoning models, we also use a number of standard prompting techniques, includ- ing ReAct (Yao et al., 2022), Reflexion (Shinn et al., 2023), and Chain-of-Thought (CoTT) (Wei et al., 2022). As the point of the Blicket Test is to discover knowledge, we do not provide with shot examples, but ask the LMs to do exploration zero-shot, based on generic
17

---

only one Published as a conference paper at COLM 2025
The prompting technique is applied both during exploration (where agent is prompted to act), and Q&A (where the agent answers True False). Details below.
prompts.
Default Prompt
Directly output the command in the format command command is included.
Ensure
ReAct Prompt
First briefly reason and think about your plan to solve the task. Then, output the command in the format '> command Ensure only one command included.
Reflexion Prompt
Please first reflect your plan solve the task. Explain your reasoning and self-evaluate any potential issues before outputting the final command in the format command". Ensure only one command is included.
Chain-of-Thought Prompt
Please provide detailed
outlining all intermediate steps leading to your decision, and then output the final command in the format '> command '. Ensure only one command is included.
B.6 Example Trajectory
Here we provide an example full trajectory in an environment with 3 object, with conjunctive (AND) rule. This is the full prompt given to the agent during the Q&A phase, which contains the entire exploration interaction history as context. In this environment, objects 1 and are Blickets.
You are in room. You
machine the center of this room.
There are also objects scattered around the room. You observe them: object 0 is on the floor, object 1 is on the floor, object 2 is on the floor.
The machine hums softly in front you, seemingly waiting. The light on the machine is currently off. You wonder if there is a relationship between the objects and the machine.
put object on machine
You put object on the machine. The light on the machine is currently off
put object 1 on machine the machine. The light the machine is currently
You put object off.
put object 2 on machine
You put object on the machine. The light on the machine is now on.
take object 0 off machine You took object off the machine. The light the machine is now on.
take object off machine
You took object I off of the machine. The light on the machine is currently off.
put object 1 machine
You put object on the machine. The light on the machine is now on.
take object 2 off machine
You took object 2 off of the machine. The light the machine is
currently off.
take object off machine
18

---

Published as a conference paper at COLM 2025
You took object 1 off of the machine. The light the machine is currently off.
> put object 2 on machine
You put object on the machine. The light on the machine is currently off.
put object You put object on the machine. The light on the machine is now on.
machine
exit Exiting the episode.
Based the information you have gathered, answer the following question Is object a blicket?
Directly output the answer in the format '> True/False Ensure only one answer is included.
B.7 Example intermediate tokens during Q&A
We provide qualitative examples of intermediate tokens below. While intermediate tokens have been used as proxy for "reasoning" in the literature, we do not wish to draw strong conclusions here. Recent evidence suggests their role in the LM's computation is illusive, for instance, they may not necessarily have to be correct for the final answer to be correct (Stechly et al., 2025).
B.7.1 GPT-4o, react prompting (correct answer)
Q: Is object 1 blicket? (true blicket indices: [1,
From the observations, the machine turns on when object and object are on the machine together. When object 1 is on the machine alone, the light is off. When object 4 is the machine alone, the light is off. However, when both objects 1 and rule for the machine to turn on requires both objects 1 and 4 to be present, indicating that both are blickets.
the machine, the
light turns on. This suggests that
Therefore, object is blicket.
True
B.7.2 GPT-4o, react prompting (incorrect answer)
Q: Is object 5 blicket? (true blicket indices: [2,
From the observations. we can see that:
When object 5 is on the machine alone, the light is off. When object 5 is the machine with object 2, the light is When object is on the machine alone, light is on.
This suggests that object is blicket because it can turn the machine by itself. Object 5 does not turn the machine by itself. indicating it is not blicket. The machine turns when object is present, regardless of whether object is also present.
Therefore, object 5 is not blicket.
False
19

---

Published as a conference paper at COLM 2025
B.7.3 DeepSeek-RI (correct answer)
Q: Is object a blicket? (true blickets: [2, 3] conjunctive)
Okay, let's try to figure out if object 2 is blicket. So, the machine turns on when certain objects (blickets) steps taken, let's recap what happened.
placed on it. From the
First, when objects 5 and were placed on the machine, the light stayed off. Then adding 1, 2, and caused the light to turn When objects and 1 were removed, the light stayed on. But when object 3 was removed, the light turned off. Then removing 2 also left it off.
[omitted "1 characters]
Then remaving turned it off. Then putting back alone didn help, but adding 2 again made it work. So yes, 2 and 3 together activate the machine. Therefore, object 2 is blicket because when combined with the machine turns So the answer should be True
B.7.4 DeepSeek-RI (incorrect answer)
Q: Is object 5 blicket? (true blickets: [5, conjunctive)
Okay, let's try to figure out if object 5 is blicket. From the actions taken in the game, here's what happened:
Initially, all objects except object 0 were on the floor, and the machine was off. The user started by putting objects 1 through 7 one by one onto the machine. Each time, the machine's 's light stayed off until object 7 was added. When all objects (1-7) were on the machine, the light turned on. Then, removing object caused the light to turn off and putting it back on turned it on again.
This suggests that the machine's rule requires the presence object 7 to activate. Since adding all other objects (1-6) didn't trigger the light until was added, it's likely that object 7 is a blicket. However, the other objects (including 5) were the machine both on (after
when the light was added). Since the presence activate the machine, and removing deactivates it regardless of 5 presence, it seems that 5 isn't blicket. The rule might be that the machine only needs at least one blicket, which is 7 here. Therefore, object 5 not blicket.
off (before
added) and when it alone (without 7) didn's
B.8 Human inference experiments
To study whether LMs biased by prior assumptions during inference, conducted a direct textual replication of the inference experiment used in Lucas et al. (2014b) and Gopnik al. (2017), designed probe causal reasoning under uncertainty. First, the "training" phase, the participants (in our case, LMs) are presented with trials that provide evidence suggesting one of three scenarios: (1) A disjunctive causal rule governs the machine; (2) conjunctive causal rule governs the machine; (3) the evidence is ambiguous and insufficient to definitively determine which causal structure governs the machine (the "baseline" condition). Below is the full trajectory of one training trial for each of the scenarios: (1) disjunctive evidence:
You are in room. You see machine the center of this room.
There are also objects scattered around the room. You observe them: object 0 is the floor, object 1 is the floor, object 2 is the floor. The machine hums softly in front of you, seemingly waiting
20

---

Published as a conference paper at COLM 2025
The light the machine is currently off. You wonder if there is relationship between the objects and the machine.
> put object 0 on machine
You put object on the machine. The light on the machine is now on.
> take object e off machine
You took object 0 off the machine. The light the machine is currently off.
put object 1 on machine
You put object on the machine. The light on the machine is currently off.
take object off machine
You took object off the machine. The light on the machine currently off.
> put object 2 on machine
You put object
the machine. The light the machine is on. take object off machine
You took object 2 off the machine. The light on the machine is currently off.
put object 0 machine You put object put object on machine
the machine. The light on the machine is now on.
You put object on the machine. The light on the machine is now on. > take object off machine
You took object 1 off the machine. The light the machine is put object 2 on machine
You put object on the machine. The light on the machine is now on.
> take object 0 off machine
You took object 0 off the machine. The light the machine is
machine You put object on the machine. The light on the machine is now on.
put object
(2) conjunctive evidence:
You are in room. You see machine the center this room.
There are also objects scattered around the room. You observe them:
object 0 is the floor, object 1 is the floor, object 2 is the floor. machine hums softly front of you, seemingly waiting The light on the machine currently off. You wonder if there a relationship between the objects and the machine.
put object 0 machine
You put object on the machine. The light on the machine is currently
off.
> take object o off machine
You took object 0 off the machine. The light the machine is currently
off.
put object on machine
You put object on the machine. The light on the machine is currently
off.
take object off machine
You took object off the machine. The light on the machine currently off.
> put object 2 on machine the machine. The light on the machine is currently
You put object
off.
>
take object off machine You took object 2 off the machine. The light on the machine is currently
off.
put object a machine
You put object on the machine. The light on the machine is currently off.
> put object 1 on machine
You put object on the machine. The light on the machine is currently off
21

---

Published as a conference paper at COLM 2025
take object off machine You took object off the machine. The light on the machine currently off.
> put object 2 on machine
You put object on the machine. The light on the machine is now on.
take object off machine You took object off the machine. The light on the machine currently off.
> put object 1 on machine
You put object off.
the machine. The light the machine is currently
(3) ambiguous evidence:
You are in room. You see a machine at the center of this room.
There are also objects scattered around the room. You observe them: object 0 is the floor, object 1 is the floor, object is the floor. machine hums softly front of you, seemingly waiting The light on the machine is currently off. You wonder if there is a relationship between the objects and the machine.
put object 0 machine
You put object on the machine. The light on the machine is currently
off.
>
take object o off machine
You took object 0 off the machine. The light the machine is currently off.
> put object 0 on machine
You put object on the machine. The light on the machine is currently
off.
take object off machine
You took object 0 off the machine. The light on the machine currently off.
> put object 1 on machine the machine. The light the machine is currently
You put object off.
>
take object off machine You took object 1 off the machine. The light on the machine is currently off.
machine You put object on the machine. The light on the machine is currently
put object
off.
>
take object off machine You took object 1 off the machine. The light the machine is currently
off.
> put object on machine
You put object on the machine. The light on the machine is currently
off.
take object off machine
You took object 1 off the machine. The light on the machine currently
off.
> put object 0 on machine the machine. The light on the machine is currently
You put object off.
put object 2 on machine
You put object on the machine. The light on the machine is now on.
Next, in the "test phase", the participants (or LMs) are presented with test trial with new objects and the same machine governed by the causal rule introduced in the training trials. Below is the full trajectory for the test trial:
You are in
room. You see the same machine the one previously the center of this room.
22

---

Published as a conference paper at COLM 2025
You now have 3 different objects scattered around the room. You observe them: object is on the floor, object is on the floor, object is on the floor. The machine hums softly in front of you, seemingly waiting. The light the machine is currently off. You wonder if there is relationship between the objects and the machine.
> put object A on machine You put object A on the machine. The light on the machine is currently off.
take object off machine
You took object A off the machine. The light on the machine currently off.
>
put object A on machine
You put object
the machine. The light the machine is currently
off.
> take object off machine
You took object A off the machine. The light on the machine is currently off.
put object A machine
You put object on the machine. The light on the machine is currently
off.
>
take object A off machine You took object A off the machine. The light the machine is currently off.
> put object в on machine
You put object B on the machine. The light on the machine is currently off.
take object off machine
You took object off the machine. The light on the machine currently off.
> put object A on machine the machine. The light the machine is currently
You put object off.
put object с on machine You put object on the machine. The light on the machine is now on. put object B machine You put object
the machine. The light on the machine is now on. take object off machine
You took object B off the machine. The light on the machine is now on.
Based the information above, is object blicket?
Directly output the answer in the format '> True/False', Ensure only one answer is included.
The test trial is intentionally designed such that the ground truth be one of the two possibilities: either both A and are Blickets, and the machine follows a conjunctive rule, or only C is Blicket, and the machine follows disjunctive rule. Participants (or LMs) are then asked whether each test object (A, B, C) is a Blicket. In particular, the answer to the identity of the test object A as a potential Blicket (causal agent) depends on the what the participant (or LM) believes the underlying rule of the causal system is. As such, we quantify the model's propensity to classify as Blicket, which serves as a proxy for identifying cognitive bias. The experimental setup mirrors the original psychological study, translated into a text-based interaction format that allows LMs to reason about causal relationships.
C
Additional Results
C.1 Causal Exploration Progress
For more comprehensive interpretations of each model's exploration efficiencies, we plot ad- ditional visualizations of the results in Figure Figure 10a plots the hypotheses elimination
23

---

Published as a conference paper at COLM 2025
disuuctive. Objects
Rule: conjundive, Objects:
0.2-
Ride: disuanttin, Objecti:
Rule: conjunction. Chjects:
disjunttve.
Rull:
0.8
0.2
0.2
15
20
25
30
10
15
20
25
30
237-43
rondon.
(b) Hypothesis elimination progress normalized by the baseline random agent's 's mean progress. Normalized progress can be negative, see Equa- tion 8.
(a) Hypothesis elimination progress, measured between (no hypotheses eliminated) and 1 (all but one eliminated). Higher is better.
Figure 10: Raw and normalized hypothesis elimination progress. Error bars denote standard error.
as a progress between 0 and 1. Denote the total number of hypotheses as N, and number of hypotheses model has eliminated at time nm (f). The hypotheses elimination progress is measured as,
N 7m (1)
Pm (f)
N -
which goes when all but one hypotheses are eliminated.
We also consider the possibility that due to bounded computational resources, one task may be harder. We account for this potential difference in search difficulties by normalizing the performance relative to a baseline that takes random actions in each environment (Figure 10b). This normalized progress measured by,
Pm (t) Prandom (t)
pm
(8)
Prandom (f)
which shows the progress of a model relative to progress made by the random action baseline. Intuitively, the normalized progress measures where the model progress pm pm(() (t) is between the random baseline's progress Prandom (t) and Note that this progress can be negative if the progress model makes is slower than that of a baseline. We further observe that for the best performing models, LM agents still show a disjunctive bias in exploration efficiencies when normalized by the random action baseline. For completeness, we also report the end-of-episode progress of the performant models in Table 1.
C.2 Effect of reasoning effort on exploration
To investigate the effect of "reasoning" on the models' abilities to explore, we run additional evaluation using the GPT-o-mini series of reasoning models the 8-objects environments. We report the results in Figure 12. In the conjunctive setting (left panels), models with higher reasoning effort consistently lag behind both lower-reasoning variants and the inio oracle. We hypothesize this could be due to: (1) the reasoning models evaluated here were overfitted the verifiable tasks they were trained on; (2) reasoning amplifies existing priors in these models that favours disjunctive interpretations. Future works are needed to address these open questions.
24

---

Published as a conference paper at COLM 2025
# Objs
Model
Disj.
Conj.
Norm. Disj.
Norm. Conj.
baseline random
0.968 0.049
0.964 0.068
0
0
infoGain oracle
1.000 = 0.000
1.000
1.000 = 0.000
1.000 # 0.000
gpt-4o
1.000 0.000
0,991 0.033
1.000 # 0.000
0.756 + 0.900
8pi-4o-mini
0.982 0.073
0.911 0.113
0.439 + 2.297
-1.444 + 3.109
deepseek-chat
0.995 0.011
0.975 0.046
0.857 + 0.356
0.319 a 1.255
deepseek-reasoner
0.934 0.224
0.895 0.236
-1.085 7.063
-1.881 6.509
baseline random
0.944 0.098
0.877 = 0.183
0 1.000 0.000
0 1.000 0.000
infoGain oracle
1.000 0.000
1.000 0.000
gpt-4o
0.999 0.005
0.924 0.125
0.983 # 0.085
0.383 + 1.019
8
8pt-4o-mini
0.812 + 0.357
0.798 0.307
-2.342 6.346
-0.647 2.499
deepseek-chat
0,976 0.092
0.935 0.138
0.581 # 1.634
0.472 = 1.121
deepscek-reasoner
0.956 0.174
0.875 0.215
0.226 + 3.091
-0.014 1.752
Table 1: Absolute and normalized exploration progress (+ stdev) across models and object counts, at the end of each of episode. Disj and Conj denote absolute progress, while Norm Disj and Norm Conj denote random-normalized progress. Higher is better.
Number of Objects:
Number Objects: 8
1.0
0.8
0.6
X
0.4
X
x
xx
3000C
0.0
0.2
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Disjunctive Accuracy
Disjunctive Accuracy
infoGain oracle baseline random
deepseek deepseek-reasoner gemma3:277
cot
x default
gpt-4o gpt-fo-mini
reflexion
Figure 11: All models' disjunctive and conjunctive accuracies. Colours indicate model type, while point shapes indicate the prompt. This is an extension of the result in Figure 2b.
C.3 Inference on causal relationships
We report the trajectory statistics of the exploration data used in Section 3.4 in Table 2.
Data Source
Rule Type
Traj Length (+ Stdev)
LLM (GPT-4o)
Conjunctive
18.94 + 5.04
LLM (GPT-4o)
Disjunctive
18.06 2.35
Oracle
Conjunctive
12.70 + 4.79
Oracle
Disjunctive
14.58 4.28
Object Count
Conjunctive
32.00 + 0.00
Object Count
Disjunctive
32.00 0.00
Random
Conjunctive
32.00 + 0.00
Random
Disjunctive
32.00 + 0.00
Table 2: Trajectory lengths (mean standard deviation) across exploration data sources and rule types. Used in Figure 5.
25

---

Published as a conference paper at COLM 2025
corjunctive
disjunctive
Rule: conjunctive
disjunctive
Aypoothcss Remainno
Myyothiccss Romoinin
10+
20x
1.0
10
20
30
20
30
20
30
30
30
Steps
Steas
Steas
Steps
infoGain oracle
c3-mini (lew reasoning)
infeGain oracle
04-mini (low reasoning)
63-mini Inigh reasoning)
63-mini (modium reasoning)
thigh reasening)
64-mini (medium reasaning)
(a) Reasoning model
(b) Reasoning model o4-mini-20223 2825-84-16
Figure 12: Hypotheses elimination efficiency for the GPTT--mini series of reasoning models at varying reasoning efforts. Lower better. Error bar denotes standard error of mean.
C.4 Human exploration characteristics
300
150
10
50
Conjunctive
Disjunctive
Conjunctive
Digunctive
Figure 13: Additional exploration statistics in children, reproduced from Table 1 of Kosoy et al. (2022b) aggregating "given hypothesis" and "not given hypothesis" conditions. Left: Number of times children checked if state of the Blicket-detectnng machine. Right: Average time children played before seeing the Blicket detector go on for the first time. Error bar denote standard deviation. Two-sample t-test significance level denote at the top of each panel.
Kosoy et al. (2022a) conducted Blicket Test experiment with 4-year-old children and allowed them to freely explore different combinations of given objects to "figure out how to make the machine go". Here reproduce their results from Table 1 graphically and statistical tests to have a better understanding of how well LMs explore in comparison to human children 4.2. In summary, across different experimental conditions (whether the causal rule of the Blicket-detecting machine can be inferred from example trials, i.e. "given hypothesis", vs. when the causal rule cannot be inferred, i.e. "not given hypothesis") and measurements (number of unique combinations attempted, number of "checks" which we use as the equivalent for the number actions taken before exiting, time spent exploring), children explore similarly for conjunctive machine and disjunctive machine (Figures 13, 14).
26

---

Published as a conference paper at COLM 2025
Actions
Combinations
p.-c0.5, p-.00-00000000 -.0.0 - 06), dggsffoamt
p--0.05. 3345.L "xp p dggiiaatt
25
85
15
Net Gienn
Givon
Not Ghean
Civan
Time (6)
Time to Success [s]
BOL
ming Syrikatt
20-
100
50
Glven
Gieen
Figure 14: Additional exploration statistics in children, reproduced from Table 1 of Kosoy et al. (2022b) without aggregating. Top left: Average number of times children pressed the "check" button per trial. Top right: Average number of unique combinations attempted per trial. Bottom left: Average total time spent exploring the Blicket-detecting machine. Bottom right: Average time before successfully activating the machine for the first time in trial. Error bars denote standard deviation. Two-sample t-test significance level denote at the top of each panel.
C.5 Hypothesis Sampling Agent
C.5.1 Details of Procedure
The sampling agent approximates the InfoGain agent to reduce the entropy of the hypothesis distribution. Instead of iterating over candidate next observations and computing G(x), we simply prompt the LM to take actions that minimizes HFF). The procedure outlined below.
1. Hypotheses sampling to form set of active hypotheses If there are zero hypotheses: we prompt the agent to generate hypotheses as python functions.
You have seen the following observations far: [HISTORICAL 0BSERVATIONS]
Come up with some hypothesis about how the world works. Write each one as python function. The function should:
Take input numpy boolean vector of length #NUM_OBJECTS#, where a means object is floor, and
object is machine
27

---

Published as a conference paper at COLM 2025
Output boolean, where machine light is on First construct boolean mask indicating which object are blickets, extract just the state of these objects, then define some function over these states to produce the state of the machine.
machine light is off, and
Only consider hypothesis where
all objects that blickets needs to be put on machine.
Do not consider the "sum" operation. Only consider boolean operations.
Produce #NUM_HYPOTHESES# hypothesis, do generate the same hypothesis. Be diverse both the blicket masks and the potential functions.
Do not generate the hypothesis you have already eliminated: [ELIMINATED HYPOTHESES]
Do not generate hypothesis you have already generated: [ACTIVE HYPOTHESES]
Directly output the answer, ensure only answer is included. Return of hypotheses python functions.
We all valid (python executable) hypotheses string python code in the list of currently active hypotheses.
Eliminate hypotheses based observation history For each python function f (i.e. hypothesis) in the currently active list, we check it perfectly predicts all past observations: ie. f(x)) = Yt for all { (xi, y=)) It does not, we move the "eliminated hypotheses" list.
Prompt agent take actions that eliminate more hypotheses We prompt the agent to take actions conditioned on the observations and the list of active hypotheses:
You are currently entertaining the following list of hypothesis: [ACTIVE HYPOTHESES]
You have seen the following observations so far: [OBSERVATIONS so FAR]
Given observations so far, and the list hypotheses (hypothesis space), take an action which will disprove the existing hypothesis.
Directly output the answer, ensure only answer is included. Output the action in the format \'> action\ ' Ensure only action is included.
The agent then takes the next actions and we repeat the above procedure for exploration.
Q&A Phase During the Q&A phase, we prompt the agent to answer questions condition- ing on the remaining set of hypotheses:
You have seen the following observations so far: [HISTORICAL OBSERVATIONS]
You have disproven the following hypothesis: [ELIMINATED HYPOTHESES]
You have not yet disproven the following hypothesis:
[ACTIVE HYPOTHESES]
Based on the information above, answer the following question: [QUESTION]
Output the answer in the format \t> True/FalseN', Ensure only answer included.
28

---

Published as a conference paper at COLM 2025
In this way, the sampling agent explicitly represents its beliefs pFF) set of active hypotheses, which the LM conditions on take the most informative action.
C.5.2 Proof of Remark
Remark Let p(F) be a discrete prior distribution, and we sample without replacement from PFF) to construct empirical distribution (F)). Let (F()) be the empirical distribution after t unique samples, the entropy of monotonically increases with each additional unique sample, H(q+) < HQq=-1).
Proof. Given the already sampled set S; = ff..2...... the empirical distribution is defined as:
} f St,
q:(f) (0 otherwise.
(9)
The entropy is H(q:) -Eg, log 91] = log( log t. log is monotonically increasing function in
C.5.3
Exploration for Both Rules
See Figure 15.
Conjunctive Rule
Disjunctive Rule
Num Hypotheses Remaining
10?
Steps
Steps
hypothesis sampl
hypothesis 96)
default
(n 32)
hypothesis sampl (n = 16)
hypothesis sampl
InfoGain oracle
(n 64)
Hypothesis elimination performance of the hypothesis sampling agent for 8 objects for both disjunctive ("OR"") and conjunctive ("AND") rules.
Figure 15:
C.5.4 Hypothesis sampling agent resolves biased exploration patterns
Additionally, we evaluated whether our method of hypothesis sampling addressed the dif- ferences in exploration patterns between conjunctive and disjunctive causal structures that we observed in 4.2, wherein LMs explore more extensively in the conjunctive condition than disjunctive. We observed that sampling hypothesis significantly reduced the difference in exploration patterns between conjunctive and disjunctive conditions (Figure 16), suggesting that our method helps LM explore more systematically, much like curious child.
29

---

Published as a conference paper at COLM 2025
Combinations
20
T15
n5
.15
20
15
10
10
Conjunctive Disjunctive
Conjunctive
hypotheses generated
Steps per trial
115
10
Conjunctive Disjunctive
Conjunctive
Disjunctive
Naive
hypotheses generated
Figure 16: Amount of exploration by gpt-4o in the 8-object Blicket environment with default system prompts, measured by the number of unique combinations of objects attempted per trial (top) and number of steps taken before exiting the trial, for agents before (left) and after (right) inference-time hypothesis sampling, averaged across trials, error bar denote standard deviation. Two-sample t-test: *p- < 0.05, p < 0.01, *** p 0,001, ns: significant.
30


Reviews:
[{"review_id": "sbHUxeGHqh", "reviewer": "Reviewer_Gqdc", "summary": "As language models are being used for autonomous decision making, it is relevant to understand the biases exhibited by them. In this paper, the authors use the blocker test to understand how LMs infer causal relationships. Their findings show that models exhibit disjunctive biases similar to humans. They also propose a test time approach to de-bias the skewed prior learnt by the model to eliminate any prior hypotheses, showing that this can help reduce disjunctive bias.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This is a relevant line of work and the direction of comparing LM biases with human psychology is interesting.\n2. The mitigation approach of flattening the skewed prior seems like a promising direction in terms of results.", "reasons_to_reject": "1. The paper lacks information and is very difficult to read. In addition, it includes typos and grammatical mistakes in multiple sections(Lines 161 and Lines 171 are a few examples).\n2. While the paper mentions disjunctive and conjunctive biases in humans, this needs to be clearly articulated with examples so that one can follow better.\n3. The evaluations performed by authors are sparse in detailing the evaluation datasets, their diversity or any qualitative analyses from their evaluations.", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CTh13iedZ1", "reviewer": "Reviewer_badZ", "summary": "The paper addresses the question of causal reasoning bias in LLMs, using the Blinket test from studies on development psychology.  In a study of various open- and closed-source models, the authors find that LLMs exhibit a disjunctive bias, similarly to human adults. They then go on to explore the factors contributing to success, with the finding that information gain is a strong predictor, they find that LLMs do not explore efficiently, often revising hypotheses that should have been eliminated, but they do spend less time exploring disjunctive hypotheses. Among the included LLMs, deepseek stands as least susceptible to the disjunctive bias.", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper is very well written.\n\nThe work explores an interesting and relevant question and does so with what seems like a solid experimental design. \n\nResults are discussed (for the most part) in sufficient depth, with meaningful findings being drawn.", "reasons_to_reject": "The proposed test-time sampling method used to eliminate the casual reasoning bias is highly tailored to the Blinket test in particular and cannot be used in more general settings. As such, it is not really practical and I am not clear even on its theoretical contribution, since it seems quite obvious that such a controlled intervention would reduce the bias. \n\nThere are minor but must be addressed:\n- Details are missing on the implementation of the Oracle InfoGain agent. \n- The LLMs used in the study are never propertly introduced and their choice motivated. The authors point to Appendix B.1 but that section only contains details on the access to API-based LLMs. \n- What is qwq which appears in Figure 2 but is seemingly never introduced?\n- Some context is needed in Appendix B.2  to understand the motivation behind the different prompt variants. What does Human Conjunctive mean? Why does it make sense to include the Math Definition variant? What is the variance in accuracy between these different settings?\n- More questions below.\n\nIt would be good to see a discussion of how the human bias may have been injected into the models, considering that this type of reasoning might rarely be codified in writing. What sort of data might the LLMs have picked this bias up from? Or could there be a third hidden variable that explains both the adult human bias and the LLM bias?", "questions_to_authors": "Questions:\n- It is not clear whether the results in Figure 2 are averaged over all system messaged and prompting methods.\n- It is not clear how many \"game instances\" models are evaluated on. \n- Does the agent have any incentive to exit early? Is the number of steps needed to solve the task important in the real Blicket Test? \n- What is the point in the random agent taking any action if it will randomly answer the question at the end?\n- If I understand correctly, the Oracle InfoGain agent is not LLM-based. Can you explain how it was implemented?\n- Does the model know how many objects are blickets or does it have to discover that too?\n- What is meant by \"environment rule\" in the caption of Figure 3? Dis v Con or something else?\n- In 3.4 how long of an exploration path is presented to the LLMs? How many steps are left to the solution? \n- 211: what are these system prompts?\n\nSuggestions:\n- It might be better to visualise the conjuctive-disjunctive pair for each model side by side, to make it easier for the reader to see the consistently higher performance in the disjunctive case. This is more important than seeing how the different models compare on a per-setting basis, at least as reflected in the text.\n- It is worth pointing out in 4.1 that deepseek is a clear outlier, correctly identifying the conjunctive rule, and also showing a conjunctive inclination in the baseline setting (together with llama, interestingly)\n\n\nTypos:\n- blicket should be capitased in the abstract and IMO does not need quotation marks\n- l74: perhaps Section should be replaced with Appendix here and later on page 7\n- l161 an->a\n- l162 performs->performing\n- l200 Proportion -> proportion", "extra_scores": {}}, {"review_id": "cpnxKCoAga", "reviewer": "Reviewer_o9YV", "summary": "The authors adapt an experimental paradigm (the “Blicket test”) to investigate how large language models make causal inferences. In this text-based version, an “agent” can “explore” various actions with a machine, which follows a logical rule (disjunction or conjunction); the agent must then generalize the underlying rule. The authors find that all the LLMs tested struggle when the number of objects under investigation increases, and they also struggle more with conjunctive rules than disjunctive rules. In general, the models seem to prefer disjunctive answers (like human adults) and spend less time exploring disjunctive settings (unlike human children). The authors then demonstrate that they can artificially “flatten” the model’s distribution over hypothesis space in such a way that it behaves more optimally.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper addresses a topic of high-level importance (causal reasoning and inference in LM “agents”), and adapts an interesting psycholinguistic paradigm (the “blicket test”) that has been extensively tested and used for human children and adults. The results themselves point to potentially promising directions for future work.", "reasons_to_reject": "The underlying premise is interesting, but I think there are a few potential weaknesses.\n\nFirst, the paper does not include (as far as I can tell) a human baseline for the results. The authors do include an “oracle” model as a comparison, which is helpful, but it would also be useful to include an analysis of human data. Again, the paper makes reference to results obtained with humans, but as far as I know, we do not have results for this text-based task, which limits our ability to draw strong generalizations or comparison to human behavior.\n\nA deeper question concerns the utility of this task in particular. It has proven useful in characterizing the causal inference behaviors of human children and adults. But what is its construct validity for assessing models?\n\nFinally, I found some of the methodological details a little hard to understand, especially some of the follow-up analyses (e.g., section 5); I understand that space is limited but I think more detail here would be helpful.", "questions_to_authors": "Is response length a good proxy for “amount of reasoning”?\n\nHave the authors considered alternative prompting strategies to encourage more exploration? E.g., there is some work on how certain prompting environments encourage or discourage explore behavior (e.g., Coda-Forno et al., 2023).", "extra_scores": {}}, {"review_id": "Riz59YHp86", "reviewer": "Reviewer_w25T", "summary": "This paper presents a clear and compelling study investigating whether LLMs exhibit human-like biases in causal reasoning, specifically a disjunctive bias, using a text-based adaptation of the classic Blicket Test. It also proposes a method to encourage more scientific reasoning in LLMs through hypothesis sampling. The paper is well written, with clear motivations and well-designed experiments. Overall, I believe it offers valuable insights for the community.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This study is interesting and timely\n\n1. It is well written, and the experiments are well presented\n\n1. The authors provide strong analysis and propose potential methods for improvement.\n\n1. It offers valuable insights to the community.", "reasons_to_reject": "1. I am concerned about the logic in lines 121–124. Given the task setup, it seems that the conjunctive condition is indeed harder than the disjunctive one. Identifying all required elements (as in the conjunctive rule) is naturally more challenging than finding any one contributing factor (as in the disjunctive rule). I do not see how the InfoGain Oracle establishes that the conjunctive case is not harder for LLMs. The Oracle represents an idealized agent with perfect reasoning abilities, which LLMs do not possess. Therefore, the observed performance gap could simply reflect task difficulty rather than a disjunctive bias. I am not convinced by the conclusion that LLMs inherently favor disjunctive reasoning.\n\n1. it appears that DeepSeek Reasoner, a reasoning model, performs differently from non-reasoning models. So does this indicate reasoning models are thinking more scientifically? It would be great to see if there are significant pattern difference between reasoning and non reasoning models. \n\n1. I think this might be addressed by simple instruction tuning. And it's unclear whether such tuning would enable models to generalize to OOD causal scenarios (other than the Blicket test). It would be interesting to explore this.", "questions_to_authors": "See above", "extra_scores": {}}]

Author rebuttal:

Meta-review: This is an interesting and timely paper evaluating LMs in a "blicket test" paradigm adapted from developmental psychology. The authors find that LMs tend to struggle with inferring conjunctive causal relationships, and tend to pattern like the inference behaviors made by adults (cf. 4- or 5-year-old children). The authors then propose a way to mitigate these biases through a test-time sampling approach.

The experiments are relevant both from scientific and practical perspectives, and grounded in a literature with rich developmental and adult behavioral data. I think the findings would be of interest to the broad LM research community, so I recommend acceptance.

In light of the discussion with reviewers, for the camera-ready I would recommend (1) adding more detail about the implementation of the baselines, (2) motivating the choice of LLMs evaluated, and (3) carefully fixing typos and grammatical errors, among the other suggestions raised by the reviewers. I am also sympathetic to reviewer o9YV's point about elaborating on "to what extent are we licensed in drawing the same inferences about causal reasoning in LLMs from this design as we are when we study humans", especially for intricate tasks such as the blicket test.
Decision: Accept