Doing Experiments and Revising Rules with Natural Language and Probabilistic Reasoning
Authors: Wasu Top Piriyakulkij
Cassidy Langenfeld
Tuan Anh Le
Kevin Ellis
Venue: NeurIPS 2024 poster (2024)
Keywords: induction
LLM
active learning
Primary area: neuroscience_and_cognitive_science
TL;DR: We give a model of how to infer hidden natural language rules by doing experiments using probabilistic reasoning with language models.

Abstract: We give a model of how to infer natural language rules by doing experiments. The
model integrates Large Language Models (LLMs) with Monte Carlo algorithms for
probabilistic inference, interleaving online belief updates with experiment design
under information-theoretic criteria. We conduct a human-model comparison on a
Zendo-style task, finding that a critical ingredient for modeling the human data is to
assume that humans also consider fuzzy, probabilistic rules, in addition to assuming
that humans perform approximately-Bayesian belief updates. We also compare
with recent algorithms for using LLMs to generate and revise hypotheses, finding
that our online inference method yields higher accuracy at recovering the true
underlying rule, and provides better support for designing optimal experiments.

Doing Experiments and Revising Rules with Natural Language and Probabilistic Reasoning
Wasu Top Piriyakulkij!*
Cassidy Langenfeld'
Tuan Anh Le² Google"
Kevin Ellis
Cornell University
Abstract
We give model of how to infer natural language rules by doing experiments. The model integrates Large Language Models (LLMs) with Monte Carlo algorithms for probabilistic inference, interleaving online belief updates with experiment design under information-theoreti criteria. We conduct a suman-model comparison on a Zendo-style task, finding that critical ingredient for modeling the human data is assume that humans also consider fuzzy, probabilistic rules, in addition to assuming that humans perform approximately Bayeesian belief updates. We also compare with recent algorithms for using LLMS to generate and revise hypotheses, finding that online inference method yields higher accuracy recovering the true underlying rule, and provides better support for designing optimal experiments.
Introduction
An important way that humans grow their knowledge of the world is by experimentation and other forms of active learning. This process is most clearly present in the experimental sciences, but similar processes of active inference begin in infancy through early childhood 212 5.. Within everyday adult cognition, active experimentation helps us quickly learn to use new devices and tools.
A basic framework for modeling experimentation is to alternate between conducting a good experi- ment, and updating one's s beliefs based on those experimental results (6). These beliefs concern latent hypothesis about the regularity or trend the experimenter is investigating. This leaves open at least two computational questions. First, we need to define a hypothesis space. Second, we need efficient algorithms for belief updates and experiment generation. Such algorithms should reason about probabilistic Selicfs-conside to find experiments that optimally resolve different competing hypothesis.
many hypotheses and their associated probabilitics- in order
even Here we will introduce a model that represents hypotheses in natural language- even for problems that do not intrinsically involve human language. We do this for two reasons. First, natural language can index many human concepts, and can recursively combine them, giving an expressive hypothesis space. Second, it allows using Large Language Models (LLMs) to aid the inference task of updating beliefs after each experiment, giving tractable, approximate probabilistic inference when we view the LLM as proposal distribution for Monte Carlo estimator.
We are especially interested in comparing our model to human behavior, given the long legacy of probabilistic modeling within cognitive science [7] 8). We find nuanced picture: vanilla LLMs are humanlike on our active learning tasks (and underperform humans); our full model outperforms humans; but simple change-switchinne -wwicchin from deterministic to probabilistic hypotheses allows matching humans in overall performance, and agreement with humans on more fine- -rained metrics.
From a technical perspective, our work needs to infer natural-language hypotheses in an online setting, so that it cycle between experimentation and hypothesis formation. This differs from recent
Conresponding author: up2378cornell.com edu
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

batched approaches for hypothesis formation 99 10 11 To allow online inference, we hybridize LLMs with Sequential Monte Carlo Samplers (SMC-S: [2]). In SMC-S, one tracks modest number of hypotheses that serve (approximate) samples from the posterior. Meanwhile, the LLM focuses the sampler on small set of candidate hypotheses that deems relevant, given the data. The resulting sampler facilitates active learning by choosing an experiment which optimally "splits" the candidate hypotheses. With strategies that do not use probabilistic framing, such as tracking a single best-guess hypothesis, the active learner would have little guidance on what experiment to do next.
We will focus here active inference of basic symbolic concepts expressible in natural language, as we believe these are tractable first targets of study. Concretely, we consider tasks in the spirit of the boardgame 'Zendo', challenging but accessible game where human players actively learn binary rules combining logical and spatial relations [33 [3], as well as 'Blicket test' style tasks, inspired by studies in developmental psychology [16
that investigate how children learn the causal mechanism behind the activation of machine. See Figure]
We contribute the following:
1. An algorithm for probabilistic inference of latent natural language hypotheses. This derives from SMC-S, but uses an LLM proposal distribution allow tractable inference over natural language strings, essentially using the LLM to suggest ways of revising the belief state.
2. Model--Humaw/ModelBBBsss
comparisons, finding that (I) get a better fit to human data using natural language, instead of formal languages; (2) the model can be further made more humanlike by considering fuzzy (probabilistic) rules, and (3) that online inference also yields better accuracy at the actual task relative to recent work [10
3. Empirical findings about the ability of LLMS to revise hypotheses and propose experiments. On the domains consider, find that LLMMs effective for proposing and revising hypotheses, but do not consistently outperform random guessing when proposing experiments.
2 Model
We start with standard Bayesian optimal experiment design, which gives framework for describing both experimentation and hypothesis formation [8 [9] Our model includes natural-
hypotheses h D", experiments x X, and experiment outcomes y E y. We consider equipping with real-valued parameters 0: For example, if the hypothesized rule is fuzzy (noisy), then would control the noise level. As new experiments are proposed sequentially, we index experiments and outcomes with subscripts, i.e. 2; and for the td experiment and outcome, respectively. The objective is to identify ground-trute trunt-truthhhm h", and accurately predict the outcome of future experiments.
Hypothhsis
Experimentation
Revision
The nachine nakes aound Hypothesia 1: at least one of then la yellow object Hypothesis than objects
Time
Do D
The mechine makes sound when Kypothesis least then yellox object. or cylinder Hypotheais 2: there Teast two objects
The nachine makes sound Bypothesis 1: as least one of then. in yallow object
cylirder that is not red
Figure Alternation of experimentation and hypothesis generation on simplified version of our ActiveACRE domain. Hypotheses characterizes what causes the machine to activate (make noise).

---

The joint distribution over hypothesis h, and outcomes yI.r, given experiments T1:7, p(ualzz, h,
p(h, VIT, 0 p(h)p(0)
II 151ST
(I)
where the prior phh) favors shorter or simpler hypotheses. From eq. the posterior is
phhlanr,, vvrr) p(h) / p(0) II
p(ue 22,, h, 0)d0 ISrT
(2)
where assume the above integral is tractable, because 0 is low-dimensional. Ultimately, the purpose of the hypothesis to make predictions on new experiments. Given a test experiment Pestt an ideal learner predicts outcome Yeat distributed follows:
((yhest Trests 21:7' V1.T) Ep(hl / p(0 TIT Virr (irr)(((ess) Peess 0)
(3)
The optimal experiment for identifying maximizes the following information gain [20]:
arg max aE
E pn
[DKL(p(L/Z MIT, v)ppp(ppp1....... (...))
The above computations are intractable because they involve considering the infinitely large set of all hypotheses and experiments. We next describe our LLM-guided approximation methods.
2.1 Revising Rules: Online Inference
online approximate inference algorithm which tracks small pool of hypotheses- called particles- -that evolve over time new data collected. Tracking representative high-posterior particles allows approximate inference (eq. (2) and prediction (eq. (3) by only considering the current particles. This makes the model "boundedly rational" [11 as the bound on computation (# particles) grows large, the sampler better approximates optimal inference. To the extent that our work offers cognitive model, we are claiming that humans only consider a small number of hypotheses, which evolve in ways that approximate probabilistic reasoning. This should be seen within the tradition of using approximate inference methods to give mechanistic accounts of human learning 22
We introduce a generalization of the Sequential Monte Carlo Sampler (SMC-S)
25].
Standard SMC-S tracks n particles at each time point f, written H. = (h, ym- Each particle has weight, W, (w, y giving the approximate posterior ((h E, ( ] Upon observing new data point, the particles H, are pushed through
forward kernel qt+1 (he+11h("") which randomly perturbs the particles, to obtain new particles H++1- Next, the particles are reweighed to obtain W Finally, a resampling step can be executed prune low-weight particles and multiply high-weight particles.
experiment:
Xteir Ye+1
Faisiried
new experiment
Revi sed hypothesea
He
4t+1
weigh
resample H H
LLM proposals
Figure 2: Sequential Monte Carlo method tracks a small number of hypotheses (called particles), each of which is natural language rule, represented above by circles. After each experiment, the particles are revised in light of the new data by pushing the particles through the forward kernel. Then, the new particles are reweighed according to how well each explains the data we have seen so far. Resampling prunes low-probability hypotheses while multiplying
ones.

---

Here h is a natural language string, suggesting LLM should define de (l For example, an LLM can be prompted with hypothesis, together with the latest experiment outcome, and asked to revise that hypothesis. But calling an LLM to perturb every single particle is expensive, and unnecessary for hypotheses that already explain the data well.
We therefore design a variant of SMC-S whose forward kernel looks globally at the current set of particles and prompts an LLM to revise the worst liwestt)ikelihhoo)) particles, while keeping unchanged the best -ikeeet-- particles. This concentrates the computation
improving bad hypotheses, instead of wasting effort altering what already works. Within the context of LLMs, this can be seen as an online, probabilistic version of hypothesis refinement [9 10] Within the
context of SMC-S, this mathematically corresponds to defining forward kernel that conditions on the entire set of previous particles and all seen data points, gr 11, 21. V1:1)
Below
formalize our new SMC-S variant, which we call LLM-SMC-S, illustrated in Figure 2
Procedure: LLM-SMC-S A.44. Given He, W, where p(h|æ1a V1.1) ~ Ei W (0)
h h- |
1
1. Define unnormalized target densities 7(h) - p(h, Vie,, via) and 7hhh) = p(h, 11 F1
Sample // ~ 4++1 (-)H(-)ee, U1a+1, (1:44111144)))) ) (i.e., using LLM to revise hypotheses)
Compute the weight w' for h' following
w' A(H, HH, W.)
where A(K, HG W;)
1 72 w
Σ 1=1
(0)
(r) 199))
(5)
4 (h' H,, T1a41111 102++1,91s0 112+1
with the reverse kernel r(h(N') defined as uniform up to strings of a maximum length.
4. Repeat steps 2-3 (sampling/weighing) total of n times, and normalize the weights. Optionally, resample to generate an unweighted posterior (we always resample).
Output: He+1 and We+1, formed from n samples of N', w/ with w/ normalized from step 4, which approximate p(h| (1:41 V1++1).
The correctness of the above procedure is most easily understood using the following definition:
Definition: Proper Weighting (77) Let (h) be an unnormalized target density, which we can
evaluate. Let the corresponding normalized target density be π (h) Ze
2(h] where Z, = rh ddh is
the normalization constant. A weighted particle h, w is properly weighted with respect to y if for any function f.
Elw [()] = Z, (x
Proposition 1. If H, W input to Procedure LLM-SMC-S properly weighted with respect to 7, then the output ht, w' is properly weighted with respect to >', (Proof in Appendix A.
2.2 Doing Experiments: Active Learning
Our active learning works by doing an experiment that maximizes information gain (eq. (4) Ex- periments may be complex, such as involving putting objects or instruments in specific positions, and there might be combinatorially many possible experiments. For rich space of experiments, bounded human or cannot consider all possibilities.
We will propose experiments using an LLM, but then reassess those proposals under probabilistic criteria. Particularly, we provide an LLM with the hypotheses tracked by the SMC-S sampler at each iteration, and prompt it generate experiments that support and falsify each hypothesis. Empirically, this process yields a diverse pool of experiments. We take the best experiment proposed by the LLM, measured under the approximate posterior from SMC-S:
It+1
arg max Z xEPROMPPT(H)) H.)) #1u)2,22:)33::3))
E
[DKL(p(h)a Fa. Vias x, v))pppll Bra))
(6)
where ý approximated with the weighted particles from SMC-S
We note that whether we condition on the seen data points or not does not change the proof. The main novelty of this new variant lies in how g can he conditioned on the entire set particles H;-1
We let the particles H;, be the support for both distributions that we calculate KL divergence.

---

2.3 Instantiating the model
All of our experiments have binary outcomes (y E {0, 1}), and all of our natural language hypotheses correspond to rules that predict whether an experiment succeeds or fails (1 or 0). Although the rules predict hard all-or-none judgments, a learner can relax that constraint by assuming that the underlying rule is fuzzy (noisy). Many natural language facts and rules actually only partly hold, such as birds fly (almost always true), or birds lay eggs (true half the time). To handle the possibility of fuzzy rules, we equipped each hypothesized rule with real-valued parameters & that control the noise level. The noise parameters decompose into a pair = (e, controlling the rate of false- positives/falle-negllle
[
6 if (f) =1
ply = 1 Ir h, 0)
1-E if =
Under this formulation, hard rules corresponds to p(c) and p(6) having non-zero probability only at value 1. For probabilistic, fuzzy rules, we use Gaussian priors for p(e) and p(0), truncated [0.5,1], and with bias toward larger e. The prior p(h) is defined as inversely proportional to wordcount, giving a gentle bias toward parsimony. We investigate both hard and fuzzy rules in our experiments.
Evaluating fı(z:) requires checking the natural language string h against experiment æ. for which we use GPT-3.5 to translate the natural language h to code which is run æ. We use GPT-4 Turbo to propose hypotheses [30]. Recent studies find a similar breakdown of LLMs works well 9
3 Experimental Results
Domains. Zendo is a game where a player seeks to infer a hidden binary rule about scenes of colored shapes. Our Zendo games begin with showing the player positive example scene, followed by rounds of experimentation, where the player builds scene, and receives feedback on the scene obeys the hidden rule. After the experimentation phase, players are tested on 8 test scenes, half of which follow the hidden rule. Our setup follows Bramley et al. [13], but modified for LLMs by presenting scenes text describing each block by its color, size, orientation, groundedness, and what other blocks it touches and stacks (Figure 3
Our second domain, ActiveACRE, derives from The Abstract Causal REsonning (ACRE) dataset [17]. which in turn derives from 'blicket' tests in developmental cognitive psychology [[6]. The original ACRE is a causal induction dataset where each task is to figure out what causes the 'blicket' machine to make sounds when multiple objects are put on the machine. We add active learning to ACRE: rather than passively observe examples, our ActiveACRE allows the player to try 7 experiments, after passively witnessing the outcome of one experiment involving eight objects. The player is then tested (without further feedback) on all possible combinations of the original eight objects.
Model-Bascline comparisons. Table I contrasts the performance of different models, showing that online inference with hard rules outperforms all other models on both datasets. including a ReAcc- style baseline [31 (Direct LLM), and batched inference with refinement, an approach advocated for in recent work [10 9. To measure accuracy on Zendo, we compute the predictive posterior accuracy summed over the 8 test scenes and averaged over tasks. Because the test set on ActiveACRE
Cround truth rile: Majoniy blocks
-Block Color - redl large, Orientatioo left, Groundedneis- -grounded, Touching none
-Block x: Color green, Size medium, Orientation upright, Groundedness grounded, Touching - Block3
xt
AAA
Which of the following blocks will of?
A A As
AA
A
AAAA
A
A A
A
A
A
A
A
Tex
A
A As A AZA A
Figure 3: (a) Example Zendo scene and its serialization into text. (b) Eight experiments, each of which is a scene, with binary outcome (whether the scene makes stars come out of it). (c) Test scenes that evaluate whether model or human has correctly inferred the hidden rule.
5

---

Method
Zendo
ActiveACRE
Avg Pred Posterior Avg Pred Posterior
ROC AUC
FI
Task Solving
Human from
13
5.26
Direct LLM (31
4.60 1 0.19
0.83 0.05
0.60 1 0.02
0.86 1.044.0.4
0.00 1 0.00
Batch, Fuzzy
4.57 + 0.15
0.64 0.02
0.84 0.02
0.84
0.00 + 0.00
Online, Fuzzy (Ours)
5.35 1: 0.09
0.72 0.01
0.90 0.08
0.96
0.001
0.15 0.08
Batch, Hard
6.01 = 0.19
0.89 = 0.03
0.77
0.10 0.96
Batch w/ Refinement, Hard 90
6.18 0.14
0.73 0.04
0.91
0.15 0.08
Online. Hurd (Ours)
6.55 + 0.13
0.92 = 0.03
0.87 0.04
0.98
+
0.01 0.35 + 0.11
Table Performance Zendo and ActiveACRE, The results for Zendo are mean + standard error of predictive posterior accuracy summed over the test scenes, averaged over the tasks and 5 seeds. ActiveACRE results are mean + standard error of each metric averaged over 20 tasks. ActiveACRE is heavily class- bass-inballnned, so we compute wider variety of accuracy metrics.
are highly imbalanced, we also report ROC AUC, F1, and task solving scores. The last metric, task solving, measures whether the models perfectly solves each task. The results, especially the large gap on average task solving between our online inference algorithm and batch inference, demonstrate successful inducing the correct causal law within ACRE, and
that our online algorithm is more accurate at predicting what scenes obey the rule in Zendo. Interestingly, our most performant models-white assume hard deterministic gales-actually surpass human accuracy the question of how humanlike the model is (or isn t), which we investigate next.
[3
This raises
Model-Human comparisons. We the model on the same Zendo games that human participants did, taking human data from Bramley et al. 13 Average human accuracy 5.26/8, which surpasses a ReAct-style agent (4.60/8), falls short of our strongest model (6.55/8), and is close to the variant of our model which uses prob- abilistic fuzzy rules (5.35/8). For more fine-grained understanding of how human and model accuracy com- pare, we split accuracy across each of the 10 rules on test scenes that either obey the hidden rule (Rule Following RF condition) violate the rule (Not Rule Following or Not RF condition) (Figure 4a). With fuzzy rules, the model explains 57% of the variation in this more fine- ane-grained measurement human accuracy (R2 .57). Switching to hard rules drops this to R -10, suggesting that hard all-or-none rules do not provide as good an explanation of human behavior, even though hard rules outperform probabilistic ones in terms of accuracy. Doing batch inference instead of online inference degrades fit to R2 .05. Having the L.M play Zendo directly (ReAct (31) is only loosely correlated with human accuracy patterns (R² 225.. We last consider predicting every single human judgment every single test scene, for every single rule. The online, fuzzy rules model predicts these human judgments at the level of R² .35, and importantly, it is only with combination of online inference with fuzzy rules that gives a significant fraction explained variance (Figure 5), and which assigns the highest likelihood to the human data (
Method
LogT.
3 best model
- 1539
Batch, Fuzzy
-1660.90
Online, Fuzzy
-1478.82
Batch, Hard
-2921.00
Batch w/ Refinement, Hard
-3499.76
Online, Hard
-2209.9
Table 2: Log likelihood of human data on models summed over all test scenes of all Zendo rules.
Performance "the majority blocks red.'
We noticed across many rules a significant difference in human accuracy on RF and Not RF test scenes. Whether people finds RF or Not RF test scenes easier depends on the underlying rule. Figure 4b) illustrates this phe- nomenon and compares it against what each model thinks should be the easier condition. Online learning of fuzzy rules successfully predicts the direction of almost all of these trends, unlike the alternative models.
0.4
Online, Fuzzy Model
Human
0.3
Frequency
0.2
0.1
Hence, we hypothesize that although the hidden Zendo rules are deterministic, humans might nonetheless in- fer fuzzy rules. Real-world regularities are seldomly deterministic, so it may be rational for human learners to seek probabilistic explanations, especially when they uncertain about the underlying rule. However, fuzzy
0.0
Number Scanes Correct
Performance of human and Figure 6:
model on 'the majority of blocks is red'

---

LLM Human
Batch, Hard Model vs Human
Batch, Refine, Hard Model Human
(a)
0.26
0.8
Accurayy
Humann Acourayy
Human Accuray
0.7
0.7
0.7
0.6
0.8
0.6
0.5
04
0.4
0.4
Averaga Model Predicted Probabilty
Average Model Predicted Probabilty
Average Model Predicted Probabitty
Online, Hard Model vs Human
Batch, Fuzzy Model vs Human
Online, Fuzzy Model vs Human
0.9
R
0.10
R -
0.05
R 0.57
0.8
0.5
Human Actunccy
Human Acsumicy
Human Accuneyy
0.7
0.7
0.7
0.6
0.8
0.5
0.5
0.4
0.4
0.4
0.6
0.8
1,0
0.3
0.4
0.5
0.6
0.7
0.5
0.6
0.7
0.8
(b)
1.0
Rule
Rule
Rule
Rule
Rule
Rule
Rule
Rule
Rule
LLM
0.5
0.0
Batch,
Hard
1.0
200
0.0
Batch,
Refine.
Hard
1.0
0.0
Online
Hard
1.0
0.0
Batch, Fuzzy
1.0
0.0
Online,
Fuzzy
1.0
0.0
RF
NF RF
NERF RF NERF RF NF RF
NFRF
RF
NER RF RF
NERRF
RF
Not
RF
NF
RF Not RF
Mocel
Figure 4: Human vs model accuracy binned by rule-following (RF) and not rule-following (Not RF) test scenes. (a) Each point is a RF or Not RF accuracy for the 10 rules. (b) Rows/columns are methods/rules. Online inference with fuzzy rules (last row) most closely matches humans.
rules on their own do not suffice to explain human judgments: Only by combining with online probabilistic inference do we begin to explain the data.
Why reason in natural language instead formal language? Many Bayesian models account for human concept learning using probabilistic reasoning over formal languages such as logic 32
33 34
35 36).
Instead, our model operates over natural language. This helps address two liabilities of formal representations: expressivity and tractability. A handcrafted formal language is often insufficiently expressive, accidentally excluding many human concepts. This expressivity must be limited because, although there exist highly expressive formal languages, in practice, inference in such languages is generally intractable- a tradeoff partly addressed by using LLM proposal distributions.
To illustrate these points, we study new Zendo rule "the majority of blocks is red' wwiic is not expressible the formal language introduced by [3] We collect new human data in an IRB-approved study. Figure shows shows that both humans and our model correctly learn this rule 30% 40% of the time. This indicates both the model and humans are able represent this rule in their hypothesis space, which is unrepresentable in a formal language designed specifically for Zendo.
7

---

LLM Human
Batch, Fuzzy Model Human
Online, Fuzzz Model vs Human
1.0
J
0:13
1.0
R 0.35
Acuuray
Human Acnuraay
Acharany
Humain Ansumay
0.8
0.6
0.6
0.4
0.
0.4
02
0.2
0.0
0.0
0.6
1.0
0.0
0.6 0.8
0.0
0.4
0.6
Averraga Model Predicted Probabilty
Average Modal Predicted Probabilty
Average
Predicted Probabilyy
Figure 5: Comparing human and model prediction on each test scene after rounds of experimenta- tion; see also Table 2 Each point is prediction test scene. We only present LLM, best batch model, and best online model here. Please see the figure for all methods at Figure [4
Active learning method
Inference mothod
Proposer for InfoCiain
Numher of candidate ex experiments
Online. Fuzzy Online, Hard
LLM
4.52 0.08
4.72 0.08
Random
5.03 + 0.11
5.84 + 0.19
Random proposer
5.84 = 0.19
6.23 + 0.16
6.55 = 0.28
InfoGain
5.35 0.09 6.55 0.13
LLM proposer
5.73 0.16
6.19 0.12
6.55 0.13
Table 3: Average predictive posterior (stan- dard error computed over 5 seeds) of on- line inference models with different active learning methods on Zendo.
Table 4: Average predictive posterior (standard error computed over seeds) of online inference with hard rules model with different experiment proposers on different number of candidate experiments on Zendo
Another reason to use natural language representations is that LLMs, trained on human-ggnerian data, may to some extent capture human bias, judgement, and opinions (37 38 39]. Unlike approaches based on estimating probabilities on formal languages, incorporating LLMs into our models might therefore make them display more human- behaviors- as shown in earlier sections without access to additional human data. Indeed, Table 2isoows that our best-performen [3 's model on human data log likelihood even though the latter fits their models on both human active queries and predictions, while our model does not perform such parameter fitting.
model surpasses
Bounded rationality. To understand the effect of com- putational cost on the results, we analyze performance and human-model fit while varying the computational budget, as measured by LLM calls. Figure 7 plots human-model fit compute budget varies (see also T5b We observe an (inverted) U-shaped curve: Too little budget gives a bad fit, but overshooting also de- grades fit. This result aligns with the theory of bounded rationality (2I). which argues for considering human's limited cognitive resources, and with the rational anal-
R Score between Model and Human Predictions
0.5
0.4
Score
0.1
0.0
ysis of human processing limitations 23 40
5
10
Number of LLM per lleration
Batch, Fuzzy Online, Fuzzy
What makes good experiments: LLMs, or Informa- tion Gain? We first study the importance of the in- formation gain objective (Table 3). contrasting three different active learning methods: T.LM (prompting with the hypotheses and asking for a good experiment); Ran- dom (handcoded random generator), and InfoGain (main method, with LLM proposing experiments). Substituting InfoGain with alternative methods signifi- cantly degrades model performance. Reranking LLM proposals with information gain is important, and an LLM on its own own-does not generate experiments that are as effective.
Batch w/ Refinement Hard Online, Hard
Figure 7: R score of human vs model ac- curacy at different computational budgets. A LLM call batch-samples 15 hypotheses.
Is this explained by the strength of the LLM experiment proposer, or by the strength of the InfoGain objective? While earlier results support LLMs' effectiveness as hypothesis proposers, Table Adben- strates that a random proposer, hand-designed under reasonable assumptions, performs similarly to an I.M experiment proposer. This finding in line with 41 which argues that L.LMs may not always produce the most useful set of candidate questions.
8

---

4 Related Work
Bayesian Concept Learning in Cognitive Science. Bayesian models of few-shot learning of concepts and categories has a long legacy [32 34
33
35
45
46
47
36).
On Zendo, [13 14 [5] also engineers a probabilistic context-free grammar to define the Bayesian model to explain Zendo human data; inference is intractable, and they study various approximate inference algorithms on the task. Our work opts for natural language as the hypothesis space for its expressiveness and leverages LLMs to help with approximate inference.
Inductive Reasoning with Large Language Models. There are number of recent works on study the inductive reasoning ability of vanilla LLMs and propose a simple refinement algorithm to improve performance. M explicitly treats LLMs as importance samplers and shows that their model, with prior learned from human data and importance sampling as their inference algorithm. is human-like on some domains. Our work frames many of these works as batch inference- in the spirit of importance sampling, from a probabilistic view-which is compared in our results. Additionally, we model the full life-cycle of experimentation and hypothesis revision.
inductive reasoning with LLMs [10 9 11 157. [10 [
Active Learning with Large Language Models. The problem of performing active learning with the help of LLMs has been studied under the task of asking better questions with LLMs [58]
61 GATE [58] directly prompts LLMs to ask open-ended, informative questions. use LLMs to help propose several candidate questions, and then use expected information gain to select the question to be asked. Following these previous works, we investigate the relevance of classic criteria from active learning such as expected information gain.
59
60
41
Other works 59 60 41
Probabilistic Inference with Large Language Models. The framework of probabilistic inference has been applied to LLM-based algorithms. Language-model cascades 62 provides unifying framework for seeing recent inference-tine time L.LM algorithms [63 64
65
as reasoning with proba-
bilistic programs. Other work [66] fine-tunes CoT models by formulating the problem as maximum marginal likelihood where the marginalization over the latent chain of thought done via probabilistic inference. We use an LLM an aid for SMC-S, but others have explored using SMC as aid for which although technically very different, is conceptually complementary.
LLM decoding
667 68
5 Limitations and Next Steps
The work presented limited important ways that suggest next steps. Most immediately, many of the the hypotheses we consider are simple and stereotyped in form, and much of the promise of using natural language is that it exposes a rich, expressive representation for combining and creating new ideas [70). More ambitiously, hypothesis generation, both in science and in everyday thinking, often involves conjecturing the existence of unseen objects, not just unknown regularities, and incorporating this abductive -hicc is absent from our model-coule open many directions.
Although the work here is most directly an account of human behavior within the context of the game Zendo, our model is also more broadly inspired by the mental activities of experimental scientists as they build theories and models, weigh hypotheses, and design experiments. Two basic features of our approach reflect scientific experimentation and theory building. First, scientific theories apply only within particular regime. For example, many equations physics only apply when objects are moving slowly, and many thermodynamic equations only apply at equilibrium. Outside this regime, the theory is no longer predictive. Similarly, a variant of model like ours could hypothesize fuzzy, probabilistic rules which either predict category with high confidence, or outside the regime in which the rule applies, can fail to make a decisive prediction.
The second way in which this work reflects the practices of experimental science is that it builds its hypotheses via an incremental evolutionary process. In this way, our model best thought of where one works within an existing
as performing what is sometimes called 'normal science' paradigm, and considers piecemeal evidence - and does not model paradigm shifts 77] (what scientific theorist might pursue) or deeper conceptual changes development), both of which require deep reanalysis of broad batch of past data, rather than online incremental revision.
3 (what happens during child

---

6 Acknowledgements
We are grateful to Neil Bramley and Jan-Philipp Fränken for providing the Zendo data, helpful discussions, and comments on the manuscript. We also thank Hao Tang, Simon Alford, Celine Lee, and the anonymous reviewers for valuable comments on the manuscript. This work was supported by an NSF CAREER grant.
References
Andres H Mendez, Chen Yu, and Linda B Smith. Controlling the input: How one-year-old infants sustain visual attention. Developmental Science, page e13445, 2023.
[2] Claire Cook, Noah D. Goodman, and Laura E. Schulz. Where science starts: Spontaneous experiments in preschoolers' exploratory play. Cognition, 120(3):341 349, 2011. Probabilistic models of cognitive development.
[3] Susan Carey. Conceptual Change In Childhood. MIT Press, 1985.
[4] Laura Schulz. The origins of inquiry: Inductive inference and exploration in early childhood. Trends in cognitive sciences, 16(7):382-389, 2012.
[5] Alison Gopnik, Andrew N Meltzoff, and Patricia K Kuhl. The scientist in the crib: Minds, bruins, and how children learn. William Morrow & Co, 1999.
[6] David Klahr and Kevin Dunbar, Dual space search during scientific reasoning. Cognitive science, 12(1):1-48, 1988.
[7] Nick Chater and Mike Oaksford. The probabilistic mind: Prospects for Bayesian cognitive science. Oxford University Press, USA, 2008.
[8] Joshua Tenenbaum, Charles Kemp. Thomas L Griffiths, and Noah D Goodman. How to grow a mind: Statistics, structure, and abstraction. Science, 33 (6022)):1279-11285 2011.
[9] Linlu Qiu, Liwei Jiang, Ximing Lu, Melanic Sclar, Valentina Pyatkin, Chandra Bhagavatula, Bailin Wang. Yoon Kim, Yejin Choi, Nouha Dziri, and Xiang Ren. Phenomenal yet puzzling: Testing inductive reasoning capabilitics of language models with hypothesis refinement. arXiv preprint arXiv:2310.08550 2023.
[10] Ruocheng Wang, Eric Zelikman, Gabriel Poesia, Yewen Pu, Nick Haber, and Noah D Gaodman. Hypothesis search: Inductive reasoning with language models. arXiv preprint arXiv:2309. 05660, 2023.
[11] Kevin Ellis. Human-like few-shot learning via bayesian reasoning over natural language. NeurIPS, 2023.
[12] Pierre Moral, Arnau Doucet, and Ajay Jasra. Sequential monte carlo samplers. Journal of the Royal Statistical Society Series B: Statistical Methodology. 68(3):411 1-1444, 2006.
[13] Neil Bramley, Anselm Rothe, Josh Tenenbaum, Fei Xu, and Todd Gureckis. Grounding compositional hypothesis generation in specific instances. In Proceedings of the 40th annual conference of the cognitive science society, 2018.
[14] Jan-Philipp Fränken, Nikos Theoddorppulls, and Neil R Bramley, Algorithms adaptation in inductive inference. Cognitive Psychology, 137:101506, 2022.
[15] Neil Bramley and Fei Xu. Active inductive inference children and adults: constructivist perspective. Cognition, 238:105471, 2023.
[16] Alison Gopnik and David Sobel. Detecting blickcts: How young children use information about novel causal powers in categorization and induction. Child development, 71(5): (1:5):205-111 1222, 2000.
[17] Chi Zhang, Baoxiong Jia, Mark Edmonds, Song-Chur Zhu, and Yixin Zhu, Acre: Abstract causal reasoning beyond covariation. Proceedings of the ieeelcvf conference on computer vision and pattern recognition, pages 10643-10653, 2021.
[18] Dennis Lindley. On measure the information provided by an experiment. The Annals of Mathematical Statistics, 27(4):986-1005 1956.
[19] Tom Rainforth, Adam Foster, Desi R Ivanova, and Freddie Bickford Smith. Modern bayesian experimental design. Staristical Science, 39(1):100-110 14, 2024.
[20] Burr Settles. Active learning literature survey. 2009.
[21] Herbert A Simon. A behavioral model of rational choice. The quarterly journal of economics, pages 99 118, 1955.
[22] Adam N Sanborn, Thomas L Grifliths, and Daniel J Navarro. Rational approximations to rational models: alternative algorithms for category learning. Psychological review, 117(4):1144, 2010.
10

---

[23] Thomas L. Griffiths. Falk Lieder, and Noah D. Goodman. Rational use of cognitive resources: Levels of analysis between the computational and the algorithmic. Topics Cognitive Science. 7(2):217-229. 2015.
[24] Roger Levy, Florencia Reali, and Thomas Griffiths. Modeling the effects of memory on human online sentenee processing with partiele filters. Advances in neural information processing systems, 21, 2008.
[25] Adam N Sanborn and Nick Chater. Bayesian brains without probabilities. Trends in cognitive sciences,
20( 20(12):883-89) 2016.
[26] Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. Teaching large language models to self-debug. arXiv preprint urXiv:2304.05122 2023.
[27] Christian Naesseth, Fredrik Lindsten, and Thomas Schon. Nested sequential monte carlo methods. In International Conference on Machine Learning, pages 1292-1301. PMLR, 2015.
[28] Jun Liu and Jun S Liu. Monte Carlo strategies in scientific computing, volume 10. Springer, 2001.
[29] Tuan Anh Le. A better proof of unbiasedness of the sequential monte carlo based normalizing constant estimator, 2023.
[30] OpenAL. Gpt-4 technical report, 2023.
[31] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models, arXiv preprint arXiv:2210.03669, 2022.
[32] Noah D Goodman, Joshua B Tenenbaum, Jacob Feldman, and Thomas L Griffiths. A rational analysis of rule-based concept learning. Cognitive science, 32(1): 2008.
[33] Steven Thomas Piantadosi. Learning and the language of thought. PhD thesis, MIT, 2011.
[34] Brenden Lake, Ruslan Salakhutdinov. and Joshua B Tenenbaum. Human-level concept learning through probabilistic program induction. Science, 350(6266):1332-18566
2015.
[35] Goker Erdogan, llker Yildirim, and Robert Jacobs. From sensory signals modality modality-independen conceptual representations: A probabilistic language thought approach. PLoS computational biology, 11(11)=1004610, 2015.
[36] Mathias Sablé-Meyer, Kevin Ellis, Josh Tenenbaum, and Stanislas Dehaene. language thought the mental representation of geometric shapes. Cognitive Psychology, 101520 2022.
[37] Gati Aher, Rosa Arriaga, and Adam Tauman Kalai. Using large language models simulate multiple humans and replicate human subject studies. In International Conference on Machine Learning, pages 337-371. PMLR, 2023.
[38] Danica Dillion, Niket Tandon, Yuling Gu, and Kurt Gray. Can ai language models replace human participants? Trends in Cognitive Sciencex, 2023.
[39] Ishita Dasgupta, Andrew Lampinen, Stephanie CY Chan, Antonia Creswell, Dharshan Kumaran, James McClelland, and Felix Hill. Language models show human-like content effects on reasoning, arXiv preprint arXiv:2207.0705: 2022.
[40] John R. Anderson. The adaptive character of thought. 1990.
[41] Kunal Handa, Yarin Gal, Ellie Pavlick, Noah Goodman, Jacob Andreas, Alex Tamkin, and Belinda L.I. Bayesian preference elicitation with language models. arXiv preprint arXiv:2403.05534 2024.
[42] Marie Amalric, Liping Wang. Pierre Pica. Santiago Figueira, Mariano Sigman, and Stanislas Dehaene. The language geometry: Fast comprehension geometrical primitives and rules in human adults and preschoolers. PLoS compitational biology, 13(1 re
(3(1)ce10053)) 1052733, 2017.
[43] Kevin Bllis, Adam Albright, Armando Solar-Lezama, Joshua B Tenenbaum, and Timothy JO O'Donnell. Synthesizing theories of human language with bayesian program induction. Nature communications, 13(1):5024, 2022.
[44] Kevin Ellis, Catherine Wong, Maxwell Nye, Mathias Sahlé-Meyer, Lucas Morales, Luke Hewitt, Luc Cary, Armando LezamaL, and Joshua B. Tenenbaum. Dreamcoder: Bootstrapping inductive program synthesis with wake-sleep library learning. In PLDI, 2021.
[45] Lucas Tian, Kevin Ellis, Marta Kryven, and Josh Tenenhaum. Learning abstract structure for drawing by efficient motor program induction. Advances in Neural Information Processing Systems, 33:2686-2697, 2020.
[46] Feras A Saad, Marco F Cusumano- Towner, Ulrich Schaechtle, Martin C Rinard, and Vikash Mansinghka. Bayesian synthesis of probabilistic programs for automatic data modeling. Proceedings of the ACM on Programming Languages, 3(POPL):1-33 321, 2019.
[47] Percy Liang, Michael I. Jordan, and Dan Klein. Learning dependency- based compositional semanties. In ACL, pages 9999 2011.
11

---

[48]
Andrew G Wilson and Pavel Izmailov. Bayesian deep learning and a probabilistic perspective of general- ization. Advances neural information processing systems, 33:4697-4708, 2020.
[49] Sreejan Kumar, Carlos G Correa, Ishita Dasgupta, Raja Marjich, Michael Hu, Robert D. Hawkins, Jonathan Cohen, Nathaniel Daw, Karthik R Narasinhan, and Thomas L. Griffiths. Using natural language and program abstractions to instill human inductive biases in machines. In NeurlPS, 2022.
[50]
François Chollet. On the measure of intelligence, 2019.
[51] Pedro A Tsividis, Joao Loula, Jake Burga, Nathan Foss, Andres Campero, Thomas Pouncy, Samuel J Gershman, and Joshua B Tenenbaum. Human-level reinforcement learning through theory-based modeling, exploration, and planning. arXiv preprint arXiv:2107.12355 2021.
[52] Kevin P Murphy. Machine learning: a probabilistic perspective. MIT press, 2012.
[53] Joshua Brett Tenenbaum. Bayesian framework for concept learning. PhD thesis, Massachusetts Institute of Technology, 1999.
[54] Steven Piantadosi, Joshua B Tenenbaum, and Noah D Goodman. The logical primitives of thought: Empirical foundations for compositional cognitive models. Psychological review, 123(4):392, 2016.
[55] Jan- Jan-Philipp Fränken, Christopher G Lucas, Neil R Bramley, and Steven T Piantadosi. Modeling infant object perception as program induction. arXiv preprint arXiv:2309.07000 0009999999 2023.
[56] Yangqiaoyu Zhou, Haokun Liu, Tejes Srivastava, Hongyuan Mci, and Chenhao Tan. Hypothesis generation with large language models. arXiv preprint arXiv:2404.0433 2024.
[57] Sarah Schwettmann Tamar Shaham, Joanna Materzynska, Neil Chowdhury, Shuang Li, Jacob Andreas, David Bau, and Antonio Torralba. Find: A function description henchmark for evaluating interpretability methods. Advances in Neural Information Processing Systems, 36, 2023,
[58] Belinda Z Li. Alex Tamkin, Noah Goodman, and Jacob Andreas. Eliciting human preferences with language models. arXiv preprint arXiv:2310. 1588, 2023.
[59] Wasu Top Piriyakulkij. Volodymyr Kuleshov, and Kevin Ellis. Active preference inference using language models and probabilistic reasoning. In NeurIPS FMDM Workshop, 2023.
[60] Gabriel Grand, Valerio Pepe. Jacob Andreas, and Joshua B Tenenbaum. Loose lips sink ships: Asking questions in battleship with language-infonnt informeed program sampling, arXiv preprint 2rXiv:2402.19471, 2024.
[61]
Chinmaya Andukuri, Jan-Philipp Fränken, Tobias Gerstenberg, and Noah D Goodman. Star-gate: Teaching language models to ask clarifying questions. arXiv preprint arXiv:2403.191
2024.
[62]
David Dohan, Winnie Xu, Aitor Lewkowycz, Jacob Austin, David Bieber, Raphael Gontijo Lopes, Yuhuai Wu, Henryk Michalewski, Rif Saurous, Jascha Sohl-Dickstein, et al. Language model cascades. arXiv preprint 2iv: 10342, 2022.
[63] Maxwell Nye, Anders Andreassen, Guy Gur-Ari, Henryk Witold Michalewski, Jacob Austin, David Bieber. David Martin Dohan, Aitor Lewkowycz. Maarten Paul Bosma, David Luan, Charles Sutton, and Augustus Odena. Show your work: Scratchpads for intermediate computation with language models, 2021.
https///asxiv.rr//as2111 11t.:/.oorrgaaaa222.000001 12.00114.
[64] Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc Le, Denny Zhou, et al. Chain thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824 24844224377228 2022.
[65] Antonia Creswell. Murray Shanahan. and Irina Higgins.
Exploiting large language models for interpretable logical reasoning. arXiv preprint arxiv:2205 200.0722,, 2022.
[66] Matthew Douglas Hoffman, Du Phan, David Dohan, Sholto Douglas, Tuan Anh Le, Aaron Parisi, Pavel Sountsov, Charles Sutton, Sharad Vikram, and Rif Saurous. Training of-in-of-thhuglee via vatenl-variable inference. Advances Neural Information Processing Systems, 36, 2024.
[67] Alexander K Lew, Tan Zhi Zhi-Xuan, Gabriel Grand, and Vikash K Mansinghka. Sequential monte carlo steering of large language models using prohabilistic programs. arXiv preprint arXiv:23366.03088 2023.
[68] Stephen Zhao, Rob Brekelmans, Alireza Makhzani, and Roger Grosse. Probabilistic inference in language models via twisted sequential monte carlo. arXiv preprint arXiv:2404.17555 2024.
[69] Zirui Zhao, Wee Sun Lee, and David Hsu. Large language models as commonsense knowledge for large-scale task planning. arXiv preprint arXiv:2305.1 14078, 2023.
[70] Elizabeth Spelke. What Makes Us Smart? Core Knowledge and Natural Language, pages 277-312. 03 2003.
[71] Thomas Samuel Kuhn. The structure of scientific revolutions. 1962.
12

---

A
Appendix
Contents
A.1
Proof for the weight update of LLM-SMC-S
14
A.2
Code and data availability
14
Zendo and ACRE details
14
A.4
Algorithm details
15
A.5
Example hypothesis traces from models
16
A.6
Baseline descriptions
17
Priors
18
A.8
Computational cost
18
A.9
Human study 'majority red rule details
18
A. 10 Prompts
A.11 Supplemental results
23
13

---

A.1 Proof for the weight update of LLM-SMC-S
Proposition 1. If H, W input to Procedure LLM-SMC-S properly weighted with respect to 7, then the output lt,, wo is properly weighted with respect to 7".
be the normalized Proof. Let z = J (K'')) )ah' be the normalizing constant of and x' 2-r target. We want to show
7'(h')
flw')(h)) Za (1))
Following arguments similar
29 we have
LHS
A(W, H1, Wi)
96+1 (h") He, 2 111+1: 11ax- S(K)
(sub in the definition of w:)
(8)
-.,.W, If A(K He Wa)f(h'dhk
(write En as an integral)
ml 2 ('h')) 21hh)(((000))))) sah)an
Eu,
(sub in the definition of A)
7h(0))
(10)
B00 If (1 7(0))((00)))))00000 s(h))a)))
/>
(pull the Σ out of the
((h(0))
(11)
wi (wo 9(b(())
Enso wit
(denote the integral ((h(()))
11
(12)
(apply the proper weighting
Zx Em(1) (9(h)
property with test function g)
(13)
2z / /1(1) /
yal dh 7(h)
(sub in expression for g)
(14)
=/1 (()))(A))))())))))
(cancel terms using Zr 7/
(15)
= - rarsosyan
(r(h"hh normalized)
(16)
7 Feree
(Z ==/-//1
(17)
= RHS.
(18)
A.2
Code and data availability
Code and data available at
and- wtweviiinnrruleee///e
A.3
Zendo and ACRE details
Zendo. Zendo is game where player seeks to infer hidden binary rule about assemblies of colored blocks. The game starts by providing the player with positive scene that follows the hidden
14

---

rule. Then, the player queries an oracle as to particular scene follows the rule or not, or makes a guess about the secret rule. The game ends when they guess correctly.
Bramley et al. (2018) [3 introduces a 2D version of the Zendo game shown in Figure R The scenes consist of blocks, each with its own color (red, blue, green) and size (small, medium, large). The blocks can have different orientations and positions in a 2D scene. They may and may not touch each other. The game starts with an initial phase where rule-following scene is given, followed by 7 rounds of active learning phase where the player gets to query an oracle for ground truth prediction. At the end, the player enters a prediction phase where they are asked to give predictions for 8 test scenes (4 rule-following and 4 not rule- mile-foloownng.. Bramley et al. (2018) study human gameplay on 10 rules, collecting data from 30 participants who each play Zendo 10 times (once per rule). They use a cover story of an alien planet where some arrangements of blocks emit radiation, and the task is to figure out a rule predicting radiation emission.
Zendo is most naturally framed as a visual-physical concept learning problem. For our model, however, we will work with discrete symbolic descriptions of scenes. This makes that problem more compatible with the language-of-thoughh paradigm, and also allows using LLMs to operationalize the language of thought. We therefore modify Bramley et al. (2018)'s version of Zendo by associating each block in a scene with discrete attributes instead. The five attributes are color (red, blue, green), size (small, medium, large), orientation (upright, left, right, strange), groundedness (grounded, ungrounded, stacking), and touching (which blocks it touches / stacks). While this natural language version of the game removes continuous attributes, such as z, y position and orientation in 2D space, from its scene representations, these five attributes still maintain the complexity of the game and are sufficient for all 10 Zendo rules
The data is licensed under CC-BY 4.0.
ActiveACRE We convert the originally visual tasks into symbolic version of the tasks, similar to (.) While the ground truth rule always has the structure that the blicket machine produces noises when or more "blicket" objects (each object is either blicket a non-blicket) is placed on the machine, in contrast to 191. we do not hint the learners that the ground truth rule is of this form, which means the learners are free to think that the rule may have to do with colors, number of objects, etc. We further modify the task to incorporate elements of active learning, making the logistics similar Zendo: the game starts with 8 relevant objects, described with color
material (metal/rubber), and shape attributes (cube/sphere/eylinde))), placed on the blicket machine which causes the machine makes sounds and follows by 7 rounds of query. The prediction phase tests the models on all possible combinations of the eight objects. We call this resulting domain, ActiveACRE, Figure I partially shows what a gameplay of simplified ActiveACRE looks like.
To obtain the 8 initial objects, sample uniformly from the three attributes to get object and keep doing this until we achieve 8 unique objects. This can be done with a simple code, without external data.
A.4 Algorithm details
For all methods, unless specified, the number of LLLM calls used per each iteration is 5 with each call batch-sampling 15 natural language hypotheses.
Batch Inference. For batch, fuzzy model, we cap the number of unique hypotheses considered to 30, otherwise we would have too many hypotheses considered, since all fuzzy hypotheses have non-zero posterior probability, making inference very compute intensive.
Batch Inference with Refinement. We set the number of refinement to 2 (we have tried increasing the number of refinement to but didn't see any improvement). Following 19, this method works as follows: (1) it first siich-samples many hypotheses with LLM, (2) select the best hypothesis (in
"The 10 rules we use are "there's a red". "all are the same size". "nothing is upright", "one is blue", "there's 's a small blue", "all are blue or small", "a red is bigger than all non reds", "some touch", "a blue and a red touch" "some pieces are stacked",, and "some pieces are stacked"
15

---

Algorithm LLM-SMC-S algorithm
Let (21,31) be the first data point we observe
h(1) h(mh ( F 91)
+ plan-n )
v11 for sisn
Reweighting Resampling
H1 Resampling H1, W1) for t = 2,.... T do
The active learning algorithm gives (It, 3t)
m. ~ q(h) H T1:t, V1:t)
Rejuvenating
A(h (0)
H1-1, W Σド= w p(h, 10 (m (rth (1 ananaa)
wt - A(H.)) He-1,W for 1 < sisn
Reweighting
q(k) |
H, < Resampling H1, W()
Resampling
end for
Algorithm 2 B function pseudocode
function B( V1.:; H)
result = 0
h1 .., h4 top---lowest-ikchhhooddd.. M1.e) get & hypotheses with lowest likelihood
for i = 1..... do
Hub LIM Uttt,,t,1, vt. h4)
get neighbors (nb) of h, D filter out bad neighbors want to consider maximum of neighbors
Hub = (hnb Hub p(hnb (ri: V1:1) S p(hs v1=)))
if Hnb > then
w (i)
for < Sism n H Down Sampling( Hnb: p W. Wab: size m)
+- p(h
end if
result result Hnb end for
return result
end function
numbers of data points accounted) to be refined, (3) use LLM to output a batch of refined hypotheses, and (4) repeat the (2)-(3) steps until least one hypothesis fully accounts for all data points.
Online Inference (LLM-SMC-S) The algorithm for LLM-SMC-S is described in Algorithm For the first iteration, the initial important proposer q( (r y1 is defined be an LLM,, similar to batch inference. We define the forward kernel q in the algorithm as follows:
H()HH, List, Via) & 1/4 E (H U B( Dia, H))
(19)
The pseudocode for B can be founded at Algorithm 2 What B doing is basically look low likelihood hypotheses, prompt LLM to come up with their neighbors, and filter out bad neighbors and limit the number of chosen neighbors to m. We find that having the down-sampling step to keep the number of neighbors considered low is helpful in practice, but one remove this step make B fully deterministic. The LLM function in the pseudocode means prompting an LLM with zero temperature.
A.5 Example hypothesis traces from models
Batch. 'Blocks must touch least one other block' is proposed but immediately falsified by an existing experiment where a scene with no blocks touching is negative.
Batch with refinement. 'Blue blocks must not touch green blocks" proposed and then refined into BBlue blocks must not touch blocks of any color other than red". This hypothesis later gets falsified, without an opportunity to refine itself since the model is not online, when a scene with no blocks touching is negative.
16

---

Same Priors Model Human
Different Priors Model vs Human
0.9
0.9
R² 0.22
R 0.57
0.8
0.8
Human Accuracy
Human Accuracy
0.7
0.7
0.6
0.6
0.5
0.5
0.6 Average Model Predicted Probabilty
0.5
0.7
0.5
0.6
0.7 0.8
Average Model Predicted Probabilty
Figure 8: Human online, fuzzy model accuracy binned by rule-following (RF) and 4 not rule- following (Not RF) test scenes, This figure shows online, fuzzy model with same and different priors for and 6
Online. "There must be blue block' is proposed and added the pool of particles. Since it has higher prior than other particles (has shorter length); it keeps surviving while others get killed, despite some conforming with the data. Upon seeing a scene with a blue touching green being negative, the particle 'There must be blue block' is perturbed into 'there must be a blue block touching red block'.
A.6 Baseline descriptions
In probabilistic inference terms, both batch with and without refinement correspond to importance [P(A" 1 k'].
sampling p(h|a1. V1.1) (ppph(| h']] (g(hr (tt. (((()1a)))
The difference in the baselines lie in how q(h' (ri.s via) constructed.
Batch. ( viv)) (LLLM(Z1 where (LM(..)) prompts an LLM to return list of hypotheses
Batch with refinement. qh|[r1a (1.) U(Refieed-LLM(rp Whtt None, 0)) where
Refined-LLe LLM is defined as follows:
First, let s(,, Vits M1st) = + Ei_ [1(();)) = v.l. This simply scores what percentage data points in Fits Viet that h makes correct predictions. Then,
function Refimed-LLMMrruu F1.t. h, k):
= LLMM-wtt-h(ppp. Vaa, h) # Prompts LLM to refine h
if A: = K:
return 0
else 3h' E H, (h', Fiss var) 1:
return H E H H|ssh' Tias (1a)) = 1 else:
h = arg Fiss v1t))
return Refined- - LLM (x1a, yaat ht, l + 1)
where K is the number of refinements allowed.

---

Method
Number of LLM Calls Per Iteration
5
10
Batch, Fuzzy
4.58 + 0.12
4.57 + 0.15
4.56 + 0.16
Online, Fuzzy
5.11 0.04
5.35
+ 0.09
5.28 + 0.06
Batch, Hard
6.16
0.19
6.18 + 0.14
Batch w/ Refinement, Hard
6.15 0.16
6.18
5.83 + 0.16
Online, Hard
6.15 +: 0.26 6.55 + 0.13
6.38 + 0.11
Table Average predictive posterior (standard error computed over seeds) of models with different number of LLM calls (each LLM batch-samples 15 hypotheses) on Zendo.
A.7 Priors
Priors for and p(5) has mean of 0.7 and standard deviation of 0.1, and p(e) has mean of 0.9 and standard deviation of 0.01. Both distributions are truncated to remain within the range [0.5,
1]. We found that using different priors for and results in more human-like behavior as shown in Figure
Prior for h We let p(h) be inversely proportional to the word count of for Zendo and uniform for ActiveACRE.
For Zendo, we consider using prior that would decay exponentially in length but find that letting p(h) (word_coun((h)) )2 already makes the particles become mostly short strings. prior decaying exponentially in string length would definitely be too harsh on the hypotheses.
A.8 Computational cost
Computational Cost Analysis 5 the performance of models with different compute budgets (number of LLM calls per iteration) on Zendo. It turns out that the performance of batch inference models plateaus after just 15 hypotheses (1 LLM call), while the performance of online inference models benefits from being able to sample more hypotheses but also plateaus after 75 hypotheses (5 LLM calls).
Experiments Compute Resources We also describe here the compute resources required to reproduce the experiments. The main compute cost comes from OpenAI API which we call to prompt GPTs, The models with 1, 5, 10 L.M calls per each iteration uses up roughly $0.5, $1.5, $3 OpenAl API credit seeds to get the performance numbers method like we reported. The actual cost, however, could be lower than calculated since one cache LLM responses.
Zendo task. For Zendo, one needs
50 tasks 10 Zendo tasks on 5 different
A.9 Human study 'majority is red' rule details
20 participants from our academic department were recruited via Slack to attempt the rule "the major- their performance ($10 base rate $1.25 bonus for each correct test scence prediction - there are 8 test scenes). Figure shows the web interface displayed to participants. The full instructions given to human participants are displayed at Figures and [3
ity of blocks are red". The participants are compensated $10-$20 depending
18

---

Special blocks 1 of 2
(euample)
A
Test
Now you play with the bemmies. Press buttons at the bottom to add blocks. Move the blocks around by picking them up with the mouse (left clicking and holding)
(counterclockwise) and (clockwise) keys Right click on them to remove them (command click you are using mac trackpad).
Turn them using
When you're done moving the special blocks, click "Test" to see If stars will come out
Figure 9: Example of the web interface shown to participants.
A
AM
A
A
A
Figure 10: First figure for human participants instructions shown at Figure 12
19

---

These special blocks are caleed Blickets. Stars come of there least one small block
2. These epecial blocks are called Wozzles. Stars come out of them the blocks are all the same color
A
3. These special blocks called Daxes. Stars come out of them if none of the blocks are touching
These special blocks are called Timas. Stare come out of them blocks point in the same direction
AA
Figure 11: Second figure for human participants instructions shown at Figure
20

---

Thank you for playing my game!
In this game you will be learning about an alien planet. This planet is called Zorb. On Zorb, there are these special blocks that look like this:
[Figure 9}
These apecial blocks may look the same, but there many different kinds of special blocks on the planet Zorb. They have different names, and they work differently.
Sometimes, when the blocks set up in certain ways, stars will shoot out of them! Every kind special block has different rule for making shoot
them.
Your job is to figure out each rule for how to make stars come out of all of the different kinds of special blocks!
(Figure 10)
So there are a lot of things that might make stars come out of certain special blocke!
You might get stare from blocks different numbers, from blocks different colore, from blocks of different sizes, from blocks facing different directions, and more!
We found out that there are more kinds special blocks on Zorb! Let's call them "Bemmies' and 'Yoks', Again, you do not have to memorize the names -- we just want to emphasize that different kinds of blocks work under different rules
But don't know the rule for setting up each kind of special blocks stars will come out them. Your job is figure out the two different rules for how set up each different kind of special blocks so stars come out!
Now we're re going watch video. This video is going to show you how you the special blocks around yourself! You must watch the video to continue.
So, in the interface you can:
Press buttons the bottom add blocks
Move the blocks around by picking them up with the mouse (left clicking holding)
Turn them using the "z" (counterclockvise) and "x" (clockvise) keys Right click them remove them (command + click if you using trackpad)
When you' rou're done moving the special blocks, you're going to test them to see if stars will come out of them. If you set them up in the right way according to the rule, you'll
bunch of stars appear! Otherwise, nothing will happen.
Figure 12: (Part Instructions for participants. Please find instruction figure and Figures 10 and 11
21

---

After you move the special blocks around and test them, you're going to see if you can pick out which pictures of the blocks you think will shoot out stars. This video will show you how to do that:
(demo video)
In the video you show bemmies that will could be anywhere between one seven the pictures.
that this participant thinks that four of the pictures of (the marked in grey). The right
Adults: You will earn bonus of $1.25 for each of the pictures in the main task where you guess correctly whether it will shoot out stars (demo task performance not count). That means, if you get eight pictures correct in the main task, you will earn bonus of $10!
You must watch the video to continue.
(demo video)
Finally, you may guess the rule for how this kind of special blocks works.
For example, if looks like only shoot green, you would write something like: "all the blocks have to green"!
the blocks if all them
Warning: Your responses will be checked by human before HIT approval. Nonsensical copy-pasted answers will lead to your HIT being rejected. If you truly have ideas about rule, please just write "I do not know".
Instructions Summary:
You will look 2 different kinds of special blocks (including demo task for learning the game) that will shoot out stars they
up certain ways.
You must figure out the rule for how each kind of special blocks works.
You will up the special blocks and test them if stars will shoot out of them seven times for each type.
Your goal is to figure out which out of 8 new pictures of each kind of special blocks will shoot out stars ($1.25 bonus for each correct in the main task)
... to write down your best guess of rule for that kind of special block! Figure 13: (Part 2) Instructions for participants.
22

---

A.10 Prompts
The prompts used all of our experiments can be found in Tables to 12
For Zendo, we engineer the prompts for initial importance sampler q(h x. y) for online inference so that they only output simple rules (see Table 7 this approach helps the proposer output hypotheses with higher priors, since our prior defined by the number of words the rule. We cannot apply this trick to batch inference because, unlike online inference, it does not evolve simpler rules into more complex ones. Additionally, we also design the importance sampler prompts to avoid proposing negative rules ('there is no...) ...') (see Table 7 We found that this leads more human-like behavior and also better performance.
A.11 Supplemental results
See Table 6 and figs. 14 and 15
23

---

Method
LogL
best model
3085
Batch, Fuzzy
3352.98
Online, Fuzzy
2988.77
Batch, Hard
5842.00
Batch w/ Refinement, Hard
6999.52
Online, Hard
10419.86
Table 6: BIC scores of models on human data.
LLM vs Human
Batch, Hard vs Human
Batch, Refine, Hard Model Human
1.0
0.10
1.0
RP =
0.00
1.0
0.00
0.8
Human Ancuney
0.8
Human Antunay
0.8
0.6
0.6
0.6
Human
0.4
0.4
0.4
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1,0
0.0
0.2
0.4
0.6
0.8
1.0
Average Model Predicted Probabitty
Average Model Predicted Probabilty
Average Model Fredicted Probabitty
Online, Hard Model Human
Batch, Fuzzy Model Human
Online, Fuzzy Model vs Human
1.0
1.0
R 0.35
Human Accussy
AumenAcuunayy
0.8
0.6
0.4
0.0
0,0
0.7 Average Predicted Probabilty
Average Probabiity
Average
Probabilty
Figure 14: Comparing human and model prediction on each test scene after 7 rounds of experimenta- tion. Each point is prediction on test scene.
LLM Active Leaming vs Human
Random Active Learnnng vs Human R"
Info Gain Active Leamnnng vs Human
0.08
0.57
0.8
0.8
Humen Accurary
Human Accuney
Human Accunnyy
0.7
0.7
0,7
0.6
8.5
0.5
0.4
0.4
0.4
0.5
0.5
0.7
Model Prodicted Probabilty
Average Model Predicted Probabilty
Model Predicted Probabitty
Figure 15: Human vs online, fuzzy model accuracy binned by 4 rule-following (RF) and not rule- following (Not RF) test scenes. This figure shows online, fuzzy model with three different active learning methods: LLM, Random, and InfoGain
24

---

Method
Prompts for Zendo
Prompts for ActiveACRE
Given the following
structures described with satt_summary) of blocks in the structures: (text_c}
A group of objects may make the "blicket machine" have lights turned on or off depending on the objects it. We seek to figure out the rule underlying this. Consider the following:
Please list {num} possible rules about the attributes in structure that differentiate the good structures from the bad structures.
Batch Inference
Keep in mind that
1. All bad structures must violate the rules.
{text_c}
2. Orders of blocks in a structure do NOT matter.
Please state {num} possible rules what makes the light turned on. State them in listed number. Do not explain.
3. Do NOT propose "negative rules" such "there is no green block". short,
4. The rules concise, single sentences. Please number them from 1-(num} and do not say anything else
Please list {num} possible rules about the {att}.
A group of objects may make the "blicket machine" have lights turned on or off depending on the objects in it. We seek to figure out the rule underlying this. Consider the following:
Example
Structure: blue, blue Simple rules (Orders NOT matter):
blue block 2. All blocks are blue Do NOT propose "negative rules" such "there is no green block". Do NOT propose rules with quantifier such as "there two blue blocks"
1. There
Online Inference
{text_c}
Please state {num) possible rules what makes the light turned on. State them in listed number. Do not explain.
Task 1: (x)
Simple rules (Orders do NOT matter):
Table 7: Prompts used for the importance sampler g(h[21, y1) of all methods
25

---

Prompts for Zendo
Prompts for ActiveACRE
An object contains the following
attributes:
color (gray/rred/blueegreenn brown/cyan/ httpm//yyaa/puupe//elloooo material (metal/rubber) shape shape Ccuube/sphere/yllinng
structure one or more blocks. Each block should contain the following attributes: (att_par)
Example of rule modifications. Additional conjunction: 'The light turns on when there is cylinder present' -> 'The light turns on when there is cylinder and cube present'
Example of rule modifications: Quantifier change: 'There must be a green block' -> 'There are two green blocks'
Additional attribute: 'There must be a green block' -> 'There must be green block that is upright'
Additional disjunction: 'The light turns on when there is a cylinder present' -> 'The light turns when there is a cylinder or a cube present'
Attribute change: 'There must be a green block' -> 'There must be blue block'
Additional attribute: 'The light turns when there is cylinder present' -> 'The light turns on when there is blue cylinder present' These modifications are "local": only one clsjunction/conjuueett attoonnttttbuuteee is changed or added for each modification.
These modifications are "local": only one attribute/ quautt//qaaiiiieerrr is changed added for each modification.
Please modify the rule '(h'' Generate {num} rules for each type modification (Quantifier change, Additional attribute, Attribute change) so that the following structure is {text_y) a good structure:
Please modify the rule 'hh'' Generate {num} rules for each type of modification (Additional conjunction, Additional disjunction, Additional attribute) so that the light does (text_y) turn when the following objects are present: {x}
(x)
Note that the number of the blocks not matter.
Make the format numbered list (1., 2., 15.) Remember that the new rules should be "local" modification from the rule 'h Do not use attribute values that are not mentioned earlier. Do not say anything other than the modified rules.
Note that the number of the blocks do not matter.
Make the format numbered list (1., 2., 15.) Remember that the rules should be "local" " modification from the rule '(h)' Do not use attribute values that are not mentioned earlier. Do not say anything other than the modified rules.
Table 8: Prompts used for the forward kernel q(h' H1, Tra via) of online inference methods
26

---

Prompts for Zendo
Prompts for ActiveACRE
Given the rule '(h'', please give one group objects that makes the light turned on and another that makes the light turned off
Given the rule 'hh', please give one structure that conforms with the rule and another structure that violates with the rule.
The list of available objects are {all_objects),
structure has one of more blocks. Each block should contain the following attributes: (specc((ttckinggnno)
The format of your answer should follows:
The format of each structure should be follows: (conforms with the rule) Structure 1: (example_block)
light on group of objects: obj_1, obj_2,
light off group of objects: obj_1, obj_2,
(violates the rule) Structure (example_block)
All objects in a group must be unique. Do not say anything else.
Table 9: Prompts used for experiment proposers.
27

---

Prompts for Zendo
Prompts for ActiveACRE
Please synthesize a python program that implements the rule '(h}'
Please synthesize python program that implements the rule '(h}'
The program should takes ZendoStructure which represents structure and returns True if it's good structure and False otherwise.
The program should takes in a ACREGroup which represents group of objects and returns True if it's good group and False otherwise.
The docstrings for the classes follow:
The docstrings for classes are follow:
class ZendoStructure:
.param blocks: list of ZendoBlock
class ACREGroup:
:param objs: list ACREObject
class ZendoBlock:
:param color: str (blue/red/green)
class ACREObject:
:param color: str (gray, red, blue, green, brown, cyan, purple, yellow)
:param size: str (small/meddum/largee)
:param orientation: str (ppriight/leef/rrrhh///tttggg (groundedness_param, msm] :param touching: list of int (index starts at 1)
:param material: str (metal, rubber)
param shape str (cube, sphere, cylinder)
The signature for the synthesized program should be def rule(group: ACREGroup) bool
The signature for the synthesized program should be def rule(structure: ZendoStructure) bool
Only output the 'rule' function. Do not include anything else.
Only output the 'rule' function. Do not include anything else.
Table 10: Prompts used to translate natural language h to code.
28

---

Prompts for Zendo
Prompts for ActiveACRE
structure has Each block should contain the following attributes: (att_par)
blocks.
An object contains the following attributes: color (gray/red//blue/grenn/ proom//caanpprppee/yello)))
Consider the following rule: '{h)>
Given structure, the output is yes follows the rule (or is good structure) and no if it does not (or bad structure)
material (metal/rubber)
shape http:/sppeeeeycyideeeee
Consider the following rule: '(h)'
The given rule gives incorrect output for the following structures:
The given rule gives incorrect output for the following groups of objects:
(feedback)
{feedback}
Based the given rule, generate (num} new refined rules that fix the outputs for all mentioned structures. The new rules may involve any of the mentioned attributes (color, size, orientation, grounded, touching). Please number them from 1-(num} and do not say anything else
Based the given rule, generate {num} new refined rules that fix the outputs for all mentioned structures. Please number them from 1-(num} and do not say anything else
Table 11: Prompts used perform refinement in batch inference with refinement
29

---

Prompts for Zendo
Prompts for ActiveACRE
You are playing
inductive game with me. I'1l be the moderator, and your task is to figure out the secret rule that know by coming up with group of blocks to me whether the group conforms with the secret rule or not.
You are playing inductive game with me. I'1l be the moderator, and your task is to figure out the secret rule that know by coming up with structure blocks ask me whether it conforms with the secret rule or not.
An object contains the following attributes: color
The structure has one more blocks. Each block should contain the following attributes: {att_par)
(gray/ree/blue/green// broom/ccaam/prppeeeyylloo//
Initial prompt
material (metal/rubber) (cappeccube/aphhee//cylll The list of available of objects are fall_objecta).
To give you start, P'1l describe one structure that follows the rule:
give you a start, I'11 describe one group of objects that follows the rule:
{text_c}
Give very short summary on what you currently think the secret rule is.
(text_c}
Give very short summary on what you currently think the secret rule is.
The verdict whether the queried structure follows the rule [verdict). Give a very short summary on what you currently think the secret rule is.
The verdict whether the queried structure follows rule is {verdict). Give a very short summary on what you currently think the secret rule is.
Follow-up up prompt
Give structure you want to test whether it follows the secret rule or not. Do not include anything other than the structure.
Give one structure you want to test whether it follows the secret rule or not. Do not include anything other than the structure.
Active learning prompt
Now, do you think this structure follow the rule7An: {x) (xHnnnswer only yes or no. Give your best guess even if you uncertain. Do not explain. Just say yes no
Now, do you think this group of objects follow the rule?\n: {xAnAnswer only yes or no. Give your best guess even if you uncertain. Do not explain. Just say yes no
Prediction prompt
Table 12: Prompts used for vanilla, direct LLM method
30

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims we made in abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The last section discusses limitations.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
31

---

Justification:
We provide the formal proof of our proposition in the appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have fully described the algorithms used in the paper, with details included in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
32

---

Answer: [Yes]
Justification:
Code and data available
at htttps://githbb wttphhu...o./oopooittteoo tonp/topwaa//
com/
doing- eewrrniexxeiimtetseeeee and- sedd-revsing--aae rrrlliissgrrees///
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm ec/
public/guides/ Soseleese/CooeSuuuissiiiooooeoe sissincccyy)) for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https for more details.
//nips. //nips.cc/ppl wwcppbblc/guuiicuccccccccoeooopp wetggiddddeeeeeebbiniinnnn
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Algorithm details are deseribed in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The paper reports error bars most of them computed over 5 seeds.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
33

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have provided estimated OpenAl API cost for our experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: The research conducted in the paper conforms, in every aspect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
Our work foundational research and has no direct societal impact.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
34

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The creators of original data are all properly credited, and the license and terms of use of the data are explicitly mentioned and properly respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the Datasets.papersstt
package should be provided. For popular datasets, paperswi thcode. cww/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
35

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The code and data introduced in the paper is well documented.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [Yes]
Justification:
We have described our human study in details in the appendix.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [Yes]
Justification:
We did not anticipate any risks from participating in our study. IRB approval obtained.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
36


Reviews:
[{"review_id": "K3d2kFsNHu", "reviewer": "Reviewer_i693", "summary": "This paper describes a model of online construction of rules that explain data, where (a) hypotheses are expressed in natural language, and (b) they are updated by proposing experiments (inputs to the ground-truth rule). The method consists of an extension of SMC, where experiments and hypotheses revision are proposed by an LLMs; the best experiment is chosen by an approximate information gain criterion. The paper considers both fuzzy and hard rules, and finds that hard rules perform best in terms of task completion (and the model outperforms previous methods, especially those that propose a batch of hypotheses rather than inferring them online). Moreover, the fuzzy rules model best fits human behavior.", "questions": "* Are any of the 10 Zendo predicates particularly unlikely to be recovered? I'd be curious to see an example of the model recovering 'a red is bigger than all non reds', for instance, since this seems like a quite unlikely hypotheses to propose by itself.\n* Is the hardcoded random experiment proposer biased in any useful way? It would be good to describe it at least in the appendix. Again, for most of the Zendo rules, it'd guess that most single random experiments would be highly non informative.\n* In the result in Table 4, with a single proposal, there's nothing for the InfoGain reranking to do. So should the first column be interpreted as two random repeats of the same exact model?\n* Appendix A1: missing an integral sign in (10)?", "limitations": "Yes, adequately addressed.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "The paper is clear and well motivated. The setup of online inference, where the agent is allowed to propose experiments and observe results, is interesting and new in this line of work. Though the initial domains studied in this setup are simple, they allow us to understand the meat of the problem more clearly. I can imagine other more realistic domains, such as based on code debugging or data analysis, that might benefit from the ideas here.\n\nThe paper is careful in its choice of experiments. In Zendo and ActiveAcre, the paper compares with recent, relevant baselines: ReAct (interactive, but LLM only), batch inference (static), batch with refinement (as done in prior work), or online inference. It also accomplishes the goal of obtaining decent fit to human data, which previous related papers did not attempt to do, providing a candidate cognitive model besides the best performing model.", "weaknesses": "While the domains used here are interesting for a first study, they are still relatively simple. Most compelling would be to find a task with a richer space of hypotheses and experiments, perhaps also making better use of common sense priors embedded in LLMs.\n\nThe paper also doesn't provide much qualitative insight into what behavioral differences arise from doing online experiments compared to only doing batched inference, perhaps with refinement. While it might be hard to describe general differences, at least having a few example traces in the appendix (e.g., showing example scenarios where batched inference falls short, or outperforms the online model) would help us at least form some qualitative hypotheses on where the gains are coming from.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive feedback! Below, we address concerns and questions raised by the reviewer.\n\n> Concern 1: While the domains used here are interesting for a first study, they are still relatively simple.\n\nWe fully acknowledge this concern. \n\nOur work evaluates the models on traditional, well-studied Bayesian concept learning datasets to demonstrate that using natural language representation is beneficial even in domains where crafting a DSL is still possible and also to enable in-depth analyses.\n\nWe believe that our work is needed to convince the Bayesian concept learning community to consider using natural language representation, and it is also an important stepping stone towards fully wielding the power of natural language. \n\nWe do show the almost obvious flexibility of natural language representation in our “majority of blocks are red” experiment. We leave evaluations on more complex domains where natural language representation is a must for future work. \n\n> Concern 2: The paper also doesn't provide much qualitative insight into what behavioral differences arise from doing online experiments compared to only doing batched inference\n\nPlease see our global response where we provide actual hypothesis traces from our models. They will be included in our revised paper.\n\n> Question 1: Are any of the 10 Zendo predicates particularly unlikely to be recovered?  I'd be curious to see an example of the model recovering 'a red is bigger than all non reds', for instance\n\nThere are two Zendo rules (out of ten) that are unlikely to be recovered by our models: ‘a red is bigger than all non reds’ and ‘all are blue or small’. We note that human participants from Bramley et al. [1] also perform very poorly on these two rules, as shown in [1]'s Figure 4. \n\n> Question 2: Is the hardcoded random experiment proposer biased in any useful way? It would be good to describe it at least in the appendix. Again, for most of the Zendo rules, it'd guess that most single random experiments would be highly noninformative.\n\nWe did try to come up with a random experiment proposer that would produce useful experiments. Here is a description of it, which we will include in the appendix:\nSample number of cones uniformly from 1-5\nSample color, size, orientation, and groundedness uniformly\nSample number of touchings from geometric-distribution(p=0.3) - 1\nThen, sample random pairs of blocks and make them touch until we have the specified number of touchings. \n\n> Question 3: In the result in Table 4, with a single proposal, there's nothing for the InfoGain reranking to do. So should the first column be interpreted as two random repeats of the same exact model?\n\nThe rows in the first column still use different experiment proposers, so they are not random repeats.\n\n\n> Question 4: Appendix A1: missing an integral sign in (10)?\n\nThank you for pointing it out! We will fix it. \n\n[1] Bramley et al. 2018, Grounding compositional hypothesis generation in specific instances.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "x0pSUIzYf5", "reviewer": "Reviewer_C8N9", "summary": "The paper addresses the problem inferring rules and designing experiments based on them. To do so, 1) they propose representing rules in natural language, generated with LLMs. 2) Using Monte-Carlo algorithms to score them 3) Revising these rules and proposing new experiments with LLMs. They instantiate the problem in the domains of Zendo (binary rules) and ActiveAcre (abstract causal reasoning/ blicket detector style tasks). The authors then compare human inferences with LLM inferences to show what might explain human inferences and how their method can outperform humans in inferring underlying rules.", "questions": "See weaknesses.", "limitations": "The authors address limitations.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is well written, I enjoyed reading it!\n- The problem is well motivated, and the description of the methods is also clear.\n- The authors use good baselines to compare their method with. I liked the controls and conditions used.\n- I particularly liked the rule following analysis and visualizations presented in fig 4b\n- I also liked the discussion on using formal languages vs natural language to represent rules!", "weaknesses": "- I think the description of the baselines could be made clearer. Including the prompts in the main or using figures to describe them could be useful.\n- While I liked the discussion on formal languages vs natural languages, I would have liked to see a baseline where the model implements rules in a formal language. This need not be a DSL, but a way for the model to reason with an external symbolic solver. Eg: the language could be python (with pymc/stan/pyro), or webppl.\n    - Relatedly, [https://arxiv.org/abs/2402.17879](https://arxiv.org/pdf/2402.17879) seems like a relevant paper where rules are inferred and implemented in pymc\n- Finally, the two domains, while inspired from psychology experiments, are quite artificial. They did make it possible for the authors to do an in-depth analysis (with human responses and try different variations, analyse failures) but adding a more realistic domain the strengthen the contribution of the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive feedback! Below, we address concerns raised by the reviewer.\n\n> Concern 1: I think the description of the baselines could be made clearer.\n\nPlease see our global response where we provide more details of the baselines. They will be included in our revised paper.\n\n> Concern 2:  While I liked the discussion on formal languages vs natural languages, I would have liked to see a baseline where the model implements rules in a formal language. This need not be a DSL, but a way for the model to reason with an external symbolic solver. Eg: the language could be python (with pymc/stan/pyro), or webppl.\n\nWe did not consider such a baseline because:\n\n1. A pair of recent papers found that pure python generation underperforms NL->Python (Ellis ‘23 NeurIPS [1], Wang ‘24 ICLR [2])\n\n2. We compare with the Bramley et al. [3] Zendo model, which uses a formal language\n\nWe will mention these facts in the revision. \n\n> Concern 3: The two domains, while inspired from psychology experiments, are quite artificial. They did make it possible for the authors to do an in-depth analysis, but adding a more realistic domain the strengthen the contribution of the paper.\n\nWe fully acknowledge this concern. \n\nOur work evaluates the models on traditional, well-studied Bayesian concept learning datasets to demonstrate that using natural language representation is beneficial even in domains where crafting a DSL is still possible and also to enable in-depth analyses.\n\nWe believe that our work is needed to convince the Bayesian concept learning community to consider using natural language representation, and it is also an important stepping stone towards fully wielding the power of natural language. \n\nWe do show the almost obvious flexibility of natural language representation in our “majority of blocks are red” experiment. We leave evaluations on more complex domains where natural language representation is a must for future work. \n\n\n[1] Ellis 2023, Human-like Few-Shot Learning via Bayesian Reasoning over Natural Language.\n\n[2] Wang et al. 2024, Hypothesis Search: Inductive Reasoning with Language Models.\n\n[3] Bramley et al. 2018, Grounding compositional hypothesis generation in specific instances.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZKHBqxVuWk", "reviewer": "Reviewer_nwGu", "summary": "The authors tackle the problem of learning natural language hypotheses and collecting new experiments to enable hypothesis refinement. They propose a system that integrates LLMs and SMC/BOED to solve this problem; crucially, this method goes beyond prior work on inductive reasoning by allowing for experimentation to guide hypothesis refinement. In brief, they use a LLM to propose natural language hypotheses, SMC to reweight and resample these hypotheses based on how well they explain the data, and then use SMC’s weighted particle approximation to approximate the expected information gain and select experiments. \n\nThe authors then evaluate this approach on two task Zendo and ActiveACRE, showing improvements against previous inductive reasoning methods and showing some agreement between their proposed system and human behavior on these tasks.", "questions": "* Does LLM-SMC-S produce a consistent estimator of expectations of test functions wrt the normalized posterior (this is the guarantee enjoyed by vanilla SMC)? I think this probably follows from the properly weighted property and the fact that ratios of consistent estimators are consistent. It would be good if authors could comment on this in the paper though. \n* Does the procedure suffer from particle degeneracy? It’s fine if it does but since this is a considerable practical challenge of SMC, I think it’s worth commenting on this. \n* The derivation in A.1 has some typos. I think there’s a missing integral sign in line 10. \n* I think I would have liked to see some of the actual hypotheses proposed by the LLM in the main text. \n* I think it would be helpful to see Equation 6 is actually approximated explicitly in math. \n* Figure 2 was a bit confusing for someone who’s used to seeing plots in SMC papers where the circles correspond to particles; here I think the circles represent specific natural language hypotheses (e.g., 3 red balls) but at first I thought they represented particles.", "limitations": "The authors discuss limitations.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The work presents a principled, novel approach to an interesting problem. First, this work extends previous work on inductive hypothesis generation with LLMs by accounting for experimentation. Second, I think the design choices are quite practical, interesting, and creative. \n\n* I like the idea of considering natural language as the hypothesis space and using SMC and BOED for active experimentation/hypothesis revision is quite natural and well-motivated. SMC and BOED are well-established ideas but leveraging them for the task of natural language hypothesis generation and experimentation is new and the authors have solid approach to integrating LLMs for proposing hypotheses while still guaranteeing the correctness of their procedure. \n\n* The authors ablate the different design decisions of the system. \n\n* The presentation of the paper was mostly clear but I do have some suggestions.", "weaknesses": "I think the tasks do a fine job of evaluating the system but I'm not sure if they're the most compelling demonstrations of the value of natural language hypotheses/they might not fully utilize the power of natural language hypotheses. For example, it seems like a restricted DSL might be enough for some of these tasks; I do note that the authors do an analysis to show the benefit of natural language.\n\nAdditionally, there are a few ablations of the system that could strengthen the paper. \n\n* Ablations that illustrate the role of a natural language hypothesis space or at least some additional discussion of more realistic settings where this is a good design choice. \n\n* Number of particles/number of hypotheses \n\n* Some analysis of particle degeneracy would be helpful \n\n* How does a model with vision capabilities perform on this task (e.g., GPT-4 V)? \n\nI think the presentation can be improved a bit in some places. For example, Figure 4b could probably be replaced with a summary figure. It’s a bit difficult to parse a 6x10 grid of figures. \n\nFurthermore, it was sometimes hard to understand the details of some of the baselines (without reading the hypothesis refinement/search papers). For example, I think clearly defining the batched inference with refinement baseline (in math) would be helpful in the main text since it's a crucial method to understand.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive feedback! Below we address concerns and questions.\n\n> Concern 1: I'm not sure if they're the most compelling demonstrations of the value of natural language hypotheses/they might not fully utilize the power of natural language hypotheses. For example, it seems like a restricted DSL might be enough for some of these tasks\n\nWe fully acknowledge this concern. \n\nOur work evaluates the models on traditional, well-studied Bayesian concept learning datasets to demonstrate that using natural language representation is beneficial even in domains where crafting a DSL is still possible and also to enable in-depth analyses.\n\nWe believe that our work is needed to convince the Bayesian concept learning community to consider using natural language representation, and it is also an important stepping stone towards fully wielding the power of natural language. \n\nAs the reviewer pointed out, we do show the almost obvious flexibility of natural language representation in our “majority of blocks are red” experiment. We leave evaluations on more complex domains where natural language representation is a must for future work. \n\n> Concern 2: There are a few ablations of the system that could strengthen the paper.\n- Ablations that illustrate the role of a natural language hypothesis space or at least some additional discussion of more realistic settings where this is a good design choice.\n- Number of particles/number of hypotheses\n- Some analysis of particle degeneracy would be helpful\n- How does a model with vision capabilities perform on this task (e.g., GPT-4 V)?\n\nRealistic setting: more abstract inductive reasoning domains such as content recommendation and moral reasoning from the GATE paper [1]\n\nNumber of hypotheses: while not directly ablated, the number of hypotheses is indirectly tied to the number of LLM calls per iteration, which we performed an ablation study on\n\nDetailed analysis on LLM-SMC-S, which, we think, may be able to provide a probabilistic account of evolutionary algorithms, and extension to visual data through the use of multimodal LLM are both exciting future directions we plan to explore. \n\n> Concern 3: It was sometimes hard to understand the details of some of the baselines \n\nPlease see our global response where we provide more details of the baselines. They will be included in our revised paper.\n\n> Question 1: Does LLM-SMC-S produce a consistent estimator of expectations of test functions wrt the normalized posterior (this is the guarantee enjoyed by vanilla SMC)?  It would be good if authors could comment on this in the paper though.\n\nYes, it does. We will revise our paper accordingly.\n\n> Question 2: Does the procedure suffer from particle degeneracy? It’s fine if it does but since this is a considerable practical challenge of SMC, I think it’s worth commenting on this.\n\nFrom what we see during the experiments, particle degeneracy usually happens when the ground truth rule is discovered and included in the pool of particles. Since the ground truth rule particle has a much higher posterior than others, other particles tend to get killed. When the ground truth rule has not been discovered, the particles tend to still be diverse. \n\n> Question 3: The derivation in A.1 has some typos.\n\nThank you for pointing it out! We will fix it.\n\n> Question 4: I think I would have liked to see some of the actual hypotheses proposed by the LLM in the main text.\n\nPlease see our global response where we provide actual hypothesis traces from our models. They will be included in our revised paper.\n\n\n\n> Question 5: I think it would be helpful to see Equation 6 is actually approximated explicitly in math.\n\nEquation 6 is actually the approximation of Equation 4, and it can be computed exactly.\n\n> Question 6: Figure 2 was a bit confusing for someone who’s used to seeing plots in SMC papers where the circles correspond to particles\n\nThe figure does in fact represent particles with circles. The three red particles represent how there are three particles with the same hypothesis. The colors of these circles change when their hypotheses are revised.\n\n[1] Li and Tamkin et al. 2023. Eliciting Human Preferences with Language Models.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eYIMYaDTUk", "reviewer": "Reviewer_zQ3i", "summary": "This paper integrates a Large Language Model (LLM) within a Sequential Monte Carlo (SMC) algorithm, where the LLM functions as both the proposal distribution (revising existing particles) and the evaluator (assessing each hypothesis in light of new data). The authors applied this method to two cognitive psychology tasks: Zendo and ActiveACRE. These tasks involve formulating latent game rules, updating the probability of these rules based on new data, and proposing new experiments to gather further data.\n\nIn both domains, the Online, Hard variant of SMC consistently outperforms other SMC variants in identifying the best latent rule, while the Online, Fuzzy variant of SMC more accurately captures human data. However, compared to optimal experimentation (the InfoGain algorithm), none of the LLM-based proposers outperform the InfoGain algorithm. Furthermore, a random proposer outperforms the LLM-based methods, indicating a significant deficit in the LLM’s performance as a proposer.", "questions": "Lines 227-229: The authors claim that “Indeed, Table 2 shows that our best-performing model surpasses [13]’s model on human data […], while our model does not perform such parameter fitting.” However, the Online, Fuzzy method does contain free parameters (specifically, the two free parameters that determine the false-positive and false-negative rates for each hard rule) that can be adjusted to better fit human data. Did the authors actually fit these parameters using human data?", "limitations": "I believe the paper would be improved if the authors could independently comment on the three separate roles of the LLMs (i.e., proposer, evaluator, and active learner) in their proposed method.\n\nMinor comments:\nLine 224: “judgement” —> “judgment”", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "I find the proposed method interesting, as it effectively demonstrates the benefits of integrating Bayesian methods with LLMs. The comparison of various LLM-based methods for problem solving and explaining human data is particularly valuable.", "weaknesses": "The main weakness of the work is that both the enhancement in problem-solving capability and the correspondence with human data are not convincing. While the Online, Hard variant of LLM-based SMC clearly outperforms ReAct-style LLMs, it employs LLMs in three distinct roles: as a proposer (for revising existing hypotheses), as an evaluator (for translating natural language hypotheses into code and verifying them), and as an active learner (for proposing new experiments to perform next). The performance of the LLM as an active learner, however, is particularly weak, even underperforming a random active learner (see Tables 3 and 4). Replacing the active learner in the Online, Hard method with a random active learner would likely significantly improve performance compared to using the LLM. Consequently, it is unclear which aspects of the pipeline benefit from the inclusion of the LLM and which aspects are detrimental.\n\nSecond, the comparison between humans and models is superficial. The best-fitting Online, Fuzzy model clearly includes at least two free parameters, as detailed in Section 2.3. However, the authors did not report the values of these best-fitting parameters. Are these the only two free parameters? Did you add additional decision noise to the model to better capture human data? If so, it would be more informative to report scores such as BIC or cross-validation results rather than just the simple log likelihood of the model.\n\nFigure 7 aims to illustrate a bounded rationality model for human-likeness, suggesting that the Online, Fuzzy method best aligns with human behavior when the number of LLM calls per iteration is neither too few nor too many. However, the evidence for this claim appears weak. The bounded rationality pattern only emerges with the Online, Fuzzy method, and the authors tested only three different numbers of LLM calls per iteration (x-axis). With just three data points, it is difficult to conclusively demonstrate an inverted U-shaped relationship. Additionally, when LLMs were used with methods other than the Online, Fuzzy method, the inverted U-shaped relationship did not appear at all.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the detailed feedback!\n\n> Concern 1: The main weakness of the work is that both the enhancement in problem-solving capability and the correspondence with human data are not convincing.  It is unclear which aspects of the pipeline benefit from the inclusion of the LLM and which aspects are detrimental.\n\nWe would like to respond to this concern with two key points: \n1. LLMs are necessary for natural language hypothesis generation: \nAs discussed in the introduction, since the hypotheses are natural language, we need to leverage LLM in our proposer. Without LLMs, having natural language hypotheses would not be possible. We also compare our proposers against a non-LLM proposer of non-linguistic hypotheses from [1] in Table 2.\n2. Our experiments are controlled: \nWhile our work leverages LLM in three distinct components: the hypothesis proposer, natural-language-to-code transpiler, and active learner, we control our experiments by keeping the transpiler and active learner fixed for all methods in the main results to perform an isolated study on the hypothesis proposer and keeping the transpiler and hypothesis proposer fixed in the active learning ablation study (Table 3 and 4). \n\nWe address other points raised under concern 1 below.\n\n> Concern 1.1: While the Online, Hard variant of LLM-based SMC clearly outperforms ReAct-style LLMs, it employs LLMs in three distinct roles: as a proposer, as an evaluator, and as an active learner.\n\nIn the main results (Table 1 and 2, and Figure 4) where we show that the online models clearly outperform ReAct-style LLMs in both accuracy and human-likeness, all methods use the same transpiler and active learner.\n\n> Concern 1.2: The performance of the LLM as an active learner, however, is particularly weak, even underperforming a random active learner (see Table 3 and 4). Replacing the active learner in the Online, Hard method with a random active learner would likely significantly improve performance compared to using the LLM.\n\nWe believe there is a misunderstanding here. There are three different active learners: LLM, random, and InfoGain. InfoGain (introduced in section 2.2) involves choosing an experiment that maximizes information gain from a pool of candidate experiments proposed by the experiment proposer, which can either be LLM experiment proposer or random experiment proposer (an ablation study for the experiment proposer is done in table 4).\n\nOur work advocates for the InfoGain active learner which outperforms both LLM active learner and random active learner as shown in Table 3.\n\nAll results, except for Table 3 and 4, use InfoGain with LLM experiment proposer, which is the best active learner, as its active learner.\n\n> Concern 2:  The comparison between humans and models is superficial.\n\nWe believe this concern is caused by some misunderstandings. We clarify the misunderstandings below.\n\n> Concern 2.1:  The best-fitting Online, Fuzzy model clearly includes at least two free parameters, as detailed in Section 2.3. However, the authors did not report the values of these best-fitting parameters.\n\nWe would like to first clarify that the two noise parameters, $\\theta = (\\epsilon, \\delta)$ are not free parameters; they are random variables, as discussed in section 2 and section 2.3. Since they are random variables with priors on them, they are not optimized but rather get marginalized out as shown in Equation 2 and 3. \n\nHowever, the gaussian priors for the two noise random variables do have free parameters, namely the means and standard deviations. Thus, there are 4 free parameters in our fuzzy models and no free parameters in our hard models.\n\nAlthough in principle we could have fitted these 4 parameters, in practice, it is really expensive to do so because it involves lots of LLM calls to even evaluate a single parameter setting. Therefore, we did not fit the free parameters; we only tried two different parameter settings for all fuzzy models, and reported the results of the setting that worked best in the main paper and the results of the other parameter setting in the appendix (Figure 8). The same parameter setting is used across all fuzzy models and reported in the appendix (A.4). \n\n> Concern 2.2: It would be more informative to report scores such as BIC\n\nWe include here a table of BIC scores which we will include in our revised appendix:\n\n| Method  | BIC      |\n|---|---|\n| [1]’s best model            | 3085     |\n| Batch, Fuzzy                | 3352.93  |\n| Online, Fuzzy               | **2988.77** |\n| Batch, Hard                 | 5842.00  |\n| Batch w/ Refinement, Hard   | 6999.52  |\n| Online, Hard               | 10419.86 |\n\n> Concern 3:  The evidence for bounded rationality appears weak. When LLMs were used with methods other than the Online, Fuzzy method, the inverted U-shaped relationship did not appear at all.\n\nThe data provided in Figure 7 is meant to show that our most human-like model is consistent with the theory of bounded rationality, and we leave it for future study to thoroughly examine whether, or the degree to which, the model is boundedly rational.\n\nHowever, the fact that only our most human-like model, the online, fuzzy model, is the only model exhibiting the relationship is not a weakness; it is in fact the opposite: it helps strengthen our point. Specifically, we expect the inverted U-shaped relationship to appear only in models that sufficiently explain human data. We believe that other methods do not exhibit the inverted U-shaped relationship because they are not human-like enough to have the bounded rationality property.\n\n> Question 1: Contradicting to lines 227-229, the Online, Fuzzy method does contain free parameters that can be adjusted to better fit human data. Did the authors actually fit these parameters using human data?\n\nWe refer the reviewer to our replies to concern 2.1 which addresses this question.\n\n[1] Bramley et al. 2018, Grounding compositional hypothesis generation in specific instances.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all the reviewers for their detailed feedback. We have posted responses to each reviewer's individual comments. Here, we address some common concerns raised by the reviewers. 

> Common concern 1: Want to see qualitative differences, i.e., actual hypotheses proposed by the models

Below are example hypothesis traces of online, batch with refinement, and batch (without refinement) models on the Zendo task with the rule “a blue touch a red”. We will include them in our revised paper. 

- Batch: 'Blocks must touch at least one other block' is proposed but is immediately falsified by an existing experiment where a scene with no blocks touching is negative.
- Batch with refinement: ‘Blue blocks must not touch green blocks’ is first proposed and then immediately refined into ‘Blue blocks must not touch blocks of any color other than red’ since there is an existing data point where a scene with a blue touching a blue is negative. This hypothesis later gets falsified, without an opportunity to refine itself since the model is not online, when a scene with no blocks touching turns out to be negative. 
- Online: ‘There must be a blue block’ is proposed and added to the pool of particles. Since it has higher prior than other particles (has shorter length); it keeps surviving while others get killed, despite some conforming with the data. Upon seeing a scene with a blue touching a green being negative, the particle ‘There must be a blue block’ is perturbed into `there must be a blue block touching a red block’.

As you can see, the batch without refinement model lacks the ability to propose hypotheses that are consistent with existing data. The batch with refinement model fixes the issue but lets many of these almost-correct hypotheses get falsified without having a chance to refine itself upon seeing new experiments. The online models are the only models whose hypothesis proposal is affected by the prior. As defined by the prior, shorter hypotheses are preferred over longer hypotheses (the longer ones get killed), given that both explain data equally well. Once shorter hypotheses get falsified by a new experiment, they get perturbed into longer hypotheses that fit the data better.  

While in principle the batch with refinement model could also do the ‘there must be a blue block’ -> ‘there must be a blue block touching a red block’ transition like the online model, the influence of prior in hypothesis proposal and the ability to revise existing hypotheses upon seeing new experiments is what differentiates online models from batch with refinement models.

> Common concern 2: Baselines are not described in detail. It is hard to understand the baseline without reading prior work. 

We include more baseline details below and will include them in our revised paper. 

In probabilistic inference terms, both batch with and without refinement correspond to importance sampling $p(h|x_{1:t},y_{1:t}) = E_{p(h’|x_{1:t},y_{1:t})} [1[h = h’]] =  E_{q(h’|x_{1:t},y_{1:t})} [\frac{p(h’|x_{1:t},y_{1:t})}{q(h’|x_{1:t},y_{1:t})}1[h = h’]]$. 

The difference in the two baselines, batch without and with refinement, lies in how $q(h’|x_{1:t},y_{1:t})$ is constructed. 

- Batch: $q(h|x_{1:t},y_{1:t}) = U(LLM(x_{1:t},y_{1:t}))$ where $LLM(...)$ prompts an LLM to return a list of hypotheses 

- Batch with refinement: $q(h|x_{1:t},y_{1:t}) = U(Refined\text{-}LLM(x_{1:t},y_{1:t}, None, 0))$ where $Refined\text{-}LLM$ is defined as follows:

First, let $s(h, x_{1:t}, y_{1:t}) = \frac{1}{t}  \sum_{i=1}^t \mathbb{1}[h(x_i) = y_i]$. This simply scores what percentage of data points in $x_{1:t}, y_{1:t}$ that $h$ makes correct predictions.

function $Refined\text{-} LLM(x_{1:t}, y_{1:t}, h, k)$:

&nbsp;&nbsp;&nbsp;&nbsp; $H = LLM\text{-}with\text{-}h(x_{1:t},y_{1:t}, h)$ # Prompts LLM to refine h

&nbsp;&nbsp;&nbsp;&nbsp; if $k=K$:

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; return $\emptyset$

&nbsp;&nbsp;&nbsp;&nbsp; else if $\exists h’ \in H, s(h’, x_{1:t}, y_{1:t}) = 1$:

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; return {$h’ \in H | s(h’, x_{1:t}, y_{1:t}) = 1$}
    
&nbsp;&nbsp;&nbsp;&nbsp; else:
        
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; $h^* = argmax_{h’ \in H} (s(h’, x_{1:t}, y_{1:t}))$
        
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; return $Refined-LLM(x_{1:t}, y_{1:t}, h^*, k + 1)$

where $K$ is the number of refinements allowed.

> Common concern 3: The domains, while interesting and good for first study, are relatively simple

We fully acknowledge this concern.

Our work evaluates the models on traditional, well-studied Bayesian concept learning datasets to demonstrate that using natural language representation is beneficial even in domains where crafting a DSL is still possible and also to enable in-depth analyses.

We believe that our work is needed to convince the Bayesian concept learning community to consider using natural language representation, and it is also an important stepping stone towards fully wielding the power of natural language.

We do show the almost obvious flexibility of natural language representation in our “majority of blocks are red” experiment. We leave evaluations on more complex domains where natural language representation is a must for future work.

[1] Qiu et al. 2024, Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinement

[2] Wang et al. 2024, Hypothesis Search: Inductive Reasoning with Language Models.

Meta-review: The paper introduces a novel approach to learn natural language hypotheses and conduct active experimentation for hypothesis refinement. It leverages LLMs for hypothesis generation and evaluation within the framework of Sequential Monte Carlo (SMC) and Bayesian Optimal Experimental Design (BOED). Experiments on two cognitive psychology tasks demonstrate improvements over existing methods and some agreement with human behavior. 

The paper is strong because its method is technically principled and experiments are carefully carried out. 

The main weakness is that the experimental settings are restricted, thus not fully utilizing the power of LLMs. Reviewers have raised questions in this aspect, and called for more analysis. Authors explain that experiments in simple domains are "important stepping stone towards fully wielding the power of natural language." The rebuttal also includes some new results and analysis. k

I think the strengths of this paper overweigh its weaknesses, and thus recommend to accept it. But I strongly recommend the authors to clarify their technical motivation and experiment design, as well as include new results and analysis in the camera-ready.
Decision: Accept (poster)