Human-like Few-Shot Learning via Bayesian Reasoning over Natural Language
Authors: Kevin Ellis
Venue: NeurIPS 2023 oral (2023)
Keywords: Cognitive science
Bayesian
Language model
Induction
Psychology
Reasoning
TL;DR: We build a model of human concept learning by integrating language models with probabilistic reasoning

Abstract: A core tension in models of concept learning is that the model must carefully balance the tractability of inference against the expressivity of the hypothesis class. Humans, however, can efficiently learn a broad range of concepts. 
We introduce a model of inductive learning that seeks to be human-like in that sense.
It implements a Bayesian reasoning process where a language model first proposes candidate hypotheses expressed in natural language, which are then re-weighed by a prior and a likelihood.
By estimating the prior from human data, we can predict human judgments on learning problems involving numbers and sets, spanning concepts that are generative, discriminative, propositional, and higher-order.

Human-like Few-Shot Learning via Bayesian Reasoning over Natural Language
Kevin Ellis Cornell University kellis@cornell.co edu
Abstract
A core tension in models of concept learning is that the model must carefully balance the tractability of inference against the expressivity of the hypothesis class. Humans, however, can efficiently learn broad range of concepts. We introduce a model of inductive learning that seeks be human-like in that sense. It implements a Bayesian reasoning process where a language model first proposes candidate hypotheses expressed in natural language, which are then re-weighed by a prior and a likelihood. By estimating the prior from human data, we can predict human judgments learning problems involving numbers and sets, spanning concepts that are generative, discriminative, propositional, and higher-order.
Introduction
Human learning is rapid and broad. Consider a child learning a new routine like 'high-five:: given just 1 or 2 examples, they can learn that move, and even generalize to variations like low-fives. Or consider learning the basics of a game like Pacman, or an Al researcher learning a new technique mathematician extrapolating '1, 16, 64, .... In each of these cases, the relevant routine, rules, or concept can be learned from little data and only seconds to minutes of experience. Furthermore, the space of possible concepts effectively infinite, because concepts can compose to yield more complex constructs like 'high-five followed by a fist bump'. Thus key computational challenge to understand how intelligent system can acquire wide range of new concepts, given modest data, and granted a modest computational budget. Building Al systems that can efficiently master many concepts is also practically valuable, because data-efficiency and broad generalization remain some of the most salient gaps between human and machine intelligence [I]
like 'few-shot prompting',
Fundamentally, are concerned with the problem of induction: Inferring generalizable pattern, rule, trend, or law from specific examples. A classic approach to induction is to start with hypothesis space of possible concepts, and then probabilistically infer which hypothesis most likely generated the observed data using Bayes' Rule. This Bayesian paradigm has proved widely applicable across
both cognitive science and artificial intelligence 2 3
&
9 IO
12
13 14].
On its own, however, the Bayesian paradigm leaves important questions unanswered. Acquiring broad range of possible concepts demands expressive hypothesis space, but inference a rich space of concepts comes at steep computational cost. For increased expressivity, many Bayesian models design hypothesis spaces that resemble programming language [15 16 17 18
19 20). Posterior inference then corresponds to constructing high-probability programs. Each of these program-learning models requires custom domain-speciff programming language. Despite confining themselves to domain-specificc language, these models still require specialized inference machinery, such heuristically-desggne search moves [T0], exorbitant compute budgets 21).
Our goal is to build a model of humanlike concept learning that makes progress toward resolving the tension between intractable inference and expressive hypothesis classes. We propose new model
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

that expresses its concepts in natural language- even when the learning problem does not involve natural language. We do this for two reasons. First, language an effective representation for many human concepts. It is compositional, richly expressive, and regularizes the learner toward natural generalizations. Second, we find we can efficiently infer natural language concepts using modern based on the transformer architecture (24).
large language models 22 23
Like any Bayesian model, we will first define a prior over concepts, which in our case exerts a top-down pressure for naturally-structure language. Our model also has a bottom-up mechanism for efficiently inferring possible hypotheses, analogous to a recognition network [25]. The interaction of these top-down and bottom-up models is surprisingly effective as a model of humanlike learning: Given around 50 samples from the recognition network, our model can account for human patterns of generalization for concepts that are generative or discriminative, and propositional or first-order. We show the model can also capture fine-grained structure in human judgements: both subtle gradations of uncertainty, and also the dynamics of learning starting from few examples and going to dozens of examples. Finally, a key reason why humans can learn so efficiently is because they have good inductive bias prior 226] 27). Our model can fine-tune its prior to human judgements, effectively extracting& human-like prior from behavioral data. We find this gives a more faithful model of human generalization, and also improves average accuracy on concept-learning tasks.
We focus on abstract symbolic concepts, such as 'prime numbers less than 30', which we think are well-suited to natural language. We do not consider concepts grounded in perception and actuation, such as 'dog' or 'chewing', and hence do not provide single unified account of inductive learning.
However, for these abstract concepts, we provide a rational process model 28%. Following the Marr levels of analysis (29), this means propose algorithmic mechanisms for concept learning that rationally approximate optimal inference, subject to bounds on computation (sampling). This contrasts with computationall-eeel models, which characterize the goal of the learner, without committing to a theory of how the learner mechanistically accomplishes that goal [29]. Most Bayesian concept
learning models operate at the computational level, avoiding issues of intractability ( 15 (cf. (30).
We contribute (1) model of symbolic concept learning that supports efficient inference over a flexible hypothesis class; (2) an evaluation on human data from two different concept learning experiments; and (3) a simple recipe for extracting a humanlike prior over concepts, given raw behavioral data.
2 Background and Related Work
Bayesian Program Learning (BPL: (00) models concept learning as Bayesian inference over latent symbolic programs. BPL models first define a domain-specific programming language spanning space of possible concepts, and then infer a posterior over concepts C' given training data D via Bayes' Rule: PCIDD) о pl DDCC)p(C') Although the set of possible concepts remains hardcoded, the prior p(C') can be learned through hicrarchical Bayesian methods. Given parameters 0 indexing possible priors, and a collection of datasets D, the prior can be learned by approximately solving (2]
8 arg max p( () = arg max p(0) II EP(D())p((C) DED C
0
Bayesian models can learn from few examples, because the prior regularizes them toward reasonable generalizations. They also produce nuanced uncertainty estimates, because they represent the posterior. Program learners can also acquire any concept expressible in their domain-specifie language, but this language must be appropriately cireumscribed for inference to remain tractable. Systems such as DreamCoder (11 partly address this concern by growing the language and training neural networks to aid inference, but even then, ceneral-purpose programming languages remain out of scope. We next turn to latent language, which considers the broad hypothesis space of all natural language.
Latent Language. Using language as an internal representation for nonlinguistic tasks was introduced by the Latent Language framework [3] 32) These systems perform few-shot learning by searching for the language which minimizes a loss on the training examples. Given training onput-output examples {(z, y)}, the latent language approach infers the language C" minimizing
C* = = arggin Σ Loss(y, fo(;
CeE-
(a.y)
where fo a neural network and Σ* the set of all strings. Because there are infinitely many strings, another neural network samples finite pool of candidate concepts. Relative to Bayesian learners,

---

latent language models use maximum likelihood estimation to infer single concept, rather than construct posterior distribution. Like our approach, latent language uses language as an intermediate representation, combined with bottom-up concept proposal process. Our work adds additional Bayesian mechanics, and shows how to learn a prior on from human judgments. Learning the prior proves important for both modeling human judgments and achieving high task performance.
Induction, Abduction, and Deduction. We address inductive problems: inferring a general pattern from specific examples [3] Abduction is a related process where the reasoner infers an explanation for a specific observation. Abductive reasoning using modern language models has received much recent attention [3), which has solidified natural language promising candidate for representing abductive explanations. Deduction Deoiction--logicallly inferring the truth of proposition has also have been similarly revisited in the context of modern language models [36 37
Model
We start with basic Bayesian approach. A latent concept C' generates K observed examples, notated x Xx, according to an IID process. We abbreviate X,,..., Xx k as X1.- For our model, C'
an utterance in natural language. The learner' posterior beliefs given by Bayes's Rule,
p(C))ix) x p(C)
II p(Xa(C)
(1)
Isksk
The prior p( pC) comes from neural model. The likelihood p(X[C) is domain-specific because depends the structure of the examples. We assume the posterior in Eq. T intractable.
We model tasks that do not involve externalized language, thus beliefs over C' plaa a fundamentally auxiliary role. We instead care about the probability that test example Xiest belongs to the same concept as the training examples XI:K This posterior predictive quantity is
p( Xiest C(Xi:K) Σ (((]) I [Xrest c C]
To make the above tractable, introduce a proposal distribution o( (C]]X1:kc. We draw from q modest number of sampled concepts (tens to hundreds), writing those samples (()) ,.... C(S). By only considering concepts proposed by the infinite sum over C in Eq. becomes a finite sum over S samples. Provided those proposals account for much of the posterior mass, this is good approximation. Conventionally, q is used to construct an importance sampler [38]:
p(Xiest C)Xix))= C-p(1Xnx))
E
[Xtest E C]
Ey it)) and w(()) p(C(.) ) (xxx((()) af ((())
(((s)
20
Σ I c(+)], where (x) IsoSS
(3)
The above Monte Carlo estimate requires evaluating ((xx))))) The most powerful proposal dis- tributions disposal, such as GPT-4 (22), do not expose this functionality, we heuristically ap- proximate importance sampling by deduplicating the samples and weighing each by ((C))(XXx(()))))
p( Xiess C((x:k) 2 Ce(co) . (C)
Σ
(O) [X C], where
Ec wl(r and ih (C) (c) =((C))(xX000000
c(.]] (4)
w
Ultimately, the distribution p(C) should reflect the prior beliefs of human learners. To tune the prior to reflect human patterns of generalization, we assume access to a dataset of human judgments consisting of triples (XI:K) Xes,r)) r), where is the ratio of humans who judged X as belonging to the same concept XI-K- More generally, could be any average human rating in [0. If parametrizes the prior, then we match the prior to the human data by solving
arg max
Σ
rlogpo (Xieat 6 x)X (1 - r) log (1 Po Xiee E c(X
(5)
where po (Xrest E C(XIK)= I po po (CIXi.)). approximated via Eq.
As process model, the approach claims that a bottom-up generator proposes bounded number of candidate concepts, which are re-weighed by re-examining each datapoint, while being biased by
prior. This can be seen dual-process model [39]. well as model of bounded rationality [40].

---

Comparing hypotheses to data. The likelihood p(X]C) and the prediction 1 [X e C both require comparing natural language against nonlinguistic datum X. For the domains we consider it con- venient implement this comparison by translating the concept C' into Python using an I.MM-hich we deterministically, via greedy decoding- and and then execute the Python program on X, similar to A1 422. We do not view this latent program as essential to the general framework: Alternatvelly, another neural network could serve to link to C, which would allow fuzzier comparisons between hypotheses and data. Fig. [ diagrams these details as Bayesian network.
Concept
Program
Data X
k [K]
proposal q
Figure I: Model as Bayesian network. The program is deterministic function of C.
The Number Game
The Number Game a few-shot concept learning setup covered by classic textbooks and disserta- tions [5 43) Participants playing The Number Game are given few example numbers belonging to hidden concept, and then rate how likely it is that other numbers also belong to the same concept. Given just the example number 16, the concept could be 'square numbers', 'powers of two', 'evens', 'odds but also 16', '97 and 16', 'numbers ending 6', or infinitely many other concepts. With more examples such as 16, 8, 2, 64, humans consistently rate powers of two almost certainly in the concept, but gradations of uncertainty remain: other evens like 24 could plausibly belong in the concept, but humans rate odds like 23 as extremely unlikely. Examining human judgments for these and other concepts reveal a variety of belief states, including sharp, all-or-none concepts like 'powers of two,' but also soft graded concepts like 'numbers around 20' (Fig. blue bars).
Eight human participants rated test numbers on scale of 1-7, given different training sets of example numbers. We model the average human rating for each test number, on each training set example numbers.
Human Data. We take human data from 43
training examples: 16
between 30 and 45 between 15 and power of
M In nl
nlln
multiple of 16
less than 20
prob. test in concept
20
40
70
100
training examples: 16, 8, 2,
divisible by 8
power of 2 evenly divisible by 2 power of two power of
0
M
In mitliniiiiinn in
0
10
20
30
40
50
60
70
80
90
100
training examples: 16, 23, 19, 20
less than 30,
model
between and between 16 and 23 between 15 and 25 greater than 15 and le...
human
20
40
50
60
70
100
number
Figure 2: Number Game human judgments (blue bars) for different example data. For instance, the top plot shows that after seeing that 16 belongs to the concept, humans rate 64 as around 50% likely belong to that same concept. Bars zero correspond to missing data. Orange curves show our model's predictions. The text to the right of each plot shows samples from our model's 's approximate posterior after proposing 100 concepts using q. Human data from 43 See also Appendix Fig.

---

Prior. We consider two different prior distributions. The pretrained prior scores the log likelihood of each concept using an open source language model, specifically, CodeGen 350M 44. The tumed prior first extracts features of C' using pretrained sentence feature extractor o, specifically all-Minil 45), which outputs a 384-dimensional feature vector. The tuned prior maps those features to an unnormalized log probability via linear layer with parameters 0:
Tuned prior: PC(C') oc exp (C))
(6)
Likelihood. Evaluating p (X(C) requires first determining which numbers belong to the concept C. To efficiently enumerate those numbers, we translate C' from natural language to Python using Codex code-davinci-002 (23], a language model trained on source code. We run the Python code on the numbers 1..100 determine the members of C. Given the members of C, we assume numbers are sampled uniformly from C' with probability (1 c),, and otherwise sampled uniformly from 100:
1 1[X C] (1-1))(1-c))
E 100
(7)
|C|
Parameter fitting. We want single parameter setting that works for all of the training example sets, so we fit the above parameters to the average human judgment for each test number, and for
each set of examples. When comparing model predictions against the average human judgment particular test number, we always holdout that particular human data from the parameter fitting. We
use Adam (66 to perform maximum likelihood estimation of the parameters, following Eq. S
Temperature, Platt transform. Because human subjects rated on scale of 1-7, we introduce learnable Platt transform between the model's predicted probabilities and the human judgments 47). We also place learnable temperature parameter on the posterior.
Proposal distribution. We implement q using Codex code-davinci by adapting the cover story given the human subjects, then append the training example numbers XI.K and have it complete the natural language description of the hidden concept.
23
We prompt Codex
Alternative models. We compare against GPT-4 to understand the few-shot learning abilities of a bleeding-edge large language model. Prompted with example numbers belonging to a concept, together with test number, we measure the probability that GPT-4 responds "yes" to the test number belonging to the concept, vs responding "no". We fit a Platt transform to those probabilities. We also compare against DreamCoder (21). recent Bayesian Program Learning system. Starting with Number Game DSL,, DreamCoder learns a prior and trains a neural proposal distribution. DreamCoder considers 10 hypotheses at test time two orders of magnitude more than our model- and trains its neural network on 10" synthetic number concepts ("dreams"), We further contrast against Latent Language (11, using the same Codex-based proposal and likelihood distributions, and the same learnable Platt transform. Finally, we consider versions of our model that encode hypotheses in Python instead of natural language ('code prior'), as well version of our model that ablates the proposal distribution by generating concepts unconditioned on X1sK ('no proposal dist.').
Results. Fig. 3 shows that out-of-the-bul bbx- pretrained natural language prior offers decent fit to the human data after proposing 50-100 hypotheses. Our tuned prior achieves a very close fit to the human data, again using 50-100 50-100 samples. Switching from English to Python significantly degrades model fit, even when the Python prior is learned ("tuned code prior'). Ablating the proposal distribution- sampling hypotheses from the prior-also provides a poor fit: the space of possible number hypotheses is too vast to be randomly guessed without looking at the data.
These results establish that Imguage--gnneattin proposal distribution can support efficient inference, and accurately model the average of a small pool of human participants. Recall that we are modeling the average of 8 human judgements, and good fit this average requires 50-100 samples. Therefore. our model suggests that each human might only need to draw a couple dozen samples, which we think is psychologically plausible (and practical, from an engineering perspective). In contrast, the original Number Game model considered over 5000 hypotheses [43], and a typical BPL system such as DreamCoder still lags our model, even when it considers orders of magnitude more hypotheses.
Last, GPT-4's judgments are decidedly different from humans. This does not mean that GPT-4 is wrong in its predictions: there is no ground-truth for whether the number 30 should belong to the same concept as the number 60. To better understand how humans and models stack up against each other, next consider richer domain where accuracy can be more objectively measured.
5

---

model-humm Saallhhmann response R
tuned prior, 100 samples
1.00
tuned prior
1.00
0.75
tuned code prior no proposal dist.
pretrained prior
human rating
0.75
R==0,95
0.50
0.50
latent language DreamCoder
0.25
0.25
GPT-4
0.00
0.00
10
10
10
0.0
0.5
1.0
num samples
model prediction
Figure 3: How well different models predict held-out human judgments (R²), as function of the sampling budget (left panel, X-axis, log scale). Error bars: =SEM over rums with different seeds.
Variance across runs decreases with number of samples. See Appendix Fig. g g for further results.
Logical Concepts
We next consider concepts with more complex logical structure. Consider a concept such as bachelor, defined as "unmarried man", or Valedictorian, defined as "the person, within single school, with the highest GPA". Using the primitives of propositional logic, we can define bachelor: (Male^-Married Using the more expressive language of first-order logic, which includes quantifiers, we can define valedictorian as Valedictorian x)) = (Vy: School School(y) = GPA(z) (z) 2 GPA(v)) Discovering the discrete logical structure that best explains dataset of examples is well-known Al challenge [48 49 50). Within the cognitive sciences, understanding how people come to grasp logical relations has been proposed to be key component of understanding how people comprehend number systems, geometry, causal processes, social and kinship relations, and other domains [51 52 53 54).
For our modeling, we consider an online learning setup from (55] where learner observes stream of examples of unknown logical concept. On each example, the learner observes a fresh batch of 1-5 objects, and must pick out which objects belong to the hidden concept. The learner then gets feedback on which objects in the batch actually belonged to the concept, and then the process repeats for new batch. Fig. iillustrates this experimental setup: each object is shape defined by its size (small, medium, large), color (green, yellow, blue), and shape (triangle, rectangle, circle). Recording each human response to each shape on each batch gives fine-grained learning curve capturing how learning unfolds over dozens examples. These learning curves signal what concepts people readily learn, what patterns of mistakes they make, and what concepts remain essentially unlearnable from even dozens of examples. We obtain this human data from [55], which covers 12 concepts, collecting judgements from 1,596 human participants they attempt to learn each concept over 25 batches of examples. These 25 batches corresponds to s examples/concept, and each concept run on 20 participants. Most human learning happens the first dozen batches, 15/25 batches, which conveniently also nearly halves the cost and time running the model.
take the first
The wudsy objects in each surrounded by square:
Example
Example
Given the above examples, which of the objects in this set are wudsy?
O
O
O
Orw
Ni
O O Nm
Figure 4: Concept learning experiment from 55 (illustration used with permission). On each batch, participants label which shapes they think belong new concept (called wudsy). Previous batches are shown with the ground truth positive examples surrounded by a square. From these examples, participants might infer a simple concept like "green triangles", and select the second test object.

---

Model. Our modeling approach is similar to The Number Game, except we now have a discriminative learning problem instead of generative one, and an online learning setup where the learner observes stream of examples. To model online learning, we draw fresh proposals from & for each training batch, and perform Bayesian inference over all proposals drawn so far. To model discriminative learning, each example is now triple (B, T, Y), where B is a batch of shapes, 7 is test shape in that batch, and Y zero or one depending on whether T B is an example of the concept.
Our likelihood model assumes human subjects predict according to C with probability (1-e) and otherwise predict randomly with base rate a of labeling an example as positive. Following [55 56
we model a memory decay process where the relative importance of earlier observations falls off according to a power law with parameter $::
p(Y log logp(X (B).TT.Va)eX.x
C7 co
(8)
Σ
K log
(9)
As before, we translate hypotheses from natural language into Python using code-davinci-002,, and evaluate the likelihood term above by running Python code. We consider both pretrained and tuned priors. Our proposal distribution q comes from running GPT-4 on prompts that illustrate previous batches, either as truth tables (for propositional concepts) or as a raw list of previous observed batches (for higher-order concepts). We again place a learnable temperature on the posterior.
Bayesian Program Learning Baseline (BPL). We contrast with a strong BPL baseline. It uses grammar over expressions in first-order logic, plus predicates for shape, color, and size, totaling 28 primitives that were selected by the creator of the logical concept learning dataset (A.3.3). The BPI. baseline uses the same memory-decay likelihood model, and fits (tunes) its prior by estimating probabilities for each logical primitive. It is implemented in Fleet [57], the state-of-the-an fast parallelized MCMC over grammatically structured hypothesis spaces.
Our model differs in two important ways. First, the baseline is given first-order primitives that were chosen specifically for this dataset. While our model can use natural language expressing first-order concepts (e.g., the only for A!), can also express concepts like objects with the least common color that are unrepresentable by the baseline, and which are not in the dataset.
The second difference is that model supports efficient inference via bottom-up proposals, while this baseline performs a stochastic search over hypotheses (MCMC), requiring many more samples. takes 10 Metropolis-Hastinne which are deduplicated to yield & 45, 000 unique concepts, which provide the support of subsequent posterior inference. This means every BPL posterior informed by the total 10 sampling moves.
proposals per batch, and per learning curve, totaling 10 proposals,
Results. Our model's predictions generally align well with human judgments (Fig. 5). Using 100 proposals per batch, our model explains 81% % of the variance in human responses (R* .81,, which is much higher than GPT-4 on its own. The model also more accurate at the actual task than GPT-4, and within 3% of human-level (Fig. (C). The model also fits the human data somewhat better than the BPL baseline, even when that baseline is granted an exorbitant sampling budget. Tuming the prior proves critical, possibly because these logical concepts
complex than the earlier number concepts, and so can be expressed in a wider variety of syntactic forms. The pretrained model is highly sensitive syntax and cannot learn attend to semantic features, unlike the tuned model.
tuned prior, 100 samples
model-human response RD
1.0
tuned prior
Rt=0.81
pretrained prior
human response
no proposal dist.
0.6
latent language
GPT-4
0.5
BPL
(10*10* samples)
0.2
0.0
10a
10a
10a
0.0
0.5
1.0
num samples per batch
model response
Figure 5: Model fits on holdout data. Error bars: +SEM over 3 runs. (Error bars often close to zero)
7

---

Although our model explains most of the variation in the human responses, nontrivial variation remains. One possible reason is that we are modeling the responses of many human subjects, and different subject groups per concept, so it is difficult to capture the full variability of the human responses: Building a model that accurately reflects the judgments of population of humans may be much more difficult, and require higher sampling budget, than building a model of single person. The model also slightly underperforms humans, and so a higher fidelity model might come from simply performing the task better. (Although one can slightly surpass human accuracy by optimizing purely for task performance, doing so degrades model fit. See Fig. Fig.6C 'tune for accuracy") .) More fundamentally, the data come from many successive trials, so they likely contain memory and order effects such as garden garden-pathing [58] and anchoring [39 which are not well accounted for by the pure probabilistic inference model approximates, nor by simple memory decay model.
At the same time, our model does account for many fine- one-grained details in the behavioral data, including predicting specific patterns of successes and failures (Fig. (A).. Because our approach manifestly interpretable-itt explicitly verbalizes its hypotheses in human language-we can inspect its maximum posteriori concept at the end of each learning episode, and observe that its successes typically occur because its verbalization of the concept describes the correct first-order law. Conversely, when humans make highly selective failures, we can probe the model to suggest what alternative hypotheses humans may have incorrectly inferred (Fig. 6A, top right).
A
VyES. y.shape=x.shape - x.sizezy.size it is the largest object with its shape in the example
x.color = blue A vyES. x.sizezy.size is the largest blue object in the example
x.shape circle x.coeor=blue a circle (unless it is blue).
1.0
accuracy
W
human tuned prior
0.5
0.0
11
1111
111
10
20
30
40
0
10
20
30
40
50
0
10
20
30
40
response number
response number
response number
B
EYES. x.size > y.size it is larger than the smallest object in the example
By ES. (xzy A y.shape x.shape) By (xzy A y.color = x.color) it shares the same shape the total number of objects with another object in the example
the same color is odd
1.0
accuracy
0.0
10
20
30
10
20
30
40
10
20
30
40
50
response number
response number
response number
the total number objects with its color is greater than the total number of objects with any other color
the total number of objects its color is less than or equal to the total number of objects of other color
0.8
0.79
average accuracy
0,74
R'=0.74 74
0.7
1.0
accuracy
Jal
0.6
0.5
BPL
D.0
0.5
human
tune prior
pretrain prior
GPT-4
tune for accuracy
20
40
60
80
20
40
60
80
response number
response number
Figure 6: A. Left and middle: learning concepts isomorphic to bachelor and valedictorian. Dark ticks delimit batches. Above each plot is the ground-truth logical expression and the predicted natural language description. Right: the model can explain human mistakes by verbalizing what other concept humans might have been misled into learning. Both humans and the model seem to have learned largest blue instead of the largest and it also happens to be blue. B. Further learning curves, including somewhat worse fits, and getting the right answer despite having a slightly odd solution (rightmost). C. The model is close to human performance after extracting a prior from human judgments. D. New concepts run on new participants. All results on holdout learning curves.
8

---

Modeling new concepts. To test the flexibility of our model, we created two new concepts for evaluating both the model and a new group of human participants. These two concepts were shapes with the majority color, and shapes with the least common color. 16 human subjects participated in an IRB-approved study (A.2). Our study finds that humans rapidly learn these concepts (Fig. GD.)
We also test our model on these new concepts, but using the prior estimated earlier on the data in [59]. Our model correctly predicts that humans will learn the concept of majority color after just 2 batches of examples. It predicts learning minority color after 4 batches, while humans need just 3, but the primary result- that this concept is very learnable holds for both humans and the model.
Although these two new concepts are easy to explain in words, and could be expressed in first-order logic with the right primitives set cardinality and quantification over cilorr-neitherr concepts are learnable by the BPI. baseline. This is because both concepts are simply unrepresentable: despite being equipped with 28 primitives, those primitives were designed without anticipating these new concepts. This highlights the fact that is difficult to preprogram a sufficiently broad set of primitives that can efficiently encode all the different concepts people might learn.
6
Discussion
Putting humanlike inductive biases into machines. Humans excel at rapidly mastering new tasks and understanding new concepts in part because they have a good inductive bias prior [60, 61]. Imparting similar priors upon machines is therefore an important problem. Our work gives a recipe for training such prior by fitting it directly to human judgments, marginalizing out the natural language, meaning we never need to elicit natural language from human subjects. Because our approach simply tunes a small network top of large open-source pretrained model, and because it can be fit to raw human judgments, we hope that it can serve as a broadly applicable engineering strategy for extracting and using human priors. Recent work has also explored complimentary strategies for instilling a humanlike prior upon a machine learning system. For example, Kumar et al. 2022 [9] show that training neural networks with auxiliary linguistic tasks, and with auxiliary programming tasks, causes their representations to better align with human priors.
Bayesian Program Learning (BPL). Our model has similar motivations to BPL. Because translate the natural language into Python to compute the likelihood, it is possible to see our approach as BPL with unusual prior: (programm) ΣnL (NL))((rogrrmm)) provides a new prior for BPL,, together with the demonstration that it is possible and practical to do BPL over during-compleds language like Python. Relatedly, recent BPL modeling has found synergies between natural language and program representations for cognitive Al models g 62].
In that sense, work
Beyond BPL, combining probabilistic reasoning with expressive symbolic representations has long been an appealing paradigm 63 64 65), although the expressivity of the symbolic language must be balanced against the tractability of inference [66] 67). Guiding inference with neural model is a natural choice, but this is hard because of the lack of natural training data (though synthetic data can help: 668 69 70). Encoding knowledge in natural language allows pretrained neural models to guide inference, and it could be fruitful to examine statistical-relations AI (7]) in light of that fact.
Large Language Models. Our work suggests that an oft-of-the-boo large language model is not an effective approach to inductive reasoning, at least on its own. Bayesian mechanics are needed to dampen the unpredictability of the language model. To the extent that being Bayesian is a normative account of rational behavior, our work offers a framework for enabling language models to draw more rational inferences, and ultimately generalize in more predictable and human-like ways by tuning their prior beliefs to match human data.
The Language of Thought. Our work connects to the Language of Thought Hypothesis [72). which says that human learning and thinking relies on an inner symbolic language. This has been a productive framework for computational modeling [15] 16 18 73). In its most literal forms, sfnguage-of-thouggee models are afflicted by the curse of a compositional mind (Spelke 2022 (74): the free-form recombination of concepts yields a combinatorial explosion, which here we address by using pretrained knowledge from language to guide the learner. Whatever the true Language of Thought looks like, however, it must have wide array of composable basic concepts in order to explain the breadth, flexibility, and generality of human thinking. Natural language, even if is not actually the same as inner mental language, acts as vast reservoir of human concepts,

---

and provides flexible algebra for combining them. Therefore a reasonable near-term strategy for modeling human thinking may be use natural language as heuristic approximation to an inner Language of Thought, despite strong evidence that human thinking need not engage language- processing brain regions 775
Rational Process Modeling, As a rational process model, our account of concept learning bridges the computational and algorithmic levels of the Marr hierarchy [28] 29 We commit to a Bayesian computational--eve theory, and particular Monte Carlo algorithm as a rational approximation to that theory. One important sense in which our account is inadequate is that we do not actually explain how the bottom-up process works, or how it came to be learned. We merely require that it is stochastic and unreliable, but occasionally correct enough to not need many millions of proposals. Given those requirements. a modern large language model is reasonable surrogate for this bottom-up process, even if it its inner workings might differ greatly from human bottom-up proposal processes.
Generalizability of the theoretical framework. The hasics of the model make few commitments, yet instantiating requires selecting specific language models, engineering prompts and likelihoods, etc. More broadly, a high-resolution cognitive model, particularly a structured Bayesian one, requires domain-specific modeling choices. How much credit should we assign to the general theoretical framing, as opposed to particular engineering decisions? Although our paradigm introduces new degrees of freedom, such as which LLMssprompts use, it removes others, such as the grammatical structure of the symbolic hypothesis space. On balance, we are cautiously optimistic that the framework will generalize with less domain cppain-specific tinkering, at least for abstract symbolic domains. because it replaces hand-designed symbolic hypothesis spaces with pretrained neural models, and because reasonable "default" neural networks worked well across our experiments.
Inference, Natural Language, and Recursive Reasoning. There is a rich literature on natural language and probabilistic inference from a social or communicative perspective, such as modeling pragmatic inference [76 77 78, (9). These models often use a recursive reasoning formulation where each agent represents the beliefs of other agents, including beliefs about themselves, and so on, recursively. Our work does not consider the communicative function of natural language, and so does not engage with these issues. However, models of pedagogical inductive learning where a helpful teacher selects the training examples- teacher and learner must reason about each other's beliefs [D) These insights could be important for building versions of our model that might simulate aspects of how humans learn from each other.
from similar forms of recursive reasoning, because the
Limitations. Our model performs induction via discrete structure learning. Given the combinatorial difficulty of structure learning, it is unclear whether our approach can scale to inferring complex systems of symbolic rules. We believe recent work on iteratively refining language model outputs may be promising here [8] (2) The present form of our model is also limited to processing discrete symbolic input-outputs. Actual human thinking connects with the messy perceptual world. It would be valuable to understand whether this limitation can be addressed using multimodal language models [83] or approaches that interface separate language and vision modules 84
83 It is worth
noting that BPL models can straightforwardly interoperate with perceptional data 10 16 18. and
that many outstanding challenge problems within Al have at their core a perceptual- process, such as Bongard problems [86] and the Abstraction and Reasoning Corpus ①1.
Currently, the model relies on costly, energy-intensive models for its proposal distribution, constraint that might be mitigated by open-source models and network compression [8].
Last, strong justification for formal representations is that they allow specifying knowledge precisely and unambiguously [88]. Natural language is usually imprecise and ambiguous, which we deferred addressing by translating language into Python. It remains to be whether language models can be coaxed into producing sufficiently precise language to support representing knowledge solely in natural language, or if refinement into precise languages like Python offers the better scaling route.
Code and data available at: https//ggiuubboommmlllll///aaaa fewpannnniikeefshotttoooo fewshottlearning
Acknowledgements. We are grateful to Steven Piantadosi for providing the raw human data and Fleet results for the logical concepts, as well Joshua Tenenbaum for providing his Number Game data, and Mathias Sablé-Meyer for comments on the manuscript and work.
10

---

References
[1] François Chollet. On the measure of intelligence, 2019.
[2] Nick Chater and Mike Oaksford. The probabilistic mind: Prospects for Bayesian cognitive science. Oxford University Press, USA, 2008.
[3] Joshua B Tenenbaum, Charles Kemp. Thomas L Griffiths, and Noah D Goodman. How to grow mind: Statistics, structure, and abstraction. Science, (30022):1279-11285 2011.
[4] Ray J Solomonoff. A formal theory of inductive inference. Information and control, 7(1):1-22, 1964.
[5] Kevin Murphy. Machine learning: a probabilistic perspective. MIT press, 2012.
David JC MacKay, Information theary, inference and learning algorithms. Cambridge university press, 2003.
Dilcep George, Wolfgang Lehrach, Ken Kansky, Miguel Lázaro-Gredilla, C. C. Laan, Bhaskara Marthi, Xinghua Lou, Zhaoshi Meng. Liu, Hua-Yan Yan Wang, Alexander Lavin, and Scott Phoenix. A generative vision model that trains with high data efficiency and breaks text-based captchas. Science, 358, 2017.
[8] Zoubin Ghahramani. Probabilistic machine learning and artificial intelligence. Nature, 521(7533):452. 2015.
Sreejan Kumar, Carlos G Correa, Ishita Dasgupta, Raja Marjich, Michael Hu, Robert D. Hawkins, Jonathan Cohen, Nathanicl Daw, Karthik Narasimhan, and Thomas L. Griffiths. Using natural language and program abstractions to instill human inductive biases in machines. In NeurlPS, 2022.
[10] Brenden Lake, Ruslan Salakhutdinov, and Joshua B Tenenhaum. Human-level concept learning through prohahilistic program induction. Science, (122666666):33213330 2015.
[11] Kevin Ellis, Adam Albright, Armando Solar-Lezama, Joshua Tenenbaum, and Timothy D'oonelll. Synthesizing theories of human language with bayesian program induction. Nature communications, 13(1):5024, 2022.
[12] Pedro A Tsividis, Joao Loula, Jake Burga, Nathan Foss, Andres Campero, Thomas Pouncy, Samuel J Gershman, and Joshua B Tenenbaum. Human-level reinforcement learning through theory-based modeling, exploration, and planning. arXiv preprint arXiv:2107. 2021.
[13] Andrew G Wilson and Pavel Izmailov. Bayesian deep learning and a prohabilistic perspective of general- ization. Advances in neural information processing systems, 33:4697-4708, 2020.
[14] Noah D Goodman, Joshua B. Tenenbaum, and The ProbMods Contributors. Probabilistic Models of Cognition. http://probmods... org/v2 2016. Accessed: 2023-9-23.
[15] Steven Thomas Piantadosi. Learning and the language of thought. PhD thesis, MIT, 2011.
[16] Goker Erdogan, lIker Yildirim, and Robert Jacobs. From sensory signals to modality-indepenndee conceptual representations: A probabilistic language of thought approach. PLoS computational biology. 11(11)=e004610, 10, 2015.
[17] Mathias Sablé-Meyer, Kevin Ellis, Josh Tenenbaum, and Stanislas Dehaene. A language of thought for the mental representation of geometric shapes. Cognitive Psychology, 139:101527, 2022.
[18] Lucas Tian, Kevin Ellis, Marta Kryven, and Josh Tenenhaum. Learning abstract structure for drawing by efficient motor program induction. Advances in Neural Information Processing Systems, 33:2686-2697, 2020.
[19] Feras A Saad, Marco F Cusumano-Towwa Ulrich Schaechlle, Martin C Rinard, and Vikash Mansinghka. Bayesian synthesis of probabilistic programs for automatic data modeling. Proceedings of the ACM on Programming Languages, 3PP 2019.
[20] Percy Liang, Michael I. Jordan, and Dan Klein. Learning programs: A hierarchical bayesian approach. In ICML, 2010.
[21] Kevin Ellis, Catherine Wong, Maxwell Nye, Mathias Sablé-Meyer, Lucas Morales, Luke Hewitt, Luc Cary, Armando Solar-Lezama, and Joshua B. Tenenbaum. Dreamcoder: Bootstrapping inductive program synthesis with wake-sleep library learning. In PLDI, 2021.
[22] OpenAL. Gpt-4 technical report, 2023.

---

[23] Mark Chen, Jerry Tworek. Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, al. Evaluating large language models trained on code. arXiv preprint arXiv: 03374, 2021.
[24] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, F. ukasz Kaiser, and Illia Polasukhin. Attention is all you need. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors,Avvance Advances in Neural Information Processing Sysiems, volume 30. Curran Associates, Inc.. 2017.
[25] Geoffrey E Hinton, Peter Dayan, Brendan Frey, and Radford M Neal. The "wake-sleep" algorithm for unsupervised neural networks. Science, 268(5214):
1995.
[26] Elizaheth S Spelke and Katherine D Kinzler. Core knowledge. Developmental science, 10(1))89-96, 2007.
[27] Brenden M Lake, Tomer D Ullman, Joshua B Tenenbaum, and Samuel J Gershman. Building machines that learn and think like people. Behavioral and brain sciences, 40:c253, 2017.
[28] Thomas L Griffiths, Edward Vul, and Adam N Sanborn. Bridging levels of analysis for probabilistic models of cognition. Current Directions in Psychological Science, (114))263-2266 2012.
[29] David Marr. Vision: computational investigation into the human representation and processing of visual information. MIT press, 2010.
[30] Thomas L. Griffiths, Falk Lieder, and Noah D. Goodman, Rational use cognitive resources: Levels of analysis between the computational and the algorithmic. Topics in Cognitive Science, 7(2):217-229, 2015.
[31] Jacob Andreas, Dan Klein, and Sergey Levine. Learning with latent language. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long Popers), pages 2166-2179, 2018.
[32] Pratyusha Sharma, Antonio Torralba, and Jacob Andreas. Skill induction and planning with latent language. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1713-1726, 2022.
[33] David Hume. An enquiry concerning human understanding. 1748.
[34] Chandra Bhagavatula. Ronan Le Bras, Chaitanya Malaviya, Keisuke Sakaguchi, Ari Holtzman, Hannah Rashkin, Doug Downey, Scott Wen-tau Yih, and Yejin Choi. Abductive commonsense reasoning. arXiv preprint arXiv: 057399 2019.
[35] Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, and Yejin Choi. COMET: Commonsense transformers for automatic knowledge graph construction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4762 4779, Florence, Italy,
July 2019. Association for Computational Linguistics.
[36] Jachun Jung, Lianhui Qin, Sean Welleck, Faeze Brahman, Chandra Bhagavatula, Ronan Le Bras, and Yejin Choi. Maieutic prompting: Logically consistent reasoning with recursive explanations. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1266-1279, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics.
[37] Eric Mitchell, Joseph Noh, Siyan Li, Will Armstrong. Ananth Agarwal, Patrick I Liu, Chelsea Finn, and Christopher Manning. Enhancing senfsconni and performance of pre-trained language models through natural language inference. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1754-1768, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics.
[38] Christopher M. Bishop. Pattern Recognition and Machine Learning. Springer- New York, Inc., 2006.
[39] Daniel Kahneman. Thinking, fast and slow. maemillan, 2011.
[40] Herbert A Simon. A behavioral model of rational choice. The quarterly journal of economics, pages 99-118, 1955.
[41] Luyu Gao, Aman Madaan, Shuyan Zhou, Uni Alon, Pengfei Liu, Yiming Yang. Jamie Callan, and Graham Neubig. Pal: Program-aides language models. In International Conference on Machine Learning, pages 10764-10799 PMLR, 2023.
12

---

[42] Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W Cohen. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks, arXiv preprini 12X.522211.12588 2022.
[43] Joshua Brett Tenenbaum. Bayesian framework for concept learning. PhD thesis, Massachusetts Institute of Technology, 1999.
[44] Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. Codegen: An open large language model for code with multi-turn program synthesis. arXiv preprint, 2022.
[45] Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics. 11 2019.
[46] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.9980, 2014.
Proceedings of the 22nd international conference on Machine learning, pages 625-632. 632,, 2005. [47] Alexandru Niculescu-Min Mizil and Rich Caruana. Predicting good probabilities with supervised learning. In
[48] Stephen Muggleton. Inductive logic programming. New generation computing, 8:295-318, 1991.
[49] Andrew Cropper, Sebastijan Dumančié, Richard Evans, and Stephen H Muggleton. Inductive logic programming 30. Machine Learning, pages 1-26, 2022.
[50] Stanley Kok and Pedro Domingos. Learning the structure of markov logic networks. In Proceedings of the 22nd international conference on Machine learning, pages 441-448, 2005.
[51] Charles Kemp and Terry Regier. Kinship categorics across languages reflect general communicative principles. Science, 336(6084):0049-1000 2012.
[52] Marie Amalric, Liping Wang. Pierre Pica. Santiago Figueira, Mariano Sigman, Stanislas Dehaene. The language of geometry: Fast comprehension of geometrical primitives and rules in human adults and preschoolers. PLoS computational biology, 13(1 (0())cc00 2017.
[53] Steven T Piantadosi. The computational origin of representation. Minds and machines, 31:1-58, 2021.
[54] Lisa Feigenson, Stanislas Dehaene, and Elizabeth Spelke. Core systems of number. Trends in cognitive sciences, 8(7):307-314, 2004.
[55] Steven Piantadosi, Joshua B Tenenbaum, and Noah D Goodman. The logical primitives of thought: Empirical foundations for compositional cognitive models. Psychological review, 123(4):392, 2016.
[56] John R Anderson and Lacl J Schooler. Reflections of the cnvironment in memory. Psychological science, 2(6):396-408, 1991.
[57] Steven Piantadosi. Fleet,
2023. [Online GitHub repository]. J.
[58] Maryellen C MacDonald. Probabilistic constraints and syntactic ambiguity resolution. Language and cognitive processes, 9()::157-211, 1994.
[59] Steven T Piantadosi. The computational origin of representation and conceptual change. 2016.
[60] Jeanne Ellis Ormrod. Human learning. Pearson Higher Ed, 2016.
[61] Charles Kemp and Joshua B Tenenbaum. The discovery of structural form. Proceedings of the National Academy of Sciences, 105(31):106877 10692, 2008.
[62] Catherine Wong, Kevin Ellis, Joshua B. Tenenbaum, and Jacob Andreas. Leveraging language to learn program abstractions and search heuristics. In International Conference Machine Leurning, 2021.
[63] Taisuke Sato and Yoshitaka Kameya. Prism: language for symbolic-staiistical modeling. In IJCA, volume 97, pages 1330- 1339, 1997.
[64] Matthew Richardson and Pedro Domingos. Markov logic networks. Machine learning, 62:107-136, 2006.
[65] Robin Manhaeve, Sebastijan Dumancic, Angelika Kimmig. Thomas Demeester, and Luc De Raedt. Deepproblog: Neural probabilistic logic programming. In NIPS, 2018.
13

---

[66] Pedro Domingos and William Webb. A tractable first-order probabilistic logic. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 26, pages 1902-1909, 2012.
[67] Noah D. Goodman, Vikash K. Mansinghka, Daniel M. Roy, Keith Bonawitz, and Joshua B. Tenenbaum. Church: language for generative models. In UAI, pages 220-229 2299,22920000 2008.
[68]
Jacob Devlin, Jonathan Ucsato, Surya Bhupatiraju, Rishabh Singh, Abdel-rahman Mohamed, and Pushmeet Kohli. Robustfill: Neural program learning under noisy i/o. ICML, 2017.
[69] Tejas D Kulkarmi, Pushmect Kohli, Joshua B Tenenbaum, and Vikash Mansinghka. Picture: A probabilistic programming language for scene perception. In CVPR, 2015.
[70] Tuan Anh Le, Aulmm Güneş Baydin, and Frank Wood. Inference Compilation and Universal Probabilistic Programming, In AISTATS, 2017.
[71] Lise Getoor and Ben Taskar. Introduction to statistical relational learning. MIT press, 2007.
[72] Jerry Fodor. The language of thought, volume 5. Harvard University Press, 1975.
[73] Noah D Goodman, Joshua B Tenenbaum, Jacob Feldman, and Thomas I. Griffiths. A rational analysis of rule-based concept learning. Cognitive science, 32(1):108-154, 2008.
[74] Elizabeth S Spelke. What Bubies Know: Core Knowledge and Composition Volume 1, volume 1. Oxford University Press, 2022.
[75] Evelina Fedorenko. Michael K Behr, and Nancy Kanwisher. Functional specificity for high-level linguistic processing in the human brain. Proceedings of the National Academy of Sciences, 108(39): 16433, 2011.
[76] Noah D Goodman and Michael C Frank. Pragmatic language interpretation probabilistic inference. Trends in cognitive sciences, 20(1 11:818882,, 2016.
[77] Yewen Pu, Kevin Ellis, Marta Kryven, Josh Tenenbaum, and Armando Solar-Lezama. Program synthesis with pragmatic communication. Advances in Neural Information Processing Syxtems, 13:13249-13259, 2020.
[78] Jacob Andreas and Dan Klein. Reasoning about pragmatics with neural listeners and speakers. In Proceedings of the 2016 Conference on Empirical Methods Natural Language Processing, pages 1173-1182, Austin, Texas, November 2016. Association for Computational Linguistics.
[79] Dave Golland, Percy L Liang, and Dan Klein. A game-thcoretic approach generating spatial descriptions. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 410-419, Cambridge, MA, October 2010. Association for Computational Linguistics.
[80] Patrick Shafto, Noah D Goodman, and Thomas Griffiths. A rational account of pedagogical reasoning: Teaching by, and learning from, examples. Cognitive psychology. 71:55 89, 2014.
[81] Xinyun Chen, Maxwell Lin, Nathanacl Schärli, and Denny Zhou. Teaching large language models to self-debug, arXiv preprint arXiv:2304.05128, 2023.
[82] Sean Welleck, Ximing Lu, Peter West, Faeze Brahman, Tianxiao Shen, Daniel Khashabi, and Yejin Choi. Generating sequences by learning to self-correct. ICLR, 2023.
[83] Wonjae Kim, Bokyung Son, and Ildoo Kim. Vilt: lasiougan transformer without convolution region supervision. International Conference on Machine Learning, pages 5583-5594. PMLR, 2021.
[84] Andy Zeng, Adrian Wong, Stefan Welker, Krzysztof Choromanski, Federico Tombari, Aveek Purohit, Michacl Ryoo, Vikas Sindhwani, Johnny Lee, Vincent Vanhoucke, et al. Socratic models: Composing zero-shot multimodal reasoning with language. arXiv preprint arXiv:2204.00599 2022.
[85] Wai Keen Vong and Brenden M Lake. Cross-situationa word learning with multimodal neural networks. Cognitive science, 46(4)=e3322) 2022.
[86] M. M. Bongard. Pattern Recognition. Spartan Books, 1970.
[87] Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. Llm. int8 () 8-bit matrix multiplication for transformers scale. arXiv preprini arXiv:2208.0733 2022.
[88] Hector J Levesque. Knowledge representation and reasoning. Annual review of computer science, 1(1):255 287, 1986.
14

---

[89]
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava. Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Maric-Anne Lachaux, Thibaut Lavril, Jenya Lec, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molyhog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, lliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama Open foundation and fine-tuned chat models, 2023.
15

---

A Appendix
A.1 Supplemental Results
Fig.1 iillustrats model predictions across every Number Game concept in
43
training examples: 16
1.0
number and bet...
divisible by 8
between and 22
11
In
M nil
divisible 4
even and less than 30
prob. test concept
training examples: 16, 8, 2, 64
power is
an even power of 2 an integer power of 2 precorrect power of power 2.
01
10
0000000000000
00
01 0000000000000000
50
100
training examples: 16, 23, 19, 20
either 16, 23, 19, 20 multiple facter of... either 16, 23, 19, or 20 20 and between 16 and 23.
training examples: 60
and 60 between 40 and divisible by both 2 and 4 between 50 and
|1|
ml
n1
55
5 n
divisible by 3
training examples: 60, 80, 10, 30
evenly divisible 10... greater than less
divided by 10 or 20
multiple of 10 and i.. divisible by 60, 80, 1...
1m
100
training examples: 60, 52, 57, 55
50
0.5
60, or 52, or 57, or 55.
between 40 and more than 50, 60, 5...
0.0
100
training examples: 98, 81, 86, 93
above 80
either 86,
between 80 and 100 either 98, 81, 86, or 93 greater equal
0.0
training examples: 25, 4, 36, 81
square (1, 4, 9...)
model human
perfect square (has an... perlect square (see square number (integer) square (non- mmon-negalive...
0.5
0.0
100
lest numberr
Figure Model predictions across every Number Game concept in using 1000 g samples
16

---

Recall that we deduplicated the proposals instead of performing actual importance sampling. Fig. contrasts model fit for importance sampling and deduplication. We originally did deduplication simply because importance sampling is not possible with GPT-4, and GPT-4 proved necessary for the logical concepts. On number concepts we used code-davinci-002, from which we can construct an importance sampler because it exposes the log probability of its samples. On number concepts deduplication provides a fit that on-par (actually slightly better) compared to importance sampling (Fig.)) 8
deduplication
importance sampling
1.0
1.0
human rating
Rº=0.95
human rating
R==0.90
0.5
0.5
0.0
0.0
0.0
0.5
1.0
0.0
0.5
1.0
model prediction
model prediction
Figure 8: Monte Carlo inference using deduplication instead of importance sampling does not harm model fit human data. The above figures show Number Game models using learned prior and 100 samples, and show predictions only on holdout data.
We also studied the effect of replacing the closed-source code-davinci-002 (Codex) with the open- source 70B LLaMA2 [9). This helps in understanding if open LLMs can be used to build models like ours. Additionally, as the open models are presumably weaker, swapping out Codex for LLaMA2 serves as a way of softly ablating either the likelihood or proposal distribution, both of which use an LLM. We find that the (presumably weaker) LLaMA2 can substitute fully as likelihood. Using LLaMA2 as a proposal distribution impairs model fit in the low-sample regime (Fig. 99, but with enough samples, the weaker model comes close to 'catching up'. This highlights the fact that an explicitly Bayesian reasoning process can compensate for deficiencies in the LLM, given enough Monte Carlo samples.
model-he model-human -human response R
1.00
0.75
tuned prior, codex for proposals and likelihood
tuned prior, llama-2 for proposals and likelihood
0.50
tuned prior, llama-2 for likelihoad only
tuned prior, llama-2 for proposals only
0.25
0.00
10
10a
10a
num samples
Figure 9: Comparing versions of our model that swap out the closed-source Codex model for the open-source LLaMA2 70B model, for both the likelihood and the proposal distributions.
A.2 Human Study
16 participants were recruited primarily through a Slack message sent channel populated by members of our academic department. Participants had average age 28.1 (stddev 13.6, all over
17

---

18), and were 7 male/5 female/1 nonbinary/3 declined to answer. Participants were randomly split between the concepts of most common color least common color. Each participant went through 15 trials, and took an average of 294s to complete those 15 trials. In exchange for participating in the study, participants received $10 in Amazon gift cards. Fig. 10 illustrrates the web interface shown to our human participants, including the cover story.
Trial 1: Please read these instructions carefully
You going attempt learn the meaning new word in an alien language, which the aliens call "Wudsy." each trial, you are going see collection shapes at the bottom the webpage, and your job select which ones you think "Wudsy." Afterward, the aliens tell you which shapes "Wudsy."
The meaning of the word Wudsy the same during the whole experiment. However, is possible that whether something Wudsy depends on what other shapes the context Wudsy may may not correspond an English word.
with, one has given you any examples what counts as "Wudsy." So just do your below pick which ones you think might belong concept called "Wudsy." Right after you do so, the aliens are going to label Wudsy objects by drawing black box around them, and then you are going to get another round of guessing which objects are "Wudsy."
will go through 15 trials of guessing what counts as Wudsy, Remember that the meaning of Wudsy does not change during the experiment, but might depend on other shapes in the collection.
(trial 1/15) Click yes on the objects that you think are Wudsy, and No on the other objects. Then click Next.
O
O
O
Yes
Yes
Yes O
Yes
Yes
O
No
No
No
No
No
Next
Figure 10: Cover story and web interface for our version of the logical concept learning study, which based on [59]
A.3 Modeling
A.3.1
Temperature and Platt Transform
Adding temperature parameter T to a model corresponds to computing the posterior via
Premp (Xest C(XI:K)
Σ
(C) 1[Xes CJ., where
CefCC) ...))
(C) (w(0)) 1/T
Ec (ib(C)) 1/T and (c)) = [CCh0pCCCpCCLCCCCCCCCCCC c(8)
(10)
Adjusting the predictions of model using a Platt transform corresponds introducing parameters and which transform the predictions according to
Prlat (X (xest E ((XEK) Logistic(b + Logistic (p(Xies e c(X
(11)
18

---

For the number game, every model has its outputs transformed by a learned Platt transform. This because we are modeling human ratings instead of human responses. We expect that the ratings correspond to some monotonic transformation of the human's is subjective probability estimates, and so this transformation gives some extra flexibility by inferring the correspondence between probabilities and ratings. Logical concept models do not use Platt transforms.
A.3.2 Parameter fitting
Training consists fitting the parameters T, 9 (for the prior), (for the likelihood), a and / (for the logical concepts likelihood), and Platt transform parameters b (for the Number Game). In practice, this amounts to around 400 parameters, almost all of which come from
We fit these parameters using Adam with learning rate of 0.001. We perform 1000 epochs of training for the Number Game, and 100 epochs for logical concepts. There is a tenfold difference in the number of concepts, so this way they take about the same number of gradient steps.
For the number game do 10-fold cross validation to calculate holdout predictions. For logical concepts we use the train-test split introduced in [59], which involves running different groups of human subjects on each concept twice, with different random examples. One sequence of random examples arbitrarily designated as training data, and the other as holdout data.
All model were trained on laptop using no GPUs. Training takes between few minutes and an hour, depending on the domain and the model.
Some of the parameters that we fit, namely e, 8, cannot be negative. To enforce this we actually optimize the inverse logistic of those parameters.
A.3.3 MCMC over Logical Expressions
Fleet was used"to to perform MCMC over logical expressions with the domain-specific primitives in this file, which include:
true, false: boolean
blue, yellow, green object boolean
rectangle, circle, triangle object boolean
small, medium, large: object boolean
and,
boolean boolean boolean
V. 3: (shape boolean) 2abject boolean
filter: (object boolean) 20bject 2bbject
e: object x 2abject boolean
:: object object u (1), unique set element
empty pobject boolean
same_shape, same_color, aame_size object x object - boolean
sizes, sizes, size>, sizez, object object boolean
The model first constructed large hypothesis space by performing MCMC for minute per batch, and per learning curve. In one minute, Fleet makes approximately 10t MH proposals. There are a little more than 200 learning curves, and 25 batches per curve, for a total of about billion MCMC proposals. In the main text, we abbreviate this analysis by referring to 10 proposals.
The top 10 samples per batch and per learning curve were retained. These top 10 samples samples were then deduplicated to yield 45 thousand hypotheses. Parameter fitting and posterior estimation was performed solely over those 45 thousand hypotheses.
Quantitatively, these are vastly more proposals than the models introduced in this paper. Quantitatively, these proposals are also derived in a very different way: the hypothesis space for the BPL learer is
Runnnnn the model was graciously performed by the authors of [59] who provided us with the raw data.
19

---

actually informed by data on other learning curves, and also on the same learning curve, but in the future batches.
in this sense that the BPL model is a computational-level theory, and not a process model, because human subjects could not be proposing hypotheses using data that is going to be in the future, or on other learning curves, However, the above strategy for proposing hypotheses is a very reasonable heuristic for constructing the support of the space of plausible logical hypotheses that human learner might ever think of.
A.3.4 DreamCoder baseline
DreamCoder was initialized with the following domain-specific primitives:
1,2,3.... 100 int
squares: 2int
singleton int - 2int
interval int x int 2
intersect, union 2 2 2
powersof, multiplesof: int - 2int
converted into single task. The discrete structure of the prior was held fixed because there was no learnable common structure across the 8 tasks, but the continuous parameters of the prior the topK=100 highest posterior programs discovered for each task at each wake-sleep cycle. The neural recognition model took input 100-dimensional binary vector encoding the support of a number concept, which was then processed by an MLP with 64 hidden units. The recognition model was trained wake-sleep cycle. During the waking phase. program synthesis was provided with timeout of 60 seconds per task per cycle, enumerating approximately 10* programs per task.
Each of the 8 stimuli from 45
updated each wake-sleep cycle. We estimated those parameters from
50/50 split of replays/dreams for approximately 10S training examples per
We confirmed by manual inspection that high-likelihood programs were discovered for of the eight tasks. Therefore the difference between DreamCoder and our model cannot be attributed to failure on the part of DreamCoder to discover valid programs. Instead, we attribute the difference to differences of representation (natural language lambda caleulus), together with differences in the ways that parameters are estimated (maximizing the marginal likelihood of the tasks, vs directly fitting human judgments).
A.4 Prompting
A.4.1
Proposing hypotheses
For the number game we use the following prompt for code-davinci-002 to generate candidate concepts in natural language, given examples XI.K. The example number concepts given in the prompt come from the cover score given to human participants 43
Python
Here are few example number concepts:
The number is even
The number is between 30 and 45
The number is power of 3
The number less than 10
Here are some random examples of numbers belonging to different 2
number concept:
The above are examples of the following number concept:
-- The number is
where X is formatted by listing the numbers with comma and space between them.
20

---

For the number game we used the following prompt to generate candidate concepts in python (code baseline):
Python 3
Here are few example number concepts:
The number is even
The number is between 30 and 45
The number is power of 3
The number
less than 10
Here are some random examples of numbers belonging to different number concept:
Write python function that returns 'num belongs 4 this number concept. def check_if laaik_if_iincocoppppe (num): return
For logical concepts we used the following few-shot prompt for GPT-4 to generate candidate concepts:
Here three simple concepta, which specify when an object
2
'positive' relative to an example collection of other 2 objects. Before giving the rule for each concept, we give / examples of collections of objects, and which objects in the 2 collection 'positive'.
Concept #1:
An Example of Concept #1:
POSITIVES: (big yellow rectangle)
NEGATIVES: (big green circle), (medium yellow rectangle)
Another Example of Concept #1:
POSITIVES: (medium yellow rectangle)
NEGATIVES: (big red circle), (small green circle)
Rule for Concept #1: Something is positive if it is the biggest yellow object the example.
Concept #2:
An Example of Concept #2:
POSITIVES: (small yellow circle), (medium yellow rectangle)
NEGATIVES: (big green circle), (big blue rectangle)
Another Example of Concept #2:
POSITIVES: (big blue circle), (medium blue rectangle)
NEGATIVES: (small green circle), (medium yellow rectangle),
Rule for Concept #2: Something is positive if there is another 4 object with the same color in the example.
Concept #3:
Example Concept
POSITIVES: (small yellow circle), (medium yellow rectangle)
NEGATIVES: (big green circle), (big blue rectangle)
Another Example of Concept #3:
POSITIVES: (small blue circle), (amall blue triangle), 4 (medium blue rectangle)
NEGATIVES: (medium green triangle), (big yellow rectangle) Another Example of Concept #3:
POSITIVES: (big red rectangle), (medium red rectangle), 4 (big red triangle)
NEGATIVES: (medium green triangle), (big yellow rectangle) Rule for Concept #3: Something is positive if it is the same color 2
4 as the smallest triangle in the example.
Now here are some examples another concept called Concept #4, but this time we don't know the rule. Infer ten different possible rules, and make those ten rules as simple and 2 general you Your simple general rules might talk
21

---

about shapes, colors, and sizes, and might make comparisons 2 between these features within single example. but it doeen have to, Remember that a rule should say when 2 something is positive, and should mention the other objects / in the example, and should be consisting with what you see / below.
Concept #4: Xı.
Rule for Concept #4: Something is positive if...
Now make a numbered list of possible rules for Concept #4. Start by writing "1. Something is positive if". End each line with
4 period.
Each sample from the above prompt generates 10 possible concepts formatted as numbered list. We draw 10 times at temperature=1 construct 100 hypotheses. To obtain fewer than 100 hypotheses we take hypotheses from each sampled list in round-robin fashion. We found that asking it to generate list of hypotheses generated greater diversity without sacrificing quality, compared to repeatedly sampling single hypothesis.
The above prompt provides in-context examples of first-order rules. We also tried using different prompt for propositional concepts that illustrated the examples as truth table, and gave in-context example rules that were propositional:
Here
example concepts defined by logical rule:
Rule: a triangle.
Rule: green rectangle.
rectangle (unless that rectangle is blue). Rule: not both big and green.
Rule: big
Rule: either big or green, but not both.
Rule: either a rectangle or not yellow.
Rule: circle.
Now please produce a logical rule for a new concept. Your rule should be consistent with the following examples. It must be 2 true of all the positive examples, and not true of all the 2 organized into table 4 with one column for each feature (size, color, shape)
negative examples. The examples
Please produce a simple rule that is consistent with the above table. Make your rule as SHORT, SIMPLE, and GENERAL as possible. Do NOT make it more complicated than it has to be, 2 refer features that you absolutely do not have to refer to. Begin by writing "Rule: and then the rule, folloved by 2 a period.
Using the first order prompt for every concept gives R .80 fit to the human responses. Using both prompts gives the R² problems, and the first order prompt for the higher order problems. We strongly suspect that single prompt that just showed both propositional and higher-order in-context examples would work equally well, given that single first-order prompt works about well also, but did not try that it would have required rerunning all our GPT-4 queries, which would have had high cost.
result the main paper: the propositional prompt for propositional
On the first batch, the learner has not observed any examples. Therefore the above prompts do not apply, and we use different prompt construct an initial hypothesis space:
Here are some example concepts defined by a logical rule:
Rule: color is purple.
Bule: shape is not hexagon.
22

---

Rule: color is purple and size is small.
Rule: size tiny shape square.
Rule: size not enormous.
Rule: color is red.
Please propose
concepts, defined logical rule. These concepts can only refer to the following features: shape: triangle, rectangle, circle
color: green, blue, yellow
size: small, medium, large
Please make your rules short and simple, and please write your 4 response on a single line that begins with the text "Rule:
Provide 100 possible rules.
We generate from the above prompt at temperature=0,, and split on line breaks to obtain candidate rules.
A.4.2 Translating from natural language to Python
We translate Number Game concepts from English to Python via the following prompt for code- dov,nci-002, and generate at temperature=0) until linebreak:
Write a python function to check if number is C.
def check_number (num): return
We translate logical concepts from English to Python using series of in-context examples, again generating with temperature=0 until the text #DONE produced
def check_object (this_object, other_ other_objects).
this_object tuple of (shape, color, , size) other_objects: a list tuples of (shape, color, size)
returns: True if 'this_ is positive according to the 2 4 following rule:
Something is positive if is small object, and not 2 a green object.
# shape: string, either "circle", "rectangle", or "triangle" color: a string. either "yellov", "green", "blue" size: an int, either (small), (nedium), 3 (large) this_shape this_color, this_size this_object
'this_object is not part of 'other_objecte" to get all of the examples, you use @all_example_objctt. [thhisobject]]
defined as 'other_objece" +
# be careful as to whether you should be using 2 'all_ www.ealeeaaeeeeeeeeeeeeeeeeeeeee all. examepee_ojjcct other_
'other "ocher_object'' in your code [this_object]
# Something is positive if it is not a small object, and not a 2 green object.
#START
return (not this_size 1) (not tolor "green") #DONE
def (thjs other_objectee
"This prompt pretty long, and probably could be much shorter. Preliminary experiments suggested that few in- on-context examples were very helpful, and so increase the odds of the model working without much time spent prompt- coonptenniineriing provided large number of in-context examples.
23

---

this_object: tuple of (shape, color, size)
other_ other_objects: list of tuples of (shape, color, size)
returns: True if 'this_object' is positive according to the
4 following rule:
Something is positive if it is bigger than every other object
shape: a string, either "circle", "rectangle", or "triangle" # color: a string, either "yellov", "green", or "blue" # size: an int, either (small), 2 (medium), 3 (large) this_shape, this_color, this_size this_object
"this_objects is not part of 'other_objecta,
# to get all of the examples, you can use defined as 'other_objects +
4 www.eeeampepeeeeeeeeeeeeeeeeeeee [thii_object]'
be careful as to whether you should be using
4 wwweeerampe__beeceeeeee or 'other_objects' in your code all exameeaple__obecttt other_objects [this_object]
Something positive it bigger than every other object #START
return all( this_size > other_object [2] for other_object in 2 other_objects
#DONE
def check_object (this_object, other_objects);
this_ tuple of (shape, color, size) other a list tuples of (shape, color, size)
returns: True if 'this_object is positive according to the 2 folloving rule:
Something is positive if it is of the largest
shape: a string, either "circle", "rectangle", or "triangle" # color: a string, either "yellov", "green", or "blue" size: an int, either (small), (medium), 3 (large) this shap_s this_color, this_size this_object
'this_object is not a part of 'other_objects' # to get all of the examples, you can use 2 pail_examlle_objee [bhes_o
defined as "other_objects
be careful as to whether you should be using
4 @aallexxampe_oblee objects or 'other_object all all_ Lexample = other_objects + [this_object]
in your code
Something positive it one of the largest #START
return all( this_size >- other -ther_object [2] for all exl_exanpl_oobject 2 in ell_
#DONE
def check_object check (this_object, other_objects).
chie_tbject: tuple of (shape, color, size)
a list of tuples of (shape, color, size)
returns: True if 'this_object is positive according the following rule:
Something is positive if is smaller than every yellow
4 object
24

---

# shape: string, either "circle", "rectangle", or "triangle" color: string, either "yellow" "green", "blue" size: an int, either (small), (medium), or 3 (large) this_ this_color, this_size this_object
'this_object' is not part of 'other_objects" get all of the examples, you use @alLeexaple_oojectae 4 [tthisoobject]
defined as 'other_objecte
# be careful as to whether you should be using L wwweeeeaaeeeeeeeeeeeeeeeeeeeeeee c 'other_objects" in your code other_objects [this_object]
all all_ exameple__eeeeect
# Something is positive if it is smaller than every yellow object #START
return all( this_size other other_object [2] for other_object in other_objects if other_object [1] "yellow"
#DONE
def oteck other_objects):
***
this_object: tuple of (shape, color, size) other_objecee a list of tuples of (shape, color, size)
returns: True if 'this_object" is positive according the following rule:
Something is positive if there is another object with the 2 same color
shape: a string, either "circle", "rectangle", or "triangle" color: a string, either "yellow", "green", or "blue"
size: an int, either (small), 2 (medium), or 3 (large) this this_shape, this_color, this_size this_object
'this_object" is not part 'other_objects" to get all of the examples, you can use
4 @alleexaple_cojectte [[this_ojjec]]
defined as 'other_objecte
be careful "all__xxamle_oojecet or 'other_objects' your code all exxampleeejjectee other_objects [this_object]
whether you should be using
Something is positive if there is another object with the / same color
#START
return any color -- other [1] for other_object in / other_objects
#DONE
def check_object cbeck objct, other_objects):
objectt tuple of (shape, color, size) other_objece list of tuples of (shape, color, size)
returns: True if 'this_object' is positive according to the 2 following rule:
Something is positive if it has unique combination color
shape
# shape: a string, either "circle", "rectangle", or "triangle" color: string, either "yellov", "green", "blue" size: int, either (amall), (medium), 3 (large) this_shape, this_color, this_size this_object
# 'this_object is not part of 'other_obeecss"
25

---

# to get all of the examples, you use "all__xae saai_exrmlle__jjecee [this_object]]
defined 'other_
be careful as to whether you should be using "all_example_ sxjjete_ or 'other_objecte in your code all exa.eeeeeebobeetttt other_objects [this_object]
Something positive it has a unique combination of color 4 and shape
#START
return all( this_shape != other other_object [0] thior != other_object [1] for other_object other_objects
#DONE
def check_ (this, other_
this_object: tuple of (shape, color, size)
other objectobjects: a list of tuples of (shape, color, size)
returns: True if 'this_object' is positive according the following rule:
Something is positive if has the same color as the 4 majority of objects
shape: string, either "circle", "rectangle", or "triangle"
color: a string, either "yellow", "green", or "blue"
size: an int, either 1 (small), 2 (medium), or 3 (large) this shap_shape, this_ this_size this_object
'this_object' is not part 'other_objecta to get all of the examples, you can use 4 'all_example.chjeces [[this_ojjcc]]
defined as 'other_object"
be careful to whether you should be using 2 hall_example_objece
or 'other_objects' all obaeple other_objects [this_object]
your code
Something is positive if it has the same color as the 2 majority of objects
#START
majority_ color = max (l"yelloo"" "green", "blue"], key=lambda 2
4 color: sum (1 for obj in obbeee if obj [i] == 2 color))
return thio_ majority.color
#DONE
def check_object (this_object. other_objects):
.**
objett tuple of (shape, color, size)
other_objecta, a list of tuples of (shape, color, size)
returns:
True if 'this_object' is positive according the following rule:
Something is positive if there are at least two other 4 objects with the same shape
shape: string, either "circle", "rectangle", or "triangle"
color: a string, either "yellov". "green", or "blue"
size: an int, either (small), (medium), or 3 (large) this_shape, this_color, this_size this_object
'this_object' is not part of 'other_objecta" to get all of the examples, you can use 4 'all_example_ objects' defined as oother_objectt + 2
4 [this_object]
26

---

be careful to whether you should be using 2 'all_examlle_oberrc c 'other_objecta" in your code all all_ exa_eeeeee_oojectt other_objects [this_object]
# Something is positive if there are at least two other objects / with the same shape
#START
return sum (1 for other_ other_object in other_objects 4 other_object [o] = thip_shape) >=
#DONE
def check (thiss otjerstbj)cts):
.**
this object: tuple of (shape, color, size)
other objecta: list of tuples of (shape, color, size)
returns: True if objest is positive according to the C
following rule:
shape: a string, either "circle", "rectangle", or "triangle"
color: a string, either "yellow", "green", or "blue"
# size: an int, either (small), 2 (medium), or 3 (large) this_shape, this_color, this_size this_object
'this_object' is not a part 'other_objects"
to get all of the examples, you can use
< wtt__eeamlee_bejee 4 [this_object]]
defined as
+
be careful as to whether you should be using eww.eeeammpe_eeeeeeeeeeeeeeeeeee or 'other_objects" your code all exapeeeeeeeeeeecct....e.eeeeeee. other_objects [this_object]
#START
When modeling the concepts majority/minority color we used different prompt for converting natural language to python, because the above includes majority color as one of its in-context examples. We gave GPT-4 the following prompt for those concepts
Please write a python function to check if an object obeys a 4 logical rule. The logical rule talks about the following
4 features:
shape: string, either "circle", "rectangle", or "triangle"
color: a string, either "yellov", "green", or "blue"
size: an int, either 1 (small), 2 (medium), or 3 (large)
The python function should be called 'check_object', and inputs:
"this_object's a tuple of (shape, color, size) a list of tuples of (shape, color, size)
The logical rule should check if 'this_object' has certain / relationship with 'other_objects' Collectively, 2
correspond to all of the 2 objects, if the rule references the whole examples, it is 2 talking about that structure.
The logical rule is: C
We strongly suspect that GPT-4 with the following prompt is strictly better then Codex. The high cost of using GPT-4 makes it more practical the Codex prompt for the main experiments however.
27

---

Please start your response writing the following code, and then 2 complete the function body that it returns 'True' if and 2 only if the logical rule above holds.
def check_object
(this_object, ohcck other_objects);
objett tuple of (shape, color, size) other_objecta. a list of tuples of (shape, color, size)
returns: True if 'this_object" is positive according the 4 following rule:
C
shape: string, either "circle", "rectangle", or "triangle"
color: string, either "yellow", "green", "blue"
size: an int, either (small), (medium), or 3 (large) this_shape this_color, this_ = this_object
'this_object' is not part of 'other_object" to get all of the examples, you can use I "aal__examle_objettt 4 [thhi_ooject]]
defined as 'other_objects
be careful as to whether you should be using 2 hall_example_objecc all exa_e
'other_objecte in your code other_objects [this_object]
# return True if and only if:
C
...
A.5 GPT-4 Baselines
Our GPT-4 baseline for each domain presented the examples X1:K in string form and then asked GPT-4 to respond Yes/No as to whether a test example X belonged to the same concept. GPT-4 then queried temperature=1 collect 10 samples. Samples not beginning with 'y'/'n' were discarded, and the ratio of remaining samples that began with 'y" was computed (case insensitive).
We show below example prompts for the number and logic domains.
Here are few example number concepts:
The number is even
number is between 30 and
The number is a power of
The number is less than 10
random examples of numbers belonging poasibly 4 different number concept: 98, 81, 86, 93
Here
Question: Does the number 42 belong to the same concept the above numbers? Answer (one word, yes/no):
Logical concept example prompt:
Here are some example concepts defined by a logical rule:
Rule for Concept #1: Something is positive if it is the biggest 2
4 yellow object in the example
Rule for Concept #2: Something is positive if there is another
object with the same color the example
Bule for Concept #3: Something is positive if it is the same color /
4 as the smallest triangle in the example
28

---

Now please look at the following examples for new logical rule.
Example Concept #4: POSITIVES: none
NEGATIVES: (large yellou circle), (small green circle), / 4 (medium green circle), (small yellow triangle)
Another Example of Concept #4:
POSITIVES: (small green circle), (large green circle)
NEGATIVES: (large yellow circle), (medium blue circle)
Another Example of Concept #4:
POSITIVES: (small green rectangle)
NEGATIVES: (medium yellow circle), (medium blue rectangle), 4 (large green circle), (medium green circle)
Another Example of Concept #4:
POSITIVES: (medium green rectangle) NEGATIVES: (medium yellow circle), (small yellow 4 rectangle), (medium yellow rectangle), (medium blue 2 4 rectangle)
Another Example of Concept #4:
POSITIVES: (small green rectangle)
NEGATIVES: (large yellow rectangle), (small yellow 4 triangle), (medium green circle), (emall blue rectangle)
Another Example of Concept #4:
POSITIVES: (medium green triangle)
NEGATIVES: (medium blue triangle), (medium blue rectangle), 4 (large blue triangle), (small yellow triangle) Another Example of Concept #4:
POSITIVES: none
NEGATIVES: (small yellow circle), (large blue circle)
Another Example of Concept #4:
POSITIVES: none
NEGATIVES: (large green circle), (small blue rectangle), 2 4 (small green triangle), (medium blue rectangle) Another Example of Concept #4:
POSITIVES: (small green rectangle)
NEGATIVES: (small yellow circle), (large blue rectangle)
Now get new collection of examples for Concept #4: (medium blue triangle) (large yellow triangle) (small blue
4 rectangle) (large blue circle) (small yellow circle)
Question: Based on the above example, is a (small yellow circle) in 2 4 the concept?
Ansver (one word, just write yes/no):
A.6 Latent Language Baseline
For fair comparison, we designed our latent language baseline to be as similar to our system as possible. performs maximum likelihood estimation of single concept, rather than estimate full posterior, but uses the exact same prompts and likelihood functions as our model. The most important difference from the original latent language paper [3]] is that instead of training our own neural models for language interpretation and language generation, we use pretrained models (Codex/code-davine-- and GPT-4)..
A.7 Ablation of the proposal distribution
We ablate the proposal distribution by proposing hypotheses unconditioned on XI-K- We accomplish this by drawing concepts from the following alternative prompt, which is designed to resemble the prompt used by the full model except that it does not include X1:K:
Python
Here are a few example number concepts:
The number is even
The number between 30 and 45
29

---

The number is power of 3
The number is less than 10
The number
A.8 Pretrained prior
Our pretrained prior comes from the opensource model CodeGen 44 which was trained on source code. We chose this model because suspected that pretraining source code would give better density estimation for text describing precise rules. We formatted the rules as a natural language comment and prefixed it with small amount of domain-speent put probability mass on rules that correctly talk about numbers or shapes.
text in order to prime the model to
For the number game, we would query CodeGen for the probability of p(C) via
Here is example number concept: The number is C
For the number game's code baseline, we would query CodeGen for the probability of p(C) via
Python 3
Let 's think of number concept.
Write a python function that 4 this number concept.
if 'num" belongs
def return C
(num):
For logical concepts would query CodeGen for the probability of p(C) via
Here are
simple example shape concepta:
1. neither a triangle nor green rectangle
2. not blue and large.
# 3. if it is large, then it must be yellow.
4. small and blue
either big or green.
6.
Because the proposal distribution would generate rules beginning with the prefix "Something is positive if..." " we would remove that text before computing p(C') above.
30


Reviews:
[{"review_id": "Ko7XK7qxq7", "reviewer": "Reviewer_y5Ba", "summary": "The paper proposes an approach to scaling up intractable Bayesian models of few-shot concept learning. The key idea is to (1) train an amortized posterior distribution q(C|X_{1:K}) over concepts (represented as natural-language expressions) and then (2) make predictions about membership by marginalizing over a finite set of latent concepts sampled from q. Several variants of this approach are considered, all of which use a large language model (Codex) for the likelihood function p(X | C) and proposal distribution q. The approach is evaluated on two few-shot learning experiments: a generative number concept task, and a discriminative logical concept task. A version of the model that reweights sampled concepts according to a prior distribution trained on human judgements is found to fit human patterns better than alternative models which either (1) replace the human-tuned prior scores with an off-the-shelf language model (CodeGen), (2) replace the natural-language prior with a Python prior, or (3) entirely omitting the separate proposal distribution q and sampling proposed concepts directly from the (learned) prior instead.", "questions": "1. It wasn't clear how train-test splits were handled. The Fig. 2 caption alludes to 'held-out' judgements, but what was the split? How many examples were used to tune the prior? Were entire concepts held out, or were some human data seen for each example set? \n\n2. In Fig. 2, it looked like the 'no proposal dist' model might potentially be competitive if it was given more than 100 samples to hit on good descriptions. This is still a relatively small number of samples, and even if it is somewhat expensive, it would aid understanding to extend the x-axis one more degree of magnitude. \n\n3. It looked like data came from the extremely small, cherry-picked set of concepts used in the original Tenenbaum dataset (with N=8). However, much larger and more systematic datasets are now standard, e.g. Bigalow & Piantadosi, https://doi.org/10.5334/jopd.19, releasing 272k judgements using many more concepts. I would strongly suggest reporting generalization performance to these new concepts. \n\n4. It wasn't clear how the Latent Language model actually worked for these task; it would help to clarify precisely how it differed from the other models in this case (did it just take the single maximum likelihood concept from the proposal distribution instead of marginalizing proportional to each sample?)\n\n5. Did the likelihood function assume boolean output from the codex-derived python code? how was this ensured? what happened if the generated python code returned an error or non-boolean? \n\n6. Section 5 states: \"Except we now have a discriminative learning problem instead of a generative one\" -- except wasn't the number game treated as discriminative by the model, using an indicator function (effectively deriving a disciminative classifier for whether each number is in or out the concept?)\n\n7. I would have liked to see a more reasonable neurosymbolic BPL baseline, which actually does proper Bayesian inference (i.e. MCMC) using the latent concept space as the proposal distribution over valid programs. \n\nA related more recent paper that may be worth including:\n\n* Wong, L., Grand, G., Lew, A. K., Goodman, N. D., Mansinghka, V. K., Andreas, J., & Tenenbaum, J. B. (2023). From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought. arXiv preprint arXiv:2306.12672.", "limitations": "A clear limitation is that a central pillar of the approach, the Codex API, is now deprecated and unavailable to other researchers. It is therefore not clear whether any of the results can be replicated, creating further incentive for the authors to compare performance on a number of other amortized posteriors (proposal functions) and likelihood functions, ideally those which are open and maintained.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The manuscript presents a promising and potentially influential approach to scale up Bayesian models on few-shot learning tasks. A number of timely ideas are explored, and the results compare favorably with other recent approaches that have been well-received at NeurIPS (e.g. Bayesian program learning, and other neurosymbolic approaches). Many of the basic ideas in play are pretty familiar by now (e.g. amortizing a proposal distribution to avoid costly search over hypotheses, using natural language as a more expressive latent hypothesis space, fine-tuning on human priors to impart human-like inductive biases). But this work still 'remixes' and integrates them in an interesting way: for example, other language-guided BPL approaches (e.g. Wong et al, 2021, ICML; Andreas et al., 2017) have required a set of language annotations to fine-tune on, rather than utilizing more generic priors (although Codex would be more impractical for multi-modal tasks, conditioning on images). Evaluating on human behavior (rather than synthetic benchmarks) are another strength.", "weaknesses": "My primary concerns are around how to do credit assignment to the many potentially brittle component parts of the pipeline, and the applicability of this scaling approach outside of toy domains. One somewhat deflationary critique is that there is no Bayesian *inference* at all in this pipeline, and certainly no inference *over natural language*. The approach depends on the independent existence of a sufficiently powerful amortized posterior distribution from a very expensive inference procedure that has already been performed off-stage (i.e. training Codex). the model comparisons simply show more or less efficient ways to approximate a *predictive* distribution by marginalizing over high-probability regions of that independently pre-existing posterior. This isn't inference! Arguably, it's just wrangling the amortized product of an earlier inference, exploring different ways of leveraging human data to correct distortions in this mammoth amortized posterior and project it down to specific tasks at hand. \n\nI still think this kind of 'wrangling' work is interesting and novel, as there are clearly more or less effective ways to do it, but (1) I believe the framing of actually doing Bayesian inference over natural language is misleading (including statements like 'our work adds Bayesian inference'), and (2) I would suggest much more focus and scrutiny on the black-box, closed-source models that are the 'wizard behind the curtain,' the parts of the pipeline actually responsible for the few-shot concept induction. For example, it is stated that Codex was used 'because we hypothesized that training on source code would transfer to reasoning about numbers,' but no other choice was considered. \n\nI would like to see a bigger space of candidate posteriors compared. I would also like to see a stronger 'credit assignment' analysis testing components of the Codex pipeline (which I understand has now been deprecated, making these results difficult to reproduce?) For example, one very strong assumption is that each linguistically expressed concept maps deterministically to a single Python program, which in turn, can be evaluated deterministically on each number. There may be multiple valid programs corresponding to each linguistic utterance. What if the linguistic concept is good but the translation to python is bad? i.e. what if the best cutting-edge model were used as the proposal distribution, but a weaker code-generation model were used in the likelihood to evaluate the generated concepts on numbers? The current analysis cannot pull these apart.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the thoughtful input, kind words, and suggested improvements. We address the main issues, with **new results bolded**, but can answer all of your questions during the discussion period.\n\n> My primary concerns are around how to do credit assignment to the many potentially brittle component parts of the pipeline\n\nIn our view, the key ideas are (1) natural language as a hypothesis space, (2) Bayesian reasoning for forming predictions, and (3) LLMs for tractable inference. Achieving the full suite of results requires all ingredients, established by comparing against Bayesian Program Learning (no natural language), latent language (no Bayes), latent source code (LLMs, but no natural language), and no proposal dist (no bottom-up proposals for tractability).\n\nOur pipeline has 3 components: prior; likelihood; and proposal distribution. We ablate the prior by not tuning it and by removing it (in latent language). The proposal distribution was also ablated. The likelihood is so critical for discarding erroneous hypotheses that we did not even consider a model without it, but **we've now run a likelihood ablation (global response PDF), confirming that the likelihood computation is essential.**\n\n> what if the best cutting-edge model were used as the proposal distribution, but a weaker code-generation model were used in the likelihood\n\nTo understand what happens when the individual components are merely made weaker instead of ablated entirely, we've run **new experiments on Llama-2, a recent 70B open-source model that is thought to be weaker than Codex (see attached global response PDF).** The new data show that a weaker LLM, like Llama-2, can implement effective likelihood models (act as code-generators), but stronger LLMs are important for proposal distributions, especially in the low-sample regime. Although these Llama-2 results do not change our scientific conclusions, they help in understanding how to practically engineer systems like ours using off-the-shelf tools.\n\n> Codex... has now been deprecated, making these results difficult to reproduce\n\nCodex is deprecated but freely available for academic use (with a special application needed), or on Azure (expensive, but for anyone). Ultimately, any closed-source model hurts reproducibility, and will be eventually deprecated. To mitigate this, we've archived our OpenAI queries+responses, as noted in the author checklist, and are doing Llama-2 replications (see above).\n\n> data came from the extremely small, cherry-picked set of concepts used in the original Tenenbaum dataset (with N=8)... larger and more systematic datasets are now standard, e.g. Bigalow & Piantadosi\n\nBefore performing our study we closely examined the Bigalow & Piantadosi dataset and discussed it with one of the authors of that dataset. We concluded it was too noisy, and that many Mechanical Turk workers were not correctly following the instructions. We suggest examining their data for \"powers of 2\", which shows that participants failed to even label the provided numbers (16, 32, 2, 8) as being 100% in the concept.\n\nAlthough small, the Tenenbaum data exhibits important phenomena such as few-shot learning of both rule-based and similarity-based generalizations. It is also canonical and pedagogical, serving as a main example of Bayesian concept learning in the textbook \"Machine Learning: A Probabilistic Perspective\" (Murphy 2012). Although a bigger dataset would be desirable, we believe the Tenenbaum data effectively shows the basics of the model, before moving onto the bigger logical concepts dataset.\n\n> scaling... outside of toy domains\n\nFrom the perspective of cognitive modeling, the logical concept data is quite challenging. To the best of our knowledge, there is no other study of logical concept learning in humans which is nearly as broad, high-quality, and high-resolution, as Piantadosi et al. 2016.\n\n> how train-test splits were handled\n\nFor Number Game we split the human judgments in 10-way cross-validation, *mixing across concepts.* Originally we thought that with only 8 concepts, fitting a prior while holding out whole concepts would not work. **Based on your question we reran by testing only on held-out concepts, finding that the results change very little: $R^2$ drops from 0.95 to 0.91.** Therefore, the model learns a prior that works for novel concepts / training data. For logical concepts, we followed Piantadosi 2016, which held out specific learning curves run on independent participants. That split shows generalization to new training data, like the new Number Game result given above. Our replication experiments get their test data from running new participants on novel out-of-distribution concepts (training on Piantadosi 2016), again showing that the learned prior can transfer to never-before-seen concepts.\n\n> I would have liked to see a more reasonable neurosymbolic BPL baseline\n\nWe appreciate your suggestion of a modern neurosymbolic baseline, and **have now ran a DreamCoder comparison, which gives a decent (but not great) fit to Number Game concepts (attached pdf).**\n\n> 'no proposal dist' model might potentially be competitive if it was given more than 100 samples\n\nWe performed a new experiment, finding that **with an order of magnitude more samples ($10^3$), 'no proposal dist' agrees with the human data only at $R^2=.41$, ie, it levels off in fit, although it should eventually trend upward with enough samples.** As the number of samples tends toward infinity, 'no proposal dist' should be just as good as the full model.\n\n>  there is no Bayesian inference at all in this pipeline, and certainly no inference over natural language\n\nWe've raised this issue in the global response, and can certainly alter our word choice.\n\n> how the Latent Language model actually worked... did it just take the single maximum likelihood concept from the proposal distribution instead of marginalizing proportional to each sample?\n\nExactly, it works as you described."}, {"review_id": "pG1gGx8zUx", "reviewer": "Reviewer_ce9a", "summary": "The authors introduce a computational approach for explaining how humans perform few-shot concept learning. The proposal is that humans use Bayesian inference over natural-language definitions of concepts. In more detail, a bottom-up proposer generates a set of candidate concept definitions, and Bayesian inference is then performed over these candidates - based on a prior distribution over concept definitions and a likelihood capturing how well the definition covers the observed examples.\n\nThe paper then presents two main experiments that implement the proposal via the use of recent large language models (LLMs). The prior over natural-language concept definitions is the probability that an LLM assigns to the definition. The proposer is instantiated as the responses of an LLM prompted with the training examples. The likelihood is computed by translating the proposed concept definitions into Python code using a code-based LLM and then running the Python code on the training examples. In both case studies (the Number Game and learning compositional logical concepts), the proposed approach provides a strong fit to human data.\n", "questions": "It would be helpful to hear your thoughts on the point discussed in W3 above, under “weaknesses.”", "limitations": "The authors do a strong job of discussing and acknowledging limitations\n", "rating": 10, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "- S1: The approach provides an interesting combination of Bayesian inference and neural network models that combines complementary strengths of both approaches: the powerful inference abilities of Bayesian models and the tractability and flexibility of neural models. This combination results in a system that is arguably more powerful than either type of approach on its own, representing an important advance in computational cognitive science.\n- S2: The components of the approach are well-motivated by high-level considerations and are well-operationalized using current AI tools.\n- S3: The results are compelling: in both case studies, the proposed model shows advantages (in overall fit to human data and/or in computational tractability) over strong baselines on both the Bayesian side and the neural network side.\n- S4: Working within this proposed paradigm, the authors show how to fit a model to human data in a way that successfully transfers human priors into a neural model.\n- S5: This work will likely be useful for future researchers working in Bayesian modeling and/or neural networks as a way to take insights from one school of thought and use them to overcome weaknesses of the other school of thought.\n", "weaknesses": "- W1: Natural language is ambiguous, so natural language strings do not really provide clear concept definitions - and, by extension, inference over natural language strings cannot strictly be viewed as inference over concept definitions. (I.e., in the general case, natural language cannot be unambiguously translated into Python code). That said, as the experiments show, it is clearly close enough to work very well in at least some settings. In addition, the authors are clear in stating (in lines 286 to 293) that they do not claim that natural language is the language of thought but rather that it is a useful heuristic tool for modeling human thinking.\n- W2: The specific proposal is well-suited to concepts that can be naturally expressed in language, but is not suited to concepts that do not have a straightforward linguistic definition. The authors acknowledge this point (lines 48 to 51).\n- W3: It was a little unclear to me what the paper’s main contribution is: Is it (i) a new hypothesis about how humans perform few-shot learning, or is it (ii) a way to make tractable some previously-existing hypotheses that were previously intractable to evaluate? Both types of contribution are valuable, but it would be helpful to clarify which is/are being made here. It’s clear that the paper accomplishes (ii), but it is not obvious to me that it does (i), as the basic high-level ideas seem to be present in the cited prior work (e.g., ideas about performing Bayesian inference over a small-ish number of heuristic proposals are present in prior work about bounded rationality). It’s not a problem if (i) is not done here, but if it is done, it would be helpful to clearly state what new hypothesis is being offered, and if it is not done, it would be helpful to state explicitly that (ii) is the main type of contribution being made. One reason I’m confused here is due to differing senses of the word “model”: the paper clearly states that it proposes a new model, but I’m not sure if this is meant in the sense where “model” means “hypothesis” or the sense where it means something more like “implementation”.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the thoughtful input and enthusiastic support. We are happy to correspond more during the discussion period, but here we mainly just address the specific question you raised:\n\n> …unclear to me what the paper’s main contribution is: Is it (i) a new hypothesis about how humans perform few-shot learning, or is it (ii) a way to make tractable some previously-existing hypotheses that were previously intractable to evaluate? … It’s clear that the paper accomplishes (ii), but it is not obvious to me that it does (i)\n\nThank you for this helpful way of thinking about the different contributions of the work.\n\nOur main contribution is (ii), offering a computational model that makes inference tractable for certain very expressive hypothesis spaces that have proved valuable in cognitive modeling (Line 31: \"Our goal is to build a model of humanlike concept learning that makes progress toward resolving the tension between intractable inference and expressive hypothesis classes\"). There is also a more speculative account of our contributions (i), namely that our model offers a hypothesis for how human brains resolve the “curse of a compositional mind” (Spelke 2022: freeform recombination of concepts yields a combinatorial explosion). In this more speculative hypothesis, culturally-transmitted concepts and linguistic schemas help guide inner thought, making combinatorial thinking more tractable. We've shied away from making that claim, because it is not directly supported by our findings, though it is not contradicted by our work, either. Right now the manuscript tries to strike a balance by simply saying that \"Natural language, even if it is not actually the same as our inner mental language, acts as a vast reservoir of human concepts, and provides a flexible algebra for combining them. Thus our best near-term strategy for modeling human thinking may be  to use natural language as a *heuristic approximation* to an inner Language of Thought\" (emphasis added), later refining the claim from natural language generally to large language models specifically as a \"reasonable surrogate for this [human] bottom-up [proposal] process, even if it its inner workings might differ greatly from human bottom-up proposal processes\".\n\nWe hope that the paper's current wording manages to strike the right balance, and are happy to revise in order to more clearly signpost the actual contributions, results, and concrete hypotheses, especially given that, if accepted, we have an additional page to include discussion and add clarifications.\n\nLast, about the other weaknesses you raise:\n> [the work is] not suited to concepts that do not have a straightforward linguistic definition... the authors are clear in stating (in lines 286 to 293) that they do not claim that natural language is the language of thought but rather that it is a useful heuristic tool for modeling human thinking.\n\n> Natural language is ambiguous, so natural language strings do not really provide clear concept definitions - and, by extension, inference over natural language strings cannot strictly be viewed as inference over concept definitions. The authors acknowledge this point\n\nIndeed, we *do not* provide a unified theory of human few-shot concept learning. Thank you for pointing out that the submission is careful about clearly discussing the limits of the work."}, {"review_id": "1E1lQ5pyo7", "reviewer": "Reviewer_qc38", "summary": "This paper proposes a model for human learning of concepts from few examples (aka \"few shot\" setting) that leverages natural language as an internal concept representation. This is a key trick because it means that\n\n* an LLM can be used to act as a proposal distribution for (efficiently) generating data-dependent candidate concepts\n* a prior distribution over concept space can be tuned (estimated) from human judgments using NLP features\n* a \"likelihood function\" for determining whether a concept could have generated an observation can be implemented via a \"text2code\" LLM that derives Python code from the natural language concept representation\n\nExperiments on \"Number Game\" and logical concept learning show concept learning with \"psychologically plausible\" sample complexity, remarkable agreement with human judgments, and explainable failure modes. In comparison with state-of-the-art Bayesian Program Learning (BPL) this approach is able to search through a much smaller hypothesis space (thanks to \"high quality\" natural language candidate proposals) but also generalize to totally novel concepts which are not express-able in the BPL due to the flexibility of natural language vs the primitives available to the BPL approach.", "questions": "L92: it's a little unclear what it means to say that P(X_{test} \\in C) - it doesn't seem exactly to be P(X_{test} | C), or even P(X_{test}, C), but rather more like the posterior probability that X_{test} was generated by _the same latent concept C that generated X_{1:K}_. I guess the key assumption here is that \\mathbb{1}[X_{test} \\in C] is easily computable, but that isn't clear or established by this point in the development (later it is shown how to do this with NL-to-Python). \n\nL147: this summary of all the variations is pretty dense and takes a bit of reader effort to unpack.\n\nSection 4: Is there a stronger or more recent baseline for Number Game, vs Latent Language from 2018?", "limitations": "Limitations seem reasonable and likely avenues for future work.\n\nThe reproducibility issues with using GPT4 seem addressed by including the responses in the software/data release.", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "The paper is well-contextualized with respect to related work. It identifies a major open question (tractability vs expressivity) and addresses it via a key insight (using natural language is now much more feasible with recent LLM advances). Actually implementing this idea requires multiple non-obvious steps to bring it all together. The experiments systematically illuminate both the power and limitations of the approach in controlled but challenging problem settings. The agreement with human subject results gives the approach credibility. The comparison approach (Bayesian Program Learning) is a strong baseline. Overall I found this work to be a creative and well-executed integration of LLM advances into Bayesian cognitive modeling.", "weaknesses": "I guess if we were purely looking for some kind of \"predictive performance\" task there might be other approaches that \"outperform\" the framework here. However, the grounding in cognitive science for this work means that is probably not the right criterion, but rather this work is pursuing improved understanding of learning itself.\n\nI'd be interested to see a discussion or mention of some task or problem setting that successfully \"red teams\" this approach - ie, some problem which is adversarially chosen to be challenging or ill-suited for this framework (while still remaining in-scope within the domain of non-embodied abstract concept learning).\n\nThe approach seems critically dependent on the quality of the Python code generation aspect. From the supplemental materials it looks like a lot of prompt engineering went into getting that to work. To some extent this seems like kind of a \"weak link\" for the whole enterprise: using natural language as the concept space requires the availability of a (very general) mechanism for computing observation likelihood or concept membership given the concept. \n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the thoughtful review and enthusiastic support. Please find below our clarifications:\n\n> …mention of some task or problem setting that successfully \"red teams\" this approach - ie, some problem which is adversarially chosen to be challenging or ill-suited for this framework (while still remaining in-scope within the domain of non-embodied abstract concept learning).\n\nBecause we use a large language model as a proposal distribution, problems which LLMs struggle with would likely foil our approach. However, there is one caveat: because we draw multiple proposals (~100), and because we reweigh each of them using Bayes Rule, the model only needs a few proposals that hit the mark. Therefore, our approach could be “red teamed” by problems that cause LLMs to “fall on their face,” but probably not by problems that cause LLMs to become merely “flakey”. \n\nAlso, our current approach translates each natural language concept into Python, so it would struggle to learn concepts that are not easily expressible in a precise formal language like Python. This restriction is not strictly necessary: you could instead define the likelihood $p(X|C)$ using another neural model in order to allow learning \"fuzzier\" concepts, although this would probably come at the expense of precision. The last sentence of the paper alludes to these subtleties by saying \"We bypass natural language’s ambiguity by translating it into Python for the likelihood computation, but future work needs to determine if language models can produce language precise enough for induction, or if refining into languages like Python is more practical.\" If accepted, we'd have an extra page for expanding that point with this discussion.\n\n> …critically dependent on the quality of the Python code generation aspect. From the supplemental materials it looks like a lot of prompt engineering went into getting that to work\n\nFor the initial submission, we tried exactly two prompts for converting logical concepts into python: a short simple prompt (which was unreliable), and later a very long prompt, which proved reliable (but probably went overboard). In the weeks after the submission, we also tried prompting GPT-4 with instructions but no few-shot examples, which works better than the long Codex prompt used in the initial submission.\n\nIn our experience, though, converting simple natural language utterances into snippets of python requires little prompt-engineering, provided one is willing to use larger models such as Codex. In the global response, we also describe new results using Llama-2 70B, a very recent open source LLM, to convert natural language into Python, which required zero new prompt engineering.\n\n> I guess if we were purely looking for some kind of \"predictive performance\" task there might be other approaches that \"outperform\" the framework here. However, the grounding in cognitive science for this work means that is probably not the right criterion, but rather this work is pursuing improved understanding of learning itself.\n\nIndeed, our primary aims are scientific. After the submission deadline, we tried optimizing the logical concept model to maximize average task performance, instead of maximizing fit to humans. We found that the maximizing average performance makes the model surpass human performance, but also degrades human-model agreement. If accepted, we will include these results in the revision.\n\n> Section 4: Is there a stronger or more recent baseline for Number Game, vs Latent Language from 2018?\n\nThanks for the suggestion. We have now ran a DreamCoder baseline (a recent neurosymbolic Bayesian program learner). It achieves a decent (but not great) fit to the human Number Game data: $R^2=.75$, which should be contrasted width $R^2=.95$ for our full model.\n\n> it's a little unclear what it means to say that $P(X_\\text{test} \\in C)$ ...  I guess the key assumption here is that $\\mathbb{1}[X_\\text{test} \\in C]$ is easily computable\n\nWe like how you put it: The key assumption is that $\\mathbb{1}[X_\\text{test} \\in C]$ is easily computed, which comes from translating $C$ into Python. A footnote will be added clarifying this."}, {"review_id": "7wNw7VQzUm", "reviewer": "Reviewer_8GEn", "summary": "This paper presents a model for how humans learn abstract symbolic concepts from induction. The model uses an off-the-shelf language model as a meta-prior, which is then tuned to form a task-specific prior over hypotheses using a small number of human samples. This prior is then incorporated into a Bayesian inference setup to solve inductive reasoning tasks. The model closely adheres to human judgments and also seems to show that natural language is a better performing hypothesis space than programs.", "questions": "Major:\n- Do you have thoughts on exactly how \"strong\" you should make the prior and likelihood functions? For example, I'd imagine you might get even better raw performance by using GPT-3 as the prior model. Is there a sweet spot wrt model capabilities for matching human judgments? If so, what are the implications for your rational process model? \n\nMinor:\n- Line 36: ”regularizes the learner toward probable generalizations” Some references here would be nice—you can just duplicate the ones you have later in the text.\n- Line 48: Why is this well-suited for natural language? \n- Line 83: typo “has also” \n- Line 135: What’s the sampling technique—nucleus sampling, greedy, etc? I’d like some more detail on the implementation of the proposal distribution, because sampling and sampling technique can change the effective distribution a lot.\n- Line 142: Can you just explain how you fit the parameters right after you introduce them? I was wondering how you trained the parameters for a few paragraphs.\n\nNits:\n- Figure 3: Do you have a higher resolution screenshot?", "limitations": "Yes, the authors have addressed limitations.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "- Clear and concise\n- Thoughtful discussion\n- Novel incorporation of LLMs into cognitive modeling\n\nComments:\n- Line 162: Super cool result. I’d love if you could stay on this result a little longer and speculate why this might be.\n- Line 275: You’re basically using LLMs as a meta-prior, and then tuning it to obtain a task-specific prior. This is very interesting. ", "weaknesses": "- The abstract is vague. I’d recommend the authors expand the abstract in length and make reference to their results. \n- Some more implementation details in Sections 4 and 5 would be helpful for future readers. Please see questions below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the thoughtful review and kind words. Below we respond to some of your main points, but can discuss further during the discussion period:\n\n> Do you have thoughts on exactly how \"strong\" you should make the prior and likelihood functions?\n\nOur thoughts are that the proposal distribution $q$ needs to be the \"strongest\", because it needs to propose plausible hypotheses from scratch. In our paper, the likelihood $p(X|C)$ only needs to translate English to Python, so it requires a decent LLM or a fine-tuned smaller model (we likely could have fine-tuned a T5-like model also).\n\nIn the global response, we have attached a PDF showing analysis of using smaller open-source models for the proposal distribution and the likelihood. These new results suggest, as above, that $q$ needs to be the \"strongest\", but that a weaker $q$ can still work, provided that you take more samples. This introduces an interesting tradeoff between the \"strength\" of $q$, and the amount of compute (# samples) that are taken at test time.\n\nThe prior $p(C)$ is only responsible for giving a soft bias toward simpler, shorter language, and in our experience, does not require a \"strong\" model: we tried both a 350M open source model and tuning a ~100M model.\n\n> Is there a sweet spot wrt model capabilities for matching human judgments? If so, what are the implications for your rational process model?\n\nThat is an interesting scientific question. For example, a stronger proposal distribution might sometimes outperform humans, which could suggest certain limits on bottom-up psychological processes. Although our paper does not specifically explore the questions you raise, they would make for thrilling future work. We will add this to the discussion.\n\n> What’s the sampling technique—nucleus sampling, greedy, etc?\n\nThe proposal distribution was sampled with temperature $T=1$ and $\\text{topP}=1.0$., which effectively disables nucleus sampling. Because we are taking multiple proposals, a higher temperature made more sense to encourage diversity.\n\nFor the likelihood, which calls out to an LLM to translate English to Python, we sampled with temperature $T=0$, both because Codex was very reliable at this translation, and to avoid needless stochasticity.\n\nThe revision will mention these issues. Thanks for the catch.\n\n> Why is this well-suited for natural language? [prime numbers less than 30]\n\nPerhaps a better phrasing would be that these concepts are \"suitable to be expressed in natural language\". All that is meant is that it is possible and reasonably practical to express these concepts in words.\n\n\n\n> Can you just explain how you fit the parameters right after you introduce them? I was wondering how you trained the parameters for a few paragraphs.\n\nYes, we can move up the explanation of parameter fitting in Section 4. \n\n> Figure 3: Do you have a higher resolution screenshot?\n\nNo we don't: this image was provided to us by Piantadosi et al. 2016 (with permission).\n\n"}]

Author rebuttal:
Thank you all for your reviews, and especially for the encouragement and constructive criticism. Below we summarize some important strengths and weaknesses identified across the reviews, and overview the new results in the attached PDF that address those weaknesses.

**Strengths:** 

qc38 describes the work as an “important advance in computational cognitive science” which "identifies a major open question (tractability vs expressivity) and addresses it via a key insight (using natural language is now much more feasible with recent LLM advances)". The other reviewers find the work “promising and potentially influential” [y5Ba], "useful for future researchers" across Bayesian AI and deep learning "as a way to take insights from one school of thought and use them to overcome weaknesses of the other school of thought" [ce9a], and contributing a "super cool result" [8GEn]. Experiments “illuminate both the power and limitations” in light of ”strong baselines” [qc38], yielding "results [that] compare favorably with pother recent approaches that have been well-received at NeurIPS" [y5Ba] and “provides a strong fit to human data” [ce9a]. Reviewers mentioned the ability to explain patterns of human mistakes via "explainable failure modes"; the ability to "generalize to totally novel concepts" [qc38]; and last, the "transfer [of] human priors into a neural model" [ce9a]. From a technical perspective, reviewers describe the computational model as one which "takes classic ideas and “'remixes' and integrates them in an interesting way” [y5Ba] via "multiple non-obvious steps” [qc38].

**Weaknesses, and responses (see also attached PDF):**

- qc38 asks "Is there a stronger or more recent baseline for Number Game, vs Latent Language from 2018?", echoed by y5Ba, who requests "a more reasonable neurosymbolic BPL baseline". We have now run DreamCoder, a 2001 neurosymbolic Bayesian Program Learner. We find a nontrivial gap between DreamCoder and our model (see attached PDF), even when DreamCoder is granted 2 orders of magnitude more test-time samples. This finding further supports the conclusion that, relative to prior BPL frameworks, our model can produce more human-like predictions with far fewer samples.


- y5Ba points out that we use closed-source LLMs, which is a reproducibility hazard. qc38 has a slightly different take, noting "reproducibility issues with using GPT4 seem addressed by including the responses in the software/data release" (indeed: we are including it in the data release, as noted in the author checklist). Further addressing this reproducibility hazard is being done by replicating the results using an open source model, Llama-2. The provided PDF shows results on a Llama-2 Number Game model, showing that open LLMs can be used to build a decent model, but that they are currently slightly worse than OpenAI's LLMs as a proposal distribution. (Llama-2 on logical concepts would take more than 2 weeks to complete on the hardware available to us.)


- y5Ba asks "how to do credit assignment to the many potentially brittle component parts of the pipeline" and requests a broader set of LLMs be tried, including weaker ones for different pipeline stages. As a reminder, the pipeline includes prior, likelihood, and proposal distributions. We ablated the prior by not tuning it, and by disabling it completely (for latent language), and also ablated the proposal distribution, but we never ablated the likelihood. The attached PDF shows a new likelihood ablation, revealing that the likelihood is very important. To understand what happens when the individual components are merely made weaker instead of ablated entirely, the new Llama-2 results include data for when individual components are replaced with the weaker Llama-2. The new data show that weaker LLMs can implement effective likelihood models, but stronger LLMs are important for proposal distributions, especially in the low-sample regime. Although the Llama-2 results do not change the scientific conclusions, they are helpful in understanding how to practically engineer systems like ours using off-the-shelf tools.


**Potentially Revised Terminology:** Reviewer y5Ba suggests that the framing as "Bayesian inference over natural language" is technically incorrect, preferring to think of pretraining the LLM as "inference", and our model as predicting based on that already inferred distribution. We're happy to revise our terminology, and understand that Bayesian vocabulary can be quite nuanced (and occasionally contentious!). Right now, we think our original terminology is consistent with adjacent literature, but we can go with whatever terminology the reviewers collectively feel is best.

Meta-review: This paper presents a method that synthesizes computational cognitive science modeling with large language models, crossing discipline boundaries and making a creative new contribution. Reviewers all agree it should be published, with some very strongly advocating for its value. Authors were responsive during the rebuttal, addressing  the reviewer suggestions. Should definitely be presented at NeurIPS, likely as a spotlight or oral.
Decision: Accept (oral)