SlowGuess/NeurIPS_2023_OCR huggingface dataset:
(e)
Figure 4: Panels (a-b) show learning curves averaged across sequences as a function of time. Panel (a) shows relative error over time and (b) shows the proportion of errors lower than the "acceptance distance"  $D_{c}$  used to determine correctness in the human experiment. Note that error is defined as the distance between the prediction and the true point relative to the distance between the true point and the previous point in the sequence. Errorbars on human data points are bootstrapped  $95\%$  confidence intervals. Panels (c-f) show the correlation between human error (y-axis) and model error (x-axis) for all sequences tested. Panel (c) shows the GP-NC model; (d) shows the Bayesian Ridge regression; (e) shows the GP-SL model; and (f) shows the LoT model.
can also represent much more complex sequences at the cost of program length. The expressiveness of this grammar affords flexibility in the representations learned by the LoT model, but also presents a challenge for inference as it defines an extremely large, discrete search space. To address this challenge we defined several custom proposal moves on the program tree structure to more effectively explore the search space: in addition to subtree regeneration moves, at each rejuvenation step we include moves which swap single nodes with nodes of a different type, remove nodes and replace them with one of their children, and insert subtrees between parent and child nodes [23]. Involutive MCMC allows custom kernels as long as you can specify an invertible function that maps from any proposal back to the initial hypothesis, and we used Gen's built-in correctness checks to ensure there were no bugs in involution [21].
During inference, the posterior probability of a particle is computed based on its prior probability and likelihood. The prior is determined by the PCFG from which the program tree is generated, gamma distributions over the noise parameters and hyperparameters, and distributions over initial values of the state variables  $\theta$  and  $s$  (see the Supplementary Materials for a more detailed description). The likelihood is decomposed by the output dimension  $(x$  or  $y)$ , and for a single dimension is computed based on the probability of drawing, at each timepoint, the true location from a normal distribution centered around the predicted location at that timepoint, with the variance determined by the inferred noise parameters.
We analyze the performance of each model on our task with reference to human performance. We propose that the representational flexibility of the two program synthesis models is key to learning in this deceptively simple domain. We also aim to elucidate the nature and degree of structure in participants' inductive biases based on the explanatory power of each model.
3.3 Model comparison
The top row of Figure 4 (panels a-b) show aggregate learning curves for the models and participants over time, averaged across all sequences. Panel (a) shows relative error over time and panel (b) shows
Figure 5: Absolute error relative to the distance from the previous point across all sequences tested for LoT model (orange) and human data (gray bars). The sequence corresponding to the data is shown in the top-right of each panel.
the proportion of absolute errors less than the "acceptance distance"  $D_{c}$  used to determine correctness in the human experiment. The GP-NC model starts off with very high error and plateaus in learning at about timepoint 10, with roughly no subsequent improvement beyond. The Bayesian Ridge regression model performs similarly, though with slightly lower relative error after about timepoint 10. On the other hand, both the LoT model and the GP-SL models continuously improve, perhaps plateauing toward the final timepoint, and are both roughly in line with aggregate human performance. Panels (c-f) compare the average relative error of humans (y-axis) and each model (x-axis) across sequences, to determine the extent to which people and the model had difficulty with the same sequences. In order of lowest to highest correlation were the Ridge model ( $r = 0.23$ ;  $p < 0.001$ ), GP-NC model ( $r = 0.48$ ;  $p < 0.001$ ), GP-SL model ( $r = 0.50$ ;  $p < 0.001$ ) and the LoT model ( $r = 0.56$ ;  $p < 0.001$ ).
To quantitatively compare the fit of the models to the human data, we first fit parameters to capture two types of additional noise in the human data that the models were not intended to capture. Specifically, we assumed that participants' clicks had added motor noise and variability stemming from perceptual imprecision, and we also assumed that on some proportion of trials, participants may not have been attending or making a serious effort. We thus optimized the likelihood of participants' responses for each model assuming they were drawn from a mixture model of the form,
$$
\hat {x} _ {t + 1} \sim (1 - \theta_ {l}) \cdot \mathcal {N} (\mu_ {x, t + 1}, \sigma_ {x, t + 1} ^ {2} + \sigma_ {m} ^ {2}) + \theta_ {l} \cdot \mathcal {N} (x _ {t}, \sigma_ {l} ^ {2}) \tag {1}
$$
$$
\hat {y} _ {t + 1} \sim (1 - \theta_ {l}) \cdot \mathcal {N} \left(\mu_ {y, t + 1}, \sigma_ {y, t + 1} ^ {2} + \sigma_ {m} ^ {2}\right) + \theta_ {l} \cdot \mathcal {N} \left(y _ {t}, \sigma_ {l} ^ {2}\right), \tag {2}
$$
where the free parameters are the motor noise term  $\sigma_{m}$ , a lapse rate  $\theta_{l}$ , and noise in responses on lapse trials  $\sigma_{l}$ . The models' predictions for the  $x$  and  $y$  coordinates at time  $t + 1$  are given by  $\mu_{x,t + 1}$  and  $\mu_{y,t + 1}$ , respectively, and the predicted standard deviations are given by  $\sigma_{x,t + 1}$  and  $\sigma_{y,t + 1}$ . Because the Bayesian Ridge regression and GP models required two training points to make predictions, we compared each model's fit to the data from after the second time-point for each sequence. We used R's built-in 'optim' function for fitting [24].
Model type	σm	θl	σl	LL	ΔAIC
GP-NC	0.05	0.32	2.28	-31592	19514
Ridge	0.01	0.20	1.96	-30187	16704
GP-SL	0.16	0.20	1.60	-28416	13162
LoT	0.01	0.24	1.74	-21835	0
Table 2: Model parameters, Log Likelihood, and  $\Delta$  AIC from the best-fitting model.
Table 2 shows the fit parameters for each model, the resulting log-likelihood of the human data, and the  $\Delta$ AIC from the best-fitting model. The worst-fitting model was the GP-NC model, followed by the Bayesian Ridge regression. The GP-SL model and the LoT model both learned many more of the sequences, but the LoT model provided a significantly better fit to the human data, with a lower AIC score by 13162 (where 10 is often considered a "significant" difference).
Figure 5 shows the relative error (y-axis) at each timepoint (x-axis) made by people (gray bars/points) and the LoT model (orange line) for all sequences tested. The learning rate of the LoT model aligns quite closely with that of people for many of the sequences tested, which is more impressive considering that this figure shows model predictions absent any parameters fit to the data. Analogous figures for the other models are shown in the Supplementary Materials. Figure 6 provides a few illustrative examples of individual timepoints in various sequences of participants' (red points) and the particles of the LoT model's (blue points) predictions. These were chosen to highlight cases where the LoT model captures patterns of successes and errors in the human data, but also show cases where the predictions of the LoT model differ from people (rightmost column). The LoT model reproduces certain biases evident in the human data, such as biases toward repeating previous changes in angle or position, that result in multimodal clusters of uncertainty. The LoT model's uncertainty clusters do not always align with people's, however, and in general the model is more strongly biased toward simple functions that are a poor fit to the data early on in a sequence (e.g. learning just the function Move(), as in the failure case at the top of the rightmost column). All the models' predictions for each sequence across all timepoints are provided in the Supplementary Materials.
Our results have shown that people can learn a variety of algorithmically-structured 2D sequences from a small number of data points. Yet, while participants learned nearly all the sequences by the final timepoint, many of the sequences we tested were not learned by a Gaussian Process model which selects from several different kernels. It is therefore an interesting question, given that participants were never exposed to these exact sequences, what makes people so adept at learning seemingly arbitrary patterns that non-parametric function approximators struggled to fit. One possible answer is that people have been exposed to similar stimuli in their past, and thus can solve the task with visual pattern recognition. Yet the space of potential sequences we could have picked that might look "familiar" to many people is vast, and thus their success seems unlikely to be attributable to a lifetime of data about, e.g., the space of all possible zig-zagging sequences. Even with large amounts of relevant data, however, amortizing useful inductive biases for this task may be difficult (though see [25, 26]). As a preliminary test, we trained an LSTM to predict thousands of sequences (sampled from the LoT model), which it did relatively well, but nonetheless generalized poorly to the sequences used in our experiment (see the Supplementary Materials).
We propose instead that people's success at learning algorithmic patterns comes from powerful inference mechanisms that are broadly applicable across many domains, ranging from those that are useful for physical reasoning (e.g. perceiving occluders [27]), to those that support more abstract human inventions (e.g., geometry [28]). Namely, people are able to infer "mental programs" to represent latent structure and uncertainty in data. This idea is supported by the success of the two Bayesian program synthesis models at learning the vast majority of the sequences we tested at a rate similar to people, in aggregate. In addition to aligning with the speed of human learning, the LoT model in particular — which explicitly represents sequences as an algorithm specifying how a point's trajectory should change in angle and speed as a function of time — recapitulates many human-like biases as well. For instance, the LoT model is biased toward repetition and "simple" angles (e.g. 90-degree or 45 degree). Gaussian process kernels, on the other hand, have no biases (or very weak ones) toward geometric regularities and symmetries.
Whether other animals, including even our close primate ancestors, represent structure in visual patterns in the same way as people is a particularly interesting unsettled question [29, 30]. Recent evidence suggests, in fact, that non-human primates do not represent shapes in the same way that humans do [31]. The authors of that study used an oddball paradigm to determine how western-educated adults, kindergartners, Himba adults without formal schooling, and baboons distinguished outlier geometric shapes in a set. They found that while all human subjects showed a geometric regularity effect, such that more regular geometric shapes were more easily detected as outliers, this was not the case for the baboons. The authors interpreted this as evidence that humans engage a "logical, recursive mode of representation akin to a programming language" absent in other animals.
Though our model comparison results primarily shed light on the nature and degree of structure in people's hypotheses, a key aspect of each of our models is that they reason probabilistically over these hypotheses. We see signatures of this probabilistic reasoning in human responses as well. Qualitatively, the patterns of multimodal uncertainty evident in the human data are reflective of a distribution over distinct hypotheses. For example, Figure 1 illustrates a case in which different
Figure 6: A set of examples of human (red) and LoT model (blue) predictions (SMC particles) chosen to highlight interesting cases. Examples in the rightmost column illustrate cases where the model makes predictions that are both wrong and also diverge from people. The transparency of the LoT model points is set by the posterior probability of the particle. The previous point in the sequence is shown as a black dot and earlier points in the sequence are shown in lighter gray. The true next point (i.e., ground-truth location) is shown as a gold star.
participants' responses are consistent with alternative hypotheses involving repetition of observed sequence substructures and features of geometric regularity. This is most evident at the third timepoint, with the three previous points forming a right angle. Participants' predictions are clustered around three points: left and up, straight, or right and down. That individuals seem to pursue distinct, well specified hypotheses, while in aggregate human response patterns indicate uncertainty in inference, is well captured by accounts of human learning and decision-making which hold that human predictions are based on one or few samples from a probability distribution over hypotheses [32].
In this work, our analyses have been at the computational level, testing whether the problem being solved by participants might be specified as Bayesian inference over a particular class of programmatic hypotheses. However, other work suggests that our models might additionally capture key algorithmic aspects of human inference. In particular, Thaker et al. [33] found that SMC offers a plausible algorithmic account of how people track multiple hypotheses over time, with limited-size particle filters explaining order effects in learning. Our models also employ MCMC rejuvenation moves, which Dasgupta et al. [34] have identified as an algorithmic explanation for dependencies between people's sequentially generated hypotheses [34]. Even more specifically, Franken et al. [35] present evidence that people iteratively modify hypotheses through processes akin to the tree-surgery proposal moves utilized by the LoT model during MCMC rejuvenation. Though we adopted these specific techniques because of their usefulness in navigating large, discrete hypothesis spaces, as opposed to their plausibility as algorithmic models of human behavior, this convergence speaks to the fruitfulness of jointly developing efficient modeling techniques and algorithmic accounts of human inference.
Finally, it is worth noting that people's inductive biases may have been shaped during the experiment by the specific stimuli we tested, which all had some geometric regularity and were free of noise (although the log likelihood of participant predictions under the LoT model actually decreases very slightly as they progress through trials  $(r = -0.016, p < .05)$ ). The ability to construct and adaptively update domain-specific hypothesis spaces is a remarkable feat of human learning which the field of program synthesis has only begun to make progress toward incorporating [2, 9, 16, 36, 37]. Despite the expressiveness of the GP-SL and LoT models, they are unable to capture the range of hypotheses participants might theoretically consider (as an extreme example, we might imagine sequences which begin to spell out words). Computationally formalizing people's ability to flexibly construct and revise hypothesis spaces during online probabilistic inference is an exciting direction for future work in both cognitive science and machine learning, and the domain of 2D sequence learning provides a rich test bed for understanding the sensitivity of people's hypothesis spaces to variations in underlying data generating processes.
This work was supported by the AFOSR Grant No. FA9550-19-1-0269, the DARPA Machine Common Sense Program, the ONR Science of AI Program, and Siegel Family Endowment. We would like to thank Mathias Sablé-Meyer, Stan Dehaene, Vikash Mansinghka, and especially Feras Saad for helpful and instructive conversations.
[1] J. A. Fodor. The language of thought. Cambridge, MA: Harvard University Press, 1975. ISBN: 0-674-51030-5.
[2] Joshua S Rule, Joshua B Tenenbaum, and Steven T Piantadosi. “The Child as Hacker”. In: Trends in Cognitive Sciences 24.11 (2020), p. 901.
[3] Noah D Goodman et al. “A rational analysis of rule-based concept learning”. In: Cognitive science 32.1 (2008), pp. 108–154.
[4] Steven T Piantadosi, Joshua B Tenenbaum, and Noah D Goodman. “The logical primitives of thought: Empirical foundations for compositional cognitive models.” In: Psychological review 123.4 (2016), p. 392.
[5] Brenden M Lake, Ruslan Salakhutdinov, and Joshua B Tenenbaum. "Human-level concept learning through probabilistic program induction". In: Science 350.6266 (2015), pp. 1332-1338.
[6] Steven T Piantadosi, Joshua B Tenenbaum, and Noah D Goodman. "Bootstrapping in a language of thought: A formal model of numerical concept learning". In: Cognition 123.2 (2012), pp. 199-217.
[7] Francis Mollica and Steven T Piantadosi. "Logical word learning: The case of kinship". In: Psychonomic Bulletin & Review (2019), pp. 1-34.
[8] Marie Amalric et al. "The language of geometry: Fast comprehension of geometrical primitives and rules in human adults and preschoolers". In: PLoS computational biology 13.1 (2017), e1005273.
[9] Joshua Rule et al. “Learning list concepts through program induction”. In: Proceedings of the Cognitive Science Society. 2018.
[10] Anselm Rothe, Brenden M Lake, and Todd Gureckis. "Question asking as program generation". In: Advances in neural information processing systems 30 (2017).
[11] Eric Schulz et al. "Compositional inductive biases in function learning". In: Cognitive psychology 99 (2017), pp. 44-79.
[12] Thomas L Griffiths et al. "Modeling human function learning with Gaussian processes". In: Advances in neural information processing systems 21 (2008).
[13] Christopher G Lucas et al. “A rational model of function learning”. In: Psychonomic bulletin & review 22.5 (2015), pp. 1193–1215.
[14] David Duvenaud et al. "Structure discovery in nonparametric regression through compositional kernel search". In: International Conference on Machine Learning. PMLR. 2013, pp. 1166-1174.
[15] Feras A. Saad et al. "Bayesian Synthesis of Probabilistic Programs For Automatic Data Modeling". In: Proceedings of the ACM on Programming Languages 3.POPL (2019), pp. 37.1-37.32. DOI: 10.1145/3290350.
[16] Michael Y Li et al. "Learning to Learn Functions". In: Cognitive Science 47.4 (2023), e13262.
[17] Mathias Sablé-Meyer et al. “A language of thought for the mental representation of geometric shapes”. In: Cognitive psychology. 139 (2022-12). ISSN: 0010-0285.
[18] J Douglas Carroll. "Functional learning: The learning of continuous functional mappings relating stimulus and response continua". In: ETS Research Bulletin Series 1963.2 (1963), pp. i-144.
[19] Kyunghee Koh and David E Meyer. "Function learning: induction of continuous stimulus-response relations." In: Journal of Experimental Psychology: Learning, Memory, and Cognition 17.5 (1991), p. 811.
[20] Edward L DeLosh, Jerome R Busemeyer, and Mark A McDaniel. "Extrapolation: the sine qua non for abstraction in function learning." In: Journal of Experimental Psychology: Learning, Memory, and Cognition 23.4 (1997), p. 968.
[21] Marco F. Cusumano-Towner et al. "Gen: A General-purpose Probabilistic Programming System with Programmable Inference". In: Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation. PLDI 2019. Phoenix, AZ, USA: ACM, 2019, pp. 221-236. ISBN: 978-1-4503-6712-7. DOI: 10.1145/3314221.3314642. URL: http://doi.acm.org/10.1145/3314221.3314642.
[22] Feras A. Saad et al. "Sequential Monte Carlo Learning for Time Series Structure Discovery". In: International Conference on Machine Learning. PMLR. 2023.
[23] Ying Jin et al. Bayesian Symbolic Regression. 2020. arXiv: 1910.08892 [stat.ME].
[24] R Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing. Vienna, Austria, 2020. URL: https://www.R-project.org/.
[25] Declan Campbell et al. "Relational Constraints On Neural Networks Reproduce Human Biases towards Abstract Geometric Regularity". In: arXiv preprint arXiv:2309.17363 (2023).
[26] Taylor W Webb et al. "The relational bottleneck as an inductive bias for efficient abstraction". In: arXiv preprint arXiv:2309.06629 (2023).
[27] Gaetano Kanizsa. Organization in vision: Essays on Gestalt perception. Praeger Publishers, 1979.
[28] Euclid. Elements. Trans. by Thomas Heath. Original work written c. 300 BC. Courier Corporation, 1956.
[29] Xinjian Jiang et al. "Production of supra-regular spatial sequences by macaque monkeys". In: Current Biology 28.12 (2018), pp. 1851–1859.
[30] Stephen Ferrigno et al. "Recursive sequence generation in monkeys, children, US adults, and native Amazonians". In: Science Advances 6.26 (2020), eaaz1002.
[31] Mathias Sablé-Meyer et al. "Sensitivity to geometric shape regularity in humans and baboons: A putative signature of human singularity". In: Proceedings of the National Academy of Sciences 118.16 (2021), e2023123118.
[32] Edward Vul et al. "One and Done? Optimal Decisions From Very Few Samples". In: Cognitive Science 38.4 (2014), pp. 599-637. DOI: https://doi.org/10.1111/cogs.12101. eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/cogs.12101. URL: https://onlinelibrary.wiley.com/doi/abs/10.1111/cogs.12101.
[33] Pratiksha Thaker, Joshua B Tenenbaum, and Samuel J Gershman. "Online learning of symbolic concepts". In: Journal of Mathematical Psychology 77 (2017), pp. 10-20.
[34] Ishita Dasgupta, Eric Schulz, and Samuel J Gershman. “Where do hypotheses come from?” In: Cognitive psychology 96 (2017), pp. 1–25.
[35] Jan-Philipp Franken, Nikos C Theodoropoulos, and Neil R Bramley. "Algorithms of adaptation in inductive inference". In: Cognitive Psychology 137 (2022), p. 101506.
[36] Kevin Ellis et al. "Learning to infer graphics programs from hand-drawn images". In: arXiv preprint arXiv:1707.09627 (2017).
[37] Kevin Ellis et al. "Dreamcoder: Growing generalizable, interpretable knowledge with wake-sleep bayesian program learning". In: arXiv preprint arXiv:2006.08381 (2020).