SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Figure 3 (right panel of orange box) shows illustrates the aleatoric component. It postulates that the participant, having inferred  $\varphi^i$ , commits to the number of clusters first, during which they "slack" with a small probability, reporting  $\hat{K}^i$  clusters according to:
$$
p _ {\mathrm {A}} \left(\hat {K} ^ {i} \mid \varphi^ {i}\right) \propto \left\{ \begin{array}{l} 1, \hat {K} = K ^ {i}; \\ \epsilon , \hat {K} ^ {i} \in \{1, \dots , K _ {\max } \} \backslash K ^ {i}. \end{array} \right. \tag {5}
$$
where  $\epsilon > 0$  is a slack parameter, and  $K_{\mathrm{max}}$  is the maximum number of reported clusters seen in an experiment across participants. If  $\hat{K}^i \neq K^i$ , then the participant removes or splits existing clusters recursively to obtain a set of slacked cluster properties  $\hat{\varphi}^i \coloneqq [\hat{\mathbf{n}}^i, \hat{\boldsymbol{\mu}}^i, \hat{\boldsymbol{\sigma}}^i] =$
$f(\varphi^i,\hat{K}^i)$  until there are  $\hat{K}^i$  clusters; Appendix B.2 presents the details of how  $f$  modifies  $\varphi^i$ . Finally, the likelihood of the reported  $\varphi^r$  given slacked cluster properties are defined as  $P(\varphi^r |\hat{\varphi}^i)\coloneqq 0$  if  $\hat{K}^i\neq K^r$  (infinity penalty for inferring the number of cluster wrong), and
$$
p _ {\mathrm {A}} \left(\boldsymbol {\varphi} ^ {r} \mid \hat {\boldsymbol {\varphi}} ^ {i}\right) := \frac {1}{\left| \mathcal {S} _ {K} \right|} \sum_ {\pi \in \mathcal {S} _ {K}} p _ {w} \left(\mathbf {w} ^ {r}; \pi \left(\hat {\mathbf {n}} ^ {i}\right)\right) p _ {\mu} \left(\boldsymbol {\mu} ^ {r}; \pi \left(\hat {\boldsymbol {\mu}} ^ {i}\right)\right) p _ {\sigma} \left(\boldsymbol {\sigma} ^ {r}; \pi \left(\hat {\boldsymbol {\sigma}} ^ {i}\right)\right) \quad \text {i f} \hat {K} ^ {i} = K ^ {r}, \tag {6}
$$
where  $S_K$  is the set of all permutations of  $K$  elements;  $p_w(\cdot ;\mathbf{n})$  is a Dirichlet distribution with concentration defined by  $\mathbf{n}$ ;  $p_{\mu}(\cdot ;\boldsymbol{\mu})$  is an isotropic normal with mean  $\boldsymbol{\mu}$  and a variance parameter; and  $p_{\sigma}(\cdot ;\boldsymbol{\sigma})$  is an isotropic log-normal with mean  $\boldsymbol{\sigma}$  and a variance parameter. See Appendix B.2 for precise definitions of these distributions and parameters involved.
Overall, the aleatoric component the takes inferred cluster properties  $\varphi^i$  from the rational component and assigns non-zero probabilities to all possible reported distributions through the slack mechanism defined by (5) and the cluster modification  $f$ . The permutation-invariant likelihood (6) regards the weight, mean, and variance vectors as independent conditional on  $\hat{\varphi}^i$ , but retains the dependences between the clusters. To approximate the marginal likelihood, we marginalize out  $\mathbf{z}^i$  and  $\hat{K}^i$  by averaging over a large number of Monte Carlo (MC) simulations. In addition, we allow visual noise by feeding in noisy observations  $\tilde{\mathbf{x}}$ , where  $\tilde{x}_t \sim p_{\mathrm{n}}(\tilde{x}_t|x_t) \coloneqq \mathcal{N}(\tilde{x}_t|x_t,\sigma_v)$ , which is also marginalized out during MC.
Altogether, the DEF estimates the likelihood of the reported  $\varphi^r$  given  $\mathbf{x}_T$  as  $p(\varphi^r | \mathbf{x}_T) \approx$
$$
\frac {1}{M} \sum_ {i = 1} ^ {M} p _ {\mathrm {A}} (\varphi^ {r} | \hat {\varphi} ^ {i} = f (\varphi^ {i}, \hat {K} ^ {i})), \quad \hat {K} ^ {i} \sim p _ {\mathrm {A}} (\hat {K} ^ {i} | \varphi^ {i}), \quad \varphi^ {i} \sim p _ {\mathrm {R}} (\varphi^ {i} | \tilde {\mathbf {x}} _ {T} ^ {i}), \quad \tilde {\mathbf {x}} _ {T} ^ {i} \sim p _ {\mathrm {n}} (\tilde {\mathbf {x}} | \mathbf {x} _ {T}), \quad (7)
$$
where  $M$  is the number of MC simulations (see Appendix B.4 for important distinction to the number of particles). We implemented this MC estimator with a fully vectorized approach using PyTorch [26], which enables easy parallelization on GPUs (Fig. 3, pink box). Compared to a conventional for-loop implementation, ours yields reliable log-likelihood estimates with orders of magnitude acceleration. This allows us to optimize the DEF by maximum-likelihood using any off-the-shelve optimization method, such as Nelder-Mead [27]. Further, it also allows us to critique different behavioral models (DEFs) by model comparison, using metrics such as Akaike Information Criterion (AIC) [28].
4 Experiments: fitting and comparing internal construct priors
We compare the following ICPs by fitting them in the DEFs that share the same class of aleatoric component: a) our proposed economical ICP as described in Section 3.1; b) its special case, the exchangeable CRP-GMM; and c) a baseline batch learning prior that describes a non-sequential cluster assignment; see Appendix B.1.3 for details. We use the Nealder-Mead optimizer to fit the DEF parameters for each participant, restarting multiple times to avoid early convergence issues [29]. We choose a large number  $M$  for MC simulations so that the estimator (7) produces small enough variance and bias tolerable for the Nelder-Mead optimizer. After fitting, we compare different DEFs using a much larger  $M$ . Details of the fitting algorithm are deferred to Appendix B.3 where we also present additional experiments validating our overall optimization scheme.
The quality of the fitted DEFs on the three experiments are shown in Fig. 4. The economical ICP produced significantly lower AICs than the exchangeable CRP-GMM in all three experiments (Exp. 1: median AICs -2627.9 vs. -2583.1,  $p < 10^{-4}$ ; Exp. 2: -1766.7 vs. -1697.9,  $p < 10^{-3}$ ; Exp. 3: -148.2 vs. -141.0,  $p < 0.01$ ; Wilcoxon signed-rank test). We compare four aspects of the predictions in Fig. 4: the error rate in predicting the number of clusters, the negative log-likelihood (NLL) of the weights, and the expected normalized error in predicting mean and log-variance predictions using the slacked predictions given  $\hat{K}^i = K^r$ . The advantage of the economical ICP is not only in correctly predicting the number of clusters but also in estimating the cluster properties. As expected, the batch ICP had much worse AICs. Further, to test whether simpler models can explain the data better, we performed an ablation study based on the CRP-GMM ICP (Appendix B.5.1). The ablated ICPs neglect various aspects of the probabilistic structures, effectively mimicking different heuristic
Figure 4: Quality of three ICPs on three experiments. Thick blue lines and whiskers are means  $\pm 1$  sem over all participants (thin lines). Red crosses are medians. Asterisk (*) indicates significance at  $p < 0.05$  by Wilcoxon signed-rank test.
strategies in forming the internal construct. None of the ablated models outperforms the economical ICP.
Example predictions of the two sequential ICPs are shown in Fig. 5A. The DEF with the economical ICP captures all of the behavioral patterns exhibited by participants, including the inconsistent number of clusters, the low overlap ratio between adjacent clusters, the moments of the reported distribution (Figs. 6 to 8), and the covariance of the reported cluster properties (Fig. 5B-D). We found largely consistent patterns in three fitted parameters across experiments (Fig. 5E-G). First, in line with our resource-rational motivation, the decay rates  $r$  in (1) are greater than 0, implying that model expansion is suppressed as  $\alpha_{t}$  decays when the internal construct becomes more complex. Second, participants' prior sd  $\sqrt{\sigma_0}$  in (2) was  $20\% - 35\%$  of the true density's sd regardless of the true scale of the observation, with a high confidence  $a_0$  (see Appendix B.5.2). This means that each cluster is kept narrow, requiring a new cluster to account for an observation far from existing clusters. It explains the bumpiness in the reported internal constructs, and also suggests that participants adaptively adjusted the cluster scale in accordance with the data scale and modality. Further, the observation that participants reported around 3 clusters is likely the trade-off between the opposite forces of a decaying  $\alpha_{t}$  and a small but confident  $\sigma_0$ . Third, we found the cluster distortion  $\beta$  in (1) to be less than 1, indicating a reduced sensitivity to cluster size when forming the internal construct online.
We studied density estimation by humans, a sophisticated capability fundamental to a variety of cognitive tasks but has so far been elusive in experiments and implicitly assumed in modeling. Through behavioral experiments, we discovered systematic inconsistencies in humans' internal constructs of environments (Section 2) and proposed a density estimation framework for modeling this rich behavioral dataset (Section 3). We explained the inconsistencies as a result of an economically-expanding internal construct prior (ICP) over internal constructs and a strong preference for narrow clusters [30]. The scale-adaptive cluster prior aligns with prior research on adaptive sensory and probability coding [31-33, 30, 34].
The density estimation framework (DEF) is a generic approach for modeling complex behavioral data. Thanks to the generality, one can compare other inference algorithms based on likelihoods and can easily incorporate priors over parameters (hyperpriors). While previous work kept the trainable parameters small and/or used error-based objective functions [15, 35, 4], our DEF can fit more parameters by maximum-likelihood, marginalizing the
Figure 5: A, Example DEF predictions for Experiment 2 when the predicted number of clusters agree with human reports. B-C, Detailed patterns of the reported cluster properties in Experiment 2, which can be successfully captured by economical DEF. In each panel, the grey heatmap denotes the distribution of responses of individual trials (collapsed across participants). The colored dots and error bars denote average response and standard error across participants in local data bins. The line denotes the regression line. B, the cluster sd increased with the cluster weight. C, The cluster sd decreased with the cluster eccentricity (distance between the cluster center and the global center). D, The cluster sd decreases with the number of reported clusters. E-G, Upper panels show the effects of key parameters in the economical ICP; lower panels show the parameters fit on the data of the three experiments. More visualizations of DEF predictions appear in Figs. 6 to 8 in Appendix A
latent cluster assignments properly, thanks to our efficient implementation and hardware acceleration. With this methodology, we showed that the economical ICP provided a closer match to human behavior than the Chinese restaurant process (CRP) which has been used pervasively in modeling online learning. With a decaying rate of expansion, the growth of an economical ICP is more in line with humans' finite memory capacity.
Our work sends important messages to the theoretical community. Dasgupta and Griffiths [32] showed that the CRP is cognitively plausible because it can be derived from a preference for low entropy in the cluster assignment distribution. We found that the fitted  $\beta < 1$  actually increases the entropy of the prior cluster weight distribution [20, 23], suggesting that the cluster count might be the main consumption of cognitive resources. Gershman et al. [16] discussed in great depth on batch versus online learning, whether the model capacity should be finite or infinite, and the hypothesis that retrospective cluster reassignment may be possible with particle representations. We provide key experimental and modeling evidence that humans may employ an online and yet finite (in expectation) model. However, with only a few number of particles, the beliefs of the other possible assignments are not well retained, so retrospective corrections may not be easily produced from these models.
In sum, the economical DEF reproduces many human patterns, not only in the loss function optimized for (quality of capturing human structure learning), but also in the moments of the whole distribution and the covariance structures of cluster properties. We note the large room for improvement in predicting the cluster variance (the right column in Fig. 4), which hints at the complexity of the task and the challenge in modeling. Our work also provides a comprehensive likelihood-based model comparisons on human density estimation, and paves way for future studies using high-dimensional reports in complex behavioral tasks.
This work was partly supported by the National Natural Science Foundation of China (32171095), National Science and Technology Innovation 2030 Major Program (2022ZD0204803), and funding from Peking-Tsinghua Center for Life Sciences to H.Z. L.K.W. was supported by Gatsby Charitable Foundation. We thank Peter Dayan and Jian Li for helpful discussions, and five anonymous reviewers for their valuable feedback.
[1] Marc O Ernst and Martin S Banks. Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 2002.
[2] Yair Weiss, Eero P Simoncelli, and Edward H Adelson. Motion illusions as optimal percepts. Nature neuroscience, 2002.
[3] Konrad P Körding and Daniel M Wolpert. Bayesian decision theory in sensorimotor control. Trends in cognitive sciences, 2006.
[4] Jake Spicer, Adam N Sanborn, and Ulrik R Beierholm. Using occam's razor and bayesian modelling to compare discrete and continuous representations in numerosity judgements. Cognitive Psychology, 2020.
[5] Aaron C Courville and Nathaniel Daw. The rat as particle filter. Advances in neural information processing systems, 2007.
[6] Thomas L Griffiths, Adam N Sanborn, Kevin R Canini, and Daniel J Navarro. Categorization as nonparametric bayesian density estimation. The probabilistic mind: Prospects for Bayesian cognitive science, 2008.
[7] Nathaniel Daw and Aaron Courville. The pigeon as particle filter. Advances in neural information processing systems, 2008.
[8] Hang Zhang, Nathaniel D Daw, and Laurence T Maloney. Human representation of visuo-motor uncertainty as mixtures of orthogonal basis distributions. Nature Neuroscience, 2015.
[9] S. J. Gershman, E. J. Horvitz, and J. B. Tenenbaum. Computational rationality: A converging paradigm for intelligence in brains, minds, and machines. Science, 2015.
[10] Kelsey Allen, Evan Shelhamer, Hanul Shin, and Joshua Tenenbaum. Infinite mixture prototypes for few-shot learning. In International Conference on Machine Learning, 2019.
[11] David J Aldous, Illdar A Ibragimov, Jean Jacod, and David J Aldous. Exchangeability and related topics. Springer, 1985.
[12] Carl Rasmussen. The infinite gaussian mixture model. Advances in neural information processing systems, 1999.
[13] Radford M Neal. Markov chain sampling methods for dirichlet process mixture models. Journal of computational and graphical statistics, 2000.
[14] John R. Anderson. The adaptive nature of human categorization. Psychological Review, 1991.
[15] Adam N Sanborn, Thomas L Griffiths, and Daniel J Navarro. Rational approximations to rational models: alternative algorithms for category learning. Psychological Review, 2010.
[16] Samuel J Gershman, David M Blei, and Yael Niv. Context, learning, and extinction. *Psychological Review*, 2010.
[17] C. Kemp and J. B. Tenenbaum. The discovery of structural form. Proceedings of the National Academy of Sciences, 2008.
[18] Samuel J Gershman and David M Blei. A tutorial on bayesian nonparametric models. Journal of Mathematical Psychology, 2012.
[19] Bennet B Murdock Jr. The serial position effect of free recall. Journal of experimental psychology, 1962.
[20] J. Sun, J. Li, and H. Zhang. Human representation of multimodal distributions as clusters of samples. PLoS Computational Biology, 2019.
[21] Konrad Körding and Daniel M. Wolpert. The loss function of sensorimotor learning. Proceedings of the National Academy of Sciences, 2004.
[22] Konrad Körding and Daniel M Wolpert. Probabilistic Inference in Human Sensorimotor Processing. In Advances in Neural Information Processing Systems, 2003.
[23] Yeon Soon Shin and Yael Niv. Biased evaluations emerge from inferring hidden causes. Nature Human Behaviour, 2021.
[24] L. Acerbi, S. Vijayakumar, and D. M. Wolpert. On the origins of suboptimality in human probabilistic inference. PLoS Computational Biology, 2014.
[25] S. J. Gershman and Y. Niv. Perceptual estimation obeys Occam's razor. Frontiers in Psychology, 2013.
[26] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 2019.
[27] John A Nelder and Roger Mead. A simplex method for function minimization. The computer journal, 7(4):308-313, 1965.
[28] Hirotogu Akaike. Information theory and an extension of the maximum likelihood principle. Selected papers of hirotugu akaike, pages 199-213, 1998.
[29] William H Press, Saul A Teukolsky, William T Vetterling, and Brian P Flannery. Numerical recipes 3rd edition: The art of scientific computing. Cambridge university press, 2007.
[30] Hang Zhang, Xiangjuan Ren, and Laurence T. Maloney. The bounded rationality of probability distortion. Proceedings of the National Academy of Sciences, 2020.
[31] Horace B. Barlow. Possible principles underlying the transformation of sensory messages. Sensory communication, 1961.
[32] Ishita Dasgupta and Thomas L Griffiths. Clustering and the efficient use of cognitive resources. Journal of Mathematical Psychology, 2022.
[33] R. Polania, M. Woodford, and C. C. Ruff. Efficient coding of subjective value. Nature Neuroscience, 2019.
[34] C. R. Sims. Efficient coding explains the universal law of generalization in human perception. Science, 2018.
[35] Samuel J Gershman and Yael Niv. Perceptual estimation obeys occam's razor. Frontiers in psychology, 2013.
Bounded rationality in structured density estimation: Supplementary material
A.1 Experiment 1
A.1.1 Participants
Experiment 1 recruited 21 participants (11 females, aged 18-25). All participants had provided informed consent before the experiment.
A.1.2 Cover story
Participants were told that they were apprentice magicians in a magical world. In this world, dangerous magic lava rocks were emitted from an unknown number of invisible volcano(es). On each trial, they observed past landing locations of lava rocks in a specific area (on the screen), and their job was to predict the probability density of future landing locations. More specifically, they were asked to draw a probability density by reporting, using click-and-drag mouse gestures, three key properties of the volcano(es), corresponding to the mean, the weight, and the standard deviation of a Gaussian component. They were told that their bonus payment depended on the accuracy of the reported predictive density.
A.1.3 Procedure and Design
On each trial, the landing positions of lava rocks were visualized as red dots and sequentially presented on a black line. The number of presented rocks (i.e., sample size) had two levels: 10 or 70. Each dot was presented for  $166\mathrm{ms}$ , followed by a  $166\mathrm{ms}$  empty screen. The landing positions were i.i.d. samples drawn from an unseen mixture distribution.
After observing all dots, participants needed to follow a two-stage procedure to report both (1) the predictive density of rocks' future landing positions, and (2) the underlying generative model. First, they marked the location of the volcano(es) by clicking on the black line. After marking all volcano(es), they pressed "F" to proceed to the next reporting stage. In the second stage, they chose a volcano by clicking the volcano icon and then moved the mouse to report the relative density of the landing position of future lava rocks emitted from this volcano. During the report, the overall density of the rock landing position was computed and presented in real-ime.
The true generative distribution set was composed of 24 distributions (Fig. 6A). The distributions were created by following a merge-from-four procedure. All distributions' overall standard deviations were about  $5.3\mathrm{cm}$ . On each trial, a uniform jitter  $\left(-5.3\mathrm{cm}, + 5.3\mathrm{cm}\right)$  was added to the mean of the true generative distribution.
Participants completed 2 (sample size levels)  $\times 4$  (true cluster number levels)  $\times 6$  (distribution subtype)  $\times 3$  (number of repeats)  $= 144$  trials in total. The experiment length was about 105 minutes.
A.1.4 Generation of true distribution set
The set of true distributions was constructed following the steps below. First, we created 6 four-cluster Gaussian mixture distributions (bottom row in Fig. 6A) in which each component had the same SD of  $0.72\mathrm{cm}$  of visual angle and equal weight. The three center-to-center distances between their adjacent Gaussian components, denoted  $[d_1,d_2,d_3]$  (from left to right, measured in cm), were chosen from the set of all permutations of  $\{3.6,4.65,5.7\} \mathrm{cm}$ .
Second, each of the 6 four-cluster Gaussian mixtures went through "merging steps", inspired by the proposal step in the dynamic clustering algorithms (e.g. Reverse-jump MCMC) in statistics [36]. In each merging step, we chose two adjacent Gaussian components to merge into one, with the post-merger new component having the same zeroth, first and second
True distribution: mixture of 3 Gaussian clusters
True distribution: mixture of 4 Gaussian clusters
Figure 6: Detailed experimental design and results in Experiment 1. A, The distribution set. B-F shows the prediction of the fitted economical DEFs compared to participants' reports under different conditions. B, The relative frequency of the reported cluster number. C, The average reported cluster number. D, The overlap ratio between the reported clusters. E, The reported moments versus the sample moments in the 10 sample-size condition, with data (left sub-panels) contrasted with model prediction (right sub-panels). Three rows are for mean, sd, and skewness. In each panel, the grey heatmap denotes the distribution of responses of individual trials (collapsed across subjects). The 5 dots and error bars denote average response and standard error across subjects in 5 local data bins. The line denotes the regression line, with shading representing  $95\%$  confidence interval. F, The reported moments versus the sample moments in the 70 sample-size condition
moments as the combination of the two pre-merger components. The to-be-merged adjacent components were chosen in such a way that the post-merger Gaussian mixtures minimize KL-divergence  $\mathrm{KL}(p_{\mathrm{pre}}||p_{\mathrm{post}})$ . By applying the merging step iteratively, the original four-cluster Gaussian mixtures were transformed into three-cluster, two-cluster, and finally one-cluster mixtures.
A.1.5 Results
See Fig. 6 for the prediction of the fitted economical DEF.
A.2 Experiment 2
Experiment 2 recruited 21 participants (13 females, aged 18-25). All participants had provided informed consent before the experiment. One participant was excluded due to
Table 1: The distribution set in Experiment 2
Distribution	Paper	Skewed or Symmetric	Number of clusters	Number of modes
1	[24]	Symmetric	1	Unimodal
2	[24]	Symmetric	-*	Unimodal
3	[25]	Symmetric	2	Unimodal
4	[21]	Skewed	2	Unimodal
5	[21]	Skewed	2	Unimodal
6	[21]	Skewed	2	Unimodal
7	[25]	Skewed	2	Unimodal
8	[23]	Skewed	2	Bimodal
9	[25]	Symmetric	2	Bimodal
10	[22]	Symmetric	2	Bimodal
11	[22]	Symmetric	3	Trimodal
12	[20]	Skewed	3	Trimodal
13	[20]	Skewed	3	Trimodal
14	[20]	Skewed	3	Trimodal
* This is a uniform distribution with smoothed edges.
their task performance being an outlier (measured by the earth-mover distance between the reported predictive density and the true density, exceeding 3 standard deviations, z-score  $= -3.3$ ).