SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Tianyuan Teng*
Center for Life Sciences,
tengtianyuan@pku.edu.cn
Li Kevin Wenliang*,
Gatsby Computational Neuroscience Unit,
kevinli@gatsby.ucl.ac.uk
Hang Zhang
School of Psychological and Cognitive
Sciences, Peking University
hang.zhang@pku.edu.cn
Learning to accurately represent environmental uncertainty is crucial for adaptive and optimal behaviors in various cognitive tasks. However, it remains unclear how the human brain, constrained by finite cognitive resources, internalise the highly structured environmental uncertainty. In this study, we explore how these learned distributions deviate from the ground truth, resulting in observable inconsistency in a novel structured density estimation task. During each trial, human participants were asked to learn and report the latent probability distribution functions underlying sequentially presented independent observations. As the number of observations increased, the reported predictive density became closer to the ground truth. Nevertheless, we observed an intriguing inconsistency in human structure estimation, specifically a large error in the number of reported clusters. Such inconsistency is invariant to the scale of the distribution and persists across stimulus modalities. We modeled uncertainty learning as approximate Bayesian inference in a nonparametric mixture prior of distributions. Human reports were best explained under resource rationality embodied in a decaying tendency towards model expansion. Our study offers insights into human cognitive processes under uncertainty and lays the groundwork for further exploration of resource-rational representations in the brain under more complex tasks.
We study the remarkable ability of humans to learn probabilistic distributions based on experiences in new environments, a skill essential for good performance in a wide range of downstream perceptual [1-4] and cognitive [5-10] tasks. This ability manifests in everyday experiences, such as when a person learns to communicate with others of varying accents
*Equal contributions. Co-responding authors. T.T. is now at Max Planck Institute for Biological Cybernetics, Germany. L.K.W. is now at Google DeepMind. H.Z. is also affiliated with Beijing Key Laboratory of Behavior and Mental Health, PKU-IDG/McGovern Institute for Brain Research, and PKU-Tsinghua Center for Life Sciences at Peking University, and Chinese Institute for Brain Research, Beijing.
and personalities, or discovers new social norms in foreign cultures. More impressively, humans can not only summarise key characteristics of new experiences but also describe those patterns in statistical terms, such as "Most people in X country typically talk about the weather, but some younger ones prefer to gossip instead". These observations indicate that humans can develop structured and probabilistic internal beliefs about their surroundings from experiences. To distinguish humans' internal beliefs from the modeling framework to be introduced later, we refer to the former as an internal construct of the external environment. We ask: how does the human mind form internal constructs from experiences?
As experiences typically arrive sequentially in real life, building a probabilistic internal construct is an online density estimation problem. An ideal solution must be both consistent in distribution estimation and invariant to the order of the experiences. Consistency requires that the learned internal construct approach the true environmental uncertainty as more data arrives, expanding in complexity when the data evidence a more complicated true distribution. Invariance to experience order means no a priori preference for any particular model based on the order in which experiences arrive. For instance, if the experiences are independent samples from the same distribution, the learned distribution should not be affected by the sample order. Computationally, these two desiderata can be achieved by choosing an appropriate internal construct prior (ICP). Many instantiations of such ICPs incorporate the Chinese restaurant process (CRP) [11-13]. The CRP can recruit more resources to account for more complicated data distributions and is exchangeable so that the order of data does not affect the internal construct. In cognitive science, the CRP has been used to model category learning [14, 15, 6], classical conditioning [5, 16], among other cognitive functions [17].
However, we argue that achieving estimation consistency and order invariance is intractable given humans' limited cognitive capacity. First, building a flexible internal construct requires an "infinite capacity" [18] that is at odds with humans' finite memory. The flexibility of the internal construct is likely bounded under resource constraints. Second, achieving order invariance online imposes significant cognitive demands: it requires memorizing all previous experiences and reallocating them into potentially different clusters for every incoming new experience. Indeed, humans do not seem to learn in an exchangeable fashion [19], which can be captured by bounded rational inference [14, 15]. Resource constraints should thus impact the quality of the acquired internal construct.
To investigate how humans actually acquire and represent a structured internal density model under resource constraints, we first designed a structured density estimation task wherein human participants observed sequentially presented samples drawn from a hidden Gaussian mixture with 1-4 clusters. Participants were asked to report the full structure of their internal construct (learned distribution) at the end of the trial, providing us with a high-dimensional behavioral dataset. We found, among others, that participants reported their internal constructs close to the true distribution, but they consistently reported more clusters when there were in fact fewer or even a single cluster in the true distribution.
Motivated by the resource constraint argument, we propose a flexible ICP with a variable propensity towards expansion and includes the exchangeable CRP as a special case. In addition, we fit all parameters of the ICPs to the behavior dataset by a novel density estimation framework (DEF), which allows us to compare between ICPs and explore other inductive biases of participants in this task by likelihood-based mode optimization and critique. Consistent with our hypotheses, the proposed ICP best captures participants' behaviors when it does not preserve order-invariance and instead expands more economically when the internal construct is already complex. Further, fitted ICP explains the inconsistent number of clusters in the reported constructs as having a strong prior preference for small cluster widths. In sum, our combined experimental and modeling contributions reveal previously unknown human inductive biases in building internal constructs of new environments, and open up doors for further theoretical explorations of flexible priors in learning rich structure representations.
Figure 1: Human behavior in a structured density estimation task. A, On each trial, human participants needed to report the latent Gaussian mixture distribution (i.e., weight, mean, and sd for each cluster) they learned from sequential samples. Panels B-F show the results of Experiment 1. B, The earth-mover distance between the reported and true distributions decreases when participants observe more samples. C, The correlation between the reported and true distributions in the first three moments. See Fig. 6E & F for detailed illustration. D, Mean number of reported clusters. Participants failed to learn the true probabilistic structure, typically reporting 2-3 clusters regardless of the true cluster number. E, Overlap ratio, which is calculated as the percentage of a cluster's area being covered by any other clusters averaged across all clusters in the report. Adjacent clusters did not overlap much. F&G, Participants' reported densities versus the true distributions on example trials of Experiment 1 (F) and Experiment 2 (G). Error bars denote 1 sem over participants. Each gray line in B-D is for one participant.
2 Human behavioral experiments of structured density estimation
The human experiments described below were approved by the Institutional Review Board of School of Psychological and Cognitive Sciences at Peking University. All participants received monetary compensation above the local minimum wage.
2.1 Experiment 1: learning visuo-spatial distributions
We designed a structured online density estimation task (Fig. 1A) to collect the internal constructs of humans formed from experiences. On each trial, participant saw a sequence of red dots with horizontal positions drawn i.i.d. from a Gaussian mixture with a varying number of clusters across trials (see Appendix A.1 for the cover story and the set of distributions used in each experiment). At the end of the sequence, they were asked to report a Gaussian mixture model that could have generated the red dots, specifying the weight, mean, and variance of each cluster through an interactive interface. As we are interested in participants' innate abilities to build internal constructs, we did not provide feedback.
We first validate the quality of the reported distribution. As participants  $(N = 21)$  saw more samples (70 versus 10), the earth-mover distance between the reported and true densities decreased (Fig. 1B,  $F(1,20) = 83.21$ ,  $p < 0.001$ ), which is a desirable trend in density
Figure 2: Human learning of numeric distributions. A, Participants were randomly assigned to two groups that differed in the sd of the true distribution (scale 1x vs. 5x). B, The reported cluster number. C, The distance between the adjacent clusters. Error bar denotes 1 sem over participants.
estimation. Figure 1C shows that the reported densities tracked the first three moments reasonably well (all slopes significantly greater than zero,  $t$ -tests  $p < 0.001$ ), although higher-order moments were harder to estimate.
A closer examination of the reported density on each trial revealed systematic inconsistencies in the learned structure. Fig. 1F shows that the reported density had more local maxima even when the true distribution only had a single Gaussian cluster. Participants tended to report around 3 clusters, irrespective of the number of clusters in the true distribution (Fig. 1D). Moreover, with increasing observations, the reported number of clusters increased  $(t(20) = 11.15$ ,  $p < 0.001)$  but did not approach the ground truth. Such inconsistencies in the learned number of clusters are striking and counterintuitive, especially when the true generative distribution is as simple as a single Gaussian, in which case more evidence actually pushed participants further away from the truth. Another notable pattern is the small overlap between the adjacent clusters (Figure 1E), which contributes to bumpiness even when the true distribution is as smooth as a Gaussian. Though counterintuitive, this pattern echoes with the finding that humans use near-orthogonal (i.e., non-overlapping) basis functions to represent learned visuo-motor distributions [8].
2.2 Experiments 2 and 3: distributions across domains and scales
To show that our findings in Experiment 1 are not specific to the selected distributions, we conducted Experiment 2 ( $N = 20$ ) to collect human internal constructs induced by a broader set of 14 representative distributions from previous studies [20-25], with the true number of clusters ranging from 1 to 3. Results in Appendix A.2 show similar findings to Experiment 1: we found reasonably good density estimation quality, and participants preferred to report 2-3 non-overlapping clusters in their internal construct, even when the true distribution is a Gaussian distribution.
Do our findings still hold when we change the domain of the distributions? In Experiment 3 ( $N = 36$ ), participants observed samples of numerical values drawn from either Gaussian distributions or unimodal Gaussian mixtures (Fig. 2A). We divided the participants into two groups, one group observed numbers drawn from the original scale ('1X'), and the other observed samples drawn from 5 times the original scale ('5X'). After observing a sequence of numbers, participants, unaware of the true densities, reported the location and weight of each latent cluster. See Appendix A.3 for details.
Consistent with Experiments 1 and 2, we found that participants in both groups of Experiment 3 often reported more complex structures than the ground truth (Fig. 2B). More observations resulted in more clusters ( $F(1, 34) = 120.12$ ,  $p < 0.001$ ). The average reported cluster number was higher (10 samples:  $2.8 \pm 0.7$ ; 70 samples:  $4.3 \pm 1.3$ ) than Experiment 1
(10 samples:  $2.6 \pm 0.4$ ; 70 samples:  $3.2 \pm 0.5$ ), possibly due to differences in visuo-spatial and numeric memory capacities. Even though the distribution scales differed by a factor as large as five, both groups used a similar number of clusters to represent the numeric distribution  $(F(1, 34) = 0.05, p = 0.831)$ . The distance between adjacent clusters differed roughly fivefold between groups, mirroring the ratio between the true scales (Fig. 2C).
Summary Through comprehensive behavioral experiments, we found that humans were able to build internal constructs of environmental uncertainties based on online experiences, but they could not capture the number of clusters in the true distributions. Learned clusters also tended to be "orthogonal" to each other, creating unsmooth features not present in the true data distribution. It is possible that these imperfections are related to human's limited cognitive resources, as discussed in Section 1, or specific inductive biases in an implicit ICP, such as a small cluster width. Due to the unusual complexity of the density estimation task and the high-dimensional human report, the detailed mechanisms are not directly obvious. We thus resort to building a mathematical model of the reported internal constructs to understand the factors contributing to these inconsistencies.
3 The density estimation framework
Our task explicitly queries the internal probabilistic model constructed under sequential experiences. Mapping from these experiences to a distribution function is a challenging and ill-posed problem. Participants need a resource-rational ICP over the space of distributions. In addition, as experimenters, we need an appropriate noise model to define a valid likelihood for maximum-likelihood parameter fitting. To this end, we propose the density estimation framework (DEF) to model the experimental data; it consists of a rational component and an aleatoric component which we detail below. An overview is shown in Fig. 3.
3.1 The rational component
The rational component performs approximate Bayesian inference given data under an ICP, examples of which have been used extensively in cognitive modelling [15, 16]. A rational ICP for this task factorizes into a prior over the latent cluster assignment of the observed dots, and a prior over the probability density function associated with each cluster. For a sequence of  $T$  dot locations  $\mathbf{x}_T = [x_1,\dots ,x_T]\in \mathbb{R}^T$ , we denote the cluster assignments for  $\mathbf{x}_t$  as  $\mathbf{z}_t = [z_1,\dots ,z_t]$ , and each  $z_{t}\in \mathbb{N}_{+}$  is the cluster number assigned to  $x_{t}$ .
The economical ICP we propose here has its prior over  $\mathbf{z}_T$  defined recursively as
$$
p _ {\mathrm {R}} \left(z _ {t + 1} = k \mid \mathbf {z} _ {t}\right) := \left\{ \begin{array}{l l} \frac {\tilde {n} _ {t , k}}{t + \alpha_ {t}}, & k \leq K _ {t} \\ \frac {\alpha_ {t}}{t + \alpha_ {t}}, & k = K _ {t} + 1 \end{array} , \quad \alpha_ {t} := \alpha_ {0} e ^ {- r K _ {t}}, \quad \tilde {n} _ {t, k} := t \frac {n _ {t , k} ^ {\beta}}{\sum_ {t = 1} ^ {K _ {t}} n _ {t , k} ^ {\beta}} \right. \tag {1}
$$
for  $t \in \{1, \dots, T\}$ , where  $z_1 = 1$ ,  $n_{t,k} \coloneqq \sum_{\tau=1}^{t} \mathbb{1}[z_{\tau} = k]$  is the number of clusters assigned to cluster  $k$  at time  $t$ , and  $K_t \coloneqq \sum_k \mathbb{1}[n_{t,k} > 0]$  is the number of non-empty clusters (model size) at time  $t$ . The expansion rate  $\alpha_t$  depends on the model size; for  $r < 0$ , new clusters are less likely to be added, which implements a form of conservative expansion. The distortion rate  $\beta \geq 0$  controls whether the different cluster sizes  $n_{t,k}$  are evened-out  $(\beta < 1)$  or exaggerated  $(\beta > 1)$ , a form of divisive normalization. As a special case, this prior recovers the conventional CRP when  $r = 0$  and  $\beta = 1$ , which is exchangeable and has been a popular choice for modeling flexible online learning. In contrast, the prior (1) is in general not exchangeable; see Appendix B.1.1 for examples. Also, the expected number of clusters is upper bounded by  $\sum_t \alpha_0 e^{-rt} < \frac{\alpha_0}{1 - e^{-r}}$  for  $r > 0$ , unlike the conventional CRP where this expectation grows as  $\log(t)$ . We provide justifications for the non-exchangeability of this prior at the end of this section.
The economical ICP assumes that each cluster is a Gaussian distribution with mean  $m$  and variance  $v$  as cluster properties. A convenient prior for Gaussians is the normal-inverse-  $\chi^2$ :
$$
p _ {\mathrm {R}} (m, v) = \mathcal {N} \left(m; \mu_ {0}, v / \lambda_ {0}\right) \operatorname {I n v} \chi^ {2} \left(v; a _ {0}, \sigma_ {0}\right), \tag {2}
$$
where the hyperparameters with subscript 0 follow standard definitions [15]. Given  $\mathbf{x}_T$ , Bayesian inference under this ICP defined by (1) and (2) yields a posterior over Gaussian mixture parameters.
Figure 3: Schematic of the density estimation framework (DEF) and likelihood evaluation. We have omitted the subscripts  $t$  and  $k$  for clarity. The DEF comprises a bounded rational component (green box) and an aleatoric component (gold box). The former infers the cluster properties  $\varphi^i$  based on noisy perceived observations  $\tilde{\mathbf{x}}_T$  under an internal construct prior (ICP), and the latter provides a well-defined likelihood function while maintaining the dependencies between the cluster properties. Under the ICP defined in (1) and the particle-based sampling algorithm in (3) and (4), the current density model either updates an existing cluster or adds an additional cluster depending on the evidence of the incoming observation. The evolution of the model corresponds to a single path sampled in the tree structure shown in the left panel of the orange box. In the aleatoric component, structured noise is added to ensure a valid likelihood function for each simulation; see Appendix B. Each prediction yields a conditional likelihood of the reported  $\varphi^r$  in each trial, and averaging over a large number of simulations yields the marginal likelihood. These simulations can proceed in parallel (pink box), which can be accelerated by running on GPUs.
Following previous modeling work[15, 5], we assume humans use a single particle to approximately perform the Bayesian updating. Specifically, the observations  $\mathbf{x}_t$  up to time  $t$  and a corresponding single-particle sequence  $\mathbf{z}_t^i$  (an "approximate sample" from the posterior  $p_{\mathrm{R}}(\mathbf{z}_t|\mathbf{x}_t)$ ) define a structured density model: a mixture of  $K_t^i = \sum_{\tau=1}^t \mathbb{1}[z_\tau^i > 0]$  clusters, where each cluster is weighted in proportion to  $n_{t,k}^i = \sum_{\tau=1}^t \mathbb{1}[z_\tau^i = k]$ , and has posterior
over density given by the usual conjugate prior update formulae:
$$
p _ {\mathrm {R}} \left(m _ {k}, v _ {k} \mid \mathbf {x} _ {t}, \mathbf {z} _ {t} ^ {i}\right) = \mathcal {N} \left(m _ {k}; \mu_ {t, k} ^ {i}, v _ {k} / \lambda_ {t, k} ^ {i}\right) \operatorname {I n v} \chi^ {2} \left(v _ {k}; a _ {t, k} ^ {i}, \sigma_ {t, k} ^ {i}\right), \tag {3}
$$
$$
a _ {t, k} ^ {i} = a _ {0} + n _ {t, k} ^ {i}, \quad \lambda_ {t, k} ^ {i} = \lambda_ {0} + n _ {t, k} ^ {i}, \quad \mu_ {t, k} ^ {i} = (\lambda_ {0} \mu_ {0} + n _ {t, k} ^ {i} \bar {\mu} _ {t, k} ^ {i}) / \lambda_ {t, k} ^ {i},
$$
$$
\sigma_ {t, k} ^ {i} = \frac {1}{a _ {t , k} ^ {i}} \left[ a _ {0} \sigma_ {0} + n _ {t, k} ^ {i} \bar {\sigma} _ {t, k} ^ {i} + \frac {\lambda_ {0} n _ {t , k} ^ {i}}{\lambda_ {0} + n _ {t , k} ^ {i}} (\mu_ {0} - \bar {\mu} _ {t, k} ^ {i}) ^ {2} \right],
$$
where  $\bar{\mu}_{t,k}^{i}$  and  $\bar{\sigma}_{t,k}^{i}$  are, respectively, the empirical mean and the (unadjusted) empirical variance of the observations assigned to cluster  $k$  at time  $t$ , according to the particle  $\mathbf{z}_t^i$ . Equation (3) shows that, given  $\mathbf{z}_t^i$ , the sufficient statistics that determine the posterior of the  $k$ 'th cluster are  $\varphi_t^i \coloneqq [n_{t,k}^i, \bar{\mu}_{t,k}^i, \bar{\sigma}_{t,k}^i]_{k=1}^{K_t^i}$ , in the sense that the conditioning variables are all functions of  $\varphi_t^i$ . So, we can equivalently write  $p_{\mathrm{R}}(m_k, v_k | \mathbf{x}_t, \mathbf{z}_t^i) = p_{\mathrm{R}}(m_k, v_k | \varphi_t^i)$  in (3). We also regard  $\varphi^i$  (not the random variables  $m$  and  $v$  in (3)) as the inferred cluster properties. When the next observation  $x_{t+1}$  arrives, it is assigned to a cluster according to the one-step posterior
$$
p _ {\mathrm {R}} (z _ {t + 1} ^ {i} | \mathbf {z} _ {t} ^ {i}, \mathbf {x} _ {t}, x _ {t + 1}) \propto p _ {\mathrm {R}} (z _ {t + 1} ^ {i} | \mathbf {z} _ {t} ^ {i}) p _ {\mathrm {R}} (x _ {t + 1} | \mathbf {z} _ {t}, z _ {t + 1}, \mathbf {x} _ {t}) = p _ {\mathrm {R}} (z _ {t + 1} ^ {i} | \mathbf {z} _ {t} ^ {i}) p _ {\mathrm {R}} (x _ {t + 1} | \phi_ {t} ^ {i}, z _ {t + 1}) \quad (4)
$$
for  $z_{t+1}^i \in [1, \dots, K_t^i + 1]$ , where  $p_{\mathbb{R}}(x_{t+1} | \phi_t^i, z_{t+1} = k)$  is the marginal likelihood of  $x_{t+1}$  if assigned to cluster  $k$ , given by evaluating at  $x_{t+1}$  the Student's  $t$ -density with degrees of freedom  $a_i$ , location  $\mu_{t,k}^i$  and scale  $(\sigma_{t,k}^i (1 + 1 / \lambda_{i,k}))^{1/2}$ . If  $z_{t+1} \leq K_t^i$ , then the  $z_{t+1}$ 'th component is updated by  $x_{t+1}$ ; if  $z_{t+1} = K_t^i + 1$ , then a new cluster is created, and the internal construct expands. Figure 3 (green box) depicts these two possibilities step at  $t = 6$ .
If the cluster prior (1) of the rational component implements the CRP, then by exchangeability, the true posterior over the model density is order-invariant with respect to observations. To obtain a particle from an order-invariant posterior, one should revise the cluster assignments of previous  $\mathbf{x}_t$  at every time step, but this requires memorizing  $\mathbf{x}_t$  which is cognitively implausible. We argue then that keeping the cluster prior (1) exchangeable is unnecessary—it forces the prior to take on a cognitively implausible property. Since (1) subsumes the CRP, we can test whether human behaviors collected in our experiments are better fit by an order-invariant prior.
3.2 The aleatoric component and maximum-likelihood model fitting
Note that each inferred particle  $\varphi^i \coloneqq \varphi_T^i$  at  $t = T$  is a single simulation of the rational component a participant may possess in their mind. Our (the experimenter's) goal is to fit the rational component to the reported cluster properties  $\varphi^r \coloneqq [\mathbf{w}^r, \boldsymbol{\mu}^r, \boldsymbol{\sigma}^r]$  collecting the weights, means, and variances of the reported clusters; the reported number of clusters implied by  $\varphi^r$  is denoted by  $K^r$ . The key challenges are worth emphasizing: a) the dimensionality of the report is variable, as it depends on the number of clusters; b) the cluster properties are unordered sets, rather than a real vector; c) the cluster assignments  $\mathbf{z}_T$  are not observed to us as experimenters and need to be marginalized out to obtain the marginal likelihood, which is intractable. We address the first two challenges by introducing an aleatoric component, a structured noise model for DEF; and we deal with the last challenge by an efficient implementation of the DEF that utilizes the parallel processing power of graphical processing units (GPUs).