Title: Bounded rationality in structured  density estimation
Venue: NeurIPS 2023
Accepted: yes
Keywords: human representation of uncertainty; Bayesian inference; bounded rationality; inductive bias; Chinese Restaurant Process
Abstract: Learning to accurately represent environmental uncertainty is crucial for adaptive and optimal behaviors in various cognitive tasks. However, it remains unclear how the human brain, constrained by finite cognitive resources, constructs an internal model from an infinite space of probability distributions. In this study, we explore how these learned distributions deviate from the ground truth, resulting in observable inconsistency in a novel structured density estimation task. During each trial, human participants were asked to form and report the latent probability distribution functions underlying sequentially presented independent observations. As the number of observations increased, the reported predictive density became closer to the ground truth. Nevertheless, we observed an intriguing inconsistency in human structure estimation, specifically a large error in the number of reported clusters. Such inconsistency is invariant to the scale of the distribution and persists across stimulus modalities. We modeled uncertainty learning as approximate Bayesian inference in a nonparametric mixture prior of distributions. Human reports were best explained under resource rationality embodied in a decaying tendency towards model expansion. Our study offers insights into human cognitive processes under uncertainty and lays the groundwork for further exploration of resource-rational representations in the brain under more complex tasks.

Bounded rationality in structured density estimation

Abstract

Learning to accurately represent environmental uncertainty is crucial for adaptive and optimal behaviors in various cognitive tasks. However, it remains unclear how the human brain, constrained by finite cognitive resources, internalise the highly structured environmental uncertainty. In this study, we explore how these learned distributions deviate from the ground truth, resulting in observable inconsistency in a novel structured density estimation task. During each trial, human participants were asked to learn and report the latent probability distribution functions underlying sequentially presented independent observations. As the number of observations increased, the reported predictive density became closer to the ground truth. Nevertheless, we observed an intriguing inconsistency in human structure estimation, specifically a large error in the number of reported clusters. Such inconsistency is invariant to the scale of the distribution and persists across stimulus modalities. We modeled uncertainty learning as approximate Bayesian inference in a nonparametric mixture prior of distributions. Human reports were best explained under resource rationality embodied in a decaying tendency towards model expansion. Our study offers insights into human cognitive processes under uncertainty and lays the groundwork for further exploration of resource-rational representations in the brain under more complex tasks.

1 Introduction

We study the remarkable ability of humans to learn probabilistic distributions based on experiences in new environments, a skill essential for good performance in a wide range of downstream perceptual [1-4] and cognitive [5-10] tasks. This ability manifests in everyday experiences, such as when a person learns to communicate with others of varying accents

and personalities, or discovers new social norms in foreign cultures. More impressively, humans can not only summarise key characteristics of new experiences but also describe those patterns in statistical terms, such as 'Most people in X country typically talk about the weather, but some younger ones prefer to gossip instead'. These observations indicate that humans can develop structured and probabilistic internal beliefs about their surroundings from experiences. To distinguish humans' internal beliefs from the modeling framework to be introduced later, we refer to the former as an internal construct of the external environment. We ask: how does the human mind form internal constructs from experiences?

As experiences typically arrive sequentially in real life, building a probabilistic internal construct is an online density estimation problem. An ideal solution must be both consistent in distribution estimation and invariant to the order of the experiences. Consistency requires that the learned internal construct approach the true environmental uncertainty as more data arrives, expanding in complexity when the data evidence a more complicated true distribution. Invariance to experience order means no a priori preference for any particular model based on the order in which experiences arrive. For instance, if the experiences are independent samples from the same distribution, the learned distribution should not be affected by the sample order. Computationally, these two desiderata can be achieved by choosing an appropriate internal construct prior (ICP). Many instantiations of such ICPs incorporate the Chinese restaurant process (CRP) [11-13]. The CRP can recruit more resources to account for more complicated data distributions and is exchangeable so that the order of data does not affect the internal construct. In cognitive science, the CRP has been used to model category learning [14, 15, 6], classical conditioning [5, 16], among other cognitive functions [17].

However, we argue that achieving estimation consistency and order invariance is intractable given humans' limited cognitive capacity. First, buiding a flexible internal construct requires an 'infinite capacity' [18] that is at odds with humans' finite memory. The flexibility of the internal construct is likely bounded under resource constraints. Second, achieving order invariance online imposes significant cognitive demands: it requires memorizing all previous experiences and reallocating them into potentially different clusters for every incoming new experience. Indeed, humans do not seem to learn in an exchangeable fashion [19], which can be captured by bounded rational inference [14, 15]. Resource constraints should thus impact the quality of the acquired internal construct.

To investigate how humans actually acquire and represent a structured internal density model under resource constraints, we first designed a structured density estimation task wherein human participants observed sequentially presented samples drawn from a hidden Gaussian mixture with 1-4 clusters. Participants were asked to report the full structure of their internal construct (learned distribution) at the end of the trial, providing us with a high-dimensional behavioral dataset. We found, among others, that participants reported their internal constructs close to the true distribution, but they consistently reported more clusters when there were in fact fewer or even a single cluster in the true distribution.

Motivated by the resource constraint argument, we propose a flexible ICP with a variable propensity towards expansion and includes the exchangeable CRP as a special case. In addition, we fit all parameters of the ICPs to the behavior dataset by a novel density estimation framework (DEF), which allows us to compare between ICPs and explore other inductive biases of participants in this task by likelihood-based mode optimization and critique. Consistent with our hypotheses, the proposed ICP best captures participants' behaviors when it does not preserve order-invariance and instead expands more economically when the internal construct is already complex. Further, fitted ICP explains the inconsistent number of clusters in the reported constrcts as having a strong prior preference for small cluster widths. In sum, our combined experimental and modeling contributions reveal previously unknown human inductive biases in building internal constructs of new environments, and open up doors for further theoretical explorations of flexible priors in learning rich structure representations.

1

2

3

4

# true cluster

0

20

40

60

overlap (%)

-5

0

5

sample

-5

0

5

report

mean (cm)

4

6

sample

4

6

report

sd (cm)

-2

0

2

sample

-2

-1

0

1

2

report

skewness

example distribution

under certian overlap level

A

B

C

D

E

F

G

# true cluster = 1

# true cluster = 2

# true cluster = 3

# true cluster = 4

# true cluster = 1

# true cluster = 2

# true cluster = 2

# true cluster = 3

reported density

true density

observations

...

observe samples one by one

click the mouse to report the

location of each latent cluster

move to report the sd/weight

of each latent cluster

1

2

3

4

# true cluster

0

1

2

3

earth-mover

distance

1

2

3

4

# true cluster

0

2

4

# reported cluster

sample size

10

70

2 Human behavioral experiments of structured density estimation

The human experiments described below were approved by the Institutional Review Board of School of Psychological and Cognitive Sciences at Peking University. All participants received monetary compensation above the local minimum wage.

2.1 Experiment 1: learning visuo-spatial distributions

We designed a structured online density estimation task (Fig. 1A) to collect the internal constructs of humans formed from experiences. On each trial, participant saw a sequence of red dots with horizontal positions drawn i.i.d. from a Gaussian mixture with a varying number of clusters across trials (see Appendix A.1 for the cover story and the set of distributions used in each experiment). At the end of the sequence, they were asked to report a Gaussian mixture model that could have generated the red dots, specifying the weight, mean, and variance of each cluster through an interactive interface. As we are interested in participants' innate abilities to build internal constructs, we did not provide feedback.

We first validate the quality of the reported distribution . As participants ( N =21) saw more samples (70 versus 10), the earth-mover distance between the reported and true densities decreased (Fig. 1B, F (1,20) = 83.21, p <0.001), which is a desirable trend in density

1670

1245

3045

2095

...

Numeric stimuli, small scale group

380

280

345

485

...

Numeric stimuli, large scale group

1x

1x

1x

5x

5x

5x

Unimodal-skew

(mixture of 2 Gaussian)

Unimodal-skew

(mixture of 2 Gaussian)

Unimodal-skew

(Mixture of two Gaussian)

1

2

# true cluster

0

2

4

# reported cluster

sample size

10

1

2

# true cluster

sample size

70

scale

1x

5x

1

2

# true cluster

0

200

400

600

distance between

adjacent clusters

sample size

10

1

2

# true cluster

sample size

70

A

B

C

estimation. Figure 1C shows that the reported densities tracked the first three moments reasonably well (all slopes significantly greater than zero, t -tests p <0.001), although higherorder moments were harder to estimate.

A closer examination of the reported density on each trial revealed systematic inconsistencies in the learned structure. Fig. 1F shows that the reported density had more local maxima even when the true distribution only had a single Gaussian cluster. Participants tended to report around 3 clusters, irrespective of the number of clusters in the true distribution (Fig. 1D). Moreover, with increasing observations, the reported number of clusters increased ( t (20)=11.15, p <0.001) but did not approach the ground truth. Such inconsistencies in the learned number of clusters are striking and counterintuitive, especially when the true generative distribution is as simple as a single Gaussian, in which case more evidence actually pushed participants further away from the truth. Another notable pattern is the small overlap between the adjacent clusters (Figure 1E), which contributes to bumpiness even when the true distribution is as smooth as a Gaussian. Though counterintuitive, this pattern echoes with the finding that humans use near-orthogonal (i.e., non-overlapping) basis functions to represent learned visuo-motor distributions [8].

2.2 Experiments 2 and 3: distributions across domains and scales

To show that our findings in Experiment 1 are not specific to the selected distributions, we conducted Experiment 2 ( N =20) to collect human internal constructs induced by a broader set of 14 representative distributions from previous studies [20-25], with the true number of clusters ranging from 1 to 3. Results in Appendix A.2 show similar findings to Experiment 1: we found reasonably good density estimation quality, and participants preferred to report 2-3 non-overlapping clusters in their internal construct, even when the true distribution is a Gaussian distribution.

Do our findings still hold when we change the domain of the distributions? In Experiment 3 ( N =36), participants observed samples of numerical values drawn from either Gaussian distributions or unimodal Gaussian mixtures (Fig. 2A). We divided the participants into two groups, one group observed numbers drawn from the original scale ('1X'), and the other observed samples drawn from 5 times the original scale ('5X'). After observing a sequence of numbers, participants, unaware of the true densities, reported the location and weight of each latent cluster. See Appendix A.3 for details.

Consistent with Experiments 1 and 2, we found that participants in both groups of Experiment 3 often reported more complex structures than the ground truth (Fig. 2B). More observations resulted in more clusters ( F (1, 34) = 120.12, p < 0.001). The average reported cluster number was higher (10 samples: 2.8 ± 0.7; 70 samples: 4.3 ± 1.3) than Experiment 1

(10 samples: 2.6 ± 0.4; 70 samples: 3.2 ± 0.5), possibly due to differences in visuo-spatial and numeric memory capacities. Even though the distribution scales differed by a factor as large as five, both groups used a similar number of clusters to represent the numeric distribution ( F (1, 34)=0.05, p =0.831). The distance between adjacent clusters differed roughly fivefold between groups, mirroring the ratio between the true scales (Fig. 2C).

Summary Through comprehensive behavioral experiments, we found that humans were able to build internal constructs of environmental uncertainties based on online experiences, but they could not capture the number of clusters in the true distributions. Learned clusters also tended to be 'orthogonal' to each other, creating unsmooth features not present in the true data distribution. It is possible that these imperfections are related to human's limited cognitive resources, as discussed in Section 1, or specific inductive biases in an implicit ICP, such as a small cluster width. Due to the unusual complexity of the density estimation task and the high-dimensional human report, the detailed mechanisms are not directly obvious. We thus resort to building a mathematical model of the reported internal constructs to understand the factors contributing to these inconsistencies.

3 The density estimation framework

Our task explicitly queries the internal probabilistic model constructed under sequential experiences. Mapping from these experiences to a distribution function is a challenging and ill-posed problem. Participants need a resource-rational ICP over the space of distributions. In addition, as experimenters, we need an appropriate noise model to define a valid likelihood for maximum-likelihood parameter fitting. To this end, we propose the density estimation framework (DEF) to model the experimental data; it consists of a rational component and an aleatoric component which we detail below. An overview is shown in Fig. 3.

3.1 The rational component

The rational component performs approximate Bayesian inference given data under an ICP, examples of which have been used extensivly in cognitive modelling [15, 16]. A rational ICP for this task factorizes into a prior over the latent cluster assignment of the observed dots, and a prior over the probability density function associated with each cluster. For a sequence of T dot locations x T = [ x 1 , . . . , x T ] ∈ R T , we denote the cluster assignments for x t as z t = [ z 1 , . . . , z t ] , and each z t ∈ N + is the cluster number assigned to x t .

The economical ICP we propose here has its prior over z T defined recursively as for t ∈ { 1 , . . . T } , where z 1 = 1 , n t,k := ∑ t τ =1 /x31 [ z τ = k ] is the number of clusters assigned to cluster k at time t , and K t := ∑ k /x31 [ n t,k > 0] is the number of non-empty clusters (model size) at time t . The expansion rate α t depends on the model size; for r < 0 , new clusters are less likely to be added, which implements a form of conservative expansion. The distortion rate β ≥ 0 controls whether the different cluster sizes n t,k are evened-out ( β < 1 ) or exaggerated ( β > 1 ), a form of divisive normalization. As a special case, this prior recovers the conventional CRP when r = 0 and β = 1 , which is exchangeable and has been a popular choice for modeling flexible online learning. In contrast, the prior (1) is in general not exchangeable; see Appendix B.1.1 for examples. Also, the expected number of clusters is upper bounded by ∑ t α 0 e -rt < α 0 1 -e -r for r > 0 , unlike the conventional CRP where this expectation grows as log ( t ) . We provide justifications for the non-exchangeability of this prior at the end of this section.

The economical ICP assumes that each cluster is a Gaussian distribution with mean m and variance v as cluster properties. A convenient prior for Gaussians is the normal-inverseχ 2 :

where the hyperparameters with subscript 0 follow standard definitions [15]. Given x T , Bayesian inference under this ICP defined by (1) and (2) yields a posterior over Gaussian mixture parameters.

simulation 1

perceived

x

1:T

simulation 2

simulation

M

x

1:T

...

p

A

(

φ

r

|

φ

1

)

p

A

(

φ

r

|

φ

2

)

p

A

(

φ

r

|

φ

M

)

p

(

θ

|

x

1:T

)

...

t = T

'>'

K i

'<'

one simulation of the density estimation framework (DEF)

inference over an internal model prior (

t

= 6)

K i

?

K i

φ

i

'='

rational component

aleatoric component

φ

i

p

A

(

φ

r

|

φ

i

)

parallel on GPU

memory

t

= 1

t

= 2

model expansion

model modification

cluster

t

= 3

t

= 4

...

...

...

...

...

create

update

update

enumerate assignments

p

R

(

z

t+1

|

φ

t

)

weight each by evidence

p

R

(

x

t+1

|

ϕ

t

,

z

t+1

) =

p

R

(

x

t+1

|

z

t+1

,

z

1:t

,

x

1:t

)

∝

α

0

exp(-

rK

t

)

∝

n

β

∝

n

β

μ

n

σ

μ

n

σ

φ

t

Kt=2

update

μ

n

σ

μ

n

σ

φ

t+1

=2

likelihood computation

next

A

0 000

→

Following previous modeling work[15, 5], we assume humans use a single particle to approximately perform the Bayesian updating. Specifically, the observations x t up to time t and a corresponding single-particle sequence z i t (an 'approximate sample' from the posterior p R ( z t | x t ) ) define a structured density model: a mixture of K i t = ∑ t τ =1 /x31 [ z i τ > 0] clusters, where each cluster is weighted in proportion to n i t,k = ∑ t τ =1 /x31 [ z i τ = k ] , and has posterior

over density given by the usual conjugate prior update formulae:

where ¯ µ i t,k and ¯ σ i t,k are, respectively, the empirical mean and the (unadjusted) empirical variance of the observations assigned to cluster k at time t , according to the particle z i t . Equation (3) shows that, given z i t , the sufficient statistics that determine the posterior of the k 'th cluster are φ i t := [ n i t,k , ¯ µ i t,k , ¯ σ i t,k ] K i t k =1 , in the sense that the conditioning variables are all functions of φ i t . So, we can equivalently write p R ( m k , v k | x t , z i t ) = p R ( m k , v k | φ i t ) in (3). We also regard φ i (not the random variables m and v in (3)) as the inferred cluster properties. When the next observation x t +1 arrives, it is assigned to a cluster according to the one-step posterior

for z i t +1 ∈ [1 , . . . , K i t + 1] , where p R ( x t +1 | ϕ i t , z t +1 = k ) is the marginal likelihood of x t +1 if assigned to cluster k , given by evaluating at x t +1 the Student's t -density with degrees of freedom a i , location µ i t,k and scale ( σ i t,k (1 + 1/ λ i,k )) 1/2 . If z t +1 ≤ K i t , then the z t +1 'th component is updated by x t +1 ; if z t +1 = K i t + 1 , then a new cluster is created, and the internal construct expands. Figure 3 (green box) depicts these two possibilities step at t = 6 .

If the cluster prior (1) of the rational component implements the CRP, then by exchangeability, the true posterior over the model density is order-invariant with respect to observations. To obtain a particle from an order-invariant posterior, one should revise the cluster assignments of previous x t at every time step, but this requires memorizing x t which is cognitively implausible. We argue then that keeping the cluster prior (1) exchangeable is unnecessaryit forces the prior to take on a cognitively implausible property. Since (1) subsumes the CRP, we can test whether human behaviors collected in our experiments are better fit by an order-invariant prior.

3.2 The aleatoric component and maximum-likelihood model fitting

Note that each inferred particle φ i := φ i T at t = T is a single simulation of the rational component a participant may possess in their mind. Our (the experimenter's) goal is to fit the rational component to the reported cluster properties φ r := [ w r , µ r , σ r ] collecting the weights, means, and variances of the reported clusters; the reported number of clusters implied by φ r is denoted by K r . The key challenges are worth emphasizing: a) the dimensionality of the report is variable, as it depends on the number of clusters; b) the cluster properties are unordered sets, rather than a real vector; c) the cluster assignments z T are not observed to us as experimenters and need to be marginalized out to obtain the marginal likelihood, which is intractable. We address the first two challenges by introducing an aleatoric component , a structured noise model for DEF; and we deal with the last challenge by an efficient implementation of the DEF that utilizes the parallel processing power of graphical processing units (GPUs).

Figure 3 (right panel of orange box) shows illustrates the aleatoric component. It postulates that the participant, having inferred φ i , commits to the number of clusters first, during which they 'slack' with a small probability, reporting ˆ K i clusters according to:

̸

where ϵ > 0 is a slack parameter, and K max is the maximum number of reported clusters seen in an experiment across participants. If ˆ K i = K i , then the participant removes or splits existing clusters recursively to obtain a set of slacked cluster properties ˆ φ i := [ˆ n i , ˆ µ i , ˆ σ i ] =

f ( φ i , ˆ K i ) until there are ˆ K i clusters; Appendix B.2 presents the details of how f modifies φ i . Finally, the likelihood of the reported φ r given slacked cluster properties are defined as P ( φ r | ˆ φ i ) := 0 if ˆ K i = K r (infinity penalty for inferring the number of cluster wrong), and

̸

where S K is the set of all permutations of K elements; p w ( · ; n ) is a Dirichlet distribution with concentration defined by n ; p µ ( · ; µ ) is an isotropic normal with mean µ and a variance parameter; and p σ ( · ; σ ) is an isotropic log-normal with mean σ and a variance parameter. See Appendix B.2 for precise definitions of these distributions and parameters involved.

Overall, the aleatoric component the takes inferred cluster properties φ i from the rational component and assigns non-zero probabilities to all possible reported distributions through the slack mechanism defined by (5) and the cluster modification f . The permutationinvariant likelihood (6) regards the weight, mean, and variance vectors as independent conditional on ˆ φ i , but retains the dependences between the clusters. To approximate the marginal likelihood, we marginalize out z i and ˆ K i by averaging over a large number of Monte Carlo (MC) simulations. In addition, we allow visual noise by feeding in noisy observations ˜ x , where ˜ x t ∼ p n (˜ x t | x t ) := N (˜ x t | x t , σ v ) , which is also marginalized out during MC.

Altogether, the DEF estimates the likelihood of the reported φ r given x T as p ( φ r | x T ) ≈

where M is the number of MC simulations (see Appendix B.4 for important distinction to the number of particles). We implemented this MC estimator with a fully vectorized approach using PyTorch [26], which enables easy parallelization on GPUs (Fig. 3, pink box). Compared to a conventional for-loop implementation, ours yields reliable log-likelihood estimates with orders of magnitude acceleration. This allows us to optimize the DEF by maximum-likelihood using any off-the-shelve optimization method, such as Nelder-Mead [27]. Further, it also allows us to critique different behavioral models (DEFs) by model comparison, using metrics such as Akaike Information Criterion (AIC) [28].

4 Experiments: fitting and comparing internal construct priors

We compare the following ICPs by fitting them in the DEFs that share the same class of aleatoric component: a) our proposed economical ICP as described in Section 3.1; b) its special case, the exchangeable CRP-GMM; and c) a baseline batch learning prior that describes a non-sequential cluster assignment; see Appendix B.1.3 for details. We use the Nealder-Mead optimizer to fit the DEF parameters for each participant, restarting multiple times to avoid early convergence issues [29]. We choose a large number M for MC simulations so that the estimator (7) produces small enough variance and bias tolerable for the Nelder-Mead optimizer. After fitting, we compare different DEFs using a much larger M . Details of the fitting algorithm are deferred to Appendix B.3 where we also present additional experiments validating our overall optimization scheme.

The quality of the fitted DEFs on the three experiments are shown in Fig. 4. The economical ICP produced significantly lower AICs than the exchangeable CRP-GMM in all three experiments (Exp. 1: median AICs -2627.9 vs. -2583.1, p < 10 -4 ; Exp. 2: -1766.7 vs. -1697.9, p < 10 -3 ; Exp. 3: -148.2 vs. -141.0, p < 0 . 01 ; Wilcoxon signed-rank test). We compare four aspects of the predictions in Fig. 4: the error rate in predicting the number of clusters, the negative log-likelihood (NLL) of the weights, and the expected normalized error in predicting mean and log-variance predictions using the slacked predictions given ˆ K i = K r . The advantage of the economical ICP is not only in correctly predicting the number of clusters but also in estimating the cluster properties. As expected, the batch ICP had much worse AICs. Further, to test whether simpler models can explain the data better, we performed an ablation study based on the CRP-GMM ICP (Appendix B.5.1). The ablated ICPs neglect various aspects of the probabilistic structures, effectively mimicking different heuristic

0

200

400

Experiment 1

­AIC

0.4

0.6

0.8

1.0

error rate on cluster count

2

1

0

1

2

NLL of weights

0.0

0.1

0.2

0.3

norm. sq. error in mean

0.8

1.0

1.2

1.4

norm. sq. error in log­variance

50

0

50

100

150

Experiment 2

0.4

0.6

0.8

1.0

2

1

0

1

2

0.0

0.1

0.2

0.3

0.8

1.0

1.2

1.4

economical

exchangeable

batch

0

50

100

Experiment 3

economical

exchangeable

batch

0.4

0.6

0.8

1.0

economical

exchangeable

batch

2

1

0

1

2

economical

exchangeable

batch

0.0

0.1

0.2

0.3

strategies in forming the internal construct. None of the ablated models outperforms the economical ICP.

Example predictions of the two sequential ICPs are shown in Fig. 5A. The DEF with the economical ICP captures all of the behavioral patterns exhibited by participants, including the inconsistent number of clusters, the low overlap ratio between adjacent clusters, the moments of the reported distribution (Figs. 6 to 8), and the covariance of the reported cluster properties (Fig. 5B-D). We found largely consistent patterns in three fitted parameters across experiments (Fig. 5E-G). First, in line with our resource-rational motivation, the decay rates r in (1) are greater than 0, implying that model expansion is suppressed as α t decays when the internal construct becomes more complex. Second, participants' prior sd √ σ 0 in (2) was 20%-35% of the true density's sd regardless of the true scale of the observation, with a high confidence a 0 (see Appendix B.5.2). This means that each cluster is kept narrow, requiring a new cluster to account for an observation far from existing clusters. It explains the bumpiness in the reported internal constructs, and also suggests that participants adaptively adjusted the cluster scale in accordance with the data scale and modality. Further, the observation that participants reported around 3 clusters is likely the trade-off between the opposite forces of a decaying α t and a small but confident σ 0 . Third, we found the cluster distortion β in (1) to be less than 1, indicating a reduced sensitivity to cluster size when forming the internal construct online.

5 Discussion

We studied density estimation by humans, a sophisticated capability fundamental to a variety of cognitive tasks but has so far been elusive in experiments and implicitly assumed in modeling. Through behavioral experiments, we discovered systematic inconsistencies in humans' internal constructs of environments (Section 2) and proposed a density estimation framework for modeling this rich behavioral dataset (Section 3). We explained the inconsistencies as a result of an economically-expanding internal construct prior (ICP) over internal constructs and a strong preference for narrow clusters [30]. The scale-adaptive cluster prior aligns with prior research on adaptive sensory and probability coding [31-33, 30, 34].

The density estimation framework (DEF) is a generic approach for modeling complex behavioral data. Thanks to the generality, one can compare other inference algorithms based on likelihoods and can easily incorporate priors over parameters (hyperpriors). While previous work kept the trainable parameters small and/or used error-based objective functions [15, 35, 4], our DEF can fit more parameters by maximum-likelihood, marginalizing the

economical

exchangeable

data

data

B

C

D

data

economical DEF

data

economical DEF

6

economical DEF

4

2

2

2

0

T

0.0

0.5

1.0

0.0

0.5

1.0

0

10

20

0

10

20

2

3

clusterweight

clusterweight

clustereccentricity

cluster eccentricity

#reported clusters

E

No extra decay

F

No distortion

G

sd of the

increasing expansion

tendency

(CRP)

tendency

decreasing expansion

“Poor get richer"

(CRP)

"Rich get richer"

true distribution

r (controls the decay rate of the cluster expansion tendency)

2

0.00

0.25

β (controls the probability distortion)

0.50

1.00

1.25

1.50

1.75

0°0

relative-  oo (controls the prior about the cluster width)

0.2

0.4

90

8'0

Exp1

Exp1-

Exp1 -

Exp2

Exp2

Exp2

Exp3(5x)

Exp3(5x)

Exp3(5×)

Exp3(1x)

Exp3(1x)

Exp3(1×)

Fitted r

2

00°0

0.25

0.50

0.75

Fitted

1.00

1.25

1.50

1.75

2.00

00

0.2

0.4

Fitted relative-√

0.6

80

latent cluster assignments properly, thanks to our efficient implementation and hardware acceleration. With this methodology, we showed that the economical ICP provided a closer match to human behavior than the Chinese restaurant process (CRP) which has been used pervasively in modeling online learning. With a decaying rate of expansion, the growth of an economical ICP is more in line with humans' finite memory capacity.

Our work sends important messages to the theoretical community. Dasgupta and Griffiths [32] showed that the CRP is cognitively plausible because it can be derived from a preference for low entropy in the cluster assignment distribution. We found that the fitted β < 1 actually increases the entropy of the prior cluster weight distribution [20, 23], suggesting that the cluster count might be the main consumption of cognitive resources. Gershman et al. [16] discussed in great depth on batch versus online learning, whether the model capacity should be finite or infinite, and the hypothesis that retrospective cluster reassignment may be possible with particle representations. We provide key experimental and modeling evidence that humans may employ an online and yet finite (in expectation) model. However, with only a few number of particles, the beliefs of the other possible assignments are not well retained, so retrospective corrections may not be easily produced from these models.

In sum, the economical DEF reproduces many human patterns, not only in the loss function optimized for (quality of capturing human structure learning), but also in the moments of the whole distribution and the covariance structures of cluster properties. We note the large room for improvement in predicting the cluster variance (the right column in Fig. 4), which hints at the complexity of the task and the challenge in modeling. Our work also provides a comprehensive likelihood-based model comparisons on human density estimation, and paves way for future studies using high-dimensional reports in complex behavioral tasks.

Acknowledgments

This work was partly supported by the National Natural Science Foundation of China (32171095), National Science and Technology Innovation 2030 Major Program (2022ZD0204803), and funding from Peking-Tsinghua Center for Life Sciences to H.Z. L.K.W. was supported by Gatsby Charitable Foundation. We thank Peter Dayan and Jian Li for helpful discussions, and five anonymous reviewers for their valuable feedback.

References

Marc O Ernst and Martin S Banks. Humans integrate visual and haptic information in a statistically optimal fashion. Nature , 2002.

Yair Weiss, Eero P Simoncelli, and Edward H Adelson. Motion illusions as optimal percepts. Nature neuroscience , 2002.

Konrad P Körding and Daniel M Wolpert. Bayesian decision theory in sensorimotor control. Trends in cognitive sciences , 2006.

Jake Spicer, Adam N Sanborn, and Ulrik R Beierholm. Using occam's razor and bayesian modelling to compare discrete and continuous representations in numerosity judgements. Cognitive Psychology , 2020.

Aaron C Courville and Nathaniel Daw. The rat as particle filter. Advances in neural information processing systems , 2007.

Thomas L Griffiths, Adam N Sanborn, Kevin R Canini, and Daniel J Navarro. Categorization as nonparametric bayesian density estimation. The probabilistic mind: Prospects for Bayesian cognitive science , 2008.

Nathaniel Daw and Aaron Courville. The pigeon as particle filter. Advances in neural information processing systems , 2008.

Hang Zhang, Nathaniel D Daw, and Laurence T Maloney. Human representation of visuo-motor uncertainty as mixtures of orthogonal basis distributions. Nature Neuroscience , 2015.

S. J. Gershman, E. J. Horvitz, and J. B. Tenenbaum. Computational rationality: A converging paradigm for intelligence in brains, minds, and machines. Science , 2015.

Kelsey Allen, Evan Shelhamer, Hanul Shin, and Joshua Tenenbaum. Infinite mixture prototypes for few-shot learning. In International Conference on Machine Learning , 2019.

David J Aldous, Illdar A Ibragimov, Jean Jacod, and David J Aldous. Exchangeability and related topics . Springer, 1985.

Carl Rasmussen. The infinite gaussian mixture model. Advances in neural information processing systems , 1999.

Radford M Neal. Markov chain sampling methods for dirichlet process mixture models. Journal of computational and graphical statistics , 2000.

John R. Anderson. The adaptive nature of human categorization. Psychological Review , 1991.

Adam N Sanborn, Thomas L Griffiths, and Daniel J Navarro. Rational approximations to rational models: alternative algorithms for category learning. Psychological Review , 2010.

Samuel J Gershman, David M Blei, and Yael Niv. Context, learning, and extinction. Psychological Review , 2010.

C. Kemp and J. B. Tenenbaum. The discovery of structural form. Proceedings of the National Academy of Sciences , 2008.

Samuel J Gershman and David M Blei. A tutorial on bayesian nonparametric models. Journal of Mathematical Psychology , 2012.

Bennet B Murdock Jr. The serial position effect of free recall. Journal of experimental psychology , 1962.

J. Sun, J. Li, and H. Zhang. Human representation of multimodal distributions as clusters of samples. PLoS Computational Biology , 2019.

Konrad Körding and Daniel M. Wolpert. The loss function of sensorimotor learning. Proceedings of the National Academy of Sciences , 2004.

Konrad Körding and Daniel M Wolpert. Probabilistic Inference in Human Sensorimotor Processing. In Advances in Neural Information Processing Systems , 2003.

Yeon Soon Shin and Yael Niv. Biased evaluations emerge from inferring hidden causes. Nature Human Behaviour , 2021.

L. Acerbi, S. Vijayakumar, and D. M. Wolpert. On the origins of suboptimality in human probabilistic inference. PLoS Computational Biology , 2014.

S. J. Gershman and Y. Niv. Perceptual estimation obeys Occam's razor. Frontiers in Psychology , 2013.

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems , 2019.

John A Nelder and Roger Mead. A simplex method for function minimization. The computer journal , 7(4):308-313, 1965.

Hirotogu Akaike. Information theory and an extension of the maximum likelihood principle. Selected papers of hirotugu akaike , pages 199-213, 1998.

William H Press, Saul A Teukolsky, William T Vetterling, and Brian P Flannery. Numerical recipes 3rd edition: The art of scientific computing . Cambridge university press, 2007.

Hang Zhang, Xiangjuan Ren, and Laurence T. Maloney. The bounded rationality of probability distortion. Proceedings of the National Academy of Sciences , 2020.

Horace B. Barlow. Possible principles underlying the transformation of sensory messages. Sensory communication , 1961.

Ishita Dasgupta and Thomas L Griffiths. Clustering and the efficient use of cognitive resources. Journal of Mathematical Psychology , 2022.

R. Polania, M. Woodford, and C. C. Ruff. Efficient coding of subjective value. Nature Neuroscience , 2019.

C. R. Sims. Efficient coding explains the universal law of generalization in human perception. Science , 2018.

Samuel J Gershman and Yael Niv. Perceptual estimation obeys occam's razor. Frontiers in psychology , 2013.

Bounded rationality in structured density estimation: Supplementary material

A Experimental details

A.1 Experiment 1

A.1.1 Participants

Experiment 1 recruited 21 participants (11 females, aged 18-25). All participants had provided informed consent before the experiment.

A.1.2 Cover story

Participants were told that they were apprentice magicians in a magical world. In this world, dangerous magic lava rocks were emitted from an unknown number of invisible volcano(es). On each trial, they observed past landing locations of lava rocks in a specific area (on the screen), and their job was to predict the probability density of future landing locations. More specifically, they were asked to draw a probability density by reporting, using clickand-drag mouse gestures, three key properties of the volcano(es), corresponding to the mean, the weight, and the standard deviation of a Gaussian component. They were told that their bonus payment depended on the accuracy of the reported predictive density.

A.1.3 Procedure and Design

On each trial, the landing positions of lava rocks were visualized as red dots and sequentially presented on a black line. The number of presented rocks (i.e., sample size) had two levels: 10 or 70. Each dot was presented for 166 ms, followed by a 166 ms empty screen. The landing positions were i.i.d. samples drawn from an unseen mixture distribution.

After observing all dots, participants needed to follow a two-stage procedure to report both (1) the predictive density of rocks' future landing positions, and (2) the underlying generative model. First, they marked the location of the volcano(es) by clicking on the black line. After marking all volcano(es), they pressed 'F' to proceed to the next reporting stage. In the second stage, they chose a volcano by clicking the volcano icon and then moved the mouse to report the relative density of the landing position of future lava rocks emitted from this volcano. During the report, the overall density of the rock landing position was computed and presented in real-ime.

The true generative distribution set was composed of 24 distributions (Fig. 6A). The distributions were created by following a merge-from-four procedure. All distributions' overall standard deviations were about 5.3 cm. On each trial, a uniform jitter ([-5.3 cm, +5.3 cm]) was added to the mean of the true generative distribution.

Participants completed 2 (sample size levels) × 4 (true cluster number levels) × 6 (distribution subtype) × 3 (number of repeats) = 144 trials in total. The experiment length was about 105 minutes.

A.1.4 Generation of true distribution set

The set of true distributions was constructed following the steps below. First, we created 6 four-cluster Gaussian mixture distributions (bottom row in Fig. 6A) in which each component had the same SD of 0.72cm of visual angle and equal weight. The three center-tocenter distances between their adjacent Gaussian components, denoted [ d 1 , d 2 , d 3 ] (from left to right, measured in cm), were chosen from the set of all permutations of { 3 . 6 , 4 . 65 , 5 . 7 } cm.

Second, each of the 6 four-cluster Gaussian mixtures went through 'merging steps', inspired by the proposal step in the dynamic clustering algorithms (e.g. Reverse-jump MCMC) in statistics [36]. In each merging step, we chose two adjacent Gaussian components to merge into one, with the post-merger new component having the same zeroth, first and second

B

True distribution:Gaussian

True distribution:mixture of 2Gaussian clusters

A

Sample size: 10

Sample size: 70

Sample size: 10

Sample size: 70

0.8

0.8

0.0

0.0

23

3

5

reported cluster

# reported cluster

>

True distribution: mixture of 3 Gaussian clusters

True distribution:mixture of 4 Gaussian clusters

MMMMM

0.8

Sample size: 10

Sample size: 70

0.8

Sample size: 10

Sample size: 70

0.0

0.0

2345

# reported cluster

#reported cluster

5

2

3

5

E

F

reported mean

Data

economical DEF

reported mean

Data

economical DEF

10

#reported cluster

3.2

10

0

economical DEF

-10

-10

2.8

data

T

T

T

-10

0

10

-10

0

10

-10

0

10

-10

0

10

2.6

sample mean

sample mean

sample mean

sample mean

10

70

reported sd

Data

economical DEF

reported sd

Data

economical DEF

samplesize

D

Sample size:10

Sample size:70

2

Overlap ratio (%)

15

20

7

2

5

8

2

5

8

2

5

8

2

5

8

sample sd

sample sd

sample sd

sample sd

10

reported skewness

Data

economical DEF

reported skewness

Data

economical DEF

5

2

0

0

2

4

3

#true cluster

#true cluster

-2

-2

0

2

-2

0

2

-2

-2

0

2

-2

0

2

sample skewness

sampleskewness

sample skewness

6sampleskewness

moments as the combination of the two pre-merger components. The to-be-merged adjacent components were chosen in such a way that the post-merger Gaussian mixtures minimize KLdivergence KL ( p pre || p post ) . By applying the merging step iteratively, the original four-cluster Gaussian mixtures were transformed into three-cluster, two-cluster, and finally one-cluster mixtures.

A.1.5 Results

See Fig. 6 for the prediction of the fitted economical DEF.

A.2 Experiment 2

Experiment 2 recruited 21 participants (13 females, aged 18-25). All participants had provided informed consent before the experiment. One participant was excluded due to

their task performance being an outlier (measured by the earth-mover distance between the reported predictive density and the true density, exceeding 3 standard deviations, z-score = -3.3).

Experiment 1 uses 14 representative Gaussian mixture distributions selected from previous studies (Table 1). The distribution set can be divided into four subtypes: unimodalsymmetric, unimodal-skewed, bimodal, and trimodal. Note that the number of modes is not necessarily equal to the number of latent Gaussian clusters. For example, a unimodal skewed distribution could be a mixture of two latent Gaussian clusters.

In each trial, participants observed 20 sequential samples drawn from one of the 14 distributions. The standard deviations of all distributions were rescaled to 4.8 cm. Each dot was presented for 166 ms, followed by a 166 ms empty screen interval. On each trial, a common uniform jitter ([-3.7 cm, +3.7 cm]) was added to the means of the clusters. To balance the skewness of the distribution in the experiment, we horizontally flipped the distribution in half of the trials. To explore participant's consistency in structure learning, we presented the same sample sequence for two times in seperate trials. Participants completed a total of 14 (distribution types) × 2 (flip or not) × 2 (number of random sequences) × 2 (number of repeats) = 112 trials. The experiment lasted about 90 minutes.

A.2.1 Results

See Fig. 7B-D&G for the prediction of the economical model.

Experiment 2 contained repeated trials with identical stimuli. Using these trials, we show in Fig. 7E that in repeated trials participants reported a different number of clusters just below 50% of the time. Similarly, our model could predict with an accuracy around 0.5, very close to the participants themselves. This shows that our model can predict human report close to the participants themselves.

To illustrate the robustness of the model fitting procedure, we run model recovery experiments. Given randomly chosen parameters for the full model, we generate 100 sets of synthetic stimuli, reset the parameters to new random values, and then fit the parameters on the synthetic dataset using the procedure described in the main paper. The results show that the recovered parameters are largely consistent with the random initial values (Fig. 7F, the average correlation between the source parameters and the fitted parameters is 0.84).

A.3 Experiment 3

Experiment 3 recruited 36 participants (21 females, aged 18-26). All participants had provided informed consent before the experiment.

SIP美

A

C

D

20

Unimodal-symmetric

# reported cluster

3.0

Overlap ratio (%)

15

Unimodal-skew

2.8

10

2.6

Bimodal

o

Uni-symm

Uni-skew-

Trimodal

Uni-symm-

Uni-skew -

Bimodal -

Trimodal

Trimodal

B

Unimodal-symm

Unimodal-skew

Bimodal

Trimodal

0.8

frequency

0.6

report

economical DEF

data

0.2

0.0

2

5

#reported cluster

G

E

reported mean

Data

economicalDEF

proportion of different K

5

0.55

0

errorrateof

0.50

the economical DEF

-5

-5

0

5

-5

0

5

0.45

sample mean

samplemean

Data

economical DEF

reported sd

10

same dist df samples

same dist, same samples

5

F

1.0

correlation

T

5

10

5

10

sample sd

sample sd

0.5

reported skewness

Data

economicalDEF

2

0.0

20

log(ao)

ug

β

log(e)

0

-2

-2

0

2

-2

sample skewnesssample skewness

In Experiment 3, participants needed to learn the distribution of numeric values and report their belief of the distribution by entering numbers on the keyboard. On each trial, the horizontal coordinates of lava rocks were shown one-by-one on the screen, with each coordinate presenting for 1.5 seconds, followed by a 0.5-second empty screen. After observing all coordinates, participants were required to first report the number of volcanoes. Then, they were required to enter the location and the relative eruption frequency of each volcano.

The true distribution set was composed of 2 unimodal distributions: one is a Gaussian distribution, and the other is a skewed Gaussian mixture with 2 wide clusters (Fig. 2A). Participants completed 3 (distribution type) × 8 (number of repeats) = 24 trials in total.

1

2

3

4

5

6

7

8

0.0

0.2

0.4

report

frequency

Unimodal-Gaussian

Sample size: 10

1

2

3

4

5

6

7

8

0.0

0.2

0.4

Unimodal-skew

Sample size: 10

1

2

3

4

5

6

7

8

0.0

0.2

0.4

report

frequency

Unimodal-Gaussian

Sample size: 70

1

2

3

4

5

6

7

8

# reported cluster

0.0

0.2

0.4

Unimodal-skew

Sample size: 70

economical DEF

data

A.3.1 Results

See Fig. 8 for the prediction of the economical model.

B Density estimation framework details

B.1 Rational component

The rational component should take the sequential observations x T and predict the internal construct properties φ r reported by a participant. From a computational perspective, any simulation-based model could be used for this component, as long as it can produce an internal construct based on x T . In this work, we adopt the rationale of Bayesian inference given a plausible generative process that could have produced the observations x T , with approximations mandated by cognitive constraints. Since the behavioral report is derived from a subjective belief of the unobserved distribution in the environment, the generative process in the participant's mind describes a subjective prior (or a construct) over distributions. As such, we call this generative process an 'internal construct prior' (ICP).

The ICP in our main model is inspired by the nonparametric Gaussian mixture model. A popular instantiation of the said model is defined through a CRP prior over the cluster assignments, and a conjugate prior over the distribution of each cluster, abbreviated as CRP-GMM. It is appealing for our purpose because the implied generative process of x T is sequential, similar to how participants observe the x T . To make this prior more flexible, we extend the cluster assignment prior by introducing additional parameters in (1) to control the expansion decay rate ( r ) and count distortion rate β . The effects of these two parameters on the prior distribution of partitions are shown in Fig. 9. While the number of clusters in the ordinary CRP grows, the decaying parameter α in our extension substantially slows down the introduction of new clusters as the model size increases. As of the distortion rate β , when β < 1 , there is weaker rich-get-richer effect, giving more evenly distributed cluster sizes and higher entropy in the cluster assignment distribution, while a β > 1 produces more extreme cluster sizes and lower entropy of the cluster assignment distribution.

B.1.1 Inexchangeability of the the economical ICP

The exchangeability of a cluster assignment prior is often desirable for online data modeling, because it ensures that the posterior of cluster assignment is invariant to the order (permutation) of data. However, in this work, we are interested in a prior that governs how humans construct a density model online. Our discussion on cognitive constraints in

Section 1 suggests that achieving order invariance requires implausible computations, so we propose to relax order invariance for the purpose of modeling human cognition.

̸

Here we show by counterexamples the proposed ICP cluster assignment prior (1) is not exchangeable. First, for the case β = 1 and r = 0 . consider the partition {{ z 1 , z 2 } , { z 3 }} . We have

which is not the same as the probability of a permuted but equivalent partition,

Then, for the case β = 1 but r = 0 , consider the partition {{ z 1 , z 3 , z 5 } , { z 2 , z 4 }} . We have

̸

B.1.2 Sufficient statistics

For completeness, we give the explicit expressions of the sufficient statistics in (3) as

We assume that participants maintain these sufficient statistics when building their internal constructs, and also report them at the end of the trial, after normalizing n i to obtain the weights. Note that, unconventionally, we denote the variance by σ .

2

10

20

30

time step

2

4

6

number of clusters

CRP,

r

=0,  =1

2

10

20

30

decays,

r

=1

2

10

20

30

distortion,

=0.5

2

10

20

30

r

=1,

=1.5

2

10

20

30

r

=1,

=0.5

1

2

3

4

5

6

cluster order

0

10

20

30

cluster size

1

2

3

4

5

6

1

2

3

4

5

6

1

2

3

4

5

6

1

2

3

4

5

6

̸

̸

B.1.3 Baseline batch ICP

As both the economical ICP in Section 3.1 and the proposed fitting algorithm (to appear in Appendix B.3) are new, it is important to compare the economical ICP and the exchangeable ICP, CRP-GMM, with another baseline ICP, all fit by the same algorithm. If we expect the baseline ICP to provide a worse description of human cognition than CRP-GMM on this task, then the proposed algorithm must be able to produce a less favorable model comparison result for this baseline after fitting them to human data. We choose a batch ICP as the baseline: it generates all observations as i.i.d. samples conditioned on cluster assignments, but, unlike the CRP, the cluster assignment prior is time-invariant and is thus exchangeable; inference over this ICP produces order-invariant posteriors. Because our task has a strong sequential nature, we expect this batch ICP to be a worse descriptor of human cognition for this task.

Specifically, this batch ICP p B maintains a truncated Poisson prior distribution over K max submodels of mixture distributions.

The K 'th submodel is a Gaussian mixture of K components. Each submodel has a symmetric Dirichlet prior over the cluster weights, and each component has mean and variance following the conjugate Gauassian-Inverseχ 2 distribution, as in (2). The truncated Poisson prior is a heuristic choice; we also tried other priors, such as a Categorical distribution supported on K = 2 and K = 3 , or a truncated Poisson with mean dependent on the sample size (10 vs 70). These choices improved the AIC only insignificantly.

Inference in each submodel The following description applies to each of the K max submodels. Given the observed noisy data ˜ x i T , each submodel produces an inferred cluster property φ i K , which are the sufficient statistics of the cluster property posteriors returned by the variational-EM algorithm (see Chapter 10.2 of [37]). We now drop the superscript i to reduce clutter, bearing in mind that all the variables are samples depending on a noisy observation ˜ x i T and a sampled K i . This algorithm alternates between the E-step: updating a Dirichlet posterior over the cluster weights; and the M-step updating a Gaussian-Inverseχ 2 posterior over the cluster means and variances. It turns out that the posterior over the cluster parameters φ K depends on a set of sufficient statistics similar to (8), except that the cluster assignments are now soft as computed by the responsibilities r t,k for each x t and k ∈ { 1 , . . . , K } during the variational E-step.

After the variational-EM procedure converges, we perform a hard cluster assignment for each observation x t by choosing the cluster with the highest responsibility,

This ensures that the cluster weights are quantized, as is the case for the particle-based inference methods on the other two ICPs-we do not want to introduce additional effects to the variational model due to having continuous support for cluster weights. Thus, at the last time step t = T , the cluster weights are determined by n k = ∑ T t =1 /x31 [ z t = k ] . The cluster mean µ k and variance σ k still depend on the original unquantized responsibilities and per-cluster sufficient statistics similar to (8) but with indicators /x31 [ z t = k ] replaced by responsibilities r t,k . As a result, for each submodel with K clusters, the inferred cluster properties are summarized by φ K := [ n k , ¯ µ k , ¯ σ k ] K k =1 .

Submodel selection After obtaining sufficient statistics for all K models, the batch rational component selects the submodel with the largest marginal likelihood given the hard cluster assignments, computed through Student's t marginal densities as in (4), minus a penalty of a weighted model size

where γ is a trainable parameter. This penalty is consistent with how AIC penalizes model complexity. Then, the submodel with K B clusters is selected, passing φ K B to the aleatoric component. As such, the batch-based rational component differs from the CRP-GMM in the following ways:

In the batch rational component, the cluster assignments are performed by variational inference rather than a particle filter;

the batch rational component performs model selection that penalizes large models after inferring multiple submodels, whereas CRP-GMM embeds the preference for smaller models in the cluster assignment prior.

Likelihood approximation These inferred cluster properties are passed to the aleatoric component, giving (slacked) number of cluster ˆ K B and the predicted ˆ φ B . This amounts to a single simulation of the batch DEF. To compute the likelihood, the batch DEF still needs to marginalize out the visual noise and the slacked ˆ K B in the aleatoric component. To this end, we run a large number of simulations, each with an independent draw of noisy observations ˜ x i T , giving the predicted cluster properties ˆ φ i B . The likelihood p ( φ r | x T ) for the batch model is then approximated by (7).

The batch rational model and the aleatoric component combine to give the batch DEF, which is used to benchmark the economical ICP and exchangeable ICP in the corresponding DEFs.

B.2 Aleatoric component

Here, we explain in more detail the aleatoric component described in Fig. 3. As discussed in Section 3.2, one challenge in modeling this dataset is that the dimensionality of the internal construct varies across trials. This requires that the model be able to produce variable-dimensional predictions. In order to fit the DEFs by maximum-likelihood, the DEF must place nonzero probability to all possible numbers of clusters. We thus defined the distribution p A ( ˆ K | K ) in (5) supported on { 1 , . . . , K max } , and restrict the maximum number of clusters to K max to be the largest number of clusters ever reported by participants in an Experiment. One can also define other slack distributions over K with decaying tails to avoid an explicit upper bound.

If there is no slack, then the predicted cluster properties are as inferred. If the participant slacks with probability ϵ and commits to a prediction ˆ K i that does not agree with the inferred K i from the rational component, they must modify the inferred φ i so that there are ˆ K i clusters. Weassume that, during modification, the participant should keep the overall distribution roughly intact. We propose the following deterministic procedure, denoted by f ( φ i , ˆ K i ) , which recursively increases or decreases the number of clusters in φ i until there are ˆ K i left.

Removing the smallest cluster. This happens whenever φ i has more clusters than ˆ K i . We simply remove the cluster with the smallest weight. An alternative is the following merging strategy: take the cluster with the smallest weight, and merge into its nearest neighbor. The merged distribution has weight equal to the sum of the weights of the clusters merged, and has mean and variances equal to the effective mean and variance of the two. More precisely, for two clusters with properties [ w 1 , ¯ µ 1 , ¯ σ 1 ] and [ w 2 , ¯ µ 2 , ¯ σ 2 ] (note that we denote the variance by σ ), the merged cluster has properties

where µ m = w 1 ¯ µ 1 + w 2 ¯ µ 2 and σ m = w 1 (¯ σ 1 +¯ µ 2 1 ) + w 2 (¯ σ 2 +¯ µ 2 2 ) -µ 2 m . Our results show that the removal strategy produced better AIC than the merging strategy.

Splitting the largest cluster. This happens whenever φ i has fewer clusters than ˆ K i . We take the cluster with the largest weight and split it into two clusters. The new clusters are centered at equal distance from and on two sides of the original cluster, and their variance is a scaled version of the variance of the original cluster. Specifically, denote the properties

10 2

10 3

10 4

M, # simulations

60

50

40

30

20

10

before fitting

10 2

10 3

10 4

14.0

14.1

14.2

14.3

after fitting

of this cluster to be split by [ w,m,σ ] , we set the cluster properties of the new clusters to be

where b σ > 0 and 0 ≤ s σ ≤ 1 are free parameters to be optimised. We also tested a version of this strategy where these two parameters are fixed at b σ = √ 3 2 and s σ = 1/4 , but this gave worse AIC.

Noise processes. The noise processes defined in (6) are given by

where ˆ K is implied from ˆ φ . The addition by 1 in (12) ensures that the mode of the Dirichlet distribution is equal to w ∝ ˆ n . The scaling parameter c changes the confidence. The variances σ µ and σ ν are free parameters.

B.3 Fitting algorithm

The MC estimator for the likelihood in (7) is unbiased. However, the corresponding loglikelihood estimator, adopted in practice for numerical stability, is only consistent and produces a nonzero bias for a finite number of simulations M . This is due to Jensen's inequality. Suppose each simulation provides a log-likelihood estimate of ℓ i conditioned on some latent variables (such as z i T and ˆ K i ). Let the true conditional likelihood be X so that e ℓ i ∼ X . then the estimated marginal log-likelihood is

Note that E [ X ] is the marginal likelihood. Fortunately, this bias is downwards, meaning that the expected MC estimate provides a lower bound on the true log-likelihood Still, we must use a large M during training and evaluation, as this reduces the variance of the empirical average in (15) and thus lowers the bias. We show in Fig. 10 the dependence of the estimated log-likelihood for different numbers of simulations, using data from a randomly

# Nelder­Mead restarts

8.0

8.2

8.4

8.6

8.8

9.0

9.2

log likelihood

Experiment 1

# Nelder­Mead restarts

7.4

7.6

7.8

8.0

8.2

8.4

log likelihood

Experiment 2

# Nelder­Mead restarts

2.0

2.2

2.4

2.6

2.8

3.0

3.2

3.4

log likelihood

Experiment 3

picked participant in Experiment 3. When the model is untrained (left panel), the bias does not completely go away even at M = 10 000 , but the variance is much smaller than using fewer M 's. When the model is well-trained, both the bias and variance of the estimated log-likelihood are substantially reduced (right panel). Thus, the likelihood estimates become more reliable as the DEF fits the data better.

Thanks to the small estimation variance, we are able to fit the parameters of the DEFs using a gradient-free optimization routine Nelder-Mead implemented in the NLOpt package 2 . During training, we estimate the log-likelihood using M = 10 000 parallel simulations on NVIDIA GTX 1080 and A100 GPUs. The latter GPU provides a likelihood estimate within 3 seconds for Experiment 2, and 5 seconds for Experiments 1 and 3. The Nelder-Mead routine typically converges to a 0.001 relative precision on parameters within 300 iterations.

We restart Nelder-Mead 10 times with parameters found from the previous optimization, as this avoids early convergence of Nelder-Mead. To further avoid local optima, we repeat this whole procedure (with 10 restarts) 10 times with different random seeds. We can then check if our algorithm has found a good solution by taking the maximum likelihood found across the 10 repeats. If this solution is reliable, then we should observe that this maximum is stable across the 10 restarts. We then take the best solution among the 10 random seeds at the last repeat as the fitted DEF parameters.

Fig. 11 shows the best (among seeds) log-likelihood across 10 restarts, averaged over all participants, for each DEF setup (see Appendix B.5.1). During the first repeat, the loglikelihood increases and stabilizes. Because we do not keep the best log-likelihood across the restarts, we see small fluctuations across the restarts. We see a very slow increase of log-likelihood for some DEFs in Experiment 3, but the relative rank of different DEFs is mostly preserved.

B.3.1 Fitting the batch DEF

For the Batch DEF, each simulation maintains K max submodels before the comparison in (10). Because the cluster weights posterior can be computed during variational inference, unlike in the CRP where we used particles, we do not need as many simulations, and so we run M = 100 simulations of each of the K max submodels. The output of the Batch rational component is then fed into the same aleatoric component for a fair comparison between the ICPs.

B.4 Particles versus MC simulations

There is an important distinction between the number of particles and the number of MC simulations. In this work, we assume that the participant uses a single particle for inferring the cluster assignments, but this is not observed from the experimenter's viewpoint. The reported internal construct may be associated with one out of many possible cluster

assignments. As such, we run many simulations of the single-particle particle filter. The number of MC simulations is M , and each simulation is indexed by i . Since we stick to the single-particle assumption throughout the paper, we do not need an additional index for the number of particles within each MC simulation.

Future work may consider multiple particles. In this case, each MC simulation will contain multiple sequences of cluster assignments. Maintaining multiple cluster assignment sequences allows re-assignment or re-weighting of the observations at each time step, achieving some retrospective correction. However, it is unknown how the participant makes a decision on reporting the cluster properties from multiple samples, especially when the number of clusters do not agree across different particles. We thus leave multiple-particle models for future work.

B.5 Additional modeling results

After training the DEFs, we evaluate the log-likelihood using a large number of simulations: 10 6 for the sequential DEFs in our main results, and 10 5 for the baseline batch DEF.

B.5.1 Ablation studies on DEF

Our initial experiment design used the exchangeable DEF (with a CRP-GMM rational component) as a reference model. We explored the model space by making small modifications to this DEF motivated by resource constraints and other heuristics. Here, we show that the two modifications in the economical DEF, namely the Decay in expansion rate α t and Distort ion in cluster count n t,k produce reliable improvements over the reference exchangeable DEF. The DEFs resulting from other modifications either did not produce reliable improvements or were insignificant compared to the reference model. All model parameters are listed in Table 2. We introduce the modifications below.

Decay : adding a model-size dependent decay rate r , as in (1).

Distort : adding an exponential transformation to the size of the clusters, as in (1).

Fixed Splitting : when ˆ K i < K i and splitting a cluster, the parameters of the splitting are fixed with b σ = √ 3/2 and s σ = 1/4

Merge Cluster : when ˆ K i < K i , instead of removing a cluster, the function f ( φ, ˆ K ) merge the smallest cluster into the cluster with closest mean. The cluster properties of the new component is computed by moment matching, as detailed in Appendix B.2.

Local MAP : instead of sampling the cluster assignment according to (4), take the cluster with largest posterior probability. This is the local MAP procedure described in [15].

Constant Variance : instead of updating the variance of each cluster according to (3), the cluster mean is fixed to the trainable parameter σ 0

Fixed Mean : instead of updating the mean of each cluster according to (3), the mean is fixed at the first observation assigned to the cluster. This checks if participants are able to update the mean or simply remember the first observations assigned to the clusters.

Fixed Mean Confidence : instead of fitting λ 0 as a parameter, we fix it at λ 0 = 0 . 01

No Visual Noise : do not add Gaussian noise to the observations.

No Distribution Prior : when computing the per-step assignment posterior in (4), the likelihood is a Student's t -distribution obtained from having a conjugate prior over the Gaussian distribution parameters. Instead of computing this likelihood using the t -distribution, this modification computes this likelihood by a Gaussian with mean µ t,k and variance σ t,k .

No Counting Prior : in (1), instead of using the n t,k maintained for each cluster, use the average cluster size.

The results of these DEFs, averaged over participants, are shown in ?? . Clearly, only Decay and Distort produced reliable improvement on AICs compared to the CRP-GMM DEF across all Experiments. All other modifications that deviate from a rational approximation

of the Bayes rule (e.g. Fixed Mean, No Counting Prior, etc.) resulted in significantly worse fit to the reported internal constructs from our participants. We also see that removing visual noise is detrimental to the quality of the fit.

The stability of the likelihood approximated from (7) is shown in Fig. 11. All DEFs fitting converged well as the number of restarts increases, except for a few worse DEFs in Experiment 3.

Proposed

Decay

Local MAP

Merge Cluster

CRP­GMM

Fixed Splitting

No Counting Prior

Fixed Mean Confidence

Fixed Mean

No Visual Noise

No Distribution Prior

2600

2400

AIC

Proposed

Decay

Local MAP

Merge Cluster

CRP­GMM

Fixed Splitting

No Counting Prior

Fixed Mean Confidence

Fixed Mean

No Visual Noise

No Distribution Prior

0.50

0.55

error rate on cluster count

Proposed

Decay

Local MAP

Merge Cluster

CRP­GMM

Fixed Splitting

No Counting Prior

Fixed Mean Confidence

Fixed Mean

No Visual Noise

No Distribution Prior

0.2

0.0

NLL on weight

Proposed

Decay

Local MAP

Merge Cluster

CRP­GMM

Fixed Splitting

No Counting Prior

Fixed Mean Confidence

Fixed Mean

No Visual Noise

No Distribution Prior

0.10

0.15

normalized MSE in cluster mean

Proposed

Decay

Local MAP

Merge Cluster

CRP­GMM

Fixed Splitting

No Counting Prior

Fixed Mean Confidence

Fixed Mean

No Visual Noise

No Distribution Prior

0.9

1.0

normalized MSE in cluster log­variance

Proposed

Decay

Local MAP

Merge Cluster

CRP­GMM

Fixed Splitting

No Counting Prior

Fixed Mean Confidence

Fixed Mean

No Visual Noise

No Distribution Prior

8

7

NLL on mean and variance

(a) Experiment 1

Proposed

Decay

No Counting Prior

CRP­GMM

Merge Cluster

Fixed Splitting

Local MAP

Fixed Mean Confidence

Fixed Mean

No Distribution Prior

No Visual Noise

1800

1700

AIC

Proposed

Decay

No Counting Prior

CRP­GMM

Merge Cluster

Fixed Splitting

Local MAP

Fixed Mean Confidence

Fixed Mean

No Distribution Prior

No Visual Noise

0.5

0.6

error rate on cluster count

Proposed

Decay

No Counting Prior

CRP­GMM

Merge Cluster

Fixed Splitting

Local MAP

Fixed Mean Confidence

Fixed Mean

No Distribution Prior

No Visual Noise

0.0

0.2

NLL on weight

Proposed

Decay

No Counting Prior

CRP­GMM

Merge Cluster

Fixed Splitting

Local MAP

Fixed Mean Confidence

Fixed Mean

No Distribution Prior

No Visual Noise

0.12

0.14

normalized MSE in cluster mean

Proposed

Decay

No Counting Prior

CRP­GMM

Merge Cluster

Fixed Splitting

Local MAP

Fixed Mean Confidence

Fixed Mean

No Distribution Prior

No Visual Noise

0.9

1.0

normalized MSE in cluster log­variance

Proposed

Decay

No Counting Prior

CRP­GMM

Merge Cluster

Fixed Splitting

Local MAP

Fixed Mean Confidence

Fixed Mean

No Distribution Prior

No Visual Noise

7.0

6.5

NLL on mean and variance

(b) Experiment 2

Proposed

Fixed Splitting

CRP­GMM

Decay

No Counting Prior

Local MAP

Fixed Mean Confidence

Merge Cluster

Fixed Mean

No Visual Noise

No Distribution Prior

150

100

AIC

Proposed

Fixed Splitting

CRP­GMM

Decay

No Counting Prior

Local MAP

Fixed Mean Confidence

Merge Cluster

Fixed Mean

No Visual Noise

No Distribution Prior

0.60

0.65

error rate on cluster count

Proposed

Fixed Splitting

CRP­GMM

Decay

No Counting Prior

Local MAP

Fixed Mean Confidence

Merge Cluster

Fixed Mean

No Visual Noise

No Distribution Prior

1.0

0.5

NLL on weight

Proposed

Fixed Splitting

CRP­GMM

Decay

No Counting Prior

Local MAP

Fixed Mean Confidence

Merge Cluster

Fixed Mean

No Visual Noise

No Distribution Prior

0.15

0.20

normalized MSE in cluster mean

Proposed

Fixed Splitting

CRP­GMM

Decay

No Counting Prior

Local MAP

Fixed Mean Confidence

Merge Cluster

Fixed Mean

No Visual Noise

No Distribution Prior

1

0

NLL on mean and variance

(c) Experiment 3

0

2

4

economical

0

2

4

6

0

5

10

15

0.150

0.175

0.200

0.225

0.250

0

0

2

4

exchangeable

0

25

50

75

100

a

0

0

2

4

6

>

0.0

0.2

0.4

0

0

5

10

B.5.2 Strong prior on cluster width

We found in most fitted ICPs that the prior cluster standard deviation is small (see Fig. 5 and Fig. 13), and the confidence indicated by its pseudocount a 0 is much greater than zero. This high confidence contrasts with the low pseudocount associated with the prior cluster mean λ 0 . This means that participants could learn the cluster mean mostly driven by the observations while failing to adapt to the cluster uncertainty. This contributes to the large number of clusters seen when there is a single Gaussian in the true data distribution. We noted that Gershman and Niv [35] also used a relatively high a 0 = 10 parameter for their task.

References

[36] Sylvia Richardson and Peter J. Green. On Bayesian analysis of mixtures with an unknown number of components (with discussion). Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 1997.

[37] Christopher M. Bishop. Pattern Recognition and Machine Learning , Springer, 2006.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe author built a new model to explain the human mental model of density estimation given sequentially observed data points. The model consists of a “Rational component” that generalizes the Chinese restaurant process. And an “Aleatoric component” that adds some error term. The model is fitted on real experimental data and showed superior performance compared to baseline.\n\nSTRENGTHS:\nThe real human experimental data application looks interesting and novel.\n\nWEAKNESSES:\nThere are some parts of the paper that are unclear to me. See the questions section below.\n\nQUESTIONS:\nExperiment:\n\nI wonder how much of the over-estimation of cluster sizes is due to the small size of samples we generated? Presumably, if the ground truth number of clusters is huge, the human being will instead under-estimate the number of clusters?\n\nRational component:\n\nIt doesn’t seem too clear to me why is the process “economical”. Could you clarify this? My best understanding of the author’s argument is that when the process is exchangeable like a CRP, it involves memorizing all the points which is cognitively implausible. But I think as long as we are doing bayesian inference and getting posterior, it implies memorizing even if things aren’t exchangeable. Another interpretation is that the author is suggesting that participants are not doing a fully bayesian posterior estimation, but some step-wise procedure.\n\nI wonder if it would be useful to give some discussion of comparing economical ICP with say pitman-yor process? (e.g. some more literature review of extensions of CRP?)\n\nAleatoric component:\n\nIn general, I am a bit confused by this part. Is it just a technical point to make MLE feasible, or is it trying to model humans making mistakes?\n\nLine 194: It isn’t clear to me why “the dimensionality of the report is variable” is a challenge that needed to be solved with an aleatoric component. Could you clarify this? I thought nonparametric Bayes prior like CRP already adapts to variable dimensions well. Are you saying that the aleatoric component is definitely needed as otherwise the model cannot be fit at all? Or just that it makes things more numerically stable or realistic?\n\nFormula (5), seems a bit too simplified, is it computationally trickier to set something that gives probability based on how close K_hat is to K_i? I understand that the author suggests that we can pick whatever slack functions but I feel this particular choice somehow seems too naive.\n\nFigure 5(a): It's unclear to me the major differences between economic vs exchangeable methods?\n\nGeneral:\n\nI wonder if there should be something that models human forgetting previous examples and upweighting examples near the end. As far as I can tell that's not in the current framework.\n\nTypo:\nLine 148: n_t,k number of “clusters” assigned to cluster k. clusters should be samples?\n\nLIMITATIONS:\nThe authors discussed about the limitations of their work in the last paragraph. I wonder if they can provide more discussions on the possible future work to address these limitations?", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors describe a density estimation task with humans wherein participants were asked to identify parameters of an unknown distribution from presented data. While participants did a better job of estimating the overall density with more samples, the authors observed a large error in the reported number of clusters in the underlying distributions (and that increased with sample size but typically fell in the range 2-3).  The participants also tended to estimate the clusters as non-overlapping.  They modeled this process with a non-parametric mixture model.   This model consists of a “rational” component and an “aleatoric” component.  The rational component uses for cluster assignments an extension of the Dirichlet process that reduces the relative probability of adding new clusters as new points are added and modulates the relative probability of assignment to an existing cluster with a form of divisive normalization.  Unlike the regular Dirichlet process, this process is not in general exchangeable, a property the authors argue is cognitively implausible and unnecessary.  The aleatoric component adds “structured noise” in order to compute an approximation to the marginal likelihood.  They compare their model against both an exchangeable version and a baseline “batch” learning prior.\n\nSTRENGTHS:\nThis is an interesting paper about the important problem of how people form internal models from observations.  The authors use an interesting approach with a non-parametric prior and provide a fitting routine that uses state-of-the-art tools for efficiency.  The experiments are clearly described and the model is (mostly) well presented.  The evidence that humans do not form models based on exchangeability is an interesting result.\n\nWEAKNESSES:\nOverall, the presentation is excellent, but the description and motivation of the aleatoric component is lacking, to me.  Adding some intuition about why you made the choices you made for the model would be helpful.  The “slack” probability seems particularly arbitrary to me, as does the noise added in equation 6.  Why are these necessary?  You mention in the appendix that “the DEF must place non-zero likelihood to mixture models with all possible numbers of clusters” which makes sense because many potential clusterings could result in the same reported number of clusters, but then you essentially remove from the estimation (equation 7) any sample in which the number of clusters does not match what is reported (line 208).  The “slack” probability seems to carry all of the weight of this possibility.  It is likely I’ve misunderstood something, but I think this section could be clearer.\n\nThe results from Figure 1 raise some interesting questions about the need for this particular model.  I can’t tell since they are bar plots, but it makes me wonder how consistent the cluster number estimate is and what its dependence is on the number of data points.  Moreover, can the data support a simpler psychological heuristic that assigns a number of clusters based on the number of observed samples, particularly given that subjects tend to estimate non-overlapping clusters?  It is not clear to me whether the baseline model accounts for this possibility.  I’m imagining a heuristic where people divide the screen into chunks and assign clusters that way, making finer divisions with more and more data points.  Is it clear that this is not what is happening?  Perhaps this is an ignorant question, but it seems to me that the estimation should be driven by some important visual priors.\n\n\nMinor:\n\nLine 129-130:  detailed mechanics are [not?] directly obvious…\nLine 150:  For r > 0, new clusters are… (not r< 0).\nLine 206:  There [are] \\hat{K}^i clusters…\n\nQUESTIONS:\nWhat is the spread of data in Figure 1, particularly panels B and D?  Are participants consistent in the number of clusters assigned?  Is there a potentially simple relationship between number of clusters and sample size?  Are there simpler null models to compare against that make some relatively add hoc cluster assignments of this nature?  Maybe the baseline model behaves this way, but it is not clear to me that that is the case.  \n\nK_max from the aleatoric component is across participants, which means the posterior takes into account observations from other participants.  Is this a practical concern?  Can you replace K_max with some large but not unwieldy number and get similar results (I assume so)?\n\nThis is beyond the scope, I think, but is there a way to get running estimates of the distribution over the course of the experiment?  Your experiments don’t directly have this information, but can you get a clue to how people update models of distributions by comparing the experiments with different sample sizes?  This seems a natural question given the result that non-exchangeability seems to be important in modeling the human behavior.\n\nLIMITATIONS:\nThe authors do not discuss societal impact nor directly discuss limitations.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a visual density estimation task for human subjects based on sequential data from gaussian mixture models, complete with experimental data analyzed through a bounded-rational model of behavior. The paper reports data on the quality of the density reported by subjects, claiming that it roughly matches the true distribution's first three moments while highlighting some systematic mismatch in the number of reported mixture components. A variant of the task is performed using a different modality (numerical data instead of visual stimuli). After the task and the data is presented, the paper develops a bounded rational model for behavior. This model is composed by a generalization of the chinese restaurant process plus a sophisticated noise component. The main difference from the CRP is that this model implements in general an \"economical\" density estimation process where the likelihood of adding new clusters decreases faster than in the CRP. The behavioral model comes with an inference scheme implemented in pytorch, which allows for gpu acceleration. This model is applied to the data, showing that the \"economical\" version explains the data better than simpler alternatives (regular CRP or batch learning where cluster assignment is not sequential). This is argued to support the idea that subjects are affected by constraints on memory capacity.\n\n\n-----\n(Scores edited following the author's rebuttal)\n\nSTRENGTHS:\n- The paper contains a large amount of novel work, including a behavioral task, a bounded rational observer, and advanced techniques for inferring the parameters of the observer and comparing it with alternatives explanations of empirical data.\n\n- The task is ambitious in that it has a high-dimensional report.\n\n- The bounded rational observer is nontrivial but well explained, despite the limited space. From a statistical standpoint, this model and its efficient implementation are valuable.\n\nWEAKNESSES:\n1. The statements that the subjects's density estimates are reasonably good do not seem sufficiently supported by the data. Line 86: \"the reported densities tracked the ﬁrst three moments reasonably well\", but figure 1 does not show the data (the values of the true and reported moments), only the estimated correlation between them. Line 105: the density estimation quality is \"reasonably good\" in experiment 3, but figure 7 does not show any data on this - it only shows that the economical DEF reproduces certain patterns in subject behavior.\n\n2. As pointed out in the paper (lines 291-292, \"the prediction errors on the number of clusters and cluster variance are still large\"), the economical internal construct prior (ICP) model is simply not very good at describing the empirical behavioral data. This is not necessarily a problem by itself, but it makes it difficult to draw any conclusions about human behavior from the application of the model to this dataset. In particular, the claim that \"We provide key experimental and modeling evidence that humans may employ an online and yet ﬁnite (in expectation) model\" does not seem adequately supported: if the economical ICP does not describe well subject behavior, it does not matter that it does comparatively better than the CRP or batch learning.\n\nQUESTIONS:\n1. Can you provide more evidence that the quality of the density estimation reports is \"reasonable\", despite the clear systematic errors in the number of components?\n\n2. Can you clarify why the model provides experimental insight into human behavior, if it does not capture basic features of the behavior like mean and variance of the components? Moreover, how is this mismatch compatible the claim that \"The DEF with the economical ICP captures all of the behavioral patterns reported in Section 2, including the inconsistent number of clusters and the low overlap ratio between adjacent clusters\"? Figure 5A (referenced in this passage) only shows three examples; I can't find any quantitative or statistical support for this claim. Alternatively, the claims about the insights derived from the application of this model to human behavior should be significantly reduced.\n\n3. Figure captions should describe what's in the figure, not (just) provide comments. For instance, the caption of Figure 1E says \"adjacent clusters in the reported distributions did not overlap much\", but does not say what is being plotted there. How is the overlap computed? The figures would be clearer if this pattern/tendency in the captions was avoided.\n\nLIMITATIONS:\nThe paper does highlight limitations in the results, although (as discussed above) other passages claim conclusions that do not seem supported by the data. Potential negative societal impact is not a concern here.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors consider the question: how do humans estimate probability distributions? They study this by performing three experiments where human subjects, where they ask participants to recover Gaussian mixture models after seeing IID samples. They find that while subjects do seem to get closer to the true mixture model as they see more samples, they consistently seem to report about three different components regardless of the ground truth number of components. They speculate that this is due to humans having limited memory.\n\nThey then propose a human model consisting of a rational component that performs approximate Bayesian inference which generalizes the standard Chinese Restaurant Process (CRP) and an _aleatoric_ component where humans pick the number of components and then merges and splits clusters until the desired number of components is reached (with some noise). They derive an efficient estimator for the likelihood under this model using Monte Carlo simulation. They then show that their model fits their experimental data much better than the standard CRP. The better performance of their model suggests that it does not seem necessary to keep the human prior exchangeable.\n\nSTRENGTHS:\n* The paper is generally clearly written, and the authors do a good job of motivating their problem and describing their results. \n* The authors' structured online density estimation task is clean and straightforward. \n* The key experimental observation (that the human subjects seem to consistently estimate around 2 or 3 clusters, regardless of ground truth) is very clearly presented and clearly of great interest to this subfield of human modeling. \n* The authors two additional experiments validating the key experimental observation, with different distributions and different domains.\n* Their proposed aleatoric component follows naturally from their key observation, and cleanly captures the inconsistent number of clusters reported by human subjects. \n* The authors experimentally validate that their proposed model generally performs better than the standard CRP (as well a batch learning baseline) for all three experiments, using five different criteria: AIC, #clusters correct, negative log likelihood, the error in means, and the error in log variances.\n * The authors give the fitted parameters and discuss how to interpret them, which I found quite interesting.\n\nWEAKNESSES:\n* All three experiments featured relatively simple Gaussian mixtures -- for example, experiment 1 has 1-4 clusters, experiment 2 has 1-3 clusters, and experiment 3 looks at _unimodal_ Gaussian mixtures/distributions. As a result, it seems hard to distinguish the interpretation where human subjects have limited memory and are using the proposed ICP from the interpretations where human subjects just happen to estimate 2--3 clusters across the board, or where they consistently tend to overestimate the number of clusters for other, unrelated reasons.\n* Similarly, the proposed ICP seems quite convoluted (e.g. the authors' likelihood estimator requires Monte Carlo simulation) and I'd be surprised if human subjects were actually using it.\n* Despite the complexity of the proposed ICP and the relative simplicity of the distributions being studied, the error bars for fitting human predictions are still relatively large (as shown in Figure 4), suggesting that it does not accurately capture the underlying human behavior. \n* I found figure 3 confusing; in particular, while the caption was quite helpful to my understanding, I still don't think I understand the rational or aleatoric component parts of the figure, and I did not find them helpful when I was first reading the paper. \n* Similarly, I found the discussion of how the aleatoric and rational components were composed to be initially confusing.\n\nQUESTIONS:\n1. It seems like the combination of the rational and aleatoric components are quite complicated -- have you considered simpler models? For example, instead of doing online expansion, perhaps the ICP just has a fixed prior distribution over cluster count. \n2. Similarly, given the complexity of the ICP, I'm curious how the authors interpret the fact that human subjects are able to (apparently) use it. \n3. As mentioned in the weaknesses section, I'm curious about how these results generalize to more complicated inference tasks. I understand if the experiments have not been done, but what do the authors expect the results to be if you looked at experiments with more clusters or in more realistic settings?\n4. I would've appreciated more discussion about the actual implementation of the estimator -- for example, how reliable is the log-likelihood estimate, and how sensitive is it to the number of MC simulations?\n5. Similarly, I would've appreciated more analysis of when the predictions of their proposed ICP fall short of reality (as opposed to only cases where the number of clusters agree with human report). \n6. As this is a relatively theoretically oriented paper, this is not strictly necessary. However, I wonder if the authors have seen the tendency for human subjects to pick a fixed number of clusters in other contexts, perhaps due to computational constraints. Do you expect this to have implications in practice? Are there more realistic contexts where this tendency causes systematic deviation from \"normative\" behavior?\n\nIn addition, there is the following typo in the paper:\n* Line 205-6: ... until **there** $\\hat K^i$ clusters -> ... until **there are** $\\hat K^i$ clusters\n\nLIMITATIONS:\n* I commend the authors for explicitly highlighting the relatively poor fit (albeit significantly better than prior models) of their proposed ICP on their experiments. However, I would've appreciated more discussion about why this might happen, and what next steps could be taken to address this issue.\n* I think the authors should include more discussion of the limitations and weaknesses I mentioned in the weaknesses section above.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis study investigates how humans infer probability distributions from samples by combining experiments and modeling. The main contributions include a careful characterization of the behavioral tendency to overestimate the number of clusters as well as a modeling framework to identify how this behavior can arise from approximate inference. By fitting parameters from a generalized version of the Chinese restaurant process, the authors conclude that behavior can arise from a myriad of factors including strong prior expectations about the sample variance, undercounting samples, as well as a decaying tendency to form new clusters.\n\nSTRENGTHS:\nHuman experiments are well-summarized and clearly document a tendency to overcluster samples. By directly asking participants to report the data-generating distribution, the approach provides a direct readout of the participant's biases that are brought to bear when solving complex probabilistic inference problems. \n\nThe paper is well-written and easy to read except for the section introducing the modeling framework where I got lost in the notations for a while. But the illustrations were very helpful.\n\nThe modeling approach is both rigorous and appropriate for answering the questions tackled by the study.\n\nWEAKNESSES:\nI see two main weaknesses. \n\nFirst is that the insights gained from modeling seem to be quite minimal and specific to the paradigm used here. I'd like to know whether/how the conclusions might apply more broadly to naturalistic scenarios.\n\nSecond, given that the model fits are not that great, it is quite possible that there are alternative models not considered here that explain behavior better. A discussion of alternatives is lacking.\n\nQUESTIONS:\nSince the fitting procedure relies on approximate inference, I'd like to see a validation of the model fits to show that the method is able to recover the true parameters. Is this already published or included somewhere in the supplementary?\n\nBased on the best-fitting parameters, the authors point out a trade-off between two different effects. A decaying tendency to form new clusters (which should lead to underclustering) and a strong prior that underestimates the variance of the gaussians in the mixture (which should lead to overclustering). Could the trade-off simply be a result of over-parameterization? Again, some sort of validation of the model fitting procedure would help here.\n\nOn a related note, it wasn't clear to me how the model explains why humans report more clusters when sample size increases from 10 to 70. Shouldn't the decaying $\\alpha$ lead to fewer clusters as a function of sample size?\n\nGiven that data exchangeability is not a constraint for human inference, why not validate the model directly by running a new experiment where the sequence of samples is presented in a reverse order? I suppose the model will have concrete predictions for such an experiment.\n\nLIMITATIONS:\nThe authors are clear about the weakness of the model in capturing aspects of the data. But they should discuss possible alternative approaches. There should be some discussion of the limitations of this paradigm and what it means for investigating structure learning in general.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-15"}