Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: The authors present a unifying account of how the bias and variance of the Bayesian mean estimator (in an encoder-decoder setup) are determined from the prior and the encoder’s Fischer information. They then use these approximations to the bias and variance to explore the behavior of Bayesian-optimal systems across a fairly wide range of objectives (estimation, MI maximization, discrimination). Their theoretical findings recapitulate known findings, and also offer an explanation for anti-Bayesian bias (when the bias is toward the stimulus with lower prior probability). Specifically, the bias is proportional to the sum of a Bayesian effect (toward the prior) and an anti-Bayesian effect, where the dominant direction depends on the specifics of the problem. The authors show that their approximations are fairly consistent with true values for a simple problem when the encoding noise is low.

main_review: Overall, this paper is fairly well-written, clear, and well-thought out. The theoretical work appears sound, the simulations are fairly convincing, and the conclusions seem interesting. (However, being somewhat outside this exact field, I hope to leave it to others to assess the novelty of this work vis-a-vis other works that have promised to unify Bayesian and anti-Bayesian biases under a single framework.)

That said, one thing that would have made this paper more impactful to me would have been a more thorough interrogation of _why_ Bayesian or anti-Bayesian biases are optimal in specific situations. It seems like this is a deep question into which these authors have been granted a privileged viewpoint—their approximation to the bias. More specifically, I wanted the authors to spend more time unpacking the I’(x)/I(x) term, what it means, and what intuition we might gain that would help us guess which problems will have Bayesian or anti-Bayesian bias.

limitations_and_societal_impact: The discussion of limitations is adequate.
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: The paper derives a parsimonious theory for the performance of a Bayes mean estimator under optimal encoder-decoder model. After identifying the equations for bias and variance, the authors explain bayesian and anti-bayesian biases that depend on task, and the relationship between bias and derivative of variance. 
main_review: It is a very well written paper. While the mathematical results might not be completely new, but as the authors say, the connections to psychophysics are novel.

* The title should be made less general, and mention the implications for perception/psychophysics. 

* How is the constraint on Fisher Information $\int\sqrt{I(x)}dx \leq \sqrt{K}$ justified? 
How are the mathematical results affected when you use a slightly different constraint (such as $\int I(x)^q dx \leq K^q$ for arbitrary $q$ for example)? Do the main insights (such as anti-bayesian bias for certain tasks) still hold ? 

* Under what practical conditions are the assumptions of the theory violated? Listing them concisely would help a reader applying it to practical data. 

* Moreover, the utility of theory would be much more apparent if a re-interpretation of existing empirical data would be presented. Improving the paper along this line would make the paper more impactful.

* Related, simulations with more practical considerations (finite n) and correlated neurons would help in highlighting the limitations of the theory for a reader. Another limitation is the assumption that decision reflects bayesian mean estimation. 
limitations_and_societal_impact: No negative societal impact
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This is theoretical contribution investigating Wei and Stocker's so-called “law of human perception” (ref 1). The authors generalize the "law" to more include general loss functions inspired by Morais and Pillow (ref 9).
main_review: I rather enjoyed reading this theoretical paper, for its clarity and rigor of the exposition.
Originality, however, could be higher. The main results are derived combining two existing and well known papers (ref 1 and 9)

The quality of the paper is good, technical derivations are detailed enough to follow and the paper does a good job of situating the contribution among  a fair a relevant set of references.  The main objection to the Bayesian approach are not included e.g. the critique offered by Bowers, J.S. and Davis, C.J., 2012. Bayesian just-so stories in psychology and neuroscience. Psychological bulletin, 138(3), p.389.    See also the well-tempered rebuttal of this critique in Griffiths, T.L., Chater, N., Norris, D. and Pouget, A., 2012. How the Bayesians got their beliefs (and what those beliefs actually are): comment on Bowers and Davis (2012).

Clarity is good. The paper is well written. 

Significance is mainly in reviewing and putting the works of ref 1 and ref 9 in a joint exposure. 
limitations_and_societal_impact: See above - no adversarial impact on society detected
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper investigates the bias (an the variance) of the Bayesian-mean decoder. The authors found that the bias is driven by two components: The variations of the priors and variations of the Fisher information, and the exact relation is driven by the encoding objective. Simulations are used to back the results.

Disclaimer: I am an experimental visual psychophysicist with experience in Deep Learning. Thus the paper falls a bit outside my area of expertise (as indicated to the AC). I am focusing on the general aspects of this paper.
main_review: Quality:

I think this paper could be of great interest to the experimental/psychophysical community in general. However, I am missing one crucial point here; that is a comparison to experimental data. A comparison to the experimental data would make the paper much stronger. Right now, the mathematical details make an accurate impression, but without empirical support, it is difficult to judge the evidence for this theory. For this purpose, the authors could easily use existing datasets.

From my point of view, using empirical data would make the paper much more interesting; for example, if one determines the mean and variance for some stimuli experimentally, one can draw conclusions about the corresponding coding strategy. Vice versa, one could try to enforce different encoding strategies and verify the predicted relation of mean and variance.

Clarity:
Overall, the quality of the paper is very high. The previous work is presented clearly. Also the mathematical parts are well structured.
limitations_and_societal_impact: The authors describe the limitations of the work implicitly. Although, they are never explicitly stated.
review_title: null

Meta-review: This is a very clearly written paper that introduces a "parsimonious theory for the performance of a Bayes mean estimator under optimal encoder-decoder model. After identifying the equations for bias and variance, the authors explain bayesian and anti-bayesian biases that depend on task, and the relationship between bias and derivative of variance." (Reviewer hpiu)

The main results are derived combining existing work, but the connections to psychophysics are novel. The findings (for example the emergence of contradictory biases under a consistently-optimal encoding-decoding model, that depend on the cost of errors in the task) are very interesting. Supported also by the highly positive sentiment of the reviews, I anticipate that the paper will be perceived with interest by a broad audience, hence I suggest a spotlight.  
Decision: Accept (Spotlight)