SlowGuess/NeurIPS_2023_OCR huggingface dataset:
- For the one-to-one code, we let  $n_{\mathrm{g}} = n_{\mathrm{odor}}$ , and simply set  $\Gamma = \mathbf{I}_{n_{\mathrm{odor}}} / \max (A_{ij})$ .
- For the naively distributed code, we let  $n_{\mathrm{g}} = 5n_{\mathrm{odor}}$ , and choose  $\Gamma$  as follows: We sample a sparse, non-negative random matrix  $\mathbf{Q} \in \mathbb{R}^{n_{\mathrm{odor}} \times n_{\mathrm{g}}}$  with entries that are non-zero with density 0.15, and the non-zero entries are drawn uniformly on [0, 1]. Then, we define  $\Gamma = \mathbf{Q} / \max \{|(\mathbf{A}\mathbf{Q})_{ij}|\}$ .
- For the geometry-aware code, we let  $n_{\mathrm{g}} = 5n_{\mathrm{odor}}$ , and choose  $\Gamma$  as follows: We sample a sparse, non-negative random matrix  $\mathbf{Q} \in \mathbb{R}^{n_{\mathrm{odor}} \times n_{\mathrm{g}}}$  with entries that are non-zero with density 0.15, and the non-zero entries are drawn uniformly on [0, 1]. Then, we compute an approximate inverse square root of the low-rank matrix  $\mathbf{A}^{\top}\mathbf{A}$  as  $\mathbf{B} = (\mathbf{A}^{\top}\mathbf{A} + a\mathbf{I}_{n_{\mathrm{odor}}})^{-1/2}$  for a small positive regularizing constant  $a$ . In our simulations, we set  $a = 0.5$ ; we find empirically that our results are not substantially sensitive to small variations in  $a$ . Finally, we let  $\Gamma = \mathbf{BQ} / \max \{|(\mathbf{ABQ})_{ij}|\}$ .
This yields a naively distributed synaptic weight matrix  $\mathbf{A}\Gamma$  that is entirely non-negative, while the geometry-aware weight matrix has a small number of negative elements due to the fact that the inverse of a non-negative matrix need not be non-negative. We see that the behavior of this circuit is
Figure G.3: Fast uncertainty estimation using Langevin sampling of the posterior, showing estimates for individual odorants. Here, we use a simple concentration estimation task in which 5 randomly-selected 'low' odorants out of a panel of 1000 appear at concentration 10 at time 0 s, and then a further 5 randomly-selected 'high' odorants appear at concentration 40 at time 1 s. A. Smoothed timeseries of instantaneous concentration estimates for low, high, and background odorants, for models with one-to-one (top), naively distributed (middle), and geometry-aware (bottom) codes. Background odorant estimates are shown as mean  $\pm$  standard deviation over odorants. Dashed lines show true concentrations over time. B. Cumulative estimates of concentration mean for low (top) and high (bottom) odorants after the onset of the low odorants for one-to-one, naively distributed, and geometry-aware codes. Black lines indicate baseline estimates of the posterior mean. Thick colored lines indicate means over odorants, while thin lines show traces for individual odorants. C. As in B, but for the estimated variance. D. As in B, but after the onset of high odorants. E. As in C, but after the onset of high odorants. See Appendix G for details of our numerical methods.
similar to that observed in Fig. 4. As mentioned in the Discussion, an important topic for future work will be to devise a method to choose  $\Gamma$  that yields a fully non-negative, sparse synaptic weight matrix  $\mathbf{A}\Gamma$ .
H Code and data availability
All code and data required to reproduce the figures presented is available under an MIT License at https://github.com/Pehlevan-Group/olfaction-geometry/.
Figure G.4: Langevin sampling of the posterior using sparse non-negative distributed codes. Here, we use a simple concentration estimation task in which 5 randomly-selected 'low' odors out of a panel of 500 appear at concentration 10 at time 0 s, and then a further 5 randomly-selected 'high' odors appear at concentration 40 at time 1 s. This figure replicates Fig. 4, except that the code is distributed using a sparse non-negative random matrix rather than an orthogonal matrix. Smoothed timeseries of instantaneous concentration estimates for low, high, and background odors, for models with one-to-one (top), naively distributed (middle), and geometry-aware (bottom) codes. Background odor estimates are shown as mean  $\pm$  standard deviation over odors. Dashed lines show true odor concentrations over time. B. Cumulative estimates of odor concentration mean for low (top) and high (bottom) odors after the onset of the low odors for one-to-one, naively distributed, and geometry-aware codes. Black lines indicate baseline estimates of the posterior mean. Thick lines indicate means over odors, and thin lines individual odors. C. As in B, but for the estimated variance. D. As in B, but after the onset of high odors. E. As in C, but after the onset of high odors. See Appendix G for details of our numerical methods.