SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \operatorname {R e} (b) = \mathbf {v} ^ {\dagger} \frac {\operatorname {d i a g} \left(\mathbf {p} ^ {*}\right) \left(\mathbf {A} \boldsymbol {\Gamma} \boldsymbol {\Gamma} ^ {\top} \mathbf {A} ^ {\top}\right) + \left(\mathbf {A} \boldsymbol {\Gamma} \boldsymbol {\Gamma} ^ {\top} \mathbf {A} ^ {\top}\right) \operatorname {d i a g} \left(\mathbf {p} ^ {*}\right)}{2} \mathbf {v} (C.44) \\ = \mathbf {v} ^ {\dagger} \left(\mathbf {A} \boldsymbol {\Gamma} \mathbf {T} ^ {\top} \mathbf {A} ^ {\top}\right) \mathbf {v} + \mathbf {v} ^ {\dagger} \frac {\operatorname {d i a g} (\tilde {\lambda}) \left(\mathbf {A} \boldsymbol {\Gamma} \mathbf {T} ^ {\top} \mathbf {A} ^ {\top}\right) + \left(\mathbf {A} \boldsymbol {\Gamma} \mathbf {T} ^ {\top} \mathbf {A} ^ {\top}\right) \operatorname {d i a g} (\tilde {\lambda})}{2} \mathbf {v}, (C.45) \\ \end{array}
$$
$$
\begin{array}{l} \operatorname {I m} (b) = \mathbf {v} ^ {\dagger} \frac {\operatorname {d i a g} \left(\mathbf {p} ^ {*}\right) \left(\mathbf {A} \boldsymbol {\Gamma} \boldsymbol {\Gamma} ^ {\top} \mathbf {A} ^ {\top}\right) - \left(\mathbf {A} \boldsymbol {\Gamma} \boldsymbol {\Gamma} ^ {\top} \mathbf {A} ^ {\top}\right) \operatorname {d i a g} \left(\mathbf {p} ^ {*}\right)}{2 i} \mathbf {v} (C.46) \\ = \mathbf {v} ^ {\dagger} \frac {\operatorname {d i a g} (\tilde {\lambda}) \left(\mathbf {A} \boldsymbol {\Gamma} \boldsymbol {\Gamma} ^ {\top} \mathbf {A} ^ {\top}\right) - \left(\mathbf {A} \boldsymbol {\Gamma} \boldsymbol {\Gamma} ^ {\top} \mathbf {A} ^ {\top}\right) \operatorname {d i a g} (\tilde {\lambda})}{2 i} \mathbf {v}. (C.47) \\ \end{array}
$$
Then, solving the quadratic equation for  $\mu$ , we have
$$
\mu = \frac {1}{\tau_ {\mathrm {p}}} \frac {- a \pm \sqrt {a ^ {2} - 4 \tilde {\tau} b}}{2}, \tag {C.48}
$$
$$
\tilde {\tau} \equiv \frac {\tau_ {\mathrm {p}}}{\tau_ {\mathrm {g}}}. \tag {C.49}
$$
$$
\begin{array}{l} \operatorname {R e} (\mu) = - \frac {a}{2 \tau_ {\mathrm {p}}} \pm \frac {1}{2 \tau_ {\mathrm {p}}} \operatorname {R e} \sqrt {a ^ {2} - 4 \tilde {\tau} b} (C.50) \\ = - \frac {a}{2 \tau_ {\mathrm {p}}} \pm \frac {1}{2 \tau_ {\mathrm {p}}} \sqrt {\frac {\sqrt {(a ^ {2} - 4 \tilde {\tau} \operatorname {R e} (b)) ^ {2} + 1 6 \tilde {\tau} ^ {2} \operatorname {I m} (b) ^ {2}} + a ^ {2} - 4 \tilde {\tau} \operatorname {R e} (b)}{2}}. (C.51) \\ \end{array}
$$
If  $\operatorname{Im}(b) = 0$ , then we are in the same situation as we were for the two-neuron circuit, and the fixed point is thus stable.
More generally, for the system to be stable we want to have
$$
a > \sqrt {\frac {\sqrt {(a ^ {2} - 4 \tilde {\tau} \operatorname {R e} (b)) ^ {2} + 1 6 \tilde {\tau} ^ {2} \operatorname {I m} (b) ^ {2}} + a ^ {2} - 4 \tilde {\tau} \operatorname {R e} (b)}{2}}. \tag {C.52}
$$
For either sign of  $a^2 - 4\tilde{\tau}\mathrm{Re}(b)$ , one can show that this holds provided that
$$
\operatorname {R e} (b) > 0 \tag {C.53}
$$
$$
\sqrt {\tilde {\tau}} | \operatorname {I m} (b) | <   a \sqrt {\operatorname {R e} (b)}. \tag {C.54}
$$
This amounts to a combined condition on the effective prior strength  $\tilde{\lambda}$  and the relative time constants. Therefore, for sufficiently small  $\tilde{\lambda}$ , we have stability for a broad range of  $\tau_{\mathrm{p}}$ .
How can we go from stability of the  $(\mathbf{q},\mathbf{p})$  circuit to stability of the full circuit? Heuristically, this follows by decomposing the dynamics of  $\mathbf{g}$  in terms of the different subspaces. We will not attempt to do this rigorously, as our goal is only to get a rough sense of how the system should behave. By the fact that their dimensions are strictly ordered, we know that  $\mathrm{span}(\Gamma^{\top})\supset \mathrm{span}[(\mathbf{A}\Gamma)^{\top}]$ . We can first exclude components outside  $\mathrm{span}(\Gamma^{\top})$ , as they will be unchanged by the dynamics and do not affect the concentration estimate. Recalling the non-negativity constraint, components in  $\mathrm{span}(\Gamma^{\top})\backslash \mathrm{span}[(\mathbf{A}\Gamma)^{\top}]$  will decay linearly to zero. Then, the components in  $\mathrm{span}[(\mathbf{A}\Gamma)^{\top}]$  should be controllable using the argument for the  $(\mathbf{q},\mathbf{p})$  space. Giving a fully rigorous analysis of the conditions under which the MAP fixed point is stable will be an interesting avenue for future investigation.
D Derivation for the Gaussian circuit model
To compare the result of the state dependent inhibition due to estimating the MAP for a Poisson noise model with the fixed inhibition resulting from Gaussian noise we build an alternative circuit model. This circuit still shares the separation into two cell types, but the granule cells  $\mathbf{g}$  converge on the solution for a Gaussian noise model.
Our starting point is an isotropic Gaussian likelihood
$$
\mathbf {s} \mid \mathbf {c} \sim \mathcal {N} \left(\mathbf {r} _ {0} + \mathbf {A c}, \sigma^ {2} \mathbf {I} _ {n _ {\mathrm {O S N}}}\right) \tag {D.1}
$$
variance  $\sigma^2$ , and an exponential prior
$$
\mathbf {c} \sim \operatorname {E x p} (\lambda). \tag {D.2}
$$
Gradient ascent on the resulting log-posterior over  $\mathbf{c}$  leads to the dynamics:
$$
\dot {\mathbf {c}} (t) = \frac {1}{\sigma^ {2}} \mathbf {A} ^ {\top} [ \mathbf {s} - (\mathbf {r} _ {0} + \mathbf {A c}) ] - \boldsymbol {\lambda}, \tag {D.3}
$$
Distributing the code such that  $\mathbf{c}(t) = \mathbf{\Gamma}\mathbf{g}$  and splitting the inference as we did in the Poisson case gives the circuit dynamics
$$
\begin{array}{l} \mathbf {c} (t) = \boldsymbol {\Gamma} \mathbf {g} (t) \\ \tau_ {\mathrm {g}} \dot {\mathbf {g}} (t) = \frac {1}{\sigma^ {2}} (\mathbf {A} \boldsymbol {\Gamma}) ^ {\top} \mathbf {p} - \boldsymbol {\Gamma} ^ {\top} \boldsymbol {\lambda} \tag {D.4} \\ \end{array}
$$
$$
\tau_ {\mathrm {p}} \dot {\mathbf {p}} (t) = - \mathbf {p} + \mathbf {s} - (\mathbf {r} _ {0} + \mathbf {A} \boldsymbol {\Gamma} \mathbf {g}).
$$
Unlike in the Poisson case, the inhibition onto the projection cells (mitral cells) is not gated by their activity, leading to the fixed offset shown in Fig. 2B.
E Extending the sampling circuit to incorporate an  $L_{0}$  prior
In this Appendix, we consider the possibility of extending our circuit model to incorporate an  $L_{0}$  spike-and-slab prior
$$
p \left(c _ {i}\right) = \varpi e ^ {- \lambda c _ {i}} + (1 - \varpi) \delta \left(c _ {i}\right), \tag {E.1}
$$
where  $\varpi$  is the probability that the odor is present and  $\lambda$  is the rate of the exponential prior on concentrations given that the odor is present. To sample from the resulting posterior using Langevin dynamics, we will follow the approach of Fang et al. [115], who developed a Langevin algorithm to perform sparse coding with this prior given a Gaussian likelihood. This approach only works in the sampling setting; it cannot be applied to MAP estimation because the Dirac mass at  $c_{i} = 0$  means that the MAP estimate will always vanish.
In this approach, we define an auxiliary variable  $\mathbf{u}$  that is mapped to concentration estimates  $\mathbf{c}$  via element-wise soft thresholding:
$$
\mathbf {c} = f (\mathbf {u}), \tag {E.2}
$$
$$
f (u) = \left\{ \begin{array}{l l} 0 & u <   u _ {0} \\ u - u _ {0} & u \geq u _ {0} \end{array} \right. \tag {E.3}
$$
is the soft-thresholding function for threshold
$$
u _ {0} = - \frac {1}{\lambda} \log \varpi . \tag {E.4}
$$
We then posit the following Langevin dynamics for  $\mathbf{u}$ , given an observation  $\mathbf{s}$ :
$$
\dot {\mathbf {u}} (t) = \left[ \boldsymbol {\nabla} _ {\mathbf {c}} \log p (\mathbf {s} \mid \mathbf {c}) \right] _ {\mathbf {c} = f (| \mathbf {u} |)} \odot \Theta (| \mathbf {u} | - \mathbf {u} _ {0}) - \lambda \operatorname {s i g n} (\mathbf {u}) + \boldsymbol {\eta} (t) \tag {E.5}
$$
with no constraint on the sign of  $\mathbf{u}$ , where  $\pmb{\eta}(t)$  is a white Gaussian noise process. Here, all nonlinearities are applied element-wise, and  $\mathbf{u}_0 = u_0\mathbf{1}$  is a vector with all elements equal to the threshold  $u_0$ . Fang et al. [115] argue that the stationary distribution induced on  $\mathbf{c} = f(\mathbf{u})$  by these dynamics should be the desired posterior with  $L_0$  prior.
Using the Poisson likelihood gradient as computed before, we have
$$
\dot {\mathbf {u}} (t) = \left\{\mathbf {A} ^ {\top} [ \mathbf {s} \odot (\mathbf {r} _ {0} + \mathbf {A} f (| \mathbf {u} |)) - \mathbf {1} ] \right\} \odot \Theta (| \mathbf {u} | - \mathbf {u} _ {0}) - \lambda \operatorname {s i g n} (\mathbf {u}) + \boldsymbol {\eta} (t) \tag {E.6}
$$
At this stage, we can see that the soft thresholding has picked out a preferred basis. If we apply the complete recipe in a way analogous to what we did in Appendix B by writing
$$
\mathbf {u} = \boldsymbol {\Gamma} \mathbf {g}, \tag {E.7}
$$
we will have
$$
\tau_ {\mathrm {g}} \dot {\mathbf {g}} (t) = \boldsymbol {\Gamma} ^ {\top} \left[ \left\{\mathbf {A} ^ {\top} \left[ \mathbf {s} \oslash \left(\mathbf {r} _ {0} + \mathbf {A} f (| \boldsymbol {\Gamma} \mathbf {g} |)\right) - \mathbf {1} \right] \right\} \odot \Theta (| \boldsymbol {\Gamma} \mathbf {g} | - \mathbf {u} _ {0}) - \lambda \operatorname {s i g n} (\boldsymbol {\Gamma} \mathbf {g}) \right] + \boldsymbol {\xi} (t), \tag {E.8}
$$
where  $\pmb{\xi}(t)$  is a zero-mean Gaussian noise process with covariance
$$
\mathbb {E} \left[ \xi_ {j} (t) \xi_ {j ^ {\prime}} \left(t ^ {\prime}\right) \right] = 2 \tau_ {\mathrm {g}} \delta_ {j j ^ {\prime}} \delta \left(t - t ^ {\prime}\right). \tag {E.9}
$$
Here, we cannot simply regroup terms; if we introduce an additional cell type  $\mathbf{p}$  as before to compute the division we will have a sort of effective weight matrix
$$
\sum_ {k} \Gamma_ {k i} A _ {j k} \Theta \left(\left| u _ {k} \right| - u _ {0}\right) \tag {E.10}
$$
for the  $\mathbf{p}$ -to- $\mathbf{g}$  connections, and an entirely different coupling  $\mathbf{A}f(|\Gamma \mathbf{g}|)$  for the  $\mathbf{g}$ -to- $\mathbf{p}$  connections. Concretely, introducing  $\mathbf{p}$  as before, we have the system
$$
\mathbf {c} (t) = f \left(\left| \boldsymbol {\Gamma} \mathbf {g} (t) \right|\right) \tag {E.11}
$$
$$
\tau_ {\mathrm {g}} \dot {\mathbf {g}} (t) = \boldsymbol {\Gamma} ^ {\top} \left[ \left[ \mathbf {A} ^ {\top} (\mathbf {p} - \mathbf {1}) \right] \odot \Theta \left(\left| \boldsymbol {\Gamma} \mathbf {g} \right| - \mathbf {u} _ {0}\right) - \lambda \operatorname {s i g n} (\boldsymbol {\Gamma} \mathbf {g}) \right] d t + \boldsymbol {\xi} (t) \tag {E.12}
$$
$$
\tau_ {\mathrm {p}} \dot {\mathbf {p}} (t) = \mathbf {s} - \mathbf {p} \odot [ \mathbf {r} _ {0} + \mathbf {A} f (| \boldsymbol {\Gamma} \mathbf {g} |) ]. \tag {E.13}
$$
This issue illustrates the limitations of the simple, linear approach to distributing the neural code used in Appendix B. In particular, the nonlinearity in some sense picks out a preferred basis, meaning that we can no longer perform a simple linear change of coordinates to distribute the code.
F Experimental details and affinity matrix fitting
Here we briefly describe the experimental procedures used to collect the data used to fit the parameters of the affinity matrix  $\mathbf{A}$  in Fig. 1. The full experimental methods are described in the paper that first presented the data, Zak et al. [94]. All the experiments were performed in accordance with the guidelines set by the National Institutes of Health and approved by the Institutional Animal Care and Use Committee at Harvard University.
F.1 In vivo recordings from mouse OB
Adult (> 8 weeks) OMP-GCaMP3 mice of both sexes were used in this study. A craniotomy was performed to provide optical access to olfactory sensory neuron axon terminals in both olfactory bulbs. A custom-built two-photon microscope was used for in vivo imaging. Images were acquired at 16-bit resolution and 4-8 frames/s. The pixel size was  $0.6\mu m$  and the fields of view were  $720\times 720\mu m$ . Monomolecular odorants (Allyl butyrate, Ethyl valerate, Methyl tiglate, and Isobutyl propionate) were used as stimuli and delivered by a 16-channel olfactometer controlled by custom-written software in LabView. For the odorant concentration series, the initial odorant concentration was between  $0.08\% -80\%(v / v)$  in mineral oil and further diluted 16 times with air. The relative odorant concentrations were measured by a photoionization detector, then normalized to the largest detected signal for each odorant. For all experiments, the airflow to the animal was held constant at  $100mL / min$ , and odorants were injected into a carrier stream. Each odorant concentration was delivered 2-6 times in pseudorandom order.
Images were processed using both custom and available MATLAB scripts. Motion artifact compensation and denoising were done using NoRMcorre [138]. The  $\Delta F / F$  signal was calculated by finding the peak signal following odorant onset and averaging with the two adjacent points. To account for changes in respiration and anesthesia depth, correlated variability was corrected for [139]. Thresholds for classifying responding ROIs were determined from a noise distribution of blank (no odorant) trials from which three standard deviations were used for responses.
F.2 Gamma distribution fitting
In order to fit the affinity matrix  $\mathbf{A}$  to the experimentally recorded data, we normalized the response of each glomerulus to its maximum response across the panel of 32 odorants. We then vectorized the resulting matrix and fitted a Gamma distribution using the gamfit function in MATLAB. As mentioned in the main text, this resulted in a Gamma(0.37, 0.36) distribution.
G Numerical methods and supplemental figures
All numerical simulations were performed using MATLAB 9.13 (R2022b, The MathWorks, Natick, MA, USA) either on desktop workstations (CPU: Intel i9-9900K or Xeon W-2145, 64GB RAM) or on the Harvard University FASRC Cannon HPC cluster (https://www.rc.fas.harvard.edu). Our simulations were not computationally intensive, and required around that 6000 CPU-hours of total compute time.
G.1 State-dependent inhibition simulations
For the simulations to highlight the signatures of state-dependent inhibition of the Poisson network and compare its behavior with the experimental observation by Arevian et al. [95] we use a reduced network with  $n_{OSN} = 2$  and  $n_g = 10$ . We chose this reduced circuit both to reduce computational costs and to match the experimental parameters of the in-vitro experiment. The other parameters of the simulation where as follows:  $\tau_p = 0.02$ ,  $\tau_g = 0.03$ ,  $r_0 = 10$ ,  $\lambda = 2$ ,  $dt = 1e^{-4}$ . The simulation ran for 600 ms, stimulation was applied starting at  $t_{start} = 100$  ms and ended at  $t_{end} = 500$  ms. We stimulated the principal mitral cell  $MCA$  with 80 equally spaced values in  $s_A \in [1,400]$ . When the second mitral cell was active, its stimulation was set at  $s_B = 80$ . The entries of the  $\Gamma$  matrix were sampled from a normal distribution on which we applied a mask such that only 25% of its entries were non-zero. We sampled 32 'pairs' of cells by resampling the  $\Gamma$  matrix for each 'pair' of cells. The simulation for the Gaussian circuit used the same parameters except the dynamics followed those
from equation D.4. For plotting purposes, we normalized the range of firing in Figure 2B to the maximum firing rate for each circuit model.
G.2 Capacity simulations
In our capacity simulations, we use  $n_{\mathrm{OSN}} = 300$  and  $n_{\mathrm{odor}} = 500$ , 1000, or 2000. We take the odor stimulus to be a rectangular pulse, with varying numbers of randomly-selected odors appearing at concentration  $c_{j} = 40$ . As elsewhere, we take  $\tau_{p} = 20$  ms [91] and  $\tau_{g} = 30$  ms [92] to match experiment. We set the baseline rate to be  $r_0 = 1$  and the prior mean to be  $\lambda = 1$ ; based on some experimentation our results appear relatively insensitive to small variations in these choices. We integrate the MAP circuit dynamics (3) using the forward Euler method with timestep  $\Delta t = 10^{-4}$  s. To determine which odors the model estimates as being present at a given timepoint, we simply check which concentration estimates exceed 20 at that time.
As discussed in the main text, we consider three variants of the MAP circuit, defined by different choices of the matrix  $\Gamma$ :
- For the one-to-one code, we let  $n_{\mathrm{g}} = n_{\mathrm{odor}}$ , and simply set  $\Gamma = \mathbf{I}_{n_{\mathrm{odor}}} / N(A_{ij})$ .
- For the naively distributed code, we let  $n_{\mathrm{g}} = 5n_{\mathrm{odor}}$ , and choose  $\Gamma$  as follows: We sample a random matrix  $\mathbf{Q} \in \mathbb{R}^{n_{\mathrm{odor}} \times n_{\mathrm{g}}}$  with  $\mathbf{QQ}^{\top} = \mathbf{I}_{n_{\mathrm{odor}}}$  by drawing a Gaussian matrix and orthogonalizing its rows. Then, we define  $\Gamma = \mathbf{Q} / N\{|(\mathbf{AQ})_{ij}|\}$ .
- For the geometry-aware code, we let  $n_{\mathrm{g}} = 5n_{\mathrm{odor}}$ , and choose  $\Gamma$  as follows: We sample a random matrix  $\mathbf{Q} \in \mathbb{R}^{n_{\mathrm{odor}} \times n_{\mathrm{g}}}$  with  $\mathbf{QQ}^{\top} = \mathbf{I}_{n_{\mathrm{odor}}}$  by drawing a Gaussian matrix and orthogonalizing its rows. Then, we compute an approximate inverse square root of the low-rank matrix  $\mathbf{A}^{\top}\mathbf{A}$  as  $\mathbf{B} = (\mathbf{A}^{\top}\mathbf{A} + a\mathbf{I}_{n_{\mathrm{odor}}})^{-1/2}$  for a small positive regularizing constant  $a$ . In our simulations, we set  $a = 0.5$ ; we find empirically that our results are not substantially sensitive to small variations in  $a$ . Finally, we let  $\Gamma = \mathbf{BQ}/N\{|(\mathbf{ABQ})_{ij}|\}$ .
where  $N$  is a normalization function defined as
$$
N (A) = \max (A) \cdot \frac {\sqrt {n _ {g}}}{C}
$$
with  $C = 50$  identified as a reasonable choice. Our normalization convention for  $\Gamma$  is motivated by the idea that, in biology, the strength of individual synapses should be bounded. Moreover, changing the overall scale of the synaptic weights in our model corresponds to changing the effective time constant of the dynamics, which can produce a trivial speedup.
after 100 ms
A
$n_{\mathrm{odor}} = 500$
B
after 200 ms
after 1 s
$n_{\mathrm{odor}} = 1000$
C
$n_{\mathrm{order}} = 2000$
D
$n_{\mathrm{odor}} = 4000$
E
$n_{\mathrm{odor}} = 8000$
Figure G.1: A. Fraction of odors correctly detected within 100 ms (left), 200 ms (center), and 1 s (right) after odor onset as a function of the number of odors present, with  $n_{\mathrm{odor}} = 500$  possible odors, for models with one-to-one, naively distributed, and geometry-aware codes. B. As in A, but for  $n_{\mathrm{odor}} = 1000$ . This matches Fig. 3B. C, D, E. As in A, but for  $n_{\mathrm{odor}} = 2000$ , 4000, or 8000 possible odorants, respectively. Shaded patches show ±1.96 SEM over realizations throughout.
naively distributed
geometry-aware
Figure G.2: A. Heatmap with overlaid smoothed contours of the fraction of odors correctly detected as a function of number of present odors and time window out of a panel of  $n_{\mathrm{odor}} = 500$  possible odors for models with one-to-one (left), naively distributed (center), and geometry-aware (right) codes. B. As in A, but for  $n_{\mathrm{odor}} = 1000$ . This matches Fig. 3B. C, D, E. As in A, but for  $n_{\mathrm{odor}} = 2000$ , 4000, or 8000 possible odorants, respectively.
G.3 Sampling simulations
In our sampling simulations, we use  $n_{\mathrm{OSN}} = 300$ ,  $n_{\mathrm{odor}} = 500$  or 1000, and  $n_{\mathrm{g}} = 5n_{\mathrm{odor}}$ . Here, the odor stimulus was composed of two rectangular pulses: At time  $t_{\mathrm{low}} = 0$  s, five "low" odors appear at concentration 10, and remain present at that concentration until time  $t = 2$  s. Then, at time  $t_{\mathrm{high}} = 1$  s, five "high" odors appear at concentration 40, and remain present until time  $t_{\mathrm{off}} = 2$  s. As in our capacity simulations, we take  $\tau_{p} = 20$  ms [91] and  $\tau_{g} = 30$  ms [92] to match experiment, and we set the baseline rate to be  $r_0 = 1$  and the prior mean to be  $\lambda = 1$ . As for our capacity results, our results appear relatively insensitive to small variations in these choices. We integrate the sampling circuit dynamics (4) using the forward Euler-Maruyama method with timestep  $\Delta t = 10^{-5}$  s.
As in our capacity simulations, in Fig. 4 we choose  $\Gamma$  as follows:
- For the one-to-one code, we let  $n_{\mathrm{g}} = n_{\mathrm{odor}}$ , and simply set  $\Gamma = \mathbf{I}_{n_{\mathrm{odor}}} / \max (A_{ij})$ .
- For the naively distributed code, we let  $n_{\mathrm{g}} = 5n_{\mathrm{odor}}$ , and choose  $\Gamma$  as follows: We sample a random matrix  $\mathbf{Q} \in \mathbb{R}^{n_{\mathrm{odor}} \times n_{\mathrm{g}}}$  with  $\mathbf{QQ}^{\top} = \mathbf{I}_{n_{\mathrm{odor}}}$  by drawing a Gaussian matrix and orthogonalizing its rows. Then, we define  $\Gamma = \mathbf{Q} / \max \{|(\mathbf{AQ})_{ij}|\}$ .
- For the geometry-aware code, we let  $n_{\mathrm{g}} = 5n_{\mathrm{odor}}$ , and choose  $\Gamma$  as follows: We sample a random matrix  $\mathbf{Q} \in \mathbb{R}^{n_{\mathrm{odor}} \times n_{\mathrm{g}}}$  with  $\mathbf{QQ}^{\top} = \mathbf{I}_{n_{\mathrm{odor}}}$  by drawing a Gaussian matrix and orthogonalizing its rows. Then, we compute an approximate inverse square root of the low-rank matrix  $\mathbf{A}^{\top}\mathbf{A}$  as  $\mathbf{B} = (\mathbf{A}^{\top}\mathbf{A} + a\mathbf{I}_{n_{\mathrm{odor}}})^{-1/2}$  for a small positive regularizing constant  $a$ . In our simulations, we set  $a = 0.5$ ; we find empirically that our results are not substantially sensitive to small variations in  $a$ . Finally, we let  $\Gamma = \mathbf{BQ} / \max \{|(\mathbf{ABQ})_{ij}|\}$ .
In Fig. 4A, we smooth the concentration estimate timeseries using a  $100\mathrm{ms}$  moving average. In Fig. 4B-C, we show cumulative estimates of the mean and variance. Concretely, given a concentration timeseries  $\hat{c}_j(t)$ , we estimate the mean and variance as
$$
\mu_ {j} (\tau) = \frac {\Delta t}{\tau} \sum_ {t _ {\text {l o w}} \leq t \leq t _ {\text {l o w}} + \tau} \hat {c} _ {j} (t) \tag {G.1}
$$
$$
\sigma_ {j} ^ {2} (\tau) = \frac {\Delta t}{\tau} \sum_ {t _ {\text {l o w}} \leq t \leq t _ {\text {l o w}} + \tau} \hat {c} _ {j} (t) ^ {2} - \mu_ {j} (\tau) ^ {2}, \tag {G.2}
$$
respectively, where we assume  $\tau$  and  $t$  are integer multiples of  $\Delta t$ . In Fig. 4D-E, we do the same except for times after  $t_{\mathrm{high}}$ . In both cases, baselines were obtained by running the naive sampling algorithm for  $10^{8}$  steps with a burn-in period of  $10^{7}$  steps. In Supp. Fig. G.3, we reproduce Fig. 4 showing estimates for individual odorants as well as the mean across odorants.
In Supp. Fig. G.4, we show a preliminary experiment with an alternative, mostly non-negative choice for  $\Gamma$ . This gives the following three models: