SlowGuess/NeurIPS_2023_OCR huggingface dataset:
11.1 HH Model Definition
For the inference experiments (Section 5.3.1) we used a probabilistic version of the squid giant axon model [9, 43]. Our experimental setup was constructed to broadly match [19], and used a single-compartment model with dynamics defined by
$$
C _ {m} \frac {d v}{d t} = I _ {\text {e x t}} - \bar {g} _ {\mathrm {N a}} m ^ {3} h \left(v - E _ {\mathrm {N a}}\right) - \bar {g} _ {\mathrm {K}} n ^ {4} \left(v - E _ {\mathrm {K}}\right) - g _ {\text {l e a k}} \left(v - E _ {\text {l e a k}}\right) \tag {42}
$$
$$
\frac {d m}{d t} = \alpha_ {m} (v) (1 - m) - \beta_ {m} (v) m \tag {43}
$$
$$
\frac {d h}{d t} = \alpha_ {h} (v) (1 - h) - \beta_ {h} (v) h \tag {44}
$$
$$
\frac {d n}{d t} = \alpha_ {n} (v) (1 - n) - \beta_ {n} (v) n \tag {45}
$$
where  $C_m$  is the membrane capacitance;  $v$  is the potential difference across the membrane;  $I_{\mathrm{ext}}$  is the external current;  $\bar{g}_{\mathrm{Na}}$ ,  $\bar{g}_{\mathrm{K}}$ , and  $\bar{g}_{\mathrm{leak}}$  are the maximum conductances for sodium, potassium, and leak channels;  $E_{\mathrm{Na}}$ ,  $E_{\mathrm{K}}$ , and  $E_{\mathrm{leak}}$  are the reversal potentials for the sodium, potassium, and leak channels;  $m$  and  $h$  are subunit states for the sodium channels and  $n$  is the subunit state for the potassium channels. The functions  $\alpha$  and  $\beta$  that define the dynamics for  $n$ ,  $m$ , and  $h$  are defined as
$$
\alpha_ {m} (v) = \frac {- 4 - v / 1 0}{\exp (- 4 - v / 1 0) - 1}, \quad \beta_ {m} (v) = 4 \cdot \exp ((- 6 5 - v) / 1 8) \tag {46}
$$
$$
\alpha_ {h} (v) = 0. 0 7 \cdot \exp \left((- 6 5 - v) / 2 0\right), \quad \beta_ {h} (v) = \frac {1}{\exp \left(- 3 . 5 - v / 1 0\right) + 1} \tag {47}
$$
$$
\alpha_ {n} (v) = \frac {- 5 . 5 - v / 1 0}{\exp (- 5 . 5 - v / 1 0) - 1}, \quad \beta_ {n} (v) = 0. 1 2 5 \cdot \exp ((- 6 5 - v) / 8 0) \tag {48}
$$
This system of ordinary differential equations defines a nonlinear dynamical system with a four-dimensional state space: the instantaneous membrane potential  $v$  and the ion gate subunit states  $n$ ,  $m$ , and  $h$ .
As in [19], we use a probabilistic version of the original HH model that adds zero-mean Gaussian noise to both the membrane voltage  $v$  and the "unconstrained" subunit states. The observations are produced by adding Gaussian noise with variance  $\sigma_y^2$  to the membrane potential  $v$ .
Specifically, let  $\mathbf{x}_t$  be the state vector of the system at time  $t$  containing  $(v_t, m_t, h_t, n_t)$ , and let  $\varphi_{dt}(\mathbf{x})$  be a function that integrates the system of ODEs defined above for a step of length  $dt$ . Then the probabilistic HH model can be written as
$$
p \left(\mathbf {x} _ {1: T}, \mathbf {y} _ {1: T}\right) = p \left(\mathbf {x} _ {1}\right) \prod_ {t = 2} ^ {T} p \left(\mathbf {x} _ {t} \mid \varphi_ {d t} \left(\mathbf {x} _ {t - 1}\right)\right) \prod_ {t = 1} ^ {T} \mathcal {N} \left(\mathbf {y} _ {t}; \mathbf {x} _ {t, 1}, \sigma_ {y} ^ {2}\right) \tag {49}
$$
where the 4-D state distributions  $p(\mathbf{x}_1)$  and  $p(\mathbf{x}_t\mid \varphi_{dt}(\mathbf{x}_{t - 1}))$  are defined as
$$
p \left(\mathbf {x} _ {t} \mid \varphi_ {d t} \left(\mathbf {x} _ {t - 1}\right)\right) = \mathcal {N} \left(\mathbf {x} _ {t, 1}; \varphi_ {d t} \left(\mathbf {x} _ {t - 1}\right) _ {1}, \sigma_ {x, 1} ^ {2}\right) \prod_ {i = 2} ^ {4} \operatorname {L o g i t N o r m a l} \left(\mathbf {x} _ {t, i}; \varphi_ {d t} \left(\mathbf {x} _ {t - 1}\right) _ {i}, \sigma_ {x, i} ^ {2}\right). \tag {50}
$$
Figure 7: HH inference performance across different numbers of particles. (left) Log-likelihood lower bounds for proposals trained with 4 particles and evaluated across a range of particle numbers. NAS-X's inference performance decays only minimally as the number of particles is decreased, while all other methods experience significant performance degradation. (right) A comparison of SIXO and NAS-X containing the same values as the left panel, but zoomed in. NAS-X is roughly twice as particle efficient as SIXO, and outperforms SIXO by roughly 34 nats at 4 particles.
In words, we add Gaussian noise to the voltage  $(\mathbf{x}_{t,1})$  and logit-normal noise to the gate states  $n,m,$  and  $h$ . The logit-normal is defined as the distribution of a random variable whose logit has a Gaussian distribution, or equivalently it is a Gaussian transformed by the sigmoid function and renormalized. We chose the logit-normal because its values are bounded between 0 and 1, which is necessary for the gate states.
Problem Setting For the inference experiments we sampled 10,000 noisy voltage traces from a fixed model and used each method to train proposals (and possibly twists) to compute the marginal likelihood assigned to the data under the true model.
As in [19], we sampled trajectories of length 50 milliseconds, with a single noisy voltage observation every millisecond. The stability of our ODE integrator allowed us to integrate at  $dt = 0.1\mathrm{ms}$ , meaning that there were 10 latent states per observation.
Proposal and Twist Details Each proposal was parameterized using the combination of a bidirectional recurrent neural network (RNN) that conditioned on all observed noisy voltages as well as a dense network that conditioned on the RNN hidden state and the previous latent state  $\mathbf{x}_{t-1}$  [45, 49]. The twists for SIXO and NAS-X were parameterized using an RNN run in reverse over the observations combined with a dense network that conditioned on the reverse RNN hidden state and the latent being 'twisted',  $\mathbf{x}_t$ . Both the proposal and twists were learned in an amortized manner, i.e. they were shared across all trajectories. All RNNs had a single hidden layer of size 64, as did the dense networks. All models were fit with ADAM [50] with proposal learning rate of  $10^{-4}$  and twist learning rate of  $10^{-3}$ .
A crucial aspect of fitting the proposals was defining them in terms of a 'residual' from the prior, a technique known as  $\mathrm{Res}_q$  [7]. In our setting, we defined the true proposal density as proportional to the product of a unit-variance Gaussian centered at  $\varphi (\mathbf{x}_t)$  and a Gaussian with parameters output from the RNN proposal.
11.2 Experimental Results
In Figure 7 we plot the performance of proposals and twists trained with 4 particles and evaluated across a range of particle numbers. All methods except FIVO perform roughly the same when evaluated with 256 particles, but with lower numbers of evaluation particles the smoothing methods emerge as more particle-efficient than the filtering methods. To achieve NAS-X's inference perfor
Figure 8: Training HH proposals with increasing numbers of particles. HH proposal performance plots similar to Figure 7, but trained with varying numbers of particles. Increasing the number of particles at training time has a negligible effect on NAS-X performance in this setting, but caused many VI-based methods to perform worse. This could be due to signal-to-noise issues in proposal gradients, as discussed in Rainforth et al. [51].
Figure 9: Inferred voltage traces for NASMC, SIXO, and NAS-X. (top) NASMC exhibits poor performance, incorrectly inferring the timing of most spikes. (middle) SIXO's inferred voltage traces are more accurate than NASMC's with only a single mistimed spike, but SIXO generates a high number of resampling events leading to particle degeneracy. (bottom) NAS-X perfectly infers the latent voltage with no mistimed spikes, and resamples very infrequently.
mance with 4 particles, NASMC would need 256 particles, a 64-times increase. NAS-X is also more particle-efficient than SIXO, achieving on average a  $2\mathrm{x}$  particle efficiency improvement. We show the effect of changing the number of training particles in Figure 8.
The FIVO method with a parametric proposal drastically underperformed all smoothing methods as well as NASMC, indicating that the combination of filtering SMC and the exclusive KL divergence leads to problems optimizing the proposal parameters. To compensate, we also evaluated the performance of "FIVO-BS", a filtering method that uses a bootstrap proposal. This method is identical to a bootstrap particle filter, i.e. it proposes from the model and has no trainable parameters. FIVO-BS far outperforms standard FIVO, and is only marginally worse than NASMC in this setting.
In Figure 9 we investigate these results qualitatively by examining the inferred voltage traces of each method. We see that NASMC struggles to produce accurate spike timings and generates many spurious spikes, likely because it is unable to incorporate future information into its proposal or resampling method. SIXO performs better than NASMC, accurately inferring the timing of most spikes but resampling at a high rate. High numbers of resampling events can lead to particle degeneracy and poor inferences. NAS-X is able to correctly infer the voltage across the whole trace with no suprious or mistimed spikes. Furthermore NAS-X rarely resamples, indicating it has learned a high-quality proposal that does not generate low-quality particles that must be resampled away. These qualitative results seem to support the quantitative results in Figure 7 — SIXO's high resampling rate and NASMC's filtering approach lead to lower bound values.
Table 1: Train Bound comparison
Metric	NAS-X	SIXO
L256BPF	-660.7003	-636.2579
L4train	-664.3528	-668.6865
L8train	-662.8712	-653.6352
L16train	-662.0753	-644.8764
L32train	-661.5387	-639.5388
L64train	-660.8040	-636.5131
L128train	-660.5102	-633.7875
L256train	-660.3423	-632.1377
12 Model Learning in Mouse Pyramidal Neuron Model
12.1 Model Definition
For the model learning experiments in Section 5.3.2 we used a generalization of the Hodgkin-Huxley model developed for modeling mouse visual cortex neurons by the Allen Institute for Brain Science [48, 52]. Specifically we used the perisomatic model with ID 482657528 developed to model cell 480169178. The model is detailed in the whitepaper [52] and the accompanying code, but we reproduce the details here to ensure our work is self-contained.
Similar to the squid giant axon model, the mouse visual cortex model is composed of ion channels that affect the current flowing in and out of the cell. Let  $\mathcal{I}$  be the set of ions  $\{\mathrm{Na}^{+},\mathrm{Ca}^{2 + },\mathrm{K}^{+}\}$ . Each ion has associated with it
1. A set of channels that transport that ion, denoted  $C_i$  for  $i \in \mathcal{I}$ .
2. A reversal potential,  $E_{i}$ .
3. An instantaneous current density,  $I_{i}$ , which is computed by summing the current density flowing through each channel that transports that ion.
Correspondingly, let  $\mathcal{C}$  be the set of all ion channels so that  $\mathcal{C} = \bigcup_{i\in \mathcal{I}}C_i$ . Each  $c\in \mathcal{C}$  has associated with it
1. A maximum conductance density,  $\overline{g}_c$
2. A set of subunit states, referred to collectively as the vector  $\lambda_{c}$ . Let  $\lambda_{c} \in [0,1]^{d_{c}}$ , i.e.  $\lambda_{c}$  is a  $d_{c}$ -dimensional vector of values in the interval  $[0,1]$ .
3. A function  $g_{c}$  that combines the gate values to produce a number in [0, 1] that weights the maximum conductance density,  $\overline{g}_{c} \cdot g_{c}(\lambda_{c})$ .
4. Functions  $A_{c}(\cdot)$  and  $b_{c}(\cdot)$  which compute the matrix and vector used in the ODE describing  $\lambda_{c}$  dynamics.  $A_{c}$  and  $b_{c}$  are functions of both the current membrane voltage  $v$  and calcium concentration inside the cell  $[\mathrm{Ca}^{2+}]_{i}$ . If the number of subunits (i.e. the dimensionality of  $\lambda_{c}$ ) is  $d_{c}$ , then the output of  $A_{c}(v, [\mathrm{Ca}^{2+}]_{i})$  is a  $d_{c} \times d_{c}$  diagonal matrix and the output of  $b_{c}(v, [\mathrm{Ca}^{2+}]_{i})$  is a  $d_{c}$ -dimensional vector.
With this notation we can write the system of ODEs
$$
C _ {m} \frac {d v}{d t} = \frac {I _ {\text {e x t}}}{S A} - g _ {\text {l e a k}} (v - E _ {\text {l e a k}}) - \sum_ {i \in \text {i o n s}} I _ {\mathrm {i}} \tag {51}
$$
$$
I _ {\mathrm {i}} = \sum_ {c \in \mathrm {C} _ {\mathrm {i}}} \bar {g} _ {c} g _ {c} \left(\lambda_ {c}\right) (v - E _ {\mathrm {i}}) \tag {52}
$$
$$
\frac {d \lambda_ {c}}{d t} = A _ {c} (v, [ \mathrm {C a} ^ {2 +} ] _ {i}) \lambda_ {c} + b _ {c} (v, [ \mathrm {C a} ^ {2 +} ] _ {i}) \quad \forall c \in \mathcal {C} \tag {53}
$$
$$
\frac {d \left[ \mathrm {C a} ^ {2 +} \right] _ {i}}{d t} = - k I _ {\mathrm {C a} ^ {2 +}} - \frac {\left[ \mathrm {C a} ^ {2 +} \right] _ {i} - \left[ \mathrm {C a} ^ {2 +} \right] _ {\min }}{\tau}. \tag {54}
$$
Most symbols are as described earlier,  $SA$  is the membrane surface area of the neuron,  $\left[\mathrm{Ca}^{2+}\right]_i$  is the calcium concentration inside the cell,  $\left[\mathrm{Ca}^{2+}\right]_{\mathrm{min}}$  is the minimum interior calcium concentration with a value of 1 nanomolar,  $\tau$  is the rate of removal of calcium with a value of 80 milliseconds, and  $k$  and  $l$  is a constant with value
$$
k = 1 0 0 0 0 \cdot \frac {\gamma}{2 \cdot F \cdot \text {d e p t h}} \tag {55}
$$
where 10000 is a dimensional constant,  $\gamma$  is the percent of unbuffered free calcium,  $F$  is Faraday's constant, and depth is the depth of the calcium buffer with a value of 0.1 microns.
Because the concentration of calcium changes over time, this model calculates the reversal potential for calcium  $E_{\mathrm{Ca}^{2+}}$  using the Nernst equation
$$
E _ {\mathrm {C a} ^ {2 +}} = \frac {G \cdot T}{2 \cdot F} \log \left(\frac {[ \mathrm {C a} ^ {2 +} ] _ {o}}{[ \mathrm {C a} ^ {2 +} ] _ {i}}\right) \tag {56}
$$
where  $G$  is the gas constant,  $T$  is the temperature in Kelvin (308.15°),  $F$  is Faraday's constant, and  $[\mathrm{Ca}^{2+}]_o$  is the extracellular calcium ion concentration which was set to 2 millimolar.
Probabilistic Model The probabilistic version of the deterministic ODEs was constructed similarly to the probabilistic squid giant axon model — Gaussian noise was added to the voltage and unconstrained gate states. One difference is that the system state now includes  $\left[\mathrm{Ca}^{2+}\right]_i$  which is constrained to be greater than 0. To noise  $\left[\mathrm{Ca}^{2+}\right]_i$  we added Gaussian noise in the log space, analogous to the logit-space noise for the gate states.
Model Size The 38 learnable parameters of the model include:
1. Conductances  $\overline{g}$  for all ion channels (10 parameters).
2. Reversal potentials of sodium, potassium, and the non-specific cation:  $E_{\mathrm{K}^{+}}$ ,  $E_{\mathrm{Na}^{+}}$ , and  $E_{\mathrm{NSC}^{+}}$ .
3. The membrane surface area and specific capacitance.
4. Leak channel reversal potential and max conductance density.
5. The calcium decay rate and free calcium percent.
6. Gaussian noise variances for the voltage  $v$  and interior calcium concentration  $[\mathrm{Ca}^{2+}]_i$ .
7. Gaussian noise variances for all subunit states (16 parameters).
8. Observation noise variance.
The 18-dimensional state includes:
1. Voltage  $v$
2. Interior calcium concentration  $[\mathrm{Ca}^{2+}]_i$
3. All subunit states (16 dimensions)
12.2 Channel Definitions
In this section we provide a list of all ion channels used in the model. In the following equations we often use the function exprel which is defined as
$$
\operatorname {e x p r e l} (x) = \left\{ \begin{array}{l l} 1 & \text {i f} \quad x = 0 \\ \frac {\exp (x) - 1}{x} & \text {o t h e r w i s e} \end{array} \right. \tag {57}
$$
A numerically stable implementation of this function was critical to training our models.
Additionally, many of the channel equations below contain a 'temperature correction'  $q_{t}$  that adjusts for the fact that the original experiments and Allen Institute experiments were not done at the same temperature. In those equations,  $T$  is the temperature in Celsius which was  $35^{\circ}$ .
12.2.1 Transient  $\mathbf{Na}^{+}$
From Colbert and Pan [53].
$$
\lambda_ {c} = (m, h), \quad g _ {c} (\lambda_ {c}) = m ^ {3} h
$$
$$
\frac {1}{q _ {t}} \frac {d m}{d t} = \alpha_ {m} (v) (1 - m) - \beta_ {m} (v) m
$$
$$
\frac {1}{q _ {t}} \frac {d h}{d t} = \alpha_ {h} (v) (1 - h) - \beta_ {h} (v) h
$$
$$
q _ {t} = 2. 3 ^ {\left(\frac {T - 2 3}{1 0}\right)}
$$
$$
\alpha_ {m} (v) = \frac {0 . 1 8 2 \cdot 6}{\operatorname {e x p r e l} (- (v + 4 0) / 6)}, \quad \beta_ {m} (v) = \frac {0 . 1 2 4 \cdot 6}{\operatorname {e x p r e l} ((v + 4 0) / 6)}
$$
$$
\alpha_ {h} (v) = \frac {0 . 0 1 5 \cdot 6}{\operatorname {e x p r e l} ((v + 6 6) / 6)}, \quad \beta_ {h} (v) = \frac {0 . 0 1 5 \cdot 6}{\operatorname {e x p r e l} (- (v + 6 6) / 6)}
$$
12.2.2 Persistent  $\mathbf{Na}^{+}$
From Magistretti and Alonso [54].
$$
\lambda_ {c} = h, \quad g _ {c} (\lambda_ {c}) = m _ {\infty} h
$$
$$
m _ {\infty} = \frac {1}{1 + \exp (- (v + 5 2 . 6) / 4 . 6)}
$$
$$
q _ {t} = 2. 3 ^ {\left(\frac {T - 2 1}{1 0}\right)}
$$
$$
\alpha_ {h} (v) = \frac {2 . 8 8 \times 1 0 ^ {- 6} \cdot 4 . 6 3}{\operatorname {e x p r e l} ((v + 1 7 . 0 1 3) / 4 . 6 3)}, \quad \beta_ {h} (v) = \frac {6 . 9 4 \times 1 0 ^ {- 6} \cdot 2 . 6 3}{\operatorname {e x p r e l} (- (v + 6 4 . 4) / 2 . 6 3)}
$$
12.2.3 Hyperpolarization-activated cation conductance
From Kole et al. [55]. This channel uses a 'nonspecific cation current' meaning it can transport any cation. In practice, this is modeled by giving it its own special ion  $\mathrm{NSC^{+}}$  with resting potential
$E_{\mathrm{NSC^{+}}}$
$$
\lambda_ {c} = m, \quad g _ {c} (\lambda_ {c}) = m
$$
$$
E _ {\mathrm {N S C} ^ {+}} = - 4 5. 0
$$
$$
\frac {d m}{d t} = \alpha_ {m} (v) (1 - m) - \beta_ {m} (v) m
$$
$$
\alpha_ {m} (v) = \frac {0 . 0 0 1 \cdot 6 . 4 3 \cdot 1 1 . 9}{\operatorname {e x p r e l} ((v + 1 5 4 . 9) / 1 1 . 9)}, \quad \beta_ {m} (v) = 0. 0 0 1 \cdot 1 9 3 \cdot \exp (v / 3 3. 1)
$$
12.2.4 High-voltage-activated  $\mathrm{Ca^{2+}}$  conductance
From Reuveni et al. [56]
$$
\lambda_ {c} = (m, h), \quad g _ {c} (\lambda_ {c}) = m ^ {2} h
$$
$$
\frac {d h}{d t} = \alpha_ {h} (v) (1 - h) - \beta_ {h} (v) h
$$
$$
\alpha_ {m} (v) = \frac {0 . 0 5 5 \cdot 3 . 8}{\operatorname {e x p r e l} (- (v + 2 7) / 3 . 8)}, \quad \beta_ {m} (v) = 0. 9 4 \cdot \exp (- (v + 7 5) / 1 7)
$$
$$
\alpha_ {h} (v) = 0. 0 0 0 4 5 7 \cdot \exp (- (v + 1 3) / 5 0), \quad \beta_ {h} (v) = \frac {0 . 0 0 6 5}{\exp (- (v + 1 5) / 2 8) + 1}
$$
12.2.5 Low-voltage-activated  $\mathrm{Ca^{2+}}$  conductance
From Avery and Johnston [57], Randall and Tsien [58].
$$
\frac {1}{q _ {t}} \frac {d m}{d t} = \frac {m _ {\infty} - m}{m _ {\tau}}
$$
$$
\frac {1}{q _ {t}} \frac {d h}{d t} = \frac {h _ {\infty} - h}{h _ {\tau}}
$$
$$
q _ {t} = 2. 3 ^ {(T - 2 1) / 1 0}
$$
$$
m _ {\infty} = \frac {1}{1 + \exp (- (v + 4 0) / 6)}, \quad m _ {\tau} = 5 + \frac {2 0}{1 + \exp ((v + 3 5) / 5)}
$$
$$
h _ {\infty} = \frac {1}{1 + \exp ((v + 9 0) / 6 . 4)}, \quad h _ {\tau} = 2 0 + \frac {5 0}{1 + \exp ((v + 5 0) / 7)}
$$
12.2.6 M-type (Kv7)  $\mathbf{K}^{+}$  conductance
From Adams et al. [59].
$$
\alpha_ {m} (v) = 0. 0 0 3 3 \exp (0. 1 (v + 3 5)), \quad \beta_ {m} (v) = 0. 0 0 3 3 \cdot \exp (- 0. 1 (v + 3 5))
$$
12.2.7 Kv3-like  $\mathbf{K}^{+}$  conductance
$$
\frac {d m}{d t} = \frac {m _ {\infty} - m}{m _ {\tau}}
$$
$$
m _ {\infty} = \frac {1}{1 + \exp (- (v - 1 8 . 7) / 9 . 7)}, m _ {\tau} = \frac {4}{1 + \exp (- (v + 4 6 . 5 6) / 4 4 . 1 4)}
$$
12.2.8 Fast inactivating (transient, Kv4-like)  $\mathbf{K}^{+}$  conductance
From Korngreen and Sakmann [60].
$$
\lambda_ {c} = (m, h), \quad g _ {c} (\lambda_ {c}) = m ^ {4} h
$$
$$
m _ {\infty} = \frac {1}{1 + \exp (- (v + 4 7) / 2 9)}, \quad m _ {\tau} = 0. 3 4 + \frac {0 . 9 2}{\exp (((v + 7 1) / 5 9) ^ {2})}
$$
$$
h _ {\infty} = \frac {1}{1 + \exp ((v + 6 6) / 1 0)}, h _ {\tau} = 8 + \frac {4 9}{\exp ((v + 7 3) / 2 3) ^ {2})}
$$
$$
\overline {{g}} = 1 \times 1 0 ^ {- 5}
$$
12.2.9 Slow inactivating (persistent)  $\mathbf{K}^{+}$  conductance
$$
m _ {\infty} = \frac {1}{1 + \exp (- (v + 1 4 . 3) / 1 4 . 6)}
$$
$$
m _ {\tau} = \left\{ \begin{array}{l l} 1. 2 5 + 1 7 5. 0 3 \cdot e ^ {0. 0 2 6 v}, & \text {i f} v <   - 5 0 \\ 1. 2 5 + 1 3 \cdot e ^ {- 0. 0 2 6 v}, & \text {i f} v \geq - 5 0 \end{array} \right.
$$
$$
h _ {\infty} = \frac {1}{1 + \exp ((v + 5 4) / 1 1)}
$$
$$
h _ {\tau} = \frac {2 4 v + 2 6 9 0}{\exp ((\left(v + 7 5\right) / 4 8) ^ {2})}
$$
12.2.10 SK-type calcium-activated  $\mathbf{K}^{+}$  conductance
From Köhler et al. [61]. Note this is the only calcium-gated ion channel in the model.
$$
\lambda_ {c} = z, \quad g _ {c} (\lambda_ {c}) = z
$$
$$
\frac {d z}{d t} = \frac {z _ {\infty} - z}{z _ {\tau}}
$$
$$
z _ {\infty} = \frac {1}{1 + (0 . 0 0 0 4 3 / [ \mathrm {C a} ^ {2 +} ] _ {i}) ^ {4 . 8}}, \quad z _ {\tau} = 1
$$
12.3 Training Details
Dataset The dataset used to fit the model was a subset of the stimulus/response pairs available from the Allen Institute. First, all stimuli and responses were downloaded for cell 480169178. Then, sections of length 200 milliseconds were extracted from a subset of the stimuli types. The stimuli types and sections were chosen so that the neuron was at rest and unstimulated at the beginning of the trace. We list the exclusion criteria below.
1. Any "Hold" stimuli: Excluded because these traces were collected under voltage clamp conditions which we did not model.
2. Test: Excluded because the stimulus is  $0\mathrm{mV}$  for the entire trace.
3. Ramp/Ramp to Rheobase: Excluded because the cell is only at rest at the very beginning of the trace.
4. Short Square:  $250\mathrm{ms}$  to  $450~\mathrm{ms}$
5. Short Square — Triple: 1250 ms to 1450 ms.
6. Noise 1 and Noise 2:  $1250\mathrm{ms}$  to  $1450\mathrm{ms}$ ,  $9250\mathrm{ms}$  to  $9450\mathrm{ms}$ ,  $17250\mathrm{ms}$  to  $17450\mathrm{ms}$ .
7. Long Square:  $250\mathrm{ms}$  to  $450~\mathrm{ms}$
8. Square — 0.5ms Subthreshold: The entire trace.
9. Square — 2s Suprathreshold: 250 ms to 450 ms.
10. All others: Excluded.
For cell 480169178, the criteria above selected 95 stimulus/response pairs of 200 milliseconds each. Each trace pair was then downsampled to  $1\mathrm{ms}$  (from the original  $0.005\mathrm{ms}$  per step) and corrupted with mean-zero Gaussian noise of variance  $20\mathrm{mV}^2$  to simulate voltage imaging conditions. Finally, the 95 traces were randomly split into 72 training traces and 23 test traces.
**Proposal and Twist** The proposal and twist hyperparameters were broadly similar to the squid axon experiments, with the proposal being parameterized by a bidirectional RNN with a single hidden layer of size 64 and an MLP with a single hidden layer of size 64. The RNN was conditioned on the observed response and stimulus voltages at each timestep, and the MLP accepted the RNN hidden state, the previous latent state, and a transformer positional encoding of the number of steps since the last voltage response observation. The twist was similarly parameterized using an RNN run in reverse across the stimulus and response, combined with an MLP that accepted the RNN hidden state, the latent state being evaluated, and a transformer positional encoding of the number of steps elapsed since the last voltage response observation. The positional encodings were used to inform the twist and proposal of the number of steps elapsed since the last observation because the model was integrated with a stepsize of  $0.1\mathrm{ms}$  while observations were received once every millisecond.
Hyperparameter Sweeps To evaluate the methods we swept across the parameters
1. Initial observation variance:  $e^2, e^3, e^5$
2. Initial voltage dynamics variance:  $e, e^2, e^3$
3. Bias added to scales produced by the proposal:  $e^2, e^5$
We also evaluated the models across three different data noise variances (20, 10, and 5) but the results were similar for all values, so we reported only the results for variance 20. This amounted to  $3 \cdot 3 \cdot 3 \cdot 2$  different hyperparameter settings, and 5 seeds were run for each setting yielding a total of 270 runs.
When computing final performance, a hyperparameter setting was only evaluated if it had at least 3 runs that achieved 250,000 steps without NaN-ing out. For each hyperparameter setting selected for evaluation, all successful seeds were evaluated using early stopping on the train 4-particle log likelihood lower bound.