SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Basile Confavreux*
basile.confavreux@gmail.com
Poornima Ramesh*
poornimaramesh1995@gmail.com
Pedro J. Gonçalves
VIB-Neuroelectronics Research Flanders (NERF), Belgium
imec, Belgium
Jakob H. Macke†
Max Planck Institute for
Intelligent Systems, Germany
Tim P. Vogels †
Institute of Science
and Technology Austria
There is substantial experimental evidence that learning- and memory-related behaviours rely on local synaptic changes, but the search for distinct plasticity rules has been driven by human intuition, with limited success for multiple, co-active plasticity rules in biological networks. More recently, automated meta-learning approaches have been used in simplified settings, such as rate networks and small feed-forward spiking networks. Here, we develop a simulation-based inference (SBI) method for sequentially filtering plasticity rules through an increasingly fine mesh of constraints that can be modified on-the-fly. This method, filter SBI, allows us to infer entire families of complex and co-active plasticity rules in spiking networks. We first consider flexibly parameterized doublet (Hebbian) rules, and find that the set of inferred rules contains solutions that extend and refine—and also reject—predictions from mean-field theory. Next, we expand the search space of plasticity rules by modelling them as multi-layer perceptrons that combine several plasticity-relevant factors, such as weight, voltage, triplets and co-dependency. Out of the millions of possible rules, we identify thousands of unique rule combinations that satisfy biological constraints like plausible activity and weight dynamics. They can be used as a starting point for further investigations into specific network computations, and already suggest refinements and predictions for classical experimental approaches on plasticity. This flexible approach for principled exploration of complex plasticity rules in large recurrent spiking networks presents the most advanced search tool to date for enabling robust predictions and deep insights into the plasticity mechanisms underlying brain function.
1 Introduction and related work
Synaptic plasticity has received continuous attention over the past decades [1-5]. Nevertheless, how functions such as learning and memory emerge from local synaptic changes is still poorly understood. Given the experimental inaccessibility of synapses in vivo, many studies have relied
*, †: equal contributions
37th Conference on Neural Information Processing Systems (NeurIPS 2023)
on theoretical methods to shed light on the plasticity rules at play [6-12]. However, this approach does not scale [3, 4]—in particular, previous attempts at tuning complex, co-active plasticity rules have been limited by human intuition, and had to rely on theoretical frameworks based on strongly simplifying assumptions [8, 9].
Recently, several studies have aimed to address these issues by automating the discovery of plasticity rules using numerical meta-learning approaches [13-20]. Given the non-differentiability of biological systems, along with the steep compute requirements of simulating plastic networks at scale, these studies have been restricted to rate networks [13-16, 18-20] or small feed-forward spiking networks [15, 17]. Moreover, these approaches require the a priori definition and optimization of a loss function that enforces a desired network computation, as well as various regularizers to ensure the implementation of the task is biologically plausible. Lastly, the above studies typically propose single plasticity mechanisms compatible with the data or network function considered.
Here, we aim at a more comprehensive approach to discovering biologically plausible plasticity rules. We study network dynamics in large recurrent spiking networks with plasticity rules that are flexibly parameterized (using polynomials or neural networks, Fig. 1A). We introduce filter simulation-based inference (fSBI), a new meta-learning approach that can thoroughly explore the space of potential rules. This method allows us to select sets of rules using metrics that effectively constrain network dynamics to fulfill the desired conditions. Our approach successfully infers entire families of rules that robustly establish plausible dynamics in large spiking networks. We demonstrate the scalability of fSBI by applying it to neural-network-based search spaces that include additional factors known to influence plasticity, such as, e.g., synaptic weight or membrane potentials. Despite the nonlinear interactions induced by the neural network parameterization of plasticity and the increase in the number of factors, fSBI successfully recovers plausible rules. Interestingly, when considered in isolation, the inferred candidate rules often result in contradictory experimental predictions, revealing that the classical pre-post protocols may only have marginal explanatory power.
Overall, our study introduces a principled approach for meta-learning plasticity rules in complex networks. The inferred distributions of plausible rules can be used as starting points for future studies of specific network computations, and raise questions about classical theoretical and experimental protocols for understanding plasticity.
Figure 1: Flexible inference of plasticity rules in recurrent spiking networks with fSBI. A: Plastic recurrent spiking networks are simulated with parameterized plasticity rules (parameters  $\theta$ ), either as polynomials (Figs. 2, 3) or as multi-layer perceptrons (Fig. 4). B: Filter-SBI (fSBI). Starting from a prior over plasticity parameters  $\pi_0(\theta)$ , we infer a posterior given a first network metric, for instance the population firing rate,  $p_1(\theta | r_{\mathrm{exc}} \cap r_{\mathrm{inh}})$ . In contrast to other classical Bayesian inference approaches,  $\pi_1 = p_1(\theta | r_{\mathrm{exc}} \cap r_{\mathrm{inh}})$  is used as a prior for the inference in the subsequent round, given a different network metric. We then repeat this process for various network metrics.
2.1 Spiking network model
We consider a recurrent spiking network of 4096 excitatory and 1024 inhibitory leaky integrate-and-fire neurons with conductance-based synapses, relative refractoriness, adaptive threshold and NMDA currents. The model parameter values follow Zenke et al. [8] (Fig. 1A; see Supp. 6.1 for a complete description of parameters). The membrane potential dynamics of neuron  $j$  (excitatory or inhibitory) are given by
$$
\tau_ {m} \frac {\mathrm {d} V _ {j}}{\mathrm {d} t} = - \left(V _ {j} - V _ {\text {r e s t}}\right) - g _ {j} ^ {\mathrm {E}} (t) \left(V _ {j} - E _ {\mathrm {E}}\right) - g _ {j} ^ {\mathrm {I}} (t) \left(V _ {j} - E _ {\mathrm {I}}\right), \tag {1}
$$
where  $\mathrm{E}$  stands for excitation and  $\mathrm{I}$  for inhibition. A postsynaptic spike is emitted whenever the membrane potential  $V_{j}(t)$  crosses a threshold  $V_{j}^{\mathrm{th}}(t)$ , with an instantaneous reset to  $V_{\mathrm{reset}}$ . This threshold  $V_{j}^{\mathrm{th}}(t)$  is incremented by  $V_{\mathrm{spike}}^{\mathrm{th}}$  every time neuron  $j$  spikes and otherwise decays following
$$
\tau_ {\text {t h}} \frac {\mathrm {d} V _ {j} ^ {\text {t h}}}{\mathrm {d} t} = V _ {\text {b a s e}} ^ {\text {t h}} - V _ {j} ^ {\text {t h}}. \tag {2}
$$
The excitatory and inhibitory conductances,  $g^{\mathrm{E}}$  and  $g^{\mathrm{I}}$  evolve such that
$$
g _ {j} ^ {\mathrm {E}} (t) = a g _ {j} ^ {\mathrm {A M P A}} (t) + (1 - a) g _ {j} ^ {\mathrm {N M D A}} (t) \quad \text {a n d} \quad \frac {\mathrm {d} g _ {j} ^ {\mathrm {I}}}{\mathrm {d} t} = - \frac {g _ {j} ^ {\mathrm {I}}}{\tau_ {\mathrm {G A B A}}} + \sum_ {i \in \mathrm {I n h}} w _ {i j} (t) \delta_ {i} (t)
$$
$$
\text {w i t h} \quad \frac {\mathrm {d} g _ {j} ^ {\mathrm {A M P A}}}{\mathrm {d} t} = - \frac {g _ {j} ^ {\mathrm {A M P A}}}{\tau_ {\mathrm {A M P A}}} + \sum_ {i \in \operatorname {E x c}} w _ {i j} (t) \delta_ {i} (t) \quad \text {a n d} \quad \frac {\mathrm {d} g _ {j} ^ {\mathrm {N M D A}}}{\mathrm {d} t} = \frac {g _ {j} ^ {\mathrm {A M P A}} (t) - g _ {j} ^ {\mathrm {N M D A}}}{\tau_ {\mathrm {N M D A}}}, \tag {3}
$$
with  $w_{ij}(t)$  the connection strength between neurons  $i$  and  $j$  (unitless),  $\delta_k(t) = \sum \delta (t - t_k^*)$  the spike train of pre-synaptic neuron  $k$ , where  $t_k^*$  denotes the spike times of neuron  $k$ , and  $\delta$  the Dirac delta. All neurons received input from 5k Poisson neurons, with  $5\%$  random connectivity and constant rate  $r_{\mathrm{ext}}$ , randomly sampled from the uniform distribution  $\mathcal{U}(5,15)\mathrm{Hz}$  in each simulation. The recurrent connectivity was instantiated with random sparse connectivity  $(10\%)$ .
2.2 Synaptic plasticity parameterizations
In order to explore a wide range of possible plasticity rules, we defined parameterized functional forms for these rules (i.e., search spaces) enabling us to flexibly generate various rules, by changing the parameters of the function. We considered two such parameterizations:
For polynomial plasticity rules, each recurrent synapse underwent spike-timing dependent plasticity (STDP) [21]. The weight from neuron  $i$  to  $j$  (of type  $\mathbf{X}$  and  $\mathrm{Y}$ ;  $(\mathrm{X},\mathrm{Y})\in \{\mathrm{E},\mathrm{I}\}$ ) evolved such that
$$
\frac {\mathrm {d} w _ {i j}}{\mathrm {d} t} = \eta \left[ \delta_ {i} (t) \left(\alpha_ {\mathrm {X Y}} + \kappa_ {\mathrm {X Y}} x _ {j} (t)\right) + \delta_ {j} (t) \left(\beta_ {\mathrm {X Y}} + \gamma_ {\mathrm {X Y}} x _ {i} (t)\right) \right], \tag {4}
$$
with  $\eta$  a fixed learning rate. The variables  $x_{i}(t)$  and  $x_{j}(t)$  trace the pre- and post-synaptic spike trains
$$
\frac {\mathrm {d} x _ {i}}{\mathrm {d} t} = - \frac {x _ {i}}{\tau_ {\mathrm {X Y}} ^ {\text {p r e}}} + \delta_ {i} (t) \quad \text {a n d} \quad \frac {\mathrm {d} x _ {j}}{\mathrm {d} t} = - \frac {x _ {j}}{\tau_ {\mathrm {X Y}} ^ {\text {p o s t}}} + \delta_ {j} (t), \tag {5}
$$
where  $\tau_{\mathrm{XY}}^{\mathrm{pre}}$  and  $\tau_{\mathrm{XY}}^{\mathrm{post}}$  are the time constants of the traces associated with the pre- and postsynaptic neurons, respectively. Thus, the plasticity updates at each connection type depended on 6 parameters:  $\theta_{\mathrm{XY}} = [\alpha_{\mathrm{XY}}, \beta_{\mathrm{XY}}, \gamma_{\mathrm{XY}}, \kappa_{\mathrm{XY}}, \tau_{\mathrm{XY}}^{\mathrm{pre}}, \tau_{\mathrm{XY}}^{\mathrm{post}}]$ , for a total of 24 parameters across all connection types (Fig. 2A).
For the MLP plasticity rules, only the recurrent EE and IE synapses underwent spike-triggered updates (Fig. 4A). Change of the weight from neuron  $i$  to  $j$  (of type X and E,  $\mathrm{X} \in (\mathrm{E}, \mathrm{I})$ ) was computed by running forward multi-layer perceptrons (MLPs) that took as inputs several synaptic variables relevant to plasticity<sup>1</sup>:
$$
\begin{array}{l} \frac {\mathrm {d} w _ {i j}}{\mathrm {d} t} = \eta \left[ \delta_ {i} (t) \mathrm {M L P} _ {\mathrm {X E}} ^ {\mathrm {p r e}} \left(x _ {i} ^ {(2)}, x _ {j} ^ {(1)}, w _ {i j}, \langle V _ {j} \rangle , C _ {j} ^ {\mathrm {e x c}}, C _ {j} ^ {\mathrm {i n h}}\right) + \right. \\ \left. \delta_ {j} (t) \mathrm {M L P} _ {\mathrm {X E}} ^ {\text {p o s t}} \left(x _ {i} ^ {(1)}, x _ {j} ^ {(2)}, w _ {i j}, \langle V _ {j} \rangle , C _ {j} ^ {\text {e x c}}, C _ {j} ^ {\text {i n h}}\right) \right], \\ \end{array}
$$
with  $C_j^{\mathrm{exc}} = \langle g_j^{\mathrm{E}}(E_{\mathrm{E}} - V_j)\rangle$  and  $C_j^{\mathrm{inh}} = \langle g_j^{\mathrm{I}}(E_{\mathrm{I}} - V_j)\rangle$  co-dependent terms representing the activity of neighboring synapses [10] (here all synapses with the same postsynaptic neuron), and  $\langle V_j(t)\rangle$  the low-pass filtered membrane potential [11]. In this search space, time constants of synaptic traces were fixed at  $\tau_{\mathrm{EE}}^{(1)} = \tau_{\mathrm{IE}}^{(1)} = 10\mathrm{ms}$  and  $\tau_{\mathrm{EE}}^{(2)} = \tau_{\mathrm{IE}}^{(2)} = 100\mathrm{ms}$  [22, 7, 8, 10]. We chose a two-hidden-layer MLP, composed of 50 sigmoidal units in the first hidden layer, and 4 in the second. We trained only the final layer, keeping all other layers fixed. While this is standard practice with pre-trained ML
<sup>1</sup> Note that all input variables to the MLP are functions of  $t$ , which we elide to lighten notation.
models, we randomly initialised the parameters of the frozen layers, rather than pre-training them. We verified that when using random initializations for the first 2 hidden layers  $(\sim \mathcal{U}(\frac{-1}{n_{\mathrm{inp}}},\frac{1}{n_{\mathrm{inp}}})$  , where  $n_{\mathrm{inp}}$  is the number of input features at a given layer), and only training the weights, bias, and output learning rate of the final layer, previously observed plasticity rules could be approximated (Supp. 6.7).
A given plasticity rule (EE or IE) thus consisted of one MLP (with 5 parameters: 4 weights and 1 bias) to update the synaptic weight when a pre-spike occurred, another MLP updating the synaptic weight for post-spikes (Fig. 4B), and a common learning rate  $\eta_{\mathrm{XE}}$  for both MLPs, resulting in 11 parameters per rule, and a total of 22 parameters between EE and IE.
2.3 Mean-Field predictions
In the polynomial case, we performed a self-consistent analysis of the weight and activity dynamics in the network at steady state, following previous work [23, 7, 10] (more details for all connection types in Supp. 6.3). For example, at EE synapses, we could calculate an expected firing rate  $r_{\mathrm{exc}}^*$  from the parameters of the rule such that:
$$
r _ {\mathrm {e x c}} ^ {*} = \frac {- \alpha_ {\mathrm {E E}} - \beta_ {\mathrm {E E}}}{\lambda_ {\mathrm {E E}}}, \tag {7}
$$
with  $\lambda_{\mathrm{EE}} = \kappa_{\mathrm{EE}}\tau_{\mathrm{EE}}^{\mathrm{post}} + \gamma_{\mathrm{EE}}\tau_{\mathrm{EE}}^{\mathrm{pre}}$
2.4 Assessing network plausibility with metrics
To assess whether a particular plasticity rule was a suitable candidate, we simulated a spiking neuronal network (SNN) with that rule and quantified the resulting dynamics with multiple metrics on the activity and weight traces. The metric choices and values were informed by—and in line with—a range of modeling and experimental cortical studies [7, 8, 24-34]. For every simulation, we stored the raw spike trains and weight traces for 10s after 110s of simulated time (60s for the MLP rules). We grouped 15 individual metrics into four main criteria:
Stable activity dynamics: Rules were flagged as suitable when the excitatory and inhibitory population firing rates were bounded between 1 and  $50\mathrm{Hz}$ , i.e.,  $r_{\mathrm{exc}} \in [1,50]\mathrm{Hz}$  and  $r_{\mathrm{inh}} \in [1,50]\mathrm{Hz}$ , in line with a large swathe of literature reporting cortical firing rates of a few Hertz on average [24, 29, 30].
Stable weight dynamics: Fraction of synaptic weights that reached extreme values (0 or  $w_{\mathrm{max}}$ ),  $f_{w_{\mathrm{blow}}}$ ; mean absolute change in synaptic weights between simulation start and end across all synapse types,  $w_{\mathrm{creep}}$ ; mean weight at the last time-step across all synapses of a specific type,  $\langle w_{\mathrm{XY}} \rangle$ . These metrics excluded rules whose weights changed too rapidly, did not converge, or reached unrealistic values. Rules were flagged as suitable if they satisfied  $f_{w_{\mathrm{blow}}} < 0.1$ ,  $w_{\mathrm{creep}} < 0.05$ ,  $\langle w_{\mathrm{EX}} \rangle < 0.5$  and  $\langle w_{\mathrm{IX}} \rangle < 5$  simultaneously.
Near-irregular dynamics: Mean coefficient of variation of the inter-spike intervals across all excitatory neurons,  $\langle \mathrm{cv}(\mathrm{ISI}_i)\rangle$ ; auto-covariance for each spike-train, and its integral, averaged over time and the neural population,  $\langle \rho \rangle_{i,t}$ ; Fano factor for each spike train averaged over the population,  $\langle \mathrm{Fano}\rangle_i$ ; standard deviation of the firing rate of individual neurons averaged across the population,  $\langle \sigma \rangle_i$ . These metrics quantified the regularity of a spike train, but, either due to metric assumptions or hyperparameter choices, each metric exhibited a different failure mode, which could be filtered out by combining the metrics. Rules were flagged as suitable if they satisfied  $\langle \mathrm{cv}(\mathrm{ISI}_i)\rangle > 0.7$ ,  $\langle \rho \rangle_{i,t} < 0.1$ ,  $\langle \mathrm{Fano}\rangle_i \in [0.5, 2.5]$  and  $\langle \sigma \rangle_i < 5$  simultaneously.
Near-asynchronous dynamics: Standard deviation of the excitatory population firing rate,  $\sigma_{r_{\mathrm{exc}}}$ ; the Fano factor averaged across the population and then across time windows,  $\langle \mathrm{Fano} \rangle_t$ ; we also computed the population average of binned spike-trains, Fourier-transformed it and averaged the resulting power spectrum across all frequencies  $f$  (except 0),  $< S >_{i,f}$ . Rules were flagged as suitable if they satisfied  $\langle \mathrm{Fano} \rangle_t \in [0.5, 2.5], < S >_{i,f} < 1$  and  $\sigma_{r_{\mathrm{exc}}} < 0.05$  simultaneously. These bounds enforced the broadly asynchronous-irregular activity observed in cortex [24, 25, 27, 29, 30]. Ranges for metrics assessing synchrony/regularity were thus devised with independent Poisson spike trains in mind, with leeway for spatiotemporal correlations, as seen in cortical recordings [32-34].
Rules were considered plausible if they fulfilled all four broad criteria above (more details and analysis of individual metrics in Supp. 6.2).
2.5 Filter simulation-based inference
Having defined plasticity rules search spaces and network metrics, we turned to the problem of exploring these spaces, systematically filtering out unsuitable rules.
Simulation-based inference (SBI) is broadly defined as an approach for performing Bayesian inference given stochastic models (with tunable parameters  $\theta$ , and generating samples  $x$ ) in cases where the likelihood of the model  $p(x|\theta)$  is analytically inaccessible but simulation is possible [35-37]. SBI infers the posterior distribution  $p(\theta|x)$ , using samples from a prior distribution  $\theta \sim \pi(\theta)$  and corresponding simulations  $x \sim p(x|\theta)$ , i.e., it applies Bayes' rule such that  $p(\theta|x) \propto p(x|\theta) \pi(\theta)$ , where  $p(\theta|x)$  represents the distribution over model parameters most likely to have generated  $x$ .
This is an appealing framework for the problem of identifying plausible plasticity rules: we treated the parameters of the plasticity rule as parameters  $\theta$  of a stochastic plastic spiking network that generated simulations  $x$  consisting of the metrics defined in Section 2.4. We then used SBI—e.g., Neural Posterior Estimation (NPE) [35, 38, 39]—to identify the posterior distribution over plasticity rule parameters that were most likely to have resulted in plausible network dynamics. However, as we could not know beforehand which parameters were most likely to establish plausible dynamics, we had to define a broad prior over the parameters of the plasticity rule  $\pi_0(\theta)$ . In turn, the broadness of the prior meant that potentially millions of SNN simulations would be needed to find a few plausible rules, rendering this search method computationally unfeasible [40, 41].
In order to address this issue, we developed filter SBI (fSBI). In fSBI, we applied the metrics quantifying plausibility sequentially, rather than all at once (Fig. 1B). Towards this goal, we first sampled plasticity rule parameters  $\theta$  from a uniform prior and simulated an SNN with the corresponding rules. Next, we computed the various metrics defined in Section 2.4. We then used NPE to fit a posterior density over the plasticity rule parameters. Finally, we sampled all parameters that satisfied a given criterion from the posterior, e.g., stable network activity  $p(\theta | r_{\mathrm{exc}} \in [1,50]\mathrm{Hz} \cap r_{\mathrm{inh}} \in [1,50]\mathrm{Hz})$ , in the first instance, effectively "filtering out" other rules from the prior. These posterior samples were then used as the prior samples for the next round of inference, in which an additional criterion was applied to obtain a new, narrower posterior. Mathematically speaking, a filtering round  $k$  using metrics  $m_k = f_k(x)$  and corresponding conditions  $g_k(m_k)$  proceeds as follows:
$$
\pi_ {k} (\theta) = p _ {k - 1} \left(\theta \mid m _ {k - 1} s. t. \mathbb {I} _ {g _ {k - 1}} \left(m _ {k - 1}\right)\right) \tag {8}
$$
$$
p _ {k} (\theta | m _ {k}) \propto p (m _ {k} | \theta) \pi_ {k} (\theta) \tag {9}
$$
$$
\pi_ {k + 1} (\theta) = p _ {k} \left(\theta \mid m _ {k} \text {s . t .} \mathbb {I} _ {g _ {k}} \left(m _ {k}\right)\right), \tag {10}
$$
where  $\mathbb{I}$  is an indicator function over the condition  $g$ . This process is repeated until only plausible candidate rules remain, for a total of around 200k simulated rules per search space.
The distribution obtained at each filtering round is not necessarily a true Bayesian posterior, since (a) we change the metric on which we condition the posterior at each round, (b) we sample the posterior for a range of metric values rather than a single value, and (c) we also do not correct for the fact that at each round, we are sampling from the posterior of the previous round rather than from the prior [42] (more detailed analysis of this "pseudo"-posterior in Supp. 6.5).
In our attempt to discover plasticity rules compatible with a given function, we introduced fSBI, a meta learning method that sequentially filters out unfit candidates rules from vast sets of possible plasticity rules in large recurrent spiking networks (SNNs, Methods. 2.1,10). We first applied fSBI in the polynomial search space described in Section 2.2, and compared the resulting distributions of rules to those predicted by mean-field analysis. We then applied fSBI to plasticity rules which themselves were parameterized by neural networks. In both cases, fSBI inferred expected and novel rules that robustly establish plausible dynamics in SNNs, while also eliminating some theoretically viable solutions.
3.1 fSBI infers plausible rules from a polynomial search space
We initially considered 4 co-active plasticity rules, E-to-E (EE), E-to-I (EI), I-to-E (IE) and I-to-I (II) in a recurrent spiking network of 5120 conductance-based leaky integrate-and-fire neurons (Fig. 2A). The network architecture and parameters were chosen following Zenke et al. [8]. Each
Figure 2: Posterior of plausible plasticity rules with fSBI. Polynomial search-space. A: Recurrent spiking network receiving Poisson inputs at a random rate  $r_{\mathrm{ext}}$ , with each recurrent connection parameterized with an STDP plasticity rule. Dotted lines denote plastic connections. B: Starting from uniform draws  $(\pi_0)$  from the polynomial search space, fSBI progressively filtered out the rules that do not establish plausible dynamics in an SNN  $(\pi_1, \pi_2, \pi_3)$ , A.I. stands for asynchronous irregular. C: Fraction of rules sampled from each intermediate fSBI posterior that obey conditions on the corresponding network metrics. D: Principal component analysis (PCA) on samples from each intermediate fSBI posterior  $(\pi_0 \rightarrow \pi_3)$ . Variance explained along the corresponding PCA directions across rounds. E: For each plastic network simulated in fSBI (pooled across all rounds), the corresponding CV and auto-covariance (Methods. 2.4). F: Example edge-case of the autocovariance metric: network fulfills the auto-covariance condition  $(\rho < 0.1)$ , but not the CV condition  $(\langle \mathrm{cv}(\mathrm{ISI}_i) \rangle \neq 1)$ . G: Example edge-case of the CV metric: network fulfills the CV condition, but not the auto-covariance condition. H: Left: Example plausible candidate rule simulated in two networks with different input rates. Right: pre-post pairing protocol on the 4 co-active rules.