# Any neuron can perform linearly non-separable computations

Romain Cazé

► To cite this version:

Romain Cazé. Any neuron can perform linearly non-separable computations. F1000Research, 2021, 10, pp.539. 10.12688/f1000research.53961.1. hal-03408395

HAL Id: hal-03408395

https://hal.science/hal-03408395

Submitted on 20 May 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

BRIEF REPORT

# Any neuron can perform linearly non-separable computations
## [version 1; peer review: 1 approved with reservations]

Romain D. Cazé ID

CNRS IEMN UMR 8520, Villeneuve d'ascq, Haut de France, 59650, France

V1

**First published:** 06 Jul 2021, **10:**539
https://doi.org/10.12688/f1000research.53961.1

**Latest published:** 16 Sep 2021, **10:**539
https://doi.org/10.12688/f1000research.53961.2

## Abstract

Multiple studies have shown how dendrites enable some neurons to perform linearly non-separable computations. These works focus on cells with an extended dendritic arbor where voltage can vary independently, turning dendritic branches into local non-linear subunits. However, these studies leave a large fraction of the nervous system unexplored. Many neurons, e.g. granule cells, have modest dendritic trees and are electrically compact. It is impossible to decompose them into multiple independent subunits. Here, we upgraded the integrate and fire neuron to account for saturating dendrites. This artificial neuron has a unique membrane voltage and can be seen as a single layer. We present a class of linearly non-separable computations and how our neuron can perform them. We thus demonstrate that even a single layer neuron with dendrites has more computational capacity than without. Because any neuron has one or more layer, and all dendrites do saturate, we show that any dendrited neuron can implement linearly non-separable computations.

## Keywords

Dendrites, computation, linearly non-separable, neuroscience

## Open Peer Review

## Approval Status

<figure><img src="image_3.png" /></figure>

1

2

version 2

(revision)

16 Sep 2021

<figure><img src="image_4.png" /></figure>

view

<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /></figure>

version 1

06 Jul 2021

view

<figure><img src="image_7.png" /></figure>

view

1. Athanasia Papouts ID, Foundation for Research & Technology - Hellas, Heraklion, Greece

Spyridon Chavlis ID, Foundation for Research & Technology, Hellas, Greece

2. Balazs B Ujfalussy ID, Institute of Experimental Medicine, Budapest, Hungary

Any reports and responses or comments on the article can be found at the end of the article.
**Corresponding author:** Romain D. Cazé (romain.caze@protonmail.com)

**Author roles:** **Cazé RD:** Conceptualization, Formal Analysis, Investigation, Methodology, Software, Validation, Writing – Original Draft Preparation, Writing – Review & Editing

**Competing interests:** No competing interests were disclosed.

**Grant information:** This work was supported by the Centre National de la Recherche Scientifique [ANR-UWAKE].

*The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.*

**Copyright:** © 2021 Cazé RD. This is an open access article distributed under the terms of the [Creative Commons Attribution License](https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

**How to cite this article:** Cazé RD. Any neuron can perform linearly non-separable computations [version 1; peer review: 1 approved with reservations] F1000Research 2021, 10:539 https://doi.org/10.12688/f1000research.53961.1

**First published:** 06 Jul 2021, 10:539 https://doi.org/10.12688/f1000research.53961.1
# Introduction

We show here how dendrites can extend the computational capacity of all neurons, even the tiniest. We already knew that dendrites might extend the computational capacity of some pyramidal neurons. Their extended dendrites capable of dendritic spikes changed the way we saw them (see² for one of the first articles presenting this idea). More recently a study suggested that we should model these neurons as a two layer neural networks.⁶ This theoretical model was further consolidated by experiments showing that we can see a pyramidal neuron as a collection of non-linear subunits.⁷ Certain non-linearities can even allow a dendrite to implement the exclusive or (XOR).⁹ Moreover, a similar kind of non-monotonic non-linearity was found in human pyramidal neurons.⁴ But what about other neurons with modest dendrites incapable of spiking?

Pyramidal neurons only represent a fraction of all neurons. For instance, the dendrites of cerebellar stellate cells cannot emit spikes, but they do saturate¹ and they can be decomposed into multiple independent subunits - with independent membrane voltages - turning them into two-stage units like the pyramidal neuron.⁸ Previously we have shown that passive dendrites are sufficient to enable a neuron to perform linearly non-separable computations, for instance, the feature binding problem.³ We focus here on cells with a modest and passive dendritic tree. These cells form a single layer unit. In the present study, we demonstrate that these neurons can still implement a linearly non-separable computation. We use them as the simplest common denominator, as even spiking dendrites do saturate, and a 2 layer network can perform all the computation of a single layer architecture and more.

# Methods

## An integrate and fire neuron with dendrites (the DIF)

We started from a leaky integrate and fire (LIF). This model has a membrane V modelled by the following equation:

$$ \tau \frac{dv}{dt} = (v_E - v(t)) + RI_s(t) \quad (1) $$

With $τ = 20$ ms the neuron time constant, $v(t)$ the membrane voltage at time $t$ and $v_E = -65$ mV which sets the resting membrane voltage. $R = 20$ mΩ is the value of the resistance and $I_s(t)$ models the time varying synaptic inputs conductance.

$$ I_s(t) = \sum_i g_{d^i}(t)(E_s - v(t)) \quad (2) $$

This current depends on the difference between $v(t)$ the neuron voltage, equal everywhere, and $E_s$ the synaptic reversal potential (0 mV) while $g_{d^i}$ is the synaptic conductance in dendrite $i$. Each $g_{d^i}$ is bounded between 0 and $10pS$. Each $g_{d^i}$ jumps up instantaneously to its maximal value for each incoming input spike and decays exponentially with time constant $τ_s = 1$ ms. In a LIF all synaptic inputs are gathered into a single umbrella and $i = 1$. In the present study, we introduce the Dendrited Integrate and Fire (DIF) which includes at least two dendrites ($i = 2$). We cluster synaptic inputs into two groups, each targeting a dendrite (one green and one blue, see Figure 1). We used the Brian software version 2 to carry out our simulations, the code is freely available on the git repository attached with this report.¹⁰

## Boolean algebra refresher

First, let's present Boolean functions:

**Definition 1.** A Boolean function of $n$ variables is a function on $\{0,1\}^n$ into $\{0,1\}$, where $n$ is a positive integer.

Importantly, we commonly assume that neurons can only implement linearly separable computations:

**Definition 2.** $f$ is a linearly separable computation of $n$ variables if and only if there exists at least one vector $w \in \mathbb{R}^n$ and a threshold $\Theta \in \mathbb{R}$ such that:

$$ f(X) = \begin{cases} 1 & \text{if } w \cdot X \ge \Theta \\ 0 & \text{otherwise} \end{cases} $$

where $X \in \{0,1\}^n$ is the vector notation for the Boolean input variables.

# Results

## The compact feature binding problem (cFBP)

In this section, we demonstrate a class of compact linearly inseparable (non-separable) computations that we are going to study. These computations are compact because they have four input/output lines.
<figure><img src="image_8.png" /><figcaption>**Figure 1.** A dendrited integrate and fire implementing a linearly non-separable computation. (A) A leaky integrate and fire (LIF) with two dendrites making it a dendrited integrate and fire (DIF), each half of the 4 synaptic inputs targets a distinct dendrite where g locally saturates at 10pS (B) Four stimulation scenarios, filled circles stand for a >50 Hz input spike train while empty circles stand for >50 Hz input spike train. Below, we plotted the response of the DIF (black) and a LIF (grey). We purposely removed the ticks label as the frequencies depend on the parameter of the model and input regularity. The parameters of the model can vary largely without affecting the observation.⁵</figcaption></figure>

<table><caption>**Table 1.** The truth table of a non-linearly separable computation.</caption><thead><tr><td>Inputs</td><td>Output</td></tr></thead><tbody><tr><td>0011</td><td>0</td></tr><tr><td>1100</td><td>0</td></tr><tr><td>0101</td><td>1</td></tr><tr><td>1010</td><td>1</td></tr></tbody></table>

We entirely specify an example in Table 1. This computation that we call the compact feature binding problem (cFBP) is linearly inseparable.

**Proposition 1.** The cFBP is linearly inseparable (non-separable)

*Proof.* The output must be 0 for two disjoint couples (1,2) and (3,4) of active inputs. It means that $w_1 + w_2 \le \Theta$, and $w_3 + w_4 \le \Theta$, and we can add these two inequalities to obtain $w_1 + w_2 + w_3 + w_4 \le 2\Theta$. However, the output must be 1 for two other couples made of the same active inputs (1,3) and (2,4). It means that $w_1 + w_3 > \Theta$, and $w_2 + w_4 > \Theta$, and we can add these two inequalities to obtain $w_1 + w_2 + w_3 + w_4 > 2\Theta$. This yield a contradiction proving that no weights set exists solving this set of inequalities.

The cFBP is simple in two ways:

* Four input/output relations define this computation - the same number as the famous XOR (exclusive or).

* Contrary to the XOR it can be implemented with excitatory inputs and a monotone transfer function.³

We can extend the cFBP by increasing the number of inputs. In this case we deal with tuples instead of couples. As such, the cFBP corresponds to an entire family of linearly inseparable computations, and a dendrited neuron can implement them using the strategy that we will present in the next section.
A LIF with its linear integration cannot implement such a computation. While a neuron with two saturating dendrites can easily implement it. We already proved how a ball-and-stick biophysical model can implement this computation in a previous study.³

## Implementing the cFBP in a dendrited integrate and fire

We use two independently saturating conductances to implement the cFBP in a minimal extension of the LIF that we called the dendrited integrated and fire (DIF). The DIF has a single membrane voltage to account for its compactness so we might wonder how local saturation can arise in such a morphology. Saturation has two possible origins: (1) a reduction in driving force can cause saturation as in,¹ but (2) it can also be due to the intrinsic limitations in conductance per unit of surface. This latter possibility makes saturation possible in an electrically compact neuron. Even in a neuron with a small dendritic tree, the conductance is going to reach an upper bound per unit of surface and the only possibility to increase excitation consists in stimulating a larger area. We are going to employ this local bounding of the conductance to implement the cFBP in a DIF.

To do that, we only need two dendrites as shown in Figure 1A. We can interpret the 0s and 1s in the truth table in at least two ways: (1) either the pre- or post-synaptic neurons activates (2) or they reach a given spike frequency. In the following section, we will use the latter interpretation. Consequently, we consider a pre-synaptic input active when it fires above 50 Hz regular spike-train and inactive if it fires below 50 Hz (this value is arbitrary and can largely vary to match a neuron working range). We stimulate our model in four different combinations of inputs to reproduce the truth table from the previous section. You can observe on Figure 1 that locally bounding g enables implementation of cFBP. When g has no bound, the membrane voltage always reaches the spiking threshold at the same speed (LIF case). When we locally bound conductances the membrane voltage takes more time to reach threshold in the clustered case (total g = 10pS) than in the scattered case (total g = 20pS). All in all, a DIF will respond differently for the clustered and scattered case while a LIF won't. This enables a DIF to implement the cFBP while a LIF can't.

## Discussion/conclusion

In this brief report, we introduced a small extension to the leaky integrate and fire neuron: a dendrited integrate and fire neuron which can implement linearly non-separable computations. This single layer model applied to granule cells predicts that they can implement a linearly non-separable computation. These neurons have on average four dendrites, but we have shown here that two suffice. The DIF's multiple distinctly bounded g underlie this ability. For example, we need a local saturation of $g_d$ to implement the cFBP.

The experiment demonstrating this prediction seems straightforward. One would need to stimulate four distinct groups of mossy fibres following our different scenarios. We could then record how a group of granule cell respond using optogenetics reporting (i.e. calcium imaging). We predict that a significant part of granule cells might implement the cFBP. This prediction could reveal the true potential of single neurons. The next step consists of looking at the network level as already done with spiking dendrites.⁵

## Data availability

No data are associated with this article.

## Software availability

* Source code available from: https://github.com/rcaze/21_03Ca/tree/1.

* Archived source code at time of publication: https://doi.org/10.5281/zenodo.4937792.¹⁰

* License: MIT license.

## Acknowledgements

I used “we” as science is a collective endeavour. Discussions on this topic had begun as early as 2013 with my former PhD Advisor and collaborators from Institut Pasteur Paris. I also want to acknowledge M. Humphries, F Zeldenrust, A. Foust for their valuable comments on the early draft and Ms Marini-Audouard for the proof-reading before submission. An earlier version of this article can be found on bioRxiv (doi: https://doi.org/10.1101/2021.04.02.438177).
References

1. Abrahamsson T, Cathala L, Matsui K, et al.: Thin dendrites of cerebellar interneurons confer sublinear synaptic integration and a gradient of short-term plasticity. *Neuron*. **73**(6): 1159-1172.

PubMed Abstract | Publisher Full Text

2. Bartlett M: The clusteron: towards a simple abstraction to a complex neuron. *Advances in Neural Information Processing Systems*. 4.

3. Cazé RD, Humphries M, Gutkin B: Passive dendrites enable single neurons to compute linearly non-separable functions. *PLoS Comput Biol*. **9**(2): e1002867.

PubMed Abstract | Publisher Full Text | Free Full Text

4. Gidon A, Zolnik TA, Fidzinski P, et al.: Dendritic action potentials and computation in human layer 2/3 cortical neurons. *Science*. **367**(6473): 83-87.

PubMed Abstract | Publisher Full Text

5. Memmesheimer R-M, Timme M: Non-additive coupling enables propagation of synchronous spiking activity in purely random networks. *PLoS Comput Biol*. **8**(4):

e1002384.

PubMed Abstract | Publisher Full Text | Free Full Text

6. Poirazi P, Brannon T, Mel BW: Pyramidal neuron as two-layer neural network. *Neuron*. **37**(6): 989-999.

PubMed Abstract | Publisher Full Text

7. Polsky A, Mel BW, Schiller J: Computational subunits in thin dendrites of pyramidal cells. *Nat Neurosci*. **7**(6): 621-627.

PubMed Abstract | Publisher Full Text

8. Tzilivaki A, Kastellakis G, Poirazi P: Challenging the point neuron dogma: FS basket cells as 2-stage nonlinear integrators. *Nat Commun*. **10**(1): 3664.

PubMed Abstract | Publisher Full Text | Free Full Text

9. Zador AM, Claiborne BJ, Brown TH: Nonlinear pattern separation in single hippocampal neurons with active dendritic membrane. *Advances in Neural Information Processing Systems*. page 8.

10. Dr Cazé RD: rcaze/21_03Ca: Release F1000 (Version 1). *Zenodo*.
2021, June 12.

Publisher Full Text
# Open Peer Review

Current Peer Review Status: ?

## Version 1

## Reviewer Report 13 July 2021

https://doi.org/10.5256/f1000research.57398.r89096

© 2021 Papoutsi A. This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

<figure><img src="image_9.png" /></figure>

## Athanasia Papoutsi

<figure><img src="image_10.png" /></figure>

Institute of Molecular Biology & Biotechnology, Foundation for Research & Technology - Hellas, Heraklion, Greece

This Brief Report shows at a conceptual level that electrically compact neurons can solve non-linearly separable computations of four or more inputs. This is a result of the saturating responses to 'clustered' input at the dendritic level (simulating mainly the reduction of the driving force in the dendrites), and increased response to 'scattered' input at the somatic level. This study expands on previous work of the author and others and adds on the range of computations neurons can perform with their dendrites.

We have two major concerns that limit the clarity of this work:

1. Figure 1B and 3rd paragraph on page 5: The x-axis label states 'Time', and the relevant text states that "the membrane voltage takes more time to reach threshold in the clustered case (total g = 10pS) than in the scattered case (total g = 20pS)". Given that this work is based on an arbitrary thresholding of the output frequency, it is not obvious where time is involved and its meaning in the x-axis.

2. In the provided code on GitHub (line 78 in the code), the ceiling of the second dendrite (i.e., syn2 in the code) is set to 0.5 and not to 0.1. Please clarify the value used. If those different saturating thresholds were indeed used, this should be explicitly stated and reasoned in the main text.

Minor comments (not in order of importance nor appearance in the manuscript):

1. For clarity, specify that granule cells refer to the cerebellum (and not the hippocampus).

2. Correct R units to be MΩ (not mΩ).

3. Figure 1 legend: "filled circles stand for a >50 Hz input spike train while empty circles stand for >50 Hz input spike train." Change the second > to <.

4. Page 4: "This computation that we call the compact feature binding problem (cFBP) is
linearly inseparable.” Delete the second ‘the’.

**Is the work clearly and accurately presented and does it cite the current literature?**

No

**Is the study design appropriate and is the work technically sound?**

Yes

**Are sufficient details of methods and analysis provided to allow replication by others?**

Yes

**If applicable, is the statistical analysis and its interpretation appropriate?**

Yes

**Are all the source data underlying the results available to ensure full reproducibility?**

Yes

**Are the conclusions drawn adequately supported by the results?**

Yes

**Competing Interests:** No competing interests were disclosed.

**Reviewer Expertise:** computational neuroscience; dendritic computations

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

The benefits of publishing with F1000Research:

* Your article is published within days, with no editorial bias

* You can publish traditional articles, null/negative results, case reports, data notes and more

* The peer review process is transparent and collaborative

* Your article is indexed in PubMed after passing peer review

* Dedicated customer support at every stage

For pre-submission enquiries, contact research@f1000.com