Patent Description:
The present disclosure relates generally to neural networks. More particularly, the present disclosure relates to leaky spiking neural networks that perform temporal encoding.

Traditionally, artificial neural networks have been predominantly constructed from idealized neurons that use non-linear activation layers to generate continuous activation values based on a set of weighted inputs. Some neural networks have multiple sequential layers of such neurons, in which case they may be referred to as "deep" neural networks.

Non-spiking neural networks typically pass information through the network using non-linear activation layers that produce continuous-valued outputs. These non-linear activation layers are differentiable, which enables the gradient of a loss function with respect to the weights of the network to be determined. In the multi-layer case, the existence of the gradient of the loss function makes it possible to use gradient-based optimization methods in combination with the backpropagation algorithm to learn particular weight values that enable the network to accurately perform a certain task.

Gradient-based optimization techniques (e.g., gradient descent) have been highly successful in training continuous-valued neural networks. However, gradient-based techniques do not easily transfer to spiking neural networks due to the hard nonlinearity of spike generation and the discrete nature of spike communication.

Furthermore, spiking neural networks are dynamic systems in which the respective times at which various neurons spike play a critical role. This is in contrast to conventional feedforward neural networks in which time is abstracted away. In particular, state transfer in classic neural nets happens globally and synchronously.

Synchronous systems have to distribute a clock and lose some possibility of phase to be used to extend the information transfer bandwidth between the neurons. From the bandwidth viewpoint, ideally the neurons would self-synchronize. That would eliminate the clock distribution requirement and would increase the information transfer bandwidth in both hardware and software implementations of recurrent neural networks.

More particularly, unlike non-spiking neurons that output analog values, spiking neurons typically communicate using discrete spikes which are binary in nature (e.g., either a spike is output or not). Typically a spike triggers a trace of synaptic current in the receiving neuron or otherwise impacts a membrane potential of the receiving neuron. In some example formulations, the receiving neuron integrates received synaptic current over time until a firing threshold is reached, at which time the neuron itself spikes or fires. Due to their hard nonlinearity, neuron spike rates are typically non-differentiable, which has prevented widespread application of gradient-based techniques to spiking neural networks.

Thus, while backpropagation is an established general technique for training traditional non-spiking neural networks, a general technique for training spiking neural networks has not yet been established. Certain previous approaches that train spiking neural networks to produce particular spike patterns depend on the absence of any hidden layers (e.g., the input layer is directly connected to the output layer). Thus, multi-layer networks cannot be trained using these approaches.

It remains a challenge to train spiking networks, especially with multi-layer learning (e.g., deep spiking neural networks). Enabling learning within multi-layer spiking neural networks is an area of ongoing development and has potential to greatly improve the performance of spiking neural networks on different tasks. <NPL> discloses that in a feedforward spiking network that uses a temporal coding scheme where information is encoded in spike times instead of spike rates, the network input-output relation is differentiable almost everywhere. <NPL> discloses a model in which conductances were modeled as either double exponential or alpha functions of time since the last spike.

One example aspect of the present disclosure is directed to a computer system that includes one or more processors and one or more non-transitory computer readable media. The one or more non-transitory computer readable media collectively store a machine-learned spiking neural network that includes one or more spiking neurons that have an activation layer that uses a double exponential function to model a leaky input that an incoming neuron spike provides to a membrane potential of the spiking neuron. The one or more non-transitory computer readable media collectively store instructions that, when executed by the one or more processors, cause the computer system to perform operations. The operations include obtaining a network input. The operations include implementing the machine-learned spiking neural network to process the network input. The operations include receiving a network output generated by the machine-learned spiking neural network as a result of processing the network input.

In some implementations, the machine-learned spiking neural network encodes information in respective spike times associated with the one or more spiking neurons.

In some implementations, the double exponential function models the leaky input as a double exponential pulse. In some implementations, the double exponential function has the form e-t(t - <NUM> + c), where c is a hyperparameter. In some implementations, the double exponential function has the form te-t.

In some implementations, the membrane potential of each of the one or more spiking neurons, if it has not spiked, has the form ∑iwi(t - ti)eti- t, where i refers to one or more presynaptic neurons connected to the spiking neuron via one or more artificial synapses with weights wi and spiking at time points ti.

In some implementations, implementing the machine-learned spiking neural network includes determining, for each of the one or more spiking neurons, a spike time that corresponds to an earliest time at which the membrane potential of the spiking neuron is equal to a firing threshold.

In some implementations, determining, for each of the one or more spiking neurons, the spike time includes applying a Lambert W function to determine the spike time.

In some implementations, the operations further include: prior to obtaining the network input, training the machine-learned spiking neural network on training data via a gradient descent technique. In some implementations, training the machine-learned spiking neural network via the gradient descent includes determining, for each of the one or more spiking neurons, one or both of: a derivative of a spike time of such spiking neuron with respect to the time points ti; and a derivative of the spike time of such spiking neuron with respect to one or more of the weights wi, wherein the spike time corresponds to an earliest time at which the membrane potential of such spiking neuron is equal to a firing threshold. In some implementations, training the machine-learned spiking neural network via the gradient descent includes modifying, for each of the one or more spiking neurons, at least one of the weights wi based at least in part on one or both of the derivative of the spike time of such spiking neuron with respect to the time points ti and the derivative of the spike time of such spiking neuron with respect to one or more of the weights wi.

In some implementations, the machine-learned spiking neural network includes a plurality of layers, at least two of the plurality of layers including at least one of the one or more spiking neurons, and the machine-learned spiking neural network has been trained on training data using a backpropagation technique.

In some implementations, the operations further include: training the machine-learned spiking neural network on training data via a gradient descent technique to simultaneously learn both parameters of the machine-learned spiking neural network and a topology of the machine-learned spiking neural network.

Another example aspect of the present disclosure is directed to a computer-implemented method to train a spiking neural network that encodes information in respective spike times associated with a plurality of spiking neurons included in the spiking neural network. The method includes obtaining, by one or more computing devices, data descriptive of the spiking neural network that includes the plurality of spiking neurons. Each of the plurality of spiking neurons is respectively connected to one or more pre-synaptic neurons via one or more artificial synapses that have one or more weights associated therewith. Each of the plurality of spiking neurons has an activation layer that controls a respective spike time of such spiking neuron based on a membrane potential of such spiking neuron. The activation layer for each of the plurality of spiking neurons includes a double exponential that models incoming spikes received from the one or more presynaptic neurons as leaky inputs to the membrane potential. The method includes training, by the one or more computing devices, the spiking neural network based on a set of training data. Training, by the one or more computing devices, the spiking neural network includes: determining, by the one or more computing devices, a gradient of a loss function that evaluates a performance of the spiking neural network on the set of training data; and modifying, by the one or more computing devices for at least one of the plurality of spiking neurons, at least one of the one or more weights based at least in part on the gradient of the loss function.

In some implementations, each of the plurality of spiking neurons receives the incoming spikes from the one or more presynaptic neurons at respective inbound spike times. In some implementations, determining, by the one or more computing devices, the gradient of the loss function includes determining, by the one or more computing devices, for at least one of the plurality of spiking neurons, a derivative of the spike time of such spiking neuron with respect to the inbound spike times.

In some implementations, determining, by the one or more computing devices, the gradient of the loss function includes determining, by the one or more computing devices, for at least one of the plurality of spiking neurons, a derivative of the spike time of such spiking neuron with respect to one or more of the weights associated with such spiking neuron.

In some implementations, training, by the one or more computing devices, the spiking neural network further includes modifying, by the one or more computing devices for at least one of the plurality of spiking neurons, at least one synaptic delay parameter based at least in part on the gradient of the loss function.

In some implementations, the plurality of spiking neurons are arranged in a plurality of layers. In some implementations, training, by the one or more computing devices, the spiking neural network includes backpropagating, by the one or more computing devices, the loss function through the plurality of layers.

In some implementations, for each of the plurality of spiking neurons, the membrane potential, if such spiking neuron has not yet spiked, has the form Σiwi(t - ti)eti-t, where i refers to the one or more presynaptic neurons connected to such spiking neuron via one or more artificial synapses, wi refers to the one or more weights associated with the one or more artificial synapses, and ti refers to respective inbound spike times at which such spiking neuron receives the incoming spikes from the one or more presynaptic neurons.

In some implementations, the machine-learned spiking neural network includes one or more electronic circuits that include electronic components arranged to execute the machine-learned spiking neural network using electrical current.

In some implementations, for each of the one or more spiking neurons, the corresponding electronic components that model the double exponential function include two capacitors, two resistors, and one or more transistors.

These features, aspects, and advantages of various embodiments of the present disclosure will become better understood with reference to the following description and appended claims.

Generally, the present disclosure is directed to spiking neural networks that perform temporal encoding for phase-coherent neural computing. In particular, according to an aspect of the present disclosure, a spiking neural network can include one or more spiking neurons that have an activation layer that uses a double exponential function, which can also be referred to as an "alpha function," to model a leaky input that an incoming neuron spike provides to a membrane potential of the spiking neuron. The use of the double exponential function in the neuron's temporal transfer function creates a better defined maximum in time. This allows very clearly defined state transitions between "now" and the "future step" to happen without loss of phase coherence.

More particularly, the present disclosure provides biologically-realistic synaptic transfer functions, for example of the form te-t, produced by the integration of exponentially decaying kernels. In contrast with the single exponential function, the double exponential function gradually rises before slowly decaying (see, e.g., <FIG>), which allows more intricate interactions between presynaptic inputs. The double exponential function provides a biologically-plausible model for exploring the problem-solving abilities of spiking networks with temporal coding schemes. In particular, it is possible to derive exact gradients with respect to spike times using this model.

Therefore, aspects of the present disclosure are directed to spiking network models that use the double exponential function for synaptic transfer and encodes information in relative spike times. The networks can be fully trained in temporal domain using exact gradients over domains where relative spiking order is preserved. Example experimental results with models of this type have been shown capable of learning standard benchmark problems, such as Boolean logic gates and MNIST, encoded in individual spike times. To facilitate transformations of the class boundaries, synchronization pulses can be used, which are neurons that send spikes at input-independent, learned times.

The proposed model are easily able to solve temporally-encoded Boolean logic and other benchmark problems. An analysis of the behavior of the spiking network during training shows that it spontaneously displays two operational regimes that reflect a trade-off between speed and accuracy: a slow regime that is slow but very accurate, and a fast regime that is slightly less accurate but makes decisions much faster.

Thus, the present disclosure develops the idea of temporal coding in leaky neurons (e.g., leaky integrate-and-fire neurons). One primary aspect described herein is the encoding of information in the spike times of spiking neurons, rather than spike rates. In particular, the output of a neuron can be its spike time, which can depend on the timings and weights of presynaptic neurons that cause it to fire. The formulation of a neuron's spike time in the continuous time domain renders it differentiable, which enables usage of backpropagation and gradient-based techniques to learn the spike timings in the network. This also optionally allows the addition of synaptic delays, also trainable using backpropagation techniques.

As such, according to another aspect, the present disclosure provides systems that enable application of gradient-based learning algorithms to learn the double exponential time transfer function. Furthermore, the systems described herein can implement the gradient-based learning algorithm to learn to build internal states in a recurrent network, allowing the network to learn states and state transfers faster.

The present disclosure provides a number of technical effects and benefits. As one example technical effect and benefit, by encoding information in spike times, the use of spike counts or spike rates can be eliminated. Further, as described herein, the neuron spike times can be formulated as a continuous representation which is differentiable and therefore amenable to gradient-based training techniques. Use of gradient-based techniques allows precise learning within the network (e.g., at the level of single spike times) and naturally extends to multi-layer scenarios, which would not be possible in training approaches based on rate-coding. In addition, use of gradient-based techniques for training the network can be more efficient than various other existing techniques which are more computationally expensive.

Enabling efficient training of spiking neural networks with gradient-based techniques provides further technical effects and benefits. By the techniques described herein enabling the training of spiking neural networks with gradient-based techniques, spiking neural networks can be trained to perform many supervised and reinforcement learning tasks where it was previously impossible, or at least infeasible, to train spiking neural networks to perform these tasks. In many instances, implementations of spiking neural networks on neuromorphic hardware can operate with significantly less energy resources than alternatives capable of performing these tasks, e.g. perceptron-based networks.

The trained spiking neural networks described above is suited to perform speech recognition In particular, the inherently temporal nature of the trained spiking neural networks makes them particularly suited for speech recognition tasks operating on audio data.

As another example technical effect and benefit, by encoding information in continuous space spike times, the network can be enabled to operate asynchronously. This better models the human brain and enables use of differential equations. Further, in some implementations, use of an asynchronous network can enable multiple rhythms or flows of information to propagate through the network at the same time, which can allow for parallel, sequential, and/or recurrent processing of input.

As another example technical effect and benefit, by encoding information in spike times, neuron firing can be highly sparse because the time of each spike can encode a large amount of information. As such, the networks described herein can be much more efficiently implemented than networks which encode information using spike rates, which themselves can be more efficient than traditional non-spiking networks. More particularly, since temporal encoding neurons typically fire many fewer times than rate-based encoding neurons, less computing resources (e.g., energy resources, processing resources, memory resources, etc.) are required to be expended to run the network. Thus, by encoding in spike time (e.g., high information content in spikes that are sparse in time) rather than spike rate, the number of neuron spikes (e.g., each of which can consume resources) can be greatly reduced.

Use of the double exponential function in the neuron's activation layer also provides technical effects and benefits. As one example, the double exponential function better mimics actual biological neuron behavior and provides a natural inherent rhythm/speed for information propagation within the network.

As another example, the double exponential function creates a better defined maximum in time (e.g., as opposed to a square wave representation, single exponential representation, or other monotonic representation). This allows very clearly defined state transitions between "now" and the "future step" to happen without loss of phase coherence.

In addition, summing or integration of incoming spikes can happen more effectively as the incoming spike's impact is moved from the exact immediate time of receipt to a slightly delayed point in the future. This slight delay enables more information to be collected prior to neuron spiking.

The use of a double exponential function also enables differentiation to occur with a double differential instead of a single differential. The optimization surface for the double differential is often smoother than that of the single differential, which will often exhibit ripples. This smoother optimization surface can result in faster training times and better convergence, as the gradient descent technique is able to more quickly and easily locate an optimal point on the surface. Faster training and better convergence can result in savings of various resources as less computing resources (e.g., energy resources, processing resources, memory resources, etc.) are required to be expended to train the network.

Although particular emphasis is placed on use of the double exponential function in the present disclosure, other functions could be used in addition to the double exponential function. As examples, a Gaussian or a Poisson distribution could be used in a temporal activation layer. As other examples, other non-monotonic and/or unimodal functions can be used in addition to the double exponential function.

In example implementations of the proposed models, information can be encoded in the relative timing of individual spikes. The input features can be encoded in temporal domain as the spike times of individual input neurons, with each neuron corresponding to a distinct feature. More salient information about a feature can be encoded as an earlier spike in the corresponding neuron. Information can propagate through the network in a temporal fashion. Each hidden and output neuron can spike when its membrane potential rises above a fixed threshold. Similarly to the input layer, the output layer of the network can encode a result in the relative timing of output spikes. In other words, the computational process can include producing a temporal sequence of spikes across the network in a particular order, with the result encoded in the ordering of spikes in the output layer.

This model can be used to solve standard classification problems. Given a classification problem with m inputs and n possible classes, the inputs can be encoded as the spike times of individual neurons in the input layer and the result can be encoded as the index of the neuron that spikes first among the neurons in the output layer. An example drawn from class k is classified correctly if and only if the kth output neuron is the first to spike. An earlier output spike can reflect more confidence of the network in classifying a particular example, as it implies more synaptic efficiency or a smaller number of presynaptic spikes. In a biological setting, the winning neuron could suppress the activity of neighbouring neurons through lateral inhibition, while in a machine learning setting the spike times of the non-winning neurons can be useful in indicating alternative predictions of the network. The learning process aims to change the synaptic weights and thus the spike timings in such a way that the target order of spikes is produced.

<FIG> provides a graphical diagram of an example spiking neuron <NUM>. The spiking neuron <NUM> can be connected to one or more presynaptic neurons <NUM>, <NUM>, <NUM> (e.g., which may themselves be spiking neurons). The spiking neuron <NUM> can be connected to the presynaptic neurons <NUM>, <NUM>, <NUM> via artificial synapses <NUM>, <NUM>, <NUM>. The presynaptic neurons <NUM>, <NUM>, <NUM> can pass spikes to the spiking neuron <NUM> via the artificial synapses <NUM>, <NUM>, <NUM>.

Each synapse <NUM>, <NUM>, <NUM> can have an adjustable weight <NUM>, <NUM>, <NUM> (e.g., scalar weight) associated therewith. The weights <NUM>, <NUM>, <NUM> can be changed as a result of learning. As described above, techniques for performing this learning rule within the spiking neural network context have been one of the most challenging components for developing multi-layer spiking neural networks because the non-differentiability of spike trains has limited application of the backpropagation algorithm.

Referring again to <FIG>, each artificial synapse <NUM>, <NUM>, <NUM> can be either excitatory (e.g., have a positive-valued weight), which increases the membrane potential of the receiving neuron <NUM> upon receipt, or inhibitory (e.g., have a negative-valued weight), which decreases the membrane potential of the receiving neuron <NUM> upon receipt.

More particularly, the spiking neuron <NUM> can have a membrane potential <NUM>. The membrane potential <NUM> can be a continuous-valued function of time. In particular, the activity (e.g., transmitted spikes) of the presynaptic neurons <NUM>, <NUM>, <NUM> can modulate or otherwise impact the membrane potential <NUM> of spiking neuron <NUM>. The spiking neuron <NUM> can also have an activation layer <NUM>, which controls the spiking of the neuron (e.g., a spike time of the neuron <NUM>) based on the membrane potential <NUM>.

As one example, the activation layer <NUM> can generate an action potential or spike when the membrane potential <NUM> crosses a firing threshold. Thus, in one example, implementing the spiking neuron <NUM> can include determining a spike time that corresponds to an earliest time at which the membrane potential <NUM> of the spiking neuron <NUM> is equal to a firing threshold.

When the spiking neuron <NUM> fires or spikes, a spike can be sent along one or more downstream synapses <NUM> to one or more downstream neurons. Alternatively, depending on the position of the neuron <NUM> in the model structure, the spike can be an output of the network. Although one downstream synapse <NUM> is shown, the spike output by the neuron <NUM> can be sent down any number of downstream synapses <NUM>.

Although not explicitly shown in <FIG>, various additional parameters can impact the behavior of the spiking neuron <NUM> such as, for example, synaptic delay parameter(s), bias parameter(s), and/or the like.

The activation layer <NUM> of the spiking neuron <NUM> uses a double exponential function to model a leaky input that an incoming neuron spike (e.g., an incoming spike from one of the presynaptic neurons <NUM>, <NUM>, <NUM>) provides to the membrane potential <NUM> of the spiking neuron <NUM>. In particular, this is obtained by integrating over time the incoming exponential synaptic current kernels of the form ε(t) = τ-<NUM>e-τt, where τ is the decay constant. The potential of the neuronal membrane in response to a single incoming spike is then of the form u(t) = te-τt. This function has a gradual rise and a slow decay, peaking at tmax = τ-<NUM>. Every synaptic connection has an efficiency, or a weight. The decay rate has the effect of scaling the induced potential in amplitude and time, while the weight of the synapse has the effect of scaling the amplitude only.

The use of the double exponential function in the neuron's activation layer <NUM> creates a better defined maximum in time. This allows very clearly defined state transitions between "now" and the "future step" to happen without loss of phase coherence.

More particularly, in some implementations, the double exponential function can model a leaky input as a double exponential pulse. A double exponential function can be any function that adheres to the following: e-At - e-Bt, with A < B, defined positive time t. For example, in some implementations, the double exponential function can take the form e-t(t - <NUM> + c), where c is a hyperparameter. In instances in which c is set equal to <NUM>, the double exponential function can take the form te-t. <FIG> provides an example plot of a leaky input modeled using a double exponential function of this form. In some implementations, the double exponential function can be mathematically modeled using Rall's alpha function. (See Rall, Distinguishing theoretical synaptic potentials computed for different soma-dendritic distributions of synaptic input. <NUM>) In some implementations, the double exponential function may be referred to as a "dual exponential" function.

Referring again to <FIG>, using the double exponential function illustrated in <FIG>, given a set of presynaptic neurons I (e.g., <NUM>, <NUM>, <NUM>) with weights wi (e.g., <NUM>, <NUM>, <NUM>) and spiking at respective time points ti, the membrane potential <NUM> at time t (if it has not yet spiked) can be expressed as <MAT>.

On the other hand, if a neuron has spiked, then there are several methods to "reset" it. One example is to restore the membrane potential to its default value and/or let the neuron be in a refractory period where it is unable to react to incoming stimuli.

Thus, the neuron <NUM> spikes when the membrane potential <NUM> crosses the firing threshold (see <FIG>). To compute the spike time tout of a neuron: determine the minimal subset of all presynaptic inputs Itout with ti ≤ tout which cause the membrane potential to reach the threshold θ while rising: <MAT>.

This can be achieved by sorting the inputs and adding them to Itout one by one, until an incoming input arrives later than the predicted spike (if any) or there are no more inputs. Note that the set I may not simply be computed as the earliest subset of presynaptic inputs that cause the membrane voltage to cross θ. If a subset of inputs I causes the membrane to cross θ at time tout, any additional inputs that occur between the maximum ti ∈ I and tout must be considered, and tout must be recomputed.

Eq. <NUM> has two potential solutions - one on the rising part of the function and one on the decaying part. If a solution exists (in other words, if the neuron spikes), then its spike time is the earlier of the two solutions.

For a set of inputs I, denote AI = Σi∈I wieτti and BI = Σi∈I wieτtiti. The spike time tout can be computed by solving Eq. <NUM> using the Lambert W function: <MAT>.

A spike will occur whenever the Lambert W function has a valid argument and the resulting tout is larger than all input spikes. As the earlier solution of this equation is valued, the main branch of the Lambert W function can be employed. The Lambert W function is real-valued when its argument is larger than or equal to -e-<NUM>. It can be proven that this is always the case when Eq. <NUM> has a solution, by expanding Vmem(tmax) ≥ θ, where <MAT> is the peak of the membrane potential function corresponding to the presynaptic set of inputs I.

<FIG> depicts example plots of the double exponential function different sets of weights w and decay constants τ. The weight scales the function in amplitude, whereas the decay constant scales it in both amplitude and time.

<FIG> depicts example plots of potential membrane dynamics in response to excitatory and inhibitory inputs, followed by a spike. In this example, τ = <NUM>, w = {<NUM>, -<NUM>, <NUM>, <NUM>, <NUM>, <NUM>}, t = {<NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>} and the spike occurs at tout = <NUM>.

One example spiking neural network architecture according to the present disclosure can include one or more (e.g., many) spiking neurons and/or non-spiking neurons. Some or all of the spiking neurons can have the structure and function illustrated in and described with respect to <FIG>.

In some implementations, the neurons of the spiking neural network can be arranged in multiple sequential layers, including, for example, multiple sequential layers that each include spiking neurons (e.g., a "deep" spiking neural network). In one particular example, one or more layers that include spiking neurons can be followed by one or more layers that include non-spiking neurons.

The spiking network can be a feed-forward network, a recurrent network, a convolutional network, or combinations thereof. Connections between neurons in adjacent layers can be structured in an all-to-all configuration and/or in a sparse configuration.

The spiking neural network encodes information in the spike times of spikes that are output by spiking neurons of the network. Thus, the information output of a neuron is encoded in its spike time, which depends on the timings and weights of presynaptic neurons that caused it to fire. This can enable the network to operate asynchronously. This better models the human brain and enables use of differential equations and backpropagation to adjust the spike timings in the network.

In some implementations, for example in a classification problem, the input class can be determined by which neuron in the output layer spikes first. In some implementations, each spiking neuron in the network is allowed to spike only once per cycle.

Further, in some implementations, use of an asynchronous network can enable multiple rhythms or flows of information (also known as "wavefronts") to propagate through the network at the same time, which can allow for parallel, sequential, and/or recurrent processing of input. For example, multiple wavefronts can propagate through the network at different phases (e.g., different but coherent phases). Propagation of wavefronts in this manner does not rely on synchronized clocking. Instead, the wavefront is itself the clocking. In some implementations, explicit clocking policies can be imposed at or around interfaces for data input and/or output.

In one particular example, the spiking neural network can be toroidal in structure. In such implementations, wavefronts can be cyclically propagated around the toroidal network with or without additional input, output, and/or other modifications (e.g., sequential input can be input over time at each cycle).

In some implementations, the spiking neural networks can be implemented in the form of computer-readable instructions stored in a computer-readable medium which are accessed and executed by one or more processors. Alternatively or additionally, the spiking neural networks can be implemented in the form of one or more electronic circuits that include electronic components arranged to execute the machine-learned spiking neural network using electrical current. As an example, the corresponding electronic components that model the double exponential function can include two capacitors, two resistors, and one or more transistors.

As one example training technique, backpropagation techniques can be used in combination with gradient-based techniques to backpropagate a loss through multiple layers of a network. For example, the loss can be a supervised loss of a loss function that evaluates the performance of the network on a set of labeled training data. Training the spiking neural network includes determining a gradient of a loss function that evaluates a performance of the spiking neural network on the set of training data; and modifying, for at least one of the plurality of spiking neurons, at least one of the one or more weights based at least in part on the gradient of the loss function.

As one example, the spiking network can learn to solve problems whose inputs and solution are encoded in the times of individual input and output spikes. Therefore, one possible goal is to adjust the output spike times so that their relative order is correct. Given a classification problem with n classes, the neuron corresponding to the correct label should be the earliest to spike. Therefore, one example loss function that can be used seeks to minimize the spike time of the target neuron and maximize the spike time(s) of the non-target neurons. Note that this is the opposite of the usual classification setting involving probabilities, where the value corresponding to the correct class is maximised and those corresponding to incorrect classes are minimised. As one example technique to achieve this effect, the softmax function can be used on the negative values of the spike times oi (which are always positive) in the output layer: <MAT>.

Cross-entropy loss can be used the usual form: <MAT>, where yi is an element of the one-hot encoded target vector of output spike times. Taking the negative values of the spike times ensures that minimizing the cross-entropy loss minimizes the spike time of the correct label and maximizes the rest.

In some implementations, determining the gradient of the loss function (e.g., the loss described above or other loss functions) can include determining, for at least one of the plurality of spiking neurons, a derivative of the spike time of such spiking neuron with respect to the weights associated with such spiking neuron.

As one example, to minmize the cross-entropy loss described above, a training system can change the value of the weights across the network. This has the effect of delaying or advancing spike times across the network. For any presynaptic spike arriving at time tj ∈ I with weight wj, denote <MAT> and compute the exact derivative of the postsynaptic spike time with respect to any presynaptic spike time tj and its weight wj as: <MAT> <MAT>.

As the postsynaptic spike time moves earlier or later in time, when Itout changes to include or exclude presynaptic spikes, the landscape of the loss function also changes. Furthermore, the loss function exhbits discontinuities where an output neuron stops spiking. This problem can be countered using a penalty, as described below. In practice, optimization is possible in spite of these challenges.

In some implementations, one or more synaptic delay parameters associated with the neuron can be trained using this gradient. As such, in some implementations, determining the gradient of the loss function can include determining, for at least one of the plurality of spiking neurons, a derivative of the spike time of such spiking neuron with respect to the weights associated with such spiking neuron and the inbound spike times associated with inbound spikes received by such neuron.

Additional example details regarding the derivation of the above gradient expressions are contained in <CIT>.

In some implementations, in order to adjust the class boundaries in the temporal domain, a temporal form of bias can be used to adjust spike times, i.e. to delay or advance them in time. In this model, synchronization pulses can act as additional inputs across some or all of the layers of the network, in order to provide temporal bias across the network. These can be thought of as similar to internally-generated rhythmic activity in biological networks, such as alpha waves in the visual cortex or theta and gamma waves in the hippocampus.

A set of pulses can be connected to all neurons in the network, to neurons within individual layers, or to individual neurons. A per-neuron bias is biologically implausible and more computationally demanding, hence some of the proposed models use either a single set of pulses per network, to solve easier problems, or a set of pulses per layer, to solve more difficult problems. All pulses can be fully connected to either all non-input neurons in the network or to all neurons of the non-input layer they are assigned to.

Each pulse can spike at a predefined and trainable time, providing a reference spike delay. Each set of pulses can be initialized to spike at times evenly distributed in the interval (<NUM>,<NUM>). Subsequently, the spike time of each pulse can be learned using Eq. <NUM>, while the weights between pulses and neurons are trained using Eq. <NUM>, in the same way as all other weights in the network.

Example experiments were conducted on fully connected feedforward networks with topology n_hidden (a vector of hidden layer sizes). Adam optimization was used with mini-batches of size batch_size to minimise the cross-entropy loss. The Adam optimizer performed better than stochastic gradient descent. Different learning rates were used for the pulse spike time (learning_rate_pulses) and the weights of both pulse and non-pulse neurons (learning_rate). A fixed firing threshold (fire_threshold) and decay constant (decay_constant) were used.

Network weight initialisation is crucial for the subsequent training of the network. In a spiking network, it is important that the initial weights are large enough to cause at least some of the neurons to spike; in absence of spike events, there will be no gradient to use for learning. Therefore, in some implementations, a modified form of Glorot initialization can be used where the weights are drawn from a normal distribution with standard deviation <MAT> (as in the original scheme) and custom mean µ = multiplier × σ. If the multiplication factor of the mean is <NUM>, this is the same is the original Glorot initialization scheme. Different multiplication factors can be set for pulse (pulse_init_multiplier), and non-pulse (nonpulse_init_multiplier) weights. This allows the two types of neurons to pre-specialise into inhibitory and excitatory roles. In biological brains, internal oscillations are thought to be generated through inhibitory activities that regulate the excitatory effects of incoming stimuli.

Some example possible hyperparameters of the model are shown in the table below. The first column shows the default parameters chosen to solve Boolean logic problems. The second column shows the search range used in the hyperparameter search. Asterisks ( *) mark ranges that were probed according to a logarithmic scale; all others were probed linearly. The last column shows the value chosen from these ranges to solve an example MNIST-based experiment.

Despite careful initialization, in some instances, the network might still become quiescent during training. This problem can be prevented by adding a fixed small penalty (penalty _no_spike) to the derivative of all presynaptic weights of a neuron that has not fired. In practice, after the training phase, some of the neurons will spike too late to matter in the classification and thus they do need to spike at all.

Another problem is that the gradients become very large as a spike becomes closer to, but not sufficient for the postsynaptic neuron to reach the firing threshold. In this case, in Eq. <NUM> and <NUM>, the value of the Lambert W function will approach its minimum (-<NUM>) as its argument approaches -e-<NUM>, the denominator of the derivatives will approach zero and the derivatives will approach infinity. To counter this, the derivatives can be clipped to a fixed value clip_derivative. Note that this behavior will occur in any activation function that has a maximum (hence, a biologically-plausible shape), is differentiable, and has a continuous derivative.

In addition to these hyperparameters, several other heuristics for the spiking net can optionally be used. These include weight decay, adding random noise during training to the spike times of either the inputs or all non-output neurons in the network, averaging over brightness values in a convolutional-like manner and adding additional input neurons responding to the inverted version of the image, akin to the on/off bipolar cells in the retina. Additionally, in some implementations, presynaptic neurons can be removed from the presynaptic set once their individual contribution to the potential decayed below a decay threshold. This can be achieved by solving an equation similar to Eq. <NUM> for reaching a decay threshold on the decaying part of the function, using the -<NUM> branch of the Lambert W function.

<FIG> depicts a block diagram of an example computing system <NUM> according to example embodiments of the present disclosure. The system <NUM> includes a user computing device <NUM>, a server computing system <NUM>, and a training computing system <NUM> that are communicatively coupled over a network <NUM>.

The user computing device <NUM> can be any type of computing device, such as, for example, a personal computing device (e.g., laptop or desktop), a mobile computing device (e.g., smartphone or tablet), a gaming console or controller, a wearable computing device, an embedded computing device, or any other type of computing device.

The user computing device <NUM> includes one or more processors <NUM> and a memory <NUM>. The one or more processors <NUM> can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, a FPGA, a controller, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected. The memory <NUM> can include one or more non-transitory computer-readable storage mediums, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory <NUM> can store data <NUM> and instructions <NUM> which are executed by the processor <NUM> to cause the user computing device <NUM> to perform operations.

In some implementations, the user computing device <NUM> can store or include one or more spiking neural networks <NUM>. For example, the spiking neural networks <NUM> can be or can otherwise include spiking neurons as described herein. Neural networks can include feed-forward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks), convolutional neural networks or other forms of neural networks. Example spiking neural networks <NUM> are discussed with reference to <FIG> and <FIG>.

In some implementations, the one or more spiking neural networks <NUM> can be received from the server computing system <NUM> over network <NUM>, stored in the user computing device memory <NUM>, and then used or otherwise implemented by the one or more processors <NUM>. In some implementations, the user computing device <NUM> can implement multiple parallel instances of a single spiking neural network <NUM>.

Additionally or alternatively, one or more spiking neural networks <NUM> can be included in or otherwise stored and implemented by the server computing system <NUM> that communicates with the user computing device <NUM> according to a client-server relationship. For example, the spiking neural networks <NUM> can be implemented by the server computing system <NUM> as a portion of a web service. Thus, one or more networks <NUM> can be stored and implemented at the user computing device <NUM> and/or one or more networks <NUM> can be stored and implemented at the server computing system <NUM>.

The user computing device <NUM> can also include one or more user input component <NUM> that receives user input. For example, the user input component <NUM> can be a touch-sensitive component (e.g., a touch-sensitive display screen or a touch pad) that is sensitive to the touch of a user input object (e.g., a finger or a stylus). The touch-sensitive component can serve to implement a virtual keyboard. Other example user input components include a microphone, a traditional keyboard, or other means by which a user can provide user input.

The server computing system <NUM> includes one or more processors <NUM> and a memory <NUM>. The one or more processors <NUM> can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, a FPGA, a controller, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected. The memory <NUM> can include one or more non-transitory computer-readable storage mediums, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory <NUM> can store data <NUM> and instructions <NUM> which are executed by the processor <NUM> to cause the server computing system <NUM> to perform operations.

As described above, the server computing system <NUM> can store or otherwise include one or more machine-learned spiking neural networks <NUM>. For example, the networks <NUM> can be or can otherwise include various machine-learned models. Example machine-learned models include neural networks or other multi-layer non-linear models. Example neural networks include feed forward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks. Example networks <NUM> are discussed with reference to <FIG> and <FIG>.

The user computing device <NUM> and/or the server computing system <NUM> can train the networks <NUM> and/or <NUM> via interaction with the training computing system <NUM> that is communicatively coupled over the network <NUM>. The training computing system <NUM> can be separate from the server computing system <NUM> or can be a portion of the server computing system <NUM>.

The training computing system <NUM> includes one or more processors <NUM> and a memory <NUM>. The one or more processors <NUM> can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, a FPGA, a controller, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected. The memory <NUM> can include one or more non-transitory computer-readable storage mediums, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory <NUM> can store data <NUM> and instructions <NUM> which are executed by the processor <NUM> to cause the training computing system <NUM> to perform operations. In some implementations, the training computing system <NUM> includes or is otherwise implemented by one or more server computing devices.

The training computing system <NUM> can include a model trainer <NUM> that trains the machine-learned networks <NUM> and/or <NUM> stored at the user computing device <NUM> and/or the server computing system <NUM> using various training or learning techniques, such as, for example, backwards propagation of errors. In some implementations, performing backwards propagation of errors can include performing truncated backpropagation through time. The model trainer <NUM> can perform a number of generalization techniques (e.g., weight decays, dropouts, etc.) to improve the generalization capability of the models being trained.

In particular, the model trainer <NUM> can train the spiking neural networks <NUM> and/or <NUM> based on a set of training data <NUM>. In some implementations, the model trainer <NUM> can performed supervised learning techniques to train the networks based on the training data <NUM>. The model trainer <NUM> can perform any of the techniques or operations described in the Example Training Techniques section above.

In some implementations, if the user has provided consent, the training examples can be provided by the user computing device <NUM>. Thus, in such implementations, the network <NUM> provided to the user computing device <NUM> can be trained by the training computing system <NUM> on user-specific data received from the user computing device <NUM>. In some instances, this process can be referred to as personalizing the model.

The model trainer <NUM> includes computer logic utilized to provide desired functionality. The model trainer <NUM> can be implemented in hardware, firmware, and/or software controlling a general purpose processor. For example, in some implementations, the model trainer <NUM> includes program files stored on a storage device, loaded into a memory and executed by one or more processors. In other implementations, the model trainer <NUM> includes one or more sets of computer-executable instructions that are stored in a tangible computer-readable storage medium such as RAM hard disk or optical or magnetic media.

<FIG> illustrates one example computing system that can be used to implement the present disclosure. Other computing systems can be used as well. For example, in some implementations, the user computing device <NUM> can include the model trainer <NUM> and the training dataset <NUM>. In such implementations, the networks <NUM> can be both trained and used locally at the user computing device <NUM>. In some of such implementations, the user computing device <NUM> can implement the model trainer <NUM> to personalize the networks <NUM> based on user-specific data.

<FIG> depicts a block diagram of an example computing device <NUM> according to example embodiments of the present disclosure. The computing device <NUM> can be a user computing device or a server computing device.

<FIG> depicts a block diagram of an example computing device <NUM> according to example embodiments of the present disclosure. The computing device 1can be a user computing device or a server computing device.

The central intelligence layer includes a number of machine-learned models. For example, as illustrated in <FIG>, a respective machine-learned model (e.g., a model) can be provided for each application and managed by the central intelligence layer. In other implementations, two or more applications can share a single machine-learned model. For example, in some implementations, the central intelligence layer can provide a single model (e.g., a single model) for all of the applications. In some implementations, the central intelligence layer is included within or otherwise implemented by an operating system of the computing device <NUM>.

Claim 1:
A computer system (<NUM>, <NUM>) comprising:
one or more processors (<NUM>, <NUM>); and
one or more non-transitory computer readable media (<NUM>, <NUM>) that collectively store:
a machine-learned spiking neural network (<NUM>, <NUM>) that comprises one or more spiking neurons that have an activation layer that uses a double exponential function to model a leaky input that an incoming neuron spike provides to a membrane potential of the spiking neuron; and
instructions (<NUM>, <NUM>) that, when executed by the one or more processors, cause the computer system to perform operations, the operations comprising:
obtaining a network input;
implementing the machine-learned spiking neural network (<NUM>, <NUM>) to process the network input; and
receiving a network output generated by the machine-learned spiking neural network (<NUM>, <NUM>) as a result of processing the network input;
wherein the network input comprises audio data and the network output is a speech recognition output; or
wherein the network input comprises audio or video data and the network output comprises a classification of the network input.