Patent ID: 12210957

DETAILED DESCRIPTION

The invention can be implemented in numerous ways, including as a process; an apparatus; a system; a composition of matter; a computer program product embodied on a computer readable storage medium; and/or a processor, such as a processor configured to execute instructions stored on and/or provided by a memory coupled to the processor. In this specification, these implementations, or any other form that the invention may take, may be referred to as techniques. In general, the order of the steps of disclosed processes may be altered within the scope of the invention. Unless stated otherwise, a component such as a processor or a memory described as being configured to perform a task may be implemented as a general component that is temporarily configured to perform the task at a given time or a specific component that is manufactured to perform the task. As used herein, the term ‘processor’ refers to one or more devices, circuits, and/or processing cores configured to process data, such as computer program instructions.

A detailed description of one or more embodiments of the invention is provided below along with accompanying figures that illustrate the principles of the invention. The invention is described in connection with such embodiments, but the invention is not limited to any embodiment. The scope of the invention is limited only by the claims and the invention encompasses numerous alternatives, modifications and equivalents. Numerous specific details are set forth in the following description in order to provide a thorough understanding of the invention. These details are provided for the purpose of example and the invention may be practiced according to the claims without some or all of these specific details. For the purpose of clarity, technical material that is known in the technical fields related to the invention has not been described in detail so that the invention is not unnecessarily obscured.

A method for performing learning is described. Using a learning network, a free inference is performed for input signals. The input signals correspond to target output signals. The learning network includes inputs that receive the input signals, weights, neurons, and outputs. The weights interconnect the neurons. The learning network is also described by an energy for the free inference. The energy includes an interaction term corresponding to interactions between the neurons. The interaction term consists of neuron pair interactions. The free inference results in output signals for the outputs. A first portion of the weights corresponds to data flow for the free inference. A biased inference is performed using the learning network by providing the input signals to the inputs and providing bias signals to the outputs. The bias signals are based on the target output signals and the output signals from the free inference. The bias signals are fed back to the learning network through a second portion of weights corresponding to a transpose of the first portion of the weights. At multiple locations in the learning network, learning network equilibrium states are determined for the for the biased inference. The weights are updated based on the learning network equilibrium states.

In some embodiments, the interaction term includes a quadratic portion. The quadratic portion is proportional to −Σi=1NΣj=1NWijuiuj, where Wijis a weight of the plurality of weights between neuron i of the plurality of neurons and neuron j of the plurality of neurons, uiis a neuron output signal of neuron i, ujis a neuron output signal of neuron j, and N is a total number of neurons.

The neurons may be arranged in neuron layers. The weights may be arranged in at least one weight layer. The weight layer(s) are interleaved with the neuron layers. In some embodiments, each of the neurons receives a neuron input signal and performs a thresholding function for the neuron input signal to provide a neuron output signal. The thresholding function may be a hysteretic thresholding function. In some embodiments, performing the free inference, providing the input signals and bias signals to perform the biased inference, determining the learning network equilibrium states, and updating the weights are iteratively performed. Performing the free inference, providing the input signals and bias signals to perform the biased inference, determining the learning network equilibrium states, and updating the plurality of weights may be iteratively performed such that the hysteretic thresholding function amplifies noise for a first portion of the iterations and suppresses noise for a second portion the iterations.

In some embodiments, the input signals are part of a plurality of split input signals. In such embodiments, the performing, providing, determining, and updating steps are also performed for a remaining portion of the split input signals. The method also includes concatenating final output signals for the plurality of split input signals.

A learning network is described. The learning network includes a vector matrix multiplication (vector MM) unit and a neuron layer. The vector MM unit is programmable and sparsely coupled. A first portion of the vector MM unit includes weights corresponding to a weight matrix. A second portion of the vector MM unit includes the weights and corresponds to a transpose of the weight matrix. The neuron layer includes inputs, outputs, and neurons coupled between the inputs and the outputs. The neuron layer is coupled with the vector MM unit such that the inputs receive weighted input signals from the first portion of the vector MM unit and such that the outputs provide neuron output signals to the second portion of the vector MM unit. The weights connect neuron pairs. The learning network is configured to receive input signals corresponding to target outputs, to provide learning network output signals in response to the input signals, to receive bias signals based on the target outputs and the learning network output signals, and to update the weights based on the bias signals, the target outputs, and the learning network output signals.

In some embodiments, the learning network is described by an energy including an interaction term corresponding to interactions between the neurons in the neuron layer. The interactions consist of neuron pair interactions. Thus, in some embodiments, the interaction term includes a quadratic portion only. The quadratic portion may be proportional to −Σi=1NΣj=1NWijuiuj, where Wijis a weight of the plurality of weights between neuron i of the plurality of neurons and neuron j, uiis a neuron output signal of neuron i, ujis a neuron output signal of neuron j, and N is a total number of neurons, the inference resulting in a plurality of output signals for the plurality of outputs.

In some embodiments, each of the neurons receives a weighted input signal and performs a thresholding function to provide a neuron output signal. The thresholding function may be a hysteretic thresholding function. In some embodiments, the hysteretic thresholding function amplifies noise for a first portion of a plurality of iterations and suppresses noise for a second portion the plurality of iterations.

The vector MM unit may include a sparsely coupled crossbar array. A first portion of the sparsely coupled crossbar array corresponds to the first portion of the vector MM unit, while a second portion of the sparsely coupled crossbar array corresponds to the second portion of the vector MM unit. In another embodiment, the vector MM unit may include a sparsely coupled crossbar array to which the neuron layer is coupled such that the inputs of the neuron layer receive the weighted input signals from a matrix configuration of the vector MM unit. The neuron layer is also coupled to the sparsely coupled crossbar array such that the neuron output signals are provided to the sparsely coupled crossbar array in a transposed matrix configuration. The vector MM unit may be expanded to have multiple first portions and multiple second portions corresponding to different weight matrices and their transposes. Each set of first and second portions correspond to a different neuron layer.

In some embodiments, the input signals are a portion of a number of split input signals. In such embodiments, the learning network further includes a splitter and a concatenation unit coupled with the vector MM unit. The splitter selects the input signals from the split input signals. The concatenation unit is also coupled with the neuron layer and combines the neuron output signals.

A learning network including system inputs, at least one vector matrix multiplication (MM) unit coupled with the learning network inputs, a plurality of neuron layers, and a plurality of system outputs, The system inputs are configured to receive input signals. The vector MM unit(s) are coupled with the system inputs. The vector MM unit(s) include a programmable and sparsely coupled crossbar array. Each of the vector MM unit(s) includes a first portion and a second portion The first portion includes weights corresponding to a weight matrix. The second portion includes weights corresponding to a transpose of the weight matrix. The at least one vector MM unit is interleaved and coupled with the neuron layers. Thus, the vector MM unit(s) may be coupled with the system input(s) directly or through one or more neuron layers. Each of the neuron layers includes inputs, outputs, and neurons coupled between the inputs and the outputs. Each of at least a portion of the neuron layers is coupled such that the inputs receive weighted signals from the first portion of a vector MM unit and such that the outputs provide neuron output signals to the second portion of the vector MM unit. In some embodiments, the inputs of the first neuron layer correspond to or are coupled to the system inputs. In some embodiments, the outputs of the last neuron layer correspond to or are coupled to the system outputs. Thus, the system outputs are coupled with a final neuron layer of the neuron layers. The system outputs are configured to provide output signals based on the input signals and to receive bias signals. The input signals correspond to target outputs. The bias signals are based on the target outputs and the output signals. The weights are configured to be updated based on the bias signals, the target outputs, and the output signals. In some embodiments, each of the neurons performs a thresholding function to provide a neuron output signal. In some such embodiments, the thresholding function includes a hysteretic thresholding function.

FIG.1depicts an embodiment of learning network100. Learning network100may be an artificial neural network that may be trained via machine learning. Learning network100includes multiple neuron layers120-1,120-2,120-3, and120-4(collectively or generically120) of neurons that are interleaved with vector matrix multiplication (MM) units110-1,110-2, and110-3(collectively or generically110). Learning network100also includes system inputs102, system outputs104, and bias130. Also shown are inputs102,122-2,122-3, and122-4(collectively or generically inputs122) and outputs124-1,124-2,124-3, and104(collectively or generically outputs124) for each neuron layer120. In the embodiment shown, the system inputs102are inputs for first neuron layer120-1. Similarly, system outputs104are the outputs for the last neuron layer120-4. System inputs102, system outputs104, bias130, inputs122, and outputs124are shown as single arrows. In general, however, multiple system inputs, multiple system outputs, multiple bias lines, multiple inputs for each neuron layer, and multiple outputs from each neuron layer are present. Moreover, the arrows illustrate information flow during a typical inference (discussed below). However, information may flow in both directions. For example, in a biased inference, information may flow in the reverse direction as well as in the forward direction. Although four neuron layers120and three vector MM units110are shown, another number of neuron layer(s) and/or another number of vector MM units may be used. For example, a learning network may include two neuron layers having a single vector MM unit between the layers. In another example, ten neuron layers interleaved with nine vector MM units may be used. Further, although system inputs102are shown as being connected provide to first neuron layer120-1, in some embodiments, a vector MM unit (not shown) may be connected system inputs102. In such embodiments, the inputs122to neuron layer120-1are the outputs of such a vector MM unit.

Vector MM units110may be considered to be analogous to synapses in a biological neural network. Each vector MM unit110includes programmable components, such as a memristors, nonvolatile memory cells, and/or other programmable resistors, for which the impedance (or resistance) can be adjusted. The impedances can be considered to be the programmable weights applied to signals by vector MM units110. In some embodiments, the programmable components of vector MM units110are sparsely connected (not all weights connected to all of its neighbors). In other embodiments, the programmable components of vector MM units110may be fully connected (each weight or programmable components is connected to all of its neighbors). In some embodiments, the connections between programmable components of vector MM units110are programmable. Each of vector MM units110may be the same as or different from other vector MM units110. For example, vector MM unit110-2may include a different number of input lines, outputs, and weights than vector MM unit110-1. In another example, vector MM unit110-2may have the weights connected differently than vector MM unit110-1whether or not the vector MM units110-1and110-2have the same number of weights. The configuration of each vector MM unit110depends upon factors such as the number and configuration of neurons in the corresponding neuron layer120. In some embodiments, a separate component is used to update the programmable components (i.e. weights) in vector MM units110. In some embodiments, vector MM units110includes logic used to update the programmable components. In some embodiments, vector MM units110are crossbar arrays. A first portion of each vector MM unit110may be configured such that the weights (i.e. programmable components) correspond to a weight matrix. A second portion of each vector MM unit110includes the weights and corresponds to a transpose of the weight matrix. Thus, the first portion of each vector MM unit110may provide a vector matrix multiplication for the weights, corresponding to information flow in the forward direction for an inference (and biased inference). The second portion of each vector MM unit110may provide a vector matrix multiplication for the transpose of the weights, corresponding to information flow in the reverse direction for a biased inference.

Each neuron layer120includes inputs122, outputs124, and neurons (not explicitly depicted inFIG.1) coupled between inputs122and the outputs124. The neurons of neuron layer120receive weighted input signals from corresponding vector MM units110, combine the weighted input signals based on function(s) for the neurons, and provide one or more resulting output signals on outputs124. In some embodiments, neuron layers120are coupled with the previous vector MM unit110such that inputs122receive weighted input signals from the first portion of the previous vector MM unit110and such that outputs124provide neuron output signals to the second portion of vector MM unit110. The weights in each vector MM unit110connect neuron pairs. Neuron layer120-1receives (unweighted) input signals via inputs102and combines the input signals based on a function of input signals. Neuron layers120-1,120-2and120-3provide their output signals to another layer of vector MM units110. Neuron layer120-4provides the output signals for learning network100. In some embodiments, neuron layers120are the same. In other embodiments, neuron layer(s)120may differ from other neuron layer(s)120. For example, different numbers of neurons may be present in different neuron layers120. Similarly, the function(s) used by neurons in a neuron layer120to operate on input signals may be the same as or different from the function(s) used in other neuron layers. In some embodiments, function(s) used by neurons in each neuron layer120include or consist of a thresholding function, as discussed below. Stated differently, the state (and thus output signal) of a neuron may be determined based on whether the input signal meets or exceeds particular threshold(s). For example, a simple thresholding function may be such that if the input signal (e.g., a current) to a particular neuron is greater than a threshold, θ, then the neuron has an output of logical +1 (e.g., a voltage of +V or current of +I). If the input signal is less than the threshold, the neuron may have an output signal of −1 (e.g., a voltage of −V or current of −I).

Also shown in learning network100is bias130. Bias130provides bias signals to system outputs104. Bias130is indicated by a dashed line because the bias signals are selectively provided to system outputs104. Stated differently, bias signals are only sometimes provided to system outputs104. In some embodiments, this may include selectively coupling (or decoupling) bias130with system outputs104.

Learning network100is configured to receive input signals corresponding to target outputs, to provide learning network output signals in response to the input signals, to receive bias signals based on the target outputs and the learning network output signals, and to update the weights based on the bias signals, the target outputs, and the learning network output signals. Learning network100is configured such that training can be accomplished without explicitly calculating gradients as required by back-propagation. More specifically, learning network100may be trained using activity-difference training.

Activity-difference training uses locally available information within the learning network to identify weight changes, without needing to explicitly calculate errors and propagate them across learning network100. Activity-difference training is typically numerical and probabilistic in nature, and has strong recent evidence as being biologically plausible. Activity-difference training involves feeding learning network100training data as input signals to system inputs102in a “free” phase (in the absence of biasing provided by bias130to system outputs104). Thus, a free inference may be considered to be performed. The input signals for the training data have corresponding target output signals (i.e. the desired output signals for the input signals). In another, bias phase, bias signals are provided to system outputs104via bias130such that system outputs104are closer or at the target output signals in response to the input signals being provided to system inputs102. Thus, a biased inference may be considered to be performed. In some embodiments, system outputs104are nudged to be closer to the target output signals for the input signals provided to system inputs102. In some embodiments, the bias signals clamp system outputs104at the target output signals during the bias phase. In some embodiments, the bias signal may be proportional to the desired target output. In other embodiments, the bias signal may be proportional to the error (the difference between the desired output and the free phase output). Thus, the bias signals provided via bias130may “nudge” or “clamp” output signals to be closer to or at the target output signals. The difference between the free and the bias phases encode the weight changes desired. For example, the desired weight changes may be determined by monitoring various portions of learning network100during the free and bias phases to determine the state of particular nodes of learning network. The weights in vector MM units110may be adjusted in accordance with the difference between the states during the free and bias phases. This process of carrying out the free phase, bias phase, and monitoring of the network may be iteratively implemented until equilibrium (or a state sufficiently close to equilibrium is reached).

Learning network100is configured such that it may be considered to be described by an “energy”. A global minimum in the energy represents the ideal trained configuration. Thus, the term energy is intended to relate to training of learning network100rather than physical forms of energy (e.g., energy dissipated by learning network100in the form of heat). The energy may also be considered to describe the operation of neurons in neuron layers120as well as weights in vector MM units110on input signals that results in the corresponding output signals. The energy includes the function which the neurons in neuron layers120utilize to provide output signals (e.g., the state of the neuron) as well as the weights for vector MM units110. Activity-difference training utilizes local information to approach or reach the global minimum in the energy. For example,FIG.2depicts an embodiment of the energy210in a space in which the horizontal plane corresponds to weights and the vertical axis corresponds to the error from the ideal trained configuration. A particular form of activity-difference training occurs along curve220in the surface formed by energy210. Using activity-difference training, the state of learning network100may proceed from222, to224and to at or near the global minimum at226.

Referring back toFIG.1, for learning network100, the energy includes an interaction term that corresponds to interactions between the neurons in the neuron layers. The interactions consist of neuron pair interactions. Thus, in some embodiments, the interaction term includes a quadratic portion only. This interaction component may be an Ising component for pairs of neuron interacting via components of a weight matrix. The quadratic portion of the interaction component of the energy may be proportional to −Σi=1NΣj=1NWijuiuj, where Wijis a weight of vector MM unit110between neuron i and neuron j, uiis a neuron output signal (or state) of neuron i in neuron layers120, ujis a neuron output signal (or state) of neuron j in neuron layers120, and N is a total number of neurons. In some embodiments, therefore, the energy, E, for learning network100is given by:

E=-12⁢∑i=1N⁢∑j=1N⁢Wi⁢j⁢ui⁢uj-∑i=1N⁢bi⁢ui+∑i=1N⁢θi⁢ui(1)
The energy could also be considered to take the form:

E=∑i=1N⁢12⁢ui2-12⁢∑i=1N⁢∑j=1N⁢Wi⁢j⁢ui⁢uj-∑i=1N⁢bi⁢ui+∑i=1N⁢θi⁢ui(2)
In the energy formulation of both equations (1) and (2), the Ising component represents the interaction between pairs of neuron (represented by their states uiand uj), which interact via components of weight matrix W of vector MM units110. Individual, neurons have a threshold θ and a bias b. Such energies can be accounted for by the functions for individual neurons. The first term in equation (2) represents a status of a single neuron independent of interactions with other neurons. The linear terms in the energy may be accounted for in the configuration of the neurons. Consequently, the Ising component may be considered to be the energy of interest represented by energy210inFIG.2.

The energies of equations (1) and (2) correspond to the free phase, or free inference. For the biased phase, an additional term corresponding to the neuron bias signal due to the bias signal provided via bias130is added. The bias term may be considered to have the form:

+∑i=1N⁢β2⁢(ui-di)2(3)
In the bias phase, diis the desired output signal of neuron i. In the embodiment described by equation 3, the bias term corresponds to the nudge applied via bias130and consists of a positive penalty to the energy for any deviation of the neuron's state uifrom the desired output di. In another embodiment, the bias signal may also be proportional to di. Other functions of the desired output might be used for the bias signal in other embodiments. During the biased phase, the total energy is the sum of equations (1) and (3) (or equations (2) and (3)).

The neurons in the above formulation may be continuous or discrete. For continuous neurons, the output state, ui, of neuron i is a continuous function of the input. For a discrete neuron, the state, ui, of neuron i, is not continuous. Further, in some embodiments, the neuron utilizes a thresholding function.

FIGS.3A-3Edepict thresholding functions and how they may operate in learning networks such as learning network100.FIGS.3A,3B, and3Cdepict embodiments thresholding functions300A,300B, and300C, respectively, that may be used for neurons in learning network100or an analogous learning network.FIGS.3D-3Eindicate operation of the neurons using the thresholding functions. Referring toFIG.3A, graph300A depicts a simple thresholding function. When the neuron input signal is below the threshold (w), the neuron provides one output signal. When the neuron input signal is above the threshold, the neuron provides another output signal. The transition at the threshold is indicated by the dual-headed arrow crossing the input axis inFIG.3A.

Referring toFIG.3B, graph300B depicts a hysteretic thresholding function with a threshold of +|w| (i.e. the threshold, w, is positive). When the neuron input signal is increasing and passes the threshold +w, the neuron output signal transitions from low to high. When the neuron input signal is decreasing and passes the threshold −w, the neuron output signal transitions from high to low. Such a thresholding function suppresses noise in learning network100. These transitions are indicated by the arrows crossing the input axis inFIG.3B. This can be understood by considering a neuron input signal near zero. If the neuron input signal increases or decreases slightly, the neuron output signal does not change. Thus, noise is suppressed.

In contrast, graph300C depicts a hysteretic thresholding function which enhances noise. The hysteretic thresholding function has a threshold of −|w| (i.e. the threshold, w, is negative). When the neuron input signal is increasing and passes the threshold −|w|, the neuron output signal transitions from low to high. When the neuron input signal is decreasing and passes (i.e. becomes less than) the threshold +|w|, the neuron output signal transitions from high to low. Such a thresholding function enhances noise in learning network100. This can be seen by considering a neuron input signal near zero. If the neuron input signal increases or decreases slightly, the neuron output signal transitions between the low and high output. Thus, fluctuations are enhanced and may be created if there are no other induced fluctuations.

The effect of noise for neurons having positive and negative thresholds is indicated in graph300D ofFIG.3D. Input signal310crosses the threshold, ±[w], at multiple times and exhibits noise. The neuron output signal312has a positive threshold (+|w|). More specifically, neuron output signal312transitions from low to high for input signal310increasing and passing threshold +|w|. Neuron output signal312transitions from high to low for input signal310decreasing and passing threshold −|w|. Neuron output signal312thus suppresses noise. The neuron output signal314has a negative threshold (−|w|). More specifically, neuron output signal314transitions between low and high multiple times for input signal310being between −|w| and +|w|. Neuron output signal314is stable only for input signal310being less that −|w| or greater than +|w|. Neuron output signal314thus enhances noise.

Hysteretic thresholding functions analogous to those of graphs300B and300C may improve performance of learning network100and/or analogous learning networks. Use of such thresholding functions can mitigate noise and improve training such that learning network100is more likely to settle at a global minimum in the energy. For example, early iterations, hysteretic functions having negative thresholds analogous to that shown inFIG.3Cmay be used for neurons in neuron layers110. As a result, fluctuations due to intrinsic noise are enhanced and activity-difference training of learning network may not become trapped in a local minimum of the energy. As the activity-difference training progresses, it is more likely that learning network100is approaching a global minimum. Thus, the neurons of neuron layer(s)110may be updated to use the hysteretic thresholding function having positive thresholds, analogous to that depicted in graph300B. Consequently noise may be suppressed and learning network100may more rapidly approach the global minimum in the energy during activity-difference training. Such a case is indicated inFIG.3E. Neuron input signal322has intrinsic noise, which is enhanced by utilizing thresholding function(s) have negative threshold(s) for neurons during starting iterations. Later, the thresholding functions are transitioned to positive threshold(s). This suppresses noise.

Thus, the term in the energy corresponding to interaction between the neurons for learning network100is the Ising term. As a result, vector MM units110may be configured as a simple crossbar array of programmable resistors. Such a crossbar array may be fully or sparsely coupled. The remaining terms are linear and can be implemented by manipulating the neuron function. Thus, the energy describing learning network100and for which activity-difference training is desired to be performed may be directly mapped to available hardware and allows for simplified implementation via clocking (discrete-time) and use of simple binary neuron functions. As a result, training of learning network100may be simplified and may utilize less power. Training may be further improved by selecting and updating the functions used by neurons in learning network100to first amplify, then later suppress noise. Thus, performance of learning networks such as learning network100may be improved. Further, such a learning network may be readily implemented, for example utilizing analog crossbars of multi-bit programmable memristive switches in vector MM units110. Other types of electronic switches such as flash memory, oxide-based memristors, phase change memories, ferroelectric memory, may also be used in some embodiments. Thus, a learning network100having improved performance and that is readily fabricated may be achieved.

FIG.4depicts an embodiment of method400for performing activity-difference training on a learning network, such as learning network100. Although particular steps are depicted, method400may include additional and/or other steps. Similarly, steps of method400may include substeps. Method400is described in the context of learning network100. However, nothing prevents method400from being used in conjunction with other networks having an analogous energy function. For example, other learning networks including neurons (e.g., neurons that utilize thresholding functions) and/or vector MM units may be used.

A free inference is performed, at402. To perform the free inference, input signals are provided to the system inputs of a learning network. Because these input signals are part of training data, target output signals correspond to the input signals. The learning network includes system inputs that receive the input signals, neurons, weights interconnecting the neurons, and system outputs that provide output signals. The learning network used for method400is also described by an energy analogous to that described above for learning network100. For example, the interaction term consists of neuron pair interactions and is a quadratic term analogous described above. In some embodiments, neurons respond to inputs based on thresholding functions, such as hysteretic thresholding functions. A first portion of the weights in the learning network corresponds to data flow for the free inference. The free inference results in output signals provided at the system outputs. The outputs signals are a result of the functions performed by the neurons and the interactions between the neurons (e.g., the output signals are based on the energy). The output signals are typically different from the target outputs.

A biased inference is performed by the learning network, at404. Performing the biased inference includes the same input signals being provided to the system inputs of the learning network and providing bias signals to the system outputs of the learning network. The bias signals are based on the target output signals and the output signals of the free inference. The bias signals nudge the output signals to be closer to (or clamp the output signals to be at) the target outputs. The bias signals are fed back to the learning network through a second portion of the weights in the learning network. The second portion of the weights corresponds to a transpose of the first portion of the weights.

At locations in the learning network, a learning network equilibrium states are determined for the biased inference and for the free inference, at406. Although indicated as being performed after402and404,406is performed in parallel with402and404. Thus, the states at which locations in the learning network settle (i.e. equilibrium states for the locations) for the free inference are determined at406during or shortly after402. Similarly, the equilibrium states at which the same locations settle for the for the biased inference are determined at406during or shortly after404. Locations at which the equilibrium states are determined may include but may not be limited to the inputs to each neuron layer (i.e. the outputs of a corresponding weight layer) and/or the outputs from each neuron layer (i.e. the inputs to a next weight layer).

The weights are adjusted based on the learning network equilibrium states, at408. The adjustment is based on the equilibrium states determined at406. The update performed at408is thus based upon the target output signals, output signals, and input signals. In some embodiments,402,404,406, and408may be considered to form one iteration, or epoch, for learning using method400.

The functions utilized by the neurons in the learning network to provide their neuron output signals may be updated, at410. For example, for a first iteration of method400, the neurons may employ thresholding function(s) having a negative threshold(s) in order to amplify noise. At some subsequent iteration, the functions performed by the neurons may be changed to thresholding function(s) having positive threshold(s) to suppress noise.

In some embodiments,402,404,406,408, and410are iteratively repeated, at412. Iterations may continue until a particular milestone is reached. For example, method400may terminate in response to the changes to the weights in408being at or below some threshold or some other measure of the learning network reaching an equilibrium status being achieved. In some embodiments, a measure of how close the output signals are to the target output signals may be used. In some embodiments, a specified number of iterations may be used to determine when to terminate the method. Thus, method400is repeated until it is determined via412that training is complete.

For example, if method400is used in connection with learning network100, input signals are provided to system inputs102and a free inference performed. A biased inference is also performed at404by providing the input signals to system inputs102and providing the bias signals to system outputs104via bias130. Equilibriums states at various locations in learning network are sampled for the free and biased inferences, at406. For example, inputs122-2,122-3, and122-4to neuron layers120as well as outputs124-1,124-2, and124-3may be monitored during or after the free and biased inferences. At408, weights in vector MM units110are adjusted based on the equilibrium states determined at406, as well as based on the target output signals and the actual outputs signals on outputs104. In order to determine the adjustment to the weights, a calculation may be performed as part of408. The calculation takes into account the free and biased inference states. In some embodiments, the calculation utilizes a Hebbian-like contrastive rule. In other embodiments, other techniques may be used. The function(s) used by neurons in neuron layers120may optionally be updated, at410. For example, the thresholding function may be changed from the noise enhancing function shown in graph300C to the noise suppressing function shown in graph300B. At412, processes402,404,406,408,410may be iteratively repeated. Thus, learning network100may be trained via method400.

Because activity-difference training is used, the training may be accomplished without requiring gradients to be explicitly calculated. Instead, training is probabilistic in nature. Further, training may be more efficiently accomplished. Because the thresholding function used for neurons in layers120may be updated (e.g., to less negative thresholds and/or from negative to positive thresholds), training may be completed more rapidly with less probability of learning network100being trapped in a local minimum of the energy instead of evolving toward the global minimum. Thus, training of a learning network may be more accomplished more rapidly and with less power consumed and may result in improved detection abilities of the learning network.

FIG.5depicts an embodiment of learning network500that is analogous to learning network100. More specifically, learning network500may be described by an energy analogous to that described above with respect to learning network100. For example, the interaction term for the energy may consist of neuron pair interactions. Thus, the interaction term for the energy of learning network500is a quadratic term. Neurons in learning network500may utilize thresholding functions, such as hysteretic thresholding functions. As such, learning network500may enhance and/or suppress noise as described in the context of graphs300B-300E, learning network100, and/or method400. Learning network500may be an artificial neural network that may be trained using activity-difference training analogous to that described for method400.

Learning network500includes vector MM unit510and hysteretic comparators520that are analogous to vector MM units110and neuron layers120. Thus, learning network500may be viewed as one implementation of learning network100. Although various lines in learning network500are depicted using single arrows, multiple conductive lines carrying multiple signals are typically present. Although a particular configuration is shown for learning network500, other configurations may be possible.

Vector MM unit510includes two crossbar arrays512and514, corresponding multiplexers513and515, as well as difference amplifier516. Crossbar arrays512and514utilize memristors and/or other programmable components for weights. In order to support both positive and negative weights in vector MM unit, two crossbar arrays512and514are used. Consequently, difference amplifier516may be used to subtract signals corresponding to negative weights (i.e. the input signals to vector MM unit510multiplied by the weights of crossbar array514) from the signals corresponding to positive weights (i.e. the input signals to vector MM unit510multiplied by the weights of crossbar array512). The neurons may be represented by components520,540,550, and560. Transimpedance amplifier (TIA)540converts signals represented as currents to voltage and may amplify the voltage signals as desired. Analog to digital converter (ADC)550converts the analog signals into digital format. Hysteretic comparators520perform the thresholding functions (e.g., using positive and/or negative thresholds). Hysteretic comparators520may also be programmed with the desired thresholding function. Thus, hysteretic comparators520may be considered to function as neurons. The output signals from hysteretic comparators520are converted into analog signals by digital to analog converter (DAC)560and used to drive system510.

Thus, learning network500may be used for a single layer (e.g., a vector MM unit and corresponding neuron layer) or may be reprogrammed and used iteratively to represent multiple layers. Learning network500functions in an analogous manner to learning network100. Consequently, learning network500may have analogous benefits to learning network100.

FIG.6depicts an embodiment of learning network600that is analogous to learning network100. More specifically, learning network600may be described by an energy analogous to that described above with respect to learning network100. For example, the interaction term for the energy may consist of neuron pair interactions. Thus, the interaction term for the energy of learning network600is a quadratic term. Learning network600may be an artificial neural network that may be trained using activity-difference training analogous to that described for method400. Learning network600includes system inputs602, system outputs604, sparsely coupled crossbar array610, neuron layers620-1,620-2,620-3, and620-4(collectively or generically620), bias630and feedback640. System inputs602, system outputs604, sparsely coupled crossbar array610, neuron layers620, and bias630are analogous system inputs102, system outputs104, vector MM units110, neuron layers120, and bias130. Thus, learning network600may be viewed as one implementation of learning network100.

In the embodiment shown, the system inputs602, system outputs604, bias630and feedback640are shown as single arrows. However, in general multiple system inputs, multiple system outputs, multiple feedback lines and multiple bias lines are present. Although a particular configuration is shown (e.g., four neuron layers620and a particular number of submatrices in sparsely coupled crossbar array610) other configurations may be possible. Further, although system inputs602are shown as being connected provide to first neuron layer620-1, in some embodiments, inputs are provided to sparsely coupled crossbar array610.

Sparsely coupled crossbar array610can be divided into portions that can be viewed as matrices and their transposes. Thus, sparsely coupled crossbar array610includes regions610-1,610-2, and610-3configured as matrices w1, w2, and w3. Regions610-1,610-2, and610-3provide a vector matrix multiplication of matrices w1, w2, and w3, respectively, with input signals v1, v2, and v3, respectively. Sparsely coupled crossbar array610also includes regions612-1,612-2, and612-3configured as weight matrix transposes w1T, w2T, and w3T, respectively. Thus, regions612-1,612-2, and612-3provide a vector matrix multiplication of weight matrix transposes w1T, w2T, and w3T, respectively, with input signals v2, v3, and v4, respectively.

For example, region610-1includes weights (i.e. programmable resistances) that are coupled between lines of this portion of crossbar array610. The weights for regions610-1are analogous to weights in a vector MM unit between neuron layer620-1and neuron layer620-2. Thus, region610-1may be considered analogous to vector MM unit120-1. Region612-1includes weights that are coupled between lines of this portion of a crossbar array that correspond to the transpose weights in a vector MM unit between neuron layer620-1and neuron layer620-2. Regions610-2and610-3and regions612-2and612-3, respectively, are similarly configured. Thus, sparsely coupled crossbar array610is configured to provide a matrix multiplication of matrices and, across the diagonal (indicated by the dotted line inFIG.6), their transposes.

As previously mentioned, crossbar array610is sparsely coupled. Individual regions610-1,610-2,610-3,612-1,612-2, and612-3may be sparsely or fully coupled depending upon the configuration of individual matrices and their transposes. Neuron layers620provide their output signals (u1, u2, u3, and u4) to the inputs as input signals (v1, v2, v3, and v4, respectively) via feedback640. For example, output signals u1of neuron layer620-1are provided via feedback640as input signals v1. Thus, crossbar array610may provide a symmetric, zero-diagonal weight matrix for learning network600having an energy including an interaction term consisting of neuron pair interactions.

Neuron layers620are analogous to neuron layers120. Thus, each neuron layer620includes neurons that are interconnected via the corresponding weight matrix. For example, neuron layer620-1includes neurons that are interconnected with neurons in layer620-2via region610-1of crossbar array610. Thus, the outputs of one neuron layer may be coupled with the inputs of another neuron layer via feedback640. Neurons in neuron layers620also utilize thresholding functions, such as hysteretic thresholding functions. As such, learning network600may enhance and/or suppress noise as described in the context of graphs300B-300E, learning network100, and/or method400.

Crossbar array610in conjunction with neuron layers620, system inputs602, system outputs604, bias630and feedback640provides an equivalent of a multi-layered deep neural network that may be trained via activity-difference training. For example, a free inference may be performed (e.g., at402of method400). To do so, the neuron output signals u1from neuron layer620-1are provided via feedback640as input signals v1. Input signals v1are multiplied by weight matrix w1of region610-1. The weighted input signals from the matrix multiplication of w1are provided to neurons620-2, resulting in neuron output signals u2. Neuron output signals u2from neuron layer620-2are provided via feedback640as input signals v2. Input signals v2are provided to matrix w2of region610-2and to matrix w1Tof region612-1. Input signals v2are multiplied by weight matrix w2of region610-2and by w1Tof region612-1. The weighted input signals from the matrix multiplication with w2are provided to neurons620-3, resulting in neuron output signals u3. The neuron output signals u3from neuron layer620-3are provided via feedback640as input signals v3. Input signals v3are provided to matrix w3of region610-3and to matrix w2Tof region612-2. Input signals v3are multiplied by weight matrix w3of region610-3and by w2Tof region612-2. The weighted inputs from the matrix multiplication with w3are provided to neurons620-4, resulting in neuron output signals u4. Neuron output signals u4may be provided as system outputs604as well as fed back as input signals v4. Input signals v4are provided to matrix w3Tof region612-3and multiplied by weight matrix w3Tof region612-3. Use of the transpose matrices in regions612-1,612-2, and612-3aids in carrying the bias signal to from later layers to earlier layers (e.g. from neuron layer620-3to neuron layer620-2).

For a biased inference (e.g.,404of method400), bias signals are provided via bias630and fed back via feedback640. The bias signals provided via bias630at the last set of neurons620-4(e.g., at system outputs604) may nudge the output signals on system outputs604to be closer to the target output signals. In some embodiments, the bias signals may clamp the output signals at the target output signals. Thus, the bias signals may be considered to provide a version of clamping (i.e. from little clamping to full clamping). With each subsequent cycle of updating the outputs, the bias information is propagated backwards across the equivalent layers of the multi-layered network. Locations in learning network600may be sampled for the biased and free inferences to determine equilibrium states of learning network600(e.g., at406of method400). Weights in regions610-1,610-2,610-3,612-1,612-2, and612-3are updated based on the free and biased inferences as well as the equilibrium states (e.g., at408of method400). Further, the thresholding functions utilizes by neurons in neuron layers620may be updated. Thus, learning network600may be trained using activity-difference training.

Thus, learning network600may be trained by activity-difference training. This process uses only locally available information. Explicitly calculating and propagate gradients for minimizing errors (which is typical in back-propagation) is unnecessary. Learning network600, with one matrix multiplication cycle per layer, may converge on a minimum in the energy (i.e. complete training) with accuracies that are substantially similar to traditional networks requiring a large overhead (i.e. a large number of matrix multiplication cycles to converge on the minimum). Although learning network600utilizes a single sparse crossbar array610including bidirectional/symmetric weight matrices, other equivalent architectures are possible. Some other embodiments described herein may utilize such other architectures.

Circuit noise and temporal variabilities have proven prohibitive in adoption of many hardware platforms for activity-difference training, especially for high-precision and storage applications. However, the thresholding functions described herein may be used in conjunction with neuron layers620to facilitate convergence on a global energy minimum for activity-difference training. Such noisy hardware, in a broad sense, would be unsuitable for backpropagation due to its analytical and exact nature. Thus, learning network600may be well suited for manufacturable post-CMOS hardware.

Learning network600thus creates a bidirectional energy-based equivalent of a deep neural network, such as learning network100. Learning network600also allows the adjustment weights of regions610-1,610-2, and610-3to be treated as an optimization problem. Method400may be used to train learning network600. Thus, the benefits of method400and/or learning network100may also be achieved in learning network600. Moreover, learning network600may be either clocked (synchronized) or asynchronous. If operated asynchronously, latencies may be minimized drastically relative to the clocked embodiments. Consequently, learning network600is an efficient system for performing learning using activity-difference training that may be readily implemented.

FIGS.7-14depict embodiments of learning networks700,800,900,1000,1100,1200,1300, and1400, respectively. Learning networks700,800,900,1000,1100,1200,1300, and1400are analogous to learning networks100,500, and/or600. Thus, learning networks700,800,900,1000,1100,1200,1300, and1400are described by an energy function analogous to those discussed above with respect to learning networks100,500, and600; may include in their energy function an interaction term that consists of neuron pair interactions (e.g., that are quadratic and may be implemented using a crossbar array); include neurons that use thresholding functions, such as hysteretic thresholding functions; may enhance and/or suppress noise as described in the context of graphs300B-300E, learning networks100,500, and600and/or method400; and may be artificial neural networks trained using activity-difference training analogous to that described for method400. In some embodiments, learning networks700,800,900,1000,1100,1200,1300, and1400may be viewed as implementations of learning network(s)100,500, and/or600.FIGS.9-11relate to synchronized learning systems, whileFIGS.12-14relate to asynchronous learning systems.

Referring toFIG.7, learning network700is a schematic indicating the data flow for an embodiment of a learning network analogous to learning network600. Learning network700includes three layers750-1,750-2, and750-3(collectively or generically750). Layer750-1includes a first set of weights (i.e. a vector MM unit)710-1and neuron layer720-1. Layer750-2includes weights (i.e. a vector MM unit)710-2and neuron layer720-2. Layer750-3includes weights (i.e. a vector MM unit)710-3and neuron layer720-3. Weights710-1,710-2, and710-3(collectively or generically710) are analogous to crossbar array610. Neuron layers720-1,720-2, and720-3(collectively or generically720) are analogous to neuron layers620. The forward data paths are represented by solid lines, the feedback paths (analogous to feedback640) are represented by dotted lines and the reverse path (e.g., for bias signals) are represented by dashed lines. The reverse paths (dashed lines) are provided to the inputs that correspond to the transpose of the matrix for the corresponding weights710. The forward paths are provided to inputs that correspond to the matrix for corresponding weights710. Thus, the forward path results in a vector matrix multiplication by the weight matrix, while the reverse path results in a vector matrix multiplication by the transpose of the weight matrix. The data paths shown in learning network700may be implemented using a single weight matrix and bi-directional hardware (i.e. analogous to learning system600) or by distinct data flow paths including distinct weight matrices (e.g., one crossbar array corresponding to the weight matrix and another crossbar array corresponding to the transpose of the weight matrix) and corresponding hardware (e.g., analogous to learning system500). Thus, as indicated above, the architecture of learning network600may be implemented in a number of ways.

FIG.8depicts a high level diagram of an embodiment of learning network800including layers850-1,850-2,850-3, and850-4(collectively or generically850), global buffer860, I/O interface870, and monitoring controller880. For clarity, only some components of learning network800are shown. Learning network800may be considered to be a chip level view of a learning network implementing learning network100and/or600. Each layer850includes a layer of weights (e.g., a vector MM unit such as a crossbar array) and a layer of neurons. I/O interface870monitors and manages inputs to and outputs from learning network800.

Learning network800may also be used and account for noise. As described in the context of learning networks100,500,600, and700, as well as graphs300A-300E, noise may be amplified or suppressed via the thresholds of hysteretic threshold functions. In some embodiments, noise may be injected via the threshold, which exploits noise from control circuits. Noise may also be injected via neurons (e.g., via noisy neurons), via synapses (e.g., via synapses with temporal and/or non-temporal variations) and via non-zero diagonals for weight matrices (representing self-feedback of neurons, which introduce energy ascent and therefore escaping local minima).

Injection of noise in energy-based systems may be important because real-world problems generally contain many local minima (incorrect solutions for activity-difference training) in addition to the global minimum (the correct solution for activity difference-training). Thus, the energy function desired to be minimized by activity-different training may include local minima which do not represent a state of the learning network being optimized by training. Such local minima can trap the learning network, leading to non-optimized training and poor performance of the learning network. To avoid this, tunable noise can be used to perturb the system out of the local minima. In such a process, the initial magnitude of noise would be large (to perturb the system out of all the local minima), and the final noise magnitude will be low (to trap the system in the global minimum after it has reached one).

In some embodiments, the injection of noise, the use of noise in training, and error correction may be performed by learning network800. In order to employ noise to improve the training of learning networks, monitoring controller880may be used. In particular, monitoring controller880may examine the state of learning system800via local buffers (e.g., buffer986depicted inFIG.9) and global buffers (e.g. global buffer860). In such an embodiment, data from local buffers986and global buffers860may be used to infer the weights (e.g. in vector MM units910) for some or all layers850. The evolution of the weights may be used to identify the energy of learning network800and the evolution of this energy. The evolution of learning system800via activity-difference training may be paused and restarted at different initial conditions. This process can be used to explore many different parts of the energy landscape, which may be useful in exploring highly multi-dimensional spaces. Thus, learning network800may ensure that activity-difference training does not result in learning network800becoming trapped in a local minimum. Consequently, training of learning network800, and thus learning network(s)100,500,600, and/or700, may be improved.

Learning network800may also utilize monitoring controller880and a similar process to account for errors. More specifically, the state of learning network800may be monitored in a similar manner (e.g., via global buffer860and local buffers986) to maintain the integrity of the information flowing through the different layers of the learning network800. If such information is corrupted during propagation (e.g., the training data), then training of learning network800would be solving a problem that we did not ask it to solve. Training of learning network800may be suspended and restarted. Hence, periodic monitoring of learning network800may keep the errors from propagating. Thus, performance of learning network800may be improved.

FIG.9depicts an embodiment of learning network900that may be used in implementing learning network800. For simplicity, not all components of learning network900are shown. Learning network includes system inputs902, system outputs904, layers950-1,950-2,950-3, and950-4(collectively or generically950), input register (IR)980coupled to inputs902, level splitter (LS)982coupled to IR980, max pool984, buffer986, and output register988. A more detailed view of one layer950-3is also shown. Other layers950may be constructed analogously. Layer950-3includes vector MM unit910and neuron layer920. Vector MM unit910may be a crossbar array. Neuron layer920includes neurons that may use thresholding functions such as hysteretic thresholding functions. Also shown is settable threshold, β, for neuron layer920. LS982is used to split input signals to match the sizes of vector MM units910.

FIG.10depicts an embodiment of a portion of learning network1000that may be trained using activity-difference training and is analogous to learning networks100and600. More specifically, learning network1000may be used to perform the vector matrix multiplication by vector MM unit910of layers950depicted inFIG.9. Learning network1000includes layers1050-1and1050-2(collectively or generically1050) that are analogous to layers950of learning network900. Each layer1050includes input registers (IR)1080, bit splitter (BS)1082, OR gate1088, analog to digital converters (ADCs)1090, shift and add (S-A)1092. In order to provide vector MM units910ofFIG.9, learning network1000includes multiple crossbar arrays (XBs)1052, digital-to-analog converters (DACs)1054, and sample-and-hold circuits (S&Hs)1056. For simplicity, only one DAC1054and S&H1056is labeled in each layer1050. XBs1052are used to perform the vector matrix multiplication operations. Layers1050-1and1050-2share XBs1052, as indicated by dashed lines inFIG.10. The configurations of the S&H1056and DAC1052differs between layers1050-1and1050-2. Consequently, layer1050-1performs a vector matrix multiplication of the weight matrix represented by XBs1056. In contrast, layer1050-2performs a vector matrix multiplication of the transpose of the weight matrix represented by XBs1056. In some embodiments, layers1050-1and1050-2may be manufactured as two neighboring layers in a chip architecture, which is typical in CMOS design. Layers1050-1and1050-2may, for example, be used in implementing regions610-1and612-1of learning network600.

In layers1050, each multiply and accumulate operation (i.e. each vector multiplication operation) includes splitting a given array or matrix to be multiplied into several bits and distributing the operation on each bit to different hardware pieces (e.g., different XBs1052). Consequently, learning network100may retain the desired accuracy. For example, each hardware operation may maintain more than 1 bit of accuracy. Further, multiplications of both the weight matrix and its transpose are performed on the same XB1052. As a result, inter-array errors that might otherwise be introduced may be mitigated or avoided. Thus, use of learning network1000may provide additional benefits to learning networks such as learning networks100,500, and/or600.

In alternate embodiments, each layer may include its own XBs.FIG.11depicts learning network1100in which each layer1150includes its own XBs. Layer1150includes IR1180, BS1182, OR1188, ADCs1190, S-A1192, XBs1152(of which only one is labeled), DACs1154(of which only one is labeled), and S&Hs1156(of which only one is labeled) that are analogous to IR1080, BS1082, OR1088, ADCs1090, S-A1092, XBs1052, DACs1054, and S&Hs1056, respectively, of layer1050-1. However, each layer1150in learning network1100includes its own XBs1152. For example, one layer1150might be used in implementing region610-1of learning network600, while another layer1150may be used in implementing region612-1of learning network600. In such an embodiment, the XBs1152for such layers would be configured as matrix transposes. Learning network1100shares the benefits of learning network100with respect to accuracy. However, inter-array variations may adversely affect performance of layer1150.

Learning networks800,900,1000, and1100may be synchronous networks. In some embodiments, the clock progression for learning networks800,900,1000, and/or1100may proceed from the chip level (e.g., the level depicted inFIG.8), to the layer level (e.g., the levels depicted inFIG.9), the vector matrix multiplication level (e.g., the level depicted inFIGS.10-11), to the layer level (e.g., the levels depicted inFIG.9), and back to the chip level (e.g., the level depicted inFIG.8). For example, operations for the I/O interface870and global buffer860may be performed first. Layer level operations for IR980, buffer986, and LS982may be performed next. Vector matrix multiplication operations for IR1080/1180, BS1082/1182, DACs1054/1154, XB1052/1152, S&H1056/1156, ADCs1090/1190, S-A1092/1192, and OR1088/1188may then be performed. Layer level operations for neurons920, buffer986, max pool984, and OR988may follow. Chip level operations for the global buffer860and I/O interface870may complete the progression.

Although learning networks900,1000, and1100are clocked, in some embodiments, learning networks can utilize the activity-difference training and inference processes described herein without a clocking signal (i.e., in an all-analog, asynchronous or continuous-time fashion). The same general chip-level structure of learning network800depicted inFIG.8may be utilized in some asynchronous embodiments.FIGS.12-14depict embodiments of portions of learning networks1200,1300, and1400that may be asynchronous.

FIG.12depicts an embodiment of learning network1200that may be used in implementing learning network800for asynchronous learning. Learning network1200is also analogous to, for example, learning networks100,600,1000, and/or1100. For simplicity, not all components of learning network1200are shown. Learning network includes inputs1202, outputs1204, crossbars1252-1,1252-2,1252-3, and1252-4(collectively or generically1252), input buffer (IB)1280coupled to inputs1202, weight splitter (WS)1282coupled to IB1280, output buffer1288, and weight and add (W-A)1292. Learning network1200may be used to asynchronously perform the vector matrix multiplication. Thus, learning network1200may be considered an asynchronous vector matrix multiplication unit (a-VMM).

FIG.13depicts an embodiment of learning network1300that may be used in asynchronously implementing learning network800. For simplicity, not all components of learning network1300are shown. Learning network includes system inputs1302, system outputs1304, layers1350-1,1350-2,1350-3, and1350-4(collectively or generically1350), input buffer (IB)1380coupled to inputs1302, weight splitter (WS)1382coupled to IB1380, max pool1384, buffer1386, and output buffer1388. A more detailed view of one layer1350-3is also shown. Other layers1350may be constructed analogously. Layer1350-3includes asynchronous vector MM unit (a-VMM)1310and asynchronous neuron layer (a-N)1320. Asynchronous vector MM unit1310may be analogous to asynchronous vector MM unit1200. Asynchronous neuron layer1320includes neurons that may use thresholding functions such as hysteretic thresholding functions. Also shown is settable threshold, β, for neuron layer1320.

Learning networks1200and1300are analogous to learning networks1000/1100and900, respectively. However, asynchronous learning networks1200and1300generally do not contain data flow in digital formats (i.e., data flows in analog fashions) and there is no clocking signal. Because there is no clocking, the signals propagate through learning networks1200and1300much faster than in a synchronized learning network. Further, learning networks1200and1300may find solutions to training and inference problems very rapidly. However, as in all asynchronous systems, there may be issues with signal degradation, stability, oscillations, an/or other issues that may be desired to be accounted for.

FIG.14depicts regenerative asynchronous learning network1400. Learning network1400includes layers1400-1,1450-2,1450-3through1450-n,1450-n+1through1450-n+m. Also shown are two regenerative circuits1490. Other numbers of layers and/or regenerative circuits are possible in other embodiments. Learning network1400also includes monitoring controller1480, global buffer1460, and I/O interface1470that are analogous to monitoring controller880, global buffer860, and I/O interface870, respectively. Regenerators1490may include or consist of digital-analog converters, buffers, clocking circuits, signal reconstruction circuits, signal monitoring circuits, and analogous components. Thus, regenerators1490may be considered repeaters used to enhance signals. As such, learning network1400may be considered to represent a middle ground between the synchronous learning networks900,1000, and/or1100and asynchronous learning networks1200and/or1300. Thus, activity-difference training may be accomplished on synchronous learning networks (e.g., learning networks900,1000, and/or1100using digital clocking) as well as on asynchronous learning networks (e.g., learning networks1200,1300, and/or1400). Thus, the benefits of activity-difference training described herein may be realized by a number of configurations.

In some embodiments, particularly where the problem sizes are large, hardware for learning networks may be desired to be reused. For example, if the problem size exceeds the available hardware resources, activity difference training described herein may be performed on a subset of the available hardware resources. Similarly, if the desired precision for a problem exceeds the abilities of the available hardware resources, activity difference training may be provided on portions of the problem. Thus, the available hardware resources (or some subset thereof) may be reused to solve subsets of the problem. The results may be combined to complete the solution.

For example,FIG.15depicts an embodiment of method1500for performing activity-difference training on a learning network, such as learning network100, in which hardware resources may be reused to solve subsets of the problem. Although particular steps are depicted, method1500may include additional and/or other steps. Similarly, steps of method1500may include substeps. Method1500is described in the context of learning network100. However, nothing prevents method1500from being used in conjunction with other networks having an analogous energy function (e.g., networks500,600,700,800,900,1000,1100,1200,1300and/or1400). For example, other learning networks including neurons (e.g., neurons that utilize thresholding functions) and vector MM units may be employed.

The input signals (i.e., training data) are split into portions that can be solved using the available hardware resources, at1502. In some embodiments, only some input signals may be used for an iteration of method1500. In some embodiments, input signals may be split based upon the precision for particular iterations. Also at1502, the input signal(s) for the current iteration are selected. Activity-difference training is performed on the input signals selected, at1504. In some embodiments, method400may be performed for1504. The activity-difference training performed at1504may utilize all or only some of the available hardware resources. The results of the activity-difference training are stored, at1506. For example, the intermediate solutions corresponding to the subset of the problem may be stored in a memory unit.

At1508, processes1502(e.g., selection of the input signal(s) for the current iteration),1504, and1506are repeated until the problem has been solved.1508may include reprogramming the hardware to represent the next subset of the problem, on which the activity-difference training continues. This process is repeated until the entire problem (or a desired portion of the entire problem) is solved. At1510, the intermediate results that have been stored are concatenated to provide a final result. In some embodiments, method1500may utilize special processes, which may be managed by components such as an on-chip processor or an off-chip computer.

For example, at1502the input signal(s) to be used in connection with learning network100may be split. At1504, the input signals(s) for the current iteration of method1500are provided to system inputs102and activity-difference training using method400performed. The intermediate result for the input signal(s) used in1504are stored, at1506. This may be performed using portions of a computer system (e.g., a memory unit) not depicted inFIG.1. At1508, learning network100may be reprogrammed for the current iteration. Also at1508, processes1502,1504, and1506are repeated for learning network100. The results may be concatenated to solve the desired problem, at1510. Thus, learning network100may be reused in order to attack problems which may exceed the available resources and/or precision of learning network100. Consequently, the benefits described herein may be achieved for problems that are high-precision and/or large.

FIG.16depicts embodiment of learning networks1600-1and1600-2that may be trained using activity-difference training. For simplicity, not all components of learning networks1600-1and1600-2are shown. Learning networks1600-1and1600-2are analogous to learning networks100,500, and/or600. Learning network1600-1is a large network including layers1650-1(of which only one is labeled). Each layer1650-1includes an asynchronous vector MM unit1610-1(of which only one is labeled) and asynchronous neurons1620-1(of which only one is labeled). Learning network1600-2is significantly smaller than learning network1600-1. Learning network1600-2including layers1650-2(of which only one is labeled). Each layer1650-2includes an asynchronous vector MM unit1610-2(of which only one is labeled) and asynchronous neurons1620-2(of which only one is labeled). Both networks1600-1and1600-2may be used in to attack large and/or high precision problems. However, learning network1600-2may reuse hardware for problems which learning network1600-1may address without hardware reuse.

Learning network1600-2may manage problems otherwise requiring larger learning network1600-1by sharing limited resources to break down a problem into smaller pieces. Stated differently, learning network1600-2may be used in conjunction with method1500. In some embodiments, the problem is broken into smaller portions sequentially (e.g., splitting by layers). Learning network1600-2may then be trained sequentially on each portion of the problem via method1500. Similarly, a problem may be broken into smaller “more important” and “less important” parts, or a gradient thereof. For instance, the most significant bits (MSBs), which carry the more important representations of the problem, may be split from the least significant bits (LSBs), which carry information that is less important. Activity-difference training of only the MSB(s) part of the problem is performed on smaller learning network1600-2. Learning network1600-2is reused in subsequent cycles to train lesser significant bits. In some embodiments, a controlling/monitoring computer (not shown inFIG.16) identifies the point in time in method1500(i.e., in1504) when the results yield diminishing returns, thereby stopping the training to provide an intermediate result. Using, hardware sharing or reuse enables smaller learning network1600-2to be used for a shorter duration of time by making the problem smaller. Thus, the benefits of activity-difference training on learning networks described herein may be maintained.

FIG.17depicts an embodiment of a portion of learning network1700that may be trained using activity-difference training and is analogous to learning networks100,600, and1600. More specifically, learning network1700may be used to perform the functions of layers1650-2depicted inFIG.16. Learning network1700includes layers1750-1and1750-2(collectively or generically1750). Each layer1750includes input registers (IR)1780(of which only one is labeled), bit splitter (BS)1782(of which only one is labeled), crossbar array (XB)1752(of which only one is labeled), and digital to analog converter (DAC)1756(of which only one is labeled)). Learning network1700also includes shared OR gate1788, shared analog to digital converters (ADCs)1790, shared shift and add (S-A)1792, shared neurons1720having threshold β, and arithmetic core1795. Learning network1700extends the aforementioned use of in-memory computing crossbars to solve for energy minimization to include the summation of two vector matrix multiplication outputs at A and the application of a non-linear activation function at neurons1720in the analog domain. The output is provided to arithmetic core1795. In some embodiments, multiple crossbars may be used in each layer1750. Use of multiple crossbar arrays may, for example, support operation with binary nonvolatile memory cells in the crossbars. In such embodiments, the bits of the weights can be distributed across multiple crossbars.

FIG.18depicts an embodiment of a portion of learning network1800that may be trained using activity-difference training and is analogous to learning networks100,600, and1600. More specifically, learning network1800may be used to perform the functions of vector MM units1610-2depicted inFIG.16. Further, learning network1800may be considered to be adapted to breaking a problem up based upon precision (e.g. from most significant bit to least significant bit).

Learning network1800includes layers1850-1through1850-2(collectively or generically1850). Each layer1850includes input registers (IR)1880, crossbar arrays (XBs)1852(of which only one is labeled), and digital to analog converters (DACs)1856(of which only one is labeled)), sample-and-hold circuits (S&Hs)1856(of which only one is labeled), OR gate1888, analog to digital converters (ADCs)1890, shift and add (S-A)1892, and processor1896. In some embodiments, XBs1852utilize nonvolatile memory elements as programmable impedances. Thus, XBs1852may provide binary weights. Processor1896may be a RISC processor including arithmetic core1895and memory1898. Non-linear activations and finite difference methods may be executed by processor1896.

Thus, learning network1800is self-contained and may locate minima (e.g., in the energy function) a network with binary weights present in XBs1852. For an arbitrary multilayer, deep neural network with weights (synapses) of N-bit precision, N layers1850are used. Each layer includes binary weights in XBs1852representing a bit of the full network. Learning network1800makes use of in-memory computing through crossbars with NVM to accelerate the multiplications in a finite difference solver.

Learning network1800may be part of a system that divides problems based on the significance of the bits. For example,FIG.19depicts learning network1900in which learning system1800may be used. Learning network1900includes cache1910, arithmetic core1920and layers1950-1through1950-8(collectively or generically1950). Layer1950-1through1950-8corresponding to layers1850-1through1850-2. Arithmetic core1920takes the equilibrium points of each of layers1950and uses a process (which may include but is not limited to averaging, linear operations and/or non-linear operations) to combine the outputs into one set of equilibrium points for the original n-bit network. Thus, the benefits of activity-difference training on learning networks described herein may be maintained for high precision problems.

Various features of learning networks have been described in conjunction with networks100,500,600,700,800,900,1000,1100,1200,1300,1400,1600,1700,1800, and1900. Similarly, various processes have been described in the context of with methods400and1500. Various features described herein may be combined in manners not explicitly depicted or discussed herein.

Although the foregoing embodiments have been described in some detail for purposes of clarity of understanding, the invention is not limited to the details provided. There are many alternative ways of implementing the invention. The disclosed embodiments are illustrative and not restrictive.