DNN TRAINING ALGORITHM WITH DYNAMICALLY COMPUTED ZERO-REFERENCE

A computer implemented method includes performing a gradient update for a stochastic gradient descent (SGD) of a deep neural network (DNN) using a first set of hidden weights stored in a first matrix comprising a Resistive Processing Unit (RPU) crossbar array. A second matrix comprising a second set of hidden weights is stored in a digital medium. A third matrix comprising a set of reference values is computed upon a transfer cycle of the first set of weights from the first matrix to the second matrix, accounting for a sign-change (chopper). The third matrix is stored in the digital medium. A third set of weights is updated for the DNN from the second matrix when a threshold is reached for the second set of weights, in a fourth matrix comprising a RPU crossbar array.

BACKGROUND

Technical Field

The present disclosure generally relates to Deep Learning, and more particularly, to systems and methods of training a Deep Neural Network using hardware elements.

Description of the Related Art

A deep neural network (DNN) can be embodied in an analog cross-point array of resistive devices such as resistive processing units (RPUs). RPU devices generally include a first terminal, a second terminal and an active region. A conductance state of the active region identifies a weight value of the RPU, which can be updated/adjusted by application of a signal to the first/second terminals.

DNN based models have been used for a variety of different cognitive based tasks such as object and speech recognition and natural language processing. DNN training is salient in providing a high level of accuracy when performing such tasks. Training large DNNs is a computationally intensive task. Most popular methods of DNN training, such as backpropagation and stochastic gradient decent (SGD), involve the RPUs to be “symmetric” to work accurately. Typical systems assume the symmetry point is correctly estimated and stored initially to a reference device array. The symmetry point may be estimated incorrectly and can also be written incorrectly including noise.

SUMMARY

According to an embodiment of the present disclosure, a computer implemented method includes performing a gradient update for a stochastic gradient descent (SGD) of a deep neural network (DNN) using a first set of hidden weights stored in a first matrix comprising a Resistive Processing Unit (RPU) crossbar array. A second matrix comprising a second set of hidden weights is stored in a digital medium. A third matrix comprising a set of reference values is computed upon a transfer cycle of the first set of weights from the first matrix to the second matrix, accounting for a sign-change (chopper). The third matrix is stored in the digital medium. A third set of weights is updated for the DNN from the second matrix when a threshold is reached for the second set of weights, in a fourth matrix comprising a RPU crossbar array. The device has the technical effect of increasing efficiency and accuracy of system computations on data used in RPU systems.

In one embodiment, which may be combined with the preceding embodiment, the second set of weights accounts for a set of previous reference values from a prior iteration of the transfer cycle. This allows more efficient computing capabilities.

In one embodiment, which may be combined with the preceding embodiments, a fifth matrix, stored in the digital medium, is configured to compute a next set of reference values from values read from the first matrix, during a chopper cycle. The fifth matrix is configured to partially update the third matrix, after the chopper cycle is completed. This enables greater accuracy of data manipulation.

In one embodiment, which may be combined with the preceding embodiments, the computing for the SGD includes a fifth matrix comprising a set of previous reference values, and storing the fifth matrix in the digital medium. This allows more efficient computing capabilities.

In one embodiment, which may be combined with the preceding embodiments, the assigning the set of reference values to the set of previous reference values in the digital medium occurs at a chopper switching time. This allows more accurate computing capabilities.

In one embodiment, which may be combined with the preceding embodiments, the resetting the set of reference values to zero occurs at the chopper switching time. This allows more efficient computing capabilities.

In one embodiment, which may be combined with the preceding embodiments, the device is configured to switch a sign of the chopper at the chopper switching time. This enables greater accuracy of data manipulation.

In one embodiment, which may be combined with the preceding embodiments, no RPU crossbar array is used for storing the set of reference values. This enables more efficient use of space in the IC array.

In one embodiment, which may be combined with the preceding embodiments, a set of previous reference values are set to a recent read-out weight vector. This enables more efficient use of space in the IC array.

According to an embodiment of the present disclosure, a non-transitory computer readable storage medium tangibly embodying a computer readable program code having computer readable instructions to solve a machine learning task, that, when executed, the instructions cause a computer device to carry out a method. The method includes performing a gradient update for a stochastic gradient descent (SGD) of a deep neural network (DNN) using a first set of weights stored in a first matrix comprising a Resistive Processing Unit (RPU) crossbar array. A second matrix comprising a second set of weights is stored in a digital medium. A third matrix comprising a set of reference values is computed for the SGD, upon a transfer cycle of the first set of weights from the first matrix to the second matrix, accounting for a chopper. The third matrix is stored in the digital medium. A third set of weights is updated for the DNN from the second matrix when a threshold is reached for the second set of weights, in a fourth matrix comprising a RPU crossbar array. The device has the technical effect of increasing efficiency and accuracy of system computations on data used in RPU systems.

According to an embodiment of the present disclosure a device including a first matrix comprises a Resistive Processing Unit (RPU) crossbar array with a first set of weights configured for a gradient update for a stochastic gradient descent (SGD) of a deep neural network (DNN). The device includes a second matrix comprising a second set of weights stored in a digital medium. Further the device includes a third matrix comprising a set of reference values computed for the SGD, stored in the digital medium, wherein the set of reference values is computed upon a transfer cycle of the first set of weights from the first matrix to the second matrix, accounting for a chopper. The device may also include a fourth matrix comprising a RPU crossbar array storing a third set of weights for the DNN that are updated from the second matrix when a threshold is reached for the second set of weights. The device has the technical effect of increasing efficiency and accuracy of system computations on data used in RPU systems.

In one embodiment, which may be combined with the preceding embodiment, the second set of weights accounts for a set of previous reference values from a prior iteration of the transfer cycle. This allows more efficient computing capabilities.

In one embodiment, which may be combined with the preceding embodiments, the set of reference values accounts for a switching frequency. This enables greater accuracy of data manipulation.

In one embodiment, which may be combined with the preceding embodiments, a fifth matrix comprising a set of previous reference values computed for the SGD, is stored in the digital medium. This allows more efficient computing capabilities.

In one embodiment, which may be combined with the preceding embodiments, the device assigns the set of reference values to the set of previous reference values in the digital medium at a chopper switching time. This allows more efficient computing capabilities.

In one embodiment, which may be combined with the preceding embodiments, the device resets the set of reference values to zero at the chopper switching time. This allows more efficient computing capabilities.

In one embodiment, which may be combined with the preceding embodiments, the device switches a sign of the chopper at the chopper switching time. This enables greater accuracy of data manipulation.

In one embodiment, which may be combined with the preceding embodiments, no RPU crossbar array is used for storing the set of reference values. This enables more efficient use of space in the IC array.

In one embodiment, which may be combined with the preceding embodiments, a set of previous reference values is set to a recent read-out weight vector. This enables more efficient use of space in the IC array.

DETAILED DESCRIPTION

Overview

Provided herein are DNN training techniques with asymmetric RPU devices. The DNN is trained by using two tunable resistive device arrays and two or three digital memory arrays. The methods may include using an RPU crossbar array to represent the weights of the DNN. An additional crossbar array per weight may be used to compute the gradient update, without the need for a third tunable RPU array used to store a reference. Further, updates of both RPU arrays may occur according to the algorithms described herein.

In typical systems, the symmetry point, of each device may be incorrectly estimated. The symmetry point is the conductance where the conductance change response to a single pulsed update in the positive direction is on average of the same as in the negative direction. The symmetry point may be wrongly written onto the reference device with noise so that a wrong value is subtracted during gradient value readout. The update device may be variable so that its symmetry point is unstable and moves with time. Additionally, oftentimes the input is too sparse or the number of devices is too large so that the symmetry point is only reached slowly and transient offsets remain. Additionally adding a dedicated reference device array is costly in integrated circuit chip area. Embodiments overcome these limitations by using a digital memory for storing a metrics used dynamically on the fly to estimate the reference.

Accordingly, one or more of the methodologies discussed herein may obviate a need for time consuming data processing by the user. This may have the technical effect of reducing computing resources used by one or more devices within the system. Examples of such computing resources include, without limitation, processor cycles, network traffic, memory usage, storage space, and power consumption.

It should be appreciated that aspects of the teachings herein are beyond the capability of a human mind. It should also be appreciated that the various embodiments of the subject disclosure described herein can include information that is impossible to obtain manually by an entity, such as a human user. For example, the voltage inputs, and conductance storage values discussed herein, are impossible for a human user to perform.

Turning now to the figures,FIG.1is a schematic diagram illustrating a DNN100having a weight matrix W102, an A matrix112, a μpastmatrix113, and a hidden matrix H114. The weight matrix W102is iteratively trained using the A matrix112, the μpastmatrix113, and the hidden matrix114, as indicated by the arrow direction shown inFIG.1. As highlighted above, the weight matrix W102can be embodied in an analog cross-point array of RPUs. See, for example, the schematic diagram shown inFIG.2.

As shown inFIG.2, each parameter (weight wij) of algorithmic (abstract) weight matrix102is mapped to a single RPU device (RPUij) on hardware, namely a physical cross-point array104of RPU devices. Cross-point array104includes a series of conductive row wires106and a series of conductive column wires108oriented orthogonal to, and intersecting, the conductive row wires106. The intersections between the row and column wires106and108are separated by RPUs110forming cross-point array104of RPU devices. Each RPU110can include a first terminal, a second terminal, and an active region. A conduction state of the active region identifies a weight value of the RPU110, which can be updated/adjusted by application of a signal to the first/second terminals. Further, three-terminal (or even more terminal) devices can serve effectively as two-terminal resistive memory devices by controlling the extra terminals.

Each RPU110(RPUij) is uniquely identified based on its location in (i.e., the ithrow and jthcolumn) of the cross-point array104. For instance, working from the top to bottom, and from the left to right of the cross-point array104, the RPU at the intersection of the first-row wire106and the first column wire108is designated as RPU11, the RPU at the intersection of the first row wire106and the second column wire108is designated as RPU12, and so on. Further, the mapping of the parameters of weight matrix102to the RPUs of the cross-point array104follows the same convention. For instance, weight ui1of weight matrix102is mapped to RPUi1of the cross-point array104, weight wi2of weight matrix102is mapped to RPUi2of the cross-point array104, and so on.

The RPUs110of the cross-point array104, in effect, function as the weighted connections between neurons in the DNN. The conduction state (e.g., resistance) of the RPUs110can be altered by controlling the voltages applied between the individual wires of the row and column wires106and108, respectively. Data is stored by alteration of the RPU's conduction state. The conduction state of the RPUs110is read by applying a voltage and measuring the current that passes through the target RPU110. All of the operations involving weights are performed fully in parallel by the RPUs110.

In machine learning and cognitive science, DNN based models are a family of statistical learning models inspired by the biological neural networks of animals, and in particular the brain. These models may be used to estimate or approximate systems and cognitive functions that depend on many inputs and weights of the connections which are generally unknown. DNNs are often embodied as so-called “neuromorphic” systems of interconnected processor elements that act as simulated “neurons” that exchange “messages” between each other in the form of electronic signals. The connections in DNNs that carry electronic messages between simulated neurons are provided with numeric weights that correspond to the strength or weakness of a given connection. These numeric weights can be adjusted and tuned based on experience, making DNNs adaptive to inputs and capable of learning. For example, a DNN for handwriting recognition is defined by a set of input neurons which may be activated by the pixels of an input image. After being weighted and transformed by a function determined by the network's designer, the activations of these input neurons are then passed to other downstream neurons. This process is repeated until an output neuron is activated. The activated output neuron determines which character was read.

The DNN100illustrated inFIG.1is trained by updating the weight values Wijthrough the A matrix112and then summing the resulting output from the A matrix112into the hidden matrix114until an element of the hidden matrix114(i.e., Hij) reaches a threshold value, as explained in detail below. Before and after the weight values are updated in the A matrix112, however, a chopper116multiplies the inputs and outputs signals by a chopper value. The chopper value at a given time is equal to either a positive one (+1) or a negative one (−1). The chopper116randomly or regularly flips between the chopper values, such that for part of the training period the updates are applied to the A matrix114with an opposite sign. This sign flip by the chopper116means that any “bias” contributed to the weight value by the A matrix112has one sign (i.e., positive or negative) for some periods of the training time, and the other sign (i.e., negative or positive) for other periods of the training time. The chopping period or switching probability may also be assigned by a user. Bias can be inherent in any analog system, including non-ideal RPUs that may be used in the DNN100.

For training purposes, such an ideal device perfectly implements the DNN training process of backpropagation and stochastic gradient decent (SGD). Backpropagation is a training process performed in three cycles: a forward cycle, a backward cycle, and a weight update cycle which are repeated multiple times until a convergence criterion is met. Stochastic gradient decent (SGD) uses the backpropagation to calculate the error gradient of each parameter (weight wij).

To perform backpropagation, DNN based models are composed of multiple processing layers that learn representations of data with multiple levels of abstraction. For a single processing layer where N input neurons are connected to M output neurons, the forward cycle involves computing a vector-matrix multiplication (y=Wx) where the vector x of length N represents the activities of the input neurons, and the matrix W of size M×N stores the weight values between each pair of the input and output neurons. The resulting vector y of length M is further processed by performing a non-linear activation on each of the resistive memory elements and then passed to the next layer.

Once the information reaches to the final output layer, the backward cycle involves calculating the error signal and backpropagating the error signal through the DNN. The backward cycle on a single layer also involves a vector-matrix multiplication on the transpose (interchanging each row and corresponding column) of the weight matrix (z=WTδ), where the vector δ of length M represents the error calculated by the output neurons and the vector z of length N is further processed using the derivative of neuron non-linearity and then passed down to the previous layers.

Lastly, in the weight update cycle, the weight matrix W is updated by performing an outer product of the two vectors that are used in the forward and the backward cycles. This outer product of the two vectors is often expressed as W←W+η(δxT), where η is a global learning rate.

All of the operations performed on the weight matrix W during this backpropagation process can be implemented with the cross-point array104of RPUs110having a corresponding number of M rows and N columns, where the stored conductance values in the cross-point array104form the matrix W. In the forward cycle, input vector x is transmitted as voltage pulses through each of the column wires108, and the resulting vector y is read as the current output from the row wires106. Similarly, when voltage pulses are supplied from the row wires106as input to the backward cycle, then a vector-matrix product is computed on the transpose of the weight matrix WT. Finally, in the update cycle voltage pulses representing vectors x and δ are simultaneously supplied from the column wires108and the row wires106. In this configuration, each RPU110performs a local multiplication and summation operation by processing the voltage pulses coming from the corresponding column wire108and row wire106, thus achieving an incremental weight update.

A symmetric RPU may implement backpropagation and SGD perfectly. Namely, with such ideal RPUs wij←wij+ηΔwij, where wijis the weight value for the ithrow and jthcolumn of the cross-point array104.

FIG.3is a diagram illustrating an example method300for training a DNN according to an embodiment. During training, the weight updates are accumulated first on an A matrix. The A matrix is a hardware component made up of rows and columns of RPUs that have symmetric behavior around the zero point. The weight updates from the A matrix are then selectively moved to a weight matrix W. The weight matrix W is also a hardware component made up of rows and columns of RPUs. The training process iteratively determines a set of parameters (weights wij) that maximizes the accuracy of the DNN. The matrix W is initialized to randomly distributed values using the common practices applied for DNN training. The hidden matrix H, stored digitally, is initialized to zero.

During training, the weight updates are performed on the A matrix. Then, the information processed by A matrix is accumulated in the hidden matrix H (a separate matrix effectively performing a low pass filter). The values of the hidden matrix H that reach an update threshold are then applied to the weight matrix W. The update threshold effectively minimizes noise produced within the hardware of the A matrix. For elements of the A matrix that are initialized with a bias, however, the update threshold will be reached prematurely since each iteration from the element carries a consistent update (either positive or negative) that is based on the bias, and not based on the weight updates associated with training the DNN. The chopper value negates the bias by flipping the sign of the bias for certain periods of time, during which time the bias is summed to the hidden matrix H with the opposite sign. Specifically, some period of time will sum the weight value plus a positive bias to the hidden matrix H while other time periods sum the weight value plus a negative bias to the hidden matrix H. A random flipping of the chopper value means that the time periods with positive bias tend to even out with the time periods with negative bias. Therefore, the hardware bias and noise associated with non-ideal RPUs are tolerated (or absorbed by H matrix), and hence give fewer test errors compared to the standard SGD technique, a hidden matrix H alone, or other training techniques using asymmetric devices, even with a fewer number of states.

The method300initializes the A matrix, the digital compute value p, the hidden matrix H (also stored in a digital buffer), and the weight matrix W in block302. Initializing the A matrix includes, for example, setting all of the values to zero. The array A can be embodied in one interconnected array.

FIGS.4A-4Bare diagrams illustrating interconnected arrays with a digital memory used for estimating reference values on the fly. As illustrated, μ represents the recent past of the gradient update matrix A. In some embodiments, the recent past μ, may be used in a difference calculation in digital storage or memory resulting in a value ω that is used to update H. This creates a floating-point representation of the reference value. The reference value, in this case changes over time according to method300. This dynamic updating and on the fly calculation of the reference value helps eliminate bias in previous systems using a hardware reference RPU matrix for the reference value.

Initialization of the hidden matrix H includes zeroing the current values stored in the matrix or allocating digital storage space on a connected computing device. Initialization of the weight matrix W includes loading the weight matrix W with random values so that the training process for the weight matrix W may begin. ω is assigned based on a read from the A matrix of each column or row, where ω is the digitally converted values processed after using the ADC.

The digital H is a hidden matrix used to filter the gradient values computed onto A. The ω is a read of the analog A matrix, which may be read each column or row, by putting a unit vector (e.g. [1 0 0 0]) with voltages in. The weights of that column in current units will be retrieved, which is changed back to digital by using an ADC. X is the scaling factor, or the learning rate. S is used for the changing chopper value which switches between negative and positive.

Once a threshold is met on H, a pulse is then sent to the weight matrix, W. In other words, the gradient is placed onto the A crossbar RPU. As placed the gradient includes a lot of noise. The gradient is read again, applying a chopper and subtracting the reference values to remove any bias, and then added onto a filter matrix, filtering the noise out. The gradient is then integrated over time, and once the gradient reaches a threshold, the weight is updated. So, therefore, the weight W is only seldomly modified, without any bias applied. This drastically improves the noise properties and accuracy of prior art RPU algorithms.

The method300includes determining activation values by performing a forward cycle using the weight matrix W (block304).FIG.5is a diagram illustrating a forward cycle being performed according to an embodiment. The forward cycle involves computing a vector-matrix multiplication (y=Wx) where the activation values embodied as an input vector x represents the activities of the input neurons, and the weight matrix W stores the weight values between each pair of the input and output neurons.FIG.5shows that the vector-matrix multiplication operations of the forward cycle are implemented in a cross-point array502of RPU devices, where the stored conductance values in the cross-point array502forms the matrix.

The input vector x is transmitted as voltage pulses through each of the conductive column wires512, and the resulting output vector y is read as the current output from the conductive row wires510of cross-point array502. An analog-to-digital converter (ADC)513is employed to convert the analog output vectors516from the cross-point array502to digital signals.

The method300also includes determining error values by performing a backward cycle on the weight matrix W (block306).FIG.6is a diagram illustrating a backward cycle being performed according to an embodiment. Generally, the backward cycle involves calculating the error value δ and backpropagating that error value δ through the weight matrix W via a vector-matrix multiplication on the transpose of the weight matrix W (i.e., z=WTδ, where WTindicates the transpose of the matrix W), where the vector δ represents the error calculated by the output neurons and the vector z is further processed using the derivative of neuron non-linearity and then passed down to the previous layers.

FIG.6illustrates that the vector-matrix multiplication operations of the backward cycle are implemented in the cross-point array502. The error value δ is transmitted as voltage pulses through each of the conductive row wires510, and the resulting output vector z is read as the current output from the conductive column wires512of the cross-point array502. When voltage pulses are supplied from the row wires510as input to the backward cycle, then a vector-matrix product is computed on the transpose of the weight matrix W. As also shown inFIG.6, the ADC513is employed to convert the (analog) output vectors518from the cross-point array502to digital signals.

The method300also includes applying a chopper value to the activation values or the error values (block308). The chopper values may be applied by a chopper (e.g., chopper116fromFIG.1), which is included for each row wire and each column wire in the A matrix502. In certain embodiments, the cross point array502may have choppers only on the column wires506, or only on the row wires504. After the chopper values are applied to the activation values or the error values, the method300also includes updating the A matrix with the activation values, error values, (input vectors x and δ), and chopper values (block310).

FIG.7is a diagram illustrating the array A502being updated with x propagated in the forward cycle and δ propagated in the backward cycle according to an embodiment. Each row and column has a chopper value550applied to the respective wire. The sign of the chopper value550is represented as “+” for positive chopper value (i.e., no change to the activation value or error value) or an “X” for a negative chopper value (i.e., sign change to the activation value or error value). The updates are implemented in cross-point array502by transmitting voltage pulses representing vector x (from the forward cycle) and vector δ (from the backward cycle) simultaneously supplied from the conductive column wires506and conductive row wires504, respectively. In this configuration, each RPU in cross-point array502performs a local multiplication and summation operation by processing the voltage pulses coming from the corresponding conductive column wires506and conductive row wires504, thus achieving an incremental weight update. The forward cycle (block304) the backward cycle (block306) and updating the A matrix with the input vectors from the forward cycle and the backward cycle (block310) may be repeated a number of times to improve the updated values of the A matrix.

The method300also includes reading an ithcolumn of A by performing a forward cycle on the A matrix using an input vector ei, (i.e., y′=Aei) and the chopper values (block312). At each time step a new input vector eiis used and the sub index i denotes that time index. As will be described in detail below, according to an example embodiment, input vector eiis a one hot encoded vector. For instance, as is known in the art, a one hot encoded vector is a group of bits having only those combinations having a single high (1) bit and all other bits a low (0). To use a simple, non-limiting example for illustrative purposes, assume a matrix of the size 4×4, the one hot encoded vectors will be one of the following vectors: [1 0 0 0], [0 1 0 0], [0 0 1 0] and [0 0 0 1]. At each time step a new one hot encoded vector is used and the sub index i denotes that time index. It is notable, however, that other methods are also contemplated herein for choosing input vector ei.

FIG.8is a diagram illustrating reading an ithcolumn of A by performing a forward cycle y′=Aeion the A matrix with chopper values according to an embodiment, where eiis the i-the unit vector. Alternatively, Y′=ATei(a transposed read), could be performed. The input vector eiis transmitted as voltage pulses through each of the conductive column wires506, and the resulting output vector y′ is read as the current output from the conductive row wires504of cross-point array502. Each column wire506and row wire504is read with the same chopper value (i.e., positive or negative) with which the A matrix was updated. For example, the first column wire506ilhas a positive chopper value (+) inFIG.7andFIG.8, the second column wire506i2has a negative chopper value (X) inFIG.7andFIG.8, and the first row wire504i1has a negative chopper value (X) inFIG.7andFIG.8. When voltage pulses are supplied from the column wires506as input to this forward cycle, then a vector-matrix product is computed. The method300includes updating a hidden matrix H (block314).

FIG.9is a diagram illustrating the hidden matrix H902being updated with the values calculated in the forward cycle of the A matrix904. The hidden matrix H902is a digital matrix rather than a physical device like the A matrix and the weight matrix W, that stores an H value906(i.e., Hij) for each RPU in the A matrix (i.e., each RPU located at Aij). As the forward cycle is performed, an output vector y′eiTis produced, alternatively called ω. This output vector is used to compute the other digital matrices as detailed below, and is also used to update the hidden matrix H. Thus, each time the output vector is read, the hidden matrix H902changes. For those RPUs with low noise levels, the H value906will grow consistently. For constant gradients and inputs, the growth of the value may be in the positive or negative direction depending on the value of the output vector ω. If the output vector ω includes significant noise, then its values are likely to be positive for one iteration and negative for another. This combination of positive and negative output vector ω values means that the H value906will grow more slowly and more inconsistently.

The hidden matrix value may be updated on the fly using the digital storage storing and updating a value of μ as such:

For the digital compute in each transfer cycle do (for each element i of one read-out vector k):

Where ω is the read-out weight vector, hikis the digital buffer value, skthe current chopper sign, λ is a learning rate and μ is a floating point reference which changes over time and on various iterations. K may be increased with wrap around every ns updates onto M.

Each time a vector k is read, after the digital compute, the buffer (with threshold) may be written to the weight matrix W. γ is a user-defined parameter and positive or zero and usually set to 2/p where p is the switching frequency, assuming regular switching.

As the H values906grow, the method300includes tracking whether the H values906have grown larger than a threshold (block316). If the H value906at a particular location (i.e., Hij) is not larger than the threshold (block316“No”), then the method300repeats from performing the forward cycle (block304) through updating the hidden matrix H (block314) and potentially flipping the chopper value (block320-322). If the H value906is larger than the threshold (block316“Yes”), then the method300proceeds to transmitting input vector eito the weight matrix W, but only for the specific RPU (block318). As mentioned above, the growth of the H value906may be in the positive or negative direction, so the threshold is also a positive or negative value.FIG.10is a schematic diagram of the hidden matrix H902being selectively applied back to the weight matrix W1010according to an embodiment.

FIG.10shows a first H value1012, and a second H value1014that have reached over the threshold value and are being transmitted to the weight matrix W1010. The first H value1012reached the positive threshold, and therefore carries a positive one: “1” for its row in the input vector1016. The second H value1014reached the negative threshold, and therefore carries a negative one: “−1” for its row in the input vector1016. The rest of the rows in the input vector1016carry zeroes, since those values (i.e., H values906) have not grown larger than the threshold value. The threshold value may be much larger than the values being added to the hidden matrix H. For example, the threshold may be ten times or one hundred times the expected strength of the updated values per cycle. Since no bias is added onto the H matrix because of on-the-fly computed reference values, the threshold does typically not need to be overly large. Higher threshold values reduce the frequency of the updates performed on weight matrix W. The filtering function performed by the H matrix, however, decreases the error of the objective function of the neural network. These updates can only be generated after processing many data examples and therefore also increase the confidence level in the updates. This technique enables training of the neural network with noisy RPU devices having only limited number of states even with shifting or unstable symmetry points. After the H value is applied to the weight matrix W, the H value906is reset to zero, and the iteration of the method300continues.

The method300also includes flipping the sign of the chopper value at a flip percentage (block320). The chopper value, in certain embodiments, is flipped only after the chopper product is added to the hidden matrix H. That is, the chopper value is used twice: once when the activation values and error values are written to the A matrix; and once when the forward cycle is read from the A matrix. The chopper value should not be flipped before the H matrix is updated. The flip percentage may be defined as a user preference such that after each chopper product is added to the hidden matrix H, the chopper has a percentage chance of flipping the chopper value. For example, a user preference may be fifty percent, such that half of the time, the chopper value has a chance of changing the sign (i.e., positive to negative or negative to positive) after the chopper product is calculated. In other embodiments the chopper may be flipped every three or four times through the cycle, for example.

When the chopper is determined to be flipped (Yes, block320) then the digital buffer values are further updated for on the fly reference estimation. For example, the following updates may occur:

μikpast←μik,μ past is updated with the current μ value for the ithrow and kthcolumn.

μaik←0, the value of μikis reset.

sk←−s, the chopper value is flipped.

The memory space usage can be reduced to 2 times, when μpastikis set to the last w value during reset (for example, omitting the running mean with γ=1).

After the chopper value is flipped, and the μpastis updated, the method300continues by determining whether training is complete. If the training is not complete, for example a certain convergence criterion is not met (block324“No”), then the method300repeats starting again by performing the forward cycle γ=Wx. For instance, by way of example only, the training can be considered complete when no more improvement to the error signal is seen. When training is completed (block324“Yes”), the method300ends.

As highlighted above, according to an example embodiment, the input vector eiis a one hot encoded vector which is a group of bits having only those combinations with a single high (1) bit and all other bits a low (0). See, for example,FIG.11. As shown inFIG.11, given a matrix of the size 4×4, the one hot encoded vectors will be one of the following vectors: [1 0 0 0], [0 1 0 0], [0 0 1 0] and [0 0 0 1]. At each time step a new one hot encoded vector is used, denoted by the sub index i at that time index.

FIG.12is a diagram illustrating an example detailed algorithm according to an embodiment of the present disclosure.FIG.13is a diagram illustrating an example detailed sub-algorithm according to an embodiment of the present disclosure.

Turning now toFIG.14, a block diagram is shown of an apparatus1400for implementing one or more of the methodologies presented herein. By way of example only, apparatus1400can be configured to control the input voltage pulses applied to the arrays and/or process the output signals from the arrays.

Apparatus1400includes a computer system1410and removable media1450. Computer system1410includes a processor device1420, a network interface1425, a memory1430, a media interface1435and an optional display1440. Network interface1425allows computer system1410to connect to a network, while media interface1435allows computer system1410to interact with media, such as a hard drive or removable media1450.

Processor device1420can be configured to implement the methods, steps, and functions disclosed herein. The memory1430could be distributed or local and the processor device1420could be distributed or singular. The memory1430could be implemented as an electrical, magnetic or optical memory, or any combination of these or other types of storage devices. Moreover, the term “memory” should be construed broadly enough to encompass any information able to be read from, or written to, an address in the addressable space accessed by processor device1420. With this definition, information on a network, accessible through network interface1425, is still within memory1430because the processor device1420can retrieve the information from the network. It should be noted that each distributed processor that makes up processor device1420generally contains its own addressable memory space. It should also be noted that some or all of computer system1410can be incorporated into an application-specific or general-use integrated circuit. Optional display1440is any type of display suitable for interacting with a human user of apparatus1400. Generally, display1440is a computer monitor or other similar display.

CONCLUSION

Aspects of the present disclosure are described herein with reference to call flow illustrations and/or block diagrams of a method, apparatus (systems), and computer program products according to embodiments of the present disclosure. It will be understood that each step of the flowchart illustrations and/or block diagrams, and combinations of blocks in the call flow illustrations and/or block diagrams, can be implemented by computer readable program instructions.

While the foregoing has been described in conjunction with example embodiments, it is understood that the term “example” is merely meant as an example, rather than the best or optimal. Except as stated immediately above, nothing that has been stated or illustrated is intended or should be interpreted to cause a dedication of any component, step, feature, object, benefit, advantage, or equivalent to the public, regardless of whether it is or is not recited in the claims.