Patent Description:
Non-Volatile Memory (NVM)-based crossbar architectures provide an alternative mechanism for performing MAC operations in machine-learning algorithms, particularly, neural-networks. The mixed-signal approach using NVM-bit cells relies upon Ohm's law to implement multiply operations by taking advantage of the resistive nature of emerging NVM technologies (e.g., phase change memory (PCM), resistive random-access memory (RRAM), correlated electron random access memory (CeRAM), and the like). An application of a voltage-bias across an NVM-bit cell generates a current that is proportional to the product of the conductance of the NVM element and the voltage-bias across the cell.

Currents from multiple bit cells are added in parallel to implement an accumulated sum. Thus, a combination of Ohm's law and Kirchoff's current law implements multiple MAC operations in parallel. These, however, can be energy-intensive when implemented using explicit multipliers and adders in the digital domain.

MAC operations are utilized in ML applications, such as artificial neural networks (ANNs) including deep neural networks (DNNs), convolutional neural networks (CNNs), etc. MAC acceleration utilizing NVM crossbars requires programming NVM elements with precision conductance levels that represent a multi-bit weight parameter. Due to inherent device limitations, the bit-precision that can be represented is limited to <NUM> or <NUM> bits, which provides <NUM> to <NUM> distinct conductance levels. This complicates the weight programming step since the entire crossbar array of NVM bits needs to be precisely programmed (capacities of <NUM> - 10Mb are typical).

Specific embodiments of the present disclosure will now be described in detail regarding the accompanying figures. For simplicity and clarity of illustration, where considered appropriate, reference numerals may be repeated among the figures to indicate corresponding or analogous elements. In addition, numerous specific details are set forth in order to provide a thorough understanding of the examples described herein. However, it will be understood by those of ordinary skill in the art that the examples described herein may be practiced without these specific details. In other instances, well-known methods, procedures and components have not been described in detail so as not to obscure the examples described herein. Also, the description is not to be considered as limiting the scope of the examples described herein.

In accordance with the present disclosure, there is provided an improved technique for refactoring MAC operations to reduce programming steps in such systems described above.

In one embodiment of the present disclosure, an apparatus includes a summing array and a multiplying array. The summing array includes a plurality of non-volatile memory elements arranged in columns. Each non-volatile memory element in the summing array is programmed to a high resistance state or a low resistance state based on weights of a neural network. The summing array is configured to generate a summed signal for each column based, at least in part, on a plurality of input signals. The multiplying array is coupled to the summing array, and includes a plurality of non-volatile memory elements. Each non-volatile memory element in the multiplying array is programmed to a different conductance level based on weights of a neural network. The multiplying array is configured to generate an output signal based, at least in part, on the summed signals from the summing array.

ANNs, such as DNNs, CNNs, etc., are a popular solution to a wide array of challenging classification, recognition and regression problems. However, many ANN models require a large number of calculations involving a large number of weights and activations, which presents a significant challenge with respect to access, storage and performance, particularly for mobile and other power or storage-constrained devices. An ANN hardware accelerator accelerates these calculations, such as, for example, convolution operations performed by CNNs.

An ANN models the relationships between input data or signals and output data or signals using a network of interconnected nodes that is trained through a learning process. The nodes are arranged into various layers, including, for example, an input layer, one or more hidden layers, and an output layer. The input layer receives input data, such as, for example, image data, and the output layer generates output data, such as, for example, a probability that the image data contains a known object. Each hidden layer provides at least a partial transformation of the input data to the output data. A DNN has multiple hidden layers in order to model complex, nonlinear relationships between input data and output data.

In a fully-connected, feedforward ANN, each node is connected to all of the nodes in the preceding layer, as well as to all of the nodes in the subsequent layer. For example, each input layer node is connected to each hidden layer node, each hidden layer node is connected to each input layer node and each output layer node, and each output layer node is connected to each hidden layer node. Additional hidden layers are similarly interconnected. Each connection has a weight value, and each node has an activation function, such as, for example, a linear function, a step function, a sigmoid function, a tanh function, a rectified linear unit (ReLU) function, etc., that determines the output of the node based on the weighted sum of the inputs to the node. The input data propagates from the input layer nodes, through respective connection weights to the hidden layer nodes, and then through respective connection weights to the output layer nodes.

More particularly, at each input node, input data is provided to the activation function for that node, and the output of the activation function is then provided as an input data value to each hidden layer node. At each hidden layer node, the input data value received from each input layer node is multiplied by a respective connection weight, and the resulting products are summed or accumulated into an activation value that is provided to the activation function for that node. The output of the activation function is then provided as an input data value to each output layer node. At each output layer node, the output data value received from each hidden layer node is multiplied by a respective connection weight, and the resulting products are summed or accumulated into an activation value that is provided to the activation function for that node. The output of the activation function is then provided as output data. Additional hidden layers may be similarly configured to process data.

<FIG> depicts ANN <NUM>, in accordance with an embodiment of the present disclosure.

ANN <NUM> includes input layer <NUM>, one or more hidden layers <NUM>, <NUM>, <NUM>, etc., and output layer <NUM>. Input layer <NUM> includes one or more input nodes <NUM>, <NUM>, <NUM>, etc. Hidden layer <NUM> includes one or more hidden nodes <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, etc. Hidden layer <NUM> includes one or more hidden nodes <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, etc. Hidden layer <NUM> includes one or more hidden nodes <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, etc. Output layer <NUM> includes one or more output nodes <NUM>, <NUM>, etc. Generally, ANN <NUM> includes N hidden layers, input layer <NUM> includes "i" nodes, hidden layer <NUM> includes "j" nodes, hidden layer <NUM> includes "k" nodes, hidden layer <NUM> includes "m" nodes, and output layer <NUM> includes "o" nodes.

In one embodiment, N equals <NUM>, i equals <NUM>, j, k and m equal <NUM> and o equals <NUM> (depicted in <FIG>). Input node <NUM> is coupled to hidden nodes <NUM> to <NUM>, input node <NUM> is coupled to hidden nodes <NUM> to <NUM>, and input node <NUM> is coupled to hidden nodes <NUM> to <NUM>. Hidden node <NUM> is coupled to hidden nodes <NUM> to <NUM>, hidden node <NUM> is coupled to hidden nodes <NUM> to <NUM>, hidden node <NUM> is coupled to hidden nodes <NUM> to <NUM>, hidden node <NUM> is coupled to hidden nodes <NUM> to <NUM>, and hidden node <NUM> is coupled to hidden nodes <NUM> to <NUM>. Hidden node <NUM> is coupled to hidden nodes <NUM> to <NUM>, hidden node <NUM> is coupled to hidden nodes <NUM> to <NUM>, hidden node <NUM> is coupled to hidden nodes <NUM> to <NUM>, hidden node <NUM> is coupled to hidden nodes <NUM> to <NUM>, and hidden node <NUM> is coupled to hidden nodes <NUM> to <NUM>. Hidden node <NUM> is coupled to output nodes <NUM> and <NUM>, hidden node <NUM> is coupled to output nodes <NUM> and <NUM>, hidden node <NUM> is coupled to output nodes <NUM> and <NUM>, hidden node <NUM> is coupled to output nodes <NUM> and <NUM>, and hidden node <NUM> is coupled to output nodes <NUM> and <NUM>.

<FIG> depicts a high-level block diagram <NUM> of an ANN that uses MAC operations, in accordance with an embodiment of the present disclosure.

The MAC operation may be represented by Equation <NUM>, where element b and element c are multiplied together, added to the value, a, stored in the accumulator, and then stored as a new value, a, in the accumulator.

The composition of a group of MACs may represent dot-products and vector-matrix multiplication. <FIG> represents a simple, multi-layer fully connected ANN <NUM>, such as ANN <NUM>. Each of the neurons in first layer <NUM>-<NUM> performs a MAC operation between input vector <NUM> and the corresponding weights in matrix <NUM>. An activation function is applied to the partial result, and an output vector is generated, which is provided as the input vector to the next layer, i.e., second layer <NUM>-<NUM>. This process takes place for every neuron in each layer. For example, the last layer <NUM>-L performs MAC operations between the output vector of the preceding layer and the corresponding weights in matrix <NUM>. An activation function is applied to the partial result, and an output vector <NUM> is generated. In many embodiments, an output layer may follow layer <NUM>-L.

A CNN may be used for classification or recognition applications, such as image recognition, speech recognition, etc. A CNN has an input layer, an output layer and multiple hidden layers including convolutional layers, pooling layers, normalization layers, fully-connected layers, etc. Each convolutional layer applies a sliding dot product or cross-correlation to an input volume, applies an activation function to the results, and then provides the activation or output volume to the next layer. Convolutional layers typically use the ReLU function as the activation function. In certain embodiments, the activation function is provided in a separate activation layer, such as, for example, a ReLU layer. A pooling layer reduces the dimensions of the output volume received from the preceding convolutional layer, and may calculate an average or a maximum over small clusters of data, such as, for example, 2x2 matrices. In certain embodiments, a convolutional layer and a pooling layer may form a single layer of a CNN. The fully-connected layers follow the convolutional and pooling layers, and include a flatten layer and a classification layer, followed by a normalization layer that includes a normalization function, such as the SoftMax function. The output layer follows the last fully-connected layer; in certain embodiments, the output layer may include the normalization function.

<FIG> depicts CNN <NUM>, in accordance with an embodiment of the present disclosure.

CNN <NUM> includes input layer <NUM>, one or more hidden layers, such as convolutional layer <NUM>-<NUM>, pooling layer <NUM>-<NUM>, hidden (flatten) layer <NUM>, hidden (classification) layer <NUM>, etc., and output layer <NUM>. Many other variations of input, hidden and output layers are contemplated.

Input layer <NUM> includes one or more input nodes <NUM>, etc., that present the input data, such as a color image, as an input volume to the first convolutional layer, e.g., convolutional layer <NUM>-<NUM>. The input volume is a three-dimensional matrix that has a width, a height and a depth. For example, input data that represent a color image are presented as an input volume that is <NUM> pixels x <NUM> pixels x <NUM> channels (red, green, blue); other input volume dimensions may also be used, such as 32x32x3, 64x64x3, 128x128x3, etc., 32x32x1, 64x64x1, 128x128x1, 512x512x1, etc..

Convolutional layer <NUM>-<NUM> is locally-connected to input layer <NUM>, and includes a plurality of nodes that are connected to local regions in the input volume (not depicted for clarity). For a CNN that uses a standard convolution, each node computes a dot product between the node's weights and the respective local region of the input volume. An activation function is then applied to the results of each convolution calculation to produce an output volume that is provided as an input volume to the subsequent layer. The activation function may be applied by each convolutional layer node or by the nodes of a subsequent locally-connected ReLU layer.

Pooling layer <NUM>-<NUM> is locally-connected to convolutional layer <NUM>-<NUM>, and includes a plurality of nodes that are connected to local regions in the input volume (not depicted for clarity). Pooling layer <NUM>-<NUM> also produces an output volume that is provided as the input volume to the subsequent layer, such as, for example, another convolutional layer <NUM>-<NUM>, a flatten layer <NUM>, etc. In certain embodiments, convolutional layer <NUM>-<NUM> and pooling layer <NUM>-<NUM> form a single hidden layer <NUM>. Similarly, in certain embodiments, convolutional layer <NUM>-<NUM>, a ReLU layer and pooling layer <NUM>-<NUM> form a single hidden layer <NUM>. Generally, the output volumes of the convolutional and pooling layers may be described as feature maps, and one or more single hidden layers <NUM> form a feature learning portion of CNN <NUM>.

Hidden layer <NUM> is a "flatten" layer that is locally-connected to pooling layer <NUM>-<NUM>, and includes one or more hidden (flatten) nodes <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, etc. Hidden (flatten) layer <NUM> "flattens" the output volume produced by the preceding pooling layer <NUM>-<NUM> into a column vector, which is provided to the subsequent, fully-connected hidden layer <NUM>.

Hidden layer <NUM> is a classification layer that is fully-connected to hidden (flatten) layer <NUM>, and includes one or more hidden (classification) nodes <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, etc..

Output layer <NUM> includes one or more output nodes <NUM>, <NUM>, etc., and is fully-connected to hidden (classification) layer <NUM>. Fully-connected output layer <NUM> receives the classification results output by hidden (classification) layer <NUM>, and each node outputs a predicted class score. A normalization function, such as a Softmax function, may be applied to the predicted class scores by output layer <NUM>, or, alternatively, by an additional layer interposed between hidden (classification) layer <NUM> and output layer <NUM>.

<FIG> depicts a convolution operation <NUM> within a convolutional layer <NUM> of a CNN <NUM>, in accordance with an embodiment of the present disclosure. This convolutional layer generally includes "M" filters, "C" input channels, "C" input feature maps (i.e., one input feature map for each input channel) and "M" output feature maps (i.e., one output feature map for each filter). Each filter has "C" weight sets (i.e., each filter has a weight set for each input channel), and is convolved across the input feature maps to produce an output feature map corresponding to that filter. Convolutional layers generally require the movement of large amounts of data, generate a significant computational load, and require buffers of considerable size to store intermediate values.

More particularly, convolutional operation <NUM> includes "M" filters <NUM><NUM>, <NUM><NUM>,. , <NUM>M, input feature maps <NUM> that include "C" input data matrices <NUM><NUM>, <NUM><NUM>,. , <NUM>N (i.e., N equals C) and output feature maps <NUM> that includes "M" output data matrices <NUM><NUM>, <NUM><NUM>,. For illustration purposes, the weight set within each filter includes a 2x2 weight matrix, each input feature map includes a 6x6 input data matrix, and each output feature map includes a 5x5 output data matrix. The total number of operations are 2x2xNx5x5xM for the specific arrangement shown in <FIG>.

<FIG> depicts another convolutional layer operation <NUM> for CNN <NUM>, in accordance with an embodiment of the present disclosure. Convolution layer operation <NUM> is a simplified version of convolution layer operation <NUM> that includes three input channels, i.e., "C" and "N," equal <NUM>, and a single filter <NUM><NUM>, i.e., "M" equals <NUM>.

Input feature maps <NUM> (6x6x3) include input data matrix <NUM><NUM>, input data matrix <NUM><NUM> and input data matrix <NUM><NUM>, filter <NUM><NUM> (2x2x3) includes weight matrix <NUM><NUM> (w<NUM>), weight matrix <NUM><NUM> (w<NUM>), and weight matrix <NUM><NUM> (w<NUM>), and output data matrix <NUM><NUM> (5x5x1) includes an output data matrix. Filter <NUM><NUM> is convolved with input feature maps <NUM> to produce an output feature map <NUM> with a single output data matrix <NUM><NUM>. In this example, the output data matrix element o<NUM> is the sum of the dot products of filter <NUM><NUM> (w<NUM>) and the upper left quadrant of input data matrix <NUM><NUM> (a<NUM>q1), filter <NUM><NUM> (w<NUM>) and the upper left quadrant of input data matrix <NUM><NUM> (a<NUM>q1), and filter <NUM><NUM> (w<NUM>) and the upper left quadrant of input data matrix <NUM><NUM> (a<NUM>q1).

More particularly, the dot product of filter <NUM><NUM> (w<NUM>) and the upper left quadrant of input data matrix <NUM><NUM> (a<NUM>q1) is equal to w<NUM><NUM> • a<NUM><NUM> + w<NUM><NUM> • a<NUM><NUM> + w<NUM><NUM> • a<NUM><NUM> + w<NUM><NUM> • a<NUM><NUM>. The dot products of filter <NUM><NUM> (w<NUM>) and the upper left quadrant of input data matrix <NUM><NUM> (a<NUM>q1), and filter <NUM><NUM> (w<NUM>) and the upper left quadrant of input data matrix <NUM><NUM> (a<NUM>q1) are calculated in the same manner, i.e., the dot product of filter <NUM><NUM> (w<NUM>) and the upper left quadrant of input data matrix <NUM><NUM> (a<NUM>q1) is equal to w<NUM><NUM> • a<NUM><NUM> + w<NUM><NUM> • a<NUM><NUM> + w<NUM><NUM> • a<NUM><NUM> + w<NUM><NUM> • a<NUM><NUM>, and the dot product of filter <NUM><NUM> (w<NUM>) and the upper left quadrant of input data matrix <NUM><NUM> (a<NUM>q1) is equal to w<NUM><NUM> • a<NUM><NUM> + w<NUM><NUM> • a<NUM><NUM> + w<NUM><NUM> • a<NUM><NUM> + w<NUM><NUM> • a<NUM><NUM>.

Output data matrix element o<NUM> is the sum of the dot products of filter <NUM><NUM> (w<NUM>) and the next upper quadrant of input data matrix <NUM><NUM>, filter <NUM><NUM> (w<NUM>) and the next upper quadrant of input data matrix <NUM><NUM>, and filter <NUM><NUM> (w<NUM>) and the next upper quadrant of input data matrix <NUM><NUM>. The "next" upper quadrant in each input data matrix <NUM><NUM>, <NUM><NUM> and <NUM><NUM> has been shifted one column to the right relative to the first upper quadrant.

More particularly, the dot product of filter <NUM><NUM> (w<NUM>) and the next upper left quadrant of input data matrix <NUM><NUM> is equal to w<NUM><NUM> • a<NUM><NUM> + w<NUM><NUM> • a<NUM><NUM> + w<NUM><NUM> • a<NUM><NUM> + w<NUM><NUM> • a<NUM><NUM>. The dot products of filter <NUM><NUM> (w<NUM>) and the next upper left quadrant of input data matrix <NUM><NUM>, and filter <NUM><NUM> (w<NUM>) and the next upper left quadrant of input data matrix <NUM><NUM> are calculated in the same manner, i.e., the dot product of filter <NUM><NUM> (w<NUM>) and the upper left quadrant of input data matrix <NUM><NUM> (a<NUM>q1) is equal to w<NUM><NUM> • a<NUM><NUM> + w<NUM><NUM> • a<NUM><NUM> + w<NUM><NUM> • a<NUM><NUM> + w<NUM><NUM> • a<NUM><NUM>, and the dot product of filter <NUM><NUM> (w<NUM>) and the upper left quadrant of input data matrix <NUM><NUM> (a<NUM>q1) is equal to w<NUM><NUM> • a<NUM><NUM> + w<NUM><NUM> • a<NUM><NUM> + w<NUM><NUM> • a<NUM><NUM> + w<NUM><NUM> • a<NUM><NUM>.

The remaining output data matrix elements o<NUM> to o<NUM> are calculated in a similar manner. After output data matrix element o<NUM> has been calculated, the "next" quadrant in each input data matrix <NUM><NUM>, <NUM><NUM> and <NUM><NUM> is shifted all the way to the leftmost column and down one row for the calculation of output data matrix element o<NUM>. Output data matrix elements o<NUM> to o<NUM> are then calculated by shifting the quadrant in each input data matrix <NUM><NUM>, <NUM><NUM> and <NUM><NUM> one column to the right. The remaining output data matrix elements o<NUM> to <NUM>, o<NUM> to <NUM>, and <NUM> to <NUM> are calculated in a similar manner.

Typically, native convolution operations are not performed by a CNN due to the complicated dataflow and expensive datapaths that are usually required. Instead, native convolution operations are converted into generic matrix multiplication (GEMM) operations, and then the GEMM operations are executed more efficiently by a central processing unit (CPU), specialized processor, hardware accelerator processing engine, etc., using optimized software libraries or specialized hardware. More particularly, an "IM2COL" software function may be used to convert the filter (weight) matrix and the input feature map (IFM) matrix for each convolution operation into an expanded format that is compatible with a GEMM operation. The IM2COL versions of each filter (weight) matrix and each IFM matrix are generated and stored in memory, and then loaded from memory and processed by the GEMM operation.

<FIG> depicts a converted convolutional layer operation for a CNN, in accordance with an embodiment of the present disclosure.

In one embodiment, the convolutional layer operation for CNNs executing on central processor units (CPUs) may be converted into generic matrix multiplication (GEMM) operations. Convolution layer operation <NUM> is converted into a GEMM operation by converting filter <NUM><NUM> into converted weight matrix <NUM> (1x12) and input feature maps <NUM> into converted input data matrix <NUM> (12x25). After multiplying converted weight matrix <NUM> and converted input data matrix <NUM>, converted output data matrix <NUM> (1x25) is then reformed into output data matrix <NUM><NUM> (5x5). For ease of illustration, converted input data matrix <NUM> is depicted in abbreviated form.

In this example, converted output data matrix element o<NUM> is the sum of the dot products of the first (i.e., only) row of converted weight matrix <NUM> and the first column of converted input data matrix <NUM>. As shown in <FIG>, the converted weight matrix <NUM> includes filter <NUM><NUM> (w<NUM>), filter <NUM><NUM> (w<NUM>), and filter <NUM><NUM> (w<NUM>), while the first row of converted input data matrix <NUM> includes the elements of the upper left quadrant of input data matrix <NUM>, (a<NUM>q1), the upper left quadrant of input data matrix <NUM><NUM> (a<NUM>q1), and the upper left quadrant of input data matrix <NUM><NUM> (a<NUM>q1).

More particularly, the converted output data matrix element o<NUM> is equal to w<NUM><NUM> • a<NUM><NUM> + w<NUM><NUM> • a<NUM><NUM> + w<NUM><NUM> • a<NUM><NUM> + w<NUM><NUM> • a<NUM><NUM> + w<NUM><NUM> • a<NUM><NUM> + w<NUM><NUM> • a<NUM><NUM> + w<NUM><NUM> • a<NUM><NUM> + w<NUM><NUM>. a<NUM><NUM> + w<NUM><NUM> • a<NUM><NUM> + w<NUM><NUM> • a<NUM><NUM> + w<NUM><NUM> • a<NUM><NUM> + w<NUM><NUM> • a<NUM><NUM>. As shown above, the converted output data matrix element o<NUM> is equal to the output data matrix element o<NUM>.

<FIG> depicts a portion of the convolutional layer operation of <FIG> implemented using NVM crossbar <NUM>, in accordance with an embodiment of the present disclosure.

NVM crossbar <NUM> includes twelve row signal lines <NUM>, i.e., row signal lines <NUM><NUM>,. , <NUM><NUM>, and a single column signal line <NUM>. A bitcell is disposed at each intersection of row signal lines <NUM> and the column signal line <NUM>. Each bitcell includes one or more NVM elements that are programmed to represent a conductance value that is proportional to a particular of weight of filter <NUM><NUM>, i.e., w<NUM><NUM>, w<NUM><NUM>, etc. The conductances are represented by g<NUM><NUM>, g<NUM><NUM>, g<NUM><NUM>, g<NUM><NUM>, g<NUM><NUM>, g<NUM><NUM>, g<NUM><NUM>, g<NUM><NUM>, g<NUM><NUM>, g<NUM><NUM>, g<NUM><NUM>, and g<NUM><NUM>, which are proportional to weights w<NUM><NUM>, w<NUM><NUM>, w<NUM><NUM>, w<NUM><NUM>, w<NUM><NUM>, w<NUM><NUM>, w<NUM><NUM>, w<NUM><NUM>, w<NUM><NUM>, w<NUM><NUM>, w<NUM><NUM>, and w<NUM><NUM>, respectively.

In order to calculate the first element of converted output data matrix <NUM>, i.e., o<NUM>, input data from the first column of converted input data matrix <NUM> are provided to crossbar <NUM> as inputs a<NUM><NUM>, a<NUM><NUM>, a<NUM><NUM>, a<NUM><NUM>, a<NUM><NUM>, a<NUM><NUM>, a<NUM><NUM>, a<NUM><NUM>, a<NUM><NUM>, a<NUM><NUM>, a37 and a<NUM><NUM> along row signal lines <NUM><NUM>,. , <NUM><NUM>, respectively. The input data are converted to analog voltages v<NUM><NUM>, v<NUM><NUM>, v<NUM><NUM>, v<NUM><NUM>, v<NUM><NUM>, v<NUM><NUM>, v<NUM><NUM>, v<NUM><NUM>, v<NUM><NUM>, v<NUM><NUM>, v<NUM><NUM> and v<NUM><NUM> by digital-to-analog convertors (DACs) <NUM>, and provided to column signal line <NUM>. Column signal line <NUM> conveys a bit line (BL) signal, which is proportional to the accumulated dot-products of the input data and weights along column signal line <NUM>. The BL signal is then digitized using an analog-to-digital converter (ADC); bias, scaling and activation functions may be applied to the digitized signal to obtain the first element of converted output data matrix <NUM>, i.e., o<NUM>.

<FIG> depicts an architecture <NUM> for a convolutional layer operation implemented using NVM crossbar <NUM>, in accordance with an embodiment of the present disclosure.

NVM crossbar <NUM> includes row signal lines <NUM>, i.e., row signal lines <NUM><NUM>, <NUM><NUM>,. , <NUM>L, and column signal lines <NUM>, i.e., column signal lines <NUM><NUM>, <NUM><NUM>,. A bitcell <NUM> is disposed at each intersection of row signal lines <NUM> and column signal lines <NUM>, i.e., bitcells <NUM><NUM>,. Each bitcell <NUM> includes one or more NVM elements that are programmed to represent a conductance value that is proportional to a particular of weight of a particular filter <NUM>, such as, for example, W<NUM><NUM> of filter <NUM><NUM>, etc. In the first column, i.e., column signal line <NUM><NUM>, the conductances are depicted as G<NUM><NUM>, G<NUM><NUM>,. , GC<NUM>, GC<NUM>.

It is generally useful to encode multiple linearly-separated resistance levels within an individual bit cell due to the wide separation between low-resistance state (LRS) RON and the high-resistance state (HRS) ROFF. In the case of correlated electron material RAM (CeRAM), the ratio of HRS/LRS is at least <NUM>-orders of magnitude, which supports <NUM>-bit encoding (i.e., <NUM> resistance levels).

Data from the input feature maps <NUM> are provided to crossbar <NUM> as inputs I<NUM><NUM>,. , IC<NUM> along the row signal lines <NUM><NUM>,. , <NUM>L, converted to analog voltages V<NUM><NUM>,. ,VC<NUM> by a digital-to-analog convertor (DAC), and conveyed across the NVM cell. In this embodiment, L is equal to four times the number of input channels, i.e., 2x2xC. The DAC includes DAC elements <NUM><NUM>,. The column signal lines <NUM><NUM>,. , 312nn convey corresponding bit line (BL) signals BL<NUM>,. , BLM, each of which is proportional to the accumulated dot-products of the input data and weights along that column signal line. Each BL signal is then digitized using an analog-to-digital converter (ADC); bias, scaling and activation functions may be applied to the digitized signals to obtain the output data for output feature maps <NUM>.

Mapping a convolutional layer operation to an NVM crossbar with M filters and C input channels is generally depicted In <FIG>. In certain embodiments, the weights are stationary, i.e., the weights are programmed into the NVM crossbar once and do not change during the course of inference operations. Typical NVM elements, such as, for example, Phase-Change Memory and Resistive RAM, have a limited write "endurance," and it is possible to write to them a limited number of times (e.g., approximately <NUM><NUM> times) after which the devices exhibit functional failure. Other NVM elements, such as, for example, Magnetic RAM and CeRAM, may exhibit relatively higher endurance (near <NUM><NUM>), but continuous operation may lead to a limited lifetime. Hence, such lifetime limits pose significant constraints on accelerator architectures that rely upon updating weights on a per-inference cycle.

For example, for an loT-class accelerator operating at <NUM>, an accelerator with an endurance of <NUM><NUM> may have a lifetime of <NUM> second, while an accelerator with an endurance of <NUM><NUM> may have a lifetime of <NUM>,<NUM> seconds or <NUM> days for a peak usage case. Unfortunately, such weights cannot be practically streamed from an external DRAM and must be fixed on-chip. Further, NVM bitcells suffer from high write-powers, and consequent expensive power consumption, which may effect the efficient performance of update operations. Thus, the write phase can be problematic and take a long time to complete.

Such arrangements differ from SRAM behavior, which has significantly higher write endurance, and may not be amenable to reprogramming the weights during inference. As a consequence, the entire ANN may need to be unrolled into an on-chip crossbar and fixed during inference. While this option has the advantage of eliminating DRAM power consumption, it may undesirably limit the maximum size of the ANN that can be programmed on-chip. Further, this option also incurs an area penalty as mapping larger ANNs requires instantiation of crossbars that are megabits in capacity. This consumes higher area and increases susceptibility to chip-failures due to yield loss. Moreover, instantiating multiple crossbars requires instantiation of multiple ADCs/DACs, all of which need to be programmed, trimmed and compensated for drift.

An NVM / CeRAM element is a particular type of random access memory formed, wholly or in part, from a correlated electron material. The CeRAM may exhibit an abrupt conductive or insulative state transition arising from electron correlations rather than solid state structural phase changes such as, for example, filamentary formation and conduction in resistive RAM devices. An abrupt conductor / insulator transition in a CeRAM may be responsive to a quantum mechanical phenomenon, in contrast to melting / solidification or filament formation.

A quantum mechanical transition of a CeRAM between an insulative state and a conductive state may be understood in terms of a Mott transition. In a Mott transition, a material may switch from an insulative state to a conductive state if a Mott transition condition occurs. When a critical carrier concentration is achieved such that a Mott criteria is met, the Mott transition will occur and the state will change from high resistance / impedance (or capacitance) to low resistance/impedance (or capacitance).

A "state" or "memory state" of the CeRAM element may be dependent on the impedance state or conductive state of the CeRAM element. In this context, the "state" or "memory state" means a detectable state of a memory device that is indicative of a value, symbol, parameter or condition, just to provide a few examples. In a particular implementation, a memory state of a memory device may be detected based, at least in part, on a signal detected on terminals of the memory device in a read operation. In another implementation, a memory device may be placed in a particular memory state to represent or store a particular value, symbol or parameter by application of one or more signals across terminals of the memory device in a "write operation.

A CeRAM element may comprise material sandwiched between conductive terminals. By applying a specific voltage and current between the terminals, the material may transition between the aforementioned conductive and insulative states. The material of a CeRAM element sandwiched between conductive terminals may be placed in an insulative state by application of a first programming signal across the terminals having a reset voltage and reset current at a reset current density, or placed in a conductive state by application of a second programming signal across the terminals having a set voltage and set current at set current density.

With respect to NVM crossbars <NUM> and <NUM>, the accumulated dot-products of the input data and weights along each column signal line <NUM> may be represented by Equation <NUM>, which represents a simple MAC operation. Each input data value (i.e., voltage Vi) is multiplied by the appropriate weight (i.e., conductance Gi), and the products are accumulated into a result (i.e., the current along the BL).

Embodiments of the present disclosure advantageously refactor the MAC operation for each column signal line <NUM> into an initial summation of voltages Vi that are to be multiplied by a particular conductance value Gk, followed by a MAC operation that accumulates the product of the each voltage sum and that particular conductance value Gk. In certain embodiments, the conductance value may be represented by a <NUM>-bit (k-bit) number, which provides for <NUM> different values (K values), such as, for example, G<NUM>, G<NUM>, G<NUM>, G<NUM>, G<NUM>, G<NUM>, G<NUM>, G<NUM>, G<NUM>, G<NUM>, G<NUM>, G<NUM>, G<NUM>, G<NUM>, G<NUM> and G<NUM>. Other bit sizes are also contemplated, such as, for example, <NUM>-bits, <NUM>-bits, etc..

For example, with respect to <FIG>, the conductance g<NUM><NUM> may have the value G<NUM>, the conductance g<NUM><NUM> may have the value G<NUM>, the conductance g<NUM><NUM> may have the value G<NUM>, the conductance g<NUM><NUM> may have the value G<NUM>, the conductance g<NUM><NUM> may have the value G<NUM>, the conductance g<NUM><NUM> may have the value G<NUM>, the conductance g<NUM><NUM> may have the value G<NUM>, the conductance g<NUM><NUM> may have the value G<NUM>, the conductance g<NUM><NUM> may have the value G<NUM>, the conductance g<NUM><NUM> may have the value G<NUM>, the conductance g<NUM><NUM> may have the value G<NUM>, and the conductance g<NUM><NUM> may have the value G<NUM>.

In this example, conductances g<NUM><NUM> and g<NUM><NUM> have the same conductance G<NUM>, conductances g<NUM><NUM> and g<NUM><NUM> have the same conductance G<NUM>, conductances g<NUM><NUM> and g<NUM><NUM> have the same conductance G<NUM>, conductances g<NUM><NUM> and g<NUM><NUM> have the same conductance G<NUM>. Accordingly, V<NUM><NUM> and V<NUM><NUM> may be summed first, and then multiplied by conductance G<NUM> during the subsequent MAC operation, V<NUM><NUM> and V<NUM><NUM> may be summed first, and then multiplied by conductance G<NUM> during the subsequent MAC operation, V<NUM><NUM> and V<NUM>, may be summed first, and then multiplied by conductance G<NUM> during the subsequent MAC operation, and V<NUM><NUM> and V<NUM><NUM> may be summed first, and then multiplied by conductance G<NUM> during the subsequent MAC operation.

Additionally, rather summing only the voltages Vi that are to be multiplied by a particular conductance value Gk, the remaining voltages may be set to zero (<NUM>) and then all of the voltages may be initially summed. In the above example, for the conductance G<NUM>, the other <NUM> voltages are set to zero, i.e., V<NUM><NUM>, V<NUM><NUM>, v<NUM><NUM>, v<NUM><NUM>, v<NUM><NUM>, v<NUM><NUM>, V<NUM><NUM>, v<NUM><NUM>, V<NUM><NUM> and V<NUM><NUM>, and then all of the voltages are summed. Generally, this process is repeated for each conductance value, including conductances that are to be multiplied by a single voltage. In the above example, for the conductance G<NUM>, the other <NUM> voltages are set to zero, i.e., V<NUM><NUM>, V<NUM><NUM>, v<NUM><NUM>, v<NUM><NUM>, v<NUM><NUM>, v<NUM><NUM>, v<NUM><NUM>, V<NUM><NUM>, v<NUM><NUM>, V<NUM><NUM> and V<NUM><NUM>, and then all of the voltages are summed.

The refactored MAC operation for each column signal line <NUM> may be represented by Equations <NUM> and <NUM>. <MAT> <MAT> where, for each input data value L, Vi is set to zero (<NUM>) when Gk does not correspond to the appropriate weight for that input data value.

Advantageously, refactoring the MAC operation, as represented by Eq. <NUM>, leads to a simpler implementation where all input multiplicands are initially added to conditionally add togetherthe input activations depending on whetherthey factor into the MAC operation with a specific weight value. The initial addition operation can be done using NVM elements. However, in accordance with embodiments of the present disclosure, these NVM elements need not be precisely programmed. A binary weight encoding (RON / ROFF) is utilized to connect an input activation to a weight value without need for precision programming.

<FIG> depicts a schematic of an architecture <NUM> for a refactored MAC operation, in accordance with an embodiment of the present disclosure. In this embodiment, architecture <NUM> includes DAC block <NUM>, crossbar <NUM> and a multiplier / scaling module <NUM>.

Generally, crossbar <NUM> includes MxN elements. In this embodiment, M equals <NUM>, N equals <NUM> and crossbar <NUM> includes <NUM> elements, nodes or bitcells <NUM><NUM>,<NUM>,. , <NUM><NUM>,<NUM>, provided at each junction of <NUM> row signal lines <NUM><NUM>,. , <NUM><NUM> and <NUM> column signal lines <NUM><NUM>,. , <NUM><NUM>. Each respective bitcell <NUM><NUM>,<NUM>,. , <NUM><NUM>,<NUM> includes one or more NVM elements, represented by sij, to store the state (e.g., RON / ROFF) for that bitcell. Bitcells <NUM> are switchable between a first impedance state and a second impedance state (RON / ROFF).

Multiplier / scaling module <NUM> includes <NUM> bitcells <NUM><NUM>,. , <NUM><NUM> that have programmable conductances or impedances g<NUM>, g<NUM>, g<NUM>, g<NUM>, gs, g<NUM>, g<NUM>, g<NUM>, g<NUM>, g<NUM>, g<NUM>, g<NUM>, g<NUM>, g<NUM>, g<NUM> and g<NUM> that are configured to correspond to <NUM> different weights values w<NUM>, w<NUM>, w<NUM>, w<NUM>, w<NUM>, w<NUM>, w<NUM>, w<NUM>, w<NUM>, w<NUM>, w<NUM>, w<NUM>, w<NUM>, w<NUM>, w<NUM> and w<NUM>, respectively.

Input data, i.e., a<NUM><NUM>, a<NUM><NUM>, a<NUM><NUM>, a<NUM><NUM>, a<NUM><NUM>, a<NUM><NUM>, a<NUM><NUM>, a<NUM><NUM>, a<NUM><NUM>, a<NUM><NUM>, a<NUM><NUM> and a<NUM><NUM>, are provided to DAC block <NUM>, converted to analog voltages v<NUM><NUM>, v<NUM><NUM>, v<NUM><NUM>, v<NUM><NUM>, v<NUM><NUM>, v<NUM><NUM>, v<NUM><NUM>, v<NUM><NUM>, v<NUM><NUM>, v<NUM><NUM>, v<NUM><NUM> and v<NUM><NUM>, and then provided to respective row signal lines <NUM><NUM>,. , <NUM><NUM>. The voltages are first summed together in the crossbar <NUM> by placing the bitcells <NUM><NUM>,. , <NUM>MN into RON / ROFF states, where the NVM elements do not require precision programming.

In the example described above, bitcells <NUM> are that associated with voltages that are to be summed for a particular conductance value are set to the RON state. These bitcells <NUM> include NVM elements s<NUM><NUM>, s<NUM><NUM>, s<NUM><NUM>, s<NUM><NUM>, s<NUM><NUM>, s<NUM><NUM>, s<NUM><NUM>, s<NUM><NUM>, s<NUM><NUM>, s<NUM><NUM>, s<NUM><NUM> and s<NUM><NUM> (identified by dotted circles in <FIG>). The remaining NVM element are set to the ROFF state, which provides a zero voltage value.

A summed signal from each column signal line <NUM><NUM>,. , <NUM><NUM> is input to a respective bitcell <NUM><NUM>,. , <NUM><NUM> of the multiplier / scaling module <NUM>, and the MAC operation occurs in a single column in the multiplier / scaling module <NUM> where all elements are precisely programmed (<NUM> elements in total).

Generally, converted input data matrix <NUM> includes the total number of input data values that are provided to DAC block <NUM>, one column of input data values at a time. In the example depicted in <FIG>, the total number of input data values in input data matrix is <NUM> (i.e., 12x25). Accordingly, the input data signals (e.g., a<NUM><NUM>,. , a<NUM><NUM>) are a set of M (e.g., <NUM>) input signals selected from a total number of input signals (e.g., <NUM>) that is significantly greater than N (e.g., <NUM>).

<FIG> depicts a schematic of an architecture <NUM> for a refactored MAC operation, in accordance with an embodiment of the present disclosure. In this embodiment, architecture <NUM> includes DAC block <NUM> of M DACs <NUM><NUM>,. , <NUM>M, crossbar <NUM> and multiplier / scaling module <NUM>.

In many embodiments, architecture <NUM> encodes the weights for a single filter <NUM>, and generates a single, output feature map <NUM> for that filter <NUM>. In other embodiments, a separate crossbar <NUM> and multiplier / scaling module <NUM> pair may be provided for each filter <NUM> to generate the output feature map <NUM> for that filter <NUM>. Each crossbar <NUM> and multiplier / scaling module <NUM> pair may be coupled to a common DAC block <NUM>.

Crossbar <NUM> includes MxN elements, nodes or bitcells <NUM><NUM>,. , <NUM>MN, provided at each junction of row signal lines <NUM><NUM>, <NUM><NUM>, <NUM><NUM>,. , <NUM>M and column signal lines <NUM><NUM>, <NUM><NUM>, <NUM><NUM>,. , <NUM>N as shown. Each respective bitcell <NUM><NUM>,. , <NUM>M,N includes one or more NVM elements to store the state (e.g., RON / ROFF) for that bitcell. As described above, bitcells <NUM> are switchable between a first impedance state and a second impedance state (RON / ROFF).

Multiplier / scaling module <NUM> includes bitcells <NUM><NUM>,. , <NUM>N that have programmable conductances or impedances that are programmed to correspond to weights w_0, w_1, w_K-<NUM>. Input data, such as, for example, activations a_0, a_1,. , a_M-<NUM>, along respective row signal lines <NUM><NUM>, <NUM><NUM>, <NUM><NUM>,. , 410nn are first summed together in the crossbar <NUM> by placing the bitcells <NUM><NUM>,. , <NUM>MN into RON / ROFF states, as described above, where the NVM elements do not require precision programming. In this embodiment, certain bitcells among bitcells <NUM><NUM>,<NUM>,. , <NUM>MN are switched between impedance states as shown. A summed signal from each column N is input to a respective bitcell <NUM><NUM>,. , <NUM>N of the multiplier / scaling module <NUM>, and the final MAC computation occurs in a single column in the multiplier / scaling module <NUM> where all elements are precisely programmed (N elements in total).

<FIG> depicts a flow diagram of a low-precision programming process <NUM> for a refactored MAC operation, in accordance with an embodiment of the present disclosure.

In the embodiment depicted in <FIG>, crossbar <NUM> includes MxN elements and multiplier / scaling module <NUM> includes N different weight values; k is the bit size of each weight value (i.e., the number of bits), and N = K = <NUM>k. MxN elements are low-precision (or binary) programmed NVM elements, and N elements are high-precision programmed NVM elements.

In block <NUM>, the process is initialized and proceeds to block <NUM> of the low-precision write loop. In block <NUM>, the indices of the elements of crossbar <NUM> are updated. In block <NUM>, the binary resistance or conductance Gb for each element of crossbar <NUM> is read. Then, in block <NUM>, Gb for each element of crossbar <NUM> is written. The process terminates at block <NUM>. In this embodiment, the MxN elements are programmed to either a "<NUM>" (ROFF) or a "<NUM>" (RON). From here, the elements of multiplier / scaling module <NUM> are precision programmed.

<FIG> depicts a flow diagram of a high-precision programming process <NUM> for multiplier / scaling module <NUM>, in accordance with an embodiment of the present disclosure.

In block <NUM>, the process is initialized and proceeds to block <NUM> of the high-precision write loop. In block <NUM>, the indices of the elements of multiplier / scaling module <NUM> are updated. In block <NUM>, the multilevel resistance or conductance G for each element of multiplier / scaling module <NUM> is read. A high-precision operation changes the conductance or resistivity to a very precise known value. Then, in block <NUM>, the multilevel G is tuned for each element of multiplier / scaling module <NUM>. Then, in block <NUM>, the multilevel G is read, and the correct multilevel G verified in block <NUM>. The process loops back to block <NUM> if the multilevel G is correct, otherwise, the process proceeds back to block <NUM> for further tuning. The process terminates at block <NUM>.

<FIG> depicts a flow diagram <NUM> of a method for performing multiply-accumulate acceleration in a neural network, in accordance with an embodiment of the present disclosure.

The method is initialized at block <NUM>, and proceeds to block <NUM> which corresponds to the low-precision programming process <NUM> shown in <FIG>. With respect to <FIG>, in the low-precision programming process <NUM>, the input activations a_0, a_1,. , a_M-<NUM> are applied to a summing array, i.e., crossbar <NUM>. A summed signal is generated by each column of NVM elements in the summing array by the low-precision programming process <NUM>. The process proceeds to block <NUM>, which corresponds to the high-precision programming process <NUM> shown in <FIG>. In the high-precision programming process <NUM>, the summed signal is input to a multiplying array, i.e., multiplier / scaling module <NUM>, having a plurality of NVM elements. In block <NUM>, each NVM element in the multiplying array is precisely programmed to a resistance or conductance level proportional to a weight, as shown in the flow diagram of <FIG>. The process terminates at block <NUM>.

Reference in the specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the system.

Embodiments of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure.

Some portions of the detailed descriptions, like the processes may be presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. An algorithm may be generally conceived to be steps leading to a desired result. The steps are those requiring physical transformations or manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated.

Unless specifically stated otherwise as apparent from the following discussion, it is appreciated that throughout the description, discussions utilizing terms such as "processing" or "computing" or "calculating" or "determining" or "deriving" or "displaying" or the like, refer to the action and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices.

The operations described herein can be performed by an apparatus. This apparatus may be specially constructed for the required purposes, or it may comprise a general-purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a computer readable storage medium, such as, but is not limited to, any type of disk, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, or any type of media suitable for storing electronic instructions.

Accordingly, embodiments and features of the present disclosure include, but are not limited to, the foregoing and following combinable embodiments.

In one embodiment, an apparatus includes a crossbar and a multiplier. The crossbar includes a plurality of crossbar nodes arranged in an array of rows and columns, each crossbar node is programmable to a first resistance level or a second resistance level, and the crossbar is configured to sum a plurality of analog input activation signals over each column of crossbar nodes and output a plurality of summed activation signals. The multiplier is coupled to the crossbar and includes a plurality of multiplier nodes, each multiplier node is programmable to a different conductance level proportional to one of a plurality of neural network weights, and the multiplier is configured to sum the plurality of summed activation signals over the multiplier nodes and output an analog output activation signal.

In another embodiment of the apparatus, each crossbar node includes one or more non-volatile elements (NVMs), and each multiplier node includes a plurality of NVMs.

In another embodiment of the apparatus, the crossbar includes M rows, N columns and MxN crossbar nodes, and the multiplier includes N multiplier nodes.

In another embodiment of the apparatus, the apparatus further includes a plurality of digital-to-analog converters (DACs) coupled to the crossbar, each DAC being configured to receive a digital input activation signal and output one of the plurality of analog input activation signals.

In another embodiment of the apparatus, each non-volatile memory element in each multiplier node is programmed to the different conductance level based on a bit size, k, of the neural network weights.

In another embodiment of the apparatus, a total number of conductance levels is equal to <NUM>k.

In one embodiment, a method includes generating, in a summing array including a plurality of non-volatile memory elements arranged in columns, a summed signal for each column based, at least in part, on a plurality of input signals, each non-volatile memory element in the summing array being programmed to a high resistance state or a low resistance state based on weights of a neural network; and generating, in a multiplying array including a plurality of non-volatile memory elements, an output signal based, at least in part, on the summed signals from the summing array, each non volatile memory element in the multiplying array being programmed to a different conductance level based on the weights of the neural network.

In another embodiment of the method, each non-volatile memory element in the summing array is configured to receive an input signal and add the input signal to the summed signal for a respective column based on the programmed resistance state; each non-volatile memory element in the multiplying array is configured to receive a summed signal from a different column of the summing array and adjust the summed signal based on the programmed conductance level to generate an adjusted summed signal; and the output signal is a combination of the adjusted summed signals.

In another embodiment of the method, each non-volatile memory element in the summing array is configured to add the input signal to the summed signal for a respective column when programmed to the low resistance state, and not add the input signal to the summed signal for the respective column when programmed to the high resistance state.

In another embodiment of the method, the summing array is a crossbar having M rows, N columns and MxN non-volatile memory elements, and where the multiplying array has N elements.

In another embodiment of the method, the plurality of input signals is a set of M input signals selected from a total number of input signals that is significantly greater than N.

In another embodiment of the method, each non-volatile memory element in the multiplying array is programmed to the different conductance level based on a bit size, k, of the weights of the neural network.

In another embodiment of the method, a total number of conductance levels is equal to <NUM>k.

In one embodiment, another apparatus includes a summing array including a plurality of non-volatile memory elements arranged in columns, each non-volatile memory element in the summing array being programmed to a high resistance state or a low resistance state based on weights of a neural network, the summing array configured to generate a summed signal for each column based, at least in part, on a plurality of input signals; and a multiplying array, coupled to the summing array, including a plurality of non-volatile memory elements, each non volatile memory element in the multiplying array being programmed to a different conductance level based on the weights of the neural network, the multiplying array configured to generate an output signal based, at least in part, on the summed signals from the summing array.

In another embodiment of the apparatus, each non-volatile memory element in the summing array is configured to receive an input signal and add the input signal to the summed signal for a respective column based on the programmed resistance state; each non-volatile memory element in the multiplying array is configured to receive a summed signal from a different column of the summing array and adjust the summed signal based on the programmed conductance level to generate an adjusted summed signal; and the output signal is a combination of the adjusted summed signals.

In another embodiment of the apparatus, each non-volatile memory element in the summing array is configured to add the input signal to the summed signal for the respective column when programmed to the low resistance state, and not add the input signal to the summed signal for the respective column when programmed to the high resistance state.

In another embodiment of the apparatus, the summing array is a crossbar having M rows, N columns and MxN non-volatile memory elements, and where the multiplying array has N elements.

In another embodiment of the apparatus, the plurality of input signals is a set of M input signals selected from a total number of input signals that is significantly greater than N.

In another embodiment of the apparatus, each non-volatile memory element in the multiplying array is programmed to the different conductance level based on a bit size, k, of the weights of the neural network.

It will be appreciated that the examples and corresponding diagrams used herein are for illustrative purposes only. Different configurations and terminology can be used without departing from the principles expressed herein. For instance, components and modules can be added, deleted, modified, or arranged with differing connections without departing from these principles.

In the following detailed description of embodiments of the present disclosure, numerous specific details are set forth in order to provide a more thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure may be practiced without these specific details.

It is to be understood that the terminology used herein is for the purposes of describing various embodiments in accordance with the present disclosure and is not intended to be limiting. The terms "a" or "an," as used herein, are defined as one or more than one. The term "plurality," as used herein, is defined as two or more than two. The term "another," as used herein, is defined as at least a second or more. The terms "including" and/or "having," as used herein, are defined as comprising (i.e., open language). The term "coupled," as used herein, is defined as connected, although not necessarily directly, and not necessarily mechanically. The term "providing" is defined herein in its broadest sense, e.g., bringing/coming into physical existence, making available, and/or supplying to someone or something, in whole or in multiple parts at once or over a period.

As used herein, the terms "about" or "approximately" apply to all numeric values, irrespective of whether these are explicitly indicated. Such terms generally refer to a range of numbers that one of skill in the art would consider equivalent to the recited values (i.e., having the same function or result). These terms may include numbers that are rounded to the nearest significant figure. In this document, any references to the term "longitudinal" should be understood to mean in a direction corresponding to an elongated direction of a personal computing device from one terminating end to an opposing terminating end.

Claim 1:
An apparatus, comprising:
a crossbar including a plurality of crossbar nodes arranged in an array of rows and columns, each crossbar node being programmable to either a high resistance state or a low resistance state, the crossbar being configured to sum a plurality of analog input activation signals over each column of crossbar nodes and output a plurality of summed activation signals; and
a multiplier, coupled to the crossbar, including a plurality of multiplier nodes, each multiplier node being programmable to a different conductance level proportional to one of a plurality of neural network weights, the multiplier being configured to input the plurality of summed activation signals to the multiplier nodes, perform a multiply-and-accumulate (MAC) operation, and output an analog output activation signal.