Patent Description:
Numerous embodiments are disclosed for programmable output blocks for use with a vector-by-matrix multiplication (VMM) array within an artificial neural network.

<FIG> illustrates an artificial neural network, where the circles represent the inputs or layers of neurons. The connections (called synapses) are represented by arrows, and have numeric weights that can be tuned based on experience. This makes neural networks adaptive to inputs and capable of learning. Typically, neural networks include a layer of multiple inputs. There are typically one or more intermediate layers of neurons, and an output layer of neurons that provide the output of the neural network. The neurons at each level individually or collectively make a decision based on the received data from the synapses.

One of the major challenges in the development of artificial neural networks for high-performance information processing is a lack of adequate hardware technology. Indeed, practical neural networks rely on a very large number of synapses, enabling high connectivity between neurons, i.e. a very high computational parallelism. In principle, such complexity can be achieved with digital supercomputers or specialized graphics processing unit clusters. However, in addition to high cost, these approaches also suffer from mediocre energy efficiency as compared to biological networks, which consume much less energy primarily because they perform low-precision analog computation. CMOS analog circuits have been used for artificial neural networks, but most CMOS-implemented synapses have been too bulky given the high number of neurons and synapses required.

Applicant previously disclosed an artificial (analog) neural network that utilizes one or more non-volatile memory arrays as the synapses in <CIT>, published as <CIT>.

The non-volatile memory arrays operate as an analog neuromorphic memory. The neural network device includes a first plurality of synapses configured to receive a first plurality of inputs and to generate therefrom a first plurality of outputs, and a first plurality of neurons configured to receive the first plurality of outputs. The first plurality of synapses includes a plurality of memory cells, wherein each of the memory cells includes spaced apart source and drain regions formed in a semiconductor substrate with a channel region extending there between, a floating gate disposed over and insulated from a first portion of the channel region and a non-floating gate disposed over and insulated from a second portion of the channel region. Each of the plurality of memory cells is configured to store a weight value corresponding to a number of electrons on the floating gate. The plurality of memory cells is configured to multiply the first plurality of inputs by the stored weight values to generate the first plurality of outputs.

Each non-volatile memory cells used in the analog neuromorphic memory system must be erased and programmed to hold a very specific and precise amount of charge, i.e., the number of electrons, in the floating gate. For example, each floating gate must hold one of N different values, where N is the number of different weights that can be indicated by each cell. Examples of N include <NUM>, <NUM>, <NUM>, <NUM>, and <NUM>.

One challenge in systems utilizing VMM arrays is the ability to accurately measure the output of VMM array and to transfer that output to another stage, such as the input block of another VMM array. Numerous approaches are known, but each has certain drawbacks, such as loss of information through leakage current.

What is needed are improved output blocks for receiving output current from a VMM array and converting the output current into a form that is better suited for being transferred to another stage of electronics.

Numerous embodiments are disclosed for programmable output blocks for use with a VMM array within an artificial neural network. In one embodiment, the gain of an output block can be configured by a configuration signal. In another embodiment, the resolution of an ADC in the output block can be configured by a configuration signal.

<CIT> discloses that a DPE memristor crossbar array system includes a plurality of partitioned memristor crossbar arrays. Each of the plurality of partitioned memristor crossbar arrays includes a primary memristor crossbar array and a redundant memristor crossbar array. The redundant memristor crossbar array includes values that are mathematically related to values within the primary memristor crossbar array. In addition, the plurality of partitioned memristor crossbar arrays includes a block of shared analog circuits coupled to the plurality of partitioned memristor crossbar arrays. The block of shared analog circuits is to determine a dot product value of voltage values generated by at least one partitioned memristor crossbar array of the plurality of partitioned memristor crossbar arrays.

<CIT> discloses that an analog neural network element includes one or more EEPROMs as analog, reprogrammable synapses applying weighted inputs to positive and negative term outputs which are combined in a comparator. In one embodiment a pair of EEPROMs is used in each synaptic connection to separately drive the positive and negative term outputs. In another embodiment, a single EEPROM is used as a programmable current source to control the operation of a differential amplifier driving the positive and negative term outputs. In a still further embodiment, an MNOS memory transistor replaces the EEPROM or EEPROMs. These memory elements have limited retention or endurance which is used to simulate forgetfulness to emulate human brain function. Multiple elements are combinable on a single chip to form neural net building blocks which are then combinable to form massively parallel neural nets.

<CIT> discloses a general purpose programmable neural computer which parallel processes analog data. The neural computer comprises neural elements for outputting an analog signal in response to at least one input signal, synaptic circuits interfaced with the neural elements for modifying gains of the neural elements, and switching circuits interfaced with the synaptic circuits and the neural circuits for routing signals between the synapse circuits and the neural circuits and for modifying the synaptic time constants, thereby changing connection architecture of the general purpose analog computer as desired. In this manner, the neural computer of the invention can be programmed to learn different confirurations as well as different synaptic values.

Digital non-volatile memories are well known. For example, <CIT> ("the '<NUM> patent") discloses an array of split gate non-volatile memory cells, which are a type of flash memory cells. Such a memory cell <NUM> is shown in <FIG>. Each memory cell <NUM> includes source region <NUM> and drain region <NUM> formed in semiconductor substrate <NUM>, with channel region <NUM> there between. Floating gate <NUM> is formed over and insulated from (and controls the conductivity of) a first portion of the channel region <NUM>, and over a portion of the source region <NUM>. Word line terminal <NUM> (which is typically coupled to a word line) has a first portion that is disposed over and insulated from (and controls the conductivity of) a second portion of the channel region <NUM>, and a second portion that extends up and over the floating gate <NUM>. The floating gate <NUM> and word line terminal <NUM> are insulated from the substrate <NUM> by a gate oxide. Bitline <NUM> is coupled to drain region <NUM>.

Memory cell <NUM> is programmed (where electrons are placed on the floating gate) by placing a positive voltage on the word line terminal <NUM>, and a positive voltage on the source region <NUM>. Electron current will flow from the source region <NUM> towards the drain region <NUM>. The electrons will accelerate and become heated when they reach the gap between the word line terminal <NUM> and the floating gate <NUM>. Some of the heated electrons will be injected through the gate oxide onto the floating gate <NUM> due to the attractive electrostatic force from the floating gate <NUM>.

<FIG> shows memory cell <NUM>, which is similar to memory cell <NUM> of <FIG> with the addition of control gate (CG) <NUM>. Control gate <NUM> is biased at a high voltage, e.g., 10V, in programming, low or negative in erase, e.g., 0v/-8V, low or mid range in read, e.g., 0v/<NUM>. Other terminals are biased similarly to that of <FIG>.

<FIG> shows memory cell <NUM>, which is similar to memory cell <NUM> of <FIG> except that memory cell <NUM> does not contain an erase gate EG. An erase is performed by biasing the substrate <NUM> to a high voltage and biasing the control gate CG <NUM> to a low or negative voltage. Alternatively, an erase is performed by biasing word line <NUM> to a positive voltage and biasing control gate <NUM> to a negative voltage. Programming and reading is similar to that of <FIG>.

<FIG> depicts a three-gate memory cell <NUM>, which is another type of flash memory cell. Memory cell <NUM> is identical to the memory cell <NUM> of <FIG> except that memory cell <NUM> does not have a separate control gate. The erase operation (whereby erasing occurs through use of the erase gate) and read operation are similar to that of the <FIG> except there is no control gate bias applied. The programming operation also is done without the control gate bias, and as a result, a higher voltage must be applied on the source line during a program operation to compensate for a lack of control gate bias.

<FIG> depicts stacked gate memory cell <NUM>, which is another type of flash memory cell. Memory cell <NUM> is similar to memory cell <NUM> of <FIG>, except that floating gate <NUM> extends over the entire channel region <NUM>, and control gate <NUM> (which here will be coupled to a word line) extends over floating gate <NUM>, separated by an insulating layer (not shown). The erase, programming, and read operations operate in a similar manner to that described previously for memory cell <NUM>.

The methods and means described herein may apply to other non-volatile memory technologies such as SONOS (silicon-oxide-nitride-oxide-silicon, charge trap in nitride), MONOS (metal-oxide-nitride-oxide-silicon, metal charge trap in nitride), ReRAM (resistive ram), PCM (phase change memory), MRAM (magnetic ram), FeRAM (ferroelectric ram), OTP (bi-level or multi-level one time programmable), CeRAM (correlated electron ram), etc. The methods and means described herein may apply to volatile memory technologies used for neural network such SRAM, DRAM, and/or volatile synapse cell, without limitation.

S0 is the input layer, which for this example is a 32x32 pixel RGB image with <NUM> bit precision (i.e. three 32x32 pixel arrays, one for each color R, G and B, each pixel being <NUM> bit precision). The synapses CB1 going from input layer S0 to layer C1 apply different sets of weights in some instances and shared weights in other instances, and scan the input image with 3x3 pixel overlapping filters (kernel), shifting the filter by <NUM> pixel (or more than <NUM> pixel as dictated by the model). Specifically, values for <NUM> pixels in a 3x3 portion of the image (i.e., referred to as a filter or kernel) are provided to the synapses CB1, where these <NUM> input values are multiplied by the appropriate weights and, after summing the outputs of that multiplication, a single output value is determined and provided by a first synapse of CB1 for generating a pixel of one of the layers of feature map C1. The 3x3 filter is then shifted one pixel to the right within input layer S0 (i.e., adding the column of three pixels on the right, and dropping the column of three pixels on the left), whereby the <NUM> pixel values in this newly positioned filter are provided to the synapses CB1, where they are multiplied by the same weights and a second single output value is determined by the associated synapse. This process is continued until the 3x3 filter scans across the entire 32x32 pixel image of input layer S0, for all three colors and for all bits (precision values). The process is then repeated using different sets of weights to generate a different feature map of C1, until all the features maps of layer C1 have been calculated.

<FIG> is a block diagram of an array that can be used for that purpose. Vector-by-matrix multiplication (VMM) array <NUM> includes non-volatile memory cells and is utilized as the synapses (such as CB1, CB2, CB3, and CB4 in <FIG>) between one layer and the next layer. Specifically, VMM array <NUM> includes an array of non-volatile memory cells <NUM>, erase gate and word line gate decoder <NUM>, control gate decoder <NUM>, bit line decoder <NUM> and source line decoder <NUM>, which decode the respective inputs for the non-volatile memory cell array <NUM>. Input to VMM array <NUM> can be from the erase gate and wordline gate decoder <NUM> or from the control gate decoder <NUM>. Source line decoder <NUM> in this example also decodes the output of the non-volatile memory cell array <NUM>. Alternatively, bit line decoder <NUM> can decode the output of the non-volatile memory cell array <NUM>.

Non-volatile memory cell array <NUM> serves two purposes. Second, the non-volatile memory cell array <NUM> effectively multiplies the inputs by the weights stored in the non-volatile memory cell array <NUM> and adds them up per output line (source line or bit line) to produce the output, which will be the input to the next layer or input to the final layer. By performing the multiplication and addition function, the non-volatile memory cell array <NUM> negates the need for separate multiplication and addition logic circuits and is also power efficient due to its in-situ memory computation.

The output of non-volatile memory cell array <NUM> is supplied to a differential summer (such as a summing op-amp or a summing current mirror) <NUM>, which sums up the outputs of the non-volatile memory cell array <NUM> to create a single value for that convolution. The differential summer <NUM> is arranged to perform summation of positive weight and negative weight.

The summed up output values of differential summer <NUM> are then supplied to an activation function circuit <NUM>, which rectifies the output. The activation function circuit <NUM> may provide sigmoid, tanh, or ReLU functions. The rectified output values of activation function circuit <NUM> become an element of a feature map of the next layer (e.g. C1 in <FIG>), and are then applied to the next synapse to produce the next feature map layer or final layer. Therefore, in this example, non-volatile memory cell array <NUM> constitutes a plurality of synapses (which receive their inputs from the prior layer of neurons or from an input layer such as an image database), and summer <NUM> and activation function circuit <NUM> constitute a plurality of neurons.

The input to VMM array <NUM> in <FIG> (WLx, EGx, CGx, and optionally BLx and SLx) can be analog level, binary level, digital pulses (in which case a pulses-to-analog converter PAC may be needed to convert pulses to the appropriate input analog level) or digital bits (in which case a DAC is provided to convert digital bits to appropriate input analog level) and the output can be analog level, binary level, digital pulses, or digital bits (in which case an output ADC is provided to convert output analog level into digital bits).

<FIG> is a block diagram depicting the usage of numerous layers of VMM arrays <NUM>, here labeled as VMM arrays 32a, 32b, 32c, 32d, and 32e. As shown in <FIG>, the input, denoted Inputx, is converted from digital to analog by a digital-to-analog converter <NUM>, and provided to input VMM array 32a. The input D/A conversion for the first layer could be done by using a function or a LUT (look up table) that maps the inputs Inputx to appropriate analog levels for the matrix multiplier of input VMM array 32a. The input conversion could also be done by an analog to analog (A/A) converter to convert an external analog input to a mapped analog input to the input VMM array 32a. The input conversion could also be done by a digital-to-digital pules (D/P) converter to convert an external digital input to a mapped digital pulse or pulses to the input VMM array 32a.

The output generated by input VMM array 32a is provided as an input to the next VMM array (hidden level <NUM>) 32b, which in turn generates an output that is provided as an input to the next VMM array (hidden level <NUM>) 32c, and so on. The various layers of VMM array <NUM> function as different layers of synapses and neurons of a convolutional neural network (CNN). Each VMM array 32a, 32b, 32c, 32d, and 32e can be a stand-alone, physical non-volatile memory array, or multiple VMM arrays could utilize different portions of the same physical non-volatile memory array, or multiple VMM arrays could utilize overlapping portions of the same physical non-volatile memory array. Each VMM array 32a, 32b, 32c, 32d, and 32e can also be time multiplexed for various portion of its array or neurons. The example shown in <FIG> contains five layers (32a,32b,32c,32d,32e): one input layer (32a), two hidden layers (32b,32c), and two fully connected layers (32d,32e). One of ordinary skill in the art will appreciate that this is merely exemplary and that a system instead could comprise more than two hidden layers and more than two fully connected layers.

The non-volatile reference memory cells and the non-volatile memory cells described herein are biased in weak inversion: <MAT> where <MAT>.

For an I-to-V log converter using a memory cell (such as a reference memory cell or a peripheral memory cell) or a transistor to convert input current into an input voltage: <MAT>.

Here, wp is w of a reference or peripheral memory cell.

For a memory array used as a vector matrix multiplier VMM array, the output current is: <MAT> , namely <MAT> <MAT>.

Here, wa = w of each memory cell in the memory array.

Alternatively, the flash memory cells of VMM arrays described herein can be configured to operate in the linear region: <MAT> <MAT>.

Other embodiments for VMM array <NUM> of <FIG> are described in <CIT>. As described in that application, a sourceline or a bitline can be used as the neuron output (current summation output). Alternatively, the flash memory cells of VMM arrays described herein can be configured to operate in the saturation region: <MAT> <MAT>.

Alternatively, the flash memory cells of VMM arrays described herein can be used in all regions or a combination thereof (sub threshold, linear, or saturation).

VMM array <NUM> implements uni-directional tuning for non-volatile memory cells in memory array <NUM>. That is, each non-volatile memory cell is erased and then partially programmed until the desired charge on the floating gate is reached. This can be performed, for example, using the precision programming techniques described below. If too much charge is placed on the floating gate (such that the wrong value is stored in the cell), the cell must be erased and the sequence of partial programming operations must start over. As shown, two rows sharing the same erase gate (such as EG0 or EG1) need to be erased together (which is known as a page erase), and thereafter, each cell is partially programmed until the desired charge on the floating gate is reached.

EG lines FGR0, EG0, EG1 and EGR1 are run vertically while CG lines CG0, CG1, CG2 and CG3 and SL lines WL0, WL1, WL2 and WL3 are run horizontally.

In VMM array <NUM>, the inputs INPUT<NUM>. , INPUTN are received on bit lines BL<NUM>,. BLN, respectively, and the outputs OUTPUT<NUM>, OUTPUT<NUM>, OUTPUT<NUM>, and OUTPUT<NUM> are generated on source lines SL<NUM>, SL<NUM>, SL<NUM>, and SL<NUM>, respectively.

In this example, the inputs INPUT<NUM>, INPUT<NUM>, INPUT<NUM>, and INPUT<NUM> are received on source lines SL<NUM>, SL<NUM>, SL<NUM>, and SL<NUM>, respectively, and the outputs OUTPUT<NUM>,. OUTPUTN are generated on bit lines BL<NUM>,.

In this example, the inputs INPUT<NUM>,. , INPUTM are received on word lines WL<NUM>,. , WLM, respectively, and the outputs OUTPUT<NUM>,. OUTPUTN are generated on bit lines BL<NUM>,.

In this example, the inputs INPUT<NUM>,. , INPUTn are received on bit lines BL<NUM>,. , BLN, respectively, and the outputs OUTPUT<NUM> and OUTPUT<NUM> are generated on erase gate lines EG<NUM> and EG<NUM>.

In this example, the inputs INPUT<NUM>,. , INPUTN are received on the gates of bit line control gates <NUM>-<NUM>, <NUM>-<NUM>,. , <NUM>-(N-<NUM>), and <NUM>-N, respectively, which are coupled to bit lines BL<NUM>,. , BLN, respectively. Exemplary outputs OUTPUT<NUM> and OUTPUT<NUM> are generated on erase gate lines SL<NUM> and SL<NUM>.

<FIG> depicts neuron VMM array <NUM>, which is particularly suited for memory cells <NUM> as shown in <FIG>, memory cells <NUM> as shown in <FIG>, and memory cells <NUM> as shown in <FIG>, and is utilized as the synapses and parts of neurons between an input layer and the next layer. In this example, the inputs INPUT<NUM>,. , INPUTM are received on word lines WL<NUM>,. , WLM, and the outputs OUTPUT<NUM>,. , OUTPUTN are generated on bit lines BL<NUM>,. , BLN, respectively.

<FIG> depicts neuron VMM array <NUM>, which is particularly suited for memory cells <NUM> as shown in <FIG>, memory cells <NUM> as shown in <FIG>, and memory cells <NUM> as shown in <FIG>, and is utilized as the synapses and parts of neurons between an input layer and the next layer. In this example, the inputs INPUT<NUM>,. , INPUTM are received on control gate lines CG<NUM>,. Outputs OUTPUT<NUM>,. , OUTPUTN are generated on source lines SL<NUM>,. , SLN, respectively, where each source line SLi is coupled to the source line terminals of all memory cells in column i.

<FIG> depicts VMM system <NUM>. VMM system <NUM> comprises VMM array <NUM> (which can be based on any of the VMM design discussed previously, such as VMM <NUM>, <NUM>, <NUM>, <NUM>, and <NUM>, or other VMM designs), low voltage row decoder <NUM>, high voltage row decoder <NUM>, reference cell low voltage column decoder <NUM> (shown in the column direction, meaning that it provides input to output conversion in the row direction), bit line multiplexor <NUM>, control logic <NUM>, analog circuitry <NUM>, neuron output block <NUM>, input VMM circuit block <NUM>, predecoders <NUM>, test circuit <NUM>, erase-program control logic EPCTL <NUM>, analog and high voltage generation circuitry <NUM>, bit line PE driver <NUM>, redundancy arrays <NUM> and <NUM>, NVR sectors <NUM>, and reference sectors <NUM>. The input circuit block <NUM> serves as interface from an external input to the input terminals of the memory array. The neuron output block <NUM> serves as an interface from the memory array output to the external interface.

Low voltage row decoder <NUM> provides a bias voltage for read and program operations and provides a decoding signal for high voltage row decoder <NUM>. High voltage row decoder <NUM> provides a high voltage bias signal for program and erase operations. Reference cell low voltage column decoder <NUM> provides a decoding function for the reference cells. Bit line PE driver <NUM> provides a controlling function for bit lines during program, verify, and erase operations. Analog and high voltage generation circuitry <NUM> is a shared bias block that provides the multiple voltages needed for the various program, erase, program verify, and read operations. Redundancy arrays <NUM> and <NUM> provide array redundancy for replacing a defective array portion. NVR (non-volatile register aka info sector) sectors <NUM> are sectors that are array sectors used to store user info, device ID, password, security key, trimbits, configuration bits, manufacturing info, without limitation.

<FIG> depicts analog neuro memory system <NUM>. Analog neuro memory system <NUM> comprises macro blocks 3301a, 3301b, 3301c, 3301d, 3301e, 3301f, <NUM>, and <NUM>; neuron output (such as summer circuit and a sample and hold S/H circuit) blocks 3302a, 3302b, 3302c, 3302d, 3302e, 3302f, <NUM>, and <NUM>; and input circuit blocks 3303a, 3303b, 3303c, 3303d, 3303e, 3303f, <NUM>, and <NUM>. Each of macro blocks 3301a, 3301b, 3301c, 3301d, 3301e, and 3301f is a VMM sub-system containing a VMM array comprising rows and columns of non-volatile memory cells such as flash memory cells. Neuro memory sub-system <NUM> comprises macro block <NUM>, input block <NUM>, and neuron output block <NUM>. Neuro memory sub-system <NUM> may have its own digital control block.

Analog neuro memory system <NUM> further comprises system control block <NUM>, analog low voltage block <NUM>, high voltage block <NUM>, and timing control circuit <NUM>, discussed in further detail below with respect to <FIG>.

System control block <NUM> may include one or more microcontroller cores such as ARM/MIPS/RISC_V cores to handle general control function and arithmetic operations. System control block <NUM> also may include SIMD (single instruction multiple data) units to operate on multiple data with a single instruction. It may include DSP cores. It may include hardware or software for performing functions such as pooling, averaging, min, max, softmax, add, subtract, multiply, divide, log, anti-log, ReLu, sigmoid, tanh, and data compression, without limitation. It may include hardware or software to perform functions such as activation approximator/quantizer/normalizer. It may include the ability to perform functions such as input data approximator/quantizer/normalizer. It may include hardware or software to perform functions of an activation approximator/quantizer/normalizer. The control block of the neuro memory sub-system <NUM> may include similar elements of the system control block <NUM> such as microcontroller cores, SIMD cores, DSP cores, and other function units.

In one embodiment, neuron output blocks 3302a, 3302b, 3302c, 3302d, 3302e, 3302f, <NUM>, and <NUM> each includes a buffer (e.g., op amp) low impedance output type circuit that can drive a long, configurable interconnect. In one embodiment, input circuit blocks 3303a, 3303b, 3303c, 3303d, 3303e, 3303f, <NUM>, and <NUM> each provide summing, high impedance current outputs. In another embodiment, neuron output blocks 3302a, 3302b, 3302c, 3302d, 3302e, 3302f, <NUM>, and <NUM> each includes an activation circuit, in which case an additional low impedance buffer is needed to drive the outputs.

In another embodiment, the neuron output blocks 3302a, 3302b, 3302c, 3302d, 3302e, 3302f, <NUM>, and <NUM> each comprises an analog-to-digital conversion block that outputs digital bits instead of analog signals. In this embodiment, input circuit blocks 3303a, 3303b, 3303c, 3303d, 3303e, 3303f, <NUM>, and <NUM> each comprises a digital-to-analog conversion block that receives digital bits from the respective neuron output blocks and converts the digital bits into analog signals.

Thus, neuron output blocks 3302a, 3302b, 3302c, 3302d, 3302e, 3302f, <NUM>, and <NUM> receives output current from macro blocks 3301a, 3301b, 3301c, 3301d, 3301e, and 3301f and optionally converts that output current into an analog voltage, digital bits, or one or more digital pulses where the width of each pulse or the number of pulses varies in response to the value of the output current. Similarly, input circuit blocks 3303a, 3303b, 3303c, 3303d, 3303e, 3303f, <NUM>, and <NUM> optionally receives analog current, analog voltage, digital bits, or digital pulses where the width of each pulse or the number of pulses varies in response to the value of the output current and provides analog current to macro blocks 3301a, 3301b, 3301c, 3301d, 3301e, and 3301f. Input circuit blocks 3303a, 3303b, 3303c, 3303d, 3303e, 3303f, <NUM>, and <NUM> optionally comprises a voltage-to-current converter, an analog or digital counter for counting the number of digital pulses in an input signal or the length of the width of a digital pulse in an input signal, or a digital-to-analog converter.

Optionally, neuron output blocks 3302a, 3302b, 3302c, 3302d, 3302e, 3302f, <NUM>, and <NUM> can apply a programmable gain when they convert output current into an analog voltage, digital bits, or one or more digital pulses. This can be referred to as a programmable neuron.

<FIG> depicts an example of programmable neuron output block <NUM>, which receives output neuron current, Ineu, from a VMM array, gain configuration <NUM>, and generates output <NUM>, which represents output neuron current Ineu with a gain of G, where the value of G is set in response to gain configuration <NUM>. Gain configuration <NUM> can be an analog signal or digital bits. In one embodiment, programmable neuron output block <NUM> comprises gain control circuit <NUM>, which in turn comprises variable resistor <NUM> or variable capacitor <NUM> that is controlled by gain configuration <NUM> to generate the gain G. Output <NUM> can be an analog voltage, an analog current, digital bits, or one or more digital pulses. In some embodiments, gain configuration <NUM> is used to trim the gain, G, of programmable neuron output block <NUM> to compensate for undesirable phenomena such as leakage current.

Optionally, each programmable neuron output block <NUM> can be provided with a different gain configuration <NUM>. This would allow, for instance, a different gain (such as for scaling the array output) to be implemented at different layers of a neural network.

In a comparative example not falling under the scope of the claims, gain configuration <NUM> depends in part on the input size, for example, meaning how many rows are enabled to generate the output neuron current, Ineu.

In a comparative example not falling under the scope of the claims, gain configuration <NUM> depends in part on the value of all the rows that are input to the VMM array. For example, for an <NUM>-bit row input for the VMM array, the maximum value is <NUM> (<NUM>^<NUM>) as input for one row, <NUM> for <NUM> rows, etc. For example, if <NUM> rows are enabled, a determination is made as to the total value of these rows, and gain configuration <NUM> is modified in response to this value.

According to the invention, gain configuration <NUM> depends on the output neuron range. For example, if output neuron current, Ineu is within a first range, then a first gain G1 is applied by gain configuration <NUM>; if output neuron current, Ineu is within a second range, then a second gain, G2, is applied through gain configuration <NUM>. While this has been described in relation to ranges, those skilled in the art will recognize that a larger number of ranges may be implemented without limitation.

The prior art includes a concept known as long short-term memory (LSTM). LSTM units often are used in neural networks. LSTM allows a neural network to remember information over predetermined arbitrary time intervals and to use that information in subsequent operations. A conventional LSTM unit comprises a cell, an input gate, an output gate, and a forget gate. The three gates regulate the flow of information into and out of the cell and the time interval that the information is remembered in the LSTM. VMMs are particularly useful in LSTM units.

<FIG> depicts an exemplary LSTM <NUM>. LSTM <NUM> in this example comprises cells <NUM>, <NUM>, <NUM>, and <NUM>. Cell <NUM> receives input vector x<NUM> and generates output vector h<NUM> and cell state vector c<NUM>. Cell <NUM> receives input vector x<NUM>, the output vector (hidden state) h<NUM> from cell <NUM>, and cell state c<NUM> from cell <NUM> and generates output vector h<NUM> and cell state vector c<NUM>. Cell <NUM> receives input vector x<NUM>, the output vector (hidden state) h<NUM> from cell <NUM>, and cell state c<NUM> from cell <NUM> and generates output vector h<NUM> and cell state vector c<NUM>. Cell <NUM> receives input vector x<NUM>, the output vector (hidden state) h<NUM> from cell <NUM>, and cell state c<NUM> from cell <NUM> and generates output vector h<NUM>. Additional cells can be used, and an LSTM with four cells is merely an example.

<FIG> depicts an exemplary implementation of an LSTM cell <NUM>, which can be used for cells <NUM>, <NUM>, <NUM>, and <NUM> in <FIG>. LSTM cell <NUM> receives input vector x(t), cell state vector c(t-<NUM>) from a preceding cell, and output vector h(t-<NUM>) from a preceding cell, and generates cell state vector c(t) and output vector h(t).

LSTM cell <NUM> comprises sigmoid function devices <NUM>, <NUM>, and <NUM>, each of which applies a number between <NUM> and <NUM> to control how much of each component in the input vector is allowed through to the output vector. LSTM cell <NUM> also comprises tanh devices <NUM> and <NUM> to apply a hyperbolic tangent function to an input vector, multiplier devices <NUM>, <NUM>, and <NUM> to multiply two vectors together, and addition device <NUM> to add two vectors together. Output vector h(t) can be provided to the next LSTM cell in the system, or it can be accessed for other purposes.

<FIG> depicts an LSTM cell <NUM>, which is an example of an implementation of LSTM cell <NUM>. For the reader's convenience, the same numbering from LSTM cell <NUM> is used in LSTM cell <NUM>. Sigmoid function devices <NUM>, <NUM>, and <NUM> and tanh device <NUM> each comprise multiple VMM arrays <NUM> and activation circuit blocks <NUM>. Thus, it can be seen that VMM arrays are particular useful in LSTM cells used in certain neural network systems. The multiplier devices <NUM>, <NUM>, and <NUM> and the addition device <NUM> are implemented in a digital manner or in an analog manner. The activation function blocks <NUM> can be implemented in a digital manner or in an analog manner.

An alternative to LSTM cell <NUM> (and another example of an implementation of LSTM cell <NUM>) is shown in <FIG>. In <FIG>, sigmoid function devices <NUM>, <NUM>, and <NUM> and tanh device <NUM> share the same physical hardware (VMM arrays <NUM> and activation function block <NUM>) in a time-multiplexed fashion. LSTM cell <NUM> also comprises multiplier device <NUM> to multiply two vectors together, addition device <NUM> to add two vectors together, tanh device <NUM> (which comprises activation circuit block <NUM>), register <NUM> to store the value i(t) when i(t) is output from sigmoid function block <NUM>, register <NUM> to store the value f(t) * c(t-<NUM>) when that value is output from multiplier device <NUM> through multiplexor <NUM>, register <NUM> to store the value i(t) * u(t) when that value is output from multiplier device <NUM> through multiplexor <NUM>, and register <NUM> to store the value o(t) * c~(t) when that value is output from multiplier device <NUM> through multiplexor <NUM>, and multiplexor <NUM>.

Whereas LSTM cell <NUM> contains multiple sets of VMM arrays <NUM> and respective activation function blocks <NUM>, LSTM cell <NUM> contains only one set of VMM arrays <NUM> and activation function block <NUM>, which are used to represent multiple layers in the embodiment of LSTM cell <NUM>. LSTM cell <NUM> will require less space than LSTM <NUM>, as LSTM cell <NUM> will require <NUM>/<NUM> as much space for VMMs and activation function blocks compared to LSTM cell <NUM>.

It can be further appreciated that LSTM units will typically comprise multiple VMM arrays, each of which requires functionality provided by certain circuit blocks outside of the VMM arrays, such as a summer and activation circuit block and high voltage generation blocks. Providing separate circuit blocks for each VMM array would require a significant amount of space within the semiconductor device and would be somewhat inefficient.

An analog VMM implementation can be utilized for a GRU (gated recurrent unit) system. GRUs are a gating mechanism in recurrent neural networks. GRUs are similar to LSTMs, except that GRU cells generally contain fewer components than an LSTM cell.

<FIG> depicts an exemplary GRU <NUM>. GRU <NUM> in this example comprises cells <NUM>, <NUM>, <NUM>, and <NUM>. Cell <NUM> receives input vector x<NUM> and generates output vector h<NUM>. Cell <NUM> receives input vector x<NUM>, the output vector h<NUM> from cell <NUM> and generates output vector h<NUM>, Cell <NUM> receives input vector x<NUM> and the output vector (hidden state) h<NUM> from cell <NUM> and generates output vector h<NUM>. Cell <NUM> receives input vector x<NUM> and the output vector (hidden state) h<NUM> from cell <NUM> and generates output vector h<NUM>. Additional cells can be used, and an GRU with four cells is merely an example.

<FIG> depicts an exemplary implementation of a GRU cell <NUM>, which can be used for cells <NUM>, <NUM>, <NUM>, and <NUM> of <FIG>. GRU cell <NUM> receives input vector x(t) and output vector h(t-<NUM>) from a preceding GRU cell and generates output vector h(t). GRU cell <NUM> comprises sigmoid function devices <NUM> and <NUM>, each of which applies a number between <NUM> and <NUM> to components from output vector h(t-<NUM>) and input vector x(t). GRU cell <NUM> also comprises a tanh device <NUM> to apply a hyperbolic tangent function to an input vector, a plurality of multiplier devices <NUM>, <NUM>, and <NUM> to multiply two vectors together, an addition device <NUM> to add two vectors together, and a complementary device <NUM> to subtract an input from <NUM> to generate an output.

<FIG> depicts a GRU cell <NUM>, which is an example of an implementation of GRU cell <NUM>. For the reader's convenience, the same numbering from GRU cell <NUM> is used in GRU cell <NUM>. As can be seen in <FIG>, sigmoid function devices <NUM> and <NUM>, and tanh device <NUM> each comprise multiple VMM arrays <NUM> and activation function blocks <NUM>. Thus, it can be seen that VMM arrays are of particular use in GRU cells used in certain neural network systems. The multiplier devices <NUM>, <NUM>, <NUM>, the addition device <NUM>, and the complementary device <NUM> are implemented in a digital manner or in an analog manner. The activation function blocks <NUM> can be implemented in a digital manner or in an analog manner.

An alternative to GRU cell <NUM> (and another example of an implementation of GRU cell <NUM>) is shown in <FIG>. In <FIG>, GRU cell <NUM> utilizes VMM arrays <NUM> and activation function block <NUM>, which when configured as a sigmoid function applies a number between <NUM> and <NUM> to control how much of each component in the input vector is allowed through to the output vector. In <FIG>, sigmoid function devices <NUM> and <NUM> and tanh device <NUM> share the same physical hardware (VM1/<NUM> arrays <NUM> and activation function block <NUM>) in a time-multiplexed fashion. GRU cell <NUM> also comprises multiplier device <NUM> to multiply two vectors together, addition device <NUM> to add two vectors together, complementary device <NUM> to subtract an input from <NUM> to generate an output, multiplexor <NUM>, register <NUM> to hold the value h(t-<NUM>) * r(t) when that value is output from multiplier device <NUM> through multiplexor <NUM>, register <NUM> to hold the value h(t-<NUM>) *z(t) when that value is output from multiplier device <NUM> through multiplexor <NUM>, and register <NUM> to hold the value h^(t) * (<NUM>-z(t)) when that value is output from multiplier device <NUM> through multiplexor <NUM>.

Whereas GRU cell <NUM> contains multiple sets of VMM arrays <NUM> and activation function blocks <NUM>, GRU cell <NUM> contains only one set of VMM arrays <NUM> and activation function block <NUM>, which are used to represent multiple layers in the embodiment of GRU cell <NUM>. GRU cell <NUM> will require less space than GRU cell <NUM>, as GRU cell <NUM> will require <NUM>/<NUM> as much space for VMMs and activation function blocks compared to GRU cell <NUM>.

It can be further appreciated that GRU systems will typically comprise multiple VMM arrays, each of which requires functionality provided by certain circuit blocks outside of the VMM arrays, such as a summer and activation circuit block and high voltage generation blocks. Providing separate circuit blocks for each VMM array would require a significant amount of space within the semiconductor device and would be somewhat inefficient.

The input to the VMM arrays can be an analog level, a binary level, or digital bits (in this case a DAC is needed to convert digital bits to appropriate input analog level) and the output can be an analog level, a binary level, or digital bits (in this case an output ADC is needed to convert output analog level into digital bits).

For each memory cell in a VMM array, each weight w can be implemented by a single memory cell or by a differential cell or by two blend memory cells (average of <NUM> cells). In the differential cell case, two memory cells are needed to implement a weight w as a differential weight (w = w+ - w-). In the two blend memory cells, two memory cells are needed to implement a weight w as an average of two cells.

<FIG> depicts integrating dual-mixed slope analog-to-digital converter (ADC) <NUM> applied to an output neuron, INEU <NUM>, to convert the output neuron current into digital pulses or digital output bits.

In one embodiment, ADC <NUM> converts an analog output current in a neuron output block (such as neuron output blocks 3302a, 3302b. 3302c, 3302d, 3302e, 3302f, <NUM>, and <NUM> in <FIG>) into a digital pulse whose width varies in proportion to the magnitude of the analog output current in the neuron output block. An integrator comprising integrating op-amp <NUM> and integrating capacitor <NUM> integrates a memory array current INEU <NUM> (which is the output neuron current) versus a reference current IREF <NUM>.

Optionally, IREF <NUM> can comprise a bandgap filter with a temperature coefficient of <NUM> or with a temperature coefficient that tracks the neuron current, INEU <NUM>. The latter optionally can be obtained from a reference array containing values determined during a testing phase.

Optionally, a calibration step can be performed while the circuit is at or above operating temperature to offset any leakage current that is present within the array or a control circuit, and that offset value thereafter can be subtracted from Ineu in <FIG> or <FIG>.

During an initialization phase, switch <NUM> is closed. Vout <NUM> and the input to the negative terminal of operational amplifier <NUM> then will become VREF. Thereafter, as shown in <FIG>, switch <NUM> is opened and during a fixed time period tref, the neuron current INEU <NUM> is up-integrated. During the fixed time period tref, Vout rises, and its slope changes as neuron current changes. Thereafter, during a period tmeas, a constant reference current IREF is down integrated for a time period tmeas (during which period Vout falls), where tmeas is the time required to down integrate Vout to VREF.

Output EC <NUM> will be high when VOUT > VREFV and will be low otherwise. EC3405 therefore generates a pulse whose width reflects the period tmeas, which in turn is proportional to the current INEU <NUM>. In <FIG>, EC3405 is shown as waveform <NUM> in the example where tmeas = Ineu1, and waveform <NUM> in the example where tmeas = Ineu2. Thus, the output neuron current INEU <NUM> is converted into a digital pulse EC <NUM>, where the width of digital pulse EC <NUM> varies in proportion to the magnitude of output neuron current INEU <NUM>.

The current INEU <NUM> is = tmeas/tref* IREF. For example, for a desired output bit resolution of <NUM> bits, tref is a time period equal to <NUM> clock cycles. The period tmeas varies from a period equal to <NUM> to <NUM> clock cycles depending on the value of INEU <NUM> and the value of Iref. <FIG> shows examples of two different values for INEU <NUM>, one where INEU <NUM> = Ineu1 and one with INEU <NUM> = Ineu2. Thus, the neuron current INEU <NUM> affects the rate and slope of charging.

Optionally, the output pulse EC <NUM> can be converted into a series of pulses of uniform period for transmission to the next stage of circuitry, such as the input block of another VMM array. At the beginning of period tmeas, output EC <NUM> is input into AND gate <NUM> with reference clock <NUM>. The output will be pulse series <NUM> (where the frequency of the pulses in pulse series <NUM> is the same as the frequency of clock <NUM>) during the period when VOUT > VREF. The number of pulses is proportional to the period tmeas, which is proportional to the current INEU <NUM>.

Optionally, pulse series <NUM> can be input to counter <NUM>, which will count the number of pulses in pulse series <NUM> and will generate count value <NUM>, which is a digital count of the number of pulses in pulse series <NUM>, which is directly proportional to neuron current INEU <NUM>. Count value <NUM> comprises a set of digital bits In another embodiment, integrating dual-slope ADC <NUM> can convert neuron current INEU <NUM> into a pulse where the width of the pulse is inversely proportionally to the magnitude of neuron current INEU <NUM>. This inversion can be done in a digital or analog manner, and converted into a series of pulses, or digital bits for output to follow on circuitry.

<FIG> shows integrating dual-mixed slope ADC <NUM> applied to an output neuron, INEU <NUM>, to convert the cell current into a digital pulse of varying width or into a series of digital output bits. For example, ADC <NUM> can be used to convert an analog output current in a neuron output block (such as neuron output blocks 3302a, 3302b. 3302c, 3302d, 3302e, 3302f, <NUM>, and <NUM> in <FIG>) into a set of digital output bits. An integrator comprising integrating op-amp <NUM> and integrating capacitor <NUM> integrates a neuron current INEU <NUM> versus a reference current IREF <NUM>. Switch <NUM> can be closed to reset the VOUT.

During an initialization phase, switch <NUM> is closed, and VOUT is charged to a voltage VBIAS.

Thereafter, as shown in <FIG>, switch <NUM> is opened, and during a fixed time tref, the cell current INEU <NUM> is up integrated. Thereafter, reference current IREF <NUM> is down integrated for a time tmeas until Vout falls to ground. The current INEU <NUM> = tmeas Ineu/tref* IREF. For example, for a desired output bit resolution of <NUM> bits, tref is a time period equal to <NUM> clock cycles. The period tmeas varies from a period equal to <NUM> to <NUM> clock cycles depending on the value of INEU <NUM> and Iref. <FIG> shows examples of two different Ineu values, one with current Ineu1 and one with current Ineu2. Thus, the neuron current INEU <NUM> affects the rate and slope of charge and discharge.

Output <NUM> will be high when VOUT > VREF and will be low otherwise. Output <NUM> therefore generates a pulse whose width reflects the period tmeas, which in turn is proportional to the current INEU <NUM>. In <FIG>, output <NUM> is shown as waveform <NUM> in the example where tmeas = Ineu1, and waveform <NUM> in the example where tmeas = Ineu2. Thus, the output neuron current INEU <NUM> is converted into a pulse, output <NUM>, where the width of the pulse varies in proportion to the magnitude of output neuron current INEU <NUM>.

Optionally, output <NUM> can be converted into a series of pulses of a uniform period for transmission to the next stage of circuitry, such as an input block of another VMM array. At the beginning of period tmeas, output <NUM> is input into AND gate <NUM> with reference clock <NUM>. The output will be pulse series <NUM> (where the frequency of the pulses in pulse series <NUM> is the same as the frequency of reference clock <NUM>) during the period when VOUT > VREF. The number of pulses is proportional to the period tmeas, which is proportional to the current INEU <NUM>.

Optionally, pulse series <NUM> can be input to counter <NUM>, which will count the number of pulses in pulse series <NUM> and will generate count value <NUM>, which is a digital count of the number of pulses in pulse series <NUM>, which is directly proportional to neuron current INEU <NUM> as illustrated by waveforms <NUM>, <NUM>. Count value <NUM> comprises a set of digital bits.

In another embodiment, integrating dual-slope ADC <NUM> can convert neuron current INEU <NUM> into a pulse where the width of the pulse is inversely proportionally to the magnitude of neuron current INEU <NUM>. This inversion can be done in a digital or analog manner, and converted into one or more pulses, or digital bits for output to follow on circuitry.

<FIG> depicts count value <NUM> (digital bits) for two neuron current values Ineu1 and Ineu2, respectively, for INEU <NUM>.

<FIG> depict waveforms associated with exemplary methods <NUM> and <NUM> performed in a VMM during operation. In each method <NUM> and <NUM>, word lines WL0, WL1, and WL2 receive a variety of different inputs, which optionally can be converted into analog voltage waveforms to apply to the word lines. In these examples, the voltages VC represents the voltage on integrating capacitor <NUM> or <NUM> in <FIG> and <FIG>, respectively, in ADC <NUM> or <NUM> in an output block of a first VMM, and OT pulse (= '<NUM>') represents the period in which the output of the neuron (that is proportional to value of the neuron) is captured using integrating dual-slope ADC <NUM> or <NUM>. As described with reference to <FIG> and <FIG>, the output of the output block can be a pulse of width that varies in proportion to the output neuron current of the first VMM, or it can be a series of pulses of uniform width where the number of pulses varies in proportion to the neuron current of the first VMM. Those pulses then can be applied as inputs to a second VMM.

During method <NUM>, the series of pulses (such as pulse series <NUM> or pulse series <NUM>), or an analog voltage derived from the series of pulses, are applied into the wordlines of the second VMM array. Alternatively, the series of pulses, or an analog voltage derived from the series of pulses, can be applied to the control gates of cells within the second VMM array. The number of pulses (or clock cycles) directly correspond to the magnitude of the input. In this particular example, the magnitude of the input on WL1 is 4X vs. that of WL0 (<NUM> pulses vs. <NUM> pulse).

During method <NUM>, single pulses of varying width (such as EC <NUM> or output <NUM>), or an analog voltage derives from the single pulses, are applied into the wordlines of the second VMM array, but the pulses have a variable pulse width. Alternatively, the pulses, or an analog voltage derives from the pulses, can be applied to the control gates. The width of a single pulse directly corresponds to the magnitude of the input. For example, the magnitude of the input on WL1 is 4X vs. that of WL0 (WL1 pulse width is 4x vs. that of WL0).

Furthermore, with reference to <FIG>, timing control circuit <NUM> can be used to manage the power of the VMM systems by managing the output and input interfaces of the VMM array and sequentially splitting the conversion of various outputs or various inputs. <FIG> depicts power management method <NUM>. The first step is receiving a plurality of inputs for a vector-by-matrix multiplication array (step <NUM>). The second step is organizing the plurality of inputs into a plurality of sets of inputs (step <NUM>). The third step is sequentially providing each of the plurality of sets of inputs to the array (step <NUM>).

An embodiment of power management method <NUM> is the following. The inputs to the VMM systems (such as wordlines or control gates of VMM arrays) can be applied sequentially over time. For instance, for a VMM array with <NUM> wordline inputs, the wordline inputs can be divided into <NUM> groups, WL0-<NUM>, WL128-<NUM>, WL256-<NUM>, and WL383-<NUM>. Each group can be enabled at different times and an output read operation can be performed (converting neuron current into digital bits) for the group corresponding to one of the four groups of word lines, such as by the output integrating circuits in <FIG>. The output digital bit results are then combined together after each of the four groups are read in sequence. This operation can be controlled by timing control circuit <NUM>.

In another embodiment, timing control circuit <NUM> performs power management in a vector-by-matrix multiplication system, such as analog neuro memory system <NUM> in <FIG>. Timing control circuit <NUM> can cause inputs to be applied to VMM sub-systems <NUM> sequentially over time, such as by enabling input circuit blocks 3303a, 3303b. 3303c, 3303d, 3303e, 3303f, <NUM>, and <NUM> at different times. Similarly, timing control circuit <NUM> can cause outputs from VMM sub-systems <NUM> to be read sequentially over time, such as by enabling neuron outpu8t blocks 3302a, 3302b. 3302c, 3302d, 3302e, 3302f, <NUM>, and <NUM> at different times.

<FIG> depicts power management method <NUM>. The first step is receiving a plurality of outputs from a vector-by-matrix multiplication array (step <NUM>). The next step is organizing the plurality of outputs from the array into a plurality of sets of outputs (step <NUM>). The next step is sequentially providing each of the plurality of sets of outputs to a converter circuit (step <NUM>).

An embodiment of power management method <NUM> is the following. Power management can be implemented by timing control circuit <NUM> by sequential reading groups of neuron outputs at different times, that is, by multiplexing the output circuits (such as the output ADC circuits) across multiple neuron outputs (bitlines). The bitlines can be placed into different groups, and the output circuit operates on one group at a time in sequential fashion, under control of timing control circuit <NUM>.

<FIG> depicts power management method <NUM>. In a vector-by-matrix multiplication system comprising a plurality of arrays, the first step is receiving a plurality of inputs. The next step is sequentially enabling one or more of the plurality of arrays to receive some or all of the plurality of inputs (step <NUM>).

An embodiment of power management method <NUM> is the following. Timing control circuit <NUM> can operate on one neural network layer at a time. For example, if one neural network layer is represented in a first VMM array and a second neural network layer is represented in a second VMM array, the output read operations (such as where the neuron output is converted into digital bits) can be performed sequentially on one VMM array at a time, thereby managing the power of the VMM system.

In another embodiment, timing control circuit <NUM> can operate by enabling sequentially a plurality of the neuro memory sub-systems <NUM> or a plurality of macros <NUM> as shown in <FIG>.

In another embodiment, timing control circuit <NUM> can operate by enabling sequentially a plurality of the neuro memory sub-systems <NUM> or a plurality of macros <NUM> as shown in <FIG> without discharging the array biases (e.g., biases on wordline WLs and/or bitline BLs for control gate CGs as inputs and bitltine BLs as outputs, or biases on control gates CGs and/or bitline BLs for wordlines WLs as inputs and bitlines BLs as outputs) during the inactive period (meaning time off period between the on and off sequential enabling). This is to save power from unnecessary discharging and charging up again the array biases that are to be used multiple times during one or more read operations (e.g., during inference or classification operations).

<FIG> depicts various circuits that can be used in a VMM input block, such as input circuit blocks 3303a, 3303b. 3303c, 3303d, 3303e, 3303f, <NUM>, and <NUM> in <FIG> or neuron output block, such as neuron output blocks 3302a, 3302b. 3302c, 3302d, 3302e, 3302f, <NUM>, and <NUM> in <FIG>.

<FIG> depicts pulse-to-voltage converter <NUM>, which optionally can be used to convert the digital pulses generated by integrating dual-slope ADC <NUM> or <NUM> into a voltage, which, for example, can be applied as an input (for example, on a WL or CG line) of the VMM memory array. Pulse-to-voltage converter <NUM> comprises reference current generator <NUM> that generates reference current IREF, capacitor <NUM>, and switch <NUM>. The input is used to control switch <NUM>. When a pulse is received on the input, the switch is closed, and charge accumulates on capacitor <NUM>, such that the voltage of capacitor <NUM> after the input signal is complete will be indicative of the number of pulses received. The capacitor optionally can be a wordline or control gate capacitance.

<FIG> depicts current-to-voltage converter <NUM>, which optionally can be used to convert a neuron output current into a voltage, which, for example, can be applied as an input (for example, on a WL or CG line) of the VMM memory array. Current-to-voltage converter <NUM> comprises current generator <NUM> that here represents the neuron current received, ineu (or Iin), and variable resistor <NUM>. The output, Vout, will increase in size as the neuron current increases. Variable resistor <NUM> can be adjusted to increase or decrease the maximum range of Vout as desirable.

<FIG> depicts current-to-voltage converter <NUM>, which optionally can be used to convert a neuron output current into a voltage, that for example, can be applied as an input (for example, on a WL or a CG line) of the VMM memory array. Current-to-voltage converter <NUM> comprises op amp <NUM>, capacitor <NUM>, switch <NUM>, switch <NUM>, and current source <NUM> that here represents the neuron current ICELL. During operation, switch <NUM> will be open, and switch <NUM> will be closed. The output, Vout, will increase in amplitude in proportion to the magnitude of the neuron current ICELL <NUM>.

<FIG> depicts current-to-logarithmic voltage converter <NUM>, which optionally can be used to convert a neuron output current into a logarithmic voltage, that for example, can be applied as an input (for example, on a WL or a CG line) of the VMM memory array. Current-to-logarithmic voltage converter <NUM> comprises memory cell <NUM>, switch <NUM> (which selectively connects the word line terminal of memory cell <NUM> to the node generating Vout), and current source <NUM> that here represents the neuron current IiN. During operation, switch <NUM> will be closed, and the output, Vout, will increase in amplitude in proportion to the magnitude of the neuron current iIN.

<FIG> depicts current-to-logarithmic voltage converter <NUM>, which optionally can be used to convert a neuron output current into a logarithmic voltage, which, for example, can be applied as an input (for example, on a WL or a CG line) of the VMM memory array. Current-to-logarithmic voltage converter <NUM> comprises memory cell <NUM>, switch <NUM> (which selectively connects the control gate terminal of memory cell <NUM> to the node generating Vout), and current source <NUM> that here represents the neuron current IiN. During operation, switch <NUM> will be closed, and the output, Vout, will increase in amplitude in proportion to the magnitude of neuron current IiN.

<FIG> depicts digital data to voltage converter <NUM>, which optionally can be used to convert digital data (i.e., of <NUM> and <NUM>) into a voltage, that for example, can be applied as an input (for example, on a WL or CG line) of the VMM memory array. Digital data to voltage converter <NUM> comprises capacitor <NUM>, adjustable current source <NUM> (which here is current from a reference array of memory cells), and switch <NUM>. The digital data controls switch <NUM>. For example, switch <NUM> can close when the digital data is a "<NUM>" and open when the digital data is a "<NUM>". The voltage accumulated on capacitor <NUM> will be the output OUT and will correspond to the value of the digital data. Optionally, the capacitor can be wordline or control gate capacitance.

<FIG> depicts digital data to voltage converter <NUM>, which optionally can be used to convert digital data (i.e., of <NUM> and <NUM>) into a voltage, that for example, can be applied as an input (for example, on a WL or CG line) of the VMM memory array. Digital data to voltage converter <NUM> comprises variable resistor <NUM>, adjustable current source <NUM> (which here is current from a reference array of memory cells), and switch <NUM>. The digital data controls switch <NUM>. For example, switch <NUM> can close when the digital data is a "<NUM>" and open when the digital data is a "<NUM>". The output voltage will correspond to the value of the digital data.

<FIG> depicts reference array <NUM> that can be used to provide the reference current of adjustable current sources <NUM> and <NUM> in <FIG> and <FIG>.

<FIG> depicts components for verifying, after a programming operation, that a flash memory cell in a VMM contains the appropriate charge corresponding to the W value that is intended to be stored in that flash memory cell.

<FIG> depicts digital comparator <NUM>, which receives a reference set of W values as digital inputs and the sensed W digital values from a number of programmed flash memory cells. Digital comparator <NUM> generates a flag if there is a mismatch, which would indicate that one or more flash memory cells has not been programmed with the correct value.

<FIG> depicts digital comparator <NUM> from <FIG> in cooperation with a converter <NUM>. The sensed W values are provided by multiple instantiations of converter <NUM>. Converter <NUM> receives a cell current, ICELL, from a flash memory cell and converts the cell current into digital data that can be provided to digital comparator <NUM> using one or more of the converters described previously, such as ADC <NUM> or <NUM>).

<FIG> depicts analog comparator <NUM>, which receives a reference set of W values as analog inputs and the sensed W analog values from a number of programmed flash memory cells. Analog comparator <NUM> generates a flag if there is a mismatch, which would indicate that one or more flash memory cells has not been programmed with the correct value.

<FIG> depicts analog comparator <NUM> from <FIG> in cooperation with a converter <NUM>. The sensed W values are provided by converter <NUM>. Converter <NUM> receives digital values of sensed W values and converts them into an analog signal that can be provided to analog comparator <NUM> using one or more of the converters described previously (such as pulse-to-voltage converter <NUM>, digital data to voltage converter <NUM>, or digital data to voltage converter <NUM>).

<FIG> depicts output circuit <NUM>. It can be appreciated that if the output of a neuron is digitized (such as by using integrating dual-slope ADC <NUM> or <NUM>, described previously), then one still may need to perform an activation function to the neuron output. <FIG> depicts an embodiment where activation occurs before the neuron output is converted into a pulse of variable width or a pulse series. Output circuit <NUM> comprises activation circuit <NUM> and current-to-pulse converter <NUM>. Activation circuit receives Ineuron values from various flash memory cells and generates Ineuron_act, which is a summation of the received Ineuron values. Current-to-pulse converter <NUM> then converts Ineuron_act into a series of digital pulses and/or digital data that represents a count of a series of digital pulses. Other converters described previously (such integrating dual-slope ADC <NUM> or <NUM>) can be used instead of the converter <NUM>.

In another embodiment, the activation can occur after the digital pulses are generated. In that embodiment, the digital output bits are mapped to a new set of digital bits using an activation mapping table or function implemented by activation mapping unit <NUM>. Examples of such a mapping are shown graphically in <FIG> and <FIG>. The activation digital mapping can simulate the sigmoid, tanh, ReLu, or any activation function. Further, the activation digital mapping can quantize the output neuron.

<FIG> depicts an example of charge summer <NUM> that can be used to sum the output of a VMM during a verifying operation after a programming operation, to obtain a single analog value that represents the output, and that can optionally be then converted into digital bit values. Charge summer <NUM> comprises current source <NUM> and a sample-and-hold circuit comprising switch <NUM> and sample-and-hold (S/H) capacitor <NUM>. As shown for an example of a <NUM>-bit digital value, there are <NUM>/H circuits to hold the value from <NUM> evaluation pulses, where the values are summed up at the end of the process. S/H capacitors <NUM> are selected with ratios that are associated with the <NUM>^n*DINn bit position for that S/R capacitor; for example, C_DIN3 = x8 Cu, C_DIN2 = x4 Cu, C_DIN1 = x2 Cu, DIN0 = x1 Cu. The current source <NUM> is also ratioed accordingly.

<FIG> depicts current summer <NUM> that can be used to sum the output of a VMM during a verifying operation after a programming operation. Current summer <NUM> comprises current source <NUM>, switch <NUM>, switches <NUM> and <NUM>, and switch <NUM>. As shown for an example of a <NUM>-bit digital value, there are current source circuits to hold the value from <NUM> evaluation pulses, where the values are summed up at the end of the process. The current source is ratioed based on the <NUM>^n*DINn bit position; for example, I_DIN3 = x8 Icell unit, I_DIN2 = x4 Icell unit, I_DIN1 = x2 Icell unit, I_DIN0 = x1 Icell unit.

<FIG> depicts digital summer <NUM>, which receives a plurality of digital values, sums them together and generates an output DOUT representing the sum of the inputs. Digital summer <NUM> can be used during a verifying operation after a programming operation. As shown for an example of a <NUM>-bit digital value, there are digital output bits to hold the value from <NUM> evaluation pulses, where the values are summed up at the end of the process. The digital outputs are digitally scaled based on the <NUM>^n*DINn bit position, for example, DOUT3 = x8 DOUT0, _DOUT2 = x4 DOUT1, I_DOUT1 = x2 DOUT0, I_DOUT0 = DOUT0.

<FIG> depict digital bits-to-pulse width converter <NUM> to be used within an input block, row decoder, or output block. The pulse width output from digital bits-to-pulse width converter <NUM> is proportional to its value as described above in relation to <FIG>. Digital bits-to-pulse width converter comprises binary counter <NUM>. The state Q [N:<NUM>] of binary counter <NUM> can be loaded by serial or parallel data in a loading sequence. Row control logic <NUM> outputs a voltage pulse with a pulse-width that is proportional to the value of the digital data inputs provided from blocks such as integrating ADC in <FIG> and <FIG>.

<FIG> shows the waveform for the output pulse width which has the width proportional to its digital bit values. First, the data in the received digital bits is inverted, and the inverted digit bits are loaded either serially or in parallel into counter <NUM>. Then the row pulse-width is generated by the row control logic <NUM> as shown in the waveform <NUM> by counting in a binary manner until it reaches the maximum counter value.

Optionally, a pulse series-to-pulse converter can be used to convert the output comprising a pulse series (such as signals <NUM> or <NUM> in <FIG> and signals <NUM> or <NUM> in <FIG> into a single pulse whose width varies in proportion to the number of pulses in the pulse series (such as signals WL0, WL1, and WLe in <FIG>) to be used as an input to a VMM array that will be applied to wordline or control gates within the VMM array. An example of a pulse series-to-pulse converter is a binary counter with control logic.

An example is as shown in the Table <NUM> for <NUM>-bit digital inputs:.

Another embodiment is using a up binary counter and a digital comparison logic. Namely the output pulse width is generated by counting a up binary counter until the digital outputs of the binary counter is same as the digital input bits.

Another embodiment is using a down binary counter. First the down binary counter is loaded serially or in parallel with the digital data input pattern. Then the output pulse width is generated by counting down the down binary counter until the digital outputs of the binary counter reaches minimum value, namely '<NUM>' logic state.

In another embodiment, the resolution of an analog-to-digital converter can be configured by a control signal. <FIG> depicts programmable ADC <NUM>. Programmable ADC <NUM> receives an analog signal, such as the output neuron current Ineu, and converts it to output <NUM>, which comprises a set of digital bits. Programmable ADC <NUM> receives configuration <NUM>, which can be an analog control signal or a set of digital control bits. In one example the resolution of output <NUM> is determined by configuration <NUM>. For instance, if the configuration <NUM> has a first value, the output can be a set of <NUM> bits, but if configuration <NUM> has a second value, the output can be a set of <NUM> bits.

A coarse level sensing circuit (not shown) can be used to sample a plurality of array output currents, and based on the value of this current, the gain (scaling factor) can be configured.

The gain can be configured for each particular neural network, and the gain can be set during neural network training for best performance.

In another embodiment, an ADC can be a hybrid of the architectures describe above. For example, a first ADC can be a hybrid of an SAR ADC and a slope ADC; a second ADC can be a hybrid of an SAR ADC and a ramp ADC; and a third ADC can be a hybrid of an algorithmic ADC and a ramp ADC; without limitation.

<FIG> depicts hybrid output conversion block <NUM>. Output block <NUM> receives differential signals IW+ and IW-. Successive approximation register ADC <NUM> receives differential signals IW+ and IW- and determines the higher order digital bits (e.g., the most significant bits B7-B4 in an <NUM>-bit digital representation) that best correspond to the analog value represented by IW+ and IW-. Once SAR ADC <NUM> determines those higher order bits, an analog signal representing the signal IW+ and IW- minus the value represented by the higher order bits is provided to serial ADC block <NUM> (such as slope ADC or ramp ADC), which then determines the lower order bits (e.g., the least significant bits B3-B0 in an <NUM>-bit digital representation) corresponding to that difference. The higher order bits and the lower order bits are then combined in serial fashion to create the digital output representing the input signal IW+ and IW-.

<FIG> depicts output block <NUM>. Output block <NUM> receives differential signals IW+ and IW-. Algorithmic ADC <NUM> determines the higher order bits (e.g., bits B7-B4 in an <NUM>-bit digital representation) corresponding to IW+ and IW-, and serial ADC block <NUM> then determines the lower order bits (e.g., bits B3-B0 for an <NUM>-bit digital representation).

<FIG> depicts output block <NUM>. Output block <NUM> receives differential signals IW+ and IW-. Output block <NUM> comprises a hybrid ADC, which converts differential signals IW+ and IW- into digital bits by combining different conversion scheme (such as in <FIG>) into one circuit.

<FIG> depicts configurable serial ADC <NUM>. It includes integrator <NUM> which integrates the output neuron current Ineu into the integrating capacitor <NUM> (Cint).

In one embodiment, VRAMP <NUM> is provided to the inverting input of comparator <NUM>. In this case IREF <NUM> is off. The digital output (count value) <NUM> is produced by ramping VRAMP <NUM> until the comparator <NUM> switches polarity, with counter <NUM> counting clock pulses from the beginning of the ramp until the comparator <NUM> switches polarity, at which point counter <NUM> provides digital output (count value) <NUM>.

In another embodiment, VREF <NUM> is provided to the inverting input of comparator <NUM>. VOUT 6203is ramped down by ramp current <NUM> (IREF) until VOUT <NUM> reaches VREF <NUM>, at which point the EC <NUM> signal disables the count of counter <NUM>, at which point counter <NUM> provides digital output (count value) <NUM>. The (n-bit) ADC <NUM> is configurable to have a lower precision (fewer than n bits) or a higher precision (more than n bits), depending on the target application. The configurability of precision is done by configuring the capacitance of capacitor <NUM>, the current <NUM> (IREF), the ramping rate of VRAMP <NUM>, or the clocking frequency of clock <NUM>, without limitation.

In another embodiment, the ADC circuit of a VMM array is configured to have a precision lower than n bits and the ADC circuits of another VMM array is configured to have high a precision greater than bits.

In another embodiment, one instance of serial ADC circuit <NUM> of one neuron circuit is configured to combine with another instance of serial ADC circuit <NUM> of the next neuron circuit to produce an ADC circuit with higher than n-bit precision, such as by combining the integrating capacitor <NUM> of the two instances of serial ADC circuits <NUM>.

<FIG> depicts a configurable neuron SAR (successive approximation register) ADC <NUM>. This circuit is a successive approximation converter based on charge redistribution using binary capacitors that converts a voltage input Vin to digital outputs <NUM>, based on a reference voltage VREF. It includes a binary capacitor DAC (CDAC) <NUM>, op-amp/comparator <NUM>, and SAR logic and registers <NUM>. As shown GndV <NUM> is a low voltage reference level, for example ground level. SAR logic and registers <NUM> provide digital outputs <NUM>. Other non-binary capacitor structures can be implemented with weighted reference voltages or correction with the outputs.

<FIG> depicts a pipelined SAR ADC circuit <NUM> that can be used to combine with the next SAR ADC to increase the number of bits in a pipelined fashion. SAR ADC circuit <NUM> comprises binary CDAC <NUM>, op-amp/comparator <NUM>, op-amp/comparator <NUM>, and SAR logic and registers <NUM>. As shown, GndV is a low voltage reference level, for example ground level. SAR logic and registers <NUM> provide digital outputs <NUM>. Vin is in the input voltage and VREF is a reference voltage. Vresidue is generated by capacitor <NUM> and is provided as an input to the next stage of an SAR ADC conversion sequence.

<FIG> depicts hybrid SAR + serial ADC circuit <NUM> that can be used to increase the number of bits in a hybrid fashion. SAR ADC circuit <NUM> comprises binary CDAC <NUM>, op-amp/comparator <NUM>, and SAR logic and registers <NUM>. As shown GndV is a low voltage reference level, for example ground level during the SAR ADC operation. SAR logic and registers <NUM> provides digital outputs. Vin is the input voltage and VREF is a reference voltage. The VREFRAMP is used as a reference ramping voltage during the serial ADC operation in place of the GndV input to the op-amp/comparator <NUM>.

Other embodiments of hybrid ADC architectures are SAR ADC plus sigma delta ADC, Flash ADC plus serial ADC, Pipelined ADC plus serial ADC, Serial ADC plus SAR ADC, and other architectures.

<FIG> depicts Algorithmic ADC output block <NUM>. Output block <NUM> comprises sample-and-hold circuit <NUM>, 1b analog-to-digital converter <NUM>, digital-to-analog converter <NUM>, summer <NUM>, operational amplifier <NUM>, and switches <NUM> and <NUM>, configured as shown.

In another embodiment, a sample-and-hold circuit is used for the input to each row in a VMM array. For example, if the input comprises a DAC, the DAC can comprise a sample-and-hold circuit.

Claim 1:
A programmable neuron output block (<NUM>) for generating an output (<NUM>) of a neural network memory array, comprising:
one or more nodes for receiving current from a neural network memory array in response to inputs applied to rows of the neural network memory array; and
a gain configuration circuit (<NUM>) for receiving a gain configuration signal (<NUM>) and applying a gain factor to the received current in response to the gain configuration signal to generate an output, wherein the gain configuration signal depends on values of the inputs applied to rows of the neural network memory array.