Patent Description:
Numerous embodiments of a precision tuning method and apparatus are disclosed for precisely and quickly depositing the correct amount of charge on the floating gate of a non-volatile memory cell within a vector-by-matrix multiplication (VMM) array in an artificial neural network. The invention is defined in the appended independent claim <NUM>. Preferred embodiments of the invention are set out in the appended dependent claims.

Non-patent literature document Mahmoodi M Reza ET AL: "An Ultra-Low Energy Internally Analog, Externally Digital Vector-Matrix Multiplier Based on NOR Flash Memory Technology", <NUM>55TH ACM/ESDA/IEEE Dsgn. (DAC) discloses a neural network comprising a vector-by-matrix multiplication array of non-volatile memory cells, wherein a weight value w is stored as a differential pair w+ and w- in a first non-volatile memory cell and a second non-volatile memory cell in the array according to the formula w = (w+) - (w-).

<FIG> illustrates an artificial neural network, where the circles represent the inputs or layers of neurons. The connections (called synapses) are represented by arrows, and have numeric weights that can be tuned based on experience. This makes the artificial neural network adaptive to inputs and capable of learning. Typically, artificial neural networks include a layer of multiple inputs. There are typically one or more intermediate layers of neurons, and an output layer of neurons that provide the output of the neural network. The neurons at each level individually or collectively make a decision based on the received data from the synapses.

One of the major challenges in the development of artificial neural networks for high-performance information processing is a lack of adequate hardware technology. Indeed, practical artificial neural networks rely on a very large number of synapses, enabling high connectivity between neurons, i.e. a very high computational parallelism. In principle, such complexity can be achieved with digital supercomputers or specialized graphics processing unit clusters. However, in addition to high cost, these approaches also suffer from mediocre energy efficiency as compared to biological networks, which consume much less energy primarily because they perform low-precision analog computation. CMOS analog circuits have been used for artificial neural networks, but most CMOS-implemented synapses have been too bulky given the high number of neurons and synapses.

Applicant previously disclosed an artificial (analog) neural network that utilizes one or more non-volatile memory arrays as the synapses in <CIT>, published as US Patent Publication <CIT>. The non-volatile memory arrays operate as an analog neuromorphic memory. The term neuromorphic, as used herein, means circuitry that implement models of neural systems. The analog neuromorphic memory includes a first plurality of synapses configured to receive a first plurality of inputs and to generate therefrom a first plurality of outputs, and a first plurality of neurons configured to receive the first plurality of outputs. The first plurality of synapses includes a plurality of memory cells, wherein each of the memory cells includes spaced apart source and drain regions formed in a semiconductor substrate with a channel region extending there between, a floating gate disposed over and insulated from a first portion of the channel region and a non-floating gate disposed over and insulated from a second portion of the channel region. Each of the plurality of memory cells is configured to store a weight value corresponding to a number of electrons on the floating gate. The plurality of memory cells is configured to multiply the first plurality of inputs by the stored weight values to generate the first plurality of outputs. An array of memory cells arranged in this manner can be referred to as a vector by matrix multiplication (VMM) array.

Each non-volatile memory cell used in the VMM array must be erased and programmed to hold a very specific and precise amount of charge, i.e., the number of electrons, in the floating gate. For example, each floating gate must hold one of N different values, where N is the number of different weights that can be indicated by each cell. Examples of N include <NUM>, <NUM>, <NUM>, <NUM>, and <NUM>. One challenge is the ability to program selected cells with the precision and granularity required for different values of N. For example, if a selected cell can include one of <NUM> different values, extreme precision is required in program operations.

What is needed are improved programming systems and methods suitable for use with a VMM array in an analog neuromorphic memory.

Numerous embodiments of a precision tuning algorithm and apparatus are disclosed for precisely and quickly depositing the correct amount of charge on the floating gate of a non-volatile memory cell within a VMM array in an analog neuromorphic memory system. Selected cells thereby can be programmed with extreme precision to hold one of N different values.

A neural network comprises a vector-by-matrix multiplication array of non-volatile memory cells, wherein a weight value w is stored as a differential pair w+ and win a first non-volatile memory cell and a second non-volatile memory cell in the array according to the formula w = (w+) - (w-), where w+ and w- include a non-zero offset value.

In an embodiment, a neural network comprises a vector-by-matrix multiplication array of non-volatile memory cells, the array organized into rows and columns of non-volatile memory cells, wherein a weight value w is stored as a differential pair w+ and w- in a first non-volatile memory cell and a second non-volatile memory cell according to the formula w = (w+) - (w-), where the storage of w+ and w- values are approximately evenly spread among all columns in the array.

In another embodiment, A method of programming, verifying, and reading a zero value in a differential pair of non-volatile memory cells in a vector-by-matrix multiplication array comprises programming a first cell, w+, in the differential pair to a first current value, verifying the first cell by applying a voltage to a control gate terminal of the first cell equal to a first voltage plus a bias voltage, programming a second cell, w-, in the differential pair to the first current value, verifying the second cell by applying a voltage to a control gate terminal of the second cell equal to the first voltage plus the bias voltage, reading the first cell by applying a voltage to the control gate terminal of the first cell equal to the first voltage,
reading the second cell by applying a voltage to the control gate terminal of the second cell equal to the first voltage, and calculating a value w according to the formula w = (w+) - (w-).

In another embodiment, a method of programming, verifying, and reading a zero value in a differential pair of non-volatile memory cells in a vector-by-matrix multiplication array comprises programming a first cell, w+, in the differential pair to a first current value, verifying the first cell by applying a voltage to a control gate terminal of the first cell equal to a first voltage plus a bias voltage, programming a second cell, w-, in the differential pair to the first current value, verifying the second cell by applying a voltage to a control gate terminal of the second cell equal to the first voltage plus the bias voltage, reading the first cell by applying a voltage to the control gate terminal of the first cell equal to the first voltage, reading the second cell by applying a voltage to the control gate terminal of the second cell equal to the first voltage, and calculating a value w according to the formula w = (w+) - (w-).

In another embodiment, a neural network comprises a vector-by-matrix multiplication array of non-volatile memory cells, the array organized into rows and columns of non-volatile memory cells, wherein a weight value w is stored as a differential pair w+ and w- according to the formula w = (w+) - (w-), wherein w+ is stored as a differential pair in a first non-volatile memory cell and a second non-volatile memory cell in the array and w- is stored as a differential pair in a third non-volatile memory cell and a fourth non-volatile memory cell in the array, wherein the storage of w+ and w- values are offset by a bias value.

In another embodiment, a neural network comprises a vector-by-matrix multiplication array of non-volatile memory cells, the array organized into rows and columns of non-volatile memory cells, wherein a weight value w is stored as a differential pair w+ and w- in a first non-volatile memory cell and a second non-volatile memory cell according to the formula w = (w+) - (w-), wherein values for w+ are selected from a first range of non-zero values and the values for w- are selected from a second range of non-zero values, wherein the first range and second range do not overlap.

In another embodiment, a method of reading non-volatile memory cells in a vector-by-matrix multiplication array comprises reading a weight stored in a selected cell in the array, comprising: applying a zero voltage bias to a control gate terminal of the selected cell, and sensing a neuron output current comprising current output from the selected cell.

In another embodiment a method of operating a non-volatile memory cell in a vector-by-matrix multiplication array comprises reading the non-volatile memory cell by applying a first bias voltage to a control gate of the non-volatile memory cell, and applying a second bias voltage to the control gate of the non-volatile memory cell during one or more of a standby operation, a deep power down operation, or a testing operation.

Digital non-volatile memories are well known. For example, <CIT> ("the '<NUM> patent"), discloses an array of split gate non-volatile memory cells, which are a type of flash memory cells. Such a memory cell <NUM> is shown in <FIG>. Each memory cell <NUM> includes source region <NUM> and drain region <NUM> formed in semiconductor substrate <NUM>, with channel region <NUM> there between. Floating gate <NUM> is formed over and insulated from (and controls the conductivity of) a first portion of the channel region <NUM>, and over a portion of the source region <NUM>. Word line terminal <NUM> (which is typically coupled to a word line) has a first portion that is disposed over and insulated from (and controls the conductivity of) a second portion of the channel region <NUM>, and a second portion that extends up and over the floating gate <NUM>. The floating gate <NUM> and word line terminal <NUM> are insulated from the substrate <NUM> by a gate oxide. Bitline terminal <NUM> is coupled to drain region <NUM>.

Memory cell <NUM> is programmed (where electrons are placed on the floating gate) by placing a positive voltage on the word line terminal <NUM>, and a positive voltage on the source region <NUM>. Electron current will flow from the source region <NUM> (source line terminal) towards the drain region <NUM>. The electrons will accelerate and become heated when they reach the gap between the word line terminal <NUM> and the floating gate <NUM>. Some of the heated electrons will be injected through the gate oxide onto the floating gate <NUM> due to the attractive electrostatic force from the floating gate <NUM>.

<FIG> shows memory cell <NUM>, which is similar to memory cell <NUM> of <FIG> with the addition of control gate (CG) terminal <NUM>. Control gate terminal <NUM> is biased at a high voltage, e.g., 10V, in programming, low or negative in erase, e.g., 0v/-8V, low or mid range in read, e.g., 0v/<NUM>. Other terminals are biased similarly to that of <FIG>.

<FIG> shows memory cell <NUM>, which is similar to memory cell <NUM> of <FIG> except that memory cell <NUM> does not contain an erase gate EG terminal. An erase is performed by biasing the substrate <NUM> to a high voltage and biasing the control gate CG terminal <NUM> to a low or negative voltage. Alternatively, an erase is performed by biasing word line terminal <NUM> to a positive voltage and biasing control gate terminal <NUM> to a negative voltage. Programming and reading is similar to that of <FIG>.

<FIG> depicts a three-gate memory cell <NUM>, which is another type of flash memory cell. Memory cell <NUM> is identical to the memory cell <NUM> of <FIG> except that memory cell <NUM> does not have a separate control gate terminal. The erase operation (whereby erasing occurs through use of the erase gate terminal) and read operation are similar to that of the <FIG> except there is no control gate bias applied. The programming operation also is done without the control gate bias, and as a result, a higher voltage must be applied on the source line terminal during a program operation to compensate for a lack of control gate bias.

<FIG> depicts stacked gate memory cell <NUM>, which is another type of flash memory cell. Memory cell <NUM> is similar to memory cell <NUM> of <FIG>, except that floating gate <NUM> extends over the entire channel region <NUM>, and control gate terminal <NUM> (which here will be coupled to a word line) extends over floating gate <NUM>, separated by an insulating layer (not shown). The erase, programming, and read operations operate in a similar manner to that described previously for memory cell <NUM>.

The methods and means described herein may apply to other non-volatile memory technologies such as SONOS (silicon-oxide-nitride-oxide-silicon, charge trap in nitride), MONOS (metal-oxide-nitride-oxide-silicon, metal charge trap in nitride), ReRAM (resistive ram), PCM (phase change memory), MRAM (magnetic ram), FeRAM (ferroelectric ram), OTP (bi-level or multi-level one time programmable), and CeRAM (correlated electron ram), without limitation. The methods and means described herein may apply to volatile memory technologies used for neural network such as SRAM, DRAM, and other volatile synapse cells, without limitation.

S0 is the input layer, which for this example is a 32x32 pixel RGB image with <NUM> bit precision (i.e. three 32x32 pixel arrays, one for each color R, G and B, each pixel being <NUM> bit precision). The synapses CB1 going from input layer S0 to layer C1 apply different sets of weights in some instances and shared weights in other instances, and scan the input image with 3x3 pixel overlapping filters (kernel), shifting the filter by <NUM> pixel (or more than <NUM> pixel as dictated by the model). Specifically, values for <NUM> pixels in a 3x3 portion of the image (i.e., referred to as a filter or kernel) are provided to the synapses CB1, where these <NUM> input values are multiplied by the appropriate weights and, after summing the outputs of that multiplication, a single output value is determined and provided by a first synapse of CB1 for generating a pixel of one of the layers of feature map C1. The 3x3 filter is then shifted one pixel to the right within input layer S0 (i.e., adding the column of three pixels on the right, and dropping the column of three pixels on the left), whereby the <NUM> pixel values in this newly positioned filter are provided to the synapses CB1, where they are multiplied by the same weights and a second single output value is determined by the associated synapse. This process is continued until the 3x3 filter scans across the entire 32x32 pixel image of input layer S0, for all three colors and for all bits (precision values). The process is then repeated using different sets of weights to generate a different feature map of C1, until all the features maps of layer C1 have been calculated.

<FIG> is a block diagram of a system that can be used for that purpose. VMM system <NUM> includes non-volatile memory cells and is utilized as the synapses (such as CB1, CB2, CB3, and CB4 in <FIG>) between one layer and the next layer. Specifically, VMM system <NUM> comprises VMM array <NUM> comprising non-volatile memory cells arranged in rows and columns, erase gate and word line gate decoder <NUM>, control gate decoder <NUM>, bit line decoder <NUM> and source line decoder <NUM>, which decode the respective inputs for the non-volatile memory cell array <NUM>. Input to VMM array <NUM> can be from the erase gate and wordline gate decoder <NUM> or from the control gate decoder <NUM>. Source line decoder <NUM> in this example also decodes the output of VMM array <NUM>. Alternatively, bit line decoder <NUM> can decode the output of VMM array <NUM>.

VMM array <NUM> serves two purposes. First, it stores the weights that will be used by the VMM system <NUM>. Second, VMM array <NUM> effectively multiplies the inputs by the weights stored in VMM array <NUM> and adds them up per output line (source line or bit line) to produce the output, which will be the input to the next layer or input to the final layer. By performing the multiplication and addition function, VMM array <NUM> negates the need for separate multiplication and addition logic circuits and is also power efficient due to its in-situ memory computation.

The output of VMM array <NUM> is supplied to a differential summer (such as a summing opamp or a summing current mirror) <NUM>, which sums up the outputs of VMM array <NUM> to create a single value for that convolution. The differential summer <NUM> is arranged to perform summation of both positive weight and negative weight inputs to output the single value.

The summed up output values of differential summer <NUM> are then supplied to an activation function circuit <NUM>, which rectifies the output. The activation function circuit <NUM> may provide sigmoid, tanh, ReLU functions, or any other non-linear function. The rectified output values of activation function circuit <NUM> become an element of a feature map of the next layer (e.g. C1 in <FIG>), and are then applied to the next synapse to produce the next feature map layer or final layer. Therefore, in this example, VMM array <NUM> constitutes a plurality of synapses (which receive their inputs from the prior layer of neurons or from an input layer such as an image database), and summer <NUM> and activation function circuit <NUM> constitute a plurality of neurons.

The input to VMM system <NUM> in <FIG> (WLx, EGx, CGx, and optionally BLx and SLx) can be analog level, binary level, digital pulses (in which case a pulses-to-analog converter PAC may be needed to convert pulses to the appropriate input analog level) or digital bits (in which case a DAC is provided to convert digital bits to appropriate input analog level) and the output can be analog level, binary level, digital pulses, or digital bits (in which case an output ADC is provided to convert output analog level into digital bits).

The output generated by input VMM system 32a is provided as an input to the next VMM system (hidden level <NUM>) 32b, which in turn generates an output that is provided as an input to the next VMM system (hidden level <NUM>) 32c, and so on. The various layers of VMM system <NUM> function as different layers of synapses and neurons of a convolutional neural network (CNN). Each VMM system 32a, 32b, 32c, 32d, and 32e can be a stand-alone, physical system comprising a respective non-volatile memory array, or multiple VMM systems could utilize different portions of the same physical non-volatile memory array, or multiple VMM systems could utilize overlapping portions of the same physical non-volatile memory array. Each VMM system 32a, 32b, 32c, 32d, and 32e can also be time multiplexed for various portion of its array or neurons. The example shown in <FIG> contains five layers (32a,32b,32c,32d,32e): one input layer (32a), two hidden layers (32b,32c), and two fully connected layers (32d,32e). One of ordinary skill in the art will appreciate that this is merely exemplary and that a system instead could comprise more than two hidden layers and more than two fully connected layers.

The non-volatile reference memory cells and the non-volatile memory cells described herein are biased in weak inversion: <MAT> where w = e (- Vth)/nVt
where Ids is the drain to source current; Vg is gate voltage on the memory cell; Vth is threshold voltage of the memory cell; Vt is thermal voltage = k*T/q with k being the Boltzmann constant, T the temperature in Kelvin, and q the electronic charge; n is a slope factor = <NUM> + (Cdep/Cox) with Cdep = capacitance of the depletion layer, and Cox capacitance of the gate oxide layer; Io is the memory cell current at gate voltage equal to threshold voltage, Io is proportional to (Wt/L)*u*Cox* (n-<NUM>) * Vt<NUM> where u is carrier mobility and Wt and L are width and length, respectively, of the memory cell.

For an I-to-V log converter using a memory cell (such as a reference memory cell or a peripheral memory cell) or a transistor to convert input current Ids, into an input voltage, Vg: <MAT> Here, wp is w of a reference or peripheral memory cell.

For an I-to-V log converter using a memory cell (such as a reference memory cell or a peripheral memory cell) or a transistor to convert input current Ids, into an input voltage, Vg: <MAT>.

Here, wp is w of a reference or peripheral memory cell.

For a memory array used as a vector matrix multiplier VMM array, the output current is: <MAT> namely <MAT> <MAT> <MAT> Here, wa = w of each memory cell in the memory array.

Alternatively, the non-volatile memory cells of VMM arrays described herein can be configured to operate in the linear region: <MAT> <MAT> meaning weight W in the linear region is proportional to (Vgs-Vth).

Alternatively, the memory cells of VMM arrays described herein can be configured to operate in the saturation region: <MAT> Wα (Vgs-Vth)<NUM>, meaning weight W is proportional to (Vgs-Vth)<NUM>.

Alternatively, the memory cells of VMM arrays described herein can be used in all regions or a combination thereof (sub threshold, linear, or saturation).

Other embodiments for VMM array <NUM> of <FIG> are described in <CIT>. As described in that application, a sourceline or a bitline can be used as the neuron output (current summation output).

VMM array <NUM> implements uni-directional tuning for non-volatile memory cells in memory array <NUM>. That is, each non-volatile memory cell is erased and then partially programmed until the desired charge on the floating gate is reached. This can be performed, for example, using the precision programming techniques described below. If too much charge is placed on the floating gate (such that the wrong value is stored in the cell), the cell must be erased and the sequence of partial programming operations must start over. As shown, two rows sharing the same erase gate (such as EG0 or EG1) need to be erased together (which is known as a page erase), and thereafter, each cell is partially programmed until the desired charge on the floating gate is reached.

In VMM array <NUM>, the inputs INPUT<NUM>. , INPUTN are received on bit lines BL<NUM>,. BLN, respectively, and the outputs OUTPUT<NUM>, OUTPUT<NUM>, OUTPUT<NUM>, and OUTPUT<NUM> are generated on source lines SL<NUM>, SL<NUM>, SL<NUM>, and SL<NUM>, respectively.

In this example, the inputs INPUT<NUM>, INPUT<NUM>, INPUT<NUM>, and INPUT<NUM> are received on source lines SL<NUM>, SL<NUM>, SL<NUM>, and SL<NUM>, respectively, and the outputs OUTPUT<NUM>,. OUTPUTN are generated on bit lines BL<NUM>,.

In this example, the inputs INPUT<NUM>,. , INPUTM are received on word lines WL<NUM>,. , WLM, respectively, and the outputs OUTPUT<NUM>,. OUTPUTN are generated on bit lines BL<NUM>,.

In this example, the inputs INPUT<NUM>,. , INPUTn are received on vertical control gate lines CG<NUM>,. , CGN, respectively, and the outputs OUTPUT<NUM> and OUTPUT<NUM> are generated on source lines SL<NUM> and SL<NUM>.

In this example, the inputs INPUT<NUM>,. , INPUTN are received on the gates of bit line control gates <NUM>-<NUM>, <NUM>-<NUM>,. , <NUM>-(N-<NUM>), and <NUM>-N, respectively, which are coupled to bit lines BL<NUM>,. , BLN, respectively. Exemplary outputs OUTPUT<NUM> and OUTPUT<NUM> are generated on source lines SL<NUM> and SL<NUM>.

<FIG> depicts neuron VMM array <NUM>, which is particularly suited for memory cells <NUM> as shown in <FIG>, memory cells <NUM> as shown in <FIG>, and memory cells <NUM> as shown in <FIG>, and is utilized as the synapses and parts of neurons between an input layer and the next layer. In this example, the inputs INPUT<NUM>,. , INPUTM are received on word lines WL<NUM>,. , WLM, and the outputs OUTPUT<NUM>,. , OUTPUTN are generated on bit lines BL<NUM>,. , BLN, respectively.

<FIG> depicts neuron VMM array <NUM>, which is particularly suited for memory cells <NUM> as shown in <FIG>, memory cells <NUM> as shown in <FIG>, and memory cells <NUM> as shown in <FIG>, and is utilized as the synapses and parts of neurons between an input layer and the next layer. In this example, the inputs INPUT<NUM>,. , INPUTM are received on control gate lines CG<NUM>,. Outputs OUTPUT<NUM>,. , OUTPUTN are generated on vertical source lines SL<NUM>,. , SLN, respectively, where each source line SLi is coupled to the source lines of all memory cells in column i.

<FIG> depicts neuron VMM array <NUM>, which is particularly suited for memory cells <NUM> as shown in <FIG>, memory cells <NUM> as shown in <FIG>, and memory cells <NUM> as shown in <FIG>, and is utilized as the synapses and parts of neurons between an input layer and the next layer. In this example, the inputs INPUT<NUM>,. , INPUTM are received on control gate lines CG<NUM>,. Outputs OUTPUT<NUM>,. , OUTPUTN are generated on vertical bit lines BL<NUM>,. , BLN, respectively, where each bit line BLi is coupled to the bit lines of all memory cells in column i.

The prior art includes a concept known as long short-term memory (LSTM). LSTMs often are used in artificial neural networks. LSTM allows an artificial neural network to remember information over predetermined arbitrary time intervals and to use that information in subsequent operations. A conventional LSTM comprises a cell, an input gate, an output gate, and a forget gate. The three gates regulate the flow of information into and out of the cell and the time interval that the information is remembered in the LSTM. VMMs are particularly useful in LSTMs.

<FIG> depicts an exemplary LSTM <NUM>. LSTM <NUM> in this example comprises cells <NUM>, <NUM>, <NUM>, and <NUM>. Cell <NUM> receives input vector x<NUM> and generates output vector h<NUM> and cell state vector c<NUM>. Cell <NUM> receives input vector x<NUM>, the output vector (hidden state) h<NUM> from cell <NUM>, and cell state c<NUM> from cell <NUM> and generates output vector h<NUM> and cell state vector c<NUM>. Cell <NUM> receives input vector x<NUM>, the output vector (hidden state) h<NUM> from cell <NUM>, and cell state c<NUM> from cell <NUM> and generates output vector h<NUM> and cell state vector c<NUM>. Cell <NUM> receives input vector x<NUM>, the output vector (hidden state) h<NUM> from cell <NUM>, and cell state c<NUM> from cell <NUM> and generates output vector h<NUM>. Additional cells can be used, and an LSTM with four cells is merely an example.

<FIG> depicts an exemplary implementation of an LSTM cell <NUM>, which can be used for cells <NUM>, <NUM>, <NUM>, and <NUM> in <FIG>. LSTM cell <NUM> receives input vector x(t), cell state vector c(t-<NUM>) from a preceding cell, and output vector h(t-<NUM>) from a preceding cell, and generates cell state vector c(t) and output vector h(t).

LSTM cell <NUM> comprises sigmoid function devices <NUM>, <NUM>, and <NUM>, each of which applies a number between <NUM> and <NUM> to control how much of each component in the input vector is allowed through to the output vector. LSTM cell <NUM> also comprises tanh devices <NUM> and <NUM> to apply a hyperbolic tangent function to an input vector, multiplier devices <NUM>, <NUM>, and <NUM> to multiply two vectors together, and addition device <NUM> to add two vectors together. Output vector h(t) can be provided to the next LSTM cell in the system, or it can be accessed for other purposes.

<FIG> depicts an LSTM cell <NUM>, which is an example of an implementation of LSTM cell <NUM>. For the reader's convenience, the same numbering from LSTM cell <NUM> is used in LSTM cell <NUM>. Sigmoid function devices <NUM>, <NUM>, and <NUM> and tanh device <NUM> each comprise multiple VMM arrays <NUM> and activation circuit blocks <NUM>. Thus, it can be seen that VMM arrays are particular useful in LSTM cells used in certain neural network systems.

An alternative to LSTM cell <NUM> (and another example of an implementation of LSTM cell <NUM>) is shown in <FIG>. In <FIG>, sigmoid function devices <NUM>, <NUM>, and <NUM> and tanh device <NUM> may share the same physical hardware (VMM arrays <NUM> and activation function block <NUM>) in a time-multiplexed fashion. LSTM cell <NUM> also comprises multiplier device <NUM> to multiply two vectors together, addition device <NUM> to add two vectors together, tanh device <NUM> (which comprises activation circuit block <NUM>), register <NUM> to store the value i(t) when i(t) is output from sigmoid function block <NUM>, register <NUM> to store the value f(t) * c(t-<NUM>) when that value is output from multiplier device <NUM> through multiplexor <NUM>, register <NUM> to store the value i(t) * u(t) when that value is output from multiplier device <NUM> through multiplexor <NUM>, and register <NUM> to store the value o(t) * c~(t) when that value is output from multiplier device <NUM> through multiplexor <NUM>, and multiplexor <NUM>.

Whereas LSTM cell <NUM> contains multiple sets of VMM arrays <NUM> and respective activation function blocks <NUM>, LSTM cell <NUM> contains only one set of VMM arrays <NUM> and activation function block <NUM>, which are used to represent multiple layers in the embodiment of LSTM cell <NUM>. LSTM cell <NUM> will require less space than LSTM <NUM>, as LSTM cell <NUM> will require <NUM>/<NUM> as much space for VMMs and activation function blocks compared to LSTM cell <NUM>.

It can be further appreciated that LSTM units will typically comprise multiple VMM arrays, each of which requires functionality provided by certain circuit blocks outside of the VMM arrays, such as a summer and activation circuit block and high voltage generation blocks. Providing separate circuit blocks for each VMM array would require a significant amount of space within the semiconductor device and would be somewhat inefficient. The embodiments described below therefore attempt to minimize the circuitry required outside of the VMM arrays themselves.

An analog VMM implementation can be utilized for a GRU (gated recurrent unit). GRUs are a gating mechanism in recurrent artificial neural networks. GRUs are similar to LSTMs, except that GRU cells generally contain fewer components than an LSTM cell.

<FIG> depicts an exemplary GRU <NUM>. GRU <NUM> in this example comprises cells <NUM>, <NUM>, <NUM>, and <NUM>. Cell <NUM> receives input vector x<NUM> and generates output vector h<NUM>. Cell <NUM> receives input vector x<NUM>, the output vector h<NUM> from cell <NUM> and generates output vector h<NUM>. Cell <NUM> receives input vector x<NUM> and the output vector (hidden state) h<NUM> from cell <NUM> and generates output vector h<NUM>. Cell <NUM> receives input vector x<NUM> and the output vector (hidden state) h<NUM> from cell <NUM> and generates output vector h<NUM>. Additional cells can be used, and an GRU with four cells is merely an example.

<FIG> depicts an exemplary implementation of a GRU cell <NUM>, which can be used for cells <NUM>, <NUM>, <NUM>, and <NUM> of <FIG>. GRU cell <NUM> receives input vector x(t) and output vector h(t-<NUM>) from a preceding GRU cell and generates output vector h(t). GRU cell <NUM> comprises sigmoid function devices <NUM> and <NUM>, each of which applies a number between <NUM> and <NUM> to components from output vector h(t-<NUM>) and input vector x(t). GRU cell <NUM> also comprises a tanh device <NUM> to apply a hyperbolic tangent function to an input vector, a plurality of multiplier devices <NUM>, <NUM>, and <NUM> to multiply two vectors together, an addition device <NUM> to add two vectors together, and a complementary device <NUM> to subtract an input from <NUM> to generate an output.

<FIG> depicts a GRU cell <NUM>, which is an example of an implementation of GRU cell <NUM>. For the reader's convenience, the same numbering from GRU cell <NUM> is used in GRU cell <NUM>. As can be seen in <FIG>, sigmoid function devices <NUM> and <NUM>, and tanh device <NUM> each comprise multiple VMM arrays <NUM> and activation function blocks <NUM>. Thus, it can be seen that VMM arrays are of particular use in GRU cells used in certain neural network systems.

An alternative to GRU cell <NUM> (and another example of an implementation of GRU cell <NUM>) is shown in <FIG>. In <FIG>, GRU cell <NUM> utilizes VMM arrays <NUM> and activation function block <NUM>, which when configured as a sigmoid function applies a number between <NUM> and <NUM> to control how much of each component in the input vector is allowed through to the output vector. In <FIG>, sigmoid function devices <NUM> and <NUM> and tanh device <NUM> share the same physical hardware (VMM arrays <NUM> and activation function block <NUM>) in a time-multiplexed fashion. GRU cell <NUM> also comprises multiplier device <NUM> to multiply two vectors together, addition device <NUM> to add two vectors together, complementary device <NUM> to subtract an input from <NUM> to generate an output, multiplexor <NUM>, register <NUM> to hold the value h(t-<NUM>) * r(t) when that value is output from multiplier device <NUM> through multiplexor <NUM>, register <NUM> to hold the value h(t-<NUM>) *z(t) when that value is output from multiplier device <NUM> through multiplexor <NUM>, and register <NUM> to hold the value h^(t) * (<NUM>-z(t)) when that value is output from multiplier device <NUM> through multiplexor <NUM>.

Whereas GRU cell <NUM> contains multiple sets of VMM arrays <NUM> and activation function blocks <NUM>, GRU cell <NUM> contains only one set of VMM arrays <NUM> and activation function block <NUM>, which are used to represent multiple layers in the embodiment of GRU cell <NUM>. GRU cell <NUM> will require less space than GRU cell <NUM>, as GRU cell <NUM> will require <NUM>/<NUM> as much space for VMMs and activation function blocks compared to GRU cell <NUM>.

It can be further appreciated that systems utilizing GRUs will typically comprise multiple VMM arrays, each of which requires functionality provided by certain circuit blocks outside of the VMM arrays, such as a summer and activation circuit block and high voltage generation blocks. Providing separate circuit blocks for each VMM array would require a significant amount of space within the semiconductor device and would be somewhat inefficient. The embodiments described below therefore attempt to minimize the circuitry required outside of the VMM arrays themselves.

The input to the VMM arrays can be an analog level, a binary level, timing pulses, or digital bits and the output can be an analog level, a binary level, timing pulses, or digital bits (in this case an output ADC is needed to convert output analog level current or voltage into digital bits).

For each memory cell in a VMM array, each weight w can be implemented by a single memory cell or by a differential cell or by two blend memory cells (average of <NUM> or more cells). In the differential cell case, two memory cells are needed to implement a weight w as a differential weight (w = w+ - w-). In the two blend memory cells, two memory cells are needed to implement a weight w as an average of two cells.

<FIG> depicts a block diagram of VMM system <NUM>. VMM system <NUM> comprises VMM array <NUM>, row decoders <NUM>, high voltage decoders <NUM>, column decoders <NUM>, bit line drivers <NUM>, input circuit <NUM>, output circuit <NUM>, control logic <NUM>, and bias generator <NUM>. VMM system <NUM> further comprises high voltage generation block <NUM>, which comprises charge pump <NUM>, charge pump regulator <NUM>, and high voltage level generator <NUM>. VMM system <NUM> further comprises algorithm controller <NUM>, analog circuitry <NUM>, control logic <NUM>, and test control logic <NUM>. The systems and methods described below can be implemented in VMM system <NUM>.

Input circuit <NUM> may include circuits such as a DAC (digital to analog converter), DPC (digital to pulses converter), AAC (analog to analog converter, such as current to voltage converter), PAC (pulse to analog level converter), or any other type of converter. Input circuit <NUM> may implement normalization, scaling functions, or arithmetic functions. Input circuit <NUM> may implement a temperature compensation function for the input. Input circuit <NUM> may implement an activation function such as ReLU or a sigmoid function.

Output circuit <NUM> may include circuits such as a ADC (analog to digital converter, to convert neuron analog output to digital bits), AAC (analog to analog converter, such as current to voltage converter), APC (analog to pulses converter), or any other type of converter. Output circuit <NUM> may implement an activation function such as ReLU or a sigmoid function. Output circuit <NUM> may implement normalization, scaling functions, or arithmetic functions for neuron outputs. Output circuit <NUM> may implement a temperature compensation function for neuron outputs or array outputs (such as bitline outputs), as described below.

<FIG> depicts tuning (program or erase the memory cells to a target) correction method <NUM>, which can be executed by algorithm controller <NUM> in VMM system <NUM>. Tuning correction method <NUM> generates an adaptive target based on the final error resulting from a cell output and the cell original target. The method begins, typically in response to a tuning command being received (step <NUM>). An initial current target (used for a program/verify algorithm) for a selected cell or group of selected cells, Itargetv(i), is determined using a predictive target model such by using a function or a look-up table, and variable DeltaError is set to <NUM> (step <NUM>). The target function, if used, would be based upon the I-V program curve of the selected memory cell or group of cells. The target function also depends on various variations that are caused by the array characteristics such as the degree of program disturb that the cell exhibits (which depends on the cell address within a sector and the cell level, where the cell is subjected to more program time in an inhibit condition if it exhibits relatively greater disturb, where cell that has a higher current typically has more disturb), cell to cell coupling, and various types of array noise. These variations can be characterized on the silicon over PVT (process, voltage, temperature). The look-up table, if used, can be characterized in the same manner to emulate the I-V curve and various variations.

Next, a soft erase is performed on all cells in the VMM, which erases all cells to an intermediate weakly erased level such that each cell would draw current of, for example, approximately <NUM>-<NUM>µA during a read operation (step <NUM>). The soft erase is performed, for example, by applying incremental erase pulse voltages to the cells until an intermediate cell current is reached. Next, a programming operation (such as by deep programming or by coarse/fine programming to a target) is performed on all unused cells (step <NUM>) such as to get to < pA current level or to an equivalent zero weight. Then target adjustment (correction) based on the error result is performed. If DeltaError > <NUM>, meaning the cell has undergone an overshoot in programming, Itargetv (i+<NUM>) is then set to Itarget + theta* DeltaError, where theta, for example, is <NUM> or a number close to <NUM> (step 3405A).

The Itarget (i+<NUM>) can also be adjusted basing on the previous Itarget(i) with appropriate error target adjustment/correction. If DeltaError < <NUM>, meaning that the cell has undergone an undershoot in programming, meaning the cell current does not reach the target yet, then Itargetv (i+<NUM>) is set to the previous target Itargetv (i) (step 3405B).

Next, a coarse and/or fine program and verify operation is performed (step <NUM>). Multiple adaptive coarse programming methods can be used to speed up the programming such as by targeting multiple gradually smaller coarse targets before executing the precision (fine) programming step. The adaptive precision programming is done, for example, with fine (precision) incremental program voltage pulses or constant program timing pulses. Examples of systems and methods for performing coarse programming and fine programming are described in <CIT>, and titled, "Precise Programming Method and Apparatus for Analog Neural Memory in a Deep Learning Artificial Neural Network,".

Icell is measured in a selected cell (step <NUM>). The cell current, for example, can be measured by an ammeter circuit. The cell current, for example, can be measured by an ADC (Analog-to-Digital converter) circuit, where in this case the output is represented by digital bits. The cell current, for example, can be measured by an I-to-V (Current-to-Voltage converter) circuit, where in this case the output is represented by an analog voltage. DeltaError is calculated, which is Icell - Itarget, which represents the difference between the actual current in the measured cell (Icell) and the target current (Itarget). If | DeltaError | < DeltaMargin, then the cell has achieved the target current within a certain tolerance (DeltaMargin), and the method is concluded (step <NUM>). | DeltaError | = abs (DeltaError) = absolute value of DeltaError. If not, then the method returns to step <NUM> and performs the steps sequentially again (step <NUM>).

<FIG> and <FIG> depict tuning correction method <NUM>, which can be executed by algorithm controller <NUM> in VMM system <NUM>. With reference to <FIG>, the method starts (step <NUM>), which typically occurs in response to a tuning command being received. The entire VMM array is erased such as by a soft erase method (step <NUM>). A programming operation (such as by deep programming or by coarse/fine programming to a target) is performed on all unused cells (step <NUM>) to get cell currents < pA level or to an equivalent zero weight. All cells in the VMM array are programmed to an intermediate value, such as <NUM>-<NUM>µA, using a coarse and/or fine programming cycle (step <NUM>) Examples of systems and methods for performing coarse programming and fine programming are described in <CIT>, and titled, "Precise Programming Method and Apparatus for Analog Neural Memory in a Deep Learning Artificial Neural Network,". A predictive target is set for used cells using a function or look-up table (step <NUM>) as described above. Next, sector tuning method <NUM> is performed for each sector in VMM (step <NUM>). A sector typically consists of two or more adjacent rows in the array.

<FIG> depicts adaptive target sector tuning method <NUM>. All cells in a sector are programmed to final desired values (e.g., 1nA - 50nA) using individual or combinations program/verify (P/V) methods such as the following: (<NUM>) coarse/fine/constant P/V cycles; (<NUM>) CG+ (CG increment only) or EG+ (EG increment only) or complementary CG+/EG- (CG increment and EG decrement); and (<NUM>) deepest programmed cells first (such as progressive grouping, meaning grouping cells into groups, groups with cells with lowest current programmed first) (step 3508A). Next, a determination is made as to whether Icell < Itarget. If yes, then the method proceeds to step <NUM>. If no, then the method repeats step 3508A. In step <NUM>, , DeltaError is measured, which equals Icell measured - Itarget (i+<NUM>) (step <NUM>). A determination is made as to whether | DeltaError | < DeltaMargin (step <NUM>). If yes, the method is done (step <NUM>). If not, a target adjustment is performed. If DeltaError > <NUM>, meaning the cell has undergone an overshoot in programing, the target is adjusted by setting a new target to Itarget + theta* DeltaError, where theta typically = <NUM> (step 3512A). The Itarget (i+<NUM>) can also be adjusted based on previous Itarget(i) with appropriate error target adjustment/correction. If DeltaError <<NUM>, meaning the cell has undergone an undershoot in programing, meaning the cell has not reached the target yet, the target is adjusted by keeping the previous target, meaning Itargetv (i+<NUM>) = Itargetv (i) (step 3512B). Soft erase sector (step <NUM>). Program all cells in sector to intermediate value (step <NUM>), and return to step <NUM>.

A typical neural network may have positive weight w+ and negative weight w- and a combined weight = w+ - w-. w+ and w- are implemented by a memory cell each (Iw+ and Iw-respectively) and the combined weight (Iw = Iw+ - Iw-, a current subtraction) can be performed at the peripheral circuit level (such as by using an array bitline output circuit). Hence, a weight tuning embodiment for the combined weight can comprise tuning both the w+ cell and the w-cell at the same time, tuning the w+ cell only, or tuning the w- cell only, as in the example shown in the Table <NUM>. The tuning operation is performed using the program/verify and error target adjustment methods described previously with respect to <FIG>/<FIG>/<FIG>. The verify operation can be performed for the combined weight only (e.g., measuring/reading the combined weight current but not individual positive w+ cell current or w- cell current), w+ cell current only, or w- cell current only.

For example, for a combined Iw of <NUM> nA, Iw+ can be <NUM> nA and Iw- can be <NUM> nA; or, Iw+ can be <NUM> nA and Iw- can be <NUM> nA, meaning both positive weight Iw+ and negative weight Iw- are not zero (e.g., where a zero would signify a deeply programmed cell). This may be preferable in certain operating conditions, as it would cause both Iw+ and Iw- to be less susceptible to noise.

Thus, differential weight mapping according to the formula w = (w+) - (w-) can be used to store a tuning value w for use in a neural network. The mapping of w+ and w- can be optimized to combat particular problems that arise in VMM arrays in a neural network, for example, by including an offset value in w+ and w- at the time each value is stored.

The weight mapping is optimized to reduce RTN noise. For example, in the instance where the desired value of w is <NUM> nA (zero w or not used memory cells), one possible mapping is (w+)= <NUM> nA and (w-)=<NUM> nA, and another possible mapping is (w+)=<NUM> nA and (w-) = <NUM> nA, where <NUM> nA here is an example of a non-zero offset value that is added to each w+ and w- value prior to storage. Including such a non-zero offset value will ultimately consume more power and incur greater inaccuracy during the tuning process, but it will minimize the impact of any noise. Similarly, in the instance where the desired value of w is <NUM> nA, one possible mapping is (w+) = <NUM> nA and (w-) = <NUM> nA, and another possible mapping is (w+)= <NUM> nA and (w-)=<NUM> nA. The latter will consume more power and may incur greater inaccuracy during the tuning process but will minimize the impact of any noise.

In another embodiment, for a zero weight w, both w+ and w- can be tuned to be approximately zero, such as by using a method of bias offset voltage for verify operation, which will now be described. casein this method, both w+ and w- are tuned to, for example, 5nA at a control gate voltage (VCG) that is higher than a normal CG voltage used for inference (reading). For example, for dVCG/Ir = 2mV/1nA, if a value of VCG = <NUM>. 5V is used in inference, then in the tuning algorithm (program verify algorithm for weight tuning), a value of VCG = <NUM>. 510V will be used to verify the zero weight cell reaching a target of <NUM> nA. In the inference operation, because VCG = <NUM>. 5V is used, the cell current is shifted down 2mV per <NUM> nA, hence <NUM> nA in a verify operation becomes ~ 0nA in an inference operation.

In another embodiment, for a zero weight w, both w+ and w- can be tuned to be having a negative current, such as by using a negative current tuning method of bias offset voltage for verify operation, which will now be described. For this case, both w+ and w- are tuned to, for example, -10nA at a control gate voltage (VCG) that is higher than a normal CG voltage used for inference (reading). For example, for dVCG/Ir = 2mV/1nA, if a value of VCG = <NUM>. 5V is used in inference, then in the tuning algorithm (program verify algorithm for weight tuning), VCG = <NUM>. 530V is used to verify the zero weight cell reaching target of 5nA. In the inference operation, because VCG = <NUM>. 5V is used, the cell current is shifted down 2mV per <NUM> nA, hence 5nA in a verify operation becomes ~ - 10nA in an inference operation.

A similar method of an offset bias condition or bias (voltage and/or current and/or timing and/or temperature) sequence can be used for the purpose of test screening to detect abnormal bits such as cells that are susceptible to significant noise (such as RTN noise, thermal noise, or any other noise sources). Basically, there is a bias condition or a bias sequence that can be used to detect the noise better by attenuating the noise to a greater degree (noise attenuation test) from the memory cells than other bias conditions or bias sequences. For example, for a <NUM> nA memory cell, it might be advantageous to detect undesired behavior from placing this cell into another condition by modulating the bias condition for this cell, for example such by changing the control gate bias voltage in test screening. For example, due to a tester limitation or a circuit limitation, it can be advantageous to detect the bits/cells susceptible to noises (such as RTN noise) at a higher current level by modulating the bias condition.

Another method of screening or verifying noise levels, such as RTN noise screening, for the memory cell is sampling the memory cell outputs (such as by measuring the output numerous times, such as <NUM>/<NUM>/. /<NUM> times). The screening criteria is such that the value of any sample instance is greater the average of the samples by a certain amount. Another screening criteria is that the value of one sample is greater the next sample by a certain amount. These techniques were described by applicant in <CIT>, and titled, "Precise Programming Method and Apparatus for Analog Neural Memory in a Deep Learning Artificial Neural Network,".

A method of tuning the weight of a memory cell (program or erase the cell) which incorporate some of the above weight assignments, can include soft erasing the cell, then programming the zero weight cells (such as above), then performing coarse and fine and/or ultra fine tuning algorithm with noise screening, such as described above. Techniques for coarse, fine, and ultra fine tuning algorithms were previously disclosed by application in <CIT>, and titled, "Precise Data Tuning Method and Apparatus for Analog Neural Memory in an Artificial Neural Network,".

In another embodiment, noise contribution can be reduced by using a method of read (inference) or verify that comprises bias condition sequencing. For example, an accumulation condition is performed on memory cells before a read or verify operation is performed.

In another embodiment, noise contribution can be reduced by applying a negative voltage range on the control gate. In another embodiment, background data for the array for the zero weight and not used cells can be a particular pattern to reduce variation. For example, a high current level background may be desirable for noise such as RTN noise reduction. For example, a low current level background may be desirable for noise such as data drift. In the standby or deep power down, the array is put in the right condition by modulating the control gate voltage, such as by using a particular voltage level as compared to the control gate voltage used during a verify operation, meaning control gate levels can be set to lower the current or raise the current level during a standby or deep power down operation.

In another embodiment, a method of read or verify is performed by applying <NUM> V, about <NUM> V, or a low bias voltage on the control gate during a read or verify operation. Wordlines are used instead of control gate lines to receive the row data inputs (activation values), such as through pulse width modulated inputs or analog voltages applied to the wordlines.

In another embodiment, a "<NUM>" value for w (zero w or not used cells) can be defined as < <NUM> nA or another pre-determined threshold. That is, if (w+) - (w-) < <NUM> nA, then w is given a value of "<NUM>. " This provides greater tolerance each time w = <NUM> and will be more robust against inaccuracies caused by noise, temperature variation, or other forces.

In another embodiment, the weight mapping is optimized to reduce temperature variation. For example, in the instance where the desired value of w is <NUM> nA, one possible mapping is (w+)= <NUM> nA and (w-)=<NUM> nA, and another possible mapping is (w+)=<NUM> nA and (w-) = <NUM> nA. The latter will consume more power and may incur greater inaccuracy during the tuning process but will minimize temperature variation.

In another embodiment, the weight mapping is optimized to reduce the total noise or temperature variation for a neuron. For example, the number of stored w+ and w- values (such as for a 5nA can be implemented as <NUM> nA - 25nA or 50nA - 45nA or 80nA - 75nA) per bit line can be mapped to be balanced among all bit lines so that the number of stored values per bitline is approximately the same for all bit lines.

In another embodiment, the weight mapping is optimized to reduce the total noise for a neuron (bitline). For example, the number of stored w+ and w- values (cells) per bit line can be balanced among all bit lines so that the total noise contribution of all the weights (cells) in a neuron (bitline) is optimal (having least noise).

Tables 9A and 9B show an exemplary embodiment for <NUM> levels (states) in nA. Meaning a memory cell can have <NUM> levels as shown. Table 9A depicts a situation where w can be one of <NUM> different positive values, and Table 9B depicts a situation where w can be one of <NUM> different negative values, according to the formula Iw = Iw+ - Iw-. The current range as shown is from <NUM>-80nA.

Tables 10A and 10B show an embodiment that compresses the dynamic total current range of the levels from <NUM>-80nA to 40nA to 85nA.

For Tables 10A/11A/12A , the Iw- can be tuned with a coarse or fine or ultra fine tuning step (e.g. program and verify Iw- cell ) and Iw+ can be tuned with a fine or ultra fine step (e.g., program and verify Iw = (Iw+ - Iw-), or just by verify Iw+ cell only). For the Table 10B/11B/12B, the Iw+ can be tuned with a coarse or fine or ultra fine tuning step (e.g. program and verify Iw+ cell ) and Iw- can be tuned with a fine or ultra fine step (e.g., program and verify Iw = (Iw+ - Iw-), or just by verify Iw- cell only).

This is advantageous in terms of reducing variation and mismatch due to process, temperature, noise, operating stress, or operating conditions, which is similar to the concepts shown in <FIG>.

As shown in Tables 10A and 10B, values in the lower half of the table are shifted up by a positive amount (offset bias) so that the total range of those values is approximately the same as the top half of the table. The offset is approximately equal to half of the maximum current (level).

Tables 11A and 11B show an embodiment that is similar to that of Figures 10A and 10B with a zero weight (w = <NUM>) equal to an offset bias value. As example shows one range of weight values for one offset bias value, two sub-ranges of weights for two offset values.

Table <NUM> shows an embodiment that uses a varying offset for positive weight and negative weight.

Table <NUM> shows an embodiment that uses a common offset for positive weight and negative weight.

Table <NUM> shows an embodiment that uses a increasing offset for weight.

Table <NUM> shows an embodiment that uses a decreasing offset for weight. It also shows a constant offset for Iw+. It also shows a maximum constant value for Iw+, basically shift all the weights toward the maximum value.

Tables 12A and 12B show an embodiment that is similar to that of Figure 10A and 10B, with each w+ or w+ value implemented by two memory cells to reduce the total dynamic range further approximately by another half.

It is to understood that the values for w+ and w- provided in the embodiments above are mere examples and that other values can be used in accordance with the disclosed concepts. For example, the offset bias value can be any value to shift the value for each level or it can be a fixed value for all levels. In effect, each w is implemented as a differential cells, which can be effective in minimizing variation or mismatch from process, temperature, noise (such as RTN or supply noise), stress, or operating condition.

<FIG> illustrates the data behavior (I-V curve) over temperature (in subthreshold region as example), <FIG> illustrates problems created by data drift during operation of a VMM system, and <FIG> depict blocks for compensating for data drift and, as to <FIG>, for compensating for temperature changes.

<FIG> depicts the known characteristic of a VMM system, which is that as operating temperature increases, the sensed current in any given selected non-volatile memory cell in the VMM array increases in the sub-threshold region, decreases in the saturation region, or generally decreases in the linear region.

<FIG> shows array current distribution over time usage (data drift), and it shows that the collective output from a VMM array (which is the sum of the current from all bit lines in the VMM array) shifts to the right (or left, depending on the technology utilized) over operating time usage, meaning that the total collective output will drift over lifetime usage of the VMM system. This phenomenon is known as data drift, as the data will drift due to a usage condition and degradation due to an environment factor.

<FIG> depicts bitline compensation circuit <NUM>, which may include injecting a compensation current, iCOMP, to the output of bitline output circuit <NUM> to compensate for data drift. The bitline compensation circuit <NUM> may include scaling up or down the output by a scaler circuit based on a resistor or capacitor network. The bitline compensation circuit <NUM> may include shifting or offsetting the output by a shifter circuit based on its resistor or capacitor network.

<FIG> depicts a data drift monitor <NUM>, which detects the amount of data drift. That information is then used as an input to bitline compensation circuit <NUM> so that the appropriate level of iCOMP can be selected.

<FIG> depicts bitline compensation circuit <NUM>, which is an embodiment of bitline compensation circuit <NUM> in <FIG>. Bitline compensation circuit <NUM> comprises adjustable current source <NUM> and adjustable current source <NUM>, which together generate iCOMP, where iCOMP is equal to the current generated by adjustable current source <NUM> minus the current generated by adjustable current source <NUM>.

<FIG> depicts bitline compensation circuit <NUM>, which is an embodiment of bitline compensation circuit <NUM> in <FIG>. Bitline compensation circuit <NUM> comprises operational amplifier <NUM>, adjustable resistor <NUM>, and adjustable resistor <NUM>. Operational amplifier <NUM> receives a reference voltage, VREF, on its non-inverting terminal and VINPUT on its inverting terminal, where VINPUT is the voltage received from bitline output circuit <NUM> in <FIG>, and generates an output of VOUTPUT, where VOUTPUT is a scaled version of VINPUT to compensate for data drift basing the ratio of the resistor <NUM> and <NUM>. By configuring the value of the resistor <NUM> and/or <NUM>, VOUTPUT can be scaled up or down.

<FIG> depicts bitline compensation circuit <NUM>, which is an embodiment of bitline compensation circuit <NUM> in <FIG>. Bitline compensation circuit <NUM> comprises operational amplifier <NUM>, current source <NUM>, switch <NUM>, and adjustable integrating output capacitor <NUM>. Here, current source <NUM> actually is the output current on a single bitline or the collection of multiple of bitlines (such as one for summing positive weights, w+, and one for summing negative weights, w-) in the VMM array. Operational amplifier <NUM> receives a reference voltage, VREF, on its non-inverting terminal and VINPUT on its inverting terminal, where VINPUT is the voltage received from bitline output circuit <NUM> in <FIG>. Bitline compensation circuit <NUM> acts as an integrator, which integrates the current Ineu across the capacitor <NUM> in an adjustable integration time to generate an output voltage VOUTPUT, where VOUTPUT = Ineu*integration time/ C<NUM>, where C<NUM> is value of the capacitor <NUM>. Hence, the output voltage VOUTPUT is proportional to the (bitline) output current Ineu, proportional to the integration time. and inversely proportional to the capacitance of capacitor <NUM>. Bitline compensation circuit <NUM> generates an output of VOUTPUT, where the value of the VOUTPUT is scaled based on the configured value of the capacitor <NUM> and/or the integration time to compensate for data drift.

<FIG> depicts bitline compensation circuit <NUM>, which is an embodiment of bitline compensation circuit <NUM> in <FIG>. Bitline compensation circuit <NUM> comprises current mirror <NUM> with an M:N ratio, meaning the ICOMP = (M/N) * iinput. Current mirror <NUM> receives current iINPUT and mirrors that current and optionally scales that current to generate iCOMP. Hence, by configuring the M and/or N parameters, iCOMP can be scaled upward or downward.

<FIG> depicts bitline compensation circuit <NUM>, which is an embodiment of bitline compensation circuit <NUM> in <FIG>. Bitline compensation circuit <NUM> comprises operational amplifier <NUM>, adjustable scaling resistor <NUM>, adjustable shifting resistor <NUM>, and adjustable resistor <NUM>. Operational amplifier <NUM> receives a reference voltage, VREF, on its non-inverting terminal and VIN on its inverting terminal. VIN is generated in response to VINPUT and Vshft, where VINPUT is the voltage received from bitline output circuit <NUM> in <FIG> and Vshft is a voltage intended to implement a shift between VINPUT and VOUTPUT. Thus, VOUTPUT is a scaled and shifted version of VINPUT to compensate for data drift.

<FIG> depicts bitline compensation circuit <NUM>, which is an embodiment of bitline compensation circuit <NUM> in <FIG>. Bitline compensation circuit <NUM> comprises operational amplifier <NUM>, input current source Ineu <NUM>, current shifter <NUM>, switches <NUM> and <NUM> and adjustable integrating output capacitor <NUM>. Here, current source <NUM> actually is the output current, Ineu, on a single bitline or the multiple bitlines in the VMM array. Operational amplifier <NUM> receives a reference voltage, VREF, on its non-inverting terminal and IIN on its inverting terminal, where IIN is the sum of Ineu and a current output by current shifter <NUM>, and generates an output of VOUTPUT, where VOUTPUT is scaled (basing on the capacitor <NUM>) and shifted (basing on Ishifter <NUM>) to compensate for data drift.

<FIG> depict various circuits that can be used to provide the W value to be programmed or read into each selected cell during a programming or reading operation.

<FIG> depicts neuron output circuit <NUM>, which comprises adjustable current source <NUM> and adjustable current source <NUM>, which together generate IOUT, where IOUT is equal to the current generated by adjustable current source <NUM>, IW+, minus the current generated by adjustable current source <NUM>, IW-. The adjustable current Iw+ <NUM> is a scaled current of the cell current or neuron current (such as bitline current) to implement positive weight. The adjustable current Iw- <NUM> is a scaled current of the cell current or neuron current (such as bitline current) to implement negative weight. The current scaling is done such as by a M:N ratio current mirror circuit, Iout = (M/N)* Iin.

<FIG> depicts neuron output circuit <NUM>, which comprises adjustable capacitor <NUM>, control transistor <NUM>, switch <NUM>, switch <NUM>, and adjustable current source <NUM> Iw+, which is a scaled output current of the cell current or (bitline) neuron current such as by a M:N current mirror circuit. The transistor <NUM> is used for example to impose a fixed bias voltage on the current <NUM>. The circuit <NUM> generates VOUT, where VOUT is inversely proportional to the capacitor <NUM>, proportional to an adjustable integration time (time switch <NUM> closed and the switch <NUM> opened) and proportional to the current generated by adjustable current source <NUM>, IW+. VOUT is equal to V+ - ((Iw+*integration time )/ C<NUM>), where C<NUM> is value of the capacitor <NUM>. The positive terminal, V+, of the capacitor <NUM> is connected to a positive supply voltage and the negative terminal, V-, of the capacitor <NUM> is connected to the output voltage VOUT.

<FIG> depicts neuron circuit <NUM>, which comprises capacitor <NUM> and adjustable current source <NUM>, which is a scaled current of cell current or (bitline) neuron current such as by a M:N current mirror. The circuit <NUM> generates VOUT, where VOUT is inversely proportional to the capacitor <NUM>, proportional to an adjustable integration time (time the switch <NUM> opened ) and proportional to the current generated by adjustable current source <NUM>, IWi. The capacitor <NUM> is re-used from the neuron output circuit <NUM> after it completes its operation of integrating the current Iw+. Then the positive and negative terminals (V+ and V-) are exchanged in the neuron output circuit <NUM>, in which the positive terminal is connected to the output voltage VOUT, which is de-integrated by the current Iw-. The negative terminal is held at the previous voltage value by a clamp circuit (not shown). In effect, the output circuit <NUM> is used for positive weight implementation and the circuit <NUM> is used for negative weight implementation with the final charge on the capacitor <NUM> representing the combined weight (Qw = Qw+ - Qw-) effectively).

<FIG> depicts neuron circuit <NUM>, which comprises adjustable capacitor <NUM>, switch <NUM>, control transistor <NUM>, and adjustable current source <NUM>. The circuit <NUM> generates VOUT, where VOUT is inversely proportional to the capacitor <NUM>, proportional to an adjustable integration time (time the switch <NUM> opened), and proportional to the current generated by adjustable current source <NUM>, IW-. The negative terminal V- of the capacitor <NUM> is, for example, equal to ground. The positive terminal V+ of the capacitor <NUM> is, for example, initially pre-charged to a positive voltage before integrating the current Iw-. The neuron circuit <NUM> can be used in place of the neuron circuit <NUM> together with the neuron circuit <NUM> to implement the combined weight (Qw = Qw+ - Qw-).

<FIG> depicts neuron circuit <NUM>, which comprises operational amplifiers <NUM> and <NUM>; adjustable current sources Iw+ <NUM> and Iw- <NUM>; and adjustable resistors <NUM>, <NUM>, and <NUM>. Neuron circuit <NUM> generates VOUT, which is equal to R<NUM> * (Iw+- Iw-). The adjustable resistor <NUM> implements the scaling of the output. The adjustable current sources Iw+ <NUM> and Iw- <NUM> also implement the scaling of the output such as by a M:N ratio current mirror circuit (Iout = (M/N)* Iin).

<FIG> depicts neuron circuit <NUM>, which comprises operational amplifiers <NUM> and <NUM>; switches <NUM> and <NUM>; adjustable current sources Iw- <NUM> and Iw+ <NUM>; adjustable capacitors <NUM>, <NUM>, and <NUM>. Neuron circuit <NUM> generates VOUT, which is proportional to (Iw+ - Iw-), proportional to an integration time (time switches <NUM> and <NUM> opened), and inversely proportional to the capacitance of capacitor <NUM>. The adjustable capacitor <NUM> implements the scaling of the output. The adjustable current source Iw+ <NUM> and Iw- <NUM> also implement the scaling of the output such as by a M:N ratio current mirror circuit (Iout = (M/N)* Iin). The integration time can also adjust the output scaling.

<FIG> depict block diagrams of an output circuit such as output circuit <NUM> in <FIG>.

In <FIG>, output circuit <NUM> comprises ADC circuit <NUM>, which is used to digitize analog neuron output <NUM> directly to provide digital output bits <NUM>.

In <FIG>, output circuit <NUM> comprises neuron output circuit <NUM> and ADC <NUM>. Neuron output circuit <NUM> receives neuron output <NUM> and shapes it before being digitized by the ADC circuit <NUM> to generate outputs <NUM>. Neuron Output circuit <NUM> can be used for normalization, scaling, shifting, mapping, arithmetic operations, activation, and/or temperature compensation such as described previously. ADC circuit can be serial (sloped or ramp or counting) ADC, SAR ADC, piped line ADC, Sigma Delta ADC, or any type of ADC.

In <FIG>, output circuit comprises neuron output circuit <NUM>, which receives neuron output <NUM>, and converter circuit <NUM> is for converting output from neuron output circuit <NUM> into output <NUM>. Converter <NUM> can comprise an ADC, AAC (analog to analog converter, such as current to voltage converter), APC (analog to pulses converter), or any other type of converter. ADC <NUM> or converter <NUM> can be used to implement an activation function by for example bit mapping (e.g., quantization) or clipping (e.g., clipped ReLU). ADC <NUM> and converter <NUM> can be configurable such as for lower or higher precision (e.g., lower or higher number of bits), lower or higher performance (e.g., slower or faster speed), etc..

Another embodiment for scaling and shifting is by configuring ADC (Analog-to-Digital) conversion circuits (such as serial ADC, SAR ADC, piped-line ADC, slope ADC, etc.) that are used to convert the array (bitline) output to digital bits such as having less or more bit precision and then manipulating the digital output bits, such as through normalization (e.g., <NUM>-bit to <NUM>-bit), shifting, or re-mapping according to a certain function (e.g., linear or non-linear, compression, non-linear activations, etc.). Examples of ADC conversion circuits are described in <CIT>, and titled, "Precise Programming Method and Apparatus for Analog Neural Memory in a Deep Learning Artificial Neural Network," which is incorporated by reference herein.

Table No. <NUM> depicts an alternative approach to performing read, erase, and program operations:.

The read and erase operation are similar to previous tables. The two methods for programming are however implemented by Fowler-Nordheim (FN) tunneling mechanism.

An embodiment for scaling on the input can be done such as by enabling a certain number of rows of the VMM at a time, then combines the results altogether.

Another embodiment is scaling the input voltage, and appropriately re-scaling the output for normalization.

Another embodiment for scaling pulsewidth modulation input is by modulating timing of the pulsewidth. An example of this technique is described in <CIT>, and titled, "Configurable Input Blocks and Output Blocks and Physical Layout for Analog Neural Memory in Deep Learning Artificial Neural Network,".

Another embodiment for scaling the input is by enabling an input binary bit one at a time, for example, for <NUM>-bit input IN7:<NUM>, evaluate IN0,IN1,. ,IN7 respectively in sequential order, then combine the output results together with appropriate binary bit weighting. An example of this technique is described in <CIT>, and titled, "Configurable Input Blocks and Output Blocks and Physical Layout for Analog Neural Memory in Deep Learning Artificial Neural Network,".

Optionally, in the embodiments described above, measuring cell current for the purpose of verifying or reading the current can be taking the average or multiple measurements, e.g., <NUM>-<NUM> times, to reduce the impact of noise (such as RTN or any random noise) and/or to detect any outlier bits that are defective and need to be replaced by a redundant bit.

Claim 1:
An analog neuromorphic memory system comprising:
a vector-by-matrix multiplication array of non-volatile memory cells, wherein a weight value w is stored as a differential pair w+ and w- in a first non-volatile memory cell and a second non-volatile memory cell in the array according to the formula w = (w+) - (w-), where w+ and w- each include a non-zero offset value stored in the first non-volatile memory cell and the second non-volatile memory cell respectively, wherein for w=<NUM>, w+ is the non-zero offset value and w- is the non-zero offset value.