Patent Description:
Numerous embodiments of input circuitry for an analog neural memory in a deep learning artificial neural network are disclosed.

<FIG> illustrates an artificial neural network, where the circles represent the inputs or layers of neurons. The connections (called synapses) are represented by arrows and have numeric weights that can be tuned based on experience. This makes neural networks adaptive to inputs and capable of learning. Typically, neural networks include a layer of multiple inputs. There are typically one or more intermediate layers of neurons, and an output layer of neurons that provide the output of the neural network. The neurons at each level individually or collectively make a decision based on the received data from the synapses.

One of the major challenges in the development of artificial neural networks for highperformance information processing is a lack of adequate hardware technology. Indeed, practical neural networks rely on a very large number of synapses, enabling high connectivity between neurons, i.e., a very high computational parallelism. In principle, such complexity can be achieved with digital supercomputers or specialized graphics processing unit clusters. However, in addition to high cost, these approaches also suffer from mediocre energy efficiency as compared to biological networks, which consume much less energy primarily because they perform low-precision analog computation. CMOS analog circuits have been used for artificial neural networks, but most CMOS-implemented synapses have been too bulky given the high number of neurons and synapses. Patent publication <CIT> discloses a CMOS-implemented neural network.

Applicant previously disclosed an artificial (analog) neural network that utilizes one or more non-volatile memory arrays as the synapses in <CIT>. The non-volatile memory arrays operate as an analog neural memory. The neural network device includes a first plurality of synapses configured to receive a first plurality of inputs and to generate therefrom a first plurality of outputs, and a first plurality of neurons configured to receive the first plurality of outputs. The first plurality of synapses includes a plurality of memory cells, wherein each of the memory cells includes spaced apart source and drain regions formed in a semiconductor substrate with a channel region extending there between, a floating gate disposed over and insulated from a first portion of the channel region and a non-floating gate disposed over and insulated from a second portion of the channel region. Each of the plurality of memory cells is configured to store a weight value corresponding to a number of electrons on the floating gate. The plurality of memory cells is configured to multiply the first plurality of inputs by the stored weight values to generate the first plurality of outputs.

Non-volatile memories are well known. For example, <CIT> ("the '<NUM> patent") discloses an array of split gate non-volatile memory cells, which are a type of flash memory cells. Such a memory cell <NUM> is shown in <FIG>. Each memory cell <NUM> includes source region <NUM> and drain region <NUM> formed in semiconductor substrate <NUM>, with channel region <NUM> there between. Floating gate <NUM> is formed over and insulated from (and controls the conductivity of) a first portion of the channel region <NUM>, and over a portion of the source region <NUM>. Word line terminal <NUM> (which is typically coupled to a word line) has a first portion that is disposed over and insulated from (and controls the conductivity of) a second portion of the channel region <NUM>, and a second portion that extends up and over the floating gate <NUM>. The floating gate <NUM> and word line terminal <NUM> are insulated from the substrate <NUM> by a gate oxide. Bitline <NUM> is coupled to drain region <NUM>.

Memory cell <NUM> is erased (where electrons are removed from the floating gate) by placing a high positive voltage on the word line terminal <NUM>, which causes electrons on the floating gate <NUM> to tunnel through the intermediate insulation from the floating gate <NUM> to the word line terminal <NUM> via Fowler-Nordheim (FN) tunneling.

Memory cell <NUM> is programmed by source side injection (SSI) with hot electrons (where electrons are placed on the floating gate) by placing a positive voltage on the word line terminal <NUM>, and a positive voltage on the source region <NUM>. Electron current will flow from the drain region <NUM> towards the source region <NUM>. The electrons will accelerate and become heated when they reach the gap between the word line terminal <NUM> and the floating gate <NUM>. Some of the heated electrons will be injected through the gate oxide onto the floating gate <NUM> due to the attractive electrostatic force from the floating gate <NUM>.

If the floating gate <NUM> is positively charged (i.e., erased of electrons), then the portion of the channel region <NUM> under the floating gate <NUM> is turned on as well, and current will flow across the channel region <NUM>, which is sensed as the erased or "<NUM>" state. If the floating gate <NUM> is negatively charged (i.e., programmed with electrons), then the portion of the channel region under the floating gate <NUM> is mostly or entirely turned off, and current will not flow (or there will be little flow) across the channel region <NUM>, which is sensed as the programmed or "<NUM>" state.

Table No. <NUM> depicts typical voltage and current ranges that can be applied to the terminals of memory cell <NUM> for performing read, erase, and program operations:.

Other split gate memory cell configurations, which are other types of flash memory cells, are known. For example, <FIG> depicts a four-gate memory cell <NUM> comprising source region <NUM>, drain region <NUM>, floating gate <NUM> over a first portion of channel region <NUM>, a select gate <NUM> (typically coupled to a word line, WL) over a second portion of the channel region <NUM>, a control gate <NUM> over the floating gate <NUM>, and an erase gate <NUM> over the source region <NUM>.

<FIG> depicts a three-gate memory cell <NUM>, which is another type of flash memory cell. Memory cell <NUM> is identical to the memory cell <NUM> of <FIG> except that memory cell <NUM> does not have a separate control gate. The erase operation (whereby erasing occurs through use of the erase gate) and read operation are similar to that of the <FIG> except there is no control gate bias applied. The programming operation also is done without the control gate bias, and as a result, a higher voltage must be applied on the source line during a program operation to compensate for a lack of control gate bias.

<FIG> depicts stacked gate memory cell <NUM>, which is another type of flash memory cell. Memory cell <NUM> is similar to memory cell <NUM> of <FIG>, except that floating gate <NUM> extends over the entire channel region <NUM>, and control gate <NUM> (which here will be coupled to a word line) extends over floating gate <NUM>, separated by an insulating layer (not shown). The erase is done by FN tunneling of electrons from FG to substrate, programming is by channel hot electron (CHE) injection at region between the channel <NUM> and the drain region <NUM>, by the electrons flowing from the source region <NUM> towards to drain region <NUM> and read operation which is similar to that for memory cell <NUM> with a higher control gate voltage.

The methods and means described herein may apply to other non-volatile memory technologies such as FINFET split gate flash or stack gate flash memory, NAND flash, SONOS (silicon-oxide-nitride-oxide-silicon, charge trap in nitride), MONOS (metal-oxide-nitride-oxide-silicon, metal charge trap in nitride), ReRAM (resistive ram), PCM (phase change memory), MRAM (magnetic ram), FeRAM (ferroelectric ram), CT (charge trap) memory, CN (carbontube) memory, OTP (bi-level or multi-level one time programmable), and CeRAM (correlated electron ram), without limitation.

Specifically, the memory state (i.e., charge on the floating gate) of each memory cell in the array can be continuously changed from a fully erased state to a fully programmed state, independently and with minimal disturbance of other memory cells.

S0 is the input layer, which for this example is a 32x32 pixel RGB image with <NUM> bit precision (i.e. three 32x32 pixel arrays, one for each color R, G and B, each pixel being <NUM> bit precision). The synapses CB1 going from input layer S0 to layer C1 apply different sets of weights in some instances and shared weights in other instances and scan the input image with 3x3 pixel overlapping filters (kernel), shifting the filter by <NUM> pixel (or more than <NUM> pixel as dictated by the model). Specifically, values for <NUM> pixels in a 3x3 portion of the image (i.e., referred to as a filter or kernel) are provided to the synapses CB1, where these <NUM> input values are multiplied by the appropriate weights and, after summing the outputs of that multiplication, a single output value is determined and provided by a first synapse of CB1 for generating a pixel of one of the feature maps of layer C1. The 3x3 filter is then shifted one pixel to the right within input layer S0 (i.e., adding the column of three pixels on the right, and dropping the column of three pixels on the left), whereby the <NUM> pixel values in this newly positioned filter are provided to the synapses CB1, where they are multiplied by the same weights and a second single output value is determined by the associated synapse. This process is continued until the 3x3 filter scans across the entire 32x32 pixel image of input layer S0, for all three colors and for all bits (precision values). The process is then repeated using different sets of weights to generate a different feature map of layer C1, until all the features maps of layer C1 have been calculated.

The purpose of the pooling function P1 is to average out the nearby location (or a max function can also be used), to reduce the dependence of the edge location for example and to reduce the data size before going to the next stage. The synapses CB2 going from layer S1 to layer C2 scan maps in layer S1 with 4x4 filters, with a filter shift of <NUM> pixel.

<FIG> is a block diagram of an array that can be used for that purpose. Vector-by-matrix multiplication (VMM) array <NUM> includes non-volatile memory cells and is utilized as the synapses (such as CB1, CB2, CB3, and CB4 in <FIG>) between one layer and the next layer. Specifically, VMM array <NUM> includes an array of non-volatile memory cells <NUM>, erase gate and word line gate decoder <NUM>, control gate decoder <NUM>, bit line decoder <NUM> and source line decoder <NUM>, which decode the respective inputs for the non-volatile memory cell array <NUM>. Input to VMM array <NUM> can be from the erase gate and wordline gate decoder <NUM> or from the control gate decoder <NUM>. Source line decoder <NUM> in this example also decodes the output of the non-volatile memory cell array <NUM>. Alternatively, bit line decoder <NUM> can decode the output of the non-volatile memory cell array <NUM>.

Non-volatile memory cell array <NUM> serves two purposes. Second, the non-volatile memory cell array <NUM> effectively multiplies the inputs by the weights stored in the non-volatile memory cell array <NUM> and adds them up per output line (source line or bit line) to produce the output, which will be the input to the next layer or input to the final layer. By performing the multiplication and addition function, the non-volatile memory cell array <NUM> negates the need for separate multiplication and addition logic circuits and is also power efficient due to its in-situ memory computation.

The output of non-volatile memory cell array <NUM> is supplied to a differential summer (such as a summing op-amp or a summing current mirror) <NUM>, which sums up the outputs of the non-volatile memory cell array <NUM> to create a single value for that convolution. The differential summer <NUM> is arranged to perform summation of positive weight and negative weight.

The summed-up output values of differential summer <NUM> are then supplied to an activation function block <NUM>, which rectifies the output. The activation function block <NUM> may provide sigmoid, tanh, or ReLU functions. The rectified output values of activation function block <NUM> become an element of a feature map as the next layer (e.g. C1 in <FIG>), and are then applied to the next synapse to produce the next feature map layer or final layer. Therefore, in this example, non-volatile memory cell array <NUM> constitutes a plurality of synapses (which receive their inputs from the prior layer of neurons or from an input layer such as an image database), and summing op-amp <NUM> and activation function block <NUM> constitute a plurality of neurons.

The input to VMM array <NUM> in <FIG> (WLx, EGx, CGx, and optionally BLx and SLx) can be analog level, binary level, or digital bits (in which case a DAC is provided to convert digital bits to appropriate input analog level) and the output can be analog level, binary level, or digital bits (in which case an output ADC is provided to convert output analog level into digital bits).

<FIG> is a block diagram depicting the usage of numerous layers of VMM arrays <NUM>, here labeled as VMM arrays 32a, 32b, 32c, 32d, and 32e. As shown in <FIG>, the input, denoted Inputx, is converted from digital to analog by a digital-to-analog converter <NUM> and provided to input VMM array 32a. The input D/A conversion for the first layer could be done by using a function or a LUT (look up table) that maps the inputs Inputx to appropriate analog levels for the matrix multiplier of input VMM array 32a. The input conversion could also be done by an analog to analog (A/A) converter to convert an external analog input to a mapped analog input to the input VMM array 32a.

The output generated by input VMM array 32a is provided as an input to the next VMM array (hidden level <NUM>) 32b, which in turn generates an output that is provided as an input to the next VMM array (hidden level <NUM>) 32c, and so on. The various layers of VMM array <NUM> function as different layers of synapses and neurons of a convolutional neural network (CNN). Each VMM array 32a, 32b, 32c, 32d, and 32e can be a stand-alone, physical non-volatile memory array, or multiple VMM arrays could utilize different portions of the same physical non-volatile memory array, or multiple VMM arrays could utilize overlapping portions of the same physical non-volatile memory array. The example shown in <FIG> contains five layers (32a,32b,32c,32d,32e): one input layer (32a), two hidden layers (32b,32c), and two fully connected layers (32d,32e). One of ordinary skill in the art will appreciate that this is merely exemplary and that a system instead could comprise more than two hidden layers and more than two fully connected layers.

<FIG> depicts neuron VMM array <NUM>, which is particularly suited for memory cells <NUM> as shown in <FIG> and is utilized as the synapses and parts of neurons between an input layer and the next layer.

As described herein for neural networks, the non-volatile memory cells of VMM array <NUM>, i.e., the memory cells <NUM> of VMM array <NUM>, are preferably configured to operate in a sub-threshold region.

The non-volatile reference memory cells and the non-volatile memory cells described herein are biased in weak inversion (sub threshold region): <MAT> where <MAT> where Ids is the drain to source current; Vg is gate voltage on the memory cell; Vth is threshold voltage of the memory cell; Vt is thermal voltage = k*T/q with k being the Boltzmann constant, T the temperature in Kelvin, and q the electronic charge; n is a slope factor = <NUM> + (Cdep/Cox) with Cdep = capacitance of the depletion layer, and Cox capacitance of the gate oxide layer; Io is the memory cell current at gate voltage equal to threshold voltage, Io is proportional to (Wt/L)*u*Cox* (n-<NUM>) * Vt<NUM> where u is carrier mobility and Wt and L are width and length, respectively, of the memory cell.

For an I-to-V log converter using a memory cell (such as a reference memory cell or a peripheral memory cell) or a transistor to convert input current into an input voltage: <MAT> where, wp is w of a reference or peripheral memory cell.

For a memory array used as a vector matrix multiplier VMM array with the current input, the output current is: <MAT> namely <MAT> <MAT> Here, wa = w of each memory cell in the memory array. Vthp is effective threshold voltage of the peripheral memory cell and Vtha is effective threshold voltage of the main (data) memory cell. Note that the threshold voltage of a transistor is a function of substrate body bias voltage and the substrate body bias voltage, denoted Vsb, can be modulated to compensate for various conditions, on such temperature. The threshold voltage Vth can be expressed as: <MAT>.

Where Vth0 is threshold voltage with zero substrate bias, φF is a surface potential, and gamma is a body effect parameter.

Alternatively, the flash memory cells of VMM arrays described herein can be configured to operate in the linear region: <MAT> <MAT> meaning weight W in the linear region is proportional to (Vgs-Vth).

For an I-to-V linear converter, a memory cell (such as a reference memory cell or a peripheral memory cell) or a transistor operating in the linear region can be used to linearly convert an input/output current into an input/output voltage.

Alternatively, the memory cells of VMM arrays described herein can be configured to operate in the saturation region: <MAT> Wα (Vgs-Vth)<NUM>, meaning weight W is proportional to (Vgs-Vth)<NUM>.

Alternatively, the memory cells of VMM arrays described herein can be used in all regions or a combination thereof (sub threshold, linear, or saturation) for each layer or multi layers of a neural network.

Other embodiments for VMM array <NUM> of <FIG> are described in <CIT>. As described in that application. a sourceline or a bitline can be used as the neuron output (current summation output).

<FIG> depicts neuron VMM array <NUM>, which is particularly suited for memory cells <NUM> as shown in <FIG> and is utilized as the synapses between an input layer and the next layer.

Table No. <NUM> depicts operating voltages and currents for VMM array <NUM>.

<FIG> depicts neuron VMM array <NUM>, which is particularly suited for memory cells <NUM> as shown in <FIG> and is utilized as the synapses and parts of neurons between an input layer and the next layer. Here, the inputs are provided on the word lines (WLA0, WLB0, WLA1, WLB2, WLA2, WLB2, WLA3, WLB3), and the output emerges on the source line (SL0, SL1) during a read operation.

<FIG> depicts neuron VMM array <NUM>, which is particularly suited for memory cells <NUM> as shown in <FIG> and is utilized as the synapses and parts of neurons between an input layer and the next layer.

VMM array <NUM> implements uni-directional tuning for non-volatile memory cells in memory array <NUM>. That is, each non-volatile memory cell is erased and then partially programmed until the desired charge on the floating gate is reached. If too much charge is placed on the floating gate (such that the wrong value is stored in the cell), the cell is erased and the sequence of partial programming operations starts over. As shown, two rows sharing the same erase gate (such as EG0 or EG1) are erased together (which is known as a page erase), and thereafter, each cell is partially programmed until the desired charge on the floating gate is reached.

Table No. <NUM> depicts operating voltages and currents for VMM array <NUM>.

<FIG> depicts neuron VMM array <NUM>, which is particularly suited for memory cells <NUM> as shown in <FIG> and is utilized as the synapses and parts of neurons between an input layer and the next layer. In VMM array <NUM>, the inputs INPUT<NUM>. , INPUTN are received on bit lines BL<NUM>,. BLN, respectively, and the outputs OUTPUT<NUM>, OUTPUT<NUM>, OUTPUT<NUM>, and OUTPUT<NUM> are generated on source lines SL<NUM>, SL<NUM>, SL<NUM>, and SL<NUM>, respectively.

<FIG> depicts neuron VMM array <NUM>, which is particularly suited for memory cells <NUM> as shown in <FIG> and is utilized as the synapses and parts of neurons between an input layer and the next layer. In this example, the inputs INPUT<NUM>, INPUT<NUM>, INPUT<NUM>, and INPUT<NUM> are received on source lines SL<NUM>, SL<NUM>, SL<NUM>, and SL<NUM>, respectively, and the outputs OUTPUT<NUM>,. OUTPUTN are generated on bit lines BL<NUM>,.

In this example, the inputs INPUT<NUM>,. , INPUTM are received on word lines WL<NUM>,. , WLM, respectively, and the outputs OUTPUT<NUM>,. OUTPUTN are generated on bit lines BL<NUM>,.

In this example, the inputs INPUT<NUM>,. , INPUTn are received on vertical control gate lines CG<NUM>,. , CGN, respectively, and the outputs OUTPUT<NUM> and OUTPUT<NUM> are generated on source lines SL<NUM> and SL<NUM>.

In this example, the inputs INPUT<NUM>,. , INPUTN are received on the gates of bit line control gates <NUM>-<NUM>, <NUM>-<NUM>,. , <NUM>-(N-<NUM>), and <NUM>-N, respectively, which are coupled to bit lines BL<NUM>,. , BLN, respectively. Exemplary outputs OUTPUT<NUM> and OUTPUT<NUM> are generated on source lines SL<NUM> and SL<NUM>.

<FIG> depicts neuron VMM array <NUM>, which is particularly suited for memory cells <NUM> as shown in <FIG>, memory cells <NUM> as shown in <FIG>, and memory cells <NUM> as shown in <FIG>, and is utilized as the synapses and parts of neurons between an input layer and the next layer. In this example, the inputs INPUT<NUM>,. , INPUTM are received on word lines WL<NUM>,. , WLM, and the outputs OUTPUT<NUM>,. , OUTPUTN are generated on bit lines BL<NUM>,. , BLN, respectively.

<FIG> depicts neuron VMM array <NUM>, which is particularly suited for memory cells <NUM> as shown in <FIG>, memory cells <NUM> as shown in <FIG>, and memory cells <NUM> as shown in <FIG>, and is utilized as the synapses and parts of neurons between an input layer and the next layer. In this example, the inputs INPUT<NUM>,. , INPUTM are received on control gate lines CG<NUM>,. Outputs OUTPUT<NUM>,. , OUTPUTN are generated on vertical source lines SL<NUM>,. , SLN, respectively, where each source line SLi is coupled to the source lines of all memory cells in column i.

<FIG> depicts neuron VMM array <NUM>, which is particularly suited for memory cells <NUM> as shown in <FIG>, memory cells <NUM> as shown in <FIG>, and memory cells <NUM> as shown in <FIG>, and is utilized as the synapses and parts of neurons between an input layer and the next layer. In this example, the inputs INPUT<NUM>,. , INPUTM are received on control gate lines CG<NUM>,. Outputs OUTPUT<NUM>,. , OUTPUTN are generated on vertical bit lines BL<NUM>,. , BLN, respectively, where each bit line BLi is coupled to the bit lines of all memory cells in column i.

The prior art includes a concept known as long short-term memory (LSTM). LSTM units often are used in neural networks. LSTM allows a neural network to remember information over predetermined arbitrary time intervals and to use that information in subsequent operations. A conventional LSTM unit comprises a cell, an input gate, an output gate, and a forget gate. The three gates regulate the flow of information into and out of the cell and the time interval that the information is remembered in the LSTM. VMMs are particularly useful in LSTM units.

<FIG> depicts an exemplary LSTM <NUM>. LSTM <NUM> in this example comprises cells <NUM>, <NUM>, <NUM>, and <NUM>. Cell <NUM> receives input vector x<NUM> and generates output vector h<NUM> and cell state vector c<NUM>. Cell <NUM> receives input vector x<NUM>, the output vector (hidden state) h<NUM> from cell <NUM>, and cell state c<NUM> from cell <NUM> and generates output vector h<NUM> and cell state vector c<NUM>. Cell <NUM> receives input vector x<NUM>, the output vector (hidden state) h<NUM> from cell <NUM>, and cell state c<NUM> from cell <NUM> and generates output vector h<NUM> and cell state vector c<NUM>. Cell <NUM> receives input vector x<NUM>, the output vector (hidden state) h<NUM> from cell <NUM>, and cell state c<NUM> from cell <NUM> and generates output vector h<NUM>. Additional cells can be used, and an LSTM with four cells is merely an example.

<FIG> depicts an exemplary implementation of an LSTM cell <NUM>, which can be used for cells <NUM>, <NUM>, <NUM>, and <NUM> in <FIG>. LSTM cell <NUM> receives input vector x(t), cell state vector c(t-<NUM>) from a preceding cell, and output vector h(t-<NUM>) from a preceding cell, and generates cell state vector c(t) and output vector h(t).

LSTM cell <NUM> comprises sigmoid function devices <NUM>, <NUM>, and <NUM>, each of which applies a number between <NUM> and <NUM> to control how much of each component in the input vector is allowed through to the output vector. LSTM cell <NUM> also comprises tanh devices <NUM> and <NUM> to apply a hyperbolic tangent function to an input vector, multiplier devices <NUM>, <NUM>, and <NUM> to multiply two vectors together, and addition device <NUM> to add two vectors together. Output vector h(t) can be provided to the next LSTM cell in the system, or it can be accessed for other purposes.

<FIG> depicts an LSTM cell <NUM>, which is an example of an implementation of LSTM cell <NUM>. For the reader's convenience, the same numbering from LSTM cell <NUM> is used in LSTM cell <NUM>. Sigmoid function devices <NUM>, <NUM>, and <NUM> and tanh device <NUM> each comprise multiple VMM arrays <NUM> and activation function blocks <NUM>. Thus, it can be seen that VMM arrays are particular useful in LSTM cells used in certain neural network systems. The multiplier devices <NUM>, <NUM>, and <NUM> and the addition device <NUM> are implemented in a digital manner or in an analog manner. The activation function blocks <NUM> can be implemented in a digital manner or in an analog manner.

An alternative to LSTM cell <NUM> (and another example of an implementation of LSTM cell <NUM>) is shown in <FIG>. In <FIG>, sigmoid function devices <NUM>, <NUM>, and <NUM> and tanh device <NUM> share the same physical hardware (VMM arrays <NUM> and activation function block <NUM>) in a time-multiplexed fashion. LSTM cell <NUM> also comprises multiplier device <NUM> to multiply two vectors together, addition device <NUM> to add two vectors together, tanh device <NUM> (which comprises activation function block <NUM>), register <NUM> to store the value i(t) when i(t) is output from sigmoid function block <NUM>, register <NUM> to store the value f(t) * c(t-<NUM>) when that value is output from multiplier device <NUM> through multiplexor <NUM>, register <NUM> to store the value i(t) * u(t) when that value is output from multiplier device <NUM> through multiplexor <NUM>, and register <NUM> to store the value o(t) * c~(t) when that value is output from multiplier device <NUM> through multiplexor <NUM>, and multiplexor <NUM>.

Whereas LSTM cell <NUM> contains multiple sets of VMM arrays <NUM> and respective activation function blocks <NUM>, LSTM cell <NUM> contains only one set of VMM arrays <NUM> and activation function block <NUM>, which are used to represent multiple layers in the embodiment of LSTM cell <NUM>. LSTM cell <NUM> will require less space than LSTM <NUM>, as LSTM cell <NUM> will require <NUM>/<NUM> as much space for VMMs and activation function blocks compared to LSTM cell <NUM>.

It can be further appreciated that LSTM units will typically comprise multiple VMM arrays, each of which requires functionality provided by certain circuit blocks outside of the VMM arrays, such as a summer and activation function block and high voltage generation blocks. Providing separate circuit blocks for each VMM array would require a significant amount of space within the semiconductor device and would be somewhat inefficient. The embodiments described below therefore reduce the circuitry required outside of the VMM arrays themselves.

An analog VMM implementation can be utilized for a GRU (gated recurrent unit) system. GRUs are a gating mechanism in recurrent neural networks. GRUs are similar to LSTMs, except that GRU cells generally contain fewer components than an LSTM cell.

<FIG> depicts an exemplary GRU <NUM>. GRU <NUM> in this example comprises cells <NUM>, <NUM>, <NUM>, and <NUM>. Cell <NUM> receives input vector x<NUM> and generates output vector h<NUM>. Cell <NUM> receives input vector x<NUM>, the output vector h<NUM> from cell <NUM> and generates output vector h<NUM>. Cell <NUM> receives input vector x<NUM> and the output vector (hidden state) h<NUM> from cell <NUM> and generates output vector h<NUM>. Cell <NUM> receives input vector x<NUM> and the output vector (hidden state) h<NUM> from cell <NUM> and generates output vector h<NUM>. Additional cells can be used, and an GRU with four cells is merely an example.

<FIG> depicts an exemplary implementation of a GRU cell <NUM>, which can be used for cells <NUM>, <NUM>, <NUM>, and <NUM> of <FIG>. GRU cell <NUM> receives input vector x(t) and output vector h(t-<NUM>) from a preceding GRU cell and generates output vector h(t). GRU cell <NUM> comprises sigmoid function devices <NUM> and <NUM>, each of which applies a number between <NUM> and <NUM> to components from output vector h(t-<NUM>) and input vector x(t). GRU cell <NUM> also comprises a tanh device <NUM> to apply a hyperbolic tangent function to an input vector, a plurality of multiplier devices <NUM>, <NUM>, and <NUM> to multiply two vectors together, an addition device <NUM> to add two vectors together, and a complementary device <NUM> to subtract an input from <NUM> to generate an output.

<FIG> depicts a GRU cell <NUM>, which is an example of an implementation of GRU cell <NUM>. For the reader's convenience, the same numbering from GRU cell <NUM> is used in GRU cell <NUM>. As can be seen in <FIG>, sigmoid function devices <NUM> and <NUM>, and tanh device <NUM> each comprise multiple VMM arrays <NUM> and activation function blocks <NUM>. Thus, it can be seen that VMM arrays are of particular use in GRU cells used in certain neural network systems. The multiplier devices <NUM>, <NUM>, <NUM>, the addition device <NUM>, and the complementary device <NUM> are implemented in a digital manner or in an analog manner. The activation function blocks <NUM> can be implemented in a digital manner or in an analog manner.

An alternative to GRU cell <NUM> (and another example of an implementation of CRU cell <NUM>) is shown in <FIG>. In <FIG>, GRU cell <NUM> utilizes VMM arrays <NUM> and activation function block <NUM>, which when configured as a sigmoid function applies a number between <NUM> and <NUM> to control how much of each component in the input vector is allowed through to the output vector. In <FIG>, sigmoid function devices <NUM> and <NUM> and tanh device <NUM> share the same physical hardware (VMM arrays <NUM> and activation function block <NUM>) in a time-multiplexed fashion. GRU cell <NUM> also comprises multiplier device <NUM> to multiply two vectors together, addition device <NUM> to add two vectors together, complementary device <NUM> to subtract an input from <NUM> to generate an output, multiplexor <NUM>, register <NUM> to hold the value h(t-<NUM>) * r(t) when that value is output from multiplier device <NUM> through multiplexor <NUM>, register <NUM> to hold the value h(t-<NUM>) *z(t) when that value is output from multiplier device <NUM> through multiplexor <NUM>, and register <NUM> to hold the value h^(t) * (<NUM>-z(t)) when that value is output from multiplier device <NUM> through multiplexor <NUM>.

Whereas GRU cell <NUM> contains multiple sets of VMM arrays <NUM> and activation function blocks <NUM>, GRU cell <NUM> contains only one set of VMM arrays <NUM> and activation function block <NUM>, which are used to represent multiple layers in the embodiment of GRU cell <NUM>. GRU cell <NUM> will require less space than GRU cell <NUM>, as GRU cell <NUM> will require <NUM>/<NUM> as much space for VMMs and activation function blocks compared to GRU cell <NUM>.

It can be further appreciated that GRU systems will typically comprise multiple VMM arrays, each of which requires functionality provided by certain circuit blocks outside of the VMM arrays, such as a summer and activation function block and high voltage generation blocks. Providing separate circuit blocks for each VMM array would require a significant amount of space within the semiconductor device and would be somewhat inefficient. The embodiments described below therefore reduce the circuitry required outside of the VMM arrays themselves.

The input to the VMM arrays can be an analog level, a binary level, a pulse, a time modulated pulse, or digital bits (in this case a DAC is needed to convert digital bits to appropriate input analog level) and the output can be an analog level, a binary level, a timing pulse, pulses, or digital bits (in this case an output ADC is needed to convert output analog level into digital bits).

In general, for each memory cell in a VMM array, each weight W can be implemented by a single memory cell or by a differential cell or by two blend memory cells (average of <NUM> cells). In the differential cell case, two memory cells are needed to implement a weight W as a differential weight (W = W+ - W-). In the two blend memory cells, two memory cells are needed to implement a weight W as an average of two cells.

<FIG> depicts VMM system <NUM>. In some embodiments, the weights, W, stored in a VMM array are stored as differential pairs, W+ (positive weight) and W- (negative weight), where W = (W+) - (W-). In VMM system <NUM>, half of the bit lines are designated as W+ lines, that is, bit lines connecting to memory cells that will store positive weights W+, and the other half of the bit lines are designated as W- lines, that is, bit lines connecting to memory cells implementing negative weights W-. The W- lines are interspersed among the W+ lines in an alternating fashion. The subtraction operation is performed by a summation circuit that receives current from a W+ line and a W- line, such as summation circuits <NUM> and <NUM>. The output of a W+ line and the output of a W- line are combined together to give effectively W = W+ - W- for each pair of (W+, W-) cells for all pairs of (W+, W-) lines. While the above has been described in relation to W- lines interspersed among the W+ lines in an alternating fashion, in other embodiments W+ lines and W- lines can be arbitrarily located anywhere in the array.

<FIG> depicts another embodiment. In VMM system <NUM>, positive weights W+ are implemented in first array <NUM> and negative weights W- are implemented in a second array <NUM>, second array <NUM> separate from the first array, and the resulting weights are appropriately combined together by summation circuits <NUM>.

<FIG> depicts VMM system <NUM>. the weights, W, stored in a VMM array are stored as differential pairs, W+ (positive weight) and W- (negative weight), where W = (W+) - (W-). VMM system <NUM> comprises array <NUM> and array <NUM>. Half of the bit lines in each of array <NUM> and <NUM> are designated as W+ lines, that is, bit lines connecting to memory cells that will store positive weights W+, and the other half of the bit lines in each of array <NUM> and <NUM> are designated as W- lines, that is, bit lines connecting to memory cells implementing negative weights W-. The W- lines are interspersed among the W+ lines in an alternating fashion. The subtraction operation is performed by a summation circuit that receives current from a W+ line and a W- line, such as summation circuits <NUM>, <NUM>, <NUM>, and <NUM>. The output of a W+ line and the output of a W- line from each array <NUM>, <NUM> are respectively combined together to give effectively W = W+ - W- for each pair of (W+, W-) cells for all pairs of (W+, W-) lines. In addition, the W values from each array <NUM> and <NUM> can be further combined through summation circuits <NUM> and <NUM>, such that each W value is the result of a W value from array <NUM> minus a W value from array <NUM>, meaning that the end result from summation circuits <NUM> and <NUM> is a differential value of two differential values.

Each non-volatile memory cells used in the analog neural memory system is to be erased and programmed to hold a very specific and precise amount of charge, i.e., the number of electrons, in the floating gate. For example, each floating gate should hold one of N different values, where N is the number of different weights that can be indicated by each cell. Examples of N include <NUM>, <NUM>, <NUM>, <NUM>, and <NUM>.

Similarly, a read operation should be able to accurately discern between N different levels.

There is a need in VMM systems for improved input blocks that can be used to quickly and accurately apply currents or voltages to one or more lines of rows of cells to be programmed, read, or erased.

The present invention is defined in independent claim <NUM>.

<FIG> depicts a block diagram of VMM system <NUM>. VMM system <NUM> comprises VMM array <NUM>, row decoder <NUM>, high voltage decoder <NUM>, column decoder <NUM>, bit line drivers <NUM>, input circuit <NUM>, output circuit <NUM>, control logic <NUM>, and bias generator <NUM>. VMM system <NUM> further comprises high voltage generation block <NUM>, which comprises charge pump <NUM>, charge pump regulator <NUM>, and high voltage analog precision level generator <NUM>. VMM system <NUM> further comprises (program/erase, or weight tuning) algorithm controller <NUM>, analog circuitry <NUM>, control engine <NUM> (that may include special functions such as arithmetic functions, activation functions, embedded microcontroller logic, without limitation), and test control logic <NUM>. The systems and methods described below can be implemented in VMM system <NUM>.

The input circuit <NUM> may include circuits such as a DAC (digital to analog converter), DPC (digital to pulses converter, digital to time modulated pulse converter), AAC (analog to analog converter, such as a current to voltage converter, logarithmic converter ), PAC (pulse to analog level converter), or any other type of converters. The input circuit <NUM> may implement normalization, linear or non-linear up/down scaling functions, or arithmetic functions. The input circuit <NUM> may implement a temperature compensation function for input levels. The input circuit <NUM> may implement an activation function such as ReLU or sigmoid. The output circuit <NUM> may include circuits such as a ADC (analog to digital converter, to convert neuron analog output to digital bits), AAC (analog to analog converter, such as a current to voltage converter, logarithmic converter), APC (analog to pulse(s) converter, analog to time modulated pulse converter), or any other type of converters. The output circuit <NUM> may implement an activation function such as rectified linear activation function (ReLU) or sigmoid. The output circuit <NUM> may implement statistic normalization, regularization, up/down scaling/gain functions, statistical rounding, or arithmetic functions (e.g., add, subtract, divide, multiply, shift, log) for neuron outputs. The output circuit <NUM> may implement a temperature compensation function for neuron outputs or array outputs (such as bitline output) so as to keep power consumption of the array approximately constant or to improve precision of the array (neuron) outputs such as by keeping the IV slope approximately the same.

<FIG> depicts input block <NUM> to be used to provide inputs to VMM array <NUM>. Input block <NUM> comprises global digital-to-analog converter (DAC) <NUM>; row registers <NUM>-<NUM> through <NUM>-n, each corresponding to a one of the rows numbered <NUM> through n in the array; digital comparator blocks <NUM>-<NUM> through <NUM>-n; row sample-and-hold buffers <NUM>-<NUM> through <NUM>-n, each corresponding to one of the rows numbered <NUM> through n; and output signals <NUM>-<NUM> through <NUM>-n, each corresponding to one of the rows numbered <NUM> through n, and denoted CGINO, CGIN1. , CGINn-<NUM> and CGINn, respectively. Signal GDACsup is the global DAC signal supplied by the global DAC <NUM>. The signals CGIN0-n couples to the row inputs of the array <NUM>.

Digital comparator blocks <NUM> compare the value stored in the associated row register <NUM> against CLKCOUNTx, which is a result of counting a clock signal during an interval; if it matches, then the corresponding row S/H <NUM> is enabled to sample the value from the global DAC <NUM> into the respective row S/H buffer. This technique will be referred to as global row DAC sampling. Each row in VMM array <NUM> has a corresponding row register <NUM>, digital comparator block <NUM>, and row S/H <NUM>.

During operation, row registers <NUM>-<NUM> through <NUM>-n are loaded with digital input bits DINx (where x is the number of bits, such as <NUM> or <NUM> bits) for that particular row and receives a clock signal, CLK. The CLK signal is used to load in the data from the DINx into the row registers <NUM>-x. Global digital-to-analog converter <NUM> is shared by all rows, and in a time-multiplexed fashion, performs a digital-to-analog conversion on the digital bits DINx stored in a particular row register <NUM>. The conversion is done by comparing the digital input bits of a particular row versus CLKCOUNTx, which is digital counting value, by each of the digital comparator blocks <NUM>). When the digital counting values of the global DAC <NUM> match the contents of the respective row register <NUM>, the corresponding row sample-and-hold buffer <NUM> for that row samples the analog output from global digital-to-analog converter <NUM> and holds that value, which is then applied as output signal <NUM> for that particular row. Output signal <NUM> can be applied, for example, to a control gate line or a word line during a programming operation in that particular row, in the manner described above with respect to other Figures.

In another embodiment, the row sample-and-hold buffer <NUM> can be shared between multiple rows by time multiplexing the row sample-and-hold buffers.

<FIG> depicts input block <NUM> to be used to provide inputs to VMM array <NUM>. Input block <NUM> comprises global digital-to-analog converter (DAC) <NUM>; row registers <NUM>-<NUM> through <NUM>-n, each corresponding to a respective one of the rows numbered <NUM> through n in the VMM array; digital multiplexer (mux) blocks <NUM>-<NUM> through <NUM>-n, each corresponding to a respective one of the rows numbered <NUM> through n; row sample-and-hold (S/H) buffers <NUM>-<NUM> through <NUM>-n, each corresponding to a respective one of the rows numbered <NUM> through n; and output signals <NUM>-<NUM> through <NUM>-n, denoted CGINO, CGIN1. , CGINn-<NUM> and CGINn, respectively, each corresponding to a respective one of the rows numbered <NUM> through n. The digital mux blocks <NUM> are used to multiplex out the data of the row registers <NUM> into the bus GDAC_DINx, which is applied as an input to the global DAC <NUM>. The corresponding row S/H buffer samples the value from the global DAC into the local S/H. Each row has its own row register <NUM>, S/H buffer <NUM> and output signal <NUM>.

During operation, row registers <NUM>-<NUM> through <NUM>-n are loaded with digital input bits DINx (where x is the number of bits, such as <NUM> or <NUM> bits) for that particular row and receives a clock signal, CLK. The CLK signal is used to load in the data from the DINx into the row registers <NUM>-x. Global digital-to-analog converter <NUM> is shared by all rows, and in a time-multiplexed fashion, performs a digital-to-analog conversion on the digital bits DINx stored in a particular row register <NUM>. The conversion is done by multiplexing the data of the row registers into the data input (bus GDAC_DINx) of the global DAC <NUM>. The multiplexing of the row register data into the data input bus GDAC_DINx is enabled by the signal EN-x <NUM>-x for each row. The corresponding row sample-and-hold buffer <NUM> samples the analog output from global digital-to-analog converter <NUM> and holds that value, which is then applied as output signal <NUM> for that particular row. Output signal <NUM> can be applied, for example, to a control gate line or a word line during a programming operation in that particular row, in the manner described above with respect to other Figures. In another embodiment, the row sample-and-hold buffer <NUM> can be shared between multiple rows by time multiplexing the row sample-and-hold buffers.

<FIG> depicts input block <NUM> to be used to provide inputs to VMM array <NUM>. Input block <NUM> is similar to input block <NUM> but also provides a row decoder function to select one or more rows for an operation. Input block <NUM> comprises global digital-to-analog converter and row decoder <NUM>; row registers <NUM>-<NUM> through <NUM>-n, each corresponding to a respective one of the rows numbered <NUM> through n in the VMM array; digital comparator blocks <NUM> through <NUM>-n each corresponding to a respective one of the rows numbered <NUM> through n in the VMM array; row sample-and-hold buffers <NUM>-<NUM> through <NUM>-n; and output signals <NUM>-<NUM> through <NUM>-n, denoted CGINO, CGIN1. , CGINn-<NUM> and CGINn, respectively, each corresponding to a respective one of the rows numbered <NUM> through n. The digital comparator blocks <NUM> compare the value stored in the respective row register <NUM> against CLKCOUNTx, which is a counting value. When the digital counting values of the global DAC <NUM> match the contents of the respective row register <NUM>. the respective row S/H buffer <NUM> samples the value from the global DAC <NUM> into the respective S/H buffer <NUM>. Each row has its own row register <NUM>, digital comparator block <NUM>, and row S/H buffer <NUM>.

During operation, row registers <NUM>-<NUM> through <NUM>-n are loaded with digital input bits DINx (where x is the number of bits, such as <NUM> or <NUM> bits) for the associated row and receives a clock signal, CLK. The CLK signal is used to load in the data from the DINx into the row registers <NUM>-x. Global digital-to-analog converter a <NUM> (which consists of a plurality of global digital-to-analog converters, such as <NUM>-<NUM> and <NUM>-<NUM>) is shared by all rows. In one embodiment, global DAC <NUM>-<NUM> operates on even rows and global DAC and row decoder <NUM>-<NUM> operates on odd rows. Global digital-to-analog converter <NUM> receives a row addresses through the row data-in bus GDAC_DINx and selects the corresponding ( rows. It then performs a digital-to-analog conversion on the digital bits DINx stored in the relevant row register(s) <NUM> (through the GDAC_DINx bus). The corresponding row(s) sample-and-hold buffer <NUM> for that row(s) samples the analog output from global digital-to-analog converter <NUM> and holds that value, which is then applied as output signal <NUM> for that particular row. Output signal <NUM> can be applied, for example, to a control gate line or a word line during a programming operation in that particular row or rows, in the manner described above with respect to other Figures.

<FIG> depicts waveforms <NUM> that illustrate exemplary linear voltage levels for exemplary sample-and-hold actions by row sample-and-hold buffer <NUM> in <FIG>, row sample-and-hold buffer <NUM> in <FIG> or row sample-and-hold buffer <NUM> in <FIG>. This is suitable for memory cells operating in linear region. The signal GDACsup <NUM> is the supplied voltage from the global linear DAC such as from the circuit blocks <NUM> in <FIG>, <NUM> in <FIG>, 3601x in <FIG>. It shows the linear steps to illustrate this is a linear DAC.

<FIG> depicts waveforms <NUM> that illustrate exemplary logarithmic voltage levels for exemplary sample-and-hold actions by row sample-and-hold buffer <NUM> in <FIG>, row sample-and-hold buffer <NUM> in <FIG> or row sample-and-hold buffer <NUM> in <FIG>. This is suitable for memory cells operating in sub threshold region. Alternatively, the global DAC voltage waveform can be done for memory cells operating in saturation region. The signal GDACsup <NUM> is the supplied voltage from the global (sub threshold, linear, saturation) DAC such as from the circuit blocks <NUM> in <FIG>, <NUM> in <FIG>, 3601x in <FIG>. It shows the log steps to illustrate this is a log DAC.

<FIG> depicts sample-and-hold buffer <NUM>, which can be used for row sample-and-hold buffer <NUM> in <FIG>, row sample-and-hold buffer <NUM> in <FIG>, or row sample-and-hold buffer <NUM> in <FIG>. Sample-and-hold buffer <NUM> comprises switch <NUM>, capacitor <NUM>, and buffer <NUM>, the buffer <NUM> can be done as a unity buffer using an operation amplifier. During operation, switch <NUM> is closed (enabled, such as by the true result of the comparison of the digital comparator and the digital counting values), which allows an analog value (global DAC value) to be stored (held) in capacitor <NUM>. A value reflective of that stored value can then be output from buffer <NUM> which drives a respective row input of the array. Capacitor <NUM> can be an actual capacitor, or it can be an intrinsic capacitance found in a wire, for example.

<FIG> depicts sample-and-hold buffer <NUM>, which can be used for row sample-and-hold buffer <NUM> in <FIG>, row sample-and-hold buffer <NUM> in <FIG> or row sample-and-hold buffer <NUM> in <FIG>. Sample-and-hold buffer <NUM> comprises switch <NUM> and capacitor <NUM>. During operation, switch <NUM> is closed (enabled, such as by the true result of the comparison of the digital comparator and the digital counting value), which allows an analog value to be stored (held) in capacitor <NUM>. A value reflective of that value can then be output from capacitor <NUM>. Capacitor <NUM> can be an actual capacitor, or it can be an intrinsic capacitance found in a wire, for example.

<FIG> depicts input block <NUM>. Input block <NUM> comprises digital-to-analog converter <NUM>, voltage-to-current converter <NUM>, and current to voltage logarithmic converter <NUM>. Input block <NUM> can be added to any of the preceding input blocks when it is desired to have an input voltage that varies according to a logarithmic function, which is useful, for example, if the memory cells in a VMM array are operating in the sub-threshold range. Digital-to-analog converter receives a digital input, DINx, and generates an analog voltage, Vout. Voltage-to-current converter <NUM> linearly converts analog voltage Vout into a current Iout. Current to voltage logarithmic converter <NUM> converts the current Iout into a voltage Vlog according to a logarithmic function: For example, Vlog = A*log (Iout), where A is a constant. The input block <NUM> can be used as a global DAC as in <FIG>, <FIG> and <FIG>. It can generate waveforms <NUM> shown in <FIG>. For the global DAC in <FIG>, CLK is used to generate the DAC output voltages in a stepwise fashion from for example <NUM> to <NUM> steps for 8bit DAC. For global DAC in <FIG> and <FIG>, DINx is used to generate the DAC output voltage.

<FIG> depicts input block <NUM>. Input block <NUM> comprises a digital-to-analog converter, which can be used as the global DAC. This can be used for example for memory cells operating in linear region. The input block <NUM> can be used as a global DAC as in <FIG> and <FIG>. It can generate waveforms <NUM> shown in <FIG>,.

<FIG> depicts input block <NUM>. Input block <NUM> comprises current digital-to-analog converter <NUM> and current to voltage logarithmic converter <NUM>. Input block <NUM> can be added to any of the preceding input blocks when it is desired to have an input voltage that varies according to a logarithmic function, which is useful, for example, if the memory cells in a VMM array are operating in the sub-threshold range. Current digital-to-analog converter <NUM> receives a digital input, DINx, as in previous examples and generates an analog current, Iout. Current to voltage logarithmic converter <NUM> converts the current Iout into a voltage Vlog, for example using a memory cell or a MOS transistor, according to a logarithmic function: For example, Vlog = A*log (Iout) + Vlogoffset, where A is a constant and Vlogoffset is another constant, for example, to take care of the turn on or off voltage of the memory cell or some other offset voltage from array or decoding circuity or the input/output circuit itself.

<FIG> depicts input block <NUM>. Input block <NUM> comprises current digital-to-analog converter <NUM> and current to voltage linear converter <NUM>. Input block <NUM> can be added to any of the preceding input blocks when it is desired to have an input voltage that varies according to a linear function, which is useful, for example, if the memory cells in a VMM array are operating in the linear range. Current digital-to-analog converter <NUM> receives a digital input, DINx, as in previous examples and generates an analog current, Iout. Current to voltage linear converter <NUM> converts the current Iout into a voltage Vlin, for example using a memory cell or a MOS transistor, according to a linear function: For example, Vlin = A* Iout +Vlinoffet, where A is a constant and Vlogoffset is another constant, for example to take care of the turn on or off voltage of the memory cell or some other offset voltage from array or decoding circuity or the input/output circuit itself.

<FIG> depicts adjustable 2D thermometer code current digital-to-analog converter <NUM>. Adjustable 2D thermometer code current digital-to-analog converter <NUM> comprises control logic <NUM> and 2D array <NUM> comprising an array of i rows and j columns of devices <NUM>, where a particular device <NUM> is noted by the label <NUM>-(row)(column). The particular devices <NUM> may be current mirrors. As shown, there are <NUM> current mirrors (devices <NUM>) in the 2D array <NUM>. The adjustable 2D thermometer code current digital-to-analog converter <NUM> converts a <NUM> digital input code into an output current <NUM>, Iout, with value from <NUM> to <NUM> times Ibiasunit which is provided from the bias source <NUM>.

For example, bias source <NUM> can provide a current Ibiasunit of 1nA, which is mirrored into devices <NUM>. Here, the first row consists of devices <NUM>-<NUM> to <NUM>-1j and is enabled sequentially from left to right, one device <NUM> at a time. Then the next row is enabled in a sequential manner from left to right to add to the first row, meaning <NUM> then <NUM> then <NUM> then <NUM> devices <NUM> are enabled. Hence, by sequentially enabling devices <NUM>, any transistor mismatch associated with conventional binary decoding can be reduced. The sum of the enabled devices <NUM> is then output as an output current <NUM>. The shown <NUM> x <NUM>2D thermometer code current digital-to-analog converter <NUM> could be any other dimension such as <NUM> x <NUM> or <NUM> x <NUM>.

<FIG> depicts reference sub-circuit <NUM>, which is can be used for device <NUM> in <FIG>. Reference sub-circuit <NUM> comprises NMOS transistors <NUM> and <NUM>, configured as shown. The transistor <NUM> is a current mirror bias transistor and transistor <NUM> is an enabling transistor (to enable the bias transistor <NUM> to be connected to output node OUTPUT), master bias current that is used to mirror to the transistor <NUM> is not shown.

<FIG> depicts a SAR (successive approximation register) digital -to-analog converter <NUM>. This circuit is a digital -to-analog converter that relies upon charge redistribution using binary capacitors. It includes a binary CDAC (DAC basing on capacitors) <NUM>, op-amp/comparator <NUM>, and SAR logic <NUM>. GndV is a low voltage reference level, for example, ground level. This is, for example, used in <FIG>.

<FIG> depicts voltage-to-current converter <NUM>, which comprises op amp <NUM>, NMOS transistor <NUM>, and resistor <NUM>, configured as shown. Voltage-to-current converter <NUM> receives input voltage VIN and generates output current IOUT. This, for example, can be used in combination with DAC circuit <NUM> to convert voltage into current. This current can be used to convert to a voltage using a log I to V converter such as in <FIG>/<FIG>/<FIG>.

<FIG> depicts logarithmic current-to-voltage converter <NUM>. Logarithmic current-to-voltage converter <NUM> comprises switch <NUM> and NMOS transistor <NUM>, configured as shown. The NMOS <NUM> operates in the sub-threshold region. Logarithmic current-to-voltage converter <NUM> receives input I-in and generates output Vlog.

<FIG> depicts logarithmic current-to-voltage converter <NUM>. Logarithmic current-to-voltage converter <NUM> comprises switches <NUM> and <NUM> and memory cell <NUM>, configured as shown. The memory cell <NUM> operates in the sub-threshold region. Logarithmic current-to-voltage converter <NUM> receives input I-in and generates output Vlog.

<FIG> depicts logarithmic current-to-voltage converter <NUM>. Logarithmic current-to-voltage converter <NUM> comprises switches <NUM> and <NUM>, NMOS transistor <NUM>, and memory cell <NUM>, configured as shown. The memory cell <NUM> operates in the sub-threshold region. The transistor <NUM> imposes a constant bias on the drain of memory cell <NUM>. Logarithmic current-to-voltage converter <NUM> receives input I-in and generates output Vlog. In <FIG> and <FIG>, the log I to V conversion is done to replicate the log I vs V slope of the memory cell current. The cell output current is then = K* I-in, the input current is from the output of the log DAC, which basically represent the value of the input activation (row value). K is determined by the floating charge in the memory cell, which basically represents the weight in neural network.

Claim 1:
An input block for providing an input to a vector-by-matrix multiplication array in a neural memory system, the vector-by-matrix multiplication array comprising non-volatile memory cells arranged in rows and columns, the input block comprising:
a global digital-to-analog converter;
a plurality of row sample-and-hold buffers, each row sample-and-hold buffer corresponding to a row in the array;
characterized in that the input block further comprises
a plurality of row registers, each row register corresponding to a row in the array; and
wherein when a row is selected, the global digital-to-analog converter converts a digital value stored in a row register corresponding to the selected row into an analog value that is sampled and held by the row sample-and-hold buffer corresponding to the selected row and applied to a line coupled to the selected row.