Patent Description:
Numerous embodiments of analog neural memory arrays and associated circuitry that enable concurrent write and verify operations are disclosed.

<FIG> illustrates an artificial neural network, where the circles represent the inputs or layers of neurons. The connections (called synapses) are represented by arrows, and have numeric weights that can be tuned based on experience. This makes the artificial neural network adaptive to inputs and capable of learning. Typically, artificial neural networks include a layer of multiple inputs. There are typically one or more intermediate layers of neurons, and an output layer of neurons that provide the output of the neural network. The neurons at each level individually or collectively make a decision based on the received data from the synapses.

One of the major challenges in the development of artificial neural networks for high-performance information processing is a lack of adequate hardware technology. Indeed, practical artificial neural networks rely on a very large number of synapses, enabling high connectivity between neurons, i.e. a very high computational parallelism. In principle, such complexity can be achieved with digital supercomputers or specialized graphics processing unit clusters. However, in addition to high cost, these approaches also suffer from mediocre energy efficiency as compared to biological networks, which consume much less energy primarily because they perform low-precision analog computation. CMOS analog circuits have been used for artificial neural networks, but most CMOS-implemented synapses have been too bulky given the high number of neurons and synapses.

Applicant previously disclosed an artificial (analog) neural network that utilizes one or more non-volatile memory arrays as the synapses in <CIT>, published as <CIT>. The non-volatile memory arrays operate as an analog neuromorphic memory. The term neuromorphic, as used herein, means circuitry that implement models of neural systems. The analog neuromorphic memory includes a first plurality of synapses configured to receive a first plurality of inputs and to generate therefrom a first plurality of outputs, and a first plurality of neurons configured to receive the first plurality of outputs. The first plurality of synapses includes a plurality of memory cells, wherein each of the memory cells includes spaced apart source and drain regions formed in a semiconductor substrate with a channel region extending there between, a floating gate disposed over and insulated from a first portion of the channel region and a non-floating gate disposed over and insulated from a second portion of the channel region. Each of the plurality of memory cells is configured to store a weight value corresponding to a number of electrons on the floating gate. The plurality of memory cells is configured to multiply the first plurality of inputs by the stored weight values to generate the first plurality of outputs. An array of memory cells arranged in this manner can be referred to as a vector by matrix multiplication (VMM) array. Optionally, the memory cells are non-volatile memory cells. <CIT>shows another example of a neuromorphic memory circuit. <CIT> discloses concurrent write and verify operations in a non-volatile memory.

Examples of different non-volatile memory cells that can be used in VMMs will now be discussed.

Various types of known non-volatile memory cells can be used in the VMM arrays. For example, <CIT> ("the '<NUM> patent") discloses an array of split gate non-volatile memory cells, which are a type of flash memory cells. Such a memory cell <NUM> is shown in <FIG>. Each memory cell <NUM> includes source region <NUM> and drain region <NUM> formed in semiconductor substrate <NUM>, with channel region <NUM> there between. Floating gate <NUM> is formed over and insulated from (and controls the conductivity of) a first portion of the channel region <NUM>, and over a portion of the source region <NUM>. Word line terminal <NUM> (which is typically coupled to a word line) has a first portion that is disposed over and insulated from (and controls the conductivity of) a second portion of the channel region <NUM>, and a second portion that extends up and over the floating gate <NUM>. The floating gate <NUM> and word line terminal <NUM> are insulated from the substrate <NUM> by a gate oxide. Bitline terminal <NUM> is coupled to drain region <NUM>.

Memory cell <NUM> is programmed (where electrons are placed on the floating gate) by placing a positive voltage on the word line terminal <NUM>, and a positive voltage on the source region <NUM>. Electron current will flow from the drain region <NUM> towards the source region <NUM> (source line terminal). The electrons will accelerate and become energized (heated) when they reach the gap between the word line terminal <NUM> and the floating gate <NUM>. Some of the heated electrons will be injected through the gate oxide onto the floating gate <NUM> due to the attractive electrostatic force from the floating gate <NUM>.

<FIG> shows memory cell <NUM>, which is similar to memory cell <NUM> of <FIG> with the addition of control gate (CG) terminal <NUM>. Control gate terminal <NUM> is biased at a high voltage, e.g., 10V, in programming, low or negative in erase, e.g., 0v/-8V, low or mid range in read, e.g., 0v/<NUM>. Other terminals are biased similarly to that of <FIG>.

<FIG> shows memory cell <NUM>, which is similar to memory cell <NUM> of <FIG> except that memory cell <NUM> does not contain an erase gate EG terminal. An erase is performed by biasing the substrate <NUM> to a high voltage and biasing the control gate CG terminal <NUM> to a low or negative voltage. Alternatively, an erase is performed by biasing word line terminal <NUM> to a positive voltage and biasing control gate terminal <NUM> to a negative voltage. Programming and reading is similar to that of <FIG>.

<FIG> depicts a three-gate memory cell <NUM>, which is another type of flash memory cell. Memory cell <NUM> is identical to the memory cell <NUM> of <FIG> except that memory cell <NUM> does not have a separate control gate terminal. The erase operation (whereby erasing occurs through use of the erase gate terminal) and read operation are similar to that of the <FIG> except there is no control gate bias applied. The programming operation also is done without the control gate bias, and as a result, a higher voltage must be applied on the source line terminal during a program operation to compensate for a lack of control gate bias.

<FIG> depicts stacked gate memory cell <NUM>, which is another type of flash memory cell. Memory cell <NUM> is similar to memory cell <NUM> of <FIG>, except that floating gate <NUM> extends over the entire channel region <NUM>, and control gate terminal <NUM> (which here will be coupled to a word line) extends over floating gate <NUM>, separated by an insulating layer (not shown). Programming is performed using hot electron injection from channel <NUM> to floating gate <NUM> in the channel region next to the drain region <NUM>, and erasing is performed using by Fowler-Nordheim electron tunneling from floating gate <NUM> to substrate <NUM>. The read operations operate in a similar manner to that described previously for memory cell <NUM>.

"Read <NUM>" is a read mode in which the cell current is output on the bit line. "Read <NUM>" is a read mode in which the cell current is output on the source line terminal. Optionally, in arrays comprising rows and columns of memory cells <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, or <NUM>, source lines can be coupled to one row of memory cells or to two adjacent rows of memory cells. That is, source line terminals can be shared by adjacent rows of memory cells.

<FIG> depicts twin split-gate memory cell <NUM>. Twin split-gate memory cell <NUM> comprises floating gate (FG) <NUM> disposed over and insulated from the substrate <NUM>, a control gate <NUM> (CG) disposed over and insulated from the floating gate <NUM>, an erase gate <NUM> (EG) disposed adjacent to and insulated from the floating and control gates <NUM>/<NUM> and disposed over and insulated from the substrate <NUM>, where the erase gate <NUM> is created with a T shape such that a top corner of the control gate <NUM> faces the inside corner of the T shaped erase gate <NUM> to improve erase efficiency, and a drain region <NUM> (DR) in the substrate <NUM> adjacent the floating gate <NUM> (with a bit line contact <NUM> (BL) connected to the drain diffusion regions <NUM> (DR)). The memory cells are formed as pairs of memory cells (A on the left and B on the right), sharing a common erase gate <NUM>. This cell design differs from the memory cells discussed above with reference to <FIG> at least in that it lacks a source region under the erase gate <NUM>, lacks a select gate (also referred to as a word line), and lacks a channel region for each memory cell. Instead, a single continuous channel region <NUM> extends under both memory cells (i.e. extends from the drain region <NUM> of one memory cell to the drain region <NUM> of the other memory cell). To read or program one memory cell, the control gate <NUM> of the other memory cell is raised to a sufficient voltage to turn on the underlying channel region portion via voltage coupling to the floating gate <NUM> there between (e.g. to read or program cell A, the voltage on FGB is raised via voltage coupling from CGB to turn on the channel region portion under FGB). Erasing is performed using Fowler Nordheim electron tunneling from floating gate <NUM> to erase gate <NUM>. Programming is performed using hot electron injection from channel region <NUM> to floating gate <NUM>, this is indicated as PROGRAM <NUM> in Table <NUM>. Alternatively, programming is performed using Fowler Nordheim electron tunneling from erase gate <NUM> to floating gate <NUM>, this is indicated as PROGRAM <NUM> in Table <NUM>. Alternatively programming is performed using Fowler Nordheim electron tunneling from channel region <NUM> to floating gate <NUM>, in this case the condition is similar to PROGRAM <NUM> except that the substrate <NUM> is biased at a low voltage or negative voltage while erase gate <NUM> is biased at a low positive voltage.

Table No. <NUM> depicts typical voltage ranges that can be applied to the terminals of twin split-gate memory cell <NUM> for performing read, erase, and program operations:.

The methods and means described herein may apply to other non-volatile memory technologies such as FINFET split gate flash or stack gate flash memory, NAND flash, SONOS (silicon-oxide-nitride-oxide-silicon, charge trap in nitride), MONOS (metal-oxide-nitride-oxide-silicon, metal charge trap in nitride), ReRAM (resistive ram), PCM (phase change memory), MRAM (magnetic ram), FeRAM (ferroelectric ram), OTP (bi-level or multi-level one time programmable), and CeRAM (correlated electron ram), without limitation. The methods and means described herein may apply to volatile memory technologies used for neural network such as SRAM, DRAM, and other volatile synapse cells, without limitation.

S0 is the input layer, which for this example is a 32x32 pixel RGB image with <NUM> bit precision (i.e. three 32x32 pixel arrays, one for each color R, G and B, each pixel being <NUM> bit precision). The synapses CB1 going from input layer S0 to layer C1 apply different sets of weights in some instances and shared weights in other instances, and scan the input image with 3x3 pixel overlapping filters (kernel), shifting the filter by <NUM> pixel (or more than <NUM> pixel as dictated by the model). Specifically, values for <NUM> pixels in a 3x3 portion of the image (i.e., referred to as a filter or kernel) are provided to the synapses CB1, where these <NUM> input values are multiplied by the appropriate weights and, after summing the outputs of that multiplication, a single output value is determined and provided by a first synapse of CB1 for generating a pixel of one of the feature maps of layer C1. The 3x3 filter is then shifted one pixel to the right within input layer S0 (i.e., adding the column of three pixels on the right, and dropping the column of three pixels on the left), whereby the <NUM> pixel values in this newly positioned filter are provided to the synapses CB1, where they are multiplied by the same weights and a second single output value is determined by the associated synapse. This process is continued until the 3x3 filter scans across the entire 32x32 pixel image of input layer S0, for all three colors and for all bits (precision values). The process is then repeated using different sets of weights to generate a different feature map of layer C1, until all the features maps of layer C1 have been calculated.

The purpose of the pooling function P1 is to average out the nearby location (or a max function can also be used), to reduce the dependence of the edge location for example and to reduce the data size before going to the next stage. The synapses CB2 going from layer S1 to layer C2 scan maps in layer S1 with 4x4 filters, with a filter shift of <NUM> pixel.

<FIG> is a block diagram of a system that can be used for that purpose. VMM system <NUM> includes non-volatile memory cells and is utilized as the synapses (such as CB1, CB2, CB3, and CB4 in <FIG>) between one layer and the next layer. Specifically, VMM system <NUM> comprises VMM array <NUM> comprising non-volatile memory cells arranged in rows and columns, erase gate and word line gate decoder <NUM>, control gate decoder <NUM>, bit line decoder <NUM> and source line decoder <NUM>, which decode the respective inputs for the non-volatile memory cell array <NUM>. Input to VMM array <NUM> can be from the erase gate and wordline gate decoder <NUM> or from the control gate decoder <NUM>. Source line decoder <NUM> in this example also decodes the output of VMM array <NUM>. Alternatively, bit line decoder <NUM> can decode the output of VMM array <NUM>.

VMM array <NUM> serves two purposes. First, it stores the weights that will be used by the VMM system <NUM>. Second, VMM array <NUM> effectively multiplies the inputs by the weights stored in VMM array <NUM> and adds them up per output line (source line or bit line) to produce the output, which will be the input to the next layer or input to the final layer. By performing the multiplication and addition function, VMM array <NUM> negates the need for separate multiplication and addition logic circuits and is also power efficient due to its in-situ memory computation.

The output of VMM array <NUM> is supplied to a differential summer (such as a summing op-amp or a summing current mirror) <NUM>, which sums up the outputs of VMM array <NUM> to create a single value for that convolution. The differential summer <NUM> is arranged to perform summation of both positive weight and negative weight inputs to output the single value.

The summed up output values of differential summer <NUM> are then supplied to an activation function circuit <NUM>, which rectifies the output. The activation function circuit <NUM> may provide sigmoid, tanh, ReLU functions, or any other non-linear function. The rectified output values of activation function circuit <NUM> become an element of a feature map of the next layer (e.g. C1 in <FIG>), and are then applied to the next synapse to produce the next feature map layer or final layer. Therefore, in this example, VMM array <NUM> constitutes a plurality of synapses (which receive their inputs from the prior layer of neurons or from an input layer such as an image database), and summer <NUM> and activation function circuit <NUM> constitute a plurality of neurons.

The input to VMM system <NUM> in <FIG> (WLx, EGx, CGx, and optionally BLx and SLx) can be analog level, binary level, digital pulses (in which case a pulses-to-analog converter PAC may be needed to convert pulses to the appropriate input analog level) or digital bits (in which case a DAC is provided to convert digital bits to appropriate input analog level) and the output can be analog level (e.g., current, voltage, or charge), binary level, digital pulses, or digital bits (in which case an output ADC is provided to convert output analog level into digital bits).

The output generated by input VMM system 32a is provided as an input to the next VMM system (hidden level <NUM>) 32b, which in turn generates an output that is provided as an input to the next VMM system (hidden level <NUM>) 32c, and so on. The various layers of VMM system <NUM> function as different layers of synapses and neurons of a convolutional neural network (CNN). Each VMM system 32a, 32b, 32c, 32d, and 32e can be a stand-alone, physical system comprising a respective non-volatile memory array, or multiple VMM systems could utilize different portions of the same physical non-volatile memory array, or multiple VMM systems could utilize overlapping portions of the same physical non-volatile memory array. Each VMM system 32a, 32b, 32c, 32d, and 32e can also be time multiplexed for various portion of its array or neurons. The example shown in <FIG> contains five layers (32a,32b,32c,32d,32e): one input layer (32a), two hidden layers (32b,32c), and two fully connected layers (32d,32e). One of ordinary skill in the art will appreciate that this is merely exemplary and that a system instead could comprise more than two hidden layers and more than two fully connected layers.

As described herein for neural networks, the non-volatile memory cells of VMM array <NUM>, i.e. the memory cells <NUM> of VMM array <NUM>, are preferably configured to operate in a sub-threshold region.

The non-volatile reference memory cells and the non-volatile memory cells described herein are biased in weak inversion: <MAT> where w = e (- Vth)/nVt
where Ids is the drain to source current; Vg is gate voltage on the memory cell; Vth is threshold voltage of the memory cell; Vt is thermal voltage = k*T/q with k being the Boltzmann constant, T the temperature in Kelvin, and q the electronic charge; n is a slope factor = <NUM> + (Cdep/Cox) with Cdep = capacitance of the depletion layer, and Cox capacitance of the gate oxide layer; Io is the memory cell current at gate voltage equal to threshold voltage, Io is proportional to (Wt/L)*u*Cox* (n-<NUM>) * Vt<NUM> where u is carrier mobility and Wt and L are width and length, respectively, of the memory cell.

For an I-to-V log converter using a memory cell (such as a reference memory cell or a peripheral memory cell) or a transistor to convert input current Ids, into an input voltage, Vg: <MAT> Here, wp is w of a reference or peripheral memory cell.

For an I-to-V log converter using a memory cell (such as a reference memory cell or a peripheral memory cell) or a transistor to convert input current Ids, into an input voltage, Vg: <MAT> where, wp is w of a reference or peripheral memory cell.

For a memory array used as a vector matrix multiplier VMM array, the output current is: <MAT> namely <MAT> <MAT> <MAT> Here, wa = w of each memory cell in the memory array.

Alternatively, the non-volatile memory cells of VMM arrays described herein can be configured to operate in the linear region: <MAT> <MAT> meaning weight W in the linear region is proportional to (Vgs-Vth).

Alternatively, the memory cells of VMM arrays described herein can be configured to operate in the saturation region: <MAT> Wα (Vgs-Vth)<NUM>, meaning weight W is proportional to (Vgs-Vth)<NUM>.

Alternatively, the memory cells of VMM arrays described herein can be used in all regions or a combination thereof (sub threshold, linear, or saturation) for each layer or multi layers of a neural network.

VMM array <NUM> implements uni-directional tuning for non-volatile memory cells in memory array <NUM>. That is, each non-volatile memory cell is erased and then partially programmed until the desired charge on the floating gate is reached. If too much charge is placed on the floating gate (such that the wrong value is stored in the cell), the cell is erased and the sequence of partial programming operations starts over. As shown, two rows sharing the same erase gate (such as EG0 or EG1) are erased together (which is known as a page erase), and thereafter, each cell is partially programmed until the desired charge on the floating gate is reached.

The input to the VMM arrays can be an analog level, a binary level, timing pulses, or digital bits and the output can be an analog level, a binary level, timing pulses, or digital bits (in this case an output ADC is needed to convert output analog level current or voltage into digital bits).

For each memory cell in a VMM array, each weight W can be implemented by a single memory cell or by a differential cell or by two blend memory cells (average of <NUM> or more cells). In the differential cell case, two memory cells are needed to implement a weight W as a differential weight (W = W+ - W-). In the two blend memory cells, two memory cells are needed to implement a weight W as an average of two cells.

One challenge with prior art VMM systems is that storing the desired analog value in a cell-which comprises one or more sequences of write and verify operations-can take a relatively large amount of time, since much greater precision is required than in a traditional memory array where only two possible values (i.e., <NUM> and <NUM>) can be stored. Here, a write operation comprises an erase operation, a program operation, or both and erase operation and a program operation, and a verify operation is a read operation to confirm that the write operation was correctly performed.

What is needed is an improved VMM system that has a reduced timing overhead for storing the desired analog value in a selected non-volatile memory cell. Furthermore, it is desirable to have a VMM system that can write memory cells in a VMM array (for example, to perform weight updates in a neural network) while concurrently reading other memory cells in the VMM array (for example, to perform a verify operation during a weight tuning operation or to perform inference for a neural network).

SUMMARY OF THE INVENTION The present invention is defined in the appended independent claims <NUM> and <NUM> to which reference should be made.

Numerous embodiments of VMM systems circuitry that enable concurrent write and verify or read operations are disclosed. A "write" refers to an operation that changes the weight stored in a memory cell such as by programming the cell and/or erasing the cell. A "verify" refers to an operation that senses an output (such as cell current) of a memory cell and confirms that it has reached a target. A "read" refers to an operation that senses an output of the memory cell. In some embodiments, concurrent write and verify or read operations occur among different banks of memory.

The VMM systems of an analog neural network of the present invention utilize a combination of CMOS technology and non-volatile memory arrays.

<FIG> depicts a block diagram of VMM system <NUM>. VMM system <NUM> comprises bank <NUM> and bank <NUM>. Bank <NUM> comprises VMM array <NUM>, DAC (digital-to-analog converter) and row decoders (row input circuit) <NUM>, high voltage decoders <NUM>, and column decoders <NUM>. Bank <NUM> comprises VMM array <NUM>, DAC and row decoders <NUM>, high voltage decoders <NUM>, and column decoders <NUM>. The DAC of the DAC and row decoders <NUM> is used to convert row input digital bits into an analog bias/timing to be applied to the row input of the VMM array <NUM> such as during neural read (inference operation of a neural network).

VMM system <NUM> further comprises high voltage HV generation block <NUM>, which comprises charge pump <NUM>, charge pump regulator <NUM>, and high voltage level generator <NUM>. VMM system <NUM> further comprises low voltage LV generation block <NUM>, algorithm controller <NUM>, analog circuitry <NUM>, control logic <NUM>, and test control logic <NUM>.

VMM system <NUM> further comprises sensing circuitry (bit line/column output circuit) <NUM> and (column) writing circuitry <NUM>, both of which are shared by VMM <NUM> and VMM <NUM>. The sensing circuitry <NUM> includes circuitry for verify and for neural read (for inference, such as including analog-to-digital ADC circuits). The writing circuitry <NUM> controls programming/erasing of the memory cell to a weight target (a cell current target) such as controlling the bit line bias voltage or program current levels. Thus, in <FIG>, sensing circuitry <NUM> is coupled to VMM array <NUM> during a verify or read operation of VMM array <NUM>, and writing circuitry <NUM> is coupled to VMM array <NUM> during a writing operation (program or erase) of VMM array <NUM>. Similarly, in <FIG>, sensing circuitry <NUM> is coupled to VMM array <NUM> during a verify or read operation of VMM array <NUM>, and writing circuitry <NUM> is coupled to VMM array <NUM> during a writing operation of VMM array <NUM>.

With reference again to <FIG>, block <NUM> is a logical unit comprising bank <NUM>, bank <NUM>, sensing circuitry <NUM>, and writing circuitry <NUM>.

<FIG> depicts VMM system <NUM>, which is a variation of VMM system <NUM> from <FIG>. VMM system <NUM> comprises the same components as VMM system <NUM> with twice as many VMM arrays and associated circuitry. While VMM system <NUM> is depicted having twice as many VMM arrays, this is not meant to be limiting in any way, and any number of VMM arrays may be provided without exceeding the scope. Thus, VMM system <NUM> further comprises bank <NUM> and bank <NUM>. Bank <NUM> comprises VMM array <NUM>, DAC and row decoders (row input circuit) <NUM>, high voltage decoders <NUM>, and column decoders <NUM>. Bank <NUM> comprises VMM array <NUM>, DAC and row decoders <NUM>, high voltage decoders <NUM>, and column decoders <NUM>. VMM system <NUM> further comprises sensing circuitry <NUM> and writing circuitry <NUM>, both of which are shared by VMM <NUM> and VMM <NUM>.

High voltage generation block <NUM>, low voltage generation block <NUM>, algorithm controller <NUM>, analog circuitry <NUM>, control logic <NUM>, and test control logic <NUM> are shared by VMM arrays <NUM>, <NUM>, <NUM>, and <NUM>.

Block <NUM> is a logical unit comprising bank <NUM>, bank <NUM>, sensing circuitry <NUM>, and writing circuitry <NUM>.

<FIG> again depicts VMM system <NUM>. Here, block <NUM> is a logical unit comprising banks <NUM>, <NUM>, <NUM>, and <NUM>; sensing circuitry <NUM> and <NUM>; and writing circuitry <NUM> and <NUM>. Thus, as used herein, a "block" can consist of two banks or more than two banks.

<FIG> depicts control signals <NUM> for performing concurrent verify and write operations across multiple banks, such as those shown in <FIG>, which can be used in an analog neural memory system. Control signals <NUM> are generated by a combination of control logic <NUM>, algorithm controller <NUM>, high voltage generation block <NUM> and low voltage generation block <NUM>, which can be referred to together, or in part, as control circuitry <NUM>. In this example, the banks are divided into sets - set <NUM> (which comprises a first bank and possibly other banks) and set <NUM> (which comprises a second bank and possibly other banks). In response to control signals <NUM>, when set <NUM> of banks is being written (denoted wrt) using writing circuitry (such as writing circuitry <NUM> or <NUM>, not shown), set <NUM> of banks is being verified (read) (denoted vfy) using sensing circuitry (such as sensing circuitry <NUM> or <NUM>, not shown). In response to control signals <NUM>, when set <NUM> of banks is being verified (or read), set <NUM> of banks is being written (programmed or erased). By operating concurrently on the banks instead of sequentially, the total amount of time for the program and verify process is cut by as much as half.

<FIG> depicts control signals <NUM> for performing concurrent verify and write operations across multiple banks. Control signals <NUM> comprising control signals <NUM> and <NUM> are generated by a combination of control logic <NUM>, algorithm controller <NUM>, high voltage generation block <NUM> and low voltage generation block <NUM>, which again can be referred to together or in part as control circuitry <NUM>. In this example, the banks are divided into sets - set <NUM> (which, for example, can comprise a first bank and possibly other banks), set <NUM> (which, for example, can comprise a second bank and possibly other banks), set <NUM> (which, for example, can comprise a third bank and possibly other banks) and set <NUM> (which, for example, can comprise a fourth bank and possibly other banks). In response to control signals <NUM>, when set <NUM> of banks is being written using writing circuitry (such as writing circuitry <NUM> or <NUM>, not shown), set <NUM> of banks is being verified (or read) using sensing circuitry (such as sensing circuitry <NUM> or <NUM>, not shown). In response to control signals <NUM>, when set <NUM> of banks is being verified (or read), set <NUM> of banks is being written and when set <NUM> of banks is being written, set <NUM> of banks is being verified (or read). In response to control signals <NUM>, when set <NUM> of banks is being written and verified (or read), set <NUM> of banks is being read such as for inference neural network operation (or verify), and when set <NUM> of banks is being written, set <NUM> of banks is being verified (or read). By operating concurrently on the blocks instead of sequentially, the total amount of time for the program and verify process is cut by as much as half. And by operating concurrently on the blocks, some portion of neural networks can perform neural read (for inference) while others are performing write operations, such as for weight update.

An example of the implementation of <FIG> is shown in <FIG>. VMM system <NUM> comprises an array of blocks, with the first row of the array of blocks comprising blocks <NUM><NUM>,<NUM>,. , <NUM><NUM>,m, and the last row of the array of blocks comprising blocks <NUM>n, <NUM>,. , <NUM>n,m, where m is an integer number of columns of blocks in VMM system <NUM> and n is an integer number of rows of blocks in VMM system <NUM>. In this example, blocks <NUM><NUM>,<NUM>, <NUM>n,<NUM>, <NUM>n. m (as well as certain other blocks in the array that are not shown) are in set <NUM> of blocks, and block <NUM><NUM>,m (as well as certain other blocks in the array that are not shown) are in set <NUM> of blocks, with reference to <FIG>. At the particular moment in time captured in <FIG>, the blocks in set <NUM> (which can comprise, for example, a first block comprising two or more banks of non-volatile memory cells, each bank comprising an array of non-volatile memory cells) are being written, and the blocks in set <NUM> (which can comprise, for example, a second block comprising two or more banks of non-volatile memory cells, each bank comprising an array of non-volatile memory cells) are being verified (read).

An example of the implementation of <FIG> is shown in <FIG>. VMM system <NUM> comprises VMM arrays <NUM> and <NUM>; sensing circuitry <NUM>; writing circuitry <NUM>; first and second write multiplexors <NUM> and <NUM>; and first and second read (verify) multiplexors <NUM> and <NUM>. First write multiplexor <NUM>, when enabled, couples VMM array <NUM> to writing circuitry <NUM>. Second write multiplexor <NUM>, when enabled, couples VMM array <NUM> to writing circuitry <NUM>. First read multiplexor <NUM>, when enabled, couples VMM array <NUM> to sensing circuitry <NUM>. Second read multiplexor <NUM>, when enabled, couples VMM array <NUM> to sensing circuitry <NUM>. The above has been described with two write multiplexors, and two read multiplexors, it being understood that this is not meant to be limiting. A single multiplexor may be used in place of the two separate multiplexors, without exceeding the scope.

Another example of the implementation of <FIG> is shown in <FIG>. VMM system <NUM> comprises VMM arrays <NUM> and <NUM>; sensing circuitry <NUM>; writing circuitry <NUM>; first and second write multiplexors <NUM> and <NUM>; first and second read (verify) multiplexors <NUM> and <NUM>; and column multiplexor <NUM>. First write multiplexor <NUM>, when enabled, couples VMM array <NUM> to writing circuitry <NUM>. Second write multiplexor <NUM>, when enabled, couples VMM array <NUM> to writing circuitry <NUM>. First read multiplexor <NUM>, when enabled, couples VMM array <NUM> to sensing circuitry <NUM>. Second read multiplexor <NUM>, when enabled, couples VMM array <NUM> to sensing circuitry <NUM>. Column multiplexor <NUM>, when enabled, connects bit lines in VMM array <NUM> to bit lines in VMM array <NUM>, so that both VMM arrays <NUM> and <NUM> are then coupled to the same bit lines and can be controlled as a single array. The above has been described with two write multiplexors, and two read multiplexors, it being understood that this is not meant to be limiting. A single multiplexor may be used in place of the two separate multiplexors, without exceeding the scope.

Claim 1:
An analog neural memory system (<NUM>) comprising:
a first bank (<NUM>) comprising a first array of non-volatile memory cells;
a second bank (<NUM>) comprising a second array of non-volatile memory cells;
writing circuitry (<NUM>) shared by the first bank and the second bank;
sensing circuitry (<NUM>) shared by the first bank and the second bank;
a first write multiplexor (<NUM>, <NUM>) to couple the first bank to the writing circuitry;
a second write multiplexor (<NUM>, <NUM>) to couple the second bank to the writing circuitry;
a first read multiplexor (<NUM>, <NUM>) to couple the first bank to the sensing circuitry;
a second read multiplexor (<NUM>, <NUM>) to couple the second bank to the sensing circuitry; and
control circuitry (<NUM>) for concurrently performing a write operation using the writing circuitry on one of the first bank and the second bank through one of the first write multiplexor and the second write multiplexor and a verify operation using the sensing circuitry on the other of the first bank and the second bank through one of the first read multiplexor and the second read multiplexor.