Patent ID: 12242949

DETAILED DESCRIPTION

According to embodiments, multiply-accumulate (MAC) results can be generated by storing sets of multiplier values (e.g., weights) in memory cell rows of a compute-in-memory (CIM) array. Multiplicand values (e.g., input values) can be applied in a column wise fashion to the CIM array. In response to multiplicand values, memory cells can generate a cell current or voltage that varies according to the stored weight values. Cell currents or voltages can be accumulated as analog MAC values on a conductive line (e.g., source line) common to each row. Analog MAC values from each row can be multiplexed to an analog-to-digital converter (ADC) to generate a digital MAC value.

According to embodiments, MAC results from CIM arrays can include operations of artificial neurons. CIM arrays can be connected to one another with a programmable switch fabric to form an artificial neural network.

According to embodiments, multiplicand (input) values can be signed. For an input value of one sign, a bit line can be driven to one voltage (e.g., positive). For an input value of another sign, the bit line can be driven to another voltage (e.g., zero or negative). In some embodiments, weight values are stored as pairs, with positive weights (+ve) being stored in one memory cell and negative weights (−ve) being stored in an adjacent memory cell. For a positive input value, the corresponding bit line pair can be driven to different values (e.g., BL0=VHI, BL1=VLOW). For a negative input value, the corresponding bit line pair can be driven to opposite values (e.g., BL0=VLOW, BL1=VHI).

According to embodiments, the conductive lines that accumulate MAC results (e.g., source lines) can have a lower resistance than the bit lines.

According to embodiments, memory cells can be narrower in the row direction than the column direction. That is, memory cells can have a row height greater than a column width. In some embodiments, a memory cell height (i.e., the cell dimension in the column direction) can be no less than three times the memory cell width (i.e., the cell dimension in the row direction).

According to embodiments, in a MAC operation, bit lines can be driven according to an input value. Word line driver circuits can activate word lines of one or more rows of a CIM array. An activated word line can connect a programmable element of each memory cell in the row to a corresponding bit line. A potential difference between a bit line and source line can generate a current through the memory cell that varies according to the programmable element. As a source line can be connected to one per row, a resulting current on the source line can be an accumulation of all currents generated by the memory cell of the row.

According to embodiments, a CIM array can be a nonvolatile memory array, with weight values being stored in nonvolatile fashion. In some embodiments, memory cells can include an insulated gate field effect transistor (IGFET) structure having a programmable threshold voltage. In some embodiments, memory cells can be two transistor memory cells, having a select transistor and a programmable transistor.

In the various embodiments below, like items are referred to by the same reference characters, but with the leading digit(s) corresponding to the figure number.

FIG.1is a block diagram of a CIM device100according to an embodiment. A CIM device100can include a nonvolatile memory (NVM) cell array102, word line drivers104-0to -n, bit line driver106, a source line multiplexer (MUX) section108, and ADC section112. A NVM cell array102can include NVM cells110arranged into rows (one shown as114) and columns (one shown as116). NVM cells110of a same column can be connected to a same bit line (one shown as118). NVM cells110of a same row can be connected to a same word line WL0-WLn (or set of word lines) and same source line SL0-SLn. Unlike conventional approaches, source lines (SL0-SLn) are not commonly connected, but rather separate from one another. In one embodiment, NVM cells may be silicon-oxide-nitride-oxide-silicon (SONOS) based charge-trapping memory cells capable of being programmed and retaining charges to represent multiple (more than two) states or levels. In other embodiments, NVM cells may be other multi-level memory cells, such as floating gate, R-RAM, etc.

NVM cells110can be programmed to a predetermined multi-level current or voltage (e.g., drain current or threshold voltage) to the corresponding source line (SL0-SLn) when selected. Such current or voltage is represented by Gi,j, where i=1 to k, and j=1 to n. In some embodiments, NVM cells110can be programmed to store sets of multiplier terms (e.g., kernels) for MAC operations. However, in contrast to conventional approaches, such sets (one represented by120) can be stored in a row-wise fashion and not a column-wise fashion. According to embodiments, NVM cells110can be programmed between at least three different states or analog levels, to provide or represent different analog weight values. That is, NVM cells110are not programmed with binary values.

Word line drivers (104-0to -n) can drive one or more word lines for each row114to select the NVM cells110of the row. In some embodiments, such as in a two-transistor (2T) configuration or 1.5T (split gate) configuration, there can be more than one word line per row114(e.g., a select gate word line and control gate word line), and such different word lines can be driven separately according to mode of operation. However, in other embodiments there can be one word line per row114. Bit line driver106can drive bit lines118according to input (e.g., multiplicand) values (X1, X2. . . )124. Different bit lines can represent different input terms. In some embodiments, bit lines118can be driven between two different voltages (i.e., input terms can be binary values). In other embodiments, bit lines118can be driven between more than two different states or driven to an analog value. Accordingly, a bit line driver106can include any of: digital driver circuits106A, multi-bit driver circuits106B or analog driver circuits106C. Digital driver circuits106A can drive a bit line between two voltage levels in response to a binary input value X1[0]. Multi-bit driver circuit106B can drive a bit line between three or more levels. In some embodiments, a bit line driver106B can include a digital-to-analog converter (DAC), that can generate different output voltages (V0to Vi) in response to a multi-bit input value X1[0:n] (where n≥1). Analog driver circuits106C can drive a bit line to an analog voltage Vout in response to an analog input voltage Vin, which may or may not be the same as Vout.

MUX section108can include one or more MUXs for selectively connecting source lines (SL0to SLn) to ADC section112. ADC section112can convert a voltage or current on a selected source line (SL0to SLn) into a digital value.

Having described the general sections of a CIM device100, MAC operations will now be described. Kernels (e.g.,120) can be stored in rows of NVM cell array102. Input values124can be driven on bit lines by BL driver106. Row drivers (104-0to -n) can drive a word line (WL0to -n). In response, a current path can be enabled through the NVM cells110of the selected row, between each bit line118and the source line (SL0to SLn) for the row. Such cell currents are thus summed on the source line as an analog MAC result (122-0to -n).

MUX section108can connect the source line (SL0to SLn) to the ADC section112. The ADC section122can convert the selected analog MAC result (122-0to -n) into a digital MAC result126. Such an operation can be repeated to generate MAC results (122-0to -n) for each row in response to a same set of input values on bit lines118.

A CIM device100can have various features that differ from conventional approaches. According to embodiments, NVM cell array102can store kernels in rows, as opposed to columns. Input values can be applied via bit lines, rather than word lines. Further, NVM cell array102can have source lines (SL0to SLn) dedicated to each row, rather than unified source lines as in the conventional case. MAC results can be summed on source lines, and not bit lines. Similarly, rows (e.g., source lines) can be MUXed to ADC circuits as opposed to columns (e.g., bit lines).

In some embodiments, bit line driver106can provide inputs for neurons and kernels120can be weight values for such neuron inputs. Each analog MAC result (122-0to -n) can correspond to a different neuron in response to a same input value set (e.g., neurons of a same hidden layer).

Embodiments can include memory cells of any suitable type that can be programmed between more than two states, where each different state can provide a different current or voltage response.FIGS.2A to2Eare diagram showing examples of NVM cells that can be included in embodiments.

FIG.2Ais a block diagram of a memory cell210A that can be included in CIM arrays according to embodiments. A memory cell210A can be connected to a bit line218, a word line228and a source line230. Memory cell210A can be programmed with a weight value, which can dictate the magnitude of a current flow through the memory cell210A. In response to a voltage on word line228, a conductive path can be enabled between bit line218and source line230. Bit line218can be at a voltage VBLwhich can vary according to an input value Xn. Source line230can be at a source line potential VSL. A current (I) flowing through the memory cell210A can vary according to the stored weight value, and in some embodiments I=Xn*Weight. A direction of current I can vary according to the values of VBLand VSL.

FIG.2Bis a schematic diagram of one transistor (1T) cell210B that can be included in CIM arrays according to embodiments. A 1T cell210B can have an IGFET structure with a threshold voltage (Vt) programmable between no less than three values. A threshold voltage (Vt) can be established with any suitable structure, including a floating gate or other charge trapping mechanism, such as a SONOS type device. A SONOS type device can include silicon substrate, ONO gate dielectric, and silicon (e.g., polysilicon gate), however a SONOS type device can also be subject to variation in gate types, substrate types, and gate dielectric structures. A 1T cell210B can be selected as described for the cell ofFIG.2A. A resulting cell current Icell can vary according to the programmed Vt.

FIG.2Cshows a two-device memory cell210C that can be included in CIM arrays according to embodiments. Memory cell210C can include an access device232and a programmable element234. In response to a voltage on a word line228, select device232can enable a conductive path between bit line218and programmable element234. A programmable element234can be programmed between three or more weight values. In some embodiments, programmable element234can be connected to one or more other nodes (shown as236) to enable the programmable element and/or to program a weight value into programmable element. A resulting current can vary according to the weight value programmed in the programmable element. A programmable element can be a two-terminal device or a three-terminal device.

FIG.2Dshows a two-transistor (2T) memory cell210D that can be included in CIM arrays according to embodiments. Memory cell210D can include a select transistor M20and a programmable transistor M22. Select transistor M20can be enabled by a voltage on a select gate (SG) word line228-0to provide a current path between bit line218programmable transistor M22. Programmable transistor M22can have a programmable Vt as described herein and equivalents. In some embodiments, programmable transistor can be a SONOS type device. In some embodiments, in a current generating (e.g., MAC) operation, a control gate228-1can be biased to generate the desired weighted current (Icell). WhileFIG.2Dshows transistors of n-type conductivity, alternate embodiments can include transistors of p-type conductivity, as well as enhancement or depletion mode transistors.

FIG.2Eshows another 1T memory cell210E that can be included in CIM arrays according to embodiments. Memory cell210E can include a select transistor M20and a programmable element234′. Select transistor M20can be enabled by a voltage on a select gate (SG) word line228to provide a current path between bit line218programmable element234′. Programmable element234′ can be programmed between three or more resistance states. Programmable element234′ can take any suitable form, including but not limited: to a ferroelectric random access memory (FRAM) element, magnetoresistive RAM (MRAM) elements, phase change RAM (PCM) elements, or resistive RAM (RRAM) elements.

FIG.3is a block diagram of a CIM device300according to another embodiment. In some embodiments, CIM device300can be one implementation of that shown inFIG.1. A CIM device300can include items like those ofFIG.1, and such like items can operate in a same or similar fashion.

InFIG.3, weight values can have positive components and negative components. Positive and negative weight value are stored in adjacent memory cells (two shown as310-0/1) of a column pairs (one shown as336). For example, memory cell310-0can store a positive component (+ve) of weight W11, while memory cell310-1can store a negative weight component (−ve) of weight W11(or vice versa).

According to embodiments, a CIM device300can store sets of weight values in rows. In addition, weight values can be programmable by rows. This is in contrast to conventional approaches, which can store sets of weight values in columns. By storing weight values in rows, embodiments can update weight value sets faster and with less disturb possibilities as compared to conventional devices. Rather that program multiple rows to update one set of weight values, embodiments can program a single row.

Bit line driver circuits306can drive bit lines of a column pair336between different voltages. Such a feature can enable input values to have a polarity. In some embodiments, if an input value is positive (e.g., X1), one bit line318-0can be driven to a first voltage while the other bit line318-1is driven to a second voltage. However, if an input value is negative (e.g., −X1), one bit line318-0can be driven to the second voltage while the other bit line318-1is driven to the first voltage. This is in contrast to conventional approaches that may have to store negative versions of weight values in a second column. In some embodiments, bit line driver circuits306can deselect a column pair, by driving both columns to a deselect voltage. A deselect voltage can be a voltage that will generate essentially no current in the memory cells selected by a word line.

In operation, sets of rows can be connected to MUXs308-0to308-iwith corresponding source lines SL0to SLn. Each MUX (308-0to308-i) can connect a source line (SL0to SLn) to ADC circuits in response to select signals SL_SEL0to -i. An analog current on the selected source line (SL0to SLn) can be integrated by a corresponding integrator338-0to -i. The integrated charge can be converted into a digital value (DIG. MAC VALUE) by a corresponding ADC circuit312-0to -i.

In some embodiments, memory cells can be longer in the column direction than in the row direction. In a conventional device, MUXs can have inputs connected to columns, and extend in the row direction. In contrast, according to embodiments, MUXs can have inputs connected to rows and extend in the column direction. As a result, MUXs can have more area per input than a conventional device.

FIG.3shows kernels (weights sets)320-0to320-2arranged into rows. Kernel320-0includes memory cells that store weights W11, W21, W31and W41. With the applications of input values X1-X4, a current can be generated on a source line SL0corresponding to a MAC result (X1*W11+X2*W21+X3*W31+X4*W41), which can be a summation operation for a neuron (H1). Kernels320-1/2provide similar results on source lines SL1and SL2, respectively. Such an arrangement can enable kernels to be updated with single row programming operations. This is in contrast to conventional approaches that store kernels a column direction, requiring the programming of multiple rows to update a kernel, introducing the possibility of disturbing the states of other memory cells, as well as increasing a wear rate for the memory cell array.

FIG.4is a table showing bit line voltages for providing signed input values according to an embodiment.FIG.4shows voltages for a bit line pair (VBL+, VBL−). For an input value having a positive polarity (Xj(pos)), a first bit line voltage (VBL+) can be relatively higher (H) than a second bit line voltage (VBL−=L). Conversely, for an input value having a negative (Xj(neg)), VBL+=L and VBL−=H. In some embodiments, a column pair can be deselected by driving a bit line pair to a deselect voltage, which in some embodiments can be a source line voltage (VSL).

FIGS.5A and5Bare diagrams showing memory cell pair operations for positive and negative inputs.FIGS.5A and5Bshow memory cells510-0/1that each include a select transistor M51/M52and a programmable transistor M50/M53. Programmable transistor M50of memory cell510-0can be programmed to store a positive weight value (ve+) while programmable transistor M53of memory cell510-1can be programmed to store a negative weight value (ve−). Each select transistor M51/M52can have a drain connected to a corresponding bit line518-0/1, a gate connected to a select word line540-1and sources connected to the drain of the corresponding programmable transistor M50/M53. Each programmable transistor M50/M53can have a gate connected to a program word line540-0and sources connected to a source line542. Referring toFIG.5A, when an input value Xj is positive, bit line518-0can be driven to VBL+, while bit line518-1can be driven to VBL−. Select word line540-1can be driven to a potential that turns on select transistors M51/M52enabling a current to flow through the select transistor M51/M52that varies according to a programmed state of the corresponding programmable transistor M50/M53. In some embodiments this can also include biasing program word line540-0. A bit line voltage VBL+can be greater than a source line voltage VSL, thus a positive weight current component Ive+ can flow through the memory cell510-0from the bit line518-0to the source line542. A bit line voltage VBL−can be less than a source line voltage VSL, thus a negative weight current component Ive− can flow through the memory cell510-1from the source line542to the bit line518-1.

Referring toFIG.5B, when an input value Xj is negative, bit line518-0can be driven to VBL−, while bit line518-1can be driven to VBL+. Consequently, weight current components Ive+, IVe− can flow in the opposite direction to that ofFIG.5A.

In some embodiments, memory cell pair510-0/1can be selected together, with current components Ive+, Ive− acting against one another at the corresponding source line542. However, in other embodiments, a MAC generation operation can be a two-step process, with one set of current components being selected while the other set is deselected. For example, in a first step, a bit line518-0could be driven to VBL+(or VBL−), while bit line518-1is driven to VSL. A source line542can thus generate positive weight currents. In a second step, the other bit line518-1could be driven to VBL+(or VBL−), while bit line518-0is driven to VSL. A source line542can thus generate negative weight currents.

FIG.6is a cross sectional diagram of a memory cell610that can be included in embodiments. A memory cell610can include a select transistor M62and a programmable transistor M63formed with a substrate656. Select transistor M62can have a drain646connected to receive a bit line voltage VBL, a select gate (SG) connected to, or formed as part of a select word line640-1, and a diffusion644serving as a source. A select gate SG can be formed over a gate dielectric648. Select gate (SG) can be driven to a voltage VWL, which can vary according to operations.

Programmable transistor M63can have a drain from diffusion644, a control gate (CG) connected to, or formed as part of a program word line640-0, and a source connected to, or formed as part of a source line642. A control gate (CG) can be formed over a charge storage gate dielectric650, which in some embodiments can include a layer of silicon nitride formed between layers of silicon oxide (i.e., ONO). Programmable transistor M63can have a SONOS type structure. A control gate (CG) can be driven to a voltage VWLNV, which can vary according to operation. In some embodiments, a magnitude and/or number of pulses for VWLNV, can be used to program a weight value into programmable transistor. Source line642can be driven to a voltage VSL, which can also vary according to operation.

In the embodiment shown, select and programmable transistors M62/M63can be n-channel devices. However, alternate embodiments can include different conductivity type transistors. Select and programmable transistors (M62/M63) can have insulated gate field effect transistor type structures. However, alternate embodiments can have different transistor structure types.

Select and programmable transistors M62/M63can be formed in a first region652, which in some embodiments can be a well doped to an opposite conductivity type to source/drains of M62/M63(e.g., p-type well). According to embodiments, first region652can be driven to a voltage VSPW, which can vary according to operation. In the embodiment shown, first region652may itself be contained in a second region654. In some embodiments, a second region654can be a deep well doped to an opposite conductivity type to first region652. Such an arrangement can enable first region652to be biased to voltages outside of a power supply voltage for a device. For example, first region652can be driven to a negative voltage to erase and/or program programmable transistor M63to store a weight value.

FIG.7is a top plan view of a portion of an NVM cell array702that can be included in embodiments.FIG.7shows memory cell regions752formed in a substrate756. Memory cell regions752can each correspond to a memory cell of a NVM array in a CIM device. Memory cell regions752can be disposed in a row direction758(e.g., parallel to word lines and essentially perpendicular to bit lines). As shown, a row of memory cell regions752can be formed within second region754. Such an arrangement can enable a substrate for a row of memory cells to be driven to program/erase bias voltages, without having to drive other rows. This can advantageously enable erasing and programming of memory cell rows with little or no risk of disturbing memory cells outside of the row being programmed.

FIG.8is a block diagram showing various programming circuits that can be included in embodiments. Programming circuits can apply conditions to memory cells that results in a current response equivalent to a multiplication operation. While memory cells can take any suitable form,FIG.8shows circuits that can be included for programmable transistors.

Section860-0shows a word line driver circuit that can apply word line (e.g., control gate) conditions to establish a threshold voltage of a programmable transistor. A digital-to-analog converter (DAC)864can generate a control gate voltage VCGin response to a digital weight value. Word line driver804-0can drive one or more word lines (840-0/1) with the control gate voltage VCG. In addition or alternatively, a pulse generator868can alter a number and/or duration of voltage pulses based on the digital weight value862. Word line driver804-0can drive one or more word lines (840-0/1) based on such pulses.

Section860-1shows a bit line voltage generator that can be included in embodiments. A DAC870can generate a drain voltage Vdrain in response to a digital weight value862. The drain voltage (Vdrain) can be driven on a bit line. In some embodiments, a drain voltage (Vdrain) can be pulsed, have a number of pulses and/or a pulse duration that varies in response to digital weight value862.

Section860-2shows a substrate voltage generator that can be included in embodiments. A DAC872can generate a substrate voltage (VSPW, VDNW) in response to a digital weight value862. A DAC872can generate more than one substrate voltage. Further, such substrate voltages can vary in polarity.

Embodiments can include a NVM cell array for a CIM device having source lines that extend in the row direction (i.e., parallel to word lines). Further, each source line can be connected to only one row of memory cells. Bit lines can extend essentially perpendicular to the source lines.

FIG.9is a top plan view of a CIM device900according to an embodiment. CIM can include NVM cell array902having rows connected to ADC-MUX circuits908/912and columns connected to bit line driver906. NVM cell array902can include memory cells (one shown as910) having a cell height910H that is greater than cell width910W. Each row of memory cells can be connected to a source line (one shown as942). Each column of memory cells can be connected to a bit line (one shown as918).

A bit line918can have a bit line resistance RBLand a source line942can have a source line resistance RSL. According to embodiments, RBL>RSL. Such an arrangement can allow for greater dynamic sensing range for an integrator stage in ADC conversions as compared to conventional cases in which MAC results are provided on bit lines. In some embodiments, a bit line518can have a length bit LBLand a source line942can have a LSL, with LBL>LSL.

FIG.10is a top plan view of a portion of a NVM array1002that can be included in embodiments. NVM array1002can include memory cells (three shown as1010) arranged into rows and columns. Memory cells1010can be 2T type memory cells, including a select transistor and a SONOS type transistor. Memory cells of a same row can be connected to a same control gate WL1040-1and a same SONOS gate WL1040-0. Memory cells1010of a same column can be connected to a bit line (the location of two shown as1018). In some embodiments, memory cells can have a about a 4:1 aspect ratio, being four times longer in the column direction than in the row direction.

Source lines1042can extend parallel to word lines (1040-0/1). Source lines1042and bit lines1018can be formed by metallization layers (not shown). Alternatively, all or a portion of a source line1042can be formed by a diffusion region within a substrate. Bit lines1018can be connected to memory cells1010by bit line contacts (two shown as1074).FIG.10shows a region that includes a portion of two rows bounded by isolation structures1076.

As described herein, bit lines1018can receive input terms (Xj, Xk), which can result in the generation of currents on source lines1042. Such currents can correspond to weight values stored by SONOS type transistors in the memory cells. Currents can be analog MAC results1026′, which can be converted into digital values.

FIGS.11A and11Bare block diagrams showing operations of a CIM device according to embodiments.FIGS.11A and11Bshow a CIM device1100having a CIM array1102, row driver1104, bit line driver1106, and data buffer1179. A CIM array1102can include one or more NVM arrays according to any of the embodiments disclosed herein, or equivalents. Row driver1140can drive word lines of CIM array1102to various potential to access (e.g., generate a MAC result) and program (including erase) memory cells therein. Similarly, bit line driver1106can bit lines of CIM array1102to various potential to access and program memory cells therein. Data buffer1179can store data values that are to be driven on the bit lines by bit line driver1106.

FIG.11Ashows a programming operation that can load weight sets (kernels) into rows of CIM array1102. According to weight values1120-0to -y stored in data buffer1179, bit lines can be driven by bit line driver1178-0according to programming operations1178-0. In some embodiments, bit lines can be driven based on a weight value to be stored in a memory cell. Row driver1104can also drive word lines according to programming operations1178-1. Like bit lines, in some embodiments, word lines can be driven by row driver1104according to weight values to be stored. Source lines (not shown) can also be driven in a programming operation. As but one example, a source line of a row to be programmed can be driven to a different potential than source lines of rows that are not programmed. It is understood that such a programming step can program memory cells to any of multiple states to provide a current that can vary over a range of values.

In some embodiments, kernels can be programmed on a row-by-row basis. A set of weight values (KERNEL0)1120-0can be programmed into one row, followed by a next set (KERNEL1)1120-1programmed into a next row, etc. Kernels can be of various sizes. Accordingly, a row can fit more than one kernel and/or a kernel may fill only a portion of a row.

FIG.11Bshows a MAC generation operation according to an embodiment. InFIG.11Bit is assumed that the kernels1120-0to -y have been programmed into CIM array1002. According to input values1124-0to -p stored in data buffer1179, bit lines can be driven by bit line driver1178-2. However, bit lines can be driven to voltages according to MAC operations1178-2, which can be different from those for program operations. As described herein, in some embodiments, input values (1124-0to -p) can be binary values. Row driver1104can drive word lines to select one or more rows to receive the input values on bit lines. Such row driver1104operations can be MAC operations, which again, can differ from program operations.

Input values (1124-0to -p) driven on bit lines can result in currents flowing between bit lines and memory cells of a row, to generate an analog MAC result (1126-0′ to -y′) on source lines. Various input values (1124-0to -p) can be applied to weight sets selected by row driver1104.

According to embodiments, CIM arrays can be configured to generate MAC results for neurons of an artificial neural network (NN). In particular, input values to a NN layer can be applied, and MAC results for each neuron of NN layer can be generated on a different row of the CIM array.

FIGS.12A and12Bshow how different sets of input values can be applied to a CIM array to generate MAC results for layers in different NN1282-0/1.FIGS.12A and12Bshow a CIM device1200with a CIM array1202having rows connected to MUXs1208-0to -3. CIM array1202stores weight values for different neurons on rows connected to different MUXs. In the embodiment shown, NN1282-0includes a hidden layer formed by neurons H11, H12and H13. Weights for these neurons1220-0,1220-1and1220-2are stored in rows connected to MUXs1208-0,1208-1and1208-2, respectively.

FIG.12Ashows input values X1, X2, X3for NN1282-0applied via bit lines to CIM array1202. MUXs1208-0to -3can be configured to select rows corresponding to the hidden layer neurons. Thus, a MAC result (H11sum) corresponding to neuron H11can be generated with weight set H11, and output via MUX1208-0. In a similar fashion, weight set H12can generate MAC result (H12) corresponding to neuron H12which is output by MUX1208-1. Weight set H13can generate MAC result (H12) corresponding to neuron H12which is output by MUX1208-1.

FIG.12Bshows the same CIM array1202generating MAC results for a layer of a different NN1282-1. MUXs1208-0to -3can switch rows corresponding to the hidden layer (H21, H22, H23, H24) to output MAC results (H21sum, H22sum, H23sum, H24sum) for such neurons.

The various MAC results can be analog results and can be converted into digital values by ADC circuits (not shown).

FIGS.12C and12Dshow how a CIM array can execute MAC results for layers in a same NN1282-2with iterative operations.FIGS.12C/D show a CIM device1200having a structure like that ofFIGS.12A/B, but further shows ADC circuits1212-0to -2connected to outputs of MUXs1208-0to -2, respectively, as well as an optional activation function circuit1284and a data buffer1279.

NN1282-2can include a hidden layer formed by neurons H31, H32and H33and an output layer neuron OUT. Weights for the hidden layer neurons1220-0,1220-1and1220-2are stored in rows connected to MUXs1208-0,1208-1and1208-2, respectively. Weights for output neuron OUT can be stored in another row connected to MUX1208-0.

FIG.12Cshows input values X1, X2, X3applied via bit lines to CIM array1202to generate corresponding MAC results. Such MAC results can be converted into digital values by ADC circuits (1212-0to -2), and optionally, applied to activation function(s)1284corresponding to each neuron. Activation can include any suitable function, including but not limited to, a sigmoid, tanh, rectified linear unit (ReLU), exponential linear unit (ELU), or maxout function. Activation functions1284can be implemented in any suitable manner, including but not limited to, a processor executing instructions, custom logic or programmable logic. Alternatively, activation functions can be implemented with analog circuits prior to ADC conversion.

Final neuron outputs can be stored in input buffer1279as values H31out, H32out and H33out.

FIG.12Dshows output values from hidden layer (H31out, H32out and H33out) being applied as input values for weights1220-4of output neuron OUT. This can generate a corresponding analog MAC results output by MUX1208-0to ADC circuit1212-0. Optionally, a resulting digital value can be applied to an activation function1284′. A resulting digital value can be an NN output value1226for NN1282-2.

FIG.13shows a CIM architecture1386according to an embodiment. A CIM architecture1386can be realized as a system-on-chip (SoC) integrated circuit device, composed of one or more integrated circuit substrates formed in a package.

Architecture1386can include a configurable portion1393and a control portion1390in communication over a bus system1395. A configurable portion1393can include processing elements (PEs) (one shown as1392) formed within a configurable fabric to enable the PEs to be interconnected to one another as desired. A PE1392can include one or more CIM devices, as disclosed herein, or equivalents. APE1392can further include additional circuits for enabling functions related to generating MAC results, including but not limited to, processing input values before they are applied to generate MAC results, as well as processing digital MAC results. In some embodiments, PE blocks (e.g.,1392) can all be formed with a single monolithic die. In the embodiments shown, a configurable fabric can include configurable buses (one shown1388-0) and switch blocks (1388-1). Configurable buses1388-0can enable programmable connections to inputs and/or outputs of PEs. Switch blocks1388-1can enable programmable connections between configurable buses1388-0.

A control portion1390can include a pooling block1390-0, an accumulation unit1390-1, an activation block1390-2, a global CPU1390-3and memory (SRAM)1390-4. A pooling block1390-0can perform pooling operations on data values, including but not limited to, aggregating data values sets according to a pooling feature. Pooling features can include but are not limited to, deriving a maximum value, a minimum value, an average value, or a mean value for a data set. Accumulation unit1390-1can combine outputs from multiple PEs with a bias value and generate outputs which can be further used either by one or more PEs or given as an input to pooling block1390-0or activation block1390-2. Activation block1390-2can perform activation functions (e.g., ReLu, tanh, sigmoid, etc.) on the output generated by PEs and its output can be fed to pooling block1390-0or PEs. A global CPU1390-3can control architecture1386based on instructions and manages internal operation of the architecture1386, which in some embodiments can be a single integrated circuit. Memory1390-4can be used by architecture for any suitable function, including by not limited to, storing configuration data for configuring buses1388-0and switch blocks1388-1, and weights for PEs. In the embodiment shown, memory can include SRAM, but embodiment can include any suitable memory type.

FIG.14is a block diagram of a PE1492according to an embodiment. A PE1492can be included in an architecture like that ofFIG.13. A PE1492can include CIM blocks (four shown as1494), a control block1492-0, I/O interface1492-1, input activation buffer1492-2, input traffic control1492-3, an input bus system1492-4, an output bus system1492-5, data path control1492-6, accumulation section1492-7, output activation section1492-8, output buffer1492-9, and timing control block1492-10.

CIM blocks1494can include a CIM array1402, bit line driver1406, MUXs1408, word line drivers1040-0/1, and ADCs1412. Such items can take the form of any of those described herein or equivalents. In addition, CIM blocks1494can include a page latch1494-0, integrator/amplifier circuits1494-1, shift/add circuits1492-2, and block registers1494-3. Page latch1494-0can store a page of data from CIM array1402. CIM array1402can be programmed with non-binary weight data. While programming the weights in the CIM array1402, the input weight data can be stored in a page latch1494-0, which is then used during programming operations. In inference operation, the integrator/amplifier circuits1494-1can integrate current values provide by MUXs1408and amplify a resulting value prior to ADC conversion.

Shift/add circuits1492-2can modify digital MAC results. Block registers1494-3can store output values of CIM blocks1494, for subsequent output from PE1492. In some embodiments, block registers1494-3can also store input data to CIM blocks1494. Thus, output data from one CIM block1494can be provided as input data to another CIM block1494.

A control block1492-0can control operations of a PE1492. In some embodiments, a control block can include a CPU and corresponding ROM and RAM. I/O interface1492-1can receive input data for PE1492. Input activation buffer1492-2can store input data, and in some embodiments can selectively enable input data to be applied to CIM arrays, based on predetermined criteria. In some embodiments, input activation buffer1492-4can serve as one or more neuron input activation functions. Input traffic control1492-3can control which input data are applied to which CIM blocks1494. Input traffic control1492-3can steer input data with any suitable method, including but not limited to, by a destination value accompanying the input data or time division multiplexing. Input bus system1492-4can provide a data path for input data to each CIM block1494. Output bus system1492-5can provide a data path for output from each CIM block1494. Input and output bus systems1492-4/5can be parallel buses, serial buses, or a combination of both.

Data path control1492-6can selectively pass output data from CIM blocks1494. Accumulation section1492-7can accumulate output values from CIM blocks1494. Output activation buffer1492-8can store output data for PE1492and perform activation operations (e.g., ReLu, tanh, sigmoid). In some embodiments, output activation buffer1492-8can serve as one or more neuron output activation functions. In some embodiments, output activation buffer1492-8can perform other operations on output data, including pooling or other aggregation or filtering functions. Output buffer1492-9can drive output data on output connections (e.g., on configurable switch fabric). Timing and control block1492-10can generate timing and control signal for coordinating operations of PE1492.

FIG.15includes diagrams showing how a CIM architecture1592can be configured to realize a NN1582. CIM architecture1592can include PEs1592-0to -4, configurable buses1588-00to -04and switch blocks1588-10to -13. In the embodiment shown, one PE1592-0can have a CIM array programmed with weight values corresponding to some neurons (H1, H2) of a hidden layer. PE1592-3can be programmed with weight values corresponding to another neuron (H3) of the same hidden layer. PE1592-1can have a CIM array programmed with weight values corresponding to a follow-on layer to the hidden layer. In the embodiment shown, this can be an output layer neuron (OUT).

Input values X1, X2, X3can be provided as input values to PE1592-0by configuring switch blocks1588-13,1588-10and configurable bus1588-01. Input values X1, X2, X3can also be provided as input values to PE1592-3by configuring switch block1588-13and configurable bus1588-03. PE1592-0can be configured to execute MAC operations on input values X1, X2, X3corresponding to neurons H1, H2. In some embodiments, PE1592-0can also execute input and/or output activation functions for the neurons H1, H2. Similarly, PE1592-3can be configured to execute MAC operations on input values X1, X2, X3corresponding to neuron H3, and optionally, execute input and/or output activation functions for the neurons H3.

Output values from PE1592-0, corresponding to outputs of neurons H1, H2, can be provided as input values to PE1592-1by configuring switch block1588-11and configurable buses1588-03and1588-02. Output values from PE1592-3, corresponding to the output of neuron H3can be provided as input values to PE1592-1by configuring switch blocks1588-14,1588-11and configurable buses1588-02and1588-04. PE1592-1can be configured to execute MAC operations corresponding to neuron OUT. In some embodiments, PE1592-1can also execute input and/or output activation functions for neuron OUT. Output values from PE1592-1, corresponding to an output from NN1582, can be connected to an output of a system, or to another PE by configuring configurable bus1588-05and switch block1588-12.

While embodiments above have shown various systems, devices and corresponding methods, additional methods will be described with reference to flow diagrams.

FIG.16is a flow diagram of a MAC generation method1696according to an embodiment. A method1696can include storing multiplier values in NVM cells of an NVM array1696-0. Such an action can include programming NVM cells to store values having a range of greater than two. When selected in a MAC generation operation, a NVM cell can generate a current corresponding to a product of its stored multiplier and an input value.

A method1696can include applying multiplicand values to columns of the NVM array to generate currents for rows of NVM cells1696-1. Such an action can include applying multiplicand values via bit lines of an NMV cell array. Currents for NVM cells of selected rows can be combined to generate MAC results of multiplicand/multiplier pairs1696-2. Selected rows can be connected to ADCs with MUXs1696-3. Such an operation can enable an ADC circuit to be shared by multiple rows of NVM cells. Currents of selected rows can be converted into digital values by ADCs1696-4. Such conversion can be according to any suitable method and can include integrating the current with an integrating capacitor. As noted herein, for embodiments having both positive and negative multiplier values (e.g., weights), an ADC conversion can be one step taking an overall current generated by positive and negative weights. However, in other embodiments, conversion can be a two-step process, converting a positive weight value, converting the corresponding negative weight value, and subtracting the negative weight value from the positive weight value.

FIG.17is a flow diagram1796of a method for generating MAC results with input (e.g., multiplicand values) that can have different polarities. A method1796can include programming NVM cell pairs with positive and negative weights. Such an action can include programming two weight values for each expected input value, where the positive weight value will increase a MAC result while the negative weight value will decrease the MAC result. A method1796can include applying input values on bit line pairs that vary according to polarity1796-1. Such an action can include driving bit lines pairs to different voltages based on whether the input value is positive or negative. In some embodiments, for a positive input value, a first bit line can be driven to a relatively high voltage, while a second bit line can be driven to a relatively low voltage. For a negative input values, voltage levels on the bit line pair can be switched.

Access devices in NVM cells can be enabled to cause current to flow through the NVM cells from bit lines to row source lines1796-2. In some embodiments, such an action can include activating word lines for rows of NVM cells. Further, source lines can be dedicated to rows. A method1796can include multiplexing one of multiple source lines to an ADC circuit1796-3. Currents on source lines can be converted into digital values1796-4.

FIG.18shows a method1896of performing NN operations on a CIM device according to an embodiment. A method1896can include programming weight values for neurons of a NN layer into rows of a CIM array of NVM cells1896-0. Input values for the NN layer can be applied to columns of the CIM array to generate row currents. Row currents can correspond to a MAC result for the neurons of the NN layer1896-1.

Currents on CIM rows can be converted into digital results1896-2. Activation functions can be applied to the digital results1896-3. The digital results can then be applied as input values for CIM rows corresponding to a different layer of the NN, or to a different NN1896-4. Such an action can include enabling programmable paths between different CIM arrays of a CIM device.

Embodiments can provide various advantages over conventional approaches.

The number of MUXs used to connect analog currents to ADC circuits can be reduced, particularly if NVM cell aspect ratio is greater in the column direction than the row direction. For NVM arrays having NVM cells with a 4:1 aspect ratio, a number of MUXs can be reduced by about a factor of four.

Embodiments can provide for a shorter integration paths for ADC conversions. In conventional approaches, a MAC current value can be provided on bit lines. In contrast, embodiments can provide such a current on source lines, which can be shorter than bit lines.

Similarly, embodiments can provide greater integrating current range, as a source line can have a lower resistance than a bit line. For NVM arrays having NVM cells with a 4:1 aspect ratio, the IR drop in the conversion path can be reduced by about a factor of four.

According to embodiments, MAC operations can be executed with signed input values without having to repeat positive and negative columns in an array, providing for more efficient use of CIM array space.

According to embodiments, a CIM device can have increased accuracy over conventional approaches. Weight value sets (e.g., kernels) can be updated on a row-wise basis. This can enable a weight value set to be updated with a single programming operation, as opposed to multiple such operations when one kernal's weight sets are stored in multiple columns and rows. Along these same lines, updating weight values can be simpler and faster.

Other advantages would be well understood by those skilled in the arts.

Embodiments can enjoy wide applications in various fields to provide fast, easily updated MAC operations, in a highly parallel fashion. Embodiments can be advantageously employed to provide neural networks which can execute fast, power efficient inference operations. Further, neural networks can undergo faster learning operations as neuron weight values can be quickly and accurately updated.

Other applications would be well understood by those skilled in the arts.

It should be appreciated that reference throughout this specification to “one embodiment” or “an embodiment” means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Therefore, it is emphasized and should be appreciated that two or more references to “an embodiment” or “one embodiment” or “an alternative embodiment” in various portions of this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures or characteristics may be combined as suitable in one or more embodiments of the invention.

Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure aiding in the understanding of one or more of the various inventive aspects. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claims require more features than are expressly recited in each claim. Rather, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.