Patent Description:
Integrated Circuits (ICs) are widespread in modern electronics and may be used to implement a wide range of processing and memory devices. In-memory computing technology is a developing area which aims to provide improvements in computational performance. Traditional systems tend to store data in memory that is separate from the processor which performs tasks such as arithmetic and logic functions. With the increase in data required for certain applications, such as with neural network processing, data movement between the processor and memory may present one of the more critical performance and energy bottlenecks. In-memory computing can improve processing performance through the use of memory technologies that are also able to perform certain computational tasks such as the arithmetic and/or logical functions.

<NPL>,discloses computing-in-memory based on embedded nonvolatile memory is a promising candidate for energy-efficient multiply-and-accumulate operations in artificial intelligence edge devices. This document discloses a method of serial input per row comprising:.

<NPL>, discloses an in-memory computing macro. The macro is an module with the circuits embedded in bitcells and peripherals to perform hardware acceleration for neural networks with binarized weights and activations.

<CIT>, discloses systems and methods for reducing power in in-memory computing, matrix-vector computations, and neural networks.

<NPL>, discloses a multi-bit precision in-memory computing architecture, based on the voltage scaling and charge sharing scheme, for artificial intelligence edge devices.

Optional features or steps are set out by the dependent claims.

Example embodiments will now be described, with reference to the accompanying drawings, in which:.

Example embodiments relate to integrated circuits (ICs) and integrated circuit fabrication techniques applicable to In-Memory Computing (IMC), also sometimes referred to as Computing-in-Memory (CIM) or Processing-in-Memory (PIM). More specifically, the present disclosure describes techniques for implementing a multiply accumulate (MAC) unit useful for In-Memory Computing.

The multiply-accumulate (MAC) operation is a building block of a variety of computational processes used in digital signal processing and machine learning (ML) algorithms. For example, the convolution function, also sometimes referred to as cross-correlation or sliding dot product, is produced by determining the integral of the product of the two functions for all possible values of shift. Convolution is used in a variety of computational operations, including signal processing, image processing, neural networks, and others.

In computer systems, convolution is a mathematical operation between two matrices often referred as an input window and a filter. For example, given a 10x10 matrix (the input window) and a 3x3 matrix (the filter), the process is to multiply the input window and the filter for each possible combination of positions as you superimpose the filter matrix over the input matrix. This can be accomplished by overlaying the filter at the first position, multiplying the overlapping values and adding them together to get the output, referred to herein as a dot product or partial sum. Next, the filter is shifted one or more column/row over and the same calculations are repeated. Once at the end of the row/column, the filter is shifted one or more row/column and the process repeated. This continues until the end of the input matrix is reached.

The efficiency of performing the multiple-accumulate operation often determines the overall performance of a hardware accelerator design. Some digital designs use an SRAM to store the matrix operands and digital logic to perform arithmetic operations. In such designs, the memory access presents a major bottleneck and often dominates power consumption.

By contrast, SRAM based in-memory computing circuits can use a dense array of SRAM-based bit cells, to perform a massive number of multiplication and summation operations simultaneously in mixed digital and analog domain. Performing computational operations at the memory cell removes the memory access bottleneck and greatly reduces the power consumption of accessing data. Accordingly, in-memory computing circuits can offer orders of magnitude higher computational throughput and energy efficiency as compared to digital logic designs. The computation performed by mixed signal circuits also presents a significant power and area saving.

However, many in-memory computing designs suffer from a limited precision. For example, many in-memory computing designs can only handle operands of <NUM> bits or less without losing accuracy. The present disclose describes various techniques that improve the in-memory computing design to achieve a higher precision and accuracy.

There are two broad categories of CIM computing mechanisms, current domain and charge domain techniques. In both techniques, the multiplication is performed within each bitcell while the accumulation is performed inter-bitcell (row, column, or both). In current domain CIM designs, the accumulation is achieved by applying multiple discharging paths to a constant supply or a single charge reservoir, where the total current or total charge displaced through timed current is used to represent the MAC result. In charge domain CIM designs, multiple charged/depleted capacitors are used to redistribute or couple the stored charges to achieve an analog voltage that represents the MAC result.

Multi-bit precision input can be represented in the current domain more easily than in charge domain, making it faster than multi-bit precision charge domain designs. However, current domain designs suffer more severely than charge domain designs from non-idealities such as IR drop and manufacturing variability, especially dopant variability. Due to the inherent noise in current, current domain techniques tend to be less accurate than charge domain techniques.

Charge domain techniques have the advantage of accuracy over the current domain techniques, since the accumulation mainly suffers geometric variability of the capacitors which is lower than current. However, multi-bit precision input cannot be represented by pulse width in charge domain designs. Instead, a multi-cycle operation is used, making it a slower approach.

Despite the higher accuracy, most charge domain CIM designs are used for low precision applications. The main issue is within bit-significance representation of both inputs and weights: the finer the bit-significance, the higher the energy used to perform CIM as the unit capacitance has physical limitations. Furthermore, the conventional circuitry used to integrate charges of differing bit-significance grows exponentially in area and energy cost. This patent presents novel techniques and methods to overcome those issues.

Embodiments of the present techniques provide a MAC accelerator apparatus with charge domain CIM modules. The top-level accelerator manages the data organization and mapping of various forms of machine learning layers to the CIM macros in order to perform MAC operations with multibit inputs, multibit weights, and multibit output precisions. The CIM modules compute in the charge domain, where the inputs are applied to the macro bit-serially whereas the weight bits are distributed across multiple columns of memory bitcells. The bitcells perform a binary multiplication in the digital domain, and the result determines the state of an output capacitor (charged or depleted). The output capacitors for a column of bitcells determine the charge stored to a holding capacitor for that column. The holding capacitors for adjacent bitcell columns form a capacitive voltage divider, resulting in an analog voltage that represents the MAC result. Sample-and-hold (S&H) circuits at the end of each column accumulate over multiple input cycles, representing the sum of each binary MAC result, accounting for the input bit significance. The capacitive voltage dividers form a C-2C ladder, which accumulate these charges over the columns to represent the final MAC results, which are summations of the partial results for each bit of the weights. The final analog voltage may then be digitized using a successive approximation register (SAR) Analog to Digital converter (ADC). Embodiments of the present techniques may be better understood with reference to <FIG> and the accompanying text.

<FIG> is a block diagram of a compute-in-memory (CIM) module for performing a multiple-and-accumulate operation in accordance with embodiments. The CIM module <NUM> includes a bitcell array <NUM> that stores weight data and also performs bitwise multiplication of each weight and corresponding inputs from an input window. The state of each bitcell <NUM> determines whether it represents a value of zero or a value of one. Although only two bitcells <NUM> are shown, it will be appreciated that the bitcells array <NUM> can include any suitable number of bitcells, for example, several hundred bitcells. Bitcell states may be set or reset by a bit line Read/Write circuitry <NUM>. Each bitcell <NUM> in the bitcell array <NUM> is configured to multiply its stored weight bit with a corresponding bit of input received from an input buffer <NUM>. As used herein, the term input or input value is used to refer to the entire N-bit input, whereas the term input bit refers to a single bit of the input value. Similarly, the term weight or weight value is used to refer to the entire N-bit weight, whereas the term weight bit refers to a single bit of the weight value.

In the embodiment shown in <FIG>, the bitcell array <NUM> is divided into regions <NUM>, each of which can have a set of columns representing a different filter. Although only one region <NUM> is depicted, it will be appreciated that the bitcell array <NUM> may be divided into several regions of the same size as region <NUM>. The number of columns in each region depends on the number of bits used to represent the filter weights. For purposes of the present description, each filter weight is represented by an <NUM>-bit binary number occupying <NUM> columns of the bitcell array. Thus, each region <NUM> is eight columns of bitcells wide. However, the present techniques can be implemented for any suitable bit precision, including <NUM> bits or others. The weights in a particular region <NUM> represent a single filter which has been unrolled from the matrix form into one column. In some embodiments, the filter may be copied multiple times to the same column to account for multiple channels of input. For example, in a system with a <NUM>-by-<NUM> filter matrix and four input channels, each column with have <NUM> bitcells (<NUM> weights per filter times <NUM> channels). In the bitcell array <NUM>, weight bit significance is represented by the position of the bit in the <NUM>-bit region.

Input values received from the input buffer <NUM> can have the same number of bits as the weights, but the input buffer is to serially send input bits of the multibit input to the bitcell array <NUM> one bit at a time starting with the least significant bit and ending with the most significant bit for a total of eight input cycles in the case of an <NUM>-bit system. At each input cycle, the same input bit is fed into a full row of bitcells <NUM>, but different rows can receive a different input bit depending on the input value received from the input window at that row position. The bitcells <NUM> process the input bits in parallel.

At each input cycle, each bitcell <NUM> multiplies the input bit with its own weight bit and outputs a zero charge if the result of the multiplication is zero or a non-zero charge if the result of the multiplication is one. All of the outputs in a single column are effectively summed in the analog domain through a capacitive coupling with a common capacitor plate. The total charge on the column is accumulated by holding capacitor, which is disposed at the end of the column and included in a sample-and-hold circuit <NUM>. Each column of bitcells <NUM> is associated with a separate holding capacitor, and the separate holding capacitors form a C-2C ladder that represents the resulting output in analog form. After <NUM> input cycles, the final MAC result stored to the C-2C ladder represents the multiplication of the <NUM>-bit filter weight by the <NUM>-bit input value. The final result can then be converted to a digital value using a converter <NUM>. Various components of the CIM module are described further in relation to <FIG>.

<FIG> is a circuit diagram showing two adjacent bitcells in a single column of the bitcell array in accordance with embodiments. Each bitcell <NUM> includes a bit storage unit <NUM> for storing the state of the bitcell (<NUM> or <NUM>). The bit storage unit <NUM> stores a single bit of the multibit weight. The two bitcells <NUM> shown in <FIG> represent the same bit position of two different weights. Other bits of each weight are stored to adjacent bitcells in the same row (not shown). In the example of <FIG>, the bit storage unit <NUM> is a Static Random-Access Memory (SRAM) circuit with two cross-coupled inverters positioned between two access transistors. Other types of bit storage units may also be used. The bit storage unit <NUM> has two stable states that are used to denote zero and one.

Access (read and write) to the bit storage unit <NUM> is enabled by the write word line (WWL) which controls the two access transistors. The access transistors control whether the bit storage unit <NUM> should be coupled to the write bit lines WBL and WBL(B). The write bit lines WBL and WBL(B) are bitwise complements that control the state of the bit storage unit <NUM>, i.e., whether the bit storage unit <NUM> stores a one or a zero.

The output of the bit storage unit <NUM>, Q and Q(B), is accessed at the outputs of the two cross-coupled inverters. The values on the two output lines will be bitwise complements that represent the value that was stored to the bit storage unit <NUM>. For example, if the bit storage unit <NUM> stores a value of <NUM>, then Q will equal <NUM> and Q(B) will equal zero. If the bit storage unit <NUM> stores a value of <NUM>, then Q will equal <NUM> and Q(B) will equal <NUM>.

The bitcell <NUM> also includes a multiplication unit <NUM> that performs a bitwise multiplication of the input bit and the weight bit. The multiplication unit <NUM> includes three transistors, labeled A, B, and C and is coupled to an output capacitor <NUM> that holds the charge representing the multiplication result. The multiplication unit <NUM> is controlled by the input bit on the MAC word line, MAC-WL(B), which represents the input received from the input buffer <NUM> (<FIG>) after being inverted. If the input value is <NUM>, the inverted input, MAC-WL(B), will be <NUM>, which turns on transistor A and turns off transistor B. At this point, if the bit stored to the bit storage unit is <NUM>, the non-inverted output from the bit storage unit, Q, will be stored to the output capacitor <NUM>. Otherwise, if the bit stored to the bit storage unit is <NUM>, the inverted output Q(B) will turn on transistor C, thereby connecting the output capacitor <NUM> to ground.

If the input value is <NUM>, the inverted input, MAC-WL(B), will be <NUM>, which turns off transistor A and turns on transistor B, in which case the output capacitor <NUM> will be coupled to ground regardless of the value of the bit stored to the bit storage unit <NUM>. To summarize, if the input bit and the weight bit are both <NUM>, then the output capacitor <NUM> will store a charge representing <NUM>. Otherwise, if either the input bit or the weight bit equal <NUM>, the output capacitor <NUM> will be discharged.

As mentioned above, each bitcell <NUM> includes an output capacitor <NUM> to store the multiplication result. Each output capacitor <NUM> includes a driven plate connected to the output of the multiplication unit <NUM> and a common plate which is effectively shared between the output capacitors <NUM> in the same column due to being coupled to one another through the MAC bit line, MAC-BL. The driven plate will be either set to the supply voltage or ground depending on the multiplication result of the individual bitcell <NUM>, while the common plate (MAC-BL) is allowed to settle to an intermediate voltage representing the sum of the multiplication results for all of the bitcells <NUM> in the column. The relationship between the voltage at the common plate and the ratio of high and low capacitors will be a linear function. Thus, the voltage at the common plate indicates the ratio of high to low output capacitors and, by extension, the summation of all of the multiplication results. Accordingly, the binary digital multiplication result produced by the bitcells <NUM> in a single column results in an analog voltage at the MAC bit line, MAC-BL, that represents the sum of all the multiplication results provided by the bitcells <NUM> in the column. The voltage at the MAC bit line, MAC-BL, is stored to a holding capacitor as described in relation to <FIG>.

The charge coupling technique described herein improves the speed, energy efficiency, and accuracy compared to charge sharing techniques for various reasons. In charge sharing, the charge stored to a capacitor is transferred to another capacitor. In charge coupling, a series of parallel capacitors share a common capacitive plate such that the combined charges from all of the capacitors in the series effect the voltage on the common plate. The charge coupling technique described above is an improvement over charge sharing techniques for various reasons. For example, the capacitive coupling mechanism described above is driven by the supply voltage, whereas the charge sharing technique is passive. Accordingly, the steady state voltage can be reached much faster with capacitive coupling compared to charge sharing. Additionally, charge sharing techniques use switching to transfer charges from one capacitor to another, which causes higher energy use as well as introducing charge injection error. Furthermore, in capacitive coupling, the output capacitor is not fully charged or depleted to represent the multiplication result. Rather, the driven plate is set to the supply or ground voltage while the common plate is allowed to settle to an intermediate voltage. Accordingly, the energy to charge or discharge the column of output capacitors <NUM> is lower than that of a charge sharing circuit.

<FIG> is a circuit diagram of a sample-and-hold circuit that is used to store the partial MAC result described in relation to <FIG>. The partial MAC result is the voltage on the MAC bit line, MAC-BL, after all of the bitcells in the column have performed the multiplication and the output capacitor has settled to a steady state. The partial MAC result is referred to as partial because it represents the summation of the multiplication results for all of the bitcells in a single column (i.e., single weight bit position) and a single input cycle (i.e., single input bit position). As described above, the multiplication of the inputs and the weights is conducted for each input bit individually. Accordingly, the partial MAC results are stored for each input cycle and added to the partial MAC results for each of the previous input cycles until the last bit of the input value has been processed.

The sample-and-hold circuit <NUM> includes a sampling capacitor <NUM> and the holding capacitor <NUM>, as well as a number of switches labeled A, B, and C. The sample-and-hold circuit <NUM> operates in such way that the partial MAC results at each input cycle have an effect on the final charge stored at the holding capacitor <NUM> commensurate with the corresponding input bit position. This is accomplished by processing the input values in order from the least significant bit to the most significant bit and, at each input cycle, halving the charge stored to the holding capacitor <NUM> in previous cycles.

Prior to the first input cycle, the holding capacitor <NUM> is reset by closing switch C while switch B is held open. To sample the partial MAC results, the switch A is closed while switches B and C are open, allowing the sampling capacitor <NUM> to be charged by the voltage at the MAC bit line, MAC-BL. Next, switch A is opened and switch B is closed, which causes the charges on the sampling capacitor <NUM> and the holding capacitor <NUM> to be shared equally. At the end of the first input cycle representing the least significant bit, the holding capacitor <NUM> will hold a charge equal to half the charge originally stored to the sampling capacitor <NUM> during the cycle.

Next, the sampling capacitor <NUM> and the MAC bit line, MAC-BL, are reset using the reset switch, RST_M, (<FIG>) in preparation for the next input cycle. Once multiplication results have been obtained for the next input cycle, the process repeats. Specifically, the switch A is closed while switches B and C are open, allowing the sampling capacitor <NUM> to be charged by the voltage at the MAC bit line, MAC-BL. Next, switch A is opened and switch B is closed, which causes the charges on the sampling capacitor <NUM> and the holding capacitor <NUM> to be shared equally. At this point half of the charge from the sampling capacitor <NUM> is effectively shared with the holding capacitor <NUM>, and half of the charge from the holding capacitor <NUM> is effectively shared with the sampling capacitor <NUM>. Accordingly, the charge stored to the holding capacitor <NUM> during the first input cycle will be halved again during the second input cycle, while the charge representing the second least significant bit has only been halved once. Repeating the same process for each input cycle causes each successive bit of input to have double the effect on the charge stored to the holding capacitor <NUM>. In an <NUM>-bit system for example, the least significant bit will have been halved <NUM> times, the second least significant bit will have been halved <NUM> times, and so on, while the most significant bit will have been halved only once. At the end of the last input cycle, each input cycle's contribution to the final charge stored to the holding capacitor will be proportional to the input cycle's bit significance. In this way, input bit significance is accounted for in the time domain, i. e, depending on the order by which the input bits are processed.

At the end of the eight cycles, the charge on the holding capacitor <NUM> is an analog value that represents the sum of the multiplication results for an entire column of weight bits for all eight input cycles. As described further in relation to <FIG>, the holding capacitor <NUM> will be one holding capacitor in a C-2C ladder. Each column of bitcells <NUM> will have its own holding capacitor <NUM> that stores the results for that corresponding column and forms part of the C-2C ladder.

In some embodiments, the system may use <NUM>'s compliment arithmetic. In such embodiments, the most significant bit of the input value will be inverted as described in relation to <FIG>.

<FIG> is a circuit diagram of a sample-and-hold circuit that is used to store MAC results in systems that use <NUM>'s complement arithmetic. The sample-and-hold circuit <NUM> includes the sampling capacitor <NUM>, the holding capacitor <NUM>, and switches A, B, and C, as described in <FIG>. However, the sample-and-hold circuit <NUM> of <FIG> also includes an inverter <NUM> used to invert the partial MAC result generated for the most significant bit during the last input cycle. The inverter <NUM> includes a second sampling capacitor <NUM> and additional switches labeled D, E, F, and G. The second sampling capacitor <NUM> is coupled to a biased voltage inversion circuit and is used to invert with constant bias the multiplication results received from the output capacitors during a last input cycle corresponding to a most significant bit of the input value.

As the process described in <FIG>, the holding capacitor <NUM> is reset prior to the first input cycle using switch C. In this example, the holding capacitor <NUM> is reset by closing switch C while switches B are G are is held open.

For all but the last input cycle, switches A and B operate in the same manner as described in relation to <FIG>, while switch D remains open. For the last input cycle, switches A and B remain open while the inverter <NUM> is used to sample and invert the partial MAC result, MAC-BL, before storing it to the holding capacitor <NUM>. To sample the MAC result for the last input cycle, switches G and E are held closed while switches F and D remain open, which allows the second sampling capacitor <NUM> to be charged by the voltage at the MAC bit line, MAC-BL. Next, switches G and E are switched opened and then switch F and D is closed. The voltage of across the capacitor <NUM> remains constant but shifts with respect to the rest of the circuit, i.e. the bottom plate voltage shifts from VMAC-BL to VSS (ground reference), and the top plate voltage shifts the same amount from VDD to VDD-VMAC-BL, thus inverting the voltage with a known constant bias voltage, VDD, to keep the voltage positive.

As mentioned above in relation to <FIG>, the charge on the holding capacitor <NUM> at the end of the eight input cycles is an analog value that represents the sum of the multiplication results for an entire column of weight bits and all eight input cycles.

<FIG> is a circuit diagram of a sample-and-hold circuit that uses ping-pong buffering and is used to store MAC results in systems that use <NUM>'s complement arithmetic. The sample-and-hold circuit <NUM> of <FIG> includes a first sampling circuit <NUM>, a second sampling circuit <NUM>, a first holding capacitor <NUM>, and a second holding capacitor <NUM>.

Both sampling circuits <NUM> and <NUM> operate in the same manner as the sample-and-hold circuit <NUM> described in relation to <FIG>. Switch lettering is the same for each sample-and-hold circuit, with the exception that the switches of the second sampling circuit <NUM> are designated with the prime symbol, i.e., switch A is equivalent to switch A', switch B is equivalent to switch B', etc..

As mentioned above, each input value is multiplied by the weights one input bit at a time. The processing of a single input bit takes place during a single input cycle. So for an <NUM>-bit system, the full input value is processed in <NUM> input cycles.

The effect of using two sampling circuits is that after the first sampling circuit <NUM> has sampled the MAC results for one bit of a particular input value (e.g., after <NUM> cycle), the second sampling circuit <NUM> can be operated to sample new MAC results for the next bit of the input value while the MAC results sampled to the first sampling circuit <NUM> are coupled to a holding capacitor. Thus, accumulation of the MAC result ping pongs between the two sampling circuits <NUM><NUM> for alternating input bits.

The effect of using two holding capacitors is that after completing the sample-and-hold operation of storing the MAC results for a particular input value (e.g., after <NUM> cycle), the second holding capacitor <NUM> can be operated to hold the new MAC results of the second input value, while the first holding capacitor <NUM> is coupled to ADC circuits for further processing.

For the first input value's odd cycles, the charge on the MAC bit line, MAC-BL, is coupled to the first sampling circuit <NUM> through switch A in case of non-most-significant-bit or switches D, E, and F for most-significant bit (not applicable if input bit precision is even) as described in relation to the sample-and-hold circuit <NUM> of <FIG>, while switches A' and D', E', and F' remain open, and the sampling circuit <NUM> is coupled to holding capacitor <NUM>.

For the first input value's even cycles, the charge on the MAC bit line, MAC-BL, is coupled to the second sampling circuit <NUM> through switches A' in case of non-most-significant-bit or D', E', and F' for most-significant bit (not applicable if input bit precision is odd) as described in relation to the sample-and-hold circuit <NUM> of <FIG>, while switches A and D, E, and F remain open, and the sampling circuit <NUM> is coupled to holding capacitor <NUM>.

For the first input value, the sampling circuits <NUM> and <NUM> are operated in alternate cycles, while the first holding capacitor <NUM> operates as described in relation to the sample-and-hold circuit <NUM> of <FIG> until all of the bits of the input value have been processed and the first partial MAC result has been accumulated to the first holding capacitor <NUM>. This ping-pong process between the two sampling circuits <NUM><NUM>, enables sampling of the MAC-BL voltage and charge sharing with the first holding capacitor <NUM> to proceed in parallel, thereby increasing speed at which the input can be processed.

For the second input value's odd cycles, the charge on the MAC bit line, MAC-BL, is coupled to the first sampling circuit <NUM> through switch A in case of non-most-significant-bit or switches D, E, and F for most-significant bit (not applicable if input bit precision is even) as described in relation to the sample-and-hold circuit <NUM> of <FIG>, while switches A' and D', E', and F' remain open, and the sampling circuit <NUM> is coupled to the second holding capacitor <NUM>.

For the second input value's even cycles, the charge on the MAC bit line, MAC-BL, is coupled to the second sampling circuit <NUM> through switch A' in case of non-most-significant-bit or switches D', E', and F' for most-significant bit (not applicable if input bit precision is odd) as described in relation to the sample-and-hold circuit <NUM> of <FIG>, while switches A and D, E, and F remain open, and the sampling circuit <NUM> is coupled to the second holding capacitor <NUM>.

For the second input value, the sampling circuits <NUM> and <NUM> are operated in alternate cycles, while the second holding capacitor <NUM> operates as described in relation to the sample-and-hold circuit <NUM> of <FIG> until all of the input bits of the input value have been processed and the second partial MAC result has been accumulated to the second holding capacitor <NUM>. This ping-pong process between the two sampling circuits <NUM><NUM>, enables sampling MAC-BL voltage and charge sharing with the second holding capacitor <NUM> to proceed in parallel, thereby increasing speed at which the input can be processed.

While the second input's MAC result is being accumulated to the second holding capacitor <NUM> by the sampling circuits of <NUM><NUM>, the first MAC result stored to the first holding capacitor <NUM> can simultaneously undergo further processing. After the sampling circuits <NUM><NUM> have finished accumulating the MAC result for the second input value to the second holding capacitor <NUM>, the sampling circuits <NUM><NUM> can be operated to accumulate the third MAC result to the first holding capacitor <NUM> while the second holding capacitor <NUM> undergoes further processing. This ping-pong process between the two holding capacitors <NUM><NUM> continues until the full input window has been processed. Ping-ponging between the two holding capacitors <NUM><NUM>, enables charge accumulation and analog to digital conversion of the MAC results to proceed in parallel, thereby increasing the speed at which the input window can be processed.

<FIG> is a circuit diagram of an analog MAC result circuit used to accumulate the partial MAC results generated by the bitcell array in accordance with embodiments. In the described embodiment, the MAC result circuit <NUM> is a C-2C ladder. However, it will be appreciated that other circuits may be used to accumulate the MAC results. The MAC result circuit <NUM> of <FIG> includes rungs of parallel sample-and-hold circuits <NUM> separated by capacitors <NUM> to form a series of voltage divider circuits. Each sample-and-hold circuit <NUM> may be one of the sample-and-hold circuits <NUM>, <NUM>, or <NUM> shown in <FIG>, <FIG>, and <FIG>.

During the processing of the input bits, charge is accumulated to the holding capacitor included in each of the sample-and-hold circuits <NUM>. Once all of the inputs have been processed and it is time to convert the analog MAC result to digital, the sample-and-hold circuit <NUM> is operated so that the holding capacitor is coupled into the C-2C ladder. In other words, each sample-and-hold circuit <NUM> will be appear electrically to simply be the holding capacitor <NUM> (or <NUM> or <NUM>).

Each sample-and-hold circuit <NUM> correlates with one of the columns from the bitcell array <NUM>. After the <NUM> input cycles described above, each sample-and-hold circuit <NUM> will hold a charge on its holding capacitor <NUM>, <NUM>, or <NUM> that represents the partial MAC results for a single column of bitcells, i.e., a single weight bit position. Together the sample-and-hold circuits <NUM> represent the full MAC results, Vout, for the entire <NUM>-bit weight, which can be measured at the output port <NUM>. The MAC result, Vout, is an analog voltage that can be sent to the converter <NUM>, which converts the analog value to a digital value. The converter <NUM> may the converter described in relation to <FIG>.

As mentioned above, bit significance of the weights is determined by the bit position. In this circuit, the leftmost sample-and-hold circuit <NUM> represents the most significant bit and the rightmost sample-and-hold circuit <NUM> represents the least significant bit. The arrangement of the sample-and-hold circuits <NUM> with the additional capacitors creates a voltage divider circuit so that the effect of each successive sample-and-hold circuit <NUM>, as measured at the output port <NUM>, is reduced by half moving from the most significant bit to the least significant bit. In this way, the bit significance can be represented using equally sized holding capacitors, rather than capacitors that increase exponentially in size. This enables bit-significance presentation at linear cost, with better matching characteristics and higher precision.

<FIG> is a circuit diagram of an example converter for converting the MAC results from analog to digital. The converter <NUM> may be a successive approximation register (SAR) analog to digital converter (ADC), which includes a digital to analog converter (DAC) <NUM>, a comparator <NUM>, and a controller <NUM>. The DAC <NUM> may be a C-2C ladder, which may be used to generate an analog voltage whose value is determined based on the positions of a set of switches. The controller <NUM> controls the switches of the C-2C ladder to generate an analog voltage, VREF.

The comparator <NUM> can include two inverter circuits arranged back-to-back. The reference voltage, VREF, from the DAC <NUM> is coupled to one side of the inverter circuits through a first switch while the output voltage, VOUT, from the C-2C ladder of <FIG> is coupled to the opposite side of the inverter circuits through a second switch. According to this configuration, the comparator <NUM> can be in one two states, which is determined based on which voltage is higher, VREF or VOUT.

The controller <NUM> controls the switches of the DAC <NUM> to arrive at a set of switch positions that generate an analog voltage approximating the output voltage, VOUT. The switch positions are determined in a series of steps progressing from the most significant bit (the leftmost switch) to the least significant bit (the rightmost switch). At each step, the comparator <NUM> compares the reference voltage with output voltage from the C-2C ladder of <FIG> to determine which voltage is higher and adjusts the corresponding switch accordingly.

At the end of the process, the positions of the DAC switches represent the digital value corresponding to the analog output voltage from the C-2C ladder. This digital value represents the results of the multiply accumulate operation for a single input value. The digital value may be stored by the controller <NUM> to memory or sent to another computing element for further processing.

<FIG> is a block diagram of a compute-in-memory (CIM) device with multiple compute-in-memory modules as described above in accordance with embodiments. The CIM device shown in <FIG> demonstrates the ability to combine multiple CIM modules <NUM> to handle a wide variety of system configurations. The CIM device <NUM> of <FIG> is configured to process an input window measuring <NUM> input values across, and each CIM module is configured to handle <NUM> input values. Therefore, <NUM> CIM modules are used to cover the entire <NUM> bit input window. Input is received from a vector data editor (VDE) which handles data arbitration to and from the CIM macros.

Additionally, the system is also configured to process the input using a total of <NUM> filters, while each CIM module <NUM> in this example can only hold a total of <NUM> filters. To extend the capability of the system from <NUM> filters to <NUM> filters, the CIM modules <NUM> are grouped in modules sets <NUM>, each of which includes <NUM> separate CIM modules <NUM> that share the <NUM> bits of input from the VDE.

The MAC results from each set of CIM modules <NUM> may be added together using an arithmetic unit array (AUA) <NUM>. The arithmetic unit array <NUM> is an adder tree that can be controlled to sum the MAC results in a variety of ways, depending on the system design and the desired results. In some configurations, the MAC results from each CIM module set <NUM> is received individually through the A1 multiplexer. For the sake of clarity, only one signal line to the A1 multiplexer is shown. However, it will be appreciated that additional signal lines will connect each of the CIM module sets <NUM> to the A1 multiplexer. The A1 multiplexer can report the results from each CIM module set <NUM> to the VDE <NUM> individually in serial fashion.

In some configurations, the MAC results from pairs of CIM module sets <NUM> added together at a summing node before being reported to the VDE <NUM> through the A2 multiplexer. Again, although one signal line is shown to the A2 multiplexer, there will be a signal line from each summing node corresponding with the pairs of CIM module sets. The A2 multiplexer can report the summed results from each pair of CIM module sets <NUM> to the VDE serially in <NUM> steps.

In similar fashion, the A3 multiplexer can be selected to report the summed results from half of the CIM module sets <NUM> to the VDE <NUM> in two steps. Additionally, all of the CIM module sets <NUM> may be added together and reported to the VDE <NUM> in one step. The master multiplexer <NUM> can be used to determine which reporting configuration will be applied.

It will be appreciated that the specific details described in <FIG> are only example implementation details, various changes may be made to the number of CIM module sets <NUM>, the number of CIM modules <NUM> per CIM module set <NUM>, the number of inputs per CIM module <NUM>, and others without departing from the scope as defined by the appended claims.

<FIG> is a block diagram of an example CIM system in accordance with embodiments. The CIM system <NUM> includes the CIM device <NUM> shown in <FIG>, which includes the CIM modules <NUM> and the arithmetic unit array <NUM>. The CIM device <NUM> receives input data from a vector data unit (VDU) <NUM>, which includes the vector data editor <NUM> shown in <FIG>. The vector data unit <NUM> also includes a programmable vector load/store unit (VLSU) <NUM> coupled to the vector data editor <NUM>. The VLSU <NUM> controls read and write operations and buffer the inputs or results.

The CIM system <NUM> also includes a control unit (CU) <NUM> that controls the operation of the CIM device <NUM> in accordance with program instructions stored to a program memory (PMEM) <NUM>. For example, the control unit <NUM> can generate control signals that configure the CIM modules <NUM> by setting the weight values of the bitcells included in the bitcell arrays. The control unit <NUM> can also generate signals for configuring the arithmetic unit array <NUM> to determine how the MAC results are summed and reported. The control unit <NUM> may be any suitable type of logic unit, such as a microcontroller, an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), or an arrangement of logic gates implemented in one or more integrated circuits, for example.

Input data and filter weight data may be stored to a data memory (DMEM) <NUM> and communicated to the CIM device <NUM> through the vector data editor <NUM>. The data memory <NUM> may be any volatile or non-volatile data storage device, such as a hard drive, solid state drive, and others. Memory access to the data memory <NUM> may be controlled by a data bus or arbiter <NUM>.

The CIM system <NUM> can also include a general purpose processor <NUM>, which may be any suitable type of central processing unit (CPU) or reduced instruction set computer (RISC). The processor <NUM> can be used to provide communication between the CIM system <NUM> and other computing units or peripheral devices. For example, the processor <NUM> may be coupled to a network that allows the CIM system <NUM> to communicate with additional remote computing systems and devices. The processor <NUM> may also be used to specify the parameters of the CIM operations, such as providing the input data and filter weight data to be processed. The processor <NUM> can also be used to initiate mathematical operations on the data and receive results.

<FIG> is a process flow diagram summarizing an example method for performing a multiply accumulate (MAC) operation using an in-memory computing device in accordance with embodiments. The method <NUM> is performed using a CIM module as described in any of the examples above. The CIM module may be controlled by a general purpose computer comprising one or more processors and digital memory. The CIM module may also be a part of a computing system, such as the computing system <NUM> shown in <FIG>. The method may begin at block <NUM>.

At block <NUM>, weight values are received. The weight values are multibit values having M bits each, wherein M may equal <NUM>, <NUM>, <NUM>, <NUM>, or any other suitable value including values in between. A weight value is received for each row of a bitcell array. The weights may be weights that are included in one or more filters, for example.

At block <NUM>, the weight values are used to populate a bitcell array by storing each weight bit to a memory unit of a bitcell so that the entire weight value is distributed across M adjacent columns of bitcells. The weights are positioned in the bitcell array so that the weight values of a same filter are arranged in columns, with each row being a different weight value. In embodiments in which the bitcell array can handle more than one filter, the additional filter weights may be positioned in adjacent columns.

At block <NUM>, input values are received. The input values may be values received from an input window, which may be a data file such as an image file, for example. The input values may be received for example by an input buffer of the bitcell array. The input values are multibit values having N bits each, wherein N may equal <NUM>, <NUM>, <NUM>, <NUM>, or any other suitable value including values in between. In some embodiments, the weight values and the input values have the same number of bits (M=N). However, the input and the weights may also have different numbers of bits.

At block <NUM>, a counter, i, may be set. As described above, the N input bits of each input value are fed into the bitcell array serially, i.e., one bit at a time from the least significant bit to the most significant bit. Accordingly, N input cycles will be used to generate the MAC results for each input value. As used herein, the Nth bit refers to the least significant bit and is the first bit to be processed, and bit <NUM> is the most significant bit, which is the last bit to be processed.

At block <NUM>, the sampling capacitor is reset. As described above, partial MAC results for each input bit are sampled by a sample-and-hold circuit. Prior to each sampling of the MAC bitline, MAC-BL, the sampling capacitor is reset, i.e., discharged. It will be appreciated that the sample-and-hold circuit includes more than one sampling capacitor.

At block <NUM>, each of the M weight bits of the weight value is multiplied by the ith input bit. The result for each bitcell is stored to an output capacitor. The same multiplication process may occur in parallel for each bitcell of the bitcell array.

At block <NUM>, the column of output capacitors is sampled using the sampling capacitor. As described above, each output capacitor in a column shares a common capacitor plate which is conductively coupled via the MAC bitline, MAC-BL. Accordingly, the voltage on the MAC bitline is linearly proportional to the ratio of charged and discharged output capacitors, i.e., the ratio of <NUM> to <NUM>.

At block <NUM>, the sampling capacitor is coupled to a holding capacitor that accumulates the results for each weight bit column across multiple input cycles. Coupling the sampling capacitor to the holding capacitor causes the total charge total charge stored by both the sampling capacitor and the holding capacitor is shared between both capacitors. In other words, half of the charge on the sampling capacitor is shared with the holding capacitor and half of the charge in the holding capacitor is shared with the sampling capacitor. Accordingly, for each input cycle, the holding capacitor's charge due to previous input cycles is cut in half, which accounts for the input bit significance.

If i is greater than one, then the process flow advances from block <NUM> to block <NUM>, and blocks <NUM> through <NUM> are repeated for the next input bit (i = i - <NUM>). If i equals one, then the Nth bit has been processed and the holding capacitors have accumulated the partial MAC results for each of the input bits. At this point, the analog MAC result representing the summed products of the input values and the weight values for each row of the bitcell array have been stored to the holding capacitors. The process flow may then advance to block <NUM>.

At block <NUM>, the analog MAC result stored to the holding capacitors is converted to a digital value. At the same time that the analog to digital conversion is taking place, an additional process flow may also branch off and return to block <NUM>. At block <NUM>, a new input value is received and the MAC process repeats for the new input value using different sample-and-hold circuits.

Blocks <NUM> through <NUM> may be repeated for each input value until an entire input window is processed. The results may be stored to memory as an output feature map and may be subject to further processing depending on the design details of a specific implementation.

It will be appreciated that the process described herein is a simplified version of the techniques described above. The method <NUM> may be repeated for multiple channels of input and/or multiple filters. Additionally, it will be appreciated that the processes described are performed by machinery, including in-memory computing circuits, digital circuits, or a combination thereof.

Claim 1:
An apparatus for performing a convolution comprising:
a bitcell array (<NUM>) to perform a multiply-accumulate (MAC) operation using a multibit weight and a multibit input, wherein the bitcell array comprises a plurality of bitcells (<NUM>), and wherein each bitcell of the bitcell array comprises a corresponding memory unit (<NUM>) to store a corresponding weight bit of the multibit weight, a corresponding multiplication unit (<NUM>) to multiply the corresponding weight bit by an input bit of the multibit input, and a corresponding output capacitor (<NUM>) to store a result of the multiplication;
sample-and-hold circuits (<NUM>, <NUM>, <NUM>) coupled to the output capacitors of corresponding columns of bitcells, each sample-and-hold circuit comprising a corresponding holding capacitor (<NUM>, <NUM>, <NUM>) to store a charge representing a partial MAC result for the corresponding column;
an input buffer (<NUM>) to serially send input bits of the multibit input to the bitcell array to be multiplied by the multibit weights in a series of input cycles, wherein each sample-and-hold circuit is configured to, in each input cycle of said series of input cycles:
sample a voltage on a corresponding column line coupled to the corresponding output capacitors in order to provide a sampled charge;
if the input cycle is after the first occurring input cycle of the series, reduce the charge stored at the end of the previous cycle in the corresponding holding capacitor by half;
add half of the sampled charge to the charge of the corresponding holding capacitor; and
an analog MAC result circuit (<NUM>) comprising the holding capacitors, each holding capacitor associated to a corresponding weight bit significance, the analog MAC result circuit configured to generate, from the charges stored in the holding capacitors at the end of the series of input cycles, an analog MAC result being a summed product of the multibit input and the multibit weight.