Patent Publication Number: US-11379714-B2

Title: Architecture of in-memory computing memory device for use in artificial neuron

Description:
CROSS-REFERENCE TO RELATED APPLICATION 
     This application claims priority under 35 USC 119(e) to U.S. provisional application No. 62/677,189, filed on May 29, 2018, the content of which is incorporated herein by reference in its entirety. This application also claims priority under 35 USC 119(e) to U.S. provisional application No. 62/728,753, filed on Sep. 8, 2018, the content of which is incorporated herein by reference in its entirety. 
    
    
     BACKGROUND OF THE INVENTION 
     Field of the Invention 
     The invention relates to in-memory computing (IMC), and more particularly, to the architecture of IMC memory device suitable for use in artificial neurons. 
     Description of the Related Art 
     An artificial neural network (ANN) is based on a collection of connected neurons. When processing and propagating input signals, the input values (hereinafter called “synapse values”) supplied to the neuron&#39;s synapses are each modulated by the synapses&#39; respective weight values. The effect of this process is to pass a portion of the synapse value through the synapse, which is proportional to the weight value. In this way, the weight value modulates the connection strength of the synapse. The result is then summed with the other similarly processed synapse values. Respective neurons receive the weighted input from the neuron in the previous stage and calculate the sum of the products. A propagation function for each neuron can be described mathematically as follows: r=Σ i=0   M W i *X i  where r is the output value of a given neuron&#39;s propagation function, “Xi” is the synapse value supplied/inputted to the neuron&#39;s synapse i, W i  is the weight value for modulating the synapse value at the neuron&#39;s synapse i, and the total number of the neuron&#39;s synapses is (M+1). 
     At present, neural networks are often executed by simulation software, using personal computers. However, as the size of the network increases, the software becomes more complex and the processing time increases. On the other hand, the drop in RAM prices in the current market contributes to the increasing popularity of in-memory computing technology. This has made in-memory computing economical among a wide variety of applications. In-memory computing (IMC) stores data in RAM instead of hard disks. This eliminates the I/O requirements and speeds data access because RAM-stored data is available instantaneously, while data stored on disks is limited by disk speeds. RAM storage and parallelization are two key features of IMC. The Applicant is making use of this technology in the artificial neural network. 
     What is needed is an IMC memory device capable of being parallel accessed and processing at high speed and with low power consumption. 
     SUMMARY OF THE INVENTION 
     In view of the above-mentioned problems, an object of the invention is to provide an in-memory computing (IMC) memory device using a digital DAC-bias loop to provide a constant current source and a voltage bias of the constant current source, eliminating the need of an analog bandgap circuit. 
     One embodiment of the invention provides an in-memory computing (IMC) memory device. The IMC memory device comprises an array of memory cells, a plurality of first word lines, a plurality of first bit lines, (M+1) input circuits, a first wordline driver and an evaluation circuitry. The array of memory cells is arranged in rows and columns and vertically divided into (M+1) lanes. The first word lines are arranged corresponding to the respective memory cell rows, each connected to the memory cells in a corresponding row. The first bit lines are arranged corresponding to the respective memory cell columns, each connected to the memory cells in a corresponding column. The (M+1) input circuits have (M+1) data input terminals and are coupled to the first bit lines. Each lane comprises P memory cell columns and a corresponding input circuit. P memory cells in each row for each lane stores a weight value W i . The input circuit in each lane charges a predefined first bit line with a default amount of charge proportional to an input synapse value X i  at its data input terminal and then distributes the default amount of charge to the other second bit lines with a predefined ratio based on a constant current. The first wordline driver activates one of the first word lines to retain a final amount of charge in each first bit line that is equivalent to a first product of its distributed amount of charge and a bit value stored in its corresponding memory cell. The evaluation circuitry is configured to selectively couple a selected number of the first bit lines to an accumulate line and convert an average voltage at the accumulate line into a digital value in response to a set of (M+1) input synapse values at the (M+1) data input terminals and the activated first word line. The average voltage is associated with the accumulation of the final amount of charge in each selected first bit line. 
     Another embodiment of the invention provides an in-memory computing memory block applied in an artificial neuron. The memory block comprises Q1 in-memory computing (IMC) memory devices that are arranged in a matrix of Q2 rows and Q3 columns, where Q1&gt;1, Q2&gt;0 and Q3&gt;0. 
     Further scope of the applicability of the present invention will become apparent from the detailed description given hereinafter. However, it should be understood that the detailed description and specific examples, while indicating preferred embodiments of the invention, are given by way of illustration only, since various changes and modifications within the spirit and scope of the invention will become apparent to those skilled in the art from this detailed description. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The present invention will become more fully understood from the detailed description given hereinbelow and the accompanying drawings which are given by way of illustration only, and thus are not limitative of the present invention, and wherein: 
         FIG. 1A  is a schematic diagram showing an IMC SRAM device according to an embodiment of the invention. 
         FIG. 1B  is an enlarged view of Lane i in  FIG. 1A . 
         FIG. 1C  is a circuit diagram showing a SRAM cell  110  according to an embodiment of the invention. 
         FIG. 2A  is a diagram showing the digital DAC-bias loop according to an embodiment of the invention. 
         FIGS. 2B and 2C  show two examples of charge sharing mechanism with a binary-weighted ratio based on multiple amounts of charge Q i , 1/2Q i , 1/4Q i , and 1/8Q i . 
         FIG. 3A  is a schematic diagram showing a DAC according to an embodiment of the invention. 
         FIG. 3B  is an exemplary time diagram showing the clock counts in DTC  310  and three time-based signals XON 1 ˜XON 3 . 
         FIG. 4A  is a schematic diagram showing an ADC according to an embodiment of the invention. 
         FIG. 4B  is a timing diagram showing six voltage signals V YL , V Yref , EVϕ 3 , EVϕ 4 , SA_OUT and T 2 D_IN. 
         FIG. 5  is a schematic diagram showing an accumulation circuit according to an embodiment of the invention. 
         FIGS. 6A-6C  are three exemplary configurations showing four IMC SRAM devices of 64-input×128-row are organized to from a logical IMC memory block with different numbers of inputs (64 inputs, 128 inputs, 256 inputs) according to an embodiment of the invention. 
         FIG. 6D  is an exemplary configuration showing four IMC SRAM devices of 64-input×128-row are organized to form two independent logical IMC memory blocks of 64-input×256-row according to another embodiment of the invention. 
     
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
     As used herein and in the claims, the term “and/or” includes any and all combinations of one or more of the associated listed items. The use of the terms “a” and “an” and “the” and similar referents in the context of describing the invention are to be construed to cover both the singular and the plural, unless otherwise indicated herein or clearly contradicted by context. 
     A feature of the invention is to use a digital DAC-bias loop to provide a constant current source I DACI  and a voltage bias V ibias  of the constant current source I DACI , eliminating the need of an analog bandgap circuit. Another feature of the invention is to charge a predefined computing bit line XB in each lane to the amount of charge Q i  by a DAC that generates a time-based signal XON with a pulse width (PW) (or pulse on period). Here, the amount of charge Qi and the pulse width are proportional to its input data (i.e., synapse value) X i . Another feature of the invention is to use a charge sharing mechanism to distribute the amount of charge Qi to the other computing bit lines in the corresponding lane with a binary-weighted ratio, a uniform weighted ratio or a hybrid ratio. Another feature of the invention is that an access to the memory cells for reading and writing is able to be conducted concurrently with an access to the memory cells for IMC operations due to dedicated computing word lines, dedicated computing bit lines and a dedicated output port (Y out ). Another feature of the invention is that the bit length of the weight values W i  is adjustable according to a charge weighting ratio of the computing bit lines in each lane, with which the charge sharing mechanism distributes the amount of charge Qi among the computing bit lines. Another feature of the invention is that an output voltage of the DAC  190  is adjustable according to the bit length of the weight values W i . 
       FIG. 1A  is a schematic diagram showing an IMC SRAM device according to an embodiment of the invention. Referring to  FIG. 1A , an IMC SRAM device  100  of the invention, suitable for use in artificial neurons, includes a SRAM array  10 , an analog to digital converter (ADC)  120 , a computing wordline driver  130 , a wordline driver  140 , an accumulation circuit  160 , a voltage/current adjusting circuit  170 , a column peripheral circuit  180  and M digital to analog converters (DACs)  190 . The SRAM array  10  is used to store (N+1) groups/rows of coefficients (i.e., weight values) and there are (M+1) coefficients in each group/row of coefficients. In  FIGS. 1A and 1B , four SRAM cells  110  in each row of each lane store a coefficient W i . It should be noted that storing one coefficient W i  in the four SRAM cells  110  in each row of each lane is provided by way of example and not limitations of the invention. In an alternative embodiment, the number of SRAM cells  110  in each row of each lane storing a corresponding coefficient W i  may be two, six, eight, or sixteen. For purposes of clarity and ease of description, hereinafter, the following embodiments and examples are described with the assumption that four SRAM cells  110  in each row in each lane store a corresponding coefficient W i . 
     The SRAM array  10  includes (M+1)×4×(N+1) SRAM cells  110  organized in a matrix of columns and rows with (N+1) word lines (WL j ), (N+1) computing word lines (xWL j ), (M+1)×4 bit line pairs (B ik , /B ik ) and (M+1)×4 computing bit lines (xB ik ), where 0&lt;=i&lt;=M, 0&lt;=j&lt;=N, and 0&lt;=k&lt;=3. For ease of illustration, however, only several computing word lines and several computing bit lines are illustrated in the SRAM array  10  of  FIG. 1A . The column peripheral circuit  180  controls selecting the column of the SRAM array  10  for reading and writing the coefficients. The column peripheral circuit  180  may include, without limitation, pre-charge circuitry, write drivers, read drivers and sense amplifiers. The computing wordline driver  130  activates a computing word line (xWL j ) for activating group/row j of coefficients to compute the result of a given neuron&#39;s propagation function (Σ i=0   M W i *X i ) based on a first address signal (not shown). The wordline driver  140  activates a word line (WL n ) for reading the coefficients from or writing the coefficients to the SRAM array  10  based on a second address signal (not shown), where 0&lt;=n&lt;=N. Due to dedicated computing word lines, dedicated computing bit lines and a dedicated output port (Y out ), the invention allows a concurrent SRAM and IMC access, i.e., an access to the memory cells for reading and writing able to be conducted concurrently with an access to the memory cells for IMC operations (i.e., computing the result of the propagation function). 
     The IMC SRAM device  100  may be a memory component for an associated microprocessor, digital signal processor (DSP), application-specific integrated circuit (ASIC) or larger electronic apparatus. Signal paths and a data interface of the associated apparatus may be coupled to the computing wordline driver  130 , the wordline driver  140  and the column peripheral circuit  180  of the IMC SRAM device  100  to send address information and retrieve/send data for reading/writing the coefficients to the SRAM cells  110  and for computing the result of the propagation function. Those who skilled in the art will understand coupling of the IMC SRAM device  100  to the associated apparatus. 
     Since four SRAM cells  110  in each row of each lane store a coefficient W i , the SRAM array  10  is divided into (M+1) lanes, each lane having four columns of SRAM cells  110 , a digital-to-analog converter (DAC)  190 , four computing bit lines, four bit line pairs, four switches  12   t  and three switches  12   b , as shown in  FIG. 1B .  FIG. 1B  is an enlarged view of Lane i in  FIG. 1A .  FIG. 1C  is a circuit diagram showing a SRAM cell  110  according to an embodiment of the invention. Referring to  FIG. 1C , a SRAM cell  110  of the invention includes a six-transistor (6T) SRAM cell  112  (including two inverters inv 1  and inv 2  and two transistors T 1  and T 2 ) and two transistors MX 1  and MX 2 . The 6T SRAM cell  112  is a traditional SRAM cell, so the read/write operations of the 6T SRAM cell  112  are omitted herein for the sake of brevity. The 6T SRAM cell  112  is implemented by a 1-port 6T or dual-port 6T SRAM cell. In one embodiment, the 6T SRAM cell  112  is replaced with an 8T SRAM cell. In an alternative embodiment, the 6T SRAM cell  112  is replaced with a 10T SRAM cell. Please note that the 6T SRAM cell  112  is an example of a memory cell. The memory cell can be a bitcell in accordance with any of a variety of different technologies. 
     Referring to  FIGS. 1B and 1C , each 6T SRAM cell  112  is used to store the value W ij [k] of a bit with number k in a coefficient W ij , where 0=&lt;k&lt;=3, i denotes a lane number and j denotes a row number in the SRAM array  10 . The transistor MX 2  is connected between the transistor MX 1  and the ground node, and its gate is connected to a corresponding computing word line xWL j . The transistor MX 1  is connected between a corresponding computing bit line xB ik  and the transistor MX 2 , and its gate is connected to an output node n 1  of a latch formed by the two cross-coupled inverters inv 1  and inv 2 . The two transistors MX 1  and MX 2  are used to discharge the capacitor C xB  (not shown) of the computing bit line xB ik , which evaluates the product of W ij [k] and Q i [k], where Q i [k] denotes the amount of charge in the capacitor C xB  (not shown) of the computing bit line xB ik . For example, if a computing word line (e.g., xWL 1 ) is asserted/activated by the computing wordline driver  130  and the synapse values X i  are inputted to the SRAM array  10 , group/row  1  of coefficients is selected to compute the result of the propagation function; if the value W i1 [k] in a 6T SRAM cell  112  is equal to 0, its transistors MX 1  and MX 2  discharge the capacitor C xB  of the computing bit line xB ik  to the ground; otherwise, the computing bit line xB ik  retains the amount of charge Q i [k] in its capacitor C xB . Asserting/activating the computing word line xWL 1  is equivalent to evaluating the product of W i1 [k] and Q i [k] for group  1  of coefficients. In  FIG. 1A , in a normal mode, (M+1)×4 control signals AE ik  are applied to (M+1)×4 switches  12   t  to turn on all the switches  12   t  for connecting all the computing bit lines (xB ik ) to the horizontal line YL, where 0&lt;=i&lt;=M, and 0&lt;=k&lt;=3. After all the computing bit line xB ik  are connected to the horizontal line YL, the average voltage of the line YL is calculated as follows: V YL =(Σ i=0   M W i *Q i )/(C xB *(M+1)*4), where (M+1)×4 is the total number of the computing bit lines (xB ik ) and C xB  denotes the capacitance of each computing bit line. According to a reference voltage V ref  at a reference computing bit line Y ref , the ADC  120  converts the average voltage V YL  into a digital value Y mea . According to two predefined weights C 1  and C 2 , the accumulation circuit  160  receives the digital value Y mea  for a currently selected/activated computing word line, calculates an accumulate value and outputs one of the digital value Y mea  and the accumulate value as an output digital value Y out  (will be described later). Please note that the accumulation circuit  160  is optional, and thus it is represented by dashed lines in  FIG. 1A . 
     Without using an analog bandgap circuit, the invention uses a digital DAC-bias loop to provide a constant current source I DACI  and a bias voltage V ibias  of the constant current source I DACI  for all the DACs  190 . Based on the digital DAC-bias loop, the constant current source I DACI  and its bias voltage V ibias  are insensitive to PVT variation.  FIG. 2A  is a diagram showing the digital DAC-bias loop according to an embodiment of the invention. Referring to  FIG. 2A , the digital DAC-bias loop includes a multiplexer  173 , a DAC  190 , a switch  12   t , a computing bit line xB ik  with a capacitor C xB , an ADC  120  and a voltage/current adjusting circuit  170 . The voltage/current adjusting circuit  170  includes a current controller  171 , a charge pump  172  and a capacitor C ibias . The digital DAC-bias loop is a digital-controlled closed loop that adjusts a constant current source I DACI  from the charge pump  172  and a bias voltage V ibias  of the capacitor C ibias  so that the input digital value D in  of the DAC  190  and the output digital value Y mea  from the ADC  120  are close to each other. For example, in a calibration mode, the value X tst  is set to its maximum value (e.g., 15) by the current controller  171  and selected as the output D in  to the DAC  190  via the MUX  173 ; the DAC  190  then charges the capacitor C xB  with a current I DACO  (see  FIG. 3A ) so that an analog voltage V YL  is produced at the horizontal line YL. Afterward, the ADC  120  outputs the digital value Y mea  to the current controller  171  according to the analog voltage V YL . If the value Y mea  is greater than the value X tst , the current controller  171  increases the pulse width of the signal DN to decrease the amplitude of I DACI ; otherwise, the current controller  171  increases the pulse width of the signal UP to increase the amplitude of I DACI . Meanwhile, the current controller  171  also adjusts the digital value X tst . In this manner, the steps are repeated until the value Y mea  is equal to the value X tst . In a normal mode, the input synapse value X M  is selected as the output D in  to the DAC  190  via the MUX  173 . In an embodiment, the capacitor C ibias  is implemented by a metal-oxide-metal (MOM) capacitor. 
     As indicated above, each lane includes a DAC  190  and four computing bit lines (xB i0 ˜xB i3 ) as shown in  FIGS. 1A and 1B . At first, the DAC  190  charges the capacitor C xB  of the first computing bit lines xB i3  with the amount of charge Q i  proportional to X i  in each lane. Then, the invention uses a charge sharing mechanism to distribute the charge Q i  to other computing bit lines (xB i0 ˜xB i2 ) with a binary-weighted ratio, a uniform-weighted ratio or a hybrid ratio.  FIGS. 2B and 2C  show two examples of charge sharing mechanism with a binary-weighted ratio based on multiple amounts of charge Q i , 1/2Q i , 1/4Q i  and 1/8Q i . For the example of  FIG. 2B , after the synapse values X i  are inputted, the charge sharing process is conducted and divided into four phases as follows. Phase ϕ 1 - 1 : the DAC  190  charges the capacitor C xB  of the first computing bit lines xB i3  with the amount of charge Q i  proportional to X i  in each lane and the capacitors C xB  of the other computing bit lines (xB i2 ˜xB i0 ) are empty. Here, when the capacitor C xB  of the first computing bit lines xB i3  is full, its maximum amount of charge is equal to Q Max , where Q i &lt;=Q Max . Phase ϕ 1 - 2 : the charge stored in the capacitor C xB  of the first computing bit lines xB i3  is distributed to the second computing bit lines xB i2  so that the amounts of charge stored in the capacitor C xB  of the first computing bit lines xB i3  and in the capacitor C xB  of the second computing bit lines xB i2  are 1/2*Q i . Phase φ 1 - 3 : the charge stored in the capacitor C xB  of the second computing bit lines xB i2  is distributed to the second computing bit lines xB i1  so that the amounts of charge stored in the capacitor C xB  of the second computing bit lines xB i2  and in the capacitor C xB  of the third computing bit lines xB i1  are 1/4*Q i . Phase φ 1 - 4 : the charge stored in the capacitor C xB  of the third computing bit lines xB i1  is distributed to the fourth computing bit lines xB i0  so that the amounts of charge stored in the capacitor C xB  of the third computing bit lines xB i1  and in the capacitor C xB  of the fourth computing bit lines xB i0  are 1/4*Q i . Thus, the multiplication operation of W ij *Q i  in lane i and row j is performed as follows: W ij *Q i =(Σ k=0   3  W ij [k]*Q i [k])=W ij [3]*1/2*Q i +W ij [2]*1/4*Q i +W ij [1]*1/8*Q i +W ij [0]*1/8*Q i =Q i *(1/2*W ij [3]+1/4*W ij [2]+1/8*W ij [1]+1/8*W ij [0]). 
     For the example of  FIG. 2C , after the synapse values Xi are inputted, the charge sharing process is conducted and divided into three phases as follows. Phase ϕ 2 - 1 : the DAC  190  simultaneously charges the two capacitors C xB  of the first computing bit lines xB i3  and the second computing bit lines xB i2  with the amount of charge 2*Q i  proportional to X i  in each lane and the capacitors C xB  of the other computing bit lines (xB i1 ˜xB i0 ) are empty. Here, when the capacitors C xB  of the first computing bit lines xB i3  and the second computing bit lines xB i2  are full, their maximum amounts of charge are equal to 2*Q Max , where 2*Q i &lt;=2*Q Max . Phase ϕ 2 - 2 : the charge stored in the capacitor C xB  of the second computing bit lines xB i2  is distributed to the third computing bit lines xB i1  so that the amounts of charge stored in the capacitor C xB  of the second computing bit lines xB i2  and in the capacitor C xB  of the third computing bit lines xB i1  are 1/2*Q i . Phase ϕ 2 - 3 : the charge stored in the capacitor C xB  of the third computing bit lines xB i1  is distributed to the fourth computing bit lines xB i0  so that the amounts of charge stored in the capacitor C xB  of the third computing bit lines xB i1  and in the capacitor C xB  of the fourth computing bit lines xB i0  are 1/4*Q i . Thus, the multiplication of W ij *Q i  in lane i and row j is performed as follows: W ij *Q i =(Σ k=0   3  W ij [k]*Q i [k])=W ij [3]*Q i +W ij [2]*1/2*Q i +W ij [1]*1/4*Q i +W ij [0]*1/4*Q i =Q i *(W ij [3]+1/2*W ij [2]+1/4*W ij [1]+1/4*W ij [0]). 
     The invention utilizes the charge sharing mechanism to achieve a charging weighting ratio of 1/2:1/4:1/8:1/8 in  FIG. 2B  and a charging weighting ratio of 1:1/2:1/4:1/4 in  FIG. 2C  for the four computing bit lines in each lane; the ratios are equivalent to a binary-weighted ratio of 4:2:1:1 after normalization. In an alternative embodiment, similar to  FIGS. 2B and 2C , the invention utilizes the charge sharing mechanism to achieve a charging weighting ratio of 1/4:1/4:1/4:1/4 for the four computing bit lines in each lane; the ratio is equivalent to a uniform-weighted ratio of 1:1:1:1 after normalization. Likewise, the invention may utilize the charge sharing mechanism to achieve a charging weighting ratio of 3/8:3/8:1/8:1/8 for the four computing bit lines in each lane; the ratio is equivalent to a hybrid ratio of 3:3:1:1 after normalization. 
     As to the uniform-weighted ratio of 1:1:1:1, the four computing bit lines xB ik  together with four SRAM cells  110  in each row of each lane are used to represent its corresponding weight value W i  ranging from 0 to 4 (equivalent to 2-bit binary numbers Wi). In this example, the bit length of the weight values W i  is two. As shown in Table 1, the W ij [k] value in each of the four SRAM cells  110  in each row for each lane has the same weight of 1 assigned to it, where 0&lt;=k&lt;=3. 
     
       
         
           
               
               
               
               
               
             
               
                 TABLE 1 
               
               
                   
               
               
                 W ij [3] × 1 
                 W ij [2] × 1 
                 W ij [1] × 1 
                 W ij [0] × 1 
                 W i  value 
               
               
                   
               
             
            
               
                 0 
                 0 
                 0 
                 0 
                 0 
               
               
                 0 
                 0 
                 0 
                 1 
                 1 
               
               
                 0 
                 0 
                 1 
                 0 
                 1 
               
               
                 0 
                 0 
                 1 
                 1 
                 2 
               
               
                 0 
                 1 
                 1 
                 0 
                 2 
               
            
           
           
               
            
               
                 - - - 
               
            
           
           
               
               
               
               
               
            
               
                 1 
                 1 
                 1 
                 0 
                 3 
               
               
                 1 
                 1 
                 1 
                 1 
                 4 
               
               
                   
               
            
           
         
       
     
     If the bit length of the weight values Wi equal to two is long enough, the binary-weighted ratio of 1:1:1:1 would be applied to the four computing bit lines xB ik  for each lane so that there are only two voltage states (representing two digits 0 and 1) for each computing bit line. Thus, it saves power consumption of the DAC  190 . In a scenario that V xB =0.5V, the DAC  190  operates in a low power mode and each computing bit line has two voltage states (0V and 0.5V) to represent two different digits (0 and 1). 
     As to the binary-weighted ratio of 4:2:1:1, the four computing bit lines xB ik  together with four SRAM cells  110  in each row of each lane are used to represent the weight value Wi ranging from 0 to 8 (equivalent to 3-bit binary numbers Wi). In this example, the bit length of the weight values W i  is three. As shown in Table 2, the W ij [k] value in each of the four SRAM cells  110  in each row for each lane has a corresponding weight (4, 2, 1 or 1) assigned to it, where 0&lt;=k&lt;=3. 
     
       
         
           
               
               
               
               
               
             
               
                 TABLE 2 
               
               
                   
               
               
                 W ij [3] × 4 
                 W ij [2] × 2 
                 W ij [1] × 1 
                 W ij [0] × 1 
                 W i  value 
               
               
                   
               
             
            
               
                 0 
                 0 
                 0 
                 0 
                 0 
               
               
                 0 
                 0 
                 0 
                 1 
                 1 
               
               
                 0 
                 0 
                 1 
                 0 
                 1 
               
               
                 0 
                 0 
                 1 
                 1 
                 2 
               
               
                 0 
                 1 
                 0 
                 0 
                 2 
               
               
                 0 
                 1 
                 0 
                 1 
                 3 
               
            
           
           
               
            
               
                 - - - 
               
            
           
           
               
               
               
               
               
            
               
                 1 
                 1 
                 1 
                 0 
                 7 
               
               
                 1 
                 1 
                 1 
                 1 
                 8 
               
               
                   
               
            
           
         
       
     
     In a case that the binary-weighted ratio of 4:2:1:1 is applied to the four computing bit lines xB ik  for each lane, because there are up to four voltage states (representing four digital values 0, 1, 2 and 4) for each computing bit line, the output voltage V XB  of the DAC  190  in  FIG. 3  has to be further raised compared to the case of two voltage states (representing two digital values 0 and 1) for each computing bit line, otherwise, it would be difficult to discriminate among the four voltage states of each computing bit line. In a scenario that V XB =1.1V, the DAC  190  operates in a high power mode and each computing bit line has four voltage states (0V, 0.5V, 0.8V and 1.1V) to represent four different digital values (0, 1, 2, 4). In this scenario, power consumption is greater than that in the case of the binary-weighted ratio equal to 1:1:1:1. 
       FIG. 3A  is a schematic diagram showing a DAC according to an embodiment of the invention. Referring to  FIG. 3A , the DAC  190  includes a digital to time converter (DTC)  310  and a time to analog converter (TAC)  320 . The DAC  190  converts a synapse value X i  in a floating point form, e.g., X i =mantissa[5:0]*2 (0+exponent[1:0]) , into an analog voltage V xB . Each synapse value X i  is represented in a floating point form for a greater value range and keeping the relative error constant. Its mantissa part (mantissa[5:0]) is used to control the pulse width (PW) (or pulse on period) of a time-based pulse signal XON. Specifically, the DTC  310  converts the mantissa part (mantissa[5:0]) of the synapse value X i  into a time-based pulse signal XON with its pulse width proportional to its mantissa part (mantissa[5:0]). According to the exponent part (exponent[1:0]) of Xi, the TAC  320  converts the time-based signal XON into the output voltage V xB . Specifically, the signal XON is used to control the switch S 3  so that at least one of the four current sources (I 1 ˜I 4 ) can charge the capacitor C xB  of the computing bit line xB i3  to the voltage V xB  in the TAC  320 . The exponent part (exponent[1:0]) of X i  are respectively used to control the switches S 1  and S 2 , i.e., controlling the amount of charging current. Referring to  FIGS. 2A and 3A , I DACI =I 1 +I 2 +I 3 +I 4  and I 1 =I 2 =I 3 =I 4 . For a steady flow of charge through a surface, the amount of charge Q xB  transferred through the surface over a time period T is defined by: Q xB =I DACO *T=I DACO *Xi.mantissa[5:0]*2 (0+xi.exponent[1:0]) *UnitTime 1 , where I DACO  denotes the current passing through the switch S 3  and UnitTime 1  denotes the internal clock cycle in DTC  310 . The less the UnitTime 1 , the more precise the amount of charge Q xB  is and the more difficult it is to implement the circuit. 
       FIG. 3B  is an exemplary time diagram showing clock counts in DTC  310  and three time-based pulse signals XON 1 ˜XON 3 . Referring to  FIG. 3B , three time-based pulse signals XON 1 ˜XON 3  corresponding to three synapse values X i  are provided by way of example. In a case that X i =23, it can be expressed in a floating point form in the following way: X i =23*2 (0+0) ; after its mantissa part (mantissa[5:0]=23) is inputted to the DTC  310 , the DTC  310  produces the time-based pulse signal XON 1  with its pulse width equal to 23 clock cycles. In a case that X i =15, it can be expressed in a floating point form in the following way: X i =15*2 (0+0) ; after its mantissa part (mantissa[5:0]=15) is inputted to the DTC  310 , the DTC  310  produces the time-based pulse signal XON 2  with its pulse width equal to fifteen clock cycles. In a case that X i =2, it can be expressed in a floating point form in the following way: X i =2*2 (0+0) ; after its mantissa part (mantissa[5:0]=2) is inputted to the DTC  310 , the DTC  310  produces the time-based pulse signal XON 3  with its pulse width equal to two clock cycles. 
       FIG. 4A  is a schematic diagram showing an ADC according to an embodiment of the invention. Referring to  FIG. 4A , the ADC  120  includes an analog to time converter (ATC)  410  and a time to digital converter (TDC)  420 . The ATC  410  includes a sense amplifier  411 , two AND gates  412 , an OR gate  413  and an inverter  415 . The TDC  420  includes a digital counter  421 , a storage device  422  and a subtractor  423 . The sense amplifier  411  compares the average voltage V YL  at the horizontal line YL with the voltage ramp V Yref  at a reference computing bit line Y ref  to produce an output signal SA_OUT. The voltage ramp V Yref  is obtained by charging the reference computing bit line Y ref  with the output current from the DAC  190 . To cancel the input offset voltage V os  of the sense amplifier  411 , an evaluation-enable signal EVϕ 3  related to a sensing period ϕ 3  and an evaluation-enable signal EVϕ 4  related to a sensing period ϕ 4  are applied for measuring the digital value Y mea . 
       FIG. 4B  is a timing diagram showing six voltage signals V YL , V Yref , EVϕ 3 , EVϕ 4 , SA_OUT and T 2 D_IN. Referring to  FIGS. 4A and 4B , after the synapse values Xi are inputted to the IMC SRAM device  100 , a computing word line is activated and the average voltage V YL  at the line YL is obtained, an evaluation-enable signal EVϕ 3  is applied to the switch S 3  in the DAC  190  connected to the reference computing bit line Y ref  and an evaluation-enable signal EVϕ 4  is applied to the switch S 3  in the DAC  190  connected to one computing bit line, such as xB 00 . Please be noted that the reference computing bit line Y ref  is selected from the computing bit lines xB ik  (not xB 00 ) and the reference computing bit line Y ref  has to be discharged to the ground voltage before the evaluation-enable signal EVϕ 3  is applied to the switch S 3  in the DAC  190  connected to the reference computing bit line Y ref . 
     During the sensing period ϕ 3 , the switch S 3  connected to the reference computing bit line Y ref  is turned on, the switch S 3  connected to the computing bit line xB 00  and the switch  12   t  are turned off and the non-inverting input terminal and the inverting input terminal of the sense amplifier  411  are respectively connected to the lines YL and Y ref . Accordingly, the capacitor C xB  of the reference computing bit line Y ref  is charged by the DAC  190 , so the voltage V Yref  increases as time elapses. When the voltage V YL  is greater than the voltage V Yref , the output signal SA_OUT of the sense amplifier  411  is at the “high” voltage state. As soon as the voltage V Yref  is greater than the voltage V YL , the output signal SA_OUT of the sense amplifier  411  flips to the ground voltage state. After the voltage V Yref  reaches its maximum voltage V BLMAX , the sensing period ϕ 3  of the evaluation-enable signal EVϕ 3  is terminated and then the sensing period ϕ 4  starts. During the sensing period ϕ 4 , the switches S 3  connected to the computing bit line xB 00  and the switch  12   t  are turned on, the switch S 3  connected to the reference computing bit line Y ref  is turned off and the inverting input terminal and the non-inverting input terminal of the sense amplifier  411  are respectively connected to the lines YL and Y ref . Accordingly, the capacitor C xB  of the reference computing it line YL is charged by the DAC  190 , so the voltage V YL  increases as time elapses. As soon as the voltage V YL  is greater than the voltage V Yref , the output signal SA_OUT of the sense amplifier  411  flips back to the high voltage state. 
     Mathematically, the amount of charge Q on each plate of a capacitor is defined by Q=C*V, where C denotes the capacitance of the capacitor and V denotes the voltage between the plates. As mentioned above, the amount of charge Q transferred through the surface over a time period T is defined by: Q=I*T. Accordingly, Q=I*T=&gt;T=C*V/I. During the sensing period ϕ 3 , PRD 1  is the time period that that it takes for the voltage V Yref  to increase from 0V to V YL (=V AVE ). Thus, PRD 1 =C xB *(V YL −V os )/I DACO , where V os  denotes the input offset voltage of the sense amplifier  411 . During the sensing period ϕ 4 , PRD 2  is the time period that it takes for the voltage V YL  to increase from V YL (=V AVE ) to V BLMAX . Thus, PRD 2 =C xB *(V BLmax −(V YL +V os ))/I DACO . According to the input signal T 2 D_IN, the digital counter  421  firstly measures the first pulse on period PRD 1  to supply a digital output D 1  to the storage device  422 , and then measures the second pulse on period PRD 2  to produce a digital output D 2 . Afterward, the subtractor  423  subtracts D 2  from D 1  to produce Y mea . Thus, Y mea =D 1 −D 2 ; PRD 1 −PRD 2 =Y mea *UnitTiMe 2 =C xB *(2*V YL −V BLmax )/I DACO , where UnitTime 2  denotes the input clock cycle for the digital counter  421 . After D 2  is subtracted from D 1 , the result (Y mea ) of computing the propagation function (Σ i=0   M W i *X i ) is obtained and the input offset voltage V os  is cancelled. 
       FIG. 5  is a schematic diagram showing an accumulation circuit according to an embodiment of the invention. Referring to  FIG. 5 , the accumulation circuit  160  includes a storage device  161 , two multipliers  162  and an adder  163 . According to two predefined weights (C 1  and C 2 ), the accumulation circuit  160  receives the digital value Y mea  for a currently selected/activated computing word line, calculates an accumulate value (=C 1 *Y mea (n−1)+C 2 *Y mea (n)) and outputs one of the digital value Y mea  and the accumulate value as an output digital value Y out (n). Specifically, the storage device  161  receives a first input digital value Y mea (n) from the ADC  120  and a second input digital value (C 1 *Y mea (n−1)+C 2 *Y mea (n)) from the adder  163 , supplies its previous digital value Y mea (n−1) to the lower multiplier  162  and outputs one of the first and the second input digital values as the output digital value Y out (n) according to a control signal CS. In one embodiment, C 2  is varied according to the currently activated computing word line and C 1  is varied according to its previously activated computing word line. Please note that each of the storage devices  161  and  422  includes, without limitation, a D-flip-flop, a latch and a memory device. 
       FIGS. 6A-6C  are three exemplary configurations showing four same IMC SRAM devices of 64-input×128-row are organized to form a logical IMC memory block with different numbers of inputs (64 inputs, 128 inputs, 256 inputs) according to an embodiment of the invention. In the embodiment of  FIG. 6A , four same IMC SRAM devices  100  are arranged in a column and the sixty-four input terminals for synapse values X 0 ˜X 63  of the four IMC SRAM devices  100  of 64-input×128-row (i.e., M=63, N=127) are respectively connected together to form a logical IMC memory block  610  of 64 synapse value inputs and 512 rows. In this specification, “the sixty-four input terminals for synapse values X 0 ˜X 63  of the four IMC SRAM devices  100  are respectively connected together” indicates that four input terminals X 0  are connected together, four input terminals X 1  are connected together, . . . , and four input terminals X 63  are connected together, but the four input terminals X 0  are isolated from the other input terminals X 1 ˜X 63 , the four input terminals X 1  are isolated from the other input terminals X 0 , X 2 ˜X 63 , . . . , and the four input terminals X 63  are isolated from the other input terminals X 0 ˜X 62 . Accordingly, when a set of synapse values X 0 ˜X 63  are inputted to the sixty-four input terminals of the logical IMC memory block  610 , the four IMC SRAM devices  100  operate in parallel to produce four output digital values Y out . Thus, the embodiment achieves a goal of parallel processing. 
     In the embodiment of  FIG. 6B , the four same IMC SRAM devices  100  are organized to form a logical IMC memory block  620  of 128-input×256-row (i.e., M=127, N=255). In  FIG. 6B , two of the four IMC SRAM devices  100  are arranged in a left column while the other two IMC SRAM devices  100  are arranged in a right column. The sixty-four input terminals for synapse values X 0 ˜X 63  of the two IMC SRAM devices  100  in the left column (left group) are respectively connected together while the sixty-four input terminals for synapse values X 64 ˜X 127  of the two IMC SRAM devices  100  in the right column (right group) are respectively connected together. Accordingly, after a set of synapse values X 0 ˜X 127  are inputted to the logical IMC memory block  620 , the four IMC SRAM devices  100  operate in parallel to produce four output digital values. Specifically, when a first half (X 0 ˜X 63 ) of the set of synapse values X 0 ˜X 127  are inputted to the input terminals of the left column, the two IMC SRAM devices  100  in the left column operate in parallel to produce two left output digital values Y outL (=Σ i=0   63  W i *X i ); when a second half (X 64 ˜X 127 ) of the set of synapse values X 0 ˜X 127  are inputted to the input terminals of the right column, the two IMC SRAM devices  100  in the right column operate in parallel to produce two right output digital values Y outR  (=Σ i=64   127  W i *X i ). The four output digital values may be temporarily stored in another SRAM for further computations. Alternatively, one of the two left output digital values Y outL  and one of the two right output digital values Y outR  are summed up to generate the final result: Y outS =Σ i=0   127  W i *X i  (=Σ i=0   63  W i *X i +Σ i=64   127  W i *X i ). Thus, the embodiment achieves a goal of parallel processing and extending the total number of the neuron&#39;s synapses from 64 to 128. 
     In the embodiment of  FIG. 6C , the four same IMC SRAM devices  100  are organized in a row to form a logical IMC memory block  630  of 256-input×128-row. In  FIG. 6C , after a set of synapse values X 0 ˜X 255  are inputted to the logical IMC memory block  630 , the four IMC SRAM devices  100  operate in parallel to produce four output digital values. Specifically, a first quarter (X 0 ˜X 63 ) of a set of synapse values X 0 ˜X 255  are inputted to the sixty-four input terminals of the left-most IMC SRAM device  100  to produce a first output digital value Y out1 =(=Σ i=0   63  W i *X i ); a second quarter (X 64 ˜X 127 ) of the set of synapse values X 0 ˜X 255  are inputted to the sixty-four input terminals of the middle-left IMC SRAM device  100  to produce a second output digital value Y out2  (=Σ i=64   127  W i *X i ); a third quarter (X 128 ˜X 191 ) of the set of synapse values X 0 ˜X 255  are inputted to the sixty-four input terminals of the middle-right IMC SRAM device  100  to produce a third output digital value Y out3  (=Σ i=128   191  W i *X i ); a fourth quarter (X 191 ˜X 255 ) of the set of synapse values X 0 ˜X 255  are inputted to the sixty-four input terminals of the rightmost IMC SRAM device  100  to produce a fourth different output digital value Y out4  (=Σ i=192   255  W i *X i ). The four different output digital values (Y out1 , Y out2 , Y out3 , Y out4 ) of the four IMC SRAM devices  100  in  FIG. 6C  may be temporarily stored in another SRAM for further computations, or be summed up to generate the final result Y outS =Σ i=0   255  W i *X i . Thus, the embodiment achieves a goal of parallel processing and extending the total number of the neuron&#39;s synapses from 64 to 255. 
       FIG. 6D  is an exemplary configuration showing four same IMC SRAM devices of 64-input×128-row are organized to form a logic IMC memory block comprising two sub-blocks of 64-input×256-row according to an embodiment of the invention. Referring to  FIG. 6D , the logic IMC memory block  640  includes two separate/isolated sub-blocks  640 L and  640 R. Two of the four IMC SRAM devices  100  are arranged in a left column to form a left sub-block  640 L while the other two IMC SRAM devices  100  are arranged in a right column to form a right sub-block  640 R. The sixty-four input terminals for synapse values XL 0 ˜XL 63  of the two IMC SRAM devices  100  in the left sub-block  640 L are respectively connected together while the sixty-four input terminals for synapse values XR 0 ˜XR 63  of the two IMC SRAM devices  100  in the right sub-blocks  640 L are respectively connected together. After a set of synapse values (XL 0 ˜XL 63 ) are inputted to the logical IMC sub-block  640 L, its two IMC SRAM devices  100  operate in parallel to produce two output digital values Y outL  (=Σ i=0   63  W i *XL i ). After a set of synapse values (XR 0 ˜XR 63 ) are inputted to the logical IMC sub-block  640 R, its two IMC SRAM devices  100  operate in parallel to produce two output digital values Y outR  (=Σ i=0   63  W i *XR i ). Since the two logical IMC sub-blocks  640 L and  640 R are separate/isolated, they are allowed to be configured to store two different bit lengths of W i . For example, the two IMC SRAM devices  100  in the IMC sub-blocks  640 L are configured to store 2-bit W i  values and the two IMC SRAM devices  100  in the IMC sub-blocks  640 R are configured to store 3-bit W i  values. Thus, the embodiment achieves a goal of parallel processing and processing the W i  values with different bit lengths. 
     Please note that the four IMC SRAM devices  100  forming the IMC memory blocks in  FIGS. 6A ˜ 6 D are provided by way of example and not limitations of the invention. In actual implementations, any other numbers of IMC SRAM devices  100  can be used to form an IMC memory block and this also falls in the scope of the invention. The logical IMC memory blocks ( 610 ˜ 640 ) are suitable for use in artificial neurons, 
     Referring back to  FIG. 1A , in a partial average mode, the control signals AE ik  are respectively applied to the switches  12   t  to turn on a portion of the switches  12   t  and turn off the other switches  12   t ; thus, only a portion of the computing bit lines (xB ik ) are correspondingly connected to the horizontal line YL, instead of all the computing bit lines being connected to the horizontal line YL. In an embodiment of the partial average mode, an IMC SRAM device  100  of 64-input×128-row (i.e., M=63, N=127) is logically divided into two IMC sub-arrays L and R of 32-input×128-row (i.e., M=31, N=127) (not shown). In this embodiment, the coefficients (weight values) for sub-array L are stored in W ij [k], where 0&lt;=i&lt;=31, and 0&lt;=k&lt;=3; meanwhile, the coefficients for sub-array R are stored in W ij [k], where 32&lt;=i&lt;=63, and 0&lt;=k&lt;=3. In order for the IMC operation in the sub-array L to be performed, a set of thirty-two synapse values XL 0 ˜XL 31  are inputted to the thirty-two input terminals of the sub-array L and only the switches  12   t  inside the sub-array L need to be turned on (with the other switches  12   t  being turned off) via their corresponding control signals AE ik . In order for the IMC operation in the sub-array R to be performed, a set of thirty-two synapse values XR 0 ˜XR 31  are inputted to the thirty-two input terminals of the sub-array R, and only the switches  12   t  inside the sub-array R need to be turned on (with the other switches  12   t  being turned off) via their corresponding control signals AE ik . 
     While certain exemplary embodiments have been described and shown in the accompanying drawings, it is to be understood that such embodiments are merely illustrative of and not restrictive on the broad invention, and that this invention should not be limited to the specific construction and arrangement shown and described, since various other modifications may occur to those ordinarily skilled in the art.