Patent Publication Number: US-6903594-B2

Title: Capacitor-free leaky integrator

Description:
CROSS REFERENCE TO RELATED APPLICATION 
   This application claims subject matter disclosed in provisional patent application Ser. No. 60/403,481 filed Aug. 13, 2002, entitled Capacitor-Free Leaky Integrator. 

   FIELD OF THE INVENTION 
   The present invention generally relates to a leaky integrator, and, more specifically, to a capacitor-free leaky integrator capable of being used in an artificial neuron. 
   BACKGROUND 
   Artificial neural networks (ANN) are used in computing environments where mathematical algorithms cannot describe a problem to be solved. ANNs are often used for speech recognition, optical character recognition, image processing, and numerous other mathematically ill-posed computation and signal processing problems. ANNs are able to learn by example and, when receiving an unrecognized input signal, can generalize based upon past experiences. 
   A given ANN is made up, at least in part, of a number of interconnected artificial neuron circuits. The output signal of a given neuron is dependent upon a series of input signals received from a group of other artificial neurons or from sensor or transducer input devices. In the case of a pulse coded ANN, the output signal changes based upon factors like the delay between a pair of input signals. The output of an artificial neuron is often called its “activation.” This “activation” ranges from no or very low activation to high-level activation. In a pulse-coded neuron the level of activation is measured in terms of the frequency and duration of output pulses which are called “action potentials” after the name given to the outputs of biological neurons. 
   The activation of a pulse-coded artificial neuron is typically based on a nonlinear spatial and temporal sum of its inputs. Spatial sum means the sum of all the inputs that are active at the same time. Temporal sum means that all the inputs are summed with a forgetting factor commonly referred to as a leaky integral. Leaky integrators are key subsystems in pulse-mode artificial neurons as they are responsible for spatially and temporally summing the input signals to determine the artificial neuron&#39;s activation. 
   A given pulse-mode artificial neural network will have dozens to hundreds of leaky integrators. Using conventional approaches, each leaky integrator circuit requires the use of one or more integrated capacitors. Compared to the size of other leaky integrator components, a capacitor is large and requires a substantial amount of space. For example a one picofarad capacitor requires on the order of 2000 square micrometers of space on an integrated circuit fabricated using a standard process. An integrated transistor, in contrast, may require less than ten square micrometers. 
   If the capacitors can be replaced with smaller components such as transistors, the size of a pulse-mode artificial neuron can be reduced. Moreover, conventional leaky integrators have fixed time constants. If the time constant can be adapted or altered, a pulse-mode artificial neuron could better mimic a biological neuron. 

   
     DESCRIPTION OF THE DRAWINGS 
       FIG. 1  is a circuit diagram of a capacitor-free leaky integrator circuit according to an embodiment of the present invention. 
       FIG. 2  is a graph illustrating pulse response as a function of delay-resistor gate bias of the circuit shown in FIG.  1 . 
       FIG. 3  is a graph illustrating variation of pulse responses as a function of input signal level of the circuit shown in FIG.  1 . 
       FIG. 4  is a graph illustrating the integration response of the circuit shown in FIG.  1 . 
   

   DETAILED DESCRIPTION 
   Introduction 
   The leaky integrator is a key subsystem in pulse-mode artificial neurons. In most implementations reported to date the leaky integrator function has been implemented with the explicit use of integrated capacitors and with fixed time constants. More recently it has been noted that mimicking real biological systems is better accomplished if the integrator time constants are adaptable and if different time constants are realized for rising and failing edges of the circuit&#39;s pulse response. 
   The present invention provides a capacitor-free leaky integrator. Integration is performed using nonlinear resistance supplied by triode-region-biased PMOS transistors operated near the weak inversion region. A large time constant can be obtained using a low-gain non-inverting amplifier to make use of the parasitic capacitance of the transistors. 
   The Circuit 
   The invented leaky integrator circuit is shown in FIG.  1 . PMOS (Positive-channel Metal Oxide Silicone) transistors M 1  and M 2 A-M 2 E provide the nonlinear resistance. The abbreviations G, S, and D stand for gate, source, and drain respectively. M 1  and M 2 A-M 2 E are referred to as “delay resistors” since it is the parasitic capacitance and RC time constants due to these components that effects the circuit&#39;s integration response characteristics. Each of these components can have identical geometry with width-to-length (W/L) ratios of one to one (1:1). It is expected that the size of each transistor M 1  and M 2 A-M 2 E will be 3 μm/3 μm or 9 2  μm. For ease of reference purposes only, M 2 A can be referred to as a starting transistor, M 2 B-M 2 D can be referred to as intermediate transistors, and M 2 E can be referred to as a terminating transistor. 
   In an alternate embodiment PMOS transistors M 2 A-M 2 E can be replaced with a single larger PMOS transistor having a W/L ratio of 1:5 (3 μm/15 μm). This results in a less “complicated” circuit. However, using five smaller PMOS transistors allows the circuit area filled by the leaky integrator to be minimized while assisting in parameter matching during circuit fabrication. 
   NMOS (Negative-channel Metal Oxide Silicone) transistors M 3 -M 5  and PMOS transistor M 6  comprise a common-gate amplifier that utilizes the parasitic capacitance of M 1  and M 2 A-M 2 E to provide differing time constants for the rising and falling edges of an output signal produced in response to a pulsed input signal. The amplifier is designed to have a nominal small signal gain of 2.5 V/V for positive-going input signals at the quiescent bias point (Q-point). M 3 , M 4 , and M 5  have W/L ratios of 4:1 (20 μm/5 μm), 10:1 (50 μm/5 μm), and 10:1 (50 μm/5 μm) respectively. Q-point drain current for M 5  is 68 nA, of which 51 nA is drawn through M 2 , when V GR  is 1.0 volts and V GA  is 2.2 volts. Under these conditions the Q-point output voltage, Vout, is 2 volts for V IN  equals 2.6 volts. PMOS load device M 6  has W/L ratio of 4:5 (4 μm/5 μm). 
   One component of  FIG. 1  can be said to be tied to another component of  FIG. 1  if a conductive path is maintained between the two components. For example, the source of M 1  is tied to the gate of M 3  and the source of M 2 A. However, the conductive path that ties two components together need not be free of other components. 
   Pulse Response 
   Dynamic response of this circuit is a function of the delay-resistor bias V GR  and the input signal level V IN . Our input Q-point design is 2.6 volts.  FIG. 2  shows the response of the circuit as a function of V GR  to a 0.25 volt input pulse over a range of V GR  from 1.0 to 1.4 volts in 0.1 volt increments. The slowest response is obtained for V GR =1.4 volts and speeds up as V GR  is decreased. The 0.25 volt input pulse is measured peak to peak and is relative to the Q-point. 
   This response is qualitatively easy to understand. The delay resistor M 2  time constant, T, goes approximately as 
       T   =       2   ⁢     L   2         μ   ⁡     [       2   ⁢     (       V   SG     -     V   T       )       -     V   SD       ]             
 
where V SG  is the source-to-gate voltage, V T  is the threshold voltage, V SD  is the source-to-drain voltage and μ is the carrier mobility. In response to a positive-going input, V SG  increases whereas the use of multiple transistors M 2 A-M 2 E keeps V SD  from matching this increase despite the gain of the amplifier. This accounts for the relatively rapid rising edge of the circuit response. On the failing edge, however, we have decreasing V SG  and consequently an increasing T on the falling edge. At higher V GR  settings the M 2 A-M 2 E transistors leave the strong inversion region resulting in a very low conductance at the Q-point.
 
   The circuit response is a function of input signal level.  FIG. 3  illustrates the response of the circuit to input pulses of 0.1 through 0.35 volts in 50 mV steps where V GR =1.3 volts. Slower responses are obtained for the lower-level input signals. Rise time response decreases up to an input of 0.25 volts. The anomalous rise time response for 0.30 volts is due to a nonlinear transfer characteristic of the amplifier and is caused by the V SG  of M 5  leaving the strong inversion region. The apparent decrease in rise time response for 0.35 volts input is due to M 5  becoming cut off. Note that the fall time response initially decreases at higher input signal levels but then slows down as the output returns to its Q-point. This behavior is consistent with that of biological neurons. 
   The circuit&#39;s behavior as an integrator is illustrated in FIG.  4 . The input signal consists of 5 equally-spaced 0.25 volt pulses with 10 nanosecond rise and fall times and a total width of 40 nanoseconds. V GR  was set at 1.3 volts. Of note is the rapid integration of the pulse inputs and the slow decay of the response tail at the cessation of the input. Also worthy of note is the fact that the tail&#39;s decay takes place over an interval of a few microseconds despite the absence of any explicitly-integrated capacitors in the circuit. 
   Conclusion 
   These results demonstrate that passive integrated capacitor components are not necessary in pulse-mode artificial neurons. The effect they are intended to model—namely, the integration of voltage in the neural membrane—can be obtained from one or more transistors. Moreover, the integrator&#39;s time constant can be adapted to better mimic biological neurons. 
   The present invention has been shown and described with reference to the foregoing exemplary embodiments. It is to be understood, however, that other forms, details, and embodiments may be made without departing from the spirit and scope of the invention that is defined in the following claims.