Winner-Take-All Networks of O(N) Complexity
Authors: J. Lazzaro, S. Ryckebusch, M.A. Mahowald, C. A. Mead
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: We have designed, fabricated, and tested a series of compact CMOS  integrated circuits that realize the winner-take-all function. These  analog, continuous-time circuits use only O(n) of interconnect to  perform this function. We have also modified the winner-take-all  circuit, realizing a circuit that computes local nonlinear inhibition. 
Two general types of inhibition mediate activity in neural systems: subtractive in(cid:173) hibition, which sets a zero level for the computation, and multiplicative (nonlinear)  inhibition, which regulates the gain of the computation. We report a physical real(cid:173) ization of general nonlinear inhibition in its extreme form, known as winner-take-all. 
We have designed and fabricated a series of compact, completely functional CMOS  integrated circuits that realize the winner-take-all function, using the full analog  nature of the medium. This circuit has been used successfully as a component  in several VLSI sensory systems that perform auditory localization (Lazzaro and  Mead, in press) and visual stereopsis (Mahowald and Delbruck, 1988). Winner(cid:173) take-all circuits with over 170 inputs function correctly in these sensory systems. 
We have also modified this global winner-take-all circuit, realizing a circuit that  computes local nonlinear inhibition. The circuit allows multiple winners in the net(cid:173) work, and is well suited for use in systems that represent a feature space topograph(cid:173) ically and that process several features in parallel. We have designed, fabricated,  and tested a CMOS integrated circuit that computes locally the winner-take-all  function of spatially ordered input. 
704 
Lazzaro, Ryckebusch, Mahowald and Mead 
THE WINNER-TAKE-ALL CmCUIT 
Figure 1 is a schematic diagram of the winner-take-all circuit. A single wire, asso(cid:173) ciated with the potential Vc, computes the inhibition for the entire circuit; for an  n neuron circuit, this wire is O(n) long. To compute the global inhibition, each  neuron k contributes a current onto this common wire, using transistor T2 a.' To  apply this global inhibition locally, each neuron responds to the common wire volt(cid:173) age Vc, using transistor Tla.' This computation is continuous in time; no clocks  are used. The circuit exhibits no hysteresis, and operates with a time constant  related to the size of the largest input. The output representation of the circuit  is not binary; the winning output encodes the logarithm of its associated input. 
Figure 1. Schematic diagram of the winner-take-all circuit. Each neuron receives  a unidirectional current input 11;; the output voltages VI •.. VB represent the result  of the winner-take-all computation. If II; = max(II ••• IB ), then VI; is a logarithmic  function of 11;; if Ii <: 11;, then Vi ~ O. 
A static and dynamic ana.lysis of the two-neuron circuit illustrates these system  properties. Figure 2 shows a schematic diagram of a two-neuron winner-take-all  circuit. To understand the beha.vior of the circuit, we first consider the input  condition II = 12 = 1m. Transistors TIl ~d T12 have identical potentials at gate  and source, and are both sinking 1m; thus, the drain potentials VI and V2 must be  equal. Transistors T21 and T22 have identical source, drain, and gate potentials,  and therefore must sink the identical current ICI = IC2 = Ic/2. In the subthreshold  region of operation, the equation 1m = 10 exp(Vc/Vo) describes transistors Til and  T12 , where 10 is a fabrication parameter, and Vo = kT/qlt. Likewise, the equation  Ic/2 = 10 exp((Vm - Vel/Volt where Vm = VI = V2, describes transistors T21 and  T22 . Solving for Vm(Im, Ie) yields 
Vm = Voln(~:) + Voln(:;).