Computation in neuromorphic analog VLSI systems: Lecture WS 2001/2002

In this paper we present an overview of basic neuromorphic analog circuits that are typically used as building blocks for more complex neuromorphic systems. We present the main principles used by the neuromorphic engineering community and describe, as case example, a neuromorphic VLSI system for modeling selective visual attention. 1 Neuromorphic Engineering The term “neuromorphic” was coined by Carver Mead to describe very large scale integration (VLSI) systems containing electronic analog circuits that mimic neurobiological architectures present in the nervous system [18]. Neuromorphic computation is related to modeling and simulation of networks of neurons and systems using the same organizing principles found in real nervous system. In recent times the term “neuromorphic” has also been used to describe mixed analog/digital VLSI systems that implement computational models of real neural systems. These VLSI systems, rather than implementing abstract neural networks only remotely related to biological systems, in large part, directly exploit the physics of silicon (and of CMOS VLSI technology) to implement the physical processes that underlie neural computation. Neuromorphic engineering is a new discipline at the boundary between engineering and neuroscience, but which crosses many other fields, including biology, physics, computer science, psychology, physiology, etc. Analog VLSI Circuits There are some direct analogies between biological neural systems and analog VLSI neuromorphic systems: conservation of charge, amplification, exponentiation, thresholding, compression, and integration. The parallels between these two worlds run from the level of device physics to circuit architectures: Diffusion mechanisms in membrane and transistor channels Function determined by the system’s structure To point out why studying natural neural systems and implementing models of these systems, using analog VLSI technology can be potentially instrumental for improving technological progress, consider the following arguments: 1. Biology has solved the problem of using loosely coupled, globally asynchronous, massively parallel, noisy and unreliable components to carry out robust and energy efficient computation. 2. CMOS VLSI technology has been continuously improving, following Moore’s law, for almost 30 years, pushed by a hyper-competitive $300 billion global industry. But, as CMOS technology improves further and transistor’s gate lengths drop below 0.10 microns transistors start to behave like neurons we have the ability to place millions of simple processors on a single piece of silicon single chips start to have complexities that are too difficult to handle by the design tools we need to begin to worry about power consumption and power dissipation issues Computation and Power Consumption The brain has on the order on neurons and synapses. It performs on average operations per second. The power dissipation of the brain is approximately J per operation, which results in about a total mean consumption of less than 10 watts. By comparison today’s silicon digital technology can dissipate at best J of energy per operation at the single chip level. There is no way of achieving operations per second on a single chip, with today’s technology. But even if this was possible, to do that amount of computation a digital chip would consume MegaWatts (the output of a nuclear power station). Any serious attempt to replicate the computational power of brains must confront this problem. Subthreshold analog circuits are also no match for real neural circuits, but they are a factor of more power efficient than their digital counterparts. In the following Section we present some basic analog circuits that are most commonly used as elementary building blocks for constructing complex neuromorphic systems. In Section 3 we present a VLSI device containing a model of selective visual attention, that uses many of the circuits described in Section 2, as a case example. Finally in Section 4 we draw the conclusions. 2 Subthreshold Analog Circuits Perhaps the most elementary computational element of a biological neural structure is the neural cell’s membrane. The nerve membrane electrically separates the neuron’s interior from the extra-cellular fluid. It is a very stable structure that behaves as a perfect insulator. Current flow through the membrane is mediated by special ion channels (conductances) which can behave as passive or active devices. In the passive case, ion channels selectively allow ions to flow through the membrane by the process of diffusion. In electronics, it is possible to implement the same physical process by using MOS transistor devices, operated in the subthreshold region (also referred to as weak inversion) [17, 20].  From subthreshold MOS transistors . . . MOS transistors operate in the subthreshold region of operation when their gate-tosource voltage is below the transistor threshold voltage. This mode of operation of a transistor has been largely ignored by the analog/digital circuit design community, mainly because the currents that flow through the source-drain terminals of the device under these conditions are extremely low (typically of the order of nanoamperes). In subthreshold, the drain current of the transistors is related to the gate-to-source voltage by an exponential relationship. Specifically, for an n-type MOS transistor, the subthreshold current is given by: