Query: The silicon retina

Candidate papers:

abstract: The USB Revolution Robotic vision is the most fascinating and feasible application for neuromorphic engineering, since processing images in real time with low power consumption is the field's most critical requirement. Conventional machine vision systems have been implemented using CMOS (complimentary metal oxide semiconductor) imagers or CCD (charge coupled device) cameras that are interfaced to digital processing systems operating with serial algorithms. These systems often consume too much power, are too large, and compute with too high a cost.1 Though neuromorphic technology has advantages in these areas, there are some disadvantages too: current implementations have less programmable architectures, for example, than digital processing technologies. In addition, digital image processing has a long history and highly developed hardware and software for pattern recognition are readily available. We therefore think it is practical at least at the current stage of progress in neuromorphic engineering to combine neuromorphic sensors with conventional digital technology to implement, for robot vision, the computational essence of what the brain does. On this basis, we designed a neuromorphic vision system consisting of analog VLSI (very large silicon integration) neuromorphic chips and field programmable gate array (FPGA) circuits. Figure 1 shows a block diagram of the system, which consists of silicon retinas, 'simple cell' chips (named after the simple cell in the V1 area of the brain) and FPGA circuits. The silicon retina is implemented with active pixel sensors (conventionallysampled photo sensors)2 and has a concentric center surround Laplacian Gaussian like receptive field.2 Its output image is transferred to the simple cell chips serially. These chips then aggregate analog pixel outputs from the silicon retina to generate an orientation selective response similar to the simple cell response in the primary visual cortex.3 The architecture mimics the feed forward model proposed by Hubel and Wiesel,4 and efficiently computes the two dimensional Gabor like receptive field using the concentric center surround receptive field. The signal transfer from the silicon retina to the simple cell chip is performed using analog voltage, aided by analog memories embedded in each pixel of the simple cell chip. The output image of the simple cell chip is then converted into a digital signal and fed into the FPGA circuits, where the image is further processed with programReceptive field Silicon retina
author_names: Misha A Mahowald

Carver Mead
title: The silicon retina.
venue: Scientific American
year: 1991

abstract: A new rotation invariant pattern recognition system is proposed and analyzed. In this system, silicon retina cells capable of image sensing and edge extraction are used so that the system can directly process images from the real world without an extra edge detector. The rotation invariant discrete correlation function is modified and implemented in the silicon retina structure by using the current summation. Simulation results have verified the correct function of the proposed system. Moreover, an experimental chip to implement the proposed system with a 32/spl times/32 cell array has been designed and fabricated in 0.8 /spl mu/m n well CMOS process. Experimental results have successfully shown that the system works well for the arbitrary orientation pattern recognition.
author_names: Chin-Fong Chiu

Chung-Yu Wu
title: The design of rotation invariant pattern recognition using the silicon retina
year: 1997

abstract: In this paper a new stereo matching concept for event driven silicon retinae is presented. The main contribution of the proposed approach is the correlation of incoming events. As a novelty, not only the spatial information is used, but also the time of occurrence of the events as a part of the similarity measure. Stereo matching is used in depth generating camera systems for solving the correspondence problem and for 3D reconstruction of the sensed environment. In fact, using conventionally frame based cameras, this is a time consuming and computationally expensive task, especially for high frame rates and spatial resolutions. An event based silicon retina delivers events only on illumination changes and completely asynchronous in time. The sensor provides no frames, but a time continuous data stream of intensity differences and thus inherently reduces the visual information to a minimum. This paper focuses on an event based stereo matching algorithm implemented in hardware on a field programmable gate array (FPGA) that allows a reliable matching of the sparse input event data. Furthermore, the approach is compared to other standard frame based and event driven stereo methods. The results show that the achieved depth map outperforms other algorithms in terms of accuracy and the calculation performance of the hardware architecture is in the range or still higher than state of the art computing platforms.
author_names: Florian Eibensteiner

Hans Georg Brachtendorf

Josef Scharinger
title: Event driven stereo vision algorithm based on silicon retina sensors
venue: 2017 27th International Conference Radioelektronika (RADIOELEKTRONIKA)
year: 2017

abstract: In this paper, we present: (i) a novel analog silicon retina featuring auto adaptive pixels that obey the Michaelis Menten law, i.e. V=V(m) I(n)/I(n)+s(n) (ii) a method of characterizing silicon retinas, which makes it possible to accurately assess the pixels' response to transient luminous changes in a 3 decade range, as well as changes in the initial steady state intensity in a 7 decade range. The novel pixel, called M(2)APix, which stands for Michaelis Menten Auto Adaptive Pixel, can auto adapt in a 7 decade range and responds appropriately to step changes up to 3 decades in size without causing any saturation of the Very Large Scale Integration (VLSI) transistors. Thanks to the intrinsic properties of the Michaelis Menten equation, the pixel output always remains within a constant limited voltage range. The range of the Analog to Digital Converter (ADC) was therefore adjusted so as to obtain a Least Significant Bit (LSB) voltage of 2.35mV and an effective resolution of about 9 bits. The results presented here show that the M(2)APix produced a quasi linear contrast response once it had adapted to the average luminosity. Differently to what occurs in its biological counterparts, neither the sensitivity to changes in light nor the contrast response of the M(2)APix depend on the mean luminosity (i.e. the ambient lighting conditions) Lastly, a full comparison between the M(2)APix and the Delbruck auto adaptive pixel is provided.
author_names: Stefano Mafrica

Stephanie Godiot

Mohsine Menouni

Marc Boyron

Fabien Expert

Raphael Juston

Nicolas Marchand

Franck Ruffier

Stephane Viollet
title: A bio inspired analog silicon retina with Michaelis Menten auto adaptive pixels sensitive to small and large changes in light.
venue: Optics express
year: 2015

abstract: A new silicon retina structure in CMOS is proposed and analyzed, which incorporates a parasitic phototransistor as an isolated photoreceptor with many phototransistors in a common well region. The smooth function is achieved by the diffusion of photogenerated carriers in the common well. Experimental results have verified the correct functions of the new structure as a silicon retina. Small chip area and compact wiring make the new structure quite realizable with the associated neural networks in VLSI.<ETX>
author_names: Chung-Yu Wu

Chin-Fong Chiu
title: A new structure for the silicon retina
venue: 1992 International Technical Digest on Electron Devices Meeting
year: 1992

abstract: Optical flow sensors have been a long running theme in neuromorphic vision sensors which include circuits that implement the local background intensity adaptation mechanism seen in biological retinas. This paper reports a bio inspired optical motion sensor aimed towards miniature robotic and aerial platforms. It combines a 20 x 20 continuous time CMOS silicon retina vision sensor with a DSP microcontroller. The retina sensor has pixels that have local gain control and adapt to background lighting. The system allows the user to validate various motion algorithms without building dedicated custom solutions. Measurements are presented to show that the system can compute global 2D translational motion from complex natural scenes using one particular algorithm: the image interpolation algorithm (I2A) With this algorithm, the system can compute global translational motion vectors at a sample rate of 1 kHz, for speeds up to 1000 pixels/s, using less than 5 k instruction cycles (12 instructions per pixel) per frame. At 1 kHz sample rate the DSP is 12% occupied with motion computation. The sensor is implemented as a 6 g PCB consuming 170 mW of power.
author_names: Shih-Chii Liu

Minhao Yang

Andreas Steiner

Rico Moeckel

Tobi Delbruck
title: 1 kHz 2D Visual Motion Sensor Using 20$\times\$20 Silicon Retina Optical Sensor and DSP Microcontroller
venue: IEEE Transactions on Biomedical Circuits and Systems
year: 2015

abstract: Abstract In the Background section of the proposal I described the basic conceptual elements which underlie the Silicon Retina (SR) and those which underlie passive mammalian hearing. My original intention was to utilize the frequency discrimination capability manifested in mammalian hearing as the mechanism on which to base an acoustic analogue to the SR concept. Specifically, mammals have sophisticated frequency dependent sensors which transmit responses to discrete and well resolved frequencies upward to higher processing levels. Each intermediate processor transmits the precise frequency information upstream. Thus one may think of acoustic frequencies as being analogous to visual positional information, since in the ray limit of optics there is a one to one correspondence between the relative locations of external objects and the retinal sensors which are stimulated at any given time. The SR mechanism was to be implemented by comparing individual frequencies with running averages over groups of nearby frequencies, eliciting a response when the intensities of these differed by a prescribed amount. The next step in the program was to remain open minded regarding the implementation of directional acoustic arrays, with the idea pf steering an array making use of the 'alert' response created by the introduction of a non background signal. In this way it was envisioned that the 'cocktail party effect' would be achieved.
author_names: Jorge F Willemsen
title: Feasibility Assessment of a Transient Sound Sensor Based on the Silicon Retina Architecture
year: 1992

abstract: This paper provides a personal perspective on human vision and event based vision sensors, algorithms, and applications over the period of last decade. The comparison between the standards of biology and technology has been the major focus throughout. Some recent advancements in the field are also briefly described.When Mahowald and Mead built the first silicon retina with asynchronous digital output around 1992 1 conventional CMOS active pixel sensors (APS) were still research chips. It required the investment by industry of about a billion dollars to bring CMOS APS to high volume production. So it is no surprise that while the imager community has been consumed by the megapixel race to make nice photos, cameras that mimic more closely how the eye works have taken a long time to come to a useful form. These "silicon retinas" are much more complex at the pixel level than APS cameras and they pay the price in terms of fill factor and pixel size; machine vision cameras with capability of synchronous global electronic shutter are about 5um. Silicon retina pixels are roughly 10 times the area of a machine vision camera pixel. So why are silicon retinas still interesting? What was the nessesity to mimic the biological eye? Mostly because of the high cost at the system level of processing the highly redundant data from conventional cameras, and the fixed latencies imposed by the frame intervals. High performance activitydriven event based sensors could greatly benefit applications in real time robotics, where just as in nature, latency and power are very important 2 3 Still is this the closest that technology brought the field towards biology? What important ascpect are missing to make a bionic eye? These are the few aspects that are througly discussed throughout this paper. 1 Mahowald, M.A. An Analog VLSI System for Stereoscopic Vision. Kluwer, Boston (1994) 2 Lichtsteiner, P. Posch, C. Delbruck, T. A 128x128 120 dB 15 ms latency asynchronous temporal contrast vision sensor. IEEE Journal of Solid State Circuits 43(2) 566 576 (2008) 3 Liu, S.C. van Schaik, A. Minch, B.A. Delbruck, T. Event based 64 channel binaural silicon cochlea with Q enhancement mechanisms. In: IEEE ISCAS 2009, pp. 2426 2429 (2010) Comparison between Human and Silicon Retina 5
author_names: Hauptseminar Neuroengineering

Marcello Mulas
title: Comparison between Human and Silicon Retina 2
year: 2015

abstract: In this paper we present a new approach for the evaluation of event based Silicon Retina stereo matching results. The evaluation of stereo matching algorithm results is a necessary task for the development, comparison, and improvement of depth generating camera systems. In contrast to conventional frame based cameras, the silicon retina sensors delivers asynchronous events instead of synchronous intensity or color images. The polarity of the events represents either an increase (on event) or a decrease (off event) of the brightness of the projected scene point. This is the reason why existing ground truth data and evaluation platforms are not suitable for testing silicon retina stereo camera systems. For the analysis of the introduced novel evaluation method, we use an area based (sum of absolute difference) algorithm for the event driven sensor system. A conventional video camera stereo vision system is used to produce reference data. The results show that the presented method offers new opportunities for the evaluation of stereo matching results computed from silicon retina stereo data.
author_names: Jurgen Kogler

Florian Eibensteiner

M Humenberger

Margrit Gelautz

Josef Scharinger
title: Ground Truth Evaluation for Event Based Silicon Retina Stereo Data
venue: 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops
year: 2013

abstract: ii Zusammenfassung iv
author_names: Minhao Yang
title: Silicon Retina and Cochlea with Asynchronous Delta Modulator for Spike Encoding
year: 2015