Development of a computational image sensor with applications in integrated sensing and processing

The objective of this research was to build a reprogrammable computational imager utilizing on-chip analog computations for the purpose of studying the capabilities of integrated sensing and processing. Unlike conventional imaging systems, which acquire image data and perform calculations on it, this system tightly integrates the computation and sensing into one process. This allows the exploration of intelligent and efficient sensory and processing. The IC architecture and circuit designs have focused on wide dynamic range signals. The fundamental computation performed is a separable two-dimensional transform. This allows various operations, including block transformations and separable convolutions. The operations are reprogrammable and utilize analog memory and processing along with digital control. The random access to both the image plane and the computational operations allows for intraframe transform variations creating a hardware foundation for dynamic sampling and computation. One can also capture scenes with non-uniform resolution. Advantages, including utilization of feedback from processing to sensing and extensions of the technology including support for wavelets and larger transforms are also explored. In the first chapter of this thesis, I discusses the integration of a reprogrammable, analog, signal processing technology onto image sensor ICs. The advantages of having computational hardware integrated with sensors is discussed. In addition, the advantages of having reconfigurable hardware for logistical and algorithmic purposes is given. Our analog processing circuity innately provides reprogrammability and presents an obvious competition to reprogrammable digital counterparts. In the second chapter, I give a selective overview of CMOS imagers and the technologies leading up to the work here. The inherent challenges in image sensing to handle large data quantities and wide-dynamic-range signals is discussed. Previous work in the integration of processing circuitry, especially focal-plane processing, is given. As demonstrated, analog processing circuitry is compact enough to be integrated with sensing circuitry and is well-suited for processing the physical-based, analog signals from the sensing circuitry. It can provide the needed level of up-front processing that can reduce data throughput through the rest of the system. In the third chapter, I introduce subthreshold conduction in field-effect transistors and floating-gate transistors. The properties of transistors operating in the subthreshold current regime, as opposed to the above threshold regime, offer exponential-based current-voltage relationships that are advantageous for implementing computations. Reprogrammable floating-gate transistors, which serve as part of the foundation of our analog processing technology, are presented. In the fourth chapter, I present the computational, focal plane used for image sensing. It provides a critical computational ability, but avoids large and complex processing circuitry that can reduce image sensitivity. The functionality of the pixel array is discussed as well as the experimental testing and characterization of it. In the fifth chapter, I discuss the architecture of the imager. The individual subsystems of the imager are each discussed. The mathematical function and circuit implementation of each is discussed along with the issues critical to their design. In the sixth chapter, I elaborate on the circuit design challenges when processing with small currents that vary over multiple orders of magnitude. Tunable current mirrors, which provide the foundation for computation capabilities are heavily discussed. To extend the speed of the operations and provide the ability to process small input currents, logarithmic transimpedance amplifiers are given. The analytical relationship of power consumption to dynamic range is given along with circuits that improve power consumption by dynamically changinggain. Limitations of these circuits in terms of distortion and noise are given. Also, a circuit the can convert a bidirectional current input into a logarithmic representation is presented. In the seventh chapter, I discuss the limitations of the computational pixel plane, including mismatch characteristics. The aggregate effects of mismatch are studied and the process of removing the dominant components of error is given. In the eighth chapter, I discuss an application, compressive sensing, and how the unique capabilities presented by the computational image sensor are suited for it. Compressive sensing takes advantage of a small amount of a priori knowledge to greatly reduce the number of samples of a signal that must be taken. The idea is strongly related to the concept reducing data in the front-end of a system to reduce component throughputs. Even though this imager was not explicitly designed for compressive sensing, the flexibility and reprogrammability built into the design allowed the use of it in the compressive sensing framework. This exemplifies the advantages of reconfigurable and reprogrammable systems. In the ninth chapter, I analyzed the resulting images from the hardware. Unlike traditional images sensors, the outputs of this IC are computational results, not raw data. However, this IC can be configured to output raw images as well, allowing a comparison between the two. In particular, a discreet cosine transform (DCT) output is compared to a standard image output. In the tenth chapter, I summarize the contributions of my thesis work.