Chapter 1
Introduction
1.1 A Brief History of Image Sensors' Development
The history of image processing technology in electronics dates from the inventionof television system in 1927. From this time, the image processing technology has
been studied and developed mainly in terms of television broadcasting technology,
such as NTSC or PAL.
In recent years, the development of image processing technology seems to havecome at a dramatic turning point, for the development of high de nition television(HDTV) or the popularization of \multi-media." For example, in the applicationsfor HDTV, the drastical increase of the quantity of image information makes variousdiculties for the real-time image processing technology. In case of the applications
for \multi-media," the importance of image signals has coming up, and the formsof the usage of image signals are not only single direction broadcasting such astelevisions, but also the bi-directional communications such as television conference
systems. The recent image broadcasting systems, such as VOD (Video on Demand)
also need the one-to-many broadcasting and the information quality adaptivity forthe needs of users. The needs of digital image processing systems have also increased since these development of image processing systems has been based on thedevelopment and the popularization of computers and their digital signal processing
technologies.

Chapter 1Introduction1

On an image capturing devices, the solid-state imagers have been invented in
1970s, which employ the CCD (charge coupled device) as a signal transformer. The
recent development of VLSI technologies enables us to fabricate not only photodetectors and signal transfer circuits, but also a simple image processing circuits
in one image sensor chip, so called \computational sensors." An integration ofimage processing systems on image sensor has a ability to decrease the quantity ofimage information coming outside the image sensor chip, which can also overcome
the bottleneck of the channel capacity between image sensors and image processing
systems.

1.2 Previous Works on \Computational Sensors"
The conventional image processing systems have separated image sensor and image
processor connected by the signal channel. Seen in the previous section, the channel
capacity is one of the most important problems in image processing systems by theincrease of the quantity of image information. One solution for this problem is theintegration of image sensor and some of signal processing systems in one chip, which
was enabled by the highly developed VLSI technologies. Such image sensors whichhave the functions of more than simply capturing images are called \smart sensor,"or \computational sensor." Most of the recent studies on computational sensors are
made in an approach of integrating the functions of pre-processing for images insensor chips.
In this section, some of previous studies on computational sensors are summerized.

1.2.1 Early vision problem
The pre-processings for images, such as noise reduction, edge detection and so on,
are called \early vision problem," which have the following characteristics:

 useful as pre-processing for images.
 can be processed parallelly.
Chapter 1Introduction

2

Figure 1.1: Model of \silicon retina."(From [3])

 can be easily mathematically modeled.
These are the reasons why the early vision problems are often studied as the computational sensors. One of the most previous studies in computational sensors are theapproach of modeling the structure of retina of creatures[1, 2, 3, 4], The function ofearly vision processing in these image sensors are implemented by the network ofresistors as shown in Figure 1.1, which are called \silicon retina."
Vi-1 R Vi

R Vi+1

Vi-1 R Vi

R Vi+1

Ii
GEi

(a)

(b)

Figure 1.2: Models of one-dimensional resistor network.

Chapter 1

Introduction

3

Figure 1.2 shows a simple model of the resistor network. Assuming that thepotential of the node i is Vi , the current injected to node i is Ii and the resistance
between two nodes is R in Figure 1.2(a), the following equation is derived from theKirchho 's law.
2Vi 0 Vi+1 0 Vi01 = 2RIi

(1.1)

Since this equation is equivalent to the discreted Laplacian r2 , the following di erential equation is derived for the whole area.

r2 V = RI(1.2)

Assuming that Vi gives the intensity of a pixel, the distribution of Ii gives the edgedetected images.
On the other hand, the following equation is derived for the resistor network inFigure 1.2(b).
2Vi 0 Vi+1 0 Vi01 = RG(Ei 0 Vi )(1.3)

This equation is equivalent to the discrete equation of the following Helmholtz'sequation, the V gives the smoothed images of the image given by the distributionof Ei .

r2 V + GRV = GRE(1.4)

The network of non-linear resistors is expected to implement the more usefulfunctions[5, 6]. For example, the network of \resistive fuse," which is the resistor
that opens for the preset voltages shown in Figure 1.3, can implement the image
smoothing with maintaining the contrast at the edge of images[7] as shown in Figure
1.4.

1.2.2 Range nding
One of the most simple methodologies for range nding is \Light-Stripe" method,which is a kind of triangle method using the angle of rotated mirror and the outputof image sensor as shown in Figure 1.5. Some computational sensor for range nding

Chapter 1

Introduction

4

Figure 1.3: The characteristics of \resistive fuse."(From [7])Chapter 1

Introduction

5

(a)

(b)

(c)

Figure 1.4: Examples of edge enhancement and smoothing using the networks ofvarious resistive elements.(From [7]) (a)original image with noise, (b)output of thenetwork of HRES, (c)output of the network of \resistive fuse."

Figure 1.5: "Light-Stripe" method for range nding. (From [8])Chapter 1Introduction

6

Figure 1.6: Pixel circuit for fast range nding computational sensor.(From [8])are reported. For example, a range nding computational sensor whose pixels areshown in Figure 1.6 is reported, and its range nding system can archive the error
less than 0.1% with very fast nding speed[8].

1.2.3 Movie compression
On-chip image compression is adequate for the reduction of the trac in thechannel between the image sensor and the image processing systems, especially incase of the increased quantity of image signals, such as HDTV. Since the algorithmsfor movie compression needs very high ability of computing, few studies on on-chipmovie compression are previously reported. One of the reported studies on onchip movie compression is the movie-compression sensor by taking the di erences
between one pixel and its neighbor pixels for the block of 323[9]. An computational
sensor for movie compression by taking inter-frame di erence is also reported[10].

1.2.4 Circuit technologies
Many of the computational sensors employ analog signal processing circuits, notdigital signal processing circuits. The advantages of analog signal processing circuitsChapter 1

Introduction

7

are follows.

 can implement operation by the physical law using simple circuit elements, forexample, add by the Kirchho 's current law using resistor network.

 can solve equations as a equilibrium of the system about in the time of RC
time constant( s).
Analog signal processing circuits also have disadvantages as follows.

 its accuracy depends on the distribution of fabrication process.
 very sensitive for noise.
 has a diculty on design system.
 has a diculty on employing memory elements.
By the reasons above, some studies on computational sensors employing digital
signal processing circuits has been reported[11, 12, 26].

1.2.5 Fabrication technologies
Since the solid-state imager has the photo detectors on a focal plain, the ll factor,
which is the ratio of the photo detectors' area to the pixels' area, is expected to be
kept as high as possible to obtain a high light sensitivity. Computational sensors has
an essential problem of low ll factor, because each pixel has both photo detectors
and signal processing circuits.
In recent years, the stacked device structure, which is called \three dimensional
IC," can have been fabricated by the development of SOI (Silicon On Insulator)
technologies. For example, a computational sensor which has 25 photo detectors
on top layer, A/D converters for each pixel on middle layer, and signal processing
circuits on bottom layer, as shown in Figure 1.7 has been reported.
It is also studied to fabricate three-dimensional structure by pasting bulk chips,
and using copper terminal for inter connections between layers[13].

Chapter 1Introduction8Figure 1.7: Cross-sectional view of computational sensor with three dimensionalstructure.(From [11])1.3 Outline of This Thesis
Seen in the previous sections, the channel capacity between image sensors andimage processing systems will become one of the most important problems in near
future, while few studies on on-chip image compression are reported. The increaseof power consumption and the bottleneck of operation speed of signal outport will
also be the important problems in the mega-pixels image sensors.
In this thesis, the studies on a image sensor employing a tree structure for image
signals, which has a kind of data compression, that is ecient both power reduction
and high speed operation.
Chapter 2 describes the power reduction methodologies for VLSI circuits including
image sensors with the probabilistic models of power consumption.
Chapter 3 describes a kind of image encoding method using tree structure to treat
image signals. The analytical model of this method also has been described.
In Chapter 4, the applications of the method using tree structure for image signals
for some adaptivities of image sensors, such as spatial adaptivity and intensional
adaptivity. The applications for movie compression sensor has been described inthis chapter.

Chapter 1Introduction

9Chapter 5 describes the two di erence implementations of tree-structured image
sensor using CMOS technologies, which can archive high speed and low power operation. The implementation of decoder of 1:4 tree code are also discussed. The
functions and the circuits of pixel for on-sensor image processing are also discussed
in this chapter. The idea of ultra-low power image sensor which uses the energy oflight is also described.
Chapter 6 describes the evaluations for tree-structured image sensors. The evaluation of the prototype system using FPGAs and full-custom design of tree-structured
image sensors are described. The evaluation of designed full-custom tree-structured
image sensors are also discussed in this chapter.
The summary and the conclusion are discussed in Chapter 7.

Chapter 1 Introduction

10