The Great Buddha Project: Modeling Cultural Heritage for VR Systems
through Observation
Katsushi Ikeuchi, Atsushi Nakazawa, Kazuhide Hasegawa, Takeshi Ohishi
Institute of Industrial Science, The University of Tokyo{ki, nakazawa, k-hase, oishi}@cvl.iis.u-tokyo.ac.jp
Abstract
This paper overviews our research on digitalpreservation of cultural assets and digital restoration of
their original appearance. Geometric models are digitallyachieved through a pipeline consisting of scanning,registering and merging multiple range images. We havedeveloped a robust simultaneous registration method andan efficient and robust voxel-based integration method. On
the geometric models created, we have to align texture
images acquired from a color camera. We have developed
two texture mapping methods. In an attempt to restore theoriginal appearance of historical heritage objects, we havesynthesized several buildings and statues using scanned
data and literature survey with advice from experts.

number of unique features; among them is its ability todigitize relatively large objects outdoor such as the
Kamakura great Buddha, and Cambodia’s Bayon. This
presents several challenges. Also, our project consists not
only of geometric modeling but also of photometric andenvironmental modeling, as shown in Figure 1.

Geometry
Partial observation

Photometry
Color images

Environment
Illumination

1. Introduction
Currently, a large number of cultural heritage objects
around the world are deteriorating or being destroyed
because of natural weathering, disasters and civil wars.
Among them, Japanese cultural heritage objects are quite
vulnerable to fires and other natural disasters because mostof them were constructed of wood and paper.
One of the best ways to prevent them from loss anddeterioration is to digitally preserve them. Digital data ofheritage objects can be obtained by using computer vision
techniques. Once these data have been acquired, they canbe preserved permanently, and then safely passed down tofuture generations. In addition, such digital data is suitable
for many applications, including simulation, restoration,and creating multi-media contents. Such digital contents
can be viewed through the internet from anywhere in theworld, without moving the objects nor visiting the sites.
We have been working to develop such digital archival
methods by using computer vision and computer graphics
technologies [1]. Similar projects include: Stanford’sMichelangelo Project [2], IBM’s Pieta Project [3], andColumbia’s project [4], to name a few. Our project has a

Figure 1. Three components of our project
The remainder of this paper is organized as follows.
Section 2 describes the outline of the geometric pipeline
developed, a parallel simultaneous alignment algorithm
and a parallel voxel-based merging algorithm. Section 3
describes methods to align observed textures from a digital
camera with a range data for texture mapping. Section 4reports our efforts to restore the original appearance of
these objects using acquired digital data and the literature
survey. Section 5 summarizes this paper.

2. Geometric Modeling
Several computer vision techniques, such as traditional
shape-from-X and binocular stereo, or modern range
sensors, provide cloud of point information. The cloud of
point information certainly carries three-dimensional
information pertaining to observed objects. However, there
is no structural information among these points. Namely,
there is no information to represent adjacency among thepoints. The first step of geometric modeling is to convert
the cloud of points into a surface representation such as amesh model.

Since each observation provides only partial information,
we have to combine these partial mesh representations into
a whole geometric mesh representation. Thus, the second
step in geometric modeling is to align these meshes so that
their corresponding parts overlap one another (alignment).
To accomplish this step, we have developed a simultaneousalignment method to avoid accumulation of errors [3].
The third geometric modeling step is to integrate thepre-registered multiple range images and to compose one
complete geometric model, a step usually called 'merging'.
The procedure can be considered as extracting one surface
from multiple overlapped surfaces. In the merging
procedure, it is important to make the integration
framework robust against any noise which may be in thescanned range images and can also be inherited from theregistration procedure.

to align a very large data set [12]. The fundamental
algorithm, due to the utilization of a graphics hardware, isslightly different from the one described above [11].
The corresponding pairs are searched along the line ofsight to use the graphics hardware. Assume that a baserange image is the model image and that a corresponding
range image is the scene image. In Figure 4, a model image
is depicted as points and a scene image is shown as meshes.
When a line extended from a vertex of model image along
the line of sight crosses a mesh of a scene image, theintersecting point is the corresponding point. In order toeliminate false correspondences, if the distance between
corresponding points is larger than a certain thresholdvalue or if the angle of lines of sight is beyond a threshold
angle, the correspondence is removed. This correspondence
search is computed for every combination of range images,
using the Z-buffer capability of the graphics hardware.

Scanning

Alignment
Merging

Figure 2. Three steps of Geometric Modeling.
2.1 Simultaneous Registration Method
To avoid accumulation of errors, we have developed asimultaneous alignment method [11]. Traditional sequential
methods [5,8,9] such as Iterative Closest Point Algorithm(ICP) align these meshes one by one, and progressivelyalign a new partial mesh with previously aligned meshes. If
a few partial meshes can cover an object, the accumulationof alignment errors is relatively small and can be ignored.
A sequential alignment works well. However, some
cultural heritage objects are very large; we need more than
one hundred views. In such cases, the error accumulation
caused by sequential alignment methods is very large. Thus,by considering alignment of all the pairs simultaneously,
we align all partial meshes so as to reduce the errors among
all the pairs simultaneously [7,10,11]. Our simultaneousalgorithm [11] employs points and planes to evaluate
relative distance as does the Chen and Medioni and
Gagnon et.al. methods [8,10]. Further, to avoid the effects
from the outliers, the algorithm uses M-estimator, inparticular, the Lorentian function as the error measure [13].
Currently, we are extending the simultaneous alignment
algorithm into a a parallel implementation so as to be able

Figure 3.

Search corresponding points

The error measure between corresponding points is the
distance between the point and the plane. Let the vertex ofrmodel image be x and let the point corresponding to thervertex be y , the error measure between the pairs is writtenasn ⋅ ( y − x)(1)rwhere n is the vertex normal of x .
The transformation matrices of the model and scene
images are computed so that this error measure isminimized. The error evaluation function is rewritten as
r r
r
r r
(2)
RM n ⋅ {( RS y + t S ) − ( RM x + t M )}Here, the rotation matrix and the translation vector of thermodel and scene image are RM , Rs , t M , t s , respectively.
Then, the distance between the model image and the scene
image is expressed asrr

ε 2 = min ∑ (RM n ⋅ {( RS y + t S ) − ( RM x + t M )})
r

R ,t

r

r

2

(3)

i ≠ j ,kIf it is assumed that the angles of rotation are small, therotation matrix R can be approximated as 1
R =  c3
− c
 2

− c3
1
c1

c2 

− c1 
1 （4）

The translation vector is expressed as
t = (t x

tz )

ty

（5）

After some algebraic manipulations [16], (3) is rewrittenas2

ε 2 = min ∑ Aijk δ − s ijk
δ（6）i ≠ j ,ksijk = n ik ⋅ ( x ik − yijk )

（7）



 
Aijk =  0{...0Cijk 0{...0  +  0{...0− Cijk 0{...0 
 6i×1 { 6 ( l −i −1)×1   6 j×1 123 6 ( l − j −1)×1 
6×1

 
6×1

 n × yijk 
Cijk =  ik
 −n 
ik



c 2i

c 3i

t xi

（10）

t zi )t yi（11）

where the number of range images is n . By (6) δ iswritten asδ =  ∑ AT ijk Aijk i ≠ j ,k



−1

∑A

T

i ≠ j ,ks

ijk ijk

We can define and remove redundant dependencies in
the range images. A pair of range images that does notsatisfies all these three conditions will be removed as
redundant relations.
1.
2.

（9）

δ = (m0 L mn −1 )mi = (c1i（8）

Figure 4 Removing redundant dependencies（12）

For the parallel implementation of the alignment
algorithm, both time and memory performance have to beconsidered. The simultaneous alignment algorithm,
originally designed, requires all range images to be read
into memory; even when the computation is distributed
over a PC cluster, the amount of memory used on each PC
is not reduced.
We will remove redundant or weak data dependency
relations. Figure 4 shows overlapping data-dependency
relations. Each node in the graph represents one range dataand each arc represents an overlapping dependency
relations among range data. In the left graph, all the data
are overlapping each other. In order to compute alignmentof one data, we have to read, into a PC memory, all theremaining range data. By removing some of redundant
overlapping dependencies, we can transform the originalgraph into a simpler one as shown in the right figure. By
using this simpler relational graph, we only need adjacentdata with respect to a vertex for alignment of a vertex, andcan increase the efficiency in memory usage.

3.

The bounding-boxes of two range images overlap eachother.
The angle θ between ray directions of two range
images is less than a threshold value.
Two range images are adjecent to each other.

Under the assumption that initial positions of two range
images are accurately estimated, Requirement 1 is satisfied
when a sufficient overlapped region exists between two
range images. Requirement 2 is satisfied when two views
are relatively near. As the side effect, this requirement also
reduces the possibility of false correspondences. Since ourmethod utilizes the ray direction to establish
correspondences, by setting the threshold, as θ = 90° , wecan avoid correspondence pairs between one and the otherside. Requirement 3 is set up for removing non-adjacent
relation sequentially. For example, as shown in Figure 5, if
the length from I 0 to I n is larger than the length fromI m to I n (l0 m < l0 n ) , the arc between I 0 and I n isremoved. Here, the distance is evaluated from the center ofa range image’s surface.

Figure 5 Definition of the distance measure

For our parallel algorithm to read data only related, we
can set up a relational table as shown in Figure 6. The rowof the table represents scene images, and the column
represents model images. Each element d j ,k represents
dependency between a scene, j and model image, k.
1 j and k have a dependency relationd j ,k = otherwise
0
Since each pair of a scene and model image can beevaluated independently in equation (6), roughly speaking,we will assign those pairs to nodes of a PC cluster so as foreach node to have an average number of pairs. Here, weassign pairs to nodes based on scene images.

Figure 6

Pair Assignment table

The amount of memory used depends on the number ofvertices included in range images required for each node.
By assuming the number of vertices of each range image isroughly equal, the amount of memory required, M i foreach node i is proportional to the number of range
images assigned to the node ri , the number of 1’s in theassignment.
More precisely,（13）
M i = M r × ri + M c
istheaverageoftheamountofdata
contained
in eachMr
range images, and M c is the amount of memory used forcomputation and independent to the number of range
images (constant). The number of range images read into
each node is expressed as union of sets of model image
rmod el ,i and scene image rscene,i .
ri = rmod el ,i ∪ rscene ,i(14)

For the sake of computational efficiency described
below, we will re-arrange the table. The range images, read
into each node, are unions of model and scene images to beread in Eq (14). The assignments to the nodes are based on
the scene images, as shown in Figure 8; it is preferable toread the same model images with respect to the scene
images We define the distance li, j between two scene
images (i, j ) as follows:
li , j = n − ∑ (d i ,k ∪ d j ,k − d i ,k ( xor )d j ,k )(15)0≤ k ≤ n

The column of the table, the scene images, are sorted in thedescending order of this distance under the assumption thatthe adjacent scene images are more likely to share the same
set of model images.

2.1.5 Averaging computational time
Next, we will make even distribution of the computationalload over a PC cluster. The computational cost of the I
node, ti is written as:ti = t s ×∑v

jn0 ,i ≤ j ≤( n0 , i + ni )+ tm ×∑∑ (v × d ) + t (16)n0 ,i ≤ j ≤( n0 ,i + ni ) k

j ,k

j ,k

c

where ni scene images are assigned to node i from thenumber n0,i through n0, j + ni . Here, v is the numberof vertices; ts is the computation time for the number ofvertices included in scene images, and tm is computation
time for the number of vertices included in model
images. tc is independent of the number of vertices, andbecause tc is very small, tc can be considered to be 0.
Assuming that the computation time for the number ofscene images and model images ts and tm is equal to tv ,(16) is rewritten as(v j ,k × d j ,k )
ti = t v ×  ∑ v j +∑∑n0 ,i ≤ j≤( n0 ,i +ni ) n0 ,i ≤ j≤( n0 ,i +ni ) k


(17)

t s = t m = tv

(18)

That is, the computation time is proportional to the numberof vertices computed. Then, in consideration of a scene
image being assigned to multiple nodes, assuming that the
number of vertices of the divided scene image is theaverage of the number of vertices of all range images
vaverage , the number of vertices vi that are computed onnode i is expressed as follows.
vi =  ∑ v j + ∑ ∑ (v j ,k × d j ,k ) + vaverage × (nnode − 1)  / nnode0≤ j ≤ n 0≤ k ≤ n 0≤ j ≤ n(19)nnode expresses the number of nodes. By using this, vi
expressed in (19) as an index, we will assign nodes to thescene images.
We implemented our method as a server/server system.
The procedures of the computation is are as follows

Algorithm Procedure of Parallel Alignment/* Check correspondence of all pair of *//* the range images */CreatePairTable;/* Create the lists of the files for eachprocessor */CreateFileLists:
while(error > threshold){/* Client Process*/for(i = 0; i < nImage; ++i)
for(j = 0; j < nImage; ++j)
if(List[i][j]){CorrespondenceSearch(i, j);CalculationEachMatrix(i, j);}/* Server Process */CalculationMatrix(all);/* Server & client process */UpdatePosition;}

The server program holds bounding-boxes andtransformation matrices from initial position to current
position of all range images, checks all combinations, andcreates the list of computations for each node. The list ofcomputations for each client is computed at the beginningof the entire iterations based on the relational table using
the algorithm described above The client program receives
the list and reads the required range images into memory.
Then, each client computes the matrices AT ijk Aijk andAT ijk sijk in (12) independently, and sends the matrices tothe server program. The server program computes therotation matrices and translation vectors of all range
images from the matrices AT ijk Aijk and AT ijk sijk received
from the client program. The results are applied to allserver/client data. Then, each iteration process is continued
until the error falls below a certain threshold value.
Computation time is defined as the time taken for one
iteration, and an average of time of all iterations is used forthe evaluation. We used the 20 range images created fromhand model (model A) and the 15 range images measured
the great Buddha of Kamakura (model-B) for timeevaluation. Figure 4 shows the result of initial andsimultaneous alignment of Kamakura Buddha (after 20
iterations).

Figure 7. Simultaneous registration result.

We have observed the computation efficiency linearly
proportional to the number of processors. In Table 1, thecomputation times with 1 processor and 16 processors aredescribed. The computation time with 16 processors isapproximately 8.4 times faster than that with 1 processor
for the hand model, and 8.9 times for the Great Buddha of
Kamakura. As for the memory usage, we can reduce theamount of memory used decreases as the number ofprocessor increases.
Table 1 Computational time

As shown in Table 2, our method can reduce the amountof memory used in approximately 60% for the hand modeland in approximately 43% for the model of the Great

Buddha of Kamakura. See [12] in more details.
Table 2 Amount of memory ratio

2.2 Voxel-based Merging Algorithm
After all range images have been aligned, a volumetricview-merging algorithm generates a consensus surface ofthe objects from them. Our method merges a set of range
images into a volumetric implicit-surface representation,
which is converted to a surface mesh by using a variant of
the marching-cubes algorithm [14]. Unlike previous
techniques based on implicit-surface representations, ourmethod estimates the signed distance to the object surface
by determining a consensus of locally coherent
observations of the surface [15,16,17,18].
We utilize octrees to represent volumetric implicit
surfaces, thereby effectively reducing the computation andmemory requirements of the volumetric representation
without sacrificing accuracy of the resulting surface. We
originally design software that merges the aligned 20 range
data. However, recent input data is unpredictably huge, we
decide to built up a PC cluster to run this merging
software; the cluster parallel-processes the merging
algorithm for saving the computation time and utilizing alarge memory space of many PCs. We produced one
integrated digital Great Buddha with this software. The
whole data set consists of 3.3 M points, and 5 M polygonsthat can be merged in approximately 20 minutes on the PC
cluster.
We have digitally archived Japanese Buddhas, including
Asuka, Kamakura, Nara, and foreign ones, including
Thailand’s Wat Si Chum and Cambodia’s Biyon. We arecontinuing this effort toward completing the world Buddha
library, as shown in Figure 8, which digitally displaytransitions of Buddha shapes in time and region.

3. Texture Mapping and Rendering
The geometric model is vital information regarding thecultural heritage objects because it enables us to analyze
object in details. In addition to the geometric model,
surface color distribution (texture) is also very importantfor some kinds of cultural properties. Figure 9 shows oneof the Japanese national treasures, the 'Koumoku-Ten'statue. This clay statue still retains the surface colors thatwere painted at the time the figure was originally made.
For such types of cultural properties, we have to archivegeometry information, texture information and theirAsuka(9AD)

China
Afghanistan

Kamakura(12AD)

India

Nara(16AD)

ThailandCambodiaJapan
Figure 8. The World Buddha Library (Current Status)

relationship at the same time. For this purpose, we havedeveloped two kinds of texture mapping methods:
calibration-based and reflectance-edge based methods.

points in image coordinate xc={xc,yc} can be easily
calculated as:

3.1 Calibration-based Method
The problem of the texture mapping method is how todetermine the relationship between image sensors andgeometrical sensors. When a short-distance range sensors
can be used, as shown in Figure 9, the most promising
method is to calibrate the geometrical relationship between
the image sensor and the range sensor before scanning.
Assume that the coordinate system of the image sensor
is (xc,yc) and the corresponding point in range image is(X,Y,Z); the relationship between them can be described as:

The matrix C34 represents the relationship between
image coordinate and world coordinate, and it can becalculated by scanning the calibration box. Inversely, when
we map the texture images onto the geometrical triangularmesh Xn={(Xn,Yn,Zn)| 1<= n <=3} , the corresponding

Figure 9. The scanning scene of the Koumoku-Ten
clay figure using the VIVID 910

c21Xn +c22Yn +c23Zn +c24
c X +c Y +c Z +cxc = 11 n 12 n 13 n 14 , yc =c31Xn +c32Yn +c33Zn +c34
c31Xn +c32Yn +c33Zn +c34
For the modeling of the Koumoku-Ten clay figure, weused 60 range images and color images that taken at thesame time. Figure 10 shows the geometrical model (upper),the texture mapped result (lower) and synthesis results
under the different lighting conditions generated using the
texture mapped result (See Figure 11).

returned strength provides a reflectance measurement. A
reflectance image is a collection of the strength of returned
laser energy at each pixel. This reflectance image is alignedwith the range image because both images are obtained
through the same optical receiving device. Commonly
available range sensors, including ERIM, Preceptron, andour main sensor, CYRAX, provide this reflectance image.
We employ this reflectance image as a vehicle for thealignment of range images with color images [3,16].
Reflectance images share characteristics similar to color
images due to the fact that both images are somehow
related with surface roughness as shown in Figure 9. Since
our CYRAX range scanner uses a green laser diode,reflectance edges can be observed along the boundary
between two colors or material boundaries along difference
reflectance ratios for this wavelength. Since different
materials are of different colors, a discontinuity also
appears in the color images. Jump edges along small
ranges in a range image also appear as jump edges in areflectance image as well as in a color image. Occluding
boundaries are observed both in reflectance images and incolor images.
Figure 10.

Geometrical model of Koumoku-Ten andtexture-mapped model.

ReflectanceColor
Material
difference
Jump edge

Occlusion

Figure 11

Synthesized results under various lighting

conditions (Sunrise->Daytime->Sunset).

3.2 Reflectance-Edge based Method
One solution for determining the relationship between
range and color image is, as shown in the previous section,
through calibration using a calibration fixture. However,this method requires that the range and color sensors be
fixed on the fixture once the relationship is calibrated.
Further, the calibration-based method is accurate onlyaround the position occupied by the calibration fixture.
When a target object is very large, this method becomes
unreliable due to the lens distortion. Thus, we also need amethod that does not rely on calibration.
Generally speaking, range sensors often provide
reflectance images as side products of range images. The
returned timing provides a depth measurement, while the

Figure 12. Reflectance and color edges. They sharethe similar characteristics.

Prior to the alignments, we paste the necessary reflectanceedges onto the 3D geometric model. As mentioned above,since occluding boundaries vary depending on the viewing
direction, edges along the occluding boundaries are firstremoved from the reflectance images. On the other hand,edges along the current occluding boundaries will be
estimated from the 3D geometric model and the current
viewing direction. Our algorithm extracts them
automatically, and uses them for the alignment.
We align edges extracted from reflectance images withthose in color images so that the 3D position error of thoseedges is minimized by iterative calculation as shown in
Figure 13. Extracted edges are represented as a collectionof points along them. The alignment is done between 3D
reflectance points on 3D geometric model projected on theimage plane and 2D color edge points in the 2D image.
To establish correspondence, the system finds the color
image points that are nearest to the projected reflectance3D error z
3D edgepoint
Projection to the image plane

Geometric model
Nearest point

method, we can obtain the configuration P that minimizes
the error term and gives the relative relationship between
the camera and the range sensor. Figure 14 shows the
texture mapped Kamakura Buddha. Since this methodminimizes a non-linear equation, we need an initialalignment. The initial alignment is given manually using
our GUI. For the current implementation, relatively
accurate alignment is necessary for rotation, but it is not
the case for translation.

Image plane

Figure 13. Correspondence between 3D reflectanceand 2D color edges

points. This operation is similar to the ICP operation.
To determine the relative pose that coincides with theposition of 2D color edges and projected 3D reflectanceedges, we use the M-estimator.
First the distance between corresponding 2D color edge
points and 3D reflectance edge points is evaluated asshown in Figure 13 : where z i is a 3D error vector which
is on a perpendicular line from a 3D reflectance edge point
to the stretched line between the optical center and a 2D
color edge point on the image plane.
ε i = Z i sin θ
where Z i is the distance between the optical center and a
3D reflectance edge point, and θ is the angle between thecolor edge point and the reflectance edge point.
The system finds the configuration, P, which minimizes
the total error, E, where ρ is an error function. The
minimum of E ( p ) can be obtained by:∂ρ (ε i ) ∂ε i
∂E
=∑
=0
∂P
∂ε i ∂Pi
We can consider ω (ε ) as a weight function to evaluateerror terms.

ω (ε ) =

1 ∂ρ
ε ∂ε

Thus, the minimization can be derived as the following
least squared equation:∂ε∂E= ∑ ω (ε i )ε i i = 0∂ρ∂PiWe choose the Lorentzian function for this function.

 1  ε 2 ω (ε ) = 1 +    2σ  −1

By solving this equation using the conjugate gradientFigure 14. Texture Mapped Kamakura Buddha.

4. Restoring Hypothesized Original State
After we obtain the precise geometry and photometry
information of the cultural assets in the current state, we
can restore them to their hypothesized original state. In thissection, we describe one of the examples: the restoration of
the Nara Great Buddha and its main hall

4.1 Restoring Nara Great Buddha
Nara Great Buddha is the main statue of Toudaiji Temple.
Unfortunately, the current statue is a rebuilt and repaired
one because the original statue was burned and melted
down due to a couple of civil wars. Accordingly, the shapeof the current Great Buddha is different from that of theoriginal one.
By using the geometrical modeling shown in Section 2,
we have acquired the complete 3D geometrical model of
Buddha in its current state. From this model, we haveattempted to synthesize the original state by morphing the
3D geometry of the model.
From some literature inherited at the temple, we know
the sizes of various face parts such as the nose and mouth.
Using these data, we design a two-step morphingalgorithm.

First, we globally change the scale of the whole portions(e.g., Sitting Height, Face Length, Nose Length); these are
gradually modified. In the 2nd stage, vertices are moved
one by one iteratively, similar to the constraint propagation
algorithm, using smoothness and uniform constraints. The
2-stage morphing enables us to obtain the complete modelof the original Great Buddha. Figure 15 shows the 3Dmodels of the current (a) and the original Great Buddha (b).
We can easily recognize that the original Buddha is larger
and rather skinny.

Figure 16 Miniature and its 3D scanned models.

Figure 17 Partial models acquired at Tousyoudaiji.

(a) Current(b) Hypothesized
Figure 15. Restored result of Nara Great Buddha.

4.2 Restorating Toudaiji Main Halle
The main hall of the Toudaiji Temple was built during
the same decades as those of the Great Buddha (8thcentury). It was also rebuilt twice: in the 12th and 18th
centuries. In the 12th century, Tenjiku architecture wasimported from China and the main hall was rebuilt in a
totally different architecture style. The rebuilding in the
18th century followed the same new style. As a result, thestyle of the current main hall is entirely different from thatof the original building.
Fortunately, the Toudaiji temple has been displaying a
miniature model of the original hall, constructed for theParis Expo in 1900, as shown in Figure 16(a). We digitized
it using the Pulsteck TDS-1500 and scaled it up to theoriginal size as shown in Figure 16(b).
Due to the limitation of resolution, the detail partscannot be obtained precisely. According to Prof. Keisuke
Fujii, an architecture professor at the University of Tokyo,
one of the experts on building style in the era, the Toudaiji
and Toushou-daiji Temples share a similar format. Here,the main hall of Toshoudaiji Temple were also built during
the same period (8th century). After scanning various key
parts of the main hall at Toushoudaiji, as shown in Figure
14, we morphed these partial range data by expanding andshrinking the Touhoudaiji parts (Figure 17) to the scaled-up
range data of the Toudaiji (Figure 16). The process was
conducted by an extended alignment algorithm that allows
scale change as well as configuration differences. Figure
18 shows the current Nara Great Buddha main hall and the(a) Current Toudaiji main hall built in 18AD.

(b) Hypothesize 8AD Toudaiji main hall.

(c) Hypothesized 8AD Nara Buddha in the
hypothesized 8AD main hall.
Figure 18 Digitally restored Nara Great Buddha.

original one digitally restored by our method.

5. Conclusion
In this paper, we introduced our project to digitallyarchive and restore cultural heritage objects. Our project’s
main goal is to develop a method of 'modeling from reality',in which the digital model of cultural properties is createdby using various computer vision methods. For theobservation of geometrical information, we used laser
range finders and post process algorithms, including
registrating and merging the range images. For the texture
information, we have developed several texture mapping
methods. For the short distance range sensors, wecalibrated the relationship between the range sensor andthe image sensor for complete texture alignment. For thelong distance range sensors, we developed a non-calibrated
texture alignment method by using laser reflectancefeatures. Digital restoration of lost cultural heritage objects
has a big advantage compared with other restoration
methods such as physical construction of actual temples,
because we can examine various hypotheses without any
physical changes nor long building periods. We
demonstrated the effectiveness of this method through therestoration of the Nara Great Buddha and its main hall. We
are also conducting a project to create a digital library of
the world great Buddhas, including three Japanese
Buddhas, Sri Chum Buddha in Thailand, and Biyon’s inCambodia. The models and restoration results constructedso far can be viewed at [21]Acknowledgment
This research is sponsored, in part, by JST under Ikeuchi
Crest program. The Bayon in Cambodia was digitized withthe cooperation of Japanese Government Team for
Safeguarding Ankor (JSA).

References[1] K. Ikeuchi and Y. Sato, Modeling from Reality, Kluwer
Academic Press, 2001.
[2] M. Levoy et. al., “The digital Michelangelo project,”
SIGGRAPH 2000, New Orleans.
[3] J. Wasserman, Michelangelo’s Florence Pieta, Princeton
University Press 2003.
[4] I. Stamos and P. Allen, “Automatic registration of 2-D with
3-D imagery in urban environments,” ICCV2001, Vancouver.
[5] P.J. Besl and N.D. McKay, "A method for registration of 3-d
shapes," IEEE Trans. Patt. Anal. Machine Intell.,14(2):239-256, 1992.
[6] R. Benjemma and F. Schmitt, “Fast global registration of 3D
shample surfaces using a multiple-z-buffer technique,” Int.

Conf on Recent Advances in 3-D Digital Imaging andModeling, pp. 113-120, May 1997.
[7] P. Neugebauer, “Geometrical cloning of 3D objects via
simultaneous registration of multiple range images,” Int.
Conf on Shape Modeling and Application, pp.130-139,
March 1997.
[8] Y. Chen and G. Medioni, “Object modeling by registeration ofmultiple range images, Image and Vision Computing,10(3):145-155, April 1992.
[9] S. Rusinkiewicz and M. Levoy, “Efficient variants of the IPC
algorithm,” Int. Conf 3-D Digital Imaging and Modeling,pp.145-152, May 2001.
[10] H. Gagnon, M. Soucy, R. Bergevin, and D. Laurendeau,“Registeration of multiple range views for automatic 3-Dmodel building,” CVPR94, pp.581-586.
[11] K. Nishino and K. Ikeuchi, "Robust Simultaneous
Registration of Multiple Range Images", Fifth Asian
Conference on Computer Vision ACCV '02, pp454-461,
2002.
[12] T. Oishi, R. Sagawa, A. Nakazawa, R. Kurazume, and K.
Ikeuchi, “Parallel Alignment of a Large Number of Range
Images on PC Cluster,” Int. Conf 3-D Digital Imaging andModeling, Oct 2003.
[13] M. D. Wheeler and K. Ikeuchi, "Sensor Modeling,
Probablistic Hypothesis Generation, and Robust Localization
for ObjectRecognition", IEEE PAMI, 17(3): 252-265, 1995.
[14] B. Curless and M. Levoy, “A volumetric method for building
complex models from range images,” SIGGRAPH 96, New
Orleans, LA.
[15] M. Wheeler, Y. Sato, and K. Ikeuchi, “Consensus surfaces
for modeling 3D object from multiple range images,”
ICCV98.
[16] R. Sagawa, K. Nishino, M.D. Wheeler and K. Ikeuchi,"Parallel Processing of Range Data Merging", IEEE/RSJ
International Conference on Intelligent Robots and Systems,
Vol. 1, pp577-583, 2001[17] R. Sagawa, T. Masuda, and K. Ikeuchi, “Effective Nearest
Neighbor Search for Aligning and Merging Range Images,”
Int. Conf 3-D Digital Imaging and Modeling, Oct 2003[18] R. Sagawa and K. Ikeuchi, “Taking Consensus of Signed
Distance Field for Complementing Unobservable Surface,”
Int. Conf 3-D Digital Imaging and Modeling, Oct 2003[19] R. Kurazume, M. D. Wheeler, and K. Ikeuchi, "Mapping
textures on 3D geometric model using reflectance image,"
Data Fusion Workshop in IEEE Int. Conf. on Robotics andAutomation, 2001.
[20] R. Kurazume, K. Nishino, Z. Zhang, and K. Ikeuchi,"Simultaneous 2D images and 3D geometric model
registration for texture mapping utilizing reflectanceattribute," Fifth Asian Conference on Computer Vision, 2002.
[21] http://www.cvl.iis.u-tokyo.ac.jp/galley_e/.